Skip to content

Channel callback from delivery acknowledgement timeout. #681

Closed

Description

@dchompd

Consumer silently losing its channel due to RabbitMQ closing the channel because of a delivery acknowledgement timeout.

I have a channel open with a consumer working it. Messages are being being pulled off and the consumer is manually acknowledging them. I ran into the issue of a delivery acknowledgement timeout (https://www.rabbitmq.com/docs/consumers#acknowledgement-timeout) that resulted in rabbit closing the consumers channel with a PRECONDITION_FAILED channel exception. This manifested in the application silently accumulating messages with no indication that the consumer was no longer being delivered messages on the channel. When I recreate the delivery ack timeout (create q with a short timeout and simulate a job that hangs the work pool thread and never acks), I see the AMQ::Protocol::Channel::Close frame (

when AMQ::Protocol::Channel::Close then
) and also noticed there is the @on_error handler. Unfortunately, in this case, the on_error handler is never called (

bunny/lib/bunny/channel.rb

Lines 1807 to 1812 in 8b700a0

if channel_level_exception_after_operation_that_has_no_response?(method)
@on_error.call(self, method) if @on_error
else
@last_channel_error = instantiate_channel_level_exception(method)
@continuations.push(method)
end
) and the last_channel_error ends up pushed onto the continuations. If the thread working the message is hung, this error is never raised. I'm trying to understand if there is a way my channel can be notified that RabbitMQ is closing it due to the delivery acknowledgement timeout (reply code 406)? (besides monkey patching a condition into channel_level_exception_after_operation_that_has_no_response?

bunny/lib/bunny/channel.rb

Lines 1822 to 1824 in 8b700a0

def channel_level_exception_after_operation_that_has_no_response?(method)
method.reply_code == 406 && method.reply_text =~ /unknown delivery tag/
end
). My current monkey patch just adds an additional text to that line

method.reply_code == 406 && (method.reply_text =~ /unknown delivery tag/ || method.reply_text =~ /delivery acknowledgement on channel \d+ timed out/)

but I'm trying to fully understand what the best approach here would be. Thanks!

Activity

  1. locked and limited conversation to collaborators on Mar 20, 2024
  2. converted this issue into a discussion #682 on Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions