Kombu+RabbitMQ: Check whether a queue is empty
Architecture
Consider a system with DB records. Each record can be in a live or expired status; live records should be processed periodically using an external software module.
I have solved this using a classic producer - consumer architecture with Kombu and RabbitMQ. The producer fetches the records from the DB every few seconds, and the consumer handles them.
The problem
The number of live events greatly varies, and on peak hours the consumer can't handle the load and the queue is clogged with thousand of items.
I would like to make the system adaptive, so that the producer will not send new events to the consumer if the queue is empty.
What have I tried
http://<host>:<port/api/queues/<vhost>/<queue_name> . It works, but it's yet another mechanism to maintain, and I prefer an elegant solution within Kombu. How do I check whether a RabbitMQ is empty using Python's Kombu?
You can call queue_declare() on the kombu Queue object.
According to the docs the function returns:
Returns a tuple containing 3 items:
the name of the queue (essential for automatically-named queues)
message count
consumer count
Therefore you can do:
name, msg_count, consumer_count = queue.queue_declare()