Magento 2 Queue Issues

Depending on how you setup Magento 2 for cron and your queueing implementation, you may come across an issue where the cron runner process gets stuck and is repeated multiple times along with each of the queue consumers.

This can eventually starve a server of resources as it constantly adds more processes to the system which will eventually lead to the site become unstable then unresponsive.

You can diagnose this by checking htop and filtering for the Magneto 2 cron cron:run or using ps aux | grep <project_location> to see the process running from there.

The root cause is that the queues may not be configured to end the process when there are no messages to process.

Fixing Queues getting stuck

The solution is a tweak to the app/etc/env.php file around the queue configuration.

Ensure you have the following:

1
2
3
4
5
// ...
'queue' => [
    'consumers_wait_for_messages' => 0
],
// ...

Sources