How to test if Magento RabbitMQ Consumers are working properly

2021-07-04
Featured Image

How can you verify if Magento can reach and utilize the benefits of RabbitMQ?

This tutorial will help you to modify attributes of a product using Magento's async API and bulk API.

Magento settings #

We used the following settings in app/etc/env.php.

//...
'queue' => [
        'consumers_wait_for_messages' => 0,
        'amqp' => [
            'host' => 'localhost',
            'port' => '5672',
            'user' => 'magento',
            'password' => 'my-secret',
            'virtualhost' => '/',
            'ssl' => ''
        ]
    ],
    'cron_consumers_runner' => [
        'cron_run' => true,
        'max_messages' => 10000,
        'consumers' => [

        ]
    ],
//...

The consumers_wait_for_messages setting will command Magento to run the AMQP consumer’s in every minute alongside the default Magento Cron, but the consumers will instantly stop after they finished consuming all messages. You can read more about this setting here: Magento DevDocs - Configure consumer behavior.

Before we begin #

Create a test product #

On the Magento Admin create a test product.

  • Catalog → Products → New Product
  • Fill in the required attributes.
  • Click on Save.

For our test product, we set the product’s SKU to test. We are going to use this in this article.

Get your Admin token #

  • Open the Magento Swagger UI. The URL should be something similar to this.
    • https://my-magento-shop.com/swagger
  • Authenticate using twoFactorAuthGoogleAuthenticateV1 endpoint.
  • Save the Bearer token.

1 - Rename the product using Magento’s async API #

Based on catalogProductRepositoryV1 endpoint insert the /async path before /V1 in the API call.

The new API endpoint

https://my-magento-shop.com/rest/all/async/V1/products/{SKU}

Example JSON Payload

{
  "product": {
    "sku": "test",
    "name": "test-11:39:15"
  }
}

Example cURL call (with added /async path)

curl -X PUT \
  -H "accept: application/json" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer __MY_BEARER_TOKEN__" \
  -d "{ \"product\": { \"sku\": \"test\", \"name\": \"test-11:39:15\" }}" \
  "https://my-magento-shop.com/rest/all/async/V1/products/test"

Post the request using Paw or Postman

Rename - Post request with Paw or Postman

You can verify if the message arrived to the queue using RabbitMQ Admin (in the Queues section)

Rename - Check the queue status in RabbitMQ Admin


2 - Rename multiple products using async Bulk API #

Based on catalogProductRepositoryV1 add /async/bulk path before /V1 in the API call.

API Endpoint (note: we are not using SKUs in this URL)

https://my-magento-shop.com/rest/all/async/bulk/V1/products

Example JSON Payload:

[
{
  "product": {
    "sku": "test",
    "name": "test-11:44:44"
  }
},
{
  "product": {
    "sku": "16-test1",
    "name": "16-test1-11:44:44"
  }
}
]

Example cURL call (with added /async path)

curl -X PUT \
  -H "accept: application/json" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer __MY_BEARER_TOKEN__" \
  -d "[{ \"product\": { \"sku\": \"test\", \"name\": \"test-11:44:44\" }},{ \"product\": { \"sku\": \"16-test1\", \"name\": \"16-test1-11:44:44\" }}]" \
  "https://my-magento-shop.com/rest/all/async/bulk/V1/products"

Post the request using Paw or Postman

Mass Rename - Post request with Paw or Postman

Response (it is important because it contains the Bulk UUID)

Mass Rename - Post response in Paw

Check the update in RabbitMQ Admin (Queues)

Mass Rename - Check the queue status in RabbitMQ Admin

Alternatively, you can view the messages in the RabbitMQ Admin (Queues → Get messages)

Mass Rename - Check the messages in RabbitMQ admin


3 - Check the Bulk update status using Magento API #

We will use the BulkUuid from our previous request’s response:

730538f2-92cf-443e-ba07-403f63e9d16b

Using Magento /V1/bulk/{bulkUuid}/status endpoint

Example cURL call

curl -X GET \
  -H "accept: application/json" \
  -H "Authorization: Bearer __MY_BEARER_TOKEN__" \
  "https://my-magento-shop.com/rest/all/V1/bulk/730538f2-92cf-443e-ba07-403f63e9d16b/status"

Response

Post and Response

Check the status on the Magento Admin Dashboard #

Go to SystemAction LogsBulk Actions

Magento Admin Bulk Action Log

Select the action and click on Details.

Magento Admin Bulk Action Details

A super simple way to test RabbitMQ Authentication #

Requirements:

  • python3
  • pip3
pip3 install kombu

cat << "EOF" >> test.py
#!/usr/bin/env python3
import socket
from kombu import Connection
host = "localhost"
port = 5672
user = "magento"
password = "magento"
vhost = "/"
url = 'amqp://{0}:{1}@{2}:{3}/{4}'.format(user, password, host, port, vhost)
with Connection(url) as c:
    try:
        c.connect()
    except socket.error:
        raise ValueError("Received socket.error, "
                         "rabbitmq server probably isn't running")
    except IOError:
        raise ValueError("Received IOError, probably bad credentials")
    else:
        print("Credentials are valid")
EOF

chmod +x ./test.py

./test.py

Credits: Lorin Hochstein

Avatar
Janos Miko Fill your bucket by drop by drop.