SAP Enterprise Messaging / Event Mesh – Push and Pull Message using REST API

Exploring SAP Enterprise Messaging/Event Mesh (SAP EM), and listed down my understanding and scenarios tried using SCP Trial account.

Basic Concepts & Scenario Tested

• Sender can send messages to both Queue and Topic.
• Queues hold messages, while Topic don’t. Message sent to Topic immediately pass to Queue.
• Queue Subscriptions link Queue to Topic. Many-to-Many relationship between Queue and Topic.
• If sender send message to Topic1, which link to both Queue1 and Queue2, then SAP EM will store message to both Queue1 and Qeueu2.
• If sender send message A to Topic1 and message B to Topic2, but both Topic1 and Topic2 link to Queue1, then SAP EM will store both messages to Queue1.
• Sender must include HTTP header “x-qos” with value either 0 or 1, else will hit error “x-qos is invalid”.
• Receiver consume messages from Queue, either using Push(SAP EM send messages to Webhook endpoint) or Pull(Client application read from SAP EM Queue).
• For Push, one queue should configure send to one webhook endpoint only (technically possible configure one queue send to different webhooks, e.g. webhook1 and webhook2, but webhook2 will be ignored because message already consumed by webhook1. If pause webhook1 subscription, then webhook2 will receive message).
• For Push, when create webhook, can configure QoS = 0 or 1.
  • If QoS = 0, SAP EM will delete message from Queue, after SAP EM post the message to webhook endpoint, regardless if webhook processing is success or failed.
  • If QoS = 1, after SAP EM post the message to webhook endpoint..
    • If webhook return HTTP 2XX code, mean positive acknowledgement, then SAP EM delete message from Queue.
    • if webhook return others HTTP code, message remain at Queue. Current observation is SAP EM will attempt to send messages in Queue again after 15 minutes.
    • Saw blog post mentioned this: “On failure (non-2xx response from webhook) EMS resends the message every 15 minutes initially, and then gradually slows down the retry frequency and keeps on redelivering for 30 days unless and until your webhook positively acks the message”.
• For Pull, when client call consumption REST API to consume queue, can configure x-qos = 0 or 1.
  • If x-qos = 0, SAP EM will auto delete message from Queue, after SAP EM response message to calling client. Each call from client will get and delete one message.
  • If x-qos = 1, SAP EM will not auto delete message from Queue, instead will only delete after receive acknowledgement API from client that contain message id in acknowledgement API. Message id was received in X-Message-Id header of first message consumption API.
  • If x-qos = 1, first call from client will return first message with unique X-Message-Id, subsequent call will return subsequent message with unique X-Message-Id as well, until no more messages to pull from, return empty payload and without X-Message-Id. However, even thought messages still in Queue(because not acknowledge yet), not sure what way to re-read earlier messages from Queue again. If you not save X-Message-Id for later use, seem like no way to get those X-Message-Id again to acknowledge/delete from Queue. Tested again, seem like after sometime(like 10 minutes), the message is available for consumption again, but don’t know how internally SAP EM re-enabled the messages in Queue for pull consumption again.

References:
SAP Event Mesh – Concept
https://blogs.sap.com/2018/10/07/getting-started-with-sap-enterprise-messaging/
https://blogs.sap.com/2018/12/18/sap-cloud-platform-enterprise-messaging-making-s4hana-event-notification-easy/
https://blogs.sap.com/2020/05/06/cloud-foundry-enterprise-messaging-webhooks
SAP Event Mesh – REST APIs Messaging
SAP Event Mesh – REST APIs for Messaging Management
https://www.youtube.com/watch?v=BJn6AJDxLW8