Skip to content

Introduction

The Neuron ESB Rabbit MQ Adapter allows Neuron ESB to interact with Rabbit MQ in a variety of ways, with configurable levels of Transaction support and Quality of Service. Organizations can use the adapter to monitor Rabbit MQ Topics or Queues for new messages, forwarding those (i.e. publishing) to a Neuron ESB Topic where other subscribing parties (e.g. Adapter, Service or Workflow Endpoints) can receive the information. Conversely, the adapter can also be used to forward information published to or generated by Neuron ESB to either a Rabbit MQ Queue or Topic, essentially making Rabbit MQ a Neuron ESB subscriber. Both types of Rabbit MQ Transactions are supported (e.g. Transaction and Publish Confirms) as are Virtual Hosts, SSL and all the latest features of Rabbit MQ.

Before reading this document

If you are not familiar with Neuron ESB Adapters, take a moment to consult our online help: https://www.peregrineconnect.com/documentation-kb/adapter-overview/. This document assumes you have the following knowledge:

  1. What adapters are and how they are leveraged in Neuron ESB
  2. Register a Neuron ESB Adapter within a Neuron ESB Solution
  3. Create an Adapter Endpoint in a Neuron ESB Solution
  4. Set a Neuron ESB Adapter Endpoint Properties
  5. Invoke a Neuron ESB Adapter Endpoint

Prerequisites

The Neuron ESB Rabbit MQ Adapter supports version 3.5.6 and later of Rabbit MQ and version 18.1 or later of Erlang. Using the Neuron ESB installer, users can optionally select to install these during the setup process, or can download these from the locations list in our readme.html file or directly from the Rabbit MQ web site.

Supported Modes

The Rabbit MQ Adapter supports 2 modes:

  • Publish: Used to monitor a Rabbit MQ Topic or Queue for new messages using Rabbit MQ’s Consumer Event based model. The information received from Rabbit MQ (as well as any custom headers) is published to a configured Neuron Topic.
  • Subscribe: Used to send messages from Neuron ESB to a Rabbit MQ Topic or Queue. The adapter allows many of the properties controlling the destination of the message to be configured dynamically at runtime.

Rabbit MQ Adapter Properties

Every Neuron ESB Adapter has design time properties that can configured within a standard Microsoft property grid (as shown below). The Rabbit MQ Adapter properties are located on the Properties tab of the Adapter Endpoint by navigating to Connections->Endpoints->Adapter Endpoints within the Neuron Explorer. These properties are used to determine the behavior of the adapter at runtime. The Rabbit MQ Adapter Properties fall into four classes:

  • General: Applies to modes (Publish and Subscribe)
  • Publish Mode Properties: Applies only to the Publish Mode
  • Security: Applies to the Connection made at runtime to Rabbit MQ regardless of mode.
  • Subscribe Mode Properties: Applies only to the Subscribe Mode

Figure 1: Rabbit MQ Adapter Endpoint Property Grid

Rabbit MQ Properties

PropertyCategoryDescription
Connect on StartupGeneralThe controls whether or the adapter will attempt to connect to the Rabbit MQ server on startup. This only is relevant for Subscribe mode since Publish mode will immediately attempt a connection on startup.
Connection TimeoutGeneralTimeout setting for connection attempts (in seconds).
Machine NameGeneralThe name or IP address of the Rabbit MQ Server to connect to.
PortGeneralThe port number of the Rabbit MQ Server. -1 uses default port
UsernameGeneralUser Id for Rabbit MQ Server logon
PasswordGeneralThe password for Rabbit MQ Server logon.
Virtual HostGeneralVirtual Host (e.g vHost) of Rabbit MQ Server. Default is “/”
ProtocolGeneralThe Rabbit MQ Protocol to use. Will use default from Rabbit MQ server app settings if not provided. Rabbit MQ AMQP 0.9.1 is supported
Publish TopicPublishThis property is used to specify the Neuron ESB Topic to use when publishing the received message to the bus.
Auto AcknowledgePublishIf true, messages will be automatically acknowledged and removed from the queue once the party receives the message.
Prefetch SizePublishThe number of messages that will be pre-fetched from the queue to transport layer. If ‘Auto Acknowledge’ is set to true, there will be a possibility of message loss if the Rabbit MQ server fails. A value of 0 means unlimited.
Subscription TypePublishDetermines what and how to monitor on Rabbit MQ to receive messages. Queue = Receives messages from an existing Queue. Topic = Receives messages from a Queue where messages are from a specific Exchange with a Specific Routing key.
  -QueuePublishRabbit MQ Queue to receive messages from
  -ExchangePublishIf Subscription Type = Topic. Rabbit MQ Exchange to receive messages from. This is used to create a Subscription based on Exchange, Queue and Routing key.
  –SubscriptionPublishIf Subscription Type = Topic. Routing Key used to receive messages from. Acts as a subscription. Example : ‘STOCK.IBM.#’, Where ‘*’ can be used to replace one word and ‘#’ to replace 0 or more words
  –Persist SubscriptionPublishIf Subscription Type = Topic. If True, the Subscription will remain after the adapter is shut down. If false, the Subscription will be removed when adapter shuts down. The Subscription is essentially the Routing Key that the Queue is bound to.
Error ReportingPublishThis property determines how all errors are reported in Event Log and Neuron Logs. Either as Errors, Warnings or Informational. This property can be one of three values:·       Error (Default)·       Information·       Warning 
Error on PublishPublishThis property determines if monitoring of the Rabbit MQ Server continues on error and if consecutive errors are reported. The property is used to determine consecutive reporting and whether or not it should shut down on if an exception occurs in the event handler. This property can be one of three values:·       StopPollingOnError (Default): Stops the Adapter when an error is encountered.·       SuppressConsecutiveErrors: The Adapter will only report the first error and will continue to run.·       ReportAllErrors: The Adapter will only report all errors and will continue to run. 
SSL EnabledSecurityConnect to Rabbit MQ Server using only SSL
  -PortSecuritySSL Port for all Rabbit MQ server connections
  -SSL ProtocolSecuritySSL Protocol to use for all Rabbit MQ server connections.
  -Client AuthenticationSecurityRequire Neuron ESB to provide Rabbit MQ Server a client certificate to authenticate against.
     -CertificateSecuritySelect a client certificate configured in the Security section of the Neuron ESB Explorer to authenticate against the Rabbit MQ Server.
     -PassphraseSecurityPassphrase for the client certificate, if one exists
Delivery ModeSubscribePersistent (i.e. Durable) or NonPersistent. Persistence ensures message is written to disk. If using Transactions, Persistence SHOULD be used.
Routing ModeSubscribeDetermines if the message should be sent to a Queue or to a Topic (i.e. Exchange with Routing Key). Queues that are bound to the Routing Key and Exchange will receive the message. Routing Keys can be used to form Topic based subscriptions. Example : ‘STOCK.IBM.#’, Where ‘*’ can be used to replace one word and ‘#’ to replace 0 or more words
  –QueueSubscribeIf Routing Mode = Queue. Rabbit MQ Queue to send messages to
  –ExchangeSubscribeIf Routing Mode = Topic. Rabbit MQ Exchange to send messages to. Used in conjunction with Routing key
  –TopicSubscribeIf Routing Mode = Topic. Routing Key used to send messages. Acts as a Topic. Example : ‘STOCK.IBM.#’, Where ‘*’ can be used to replace one word and ‘#’ to replace 0 or more words
Transaction TypeSubscribeControls the level of reliability for messages. Either ‘None’, ‘PublisherConfirms’ or ‘Transactional’ can be selected. ‘PublisherConfirms’ uses an asynchronous Ack/Nack protocol while ‘Transactional’ forces a commit/rollback on each message published.
  -Batch SizeSubscribeOnly for use with PublishConfirms type of transactions. The number of messages that will be published to Rabbit MQ in a Publish Confirm transaction
  -Batch Confirm TimeoutSubscribeOnly for use with PublishConfirms type of transactions. The number of seconds to wait after the Batch of messages have been published to receive all ACKs/NACKs from Rabbit MQ. Should be a value between 1 and 60.
  -Inactivity TimeoutSubscribeOnly for use with PublishConfirms type of transactions. The number of minutes to wait after the last message sent before checking to determine if all ACKs/NACKs from Rabbit MQ have been received. Should be a value between 1 and 5.
Time To LiveSubscribeA value in minutes that specifies how long messages are valid for delivery by Rabbit MQ before they are expired (dead letter).
Must be RoutableSubscribeIf set to true, the message must be routable by Rabbit MQ. If the message cannot be routed to a destination queue, the failure will be recorded asynchronously and posted to the Neuron ESB Failure Message database table.

MetaData

Neuron ESB has the ability to support custom message properties, either generated by users or by the sub systems or Adapters that process the message. In messaging systems, it’s very common to add custom meta data to a message where its either impractical or impossible to alter the original inbound or outbound message. Many developers use these custom properties to assist in driving custom business logic as part of the integration process.

For example, when the Rabbit MQ Adapter receives a message from a Queue or Topic, that information is stored in the Neuron ESB Message Body property. However, the adapter adds a number of custom properties specific to the adapter to the Neuron ESB Message before its published. These properties would contain information such as the Name of the Queue or Topic the message was retrieved from as well as any Rabbit MQ custom headers which may have already existed on the message. The following shows a message retrieved by the Rabbit MQ adapter, published to a Neuron ESB Topic and received by a subscribing Neuron ESB Test Client.

article-adapter-rm2

All custom message properties generated by the Rabbit MQ adapter are all prefixed with the moniker “rmq”. These properties can be retrieved or set within any Business Process or Workflow Activity using the Set Property Process Step or, in any C# Editor, using the following:

var routingKey = context.Data.GetProperty("rmq","RoutingKey");

var customHeader = context.Data.GetProperty("rmq","Header.custom1");

If users do NOT want Neuron ESB to either generate or set custom message properties specific to the Rabbit MQ adapter (i.e. properties prefixed with “rmq”), the “Include Metadata Properties” located on the General tab of the Adapter Endpoint should be unchecked.

Publish Mode

The following properties can be generated and added to the Neuron ESB Message’s Custom Properties collection using the moniker/prefix, “rmq”.

PropertySourceDescription
ModeNeuron ESBIndicative of the Mode the Rabbit MQ Adapter is configured for. Generated by Neuron ESB
ServerNeuron ESBRabbit MQ server. Generated by Neuron ESB
PortNeuron ESBRabbit MQ server port. Generated by Neuron ESB
QueueNeuron ESBRabbit MQ Queue the message was received from. Generated by Neuron ESB
ConsumerTagRabbit MQRead from Rabbit MQ Basic Properties
DeliveryTagRabbit MQRead from Rabbit MQ Basic Properties
RoutingKeyRabbit MQRead from Rabbit MQ Basic Properties
DeliveryModeRabbit MQRead from Rabbit MQ Basic Properties
PersistentRabbit MQRead from Rabbit MQ Basic Properties
ExpirationRabbit MQOptional. Read from Rabbit MQ Basic Properties
ContentTypeRabbit MQOptional. Read from Rabbit MQ Basic Properties
ClusterIdRabbit MQOptional. Read from Rabbit MQ Basic Properties
ContentEncodingRabbit MQOptional. Read from Rabbit MQ Basic Properties
CorrelationIdRabbit MQOptional. Read from Rabbit MQ Basic Properties
AppIdRabbit MQOptional. Read from Rabbit MQ Basic Properties
MessageIdRabbit MQOptional. Read from Rabbit MQ Basic Properties
PriorityRabbit MQOptional. Read from Rabbit MQ Basic Properties
ReplyToRabbit MQOptional. Read from Rabbit MQ Basic Properties
UserIdRabbit MQOptional. Read from Rabbit MQ Basic Properties
ReplyToAddress.ExchangeNameRabbit MQOptional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present.
ReplyToAddress.ExchangeTypeRabbit MQOptional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present
ReplyToAddress.RoutingKeyRabbit MQOptional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present
ExchangeNeuron ESBRabbit MQ Exchange the message on the Queue was received from. If Queue, generated by Rabbit MQ, if Topic was configured, generated by Neuron ESB.
SubscriptionNeuron ESBRabbit MQ Subscription that was used to retrieve the message. Only if message is retrieved via Topic. Generated by Neuron ESB.
Header.*Rabbit MQOptional. Read from the Rabbit MQ Basic Properties Headers collection. * represents the name of the header retrieved.

Subscribe Mode

Dynamic Configuration Properties

When the Rabbit MQ Adapter is configured in Subscribe mode, some of the adapter’s specific custom message properties can be used to dynamically configure the Adapter Endpoint at runtime. For example, at runtime a Business Process or Workflow may execute business logic that determines what Queue, Exchange or Routing Key should be used to for the Adapter Endpoint. The properties that can be set at runtime are:

  • Exchange
  • Queue
  • RoutingKey
  • Expiration

Setting the properties can be done using the Set Properties Process Step or by using C# code in a Neuron ESB Code Editor as shown below:

context.Data.SetProperty("rmq","RoutingKey","TestTopic");

Add Custom Headers

Adding extra meta data to the message that Neuron ESB sends to a Rabbit MQ Queue or Exchange can be easily done using the “Header.*” type properties of the adapter. Recipients using Rabbit MQ can retrieve theses using the Rabbit MQ API and accessing the Headers collection of the Basic Properties of the Rabbit MQ message. This meta data can be written at runtime by using C# code in a Neuron ESB Code Editor as shown below:

context.Data.SetProperty("rmq","Header.custom1","my extra info");

Publish Confirm Transactions

The Neuron ESB Rabbit MQ Adapter supports both Transaction types that Rabbit MQ offers; their channel based Transaction model as well as their batched style Transaction model e.g. Publish Confirms. Both are Acknowledge, Negative Acknowledgement (ack/nack) based models. Users can learn more about Publish Confirms here: https://www.rabbitmq.com/confirms.html , as well as why Rabbit MQ introduced them: http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms .

The Rabbit MQ Adapter exposes several properties that can be used to finely tune the performance and throughput as well the reliability of the batch transaction such as “Batch Confirm Timeout” and “Inactivity Timeout”. These properties force Neuron ESB to call into Rabbit MQ for it to finish sending any pending acks/nacks. Messages the adapter receives nacks for or where we’re notified by Rabbit MQ that that message is undeliverable (i.e. where “Must be Routable” property is set to True), are automatically moved into the Neuron ESB Failed database table with an AdapterSubscribeException type.

Security

With CU4’s support for Rabbit MQ 3.5.6 comes support for SSL. Once SSL has been enabled, the Rabbit MQ SSL port needs to be provided and the SSL Protocol to use must be selected. Although Neuron ESB and Rabbit MQ supports both SSL2, SSL3, TLS, TLS 1.1 and TLS 1.2 by default SSL3 support is disabled by Rabbit MQ to avoid POODLE attacks. More about Rabbit MQ and its SSL support can be found here: http://www.rabbitmq.com/ssl.html

Client Authentication can also be enabled by providing a Certificate (registered within the Security section of the Neuron ESB Explorer).

Configuring SSL support is done by modifying the Rabbit MQ configuration file as well as registering several important Environment Variables for the machine. More information can be found here: http://www.rabbitmq.com/configure.html .

About the Author

Author's Name
Marty Wasznicky

President/CTO

Marty has almost 30 years of experience in the software development industry. He joined Peregrine Connect after six years as a Regional Program Manager in the Connected Systems Division at Microsoft. His responsibilities there included building out Microsoft’s BizTalk Server product integration business, managing a team of SOA/ESB/BPM field specialists and building strategic partner alliances. Marty created the Microsoft Virtual Technical Specialist program and owned the development of Microsoft’s Enterprise Service Bus Toolkit.

Read more about Peregrine Connect

articles
  • Rabbit MQ Topics

    Introduction Due to the open-source nature of RabbitMQ and constant updates, it is...

  • Port Sharing

    One of Neuron ESB’s scalability features is the ability to install multiple...

whitepapers
  • The Integration Journey to...

    The Integration Journey to Digital Transformation with Peregrine Connect

  • Saving Time and Money by...

    Neuron ESB Application Integration and Web Service Platform: A Real-World Example...

casestudies
  • Elektro Gorenjska

    Peregrine Connect Eliminates Over 30% of Point-to-Point Integrations and reduces...

  • D&H Distributing

    Modernizing operations integration to increase volume transactions by 2X

video
  • video-icons-wrapper

    Decision Test Data Mapping

    - Use decisions to drive the execution of...

  • video-icons-wrapper

    Map Testing

    Learn how to utilize FlightPath's testing functions...