Topic Networks: How Neuron ESB Implements Publish-Subscribe

Today I’d like to talk about how Neuron ESB implements publish-subscribe over topics. Topics are important in Neuron both logically and architecturally. Logically, topics model business conversations and are an important tool for organizing information. Architecturally, topics are independent networks that give the bus its distributed nature and scalability.

A Logical View of Topics

Logically speaking, topics represent business conversations and are usually named accordingly. Topics can have simple names like Orders or Inventory, or a topic can have a hierarchy of subtopics, expressed with dotted names such as Orders.New, Orders.Approved, Orders.Shipping.US, Orders.Shipping.International, and so forth. A party expresses interest in a topic with a subscription. Subscriptions may be wildcarded, as in Orders.* which expresses interest in Orders as well as all of its subtopics (Orders.New, etc.). By modeling your solution after actual business conversations and entities, you raise the likelihood that your messaging system will be a successful and long-lived one.

Together, topics and subscriptions define a virtual circuit between publishers (senders) and subscribers (receivers) about some subject of interest. In Figure 1, a publisher named OrderEntry has published a message on the topic Orders.New. Of the 3 other parties, only 2 have matching subscription: (OrderApproval and OrderFulfillment. The other subscriber, PaymentApproval, is only interested Orders.Paid and does not see messages published on Orders.New.

Figure 1: Topical Publish-Subscribe

Although bus communication is pub-sub based, that doesn’t limit you to pub-sub as a messaging pattern. Parties can also use point-to-point messaging, request-reply messaging, and itinerary-based routing.

The ability to easily add, change, and remove parties, topics, and subscriptions—even while running live—results in great flexibility and easy change management.

An Architectural View of Topics

Architecturally speaking, each topic is an individually configured network called a topic network. Figure 2 shows 3 topic networks, named Inventory, Orders, and CallCenter. Each topic network is separately assigned a communication technology to drive it, such as peer networking, WCF TCP, MSMQ, or BizTalk Server. This allows you to run topics with a mix of qualities of service, a refreshing change from messaging systems with a single “one size fits all” QoS. Each topic network can also refer to separate server infrastructure, making topics the primary division for scaling out. The distributed topic network design is very different from the classic EAI hub-and-spoke design. There is no central hub and no central message store that all messages must pass through; instead, each topic network performs its functions independently. While you could run all of your topics over the same technology and off of the same server, you could just as easily run each topic over a different technology and different servers. Moreover, some of the technologies that can power topics require little or no server infrastructure at all.

Figure 2: Topic Networks

If each topic is really a separate network, what sort of network is it, and what are its characteristics? The answer to  that is determined by the channel you select. A channel is a modular software driver that implements pub-sub for Neuron ESB using some base technology, such as WCF or BizTalk Server. The channel has complete freedom to implement pub-sub any way it wishes, as long as it provides the expected functionality. Some channels are volatile, emphasizing speed over reliability. Others channels are durable, stressing reliability over speed.

In a future article, we will dig into channels more deeply and compare the various trade-offs between them.