Neuron ESB and Endpoints

Today I’d like to talk about how Neuron ESB connects to physical endpoints in the enterprise. Two blog posts ago I talked about Neuron’s topic network architecture, and in my last blog post I focused on the internal side of the bus, where channels implement pub-sub over a variety of Microsoft technologies ranging from WCF to BizTalk Server. But just how do those publisher and subscriber nodes relate to the physical world of services and clients? And what about non-service applications? We’ll answer that here as we talk about the external side of the bus.

First off, let’s mention in passing that one way you can program against Neuron is to write .NET programs using Neuron’s API. If you do that, you are directly creating a publisher or subscriber node. We’ll discuss the API and when you might choose to use it at another time. For now, we want to focus on scenarios where you don’t want to do that. Rather, you have existing endpoints in the enterprise and you want Neuron to be able to talk to them.

External endpoints fall into 2 distinct categories: they’re either service-oriented or they aren’t.

·         Service endpoints are service-oriented applications which of course come in two roles: clients (which initiate messaging) and services (which respond to messages).

·         Adapter endpoints are non-service oriented applications such as a legacy system or a line-of-business application. Neuron can access these kinds of endpoints using adapters. Like service endpoints, adapter endpoints can be send-oriented (resembling clients) or receive-oriented (resembling services).

Neuron can communicate with both service and adapter endpoints in request-reply or 1-way messaging patterns.

Service Endpoints

Service endpoints are first- or second-generation service programs. Neuron uses WCF to interact with service endpoints and thus inherits the ability to interconnect to services via a wide variety of transports and security models.

A web client connects to the ESB through a client connector (Figure 1). The client connector is both a WCF endpoint that the client can send messages to and a bridge to a Neuron publisher node. The client connector is mapped to a Neuron publisher Id and topic: when a client sends a message, the client connector receives the message and publishes it over the bus using the assigned publisher and topic. As described in a previous post on service pass-through, the hosting port on the ESB server looks to the client just like its target service, except that the address is different.  

 Figure 1: Client and Service Connectors

A service is connected to the ESB through a service connector (Figure 1) and acts reciprocally to a client connector as you would expect. The service connector is a Neuron subscriber node and a bridge to a WCF channel by which messages can be sent to the service. The service connector is mapped to a Neuron subscriber Id. When the subscriber receives a message over the bus, the service connector relays it to the service.

The bus gives you the freedom to enjoy a great variety of configuration possibilities that would be impossible if you directly connected your clients and services.

·         Loosely-coupled routing. Although Figure 1 only shows 2 endpoints, there could be any number of them communicating together. Doing so through the bus is far more attractive than the alternative, where services become clients of other services; such tightly coupled routing requires every module to know the connection details of every other module. It’s much smoother to communicate by named topics.

·         Mediation. There’s no requirement for the client and service endpoints to use the same method of communication or the same service model. For example, you could have a client that uses username/password security interacting with a service that uses X.509 certificate security through the bus. Even programs that have a different idea about message format or version can be made to work together by having the bus perform transformations on messages as they are en route.

·         Workload Sharing. You can have more than one instance of the same service, and the bus can perform workload sharing among them.

·         Service Policy. What should happen to a message if the target service is not available to receive it? Neuron lets you specify policy to control delivery aspects such as retries and failover to alternate destinations.

·         Services and Non-Services Working Together. You’re completely free to have your clients and services interact with non-services. The bus extends the reach of your services and other I.T. assets.

Adapter Endpoints

Enterprises often have line of business applications and/or legacy systems that are core to the business but are not service-oriented. Often these applications are “silos” in the sense that they have little or no interaction with other applications in the enterprise. Neuron provides comprehensive adapter support to bring these other systems onto the bus and make them accessible to the rest of the enterprise.

Neuron supports 2 families of adapters:

·         WCF LOB Adapters are the successor to BizTalk adapters. Adapters have already been developed for Oracle database, SAP and Siebel by Microsoft, and many other adapters are in developed by Microsoft  and other members of the community. As more and more of these adapters are developed, they benefit not only WCF and BizTalk Server but Neuron ESB as well.

·         Neuron .NET Adapters are Neuron’s own adapter model. They have an interface that makes them very easy to write (the design goal was that field engineers could write one on site if necessary).Neuron ships with a versatile collection of these adapters.

For Neuron .NET adapters, an endpoint connector (Figure 2) works analogously to a service connector or a client connector. WCF LOB Adapters are a different matter: because they are designed to look like WCF bindings, they can be used with service connectors or client connectors.

Figure 2: Endpoint Connectors

Just like service endpoints, adapter endpoints can operate in a client/sender role or a service/receive role; and just like service endpoints, adapter endpoints can utilize request-reply or 1-way messaging. An adapter defines one or more modes of behavior that define what it does and which direction and pattern messaging the mode needs.

To talk to a send mode adapter endpoint, the endpoint connector acts like a service connector, relaying bus messages to the adapter. The endpoint connector has an associated Neuron subscriber on which it receives bus messages which are then passed on to the adapter for delivery.

To talk to a receive mode adapter endpoint, the endpoint connector acts like a client connector, relaying adapter messages to the bus. The endpoint connector has an associated Neuron publisher on which incoming messages from the adapter are published onto the bus over a designated topic.

Interaction with adapters is optionally transacted: both Neuron adapters and Neuron pub-sub channels may implement transactions, and if they do the hand-off between them is also linked transactionally. This makes it possible to have transacted end-to-end communication through the bus, as long as the channel and adapter components you select support transactions.

As with service/client connectors, you may set up adapter policies to control the behavior of adapter endpoints in the face of errors. You can determine retry counts and actions to take when a message can’t be delivered.

The adapters that ship with Neuron include ActiveMQ,  RFID via BizTalk Server, eConnect (Great Plains accounting), SMTP email, files, MSMQ, SQL Server, and WebSphere MQ) and we’ll be taking more about them individually down the road.

Putting it all together: Service and Adapter Endpoints

Neuron’s strong support for services and adapters lets you break down the walls between your SOA applications, legacy systems, and line of

Neuron’s strong support for services and adapters lets you break down the walls between your SOA applications, legacy systems, and line of business applications, and other groups of systems that up till now have had limited reach.

All of your systems are now connected, but with loosely-coupled routing through named pub-sub topics. Actions such as changing the connection specifics of an endpoint, adding new endpoints, or removing endpoints can be done without disrupting the rest of the enterprise and without the need to make code changes to existing programs.