Messaging-only request-response correlation

Recently I have been asked if it was possible to "couple" two services via BizTalk without using orchestrations. Let's look at that required scenario more closely:

1. A request message (ReqA) is received on the WCF-BasicHttp Receive Adapter, and it is posted to the MsgBox.
2. A solicit-response send port is activated due to a subscribtion to ReqA.
3. ReqA is mapped to ReqB.
4. ReqB is sent via the WCF-BasicHttp Send Adapter
5. RespB is received by the WCF-BasicHttp Send Adapter.
6. RespB is mapped to RespA and it is posted to the MsgBox.
6. RespB is picked up by the appropriate instance of the request-response receive port due to an instance subscription.
7. RespA is sent via the WCF-BasicHttp Receive Adapter.

The first questions that came up with me were:
1. How can the response of the outgoing webservice call be correlated to the incoming one? In the above scenario, that question would be "How can response B be correlated to response A?". I certainly didn't know that that comes out of the box with BizTalk. How? We'll see that later on.
2. What happens if we configure multple (outgoing) solicit-response send ports, subscribing to messages coming in on the one request-response receive port? Will timing issues start to play a role? Or will an exception be thrown?

Before I continue answering these questions, I would like to say that SoapUI is simply AWESOME when developing/working with web services (yes, including WCF web services). True, it's JAVA application, but still... :) It blows other webservice related stuff I've seen out of the water. Give it a try!

In order to answer question 1, we could check the instance subscription generated when a message is received on the request-response receive port. I did (by stopping -but not unenlisting- the subscribing sendport  and submitting a request), and it's as follows:

http://schemas.microsoft.com/BizTalk/2003/system-properties.EpmRRCorrelationToken == Svr:WIN2003EER2SP2;PID:3092;RRUid:{9731CF80-DDCF-4BB8-9CCA-03651243AE3E}  And
http://schemas.microsoft.com/BizTalk/2003/system-properties.RouteDirectToTP == True

Documentation on those properties is hard to come by. The BizTalk documentation states:

BTS.EpmRRCorrelationToken
Promoted by the Messaging Engine on request-response message execution. The property is promoted before messages are submitted into the MessageBox database.
Used internally by the Messaging Engine. Specifies the Server Name, Process ID and a unique GUID for a request response stream of messages.
 
BTS.RouteDirectToTP
Promoted by the Messaging Engine on messages for loop back or request-response execution. The property is promoted before messages are submitted into the MessageBox database.
Used internally by the Messaging Engine to enable loop back and request-response scenarios.

I guess anyone will agree that <censored>. However, from the fact that the above example is indeed working, we can safely assume that the EpmRRCorrelationToken property is copied to all messages involved in the communication flow. This implies that the EpmRRCorrelationToken property "survives" maps, the WCF solicit-response adapter and the XmlReceive pipeline. If you'd check the message context of the response message, published by the solicit-response port, you'll find those properties present and promoted.

Question 2 can be answered fairly quickly. I configured a second solicit-response send port using a loopback adapter (see my previous post for more info on the loopback adapter) and a map from ReqB to RespB (inbound or outbound, no difference in this case). Starting both send ports, and then firing a message into the request-response receive port (using SoapUI), will give you a routing failure in BizTalk and the following soap fault on the soap client:

"The message found multiple request response subscriptions. A message can only be routed to a single request response subscription."

Which basically means that a message just won't get processed in this scenario. Makes sense, but would you have predicted that? Searching the web found me a hotfix (KB923632), that apparently enables scenarios where multiple request response subscriptions should be allowed. I haven't got an example of where this would be needed, though. If you do, leave a comment!

How could we use this information?
One scenario I can think of right now is to have an orchestration that uses two seperate direct bound orchestration ports (one receive and one send), instead of one receiving request-response orchestration port to process the request and to submit a response. I started out with an orchestration that receives a message over a receive-only logical receive port, transforms it, and sends the transformed message over a send-only logical send port. All orchestration ports were direct bound. Oh, and after the transformation I copied all message context properties over to the transformed message. The orchestration looked like this:

The receive shape (Receive_1) had the following filter expression: (BTS.MessageType == "http://Blah.BlahBlah.Schema1#Schema1Root")
The SetContext expression shape looked like this: Message_2(*) = Message_1(*);

The result was that the response message (actually, better to say the transformed message, because we're intentionally trying to avoid request-response here) couldn't be routed back to the physical request-response port. Checking the message context revealed that the EpmRRCorrelationToken and RouteDirectToTP were present (due to copying over the properties), but they were not promoted, which is correct in hindsight (because setting properties in an expression shape within an orchestration does not promote them). So I added a correlation type and set, using these two context properties, and set the send shape to be initializing this correlation set. The result was that I got the following message:

Event Type: Error
Event Source:   BizTalk Server 2006
Event Category: BizTalk Server 2006
Event ID:   5813
Date:       8/19/2008
Time:       11:17:27 PM
User:       N/A
Computer:   WIN2003EER2SP2
Description:
A response message for two-way receive port "ReceivePort1" is being suspended as the messaging engine could not correlate the response to an existing request message. This usually happens when the host process has been recycled.
 
 MessageId:  {A817F71C-5AC0-4CEB-BAAC-C60EFED8D92E}
 InstanceID: {B931AE36-5C9F-4B23-8697-DCC79E0A76DF}

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Hmmmm, what's all this about? Apparently I hadn't satisfied all required properties, that are used internally by BizTalk for request response correlation. After some fiddling around with context properties, performing a lot of WinMerge (which also is awesome) actions to compare message contexts (of the messages coming from the solicit-response ports (see above) and the messages I was producing with my orchestration), and most importantly, by finding this excellent blog by Paolo Salvatori, describing exactly the property set I needed, I changed the expression shape and the correlation type into the following:

Expression shape:
Message_2(*) = Message_1(*);
Message_2(BTS.RouteDirectToTP) = true;
Message_2(BTS.IsRequestResponse) = true;

Correlation Type:
BTS.CorrelationToken,BTS.EpmRRCorrelationToken,BTS.IsRequestResponse,BTS.ReqRespTransmitPipelineID,BTS.RouteDirectToTP

The result was that the response was correctly picked up & processed by the instance-subscribing physical request-response port.

Conclusion
A much overlooked feature of BizTalk is the ability to route responses without the need for orchestration by using messaging only. This post described what happens internally in such a scenafrio and also described how that information can be used to create an orchestration, in which the request and response can be communicated over seperate direct bound orchestration ports instead of over one request-response port, which can be helpful (for instance) for testing the orchestration using physical FILE ports.