Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 05/21/2008
From: Dan Rosanova <(email address - cut out)>
Message:

Hi Zoe,
Do you need to send the response at all? I use SOAP a lot with One
Way web service requests. This makes the web service like a void
method call. Alternatively you could just craft a fake response and
just always send that, sort of as a 'I got your message' response.
Lastly if you really need you could do a listen shape and send the
response no matter what at the end of the listen, completed or not.

This is common in the sense that HTTP clients can set their own
timeout and once that occurs you will not be able to send them a
response at all.

Kind Regards,
-Dan

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 05/21/2008
From: Dan Rosanova <(email address - cut out)>
Message:

OK, now I'm more confused. So is receive from the HTTP request is not
an Activating Receive, i.e. it's not the first shape in the
orchestration? If not then you must be using some sort of correlation
to receive this message? I'm affraid I don't understand your issue.
Kind Regards,
-Dan

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 05/21/2008
From: "Zoe Hart" <(email address - cut out)>
Message:

Do I need the response? Yes, I think I do. My client sent me a request and
is expecting a response. Let me see if I can explain the situation more
clearly, though it's a little complicated.

t = 0 Client sends request A1. It gets into the Message Box and is queued
up for my orchestration, but for one reason or another the orchestration
doesn't pick it up immediately.
t = 10s Client times out waiting for the response and resends request,
now A2 (second instance of request A). It also gets into the Message Box and
is queued up for my orchestration.
This goes on for a while.
t = 20+min Orchestration picks up request A1 from the queue, receives it,
processes it, and sends response B1.
At this point the orchestration suspends because the IIS session in which
request A1 was received, and which is waiting for response B1, has already
timed out. This generates a "message has no subscribers" type error and the
orchestration suspends.

By comparing the request A1 timestamp and the current time I can detect when
I'm in this doomed scenario, but the compiler won't let me *not* send the
response.

I also don't entirely understand BizTalk's internal workings and why the
messages get queued when they do. My orchestration sends a message and waits
for a response - both unrelated to request A1. The Listen shape for request
A1 is further down in the orchestration. It is while the orchestration waits
for the response to this first message that requests A1, A2, ... An get
queued up even though that Listen shape is further down the line, not in
scope, and actually within a Decide shape so that the Listen shape may never
even be executed at all.

I suppose, simply to satisy the compiler, I could change my two-way
request-response port into a one-way receive port for the request and a
one-way send port for the response. Then presumably I could make the send of
the response conditional based on the timing. But that seems like a rather
stupid way to develop a request-response port.
"Dan Rosanova" <(email address - cut out)> wrote in message
news:(email address - cut out)...

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 05/21/2008
From: "Zoe Hart" <(email address - cut out)>
Message:

As I said, it's complicated. I'll step back and try to give you the big
picture.
I have an orchestration that is managing a long (really long) running
transaction between two systems - call them A and B. Sometimes something
goes wrong, an exception occurs, and the orchestration gets suspended. When
this happens, we typically have to terminate the orchestration because it
wasn't designed in a way that lets it recover on resume. System A then
continues to process the long running transaction and a manual process is
used to update System B. But System B, in response to events triggered
through the manual process, still attempts to send messages to System A
through the automated, BizTalk hosted process. These fail because there's no
orchestration subscribed to them. When they fail, System B resends them...
over and over again for two weeks. The result is a really large number of
routing failures and suspended messages every time an orchestration fails.
When something happens that causes a large number of orchestrations to fail,
the number of suspended messages aimed at those suspended/terminated
orchestrations becomes ridiculously large.

Enter my new Responder orchestration. When the main orchestration gets an
exception it drops down into my exception handler. In the exception handler
I use a StartOrchestration shape to start an instance of the Responder
orchestration. I pass in enough information so that the Responder knows what
order it is responding to messages for.

The first thing the Responder does is send a message to IT Support. The
message contains the order number, which initializes a correlation set used
to receive subsequent messages from System B. The message gets mapped to an
HTML email that contains two links, each of which comes back to the
BTSHttpReceive listener. The query string parameters basically tell the
Responder to do one of two things:

1) Terminate the original suspended orchestration and begin listening for
and acknowledging all messages from System B for this order, or
2) Do nothing - used primarily in rare cases where the original
orchestration was successfully resumed.

So the Responder orchestration sends the message and waits for the message
back that tells it what to do. If the message back tells it to take Action
1, it first terminates the original orchestration (the orchestration
InstanceId was passed in as a parameter). It then has a Listen shape
listening for any one of about six different messages from System B and, for
each message, it simply needs to send a simple request-received
acknowledgement message that keeps System B happy and prevents System B from
sending and resending the messages.

The problem is that while the Responder orchestration has sent its initial
message and is waiting for the subsequent message that will instruct it what
action to take, System B is sending messages - either messages in response
to some event triggered through the manual interface or resending the
message it was sending when the initial exception that caused the main
orchestration to fail occurred. The Responder orchestration is not yet at
the Listen shape that will receive these messages but, for reasons I don't
fully understand, the messages get queued up for the Responder
orchestration. If I right-click a dehydrated Responder orchestration in the
Admin Console, one that I know is awaiting its "instructions" message, and
Show Messages I can see the messages from System B - usually 1 to n resends
of the same message queued up. Hours can go by before IT Support reads the
mail message and clicks the link that instructs the responder to terminate
the original orchestration and respond to messages from System B. Now the
Responder orchestration moves forward, terminates the original
orchestration, and reaches the Listen shape to listen for messages from
System B. It takes the first queued up message and attempts to send an
acknowledgement, but that Send fails with:

Exception occurred when persisting state to the database.
....
Additional error information:
A batch item failed persistence Item-ID ... OperationType MAIO_CommitBatch
Status - 1061151998 ErrorInfo The published message could not be routed
because no subscribers were found.

I believe this exception occurs if the orchestration attempts to send the
response after the IIS process that received the original request has timed
out. The end result is that my Responder orchestration, which was intended
to clean up the mess that results when my main orchestration suspends, now
ends up also suspended so I've got one more thing to clean up.

Now do you understand my issue? If not, don't worry about it. As I say, it's
complicated.

Thanks for trying.
Zoe

"Dan Rosanova" <(email address - cut out)> wrote in message
news:(email address - cut out)...

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 05/22/2008
From: Dan Rosanova <(email address - cut out)>
Message:

Hi Zoe,
Good description! I think I understand exactly now. Better yet, I
think I know how you can fix this too! OK, here's my idea. In your
Responder I would do like you are (Send that first message to
correlate), but then I would have a Loop around a Listen and on one
branch have the HTTP receive (with its Response) and on the other have
the receive from IT. This way, whichever comes first is caught (which
in most cases will be the HTTP from the other system). It can freely
response and then when the IT response comes you can decide what to do
then. Allow me to use my fantastic ASCII art skills to demonstrate.


[Send to IT]
|___[Loop]___________________
| |
[Listen] |
________|_________ |
| | |
[Receive B] [Receive from IT] |
| | |
[Send to B] [Do whatever] |
|__________________| |
| |
[Evaluate if loop continues] |
|___________________________|
|
[Continue on]

I think this will do exactly what you want. At least I hope so, that
drawing took time! Please let me know if this works. The big thing
is going to be figuring out how to end the receive loop. Maybe after
to get the IT response you want to go into a loop just to do receive
response to the system B requests. The conditions of terminiation
will probably what you need to focus on then.

Kind Regards,
-Dan

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 05/27/2008
From: "Zoe Hart" <(email address - cut out)>
Message:

Dan,
I think you've got it - both the problem and the solution. I've been talking
offline with another BizTalk developer and he suggested essentially the same
solution. I'm going to start work on it sometime later today.
Thanks,
Zoe

"Dan Rosanova" <(email address - cut out)> wrote in message
news:(email address - cut out)...

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 05/28/2008
From: Dan Rosanova <(email address - cut out)>
Message:

My pleasure. Again I am impressed Zoe was able to articulate her
situation so quickly. It=92s funny after working a lot with
orchestration how hard it can be to describe something verbally that
is a pretty simple picture. Too bad the posting messed up my
beautiful ASCII drawing of the solution :(

Rate my post if you liked it! :)

Kind Regards,
-Dan

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 06/10/2008
From: Dan Rosanova <(email address - cut out)>
Message:

Oh Zoe that's tough to hear, I really thought we got it! I took a
quick look at C:\Program Files\Microsoft BizTalk Server 2006\SDK
\Samples\AdaptersDevelopment\HttpAdapter and I know it can be done in
here, I just know it can. It looks like it's in the receive adapter.
I'll see if I can get some time this week to make this happen. I
think we'd want to be able to use a promoted property to set the
response code. I can't promise anything, but I'll try. Can you tell
me if your trading partner is use HTTP POST or GET? Also, let me
confirm you're on BizTalk 2006 (not R2).

Kind Regards,
-Dan

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 06/10/2008
From: "Zoe Hart" <(email address - cut out)>
Message:

My trading partner sends requests to me via HTTP POST. I'm on BizTalk 2006
R2.

"Dan Rosanova" <(email address - cut out)> wrote in message
news:(email address - cut out)...

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 06/17/2008
From: Dan Rosanova <(email address - cut out)>
Message:

Hi Zoe,
So here's where I'm at (don=92t think I forgot about you):

You could probably do this a lot easier on R2 with WCF, but as I'm
still mostly doing R1 I looked into the adapter code sample I
mentioned and it looks like in the ProcessRequestResponse method you
can throw any exception you like or just do
context.Response.StatusCode =3D 500; near the top. Now here is where I
am stuck because I don't do much adapter development. The real issue
is that you need to conditionally return this error, so you need to
ask BizTalk what is going on with this orchestration. I really don't
know how to do this, but can bet either WMI or part of the SDK
could.

I=92ll try to give it another shot, I just don=92t want to suggest
anything that=92s 1) overly complicated or 2) performs poorly.
Generally I find that what I want to do can be done easily in BizTalk,
I just don=92t know the right way to do it sometimes. Have you
considered opening a support ticket with MS or emailing someone there
about it?

Kind Regards,
-Dan

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 06/18/2008
From: "Zoe Hart" <(email address - cut out)>
Message:

I better respond or you'll think I forgot you. I'm running R2, but haven't
made the WCF leap yet. This Responder orchestration was supposed to be one
last deliverable on this project and was supposed to be relatively simple.
Alas, it hasn't worked out that way.

I have a conference call with my client in about an hour to determine if
they want to invest more effort in getting this last "feature" of the
responder to work correctly. If not I can drop back to a prior version that
almost works. In that version I don't initialize the correlation set for the
third party messages until after I receive the response from IT. The end
result in that version is that the messages get an automatic negative
acknowledgement from BizTalk (the behavior I want during the waiting period)
because there are no subscribers for them. Later when IT indicates that
Responder should terminate the original orchestration and respond to
messages, it successfully responds to all *subsequent* messages - messages
that are still coming in because the third party is resending because of the
negative acknowledgements generated by BizTalk. The less than ideal aspect
of this version is that the messages that came in while Responder was
waiting for a response from IT were suspended and never get automatically
cleaned up. They're left to be cleaned up manually. But my client may well
decide that's good enough. I'd love to keep coding and coding until
everything works exactly as I'd planned, but that may not make good business
sense.

If they do decide to have me keep working on Responder, the custom adapter
is a likely approach. It does seem like that's where the status code is
controlled. I know in the VS.NET orchestration GUI, the port operation for a
fault message has a direction of "Fault Message". I'd be curious to see if
that attribute gets into the message context where the Http adapter could
check it. Then it would be pretty straightforward to code the adapter to
return status code HTTP 500 if the message direction is "Fault Message".

I also have not done much (OK not any) adapter development, but I have a
more experienced BizTalk developer backing me up so he can probably help me
through it. And, of course, I have you. :)

Thanks for all of your help.

Zoe

"Dan Rosanova" <(email address - cut out)> wrote in message
news:(email address - cut out)...
Hi Zoe,
So here's where I'm at (don’t think I forgot about you):

You could probably do this a lot easier on R2 with WCF, but as I'm
still mostly doing R1 I looked into the adapter code sample I
mentioned and it looks like in the ProcessRequestResponse method you
can throw any exception you like or just do
context.Response.StatusCode = 500; near the top. Now here is where I
am stuck because I don't do much adapter development. The real issue
is that you need to conditionally return this error, so you need to
ask BizTalk what is going on with this orchestration. I really don't
know how to do this, but can bet either WMI or part of the SDK
could.

I’ll try to give it another shot, I just don’t want to suggest
anything that’s 1) overly complicated or 2) performs poorly.
Generally I find that what I want to do can be done easily in BizTalk,
I just don’t know the right way to do it sometimes. Have you
considered opening a support ticket with MS or emailing someone there
about it?

Kind Regards,
-Dan

Re: Handling IIS timeouts with HTTP request-response responses

Sent date: 06/18/2008
From: "Zoe Hart" <(email address - cut out)>
Message:

As I suspected, my client is growing weary of custom BizTalk development
(and weary of this particular project that has already lasted 2, 3, 4 times
longer than originally planned). They are content with the solution that
handles messages that come in *after* someone responds to the Responder
email notificaton and leaves messages that come in during the interim period
suspended, to be cleaned up manually.

So don't develop that custom adapter on my account...

Thanks again.

Zoe

"Zoe Hart" <(email address - cut out)> wrote in message
news:(email address - cut out)...