The ESB Guidance Scatter-Gather Sample

The motivation for this post
Ever since Richard wrote his post about a loosely coupled, MsgBox utilizing, Scatter-Gather implementation I’ve been meaning to illustrate the approach taken by Microsoft Pattern and Practices within the ESB Guidance package. Now there is absolutely nothing wrong with Richards solution, and it straight to the point and easy to grasp. The learning curve of the ESB Guidance and its Itineray Processing is something that not everyone is willing to take on if they don’t see the immediate benefit for it’s use within their environment. Still, it’s always interesting to get a different solution to the same problem, so here goes.

Itinerary Processing
First of all, the Scatter-Gather sample makes heavy use of Itinerary Processing. Both the orchestration to trigger, as well as the locations to send the scattered request and gather the responses from, and finally send the aggregated response to is expressed through an itinerary. Below is a sample of such an itinerary.

<Itinerary xmlns=”http://schemas.microsoft.biztalk.practices.esb.com/itinerary”>

  <ServiceInstance uuid=”” name=”ScatterGather” type=”Orchestration” 
	state=”Pending” position=”0″ isRequestResponse=”false” xmlns=”” />

  <Services xmlns=””>

    <Service uuid=”” beginTime=”” completeTime=”” name=”ScatterGather” 
	type=”Orchestration” state=”Pending” isRequestResponse=”false” 
	position=”0″ serviceInstanceId=”” />

  </Services>

  <Services xmlns=””>

    <Service uuid=”” beginTime=”” completeTime=”” name=”DynamicTest” 
	type=”Messaging” state=”Pending” isRequestResponse=”false” 
	position=”1″ serviceInstanceId=”” />

  </Services>

  <ResolverGroups xmlns=””>

    <Resolvers serviceId=”ScatterGather0″>&lt;
	![CDATA[BRE:\policy=ResolveEndPointScatterGather;version=;useMsg=;]]&gt;&lt;
	![CDATA[UDDI:\serverUrl=http://localhost/uddi;serviceName=PurchaseOrderSubmitOrderService;serviceProvider=Microsoft.Practices.ESB]]&gt;
    </Resolvers>

    <Resolvers serviceId=”DynamicTest1″>&lt;
	![CDATA[UDDI:\serverUrl=http://localhost/uddi;serviceName=OrderFileService;serviceProvider=Microsoft.Practices.ESB;]]&gt;
    </Resolvers>

  </ResolverGroups>

</Itinerary>

This describes an itinerary that will call two locations, described in the ScatterGather0 resolver, and send the aggregated response to the location described in the DynamicTest1 resolver. Basically, a resolver is a lookup location where the endpoint is stored. In the example above this is illustrated by the BRE (Business Rules Engine) and UDDI (Universal Description Discovery and Integration registry) resolvers. Read more about resolvers at Mikaels blog.

The Broker orchestration
The Scatter-Gather sample is made up of two orchestrations, Broker.odx and ServiceDispatcher.odx. The broker has a direct subscription and is started by a filter expression based on message context properties found in the Itineray above, namely the name, type and state with the service instance. The broker is started by a XmlDocument typed message. The orchestration then loops through the resolvers for ScatterGather0 and for each resolver starts the ServiceDispatcher orchestration (using the Start Orchestration shape) passing in the received message along with the Self Correlated porttype to respond to, and transport and resolver information. See figure1 for a condensed look of Broker.odx.

The ServiceDispatcher orchestration
The ServiceDispatcher.odx receives the message and gathers some metadata. It then maps the message to the required request format (determined by a BRE policy lookup). It does the maping by using API calls rather then by using the map shape, allowing it to stay generic and untyped. It then calls an untyped passthrough dynamic request-response port, whose location is determined from the location passed in, originating in the itinerary. Remember this orchestration was started by looping the resolvers. It then maps the response to the required response format (which is made up of some status information and an Any node, and send the response, through the MessageBox, back to the broker orchestration. Should an exception occur the same response message is sent back containing information saying that a fault has happened.

Aggregating the response
The Broker.odx receives the responses in a loop using a Self Correlated Partner Orchestration Port passed in as a parameter to the ServiceDispatcher.odx (see Charles Youngs write-up on Direct Biding models for more information on how this works – it’s not new, but it’s still good). It then adds the responses to a messageAggregator message. When it has collected all the messages it executes a send pipeline called AggregatingPipeline and receives the response back (see the Aggregator sample if you are curious on how this works). Finally the FinalResponse message is then sent out through to the final location given in the resolver named DynamicTest1.

In summary
There you go, message accomplished. I hope I explained it in a way that was understandable. Like I said in the beginning, the Itinerary processing might be a hurdle to get over when implementing this solution, but that information can of course come from a different source then the itinerary, like a config file, a database or another config location. Even leaving out the itinerary this sample still contains some interesting patterns and is worth checking out. This sample isn’t available for download from this blog, but I encourage you to check out the ESB Guidance. If ESB Guidance interests you I encourage you to read Mikaels blog. Also, if you happen to be in Stockholm, Sweden on February 12th we’ll be doing a presentation about the ESB Guidance for the BizTalk Usergroup Sweden.