Storing information for BizTalk Server 2004 and BizTalk Server 2006 – a new model is exposed – Blog Entry

I've been reviewing and studying the CrossReferencing APIs because it looks like no one is using them, and I like to find new things to play with.

I got a hint from Marty Wasznicky that they were written for a customer years ago "to provide a way to easily take an inbound value and cross it over to an outbound value."

As I thought about it, I realized that I do this all the time. For example, at a health care provider, I had inbound LabCorp Medical Testing values that had a one-to-one relationship with the electronic medical records (EMR) value. So, I could possibly use this to map values from one system to value to another system.

UC1 – Map inbound values to outbound values in a Map (pre-built functoids!)

Then I was thinking about the fact that in my SAP System, the target system has values in the BAPI that are different between environments. For example, the OBJ_SYS value in the message changed. Instead of two maps with hard-coded values in them, I could use the xRef lookup to return the value that is appropriate for each system.

UC2 – Environmental Constants that change from system to system, but are shared across all servers in the same group used in Maps, Pipelines, or Orchestrations.

Then I got interested in using the APIs directly in the Orchestration. I realized that I could set a flag that turned on and off Debugging output that was specific for each component in the system Orchestration, Expression Shape, Exception, and so on. System.Diagnostics.Debug.WriteLineIf(Boolean, Debugstring, Category)

UC3 – Optional configuration values that could be used in Orchestrations to change behavior

Then I finally realized that I could use these APIs to set variables in the Orchestration for use. For example, I wanted the Category in the Debug statement to be configurable. Once I figured that out, I realized that almost any variable in an Orchestration could be initialized and changed on the fly, constrained by some limits, that I've listed below.

UC4 – Variables that can be changed in the Orchestration or Pipelines (YES PIPELINES or Custom CODE!), as desired.

The advantages of this approach are:

1. Variables are read at run time and require no restart of the Host Instance to apply the change.
2. The APIs and functoids are brutally easy to use.
3. The values take effect immediately.
4. This is the ONLY way I know of to change values in a map at run time because they provide built-in functoids for the APIs.
5. The security is controlled by the standard BizTalk Security model. No additional security management is required.
6. Unlike the BTSNTSvc.exe.config file, erroneous changes to the data will not cause the service to fail to start.
7. The config is shared simultaneously across all the servers automatically.

Disadvantages are:

1. The fields are limited to 50 characters.
2. The Name (AppID) and Values (CommonID) must be each unique. You cannot have duplicate values in either column.
3. The method to feed values into the system leaves much to be desired. It's a complex set of XML config files, and the development of them is an arcane art -> I may be induced to build a creator tool that generates them.
4. Once a value is into the system, there is only one class of values that can be deleted. A lot of the remainder can't be deleted or even updated. I have concentrated my efforts on the ones that are updatable due to this reason.
5. The data is stored in the BizTalkMgmtDb, which is not normally optimized heavily because that it is a low usage database. So you have to be careful that there is sufficient room on the where that the database resides if you plan on putting many records into the system.

Here are the four APIs that I am concentrating on:

Basically two ways to Get, by Name (AppID) and Value (CommonID)

One way to delete by Name (AppID) which deletes the Pair

One way to Create by Name (AppID) which can generate a Base64 encoded GUID for the Value so that you can simply use it as a flag to indicate that you saw the value.

                PENDING-> How to use the CrossReferencing APIs to guarantee unique messages!

Microsoft.BizTalk.CrossReferencing Namespace (some of the interesting APIs, and YES I expect someone to read about them)

public static string GetAppID (

       string idXRef,

       string appInstance,

       string commonID

)

public static string GetCommonID (

       string idXRef,

       string appInstance,

       string appID

)

public static bool RemoveAppID (

       string idXRef,

       string appInstance,

       string appID

)

public static string SetCommonID (

       string idXRef,

       string appInstance,

       string appID,

       string commonID

)

How to use these APIs?

Below is a representation of the table structure:

Ok, so you have to have an AppType, it is required for the AppInstance and xRef datatable.

• ListOfAppType.xml

    <appType>

        <name>ApplicationOne</name>

    </appType>

Then you create the App Instance that has one and only one AppInstance per AppType and is never used anywhere else.

• ListOfAppInstance.xml

    <appInstance>

        <instance>ApplicationOne_01</instance>

        <type>ApplicationOne</type>

    </appInstance>

Now we start on the right side of the tree that contains the "Group" values, idXRef

• ListOfValueXRef.xml

    <valueXRef>

        <name>Credit Term</name>

    </valueXRef>

Then we build the common table that relates idXRef (Group) to AppInstance (Application)

• ListOfValueXRef_Data.xml

    <valueXRef name="Credit Term"> <- This is the Group

        <appType name="ApplicationOne"> <- This is the Application

            <appValue commonValue="Credit Term 1">001</appValue>

            <appValue commonValue="Credit Term 2">002</appValue>

            <appValue commonValue="Credit Term 3">003</appValue>

            <appValue commonValue="Credit Term 4">004</appValue>

        </appType>

    </valueXRef>

The AppValue is the Name field and the CommonValue is the V alue field.

In the application I wrote:

Group: Credit Term

Application: ApplicationOne_01

Name: 001

Value: Credit Term 2

To create or update a value in the table:

if (CrossReferencing.GetCommonID("Credit Term", "ApplicationOne_01", "001").Length > 0)

{

CrossReferencing.RemoveAppID("Credit Term", "ApplicationOne_01", "001");

}

TheReturnString = CrossReferencing.SetCommonID("Credit Term", "ApplicationOne_01", "001", "Credit Term 2");

TheReturnString will equal "Credit Term 2" if successful.

How do I use this in an Orchestration?

One way to use this is to drive your Debugging code at run time.

For instance, I created the following entries:

<Setupfile.xml>

<?xml version="1.0" encoding="UTF-8"?>

<Setup-Files>

    <!--btsxrefimport –file=setupfile.xml-->

    <App_Type_file>ListOfAppType.xml</App_Type_file>

    <App_Instance_file>ListOfAppInstance.xml</App_Instance_file>

    <IDXRef_file>ListOfIDXRef.xml</IDXRef_file>

    <IDXRef_Data_file>ListOfIDXRefData.xml</IDXRef_Data_file>

</Setup-Files>

<ListOfAppType.xml>

<?xml version="1.0" encoding="utf-8"?>

<listOfAppType>

    <appType>

        <name>Neudesic.Integration.BizTalk.GE.InvXRef</name>

    </appType>

</listOfAppType>

<ListOfAppInstance.xml>

<?xml version="1.0" encoding="UTF-8"?>

<listOfAppInstance>

    <appInstance>

        <instance>Orchestration.Variables</instance>

        <type>Neudesic.BizTalk.GE.Variables</type>

    </appInstance>

</listOfAppInstance>

<ListOfIDXRef.xml>

<?xml version="1.0" encoding="utf-8" ?>

<listOfIDXRef>

    <idXRef>

        <name>Acetta.Janus.Integration.BizTalk.GE.Invoicing</name>

    </idXRef>

</listOfIDXRef>

<ListOfIDXRefData.xml>

<?xml version="1.0" encoding="utf-8" ?>

<listOfIDXRefData>

    <idXRef name="Neudesic.Integration.BizTalk.GE.Invoicing">

        <appInstance name="Orchestration.Variables">

            <appID commonID="true">Debug</appID>

            <appID commonID="ReceiveInvoiceFromSalesforceAndTransform">DebugCategory</appID>

        </appInstance>

    </idXRef>

</listOfIDXRefData>

Now in the expression shape access the Debug and Debug category Variables:

Finally, let's use it to determine that Debugging is enabled and we should print out to the Debug Console:

How about Maps?

The CrossReference APIs were designed with Map usage in mind.

Here I set values that I need for BAPI calls to SAP:

<?xml version="1.0" encoding="utf-8" ?>

<listOfIDXRefData>

    <idXRef name="Neudesic.Integration.BizTalk.GE.Invoicing">

        <appInstance name="X_InvoiceAPApprovedToBAPI_ACC_INVOICE_RECEIPT_POST">

            <appID commonID="2300000000">SAPDocFormatString</appID>

            <appID commonID="SFDOCNO000000000">REF_DOC_NO_FormatString</appID>

            <appID commonID="BKPFD">OBJ_TYPE</appID>

            <appID commonID="SAPDEP004">OBJ_SYS</appID>

            <appID commonID="42">COMP_CODE</appID>

            <appID commonID="T1">DOC_TYPE</appID>

        </appInstance>

    </idXRef>

</listOfIDXRefData>

This returns the OBJ_SYS parameter that varies from Production and Development, this means I don't have to recompile the Map between production and development:

At the end of the day the CrossReferencing API's and using them effectively will reduce custom code and recompilation in your Maps, Orchestrations and pipelines.