Pipelines are a component of Microsoft BizTalk Server that provides an implementation of the Pipes and Filters integration pattern. During the receiving and sending of messages, there are business reasons to perform transformations on messages to prepare them to enter or leave BizTalk Server. Pipelines enable the developer to define a series of transformations that will be performed on a message as it is being received or sent.
A common example is that you may need to compress your data before sending them to target subscriber in order to save server resources such as disk space or bandwidth, through this talk I'm going to discuss how to develop a general custom pipeline component to compress/decompress data. There are two types of pipelines, send and receive, and these match the ports in which they execute. Send pipelines are executed in send ports and in the response portion of a request/response receive port, while receive pipelines are executed in receive locations, and in the response portion of a solicit/response send port. Essentially, receive pipelines are intended to be used to transform messages that are being published to the MessageBox database, while send pipelines are intended to be used on messages which have been subscribed to and are being sent out of BizTalk Server.
| Receive Pipeline Stages |
Send Pipeline Stages |
 |
 |
- Decode: Decrypts or decodes the message data.
- Disassemble: Disassembles an interchange into smaller messages and parses message contents.
- Validate: Validates the message data, generally against a schema.
- Resolve Party: Identifies the BizTalk Server party associated with some security token in the message or message context.
|
-
Pre-assemble: Performs any message processing necessary before assembling the message.
- Assemble: Assembles the message and prepares it to be transmitted by taking steps such as adding envelopes, converting XML to flat files, or other tasks complementary to the disassemble stage in a receive pipeline.
- Encode: Encodes or encrypts the message before delivery.
|
A general pipeline component gets one message from the BizTalk Messaging Engine, processes it, and returns it to the BizTalk Server engine, it implements the following interfaces:
Now let's start developing our compression/decompression custom pipeline component:
- Open visual studio 2010 and create a new class library project then add a reference of BizTalk pipeline " Microsoft.BizTalk.Pipeline.dll" normally found under the "c:\Program Files\Microsoft BizTalk Server 2010\Microsoft.BizTalk.Pipeline.dll".
- Add new class to the project, name it "CompressionPipeline" that implements all the basic interfaces mentioned earlier, additional three class attributes will be added as follows:
[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
[System.Runtime.InteropServices.Guid("AAD4E8D2-3900-4542-88A7-43B6D6FE9080")]
[ComponentCategory(CategoryTypes.CATID_Encoder)]
public class CompressionPipeline : Microsoft.BizTalk.Component.Interop.IComponent,
IBaseComponent, IPersistPropertyBag, IComponentUI
- The "ComponentCategory" associates our component with a specific pipeline design category in the pipeline designer which tells the designer that this class represents a pipeline component.
- The "Guid" attribute supplies an explicit unique identifier to our class for COM registery and uniquely identify our component. Use visual studio tools to generate GUID by choosing "Tools" à "Create GUID".
- The third attribute is used to tell the pipeline designer that this component is encoding component as we are developing a send pipeline.
- Before implementing each interface we will add a resource file that separates resources "Strings, Pictures, Icons, …" from our actual implementation code, add a new resource file, I will add some strings to the resource file that help us implementing our interfaces as can be observed from the below screenshot:
- First, I will implement IBaseComponent interface, which allows other developers and biztalk administrators to get information about our compression pipeline component, I will use the resources created in the previous step as follows:
#region IBaseComponent Members
public string Description
{
get { return CompressionPipelineResources.CompressionPipelineDescription; }
}
public string Name
{
get { return CompressionPipelineResources.CompressionPipelineName; }
}
public string Version
{
get { return CompressionPipelineResources.CompressionPipelineVersion; }
}
#endregion
- We will need an icon for our component so open the resource file again and choose "Images" or "Icons" and copy the desired image from your local disk drive to the resource file. This will drive us to implement IComponentUI as observed from the following code snippet:
#region IComponentUI Members
public IntPtr Icon
{
get { return CompressionPipelineResources.ZIP.GetHicon(); }
}
public System.Collections.IEnumerator Validate(object projectSystem)
{
return null;
}
#endregion
As you can see from the above code that I omitted the "validate" method which verifies that all of the configuration properties are set correctly. So I configured validation to null.
- .Net introduced compression capabilities since version 2.0 as a part of the data streaming framework, there are several frameworks available that give better performance than the one provided by .Net but I prefer to use compression algorithms provided by .Net in order to eliminate dependencies and to simplify deployments. .Net provides two algorithms GZip and Deflate so we want our BizTalk administrators to be able to select the compression algorithm, our custom pipeline contains a string property that holds the definition of the compression algorithm. BizTalk supports the persistence of the user configuration by implementing IPersistPropertyBag interface as follows:
#region -- Properties --
[Browsable(true)]
[Description("Controls the compression algorithm used to encode the output stream.")]
[DefaultValue("GZip")]
public string CompressionType
{
get;
set;
}
#endregion
#region IPersistPropertyBag Members
public void GetClassID(out Guid classID)
{
classID = new System.Guid("AAD4E8D2-3900-4542-88A7-43B6D6FE9080");
}
public void InitNew()
{
}
public void Load(IPropertyBag propertyBag, int errorLog)
{
object val=null;
propertyBag.Read("CompressionAlgorithm", out val,0);
if (val != null) this.CompressionType = val.ToString();
}
public void Save(IPropertyBag propertyBag, bool clearDirty, bool saveAllProperties)
{
object val = this.CompressionType;
propertyBag.Write("CompressionAlgorithm",ref val);
}
#endregion
As you can see that "Load" and "Save" methods are used to persist our compression algorithm type, using string is not the best practice for this functionality as the BizTalk administrator or our users could mistype or misspell the algorithm type. The best way is to use enumeration and limit the user to choose from a dropdown list to avoid any mistyping, doing this is out of my scope though this talk but you can refer to Saravana Kumar's talk " Understanding Design-Time Properties for Custom Pipeline Components.doc (1.22 mb)" also you can download the full resource from here.
- The last step for our pipeline is to implement IComponent interface that does the actual stream compression work:
#region IComponent Members
public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(
Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
if (inmsg.BodyPart != null && inmsg.BodyPart.Data != null)
{
try
{
MemoryStream ms = new MemoryStream();
Stream inStr = inmsg.BodyPart.GetOriginalDataStream();
switch (this.CompressionType)
{
case "GZip":
using (GZipStream zipped = new GZipStream(ms, CompressionMode.Compress, true))
{
inStr.Seek(0, SeekOrigin.Begin);
inStr.CopyTo(zipped);
}
break;
case "Deflate":
using (DeflateStream zipped = new DeflateStream(ms, CompressionMode.Compress, true))
{
inStr.Seek(0, SeekOrigin.Begin);
inStr.CopyTo(zipped);
}
break;
}
inmsg.BodyPart.Data = ms;
inmsg.BodyPart.Data.Seek(0, SeekOrigin.Begin);
return inmsg;
}
catch (Exception ex)
{
throw new BTSException(string.Format(CompressionPipelineResources.CompressionExcpetion, ex.Message));
}
}
return inmsg;
}
#endregion
One important thing here is never to close the message input stream that's why we set the value of "LeaveOpen" of compression stream to "true" in order to leave the stream open after compression.
- The last step is to sign our custom class library "dll" then copy it to BizTalk pipeline components folder which is normally found under the path " C:\Program Files\Microsoft BizTalk Server 2010\Pipeline Components \".
Now our custom compression pipeline component is ready to be used by pipeline designer. The next step is to create a new BizTalk project to use our custom pipeline:
- Create a new BizTalk project then add a new send pipeline item, the pipeline designer will be opened. From the toolbox right click then click "Choose Item", from the tab control choose "BizTalk Pipeline Components", you will find our compression custom pipeline component, choose it and drag the component from the toolbox to the "Encode" area as below:
- The second step is to deploy our custom compression pipeline to our BizTalk server to be ready for use, right click on the BizTalk project and click properties then choose "Deployment" from property page and setup your target server as well as your target BizTalk application as can be seen from the below screenshot:
- You will have to sign the project, click "Signing" and create a new key then save your settings and close the properties, then right click on the project and choose "Deploy".
- Now our pipeline component can be used from BizTalk server, open BizTalk administration console then explore "BizTalk Application 1" you will find our custom compression component exists in pipelines artifacts. Create a new send port and configure it to use our compression pipeline component as below:
We can test our pipeline component by using visual studio in case of BizTalk unavailability by following the below steps:
- Load the custom pipeline project solution into Visual Studio.
- Change the output path for your solution to <Installation Folder>\Pipeline Components. In Solution Explorer, right-click your project, click the Build tab, and then change the Output Path by clicking the Browse button and selecting the <Installation Folder>\Pipeline Components directory.
- Change the start action for your solution. In Solution Explorer, right-click your project, click the Debug tab, click Start external program, then click … and navigate to <Installation Folder>\SDK\Utilities\PipelineTools and choose Pipeline.exe. Under Start Options, enter the command line arguments appropriate for your component for example [<Path>\YourPipeline.btp -d <Path>\YourTestFile.txt -c]
- Set your breakpoints if you want to debug the pipeline component.
- Press Ctrl + F5 to start testing our just F5 to begin debugging.
Now we have finished our compression pipeline component, you can follow the same steps to create decompression for a receiving pipeline also there is a great tool for creating custom pipeline component which adds a new pipeline project template for Visual Studio, you can find it on codeplex "BizTalk Server Pipeline Component Wizard" .
I have attached the full solution that contains both compression/decompression pipelines here [Intellecting.CustomPipelines.zip (161.46 kb)].
Currently rated 4.5 by 2 people
- Currently 4.5/5 Stars.
- 1
- 2
- 3
- 4
- 5