Simon Evans' Blog

Issues using Visual Studio 2008 with XML Spy integration

simon.evans — Tue, 11 Mar 2008 21:12:00 GMT

Today I spent several hours of my life trying to figure out what had broken the design view and CSS features of web pages in my installation of Visual Studio 2008. After a process of elimination, starting with uninstalling the Silverlight 2.0 SDK bits off my machine, I found that the culprit for rendering the excellent design features of Studio useless was the XML Spy 2008 components for Visual Studio. On the plus side, uninstalling the offending XML Spy pieces from Visual Studio fixed the issue without me having to completely reinstall Visual Studio, but nevertheless this was a frustrating few hours that I hope no one else has to endure.

Consuming Services Using Silverlight 2.0

simon.evans — Thu, 06 Mar 2008 20:15:00 GMT

I never really got that excited about Silverlight 1.0, mainly because whilst it had a great core graphics engine and did video streaming very well, it's feature set was just not rich enough to make it really useful to applications that demanded deep functionality.

One of the features that was clearly lacking was connectivity; you could not directly consume services; the main workaround was to use the connectivity features of ASP.net AJAX to consume services from Javascript. Thankfully, this problem has been rectified in Silverlight 2.0. Alongside other new features in Silverlight 2.0, I am now much more interested in what I can do with the technology.

When calling any service from within Silverlight 2.0, you have to call the service asynchronously (like AJAX). The reason this is important is because of the nature of the browser itself, which ultimately is the host you are running in.

There are two sets of services you need to consider consuming from Silverlight: services that are self describing (such as SOAP / WSDL and RSS / ATOM) and services that are not (such as REST and POX). Silverlight 2.0 can consume all of these types of services, but it does so in distinctly different ways.

The easiest services to consume in Silverlight 2.0 are SOAP based services (something sure to infuriate all the RESTafarians out there!). They are consumed in Silverlight using a WCF service proxy configured to the BasicHttpBinding (BasicProfile 1.1). They are easiest to consume, because it is possible to auto generate a service proxy from the WSDL. Unlike AJAX, XML is also the preferred encoding over JSON, because the Silverlight runtime has good XML performance in managed code unlike Javascript. When returning XML, Silverlight gives you two main approaches to parsing the asynchronous result (returned via an event handler): either using LinqToXML or using the XmlSerializer to deserialize the result into a predefined type. SOAP faults, however are not supported, because of security restrictions on the browser, which could cause some issues to existing services that use the fault mechanism to return exceptions.

Other services (such as REST) are a little harder to consume, and the one thing that did surprise me was that there is no support for the auto proxy generation used by ASP.net AJAX against the WebHttpBinding. You have to construct a Uri string manually and call the service either by using the WebClient class in the case of HTTP GET resuests (REST), or by using the HttpWebRequest class for other HTTP verbs. If the service uses JSON encoding, parsing the response can be achieved in one of two ways: either through WCF's DataContractJSONSerializer (similar in concept to the XmlSerializer), or by using LinqToJSON, which you can find here:

http://www.codeplex.com/Json

Whilst XML seems to be the first citizen of Silverlight, it is important that JSON encoding is supported easily, as many services you write may need to be consumed from both an AJAX enabled page and a Silverlight control. In these circumstances, you would definitely opt to use JSON, because the performance of XML in Javascript is poor, and the payload size is also bulkier.

RSS and ATOM feeds are consumed using a mixture of the two methods above; you need to manually construct a Uri and pass it into a WebClient which calls the feed, but Silverlight does provide you with a SyndicationFeed class to parse the response.

One final point to make is around security. As with AJAX, there is a difference between private services that are written and consumed only by your application and public services that are consumed by multiple applications across the internet.

For privates services, Silverlight uses the ASP.net forms authentication mechanism (based around an authenticated cookie) to secure your services from other clients. This means that there is no additional code (on top of your ASP.net code plus config) to write in order to secure your own services.

For public services, the same cross domain security issues apply as they do in the AJAX world. Silverlight supports its own ClientAccessPolicy.xml format as well as Flash's CrossDomain.xml file format to resolve this problem. So long as the service you call has one of these two files in its root, Silverlight will be able to call the services cross domain.

Consuming JSON enabled WCF 3.5 services using ASP.net AJAX

simon.evans — Thu, 06 Mar 2008 06:03:00 GMT

There are several blog posts out there that cover the subject of how to consume JSON enabled services from Javascript using ASP.net AJAX. Having scouted around the usual suspects, I think the best article is by Fritz Onion here:

http://www.pluralsight.com/blogs/fritz/archive/2008/01/31/50121.aspx

That said, I haven't found it all plain sailing, so I thought I'd write a blog entry covering the issues I have had to overcome in order to get this functionality working.

Service contract namespace issues

You will see from the article above that setting the namespace in the service contract attribute affects the Javascript namespace ASP.net's auto generated Javascript proxy uses to call the service. So, for example, when you use the namespace http://schemas.conchango.com/myservice/, you would instantiate a client proxy in Javascript using the following code:

var proxy = new schemas.conchango.com.myservice.IMyServiceContract;

where IMyServiceContract is the name of the service contract you are instantiating. Notice that the forward slashed are turned into dot notation by the Javascript proxy generation (and the final forward slash does matter!).

Now this all makes a lot of sense so far, until I introduced schema versioning into my namespace:

http://schemas.conchango.com/2008/03/myservice/

Versioning a schema in this way is standard practice for self describing SOAP services that expose a WSDL. It is important to ensure that you can version your service contract safely, ensuring that you don't break any consuming clients when you update your service to include new and changed functionality. It is also fundamental to publish-subscribe middleware, such as BizTalk understands who subscribes to the published message.

Here a hardnosed RESTafarian may well point out that my WCF endpoint exposed using a WebHttpBinding is indeed not a SOAP service, so this is not important. And they are right, so long as I never want to expose my WCF service through an additional SOAP based endpoint (using a binding such as BasicHttpBinding). This is one on the single most important principles of WCF and Service Orientation: to be able to expose a single service, hosted as a single instance through different endpoints so that you can communicate using different transports, encodings and security. In other words, services should be policy driven.

So I did work around the problem; when I need to version my schema I would use a new namespace such as:

http://schemas.conchango.com/myservice2/

I think this is acceptable, although not standard schema naming practice. At least I can now version my schemas properly and expose my service through SOAP and REST at the same time.

Debugging issues with IIS

In order to use ASP.net debugging with ASP.net websites and services hosted in IIS, Visual Studio requires Integrated Windows Authentication to be enabled. However, if you are developing a web site for anonymous access, you will need to enable anonymous access in IIS too. The problem here is that WCF only allows one authentication scheme to be used. If you attempt to debug a service in IIS under these conditions, you will see the following error message:

IIS specified authentication schemes 'IntegratedWindowsAuthentication, Anonymous', but the binding only supports specification of exactly one authentication scheme. Valid authentication schemes are Digest, Negotiate, NTLM, Basic, or Anonymous. Change the IIS settings so that only a single authentication scheme is used.

The simplest workaround to this problem is to switch from using IIS to Visual Studio's development web server. Otherwise I found that I was limited to running the site in IIS.

LINQ to SQL: Let the debate begin

simon.evans — Sun, 11 Nov 2007 14:15:00 GMT

So I just got back from TechEd Barcelona, and I had a great conference in no small part due to the people I went with: Merrick Chaffer, James Dawson and Paul McMillan. This is a selection of some of Conchango's finest from the world of .Net development, infrastructure and SQL Server. One of the reasons the conference was so much more enjoyable in the company of my colleagues was that I got as much out of the conversations we had outside of the sessions as I did from what I learnt in the sessions and labs.

Of all the subjects covered at this year's TechEd, none had more coverage than LINQ, and between us, none had more debate than the impact LINQ has on our worlds. This is a conversation I can see playing out across the Developer / DBA divide the world over, as the majority of solutions developed require some sort of data access to a database. David Chappell made a comment in his REST vs SOAP session (which was probably the best session I went to) that struck a real chord with me; he said "people are only passionate when something is in doubt" Conversations of this nature can tend to take a slightly religious standpoint, so I'll try to bear in mind David's comments.

So with LINQ to SQL (and LINQ to Entities) the most hotly debated point was around the auto generation of LINQ's dynamic SQL vs. using stored procedures. In the old days, one point favoring of using stored procedures was the performance gain from a cached execution plan. But since SQL Server 2005, dynamic SQL is also cached, so this reason for using stored procedures has largely gone away. One benefit dynamic SQL has over stored procedures is that you only select what you need in all cases. In a typical scenario with stored procedures and a DAL populating an object model, stored procedures will tend to be reused even when all of data is not used by the calling DAL method. There ways around this, but it largely involves an ever increasing list of stored procedures (GetObjectByX), make the solution less manageable.

What became clear in our debates on this topic is that really all depends on who you trust more: a developer writing a stored procedure or the LINQ to SQL engine? I began the week at TechEd a skeptic of LINQ to SQL, but the more I looked at what the engine produced, the more I began to trust it. Sure it won't do a better job than a highly trained SQL expert (like Paul), but is it good enough and better than your average developer? From what I have tried I would say the answer is yes. And the bottom line is, when you need that extra performance out of a finely tuned stored procedure, you can still use them.

Dealing with Message Ordering in Service Orientated Architectures

simon.evans — Wed, 10 Oct 2007 14:26:00 GMT

Message ordering is a key concept in a service orientated architecture. It is a key concept, because one of the core tenets of SOA is that a service must remain autonomous. In remaining autonomous, a service must be able to deal with any unexpected circumstance of messages sent to it, including the possibility of messages being received out of order.

There are two core strategies you can apply when designing your service: you can either force your client to send messages in the correct order, or you can enable messages to be processed out of order and implement mechanisms within your service to deal with the consequences of this. There are strengths and weaknesses to both approaches. In general, imposing ordering has a cost to the performance of the solution, because managing ordering requires you to manage the state of order. However, removing ordering requires the service to respond to the consequences of receiving the messages out of order. This has an additional development cost, which normally requires the development of compensating transactions.

When discussing the strengths and weaknesses of imposing ordering, the decisions have to be considered for two distinctly different scenarios: homogeneous and heterogeneous ordering.

Homogeneous message ordering

Homogeneous message ordering is ordering of messages with the same schema. For example you may have a message to update a customer. In this example, it is important that ordering is considered, because messages being received out of order could result in a customer being updated in an incorrect state (because the final update to the customer has been overwritten by an earlier message which has been received out of order).

Imposing ordering of such messages can be achieved in WCF using a binding that supports ordering, such WsHttpBinding (this uses the WS-ReliableMessaging policy). Likewise, BizTalk can support ordering through use of correlation. However in such ordering scenarios, be mindful of the fact that both the client and the endpoint must be order aware. So for example, if BizTalk (as the client) correlates messages being sent to a web service that is not order aware, no guarantee can be made on the order the messages are processed by the endpoint.

Enabling a service to receive homogeneous messages out of order means that the endpoint must be able to cope with messages that have expired; the service must cope with the fact that the inbound message has already been superseded by a newer message. An expired message is known as a "zombie" message. The simplest way to ensure that the endpoint is able to evaluate the message as older than the previous message is to include a published date (date time) on the outbound message. If the endpoint persists this published date as stateful data, the service can then evaluate the previous published date for the message and ignore older messages.

Heterogeneous message ordering

Heterogeneous message ordering is ordering of messages of different schemas that have a relationship to each other. For example you may have a message for each customer and each customer may have many bank accounts, each of which constitutes a separate message which is related to a parent customer message.

Imposing ordering of such messages means that the publishing system needs to maintain some understanding of related messages that have been previously sent. In BizTalk, this is achieved using correlation which requires messages to expose distinguished fields that will be used for correlation.

Whilst imposing ordering is achievable its performance cost can be avoided if the endpoint is designed to be order agnostic. When an endpoint is order agnostic, it is able to receive heterogeneous messages in any order and tie the messages together within the endpoint. So for example, the endpoint would accept an account for a customer it has yet to receive the message for. To enable the endpoint to correlate these messages, the customer and account messages would need to contain enough information about the relationship between these two entities. The endpoint then needs to be designed to be able to accept these messages in either order and deal with the consequences of this.

By enabling an endpoint to receive messages out of order, the service endpoint must also deal with the consequences of one these messages never arriving at the endpoint. So, for example, if a customer has two accounts and only one them is received by the endpoint, it may be important to deal with this account having gone missing. This scenario would require compensation.

Compensation is some form of action that compensates the publishing system when a failure has occurred. Compensation can take many forms from the simple to the sophisticated, depending on the requirements and the business value. Indeed, in some scenarios, it may be deemed reasonable not to provide any form of compensation at all. A simple form of compensation might be to email an administrator to tell them in invoke a manual procedure. A sophisticated form of compensation might be to send a compensating message back to the publishing service which would then rollback the state of customer in that system.

A comprehensive guide to using MsmqIntegrationBinding with MSMQ 3.0 in WCF

simon.evans — Mon, 17 Sep 2007 12:52:00 GMT

Overview of WCF architecture

Figure 1 illustrates the generic architecture WCF employs for all services.

Figure 1 - General WCF Architecture

The outer service represents the AppDomain of the service. A WCF service consists of contracts (data, fault and service contracts), a service instance (implementation) which is the implementation of a service contract, and configuration which contains the service endpoint information used at runtime to create the channel stack and the service instance.

A WCF service endpoint consists of two core elements: the service contract and a binding. A binding describes how the service will communicate with clients at runtime. WCF provides several default bindings out of the box, covering many common communication scenarios such as BasicHttpBinding and MsmqIntegrationBinding. In addition to creating bindings from scratch, you can also extend and customize the default bindings to work as you wish.

Service Instancing and opening communication

Figure 1 illustrates what happens when a WCF service is run. Ultimately, the aim of the service is to map messages from clients to operations within the service instance. The service instance requires a runtime host and the host must be opened for communication to occur. The ServiceHost<T> class represents this host in code. When the host's constructor is run, it uses the service endpoint configuration combined with the contracts defined in assemblies (retrieved using reflection) to create a ServiceDescription. Thus, the Service host description contains the complete description of the service, including what contracts to use, what service implementation to use for the service contract and how the service will communicate with the outside world (binding).

When the ServiceHost<T>.Open() method is run the Service Description is used to create a Channel Stack and a Service Instance. The Channel Stack is the runtime equivalent of the Binding information. The stack contains all the protocols required in order for the communication. As a minimum for any binding, this must contain at least a Transport and an Encoding. Additional protocols (such as Sessions and Transactions) can then be applied on top. For more information on the channel stack architecture, look at the following MSDN article: http://msdn2.microsoft.com/en-us/library/ms729840.aspx.

Once the service host is open, the service is ready to receive incoming requests from a client. The inbound message gets received through the channel stack and the Dispatcher is then responsible for assigning the message to the corresponding Service Instance operation, deserializing the message using the appropriate deserializer and executing the operation on the service instance. For more information on serialization , look at the following blog entry: http://blogs.conchango.com/simonevans/archive/2007/05/14/WCF-serialization-with-MSMQ.aspx.

Behaviors in WCF

Behaviors are a way of changing the default way in which either a service instance, or operation will execute when run. Behaviors either inherit from interface IServiceBehavior or IOperationBehavior. They are attached to the service description when the host is constructed, using a mixture of configuration, and/ or any additional code you choose to add. Whether you add a behavior in code or in configuration depends on whether you want the behavior to be configurable, or if you need to behavior to be permanently applied.

Examining runtime metadata using InstanceContext and OperationContext

Whilst the service instance is running, the InstanceContext class provides you with the ability to examine metadata about the instance at runtime, such as information about the channel stack and the host. The lifetime of the InstanceContext is aligned to the life time of the service instance. Likewise, the OperationContext provides you with runtime access to service operation metadata for the lifetime of the call to the service operation. OperationContext provides you with access to information such as message headers and properties for inbound and outbound messages, and the Dispatcher.

Handling Concurrency

InstanceContextMode and ConcurrencyMode are service behavior attributes applied to the Service Description when the host is constructed. Together, these two attributes are used to configure how a service will control the lifetime of service instances and how many messages a dispatcher will hand over to an instance.

The InstanceContextMode controls the lifetime of a service instance. By default, this is set to PerSession. This means the service instance will last for the lifetime of the corresponding Session (which equals PerCall if no session is available). PerCall instancing means the instance will be destroyed after the service operation completes. The final option here is Single, which means that the instance context will only be destroyed when the host is closed.

The ConcurrencyMode handles how many messages the dispatcher will hand over to the service instance. The default is Single, which means the service instance runs single threaded, but this can be changed to run multi threaded by using the setting Multi.

Additional to InstanceContextMode and ConcurrencyMode, the ServiceThrottlingBehavior controls how many concurrent service instances will be run by the host at one time (the MaxConcurrentInstances property), and how many concurrent calls can be made to a single instance (the MaxConcurrentCalls property). It is worth noting with this final property that the channel stack will automatically ready the next inbound message even when concurrent calls are set to 1.

All of these behaviors together control how concurrency and threading will be handled within a WCF service. Figures 2 to 4 below show how these variables can affect threading and concurrency in WCF.

Figure 2 – PerCall instancing, Single Threaded with 2 concurrent instances

Figure 2 shows a service that allows two concurrent service instances to run, each of which is destroyed when the call to the operation is complete (PerCall). Each service instance is configured to only allow one thread to run at a time. This means that of the three inbound messages (shown on the left), the dispatcher will handle two currently using two service instances in the process. The third message will be made ready by the channel stack for handling by the dispatcher.

Figure 3 – PerCall instancing, Single Threaded with 1 concurrent instance

Figure 3 shows what happens when the concurrent instances property is set to 1. The result is only one service instance is created and runs single threaded. A second message is made ready for the dispatcher in the channel stack and is not processed until the service instance is destroyed at the end of the first operation call.

Figure 4 - Single Instance, Multi threaded with 2 concurrent calls

Figure 4 shows how using multi threaded concurrency with a single instance would affect how the service behaves with two concurrent calls allowed. The top two inbound messages from the channel stack would make a call to an operation on a single service instance concurrently. This service instance would not be destroyed at the end of the call to the operation. The third message would be waiting in the channel stack, and as soon as the service instance made a thread free for use, the dispatcher will call the operation for the third message against the same service instance.

Service Exceptions

When a service instance throws an unhandled exception, the InstanceContext is said to be in a Faulted state, and the Faulted event will be raised. This event can be handled in the service host. Once the host is in a faulted state, no further service instances will be created until the channel stack is taken out of a faulted state. This can be achieved by closing and reopening the host (services hosted in IIS automatically handled restarting the host).

Using MSMQ with WCF

Why use MSMQ for communication?

MSMQ provides core features which can be employed to solve various architectural problems. It is a store and forward mechanism for messages, and provides core features such as durability, exactly once delivery and transactional support. MSMQ should be employed in situations where you need a buffer in communication between two endpoints, such as cases where a service intermediary is required to deal with service (un)availability and reliability.

Before looking at using MSMQ with WCF, it is important to understand some core concepts of MSMQ, particularly when it is employed in a distributed architecture, where endpoints are running on different servers. These concepts are largely abstracted away from you writing service code, but their impact is important to understand.

Firstly, the term store and forward refers to how MSMQ will send a message across machine boundaries. When a message is sent from one machine to another, MSMQ will send the message to an outbound local queue and then forward it onto the remote queue on the other machine. If the remote machine is offline, the message will remain on the local queue for a period of time until the remote queue becomes available, or the attempt to process the message times out. The store and forward mechanism deals with the requirement that the remote machine may be offline when the message is sent. If the timeout for processing the message from outbound local queue expires, the message will be removed from the local queue and can be optionally moved to a dead letter queue. This queue is the final resting place for a message ensuring that the message has not been lost.

The second concept to understand is durability. During the store and forward mechanism a message can optionally be persisted to disk. The reason for doing this would be to recover from machine failures and reboots on either the outbound queue or the remote queue.

The next concept is exactly once delivery. This ensures that an individual message is assured to be sent exactly once. This assurance should be used when it is important not to receive duplicates of the same message.

Finally, MSMQ can optionally support transactions either homogeneously using a local MSMQ transaction manager, or heterogeneously using DTC. When transactions are required, MSMQ requires that the message uses exact once delivery with durability in order to achieve the requirements of an ACIDic transaction.

It is worth also understanding the transport that MSMQ uses for sending and receiving messages. Messages are sent using TCP/IP and received using RPC. As RPC is a heavy weight protocol, it is seen as best practice to send to a remote machine queue, but receive from a local queue. Indeed remote receives on a transactional queue are not supported in MSMQ 3.0, because of the performance implications of doing so.

MsmqIntegrationBinding in Detail

MsmqIntegrationBinding is an out of the box binding for communication with MSMQ alongside NetMsmqBinding. The reason that WCF provides two bindings to communicate with MSMQ is that NetMsmqBinding will only work in pure WCF only architectures, where all clients and services are WCF based. It does however provide a much richer channel stack, including support for Sessions, which in turn gives you greater transactional control.

If you are working in a heterogeneous architecture where WCF is interoperating with other MSMQ based clients and services, MsmqIntegrationBinding is the only option available to you. MsmqIntegrationBinding is limited in the communication policies it can use, and it also requires service contracts specifically designed for use with the binding. Figure 5 below shows the WCF architecture from the point of view of MsmqIntegrationBinding.

Figure 5 - WCF architecture using MsmqIntegrationBinding

MsmqIntegrationBinding requires contracts that are designed only for its use. Unless additional behaviors are applied to the dispatcher, the service contract can contain only one service operation. For more information on extending the service contract in this way, look at the following link: http://msdn2.microsoft.com/en-us/library/ms752265.aspx. Service operations for MsmqIntegrationBinding can contain only one argument typed as MsmqMessage<T>, where T is the data contract used in the MSMQ message body.

MsmqIntegrationBinding is also limited in terms of what policies can be applied in the channel stack. The Transport used is TCPIP for sending messages and RPC for receiving messages, and it is only at the transport level can security be applied (there is no message level security available).

Beyond Transport and Encoding policies, the only optional addition to the channel stack is transactional support. For MsmqIntegrationBinding, transactions have to use auto completing transactions. This is achieved by applying an operation behavior to the service implementation with attributes TransactionScopeRequired and TransactionAutoComplete set to true. This means that the transaction (with MSMQ) will automatically be committed if the service operation completes without an unhandled exception. This is limited transactional support because it means your transactions cannot flow beyond the service operation you are running into other service operations or behaviors. The reason MsmqIntegrationBinding does not support explicit transaction control is because explicit transactions in WCF require Sessions to be enabled, which are not supported by this binding.

MSMQ comes in various different versions. MsmqIntegrationBinding supports both version 3.0 and 4.0. Version 3.0 is provided with Windows XP and Windows Server 2003. MSMQ v4.0 is provided with Windows Vista and Windows Server 2008; it provides richer functionality for custom dead letter queues and poison messaging (see later).

The following table lists key properties exposed by the MsmqIntegrationBinding, how these are used and what versions of MSMQ will support this property:

Property name	What does it do?	MSMQ version
CustomDeadLetterQueue	Specifies the name of the custom dead letter queue to use.	4.0 only
DeadLetterQueue	Specifies whether to use a system, custom or no dead letter queue.	All (3.0 does not support custom dead letter queues).
Durable	Specifies whether MSMQ will persist the message to disk, making it durable to server shutdown.	All
ExactlyOnce	Specifies whether MSMQ will ensure that a message is delivered exactly once.	All
ReceiveRetryCount	Specifies the number of times the channel will retry attempting to process the message from the queue (executing the service operation). This is used for poison message handling (see below).	All
ReceiveErrorHandling	Specifies how the channel will respond to a message which fails to be processed once all retry cycles are exhausted. This is used for poison message handling (see below).	All (3.0 only supports Drop and Fault values).
MaxRetryCycles	Specifies how many retry cycles are used in an MSMQ 4.0 retry queue.	4.0 only
RetryCycleDelay	Specifies a delay between retry cycles used by the MSMQ 4.0 retry queue.	4.0 only

Message processing failure

When an MSMQ message is processed by the channel stack and handed to the service instance from the dispatcher, the service operation must cope with exceptions. These exceptions can be separated into two distinct groups: expected exceptions and unexpected exceptions. For unexpected exceptions, WCF provides a built in mechanism called Poison Message Handling, which is covered later in this document.

Expected Exceptions

A general principle of all message based architectures is to handle the message as quickly as possible to a final destination (normally your endpoint). This is important to remember when dealing with expected exceptions. These are exceptions raised within the service operation that are expected to happen under certain known conditions. Expected exceptions are best dealt with explicitly in code, with a definitive outcome for handling the exception. For example, a message may contain data which is invalid for the service implementation's business rules. In this example, the message would never be allowed into the endpoint, so the message needs to be routed to another destination, such as another queue.

The best way to handle expected exceptions is to catch the typed exception in a try catch block, as place the action for the exception in the catch block. With MsmqIntegrationBinding it is important to remember that transactional messages can only be processed using automatic transactions, which means that your handling code needs to participate in the outer transaction if you want the message to be moved off of the queue. The metadata about the MSMQ message can be retrieved using the OperationContext.IncomingMessageProperties collection, which for MsmqIntegrationBinding are typed as MsmqIntegrationMessageProperty. The following code is an example of such a handler for an expected exception used within a service operation that executes in an automatic transaction:

try

{

// do service operation logic here:

}

catch (MyValidationException)

{

MsmqIntegrationMessageProperty integrationMessageProperty = operationContext.IncomingMessageProperties[MsmqIntegrationMessageProperty.Name] as MsmqIntegrationMessageProperty;

string label = integrationMessageProperty.Label;

if (integrationMessageProperty.Body != null)

{

using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required))

{

using (MessageQueue badQueue = new MessageQueue(badQueuePath))

{

badQueue.Send(integrationMessageProperty.Body, label, MessageQueueTransactionType.Automatic);

scope.Complete();

}

Poison Message handling for Unexpected Exceptions in MSMQ 3.0

MsmqIntegrationBinding includes a built in mechanism for dealing with unexpected exceptions called Poison Message Handling. MSDN includes a lot of useful information about poison messaging at http://msdn2.microsoft.com/en-us/library/ms789028.aspx. More specifically, there is also information about using Poison Messaging with MSMQ 3.0 at http://msdn2.microsoft.com/en-us/library/ms751472.aspx. There are however quite a few missing details from these articles.

MsmqIntegrationBinding exposes properties called ReceiveErrorHandling and ReceiveRetryCount which deal with the number of times the channel stack will attempt to process the message from the queue and what happens when this retry number is exceeded. In MSMQ 3.0, the only options for dealing with poison messages is either to Drop the message (remove it from the queue and ignore) or Fault, which will cause the service instance to be put in a Faulted state. When Fault is chosen for error handling, WCF will throw an MsmqPoisonMessageException, which can be handled by writing a custom error handler which implements IErrorHandler. This error handler can be attached to the service description (either in code or configuration) by attaching the error handler to a behavior. The above MSDN articles provide code samples for this scenario.

Any error handler that is attached to the service description is executed when any exception occurs within the channel stack. The HandleError method is called, passing in the exception that was thrown. If the error is handled by the error handler, the method returns true back to the channel stack and no other error handler is called. The standard scenario for a handler for poison message exceptions is to move the message off of the main queue to allow other messages to be processed, and reopen the service host so that communication can continue.

It is important to understand the context in which the MsmqPoisonMessageException is thrown. When this exception is thrown, receive retries have been exhausted and the InstanceContext is faulted. This means that communication is stopped in the channel stack and the host is no longer open for communication. The message transaction will therefore rollback and return to the message to the queue. The poison message handler is then able to receive this message from the queue to move it onto another queue so that normal message processing can resume once communication is reopened.

Using a poison message handler needs to be understood from the point of view of concurrency. As the original transaction is rolled back a new transaction is created to move the message from the queue, there is a possibility that other concurrent instance will attempt to process the message off of the queue before the host has been completely closed and the poison handler is run. Therefore, the poison message mechanism cannot guarantee how many times the service operation will be executed before the handler is run. The handler is running under transactional race conditions where other processing service instances might not yet have fully rolled back their transactions. Therefore the handler needs to attempt to receive the message off the queue several times to ensure the message is moved off the queue.

In MSMQ 3.0, the ReceiveRetryCount for each message processed concurrently is maintained in memory in the channel stack and can only be maintained for a maximum of 256 concurrent messages. The current count is available from the MsmqIntegrationMessageProperty.AbortCount property of the inbound message in the InstanceContext. Because this maintained in memory, an MsmqPoisonMessageException will cause all concurrent instances to be destroyed as the host is closed, and there all retry counts are reset. This means that with concurrent instances, it is feasible that messages may be processed more times than their retry count, because the count is reset during processing. This problem is resolved in MSMQ 4.0, where MSMQ stores the retry count on disk outside of the channel stack.

WCF serialization with MSMQ

simon.evans — Mon, 14 May 2007 16:03:00 GMT

By default, Windows Communication Foundation (WCF) uses the DataContractSerializer to serialize and deserialize XML in a service, but this serialization mechanism can be replaced to use the XmlSerializer. The following article on MSDN covers this in more detail:

http://msdn2.microsoft.com/en-us/library/ms733901.aspx

Whilst using the Data Contract attributes is much easier, and many more types are supported for serialization, there are various scenarios where switching to the XmlSerializer can be beneficial, such as:

Migrating services from ASMX services
Tighter control over XML serialization
Using WCF with MsmqIntegrationBinding

The latter of these three examples is core reason why I have spent a lot of time investigating the use of serializers within WCF. Msmq integration binding relies upon the old Msmq API's used by the System.Messaging namespace in order to ensure interoperability with other legacy Msmq endpoints, such as BizTalk Server 2006 (pre R2).

The scenario I have been working with is that I want to use MsmqIntegrationBinding with BizTalk Server 2006 in the short term, and move to BizTalk Server 2006 R2 and NetMsmqBinding in the future. Thus in the future I want to move towards using the DataContractSerializer, but in the short term, I need to support the old XmlSerializer, because MsmqIntegrationBinding uses this serialization method internally.

Getting MsmqIntegrationBinding to work with complex data contracts

I have seen dozens of samples on the internet for using MsmqIntegrationBinding in WCF, and in short the basic mechanics are as follows:

Create a service contract with a one way service operation that takes a single argument MsmqMessage<T>, where T is a data contract.
Ensure the T of MsmqMessage<T> is specified using the ServiceKnownType attribute.

The problem is, all the examples I have seen use a single simple data contract to demonstrate this binding. When a more complex data contract is used, things get more difficult. For example if I have two data contracts A and B, and A exposes B as a data member, the above instructions alone do not work; the message is never received by the service implementation. The reason this would not work is because data contract B is not known to the serializer and thus A could not be serialized.

After much hunting, I found the solution was to implement serialization using the old XmlSerializer, using XmlRoot and XmlArray attributes to decorate each data contract. By using the XmlSerializerFormatAttribute, you don't need to use the data contract attributes at all, as WCF will use the old serializer instead throughout the service contract.

Comparing the DataContractSerializer with the XmlSerializer

In order to ensure a smooth transition from MsmqIntegrationBinding to NetMsmqIntegration binding I have opted to support both serializers in my data contracts, with a view to retiring the XmlSerializer as some point in the future. The problem is, the serializers handle things in different ways. The table below lists the differences:

DataContractSerializer	XmlSerializer	Resolution to serialization differences
If ordering is not specified, the data members are serialized in alphabetical order	The properties are serialized in the order they appear in the class	Ensure the order argument is specified in the data member attribute in the order that the properties appear in the class.
If the data contract contains an array of objects from a different XML namespace, the serialization will automatically set the namespace of the array to be that of the objects within the array.	The namespace of the array will be that of the outer data contract.	Ensure the XmlArray attribute is used against the array property to ensure that the array is serialized the same in both serializers.

Additionally, you must ensure the XSD namespace for each DataContractAttribute and XmlRootAttribute match.

You can compare how the two serialization models match by running svcutil.exe and xsd.exe against the data contract assembly and look at the output XSD schemas.

WCF Contract Design Guidelines

simon.evans — Sun, 22 Apr 2007 21:10:00 GMT

Here at Conchango we have community days every six weeks where we get together to discuss common work interests. As a .Net Architect, one of the core communities I participate in is the .Net community. At our last community day, I presented on Windows Communication Foundation, a technology I have been working with in anger now for over a year.

Whilst the PowerPoint slideshow doesn't really cover any detail on what I presented or demonstrated, I thought I'd attach it to this blog anyway, as this blog covers the contract design guidelines I ran through with the community.

Overview

The following design guidelines can mostly be based around one single important fact about implementing a service orientated architecture (SOA); changing the design of a contract is in an expensive exercise. If changes in contract designs can be limited to a minimum, the cost of ownership of your SOA will reduce, because calling clients have to change less often.

Do all you can to design service contracts to be binding agnostic

This sounds like a simple point to make, but it's an easy one to forget, and difficult to achieve. The bottom line is that to get the most out WCF from an architect's perspective, you really want to try to ensure that non-functional requirements related to your communication needs do not become hard coded into your contract designs; if you can use configuration to drive transport, policy and encoding decisions, the cost of changing these features later becomes much cheaper than changing code.

This can be hard and at times impossible to achieve. For example, if you decide that you need to use MSMQ integration binding, your service contract will need to use the MsmqMessage<T> class as the only argument allowed in your service operation, and thus your service contract is effectively hard wired to this binding type.

However, with other examples, you can design your contracts in an agnostic way. For example, you may choose to use the Session features of WCF, and could use the IsInitiating and IsTerminating properties of the OperationContractAttrbute in your design to enforce the opening and closing of sessions. Whilst this feature is useful, it will tie you into not being able to use BasicHttpBinding, and thus if you decide later that this binding best suits your communication needs, you will need to change the design of your service contracts accordingly.

Design request data contracts to act as an envelope to service requests

As mentioned above, MsmqIntegration binding requires dedicated service contracts using the MsmqMessage<T> class. If you design a service contract for a web service, but later decide to change this into a service that receives a message from a message queue, you will need to use MsmqMessage<T>, where T is the type of a single data contract.

Consider the following service operation:

int MultiplyNumbers(int firstNumber, int secondNumber);

If you choose to implement this using MsmqIntegration binding, you will need to create an envelope data contract, with properties for the first and second numbers. To avoid having to redesign data contracts later, always wrap multiple arguments into a single request data contract, such as:

int MultiplyNumbers(MultiplicationRequestData request);

Fault contracts require two way communication

Obvious when you think about it, but if you expect service faults to be reported to the calling client, the service operation must use two way communication.

Avoid contract namespace conflicts with the application layer

Be careful how you name you data contracts, because you will normally be wrapping an application with its own API, and this can lead to namespace conflicts which require you to use a fully qualified namespace on every call to the type, which mean messy code. On my project, we have adopted a standard of using EntityNameData as the naming convention for data contracts.

Always assign dates to your schema namespaces

This is important for schema versioning in your contracts, and particularly important in you are planning you use your services with a messaging hub, such as BizTalk Server where subscription is driven off of unique schema namespaces.

Only use types that translate to XSD types in your data contracts

Data contracts should only use types that have a direct comparison in XSD, because you WSDL will otherwise contain XSD import statements to Microsoft generated schemas to represent .Net types. This means using arrays instead of collections, and sticking to enumerations, integers, strings and datatimes.

Do not take an Object Orientated approach

Remember when you are designing a data contract that it is a schema, not an object model. This means that whilst the entities in your application are likely to use inheritance, this approach does not make sense for the generated schema you are creating. Always review your generated schema using svc util to ensure that the contracts create a clean WSDL with clean XSD schemas.

Implement ordering in your data contracts

Ordering is an optional parameter of the DataContractAttribute that controls the order of elements as they appear on the XSD schema that is produced as a result of designing a data contract. If you do not specify order, the schema will be generated in alphabetical order. The affect of this is that newly added elements do not necessarily appear at the end of the schema, which can make altering related artifacts (such as BizTalk maps) more problematic.

Data contracts should define type, not business rules

It is not an in built part of WCF of define validation rules within schema (the data contract), but I am commonly asked how to extend WCF data contracts to validate properties. The reason that WCF does not support XSD restrictions through data contracts out of the box is that implementing business rules through schema design is an expensive way to implement business rules, because if a rule changes, the schema must be changed, and thus calling clients must be made aware of the change. A better way to implement business rules is within the service implementation itself. The data contract should only define the types allowed to be used by the service. This way, changes in business rules do not affect contract designs and services become cheaper to maintain going forward.

Architecting web sites using Microsoft ASP.net AJAX

simon.evans — Fri, 09 Mar 2007 13:55:00 GMT

Last Tuesday I conducted a presentation at the Microsoft Architect Insight Conference entitled "Architecting web sites using Microsoft ASP.net AJAX". You can download the presentation below.

The aim of the presentation was to get attendees thinking about how they choose to architect aspects of their website with ASP.net AJAX. The presentation was conducted with demonstrations on client and server side data binding patterns, and client and server side authentication patterns. I will be making the code for this available shortly.

It was good to see that the web, user experience and AJAX were well represented at this year's event; all too often these subjects are not considered in terms of architectural considerations, which can easily lead to applications that perform or scale poorly.

Microsoft Architect Insight Conference 2007

simon.evans — Sun, 04 Mar 2007 23:44:00 GMT

So, my blog has been pretty quiet of late. This is mainly due to work (and DIY!) commitments, but it is also due to the fact that I have busy preparing for my ASP.net AJAX tutorial for this years conference, which begins tomorrow (5^th March).

My session, entitled "Architecting web applications using Microsoft ASP.net AJAX" will cover some core architectural considerations in approaching the design and development of AJAX web sites. I will be basing my presentation on an application which I have developed to demonstrate these techniques.

Hopefully, I'll see you there, but if you are not attending, in the next week or so, expect the following content to appear on my blog:

The slide deck from the AIC presentation
A series of blog entries which diarize the development of the application I will be demonstrating
A free download of the application, including source code

I am also going to be doing an ARCast tomorrow with Ron Jacobs, for broadcast at a later date on www.skyscrapr.net, covering some topics around ASP.net AJAX, amongst other things.

Plus, at some point soon, I will be posting several articles on WCF, which is currently my daily food of thought.

The importance of Windows Communication Foundation (WCF)

simon.evans — Tue, 05 Sep 2006 19:18:00 GMT

When I first gained knowledge that Microsoft were developing "Indigo" well over two years ago, I must confess, I missed the point. I first saw demos of WCF at a conference around that time, and my initial reaction was "show me something I can't do already". And yet three months into developing with WCF and I can't stop praising the product. It is one of those rare beasts in technology that just feels so very right to use; and that is in itself a first in communication API's. I don't think anyone was truely happy with either ASMX web services, remoting or WSE.

But saying WCF is great because it feels right is underselling its importance. The real key to why WCF is so important is because the core programming model is neutral to the non functional requirements you have for communcation. This means that you can focus on designing your data and service contracts without having to make prior decisions on which transport, messaging or security protocols and policies you need to adopt.

A key part of successful software development is defined in the Lean Software Development book, which is to "decide at the last responsible moment". This concept states that making a decision too early is risky because you know too few facts to draw sensible conclusions, whereas making a decision too late could mean that you could delay whole parts of the project by doing so.

Using WCF supports making decisions at the last responsible moment, and this is a revolution for communications APIs. For example, selecting how you want to implement communication security becomes a matter of configuration; do you want to secure the communication by using a secure transport (such as HTTPS), or do you want to send the message over TCPIP and encrypt the message and use kerberos authentication? Now, this is just a matter of configuration.

Being able to just configure your service to your communication needs also enables another important scenario. It enables the development of Domain Specific Language tools (DSL) for your communication requirements. A DSL tool for WCF will enable you to use a rich tool to design and configure your service. This is the equivilent to being able to use the Windows Forms designer to design your forms; the increase in productivity is immense. There are already some software factory assets available for WCF. Firstly, you can use the Visual Studio 2005 Class Designer to design you data and service contracts. Secondly, the Service Configuration Editor enables you to manage your service configurations through a user interface. Finally, you can also get a great productivity boost from using the WCF Service Factory, which is currently a community preview release.

The final benefit of WCF is not just in your initial communication design decisions, or in productivity. WCF also delivers major benefits when changes occur in your communication needs. I say when changes occur, because as software developers, we have to accept (even if it is painful) that change happens. So, think about what that would mean if you had a web service that you then needed to turn into something you would use over the LAN using TCP and binary? You would end up rewriting your code to use .Net remoting, which would be a costly excercise that you probably wouldn't undertake, even if TCP and binary messages were the most suitable methods of communication.