It is very common to scale out web applications by deploying a large array of inexpensive commodity servers as a very economical way to achieve high scalability and availability. However, once you have put this into play for the presentation and business logic layers you get to the thorny issue of partitioning the data that drives the web application. This usually requires plenty of thought. What scheme will you use to partition the data? How will you distribute the data? How will you query across the data now that it is partitioned and federated? Actually there is a lot more to think about but that is not why I am posting.
One thing that often slips under most people's radar is web application session state. In a vast farm of web servers, how do you maintain session state when a load balancer could send you to any one of the servers for each request?
- Make the application stateless
- Maintain state in the browser (e.g. a cookie)
- Maintain state in the URL
- Maintain server affinity with the client using sticky sessions or redirecting the first ever call to a DNS distinguished server (server129.mydomain.com)
- etc.
If none of those work so well in your context you might want to apply the "partition and federate" technique to the session store itself. It is a little known fact that this is supported under ASP.Net 2.0. As an application developer you can obliviously use session state as normal because you have defined your partitioning scheme in code and configured either a SQL Server or State Server farm to scale out session. This is ideal for many scenarios, especially applications that are already built and deployed and don't have the luxury of examining and building their preferred solution from the get-go.
Check it out, it is an option worth remembering:
http://msdn.microsoft.com/msdnmag/issues/05/09/sessionstate/
"ASP.NET 2.0 provides a solution to the problems encountered when scaling up by enabling horizontal scale-out of session state stores through its state partitioning feature. State partitioning enables the session data and the associated processing load to be divided between multiple out-of-process state stores, allowing the session state load to scale as the Web farm grows and the number of concurrent sessions increases. It works by supplying a custom partitioning algorithm to SessionStateModule, which uses the algorithm to determine the state store connection string to be used for the current request based on the session ID. Both the SQLServer and the StateServer providers will then use the appropriate connection string to fetch and save the session..."