|
|
in·te·gra·tion (ĭn'tĭ-grā'shən)
n.
"the combining and coordinating of separate parts or elements into a unified whole"
-
So I (somewhat foolishly, I suspect given my overall fitness) decided to sign up to do the London to Brighton cycle ride. It's Europe's biggest and best fund raising cycling event, with 27,000 people taking part each year.
It's 56 miles in aid of the British Heart Foundation, so by sponsoring me, you'll not only be helping me look after my heart, but you'll be helping the BFH with their much needed, and pioneering work in helping us all stay healthy. I'm sure everyone reading this will know someone who has benefited from the work that the BFH fund. As of the year 2007, heart disease is the leading cause of death in the United States, England, Canada and Wales, killing one person every 34 seconds in the United States, and one adult every three minutes in the UK.
We're cycling as a team - we're called Lance's Love Children, after the cycling guru that is Lance Armstrong. Our sponsorship page can be found here, or you can donate using the widget below.
Please give generously, and remember - every little helps 
Also, please put my name in the comments so that sponsorship money is attributed to me - If I raise more than £150, then I get priority entry to next years ride.
|
-
I was recently asked by a client if the SOA design principles of Service Autonomy and Loose Coupling amounted to the same thing. I thought it was an interesting question, so after some thought, I replied with the following, which I repeat here to help others that may be intrigued by the distinction: The principle of Service Autonomy says that a service should be independent of external dependencies so that it is predictable in it's run-time behaviour, able to be maintained without disrupting other aspects of an organisation's systems landscape, and implementation decisions such as technology choice can be made in the best interests of delivering the service without affecting the usability of the service. On the surface of it, Service Autonomy would seem to imply that there should be no coupling at all. That is to say, that coupling of any kind implies the introduction of a dependency. And this is where the distinction exists. Coupling and Autonomy are not the same thing, but they are very closely related. Coupling is about the way in which dependencies are introduced, and by doing so perfect autonomy is compromised. They feel like the same thing, because a balance must be reached between them and they go hand-in-hand. Within a service boundary, there will naturally exist some tight couplings, and this is not always anathema to Service Orientation. Service Logic will be tightly bound to the service contract in order to implement the appropriate service behaviour, for example. However, it is when we introduce dependencies outside of the service boundary where we should pay careful attention to the manner of coupling. In this context, loose coupling means that the services that are being coupled maintain as much autonomy as possible. This can be achieved by careful service contract design, and through the use of standards for contract implementation, including standardised schemas. So, there is a strong relationship between Service Autonomy and Loose Coupling, they can be thought of as two sides of the same coin, but they are not the same thing.
|
-
I just read a really interesting post that sparked me off on a train of thought that I just had to share. Joe McKendrick writes an SOA blog for ZDNet (amongst other things) and often poses some interesting questions or disseminates some key nuggets of thought from the SOA blogosphere. One of his recent posts talks about how Pfizer have bridged between Service Oriented Architecture (SOA) and Master Data Management (MDM). I found this particularly interesting for two reasons. - The Microsoft ESB Guidance Toolkit has just gone RTM and been published to MSDN. Marty Wasznicky (who led the team that built this) acknowledged at the Microsoft SOA and BP Conference that this Patterns and Practices guidance toolkit was put together following a successful ESB implementation that Microsoft produced using BizTalk Server at Pfizer. Clearly this is a key plank to the SOA that Pfizer is operating.
- Conchango led the way on MDM and it's usage in SOA a couple of years ago. My colleagues Mick Horne, Rob Grigg and Jamie Thompson were working on a large scale integration project in the Oil & Gas Industry. They faced three key problems. They needed to commodify what they were building so that this could be rolled out to other markets, plus they needed to standardise the process of getting data from multiple disparate data sources, and each of these data sources had their own data dictionaries and domain specific languages. Clearly a Service Oriented approach was going to tick the first two boxes, and MDM rode to the rescue for the third problem. They called it Service Oriented Business Intelligence, and coined yet another TLA - SOBI. Their work was published in the January 2006 issue of the Microsoft Architecture Journal and presented to the Microsoft Architects Insight Conference. Jamie published a series of posts giving loads of details into how they solved the problem. You can find a summary post here.
So what does this series of links tell us? That a good idea can be a long time coming? Perhaps. The argument goes that IT, for all it's promise of "agility" "scalability" and every other -ility at the forefront of a CIO's agenda, is still just an industry like any other. So whilst Silverlight can be the darling of rotating text boxes in no time at all, no C level decision maker is going to stake the future of business application construction and data rationalisation for their organisation on SOA and MDM if it might just be the latest great idea that doesn't quite make it into the air. Perhaps this news from Pfizer shows us that it's not only our Oil & Gas client (who were in a unique position of having no other choice but to adopt a SOA + MDM route) who think that SOBI (or SOA + MDM, or one version of the truth plus a single enterprise data dictionary plus composite applications built upon services, or whatever else you want to call it) is an idea that has delivered the -ilities that it promised, and has not only taken off, but delivered down to earth benefits.
|
-
I'm currently attending the Microsoft SOA & BP Conference in Redmond. It's been a fantastic conference so far, with loads of great sessions by industry luminaries. The key news of the conference is the announcement of Microsoft's new wave of products for building Service Oriented Architecture. Will this change the way we build SOA? SOA is not a product. It is a way of building IT systems. That’s the facts of the matter. Consultancy organisations like ours have known this for a long time. We have been building Service Oriented Architectures for our clients in Retail, Media & Entertainment, Financial Services, Consumer Packaged Goods, and Energy for many years. Vendors will often try and sell a product for SOA, maybe it’s an ESB product or an Web Services product, or even claims to be an SOA product. This is not a reality. SOA is not a product. The announcement of Oslo made by Microsoft this week marks the fact that Microsoft have long realised that the tools needed to build SOA must be delivered as part of co-ordinated strategy across their application platform products. The SOA approach is different from n-tier development. The SOA approach says model your individual business process steps as granular reusable services, and expose them using open standards such as Web Services. Then provide business value through the automation and managed execution of business processes that are created through a composition of the services using a Business Process Management (BPM) tool, to be surfaced to end users through the web and on the desktop. An example might be the business process of booking a flight on an airline. The airline provide you, the consumer, access to this business process on their website. The airline implements their business process in their BPM tool as a number of steps. The Web site integrates with the BPM implementation, which executes the process step by step. Each step is provided by a service, such as a service to look up the number of available seats on the flight that you have selected, the service to take payment from your credit card, or the service to actually book your reservation. To date, Microsoft have released 5 versions of Microsoft BizTalk Server, with BizTalk Server 2006 R2 being the latest and greatest version. This has enabled thousands of companies big and small across the world to build SOA solutions. This is accomplished through deep integration with Visual Studio, SQL Server, System Center and the .NET Framework. Many of our customers have received business agility, productivity improvements and real world SOA solutions through the use of this application stack. Oslo marks a real shift in Microsoft’s approach. Currently implementing an SOA requires a lot of heavy lifting and a lot of highly skilled manpower. A lot of this work is related to the separate silos that people have to work in when building SOA. Business analysts using one set of tools, Architects using another, Developers use a third tool set and Operational people use yet another set of tools. Communication between all of these teams tends to rely on paper documentation. What Microsoft have announced is that they will unify the key products that are used to build SOA today by all of these different roles in the IT organisation, so that the tools all communicate with a common set of models. This will deliver productivity and agility improvements to both the business and IT. And, of course, cost savings to boot. Unified modelling is the new aspect promised by Oslo. Models are used to describe the Service Oriented Architecture from multiple points of view. Different views being accessed and created by different people with different tools, but importantly all of these tools and all of these people will be working on the same part of the business solution. So a business process is expressed in business terms by the business analyst, how that process will be implemented in Software will be designed by the Architect, the developer will provide the code, operations can implement the instrumentation and monitoring, and all of this is being done in the tools that these people prefer, and the tools are all communicating with each other through a common model repository. And this all works at run time as well as design time. So, when the operations guy sees that a service is underperforming, the business analyst will also be able to see this in his view, in terms that are important to him, for example that fulfilment of a particular customer’s order will not meet the agreed Service Level. And when a new requirement is expressed by the business analyst, the developer will immediately see that he needs to implement some new features. This is a revolutionary approach to building SOA. The products that will enable this will be the next wave of application platform products coming from Microsoft over the next couple of years. This new wave of products is called Oslo, and will be made up of: - BizTalk Server “6” – Process Server, the hosting infrastructure for not just WF and WCF, but any .net code
- BizTalk Services “1” – Internet Service Bus – publish/subscribe communications beyond the firewall.
- Visual Studio “10” – Development platform
- System Center “5” – Operations – infrastructure support
- .NET Framework “4” – Application building framework that will deliver the guts of these products and the SOA implementations that run on them
These are not necessarily the version numbers of the products that will deliver Oslo, but indicates the next version of the Microsoft application platform. So that's Oslo. I believe it will change not just the way we build SOA, but the way that all business sytems are built in the future. Modelling + Services. It's the future of Information Technology.
|
-
Well it's been quite a while since I last blogged about anything of note, but reading through my RSS subscriptions this morning I noticed that Richard Seroter has written a very handy utility for managing configuration data stored in the SSO database. I tend to use a combination of the BizTalk rules engine and the SSO database to store configuration information for the projects that I work on. I tend to use the rules engine where I wish to have several items returned in one operation rather than a simple name-value pair. I also tend to use the rules engine where it is beneficial to have the data returned as xml. Plus there are performance considerations and versioning implications to consider. The SSO database I find very useful especially for configuration parameters that are environment specific, and I will tend to use this database as the configuration store of first choice. On my current project we are using the Scott Colestock BizTalk Deployment Framework, and we use this for deploying the SSO data. In the past I have built specific utilities for support teams to use to alter configuration data stored in SSO, but this new tool that Richard Seroter has built is a fully comprehensive application for this purpose. I think I'll be making a lot of use of this on my projects in the future. Thanks Richard!
|
-
World rejoice! - Rich Wadsworth has just started blogging. Whilst this experienced .net architect is sure to have loads of interesting stuff to say to the world, he's kicked things off with a post extolling the virtues of Dell's QuickSet utility. And what an excellent subject to start with. As a consultant, I'm often working in different locations at different times and switching proxy servers and default printers can be a time consuming pain. QuickSet does it all automatically for you. So, I recommend you read Rich's post and make your life easier
|
-
When you build a BizTalk 2006 project, you might get build output containing warnings such as: Updating references... The dependency 'Microsoft.BizTalk.CachingService' could not be found. The dependency 'Microsoft.BizTalk.DBAccessor' could not be found. The dependency 'Microsoft.BizTalk.Bam.EventObservation' could not be found. The dependency 'Microsoft.BizTalk.Streaming' could not be found. The dependency 'Microsoft.BizTalk.XPathReader' could not be found. Performing main compilation...
This is because one of the BizTalk assemblies has been added to your project with Copy Local set to true. Most likely this is because you've added Microsoft.XLANGs.Pipelines to your project in order to execute a pipeline from within your orchestration. Go through your project references and ensure that any external assembly references that are in the GAC have Copy Local set to false.
|
-
I've claimed this blog on Technoratio. Visit my profile.
|
-
-
Another one for the online memory... Although the readme for BizTalk Server 2004 states "Only install Service Pack 1 (SP 1) on computers already running BizTalk Server 2004", you can in fact install BizTalk Server 2004 Service Pack 1 before you run configframework.exe So that should say "Only install Service Pack 1 (SP 1) on computers that have BizTalk Server 2004 already installed" Hmm... why are my blog posts always on installation issues? Oh yeah, maybe it's coz installation is a pain and there's too many fiddly bits to remember! I'll blog some BizTalk 2006 dev stuff soon!
|
-
Blogging this one as a reminder to myself, and in the hope that it might help someone out.
Visual Studio 2005, by default doesn't show the solution node in solution explorer if the solution only has one project.
To display the solution node bring up options Tools->Options, under the Projects and Solutions node, on the General page, check the 'Always show solution' box.
|
-
I came across a slightly thorny issue of interdependent hotfixes the other day, so I thought I'd better blog this one, as much as for my own memory as for other’s benefit.
I was installing BizTalk Server 2004 Service Pack 1 on a machine that was built with Windows Server 2003 Service Pack 1 as the OS. The issue flows from the fact that BizTalk Server 2004 Service Pack 1 was released before Windows Server 2003 Service Pack 1.
BizTalk Server 2004 Service Pack 1 has a pre-requisite that KB890673 is installed. The Readme for BizTalk Server 2004 Service Pack 1 refers to this as the "XM GDR Update" and the title of this Knowledge Base article is the rather unwieldy "Availability of the .NET Framework 1.1 Post-Service Pack 1 XML Web services and XML Messaging hotfix rollup package 8".
So effectively in a pre- Windows Server 2003 Service Pack 1 world the installation of this post .NET Framework 1.1 Service Pack 1 update is required.
This update roll-up (KB890673) is, as the name suggests, a roll-up of KB839588, KB842832, KB870722, KB872800, KB884021 and KB887191.
Now, Windows Server 2003 Service Pack 1 includes the .NET Framework 1.1 Service Pack 1, and also includes KB839588, KB842832, KB870722, and KB872800, but not KB884021 or KB887191.
Hmmm, I thought. KB887191 appears to be a fix for Host Integration Server, but KB890673 also mentions that this issue can occur in the .NET Framework more generally. And KB884021 refers back to KB890673.
So I was unsure if the pre-requisite for BizTalk Server 2004 Service Pack 1 was met by Windows Server 2003 Service Pack 1.
I contacted Microsoft Premier Support, and they assisted in unravelling this.
Although the list of included fixes for Windows Server 2003 Service Pack 1 does not mention KB884021 or KB887191 it does mention KB890673.
So, the result:
If you're installing BizTalk Server 2004 Service Pack 1 on Windows Server 2003 Service Pack 1, you don't need to install the "XM GDR Update" (KB890673).
|
-
So, my first proper post... I've recently been working on a way to take a running virtual machine, that has a full, single machine installation of BizTalk Server 2004 (with Sharepoint Integration and everything else), and to clone it so that virtual developer workstations can be provisioned with minimal fuss and hassle. Neat idea I hear you say, so why the blog post? Well, this isn't a straightforward task, as I'm sure you can imagine... Background My current client operates an integration development model where a central team builds, maintains and enhances a shared integration infrastructure, which provides a service to application teams. The application teams design and build orchestrations using the pipelines, visual studio templates, etc. This client also has a fairly complex Active Directory set up, the long and short of this is that the machines that developers use to build BizTalk solutions are in a different domain to the 'normal' (ie .NET) development machines, meaning that a) we can't just wack BizTalk onto their current machine, and b) we have to provision a developer machine relativly often. Also this client has made an extensive investment in Virtualisation technology in the form of VMWare ESX Server, so in theory we can just clone a running machine with BizTalk on and ba-da-bing we have a new developer workstation, right? Read on to find out how much hassle this really was, and for pointers on how to do it yourself... Theory Cloning a virtual machine is pretty simple stuff. If you use Virtual PC, basically all you have to do is copy the folder containing the virtual harddrive and vmc file, rename the vmc file, and double click the vmc to add it to virtual PC console. Voila, you now have two virtual pc's with exactly the same config. Easy? Well no. The trouble is that the config is too much the same. You can't run both at the same time (unless you isolate them from each other) because they'll conflict with each other on the network. Not only do they have the same machine name, but more importantly, underlying both the machine identity and each local user account & group on the machine is a Security Identifier (SID) that must be unique, but is obviously not once you copy the virtual machine. So what is described above is not cloning, but copying. To clone you need to do what OEM's like Dell do, and use Microsoft System Preparation Tool (sysprep). This is included on your OS cd as part of the deploy.cab file, tucked away in the <root>\support\tools folder. Now there's a lot to sysprep, and automated deployment, more of which can be found here, but basically you take a working system, create a config file for sysprep, run sysprep (which culminates in the machine shutting iteself down) copy the harddrive, (in a virtual world you copy the file that is the harddrive for the vm, whereas OEM guys copy the physical harddrive contents from one PC to another.) And when you next boot from that harddrive, Windows will prompt for a new name for the 'clone', and importantly will change all the SIDs to new ones, plus will allow all maner of settings to be changed from the admin password to the locale, either prompting the user for these, or these new parameters can be passed to sysprep in a configuration file. Does booting a PC and entering a load of personalisation sound familiar? Well anyone who's bought a PC with the OS already installed will have gone through this sysprep 'mini-setup' at the first time of booting... So, that's cloning, big deal, could have found out about that anywhere, now why the blog post? OK, sysprep is only really designed to clone the OS. Applications (like BizTalk, in fact especially BizTalk) store values that relate to the old machines identity (such as the machine name, in fact mostly the machine name, but the change in SID is also very disruptive to SQL Server logins) in places that sysprep can't know about (as it's an OS tool), and therefore can't update. So once you've run configframework the above process no longer works. That is to say that the OS clones fine, but BizTalk, Enterprise Single Sign On, Windows Sharepoint Services, to a lesser extent SQL Server, and a whole host of other stuff will not be happy bunnies. Help is at hand What I've done, therefore is to make use of the ability for sysprep to run one or more configurable commands once it completes, and to have it call a batch script that does a whole heap of stuff to 'fix' the clone. Obviously I can't give you the code, but I can point you towards what needs to be done: - There are many locations within the databases used by BizTalk where the old server’s name is hard-coded. I wrote a T-SQL script that searches through the BizTalk databases and replaces any instances of the old server’s name with the new server’s name.
- The same is applied to the Sharepoint databases
- SQL Server Analysis Services uses an mdb file to store its configuration, including the instance name of the SQL Server. I use some VBScript that opens the mdb and replaces the old entries in this file.
- There are many, many places where the old machine name is hard coded in the registry. I painstakingly searched through the whole registry to find each entry, then created a template reg file of all these entries with an identifier instead of the machine name. Then at fix time i use a small utility from the Windows NT 4.0 Server Resource Kit called munge.exe to do a find-and-replace on the template replacing the identifier with the new machine name, then I import this newly created reg file.
- Also, the performance monitor counter entries are stored in the registry as hex values for some reason. I use a VBScript to generate another Registry File that updates the performance monitor counters registry entries.
- The BizTalk BAM components store the server name in a configuration XML file.Again I use munge.exe to fix this.
- This next one caused me a lot of grief. Microsoft: *please* change sysprep so that it is IIS Metabase aware..! Anyway, IIS stores its configuration in the IIS Metabase. This is a hierarchical structure much like the registry. Each entry can have permissions set using an Access Control List (ACL) which grants or denies specific access permissions to specific users. The ACLs in this configuration store are not updated by Sysprep. I use some VBScript to update these ACLs.
- The Registry also allows Permissions to be set on each key in the form of a DACL (Discretionary ACL, if anyone knows the difference between a DACL and an ACL, let me know!). These are also not updated by Sysprep (although IMHO certainly should be, again, Microsoft: *please* change sysprep so that it updates registry ACLs...) This step updates these using a neat little utility called regperm.exe. *Update 6/9/2006* I've found a better way.
- I was using VMWare, not Microsoft Virtual PC or Virtual Server, as this is what my client has. VMWare drives the sysprep process for you, and for some reason, VMWare does not seem to honour the configuration parameters that are used to set the locale of the server when cloning. This may be a configuration issue to do with my clients installation thoug, YMMV. Anyway, due to a bug in the BAM implementation (acknowledged by Microsoft) the machine must have its locale set to ‘English (US)’ in order to be able to deploy BAM solutions. As the VMWare host is in the UK, and set to 'English (UK)', after cloning, VMWare automatically sets the locale to ‘English (UK)’ regardless of the configuration set through the cloning process. I have to use a bit of trickery, found here (ignore most of the article, it's the command about two thirds down that does the trick) to change the locale back to ‘English (US)’
- Changing the locale requires a reboot to take effect. Therefore I ,make autologon settings and runonce settings in the registry that will cause the server to automatically log on as the local Administrator account after the server reboots, and continue the clone process by executing a follow on batch script. Then I use shutdown.exe that is part of Windows Server 2003 to reboot the computer.
- Obviously, after the Server reboots and automatically logs on, I remove the registry settings that cause the automatic logon.
- After cloning the server any BAM deployments that had previously been done have to be un-deployed and re-deployed to restore them to functionality. I scripted this in a batch file.
- Because these BAM deployments are done by the local Administrator, their associated SQL Server Agent jobs are owned by the local Administrator. My clients security policy is to remove SA privelages from the local Administrators group. Once this happens (at the end of the cloning fix up proces), these jobs will fail to run. Therefore I use a bit of T-SQL to change the job ownership for all of the SQL Server Agent jobs so that they are owned by the service account that SQL Server runs under.
- And apart from removing the SA privelages from the local Administrators group, and cleaning up, that's it!
Caveats This works for me for one reason, and this is one big reason why it may not work for you: All of the service accounts (the accounts used for IIS Application Pool identities, to run BizTalk, Sharepoint, SQL and EntSSO services, etc) are domain accounts. What this means is that when syspreping, these accounts SIDs do not change, as sysprep only needs to change the SIDs for the local security accounts, groups and the machine. If you are using local accounts to run BizTalk etc under then when the SIDs change you will have a whole extra problem to fix: SQL Server identifies logons that use Windows Authentication by the SID. So when sysprep changes the SID, it obviously doesn't update SQL Server as it's not aware of server software, then when 'My Service Account' that is the identity for say a BizTalk host tries to login to SQL Server, SQL Server says, "you're not 'My Service Account', you have a different SID to the one I've got stored here, go away!" By using domain service accounts this problem is removed. "But I just want to clone a standalone VPC on my laptop" I hear you say. Well, you can fix the SQL Server logins, but luckily I didn't have to ;-), because it's not a trivial exercise... I'll leave this as an exercise for the reader! BTW, if you do work on this, please let me know how you get on :-) Conclusion After each aspect above is fixed, the macine is fully restored to perfect functioning state and does not even require a further reboot! Machines that have been cloned like this have been used without a single hitch at my client for several weeks now. Plus you can even still use Visual Studio to develop ordinary Windows Forms, ASP.NET, and Web Service applications (as long as you beat up sharepoint to stop it messing with IIS, but that you would have to do even if it wasn't a clone. I think that may be another blog post...!) Jack
|
|
|
|