Kalen Delaney just posted a great blog entry "Geek City: New geeky words". She describes a new term "technical debt" as being "the fact that the quick fix will usually end up costing more in the long run".
I see this all the time in my working life. Kalen gives a very geeky example pertaining to the use of query hints in SQL queries but the example that popped into my mind was at a much higher level than that. I am forever encountering situations where many packaged off-the-shelf applications exist within enterprises, none of which work together. The challenges that I face on a daily basis are all around finding ways of making disparate systems talk to each other in a way that makes the effort in achieving that invisible to the data consumer. These challenges exist because the enterprise has multiple systems bought from multiple vendors running on multiple platforms. This brings with it a requirement for multiple skill sets, multiple intermediary technologies to pull all this data together, and multiple headaches for the likes of you and me. The solution to all this is a well thought out and well-defined information architecture (IA) but putting that in place is so incredibly difficult that I could devote pages and pages of blog entries to it - and some day I may do that.
Here's a scenario that I can take from a recent engagement. A particular business unit in an enterprise started to use an asset-tracking system that was in use throughout the enterprise in many other business units. They realised they already had a system in place whose functionality overlapped with the new one so they decided they needed a way to make the two work together. The solution? Add a new column to the Asset table in the new system that contains the natural key of assets in the existing system. Let's consider what problems have been introduced here:
- There needs to be a mechanism for keeping the two systems in sync. That requires investment in development and support.
- There will likely be a time lag between updates thus at some point the data will be inconsistent.
- There is an effort to centralise support of these systems but that's going to be difficult if the underlying physical data model gets changed on an ad-hoc basis. Consistency has disappeared.
I often hear that the solution for these problems is "no matter, build a data warehouse". Well, I've spoken before about why that might not be the best solution. You can form your own opinion.
Often I find that the buying decisions for these systems were made by none-IT folk and I believe that that is where the problem lies. None-IT folk should not be expected to make considerations for interoperability or maintainability; their job is to get a tool that does the job and to me that is the very definition of "quick fix". We are now in an era where the IT function no longer exists to support the rest of the business, instead it is an integral part of the business so in many ways, it IS the business. The time has come where the IT director needs to take a lead when making purchasing decisions, IT should no longer be expected to just go and support something that was purchased by someone else.
Technical debt. A new term in my vocabulary. Thanks Kalen.
-Jamie