The metadata repository market has seen its share of ups and downs, and when one discusses metadata with IT oldtimers a note of cynicism can easily creep into the conversation . . .
"Great idea, too expensive, no ROI" is a frequent response. Others point to issues of keeping the metadata up to date. Some of the disillusionment with Computer Assisted Software Engineering (CASE) spilled over onto metadata, as both paradigms use the ill-defined term "repository" and both seemed to have the same totalizing, boil-the-ocean, rule-the world goals.
However, as I noted in my last post, there still is a metadata repository market. There are also market forces aligning I believe to dramatically expand its scope and relevance to enterprise IT operations.
Consider the current scope of metadata, as expressed by this Advantage Repository blurb:
Advantage Repository for Distributed Systems provides your application developers with a single source to view logical, physical and process aspects of application environments. Applications, programs, data structures and business processes can be documented, enabling easy identification of reusable components and enterprise-wide impact analysis.
This is an ambitious scope, and when one thinks through the implications of what it covers, there is little left in enterprise IT it doesn't touch. One interesting slant, however, is that it is aimed at application developers. What about support personnel?
This I argue is a fundamental weakness of the metadata repository concept. By omitting the support/operations side of the house, one omits a potentially powerful group of stakeholders, and also one then cannot reference or leverage the processes in place on that side of the IT organization.
As I've previously argued in response to David Marco,
How are ...
...servers deployed?
databases approved?
software systems installed?
All of these processes potentially affect metadata, especially in its broader sense of technical metadata. And these processes can be (indeed, must be) leveraged synergistically to keep the metadata up to date. In fact, a competing concept called the Configuration Management Database (CMDB) has been proposed by the discipline of IT Service Management (ITSM), and the $64,000 question is:
What is the relationship between the metadata repository and the configuration management database?
There is no clear consensus around the CMDB's scope, but since it is targeted more at the operations side, it often starts with lower level hardware and network components. However, the need to manage software as an asset quickly leads to the inclusion of software components and their deployments to servers. Business impact analysis is also a big concern, which means that the CMDB must address systems and business processes. Hence the conclusion:
In terms of the information stored, there is no essential difference between a Configuration Management Database and a modern metadata repository.
In order to support ITSM processes such as Change and Release Management, the CMDB products are very strong on asset lifecycle, which is often less of a metadata concern. They also tend to generalize many diverse classes of IT "things" into a master "Configuration Item" concept, which is useful but also dangerous, as I argue elsewhere.
Metadata repository vendors must start framing their products as relevant in this space. In this way they can start to leverage enterprise IT processes such as change management to ensure that the metadata is up to date.
This will require the creation of OLTP consoles that heads down analysts can enter metadata into, which is a divergence from common metadata thinking. The usual approach is to focus on scanning or harvesting metadata, preferably by automated means. But this only displaces the problem. Again, as I noted to Marco,
...someone has to be the first one to type in the name of the new server, the definition for a new release of a package, or the name of the new table. What is that process? And if the metadata administrators are not typing in the information, who is?
I don't have any evidence for this, but I do wonder if the lack of a clear process model for metadata maintenance hasn't been the biggest problem all along. Data management as a discipline is capable of generating clear process models, but when theorists start talking about expanding metadata's scope into the more technical areas of systems and components -- how is it maintained?
The idea of running IT more rationally is hitting its stride, as shown by the rapid uptake of the ITIL standard. This provides a wide set of process best practices covering all types of technical metadata, which can start to answer the challenge of keeping the stuff up to date. (Ironically, the one thing ITIL does not cover is traditional data management. Also, one is hard pressed to find the word "metadata" in the ITIL spec or any ITSM discussions.)
Unsurprisingly, a new set of vendors is emerging in this space, but as I have argued elsewhere, their information models are at best proprietary and often impoverished, nor is there any sort of standardization. Metadata's rich and precise normalized structures are what the industry needs, not minimalistic Configuration Items. (For further information on this, see my TDAN article. ) There is thus a rich opportunity for metadata vendors to take share in an emerging market space.
And finally, no post of mine would be complete without noting that the creation of proprietary information models in this space is a thing of the past. The OMG and DMTF already have most of the semantics one could need in building such systems, and are well on the way to completing the picture.
-ctb
