erp4it

Agile IT Service Management: Why it's an oxymoron

With the increasing prominence of both Agile and IT Service Management in current IT-related discourse, it's inevitable that some will ask "why not Agile ITSM"? Surely, those stick in the mud ITSM types with their Change and Release processes (so often impediments to developers) could use a little agility!

I've no doubt that some digital ink will be spilled in well meaning attempts to square this particular circle. But while there is plenty of room for continuous process improvement and Lean techniques applied to ITSM, applying the specific set of practices termed "Agile" (in the context of software development and project management) I think is misguided.

In order to make this case, I am going to refer to my industrial analogy for IT, that software development is product development. In both areas we find creative, risky, iterative, and non-deterministic processes. Agile might well find applicability in manufacturing R & D labs (at a guess, they already are doing similar things).

Agile makes a great deal of sense in terms of building novel symbolic structures (software) to meet emergent information needs. The need tends to co-evolve with the solution so an iterative approach is required. But these elegant and effective structures require much sweat of the brow. Even once a compelling presentation and experience is envisioned through well-applied Agile practices, there may remain tough ground work in creating a robust implementation.

How is this different from manufacturing, where a beautiful prototype may still require substantial investment in order to "tool up" a workable manufacturing line? And once that line is built, it will have  degrees of freedom along which the product might change - and constrained dimensions along which change is impossible without re-engineering.

And once you build the assembly line - or the transactional system - you need to run it with a high degree of reliability if it is to merit the label "service," which implies simultaneous production and consumption. Complex systems are prone to emergent chaotic dynamics, and change is the enemy (whether labeled "Agile" or not). That is invariant in any systems engineering, with theoretical foundation in Turing's proof of the insolubility of the Halting Problem along with much other math & science. Test-first development, while an excellent practice, does not solve the Halting Problem. You cannot test all paths, and the more extensive the testing harness, the more invested in it, the less the agility. Re-factoring, while another excellent practice, cannot be construed as risk-free. Any new execution path has increased probability of failure. And no reliability, no service, no Agile ITSM.

So, instead of attempting "Agile ITSM," why don't we accept them each as two poles of dynamic tension? There are some areas where rapprochement may be possible - e.g., do "Design for Manufacturing" practices have any applicability? Certainly, the philosophy of high cohesion & loose coupling is beneficial from both an Agile and an ITSM perspective.

But, to borrow from the Hindu Trimurti, Brahma the Creator and Vishnu the Preserver are fundamentally different aspects, and in the case of large scale IT need to maintain some distance in order to ward off the untimely appearance of Shiva the Destroyer.

CODA:

I am remodeling my house, and have replaced pretty much all the mechanical systems. I was confronted with a classic example of interacting systems dynamics yesterday morning. We've purchased and had installed a high efficiency boiler and "sidearm" water heater, a very nice, cutting edge system. (See my interview with Billy the boiler geek, with reference to 10Base2 networking).

By definition, "high efficiency" means that the heat is going more where we want it, i.e. into the heating pipes. It is going less where we don't want it, i.e. as "waste" heat given off by the boiler unit itself. But in the old system, that "waste" heat was just enough to maintain an ambient basement temperature just above freezing...

So, yesterday morning, we awoke to a waterless house. The basement had gotten cold enough that the water service had frozen at the point of premise entry. (Fortunately, it did not burst!) After some panic, we applied a hair dryer and all was well.

But there I had, in the small, a perfect example of the chaotic interplay of systems. The plumber did not foresee it. The boiler guy did not foresee it. The excavator (who dug a 4' trench below the frost line and installed the shiny new copper line to the city water main) did not foresee it. The general contractor did not foresee it.  It was outside of everyone's experience. And yet it happened...

I just read one of the finest, most relevant, and innovative articles i have seen in years. "The “Physics” of Notations: Toward a Scientific Basis for Constructing Visual Notations in Software Engineering" by Dan Moody. http://www.computer.org/portal/web/tse . Rigorous, scientifically grounded exploration of visual syntax, with some on target criticism of UML along the way.

Mind blowing Turing quote

The following quote is from Alan Turing's lecture on the Automated Computing Engine, an ambitious design he developed after World War II. I find it notable in its prescience; one can see the beginnings of information technology as an organizational function, with the tension between "dev" and "ops" already recognized; it also presages the development of the monitoring tools that have evolved into BSM -  and finally, a cautionary note regarding the hidden agendas that can be concealed by technical jargon.

"Roughly speaking those who work in connection with the [Automated Computing Engine] will be divided into its masters and its servants. Its masters will plan out instruction tables for it, thinking up deeper and deeper ways of using it. Its servants will feed it with cards as it calls for them. They will put right any parts that go wrong. They will assemble data that it requires. In fact the servants will take the place of limbs. As time goes on the calculator itself will take over the functions both of masters and of servants. The servants will be replaced by mechanical and electrical limbs and sense organs...

"The masters are liable to get replaced because as soon as any technique becomes at all stereotyped it becomes possible to devise a set of instruction tables which will enable the electronic computer to do it for itself. It may happen however that the masters will refuse to do this.

"They may be unwilling to let their jobs be stolen from them in this way. In that case they would surround the whole of their work with mystery and make excuses, couched in well chosen gibberish, whenever any dangerous suggestions were made. I think that a reaction of this kind is a very real danger."

"Lecture on the Automated Computing Engine," in The Essential Turing (B. Jack Copeland, ed), p. 392. [emphasis added].

-------------------------------------------------------------------------------

Always remember: it can be argued that Turing both defeated Hitler and gave us computers.

Software development is product development

Updated 1/30/2010: The distinguished Bjarne Stroustrup also attacks the shop floor metaphor in his recent CACM article: ""The idea of software development as an assembly line manned by semi-skilled interchangeable workers is fundamentally flawed and wasteful.""

--------------------------------------------------------------

"Software development is not like building bridges! It's not like a shop floor! It can never be a factory! It doesn't follow a process! It's an art! A craft! Poetry!"

Variations on the above are frequent in the world of software, systems, and solutions, especially when developers are heads down writing code.

Don't miss my TDAN article

Just realized I never posted a pointer to my new TDAN article "Lean IT Value Streams: An Entity Lifecycle Approach"  here. The article attempts to lay out an entity lifecycle based framework for defining IT value streams. It's terse and dense and probably the basis for the next edition of my book (if that happens).

Comments/criticisms most, most welcome.

Project good, maintenance bad? A contrarian view.

In the quest for IT value, a commonly cited figure is that projects (new functionality) consume a mere 25% or so of IT spending, while "maintenance" consumes the balance.

This is argued by many (including me, in the 1st edition of my book) to be a Bad Thing. Some well known authors even propose that reducing maintenance & increasing projects should be measured as evidence of IT value delivery - i.e., a Key Performance Indicator.

Today, I started to wonder if this whole point of view might be misguided.

Craig Larman discounts Critical Chain

There is an interesting Lean IT debate shaping up.

Most followers of Lean are familiar with Eli Goldratt, the widely respected author of The Goal.

Goldratt has gone on to write a number of other well known books, including one on project management called Critical Chain, which I have read and found interesting - but difficult at best to apply (runs into a computationally intractable conundrum known as the Resource Constrained Scheduling Problem). 

The Agile movement owes a great deal to Lean philosophy, and Craig Larman is a well respected author in this movement. Recently, (with co-author Vas Bodde) he published a notable work Scaling Lean and Agile Development. The book addresses a couple of trends I have been tracking: the use of Systems Dynamics modeling, and the application of Lean principles to IT management.

Systems Dynamics is explored and found useful at a conceptual level, but with skepticism towards the use of actual simulation tools. And the comments on Goldratt and Critical Chain are pointed:

"In the 1990s Goldratt extended TOC to project management for product development work...Official "project management TOC" ...involves specialized courses, tools, and coaching from the Goldratt Institute or authorized providers. It includes a relatively complex and detailed plan-and-control centralized management system with detailed task assignment to people, intensive upfront estimation and scheduling in detailed Gantt charts, and several other traditional management practices.

"Bottom line: We have seen two very large official "project management TOC" adoption attempts (and heard of one more) in companies developing software-intensive embedded systems... The practice was clearly heavy, not agile, and not lean. In all three cases, the approach was eventually found cumbersome and not very effective, and was dropped."

So, one of the leading lights of the Agile movement has thrown down the gauntlet to the TOC advocates.

I am as yet undecided in this. One can't dismiss Larman, and yet the Goldratt reasoning behind Critical Chain is also sound. Could an Evaporating Cloud assist in resolving this? Inquiring minds want to know.

IT Lifecycle Value Metrics

As I argue elsewhere (most recently on TDAN.com) there are four IT value streams:

- Application service lifecyle

- Infrastructure service lifecycle

- Technology Product Lifecycle

- IT Asset Lifecycle

I believe that these are at an appropriate level of granularity to apply Lean value stream mapping, in such a way that constraints can be identified and elevated.

But any value stream needs metrics, and they are not easily come by. We have myriads of ITSM "activity" metrics (e.g. # changes performed) but these are emphatically not "value" metrics. (Show me any relationship between # of changes performed - even # of successful changes - and the bottom line.)

Quotes

This will be a living post to collect useful quotes and citations.

-----------

"An 'information shortage' in service is equivalent to material shortage in manufacturing."

George, Michael L.. Lean Six Sigma for Service : How to Use Lean Speed and Six Sigma Quality to Improve Services and Transactions. New York: McGraw-Hill, 2003.

-----------

Do two states have the same definitions of "marriage"? Is the notion of "writing"
a play the same as the notion of "writing" a song? It is the semantic models that
give answers to questions like these. A semantic model acts as a sort of glue
between disparate, federated data sources so we can describe how they fit
together.

One reason to model knowledge about a domain in the first place is to understand the ramifications of that model, and to understand when there are conflicts between one viewpoint of the world and another.

Allemang/Semantic Web

Capability & process maturity: the antithesis of Lean?

I've had a nagging thought lately... are Lean and CMM (& its derivations), antitheses? 

An absurd thought on the surface perhaps, especially since they share common roots in quality management philosophy, and a simple Google search on Lean + CMMI yields a million hits (mostly of the "Lean Six Sigma + CMMI" variant, an important distinction).

But Lean has a laser focus on value, while capability maturity is more about evolving an organizational platform that in theory will then more effectively (consistently) produce value. It's a less direct value proposition.

I'm not talking only of the Software CMM, nor even CMMI's various flavors. COBIT also has maturity scales. And there's a broader implication even for ITIL: the question always seems to be, "are you doing *all* of the framework?"

ITIL and COBIT are more tolerant of ala carte treatment, but the original "staged" representation of CMM asserted a certain correct order of maturation.

(Digression: The Wikipedia article on CMM(I) claims there was an empirical basis for this order; I need to research CMM's roots further. Some may say that CMM is not even something to strive for; that its maturity stages are evidence of maturity, but deliberately seeking to enhance one's performance of them represents a misunderstanding of CMM. This is almost analogous to the faith vs. works argument in Christian theology - good works are a sign of grace, but can one use them to earn salvation? Any theologians out there who've also looked @ CMM?)

The staged representation was loosened with the "continuous" representation of CMMI, but I think the influence still lingers. And for any framework, the temptation to completism is apparent: you're not really doing "it" unless you are doing all of it.You're not "mature" to a certain level unless you have demonstrated all the antecedents, any more than you can be proficient in calculus without a foundation in algebra and trigonometry. And if you haven't done statistics and discrete math as well, your understanding is incomplete.

But delivering value, especially in a dynamic and competitive environment, is not the same as building an edifice of formal knowledge. The dependencies flicker in and out of sight and today's firm foundation turns into tomorrow's sand.The obstacles to value often cannot be forecast... they can only be reacted to and mitigated in the moment. Professional competence and disciplined practice may be necessary, but they are not sufficient for value delivery. And at a certain point... is the drive to "maturity" across a set of "capabilities," simply equal to the fallacy that aggregating local optima will lead to a global optimum?

I've had enough brushes with dialectics that my use of the word antithesis is deliberate. An antithesis does not obliterate the thesis; the two struggle together until a richer synthesis emerges. For a good example of an emerging synthesis, see CMMI or Agile: Why Not Embrace Both! - an excellent and related read.

How not to do Lean IT

Lean IT is headed up the hype cycle.

It's going to be interesting. I fear that the fundamental lessons of Lean will be trivialized into mere "process" improvement, with "process" translated into well known ITIL practices like Change Management.

Was this how Toyota became great? Did Ohno do nothing but look at activities (e.g. transmission assembly) in isolation and promote their improvement? Can a practice of the scale of Change or Incident Management *ever* be considered a true "value stream"? Or are we killing rabbits with a howitzer?

I think Lean IT needs to be much more ambitious in scope, and the first thing that the Lean IT practitioner should probably do is throw out their ITIL books.

Thoughts?

Happy Thanksgiving...

ITSM ambiguities: lifecycle vs. transaction; system vs. service

Apologies all for long absence. Someday I'll tell the tales. But not today.

Twitter is increasingly useful lately, for pointers to current IT Service Management (ITSM) discussion & many other topics. A couple lines of inquiry have me feeling contrarian, or at least desiring more clarity:

1. There is a basic distinction between the lifecycle of a service, and the end to end performance of that service. I still do not understand whether the concept of service "delivery" applies to either, or both. 

2. Despite all the inkbytes spilled on how "service" is different from "system" (or application) I continue to find the distinction problematic.

Meditations on Lean for IT

My ongoing research has taken me squarely into Lean. Have gone through The Machine that Changed the World and Lean Thinking (Womack/Jones)

As with ERP for IT, I am by no means the first to entertain this topic. There's a Wikipedia article that, while very well written, seems a little premature and bordering on original research (no books have been written on Lean IT to my knowledge). CA and Fujitsu have some material as well. All those are fairly recent.

Other material pops up on various Google searches, and since you all can do that I'm not going to devote a lot of time to random linking. Also avoiding commentary on other Lean IT commentary for now.

Instead I'm going to spend a few posts on Lean source material and my own exegetic interpretations of how it might apply across the IT supply chain.

Starting off with a few random/high level observations:

New directions: TOC for IT

I feel like being cryptic. 

I just registered toc4it.com, toc4it.net, and toc4it.info. 

Also, this Typepad site may be nearing the end of its run... spending more and more time over on Google. New brand, new look? If I have time.

Web 2.0 Enterprise Architecture tools

I am interested in any Web-based EA tools, especially those with a Web 2.0 collaborative slant. Requirements:

• No per seat charge
• AJAX or equivalent GUI (Flash or equivalent free, mainstream plugin OK)
• Full Wikification (objects, attributes)
• Every object must have its own URI. Tool should operate in a text mode that is URI-centric; focus objects and related collections.
• Support for Archimate or comparably terse metamodel
• Architecture DSL workbench capabilities (ideally MOF-based)
• Models expressable as text reports or graphic artifacts.
• Visio integration
• Google-like universal search
• Effective model concept packaging

Open to SaaS options. Open source would be awesome. 

-Charlie

The CMDB is an Operational Data Store

Longtime readers know that bringing some sense of architectural rigor to ITIL is an ongoing goal of this blog. The CMDB, viewed through an architectural lens, remains an ambiguous and challenged concept. Consider just two recent CMDB discussions: 

- Is a Person a Configuration Item (CI)?  Do People belong in the CMDB?

- How can a CMDB be "federated" to other systems?

In my opinion, these questions show some confusion around data and distributed systems architecture principles. By way of examining them, I'd like to take a look at a concept proposed by Bill Inmon et. al. in 1996: the Operational Data Store. 

Systems dynamics: a challenge to "enterprise" architecture?

Followers of this site and IT Skeptic may have noticed a number of debates concerning industrial theory, business process management, value chains vs. networks, and complex systems.

My analysis to date was based on well accepted principles of enterprise architecture: process, data, and systems analysis. However,  I have come to see all of these approaches as essentially static.

Barbecue is a noun...

For all us Yankees who may need to know a bit more on the subject (and if you don't know why I'm posting this, you're not paying attention...):

Why and how I blog

Some say that blogging has become passe. I don't care; I started doing it because it made sense for me, and still does. 

While I don't make a big deal about it, I'm allowed to state my employer and my position. I won't do that here, but you can dig around a bit on this site and LinkedIn and figure it out. Of course, the opinions stated here are my own and in no way represent those of my employer. In addition to that disclaimer, I write this blog as if my executive leadership were reading it. I want it to reflect well on both myself, and by extension any employer who hires me. It is a professional and collegial channel.

Profound PMBOK/PMI skepticism

Here, courtesy of my good friend and sometime partner in crime Sean Goggins. 

You could substitute ITIL throughout to much the same effect.

-Char