General

That's the line that kept going through my head after I unscrewed the lid on the semantic can of worms differentiating "DevOps" from "Agile Operations." The movement and concepts behind the terms are still relatively new and with little clear understanding of the differences (if any; some simply consider them to be two terms that describe the same things) I thought it might be a good idea to delineate what I thought each addressed and what I saw as the differences.

Unfortunately, I did a pretty poor job of it and simply sewed confusion in my wake. Fortunately, the more clear-headed and analytical Ernest Mueller took the sorry muddle and created something supremely useful out of it. Ernest's terms and definitions are sufficiently clear and succinct that I plan to adopt them in my own discussions, and I hope others do as well.

My initial contention was that Agile Operations was more focused on, and described from the point of view of, system administrators, while DevOps, while involving both sysadmins and devs, was more described and driven from the development point of view.

Ernest went back to the bible and usefully used it to break the concept down as follows:

• Agile Principles - like "business/users and developers working together." These are the core values that inform agile, like collaboration, people over process, software over documentation, and responding to change over planning.


• Agile Methods - specific process types. Iterations, Lean, XP, Scrum. "As opposed to waterfall."


• Agile Practices - techniques often found in conjunction with agile development, not linked to a given method flavor, like test driven development, continuous integration, etc.

I believe the different parts of Agile Operations that people are talking about map directly to these three levels.

• Agile Operations Principles includes things like dev/ops collaboration (DevOps definition 1 above); things like James Turnbull's 4-part model [Ed. roughly analogous to our Principles] seem to be spot on examples of trying to define this arena.


• Agile Operations Methods includes process you use to conduct operations - iterations, kanban, stuff you'd read in Visible Ops; Agile Operations definition #2 above.


• Agile Operations Practices includes specific techniques like automated build/provisioning, monitoring, anything you'd have a "toolchain" for. This contains DevOps definition #2 and Agile Operations definition #3 above.

In retrospect, I made a number of errors in my first pass. First, I tried to come at it from a 50,000 foot view, where both movements are primarily driven by folks down in the trenches. Just as with agile development before it, DevOps is coming from the ground up (which is good) not the top down. My own practice is focused on strategy, and my efforts are not in evangelizing to grunts (who generally already have the good sense to adopt these things), but to management, who have a different, broader perspective of how all these things fit together. It was probably not productive to try to discuss those with people who do not, and do not need to, share those concerns.

Second, while advancing the observation that DevOps is a primarily development driven movement (which some proponents dispute, though it is not meant as a slight of any sort), I brought up the point that on the whole, most sysadmins never interact with developers, and that from a strategic perspective, it's often wisest to avoid involving one's organization in custom software development in the first place. While I believe these things are true, and that to properly adopt lean or agile techniques at the strategic and operational levels, simplification necessarily calls for an effort to use off-the-shelf (or, these days, over-the-web) solutions first, it's a perspective that almost seems designed to threaten developers, whose livelihoods are called into question by it. While this is not, again, intended as a slight (I often tell sysadmins that it's their job to work themselves out of a job; it's not that I expect it to happen, it's that such a mindset is useful to people building an agile system), it's not going to endear such readers to my considerations.

Finally, I didn't consider the size of the disconnect between the strategic and the tactical perspectives or that my real aim is to address strategic decision-makers. I think that it is something of a historic failing of movements of this sort that they neglect this aspect of changing the paradigm; while IT in general may be unique in that it offers senior technical staff unprecedented influence on their organizations, ultimately they are not the people that are designing or implementing the organizational changes necessary to successfully bring their ideas to fruition. It's a common geek misconception that something that is logical is also straightforward. Anyone who has tried to introduce agile development or the like to staid and non-technical business-people knows otherwise.

So if I apply Ernest's rubric to my own thoughts and goals, DevOps sort of falls away from the discussion. In his vision, it seems to simply be the expression of an overarching principle(s) of Agile Operations, and I will probably get more value, and see less confusion, if I continue to espouse the principle itself without drawing in an appellation that doesn't apply in many environments where the principles would still be useful. Business silos and communications problems are hardly unique to the operations/development interface (which Ernest points out, though still in a development context); agile operations has just as much application to more traditional, and more widespread, issues between IT departments and other business units.

That doesn't mean I won't still be tracking closely with the DevOps crew; I still think a lot of the most practical methods and practices for adopting those principles to operational concerns are coming out of that group. Whether development is involved or not all the principles are aligned and it makes good sense for me to follow their thinking. Little of what I produce will probably be of much use to them, however, and with respect to the folks I am trying to reach on the management side of IT, there's little point in confusing them with DevOps references that don't apply in their environments.

Ward Cunningham, one of the godfathers of the Agile Development movement and the co-author of the Agile Manifesto, is moving over into Agile Operations territory these days with an upcoming webinar, Agile IT: A Better Approach to Application Development, Deployment, and Management. In the precis, the presenter's ask "Can IT Operations benefit from agile practices as well?" Obviously we here at Agile Operations think so, but it will be interesting to see what exactly Cunningham's take on the idea is.

The full precis gives some insight:

How often do Agile development teams finish their new web applications in record time only to wait weeks to go into production? New approaches like Agile development and Cloud deployment are enabling application teams to become faster, more responsive, and more cost-efficient. Can IT Operations benefit from agile practices as well?

Find out when you join EMA Research Director Julie Craig and New Relic VP of Sales and Marketing Mike Malloy for this one-hour Webinar where you will learn:

*How some organizations are moving new business software features into production in days, not months.
*How Operations and Applications teams can work together to form an “agile alliance.”
*How the on-demand SaaS model, which has transformed business applications, is now poised to change IT application management as well.
*How Cloud-based application management improves your team’s agility and cost effectiveness—but also demands a new approach to application management and support.

If you want to find out how your organization can benefit from Agile IT, this is a can’t-miss event!

The perspective seems to be about what you would expect from someone coming from a development shop, that the operations side of the house seems stilted and awkward in the deployment and operation phases compared to the speed and flexibility of the dev teams. It's true that a lot of IT shops haven't caught up with the devs yet in their change management and support roles, damping some of the greatest advantages of agile development, and that must be a source of great frustration to those developers. If development is cranking out new releases once every two weeks, but IT can only deploy once a month because of cumbersome release and support processes, it's defeating the purpose. It's only natural the developers would next turn to helping the operations team develop a similar tempo to allow the process to function properly.

That's a laudable goal and obviously one that I believe is entirely achievable. Yet historically I have seen many difficulties arise from development shop logic being applied directly to IT. The sensibilities and priorities are different, and it will be interesting to see if Cunningham, a life-long and accomplished development rock star, appreciates the differences or if he is simply carrying around an Agile stamp these days and slapping it on whatever business functions happen to be most frustrating at the moment.

Judging by the topics listed in the precis, I am optimistic, however. Those are exactly the factors in today's IT environment that have prompted the Agile Operations movement, and are some of the things I talk about most frequently when I am helping organizations move their IT departments in that direction. But they are also the cool, sexy new things to talk about; equally important are the more difficult and pedestrian requirements to emphasize change management processes, create a robust infrastructure that can support frequent changes, the necessity of re-organizing the department to align with a new conceptual framework of service delivery and building management support for the model, and implement controversial measures de-emphasizing IT control, such as fragile support. If you are offering up a webinar for movers and shakers I suppose you don't emphasize the less trendy topics, but it will be interesting to see what Cunningham chooses to address and how he pitches the concept. Definitely worth signing up for if you are interested in the Agile Operations concept.

The Agile Operations model is only one of many possible models around which to structure your Information Technology department, and your staff and executives are all more familiar with and better at following those other options. Why then, would you want to go Agile with your IT department?

Put plainly, you don't have much choice. At least, not if you want to still have an IT department, or anything resembling one.

This is a recurrent theme that is cropping up now among industry pundits and visionaries, and while I hear CIOs and traditionalists disagreeing with the assessment, I haven't heard them lay out a case against it other than the dubious "Our company is too incompetent to get by without us" argument.

The logic behind the disappearing IT department is simple, and it was among my motivations in starting this blog. Briefly, the fundamental efficiency of centralized utility computing combined with the improved delivery methods afforded by the Web and Internet (in the forms of pre-fabricated Software as a Service products and raw computing power as cloud computing products) will make outsourcing of most IT functions dramatically more cost-effective in most organizations than retaining in-house IT resources. At the same time these advances are being made in outsourced services, existing IT organizations are becoming more resistant to new initiatives and more encumbered by dwindling budgets than ever before. The inclination in these departments to say "no" to initiatives from the business is strong. And again, just as this is happening internally, external service providers are finding more and more seditious ways to insinuate themselves into departments bypassing the IT gatekeeper; the iPhone, web-based SaaS, and other new offerings require nothing more than a credit card to use, and the low price point of entry falls well within many departmental budgets... no new capital expenditures need flow through regular IT channels.

When you combine all of these existing trends, what you get is an IT department that will find itself struggling for relevance in five years or less, depending on how rapidly CEOs and CFOs get around to running the numbers.

Patrick Gray at TechRepublic is among those who have laid out the case most succinctly. In a recent post "The CIO is dead (Long live the CIO)" he describes how the popular shared services model many IT departments have adopted encourages comparison with outside services, and that comparison may not come off favorably to the IT department. Joe McKendrick comes at the same concept from a different angle, arguing that the IT department should become even more like an external cloud provider, and count on its better knowledge of the core business in order to compete.

Although McKendrick may have a point, I have to wonder if it is a bit late for such measures, even with inside knowledge. There is another factor, pointed out recently on TWiT by John C. Dvorak. Paraphrasing a participant at a Holland meeting of cloud-computing industry folks: "There's a lot of big companies that are getting sick and tired of their IT department, period. And they just want to get them, they just want to fire them all, but they can't quite do that, but what they can do is move to the cloud, and let these like a professional IT department that is not so meddlesome run the backend of their operation."

Now, I think Dvorak's conclusions are correct about twice a day, but his industry knowledge and insights have a much higher hit rate. When he says that executives are fed up with their IT departments, that resonates; and why not? IT has long had the upper hand in the relationship, fueled by secrecy, buzz, and complexity. When the system is broken and the CEO hears that an expensive new widget is needed to fix it, all he has been able to do is grit his teeth and reach for the checkbook. A sad fact is that there are a significant number of IT folks who have taken advantage of this to turn the department into their own techno-wonder playground, operated more for their own elucidation and interests than those of the business. A certain level of traditionalism in IT is also to blame: CIOs are by nature risk averse, and when they are faced with a choice between an innovative but potentially lucrative risk, and the predictable status quo, many will choose the status quo. This isn't necessarily a terrible trait, but when extended too far in the face of industry innovation, it has resulted in a lot of heavy, expensive technology processes which are no longer necessarily the best option for the business. CEOs are not completely blind to this. Every CIO dreads the Wednesday afternoon that the CEO comes back to the office after his weekly golf game and says, "Hey, the CEO over at Widget Industries says their e-mail never crashes and doesn't cost anything and they use such-and-such... why aren't we using that?"

It can be hard to explain even when it's justified, and CEOs hate that. When presented with an option to take those services outside with a provider who isn't going to argue with them, it's going to look pretty good.

The only palatable long-term answer to this problem is to get ahead of that curve: your IT department has to become the one that your CEO is bragging about to other envious CEOs at that weekly golf game, instead of the other way around. It has to be as responsive and efficient and innovative as all those external options. It's not enough just to know the business better; the department has to learn marketing and customer service just as well as those outside competitors. Staff have to want to come to the IT department first. It has to be easy and advantageous to bring initiatives to IT. Staff have to expect their requirements will be at least heard and probably fulfilled, and if the IT department can go to them first with those options, so much the better... they may never even get around to looking outside the business for solutions.

This is what Agile Operations is about. The internal IT department will never be able to compete with external providers on scale. In fact, AO often suggests co-opting those external providers to provide internal services, leveraging those economies through, rather than around, the IT department. What IT can do is take their better knowledge of the business and combine it with the flexibility and responsiveness inherently available when operating internally. Although it's a neglected capability, the fact is that IT has something besides business knowledge to wield as an advantage in this fight: it has the potential for objectivity. External service providers have one objective: make money. And they'll take money from a business whether it is in that businesses interest to pay them or not. The new role for the internal IT department, as Patrick Gray suggests above, is as a consultant. Not the sort of consultant that operates in effect as a vendor, with a sales agenda all laid out coming in the door, but a true consultant, an objective evaluator of solutions who can make the best possible recommendations to the business unit without bias or greed. IT must learn to associate itself intimately with the business, and to solve business problems using technology, rather than simply providing a platter of technology from which the business is expected to solve its own problems. Providing an e-mail system, for instance, is an example of a traditional IT task. Developing a technology based communication process between field and office staff might be a new Agile IT task. The former puts a big chunk of complicated technology out on the field and tells the business to go play, often with questionable results. The latter actually addresses and solves real and fundamental business problems. It becomes quantifiably advantageous, rather than simply presumably advantageous.

The IT department absolutely will not survive in its current form, and many CIOs may not make the transition either. But of all the IT department models on the table today, I am most confident in the Agile model as the method that will allow IT departments to survive internally in the most advantageous fashion. Businesses that do away with IT or the CIO internally are not going to be most successful; the hybrid model of external services and internal expert decision making is the best combination. It is a combination, however, which necessitates changes on the part of IT; AO might be the first and best of those changes to make.

Some background on the origins of Agile Operations are probably in order here, at least as it is envisioned here. It's still new, it's still evolving, but the essential elements have a longer history and the context may aid in understanding the applications to the Information Technology environment.

The concept attempts to adopt many of the lessons and precepts laid down by the Agile Software Development movement. Agile Operations, as practiced at IMS, came to pass in the SMB environment. The precept attempts to make a virtue of necessity; small organizations have never had the option of vast IT resources to handle all organizational requirements. Nonetheless, they get into deep trouble when their systems evolve as an unmanaged organic mess. The concept attempts to provide structure and rigor in the IT operations arena without the vast overhead which is traditionally associated with those functions.

This should not be confused with the recently popular "lean" movement developing in some IT organizations. Lean isn't necessarily Agile, although Agile IT organizations will certainly be lean (and lean certainly appears to have influenced the movement that has in turn influenced Agile Operations: Agile Software Development). Lean, as it is described by current proponents in IT organizations, focuses on cutbacks and trimming expenses; this isn't necessarily bad, particularly among the many bloated IT departments out there right now, but in our view it improperly emphasizes cost at the potential expense of effectiveness. Agile Operations attempts to achieve the effect first and foremost, not simply because that is the ultimate goal of most organizations (very few have mission statements reading "Our goal is to save a bundle of money somehow!") but because, in our view, the alternative is particularly dangerous at this place and time in the evolution of technology. Today, it's easier than ever for average users with nearly unrestricted Internet access to adopt any of a host of external, third-party tools in pursuit of their own job goals, and it is almost impossible for the IT department to prevent this. As these external tools become more and more powerful, and as the vendors are more and more motivated to bypass the IT department and sell to end-users directly, any IT department which puts itself in a position to oppose users or deny them resources risks becoming entirely marginalized, relegated to little more than infrastructure maintenance as business process solutions are increasingly constructed entirely outside their purview.

We believe this is dangerous not only for the IT department (in fact, we don't care about the IT department per se; within a decade, we expect to see a diminution of the artifact as a whole as companies absorb more traditional information management functions into mainstream operations organizations due to the increasing level of outsourcing and ease of platform deployment) but the company which it serves. Although we couldn't give a fig about the IT department itself, as it happens it has evolved into the sort of organization which is best suited to logically informing and advising the mainstream business in the adoption of new process efficiency technologies.

How can a software development methodology be applied in an environment of hard costs, capital expenditures, and established contracts and vendor relationships? How can you make IT as malleable as code?

Consider that a lot of thinking about agile development has been informed by existing physical processes. If you study the lean manufacturing concepts pioneered by Toyota and other "just in time" manufacturers, then you will quickly pick up on the similarities. Smaller commitments, made more rapidly, echo the just-in-time principle of TPS; other features resemble leveling, consensus-driven decisions, constant reflection, and other techniques that TPS proponents would readly recognize. The methodology has not been copied wholesale (noteably, AD not only acknowledges but embraces mistakes, where TPS seeks to squash them), reflecting the differing realities of the production fields they seek to improve. Similarly, AO differs from AD in a number of important respects; but philosophically, all three systems have a noteable resemblance.

We won't spend a lot of time talking about the core justifications behind this approach: you can find a lot of existing philosophical writing on the topic in places like this, or in books like The Toyota Way. This site, instead, will focus on implementation in the IT operating environment.

Some people are already doing it!

AgileOperations.net is a resource for CIOs and IT managers who are attempting to adjust their departments to the new realities of slashed budgets, sophisticated and demanding end-users, and vendors eager to bypass IT entirely and sell web-based services directly to the business. Some IT organizations are clinging even in this environment to old notions of tight, centralized control and rigorous blocking of outside services in favor of traditional internal IT tools. They may as well be buffalo standing on the tracks of the Northern Pacific as the first train barreled along toward Promontory Point from the east. The landscape of technology is changing, and the intelligent IT executive now has to walk a tightrope, balancing increasing user demands and external threats against the overall health of the business. Major consulting firms and market analysts are urging IT to become "lean" by continuing centralization and cost cutting, but lean doesn't necessarily mean flexible, and inflexible IT organizations will be bypassed and phased out as online services commoditize traditionally valuable internal services.

Agile Operations is a concept which combines lean, low-cost service delivery with flexible, just-in-time response to business demands, helping you keep your department relevant and competitive with trendy low-cost solutions available outside the business. In a sort of technology judo, Agile Operations seeks to use the strengths of these alternatives against them, keeping the flexibility and the savings in house and under the control of the IT department without resorting to heavy-handed prohibitions and lock-down measures which simply serve to force users further and further from a state of trust and understanding with the CIO.

In the coming months, this side will expand to include the basic precepts of Agile Operations, the theories and thinking behind the concept, and examples of uses of it in the wild. The aim will be to provide one stop shopping for the IT executive interested in researching and adopting agile tenets, and for those already making use of the concepts to expand and expound on them.