Practices

Dennis Stevens posts "Kanban and when will this be done?" talking about how to estimate feature completion in agile environments.

Stevens focuses on the Kanban pull system that has been making inroads in agile development methodologies recently, but the techniques he describes are sufficiently generic to make use of in almost any agile system.

Scheduling in an iterative, short-cycle environment can be a significant challenge for recent adopters, but in fact it's no more inherently difficult (and may actually be easier) than scheduling in more traditional environments. The major difference is in experience. Most CIOs and IT managers have a career of experience estimating traditional waterfall-managed projects and have developed some level of skill scheduling in that environment... not enough, unfortunately, to make up for the other issues befouling waterfall management in IT. Stevens has the answer for picking up that experience in a kanban system, however: measure and analyze!

By measuring cycle time required for various work items and comparing it to the "size" or complexity of the items, Stevens' method lays out an adequate rule of thumb for producing scheduling estimates of similar items. I'll refer you to his original post for specifics.

The most important point he makes, however, is that planning and scheduling should be decoupled. To whatever extent possible, the specific work items should be divorced from delivery dates, at least as far as the team working on them is concerned. This helps to avoid a whole host of negative behaviors that are associated with deadlines. It does not, however, preclude managers from using the estimating methods to produce their own timelines for scheduling purposes.

You're unlikely to ever be lucky enough to work in an environment where planning can be entirely divorced from time. Indeed, it's a vital factor in any cost/benefit analysis to be able to state when a cost is going to result in a benefit. Committing resources to a project that will result in a 50% reduction in costs within two months is a better call than committing them to a project that will result in a 75% reduction in two years. At the management level, you have to be able to make some ballpark estimates that are sufficient for business planning purposes. What you don't have to do, and what Stevens makes a good case against, is to attempt to hold your team to those numbers. Accountability is necessary, but the better way to achieve it in Agile is by putting good, trustworthy people on your team, and getting them to buy into self-enforcing standards of accountability.

The other great coping feature in the agile toolset is the iterative release concept. If you miss an estimate on a feature release, it's not as bad if you are putting out your updates once a week as if you were doing them only quarterly. You may miss, but you won't miss by much.

I wrote a couple of weeks ago on the role of governance in Agile Operations, but a recent podcast put together by ZDNet's Dana Gardner has kept the concept on my front burner this week. Read excerpts, or download the audio, at Gardner's blog here.

Gardner and his guests, Jeff Papows (President and CEO of WebLayers) and Kerrie Holley (IBM fellow and Chief Technology Officer for IBM’s SOA Center of Excellence) are more interested in discussing SOA and their relationship to governance, but on the whole the conversation is a good fit with the broader conversation about Agile approaches to governance (I've already put up some of my takes on SOA as it relates to AgileOps).

Both Holley and Papows have books coming out later this summer; Holley's, co-authored with Ali Arsanjani, is 100 SOA Questions: Asked and Answered; Papow's is the more broadly themed Glitch: The Hidden Impact of Faulty Software. Since neither of them are out yet, I will warn you that my take on Holley and Papow's arguments may be poorly informed. Nonetheless, while I find myself disagreeing with some of the ways they get there, I couldn't be more on board with their conclusions.
Continue reading "More on Agile IT governance" »

I don't talk a lot about specific technologies used in Agile Operations here, mostly because it's an approach that doesn't necessarily require specific technologies (and anyway, the DevOps crowd covers the useful tactical details pretty well) but it's sort of an assumption that Service Oriented Architectures (SOA) will figure prominently into many AgileOps plans.

SOA isn't really a technology, of course, it's a set of principles that happen to be complimentary to the AgileOps tenets and can provide powerful enablers for agility. It's not yet clear that the SOA community has grasped this alignment, or the idea that AgileOps can be as strong a selling point for SOA implementation as the reverse. But Joe McKendrick asks an old question in a new venue when he discusses taking bottom-up or top-down approaches to SOA implementation and as it happens, Agile Operations already has an answer of sorts.

The answer, of course, is that it's a false choice, and something of an anathema to agile principles in general. The reason people have such vigorous debates over bottom-up versus top-down is that there are excellent merits, and terrible faults, in both approaches, and it's easy to pick and choose which to champion based on your preferences (or, more likely, your position at the top or bottom of the organization in question) and discount the other. Agile, though, emphasizes communication, cooperation, and alignment... all things that don't come from either top or bottom, but both.

Oddly enough, McKendrick mentions this "middle-out" approach but then fails to compare it to the others. Instead, he makes points for both top-down and bottom-up:

• Top-Down: Provides governance, business-level SLAs, is active rather than re-active


• Bottom-up: Avoids rigidity, long-term investments, and detached perspectives on tactical implementation issues

From where I'm sitting, these are all good points... and none really contradict the others. There's no reason you can't have the advantages of both without the disadvantages of either. It just requires communication and coordination... things that Agile is pretty strong on.

So, if you're looking at bringing in an SOA in your organization, think hard about moving toward the use of Agile Operations at the same time. Don't force yourself to make the false choice, picking bottom-up or top-down and then finding yourself having to reconcile the mess in eighteen months. Start middle-out in the beginning and make it easier and more effective for everyone.

Much of the discussion of agile operations occurs at the grass roots level, a result of the emphasis on practicality and the mechanics of getting things done. But agile practices are sufficiently different from what has come before that they have significant impacts at the organizational management level as well. This conflict has proven a headache for both CIOs and for agile proponents in the trenches, neither of whom are able to proceed unfettered toward their goals while the other is not onboard.

Fortunately, when we get to the operational aspects, we can borrow much from the consideration that others have already given the problem from the similar situation agile developers found when first bringing their own model into play in large organizations. Even better, we can draw from much of the original thinking of lean proponents, who were focused largely on management aspects in the first place, and whose techniques and conclusions are often directly applicable in the operations environment.

Establishing the right governance model is as critical as any other factor in implementing agile operations in your organization. It's broadly recognized, if still poorly reflected in actual practice, that management support for almost any significant initiative is a critical component for success. Governance represents the effort to establish that support by interaction between the IT department and business leadership on a functional level (this is opposed to the more theoretical interaction typical between CIOs, CEOs, and CFOs, where much detail is often absent from strategic and operational discussions).

This idea often seems an anathema to dedicated agile proponents, as "governance" brings up thoughts of mindless paperwork, rubberstamping, and endless unproductive meetings that are precisely the reason they prefer agile to waterfall in the first place. But if every crisis represents an opportunity, it is also true that every new governance structure represents a way to get governance right. Bringing agile thoughts and theories to governance can transform that process as radically as it has transformed software engineering.

Continue reading "Agility and governance" »

Cloud computing and agile IT are often bandied about in the same conversations, but to what extent are the advantages of cloud computing inherent or beneficial to agile IT operations?

Christina Torode wonders "Does IT scalability translate into business agility?" over at TechTarget and concludes that "something is missing" between the two.

Of course there is something missing; a tool is not a methodology and although you should try to select the right tools for the job, simply selecting the right tools doesn't inevitably lead to the success of the project. Indeed, tools are rarely the most important factor, and different tools may be the "right" ones given the same objective but differing teams.

On the other hand, good tools can make adherence to a set of beneficial practices much easier than without them. Cloud computing, and more specifically, virtualization in general, represent killer apps for agile administrators and CIOs.

Best practices for operations have always involved developing a certain degree of portability to systems. Hardware being eminently fallible and migrations for scaling purposes all-too-common, images became a popular solution for rapidly and systematically replacing or deploying new systems in a way that allowed thorough testing and predictable behavior. Increasingly advanced imaging and scripting tools made deployment and configuration a matter of routine in well-run IT departments.

But the messy bit, the hardware itself, was still a prominent factor in the equation. In the early days of automobiles, most drivers could not simply own a car and drive to their destination unhampered by mechanical difficulties as we do today; every driver had to be a mechanic first, the degree of reliability of the car being constantly suspect, and there existing little or no infrastructure in the form of wreckers, gas stations, and repair shops to turn to with problems. In short, it was messy. Cloud computing does not remove that messiness, but it does abstract it by turning it all into somebody else's problem, just as has happened with cars. We don't all have to be mechanics now, we can just get in the vehicle and go.

On a purely technical basis, it's interesting to look at the circular nature of agile ops and cloud computing. James Urquhart at Cnet has a three part series on cloud computing and 'devops' which is as much about how agile methodologies are enabling cloud service providers as how cloud providers enable agile for the rest of us.

On a functional basis, however, cloud-based services and applications are a shortcut to most of what operations teams have been trying to achieve for years with hard-won redundancy and virtualization features. The difficulties of running the infrastructure are dealt with by specialists, and the primary challenge for the operations crew becomes allocating and monitoring the services themselves. This puts some much-needed focus on the service deliverables without the distraction of the hardware level... it allows the ops team to stop playing at being mechanics and instead just get in the car and go.

Just as with your favorite repair shop, you'll want to do due diligence on your cloud computing provider... it's not that the hardware becomes unimportant, after all, it's simply abstracted and someone still has to know how to work with what is under the hood. But past that, an ongoing understanding of how the services are provided isn't necessary, at least by most of the team. They can instead focus on services, provisioning, and reliability at the functional delivery stage. More attention there will not go amiss these days when the utility of IT departments everywhere is being called into question and most are being asked to provide the same services with fewer staff and resources.

At the end of the day, cloud computing is just another level of virtualization, so if you have already done the hard work of convincing corporate authorities that virtualization is a winner (which it is) then you don't have far to go with cloud-based services. The costs in most instances are even more attractive than with in-house virtualization, and security issues, while real, are likely to be less a concern than the computing press has so far made them out to be. The scalability of external virtualization will far exceed what most businesses can accomplish with in-house resources... and the agility the solution offers makes it the right tool for IT organizations moving toward Agile Operations.

Fragile Support is the practice of reducing the internal support function of the business in favor of simplified or externalized resources.

The basis for fragile support rests on two premises: One, it demands that internal systems be designed to function as robustly and intuitively as possible, so that they will never require much support; Two, it emphasizes pushing the support function back outside the business or onto such resources internally as can be maintained without any specific support role designated.

The key to fulfilling both those premises is not in some wondrous technology or managerial expertise, but rather in providing the appropriate incentives to both users and IT staff.

Continue reading "Fragile Support" »

Like firefighters heading into a burning building, it's a wise practice for agile administrators to know their closest two exit routes from a newly adopted system before they adopt it.

At every point in the evolution of technology one can find stories of users stuck in the ghettos of old systems, trapped there by fear, ignorance, or planning failures. Most IT professionals can safely escape the pits of the first two problems, but the third is all too common even among the supposed elites in the business.

The article linked above discusses the tar pit that Internet Explorer Six has become for many businesses large and small, and the situation illustrates a terrible trap for an agile IT shop. Agility provides options and flexibility; being stuck with an insecure, unsupported piece of legacy software is almost exactly the opposite of it.

Note that this is not the same situation as selecting and continuing to use an older system out of preference. There's nothing at all wrong with not playing the upgrade game if there are no tangible benefits; that, too, is a hallmark of agility, the capability of refusing to be forced into changes that are not necessary or desirable for the purposes of the adopting business, rather than the benefit of the vendor.

In this case, companies that would otherwise gladly upgrade from IE6 find that they are unable to because critical web-based applications they use will only run on that browser. The upgrade cycles of their vendors have not aligned, a classic problem. Microsoft is ready to move on, but others are not (SAP being one vendor cited specifically in the article whose website does not yet support IE8).

The problem is not, however, that vendors or internal developers are not marching in lockstep on upgrade cycles. Any attempt to align all the various interests and capabilities that determine the upgrade timing for all the various providers involved is doomed to failure. The real problem is that so many companies were so ready to adopt platforms that had such stringent software requirements in the first place.

The situation in which these companies find themselves was anything but inevitable. There are thousands of sites and web-based applications out there that work as well in any number of browsers, including IE8, today as they did in 2001 in IE6. The problems that businesses are having result from their decisions to make or adopt software that was tailored to the non-standard quirks of IE6... quirks not replicated in other browsers, not even in newer versions of IE.

It's tempting to buck standards when someone is offering a solution that seems to be ahead of the pack, beyond the capabilities that the standards can provide. But the fact that standards are slow to emerge also makes them slow to change, and even as they evolve users are rarely left behind.

That's not to say standards are the only answer or that they completely avoid the pitfall of rapidly evolving technologies; there's no magic bullet, and it's not always a bad thing. The argument, rather, is not to over-commit. Keep your options open, think about how you are going to get out of a proprietary system before you get into it.

I've seen a few references recently to Malcolm Gladwell's piece in the New Yorker (subscription required) taking on, in that way he does, the conventional wisdom that entrepreneurs are risk-takers and heart-breakers, hard-charging individuals with vision and boldness who forge ahead regardless of the consequences and break into markets where others wouldn't dare. Quite to the contrary, Gladwell says; most ground-breaking entrepreneurs have done everything possible to minimize risk, to maximize their chances of success by not leaving room for failure.

A brief review of the article appears in this Business Week post, which suggests that this isn't news to most successful business owners and operators. That is in line with my experience, as well, dealing with many of those entrepreneurs in my day to day business. They have usually planned hard to get where they are, focused in on what they need to do and how to avoid stumbling, and they play every card to make sure they get the execution correct.

This comes up as a topic on a blog about agile operations because it both addresses a level of planning that many in the IT world neglect, and it takes on a popular myth in the industry, that of the maverick coder or sysadmin, who by strength of will and intellect single-handedly pulls together a great system or application, frequently storming past the mere mortals in management or business units who may oppose him.

In fact, operations that run this way are ripe for disaster and if it hasn't happened to them yet, it will at some point, because no one is immune to risk, even if they prefer to act as if they are.

Successful systems are those built with a recognition of and a system for managing what are called "Normal Accidents." Dmitriy Samovskiy has a post discussing normal accidents in complex IT systems that talks about the implications of the book in the IT world and it's right on the mark.

Agile, both with respect to development, and operations, is in large part about managing the risks that we all run of suffering these normal accidents. There's almost no way to foresee the often unique and tawdry chains of events that lead to such accidents; the only way to manage them is to step back from the immediate context and ask what can be done systemically to minimize their impact. And perhaps not surprisingly, this can be done by making some very practical adjustments such as simplifying those systems in the first place, improving communications, creating processes that don't rely upon outcomes, and putting control in the hands of people who know the work best. These are the things that agile operations are about; they provide a level of meta-control over the uncontrollable, and in doing so, diminish risk and increase chances of success.

One of the most prominent activities and most significant cost centers in the average IT department is support operations. How does Agile Operations handle the support function? By avoiding it!

Bill Detwiler put out a poll on a contentious subject for CIOs: whether or not one should provide corporate IT support for staff home PCs.

As you will immediately understand if you have ever been a CIO or IT manager of any stripe, this subject comes up repeatedly and in many permutations between the IT department and other departments. The default response from IT varies depending on the IT executive's level of experience and the relative rank of the staffer making the request: new IT managers are inclined to be magnanimous and provide help wherever requested, whereas more experienced folks have seen the pit of despair that home PC support can turn into and either avoid or limit it as much as possible, except where it involves senior staff or the CEO's eight year old daughter.

So the responses to the poll are predictably across the board with “No” at 48% barely edging out “Only for corporate software” at 45%.

Detwiler comments that his response is in the “no” category and goes on to describe a typical reason why: partial corporate assistance often morphs into total unswerving desktop support, costing hours of IT staff time and resulting in no real advantage for the company as a whole. Yet many businesses find themselves sucked down this hole.

This often results from corporate software being difficult to use and not interfacing well with common home systems. Detwiler's opening examples point to this very phenomena, with home users being compelled to call the help desk when they had problems using corporate intranet software on home machines. It's not intuitively obvious where the problem lies in such situations and understandably frustrating to both support staff and users when support is unable to address the issue.

The advantages of allowing users to work from home or from their personal devices are numerous, however, and it is obviously unfair to require or encourage them to do so yet have to fend for themselves when it comes to dealing with balky and mandatory corporate software packages. How does an agile approach remedy this situation?

Agile Operations encourages broad platform compatibility and compliance with universal standards (HTML, Java, etc). While there are certain esoteric incompatibilities in these platforms as delivered by certain vendors, a good rule of thumb is that if successful multi-billion dollar companies like Amazon can reach millions of people with a technology daily, you can probably rely on it to work with most of what your staff might throw at it. If, alternately, you bring in a proprietary single-vendor solution with limited industry distribution, contrary to popular perception you are likely to have interface issues. Build web applications and applets that are black boxes; devote FTE that might otherwise go into support or training to development and operations to avoid generating problems in the first place. Dealing with support issues before they become issues always counts as a double savings: business staff don't lose work time to a bug, and your support techs don't spend time tracking the bug down and fixing it. When we're talking about this particular issue, though, the winnings are threefold: staff aren't losing time to your bugs, and your techs are neither spending time on internally generated issues or on external ones, because you can successfully enforce a “no home support” rule.

So the Agile answer is the same as that many CIOs have already arrived at, but without the more capricious side effects of either creating schizophrenic corporate software mandates or inadvertently discouraging personal resources being used for corporate projects.

Of course this is an oversimplification of the support requirement, and you're certainly not going to get away with pulling back entirely from support. You can seek to minimize it by applying the basic tenets of simplicity, process, and people to your support system. Keep it so simple that problems can't come up; design processes that deal with problems intrinsically rather than exceptionally; and focus your designs around your people and their requirements instead of esoteric technical demands.

I'm seeing more and more of this sort of story, and while I am an Apple fan myself, I have to say that they come to almost entirely the wrong conclusion about mobile device deployment... ironically, the same "wrong" conclusion that most of them decry at the beginning of the episodes prompting their switches in the first place.

From the agile perspective, standardizing on a "good" phone is just as bad as standardizing on a "poor" one. It's the standardization that is the problem, because good and poor are both extremely subjective when it comes to a device used as intimately as a mobile. I know people who loathe the iPhone and would hate to give up their Treo, or who are addicted to Blackberry and regard all other devices as interlopers. There are sure to be some in every organization... they aren't usually the ones chosen to write the articles, obviously.

Instead, standards should be placed on the services that are to be delivered, rather than how they are to be consumed. You'll find this crossing over with SOA practices as well, which shouldn't be a surprise... SOA is an important tool for the Agile IT department. But let's look at how the management of mobile devices might fit into Agile Operations. Continue reading "Getting out of the mobile device business" »

The Forrester report I wrote about here last week has made its way across other desks now and I thought Joe McKendrick's take is worth linking to for another perspective.

Joe's post repeatedly conflates operations with development, misses the larger takeaway from the Forrester report, and cites Wikipedia as a definitive source on the definition of the term, a cardinal sin when discussing new techniques or technologies. Yet he brings in some necessary connections to the increasing standardization and commiditization of technology services, and finishes up with the most salient question I've seen yet on the subject: "...what does Lean IT offer that we haven’t been trying to do already?"

He precedes this with an impressive list of various IT initiatives from SOA to ITIL which exemplify the basic principles of lean manufacturing, or at least to whatever extent is possible in an admittedly quite different field. McKendrick also draws support from the work of IBM’s Dr. Irving Wladawsky-Berger and Microsoft’s Jack Greenfield, thinkers who have both posited more mechanized, industrialized IT products and services, a list to which he may as well have added Nick Carr and his vision of utility computing as presented in The Big Switch: Rewiring the World, from Edison to Google. It's no suprise that this particular vision of information technology, with its emphasis on similarities to the historical evolution of industrialization, should find basis for comparison in techniques that have made manufacturing more efficient.

But McKendrick is entirely correct in his analysis that IT is already on the road toward creating its own, similar techniques for achieving standardized efficiencies. In some ways, Lean IT has already missed the boat. The question is not now whether or not to bring lean techniques to IT, but how to do so within the larger framework of creating a lean organization? This is really a more expansive topic than "lean IT" or "Agile Operations" but I like to think that an Agile IT organization is a step ahead on implementing the basics of lean. After all, agile as a technology concept owes much of its basis to lean thinking, and much inspiration to the Toyota Production System. An IT organization that has already been taking steps to implement these concepts is probably going to be set back by lean, with the susceptibility of narrowing the perspective to cutting costs while ignoring service improvements and other benefits.

Lean isn't just a buzzword, though it will certainly find a role as such among organizations tasked with marketing to IT departments and with dedicated players of buzzword bingo. In fact, it's an important concept that informs many of the other paradigms that IT departments are adopting to standardize, automate, and improve services, and for that reason alone it's worth contemplating. As a set-piece methodology to bring in as a tool to improve IT operations, however, it's old and incomplete, and IT departments will be better served continuing to implement other methods already well-adapted to their environment.

There has always been a lot of tension in the agile development community over the extent to which agile practices need to be adopted in order to have significant, or any, benefit. Extreme Programming proponents, for example, have long held that every single one of the system's rules must be adopted in order for it to work; it's a synergistic process not amenable to piecemeal implementation. It's not unusual to hear other Agile proponents suggest that "failures" of their favorite systems are due to an incomplete adoption of the principles and processes they espouse, an argument with varying degrees of validity which is not without the potential for bias.

My feeling with regard to the piecemeal adoption of either Agile development or Agile operations is that there are certainly components of the associated principles which must be taken up simultaneously or not at all, but that it's rarely necessary to rigidly and completely implement every single method they incorporate in order to see some benefit from them. It's true that some practices make little sense without others; the idea that you should push support functions outside the organization, for example, does nothing but cause trouble unless you have also taken to heart the need to bullet-proof and standardize your internal systems to allow that sort of easy interaction. On the other hand, you can certainly adopt rapid, iterative release cycles without pushing power out to the organization's periphery... those principles don't collide. I've taken a similar stance on the adoption of ITIL and similar frameworks. As long as you consider carefully what you will and won't implement, you can get a great deal of benefit from pulling in only certain parts at a time, where they make sense for your organization. In most large IT organizations, it's impossible to simply adopt any new system from whole cloth at a given point anyway, so there is inevitably some mixing and matching to deal with no matter your intent.

The CIO Dashboard has posted on some of the ground rules for agile implementations alongside more traditional methodologies. Although they are primarily looking at it from a development perspective, the discussion verges into system architecture, which provides some models for considering the argument from the operations perspective as well.

The three pillars they identify in what they term "Architecture Driven Agile" are:

• Architecture


• Release Planning


• Organizational Structure

All three of those factors are also critical in the adoption of Agile Operations.

A solid system architectural framework may be the most important among them. Agile Operations allows for frequent minor, iterative changes, and accepts the probability that some of these may result in upset and breakage in the production environment. It's acceptable only because it assumes that your basic core architecture can remain functional and unaffected and that your release planning (number two on their list) and change management can rapidly roll back problematic updates. The changes you make, however, have to hang inside a solid, unchanging architectural framework, or you can't be certain what is breaking, or how to roll it back, or even what your initial limits should be.

Organizational structure, which might also simply be called "security" in the operations perspective, fills a similar role in providing a framework inside which changes can be safely made and reversed. It's important for the team to coordinate their efforts with the staff they may be affecting, and that the roles and rules associated with operations management are clear on both sides. Without this, communication usually suffers, and tight control over release planning and change management is impossible. "Control" is probably the wrong word there; it's not always necessary to directly control changes so long as it's possible to strictly monitor it. A working agile system might not require a lot of overhead intervention but certainly those responsible for overseeing it should be constantly aware of what is happening.

With those elements in place, other aspects of "pure" agile may be neglected or put off without serious long-term consequence.

This is really just a quick link to an article by Oracle India VP Shailender Kumar in Network Computing discussing the advantages of application grid technology as an enabler for Agile IT operations. Kumar is discussing the implementation of grids internally, which automatically puts them outside the reach of even quite a few businesses in the SME sector, but there actually isn't any hard and fast requirement for internal implementation. Though Kumar never actually uses the word "cloud" in the article (probably it has been banned from on high at Oracle, worried that their lucrative corporate implementations are going to fly off and consolidate in EC2 somewhere) that is essentially what he is describing and advocating: utility grid computing, centralized resources that can be pooled and re-allocated on the fly to their highest and best use.

The cost advantages of any utility computing implemenation are what generally receive the most attention, but from an Agile perspective the flexibility offered in such an arrangement is at least as important. Whether your grid is built internally or rented out for less from another provider, don't make the mistake of just looking at it as a way to shave some pennies off the cost per computing cycle... take advantage of the inherent flexibility to make your organization more responsive to business requirements.