Ernest Mueller at the agile admin is asking for feedback on an outline he has developed of service delivery best practices for developers seeking to implement NoOps.
I'm not sure I have any worthwhile comments on the outline. I've worked with non-ops-oriented organizations before to help them develop service standards, but never starting from a list of best practices... or, rather, I suppose I took the best practices in my head and helped adapt them to the scenario at hand. If there is a secret sauce to developing working and worthwhile standards, I think that is it: taking generic industry best practices and figuring out what does and doesn't apply in your organization and how to adapt the rest successfully to your particular requirements.
I suppose that means you need to start with a list such as this one, and Ernest has made the right move in attempting to tie back traditional ops practices to customer service standards that developers might be more attuned to. If anything, he might move even further in that direction. I think one of the things that irritates developers about ops people in general is the apparently blind allegiance to checklists and apparently obscure mandates that seem to have little to do with running code and serving customers. While Ernest has made every effort to tie these practices back to customer service points, I can see developers tuning out somewhere mid-way through "Performance."
Ernest tips a hat to this conflict at the end of the piece where he acknowledges it was a struggle defining the list to acknowledge the limitations of the environment: "On the one hand you should only put in e.g. 'as much security as you need' but on the other hand there’s value in understanding the optimal condition."
I might come back at it as more of a recipe book, starting with the problems that developers are likely to run into and working back to the practices to eliminate them. I have a hunch that might get more traction in development circles but it may depend on the environment.
If you are a developer, or simply a clairvoyant ops guy, head over to Ernest's blog and post some feedback for him in comments.
Monday, March 26. 2012
The bureaucratic constraints on agility
Virtualization is a huge tool in the toolbox for agile admins. Few technologies offer such power, safety, and flexibility all rolled into one package. But although virtualization technologies have advanced by leaps and bounds over the past decade, business models and licensing practices have not kept up, and consequently, a lot of what we should be able to do to create a more agile IT operation is artificially constrained by outside interests.
The latest example of this is being highlighted by a service I have mentioned before, OnLive's Windows 7 desktop for iPads. I did not fully consider it at the time the service was announced, but it turns out that it was never entirely clear how OnLive was able to offer such a service while remaining in compliance with Microsoft's licensing strictures. Recently, the apparent public flouting of those strictures has caused competing virtual desktop services to cry foul.
It's not just Microsoft creating these problems. VMware has continued to struggle with licensing terms also, making tweaks last year that set off a minor firestorm among the customer base.
The very nature of the utility of virtualization has a root conflict with traditional software licensing schemes, and this undoubtedly makes it difficult to price fairly in the eyes of both provider and customer. We're not at the sweet spot yet, judging by these latest moves. But we're getting to a point where the limitations of the state of the art in information technology are no longer technological, but rather bureaucratic. That's bad news for productivity, and for information technology in general, and it would behoove vendors in that segment to recognize and streamline this obstacle to adoption.
The latest example of this is being highlighted by a service I have mentioned before, OnLive's Windows 7 desktop for iPads. I did not fully consider it at the time the service was announced, but it turns out that it was never entirely clear how OnLive was able to offer such a service while remaining in compliance with Microsoft's licensing strictures. Recently, the apparent public flouting of those strictures has caused competing virtual desktop services to cry foul.
It's not just Microsoft creating these problems. VMware has continued to struggle with licensing terms also, making tweaks last year that set off a minor firestorm among the customer base.
The very nature of the utility of virtualization has a root conflict with traditional software licensing schemes, and this undoubtedly makes it difficult to price fairly in the eyes of both provider and customer. We're not at the sweet spot yet, judging by these latest moves. But we're getting to a point where the limitations of the state of the art in information technology are no longer technological, but rather bureaucratic. That's bad news for productivity, and for information technology in general, and it would behoove vendors in that segment to recognize and streamline this obstacle to adoption.
Wednesday, March 7. 2012
Security agility
I haven't been posting about it much, but I have been putting more thought into the relationship between security and agility since I was last made aware of my deficiencies in discussing that topic.
The tension between agility and security seems obvious: rapid, iterative change leaves little time for detecting and defending against vulnerabilities that may naturally arise during the process. On further reflection, however, it's not so obvious that the bad outweighs the good in iteration. True, moving quickly may introduce vulnerabilities that could have been avoided with further consideration. However, it also means that any vulnerabilities that are introduced can be fixed more quickly. And the idea that one can simply think about, test for, and avoid security problems presumes an element of perfection in security design that does not exist. This falls into the trap of enumerating badness. Moreover, rapid iterations also tend to be relatively minor iterations. It's easier to consider the impacts of the relatively small amount of change that can possibly be made in a one or two week sprint versus a longer (and larger) process. The collaborative nature of agile also suggests that more eyes from more diverse perspectives will have an opportunity to review systems as they are being developed, making vulnerabilities shallower, sooner.
There are times, however, when insularity is a good thing... putting it into coding or ops terms, think of it as a kind of modularity. A failure in one component should not effect a well-designed separate component in a system. It's not clear that the collaborative aspects of agile operations are a net security benefit when viewed in this light. It's not clear that they are not, either. But groupthink can be a powerful problem even before you get to a truly collaborative environment.
Where this applies in a discussion of the security function is that a remarkable number of security failures occur because of familiarity with the operational aspects of code. Developers already are quite familiar with the tendency for bugs to remain undiscovered by the individual who wrote the buggy code; this is not just due to the natural presumption that if it was recognizable to him, the coder wouldn't have introduced it in the first place, but also because in his mind, how the code should work is clear and he will tend to only access it in those expected ways in which it will work as designed, and not in other ways in which it will not. This is why many agile development paradigms call for pair programming and previously established unit tests, to get the coder's mind out of that rut. There is a school of thought that vulnerabilities are just a subset of bugs. I'm not convinced it's quite so clear, but I would agree at least that they are often introduced into code in the same manner.
This presents an argument for a security function that is almost the polar opposite of DevOps: not tightly coordinated at all, but separate, and kept intentionally in the dark on the inner workings of the development and operations team. Ernest Mueller has an interesting counter-argument that continues to talk up collaboration between developers, operations teams, and security experts. I like those arguments, too, but I'm not sure how they address that basic familiarity problem, or if they even intend to do so. Reading Ernest's post and reflecting on his observation that we have some twenty years of improved tools, best practices, and the like but still code just as insecurely, I wonder if either more collaboration or more insularity really gets at the heart of the problem: a basic indifference to security.
There are folks who are approaching this problem in uniquely agile ways. With "Rugged DevOps," Gene Kim suggests that we look to the advantages that rapid, collaborative iteration can offer in the security space. There's no need now, for instance, to wait three or six months on critical patches when vulnerabilities are discovered. And if security experts are brought into the DevOps team, they can help design in robust security practices from the start. It's striking how both Kim and Mueller emphasize the need to make the utility of these security concepts and practices clear to coders for the ideas to work. That may seem obvious at some level, but it's less clear to me that it can be done at all. Both men point out that security is a drag on feature development. This isn't just a drag at the developer level, this is a drag at the organizational level, and part of the reason security receives short shrift is because there are rarely strong business forces compelling it. Sometimes, it just makes financial sense to run risks, and that's what companies do when presented with a choice between putting in a feature that will help sell the product and skimping on a security risk that might never come to light.
The tension between agility and security seems obvious: rapid, iterative change leaves little time for detecting and defending against vulnerabilities that may naturally arise during the process. On further reflection, however, it's not so obvious that the bad outweighs the good in iteration. True, moving quickly may introduce vulnerabilities that could have been avoided with further consideration. However, it also means that any vulnerabilities that are introduced can be fixed more quickly. And the idea that one can simply think about, test for, and avoid security problems presumes an element of perfection in security design that does not exist. This falls into the trap of enumerating badness. Moreover, rapid iterations also tend to be relatively minor iterations. It's easier to consider the impacts of the relatively small amount of change that can possibly be made in a one or two week sprint versus a longer (and larger) process. The collaborative nature of agile also suggests that more eyes from more diverse perspectives will have an opportunity to review systems as they are being developed, making vulnerabilities shallower, sooner.
There are times, however, when insularity is a good thing... putting it into coding or ops terms, think of it as a kind of modularity. A failure in one component should not effect a well-designed separate component in a system. It's not clear that the collaborative aspects of agile operations are a net security benefit when viewed in this light. It's not clear that they are not, either. But groupthink can be a powerful problem even before you get to a truly collaborative environment.
Where this applies in a discussion of the security function is that a remarkable number of security failures occur because of familiarity with the operational aspects of code. Developers already are quite familiar with the tendency for bugs to remain undiscovered by the individual who wrote the buggy code; this is not just due to the natural presumption that if it was recognizable to him, the coder wouldn't have introduced it in the first place, but also because in his mind, how the code should work is clear and he will tend to only access it in those expected ways in which it will work as designed, and not in other ways in which it will not. This is why many agile development paradigms call for pair programming and previously established unit tests, to get the coder's mind out of that rut. There is a school of thought that vulnerabilities are just a subset of bugs. I'm not convinced it's quite so clear, but I would agree at least that they are often introduced into code in the same manner.
This presents an argument for a security function that is almost the polar opposite of DevOps: not tightly coordinated at all, but separate, and kept intentionally in the dark on the inner workings of the development and operations team. Ernest Mueller has an interesting counter-argument that continues to talk up collaboration between developers, operations teams, and security experts. I like those arguments, too, but I'm not sure how they address that basic familiarity problem, or if they even intend to do so. Reading Ernest's post and reflecting on his observation that we have some twenty years of improved tools, best practices, and the like but still code just as insecurely, I wonder if either more collaboration or more insularity really gets at the heart of the problem: a basic indifference to security.
There are folks who are approaching this problem in uniquely agile ways. With "Rugged DevOps," Gene Kim suggests that we look to the advantages that rapid, collaborative iteration can offer in the security space. There's no need now, for instance, to wait three or six months on critical patches when vulnerabilities are discovered. And if security experts are brought into the DevOps team, they can help design in robust security practices from the start. It's striking how both Kim and Mueller emphasize the need to make the utility of these security concepts and practices clear to coders for the ideas to work. That may seem obvious at some level, but it's less clear to me that it can be done at all. Both men point out that security is a drag on feature development. This isn't just a drag at the developer level, this is a drag at the organizational level, and part of the reason security receives short shrift is because there are rarely strong business forces compelling it. Sometimes, it just makes financial sense to run risks, and that's what companies do when presented with a choice between putting in a feature that will help sell the product and skimping on a security risk that might never come to light.
Friday, January 20. 2012
The relationship between agility and perspicacity
Most consultants have those frustrating engagements from time to time where it seems like you and the client are talking past one another the whole time, where assumptions are not shared and ideas cannot be accurately conveyed no matter how verbose you are or how many different metaphors you attempt to exercise on the subject. It can feel like you are working in molasses, where every step is difficult and is likely to result in a slip back from the ideal solution to the issues at hand. Over time, I've come to realize that clients like these, no matter how determined both parties are to make the relationship work, are better referred elsewhere; those projects become time sinks which are rarely successful and leave no one happy even if actually completed.
Over the years, it has become easier for me to instinctively identify those clients, but it wasn't until I read Paul Graham's "A Word to the Resourceful" today that I began to understand why: it's exactly because they are difficult to talk to. Graham, a programmer and investor who co-founded Ycombinator, noticed that the least successful startups he funded had a common feature: they were all hard for him to talk to. Successful startups, on the other hand, he found it easy to talk to.
Graham is careful to unpack this concept and clarify that it's not because people are difficult to talk to that they are unsuccessful, but rather that the mechanism that makes them difficult to talk to is one that indicates a lack of resourcefulness or, as I would put it, perspicacity. The "resourceful" startups were able to latch onto the implications of his statements relatively easily and comprehend the background and intent without a great deal of explanation. Moreover, he found that they were willing to follow those implications all the way, even when they might lead to unpleasant conclusions. The less resourceful startups were either unable or unwilling to chase down the implications, lacking the "conversational resourcefulness" to extrapolate in the way the more successful would. That lack of mental agility doesn't indicate a dearth of intellectual horsepower, but a sort of innate conservatism or set of cultural blinders that can inhibit understanding new or unfamiliar ideas.
It strikes me that agile fits pretty neatly into that category of new and unfamiliar, and that those conversations in which I find myself flailing with certain clients is not necessarily, as I had previously supposed, a simple communications failure or a basic personality conflict. Instead, it may very well indicate a lack of perspicacity, an unwillingness to follow the implications all the way to the sometimes uncomfortable conclusions. There is much in the agile operations arena that represents a threat to conventional IT operations teams and by-the-book CIOs. That much is a given, and I routinely try to address those concerns when discussing the topic. However, it had not occurred to me that my careful assurances and discussions of alternatives weren't getting through not because the arguments themselves were being rejected, but because the premise of the original assertion was never understood. Rather than discussing the topics on their merits, such clients may be unable to understand or follow the chain of implications far enough to continue a rational conversation. Clarifications, in these cases, typically do not improve matters; you may be speaking from a level of understanding the client is unwilling to join you at... to the point where they may believe that you are confused.
None of this rules out more conventional misunderstandings or explanatory difficulties, and those all still exist and must be overcome when discussing agile operations. However, it provides a neat heuristic with which to gauge certain interactions and potentially determine early on whether it is worth the time proceeding with the conversation you had planned, or whether your client lacks the perspicacity to ever unravel what you are saying.
Over the years, it has become easier for me to instinctively identify those clients, but it wasn't until I read Paul Graham's "A Word to the Resourceful" today that I began to understand why: it's exactly because they are difficult to talk to. Graham, a programmer and investor who co-founded Ycombinator, noticed that the least successful startups he funded had a common feature: they were all hard for him to talk to. Successful startups, on the other hand, he found it easy to talk to.
Graham is careful to unpack this concept and clarify that it's not because people are difficult to talk to that they are unsuccessful, but rather that the mechanism that makes them difficult to talk to is one that indicates a lack of resourcefulness or, as I would put it, perspicacity. The "resourceful" startups were able to latch onto the implications of his statements relatively easily and comprehend the background and intent without a great deal of explanation. Moreover, he found that they were willing to follow those implications all the way, even when they might lead to unpleasant conclusions. The less resourceful startups were either unable or unwilling to chase down the implications, lacking the "conversational resourcefulness" to extrapolate in the way the more successful would. That lack of mental agility doesn't indicate a dearth of intellectual horsepower, but a sort of innate conservatism or set of cultural blinders that can inhibit understanding new or unfamiliar ideas.
It strikes me that agile fits pretty neatly into that category of new and unfamiliar, and that those conversations in which I find myself flailing with certain clients is not necessarily, as I had previously supposed, a simple communications failure or a basic personality conflict. Instead, it may very well indicate a lack of perspicacity, an unwillingness to follow the implications all the way to the sometimes uncomfortable conclusions. There is much in the agile operations arena that represents a threat to conventional IT operations teams and by-the-book CIOs. That much is a given, and I routinely try to address those concerns when discussing the topic. However, it had not occurred to me that my careful assurances and discussions of alternatives weren't getting through not because the arguments themselves were being rejected, but because the premise of the original assertion was never understood. Rather than discussing the topics on their merits, such clients may be unable to understand or follow the chain of implications far enough to continue a rational conversation. Clarifications, in these cases, typically do not improve matters; you may be speaking from a level of understanding the client is unwilling to join you at... to the point where they may believe that you are confused.
None of this rules out more conventional misunderstandings or explanatory difficulties, and those all still exist and must be overcome when discussing agile operations. However, it provides a neat heuristic with which to gauge certain interactions and potentially determine early on whether it is worth the time proceeding with the conversation you had planned, or whether your client lacks the perspicacity to ever unravel what you are saying.
Wednesday, November 30. 2011
Tipu too
It may be inevitable that, after a long enough period of time spent as a critic, a person is inspired to present their own ideas as an improvement or alternative to the concepts being critiqued. Since Rob England, the man behind the widely-read ITIL-focusedIT Skeptic blog, has been hectoring the ITIL community for going on six years now, it's about time he come out with something of his own in the service management improvement line, and he has finally done so in the form of Tipu, a framework for service management Continual Service Improvement (CSI).
Rob doesn't restrict his focus to Information Technology only, however, stressing that Tipu is applicable to any sort of service management. In that, and many of the other basic philosophical underpinnings, Tipu warms my heart; an excessive focus on technology is one of the worst things wrong with IT departments today, odd as that may sound. Tipu seeks to make process improvement a part of business as usual, an approach that has the benefit of not only being workable, but the virtue of being pretty much the only reliable mechanism for effecting change. It seeks real, genuine business needs as the underpinnings for these improvements. And nestled in amongst the fourteen principles the framework is based on, old friends like humanity, granularity, agility, and measurement are all to be found alongside companions of equal pedigree in the Agile pantheon.
But it's that list of fourteen that sets off the first warning bells about Tipu. It's not the contents of the list so much as the length... and this defect continues throughout the parts of the framework published to date. To my view, it appears excessively complex. ITIL also tends in this direction, so perhaps it is no surprise that Tipu does as well. England basically admits as much, stating, "...the Framework attempts to be more complete than ITIL or COBIT" and "The Tipu Framework represents just about all of the essential components (practices and functions) of service management best practice. None of these are optional." When you take a look at the circular graphic illustrating the fifty mandatory practices and functions, you can squint a little bit and see the nine circles of process hell awaiting you.
Don't get me wrong, managing services in any large, diverse environment is bound to get complicated; however, precisely because of that I think it's wrong-footed to start off with a framework that is not simple itself. The philosophical underpinnings of the framework acknowledge the organic and evolutionary nature of systems implementation, and lip-service is even paid to the idea that it might be implemented from the ground up. But it's difficult to take any talk of gradualism all that seriously when the framework assumes (and indeed, even seems to demand) that it be birthed whole from some senior station in the executive suite with complete control over fifty practices and functions that may, in many organizations, not actually exist at the point of the framework's guidance.
Perhaps I'm being too hard on a product that is effectively still in beta testing, and which England warns outright "...we do not recommend that you adopt it as your organisation’s reference framework." And he is equally forthright in declaring that frameworks are guidance and not law, and that, as the name suggest, they be built upon rather than placed as a structure themselves. However, it's difficult to successfully reconcile that flexible perspective with implications suggesting that, if you are really serious about service improvement, you'd better not start skipping steps. It's up to you if you want to only wear the minimum number of pieces of flair, but Brian, for instance, has 37 pieces, so I think we all know what that means.
In my experience, programs tend to come out of testing with more cruft, not less, so I won't hold my breath that Tipu will get trimmed down after a few real-world runs. On the other hand, it's open source; all the documents available so far are covered under Creative Commons licensing, so there's always the possibility that the community will take it up and run with it... provided a community emerges around it.
Rob doesn't restrict his focus to Information Technology only, however, stressing that Tipu is applicable to any sort of service management. In that, and many of the other basic philosophical underpinnings, Tipu warms my heart; an excessive focus on technology is one of the worst things wrong with IT departments today, odd as that may sound. Tipu seeks to make process improvement a part of business as usual, an approach that has the benefit of not only being workable, but the virtue of being pretty much the only reliable mechanism for effecting change. It seeks real, genuine business needs as the underpinnings for these improvements. And nestled in amongst the fourteen principles the framework is based on, old friends like humanity, granularity, agility, and measurement are all to be found alongside companions of equal pedigree in the Agile pantheon.
But it's that list of fourteen that sets off the first warning bells about Tipu. It's not the contents of the list so much as the length... and this defect continues throughout the parts of the framework published to date. To my view, it appears excessively complex. ITIL also tends in this direction, so perhaps it is no surprise that Tipu does as well. England basically admits as much, stating, "...the Framework attempts to be more complete than ITIL or COBIT" and "The Tipu Framework represents just about all of the essential components (practices and functions) of service management best practice. None of these are optional." When you take a look at the circular graphic illustrating the fifty mandatory practices and functions, you can squint a little bit and see the nine circles of process hell awaiting you.
Don't get me wrong, managing services in any large, diverse environment is bound to get complicated; however, precisely because of that I think it's wrong-footed to start off with a framework that is not simple itself. The philosophical underpinnings of the framework acknowledge the organic and evolutionary nature of systems implementation, and lip-service is even paid to the idea that it might be implemented from the ground up. But it's difficult to take any talk of gradualism all that seriously when the framework assumes (and indeed, even seems to demand) that it be birthed whole from some senior station in the executive suite with complete control over fifty practices and functions that may, in many organizations, not actually exist at the point of the framework's guidance.
Perhaps I'm being too hard on a product that is effectively still in beta testing, and which England warns outright "...we do not recommend that you adopt it as your organisation’s reference framework." And he is equally forthright in declaring that frameworks are guidance and not law, and that, as the name suggest, they be built upon rather than placed as a structure themselves. However, it's difficult to successfully reconcile that flexible perspective with implications suggesting that, if you are really serious about service improvement, you'd better not start skipping steps. It's up to you if you want to only wear the minimum number of pieces of flair, but Brian, for instance, has 37 pieces, so I think we all know what that means.
In my experience, programs tend to come out of testing with more cruft, not less, so I won't hold my breath that Tipu will get trimmed down after a few real-world runs. On the other hand, it's open source; all the documents available so far are covered under Creative Commons licensing, so there's always the possibility that the community will take it up and run with it... provided a community emerges around it.
Friday, September 23. 2011
Non-negotiable features ARE software products
Matthias Marschall lays out a common and lamentable scenario in his recent post on How Non-negotiable Features Kill Software Products: sales staff run amuck, promising deliverables to prospective customers without discussing those features or related "technical debt" to be incurred in the process of developing them. Most of us have seen this in one form or another on the technical side of the table... whether you are in development or operations, someone, at some point, has probably signed you up for something you think is a bad idea that is going to take a lot of resources without providing much long-term leverage in return.
This is pretty frustrating and, as Matthias points out, it's often ultimately self-defeating for the company doing the development: the features are rarely fully fleshed out and fail to live up to the hype they have been imbued with by the sales team, simply serving to disappoint those very same customers who signed on for the feature in the first place.
Matthias pins this problem on the idea of non-negotiable features being sold as parts of the product in the first place, but this presents a development-centric view of software sales that entirely misses the point from the customer perspective. For the customer, software with "negotiable" features is called something else: vaporware. There are a certain percentage of gullible customers who buy the sales song and dance and sign up for features they don't really need, but by and large, when users commit to purchasing software, it is because it includes features that solve business problems for them. Except for certain very specialized requirements, negotiation is the last thing they are interested in. They want to know what the product does or is going to do, and they want to be able to make a purchasing decision on that basis.
Customers might have requests and sales guys might have feature ideas they think will sell more product, but the problem Matthias describes isn't really that of "non-negotiable features." It's a poor internal process that allows sales staff to make commitments they are not accountable for, or a development process that's too inflexible to respond to important feature requests, or a lack of management control over product direction. Non-negotiable features are, from the customer perspective, just features. If the product doesn't have them, they have no reason to buy it. Resource and commitment issues internal to the developer are not generally a problem they want to be involved with.
Agile certainly has some answers to these issues, but as I have pointed out previously, there are a lot of software products and projects that simply aren't ever going to be amenable to the traditional sort of user/developer interaction envisioned by processes like XP or DevOps. Unfortunately, when you get to the point of blaming the market for being what it is, you lose the ability to rationally apply agile solutions to the actual issues.
This is pretty frustrating and, as Matthias points out, it's often ultimately self-defeating for the company doing the development: the features are rarely fully fleshed out and fail to live up to the hype they have been imbued with by the sales team, simply serving to disappoint those very same customers who signed on for the feature in the first place.
Matthias pins this problem on the idea of non-negotiable features being sold as parts of the product in the first place, but this presents a development-centric view of software sales that entirely misses the point from the customer perspective. For the customer, software with "negotiable" features is called something else: vaporware. There are a certain percentage of gullible customers who buy the sales song and dance and sign up for features they don't really need, but by and large, when users commit to purchasing software, it is because it includes features that solve business problems for them. Except for certain very specialized requirements, negotiation is the last thing they are interested in. They want to know what the product does or is going to do, and they want to be able to make a purchasing decision on that basis.
Customers might have requests and sales guys might have feature ideas they think will sell more product, but the problem Matthias describes isn't really that of "non-negotiable features." It's a poor internal process that allows sales staff to make commitments they are not accountable for, or a development process that's too inflexible to respond to important feature requests, or a lack of management control over product direction. Non-negotiable features are, from the customer perspective, just features. If the product doesn't have them, they have no reason to buy it. Resource and commitment issues internal to the developer are not generally a problem they want to be involved with.
Agile certainly has some answers to these issues, but as I have pointed out previously, there are a lot of software products and projects that simply aren't ever going to be amenable to the traditional sort of user/developer interaction envisioned by processes like XP or DevOps. Unfortunately, when you get to the point of blaming the market for being what it is, you lose the ability to rationally apply agile solutions to the actual issues.
Tuesday, February 8. 2011
Training: the last refuge of the incompetent
Training is a staple in corporate information technology systems, or, if the corporation is too cheap to provide actual training, then it is at least usually a mantra; if there isn't already quite a lot of it going on, management either promises or demands (according to their style) more. This is not, unsurprisingly, usually a priority of the IT department itself, where people still work for a living. It's usually something driven by human resources types, who have had continuing education drummed into them as a means of justification by all their own training.
Training has long been a necessary evil in IT, long enough that few people understand that any actual need for it is in fact indicative of a very poorly designed or operated system. Instead, it is seen as both desirable and effective by many managers, both inside and outside the IT department.
You can get a glimpse of both sides of this issue in today's Wall Street Journal article discussing Delta Airlines' decision to send all 11,000 of its gate agents back to customer-service classes as a first step toward addressing the industry-low customer satisfaction levels the corporation suffered last year.
The article points out that many of the ills experienced by the airline are the result of actual operational failings that resulted in cancellations or delays. The agents were simply dealing with the fall-out of those problems, and apparently doing so quite poorly. The airline does seem to realize that fixing the core problem, by improving actual service (rather than lip-service, which is the only sort that an agent can provide), is important and is something that will also be addressed.
Nonetheless, it's telling that gate agent training is the first item on the agenda. It's indicative of an all-too-popular mindset that simply training staff can overcome core deficiencies. That's not likely to work out any better for Delta than anywhere else I have seen it tried. Instead, it is likely to simply breed cynicism and swallow funds that might otherwise have gone to addressing the root problems in a more effective manner.
Of course some level of training may be required in core job skills and for customer service agents the basics of customer service have to be instilled at some point prior to taking the desk. Still; as Antoine de Saint Exupéry once said, "If you want to build a ship, don't drum up people to collect wood and don't assign them tasks and work, but rather teach them to long for the endless immensity of the sea." His point, which is one that I often make with much less eloquence, is that you have to incentive the outcomes you are looking for. If you want a guy to build a ship then yes, you probably need to show him how to pay and caulk, but what is really going to drive him is a desire to go sailing on it. Similarly, if you want agents who provide excellent customer service, then you might tell them to smile, but what is actually going to make it happen is instilling in them a genuine desire to make customers happy and provide a good experience.
My guess is that if some chunk of the $2 billion that Delta is sinking into making these improvements were taken out of trainer's pockets and instead put toward some incentive program rewarding agents for genuinely cheerful and helpful service, you would see a lot more bang for the buck out of it. A quote in the article from a passenger crystallizes the issue: "It's like they don't enjoy their jobs," he says of the employees. That's probably exactly the problem. And if they don't enjoy their jobs, no amount of telling them to smile or giving them alternative methods of dealing with customers is going to change that. Instead, you have to work to make it an enjoyable job.
This is as true in information technology as anywhere else. Something I find pretty consistently (although, admittedly, anecdotally) is that support incidents are fewer and less impactful where staff seem happier*. This isn't necessarily because the systems actually work better, but instead because they are more willing to work through or around problems. Making them happy isn't usually a matter of training, either. It's more deeply involved with the daily rhythms of their jobs. And the one thing that the IT department can generally influence in the jobs of other staff is to what degree information systems are either an aid or a hindrance in performing their duties.
My argument is that most money spent on training would be far better spent by investing in systems that are easier to use and that make work easier for employees. Conversely, when you put that money into systems that are hard to use and that present an obstacle to employees, you are creating a problem that no amount of training will solve.
In either case, training is what you end up doing when you can't find a way to do things right the first time.
*Interestingly, there's actually a curve here; there are also generally fewer support incidents in environments where the staff are very unhappy as well, or at least unhappy with the IT department support process. They get to a point where they realize nothing is actually going to be done about their problems, so they simply stop reporting them. It's in the middle, where folks still have some element of hope mixed in with their depression, that you get the most reported problems.
Training has long been a necessary evil in IT, long enough that few people understand that any actual need for it is in fact indicative of a very poorly designed or operated system. Instead, it is seen as both desirable and effective by many managers, both inside and outside the IT department.
You can get a glimpse of both sides of this issue in today's Wall Street Journal article discussing Delta Airlines' decision to send all 11,000 of its gate agents back to customer-service classes as a first step toward addressing the industry-low customer satisfaction levels the corporation suffered last year.
The article points out that many of the ills experienced by the airline are the result of actual operational failings that resulted in cancellations or delays. The agents were simply dealing with the fall-out of those problems, and apparently doing so quite poorly. The airline does seem to realize that fixing the core problem, by improving actual service (rather than lip-service, which is the only sort that an agent can provide), is important and is something that will also be addressed.
Nonetheless, it's telling that gate agent training is the first item on the agenda. It's indicative of an all-too-popular mindset that simply training staff can overcome core deficiencies. That's not likely to work out any better for Delta than anywhere else I have seen it tried. Instead, it is likely to simply breed cynicism and swallow funds that might otherwise have gone to addressing the root problems in a more effective manner.
Of course some level of training may be required in core job skills and for customer service agents the basics of customer service have to be instilled at some point prior to taking the desk. Still; as Antoine de Saint Exupéry once said, "If you want to build a ship, don't drum up people to collect wood and don't assign them tasks and work, but rather teach them to long for the endless immensity of the sea." His point, which is one that I often make with much less eloquence, is that you have to incentive the outcomes you are looking for. If you want a guy to build a ship then yes, you probably need to show him how to pay and caulk, but what is really going to drive him is a desire to go sailing on it. Similarly, if you want agents who provide excellent customer service, then you might tell them to smile, but what is actually going to make it happen is instilling in them a genuine desire to make customers happy and provide a good experience.
My guess is that if some chunk of the $2 billion that Delta is sinking into making these improvements were taken out of trainer's pockets and instead put toward some incentive program rewarding agents for genuinely cheerful and helpful service, you would see a lot more bang for the buck out of it. A quote in the article from a passenger crystallizes the issue: "It's like they don't enjoy their jobs," he says of the employees. That's probably exactly the problem. And if they don't enjoy their jobs, no amount of telling them to smile or giving them alternative methods of dealing with customers is going to change that. Instead, you have to work to make it an enjoyable job.
This is as true in information technology as anywhere else. Something I find pretty consistently (although, admittedly, anecdotally) is that support incidents are fewer and less impactful where staff seem happier*. This isn't necessarily because the systems actually work better, but instead because they are more willing to work through or around problems. Making them happy isn't usually a matter of training, either. It's more deeply involved with the daily rhythms of their jobs. And the one thing that the IT department can generally influence in the jobs of other staff is to what degree information systems are either an aid or a hindrance in performing their duties.
My argument is that most money spent on training would be far better spent by investing in systems that are easier to use and that make work easier for employees. Conversely, when you put that money into systems that are hard to use and that present an obstacle to employees, you are creating a problem that no amount of training will solve.
In either case, training is what you end up doing when you can't find a way to do things right the first time.
*Interestingly, there's actually a curve here; there are also generally fewer support incidents in environments where the staff are very unhappy as well, or at least unhappy with the IT department support process. They get to a point where they realize nothing is actually going to be done about their problems, so they simply stop reporting them. It's in the middle, where folks still have some element of hope mixed in with their depression, that you get the most reported problems.
Wednesday, November 17. 2010
Blurring the problems
I am torn by Damon Edwards' post "DevOps is not a technology problem. DevOps is a business problem."
Damon's point is that the focus of DevOps to date has been primarily about the classic problems of friction between development and deployment, but that in fact the principles that inform it can be applied more generally and with equal success to business problems beyond that particular divide.
It's a great point and a perspective that I happen to share. But please, let's not try to wedge all that into DevOps. Damon later tries to argue that DevOps is distinct from a closely related cousin, Agile Development... that although the principles involved are more or less identical, they function on different planes. He's exactly right. But the same is true of DevOps and Agile Operations... though closely related, they remain on different planes. Trying to bring Agile Ops to bear in an organization where (as with most organizations) development is not even an internal function, or when trying to apply it to areas in IT that don't really involve development at all, and you are blurring much of what makes "DevOps" effective within it's original area of focus... just as if developers had taken Agile Development and tried to simply stretch it out, as-is, to cover deployment and operations.
Damon's point is that the focus of DevOps to date has been primarily about the classic problems of friction between development and deployment, but that in fact the principles that inform it can be applied more generally and with equal success to business problems beyond that particular divide.
It's a great point and a perspective that I happen to share. But please, let's not try to wedge all that into DevOps. Damon later tries to argue that DevOps is distinct from a closely related cousin, Agile Development... that although the principles involved are more or less identical, they function on different planes. He's exactly right. But the same is true of DevOps and Agile Operations... though closely related, they remain on different planes. Trying to bring Agile Ops to bear in an organization where (as with most organizations) development is not even an internal function, or when trying to apply it to areas in IT that don't really involve development at all, and you are blurring much of what makes "DevOps" effective within it's original area of focus... just as if developers had taken Agile Development and tried to simply stretch it out, as-is, to cover deployment and operations.
Thursday, October 28. 2010
Dawn of a New Day... Ray must be talking about Agile Operations
It's been a few days since Steve Ballmer announced that Ray Ozzie will no longer be serving as Microsoft's Chief Software Architect (and indeed that Microsoft will no longer have a position of Chief Software Architect) so you've probably already had a chance to digest the implications of that move. But yesterday (although it is dated, oddly, two days from now) Ray published a capstone, or rebuttal, or perhaps just another long diatribe titled Dawn of a New Day discussing his view of the "post-PC world" and, at a coolly intellectual arms-length, his impact on Microsoft's current strategy and his thoughts on what its future strategy must be.
I say "coolly" because it sounds very much as if Ray wants to explode a little bit. He came into the position five years ago with a set of ideas that had the company on an even plane with Amazon, Apple, and Google (see his The Internet Services Disruption memo... oh, hey, there's the reason Dawn of a New Day is dated the 28th, that makes it exactly five years to the day after the first memo. Clever, Ray, very clever) with respect to a vision of how the industry was poised for sudden changes. As he alludes to discretely in his more recent memo, Microsoft has suffered a massive failure in execution in realizing any profit from this foresight, and if one evaluates even the successes that Ozzie notes against their competitors, it doesn't look like much to cheer about.
But Ray isn't bitter, and he's giving the company a few parting words of advice on his way to (hopefully) greener pastures: "Complexity Kills"
It has taken quite a while for agile development practices to make any headway at Microsoft, although they are well-entrenched there in many teams now. There's little hope that Agile Operations will penetrate any more quickly, particularly in view of the threat to the traditional revenue driving products that it presents. Because, as Ozzie is quick to point out, complexity has long represented a degree of lock-in to conventional software products, lock-in that Microsoft most of all has profited from in the PC industry.
But this is less about Microsoft, which is fast fading as a company of much interest to agile proponents (if it ever held much of their interest in the first place), and more about how they might represent the status quo in most companies that could benefit from implementing agile operations practices. As Larry Dignan notes, the identification of complexity as the killer issue in modern information technology makes the solution less about technology than about management.
This is where the real difficulty almost always lies, and modern corporate bureaucracy is almost universally oriented toward creating complexity rather than removing it. Living close by to the corporate headquarters of Microsoft and having watched it through the years, I can see this sclerosis as a significant factor in the company's current problems. It's a factor in a lot of other organizations where it may be less obvious, as well.
Agile Operations isn't going to cure Microsoft's ills, any more than it can fix problems at any company which has lost touch with its market, but it can help prevent the microcosm of corporate sclerosis from infesting and hardening the IT department, at least. And the easiest and best way to sell it to those who need to buy into it is the mantra that Ray Ozzie has adopted: Complexity Kills. It's a lesson that has been learned from manufacturing to software development, and now it's time for IT operations to learn it as well.
Complexity has served IT departments just as it has served Microsoft over the years, effectively locking the user in to IT in the same way that IT has been locked in to Microsoft. IT is now threatened by the same trends that are threatening Microsoft and for many of the same reasons... the post PC era unlocks the user.
Ozzie's vision of connected devices using continuous services is the ideal for most users, just as it is a threat to the control of the organizations. The vision will almost inevitably become reality, at some date and in some fashion. The challenge for any organization that hopes to stay relevant is to get out ahead of the trend and help make it happen instead of standing by clinging to the old patterns.
Other than pointing out this necessity, Ozzie doesn't really lay out a clear new pattern for organizations to follow. It is unpalatable for Microsoft to suggest that it may simply, inevitably, even if it makes all the right moves, become less relevant to the future of information technology. The same may be true for CIOs and IT departments. But recognizing the future and embracing it at least affords the opportunity to remain somewhat relevant. If there is no clear path for Microsoft in that world, I think that in Agile Operations, IT departments at least have some hope ahead of them.
I say "coolly" because it sounds very much as if Ray wants to explode a little bit. He came into the position five years ago with a set of ideas that had the company on an even plane with Amazon, Apple, and Google (see his The Internet Services Disruption memo... oh, hey, there's the reason Dawn of a New Day is dated the 28th, that makes it exactly five years to the day after the first memo. Clever, Ray, very clever) with respect to a vision of how the industry was poised for sudden changes. As he alludes to discretely in his more recent memo, Microsoft has suffered a massive failure in execution in realizing any profit from this foresight, and if one evaluates even the successes that Ozzie notes against their competitors, it doesn't look like much to cheer about.
But Ray isn't bitter, and he's giving the company a few parting words of advice on his way to (hopefully) greener pastures: "Complexity Kills"
It has taken quite a while for agile development practices to make any headway at Microsoft, although they are well-entrenched there in many teams now. There's little hope that Agile Operations will penetrate any more quickly, particularly in view of the threat to the traditional revenue driving products that it presents. Because, as Ozzie is quick to point out, complexity has long represented a degree of lock-in to conventional software products, lock-in that Microsoft most of all has profited from in the PC industry.
But this is less about Microsoft, which is fast fading as a company of much interest to agile proponents (if it ever held much of their interest in the first place), and more about how they might represent the status quo in most companies that could benefit from implementing agile operations practices. As Larry Dignan notes, the identification of complexity as the killer issue in modern information technology makes the solution less about technology than about management.
This is where the real difficulty almost always lies, and modern corporate bureaucracy is almost universally oriented toward creating complexity rather than removing it. Living close by to the corporate headquarters of Microsoft and having watched it through the years, I can see this sclerosis as a significant factor in the company's current problems. It's a factor in a lot of other organizations where it may be less obvious, as well.
Agile Operations isn't going to cure Microsoft's ills, any more than it can fix problems at any company which has lost touch with its market, but it can help prevent the microcosm of corporate sclerosis from infesting and hardening the IT department, at least. And the easiest and best way to sell it to those who need to buy into it is the mantra that Ray Ozzie has adopted: Complexity Kills. It's a lesson that has been learned from manufacturing to software development, and now it's time for IT operations to learn it as well.
Complexity has served IT departments just as it has served Microsoft over the years, effectively locking the user in to IT in the same way that IT has been locked in to Microsoft. IT is now threatened by the same trends that are threatening Microsoft and for many of the same reasons... the post PC era unlocks the user.
Ozzie's vision of connected devices using continuous services is the ideal for most users, just as it is a threat to the control of the organizations. The vision will almost inevitably become reality, at some date and in some fashion. The challenge for any organization that hopes to stay relevant is to get out ahead of the trend and help make it happen instead of standing by clinging to the old patterns.
Other than pointing out this necessity, Ozzie doesn't really lay out a clear new pattern for organizations to follow. It is unpalatable for Microsoft to suggest that it may simply, inevitably, even if it makes all the right moves, become less relevant to the future of information technology. The same may be true for CIOs and IT departments. But recognizing the future and embracing it at least affords the opportunity to remain somewhat relevant. If there is no clear path for Microsoft in that world, I think that in Agile Operations, IT departments at least have some hope ahead of them.
Monday, October 18. 2010
Manifestos and Marketing Agile
And speaking of the Agile Manifesto, the always thoughtful Ernest Mueller at the agile admin asks whether or not DevOps really needs its own separate manifesto, or can we perhaps all just get along and agree to apply the original to both development and operations?
I think Ernest is dead right and that there is no need to reinvent the wheel on this; I think agile operations is all about adopting exactly those values and concepts in the operations field. If we're going to borrow the ideas, can't we just borrow the manifesto and not waste time coming up with our own? Re-use is a golden concept in both development and operations.
All that said, Ernest does go on to make a few adjustments to the Twelve Principles of of Agile Software that the Manifesto authors laid out. They're good edits but probably unnecessary for anyone already in the mindset of adopting agility; if you think the concepts are good, you are probably already reading between the lines with an understanding that the delivery of the promises implied requires serviceable and flexible systems. It might seem, then, a bit of a waste to put the time into revising them, but as was discussed in the comments to the post, a "manifesto" of any sort is largely a marketing document. Even if the audience is primarily other admins or other developers, none of whom will spend a dime on the "product," there is a competition for mindshare between this concept of IT operations and others, and that mindshare is increased through marketing. A working manifesto that lays the concepts out clearly to all parties is an asset if that is your goal.
I think Ernest is dead right and that there is no need to reinvent the wheel on this; I think agile operations is all about adopting exactly those values and concepts in the operations field. If we're going to borrow the ideas, can't we just borrow the manifesto and not waste time coming up with our own? Re-use is a golden concept in both development and operations.
All that said, Ernest does go on to make a few adjustments to the Twelve Principles of of Agile Software that the Manifesto authors laid out. They're good edits but probably unnecessary for anyone already in the mindset of adopting agility; if you think the concepts are good, you are probably already reading between the lines with an understanding that the delivery of the promises implied requires serviceable and flexible systems. It might seem, then, a bit of a waste to put the time into revising them, but as was discussed in the comments to the post, a "manifesto" of any sort is largely a marketing document. Even if the audience is primarily other admins or other developers, none of whom will spend a dime on the "product," there is a competition for mindshare between this concept of IT operations and others, and that mindshare is increased through marketing. A working manifesto that lays the concepts out clearly to all parties is an asset if that is your goal.
Leading, Managing, and finding Agility
Martin Glassborow talks a little bit about leadership and management as they relate to agility, with a side of explaining how we all got here in terms of the decidedly un-agile aspect into which most IT departments have settled themselves.
Leadership is a significant issue at this stage in IT, and Glassborow has some salient observations about how it evaporated along the road to today. The complexity that IT departments have built into their services are their own biggest impediments to change; the vital nature of the services has made any changes extraordinarily fraught with risk for those who have been responsible both for building the original systems and are now on the hook for upgrading them. This has bred an extremely risk-averse culture of management in most information technology departments, a culture that my last post was largely aimed toward. At this point, they're in a position where they have to get across the river of change from old economic models to new, and the people who need to do it are the sort who will try to jump the river in two bounds. That's not now it happens.
The grunt-level DevOps movement is taking off under these conditions because the developers and sysadmins aren't as constrained as CIOs and IT managers, but Agile Operations as a whole, the top to bottom adoption of agile principles into IT department operations (not just in IT/development interaction), isn't something that is going to get off the ground without executive support. It's not something that will be managed into being; leadership will have to be exercised.
Glassborow does point out an intriguingly positive point that may work for AO adopters; he notes that, "The move to agile development techniques has been remarkably quick compared to say that of take up of object orientated techniques which can only be attributed to that there is something right and easily implementable about the techniques."
There is certainly something to that; the Agile Manifesto is remarkably easy to understand, even for executives with no intimate knowledge of coding mechanics. And while the implementation can become deeply and vastly subtle in a nearly Zen-like way, the basics are simple to adopt, and self-reinforcing. If this can be said to be true of agility in general, then there is no reason that Agile Operations should not benefit from the same factors.
Leadership is a significant issue at this stage in IT, and Glassborow has some salient observations about how it evaporated along the road to today. The complexity that IT departments have built into their services are their own biggest impediments to change; the vital nature of the services has made any changes extraordinarily fraught with risk for those who have been responsible both for building the original systems and are now on the hook for upgrading them. This has bred an extremely risk-averse culture of management in most information technology departments, a culture that my last post was largely aimed toward. At this point, they're in a position where they have to get across the river of change from old economic models to new, and the people who need to do it are the sort who will try to jump the river in two bounds. That's not now it happens.
The grunt-level DevOps movement is taking off under these conditions because the developers and sysadmins aren't as constrained as CIOs and IT managers, but Agile Operations as a whole, the top to bottom adoption of agile principles into IT department operations (not just in IT/development interaction), isn't something that is going to get off the ground without executive support. It's not something that will be managed into being; leadership will have to be exercised.
Glassborow does point out an intriguingly positive point that may work for AO adopters; he notes that, "The move to agile development techniques has been remarkably quick compared to say that of take up of object orientated techniques which can only be attributed to that there is something right and easily implementable about the techniques."
There is certainly something to that; the Agile Manifesto is remarkably easy to understand, even for executives with no intimate knowledge of coding mechanics. And while the implementation can become deeply and vastly subtle in a nearly Zen-like way, the basics are simple to adopt, and self-reinforcing. If this can be said to be true of agility in general, then there is no reason that Agile Operations should not benefit from the same factors.
Friday, October 8. 2010
Are CIOs "under siege" by expectations?
So contends Red Hat CEO Jim Whitehurst, and I know a lot of CIOs who would agree with him.
In ComputerWorld, Whitehurst said that Google Docs and other free, scalable, easy-to-use cloud-based services are conditioning users to expect vastly capable services at very little cost from their IT departments... and those departments aren't able to deliver.
As an example you've probably already heard, Whitehurst trots out the following anecdote:
It's only natural for CIOs, and for IT departments in general, to feel a little defensive about this state of affairs. It's as if they are accomplished classical pianists suddenly asked to put on a Def Leppard style arena rock show. When they protest, the audience of users says, "It's music! You're musicians aren't you?" IT can only splutter unconvincingly, "But that's just not what we do." It's particularly difficult, because both parties are basically correct... they are just talking past one another. Continue reading "Are CIOs "under siege" by expectations?" »
In ComputerWorld, Whitehurst said that Google Docs and other free, scalable, easy-to-use cloud-based services are conditioning users to expect vastly capable services at very little cost from their IT departments... and those departments aren't able to deliver.
As an example you've probably already heard, Whitehurst trots out the following anecdote:
A few months ago the CIO was asked by the chief marketing officer to provide a way for marketing employees around the world to share and build documents together, and perform other collaborative tasks.
The CIO discussed the project with his application development group, then went back to the CMO and said "we can do this, in nine months at a cost of $14 million," according to Whitehurst.
"The CMO says 'what are you talking about? I was describing my daughter's high school science project.' And they were on Google Documents, sharing information, jointly editing documents, and they're doing it for free. This is a true story. I may have been slightly off on the numbers, but a true story."
It's only natural for CIOs, and for IT departments in general, to feel a little defensive about this state of affairs. It's as if they are accomplished classical pianists suddenly asked to put on a Def Leppard style arena rock show. When they protest, the audience of users says, "It's music! You're musicians aren't you?" IT can only splutter unconvincingly, "But that's just not what we do." It's particularly difficult, because both parties are basically correct... they are just talking past one another. Continue reading "Are CIOs "under siege" by expectations?" »
Friday, May 7. 2010
Agile Operations and the CIO
The thought that the CIO role and the state of the IT organization as a stand-alone entity in the modern corporation are due for a dramatic revision soon has not exactly hit the mainstream yet, but it's on the way. Since late 2007, there have been indications that the CIO is losing what little respect and autonomy the role ever had, and the forces leading to that diminution and the corresponding fading of the IT department as a whole in the role of corporate technology provider.
The latest missive in this vein comes from the Corporate Executive Board (CEB) in their whitepaper "The Future of Corporate IT." (PDF) The CEB are projecting a 75% drop in IT headcount in the next five years in many corporations, and with the current paradigm of "big headcount = big honcho" the equation does not favor the head of such a diminished department.
CIOs seem bound to make this worse than it has to be by stalling, digging in their heels on traditional initiatives, and working to preserve their headcount and budget even in the face of solutions that could both improve their efficiency and reduce their expenses at the same time. This may work for a while--I certainly think that the 75% reduction in five year prediction is aggressive considering the corporate infighting capabilities many IT organizations have developed--but when the time comes, it's going to be a harder and more permanent fall from grace than is really necessary. Continue reading "Agile Operations and the CIO" »
The latest missive in this vein comes from the Corporate Executive Board (CEB) in their whitepaper "The Future of Corporate IT." (PDF) The CEB are projecting a 75% drop in IT headcount in the next five years in many corporations, and with the current paradigm of "big headcount = big honcho" the equation does not favor the head of such a diminished department.
CIOs seem bound to make this worse than it has to be by stalling, digging in their heels on traditional initiatives, and working to preserve their headcount and budget even in the face of solutions that could both improve their efficiency and reduce their expenses at the same time. This may work for a while--I certainly think that the 75% reduction in five year prediction is aggressive considering the corporate infighting capabilities many IT organizations have developed--but when the time comes, it's going to be a harder and more permanent fall from grace than is really necessary. Continue reading "Agile Operations and the CIO" »
Thursday, April 29. 2010
Revisiting the "faster = crappier" conundrum
One of the ancient and original objections to implementing agile was the predictable "But if we release more frequently, we won't have time to make sure we're not releasing crap!" This is probably a pretty tired argument in the ears of the agile development community, but it's starting to come out again now that agile is moving into the operations sphere.
Joe McKendrick at ZDNet is the latest to register this objection in his post "Opinion: SOA and Agile may short-change application quality." In it, he notes that "One of the concerns about automation is that in many cases, it just makes a crappy business process a much faster crappy business process" and wonders if service-oriented architectures that are pushed out with agile methods are putting the business at risk by exposing it to poorly designed processes. The solution, Joe says, is to make sure that "Application quality needs to be addressed as early in the process as possible."
This is a fine sentiment, but of course it's essentially describing the paradigm that agile is largely a reaction to: the waterfall model. Plan everything out, in detail, implement slowly and completely, deploy only after you are sure you have all the kinks worked out. All of which are great ideas... in theory. In practice, in information technology, they turn out to not always work so well, for a variety of reasons that have been well-explored elsewhere.
While the worries that McKendrick has are very real, there are also a host of concepts in the agile philosophy designed to address them. Increased communication, rigorous testing and automation designed prior to coding, minimalist and modular changes that don't necessarily implicate other parts of the system even if they do contain errors... these factors and others are specifically designed to address the risks of going faster.
The unspoken assumption remains, however, that slower and more methodical is less risky. If we can acknowledge that faster releases carry a certain risk, we should also be honest enough to acknowledge that slower has never really proven to carry any less risk. Feel free to consult the IT project failure rate study of your choosing at this point. In fact, waterfall has always carried a tremendous risk, and it's a risk that agile addresses directly precisely because it is faster: the sunk cost problem.
More methodical waterfall approaches have not proven any better at detecting and eliminating process errors of the sort that Joe fears. What they have done, however, is ensured that those process errors in many cases have become deeply and expensively embedded in systems before being detected. The cost of resolving such failures is enormous. While agile methodologies may not do any better at finding those problems ahead of deployment, what they can guarantee is that you will find out about them in production much sooner, and have an opportunity to address them more quickly and at less cost.
There is a sort of circular logic to these objections that is insidious and will likely continue to bubble up for years. Agile is itself a rebuttal to the sort of cautionary approach that Joe proposes, but it's clear that the point has not penetrated past the sort of intuitive, but incorrect, surface logic that obviously going slowly and planning ahead will lead to better results than moving quickly and not planning in detail. That's such an obvious position that it's difficult to even force people to justify it. But forcing them to confront the numbers and the complexity of technology may be the only way to move past this debate.
Joe McKendrick at ZDNet is the latest to register this objection in his post "Opinion: SOA and Agile may short-change application quality." In it, he notes that "One of the concerns about automation is that in many cases, it just makes a crappy business process a much faster crappy business process" and wonders if service-oriented architectures that are pushed out with agile methods are putting the business at risk by exposing it to poorly designed processes. The solution, Joe says, is to make sure that "Application quality needs to be addressed as early in the process as possible."
This is a fine sentiment, but of course it's essentially describing the paradigm that agile is largely a reaction to: the waterfall model. Plan everything out, in detail, implement slowly and completely, deploy only after you are sure you have all the kinks worked out. All of which are great ideas... in theory. In practice, in information technology, they turn out to not always work so well, for a variety of reasons that have been well-explored elsewhere.
While the worries that McKendrick has are very real, there are also a host of concepts in the agile philosophy designed to address them. Increased communication, rigorous testing and automation designed prior to coding, minimalist and modular changes that don't necessarily implicate other parts of the system even if they do contain errors... these factors and others are specifically designed to address the risks of going faster.
The unspoken assumption remains, however, that slower and more methodical is less risky. If we can acknowledge that faster releases carry a certain risk, we should also be honest enough to acknowledge that slower has never really proven to carry any less risk. Feel free to consult the IT project failure rate study of your choosing at this point. In fact, waterfall has always carried a tremendous risk, and it's a risk that agile addresses directly precisely because it is faster: the sunk cost problem.
More methodical waterfall approaches have not proven any better at detecting and eliminating process errors of the sort that Joe fears. What they have done, however, is ensured that those process errors in many cases have become deeply and expensively embedded in systems before being detected. The cost of resolving such failures is enormous. While agile methodologies may not do any better at finding those problems ahead of deployment, what they can guarantee is that you will find out about them in production much sooner, and have an opportunity to address them more quickly and at less cost.
There is a sort of circular logic to these objections that is insidious and will likely continue to bubble up for years. Agile is itself a rebuttal to the sort of cautionary approach that Joe proposes, but it's clear that the point has not penetrated past the sort of intuitive, but incorrect, surface logic that obviously going slowly and planning ahead will lead to better results than moving quickly and not planning in detail. That's such an obvious position that it's difficult to even force people to justify it. But forcing them to confront the numbers and the complexity of technology may be the only way to move past this debate.
Wednesday, March 10. 2010
As mobile devices proliferate, agility gains importance
A recent survey by Egnyte reveals that half of respondents regularly use smartphones to conduct business, and that 13% use them with greater frequency than they do their computers. Google Europe executive John Herlihy says that company believes that within three years, most users' primary interaction with computers will be through mobile devices and that "...desktops will be irrelevant." Herlihy points out that in Japan, already most research is conducted on smartphones rather than PCs.
Of course, these developments aren't news to CIOs or anyone else responsible for meeting the needs of an increasingly mobile workforce, but it is a solid indicator that the trend will continue. It's become fashionable in certain circles to minimize the impact of cloud computing, to deny the death of the desktop, and to claim you can't do anything really significant on your smartphone, but these are all the same people that would have told you that there were plenty of lifeboats on the Titanic and anyway, it certainly wasn't going to sink very quickly. While technically they generally have a point, they are missing the forest for the trees; mobile devices are good enough for a lot of people, and those people will use whatever is most convenient and that they are most comfortable with in order to do what they need to get done. That universal truth has ever fueled the progress of industry and there's no reason to suppose it is no longer the case. Moreover, these trends are reinforcing one another; mobile devices don't have the heft or storage capacity to accomplish internally what a desktop can (and the same is true of netbooks), so users who want to get things done on them pretty much have to use cloud-based solutions. Once you are using them on your phone, it's easier to be using them on your desktop or laptop too... everything is in one place.
The challenge to the IT department in this scenario is to provision and support users with these devices and solutions. This is a far different prospect than traditional IT provisioning and support, and many IT departments are having trouble keeping up. It's not the first time IT has had trouble staying on the same plane as users when new technologies emerge, but this time, the devices and the online solutions are sufficiently cheap that those users don't need the IT department to implement them. While IT has held the keys to the kingdom for many years, there is an increasing chance that the IT department will simply become irrelevant as users bypass it for easier, cheapers solutions. As this Wall Street Journal article outlines, that day is coming.
Indeed, the Journal article should be mandatory reading for today's CIO or IT manager, as should an older post by Wired editor Chris Anderson, "Who Needs a CIO?" Both provide a good perspective on what it is your users are thinking these days, and what they see as their options. Ignore, if you will, the things that the articles miss or flat-out get wrong about the role and function of your IT department; focus on those things that the articles get right: many of your services are cumbersome, inferior, and more expensive than widely available, easily-adopted alternatives.
Ignore, too, the fact that your services have to be more expensive and cumbersome in this context. No one expects you to have the economies of scale of Google, or the burning motivation of a small start-up development team who are working on dreams and promises of an equity stake; you're going to be slower and more expensive when you try to deliver the same services, no question.
Your alternative, then, is to get out of the business of offering those services. And this is where Agile Operations comes in, because there are still services, core services, that you must deliver. Google isn't going to run your ERP system, not yet, anyway. You have to be flexible and learn to integrate public and private platforms. Continue reading "As mobile devices proliferate, agility gains..." »
Of course, these developments aren't news to CIOs or anyone else responsible for meeting the needs of an increasingly mobile workforce, but it is a solid indicator that the trend will continue. It's become fashionable in certain circles to minimize the impact of cloud computing, to deny the death of the desktop, and to claim you can't do anything really significant on your smartphone, but these are all the same people that would have told you that there were plenty of lifeboats on the Titanic and anyway, it certainly wasn't going to sink very quickly. While technically they generally have a point, they are missing the forest for the trees; mobile devices are good enough for a lot of people, and those people will use whatever is most convenient and that they are most comfortable with in order to do what they need to get done. That universal truth has ever fueled the progress of industry and there's no reason to suppose it is no longer the case. Moreover, these trends are reinforcing one another; mobile devices don't have the heft or storage capacity to accomplish internally what a desktop can (and the same is true of netbooks), so users who want to get things done on them pretty much have to use cloud-based solutions. Once you are using them on your phone, it's easier to be using them on your desktop or laptop too... everything is in one place.
The challenge to the IT department in this scenario is to provision and support users with these devices and solutions. This is a far different prospect than traditional IT provisioning and support, and many IT departments are having trouble keeping up. It's not the first time IT has had trouble staying on the same plane as users when new technologies emerge, but this time, the devices and the online solutions are sufficiently cheap that those users don't need the IT department to implement them. While IT has held the keys to the kingdom for many years, there is an increasing chance that the IT department will simply become irrelevant as users bypass it for easier, cheapers solutions. As this Wall Street Journal article outlines, that day is coming.
Indeed, the Journal article should be mandatory reading for today's CIO or IT manager, as should an older post by Wired editor Chris Anderson, "Who Needs a CIO?" Both provide a good perspective on what it is your users are thinking these days, and what they see as their options. Ignore, if you will, the things that the articles miss or flat-out get wrong about the role and function of your IT department; focus on those things that the articles get right: many of your services are cumbersome, inferior, and more expensive than widely available, easily-adopted alternatives.
Ignore, too, the fact that your services have to be more expensive and cumbersome in this context. No one expects you to have the economies of scale of Google, or the burning motivation of a small start-up development team who are working on dreams and promises of an equity stake; you're going to be slower and more expensive when you try to deliver the same services, no question.
Your alternative, then, is to get out of the business of offering those services. And this is where Agile Operations comes in, because there are still services, core services, that you must deliver. Google isn't going to run your ERP system, not yet, anyway. You have to be flexible and learn to integrate public and private platforms. Continue reading "As mobile devices proliferate, agility gains..." »
(Page 1 of 2, totaling 19 entries)
next page »
