Agile Observations from the Trenches

Every time the topic of an Agile Process Maturity Model (or APMM for short) comes up, I simply sit back and watch the backlash grow to a furious pace, then see the APMM proponents slink off.  But here we are again, seeing the growing trend of APMM discussions, but this time, it looks like there are some pretty big players backing the movement, including a division of IBM.

As I was poking around the web and polling my Twitter folks, I found a couple of interesting links.  The first was a short post on InfoQ by Scott Ambler (dated June 15, 2006), titled Has Hell Frozen Over? An Agile Maturity Model? And then, just a single Google search later, I found the IBM website that showcases the work that has been put together in favor of this APMM.  The crazy thing?  It too was authored by Scott Ambler.  Now, don’t go misinterpreting what I am saying here, I do not think Scott is crazy, afterall 3 years have passed between the two posts, but I do find it a little peculiar when big business (like IBM) partners with an industry expert in an attempt to capitalize on any growing trend.

For those of you that have spent any amount of quality time in the Agile trenches, then you will likely know what I know, which is the idea that an overly burdensome process can easily sap an Agile project of any power and efficiency that may have been there otherwise.  Processes are like governmental positions and agencies in that once they are created, they very rarely go away.  What we have succeeded in doing with Agile is having the conversation about this truth about process and then encouraging the courage required to eliminate that which does not add value to our efforts.  Lean and mean is the way to go, but this is not an approach that is likely to play friendly with any type of review approach that will judge and evaluate success based on adherence to a pre-defined process.

Do I think that APMM could single-handedly ruin the Agile movement?  No, but as far as I understand it in its current form, I do not think that it is going to help teams become any more proficient in their approach or deliver a better product.  I believe that the man behind the curtain with this whole movement is pushing his levers in a poorly veiled attempt to appeal to large organizations that find value in the certification designation that they get to display proudly when they pass “the test.”  I have worked with more than one organization that was currently still stuck in this trap as it concerned their CMM designation.  These companies spent more time and effort worrying about keeping their current level, or moving to the next, to accurately evaluate if what they were doing and what they were delivering translated into true value.

Uhhhg, this may simple be a battle not worth fighting.  Let those companies that want their certificates showing they are at X Level of APMM go for it.  Let them spend the money.  Let them do all of this and then at the end of the day, let them continue doing the same things they were doing before the set out to rank their approach in terms of the APMM.  I do suppose that there is going to be a lot of money out there to be made for consultants who specialize in taking companies to higher levels of an APMM designation, so I might as well surrender to the movement, and then plan buying a bigger mattress under which I will stuff all of this new consulting money I plan on making.

In 1931 Alfred Korzybski gave a presentation at a meeting of the American Association for the Advancement of Science in New Orleans where he used the phrase “the map is not the territory.”  Korzybski used this phrase to mean that people in general do not have access to absolute knowledge of reality, but merely possess a subset of that knowledge that is then tinted through the lenses of their own experience.  He further added that it is important for people to know that their understanding of things, “the map”, is not a true representation of reality itself or everything represented by reality, or “the territory”.

A Belgian surrealist artist Rene Magritte used Korzybski’s notion in his work, and described the idea that was the foundation of his art as an understanding that “perception always intercedes between reality and ourselves.”  One of Magritte’s most famous works was an image of a pipe with a caption below it that read Ceci n’est pas une pipe (“This is not a pipe.”)  This piece was meant to illustrate that the painting of a pipe was not a pipe itself, but a representation of what his audience would associate with a real pipe.

So what does the phrase “the map is not the territory” mean for software development?

Plenty.  

Software development teams have a very difficult job. They must elicit requirements for a product that doesn’t exist.  In that effort, the customer then tries to describe this non-existent product, while the software development team imagines what the customer is describing.  In a waterfall approach, the first time that anyone finds out if what the development team built is what the customer wanted, needed, or asked for is during user acceptance testing.  This is very understandably why most waterfall projects do not end perfectly, or in most cases are far from perfect in their delivery.

Why does this so often happen?  Korzybski would describe this as being a suitable example of his concept that the map is not the territory.  Software developers spend a great deal of time in the requirements gathering phase attempting to construct a perfect map of the product territory.  But this map, this understanding of what the customer wants, generally results in the delivery of a product that misses the customer’s needs for the product.  It is not that the developers were not ernest enough in their requirements gathering efforts, it is not necessarily that the customer did not do their best to document their requirements, it is simply the nature of misses that occur when we try to translate what they customer is saying to what we are thinking, and then to what we ultimately end up building.  It would be like trying to construct a perfect map of the American coast based on the description provided by an explorer familiar with the area.  It just would not be a representation of any accuracy or quality.

So how do we proceed with software development knowing that this problem will continue to be a part of most software development projects?  Agile fundamentals recognize that in software development, the map is definitely not the territory.  So in light of this we only build small bits before going back to the customer to verify if our territory matches their mental map.  Showing working software is truly the only real way to ensure that what we are building is what they asked for, and more importantly, what they need and want.

The first step to avoiding the pitfalls of mismatches in understanding the requirements for a product is to open our eyes and check with our customer each step of the way.  We need to be aware that even with simple requirements, a simple product, or an impatient customer, there exists the likelihood that what we hear as developers is not going to perfectly match what the customer thinks they are saying.  

I use an exercise to demonstrate this phenomenon and recently had the opportunity to use it with a group of over 100 business analysts.  Each person participating in the exercise is given a single sheet of white paper, they are asked to hold it up in front of them, and then they are asked to close their eyes for the duration of the exercise.  Following everyone closing their eyes, I provide a series of very simple instructions (fold it in half, rip off a corner, etc.) while I am also following my own instructions.  They are playing the part of the software development team, while I play the part of the customer.  After a series of 4-5 instructions, or requirements, everyone opens their eyes, unfolds their paper, and then compares what they “built” to my requirements.  Out of 100 people participating in this exercise, would you like to guess how many folded their paper so that it matched my requirements?  If you would like to find out, I have the video of this exercise posted here (it is within the first few minutes of the presentation.)  Even with simple instructions, it can still be difficult to match the map to the territory.

A successful Agile team recognizes that their map will rarely match the customer’s territory without regularly opening their eyes and checking that their folds match the customer’s folds.

After training dozens of teams, I have found another disturbing trend, the trend of cargo cult agile teams.  The term cargo cult comes from the author Richard Feyman and was described in detail in his book Surely You’re Joking, Mr. Feyman!  The terms roots were even earlier than its use in his book, originally being used in his 1974 commencement address where he warned of learning to not fall into the trap of fooling one’s self.  And unfortunately, this is just what I am seeing more and more lately, agile teams fooling themselves into believing that they are truly utilizing agile.

So what is a cargo cult?

The term cargo cult describes the phenomenon of natives from the islands of Melanesia in the South Pacific. During the war, the islands of Melanesia served as a staging area for the military where they built up temporary operations. The natives of the island observed everything that the allied forces were doing and, more importantly, also observed that with the allied force’s actions came cargo. The natives had little or no knowledge of the civilized world from which this cargo originated, but instead incorrectly correlated the actions of the foreigners with the pre-requisites for obtaining the cargo they so desired.  Ultimately, after the war ended and the allied troops left the island, the cargo also disappeared.  There were no more shipments of the riches that the cargo represented, so the natives did what they assumed would bring this cargo back.  The natives, these cargo cults, decided that to entice the return of the cargo they must duplicate the actions and efforts of the foreigners that were so successful in obtaining such items.  So in light of their mistaken beliefs they built dirt runways, bamboo control towers, offices and planes, sewed crude uniforms, and even crafted bamboo headsets in their effort to entice the return of the cargo.

These natives learned the foreigners ‘rituals’ very well, performing them over and again in hopes that planes would return full of cargo.  Over time they learned that even though they may be able to duplicate the ritual, it does not guarantee the same result.

How does the term apply to an agile team?

Cargo cult agile teams do much of the same thing as the natives in the story above.  These ‘agile’ teams use the correct terms, they may hold stand-up meetings, they may use story points, they may even segment their work into iterations, but fundamentally their culture does not truly change to match that of a real agile team and organization.  These teams have simply replaced their old, static project process with a new static project process, but instead have labeled their new process agile.  These teams represent the possibility of an ‘agile backlash’ that I feel is in the making (something I am currently working on for this blog.)  These teams, not completely devoted to open and honest inspection of their own processes, will likely not find agile to be the panacea of processes and will instead associate the problems they find as being caused by agile.  Any agile veteran knows that these problems are more likely to simply be problems that these teams have always had, just never knew.

How do we change from cargo cult agile, to true agile?

The hallmark of a good agile team is the ability to respond to change.  Change that can come in the form of customer initiated requests for the product as well as change in the form of guidance that comes from effective retrospectives.  Good agile teams are willing to risk, they are willing to act outside of the guarantee of a positive outcome.  They decide to risk because the possibility of great things come from endeavoring outside of the strictly known software development universe.  Organizations that are steeped in process and that have a heavy handed corporate culture face a challenging endeavor in moving to an agile approach.  A culture of following a process because we have always followed this process is likely to find that agile does not play well in that sandbox.  These types of corporate cultures, those companies that are most susceptible to becoming a cargo cult in their agile approach, simply do not put in the effort to change their deeply rooted culture, where it is required to change, in order to take advantage of what agile has to offer.

So how do we avoid becoming a company using a cargo cult agile approach?

Take an honest look at the company’s current culture, and recognize where the culture and the agile principles may be at odds.  Honestly evaluate the effort that is required in order to effectively change the culture where necessary.  Be painfully aware of the danger that exists in becoming comfortable with simply going through the motions.  Implement a mechanism in which agile teams that employ self-discipline are recognized and rewarded; self-discipline that is required in order to maintain sight of the goal for each project.  Embrace your team’s ability to risk and address any portions of the culture that are at odds with a team’s ability to risk for the greater good.

Make sure that every portion of the organization that’s involved in using an agile approach has the appropriate training.  When agile is taken out of context, when only bits and pieces are explained or used, it may be difficult to get the appropriate amount of mindshare required to truly affect a paradigm shift of the company culture.  Too often I have seen the tides turn against agile simply because the people being asked to use it do not have a complete understanding of the pieces that make up the whole.  When you are able to elevate the team’s understanding of the benefits and the ‘whys’ behind the ‘whats’ of agile, it will becoming self-reinforcing, which is the best possible approach to changing a company’s culture.

Is your company using a cargo cult approach to agile?  Will you be the one to ask the questions that start the discussion?  Dare to risk!

When I first started providing Agile training for software development teams looking to abandon their waterfall approach, I found myself consistently working with mavericks, teams of developers looking to push the envelope on what was possible.  These teams were on the bleeding edge of innovation, and as such, they often were habitual practitioners of risk taking.  Not the risk that those of us with a project management background may have traditionally defined it as, but a simple willingness to explore the unknown in search for a better outcome, where such an outcome is not guaranteed.

Those teams from just a few years ago are no longer the risk takers they once were.  Those teams have been replaced by teams comprised of nervous individuals, afraid to do much of anything that does not come neatly packaged with a guaranteed outcome.  The willingness to risk has been wholly replaced by an inflexible adherence to metrics that measure the team’s ability to meet expectations and estimates, but say nothing about product quality and the importance of a happy customer.

Software development teams are losing sight of what is truly important.  These teams are worried more about satisfying internally defined processes as opposed to building great software and endeavoring to satisfy their customer.  And sadly I am seeing this trend on the rise rather than the decline.  In this battered economy, I am seeing teams of talented people under-promising in their efforts and estimates, so that they can minimize the risk of falling short on a deadline or a deliverable.  I am not saying that missing a deadline or deliverable is a good thing only that  I am seeing these teams behave from a place of fear because their employers are rewarding the wrong results and inadvertently punishing, or at the very least discouraging, the right behaviors.

How have we so quickly become adept at management from this place of fear?  Is it the economy that has created this fear based professional economy of cowards?  Partially, possibly, but I started seeing this trend before the economy passed the event horizon.  So what type of organization, company, culture, or management approach is cultivating this crop of individuals?  (I use the word ‘individuals’ loosely, based on the herd mentality I have also noticed.)  What is being put in the drinking water at these IT companies?  Probably most importantly, what can we do to reverse course on what is likely to become the silent killer of innovation and workplace happiness.

The answer is simple.

Failed efforts to improve must be celebrated.
Failed attempts at a new approach must be cheered.
Failed attempts at doing anything out the circle of comfort must be rewarded.
Failure itself must be de-stigmatized.
Failure as a term must be taken off our list of bad corporate words.
Failure must be redefined as the hallmark of a team on a path towards greatness.
To be great, we must fail.

Failure is a product of good Agile teams.
Failure is an absolute necessity for great Agile teams.
Failure redraws the lines that bound the area of comfort which typically define an average team’s actions, as they rarely or never act outside of this zone.  They rarely act outside of this zone because anything over this line represents effort without a guarantee of positive outcome.

For great Agile teams, this line, this ‘boundary’ defines the point at which they have the possibility of improving, growing, producing results that exceed their own expectations or understanding.  This line does not bound their actions, it simply provides measure against which they can judge their ability and extent to act outside this zone.  Great teams are defined by their ability to constantly re-evaluate and re-draw this line.  The ‘boundary’ for great teams fails to bind at all and becomes a reflection of the great team’s greatness.

When I began training teams on the tenets of Agile, I trained great teams.  Today I see average teams.  And the discouraging trend is towards average being the preference.

Now I am not all about simply pointing out doom and gloom trends as I see them, I would also like to offer a practical approach to how to stem this tide.  I stated above that we need to embrace failure.  Sure, I said that in a very specific way to elicit some response, but in truth if we, as a team, only pursue those actions that have a guarantee of successful outcome, then we will only ever produce the known.  Agile teams, with short design and production cycles can actually fail, and fail quickly, while still being successful.  Short term failures are then replaced with the possibility of long term innovation.

If you always do what you have always done, then expect to get what you always got. Agile teams embrace change and often strike out into the unknown for nothing more than the possibility that they may be able to create something greater than for which they could have planned.  Agile teams are those mavericks from the past.  They are the explorers.  We must encourage breaking the chains that I see are binding teams today, teams that see their boundary as insurmountable.  We are creating teams, environments, and cultures of mediocrity.  Nothing great was ever created by individuals that made their decisions from fear.

We are either growing or dying.  We are either deciding to do great things or deciding to shrink our influence.  The decision is ours to make.  And I know what I have decided.

Well, looks like I am finally joining the fray of agile community individuals that are posting their experiences, observations, and beliefs about all things agile. After telling most professional acquaintances at one point or another that I would start to blog about my agile life, I was eventually told to put up or shut up…so here it is.

I can’t say that it will be like most of the agile fare currently available on the internet. My plan is to share my unique perspectives on how agile is best implemented. Information that has been gathered during my engagements as a scrum master and the many agile classes that I have facilitated. And I can say with all honesty, I have learned more from training others on agile than in any other professional endeavor. Some of these experiences are true gems that I have wanted to share for awhile, so now I finally have the forum and opportunity. Now I get to see if what I have floating around in my head is actually worth anyone’s time to read.

So, this, my very first post, is nothing more than to announce my intentions to add to the knowledgebase of agile, a compendium of information that is continually growing, and more importantly, evolving.  That is, if agile and the agile community truly wishes to practice what we preach, we are going to have to eat our own dog food and recognize those areas of opportunity.  What better place to share these experiences than on the internet, the great equalizer.

I look forward to getting to know those of you who find some value in what I have to share.   And finally, if you are reading these words, then let me offer a sincere word of gratitude that you have taken a moment of your time to read these words.

Until next time,

Bill Gaiennie