Agile Observations from the Trenches

Agile Observations from the Trenches is sponsored by Davisbase Consulting, which offers a wide range of consulting services, Agile training courses, and customized services for organizations looking to adopt an Agile approach to their IT development efforts. This blog is written by several trainers that primarily work with teams looking to implement or improve their Agile approach to software development. To find out more about who is behind the blog, please view our 'About' section.

Welcome, and thank you for taking the time to read and participate in the discussion.

I had big dreams when I started college.  I had always wanted to go to law school, for nearly as long as I can remember, throughout grade school and then into high school, my plans were always that following college, I would enter law school.  As I approached college, I had to get more serious about my ultimate plans to make it into law school, so I had a long discussion with my father, during which we both realized that there were far too many lawyers graduating from law school and finding themselves unemployed with a law degree.  We concluded that in order for me to make it in the field of law, I would need to specialize.  In preparation for this specialization approach, I decided that I would like to try international law, and toward that decision, I enrolled in a Japanese language course.  It only took a single class for me to declare my major as East Asian Studies with a concentration in Japanese (this was the official UCLA equivalent of a Japanese major.)  Four years later I graduated with this major, but never made it to law school, never even applied.

Here I was, I had a fresh degree from a reputable university with a major in Japanese, I had big plans to get  a law degree to practice international law, but instead, I found myself in the computer software field.  So how did my plans get so derailed? 

And what does this have to do with determining whether your Agile team should use a high-tech or low-tech tool?

Plenty.

You see, nearing the end of my junior year in college, my roommate kept himself busy writing something that was very new, pretty cutting edge, he was writing websites, USING MICROSOFT NOTEPAD.  He would show me a screen full of just text, then would click a button, and VOILA, there would be a beautiful website.  I could not understand how we was turning a document in Notepad into a picture laden website.  Now although I was impressed, I was even more impressed when he told me that he was making several thousand dollars a month creating and selling these websites to car dealerships in the Los Angeles area.  For the meager budgets of college students, this was more money that I had imagined making even after I graduated.

I was hooked.  I asked (begged is probably more like it) to let me in on this business.  He told me that we would be more than happy to have me as a partner, but I would need to learn the ropes, and that meant learning how to create my own web pages.  That was it, I was now on a mission.  I went home that summer and spent most of it reading books, teaching myself all there was to know about hand-coding websites, and then creating as many as I could, each better than the one created before it.  I was honestly pretty impressed with myself, and was looking forward to joining my roommate in our business adventure together.

The summer ended and I went back to school, excited to reunite with my former roommate so that we could take the LA area by storm with our websites.  I spent an entire afternoon showcasing the websites that I had created, painstakingly by hand.  He was impressed.  He spent more time studying the actual code than the marveling at my pretty websites.

After a good deal of time, he seemed satisfied.  He gave me praise for all of the websites I created and specifically for the fact that I had done them all by hand.   He could tell that I was anxious to begin discussing how we were going to proceed with our business, but before I could ask what our next steps were, he spoke first.

He said to me “these are great, you did really well coding these sites by hand.  Now that we are ready to develop some real websites, there is this tool we can use, it’s called Dreamweaver…”

Huh?  A tool?  Then he showed me the tool, showed me just how easy it was to create websites using this tool.  I found myself getting slightly angry, after all I had spent a great deal of my summer learning how to code these sites BY HAND!  I took a few deep breathes, and then calmly asked my friend “if there was this tool that makes creating these websites so easy, then why didn’t you tell me about this BEFORE I spent all summer learning how to do this by hand?  I could have created all of these sites in a weekend, rather than it taking me all summer.”

He smiled, knowing that my reaction would likely be just as it was, and replied wisely, “you had to.  You needed to know how to create these sites by hand, how the tool would be building the site behind the scenes, so that you could get into the code and fine-tune the website so that it is just as you want it to be.  If you ONLY knew the tool, but not the basics behind it, the tool would always limit you rather than support you.”

I am not sure just how my college roommate could be so wise at such a young age, but it is a lesson I will never forget.  And he was right too.  In fact, over the following few years I thought back many times to that first summer, and was thankful I had really, truly learned the trade rather than solely relied on a tool.  This valuable, unforgettable lesson applies perfectly to a situation I see teams new to agile get themselves into.  Their situation usually starts with a question like…

Our Team Is New to Agile:  Which Software Tool Should We Use to Manage Our Project?

None.  Instead, try Sharpies, notecards, a whiteboard.  You must learn the inner workings of agile first.  You must get good at agile.  You must develop the self-discipline that is the hallmark of good and great agile teams.  If you try to find a tool to use before you develop the basic skills necessary to implement an agile approach, then the tool will always limit you rather than support you.

The tools that are currently on the market are great, some even really great.  They provide a great deal of value and they have the features and functionality to truly serve the best needs of your agile team.  Some will save you time, others will save you money, some will do both, others will do none.  But for the team fairly new to agile, none of them will likely support your team’s best efforts in getting more proficient in agile.  This post is not about pushing one software vendor over another, but about the need for your team to build their agile skills before trying to use a tool that will ultimate limit their ability to grow their own unique brand of agile approach.

When you are ready, and you really are looking for some suggestions on the best agile software packages currently on the market, come back to my blog, because I am sure that by that time I will have finally posted my list of recommendations.  Until then, spend your summer getting good at agile.  I’ll be here when you get back.

In 1931 Alfred Korzybski gave a presentation at a meeting of the American Association for the Advancement of Science in New Orleans where he used the phrase “the map is not the territory.”  Korzybski used this phrase to mean that people in general do not have access to absolute knowledge of reality, but merely possess a subset of that knowledge that is then tinted through the lenses of their own experience.  He further added that it is important for people to know that their understanding of things, “the map”, is not a true representation of reality itself or everything represented by reality, or “the territory”.

A Belgian surrealist artist Rene Magritte used Korzybski’s notion in his work, and described the idea that was the foundation of his art as an understanding that “perception always intercedes between reality and ourselves.”  One of Magritte’s most famous works was an image of a pipe with a caption below it that read Ceci n’est pas une pipe (“This is not a pipe.”)  This piece was meant to illustrate that the painting of a pipe was not a pipe itself, but a representation of what his audience would associate with a real pipe.

So what does the phrase “the map is not the territory” mean for software development?

Plenty.  

Software development teams have a very difficult job. They must elicit requirements for a product that doesn’t exist.  In that effort, the customer then tries to describe this non-existent product, while the software development team imagines what the customer is describing.  In a waterfall approach, the first time that anyone finds out if what the development team built is what the customer wanted, needed, or asked for is during user acceptance testing.  This is very understandably why most waterfall projects do not end perfectly, or in most cases are far from perfect in their delivery.

Why does this so often happen?  Korzybski would describe this as being a suitable example of his concept that the map is not the territory.  Software developers spend a great deal of time in the requirements gathering phase attempting to construct a perfect map of the product territory.  But this map, this understanding of what the customer wants, generally results in the delivery of a product that misses the customer’s needs for the product.  It is not that the developers were not ernest enough in their requirements gathering efforts, it is not necessarily that the customer did not do their best to document their requirements, it is simply the nature of misses that occur when we try to translate what they customer is saying to what we are thinking, and then to what we ultimately end up building.  It would be like trying to construct a perfect map of the American coast based on the description provided by an explorer familiar with the area.  It just would not be a representation of any accuracy or quality.

So how do we proceed with software development knowing that this problem will continue to be a part of most software development projects?  Agile fundamentals recognize that in software development, the map is definitely not the territory.  So in light of this we only build small bits before going back to the customer to verify if our territory matches their mental map.  Showing working software is truly the only real way to ensure that what we are building is what they asked for, and more importantly, what they need and want.

The first step to avoiding the pitfalls of mismatches in understanding the requirements for a product is to open our eyes and check with our customer each step of the way.  We need to be aware that even with simple requirements, a simple product, or an impatient customer, there exists the likelihood that what we hear as developers is not going to perfectly match what the customer thinks they are saying.  

I use an exercise to demonstrate this phenomenon and recently had the opportunity to use it with a group of over 100 business analysts.  Each person participating in the exercise is given a single sheet of white paper, they are asked to hold it up in front of them, and then they are asked to close their eyes for the duration of the exercise.  Following everyone closing their eyes, I provide a series of very simple instructions (fold it in half, rip off a corner, etc.) while I am also following my own instructions.  They are playing the part of the software development team, while I play the part of the customer.  After a series of 4-5 instructions, or requirements, everyone opens their eyes, unfolds their paper, and then compares what they “built” to my requirements.  Out of 100 people participating in this exercise, would you like to guess how many folded their paper so that it matched my requirements?  If you would like to find out, I have the video of this exercise posted here (it is within the first few minutes of the presentation.)  Even with simple instructions, it can still be difficult to match the map to the territory.

A successful Agile team recognizes that their map will rarely match the customer’s territory without regularly opening their eyes and checking that their folds match the customer’s folds.

After training dozens of teams, I have found another disturbing trend, the trend of cargo cult agile teams.  The term cargo cult comes from the author Richard Feyman and was described in detail in his book Surely You’re Joking, Mr. Feyman!  The terms roots were even earlier than its use in his book, originally being used in his 1974 commencement address where he warned of learning to not fall into the trap of fooling one’s self.  And unfortunately, this is just what I am seeing more and more lately, agile teams fooling themselves into believing that they are truly utilizing agile.

So what is a cargo cult?

The term cargo cult describes the phenomenon of natives from the islands of Melanesia in the South Pacific. During the war, the islands of Melanesia served as a staging area for the military where they built up temporary operations. The natives of the island observed everything that the allied forces were doing and, more importantly, also observed that with the allied force’s actions came cargo. The natives had little or no knowledge of the civilized world from which this cargo originated, but instead incorrectly correlated the actions of the foreigners with the pre-requisites for obtaining the cargo they so desired.  Ultimately, after the war ended and the allied troops left the island, the cargo also disappeared.  There were no more shipments of the riches that the cargo represented, so the natives did what they assumed would bring this cargo back.  The natives, these cargo cults, decided that to entice the return of the cargo they must duplicate the actions and efforts of the foreigners that were so successful in obtaining such items.  So in light of their mistaken beliefs they built dirt runways, bamboo control towers, offices and planes, sewed crude uniforms, and even crafted bamboo headsets in their effort to entice the return of the cargo.

These natives learned the foreigners ‘rituals’ very well, performing them over and again in hopes that planes would return full of cargo.  Over time they learned that even though they may be able to duplicate the ritual, it does not guarantee the same result.

How does the term apply to an agile team?

Cargo cult agile teams do much of the same thing as the natives in the story above.  These ‘agile’ teams use the correct terms, they may hold stand-up meetings, they may use story points, they may even segment their work into iterations, but fundamentally their culture does not truly change to match that of a real agile team and organization.  These teams have simply replaced their old, static project process with a new static project process, but instead have labeled their new process agile.  These teams represent the possibility of an ‘agile backlash’ that I feel is in the making (something I am currently working on for this blog.)  These teams, not completely devoted to open and honest inspection of their own processes, will likely not find agile to be the panacea of processes and will instead associate the problems they find as being caused by agile.  Any agile veteran knows that these problems are more likely to simply be problems that these teams have always had, just never knew.

How do we change from cargo cult agile, to true agile?

The hallmark of a good agile team is the ability to respond to change.  Change that can come in the form of customer initiated requests for the product as well as change in the form of guidance that comes from effective retrospectives.  Good agile teams are willing to risk, they are willing to act outside of the guarantee of a positive outcome.  They decide to risk because the possibility of great things come from endeavoring outside of the strictly known software development universe.  Organizations that are steeped in process and that have a heavy handed corporate culture face a challenging endeavor in moving to an agile approach.  A culture of following a process because we have always followed this process is likely to find that agile does not play well in that sandbox.  These types of corporate cultures, those companies that are most susceptible to becoming a cargo cult in their agile approach, simply do not put in the effort to change their deeply rooted culture, where it is required to change, in order to take advantage of what agile has to offer.

So how do we avoid becoming a company using a cargo cult agile approach?

Take an honest look at the company’s current culture, and recognize where the culture and the agile principles may be at odds.  Honestly evaluate the effort that is required in order to effectively change the culture where necessary.  Be painfully aware of the danger that exists in becoming comfortable with simply going through the motions.  Implement a mechanism in which agile teams that employ self-discipline are recognized and rewarded; self-discipline that is required in order to maintain sight of the goal for each project.  Embrace your team’s ability to risk and address any portions of the culture that are at odds with a team’s ability to risk for the greater good.

Make sure that every portion of the organization that’s involved in using an agile approach has the appropriate training.  When agile is taken out of context, when only bits and pieces are explained or used, it may be difficult to get the appropriate amount of mindshare required to truly affect a paradigm shift of the company culture.  Too often I have seen the tides turn against agile simply because the people being asked to use it do not have a complete understanding of the pieces that make up the whole.  When you are able to elevate the team’s understanding of the benefits and the ‘whys’ behind the ‘whats’ of agile, it will becoming self-reinforcing, which is the best possible approach to changing a company’s culture.

Is your company using a cargo cult approach to agile?  Will you be the one to ask the questions that start the discussion?  Dare to risk!

When I first started providing Agile training for software development teams looking to abandon their waterfall approach, I found myself consistently working with mavericks, teams of developers looking to push the envelope on what was possible.  These teams were on the bleeding edge of innovation, and as such, they often were habitual practitioners of risk taking.  Not the risk that those of us with a project management background may have traditionally defined it as, but a simple willingness to explore the unknown in search for a better outcome, where such an outcome is not guaranteed.

Those teams from just a few years ago are no longer the risk takers they once were.  Those teams have been replaced by teams comprised of nervous individuals, afraid to do much of anything that does not come neatly packaged with a guaranteed outcome.  The willingness to risk has been wholly replaced by an inflexible adherence to metrics that measure the team’s ability to meet expectations and estimates, but say nothing about product quality and the importance of a happy customer.

Software development teams are losing sight of what is truly important.  These teams are worried more about satisfying internally defined processes as opposed to building great software and endeavoring to satisfy their customer.  And sadly I am seeing this trend on the rise rather than the decline.  In this battered economy, I am seeing teams of talented people under-promising in their efforts and estimates, so that they can minimize the risk of falling short on a deadline or a deliverable.  I am not saying that missing a deadline or deliverable is a good thing only that  I am seeing these teams behave from a place of fear because their employers are rewarding the wrong results and inadvertently punishing, or at the very least discouraging, the right behaviors.

How have we so quickly become adept at management from this place of fear?  Is it the economy that has created this fear based professional economy of cowards?  Partially, possibly, but I started seeing this trend before the economy passed the event horizon.  So what type of organization, company, culture, or management approach is cultivating this crop of individuals?  (I use the word ‘individuals’ loosely, based on the herd mentality I have also noticed.)  What is being put in the drinking water at these IT companies?  Probably most importantly, what can we do to reverse course on what is likely to become the silent killer of innovation and workplace happiness.

The answer is simple.

Failed efforts to improve must be celebrated.
Failed attempts at a new approach must be cheered.
Failed attempts at doing anything out the circle of comfort must be rewarded.
Failure itself must be de-stigmatized.
Failure as a term must be taken off our list of bad corporate words.
Failure must be redefined as the hallmark of a team on a path towards greatness.
To be great, we must fail.

Failure is a product of good Agile teams.
Failure is an absolute necessity for great Agile teams.
Failure redraws the lines that bound the area of comfort which typically define an average team’s actions, as they rarely or never act outside of this zone.  They rarely act outside of this zone because anything over this line represents effort without a guarantee of positive outcome.

For great Agile teams, this line, this ‘boundary’ defines the point at which they have the possibility of improving, growing, producing results that exceed their own expectations or understanding.  This line does not bound their actions, it simply provides measure against which they can judge their ability and extent to act outside this zone.  Great teams are defined by their ability to constantly re-evaluate and re-draw this line.  The ‘boundary’ for great teams fails to bind at all and becomes a reflection of the great team’s greatness.

When I began training teams on the tenets of Agile, I trained great teams.  Today I see average teams.  And the discouraging trend is towards average being the preference.

Now I am not all about simply pointing out doom and gloom trends as I see them, I would also like to offer a practical approach to how to stem this tide.  I stated above that we need to embrace failure.  Sure, I said that in a very specific way to elicit some response, but in truth if we, as a team, only pursue those actions that have a guarantee of successful outcome, then we will only ever produce the known.  Agile teams, with short design and production cycles can actually fail, and fail quickly, while still being successful.  Short term failures are then replaced with the possibility of long term innovation.

If you always do what you have always done, then expect to get what you always got. Agile teams embrace change and often strike out into the unknown for nothing more than the possibility that they may be able to create something greater than for which they could have planned.  Agile teams are those mavericks from the past.  They are the explorers.  We must encourage breaking the chains that I see are binding teams today, teams that see their boundary as insurmountable.  We are creating teams, environments, and cultures of mediocrity.  Nothing great was ever created by individuals that made their decisions from fear.

We are either growing or dying.  We are either deciding to do great things or deciding to shrink our influence.  The decision is ours to make.  And I know what I have decided.

Well, looks like I am finally joining the fray of agile community individuals that are posting their experiences, observations, and beliefs about all things agile. After telling most professional acquaintances at one point or another that I would start to blog about my agile life, I was eventually told to put up or shut up…so here it is.

I can’t say that it will be like most of the agile fare currently available on the internet. My plan is to share my unique perspectives on how agile is best implemented. Information that has been gathered during my engagements as a scrum master and the many agile classes that I have facilitated. And I can say with all honesty, I have learned more from training others on agile than in any other professional endeavor. Some of these experiences are true gems that I have wanted to share for awhile, so now I finally have the forum and opportunity. Now I get to see if what I have floating around in my head is actually worth anyone’s time to read.

So, this, my very first post, is nothing more than to announce my intentions to add to the knowledgebase of agile, a compendium of information that is continually growing, and more importantly, evolving.  That is, if agile and the agile community truly wishes to practice what we preach, we are going to have to eat our own dog food and recognize those areas of opportunity.  What better place to share these experiences than on the internet, the great equalizer.

I look forward to getting to know those of you who find some value in what I have to share.   And finally, if you are reading these words, then let me offer a sincere word of gratitude that you have taken a moment of your time to read these words.

Until next time,

Bill Gaiennie