XP Implies that failure is inevitable, and we should be expected to (refactor rewrite mercilessly); however they try to minimize the impact of the failure, by only failing one day at a time.
However, since their next attempt does not use any more analyses than the previous (failed) attempt, they will continue to fail every day in a row (fail early, fail often), as can be seen here in this example of an XP Luminary trying (unsuccessfully over 5 attempts) to solve a sudoku puzzle.
Using a non XP process another author was able to solve the problem easily.
XP Claims that they “Embrace Change” and other processes have a “Fear of Change”.
It is the opinion of this author that XP experiences a “Fear of Notation (Documentation)”, “Fear of Process“, a “Fear of Excellent Programmers“, a “Fear of Design/Architecture“, and a ”Fear of Large Teams“. If you then view XP as driven by the “Five Fears”, then all of their other tenets and practices (no or minimal documentation, small teams, pair programming) become obvious as they drive out (and in fact excommunicate) the feared practices entirely in the XP dogma.
Like George H.W. Bush and his famous statement about Broccoli:
“I do not like broccoli. And I haven’t liked it since I was a little kid and my mother made me eat it. And I’m President of the United States and I’m not going to eat any more broccoli. “
How is that different from:
“I do not like Documentation. And I haven’t liked it since I was a little kid and my teacher made me write it. And I’m Scrum Lord of a .Com and I’m not going to write any more Documentation. “
Like Peter Pan in Neverland who never wants to grow up, fear of broccoli, written specs, documentation, and architecture has driven most of the available nutrition out of the XP/Scrum lifestyle.
The more one actually critically examines what is going on with agile, versus bleatingly accepting without question the Bold Assertions and Broad Generalizations that the Agilistas throw out, your cognitive processes will improve as you become a discerning individual and not merely an unquestioning sheep in a trendy (but simple) flock who’s 15 minutes of fame have lasted far too long.
Think for yourself; don’t blindly accept the party line most of which is conjecture and anecodate with little hard evidence. You’re in charge of your project, not the Agile newsgroups.
Oh and hey — don’t be afraid to try a little Broccoli! It’s good for you. Life isn’t all cotton candy you know. And just think how great it would be to write things down for a change, hire some experts, and stop playing musical chairs.
July 3, 2007 at 2:17 am
[...] Embracing Failure: Agile and Fear Driven Process « The Practice and Art of Software Composition (tags: scrum agile fear) [...]
July 5, 2007 at 10:46 pm
Morning Maestro,
*Quick disclaimer: I do not refer to user documentation here*
Is not the greatest technical documentation we, as creators of software, manufacture readable succinct source code?
In a previous role, I was thrown into a behemoth of a PHP script having no prior experience in the language. Long story short, a great developer preceded me and the code was self-documenting (i.e a novice could read and understand it whilst it was not littered with comment.)
I have no aversion to broccoli as a side but as let’s make it part of a balanced diet. “Working software over COMPLETE documentation.” I am sure the “Agilstas” you refer to would concur a project without any documentation is probably going to fail.
Kindness (and always a healthy dose of enjoyment at your posts)
Dan
July 5, 2007 at 10:55 pm
Hi Dan
When I talk about “Documentation” I’m primarily talking about things like “Specifications”, “Marketing Requirements” and “Engineering Requirements”.
Not “Documentation about the code”, rather, Documentation that should be created before the coding process begins.
XP/etc doesn’t believe in those kinds of Documents, they believe the onsite customer should be there to tell you that, and/or variations on that theme. That is what I’m arguing against when I use the word “Documentation”.
Just about every project needs some written specs, MRD, ERD, etc.
The larger the projects get the more they might need code level documentation, but that’s not what I’m talking about for the most part….
The railing against Specs/Docs/etc that the XP team does is because they originally were working on a Payroll system, C3. The payroll system was boring. They didn’t want to read the specs. Even though a Payroll system doesn’t “change”, they wanted an onsite customer to read the spec to them. Google for “C3 was a failure” and you’ll learn more about it. But ultimately their unhappiness with reading boring payroll docs led them to excommunicate docs from XP, and then they opened up the flood gates of Never never NEVER land
Software Maestro
July 12, 2007 at 7:02 am
You know there are extremists in every movement. Agile is a reaction to beauracratic, over-documented, segregated, inflexible, and over architected project management methodology that was waterfall. So naturally there will be some extremists that think there should be no design, no architecture, no documentation etc. If you think of Waterfall on the left, and these extremists on the right, the original intention of Agile was to swing towards the middle.
I think people should go read the original Agile Manifesto:
http://agilemanifesto.org/
——————————
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on
the right, we value the items on the left more.
——————————
So Agile admits up front that there are values on the right, but that there is more value in whats on the left. The items on the right are a possible means to an end, not the end itself.
July 12, 2007 at 8:23 am
If you look at zero documentation as one extreme, clearly you can see too much documentation, too early, as another extreme.
Only the lunatic fringe in the lean/agile/scrum/xp world have an outright ban on any documentation.
Of course you should do some analysis/design/documentation up front. It’s just that you shouldn’t think you can do all of it then. Just as importantly, you shouldn’t *stop* doing design/analysis just because you’re actually writing code at the same time.
Pretty much everyone agrees that you learn more and more about a system by building it. Why should you make a hard commitment to the decisions that you made at the beginning of the project when that’s the time that, by definition, you know the least about what you’re doing?
July 12, 2007 at 9:28 am
I guess by that definition then people like Kent Beck and Ron Jeffries are part of the “lunatic fringe” something I wouldn’t necessarily disagree with.
If you do the right amount of design up front, the decisions will probably not be bad ones and won’t need backtracking.
I’ve never needed to use the roomba method to solve any problems in my career…
Software Maestro
July 12, 2007 at 9:33 am
@Fregas:
You say: “Agile is a reaction to beauracratic, over-documented, segregated, inflexible, and over architected project management methodology that was waterfall.”
In reality, XP anyway was a reaction to a beauracractic project AT CHRYSLER.
Agile builds up “waterfall” as a straw bogeyman, and then posits “agile” as a reaction to a poorly run “waterfall”…
As if there are only two choices: Agile chaos hell, or Waterfall beauracratic hell, and which would you rather have?
In this case the cure is as bad or worse than the disease.
While people are checking out the “agile maninfesto” they should also check out my
http://softwaremaestro.wordpress.com/2007/07/02/does-xpscrum-violate-the-agile-manifesto/
Does XP Violate the Agile Manifesto where they can find that Agile is not consistent with itself and is anti individual.
July 13, 2007 at 11:43 am
See, I just don’t find that Agile is “Chaos Hell”. I think agile is actually closer to the middle, except for the absolute extremists. At my current company, we had almost no process whatsoever. There was some documentation and analysis up front, but very little. After that the developers went off in their “coding caves” for 1-6 months, came back and it was shit because the requirements were not understood, not completely discovered or the client had changed their mind on some things.
Don’t confuse XP with Agile either. I was talking in general terms about Agile. You were talking about one form of Agile called XP.
We’ve started doing scrum, and its actually MORE process than what we had before. We’ve started moving from the far left (Chaos) to the middle (Agile) but not quite to the far right (Waterfall). We’re hitting deadlines, we know what we’re supposed to be working on, we have some idea when certain things are going to be done. We’re starting to do development in smaller chunks and show our clients’ iterative deliverables to make sure we’re on track. Its not perfect, but its better than the chaos we had before, and better than the politically charged waterfall at my last job. Less things are slipping thru the cracks.
I haven’t done XP specifically, but i have read some of the book but I don’t see anything in there that says don’t do any documentation. XP at LEASE has User Stories which are a form of documentation. It sounds like you just had a bad experience with XP, but that doesn’t mean XP is always implemented so poorly or it doesn’t work for anyone. Also, XP focuses more on software development practices (paired programming, TDD, weekly iterations, Unit Testing, etc) and LESS on project management. Scrum deals more with Project Management. So maybe XP on its own tends to lead to chaos, because there is minimal documentation and management. Maybe for small shops its fine but larger shops needs Scrum, crystal or something else to manage it all. I have heard that many shops use XP & Scrum together: XP for their engineering practices, and Scrum for the management aspects.
I don’t think documentation is bad, but I feel it should be lean. I think the larger projects need more documentation, but documentation always tends to veer from reality very quickly. I think the general agile concepts of personal communication, feedback, small iterations, help make up for this failure. You’re right. Its not either/or all or nothing, nor was it ever meant to be.
I read your article about whether XP violates Agile. I think that anytime a methodology is inflicted on team members without their consent, e.g. they are not on board, its not very “Agile.” The book “Agile Project Management with Scrum” by Kent Shwaeber talks about this very experience, and he had to back off on some of the changes he was trying to make using Scrum. He had to ignore some processes because the team members didn’t want to do them, and he chose to honor the team and Agile principles, rather than stick obsessively to the “official scrum” process.
I disagree that things like stand up meetings, collective code ownership, etc. violate the Agile Manifesto or necessarily decompose into some form of “Agile Communism.” We move people around from project to project and from one piece of code to another all the time. Noone really has a problem with it. We do specialize, so while i may be able to modify some CSS or graphics, i’ll usually turn it over to our UI Developer if its something complicated. He in turn almost never touches and C# and ASP.NET, but i’m teaching him a little bit of it and he’s picking it up enough to do simple tasks. The other C# developers will change my code and vice versa. If there is a budget or time issue at stake, then the guy most familiar with that part of the code will usually work on it. Its really not rocket science, and it doesn’t feel like communism or stalinism. We all get along pretty well, but we’re a small team. I think individual code ownership is actually more dangerous to a business, because of two reasons:
1) if that guy quits, noone else knows his code and it will likely look very different from everyone else’s.
2) if you put the same guy on the same project or part of a project, he’ll often get burnt out.
Its just not as bad as you make it out to be. There are (and should be!) exceptions to every rule. If you adhere to anything dogmatically, it will be a curse rather than a blessing. The scrum book i mentioned even says “Change the process!” if something doesn’t work or needs to be tweaked.
July 13, 2007 at 2:22 pm
Deferring design commitments until the last responsible moment is not the same thing as acting like a Roomba.
You (rightly so) talk about the tendency of Agilists to create a “waterfall” bogeyman/strawman. It’s something that I know I’ve done in the past, and have resolved not to do. Likening an approach to a non-sentient robot vacuum cleaner? Not exactly respectful.
Anyway, what would happen if you were to temporarily discard the specific process implementations (such as Scrum and XP, both of which have their dysfunctions) and you look at Agile (and Lean, for that matter, which I think is a more useful framework) as just a collection of pre-packaged values? Would you have the same negative reactions?
July 13, 2007 at 9:18 pm
I think deferring design commitments (like Scalability) to the last moment is near suicide.
And that is how the Agilists would do it… YAGNI.. Then when you do, rewrite the whole system… Great….
As far as discarding Scrum and XP, what do you have left?
Iterative Process and some Unit Testing?
Those are things I believe and have been using long before they were published.
Everything else about so called Agile, collective code, pair programming, daily scrums, user stories instead of specs, user stories on cards, all of that nonsense, I have zero use for.
If you’re in favor of “Iterative” development — so am I — and I think it’s time for people who care about Iterative and nothing else about Agile to start espousing Iterative development.
The rest of it is just a collection of one-size-fits-all dogma thrown onto a common and well known concept and relabeled “Agile” or “Scrum” in attempt to create a Brand Name merely to make money from it.
The fact that some iterative proponents, possibly yourself included, jump on anything they see as facile and call it “Agile” merely futhers to muddy the water and promote a brand name filled with shills and posers….
I’m sure we have some common ground here
As far as “calling it roomba” is disrespectful — that is just calling a spade a spade. Test first means make the test fails and then try anything, or at most, “a little” design to make them pass. Go read Ron Jeffries laughable attempts to use that method to write a Sudoku Solver, referenced above, and you’ll see that calling it roomba is largesse in the extreme.
Software Maestro
July 13, 2007 at 9:27 pm
@Fregas:
See you validate my thinking completely, which is that the only people who would be impressed by Scrum/XP/Agile are people who had NO process before and anything was an improvement!
You say: “At my current company, we had almost no process whatsoever. There was some documentation and analysis up front, but very little. After that the developers went off in their “coding caves” for 1-6 months, came back and it was shit because the requirements were not understood, not completely discovered or the client had changed their mind on some things.”
Well of course — no process and then big bang integrations is a recipe for disaster. But does that mean daily meetings and daily integrations are the answer?
It’s like someone who has been starving to death for weeks, then finds some Cheez Whiz and Spam, and it’s like manna from heaven.
Then these same people decry fine cuisine as overly documented, and you won’t be hungry by the time you finish making it….So you just eat Spam and Cheez Whiz because it’s quick and no documentation and that way you(or the customer @/) won’t change your mind while baking something.
Obviously, I’m not convinced, and the reason I’m not convinced is because I’ve been on, managed, and even been the CEO of companies, that had a well run software development process; that had bug free code on time and on budget- without TDD, User Stories, Daily Scrums, or any of the tripe, hype and dogma recommended by Agilistas round the world.
It certainly is hard to describe the taste of chocolate to someone who hasn’t tasted it, but I can definitely tell you it doesn’t taste like Cheez Whiz.
If people don’t like it being called tripe, I’m a little sorry, but I have to be clear: when people write books filled with tripe, and call it down home cooking, those of us who know how to cook have to say something about it.
But it sure won’t stop people from crowing about how great their favorite greasy diner is.
Software Maestro
July 16, 2007 at 8:11 am
Maestro,
I think you make a counter arguement, but you ignored this part of my post:
“…and better than the politically charged waterfall at my last job”
At my previous job, it was a large company with lots of beauracracy, and our process was basically waterfall. Developers were expected to find ALL the requirements ahead of time in a slew of long meetings, write mounds of documentation (screen shots, functional specs, use-cases, etc.) and then start writing code to spec without changing anything. The customer was required to literally “sign off” on the documentation and so the programming group would not have to change anything during development.
It didn’t work. The reason it didn’t work is because people don’t know what they want/need until they see it. Prototypes and screenshots and documentation CAN HELP, but until they see working software, you’re not going to know whether you’re on track. Our software group didn’t have the political clout to say “no” when the users inevitably changed their minds on what they wanted, and even if they did, we’re there to serve them, not the other way around. So these mounds of documentation and design up front were mostly waste.
I’ve been on both sides–waterfall and chaos. I don’t do TDD, paired programming or User Stories–we’re not an XP shop. I do have daily scrum meetings and they’ve helped, without becoming a black hole of project management with long meetings that i had at the last job. They are short and to the point and keep everyone on track.
I am interested in hearing what methodolgy you use Maestro. Is it waterfall or is it something else? I’m not sure what alternative you’re proposing as your comments seem contradictory. You’re calling heavy documentation “fine-cuisine” but earlier you said that watefall was too heavy and agile was too chaotic. So what do you see as being the middle ground?
You also asked what was out there besides Scrum and XP. The answer is A LOT. While those two are undoubtedly the most popular, there is Lean, Crystal, RUP and others. They don’t get quite as much press but i think they are certainly worth looking into.
July 16, 2007 at 11:15 am
I’m not a “methodology expert” so I don’t read and study various methodologies to see what is the best.
What I do, and at somepoint I’ll write a more in depth blog about it is very straightforward:
1) Hire the Best and Brightest
2) Use the Specialists for what they are good at
3) Get a Clear up front Specification in Writing
4) Start a Design and Prototype Interfaces/performance
5) Modify Design as Necessary
6) Begin Coding all the Unknowns first, since those are the ones that are impossible to predict in terms of schedule; eliminating these makes it easier to predict a schedule going forward
7) Incrementally develop and refine the software organized at medium length releases in the 2-4 month range.
8.) Use a Bugtracker with bugs/wishes/features prioritized and hand those off to developers on a daily basis
9) Teams are organized into 1-4 persons “Squads” in charge or a particular module, providing pride of ownership and consistency of implementation and utilization of each persons skillset in a way that is most benfecial to the project.
Nothing fancy… as far as meetings I believe what is important is clear communication, not face time.
Whether it’s emails, meetings, wikis, diagrams, all of those things can work, but I strive towards clear communication and not dogmatic process..
To be clear, in my writings I’m criticizing Scrum and XP in particular, since these are the most useless, as well as most well known and most clearly overhyped and misleading a generation of programmers and managers with extremely false promises and laughably poor “logic”.
That doesn’t mean any iterative process is bad and I would hope that people can stop calling iterative agile now
Agile = Iterative + Brand Name + Hype + Dogmatic Practices
Frankly I’m not at all interested in the Hype or the Dogma, and most of what’s useful in Agile has been known for quite a long time; they are merely taking sugared water and making a brand name to sell it (Pepsi)…. Then they have a lot of voodoo and dogma to make it seem unique, a cult like clubby marketing plan, and a vast herd of sheep to keep bleating out the same platitudes all the web over.
The most important thing a programmer can do is think and be creative; it’s the art, the magic or software;
No certification, no methodology, no college can teach that; either you have it or you don’t.
Software Maestro