One of the key things I promote these days is the constant, continual peer evaluation. This helps solve a large part of the problem with automated unit tests which is that they cannot understand context. The most efficient way I have experienced to do this is the Buddy System. This is a big part of the Jonah Methodology even if they do not promote it on their site (though I think they should).

Remember back when you were a kid on a class trip and you had a buddy you had to stick with for the day (in theory to keep each other out of trouble)? The similar idea applies when developing software. The XP community has been using the Buddy System for years in the guise of Pair Programming. I’m pretty sure that the XPers have produced studies to show productivity games of having pairs of coders sharing a single keyboard but I don’t buy it. What the Buddy System does is take the benefits of having two sets of eyes on a change but without the full time consumption of pair programming.

Here is how the process works:

• Developer A produces some code
• Developer A asks Developer B to Buddy them
• Developer B then sits with A for a session (more on this in a bit)
• Developer B provides feedback on the change and when it is acceptable,
• Developer A then commits their change recording in the the commit notes who the buddy was (you can enforce this using a pre-commit hook if you are using subversion

So what sort of conversations occur during a session?

• Show it working
• Which exceptions are you anticipating and how are they handled?
• Do the existing unit tests all continue to pass?
• Which new tests have you added?
• Any Security issues?
• Can the page / process handle non-English text data?
• Are Corporate/Industry/Company standards met?
• Does it build clean?
• How have you improved the existing code?

Obviously the nature of the change will dictate which questions are asked but those are the big ones in most cases.

The concern people typically have when hearing about this the first time seems to be that it will make their development cycles longer. To borrow from the TDD crowd, it should actually reduce the time (or at the very least maintain it) since there will (in theory) be less rework / debugging associated to changes which have been buddied.

A smart build system coupled with the Buddy System are pretty strong foundations to build a quality system around.

For various reasons, not the least of which dealing with power distance, your team members / employees might not be completely honest with you on how you (and the team you lead) are doing. One thing that might help is if you get someone who is independent of the team to conduct debriefs with your group individually to find out what is working, and what isn’t.

To get this to work though, there has to be at least one ground rule and that is anonymity. People are not going to open up if they suspect that you are just going to go to their boss and say ‘Herb thinks you are an idiot.’ I’ve seen situations where people have couched their responses in ‘my boss is going to know who said this’ to which point you should point out that if that is truly the case, then they don’t need to hide because the boss already knows. If something seems to be specific to that person, ask permission to cite them.

When I was doing this sort of thing while at HP in the context of a CMMi self-assessment-y thing I started the conversation with ‘Tell me what bugs you.’ There is almost always something that bugs a person about their job and if these things start to cluster you know you have found a problem.

I think I would also try to keep these things as informal as possible. By this I mean, don’t contract McKinsey to do this, but perhaps bring someone from your local testing community in who you trust. I suspect that some form of payment might also be required to ensure a certain level of professionalism and air of importance within the organization. In lieu of payment, perhaps you could do the same for their team.

An implementation timeline for this sort of thing could be

• One week prior, mention it at the team meeting and ask the team to mull over things bad, and good they will say
• Start the actual meeting / interview day with a team meeting to introduce the person doing the interviewing, set the schedule, reaffirm the rules, etc.
• Conduct the interviews in a meeting area which guarantees privacy (as in a room with a door, not a communal lounge area)
• At the end of the day, get everyone together again and thank them for participating
• Within a week, the interviewer should compile the notes to the person who started the process
• That person then needs to address at least some of the problems, and reinforce all the items seen as positive. Without this visible feedback loop the who process will become a sham and next time you try something like this it will not be taken seriously

You might even expand this to people outside of your team and include people such as the development team lead, the product manager and others who have direct stakes in how your team functions.

Big disclaimer: this idea just popped into my head and I haven’t actually implemented it. It’s certainly interesting though.

All software, whether intentionally or not, is developed within some sort of model. Be it waterfall, agile, spiral, etc. Each has it’s proponents and detractors, but the common to them all is that they can be drawn out on a whiteboard to explain them. Typically I describe how software is actually built in the world at large (from my experience and speaking with others) is to draw something very waterfall-ish and then point out the places where you can inject Quality into the mix.

Where am I going with this? Well, through a discussion on the software-test mailing list I became aware of the Prince2 process model. Two things about it make it interesting from a QA perspective is

• All deliverables need testing – note how they do no limit this to ‘software deliverables’
• Nothing is complete until it is checked, reviewed and tested – again, note the ‘nothing’

Of course, things like Prince2 and CMM are just tools in the greater battle against bugs and are not suitable in all environments, but I think theyve got some of the philosophy correct. Now if only they had a diagram for the software-build process and not just the project-management process.

Post 2 (of 2) for catching up on my inbox

Agile Testing – 892 messages
CMMi – 617 messages

(more…)

Well, its been over a month, so…

FitNesse – 457 messages
Agile Testing – 892 messages
CMMi – 617 messages

(more…)

It’s that time again

FitNesse – 202 messages
Agile Testing – 202 messages
CMMi – 159 messages

(more…)

Chapter 11 – Software Testing

(more…)

It’s that time again

FitNesse
Agile Testing
CMMi

(more…)

I’ll be posting what I consider the interesting tidbits from the CMMi mailing list I subscribe to periodically from now on.

• In terms of systems analysis, there is no such thing as stability. There are only times when variations in the inputs are not large enough to cause the system to fail.
• SAM deal with anything that is “integrated with your product that is shipped to the customer”
• CM audit ensures “you have the stuff, the right stuff, and nothing but the stuff” that you’re supposed to have

We had Mike Cohn of Mountain Goat Software give a seminar on Agile Estimating and Planning this week. Here are my notes.

(more…)