Posted on September 9, 2012 in Uncategorized by adamNo Comments »

Beyond the Matrix is an excellent article for a number of reasons. Not only does it give a peek to how a major movie came together, but it also gives insight into the difference between writing a book and making a movie.

Here is a paragraph from page 7.

The set was rudimentary: the control room of the satellite-communication center would be completed with computer-generated imagery, imagined by the Wachowskis down to the minutest detail. The scene in the control room, for example, features an “orison,” a kind of super-smart egg-shaped phone capable of producing 3-D projections, which Mitchell had dreamed up for the futuristic chapters. The Wachowskis, however, had to avoid the cumbersome reality of having characters running around with egg-shaped objects in their pockets; it had never crossed Mitchell’s mind that that could be a problem. “Detail in the novel is dead wood. Excessive detail is your enemy,” Mitchell told me, squeezing the imaginary enemy between his thumb and index finger. “In film, if you want to show something, it has to be designed.” The Wachowskis’ solution: the orison is as flat as a wallet and acquires a third dimension only when spun. Mitchell, who had been kept in the loop throughout the process (and has a cameo in the film), was boyishly excited by the filmmakers’ “groping toward exactitude.” “I was like Augustus Gloop in the Wonka factory,” he told me. “I’ve witnessed a long sequence of decisions, which I never had to make while writing a book. Intellectually, I know it’s a replacement, but I don’t feel a loss at all.”

It is this sort of detail that appeal’s to my tester’s mind. And one which I have been noticing since hearing an interview with L. E. Modesitt, Jr. on Writing Excuses (I think it is this episode, but didn’t double check) where he discusses things like the support logistics of great fantasy battles that authors seem to forget [ignore].

And now, you will notice them too.

(You’re welcome)

Posted on August 28, 2012 in Uncategorized by adamNo Comments »

My son is presently baseball-mad and so went to training day with some Blue Jays last January as part of their Winter Tour. I have no idea how to coach baseball to kids, so I took some notes. They’ve been floating around the basement since then so I’m de-paper-ing them by putting them here.

  • Stretches
    • butt kicks
    • side strides
    • long steps
    • frankensteins (high kicks)
    • flamingos (balance on one foot, lean forward)
    • circles with arms
    • fall, then sprint
    • cross legs, then up without using arms then sprint
    • watch the pitcher, then sprint
  • Batting
    • Use a tee
  • Bunting
    • line up with the front of the box so if the ball drops straight down it doesn’t hit the plate; if it does its a strike
    • pinch the bat
    • rotate feet (into ‘athletic position’)
  • Base Running / Paths
    • touch at the front of the bag
    • run straight!
    • lean when crossing the plate to make it seem like you are further ahead than you actually are
    • game: relay where half run from home and half run from second
    • the high-5 at the bag is important
    • when going multiple bags, start running in the shape of a banana halfway to first
    • take two steps from the bag, square them, then two side steps to be in optimal position
  • Infield Fielding Drills
    • direct grounder
    • glove side
    • throwing side
    • random side
  • Outfield Fielding Drills
    • routes to ball
    • glove positions (above or below hips)
Posted on July 5, 2012 in Uncategorized by adam9 Comments »

Twitter is excellent for planting troll-bait one-offs, but the blog is better for planting long-tail ones. So this afternoon I tweeted something along the lines of this.

Unless your are interviewing someone for a job that requires [stupid] logic problems, then they don’t have any place in an interview.

And I stand by that. Though a lot of twitter seems to disagree, to one degree or another.

Context. I was at a client today and the two people I am working with came back from an interview…

Them: Do you like logic problems?
Me: Not really.
Them: Want to try the one we just gave in the interview?
Me: Sure…
Them: How long should we give him? 8 seconds? 5?
Them: Design an algorithm that solves Me: (immediately) Dunno.
Them: Give up?
Me: Nope. I didn’t come up with something immediately and don’t care enough to focus any thought on it
Them: …

This of course led to a much larger conversation about the value using these sorts of things in an interview has. My mentioned above, I don’t think they have any. (Or at least very tiny minuscule amounts to avoid the use of an absolute…).

More context. The interview is for a position that will primarily be webdriver automation using python with some manual testing thrown in there as well (and will be taking over the stuff I put in place.)

So. How does the ability to answer a logic problem help determine the candidate’s ability to do the job? Very. Little. Here is a secret; web automation is not a ‘logic’ problem. It’s not even that hard. It is way easier to be an automator than a tester.

What should you spend the time in an interview for a web automator instead of wasting everyone’s time with the logic problems? Ummmm, how about you have them automate your app. Right there. For real. Projected on the screen.

  • What oracles do they establish?
  • Which patterns do they use?
  • How ‘robust’ are their locators?
  • How is their synchronization?
  • etc.

These are all critical things to understand about an automator. Especially the one that will take the lead on the project. Notice a distinct lack of explanation why manholes are round or how to determine the odd billiard ball out in a minimum number of steps.

(Oh, and looking up syntax is absolutely ok. It shows they know where to look. We have this internet thing — we don’t need to memorize everything anymore. Now, reliance on an IDE to program for you is not ok.)

I see logic problems in an interview almost as a crutch that the interviewer is using to hide that they don’t really know what they are looking for. And as an interviewee, if a logic puzzle rears its head in an interview the bozo bit gets partially flipped on the whole situation. Do I want to work at an organization that values people who can solve logic problems? Or that are awesome automators and/or testers. (This isn’t to say there isn’t a possible intersection between the two groups, but it is not guaranteed.)

What are a few arguments I have received today in favour of logic puzzles; none of which sway me…

  • It shows a high IQ and high IQ is needed for a testing role – People play lawyers on the internet, so I’m going to play neuroscientist. So first, all being good on logic problems means is … you are good at logic problems. A lot of people spend their free time doing these things so they train their brain to spot patterns and build their own solving heuristics for them. Same for crossword-ers or word search-ers. Heck. Look at the whole ‘tester game’ thing. (Of course, the tester game phenomenon has a large degree of arrogance associated with. And no, I won’t play.) Back on topic. No, you also do not need a high IQ to be an excellent tester. Again, I could be conflating IQ with other things (only playing neuroscientist remember…) but there are more firms hiring Autistic people as testers (article).
  • It lets us see how they work through a problem(1) – Sure, but is it a class of problem they will experience at the job you are hiring for? Those are the problems you are hiring for.
  • It lets us see how they work through a problem(2) – As usual, Michael Bolton provided a Feynman reference (even if it is fictional); Round Manhole Covers, or: If Richard Feynman applied for a job at Microsoft. Its a fun read and on target in capturing what I have seen of Feynman. But he gets a job offer for marketing. Not in test. Not in web automation. His answer would tell me nothing about his ability to do those jobs, though would tell me not to engage him in an argument about manhole covers.

A slight variation on this is of course, FizzBuzz. FizzBuzz is loved online and is somehow seen as the bar that should be met to be called a programmer. I have not once had to use anything FizzBuzz like in automating apps since I started in early 2000. (And that is of course the only experience I can have.) If you are hiring someone whose responsibility includes programming, absolutely you should have them write code. But have it be relevant to the job! As mentioned above, for a web automation job, have them write web automation code! And if you must use FizzBuzz, then you should write a web app that does it and have them automate it.

Way at the beginning I mentioned that the job that originated this discussion is mainly automation and partly testing. The automation side I think I’ve beat to death. Interviewing for testing is, perhaps unsurprisingly, very simple. Have them test! Logic problems will not help you determine if they have a set of heuristics and philosophy for mobile, or payment gateways, or video, or, or, or. Unless your product is vapourware (or super crazy top secret — at which point, have them test something else), have them test your real product! Again, live! And not just silently do it pointing out [possible] bugs but out loud explaining their thoughts. (I wouldn’t have them write bug reports as an interview is a fake scenario and you’re not going to get a ‘real’ bug report.) Flashing back to Feynman, this is the useful part of the article. The thinking and rationality behind the thoughts. As an interviewer, that’s what I care about. Does it clash with mine, does it create overlaps, does it fill capability holes, etc.

But you can’t get that from the creative process solving a logic problem! As a community we claim to care about context, but interviewing outside of context seems like a waste of everyone’s time.

Posted on May 7, 2012 in Uncategorized by adam3 Comments »

I have come to realize in the last two weeks that there are two words in the Agile lexicon that really bug me. Not because they are bad per se, but that they are so commonly misused and abused. Not to mention being insanely context sensitive.

I’m doing what might be considered an agility assessment for a company and am entering the report generation phase after a number days of observation and data gathering. There is nothing too ‘OMG! MY PANTS ARE ON FIRE!!!’ but one thing that sticks out is their working definition of ‘Done’.

I don’t think anyone who has played the role of Agile coach or consultant has not written a blog or paper at some point on ‘Done vs Done Done vs Done Done Done’.

This company has four different teams which are pretty autonomous and effective and one of the things each have done is define their own understanding of Done. While this seems like a good idea on the surface it is really quite dangerous. With each team having their own definition of Done there can be no common understanding when something is Done and that information rolled up with other non-contextual information to management.

Having the development teams generate the definition of Done also means it is limited to their scope of influence, care and interaction. The net result is a very naive scope of Done. Stories that are ‘done’ frequently will go out into production without adequate analytics hooks or means of monitoring the health of the component being created.

This nicely dovetails to the other most dangerous term; The Whole Team. I was at an automation workshop this past weekend and the people there spent almost 45 minutes talking around this subject. In retrospect no one asked what everyone meant by ‘the whole team’. This is another area I suspect Agile teams need to work better on. I would bet that most people’s working definition of the whole team includes the developers, the testers and someone in some sort of product ownership role (either directly or in proxy). But what about the analytics team, or the sales folks who use the output from the analytics team? Or the Ops team, including the DBAs that need to push the code out onto the iron? How about the customer support people who deal with the brunt of the code team’s mistakes?

The list is not endless, but it is large. One thing I like from the Lean Startup [Cult|Movement] is the the story isn’t done until it in production getting validated by real users. The user is on the team whether they realize it or not. We’ll just gloss over the lack of humans in the release process…

I think there is actually some sort of circular feedback loop at play between Done and Whole Team and you cannot address just one in isolation. Instead, really figure out who your team’s stakeholders are and what skin they have in the game. Only then do you have a chance, and only a chance, of coming up with an appropriate, inclusive, accurate definition of Done.

Posted on May 1, 2012 in Uncategorized by adam1 Comment »

I’m most of the way through a client on-site engagement to do an ‘assessment’ of where their testing processes are and to suggest tweaks to how they do things. I have some broad recommendations that will morph into blog posts over the next month or so, but right now I am challenged by coming up with something that doesn’t sound too much like a consultant-y dodge around ‘When will our [functional] automation remove the need for our extensive manual regression testing?’.

This is of course a trap.

There is no way to determine this number with any degree of certainty.

But what if you really needed to provide an argument around the inability to predict this? My current, best argument around this is that we, as humans, we are really bad at predicting the future.

  • Just because you are having a certain degree of automation success now, does not mean that the rate of success will continue at all let alone along the same trajectory.
  • Is the competency of the developers going to change? For better? For worse?
  • Is the competency of the testers going to change? For better? For worse?
  • Will the market fundamentally change?
  • Will the types of stories that come into the development groups remains such that there is a high degree of [perceived] value from automation?

All of these are completely out of control of, well, everyone. Yes, even the ones about the competency of staff. Sure, you can change the way you hire and the way you train, but you can’t change the fact they are people and people are messy. There are however things that we can at least fool ourselves into thinking we have some degree of control over. Though not accuracy…

  • This particular client is migrating two older application stacks to a new one, so the source of some of the regression risk (the older stacks) will be removed from the equation (replaces, of course, with a new set of risks but the plan is they will be less).
  • With [better] exploratory testing will regressions be caught during ‘feature’ testing?
  • If the developers adopt a TDD approach to code generation rather than ad-hoc post-coding security blanket unit testing, will that decrease regressions?
  • Will the internal focus on a single browser allow for more focused testing (with breadth covered though a testing partner)?
  • Will proper management of environments lead to less false positives, less re-testing and/or idle testers?

Because we cannot predict the output of any let alone all of these things, which are all inputs into the equation that pops out a date for that magic point where we could be a half-step away from Continuous Deployment (no humans) then ‘tomorrow’, ‘July 23’, ‘Q3 2013’ are all correct. And all wrong.

But if we think of each of these are part of the overall system that results in regression duration that is deemed unacceptable and/or too long then we can address them each individually there is likely to be improvement in the overall system. To what degree and by which contributing measures is however anyone guess. And only a guess.

Posted on April 9, 2012 in Uncategorized by adamNo Comments »

So this graphic is making the rounds today.

And while it is interesting (at time of posting, Selenium has crossed QTP and the trends are going the ‘right’ way), it is important to also remember this:

Posted on April 7, 2012 in Uncategorized by adam3 Comments »

The Un-Conf is a common event type these days; no scheduled speakers, just a bunch of smart people and a space to facilitate conversation. The success of these is entirely dependent on who is there. And there is the classic Testing Conf where speakers are selected, assigned slots and talk about things directly related to Testing.

But what I think I want a Un-Testing Conf. This would would be the same format of a classic conference but without topics about Testing.


Obviously about testing.

Irony Alert

I am sick of going to Testing conferences and hearing the same sort of stuff over and over. Or something that I could quickly pick up from any number of industry related books or blogs. I want to learn new, mind blowing things. And do you know where those things hide out? Hint: its not in the testing world!

Tangent. Have a single track conference; 5 or 6 concurrent tracks is too much.

So what would my Un-Testing Conf look like? Well, it would likely be expensive since you can’t just comp the speakers a ticket and expect them to graciously accept the invitation. Currently the line-up would be something like…

I can, without even trying, explain how each and every one of the above people have important things to say that would make testers better at their job.

That is a full two days of speakers already. Who else would you like to see? The only ‘rule’ is they cannot self-identify as a Tester.

Posted on April 3, 2012 in Uncategorized by adam2 Comments »

So last week I was in New Orleans for STPCon and one of the things I did was a 15-ish minute gabfest on Set Course For Awesome (as a ‘Career Tester’). One of my supporting ideas for it is that you need to shut up, listen, and figure out what it is you are supposed to be providing information on. (Protip – its not what you think it is; or the second thing; or even likely the third thing.)

One trick I have started to intentionally using is to understand why something is the way it is. Especially for the things that stand out as particularly bone-headed. Often a series of perfectly rational thoughts happened in a perfectly logical order only to have nasty repercussions.

There are lots of examples of this, but when I was in New Orleans I took a ‘pirate tour’ (I know, shocking) and it provided yet another example of this.

  • In 1788, a candle lit a bit of curtain and started a pretty nasty fire
  • It was Good Friday, and the Cathedral bells had burlap on them so as to not ring until Easter Monday
  • New Orleans burned. Almost to the ground.
  • But they rebuilt! In stone!
  • And inside each stone building was a well — not only for drinking purposes but to have ready access to water in case of another fire
  • Mosquitos breed in water
  • There is water in every home
  • Between 1817 and 1904 there were over 41 000 yellow fever deaths
  • Yellow fever is transmitted by mosquitos

All perfectly reasonable individually, but viewed through the lens of history, 41 000 people died from yellow fever because a candle’s flame caught a curtain.

The trick, especially for those of us who see lots of companies is to trace back things and where possible address the original concern rather than just the current manifestation of the concern.

Posted on March 24, 2012 in Uncategorized by adam3 Comments »

I’ve been asked to do a 15-ish minute talk as part of Anne-Marie Charrett and Fiona Charles‘ upcoming workshop at STPCon on being a Career Tester (as compared to going into Management or Consultant — even though I have been both). As I thought about this, I think this could actually turn into a keynote worthy talk (of course, then I would actually have to practice it?!), but I have come up with what I think is basically my advice to myself of 15 years ago when I landed on this career path. Now of course, to deliver it as a double-stuffed-lightning-talk for them.

Set Course For Awesome

Set Course for Awesome by Fake Grimlock. This is also my computer desktop. And is awesome.
Testing is an awesome and rewarding career. But it is also ‘new’ even in the larger context of IT jobs which are themselves new in the grand scheme of things. This means that unlike someone in the trades, the career path for testers isn’t well defined and everyone’s will be slightly (or significantly) different. The thing that will be common to them all though is that those who are happy(est) will have taken responsibility for their career and planned it out. The plan may not work out, and will almost guaranteed to need course corrections along the way, but the act itself is useful. So set a course for awesome. (And lets not concern ourselves about the alternatives to that shall we? Great.)

Shut Up and Listen

Shut Up and Listen. Words to live by.
It was during RST with James Bach that I really understood that the role of a tester isn’t to ‘break the app’ or ‘find all the bugs’ but to provide information about the application. It wasn’t until some time later than I realized that it is actually more subtle than that. Our job is to provide information that matters. And how do we do that? Easy. We Shut Up and Listen. To what? That is also easy. To the people we are providing the information to. Now that I’ve completed PSL I can safely start quoting Gerry Weinberg so here are two useful things to remember.

  1. No matter how it looks at first, it’s always a people problem
  2. Things are the way they are because they got that way

If you can start to understand the people in the system you are operating in, and why the [larger] system operates the way it does then you can provide them the information they need. But you can’t do that unless you stop, shut up, and listen.


This venn diagram should be all you need to plan what you want to do with your life
Too often people let others drive their career trajectory or they just listlessly bounce from one place to another. This handy venn diagram should help you organize your thoughts. Make the three lists. Find the Hooray point. That is your destination. Or more correctly, your current destination as it is likely to change as you experience new things. You are not the person you were five years ago, and you are not today who you will be in five years. I actually have a large copy of this that needs framing and will go on my office wall.

Steal Like an Artist

Book. Of. The. Year.
I’m going to say this right now. In March. That Steal Like an Artist is my Tester Book of the Year for 2012. And it has no direct links to testing. Which is also part of the point. Not only are the 10 points in it completely relevant to being a tester but it comes from a different field entirely. Go outside of the testing community for inspiration. One of my first public testing talks was about being a houseleague lacrosse coach. The most useful books for my testing practice recently have been about how the brain handles stress, how to really practice something and about apprenticeship patterns. None of them were written by testers, but each made me a much better tester. Take inspiration from everything and everyone. When you are constantly looking for it, it is amazing how often you can find it.

Be Yourself

Keep Calm and Be Yourself
The trick to Stealing Like an Artist vs blind theft is that artists make what their steal their own. You will not truly succeed just regurgitating words and ideas that you heard from people at conferences or read on their blogs. The two I see all the time is the SFDPO mnemonic and the notion of testing tours. Think of your own. I know you mean well by parroting it, but internalize it, morph it, evolve it until it is yours. It is impossible for you to be anybody but yourself. So don’t try. Take what you like and what inspires you, make it part of you, and ignore the rest.

Have a Therapist

Talk to everyone you can. But have a couple close people you can bounce ideas and questions off of. Both for ‘shop’ and life in general. You might be smart, but someone is always smarter. You can’t grow if you just talk to yourself or your teddy bear. And it works the same way too. Some of the best lunch chats I’ve had have been initiated by me but I went home with less notes than the person I was trying to talk something through with. The term mentor I think is too constrained. Remember, its always a people problem and people problems require therapists.

Be Totally Fucking Amazing

This is hanging in my office
Quality this, Quality that. Blah. Blah. Blah. As a tester, you shouldn’t fall into the trap of thinking ‘Quality is Job One’. Its not. Being totally fucking amazing is job one. Your customers do not care that the code is quality, or that it follows some style guide line or has only 0.45444 bugs per 1000 lines of code. They care that it is amazing. That’s all. The same applies for your career. Whoop-dee-freakin-do that you have been at the same company for 10 years and got a commemorative clock (do they even do that any more??!?). Are you being amazing anymore? If not, why are you still there? Print out the Hooray graph, plot your course to awesome and go be totally fucking amazing.

Here is the slideshare link to share amongst yourselves.

Posted on March 22, 2012 in Uncategorized by adamNo Comments »

I just finished readying Mary Robinette Kowal‘s book Shades of Milk and Honey which is often described as ‘Jane Austen with Magic’ which so far as I can tell is accurate. (There is magic and I trust that is Austen-esque.) This is clearly a large step outside of my normal Epic Fantasy genre reading and I would have ignored the cover illustration completely in the bookstore (had it a copy, which is didn’t). And even if I had, the Jane Austen part would have scared me away based solely on reputation than actual opinion (and that I couldn’t get through ‘Pride and Prejudice and Zombies’). But I bought it soley on who wrote it. And how I know about her wouldn’t be possible even 5 years ago.

But first, a mini-review.

Having suffered through some ‘classic’ Victorian era books in highschool, I somewhat feared that this would be as equally difficult to get around just the phrasing, but it was a quick, and enjoyable read — I think I went through it in 3 sittings (though one was exceptionally long). The characters fit my mental model of how they would behave and act in ‘proper society’ and the magic system seemed perfectly natural (and thankfully the origins of which were not explained). The book itself (at least the hardcover) has the pages not chopped for even-ness which lends, I think, the appropriate amount of ‘era’ to the book.

Oh. And I only found one type-o; ‘colour’ in one sentence and ‘color’ in the next which is really an spell-checking dictionary problem.

Now for the real reason for the post; an author who is active in various social media platforms can dramatically affect how a reader experiences there work. And not just in a sales perspective (though certainly that does help). Here is the list of ways in this particular case.

  • Mary has been one of the hosts of Writing Excuses for a while now so I get a dose of what she sounds like in 15 minute chunks every Monday. What this did was ‘change’ the voice in my head while I was reading. This is not something I had noticed before but it is usually male voice, but this time it was not only female, but Mary
  • Also from Writing Excuses I knew that the Austen has used [almost] every work in one of her original works as Mary took the complete works of Austen, uniqued it and then used that as her spell-checking dictionary. Which. Is. Awesome. And while appreciating this fact runs slightly counter to one of the themes of the book which is to appreciate the beauty in art without dissecting in, it is still awesome.
  • At one point there is a puppet show in the story, and while it is explained, nothing beats actually seeing it. Which you would, if you followed her twitter and watched this interview in which she performs the show
  • Which she is well qualified to do since she is a Puppeteer first. That itself is hilarious when she is tweeting about making puppet.

  • Mary blogs frequently sometimes posting period costume which helps complete the mental picture of things while reading.

We (I) completely blew the social media aspect of promoting Beautiful Testing. Mary seems to have it nailed. And I have Glamour in Glass already on order.

« Previous PageNext Page »