Posted on December 18, 2008 in Uncategorized by adam2 Comments »

This arrived in my mailbox today:

Congratulations! We’re ready to move ahead with your project.

The project in this case is a book called Beautiful Testing that Tim Riley (Director of QA at Mozilla) and I are putting together for O”Reilly. The vision is that it will have a number of chapter length essays on how tests or testing is beautiful. (Which is a wonderfully vague term to build a book around)

We already have a number of people that readers will recognize signed on who are willing (and able) to participate and will be pinging a few others over the next week or so. Also as important are a number of people whom you might not have heard of have interesting things to say.

Like all books in the Beautiful series (which started with Beautiful Code) all the proceeds from the book goto chartity. Our is Nothing But Nets which provides mosquito netting to malaria infested areas of Africa.

(Good grief, what have I gotten myself into now? 🙂 )

Posted on December 16, 2008 in Uncategorized by adamNo Comments »

Apple released a point release of Leopard this week which means I need to actually reboot my machine. This also means that I need to go through all my open windows and capture the stuff I wanted to remember.

We recently decided to roll out own EC2 images instead of someone else’s in order to have exactly what we want in our instances and nothing else. EC2 on Rails and similar things are handy, but do I want mysql or ferret on an apache loadbalancer machine? No, I do not.

Here, in some sense and order are things to remember when building your own EC2 image collection

  • Useful article to get yourself going, but give yourself more than 750M to work with. Depending on how you wire up your instance this could bite you sooner, or later, but it will.
  • Even a bare instance will need to have Ruby and Java installed
  • Building images using ec2-bundle-vol is much handier than with FINDME
  • Make a base image and then base subsequent image of each other type
  • Keep track of the packages you install in the child images
  • When you rev the base, rev the children
  • Don’t forget to use –generate-fstab with ec2-bundle-vol
  • Install the AWS AMI and API tools and symlink to them for easy updating
  • To get your debian instance to boot you need to modify image.rb before making the image
    - exec( 'for i in console null zero ; do /sbin/MAKEDEV -d ' + dev_dir + ' -x $i ; done' )
    + exec("cd #{dev_dir} && /sbin/MAKEDEV console && /sbin/MAKEDEV std && /sbin/MAKEDEV generic")
  • Think about your port needs and build layers of security groups
  • Install the kernel modules; eventually you will need them.
  • Borrow liberally from other people’s build scripts: Eric Hammond‘s is pretty good
Posted on December 15, 2008 in Uncategorized by adamNo Comments »

The other trade rag that was queued up is Software Test & Performance. Here are my thoughts on the last three issues.

October, 2008

  • Talend seems to be a company that is trying to make a go at thingss using the value-added model to open source. The Data Quality Suite was just released and seems pretty interesting. Or at least it does with the database I have. Can I have a native Mac version please?
  • There is a good overview of the hows and whys of detecting ‘borrowed’ code by Bob Zeidman called Are There Copycats In Your Company?. It gets mathy pretty quick, but if your cares about this sort of thing (such as you when you have an closed-source application written which has functionality of an open-source one) then this might be something worth integrating into your testing process.
  • Elfriede Dustin starts a series of articles about test automation. Check out the ‘Building Testability Into
    The Application’ and ‘GUI/Interface Testing Recommendations’ sections for things that make automation easier.
  • The main idea behind Development and Testing: Two Sides of the Same Coin is that test automation projects are coding projects, just done by the test team and as such need to be treated and developed the same way. I agree, but not necessarily to the waterfall-ish model presented.

November, 2008

  • The Testers Choice winners are listed. Doesn’t seem to be much difference between last year’s winners and this. I wonder how many products get actually purchased as a result of these. I remember a story I was told by a Mercury consultant in late 1999. Mercury became the number one automation platform because it had a campaign that advertised that they were. Managers (and their influencers) saw that and figured it was a safe bet to buy the products that were best in the market. Mercury said they were, so they bought the products and the resulting revenue increase made them number one in marketshare. Nobody ever got fired for buying IBM meets test automation.
  • Step-by-Step Performance Test lists 5 types of performance tests and spends about half a page on each. Those types are:
    • Response Time
    • Load
    • Stress
    • Stability
    • Database Volume
  • Elfriede Dustin continues her series about automation failure with the various myths that surround automation. Quick, how many have you been hit with? Likely all of them if you have been around for a couple years.
    • Test Efforts are Immediately Reduced
    • Schedules are Immediately Compressed
    • Automation Tools are Easy to Use
    • All Tests Can be Automated
    • Automation Provides 100 Percent Test Coverage
    • Test Automation is the Same As Capture and Playback
    • Automated Software Testing is A Manual Tester Activity

    And of course, you yourself have to be careful of Losing Site of the Testing Goal: Finding Defects

December, 2008

  • Elfriede Dustin finishes the series on test automation by presenting the Automated Testing Lifecycle
    Methodology (ATLM). These are the phases of it.

    • Requirements Gathering— Analyze automated testing needs and develop high level test strategies
    • Design & Develop Test Cases
    • Automation framework and test script development
    • Automated test execution and results reporting
    • Program review
      • It is very waterfall; in fact, they she talks about gating the steps. I’m not sure how it would mesh with agile-ish environments.

Posted on December 14, 2008 in Uncategorized by adamNo Comments »

As the previous post might have suggested, I downloaded tomorrow’s edition of SD Times today and figured I should actually read the last three month’s worth.

Unfortunately, they have put up a login-wall to the various issues so direct links are out, but here is the backissue link.

October 1, 2008

  • LDRA‘s operations director Ian Hennell – By providing customers with a comprehensive set of tools for requirements traceability through to unit testing, LDRA’s tool suite portfolio takes developers much closer to the goal of zero-defect software development.. Aside from zeo-defect software being a myth, they look like they like have some interesting tools that look like they are aimed at the CMMi crowd. Their tools appear to also be for C/C++ and Ada which is an area that seems to get overlooked these days.
  • ReplayEngine seems like a pretty cool debugging environment by letting you step through execution forwards or backwards.

October 15, 2008

  • Rational AppScan Build Edition embeds Web application security testing into build management in order to automate the security test function. About time.
  • The article Lost in Translation is ok I guess and has lots of quotes from big companies about managing the gap between what the customer wants and the developers deliver. It’s best point I think is made by BJ Rollison: one leading cause may be that many companies overload developers and testers with the added responsibility — and burden — of pretending to understand the customer’s needs as well as the marketing people or program managers [do].. Talking to customers and getting accurate requirements from them is a black magic skill a lot of developers (and testers) lack.
  • David Linthicum lists the traits of a good SOA architect in his column, but I think they apply to any architect or senior role (including tester)
    • Can speak the language of business and IT
    • Understand how to work with complex technologies in challenging environments
    • Can drive toward a strategy through effective tactical implementations
    • Are experts in the available SOA technologies and in SOA best practices
    • Can work effectively with the resources they are given

November 1, 2008

  • SAFECode (The Software Assurance Forum for Excellence in Code) has two papers available on their site on Software Assurance: An Overview of Current Industry Best Practices and Fundamental Practices for Secure Software Development which might be interesting reading if I didn’t have a huge list of things already.
  • Is it just me or is ALM (Application Lifecycle Management) the next hot buzzword? Of course, if you rely on a single vendor / toolset for your entire lifecyle you lose the ability to use the tools that best solve your specific needs and not just the ones that will solve most of most people’s problems.

November 15, 2008
Nothing really of note except this almost throw-away blurb: IBM has created Rational Test Lab Manager, a software product that helps monitor energy use in a business. IBM executives said Rational Test Lab Manager configures and manages the use of physical and virtual machines to conserve energy by using analytics and reporting from Telelogic System Architect. Someone from IBM should contact me to integrate a lab’s carbon footprint into their application next. 🙂

December 1, 2008

  • I didn’t have a name or a reference for the standard that avionics software is measured against. Now I do: DO-178B, Software Considerations in Airborne Systems and Equipment Certification.
  • The feature article, Agile for a New Age is about scaling Agile techniques to enterprise levels and might be interesting to people who have been told they must make their teams ‘Agile’ but have more than 5 people working for them that are geographically dispersed.

December 15, 2008
For Java folks with large legacy code bases (in the Michael Feathers style; so no unit tests), Pillar Technologies wants to help with their Verde product. It will record what the VM is doing and save that as unit tests. Could be pretty trick.

Posted on December 14, 2008 in BITW by adamNo Comments »

I found this installment of BITW while downloading the current issue of SD Times.

There are actually two bugs (at minimum) on display here

  • The most obvious is that there are two entries in this select box that do not have labels and so just display as ‘s. If I was troubleshooting this I would first look for incomplete records in the database and possibly suggest a ‘no null’ condition be applied to the column. If all the records that should be displayed are, then the likely culprit would seem to be a hardcoding of the records to display. (Or could actually be both.)
  • The other is a grammatical mistake (which if you have ever seen me on IM or IRC you will note the irony of me pointing this out). ‘s indicates possession. For example, Adam’s blog. Doing this sort of dynamic pluralization opens up a world of potential hurt as there are so many edge cases (such as SD Times’ bug). It also means that you cannot (easily) internationalize the site. Interestingly, the first item in the list is correct.
Posted on December 13, 2008 in Uncategorized by adamNo Comments »

I first heard the expression All Hat and No Cattle while at HP from my boss’s boss’s boss’s boss. (or maybe their boss). As I’ve been thinking about training and such this expression has been coming back into mind. I think it sums up what a lot of people that come out of training courses and suddenly taught themselves as ‘experts’.

The problem is though, that some of use know the difference between a bull, steer and sidewinder. And with that knowledge we can see right through costume, regardless of how trick the hat is. It is far better to be honest when talking to people about your recent acquisition of a new hat and you know how to wear it, but that is about the limit of your skill-set.

(And no, I do not have cattle, nor have I participated in the breeding of them. I can however spot problems with horse’s gaits and lameness better than most it seems. I just couldn’t come up with a horse example)

Posted on December 12, 2008 in Uncategorized by adamNo Comments »

The really good testers not only think about things differently, but they also observe things are a much greater details than others. Poirot would likely have made a wonderful tester for example. Here are three videos from the Do The Test campaign that is promoting awareness of cyclists in / around London, England. How perceptive are you? (The first one is the best)

This one is the best and is in the same vein as this one

Posted on December 8, 2008 in Uncategorized by adamNo Comments »

Inspired by this post, I wonder if there will evolve a market need for small collectives of very highly skilled, talented and disciplined software testers producing high quality information for discerning clients. The discerning part implies selectivity which further implies healthy bank accounts (at lease initially) which means I’m out of the running right now. It might be interesting otherwise.

The questions that fall out of that are

  • Who would you ask to join your collective?
  • And do you know them well enough that they might consider it?

Both of those are worth working towards a positive outcome.

Posted on December 8, 2008 in Books by adamNo Comments »

I think either code reviews or static analysis is where the next evolution of testing processes is going to happen. The main problem facing code reviews is that they typically require at least one senior programmer who has seen a lot of code. This is largely because our schools focus on producing people who are more concerned about whether the code works at time of grading rather than on how nice the actual code is. Lets not even think about the long-term maintainability of it. And of course there are those of us who learned by hacking things together as teenagers; we’re a hopeless bunch. Robert C. Martin’s (Uncle Bob) Clean Code could very well be a large part of the solution.

Here’s a quote from the end of chapter 1 which summarizes things nicely: Books on art don’t promise to make you an artist. All they can do is give you some tools, techniques, and thought processes that other artists have used. Uncle Bob and a number of his friends start right at the beginning, Meaningful Names, and start building from there through Functions, Comments, Objects, etc. Each subject gets its own chapter which is then packed with lots of tidbit on the subject with subtopics getting slightly over a page on average. In my copy it is rare that one of these subtopics does not have at least one thing underlined.

One of the strongest parts of the the book are chapters 14 – 16 which are almost like peering over his shoulder as he refactors code. It is one thing to just read a couple paragraphs on something, but another completely to see it put in practice. The next chapter potentially tops those ones by providing a nice list of Smells and Heuristics that you could take with your into a code review or to build a coding standard for your organization.

Perhaps it is my paranoid, Hobbes-ian bent, but I fear this book might also cause just as much damage as it prevents. Too many people I fear will take what is presented as gospel since the authors are well known in the programming world and if they say it then it must be true (in all situations). Having seen a number of codebases, applying all the heuristics that are shown without the appropriate refactoring would result in a collection of source files that are anything but clean and easy to read.

  • Comments are always failures (page 54) – Well, let’s just remove all the comments and / or stop creating them. Too often the code I encounter would be made clearer with a generous helping of comments; or a major refactoring to make them unnecessary. But please don’t do one without the other.
  • Small! (page 34) – In the discussion around function length he retells a program written by Kent Beck where every function was just two, or three, or four lines long. … That’s how short your functions should be. Yes, it is the context of a specific example program, but how many people are going to see that and refactor their nine line function into three three line ones? Later in

Steps were taken to say that everything shown needs to be taken in the context the reader is operating in, including undoing a refactor since it made the code less fluid. I’m not sure it was expressed explicitly enough or often enough.

My only other complaint, and it is a minor one, is the inclusion of sixty pages of source code in the appendix. This sort of thing would be ideal to host online on the book’s website. That alone might have been able to reduce the code of the book a couple dollars and reduced the paper required.

Overall, I think this is a great book which would be at home in a university level course and in the real world either creating or evaluating code. This one is definitely going onto my desk at work. You can get your team’s copy of Clean Code: A Handbook of Agile Software Craftsmanship from this link. Just remember not to forget your brain when applying the techniques to your code.

Posted on December 7, 2008 in Uncategorized by adamNo Comments »

This cartoon cracks me up, but completely captures the thought process of crawling through nasty code. To borrow heavily from that cartoon I propose a new metric: WTFAIDH / x (or What The Freak Am I Doing Here per x (where x is some period of time dependent on the situation)).

WTFAIDH / x is an indicator of the need for more learning or understanding and perhaps being a bit too far out of your league. Here are some situations where you might find yourself thinking this phrase:

  • Standing at top of a ski hill
  • Playing a sport
  • Speaking in public
  • Leading a release
  • Testing a completely unfamiliar type of product

Of course, this could also be an indicator that you have now gone beyond your safety zone and are growing as a person. 4 years ago when I started playing lacrosse I asked myself that the entire game but it occurred to me the other day that I don’t ask myself it anymore. Similarly when making long-term strategy decisions, but now I know to trust myself and the number has decreased. But wow do I still get a high number when speaking at conferences. I’m actively working on decreasing that amount though.

« Previous PageNext Page »