Posted on February 26, 2010 in Uncategorized by adamNo Comments »

I’m working on automating up a client’s AJAXy application right now and managed to get myself very confused frustrated yesterday over retrieving a value that was updated through a JS event.

For context, there are a bunch of fields that can have numbers entered into them and on each key press a running total is updated via Javascript and replaced in the DOM.

Because the field that gets updated is an input one, the content of it is stored in the value attribute.

This is a RoR application, so I’m using the selenium-client gem. To get an attribute named ‘value’ using it you would do the following.

@selenium.get_attribute "//input[@id='currency_count_total_value']@value"

And that works fine — until it is updated through Javascript. On the page the field will be updated but that snippet will always return the original value since the page hasn’t actually been updated — just the DOM.

So how do you get the updated value? Well, after an embarrassing length of time the magic is to use the get_value method.

@selenium.get_value "currency_count_total_value"

Now, in theory they should do the same darn thing. Reality is different. Yesterday evening would seem to imply that get_attribute works on the most recently updated html retrieved from the server whereas get_value works directly on the DOM.

Posted on February 26, 2010 in Uncategorized by adamNo Comments »

At some point when I wasn’t paying attention Selenium grew the ability to take screenshots. The two commands of interest are captureScreenshot which takes an image of what is viewable in the the browser and captureEntirePageScreenshot which captures the entire content of the browser; visible or not.

Screenshots and automation have a dubious history. Too often they get used to determine error conditions which leads to ‘the test failed because the pixel was off by one even through it actually passed functionality wise’. That is exactly opposite to how I would recommend you use this capability.

Instead I think you should incorporate it two ways.

  1. Ack! Something went wrong! – Any time an assertion fails, take a capture of what is being displayed on the screen. This could save you the time of running back through scripts to get to the error point and if it is an intermittent fail you can capture those states.
  2. As an aid for humans – The ability to drive different browsers is one of the great selling points of Selenium. But any tool like this is great for functionality, but really bad for layout and presentation problems. By wonderful coincidence though, humans are great at it. So why not make a script to capture each page in your application and stash it somewhere for a human to look at later?

This doesn’t mean that people have not been successful doing screenshot comparisons before. At CAST 2007 in Seattle, Harry Robinson demonstrated how Google Maps used image comparison when figuring out the driving directions algorithm. And at GTAC 2009 there was a presentation on Fighting Layout Bugs – Techniques to automatically verify the work of HTML and CSS programmers which Jeff Fry summarizes quite nicely. Here is the recording of that session.



So yes, it is possible to do automation using screenshots to determine pass/fail conditions. But for most projects they infrastructure and time to set it up is not worth the return. Instead, use them as tools to assist in figuring out what went wrong in a script or to capture information for later human analysis.

Posted on February 24, 2010 in Uncategorized by adamNo Comments »
  • Selenium RC 1.0.3 is released
  • Selenium IDE 1.0.5 is released!
  • Mogo is making web testing easier; invitation alpha now on
  • Step six of Oracle ADF Development Essentials is Testing Your Oracle ADF Faces Application with Selenium. I would have switched it for step seven which is CI, but whatever.
  • FlexMonkium is a FlexMonkey -> Selenium bridge from the authors of FlexMonkey. I’m not sure how they are getting around the need to have an enterprise license for recording, but I’ll be talking to them over the next day or two.
  • Learning Python is not about Selenium, but about why learning a scripting language has helped Lanette Creamer do her job better and includes ‘Since I’ve been working more with Python I can now understand Java better! I can read other people’s code and follow it more often. I’m pleased that I can get test ideas from what I think is missing in the code, not just what I see that is there.‘. If you are using Selenium, you should know a scripting language.
  • While not new, 10 Tips for Building Selenium Integration Tests saved me some time last weekend.
  • If you are using Rails and Sauce Labs OnDemand, the folks at Pivotal Labs have written a saucelabs-adapter
  • Chris Bedford at the San Francisco Java User Group presents talking about Selenium.

  • 2010 CWE/SANS Top 25 Most Dangerous Programming Errors – do your scripts check for these?
  • Jason Huggins at the Denver Agile User Group talking about the Future of Automation

  • The next London Selenium Meetup is April 20 and will be on Selenium 2 — going to record it guys?
  • Announcing Selenesse, the modern Fitnesse -> Selenium bridge
  • Flow Control and WebDriver Se-IDE plugins
  • SeleniumHQ has a new blog
  • If you are using Se with Rails, you really owe it to yourself to use mechanize
  • The CoScripter Reusable History plugin was this week’s Sikuli. The only thing I could see steal, erm, borrow-able is the exporting to human friendly text part. Who is going to write that formatter for Se-IDE?
  • Google Summer of Code is on the horizon. What Se projects should we submit?
  • If you want the bleeding edge Se-IDE builds, you can get them out of the OpenQA Continuous Integration server. (Click the current build, then Artifacts)
Posted on February 23, 2010 in Uncategorized by adamNo Comments »

I’ve just old (young?) enough to say that I have spent the majority of my life online. I remember when search engines were ‘new’ and a time where Search Engine Optimization (SEO) was not even a thought. So I’ve seen SEO be born and enter its unruly teenage years and so far I can say pretty confidently that companies offering SEO services for high dollar per hour amounts are a scam.

SEO is however a reality. The internet is so vast that discoverability is getting harder and harder even if you write content that people want. And because it is a reality, it is yet another thing that testers need to keep in mind when testing web properties.

The Art of SEO is the new tome from four SEO ‘experts’. The fact that experts is in quotes highlights one of my main objections to any book on SEO: there are no experts and it is a constantly moving target. The rub is that none of the major search engines provide their full algorithms for search discoverability or indexing so how can anyone claim to be an expert in it? Also, these algorithms are constantly evolving — often in a direct response to things happening in the SEO realm. That said though, if you are going to have someone write a book on such a subject, these four seem like a good choice.

I’m not going to even try to hide the bias I have towards the SEO industry as a whole, so why on earth would I get a 575 page book on the subject? Simple. I wanted to make sure my ‘SEO is a scam’ claim was correct — and I maintain that it is. But that doesn’t mean you shouldn’t buy this book. While SEO itself is a scam, ‘The Art Of SEO’ is not one. Especially if you are new to the area.

There is nothing in this book that could not be found online, but the ironic value in the book is that you don’t have to search for it. If you test an internet based web application, here are thing things I think you need to know about SEO — all of which are covered in the book.

  • Spidering – How spiders work, how to control their access to your site, and how to make their job easier
  • Structure – There are ways to structure content to legitimately enhance its searchability without being scammy
  • Redirects – How to deal with moved or removed content in such a way as to inherit prior page value
  • Rich Media – Flash, images and audio all present challenges
  • Types of Search – Horizontal, Vertical and Local search all have different needs

What is also important is the couple pages about ‘Grey / Black Hat’ techniques. These are the tricks that raise your search ranking for a bit, but are often frowned upon by the search engines themselves and could get your delisted. The less that reputable SEO outsourcing companies often employ these techniques so people need to know what they are and stay away.

There are a few things I don’t agree with in the book (of course). Primarily, the notion that runs through the book is that SEO should be first and foremost when planning a website. That is suits the needs of the users almost seems secondary. Being a book on SEO though from people who make their living on SEO, this bias is understandable. I would counter that if you forget the user experience though then it doesn’t matter that people found your site as they won’t use it.

‘The Art Of SEO’ is not really a read-cover-to-cover book. Instead I would recommend picking a topic (like redirects), skimming the places where it is discussed and then see if your site needs tweaking. It is not exactly a heavy read so you should be able to get maximum value from it pretty quickly — I knew 97% of the content already and went through it in 2 swimming lessons and a snowboarding lesson.

Do I recommend ‘The Art of SEO’? That depends on your prior knowledge in the area. If you have been creating content online for awhile and have picked up how to do SEO naturally then probably not. But if you are new to testing web applications or management of them, then there is a decent amount of value in the book and it would be a good addition to your bookshelf. Especially when you compare the $56.99 Canadian price tag to what so called SEO experts charge to do stuff to your website that you are more than capable of doing for yourself.

Posted on February 23, 2010 in Uncategorized by adamNo Comments »

I’m a mentor for a class at U of T this semester where a group of students are trying to crack the ‘Flash in Selenium’ nut. They are at the point where they are having planning / status meetings and I had to be downtown anyways so attended this week’s.

The first task they had to do was to individually produce the Elevator Pitch for their project. They had 3 minutes to write the pitch and then they had to read them out loud. The discrepancies, and there were some are signs of divergences within the team. These divergences are warning signs that need to be paid attention to.

Immediately.

I advocate elevator pitches for individuals and their careers but had not seen them used in a project beyond the initial kick-off until now. I think this has officially been added to my retrospective repertoire.

Posted on February 19, 2010 in Uncategorized by adamNo Comments »

After talking to a number of people, Se-IDE is going to be growing a new pane in the Option panel to manage your plugins. Actually, it already has as of 1.0.5. In order for a plugin to utilize this new panel it has to register itself. Ideally this would be a mandatory thing, but overlays don’t allow that (so far as I know) which means it is up to the plugin developer to play nice.

Registering a plugin is easy. In one of your XUL files where you have an API object, just call the addPlugin method with the id of your plugin. This id is the one you specified in your install.rdf file.

<?xml version="1.0"?>
<?xml-stylesheet href="chrome://global/skin/" type="text/css"?>
 
<overlay id="preflight_extension_loader_overlay"
         xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"
         xmlns:html="http://www.w3.org/1999/xhtml">
    <script type="application/x-javascript" src="chrome://selenium-ide/content/api.js"/>
    <!-- <script type="application/x-javascript" src="chrome://preflight/content/preflight.js"/> -->
    <html:script type="application/javascript">
        var ide_api = new API();
        ide_api.addPlugin("preflight@adam.goucher");
    </html:script>
</overlay>

Eventually this new pane will display the plugin’s version/author information, options and even controlling whether it is enabled or disabled.

Posted on February 14, 2010 in Uncategorized by adam1 Comment »

So it has been just over 2 months since the last Smattering of Selenium and a lot has happened; not including starting my own Selenium consultancy / startup called Element 34 and inked a deal for a book on Selenium. Here are the browser tabs that I have had open for far too long that would be relevant to Selenium users.

Posted on February 12, 2010 in Uncategorized by adamNo Comments »

I’ve had this podcast in the queue for a long time. Privacy has become one of those things that testers need to always have at the back of their mind when testing. Understanding Privacy is a book by Dan Solove was on the podcast circuit promoting and had a number of excellent points around the subject in this interview.

  • People don’t understand the implications of giving up information
  • Because we don’t know how information might be used in the future, holders of the information doesn’t even know yet, making an informed decision is really hard
  • What does privacy even mean? Usually its a vague an amorphous concept
  • By contrast, the things that privacy is often traded off against, such as Security, are easy to define so they win out
  • Privacy isn’t one thing, but many things
  • Past attempts to define Privacy have been too broad (‘Right to be let alone’) or too narrow (‘intimate’ or ‘client confidentiality’)
  • Because of the diverse areas affected by Privacy, there is not just a single solution to it, but a basket of solutions for each problem
  • It is not about being paranoid, but about doing things with your eyes wide open
  • Unlike a car accident, the problems that Privacy issues cause might manifest many years down the road. The lack of immediacy is a key problem in education and recognition of risks
  • Most people think that because a company has a Privacy Policy it means that the company won’t share you information. Umm, no.
  • Don’t think that this isn’t just a Consumer problem. This is hard to manage and build out from a developer / implementation perspective.
  • There are four broad categories of problems around Privacy. The idea is that with such a taxonomy you can understand which problems (and there is often not just one) that need to be addressed.
    • Information Collection
      • Surveillance – when it inhibits personal information or used in an unrelated manner
      • Interrogation
    • Information Processing
      • Identification – linking people to information streams (think Gattaca)
      • Aggregation – combining bits of information to make something that is greater than the individual parts
      • Security – is the information kept securely?
      • Secondary Use – when data gathered for one purpose is used for others
      • Exclusion – denial of information regarding how data is used
    • Information Dissemination
      • Disclosure – spreading of facts
      • Breach of Confidentiality – betrayal of trust
      • Exposure – not reputation; but deep embarrassment
      • Accessibility of information – obscure data being made less obscure
      • Blackmail
      • Appropriation
      • Distortion
    • Invasion
      • Intrusion – spam for instance
      • Decisional Interference
Posted on February 10, 2010 in Uncategorized by adamNo Comments »

Fortune has their annual list of the 100 Best Companies To Work For out now. What I think is more important that where companies ended up, or their trending in the ranking is what the blurb that accompanies each entry has. The reason? That is the ‘why’ of the choice. The why gives insight into the culture, the very fabric of the company. Having worked at a number of organizations, a toxic culture can be no good at all.

The big patterns this year are: caring about the people who work for you, investing in people, reward community involvement and low turnover.

Here are the ‘sound bites’ I lifted from the list.

  • Trust between our employees and the company
  • Salaries were frozen, but profit sharing continued
  • 11% of the workforce, have been here more than 15 years
  • Culture of collaboration and openness
  • Good people can do their best work
  • “No asshole” rule
  • “If you tell anybody what to do here, they’ll never work for you again.”
  • On-site child care
  • Your time off is your time
  • No wonder full-time turnover rate is just 3%
  • Always “do what is right.”
  • Mentors
  • More than 1,000 new employees were added in the past two years, 40% from referrals
  • Culture of openness
  • Employee referrals account for 85% of new hires
  • Restocked its vending machines with healthy snacks and nuts, and employees get checks ranging from $100 to $300 for trying to get fit
  • Treats me as a peer
  • Fourteen “junior boards of directors” give employees at all levels input into decision-making
  • Employees who volunteer are rewarded with vacation
  • The most generous sabbatical program in American business
  • “No job is seen as better than another job, and no person is seen as better than another person.”
Posted on February 7, 2010 in Uncategorized by adam3 Comments »

For the last couple years there has been a growing collection of people rallying around the software craftmanship. From an outside perspective, it appears that they are dipping into the classic apprentice – journeyman – master model for growth (without the inclusion of guilds which is often the point of fear with this model). From its title, Apprenticeship Patterns – Guidance for the Aspiring Software Craftsman would be aimed only a the beginning of the road and only for developers. Both would be wrong. This book is of value to anyone in a knowledge role and at any point in their development.

Dave Hoover and Adewall (Ade) Oshineye have collected together a series of of almost 40 ‘patterns’ that you can use, and likely already are, to assist your growth. Each pattern follows the same structure by defining the context, giving a sample scenario where the pattern might solve a particular type of problem, a solution and action that the reader can take immediately to start implementing the pattern. Because of this structure it is easy to jump at random throughout the book and [start to] digest a pattern in a single (brief) sitting. The real power of the patterns for people starting out is in the Action section I think, though I admit that I have not followed through on most. These are the ‘get off you butt and do something about your growth’ motivators.

Having embarked on my path already I found that I was already implementing a number of these patterns which I think completely validates their inclusion in the book. But it also meant that the Solution section was what got my attention the most with lots of ideas how to refine my implementation of the patterns. Flipping through my copy of Apprenticeship Patterns I can find only 2 or 3 patterns that I did not underline or star or otherwise mark up.

My favourite patterns (both from the book and the real world)?

  • Breakable Toys – Experience is built upon failure as much as (if not more than) success
  • Find Mentors and Kindred Spirits and Rubbing Elbows – Seek out those who have gone ahead of you and strive to learn from them
  • Record What You Learn and Share What You Learn – Start writing. Start writing now!

People who know me likely won’t be that surprised with that list. Your list, almost by definition will be different as you have walked a different (though perhaps similar) road in your journey.

My only knock against the book is likely its price. It is 130 pages and $37.99 CDN. That is steep, though I suppose in-line with current pricing structures since Beautiful Testing is 320 pages for $62.99. To their credit, the patterns could at one point be all found online (minus some polish that comes from the editing process) but I couldn’t find them at time of writing. But a book like this really should be consumed in dead-tree format where you can make notes in the margins. (If someone finds the wiki link again, please post it.) And at the risk of sounding like I’m pimping my book, Amazon has been long running a deal where you can save if you buy Apprenticeship Patterns and Beautiful Testing together. While likely related to release dates and ish similar breadth of content, I’m ridiculously happy to see it.

The Agile community is in the midst of an obsession of Shu Ha Ri, and much to my annoyance I admit that there are different ways that people learn at different stages of their learning. I don’t know if there is going to be ‘Journeyman Patterns’ follow-up, but I hope there is. But even if there is, Apprenticeship Patterns is staying on the bookshelf where it is nice and handy.

Next Page »