Q: When is a button not a button?

A: When it is a YUI button

Selenium, at its core, deals with HTML things stored in the DOM. And does a pretty good job at it. But things start to get confusing when the people creating the page start to get fancy and use third-party widgets like those provided by YUI. This post is about how to fake your way through Se automation when you encounter YUI controls, specifically a YUI Button.

The big thing to understand is that YUI buttons are not buttons (to the browser), they are in fact just blobs of JS that happen to look like a button. When you ‘click’ on one, a callback is fired to execute another bit of JS to make it look like you click did something. Ahhh, AJAXy goodness.

But because there is nothing happening at the DOM level, we need to simulate both the ‘click’ and the subsequent action. For example, here is a helper function I wrote to deal with a ‘checkbox’ type YUI button for a client.

def enable_bubbles(se):
    if se.is_element_present("//span[@id='PinBalloonSelector']"):
        se.get_eval("window.ShowBalloonButton.set('checked', true)")
        se.get_eval("window.PersistBalloonPinMode()")
 
    se.wait_for_condition("selenium.isElementPresent(\"//div[@id='balloonContentDiv']/div\")", "60000")

Important things to note:

• YUI buttons are children of the window object so start your hunt there
• To ‘click’ the box on you set the attribute ‘checked’ to true. Setting to false ‘unclicks’ it
• After the click you need to call the JS function that would normally be triggered as we’re sneaking operations in under-the-hood
• Because all of this is happening in JS, you need to use the get_eval variant that your binding provides

That took me well over a day to figure out, so I figure someone else has likely run into issues where Se-IDE has recorded a button interaction one way but on playback or export it didn’t work because the button isn’t an HTML button, it is a YUI button.

So who wants to write a plugin for Se-IDE that deals with YUI controls properly? 🙂

I’ve been bandying around the idea of what I would like to attend in terms of a Se event, but figured I would see how out of touch I am with others through a quick survey.

As of this morning there were 45 responses to it, which is in itself pretty telling since I announced it on…

• Twitter and between my followers who use Se and the RT’s it got hit an audience of I would guess ~ 150 – 200
• the selenium-users mailing list which has 1736 members
• the selenium-developers mailing list which has 308 members
• IRC which usually sits between 30 – 50 these days

Overall an overwhelming lack of interest to the survey was shown. And yet, meetups like the San Francisco Selenium Meetup get ~ 100 people once a month so there appears to be a market for geeking out over Se.

The first question was about location, and to be honest, I have no idea how to interpret this other than What’s wrong with Toronto?.



Well, it is not really a shock that Silicon Valley got the most votes given the community legwork Sauce Labs has done already. The high Maybe response to Chicago is a positive though (as I like it a lot better than the valley). But Chicago in November is likely not the easiest sell. Maybe we should have it in Cozumel; Sun, Sand, Selenium!

The next question was on length. The format wasn’t an option (single track) just the length.



Useful, but I think 2 days content + 1 day tutorial is right answer as it is long enough to provide a mix of content and be able to justify travel. A one-day conference, that isn’t intended to just be a regional one, is just an extended meetup.

The third question confirmed a lot of what I thought it would, and means I’ll I have to convince a bunch of the Se2 kids to come speak.



The trick, as with any one-track one is to make the speakers diverse enough to not bore the entire audience all the time on topic they don’t understand or don’t care about.

The last non-free form question was the whole cost thing which was interesting, if not really useful information. Conference pricing is often done on a backwards basis in that you get all your fixed costs figured out, subtract your sponsorships (paid and/or expected) and then you arrive at a number. And then pray you don’t lose your shirt.



I suspect if I put down $75 as the lowest amount, it would have been the most. I’m guessing that the $200 roughly maps to the ‘1 day’ length of question 2, so doubling that to 2 days is $400 which is close to $500 and combined have the overwhelming total (all but one) then things are in agreement with what I had hoped for. SeConf would not be a huge production like Star and so wouldn’t need to charge as much. I would actually consider it a failure if we had to charge more than $500 for the conference. Tutorials, as is the common model, would be extra.

The final question was just a text box which resulted in lots of how come you didn’t pick my city? type responses.

I’ve still not wavered from the idea that there should be a dedicated Selenium Conference this year. Though we’re getting perilously close to being too late to have one this year given lead times on facilities and speaker travel planning. Especially if I am the driving force / organizer of it. I know exactly when and where the 2011 should be, but thats getting the cart in front of the horse somewhat. I may just have it in Toronto, survey results be damned; as someone pointed out yesterday, the trick is to have it happen this year. We can make it a huge success next year.

In his book, Shop Class as Soulcraft, Matthew Crawford makes the argument that the trades, the so-called blue collar occupations, have been intentionally sidelined and belittled by North American society. And we are worse for it. I picked it up at the bookstore on a whim since the cover had a motorcycle on it and had ‘craft’ in the title; two things I am currently interested in. While reading it I felt the familiar sense of deja-vu as it relates to the testing field right now and we think there is lots to be applied from the plight of electricians and plumbers in terms of our field.

A System of Rules

Testing is, at its heart, the application of a series of rules. True testers realize that this, but also know that the set of rules is ever changing based on a myriad of factors. Compare the professional tester’s mental process to that of a repairman using the following quotes from Chapter 1

• The repairman has to begin each job by getting outside his own head and noticing things; he has to look carefully and listen to the ailing machine
• Knowing what kind of problem you have on hand means knowing what features of the situation can be ignored
• A realistic solution must include ad hoc constraints known only through practice
• Creativity is knowing what to do when the rules run out or there are no rules in the first place

All of these illustrate the importance of practice, knowledge and desire to do the job effectively. To treat it as a Craftsman would any other task. Unfortunately the number of people who treat testing as a Craft is small, though vocal. The testing world is experiencing something similar to what highschool education did in the 90s with the elimination of shop classes in place of computer labs. What schools turn out now are people who know the ‘why’ of things rather than the ‘how’. Or as stated by the executive director of the California Agricultural Teachers’ Association

We have a generation of students that can answer questions on standardized tests, know factoids, but they can’t do anything

If we swap ‘testers’ for ‘students’ in that statement you have a very generalized view of what is happening in the Standardization and Certification camps. That’s great that you know the definition of a testing task, but do you know how to do it, when to do it, and more importantly, when not to do it?

The focus on standardization in the testing world is also partly the cause of the mass offshoring of testing in recent years. If a task can be broken into a series of finite, distinct rules, then the task is able to be delivered globally. Not only that, but according to MIT economist Frank Levy, there is a downward pressure on wages for jobs based on rules.

The way forward then is to treat Testing as Craft, to be delivered by people who take pride in the application of learned skill and knowledge and not as an assembly-line like task where the testers are easily replaceable cogs.

Old bikes don’t flatter you, they educate you

Crawford is by day a motorcycle repair person and is constantly confronted by new problems, scenarios and indeed contexts. He says quite clearly that “Old bikes don’t flatter you, they educate you” and then to show that discovery is not limited only to bike maintenance he quotes Iris Murdoch about learning a new language

My work is a progressive revelation of something which exists independently of me.

Software systems educate us daily, if not hourly as they reveal themselves to our inquiries. How effectively they educate us is largely dependent on how well we prepare ourselves for the eduction. Philosopher Albert Borgmann has the idea of Focal Practice which is the decided, regular and normally communal devotion to a focal thing.

Focal Practice has existed in pockets of the Testing Community for awhile though under many different names. Testing Games, Katas, Dojos and now the Weekend Testing events are all examples of Focal Practice. And seeking them out and participation in are signs of someone who wants to master the craft of testing.

A Series of Masters

In the trades, the accepted way of learning is to apprentice at the side of a Master. They offer a model to follow and pass along their accumulated knowledge. Crawford recounts his education as a ‘gearhead’ in his his teens and early twenties; all of which was informal and outside of the traditional school system. And was utterly important towards his education of the craft.

Now I saw it. Countless times since that day, a more experienced mechanic has pointed out to me something that was right in front of my face, but which I lacked the knowledge to see. It is an uncanny experience; the raw sensual data reaching my eye before and after are the same, but without the pertinent framework of meaning, the features in question are invisible. Once they have been pointed out, it seems impossible that I should not have seen them before.

The testing craft needs to develop more of these situations where the awareness lights turn on, never to turn off again. Not only through the treatment of testing as a craft, but through continued analysis of ourselves as it relates to the problem at hand. One of the big evolutions the Agile movement brought was the periodic reflection of how the process was working at a team level through retrospectives. Crawford speaks on the singular with metacognition which is thinking about your thinking. Or as he describes it

It is what you do when you top for a moment in your pursuit of a solution, and wonder whether your understanding of the problem is adequate.

Is my context [still] correct? Am I on mission? It is the mission I think I was on before?

Another important role of the Master is to allow for learning opportunities through safe failure. Failure is something that has been removed from modern learning. Especially education based on books and theoretical discussion than from hands-on doing. Would you want a plumber redoing your bathroom who has worked under someone with years of experience or who memorized a book and wrote a multiple choice test? Now think about whose hands your software is in.

Tacit Knowledge

So what makes someone a Master? I would think it has a large part to do with the accumulation ,and ability to recall, a substantial body of Tacit Knowledge. This is what Blink by Malcolm Gladwell focuses on and its inclusion in another work lends more credence to the idea.

Just as firefighters will often leave a building just before it collapses — not because it looks like it will collapse, but because it doesn’t look ‘normal’ based upon their years of experience, a Tester can find bugs not because they are obvious, but because something doesn’t look right on the screen in front of them.

Just having the tacit knowledge is not enough though. They have to be able, and willing, to pass along that knowledge which has to be done on a human-to-human basis as it is too complex to capture algorithmically.

‘Manual’ Testers

It is trendy to pour disdain on both people who practice manual testing and indeed the very notion of it much along the way the trade of plumbing has been dumbed down to the image of a slovenly character with their butt hanging out of the back of their ill fitting pants. The trades used to be something to aspire to and commanded respect, now they are the subject of jokes and ridicule. Is Testing destined to go down that same route? I hope not. But if it is to escape such a fate we need to stand up against bad testing education and practices and bring the pride and ethic to it that a Craftsman brings to their work.

Impressions of Shop Class as Soulcraft

This is not a testing book. Instead it is a reverent look at a way of life that is under attack in a systemic manner. Though sometimes preachy, if you are interested in reading about someone’s journey from being a kid who liked cars that went fast to someone who fixes motorcycles for a living and how that can relate to testing then this book is a nice quick read for you. I’m going to say now that it won’t be one of the most important books for testers of the year, but it opened my eyes to a couple new concepts and reinforced some others and for that it was well worth the money.

So in Se there is the notion of a Assert and a Verify. To recap, when an Assert fails, then the script stops immediately and goes to the teardown function. A Verify will ultimately fail the script, but the test will continue.

In Se-IDE the commands make the syntax very obvious but it is less so in Se-RC. Here is a python example to illustrate how to use both.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import unittest
 
class assertExample(unittest.TestCase):
 
    def setUp(self):
        self.verificationErrors = []
        self.selenium = selenium("localhost", 4444, "*chrome", "http://you.site/")
        self.selenium.start()
        self.open("/")
        self.wait_for_page_to_load("10000")
 
    def testSomething(self):
      se = self.selenium
      se.assertEqual("Login", se.get_title())
      try:
          self.assertEqual("Please login", se.is_element_present("welcome_message"))
      except AssertionError, e:
          self.verificationErrors.append(str(e))
 
    def tearDown(self):
        self.selenium.stop()
        self.assertEqual([], self.verificationErrors)
 
if __name__ == "__main__":
    unittest.main()

First let’s look at the Assert. In line 14 of the above example I’m using a common pattern of Asserting upon page load that I am on the correct page. If the title is not exactly Login, then the script will end and the tearDown function will execute. The rationale here is that if the script has taken you to a page that you are not expecting, then the rest of the script’s results will be suspect — if able to continue at all.

Lines 15 – 18 are the Verify. Notice how it still is based on an Assert provided by unittest, but we are catching the AssertionError that gets thrown on failure. This prevents the script from running and allows us to track the failure to the list that was created on line 6. The list contents are what will determine whether a test passes (empty) or fails (non-empty) which is done in the tearDown (line 22).

Is this the only place to use an Assert or Verify? Of course not, its just a common one. I also use Asserts when something is critical to the purpose of the script (the pass / fail criteria) or when something is worthy of extra special attention. The trick is to use both in a way that makes sense for your application and scripts.

Tonight was the May San Francisco Selenium Meetup and since I was in town I put on my combination Selenium-IDE Maintainer and Sauce Labs hat to schmooze.

The main even featured two presentations, the first by two Mozilla automation folks who talked about their testing rig. Some takeaways…

• They mentioned they use custom firefox profiles for their tests to get around self-signed certificate problems. This is one of my great Se pet peeves. DON’T USE SELF-SIGNED CERTIFICATES IN TEST! Actually, unless you are using certificate based authentication, don’t turn on SSL at all in test. And if you, really, really, really think you need SSL in test (you don’t…) then use a real, issued by a trusted CA certificate. Oh, and custom firefox profiles are a good idea in general anyways as it gives you complete control over the environment (prefs.js is your friend)
• The coolest thing about their whole rig was that it was controlled by a bot you interact with on irc. Outstanding!
• They also created a central module (they use Python) which had all their object locators in it. Not really Element-UI and not really GUI-Map, but it seems to work. Se itself should think about how to support this pattern in a clean, well-documented manner…
• XPath as a pretty deserving reputation for being slow as a locator strategy, but seeing XPath in a script is not a bug. XPath is ridiculously powerful. Bad XPath is a bug, but bad anything is a bug. Good, robust XPath is a good way to access an object.

After Mozilla was Linked-In’s turn to show off their Page Object based testing framework. I also have some notes / opinions on it too.

• Their setup method had @env = Environment.new which is a nice object to be carrying around. I don’t know what was in Environment, but I suspect I’ll be stealing the concept. Or re-stealing I guess as I did something similar 4 or 5 years ago in an old framework but stopped doing it for some reason.
• Using the link locator strategy fails when testing across locales. Duh! Why didn’t I realize that before now?
• The whole framework seemed over-engineered. This was only the send Page Object one I had seen, but that seems to be a pattern. Abstraction is absolutely necessary for success, but to what end?
• What was cool as a result of all this effort was there was no direct Selenium commands in the scripts.
• Jason [Huggins] did some like code review during the talk which provided another key thing to remember when writing scripts and that is to let specific tools do what is better when appropriate. For instance, rather than looping through a list of XPath elements in Selenium (slow), hand off the page source to Beautiful Soup or Nokogiri whose sole purpose of existing is to do that stuff (fast)

One final bit of trivia. The meetup was held at the Mozilla (as in makers of Firefox browser) offices in Mountain View. That same building housed the first offices of Netscape — which of course died a slow painful death and whose source code formed the basis of Firefox. What goes around comes around…

This week’s Writing Excuses is on Discovery Writing. I’m a discovery writer so it was important on that level, but one thing stood out that was important for testers too.

The 1/4 mark is the one to really worry about because that is where the freedom starts to disappear because you have written so much and you realize that everything else i write from here on out has to take into account all the stuff i have already written. I can’t destroy the setting and the characters I have already done…

Do we need to coin the phrase Discovery Testing? I’m not sure what the full definition would be, but I suspect it would differ from Exploratory testing in how much detail and discipline is used when following up items. And perhaps the lack of a charter as illustrated by this quote that was used on the podcast:

Writing serialized fiction is like jumping out of an airplane with a needle and thread and hoping you’ll have sewn a parachute before you hit the ground – Neil Gaiman

Swap out ‘Writing serialized fiction’ with ‘Discovery testing’ and I think you have the idea.

MX Performance Magazine has an article in the May, 2010 edition comparing the Canadian Tire Nascar Series, the Parts Canada Superbike Series and CMRC Motocross. The thing that struck me from it was not the actual series differences (which is outwardly pretty obvious), but the why the pro teams do it.

• Nascar – act as a feeder series to the main Nascar series
• Superbike – says one insider – ‘These sports are stupidly expensive at this level, and we do what we do strictly for the tertiary benefits’
• Motocross – to move product for the sponsors; what wins on Sunday sells on Monday (incidentally, my bike will be orange for that reason)

Understand why a team exists will put all of its activities in to their proper scope and priority. Misunderstanding it will result in (potentially lots) of wasted money.

The same thing with testing. If you don’t know the why of what you doing, then you are not testing, you are just flipping switches.

I’ve had the two ideas contained in this post kicking about for a bit and in James’ most recent blog he says

Next week it will be exactly 23 years since I first became a full-time software tester, and nearly 17 years since the first time I stood up at a conference and pointed out the absurdity of “traditional” testing methods.

which is a fantastic setup for both of them.

And that is, while I’ve been a full-time software tester for ~ 12 years I became a professional software tester at some point during Monday, August 29, 2005. That day is notable not only for hurricane Katrina ripping through New Orleans, but because that is when took RST (and met James as a result). So everything I have accomplished or done since then can be traced to those three days.

It was this eye-opening that I heard from several people at Star which inspired them to be a better tester but start writing and talking about testing. But it also goes further than that as people have told me that they talk and write about testing because I write and talk about it.

This network effect is extremely powerful, but it turns out that while I blame James for what I have become, it doesn’t start with him. I would guess that it actually goes back to Jerry [Weinberg]. So in effect I should blame Jerry, but since I have only met him once I’ll stick to blaming James.

Now in no means is James the only one to directly blame, though he is the catalyst, as Michael [Bolton] has directly done a lot to shape the outcome. But, to me, let’s call them ‘primary influences’. And let’s say that I am a primary influence to someone else, and them in turn to someone else.

Let’s also accept and acknowledge that influence is a two-way street which means people further away from the origin can still influence at it. See James’ Star keynote with its 5 minute discussion of Lanette as Exhibit A of this.

Confused yet?

Here’s a video.



The video is talking about the dimensions of space and time, but it works well-ish for influence also.

Note: blame throughout this post is used very tongue-in-cheek.

Canadian Business magazine had their Investor 500 issue recently which they bill as ‘a guide to the market’. The Editor’s Letter tackles head on the notion of its continued relevance in today’s information age.

…we are not in the information business, we’re in the insight business. Information is the 10-ton slab of coal. Cheap, ubiquitous and useful only to a point. Insight is the rough diamond trapped somewhere inside. We’re living in a world that is inundated with coal, which only serves to reinforce the value of the diamonds.

‘Information’ is one of the words I pick up on when going through articles since, as testers, our job is to provide quality related information. But is that really a good enough definition in light of the coal/diamond metaphor? I’m tempted to lean towards no and that we should be attempting to provide quality related insight to the stakeholders instead.

Anyone can provide information (bugs, crashes, feelings), but it takes training, dedication and experience to be able to provide insight.

Back in Part 4 of this series, Extending the Selenium API I showed how to use addPluginProvidedUserExtension method to add new commands to Se-IDE. As a result of some real-world usage, a bit of a bug (well, more of an unimplemented feature technically) was discovered. And that was the exporting of the script to one of the supported RC languages. Something that has now been solved.

So what has changed? Well, if your added functionality applies only to Se-IDE (such as random, or goto or anything else done better in the RC language) then nothing; you still only provide the URL to your JS as the first, and only argument. If you want it exported though, there is an optional second argument to specify.

<?xml version="1.0"?>
<?xml-stylesheet href="chrome://global/skin/" type="text/css"?>
 
<overlay id="preflight_extension_loader_overlay"
         xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul"
         xmlns:html="http://www.w3.org/1999/xhtml">
    <script type="application/x-javascript" src="chrome://selenium-ide/content/api.js"/>
    <script type="application/x-javascript" src="chrome://preflight/content/preflight.js"/>
    <html:script type="application/javascript">
        ide_api = new API();
        ide_api.addPluginProvidedUserExtension("chrome://preflight/content/extensions/extension-random.js", "chrome://preflight/content/extensions/extension-random.xml");
    </html:script>
</overlay>

Yes, sorry. That is an XML file that needs to be provided. I could have completely rewritten things to not take XML, but the existing code expected it and working is better than pretty more often than not. Here is the format of that XML.

<apidoc>
  <function name="randomString">
    <param name="options">type and length separated by a pipe. type can be alpha, number or alphanumeric</a></param>
    <param name="var">variable to store random string</a></param>
    <comment>Generates a random string of specified type and length</comment>
  </function>
</apidoc>

Now the astute readers will notice that the information contained in that xml looks suspiciously like that which is is the Reference tab of the Se-IDE. And you would be correct. But that is also the source of another bug that has yet to be addressed: if you want to have your functionality appear in the reference box, it will also be exported via the Se-RC formatters.

This new functionality will be available in the version 1.0.7 of Se-IDE which should be out in the next week.