SMART Objectives with the S before the T

Hello all and welcome to my blog!

I will start by introducing myself briefly and then proceed to present an age-old concept that saved my day quite a few times... Who knows, maybe you could put it to good use very soon?

So, first things first: my name is François Perron. Yes, there is a funny cedilla on the 'c'... Just call me FX.

I have been involved in system integration testing, design and management since the mid nineties. I have previously been disassembling clockwatches, motors, guitar amps and your favorite toys since forever. I like to think I am pretty good at it now.

This blog is originally intended as a reflexion on some of the lessons learned and the general tribulations of an electronics enthusiast, lab rat, engineer and consultant. It is also somewhat of an experiment in social interaction, since I expect to stirr-up the conversation with you and try to figure-out ways to captivate your interest! So if you want to suggest topic material or ask questions, feel very free to leave your comments (email info@cydone.com, use subject FX_BLOG for those who are not logged-in).

So, that's it, the word is out: we will talk about testing!!!

Now, on to the SMART objectives. Why on earth would you start a blog by presenting the concept of SMART objectives? Besides, this sounds like management stuff... right? Well, yes and no. Ok, ok. Make that a plain yes then... SMART objectives have many definitions, many presentations and even many inventors. The most recognized form, credited to Peter Druker (well, according to wikipedia anyway...), is around since 1954. Before the C language, much before the VIC-20 but certainly after the 12AX7 tube. But I digress.

SMART is an acronym for:

• Specific
• Measurable
• Attainable
• Relevent
• Time-Bound

So a SMART objective is an objective that is described by these 5 adjectives. I read this article that presents 10 steps to SMART objectives. It is a very good read and will save me from having to present the definition of these words myself. Besides, the original author synthesised the information much better than I would have. Just promise me to come back here after you read it... the punch is coming...

Now that you have those 10 steps in mind, let's hack them a bit. In fact, let's forget about all of them. Except number 3. MA/RST. That is SMART but with the S just before you talk about time.

And now suddenly, we are talking about testing. Boom. Bet ya did'nt see it coming, eh? MA/RST is one of the most important test case planning tool I have in my arsenal. MA/RST ('Ma Reset for the initiated) is a funny mind trick. It is a personnal mantra. "MA in the setup of the test case, RST as a sanity check".

Where are we going with this, you ask? Well here it is:

• Do you feel the pressure of writing good test cases, all the time? Not always easy, huh?
• Do you feel testing is an important part of the complete project, yet you end-up leaving entire parts of the big picture without any coverage?
• Do you find yourself looking around for ideas about what to test and finally end-up doing the ones that are easiest first until you run out of project time?

Of course some of these issues should be addressed within a much larger scope, and by no means do I imply that the small topic of MA/RST would be the first or the most important thing to discuss if one wanted to solve them all for good. We will definitely come back to that in one form or an other in the posts to come. But, in the meantime, I wanted to present a trick, a tactic, that can have a very large impact on your testing activities, yet remain simple to remember... 'Ma Reset should do the trick !!!

So here is how it works:

While planning the test cases, use MA/RST as a mantra to check your progress. It is a bit like a test case for your test cases. Let's see how to apply it. First we need an assignement. We all, at some point, develop a 6th sense for identifying what to test by performing some of these tasks:

• Glancing at specifications
• Looking at / playing with a prototype
• Receiveing verbal instructions (hopefully better than "check this out" or "make this work!")
• Discussing with your working mates your next project over a foosball table

Or perhaps, in the best case, we are executing the test plan that we have been carefully planning from the ground-up with our team (we will get to that in future posts, I promise!). In all cases, we are there: we have something to test that has to be described for ourselves first to organise our thoughts. In the best world, again, maybe these thoughts will be recorded in a written description under revision control (in a script header, in a test case management tool, in an annex of a test plan, YMMV) or shouted over a desk to another team member (yuk, you are late with your project, are you?).

The point is plan for MA, check with RST.

MA goes like this in my head: Mhuh, what do I have to measure? This brings questions about Measurability and Achievability:

• How can I measure this?
• What are the specific bounds for an acceptable pass condition?
• What could go wrong?
• What kind of precision/accuracy do I need?
• Is it possible (Attainable) to measure this?
• Will the result bring significant insight?
• Will the test be repeatable?
• Will the SUT (System Under Test) need set-up/tear-down activities before/after the test?

MA involves, perhaps, reading a requirement and having to interpret it to bring a measurement out. It involves looking for a possible way of performing this measurement, to verify this condition. With the experience accumulating, other thoughts emerge: can I recycle an old test and modify it for what I have to do now? After the MA, I have a clear idea of what I have to do. Perhaps the means are not very clear just yet, let alone even realistic within the work frame I currently am in.

Time for a check!

RST goes like this: Reset! Can I really do this? This brings another set of questions about Realism, Specificity and Time

• Do I have the equipement to perform this test?
• Do I have the time?
• Will it clog the test database, the tools, the lab, the network?
• Will it cost too much?
• Can I perform the setup myself realistically? If not who/what will I need?
• Can I combine this test with other tests, or is it so specific that it has to be run be itself?
• Does it bring-out enough insight into the product as to be important to a designer? To a user? To management?
• Will all the units be submitted to this test? Can I only sample some?
• In what order should this test be performed?

RST involves double checking your test case prototype against constraints that are very easily overlooked. It often brings ideas about grouping tests together in specific suites for short or long runs. It brings about the concepts of test jigs and test processes that could significantly reduce the time required for specific steps (can we run this in parallel?).

So there we are: test your test cases by submitting them to the MA/RST mantra! How could they not improve?

Stay tuned for the next post, and don't forget to comment on this one, I can't wait to learn more from you!

Thanks,

FX