Where Worlds Collide

Another of my occasional professional-related blog posts. I have considered spinning these off into a separate blog, but for the time being, it’s another category on my personal blog. As I’ve explained before, what I do in my day job isn’t completely unrelated from some of what I do outside of work.

So, the all-time favourite bug?

This is a question I’ve been asked recently. While it’s difficult to single out one bug from all my years of testing experience, this one does stand out, and comes from one of the most challenging projects from my testing career so far.

The application was a work scheduling system, part of a very comprehensive housing management product. A call centre would take calls from tenants reporting broken windows or blocked drains, then the system would schedule appointments for operatives to carry out repairs. The operatives themselves could update progress on these appointments in real time using a mobile app, allowing the system to maintain the daily work plan dynamically.

It was a complex application, involving two third-party products, one of them the mobile app, the other, which formed the focus of my own testing, being the scheduling engine itself. This was a standalone system which took XML messages containing the call information, and sent back XML containing either the appointment, or the reason why it couldn’t create one.

One thing that became apparent early on was the scope of my testing didn’t encompass just our own product and the XML interfaces. End-to-end testing soon revealed a lot of bugs in the scheduling engine itself. We had some complex business rules for dealing with things like multi-person jobs, multiple jobs which had to be performed in sequence, or jobs where the call centre wanted to assign a particular operatives rather than let the scheduling engine choose. With a near-infinite number of potential combinations of conditions I performed extensive exploratory testing on increasingly arcane permutations. A straightforward example might be “Remove wasps nest” followed by “Carry out repairs to guttering“, which required two different trades but had to be performed in the right order. Some of the less straightforward scenarios resulted in the scheduler returning gnomic utterances such as “The optimal schedule does not include this call“, or, if you were really unlucky, “Object reference is not set to an instance of this object“. Unfortunately I could never find a test case that would repeat that last one consistently.

The most significant bug occurred when the schedule filled up. The dynamic nature of the schedule meant that sometimes appointments did get missed. For example, if a previous appointment ended up taking far longer than anticipated, and the system couldn’t find another operative available in that slot because everyone else was busy, the appointment would fall out of the schedule and need to be re-scheduled. The same would happen if an operative went sick or crashed his van during the course of the day.

Unfortunately it did the same thing when I filled up the schedule and tried to schedule further appointments. Rather than reject them with “No operatives available with correct skill” as it should have done, it decided the plan would be more ‘efficient’ if it appointed the new jobs and displaced existing ones it had already committed to. Not a situation which would lead to happy tenants, who would not be impressed if they’d taken an afternoon off work for a plumber who never turned up.

Given the severity, that bug was reported to the third-party supplier and fixed before the system went live on any customer site. But that particular scenario ended up in the suite of regression tests run each time we received a new release of the third-party scheduling engine.

Regular readers of this blog don’t really need to be told that I’m a very keen music fan and amateur rock critic. Writing about a small club-based scene I’ve come to know quite a few band members over the years. I’ve even had people suggest I should quit working in the IT industry and become a full-time music writer. But while being on the fringes of the music scene is can be a great experience, I’m not convinced I want to jump ship and join the circus.

But I can see a lot of parallels between music criticism and my professional career as a software tester.

Not that I’m suggesting that testing and reviewing are exactly the same. To start with music is inherently more subjective than software. But there just as it can be a judgement call as to whether or not a piece of software is fit for purpose, it’s never completely subjective as to whether a record or performance is good, bad or indifferent. There are those that claim all opinions are equally valid when it comes to reviews, and there is no such thing as an objectively good or bad record. If you believe that, you clearly haven’t heard Lou Reed’s appalling collaboration with Metallica. It seems to me that both testing and reviewing are something many people can attempt, and just about anyone can do badly, but take skill and experience to do well. You only have to look at the reviews on websites to which anyone can post without moderation to realise there are bad reviewers out there just like there are bad testers.

To review a record or concert requires both an understanding of what the artist is trying to achieve, and an honest assessment of how well they’ve succeed in achieving it. That in turn requires the equivalent of domain knowledge. Just like a lot of indie-pop reviewers come horribly unstuck attempting to review progressive rock or metal releases, ask me to review a dubstep or free-jazz record and I wouldn’t know where to start. But just as testers from different backgrounds will approach things from different angles and uncover different bugs, a reviewer with deep specialist knowledge of a specific genre will have a quite different perspective from one whose taste is far broader. Something that’s meant to have crossover appeal benefits from both viewpoints.

Then there is the issue of speaking truth to power, which can require both courage and diplomacy. Egos even bigger than those of developers go with the territory. When an artist has poured their heart and soul into making a record, they don’t always appreciate being told how their work could have been better. Much like the way developers don’t always appreciate being told the code they’ve slaved over is riddled with bugs they they really ought to have picked up in their own unit testing. And if you’ve ever had the misfortune to work in a dysfunctionally political environment where project managers surround themselves with yes-men and tend to shoot the messenger whenever those messengers are bearers of bad news, then you’ll recognise those over-zealous fans who sometimes try to vilify anyone that attempts constructive criticism.

It’s true that there are a lot of rock critics out there who exhibit exactly the same sort of adversarial behaviour that gives some testers a bad name. Yes, writing and reading excoriating reviews of mediocre records can occasionally be cathartic, but informed and honest constructive criticism is far more valuable in the long run. Just as software testing is a vital part of making sure software is fit for purpose, constructive criticism has a role in making music better.

Perhaps it’s my tester’s ability to see patterns, but what I hope the above goes to show is that sometimes what you do in your “day job” and an apparently unrelated activity you do in your spare time can have more in common than you think. Certainly there are transferable skills, especially those softer ones which are much in demand.