Testing & Software Blog

The occasional thoughts of a freelance software tester, drawn from experience across the software development life-cycle.

Reality Avoidance

Good blog post on Waterfall, Reality Avoidance & People Who Say “No”

One of the problems some managers have with iterative software development is that, when it’s done well – seeking early and frequent feedback and acting on it, as opposed to just incrementally executing a waterfall plan – it reduces the scope for avoiding reality.

On a waterfall project, reality can be avoided for months or even years. The illusion of progress can be maintained, through form filling and the generation of reams of reports that nobody ever reads, right up until the point that software needs to be seen to be working.

If it were my money, this would scare the shit out of me – not knowing what my money’s been spent on until the last moment.

But I can see the attraction for managers. It’s not their money. And typically they get rewarded for this illusion of progress, which can go as far as pretending the software is ready the night before it’s deployed into a live environment.

I think most of us who have worked in the software industry for any length of time will be nodding at that one. Been there, done that, got the polo shirt.

The whole thing is well worth a read, with some real life war stories leading to the inevitable conclusion.

Managers need to be rewarded for testable achievements, and steered away from peddling illusions. The reason this doesn’t happen more often, I suspect, is because the value of illusion increases the further up the ranks you go. If a PM gets a pat on the back for saying “we’re on track”, the CTO gets a trip to Disneyland, and the CEO gets a new Mercedes. Hence, the delusion gets stronger as we go higher. People running governments tend to be the most delusional of all, such is their power and influence. This effect is what produces the sometimes gargantuan IT failures only governments seem capable of creating.

Indeed. Though quite how you can stop politicians behaving like politicians isn’t an easy problem to solve.

Posted in Testing & Software | Tagged | 1 Comment

Testing Challenges

I don’t often post about the testing I’m currently working on, but a current project has thrown up some interesting challenges that are worth commenting on.

The new feature I’m testing is a batch update process that replicates the functionality of some existing GUI forms, using another object in the system as the source data. The actual processing is quite complex with a lot of validation and data update rules. The GUI front-end hasn’t yet been built, so I’m starting out by testing the Oracle server-side procedures by calling them through TOAD, populating the temporary tables that drive them through some simple SQL scripts.

We’re not using the main system test environment with its strictly-controlled weekly builds, but a separate environment where the developer can apply changes as soon as required. This means there’s a much swifter turnaround than I’d become accustomed to when it comes to fixing bugs I raise. Sometimes I’ve found an issue, to see it fixed in minutes.

Because none of the code under test has reached the point of a formal build the bugs aren’t going into the bug tracking system. Initially we were just using an email thread between me, the developer and the business analyst to keep track of the bugs, but after a bit that started getting unwieldy so I started recorded them in a simple spreadsheet instead.

I’m still using an exploratory testing approach, with the added advantage that the persistent data in the “temporarily” tables can serve as a record of the test cases I’ve tested. As for the source of those test case, there are very detailed functional specifications, but I’m finding the best “oracle” is actually the equivalent GUI functionality, which I’ve tested and regression tested often enough to have memorised most of the validation rules. Indeed, a significant proportion of the bugs I’ve been finding have turned out to be specification issues rather than coding errors, or edge cases that weren’t covered by the new business rules.

It’s not the usual way I’ve been working on this project, which is very Waterfall, but I’m finding working closely with the developer and seeing bugs turned round far more rapidly is a very productive way of working. Of course, once the GUI front-end is built and it’s all included in the build and spun into the proper system test environment, I’m going to have to test it all over again. But at least we’ll have pre-emptively squashed most of the bugs, and it will take far fewer iterations for it to become stable.

Posted in Testing & Software | Tagged | Leave a comment

Context Collapse

Interesting post on the Software Testing Club on the subject of Context Collapse.

I recently heard the term “context collapse” in a podcast discussing the possible flight of the younger audience from some social media applications. It is unclear who originally coined the term in the early 2000′s, which initially referred generically to the overlapping circles on social media leading to a poster’s inability to focus on a single audience. In the podcast, the meaning was more specifically defined to identify the clash of incompatible social circles: college acquaintances, close friends, family, and work connections (especially management). That incompatibility leads to an abandonment of the media or couching postings in coded terms that are (supposedly) only understood within a specific circle.

Yes, that’s exactly why I decided to leave Facebook. I didn’t realised there was actually a term for it. The post on STC goes on to describe another case of Context Collapse involving accessibility testing, which the team eventually dealt with by getting actual disabled people to test the product. It’s a very interesting read.

Posted in Social Media, Testing & Software | Tagged | Leave a comment

Moments after I mentioned on Twitter that I was seeing very little spam in my inbox compared to a couple of years ago, my inbox started getting flooded with “Undeliverable mail” return messages because some *&$% spammer has been spoofing one of my addresses. Suffice to say that if you get spam from Russian ladies using the address umlaut@kalyr.com, it’s not from me, and there’s nothing I can do to stop it.

Posted on by Tim Hall | 1 Comment

“Someone please take my bottomless bowl of popcorn? I’ve eaten so much I think I’m going to be sick” – Bitcoin-hating Charlie Stross on the collapse of Mt Gox. The whole thing reads like the plot of Stross’ novel “Halting State”.

Posted on by Tim Hall | Leave a comment

The NHS Czech Malware Bug

A bug in the NHS Choices system sent users to a malware site. As reported in The Guardian:

“Last year, a developer accidentally put “translate.googleaspis.com” rather than “translate.googleapis.com” as the source for the JavaScript file,” an NHS Choices spokesperson told the Guardian.

The “internal coding error” sent users to the mistyped URL, of which a third-party appears to have taken advantage, registering the mistyped domain name to serve adverts and malware to unknowingly redirected visitors from the NHS Choices website since Sunday evening.

Things like that make me wonder how on earth that bug could have been missed in testing, even though t’s not easy to answer that question without some knowledge of the archtecture of the site. I would assume from the URL that it’s some form of translation functionality, and I’d have thought somebody ought to have noticed the feature wasn’t working properly and investigated it little more deeply.

What I would like to know is how the Czech malware operator managed to find the bug when NHS’s own testing didn’t.

Posted in Testing & Software | Tagged , | Leave a comment

FUNCTEST stands for Functional Test. As opposed to FUNKTEST, where you raise a bug if the drummer doesn’t have a good enough sense of rhythm

Posted on by Tim Hall | 1 Comment

What does the spec say again?

Today I came across and logged a bug, which on closer examination turned out to be a result of an ambiguously-worded line in the specification rather than a simple coding error.

I mentioned this on Twitter during a mid-morning coffee break, and got two contrasting responses.

The first was that the written specification is just the starting point of a conversation between the Business Analysts, Developers and Testers over exactly what the system should look like, and constant communication will resolve any ambiguities as the development proceeds.

The other was that a developer should not be expected to question things in an environment where even the smallest changes require signing off from multiple people with different conflicting agendas. In such circumstances it’s easy to see why a developer might make guesses rather than ask questions.

My reaction to that is that if you’re trying to develop software in an organisation as bureaucratic as in the second case, you run the risk of ending up with software that’s every bit as dysfunctional as the organisation itself.

I’ve worked on projects like that in previous lives, with great long specifications written in great detail for the benefit of the developers who were supposed to implement the thing, but completely failed to give the business stakeholders any real impression of the actual functionality. But the stakeholders went and signed it off anyway, perhaps because they wouldn’t admit, maybe even to themselves, that they didn’t really understand the thing. Needless to say that project went horribly pear-shaped and turned into a nightmare death march as the development team were buried under a mountain of change requests.

Are there still organisations that develop software like that?

While I’m still in Waterfall-land, fortunately my current project is nothing like that. In the end, I got given the task of rewriting that bit of the specification to remove those ambiguities.

Posted in Testing & Software | Tagged | 1 Comment

Testers and software engineers love to argue over whether a bug is a coding error or a missed requirement. But when it causes this much damage to people’s lives, then such hair-splitting doesn’t really matter.

Posted on by Tim Hall | Leave a comment

I always find that the frequency of clichéd buzzwords is a very useful metric in determining the credibility of any random article on the internet. This is true both for business-speak like “leverage”, or the ideological call-signs used in political and cultural blogging.

Posted on by Tim Hall | Leave a comment