Skip to main content

Risks of Manual Integration Testing in the Context Rapid Change

You have probably come across a situation like this: It's close to a release deadline. The QA team is testing. Developers as testing and fixing problems, and everyone is focused on getting the best product they can out the door on time. During this time, you may notice that someone on the QA team, while working late has found an interesting problem. And they clearly spent a lot of time investigating the problem, identifying the expected results, and the details of why what's happening is wrong. If this is a data intensive application, there may well be SQL queries included to allow you to pinpoint the issue quickly. In short, an ideal problem report.

Except for one thing. Your team found and fixed the problem hours before.  And the effort to find and document the problem could have been spent on something else.

It's hard to avoid this kind of overlap when you don't have a complete end-to end automated testing process. And it's probably impossible (for now, anyway) to create an automated test process that will completely replace exploratory  testing. Any manual testing process has to balance the timeliness with the "flow" of the testers. If you update a deployment hourly, then you'll reduce the risk of redundant bug reports,  but the testers will experience too many interruptions. If you have a paradigm where you code, deploy, and stop coding until you see the next set of issues, but that means time wasted, and probably reduced quality. The answer is better communication between the the testers and developers about the state of the application. Under ordinary circumstances, you might have this sort of exchange in your daily scrum.

You're still early in adopting agile, you'll likely have a week or two at the end of a project where you'll have more effort put into manual testing, and that manual testing will find errors. So the simple thing to do is  to query among the team before spending too much time documenting an issue.

Some of the options are:
  • Searching the issue tracking system. This sound like a good idea, but sometimes it's hard to find the right query. And it's possible that something got fixed without and issue being generated.
  • Asking.  Yelling out a question, sending and email, or posting a message in a team chat room gives you the benefit of being able to be a bit vague in your query. 
I tend to prefer the "ask the room" approach. But this isn't effective when you're working in different time zones, or at different hours. On the other hand, since software development is a collaborative activity, you'll need to address the question of how yo collaborate across time. (The simple answer is to try to avoid close coupling between activities that are likely to happen when the teams are on different schedules).

Situations like the one that I described can happen. But it's worth figuring out how often they happen, and if there are simple ways to reduce their incidence and impact. Good communication channels are importanty and sometimes the lower tech ones work better.

Comments

Popular posts from this blog

Continuous Integration of Python Code with Unit Tests and Maven

My main development language is Java, but I also some work in Python for deployment and related tools. Being a big fan of unit testing I write unit tests in Python using PyUnit. Being a big fan of Maven and Continuous Integration, I really want the  Python unit tests to run as part of the build. I wanted to have a solution that met the following criteria:
Used commonly available pluginsKeep the maven structure of test and src files in the appropriate directories.Have the tests run in the test phase and fail the build when the tests fail.
The simplest approach I came up with to do this was to use the Exec Maven Plugin by adding the following configuration to your (python) project's POM.

<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <executions> <execution> <configuration> <executable>python</executable> <workingDirectory>src/test/python</workingDirect…

Displaying Build Numbers in Grails Apps

Being a fan of Continuous Delivery, identifiable builds, and Continuous Integration: I like to deploy web apps with a visible build number, or some other way of identifying the version. For example, having the build number on the login screen for example. In the Maven/Java world, this is straightforward. Or at least I know the idioms. I struggled with this a bit while working on a Grails app,  and wanted to share my solution. There may be other, better, solutions, but the ones I found approaches that didn't quite work they way that I'd hoped.

My requirements were:
To display a build number from my CI tool, where the number was passed in on the command line. In Bamboo, for example you might configure a grails build as
-Dbuild.number=${bamboo.buildNumber} warTo only change build artifacts and not any source files.To not misuse the app version, or change the names of any artifacts.To be simple and idiomatic.I realized that that Grails itself changes the application metadata (appl…

Motivation Visibility, and Unit Testing

I've always been interested in organizational patterns (such as those in Organizational Patterns of Agile Software Development). I've recently found myself thinking a lot about motivation. I'm now reading Drive: The Surprising Truth About What Motivates Us and just finished Rob Austin's book on performance measurement. Being the parent of a three year old, I'm finding more and more that "because I said so, and I'm right" isn't too effective at home. My interests in motivation are closely related to my interest in writing software effectively. Writing software is partially a technical problem about frameworks, coding, and the like, but the harder (and perhaps more interesting) problem is how to get a group of people working together towards a common goal. Agile practices, both technical and organizational, build a framework which makes having the right amount of collaboration and feedback possible. But there's a bootstrapping process: How do yo…