Skip to main content

What Agile QA Really Does: Testing Requirements

Teams transitioning to agile struggle with understanding the role of QA. Much of what you read about agile focuses developer testing.  Every project I've worked on which had good QA people had a much higher velocity, and higher quality. And there are limits to what developer Unit Testing can test.

In an "ideal" agile environment a feature goes from User Story, to Use Case, to implementation. With tests along the way, you should be fairly confident that by the time you mark a story done that it meets the technical requirements you were working from. If someone is testing your application after this point, and the developer tests are good, it seems like they should be testing something other than the code, which has already been tested.

Even though it's important that your QA team does not become the place where "testing is done" as that will sidetrack an agile project very quickly, it is  possible that the QA Team is testing (ideally in collaboration with developers) things that the team decided that developers could not test completely. Also, in practice,  problems sometimes slip through developer tests,  and the QA team is then testing the developer tests to give feedback on the types of things the developer tests need to be better at.  

Aside from "catching the errors that slipped through" a QA team doing exploratory testing is also exploring novel paths through the application, and may discover problems during some of these paths. In this case the team and stakeholders need to have a conversation about whether the error is something that users will see and care about, or whether it is something that isn't worth fixing.

In effect, exploratory testing is testing the requirements for completeness.

If you have a QA team, and they are doing exploratory testing, they are really testing:
  • Requirements: Finding interactions between features and components that were not defined or understood when coding and developer testing began. 
  • Developer Tests: Identifying where developer tests were not as good as they could have been (and how they could be better). This is a good use of automated "integration" tests.
  • Systems tests that might be hard to test in a developer context.This might be the one set of tests that are the primary domain of QA.
And, as I've said in other places,  QA team can be extremely valuable testing user stories and requirements before they make it into a sprint backlog, by providing feedback about testability and precision. The QA team can also add value by testing how useful a product that meets the specification actually is.

Of course, this is an idealization.  But if you are disciplined about doing developer testing, you can still get a lot of value from exploratory testing, whether the people doing them are dedicated QA people, or people who are filling the role.

Comments

Popular posts from this blog

Continuous Integration of Python Code with Unit Tests and Maven

My main development language is Java, but I also some work in Python for deployment and related tools. Being a big fan of unit testing I write unit tests in Python using PyUnit. Being a big fan of Maven and Continuous Integration, I really want the  Python unit tests to run as part of the build. I wanted to have a solution that met the following criteria:
Used commonly available pluginsKeep the maven structure of test and src files in the appropriate directories.Have the tests run in the test phase and fail the build when the tests fail.
The simplest approach I came up with to do this was to use the Exec Maven Plugin by adding the following configuration to your (python) project's POM.

<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <executions> <execution> <configuration> <executable>python</executable> <workingDirectory>src/test/python</workingDirect…

Displaying Build Numbers in Grails Apps

Being a fan of Continuous Delivery, identifiable builds, and Continuous Integration: I like to deploy web apps with a visible build number, or some other way of identifying the version. For example, having the build number on the login screen for example. In the Maven/Java world, this is straightforward. Or at least I know the idioms. I struggled with this a bit while working on a Grails app,  and wanted to share my solution. There may be other, better, solutions, but the ones I found approaches that didn't quite work they way that I'd hoped.

My requirements were:
To display a build number from my CI tool, where the number was passed in on the command line. In Bamboo, for example you might configure a grails build as
-Dbuild.number=${bamboo.buildNumber} warTo only change build artifacts and not any source files.To not misuse the app version, or change the names of any artifacts.To be simple and idiomatic.I realized that that Grails itself changes the application metadata (appl…

Motivation Visibility, and Unit Testing

I've always been interested in organizational patterns (such as those in Organizational Patterns of Agile Software Development). I've recently found myself thinking a lot about motivation. I'm now reading Drive: The Surprising Truth About What Motivates Us and just finished Rob Austin's book on performance measurement. Being the parent of a three year old, I'm finding more and more that "because I said so, and I'm right" isn't too effective at home. My interests in motivation are closely related to my interest in writing software effectively. Writing software is partially a technical problem about frameworks, coding, and the like, but the harder (and perhaps more interesting) problem is how to get a group of people working together towards a common goal. Agile practices, both technical and organizational, build a framework which makes having the right amount of collaboration and feedback possible. But there's a bootstrapping process: How do yo…