Skip to main content

Posts

Showing posts from 2010

Continuous Delivery

If you didn't already know that the key to reliably deploy quality software is to take a cross-functional, full-lifecycle approach, Jez Humble and David Farley's book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation will help you to understand. Much like Jim Coplien describes in  Lean Architecture: for Agile Software Development the "secret" to successful lean projects is to work with "Everyone, All Together, from Early On."

While the authors have experience in, an a pre-disposition for, agile techniques, the principles described in  this book apply to any organization, whatever the process,  though if you take the approach to heart, you will find yourself becoming more agile, which is to say, more responsive to customer needs.

The book  covers the full lifecycle from requirements, and design, and coding, to acceptance testing, deployment and operations. There are even discussions on test design, data migratio…

What Agile QA Really Does: Testing Requirements

Teams transitioning to agile struggle with understanding the role of QA. Much of what you read about agile focuses developer testing.  Every project I've worked on which had good QA people had a much higher velocity, and higher quality. And there are limits to what developer Unit Testing can test.

In an "ideal" agile environment a feature goes from User Story, to Use Case, to implementation. With tests along the way, you should be fairly confident that by the time you mark a story done that it meets the technical requirements you were working from. If someone is testing your application after this point, and the developer tests are good, it seems like they should be testing something other than the code, which has already been tested.

Even though it's important that your QA team does not become the place where "testing is done" as that will sidetrack an agile project very quickly, it is  possible that the QA Team is testing (ideally in collaboration with de…

Risks of Manual Integration Testing in the Context Rapid Change

You have probably come across a situation like this: It's close to a release deadline. The QA team is testing. Developers as testing and fixing problems, and everyone is focused on getting the best product they can out the door on time. During this time, you may notice that someone on the QA team, while working late has found an interesting problem. And they clearly spent a lot of time investigating the problem, identifying the expected results, and the details of why what's happening is wrong. If this is a data intensive application, there may well be SQL queries included to allow you to pinpoint the issue quickly. In short, an ideal problem report.

Except for one thing. Your team found and fixed the problem hours before.  And the effort to find and document the problem could have been spent on something else.

It's hard to avoid this kind of overlap when you don't have a complete end-to end automated testing process. And it's probably impossible (for now, anyway) …

To Scrum, Prepare

Agile methods have some sort of daily all-team checkpoint meeting as part of  the process. The idea behind the Daily Scrum (Scrum) or Daily Standup (XP) is good:  replace status meetings (or someone walking around asking about status) with one short daily meeting where everyone has a chance to communicate about what they are doing and what they need help with. This ensures that there is at least one chance each day for everyone to understand the big picture of the project, and to discover unexpected dependencies.

But just having everyone in the room doesn't make for an effective, focused scrum. You need to be be prepared. Once I was on a team where the scrums started going off track. They took longer. People's updates were often "I don't remember what I did yesterday," or they became long unfocused rambles that didn't convey much information.  I suggested that we all take a few minutes before Scrum to organize our thoughts. This got a lot of resistance. "…

The Checklist as Empowerment tool

In an earlier post I talked about how many of the ideas in The Checklist Manifesto: How to Get Things Right  support for agile values. One of the  observations in the book that caught me by surprise was that checklists help people function as a team by making it easier to distribute decision making and empower individual team members. Checklists also help teams make better decisions by making it easier to distribute decision making. A team of empowered cross functional people, working together to decide how to get work done sounds a lot like the model of an agile team.

Checklists can help by institutionalizing a process where someone other than "the expert" is the center of decisions. In a discussion of a surgical checklist, Gawande discusses that nurses are the best people to own the checklist process but they needed to be able to stop a surgeon who skips a step without risking disciplinary action by a surgeon who feels free to avoid the process. Making the checklist part of…

Lean Architecture

When I was a new programmer, the career path that appealed to me was to be an software architect. The architect was the person who had the vision of how the system worked, and the work of the architect (if done correctly) set the stage for all good things in a project, coordinating the development, requirements, and anything else  that you need to build a system. One thing that troubled me was that many architects I knew didn't code, considering the coding a distraction. Having worked on a project or two early in my career with a non-coding architect who was reluctant to spend time helping the team address how difficult his vision was to execute given the languages and frameworks we were implementing with, I thought that something was amiss with the idea of a non-coding architect.

One of the off-shoots of specification heavy projects (no doubt staffed by many analysts and non-coding architects) was the agile software development movement, which has as one of it's principles mi…

Are You Done Yet?

Johanna Rothman recently wrote, commenting on Joshua Kerievsky's proposed definition of done. Both posts are worth a read, if for no other reason than to better understand why we have such a difficult time defining what "done" is, and why defining "done" is one of the major challenges for teams  trying to adopt agile practices.

Thinking about both Joshua's and Johanna's points I wonder if the difference isn't similar to a discussion of whether principles or practices are more important to be successful when adopting agile methods. On the one hand following  practices diligently allows you to develop good habits and even to get some good results early on. The challenge comes when it's time to reflect and improve on your practices. Without a good understanding of practices it's hard to optimize.

Similarly, defining done earlier in the process can cause problems if you are thinking about the meaning of "done" the wrong way.  If "…

A Review of Drive by Dan Pink

This is one of those books that describes something extremely obvious and intuitive that at the same time goes against what you were taught was "common sense." This would be a good book just for the survey of the (long) history of the study of the theory of motivation. It also concludes with a number of things you can do to create an environment that encourages mastery (as opposed to simply meeting goals) in your work and school. 


If you're an agile software developer you'll have a few aha! moments when you understand how agile practices really encourage flow and create environments where teams and individuals can be highly productive. If you're a manager, this book will encourage you to think about how teams work and how some common practices are counter-productive. 


If you're trying to understand why self organizing teams work, but with a perspective outside of software development, this is a quick read that will get you thinking and learning.


Some other books…

Motivation Visibility, and Unit Testing

I've always been interested in organizational patterns (such as those in Organizational Patterns of Agile Software Development). I've recently found myself thinking a lot about motivation. I'm now reading Drive: The Surprising Truth About What Motivates Us and just finished Rob Austin's book on performance measurement. Being the parent of a three year old, I'm finding more and more that "because I said so, and I'm right" isn't too effective at home. My interests in motivation are closely related to my interest in writing software effectively. Writing software is partially a technical problem about frameworks, coding, and the like, but the harder (and perhaps more interesting) problem is how to get a group of people working together towards a common goal. Agile practices, both technical and organizational, build a framework which makes having the right amount of collaboration and feedback possible. But there's a bootstrapping process: How do yo…

Measuring Performance for Teams

If you've ever been in an organization that had performance reviews, you may have found yourself wondering whether the measurement process used to evaluate your (and your team's) performance made sense, and if there was a better way. Even if you have goals that can be evaluated quantitatively, rather than some metric that feels arbitrary, you may feel that your personal goals may run counter to the success of the your team. For example, you may wonder whether it makes sense to help your colleague on her high priority project and risk missing a deadline for your (lower) priority one. Sometimes the problems with measurement systems are because people just don't measure well. In other cases, it's because it's impossible to measure all of the things that matter.

Rob Austin's book Measuring and Managing Performance in Organizations gives you a model to understand why measurement systems become dysfunctional, and an approach to avoid dysfunction when you are measurin…

Things about Release Management Every Programmer Should Know

As I mentioned earlier I was privileged to contribute to the book 97 Things Every Programmer Should Know: Collective Wisdom from the Experts. In addition to the contributions about coding and design, I was pleasantly surprised to see the number of items that relate to release management. While I've long been interested in how to build architectures and processes that make deploying and releasing software easy, I sometimes get the impression that these items were often though of necessary evils that could be done at the end, often by the someone who isn't doing "more valuable work." Much like awareness of agile software development made it obvious that testing and quality assurance activities work best when they are integrated throughout the development lifecycle, agile has also made it more obvious why build and release engineering is something to work on as you go. This makes a lot of sense, as ease of release is closely tied to the physical architecture of the syst…

Planning is a Gerund

One of the things teams adopting agile struggle with is deciding how much to define a plan before you start executing. Have a plan that's too well developed and you end up risking that your team may not be responsive enough to change. Too little of a plan and you may end up changing course excessively, and have no way to measure your progress towards any sort of deliverable. 
At the core of this confusion over how much to plan is the reality that plans change, and spending too much time and energy creating a plan that ends up being wrong seems wasteful. But the lack of a plan means that you have nothing to measure progress against. One way to reconcile this is to keep in mind the following quote, attributed to Dwight Eisenhower (and a variant attributed to Churchill): Plans are nothing; Planning is everything.
If we assume that as a project progresses, that things will change, we can still benefit from talking through the options as a team. Capturing the things that we don't kno…

Book Review: Modular Java

I recently read  Craig Walls' book Modular Java: Creating Flexible Applications with Osgi and Spring (Pragmatic Programmers). This book is a very detailed tutorial that walks you through setting up an application using OSGI and Spring with the help of Maven as a build tool. If you aren't familiar with any of these technologies, this book will get you started, and quickly have you feeling like you have a basic grasp of the concepts and technologies.

You'll finish the book with a desire to learn more about the technologies, and understand the power of modular applications.  As a tutorial, this is an excellent book. This is not an general guide to how to design with OSGI.  There is some background on the frameworks, and some explanation of technologies, and also pointers to sources for more information, but this book is all about learning by building. If you pick up the book hoping to learn any details about the how and why of OSGI and Spring, two useful technologies, you mig…

Are You Agile Enough?

I recently read an article in SD Times about how organizations tailor agile processes to fit into their environment, rather than feeling a need to be dogmatic. Adapting a process to work in your environment is very important to being successful. It's also  important, however to understand how the variations you are making help you move toward your goals.

Being agile is a means to an end; your goal is to develop better software more effectively, not to be able to wear a "We are Agile" badge. If you're considering adopting agile, you are probably doing do because your current approach isn't getting you where you need to be so it's worth giving the 'by the book' technique a shot before you try to adapt an agile method to your circumstances. This especially applies to when you consider omitting practices.  Like many approaches,  there is a synergy between the the core agile practices; any one can help you be better but the big wins come when you do them a…

The Checklist Manifesto as Agile Primer

I recently read Atul Gawande's book The Checklist Manifesto: How to Get Things Right and found a number of of useful lessons in the book for agile developers. Agile software development methods often have very few explicit processes, but these processes are essential, and require discipline to execute well. We're often tempted to skip steps, either because:
We think that the step doesn't apply in a particular situation orWe forgotThe former is reasonable when we really understand why the step isn't relevant, but we are often tempted to convince ourselves that a step is unnecessary before we've mastered the basic process. One common example is the Scrum Team that is tempted to skip one or more of the standard planning or review activities.  And forgetting is just something that happens when we're busy and moving quickly.

It's when we skip steps that processes break down. Think about the time's you've had a hard time tracking down a problem. How often …

Agile Portfolio Management

I've heard people criticize agile methods as being too reactive and focusing too much on the little picture and ignoring larger goals. This is a misunderstanding of a basic idea of agile. Agile methods are't about thinking small.  Agile methods are about making small steps towards a goal, applying programming an management discipline along the way. (For more, have look at an elevator pitch for agile I wrote last year.)

The basic approach of all agile methods is to
Define a goalBreak the goal into incremental bits so that you can iterate towards the goalPeriodically (at the end of each iteration) pause to evaluate both your progress towards the goal, and whether the goal makes sense. Teams often skip the second part of the this evaluation, but it's the constant evaluation of the long term goals that makes it possible to reconcile the concepts of being agile and planning.

If you have doubts about whether long-range planning in an agile environment is even possible, read Joha…

Estimation Poker

Estimation is a necessary part of software development. Product owners want to know how much work can get done by a deadline, project managers need to make commitments, and developers want to know if they committed to a reasonable amount of work.  While estimates are often inaccurate, estimates  provide landmarks along the way of a project to gauge progress. So, estimates are an inevitable, and useful part of the software development process. Many complain that the process of getting to those estimates, estimation, takes too long, so planning sessions are cut short and teams don't have enough time to discuss issues that have uncertainty. By appropriate use of planning poker, you can balance the needs for good estimates while minimizing time spent estimating.

Sometimes when a team is asked to estimate a backlog item, one or more people with expertise in an area are asked to estimate the item, but this is not the best way for an agile team  to get a good estimate.

There are benefits…

The Indivisible Task

One of the things that makes agile work well is a daily sense of progress that can be reflected in, for example,  a burn-down chart.  For burn-down charts to be meaningful, the estimate of amount of work remaining in a sprint need to be accurate. Re-estimating work remaining in a task is helpful,  but the best measure of progress is the binary "done/not done" state of the items in your backlog.

Assuming that you have a clear definition of "done" for a task,  it's easiest to measure progress when you have tasks that are small enough that you can mark them complete on a daily (or more frequent) basis. Breaking work down into a reasonable number of reasonably sized tasks is something many find challenging. (Note: I'm talking here about development tasks as part of a sprint backlog, rather than splitting User Stories in a product backlog, though there are some parallels.)

 I've worked on teams when people refused to break down large task into 1-day or smal…

97 Things Every Programmer Should Know is Done

The book 97 Things Every Programmer Should Know: Collective Wisdom from the Experts is finally available, and the title is on the mark.

Kevlin Henney, who I first met at the 1998 PLoP conference, asked me to participate in this project  in September of 2008.  I am honored to be a part of the list of contributors, which includes Kevlin,  Bob Martin, Michael Feathers, Giovanni Asproni, and many others who have important things to say about how to build great software. Kevlin did an amazing job coordinating and editing, and the the book represents an excellent cross-section of the many contributions that formed the basis for the final version.

Reading this book gives you a chance to learn from the experiences of people who've worked hard not just at writing good code, but at creating good software systems. Some of the advice may be things you already know. Some items may be surprising. Read this book to learn, be challenged, and to understand why programming isn't just about lang…

Tracking what Matters

I'm a big fan of burn-down charts for tracking sprint and release progress.  The basic idea of a burn-down chart is that the team starts with estimates for all of the tasks in the sprint, and then on daily (or more frequent) basis re-estimates the amount of work remaining.

With a burn-down chart, you are tracking the new estimate based on the work that you have done. As you work on the sprint backlog you get a better understanding of the tasks, and thus you can revise estimates for tasks that span more than one day. This is reasonable since the original estimate is, well, an estimate.

Sometimes if you spend 4 hours on an 8 hour task, you'll have 4 hours of work left. Most of the time the time left  will not be the original estimate less the time spent, but more or less. At the end of 4 hours, the remaining work estimate for the same 8 hour task could be 2 hours, or it could be 10 hours if you discovered a snag.  This is important information for everyone involved in the proje…

Banishing Blame with Agile Values

I recently got pointed to an article about some research done by researchers at Stanford and USC about the dynamics of blaming in organizations. People Like to Play the Blame Game says that it's quite easy to create a culture of blame.
Merely observing someone publicly blame an individual in an organization for a problem - even when the target is innocent - greatly increases the odds that the practice of blaming others will spread ...
The ways that you avoid a blame culture seem to fit well into an agile culture that values retrospectives, and continuous improvement. The article advises leading by example:
A manager can keep a lid on the behavior by rewarding employees who learn from their mistakes and by making a point to publicly acknowledge his or her own mistakes, Fast said. Managers may also want to assign blame, when necessary, in private and offer praise in public to create a positive attitude in the workplace.
This advice isn't just for managers; everyone on the team need…

Estimates and Priorities: Which Comes First

When developing a release plan, product owners often want to factor cost (or estimates) into where items go into the backlog. For example, you might hear "Do that soon, unless it's really hard." If this happens once in a while it's a useful way to have a conversation about a feature. If this approach is the norm and nothing gets prioritized until the team estimates it. I'd argue that the default order of operations is that features should get prioritized before anyone spends any effort on estimation. My reasons for this are both philosophical and practical.

The philosophical reason is that the Product Owner should be the one to prioritize work. By asking for the estimate first, the product owner is deferring their authority to the engineering team. This creates a risk that the team may not end up working effectively.

The practical reasons for prioritizing before estimating are:
Estimation takes time, and if you don't start with a prioritized list to estimate …

Review of Adapting Configuration Management for Agile Teams: Balancing Sustainability and Speed

Adapting Configuration Management for Agile Teams: Balancing Sustainability and Speed is a book that bridges the gap between tradition SCM and Release Engineering and Agile teams. Mario asked me to be a reviewer of the draft manuscript and I knew that Mario had great experience in establishing SCM processes in larger organizations, and that he was also a strong advocate of Agile methods. I was pleased to discover a book that, while being able to help those from a traditional release management background adapt their processes to support agile, also addresses the needs of those transitioning to agile who can benefit from using appropriate SCM processes to help them.

This book will help you to understand how SCM can be an enabler for agile, and will also help you to understand how to fulfill the SCM definition of Identification, Control, Status Accounting, and Audit and Review while still being agile.

This is what I said about the book on my books page:

This book is a good guide to both…

Sprint Boundaries and Working Weekends

A core principle of agile methods is sustainable pace. While the precise definition of this is debatable, the basic idea is that you want your work load to allow for a life outside of work, which in turn means not planning on overtime. The reality for many projects is that there will be times that the team needs to work outside of "normal" hours to meet a goal, and this is consistent with the idea of a sustainable pace if it happens occasionally, and the team decides that to meet a goal the added hours are needed.

Another core principle of agile methods is transparency. In order to improve, you need to be honest about acknowledging changes in plans, mistakes which cause more work, and misunderstandings that cause you to get things done significantly ahead of schedule.

A decision that teams make independently of the number of hours that they need to plan for is what the boundaries of a sprint should be, particularly with a 1 week sprint. This question is especially relevant …