Scrum is, according to Wikipedia, “an iterative, incremental framework for project management and agile software development.”  Here at Inova, Scrum has changed just about everything in my working life, and I would have to say the change has been overwhelmingly for the better.

In the old “waterfall” days everyone worked secluded in their own offices, and my fellow software testers and I were organized into a separate sub-department in Engineering, alongside, or some might have said underneath, Software Development.  At the beginning of a release effort, we testers spent our time reading software design documents in an effort to write extensive manual test plan documents.  These software design documents had been laboriously produced by the programmers after plowing through incomplete and often misguided software requirements documents — sometimes the programmers themselves also wrote the requirements documents!  This documentation step was a thankless process, highly abstract, and the documents produced — requirements, designs, and test plans — were practically impossible to effectively review and edit.  They were simply too abstract and ungrounded in reality.  The testers would often cheat, and write test plans by running the current version of the software, while making changes for the expected new features as we understood them.

A few weeks later there would be enough new code checked in to start testing.  Our team would do one build a week, which to me now is shocking, since we often do more than one build a day.  We’d test the software by manually installing it and running our test plans.  We’d find that the software was different than the design document, different than the test plan, and too often poorly integrated across boundaries between different developer’s code.  Most bugs were found setting up a scenario to isolate a test case, rather than from the test itself.  At the beginning the programmers would try to update their design documents to match what had actually been built and what changed due to bug fixes, but that soon stopped.  We’d spend months on test and fix cycles, being impacted by changes other programmers made to other parts of the system based on what the tester who was testing that code found.  Overall organization was loose and often code changed just to work around bugs or “fixes” in other parts of the system.  After six to fourteen months we’d release.  The rest of the company often went in to shock over what the new release did, how it did it, and what it still didn’t do or no longer did.

So we tried scrum.  At first it was difficult to adopt a totally new process and difficult not to retreat into our “silos” of knowledge.  But our process continued to evolve and pay off.  Now I am a developer, still a tester, but part of a cross-functional team that builds discrete units of functionality in three-week sprints.  Gone are the elaborate documents and the many weeks wasted trying to create them.  Also gone are our offices, now we work together in a co-location area.  The rest of the company knows what we’re doing because we demonstrate it to them every three weeks.  The silos are still around, but being broken down whenever strategic opportunities to disperse that knowledge present themselves.  We use automated builds triggered by code check in, automated tests, and continuous integration.  We still do manual testing, but now without the heavy documentation.  Best of all, we always question what we’re doing and how we can make it better, which is the best thing about the good new days.

Working with user stories in developing our products for the call center and mass notification markets has brought more focus to software development at Inova Solutions.  When a user story has been defined, estimated, and prioritized, the development team can pull the story into a development iteration and begin work on it, with the goal of completing that user story in its entirety during that one iteration.  This means each user story must be small enough to fit into one iteration’s worth of effort (usually one team of five to nine people working for one to four weeks).  How we develop a user story to get it into shape to implement in a single iteration is the topic of this and the next posting.

At Inova Solutions, our Product Owners work with departments throughout the company, survey the call center and mass notification marketplaces, and visit customer sites to come up with new product ideas and changes to our existing suite of products.  The Product Owner expresses market needs in terms of one or more user stories.  For major new features or changes there will be an epic story, encompassing the entirety of the new product or major change to an existing product.  The Product Owner then expands the epic into sets of user stories that describe the problem space that the new functionality will address.

Agile teams talk about the process for developing user stories in terms of the Card, Conversation, and Confirmation¹.  Different terms are used on different teams, but these three aspects of a user story are common in the agile development community.

Card – A high-level statement of the user story, including the user by name, a statement of the problem, and why the problem needs to be solved, usually in terms of what the user will be able to accomplish.

Conversation – The Product Owner and the development team get together to discuss the meaning, intent, and acceptance criteria for the user story. The product owner will elaborate on the environment and conditions surrounding the problem and will state the intentions and objectives of the users and customers of the story.  The team will ask questions about the story and discuss approaches for possible solutions – but only in so far as to understand the user story; solutions and designs are not provided at this time.  Often the team’s questions can lead to changes in the user story.

Confirmation – The acceptance criteria specify how we’ll know that the user story has been accomplished.  When the acceptance criteria are verified as having been met, we’re confident that we’ve met the intentions and objectives of the story.

User stories are couched in everyday terms and in the business language of the user.  This encourages communication between the customer and development teams.  At Inova Solutions, our Product Owner acts as a proxy for our customers and is not a software engineer, so putting our stories into business language facilitates the conversations around user stories.  There are also software testing tools that allow for automating acceptance tests in business language so the Product Owner can participate in developing the tests (for example, FitNesse²).

If a user story statement doesn’t encompass all it needs to be useful at the high-level, that will usually come out in the conversation.  Questions will be asked that aim at putting the user story into context with the overall epic and the other user stories defined to meet the epic.  If a question or its answer reveals that the user story or the current set of user stories is incomplete in some way, the Product Owner can clarify the user story at hand and/or add new user stories to the project.  If the user story does cover the high-level need adequately but more details are needed, that, too, will come out in the conversation and more specifics can be added as acceptance criteria.

Next time, INVEST in user stories

References

¹ Ron Jeffries, “Essential XP: Card, Conversation, and Confirmation,” XP Magazine, August 30, 2001
² FitNesse (www.fitnesse.org)

Here at Inova Solutions, we employ user stories to specify the functionality of the call center and mass notification products we develop.  We work with the Product Owner to flesh out what is needed for each marketplace, and why, given market information and company priorities. Our Product Owner, serving as a proxy for the customer, consolidates competing priorities and market information that the company knows about from all departments, and distills them into descriptions of a marketplace need from the user perspective.  When that need can be addressed by a software development effort, the Product Owner comes up with one or more user stories and enters into a conversation about the need with the development team.  We use user stories rather than a full software requirements specification so that we can be more responsive to changing marketplaces and software environments.

Because marketplaces and software environments are in continual states of flux, specifying in advance all the requirements for a project that will span months or even years is effectively impossible.  Both the Product Owner and development teams continue to learn more about the business domain and the software environment throughout the life of a project; being able to incorporate that knowledge during a project is critical to its success.

Employing user stories encourages customer involvement in our development process, both when defining the user stories and when evaluating the results, allowing for two-way feedback throughout the project. After each user story is developed, both the team and the customer have gained knowledge and can contribute to the further development of upcoming stories, taking into account the newly acquired knowledge about current and projected environment and market conditions.

With traditional software requirements specification, the functionality is often stated only in terms of the “what” and is usually proscriptive to allow for verification that the result matches the specified what.  But by being proscriptive there isn’t much opportunity for incorporating new knowledge.  With a user story, the “why” of the story provides more context than simply the what and allows for flexibility and innovation in coming up with a solution.  Breaking the project into relatively independent chunks of incremental work makes it easier to estimate the project and it allows for more flexibility when new information comes to light, whether about the market place, the software environment, or company priorities.

Mike Cohn writes extensively on Agile Software Development, and has this to say about the level of detail in user stories:

“User stories encourage the team to defer collecting details. An initial place–holding goal–level story (“A Recruiter can post a new job opening”) can be written and then replaced with more detailed stories once it becomes important to have the details. This technique makes user stories perfect for time–constrained projects. A team can very quickly write a few dozen stories to give them an overall feel for the system. They can then plunge into the details on a few of the stories and can be coding much sooner.”¹

Next time, How User Stories Work.

References
¹ Mike Cohn, Advantages of User Stories for Requirements
Mike Cohn, “User Stories Applied”, 2004, Addison Wesley, ISBN 0-321-20568-5
Mike Cohn: Agile Estimating and Planning, 2006, Prentice Hall, ISBN 0-13-147941-5

Developing software requires us to make decisions that have short and long term consequences.  We are a value-driven organization and try to evaluate cost vs. benefit at all layers of the organization.  This list represents a core set of principles I use to guide my decisions during software development.

• Simplicity first – in design and thought
  • Don’t assume anything HAS to be complex. If there is a simpler option, discuss it.
  • Don’t assume the requirements are right or can’t be changed.
  • Think about the “why,” even when the “what” or “how” seem obvious.
• Maintenance (fixing and incremental change) is a huge part and cost of development.
  • Always think about long-term costs, especially when there seems to be an easy fix and a deadline is looming.
  • If it isn’t required, then don’t do it (YAGNI – You aren’t going to need it).
  • If you write code that makes you feel dirty, you will probably regret it later.
• Structure, organization, naming and consistency are important.
  • Structure everything in layers; including projects, namespaces, classes, and functions.
  • Code must be readable and include comments or be self-documenting.
  • Naming should tell you what it is or what it does.  A lot of effort goes into naming and we still often get it wrong, but spending less time on it won’t make it better.
• Look before you leap
  • Look for existing solutions, and see what has already been done.
  • Don’t assume starting from scratch is the best option.
• Testability
  • Design for testability from the  start.
  • Unit tests are good – but understand where they get the most value.
  • Using unit tests for Graphical UIs and external interfaces may not be worth the cost.
• Performance
  • Think about memory usage and management.  Constructing objects has a cost.
  • If you start by writing efficient code, then you never learn bad habits and won’t pay the costs later.
  • String usage, logging, and loops can all be done very inefficiently.
  • It is a lot easier to focus on profiling the algorithms when the code is already efficient.

In business the importance of looking forward is universally understood. How else will you know where you are going?  Interestingly, there are a few compelling reasons to look backwards as well.

Inova Solutions uses one of the Agile software development methodologies, and these techniques demand that software be built and fully tested in small iterations, and with frequent demonstrations to stakeholders.  Each iteration delivers functionality which builds over multiple releases.  As these software projects are completed, an Agile team with a good retrospective process asks itself three questions as they prepare for the next iteration:

–        What do we want to keep on doing?

–        What do we want to stop doing?

–        What do we want to start doing?

These are simple questions, but when openly asked and answered their impact over a number of iterations is very significant.  The answers guide improvements in product and process – and these improvements are the most effective and most interesting when they come from within the team itself.  Too many teams are often ramping up on the next project as they wrap up the last one – don’t skip the retrospective even if the last project went perfectly – you could learn something very important that will help the next project to go even better!  If you want to learn more about the retrospectives and how to have one after your next project, try reading Esther Derby’s book Agile Retrospectives: Making Good Teams Great.