Managing Uncertainty in Software Development
Let's face it: The traditional method of software development has been an unqualified failure. The so-called "waterfall approach" has led the industry to a dismal success record of about one in ten (Royce, 2002). Hard to imagine --- after all, it seems like common sense: First you plan a sequence of tasks, then you track execution, and finally, you adjust for variances between planned and actual performance. Straightforward, simple, and proven. After all, it's worked well in many industries, such as the construction industry.
But software development isn't straightforward. Uncertainty creeps in at every turn. The main reason the waterfall approach's low success rate is that fails to account for the numerous causes of uncertainty that typically affect planning and managing a software development project. In the construction industry, there is little uncertainty about the steps in the process or the eventual outcome. There are well-understood construction codes, laws of physics, and properties of materials that all determine the result. Success simply depends on resource management and proper execution of the plan. But as much as we would like to believe that software development is a straightforward engineering task, it just isn't. We must, therefore, adopt a different model - one that affords the flexibility required in the challenging environment of software development. That process is User-Centered Design (UCD). UCD is defined as "an approach to designing ease of use into the total user experience with products and systems. It involves two fundamental elements - multidisciplinary teamwork and a set of specialized methods of acquiring user input and converting it into design (Vredenburg, K., Isensee, S. and Righi, C., 2002).
This article discusses some of the major causes of uncertainty in software development and describes how process can address them.
1. Customer requirements frequently change over the course of a project. Sometimes the changes are due to new tasks that users need to perform. Sometimes, new environmental factors (e.g., stories in the popular press about robots that mix martinis) cause users to have greater expectations for the product. Whatever the reason, there has to be a way to account for new requirements.
In "waterfall" type processes, customer requirements are gathered at the beginning of a project, and are virtually written in stone. The design spec is then created directly from those requirements. This makes it difficult to account for new requirements that arise later on. If the design does change, it is typically at great expense. In fact, it has been repeatedly demonstrated that the later in a project changes are made, the more costly those changes are (e.g., Mantei & Teorey, 1988).
UCD, on the other hand, is an iterative process. In addition to the requirements gathering performed at the beginning of the project, ongoing customer feedback sessions are performed throughout the project. These activities help to weed out invalid requirements and help head off "feature creep," the tendency to keep adding features to a product whether or not customers want or need them. Mauro (2002) reports that 95% of your customers will use less than 5% of the features and functions on a typical website, and 75% of the functions will never be used. By performing multiple requirements gathering sessions, the requirements from one customer can be validated with those of others, thus ensuring that the features you decide to include are those that your customers truly want and need. If only a small number of customers seem to need a feature, and there is no other compelling business requirement to include it, then that feature can be excluded from the product.
In addition validation of requirements, iterative requirements gathering sessions also frequently reveal new requirements. New requirements that arise late in the process are almost an inevitable fact of life in product development. It would be ideal to gaze into a crystal ball and identify all the requirements at the start of the project. But given the lack of said crystal ball, rather than fighting against them or ignoring new requirements, UCD has a mechanism for accounting for them. With UCD, you can get a good handle on out what customers want at the beginning, and continue to validate those requirements and evaluate your design as it evolves, rather than build a mistake and find out about it when it's already too late.
2. New competitors may enter the marketplace while a product is in development. This can upset your carefully laid product plans.
UCD stresses the importance of knowing your competition. You need to understand the products you will compete against and design your product to beat them. You need to not only know the characteristics of competing products when you begin design, but you also need to estimate how they will improve over time so that you are still ahead when you release the product and throughout the life of the product. To hit a moving target, you need to aim ahead. Because UCD is a multidisciplinary design process, your design team includes marketing and business intelligence professionals who can help to anticipate the competition's next move.
Despite your best attempts to anticipate your competition, you will sometimes be surprised. Fortunately, as described above, UCD is iterative. If a new competitor or improved product unexpectedly comes along part way through a UCD development project, you can adapt. If your market is very dynamic, you have to "expect the unexpected" and take this into account in your product plans. For example, UCD includes ongoing market research in your project plan, and should allow time in the schedule for redesign.
3. Unexpected personnel changes can introduce risk to a project. These days, few people work for one employer for their entire career. Even when employees do stay with a company, they may be frequently shifted across projects because of reorganizations, changes in business needs, and employees' personal interests or career goals. Job changing was a particular problem during the dot com boom when many employees jumped ship as often as every few months.
UCD advocates involving the whole team in decisions so information is dispersed and replicated. Work is allocated by role. Roles can be reassigned and individuals can assume more than one role. Like the internet, UCD teams are designed to survive a nuclear blast! Well, not really, but both are designed to be very adaptable.
The UCD process also produces documents and design artifacts that can be stored in a database, website, or other repository where the whole team has access to them. The team has a collective "memory" so that vital information is not lost when an individual is lost.
4. Budget cuts are often made during the course of a project. Especially given the current (terrifying) business climate, reevaluation of project priorities occurs frequently. Staff may be eliminated or reassigned, cycle times may be reduced, and as a result, the number of product features, amount of task support, and other attributes may have to be cut.
In traditional processes, such changes may be cataclysmic. Because the design is locked in early in the process, it is resistant to change. Making changes is akin to the old game of pickup sticks: Moving one stick will cause many others to move out of place. Late changes are disruptive and costly. By contrast, with UCD, the design is not written in stone early in the process, but rather, evolves over the course of the project. With UCD, the conceptual design is formulated first, followed by the detailed design. With UCD, cuts can be accommodated more easily and made deeper into the project without experiencing "project meltdown."
Other aspects of UCD that facilitate design change include customer prioritization of requirements. Because customers help the designers understand which requirements are most important, intelligent cuts can be made, rather than hacking away in the dark. Finally, because UCD requires constant customer involvement throughout the process, designers can tap customers for their feedback with regard to the planned product changes. Designers can ask, will this product still be attractive to you if these changes are made? In short, UCD allows designers to react intelligently and in a measured, informed way to budget cuts.
5. Design is a creative process. Many products need to be new and innovative to be successful. It is difficult to mandate and schedule creativity. It works best to define product requirements in terms of user needs rather than features. This gives the development team freedom to innovate. There are usually multiple design solutions to any given problem. The development team can brainstorm multiple solutions then prioritize them based on usability, cost, schedule, and other factors.
Solutions that are both creative and useful often come from the collective wisdom of team members with varying skills and viewpoints. Sometimes this diversity can be challenging. Often these disciplines have their own terminology and culture. It is important to be able to translate their ideas into terms the whole group can understand so that the benefits of the multidisciplinary nature of the UCD team can be realized.
Here are some examples of multidisciplinary team communications and their translations:
ARTIST: A black background with Garamond san serif text would be like really bitchin', dude!
TRANSLATION: Superior visual esthetics would increase customer satisfaction.PROGRAMMER: There is absolutely, positively no way to code that. Can't be done.
TRANSLATION: You'll have to buy me beer and pizza to get that one.
UCD design artifacts (for example, prototypes) provide a blueprint or specification for the design that all team members can understand and that can also communicate the design to people outside of the team. They don't all have to speak Geek. The UCD process also facilitates estimation and project management by introducing milestones, measurements, and deliverables that help keep the project on track.
6. The nature of good design is iterative: You design, you test your designs, and then you redesign based on the results. This is a proven effective method: Mauro (2002) reports that for every $10 spent on early usability engineering and usability testing activities, $100 is saved in total development costs. However, iteration brings about with it a measure of uncertainty. Every time you test a design, you run the risk of discovering it's lacking. This compels you to fix the problems and test again. OK, so quality improves. But what's quality compared to broken deadlines and project managers' gray hair? Facetiousness aside, trying to schedule iterative design is a bit like stapling Jello. You hope for fewer iterations, but you may have several. This increases the uncertainty factor, especially with regard to schedule.
What is the remedy to iteration? Embracing it. All but gone are the days of the killer app everyone had to have, despite its abysmal usability. Today, customers almost always have a choice; and given a choice, they almost always choose quality. Rather than sacrificing quality for speed, UCD considers the quality that iteration buys, and schedules for it. This does not mean speed is disregarded - far from it. Time to market is still critical. But it is only one of many aspects that constitute the Total Customer Experience. It must be weighed against all other aspects of the experience. Instead of shying away from or covering up mistakes, UCD seeks to expose and accelerate them. UCD allows you to fix mistakes early while they can be fixed, and for much less cost than later on. Even the harried project manager would rather revise a prototype than recall a shipped product!
7. Software is perceived by those who create it as open to an infinite amount of change (hence the term *soft*ware). This perception can cause problems at both ends of the spectrum. Some development teams continue to code well past the time they could have shipped the product, in search of perfection (You can almost hear the chanting: "Nirvana through Java…oooooooommmmmmm"). Others err on the other extreme and ship the product on schedule, full of flaws, and end up inflicting great pain and suffering on the poor, unfortunate souls that are the end users.
One thing that's becoming clear: The old method of software development just doesn't cut it any more. The lone hacker in the garage creating the next killer app has gone the way of the dot-com bust. Customers are getting too savvy to accept whatever the IT industry dishes out. Competitors offering a product with as many features but with a better total customer experience are lined up waiting to grab your users away from you. So what's the solution? UCD. A UCD process helps the team to set realistic goals, measure progress toward them, and manage the project to meet them. UCD introduces a measure of rigor into software development. Let's face it: IT is a business. We have to treat it as such. We have to set business, market, and user-oriented goals and work toward them. And UCD helps us do that.
8. Many projects fail because of insufficient understanding of who the users will be, what their needs are, and how the product will be used. Without this, the design process is misdirected or even random. In evolution, it works really well to make random changes then watch and see which ones survive and thrive. In product development, however, few companies will be around for millions of years, so it pays to adopt a more efficient approach. According to Charles Darwin and Don Norman, this continues to be a case of survival of the fittest. One of the key factors determining survival is ability to meet the needs of customers. Since meeting the needs of customers will determine success of your product, why not find out what these needs are right at the beginning and design the product to meet them? Then there will be no more need for genetic mutations in your product or your programming staff.
9. Many risks and failures result from developing pieces of a product and throwing them over the wall. For example, requirements are gathered by a marketing person who writes a report and throws it over the wall to the developer who designs and implements the product, who then throws it over the wall to a quality assurance group to test.
The waterfall process has been recognized as deficient for years. It is surprising, however, how many companies continue to practice it anyway. It is easier to throw things over the wall than to communicate between teams, and easier to plan a sequential development process than an iterative one. UCD isn't the simplest approach around, but it is the best.
10. There is no easy way to decide how to trade off among product quality, cost, and time to market. Product managers address this challenge by focusing primarily on the product itself, trying to get the best possible product to market as quickly and cheaply as possible. This view of the product as the thing people buy has dominated the process of design and the business decisions that drive it.
UCD gives managers and designers another way to look at the challenge of product development. Product attributes such as cost, availability, and quality are, of course, important elements of a product. But instead of looking at the product per se, UCD compels you look instead at the total customer experience. Customers don't buy a product in isolation. The purchase is only a part of a sequence of events that include experiencing the product's advertising, the sales process, ordering, delivery/receipt, setting it up, learning to use it, getting support for it, and upgrading it. If a company does a poor job in any of these customer touch points, or if they are uncoordinated, a poor customer experience may result. Warehouses are littered with the remains of quality products that have failed because another aspect of the total customer experience fell down. Successful products and successful companies are those that emphasize the total customer experience. And UCD helps to measure, with the help of customers, which aspects of the total customer experience are important to them, and what about those aspects would compel them to purchase the product.
Software development isn't easy. It's not for the faint of heart, or for those with low tolerance of ambiguity. To do it well, you have to be both disciplined enough to plan intensely, and flexible enough to change your plans at a moment's notice. A waterfall process, inherently resistant to flexibility, simply does not meet the needs of software development. A process that is iterative, multidisciplinary, and adaptive is required to be able to meet the needs of users. UCD is that process. The future success of software development will rely on the further development and widespread adoption of UCD.
For additional information about UCD, read the book - User-Centered Design: An Integrated Approach.
Carol
Righi, Ph.D.
Biography
June 17, 2002
Email the editor with
any questions or comments.
