Enterprise Agile: Software Process Improvement (SPI)

Table of Contents

1. Overview
2. Process need assessment
3. Process tailoring
4. Process creation
5. Process deployment
6. Supporting project teams
7. Summary

1. Overview

The workflow diagram for the SPI discipline is depicted in Figure 1 and the activities, roles, and artifacts which comprise the SPI discipline are depicted in the amalgamated workflow diagram of Figure 2. Note that important message of this discipline is that in order for it to succeed your organization must be as agile as possible: it is possible for enterprise-level  professionals, including process engineers, to work in an agile manner, but they must choose to do so and be allowed to do so.

Figure 1. The Software Process Improvement Discipline workflow.

Figure 2. The amalgamated workflow diagram for the SPI discipline.

2. Process Need Assessment

It’s crucial to recognize that the primary drivers for process improvement are business-oriented: implementing a process for its own sake is a bad idea. Process improvement is an enabler, not an end in itself. You must understand the business goals of your organization in order to implement an effective information technology (IT) process. For example, the goal of always being first to market with new functionality will require a process with a different emphasis than that of minimizing defects in a highly regulated industry. The point is that you must start by determining your business needs.

You may find that your process-related needs are complicated and subtle: different business units typically have different priorities, particularly in mid-to-large size organizations. You must identify and prioritize these needs and then choose to address them accordingly. How you approach this will depend on many factors: Do you have global issues that need to be addressed: for example, improved quality, faster time to market, or consistency: or are individual business units needs paramount, such as the Research and Development area’s need for quick feedback? If the latter is the case, you may need a tailored process for R&D that emphasizes quick feedback rather than low defects. Important issues to consider:

• Process culture.
• Existing process expertise.
• Coordination across groups.
• Acceptance of change.
• Acceptance of an evolutionary (iterative and incremental) approach.
• Acceptance of an agile (evolutionary & highly collaborative) approach.
• Expertise in the (new) technologies.
• Expertise in the new techniques (e.g. use cases, agile testing, agile modeling, …).

3. Process Tailoring

It is very likely that you can tailor an existing process, such as the RUP or EUP, to meet your needs. If you examine the wide array of processes in the industry, you will most likely determine that you can tailor one (or more) to meet your assessed needs. Boehm and Turner present a risk-based approach for choosing a methodological strategy, which describes how to combine agile with rigorous/prescriptive approaches. They advise that uncertain requirements, complex technologies, and low employee turnover favor agile techniques, whereas larger teams and a diverse group of stakeholders favor more prescriptive methods.

There are many processes in the industry today, and they are at varying degrees of maturity and acceptance. Pick one (or more) that will help you achieve your goals. Remember that you will still need to tailor your adopted process. Rarely will a process, even a widely accepted one like the RUP, be a perfect fit for an organization. Choose what works for you, take out what doesn’t, and tweak the rest. Do you really think that you need all 3000+ pages of the RUP product? If a process makes sense to you, and you believe it will add value to your organization, then adopt it. Otherwise, do not. Do not implement processes just because someone else does. Process tailoring is best done in an iterative manner: tailor some, implement some, and then repeat. You can spend months or even years defining and tailoring the perfect process for your organization, but you won’t know how it will work until you try it. You’ll save time and effort (and therefore money) in the long run by taking an iterative approach.

4. Process Creation

It is possible that no existing process which meets your needs is available; this is very unlikely, given the wide array of processes available today. However, it is possible that you will discover that you need to create specific, detailed procedures and guidance that reflect your exact needs. If groups are unsure about some aspect of a standard process, such as iterative development, then they should try it out. People often get so set in our ways that it becomes difficult to contemplate other ways of doing our work. If you try a standard process, you may be pleased with the results of the test and decide that you can use it after all. If this fails and you need to create your own process, by:

• Identify the issue(s) you need to address. If you don’t understand the issue, you cannot develop an effective process for it.
• Look for existing process material similar to what you need. You’ll often discover that your “unique need” is not so unique after all.
• Engage the appropriate people. Identify people within your organization who are currently fulfilling this need and work with them to identify what they actually do. This will result in processes that people accept and that reflect your environment.
• Write less process documentation than you think you need. The hard truth is that very few people will actually read the process in its entirety. People usually read relevant portions of your process when they join your organization or when they’re doing performing an activity for the first time. After that, they may only read the appropriate guidance documentation and use your standard artifact templates, ignoring the rest of your documentation altogether.
• Take a just-in-time (JIT) approach. Writing the process(es) when you need them will help you to reduce your overall effort because you’ll discover that you don’t need as much as you think.

5. Process Deployment

After you have a defined process by either tailoring an existing one or by creating a new one, you need to implement it. In the cycle of defining/tailoring and implementing, you may identify the need to have slightly different variations of the process. For instance, you may find it beneficial to define different tailorings of the process for green field (new development) and commercial off the shelf (COTS) implementations. The article Process Tailoring and Deployment Strategies covers deployment issues in detail.

6. Supporting Project Teams

After a software process has been defined and implemented, it must be maintained, and teams must be assisted in order to apply it. Immediately after deploying a new process, you should spend considerable time adjusting it, as you adapt based on lessons learned. Running a pilot project or two will definitely help you to identify major issues, but there will always be a need for improvement. This is normal; expect it.

One way to identify “lessons learned” is to hold retrospectives. How often you hold them will depend on your comfort level with your process. I prefer to hold them at the end of each iteration so that the project team can tweak their process and learn from their experiences continuously. Some people will hold them at the end of a project, often referring to them as a project post-mortem, but I find this to not work very well in practice because the identified “lessons learned” often prove to be “lessons indicated” which are never acted on. The end is simply too late.

Kerth recommends that retrospectives focus on four simple questions:

1. What did we do well that, if we don’t discuss it, we might forget?
2. What did we learn?
3. What should we do differently next time?
4. What still puzzles us?

Remember that feedback is crucial to a process improvement effort. If something is not working, you need to know about it and adjust things as soon as possible. Put in a mechanism in place to allow people to honestly provide feedback on what’s going right and, more importantly, what’s not. Do not assume that no news is good news. Motivate people to provide feedback if you must; consider an approach where people can provide feedback anonymously if necessary. If you don’t find that teams are offering suggestions to change the process, you may want to take a closer look at what they are really doing. The teams might not know
enough yet to provide you meaningful feedback: or they might be finding ways to circumvent the process.

7. Summary

In this article, you learned that:

• SPI is an ongoing effort; your process will evolve over time.
• You need to invest in, and support, SPI efforts for them to be successful.
• The goal of SPI is to ensure that your organization can define, implement, and evolve one or more appropriate processes to help you meet your IT goals.
• There are many proven software processes; one or more of them will likely help you to meet your needs.
• Your software process must be tailored to reflect your organization’s strengths, weaknesses, culture, and business needs.