CS724 — Final Term Summary (Lectures 23–45)

📘 Lecture 23 — Introduction to TSP – I

📖 Overview: This lecture introduces the Team Software Process (TSP), a framework for guiding software engineering teams to achieve extraordinary work. It begins by examining why projects fail, primarily due to people problems rather than technical issues, and explores different team paradigms and the factors that create toxic team environments. The lecture then defines what a team truly is, emphasizing the importance of roles, interdependence, and social support, and concludes by outlining the characteristics of effective teams that TSP aims to foster.

🗂️ Topics Covered

This lecture covers the reasons for project failure, which are seldom technical but often related to people problems and excessive schedule pressure. It then introduces four organizational paradigms for software engineering teams: Closed, Random, Open, and Synchronous. The concept of a "jelled team" is introduced, followed by a discussion of factors that foster a toxic team environment. The lecture then formally introduces the Team Software Process (TSP), its objectives, and its origin at the SEI. The conditions for teamwork are explored through a formal definition of a team, highlighting the necessity of roles and interdependence. Finally, the lecture details the characteristics of effective teams, with a special focus on innovation and the importance of a trusting environment.

📝 Lecture Summary

Why Projects Fail

Projects seldom fail due to technical reasons. The primary causes are people problems and excessive schedule pressure. The solution lies in fostering good communication, building good teams, and managing pressure effectively. The best team structure depends on the organization’s management style, the number of people, their skill levels, and the overall problem difficulty. The performance of a team is inversely proportional to the amount of communication that must be conducted, and the length of time the team “lives together” affects team morale.

🔑 Definition — People Problems: Issues related to team dynamics, communication, and morale that are the primary cause of project failure, rather than technical hurdles.

Constantine suggests four organizational paradigms for software engineering teams.

Closed Paradigm

A closed paradigm structures a team along a traditional hierarchy of authority. Such teams can work well when producing software that is quite similar to past efforts, but they will be less likely to be innovative.

Random Paradigm

The random paradigm structures a team loosely and depends on the individual initiative of the team members. When innovation or technological breakthrough is required, teams following the random paradigm will excel. However, such teams may struggle when “orderly performance” is required.

Open Paradigm

The open paradigm attempts to structure a team to achieve some of the controls of the closed paradigm but also much of the innovation of the random paradigm. Work is performed collaboratively, with heavy communication and consensus-based decision making. Open paradigm teams are well suited to solving complex problems but may not perform as efficiently as other teams.

Synchronous Paradigm

The synchronous paradigm relies on the natural compartmentalization of a problem and organizes team members to work on pieces of the problem with little active communication. To achieve a high-performance team, team members must have trust, the distribution of skills must be appropriate, and mavericks may have to be excluded to maintain cohesiveness.

🔑 Definition — Jelled Team: A group of people so strongly knit that the whole is greater than the sum of the parts. Once a team begins to jell, the probability of success goes way up, and the team can become unstoppable.

Factors that Foster Toxic Team Environment

Several factors can create a toxic team environment. These include a frenzied work atmosphere where team members waste energy and lose focus. High frustration caused by personal, business, or technological factors causes friction. Fragmented or poorly coordinated procedures or a poorly defined process model can become a roadblock. An unclear definition of roles leads to a lack of accountability and finger-pointing. Finally, continuous and repeated exposure to failure leads to a loss of confidence and lower morale.

Team Software Process (TSP)

The Team Software Process (TSP) is an important step in software process improvement, developed after CMM and PSP at the Software Engineering Institute (SEI). The TSP provides a disciplined context for engineering work. The principal motivator for TSP was the conviction that engineering teams can do extraordinary work, but only if they are properly formed, trained, staffed, and led. The objective of TSP is to build and guide such teams. While some small products can be developed by individuals, the scale and complexity of modern systems requires teamwork. The effectiveness of the team largely determines the quality of the engineering.

💡 Why this matters: Unlike PSP which focuses on individual discipline, TSP scales these principles to a team level, providing a structured way to manage the complexities of collaborative software development.

Conditions for Teamwork

What is a Team?

A team is a group of people who share a common goal, are committed to it, and have a common working framework. The formal definition of a team includes four key parts:

• A team consists of at least two people.
• The members are working toward a common goal.
• Each person has a specific assigned role.
• Completion of the mission requires some form of dependency (interdependence) among the group members.

Team

All four parts of the definition are important. Roles provide a sense of ownership and belonging, guide team members on how to do their jobs, prevent conflicts and wasted effort, and provide members with a degree of control over their working environment. This sense of control is a fundamental requirement for motivated team members.

Interdependence

Interdependence means that each team member depends to some degree on the performance of the other members. It improves individual performance because members can help and support each other, such as in design teams.

🔑 Definition — Interdependence: A condition where team members rely on each other’s performance to achieve a common goal, fostering collaboration and support.

Social Support of Membership

Team performance is enhanced by the social support of membership. Because of the social context, members will generally make a special effort to meet their obligations to the rest of the team. Through mutual support and interdependence, teams become more than just the sum of their individual members. To be effective, teams must be properly skilled and work as cohesive units.

Characteristics of Effective Teams

Effective teams have several common characteristics. The members are skilled. The team’s goal is important, defined, visible, and realistic. The team’s resources are adequate for the job. The members are motivated and committed to meeting the goal. The members cooperate and support each other. The members are disciplined in their work. Finally, the members are innovative.

Innovation

Innovation is more than just thinking up ideas; it requires creativity and hard work. Innovative teams must have skilled, capable, and highly motivated people who are creative, flexible, and disciplined. They must strive to meet demanding schedules while controlling costs and informing management of their progress. To be innovative, engineering teams must work in a trusting and supportive environment. When managers do not trust their teams, engineers feel antagonized and lose their commitment. While it is appropriate for management to challenge teams with aggressive goals, they must be willing to negotiate realistic commitments that engineers believe they can meet. Few people will work diligently to meet a seemingly hopeless schedule. The TSP is designed to establish the conditions that characterize effective teams.

⭐ Key Takeaways

Project failure is almost always due to people problems, not technical ones, and can be mitigated by building effective teams. Effective teams work within a defined paradigm (Closed, Random, Open, or Synchronous), each suited to different types of tasks, and avoid the factors that create a toxic environment. The Team Software Process (TSP) is a disciplined framework from the SEI designed to form, guide, and lead high-performance engineering teams. A true team is defined by common goals, assigned roles, and member interdependence, which together foster ownership, social support, and enhanced performance. Finally, for an engineering team to be innovative and effective, it must operate in a trusting environment with challenging but realistic goals and schedules.

🧠 Quick Revision Questions

1. What are the primary reasons projects fail, according to this lecture?
2. Name and briefly describe the four organizational paradigms for software engineering teams suggested by Constantine.
3. What is a "jelled team," and what is its significance?
4. List the four key parts of the formal definition of a "team" as presented in this lecture.
5. According to the lecture, what is the principal motivator for the development of the Team Software Process (TSP)?

📘 Lecture 24 — Introduction to TSP – II

📖 Overview: This lecture continues the introduction to the Team Software Process (TSP), focusing on the principles of teambuilding and the operational processes needed for effective teamwork. It explains the TSP launch process in detail, covering how teams form, plan, and commit to a project, and it ties together the CMM/CMMI, PSP, and TSP for cohesive process improvement.

🗂️ Topics Covered

The lecture covers TSP teambuilding principles, the importance of defined operational processes, the principle elements of TSP including PSP and CMM/CMMI, the integration of these methods, and the detailed nine-meeting TSP launch process that guides teams from goal setting through to management review and postmortem.

📝 Lecture Summary

Teambuilding

The lecture begins by outlining the teambuilding principles used in the TSP to establish conditions for effective teams. These principles include establishing common goals and defined roles, developing an agreed-upon strategy, and defining a common process for work. All team members participate in producing the plan and know their personal role. The team negotiates the plan with management, who then reviews and accepts it. Team members do the job as planned, communicate freely, and form a cohesive, cooperative group committed to meeting the goal. Effective team formation requires that members truly understand their tasks, agree on how to do the job, and believe their plan is achievable. These conditions are established by involving engineers in producing their own plans. The TSP provides explicit guidance that organizations need to build effective engineering teams.

Team Operational Processes

To do disciplined work, engineers need what Deming calls operational processes. These are processes that define precisely how the work is to be done, more like a script designed for team members to use. The TSP provides a defined operational process to guide engineers and managers through the team-building steps. Without specific guidance, engineers must work out team-building details themselves, which wastes time and often produces poorly functioning teams. With a defined process and plan, engineers can be highly efficient. The TSP provides the operational processes needed to form engineering teams, establish an effective team environment, and guide teams in doing the work.

Principle Elements of TSP

Before team members can participate on a TSP team, they must know how to do disciplined work. Training in the Personal Software Process (PSP) is required to provide engineers with the knowledge and skills to use the TSP. PSP training includes making detailed plans, gathering and using process data, developing earned value plans, tracking projects, measuring quality, and defining operational processes. Engineers must be trained in these skills before they can follow the TSP process. The Capability Maturity Model (CMM) or Capability Maturity Model Integration (CMMI) provides the overall improvement framework, while the PSP provides the engineering disciplines needed for using a defined, planned, and measured process.

💡 Why this matters: The TSP is not a standalone method; it requires the foundation of PSP for individual discipline and CMM/CMMI for organizational maturity.

Tying Process Improvement Efforts

The TSP couples the principles of integrated product teams with the PSP and CMM/CMMI methods to produce effective teams. In essence, the CMM/CMMI and PSP provide the context and skills for effective engineering, while the TSP guides engineers in actually doing the work. Thus, the TSP capitalizes on the preparation provided by the PSP and CMM/CMMI, while also providing explicit guidance on how to do the work.

Back to TSP

While there are many ways to build teams, they all require individuals to work together to accomplish a demanding task. In the TSP, this demanding team-building task is a four-day planning process called the team launch. In a launch, all team members develop the strategy, process, and plan for their project. After completing the launch, the team follows its own defined process to do the job.

TSP Process

The TSP process includes launching a team project, development strategy, development plan, design process, implementation process, test plan, and postmortem.

Launching a Team Project

The process of forming and building a team does not happen by accident and it takes time. Teams need to establish working relationships, determine member roles, and agree on goals. The launch is the first step. Defining roles and agreeing on who will handle each role is an essential first step in team formation. In the TSP launch, teams make an overall plan and a detailed plan for about the next three to four months. After the team members complete the next project phase or cycle, they revise the overall plan if needed and make a new detailed plan.

The TSP Launch Process

Teams generally need professional guidance to properly complete the launch process. This guidance is provided by a trained launch coach who leads the team through the launch process. Unless teams are very experienced, they generally need the support of a trained launch coach.

🔑 Definition — Launch Coach: A trained professional who leads the team through the TSP launch process, providing essential guidance for unique team problems and issues.

The launch consists of nine meetings:

• Meeting 1: The team, team leader, and launch coach meet with senior management and marketing representatives. The objective is to inform all team members about the job, describe management's goals for the team, and convince the team members that management is relying on them.
• Meetings 2 through 8: The team, team leader, and coach meet with no observers or visitors. The team is led through steps designed to produce the conditions for effective teamwork.
• Meeting 2: The team documents its goals and selects team member roles. Standard TSP roles include team leader, customer interface manager, design manager, implementation manager, test manager, planning manager, process manager, quality manager, and support manager.
• Meetings 3 and 4: The team makes the overall project strategy and plan. Engineers produce a conceptual design, devise the development strategy, define the detailed process, and determine support tools. A list of products, along with size and time estimates per step, is generated. A schedule is created from these data.
• Meeting 5: Once they have an overall plan, engineers produce quality goals and plans. This defines quality actions and provides a measurable basis for tracking quality. Team members estimate defects injected and removed in each phase, and how many will remain for system test, acceptance testing, and final delivery.
• Meeting 6: Team members make detailed next-phase plans and review the entire workload to distribute tasks evenly. This results in a balanced team plan.
• Meeting 7: Engineers identify major project risks, rank them for likelihood and impact, assign a member to track each risk, and prepare a mitigation plan for the most significant risks.
• Meeting 8: After the plan is completed, the team prepares for the management review.
• Meeting 9: The team conducts the review with management. The team explains the plan, describes how it was produced, and demonstrates that all members agree and are committed. If the plan does not meet management's objectives, the team should present alternate plans showing what could be done with added resources or changes. The principal reason for showing alternate plans is to provide management with options.

At the end of the TSP launch, the team and management should agree on how to proceed. In the final postmortem step, the team reviews the launch process and submits Process Improvement Proposals (PIPs) on suggested improvements. The team also gathers and files launch data and materials for later use.

📌 Example: If a team's plan for a software project estimates 6 months and management needs it in 5 months, the team might prepare an alternate plan showing that with two additional developers, the schedule can be met. This gives management a concrete option for decision-making.

⭐ Key Takeaways

The TSP teambuilding principles rely on involving all engineers in planning to ensure commitment, strategy, and role definition. The TSP provides operational processes—scripts that guide engineers step-by-step—to avoid ad-hoc, inefficient team-building. The TSP launch is a structured nine-meeting process led by a coach, where teams establish goals, create a balanced plan, assess quality, identify risks, and negotiate with management. PSP training is required before engineers can use TSP, and TSP works in conjunction with CMM/CMMI to provide a complete improvement framework. The final postmortem and submission of PIPs are crucial for continuous process improvement.

🧠 Quick Revision Questions

1. What are the three key conditions for effective team formation according to the TSP teambuilding principles?
2. Why is a defined operational process important for engineering teams, and what happens without it?
3. What is the purpose of the TSP team launch, and how long does it typically last?
4. In which TSP launch meeting is the "balanced team plan" created, and what does it ensure?
5. What is the purpose of the postmortem step at the end of the TSP launch process?

📘 Lecture 25 — Introduction to TSP – III

📖 Overview: This lecture covers the core TSP team working process, focusing on how a team operates after launch. It details the responsibilities of leadership, process discipline, tracking, communication, and quality management, along with guidance on goal-setting, development strategy, planning, and project postmortems. This lecture is critical for understanding how to maintain a disciplined and effective software team throughout a project.

🗂️ Topics Covered

This lecture addresses the TSP team working process, including leading the team, maintaining process discipline, tracking issues, ensuring communication and feedback, rebalancing workload, and relaunching projects. It also covers TSP quality management, goal-setting (team goals, considerations, and steps), the TSP process itself (including development strategy and risk management), the development plan, key TSP design decisions, and the importance of the postmortem.

📝 Lecture Summary

The TSP Team working Process

Once the TSP team is launched, the principal need is to ensure all team members follow the plan. This includes leading the team, maintaining process discipline, tracking issues, communication, management reporting, maintaining the plan, estimating project completion, rebalancing team workload, relaunching the project, and TSP quality management.

Leading the Team

The team leader is responsible for guiding and motivating team members, handling customer issues, and dealing with management. This includes day-to-day direction, protecting team resources, resolving team issues, conducting meetings, and reporting on work. The leader’s principal responsibility is to maintain the team’s motivation and energy and ensure it is fully effective. Productive teamwork requires a combination of specific goals, a supportive working environment, and capable coaching and leadership. TSP provides the supportive environment, and the instructor provides the coaching. Without the precise guidance of the TSP, a team could waste considerable time defining its own practices, methods, and roles.

Process Discipline

One key leadership responsibility is maintaining process discipline, where the team leader ensures engineers do the job the way they planned to do it. Almost every project faces heavy schedule and resource pressure, so there is always a temptation to cut corners. However, when teams stop following their defined processes, they have no way to tell what they are supposed to do or where they stand on the job.

Tracking Issues

Another important team leader responsibility is ensuring that all issues team members identify are managed and tracked. This includes task hours, earned value status, milestone status, and quality status.

Communication and Feedback

Learning is most effective when you follow a defined process and get rapid feedback. The TSP scripts and forms provide a defined, measured, and repeatable framework for team software engineering. TSP provides rapid performance feedback because the team produces the product in several short development cycles and evaluates results after each cycle. The team leader is responsible for maintaining open and effective team communication. When team members do not know the project’s status, understand what their teammates are doing, or know what challenges lie ahead, it is hard for them to stay motivated. Communication is a key part of maintaining the team’s energy and drive.

Rebalancing Team Workload

Unbalanced workload can cause a team to be inefficient. This occurs when some engineers have much more work than others.

Re-launching the Project

In TSP, teams periodically replan their work. This is necessary because it is difficult to make detailed plans that run for more than a few months. The relaunch is generally scheduled as part of the project and conducted whenever the team feels it would be helpful.

TSP Quality Management

To manage quality, teams must establish quality measures, set quality goals, establish plans to meet these goals, measure progress against the plans, and take remedial action when goals are not met. The elements of TSP quality management are making a quality plan, identifying quality problems, and finding and preventing quality problems.

TSP Goals and Goal-Setting

Goal setting is an essential step in team formation and should usually be done at the start of each project. Establishing goals is simple in concept but often difficult in practice. Goals should be precisely measured.

Goal-Setting Considerations: Teams should not be measured on whether or not they actually meet their goals. Instead, they should be evaluated on their willingness to set measurable and aggressive goals and on their efforts to meet them. Too much emphasis on meeting goals is counterproductive because it motivates teams to set goals they are sure they can meet.

Goal-Setting Steps:

1. Write down the goals you wish to use instead.
2. Specify how to measure these goals.
3. Describe why you have selected these goals instead of the ones provided by the TSP.
4. Give a copy of your revised goals to the team and to the instructor.
5. Have a team leader put a copy of your goals in the project notebook.

Goal Setting for TSP: The primary goals are to produce a quality product, run a productive and well-managed project, and finish on time.

TSP Process

This includes launching a team project, development strategy, development plan, design process, implementation process, test plan, and postmortem.

The Development Strategy

One principal problem in software development is figuring out how to build big systems. The steps in developing a strategy are:

1. Define the strategy criteria.
2. Determine the possible alternative strategies.
3. Identify the risks and benefits of each alternative strategy.
4. Make a comparative evaluation of these alternatives.
5. Make the strategic decision.
6. Document the selected strategy.

To develop a strategy, ask four questions:

• Based on what I know, how would I build this product?
• What are the principal components I will need to build this product?
• What functions must these components provide?
• How big do I think these components will be?

Some Risks: You may encounter functions you don’t know how to design, run into support system problems, have a product too defective for testing, lose control of the product, or have a team that cannot work effectively.

Managing Risks:

• Too large a product: Start with a small kernel product and add functions in later cycles.
• Difficult or complex functions: Prototype these functions at the beginning of the project.
• Support system problems: Build an early prototype to ensure you know how the system works.
• Testing time: If you consistently follow TSP and PSP methods, your product will have few defects.
• Product control: TSP addresses configuration management at the beginning of the project.
• Teamwork problems: Discuss problems openly; if they persist, get help from the instructor.

The Development Plan

This section covers why to make plans, creating balanced plans, tracking progress against the plan, planning in detail, handling unplanned tasks, estimating levels, and implementation planning.

The TSP Design Decisions

TSP design decisions are to provide a simple framework that builds on the foundation of the Personal Software Process (PSP), develop products in several cycles, establish standard measures for quality and performance, provide precise measures for teams and students, use role and team evaluations, require process discipline, and provide guidance on teamwork.

The Postmortem

The postmortem is a critical part of the process. It addresses why we need a postmortem, what a postmortem can do for you, and the process improvement proposal.

⭐ Key Takeaways

The most critical points from this lecture are that a successful TSP team requires a leader who actively maintains motivation, process discipline, and open communication. Effective goal-setting focuses on setting aggressive, measurable goals rather than strictly achieving them. To manage risks like size and complexity, teams should use incremental development and prototyping. Finally, the postmortem is essential for continuous process improvement, as it allows the team to learn from its experiences and propose changes for future projects.

🧠 Quick Revision Questions

1. What are the three principal responsibilities of a TSP team leader?
2. Why is it counterproductive to evaluate a team solely on whether it meets its stated goals?
3. What is the primary risk associated with a "too large product" and what is the recommended TSP strategy to manage it?
4. What four questions should a team ask when developing its development strategy?
5. Why is a postmortem an essential part of the TSP process?

📘 Lecture 26 — TSP Body of Knowledge

📖 Overview: This lecture introduces the Team Software Process (TSP) Body of Knowledge (BOK) developed by the SEI. It defines the fundamental knowledge and skills that distinguish TSP-trained professionals and provides a structured framework for practitioners, employers, and academic institutions. The lecture systematically covers the six core competency areas and their associated knowledge areas that form the foundation of TSP mastery.

🗂️ Topics Covered

The lecture begins by defining key terminology related to the body of knowledge, including competency area, knowledge area, concept, and skill. It then introduces the six TSP BOK competency areas: TSP Foundations and Fundamentals, Team Foundations, Project Planning with TSP, Project Implementation and Tracking with TSP, Gathering and Using TSP Data, and Scaling Up the TSP. Each competency area is broken down into its constituent knowledge areas with detailed explanations of their content and purpose.

📝 Lecture Summary

TSP Body of Knowledge

The SEI has drafted the Team Software Process Body of Knowledge (TSP BOK) document, which defines the fundamental knowledge and skills that set TSP-trained individuals apart from other software professionals. It helps individual practitioners assess and improve their own skills, provides employers with an objective baseline for assessing the process improvement skills and capabilities of their development team members, and guides academic institutions that want to incorporate TSP into their software engineering curriculum. The TSP body of knowledge is composed of six competency areas, each with several knowledge areas.

🔑 Definition — Competency Area: A group of closely-related knowledge areas that a practitioner is well qualified to perform intellectually or physically. 🔑 Definition — Knowledge Area: The sum or range of specific understanding and ability gained through study of a set of concepts or through experience with a set of skills. 🔑 Definition — Concept: An explanatory principle applicable to a specific instance or occurrence within a particular knowledge area. 🔑 Definition — Skill: Proficiency, facility, or dexterity of performance that is acquired or developed through training or experience in a particular knowledge area.

TSP BOK Competence Areas

The six TSP BOK competency areas are: TSP Foundations and Fundamentals, Team Foundations, Project Planning with TSP, Project Implementation and Tracking with TSP, Gathering and Using TSP Data, and Scaling Up the TSP. These six areas collectively define the full scope of knowledge required for TSP mastery.

TSP Foundations and Fundamentals

This competency area outlines the foundational knowledge on which TSP is built and describes the fundamental concepts that a TSP practitioner must understand to successfully implement and practice the TSP methodology.

The knowledge areas composing this competency area are:

• Knowledge Work: PSP and TSP are practices designed to facilitate and improve both the process and the outputs of knowledge work, which is the interpretation, development, and implementation of information by skilled professionals within a specific subject area. This knowledge area discusses the nature of knowledge work and the team and workplace characteristics required for such work.
• TSP Prerequisite Knowledge: This outlines the fundamental concepts and skills that individuals must master before implementing the TSP methodology as a member of a TSP team. Although this area calls out some of the specific Knowledge Areas of the PSP BOK, the PSP BOK in its entirety is considered to be prerequisite knowledge for implementing the TSP in practice.
• TSP Principles: This outlines the basic principles underlying the Team Software Process. The key concepts identify the elements that are common to and required for successful outcomes of work done by teams to produce software products and/or software-intensive systems.
• TSP Process Elements and Measures: This describes the process elements and measures used in the TSP. Where applicable, overlaps with or differences from PSP process elements and measures are noted.
• TSP Quality Practices: This describes the specific quality practices added in the TSP to build on the individual quality practices used by PSP practitioners.

Team Foundations

A team consists of a group of people who act in cooperation to achieve a common purpose. When teams are effective in achieving their goals, it is because they are composed of members with complementary skills that work together to create a synergistic effort; effective teams achieve a kind of gestalt, in which the members' strengths are maximized and the weaknesses minimized so that the team as a whole becomes greater than the sum of its parts. Teams are especially appropriate for doing work of a highly complex nature (such as knowledge work) and for accomplishing large-scale tasks with many interdependent subtasks.

The knowledge areas are:

• Teams and Teambuilding: This describes the characteristics of teams and explains concepts for building high-performing project teams.
• Team Types, Styles, and Dynamics: This describes some of the models of team types, team styles, and team dynamics that provide a foundational understanding of team work in the TSP.
• Team Formation and Membership: This describes important parameters that should be considered when forming and populating TSP teams.
• Team Member Responsibilities: This describes the team's specific responsibilities for helping teams to be fully effective.
• Team Member Roles: This discusses the types of roles on TSP teams and delineates the responsibilities required of the various team member roles.
• Team Leader Role: This lists the roles that a TSP team leader must fulfill and discusses some of the tasks and responsibilities that accompany the various roles.
• Coach Role: This describes the general roles and responsibilities of the TSP coach.

Project Planning with TSP

Project planning with TSP begins when an organization makes the decision to implement the new technology using one or more small pilot projects. The organization works with a certified TSP coach to choose pilot teams and to provide them with training in PSP or process improvement techniques and in TSP project planning and tracking methodologies. Once trained, the team plans the project work by participating in a TSP launch.

The TSP launch process is actually a series of teambuilding activities that are organized into ten meetings. The TSP coach works with the team's members, leader, and managers to form an understanding of the project requirements, determine team goals, select team member roles, and make plans for producing the product. The team presents its plan to management and after receiving management acceptance for the plan, or an alternative plan, begins to work on the project. The knowledge areas in this competency area describe the requirements, activities, and guideposts for implementing pilot TSP projects and launching TSP teams.

The knowledge areas are:

• Change Management Fundamentals: This describes some of the fundamental concepts of change management and technology introduction that can help to facilitate the introduction of new technologies, processes, and practices into an organization.
• Piloting TSP in an Organization: This describes the general requirements and guidelines for initiating and conducting TSP pilot projects.
• Preparing Management and Teams for TSP Implementation: This describes the training and pre-launch preparations required to prepare executives, managers, team leaders, and team members for effective participation in TSP implementations.
• The TSP Launch Meetings: This provides an overview of the TSP launch and a description of each of the meetings that make up a TSP launch.
• The TSP Relaunch: The team relaunch is nearly the same as the launch except that it is done by a team that has already completed an initial launch of the same project. This knowledge area describes how the relaunch differs from the launch, explains when and how to conduct a relaunch, and delineates the inputs and outputs for the relaunch.

Project Implementation and Tracking with TSP

Following a TSP team launch or relaunch, the team members must work to implement their plan by completing the tasks which they have been assigned. The TSP does not mandate the use of any particular approach to project implementation; rather, it stipulates that team members should all use PSP (for engineering work) or best professional practices (for other types of work done in support of the project) to enable team members to produce high-quality product components or project elements.

TSP principles also require the team members to observe process discipline by following the processes and strategies that the team agreed to use; by requiring that the team members all record their data accurately, honestly, and in real time; and that the team members individually and collectively maintain a constant focus on quality. Faithful and precise data collection enables teams to monitor their progress against the plan. Up-to-date and accurate data can be analyzed to enable team members to identify potential problems with project schedule or product quality earlier in the process, rather than later. This allows teams to address these issues, fix the problems, or seek help before small problems become big problems that can derail the project. Using data for status tracking also provides the information necessary to generate reports for its stakeholders on the team's progress, and to advise management of potential problems so that they can provide assistance or additional resources to put the team back on track.

The knowledge areas are:

• Weekly Meetings: The weekly team meeting is one of the elements that differentiates TSP from typical software projects. It keeps the team members focused on the project work and allows regular opportunities for teamworking and team building.
• Checkpoints: Checkpoints are a series of meetings between the team leader, team members, and coach that allow everyone to discuss issues or problems such as process implementation, data gathering, or data analysis, and to generate solutions for any identified problems.
• Communicating with Stakeholders: Management is the primary stakeholder of the TSP team, although there may also be internal customers, external customers, or both. In addition, the team members themselves are stakeholders in the process and its outcome. Whoever the stakeholders may be, it is important that the TSP team communicates frequently with them and reports useful and timely information about status, schedule, and product quality.
• Replanning: Plans are subject to frequent change, particularly on new or inexperienced TSP teams. When plans change frequently it is easy for the team members to lose track of the overall team plan or their personal roles in meeting the plan.
• Phase, Cycle, and Project Postmortems: To ensure that the team stays on track and maintains its motivation, it may be desirable to hold periodic short replanning meetings, in which the team leader reviews the current management priorities, and the team members summarize their status and adjust their individual and team plans as needed to address those priorities. The postmortem analyzes the collected data on schedule, task hours, estimation, quality, and process. The results are used to provide personal, team and organizational planning data for use in an upcoming launch or relaunch and to identify improvements to process elements. The focus is on using data for planning and process improvement.

Gathering and Using TSP Data

In the same way that PSP relies on individual data, the TSP methods rely on data collection and analysis to enable teams to understand what they do and how they do it, thereby enabling teams to identify effective procedures that should continue to be used and to pinpoint those areas in which improvements are needed. Data provide a baseline for making estimates and plans, tracking progress against the plan, and measuring the effects of changes implemented as part of a process improvement effort. Therefore, it is critical to the successful implementation of TSP that teams collect data as they work, for use in later analyses and estimations.

The knowledge areas are:

• Data Recording: Data are used to provide teams with a baseline for making estimates and plans, tracking progress against the plan, and measuring the effects of changes implemented as part of a process improvement effort. Therefore, it is critical to the successful implementation of TSP that teams collect data accurately and in a timely manner as the project work progresses.
• Gathering and Using Size Data: Size and time are often correlated, and when they are, size estimates can be used to estimate effort and then create plans based on the size and effort estimates. Size data are also useful for tracking development effort and assessing product quality (when defect data are normalized based on size).
• Gathering and Using Schedule Data: Schedule data are used to predict the project's likely completion date and to track actual progress against the planned schedule.
• Gathering and Using Quality Data: The primary focus of the TSP is producing a high quality product, and it is required to learn the principles, measures, tools, and techniques for using data to manage quality.
• Gathering and Analyzing Postmortem Data: The postmortem provides the team with an opportunity to learn from its work to improve their product quality, design practices, planning and tracking, and teamwork.

By gathering, compiling, and analyzing the available data at the end of a phase, cycle or project when it can most economically, rapidly, and accurately be obtained, the team members can learn from their strengths and weaknesses and capitalize on this learning when they start another phase, cycle or project. This knowledge area discusses the types of data that are gathered and analyzed, identifies the various team member responsibilities for gathering and preparing data for the postmortem, explains how data are used in the postmortem meeting to identify strengths and areas for improvement, and lists the data that should be captured in the postmortem report.

Scaling Up the TSP

Successful introduction of TSP into an organization begins with one or two small-scale pilot projects. Pilot projects provide the management support and compelling evidence needed to convince the organization's general population that the TSP methods are effective and will be beneficial to the organization. If the pilot projects are successful, management often decides to implement TSP in one or more divisions or organizations. As with the introduction of the pilot projects, special considerations must be addressed to increase the likelihood that organization-wide implementation will be successful.

Whether or not organizations choose to implement TSP throughout the company, they all have unique needs that may require some tailoring of the TSP applications. This is particularly true if the TSP project team requires more than 15 to 20 members, if the members of the team have different professional capabilities or specialties that must work together to produce the product, or if some of the team members work at locations apart from most of the team. The knowledge areas in this competency area describe the activities of scaling up the TSP for entire organizations or very large TSP project teams, and the adaptations to the basic TSP process that may be needed to address the needs of specialized TSP teams.

The knowledge areas are:

• Organizational Implementation: This describes the process of scaling the TSP implementation up from use on a few pilot projects to full introduction of TSP across the organization.
• TSP Process Variations: In development work, teams typically are classified as either project teams or functional teams. A project team is one that is formed to accomplish a specific project objective and, when that objective has been completed, the team is either disbanded or given another project assignment. A functional team is one that has a continuing mission responsibility. Additional variations in team type are due to team size or physical location. TSP can be adapted to fit the needs of functional teams (TSPf), integrated project teams (TSPI), distributed teams (TSPd), multiple TSP teams working in tandem (TSPm), TSP teams that also use CMMI (TSP+), and academic (student) teams (TSPi).
• Large-scale TSP Teams: This describes the characteristics and considerations unique to large-scale TSP teams.

⭐ Key Takeaways

The TSP Body of Knowledge is structured into six distinct competency areas, each containing several knowledge areas that define the complete skill set required for TSP practitioners. The foundation begins with understanding the nature of knowledge work, TSP principles, quality practices, and mastering PSP as prerequisite knowledge. Team dynamics are critical, as effective teams achieve a gestalt where collective strengths are maximized and weaknesses minimized through proper formation, roles, and leadership. Data collection and analysis are fundamental to TSP success, enabling accurate planning, tracking, quality management, and postmortem learning. The scaling of TSP from small pilot projects to organization-wide implementation requires careful change management and process variations (TSPf, TSPI, TSPd, TSPm, TSP+, TSPi) to accommodate different team sizes, types, and locations.

🧠 Quick Revision Questions

1. What are the six competency areas that compose the TSP Body of Knowledge?
2. How does the TSP define a competency area and how does it differ from a knowledge area?
3. What is the role of pilot projects in the TSP implementation process, and how do they relate to scaling up the TSP?
4. What is a team gestalt and why is it important for effective TSP teams?
5. What are the five main process variations of TSP (TSPf, TSPI, TSPd, TSPm, TSP+) and what type of team does each serve?

📘 Lecture 27 — Software Process Improvement Using PDCA

📖 Overview: This lecture introduces the PDCA (Plan-Do-Check-Act) cycle, also known as the Shewart cycle, as a structured and cyclic model for implementing software process improvement (SPI) programs in organizations. It explains the four phases in detail and presents a spiral model that combines PDCA with the classic spiral process model, illustrating a staged progression from initial project-level adoption to widespread organizational use. The lecture emphasizes critical aspects such as planning for risks, structuring for success by identifying key people, measuring success, and leveraging achievements to ensure lasting improvement.

🗂️ Topics Covered

The lecture covers the Shewart/PDCA cycle and its combination with the spiral model for process improvement adoption across five cycles (from first-time practice usage to optimization). It then details the Plan quadrant, including identifying risks, business planning, core-competence planning, and management commitment influencers. Next, it examines the Do quadrant, focusing on structuring for success and the roles of key people like project managers, champions, sponsors, opinion leaders, and infrastructure members. The Check quadrant discusses measuring success using primitive metrics, and the Act quadrant covers leveraging success through revision, spreading improved processes, and summarizing the best way to extend adoption.

📝 Lecture Summary

Software Process Improvement Using PDCA

There are a number of models for instituting software process improvement (SPI) programs in organizations. All these models have to be incorporated in an orderly and cyclic manner, and these introductions cannot and must not be made abruptly.

Shewart cycle

The Shewart cycle is one such way of introducing a software process improvement program in an organization. It was popularized by W. Edwards Deming. The Shewart cycle has four steps: Plan, Do, Check, Act (PDCA). The PDCA model can be combined with the famous spiral process model, specially adapted for process improvements, resulting in a new spiral process improvement model for software.

Cycle I: SPI Spiral Model

The project plans a new practice (Plan), tries the new practice (Do), achieves success (Check), and then describes the practice as part of that success (Act).

Cycle II: SPI Spiral Model

Other projects plan to use the practice (Plan), with minimal new documentation added (Do). Other successes are linked to the practice (Check), and a decision for "organization-wide" use is made, with successes told widely (Act).

Cycle III: SPI Spiral Model

People are assigned to document, train, and support (Plan). A training manual, process support by staff, and initial process metrics are created (Do). More projects use the practice with few hard problems, and initial measurable results emerge (Check). Procedures are revised, expected results are quantified, and management refers to the practice as a "standard" (Act).

Cycle IV: SPI Spiral Model

All projects plan to use the practice, with urgent requests for training and improvements planned (Plan). Support processes are standardized, initial automation is implemented, and metrics are used to monitor effectiveness (Do). Most projects use the practice, which is evaluated against measurable goals, and most people are convinced to value it (Check). Procedures are revised, ongoing responsibility for effective use is assigned, and management reviews effectiveness (Act).

Cycle V: SPI Spiral Model

Plans are made to optimize the infrastructure (Plan). The number of infrastructure personnel is reduced to optimal ongoing levels (Do). Metrics are monitored to ensure no loss of benefits (Check). Effectiveness is reviewed, and the process is tuned (Act).

The figure shows a progression from first-time practice usage (in the center) to widespread organizational adoption. The model includes several patterns across the plan/do/check/act quadrants, reflecting increasing people and resource investments, greater management understanding, attention, and commitment, and better control with process metrics.

Plan

Most new improvements are first tried by one project team. If successful, these improvements are spread to other projects. As usage spreads, planning aspects include ways to provide better documentation, training, and support. Eventually, demand outpaces the ability of the people helping new adopters, and plans for an ongoing infrastructure are developed and tuned.

Plan: Identifying Risks and Deciding What to Do

While PDCA always starts with some form of commitment, the nature and strength of that commitment is constantly challenged as plans solidify. Business planning includes changes that make good business sense and link to your goals. You need to plan the scope, timing, cost, and payback of each change. Without such scoping, you have no basis for comparing proposed process-improvement projects to any other business proposals. You must understand your organization's readiness and enthusiasm for each proposed change. Your culture, your existing processes, and your people's underlying beliefs all affect changes. Focus on developing core-competence is extremely important for organizations in the software business. Core-competence planning is just as important as business planning in setting an organization's long-term strategy. The key results of core-competence planning are a set of clearly stated goals and action plans with clear links to business needs. An investment model of your software processes will help you to evaluate trade-offs. No matter how well you analyze the business urgency of a particular process improvement and its investment value, your organization might just not be ready for it. A good subjective assessment should give you some sense of how widespread feelings are about key issue areas and how motivated people are to change.

A table shows Management Commitment Influencers which include Business (Strategic Vision, Business Approach, Core Competence) and Tactical (Customer Perceptions, Market Share, Product Cycle Time, Profitability) aspects, and Organizational (Organizational Maturity, Process Improvement Infrastructure) and Tactical (Organizational Inertia, Stability, Cost/Time Alignment) aspects. When these are not positive, they can distract and undermine your manager's best intentions. Strategic business aspects most reflect the leadership of your organization's management. Tactical business aspects are a constant source of tension, tied to commonly accepted evaluations of business health. Strategic organizational aspects can be the strongest negative influencers. Tactical organizational aspects are the most controllable from lower in the organization. A strong selection process with a well-thought-out balance of tactical and strategic projects with clearly defined goals shows people your rationale and helps to motivate them toward successful change.

Do

Progress in this quadrant parallels the stages in the Plan quadrant. Adoption by second-ring projects starts a trend to improve and adapt documentation and training. By the third ring, users have formed realistic expectations for the new process and use metrics. In the outermost two rings, good support is common, automation improves process efficiency and quality, and the number of infrastructure personnel is finally reduced to optimal ongoing levels.

Do: Structuring for Success

Software managers often respond to proposed process-improvement projects with skepticism. One of the most powerful approaches is to work with people to mentally shift to a future desired state. The spiral model shows that improvements progress by stages. One aspect is the key people in an organization that require special attention.

Project Managers play a key role by ensuring that all involved people work together productively. A project manager is generally also the project's champion. A sponsor is a person in a higher organizational position who ensures funding and high-level public enthusiasm. Sometimes there are less obvious technical leaders, called opinion leaders. Before other people will consider a change, they look to whether opinion leaders accept it. When people first work with a new process, ready access to infrastructure members to answer questions and remove roadblocks is invaluable. The single most important step you can take is to create and manage from a good process-improvement project plan. A set of storyboard pictures or maps of where you plan to go is also a very useful way to start a project plan. A typical storyboard outline includes sections for Situation, Challenges, Plan (Project Selection Criteria, Planned Changes, Objectives, Cost/Benefit Analysis, Project Milestones), Do (Methods for Measuring and Validating), Check (Approach for Standardizing Process Changes), and Act.

🔑 Storyboard: A visual outline or map used to organize thoughts for a process-improvement project plan, often used to create a slideset that evolves over the adoption of the process improvement.

Check

After the first ring project team uses a new practice, they describe their "success" in terms matching whatever measurements they took. As other projects adopt the new process in later spiral rings, improvement metrics become more precise until process performance becomes well understood and is measured regularly to ensure optimum performance is maintained.

Check: Measuring Success

If you started with a storyboard that included the Check part, you are probably in good shape. The amount of effort needed to set up measures depends on what measures already exist. If you do not measure these four primitive metrics, it will be difficult to frame expectations and even more difficult to describe success.

• Non-Comment Source Statements (NCSS)
• Engineering Months (EM)
• Defects
• Calendar Months (CM)

🔑 Primitive Metrics (NCSS, EM, Defects, CM): Four fundamental measurements (Non-Comment Source Statements, Engineering Months, Defects, Calendar Months) that are essential for framing expectations and describing success for a software process improvement project.

Act

The speed at which adoptions of a new practice occur will heavily depend on how credible early adopters are, how good a job they did of measuring their processes, how well they can communicate their success, and how well their improvement matches the organization's readiness. Gaining ever-increasing management commitment as you move outward on these segments suggests the critical elements of follow-through and "sales" that are necessary for process improvements.

Act: Leveraging Success

The essence of the "Act" step is to revise and spread improved processes. This is done by describing successes so different audiences understand and get excited. No process should ever be "final". This means we must model improvements on previous successes, create similar situations and environments, and reinforce behaviors that move people as quickly as possible to improved conditions. The best way to summarize what is needed to extend best-practice adoption and effective usage outward is:

• Proactively identify and seek support of process-improvement champions and sponsors.
• Reinforce management awareness and commitment with a strong business case.
• Build an infrastructure strong enough to achieve and hold software core competence.
• Measure the extent of adoption of each desired process improvement until it is operating effectively, efficiently, and across all appropriate parts of the organization.

Conclusions

Use the PDCA model to help you make sure you have clear goals and plans for what you want at the end of each of multiple spirals, understand how much you can expect to successfully do in stages, measure progress and results that emphasize your incremental successes, and evaluate management and organizational commitment influencers and regularly reenlist and reinforce support on your continued exciting journey.

⭐ Key Takeaways

The PDCA (or Shewart) cycle is a structured, four-phase approach to introducing software process improvements: Plan (identify risks and decide), Do (train, adapt, and structure for success), Check (evaluate results and measure success using primitive metrics like NCSS, EM, Defects, and CM), and Act (revise, spread, and leverage success). The spiral model combines PDCA with a progressive, cyclic framework, moving from a single project trying a new practice in the center to widespread organizational adoption and optimization at the outer rings, reflecting increasing investment, management commitment, and process control. Success in the "plan" phase requires careful risk identification, business and core-competence planning, and evaluation of management commitment influencers, while the "do" phase hinges on identifying and leveraging key people (project managers, champions, sponsors, opinion leaders, infrastructure members) and using storyboards for effective project planning. The end goal is to shift thinking from solely tactical process-improvement projects to strategically oriented organizational capabilities that evolve key abilities from individual knowledge to robust organizational infrastructure.

🧠 Quick Revision Questions

1. What are the four steps of the Shewart (PDCA) cycle, and what is the primary action that must be taken in each step according to the lecture?
2. Describe the progression of adoption from Cycle I to Cycle V in the spiral model for process-improvement adoption.
3. List the four primitive metrics (the minimum measurements) required to frame expectations and describe success for a process improvement project.
4. What are the four types of key people identified in the "Do: Structuring for Success" section, and what is the primary role of a "sponsor" compared to a "champion"?
5. Name the twelve management commitment influencers listed in the table, and categorize them into the four main groups (Business/Strategic, Business/Tactical, Organizational/Strategic, Organizational/Tactical).

📘 Lecture 28 — Software Process Improvement Using SEI’s IDEAL Model

📖 Overview: This lecture introduces the IDEAL model, a structured organizational improvement framework developed by the SEI. It serves as a roadmap for initiating, planning, and implementing improvement actions in software process improvement. The model provides a disciplined engineering approach for continuous improvement, focusing on managing the improvement program and establishing a long-term improvement strategy.

🗂️ Topics Covered

The lecture covers the IDEAL model's five phases: Initiating, Diagnosing, Establishing, Acting, and Learning. For each phase, detailed activities are explained. The initiating phase covers stimulus for change, setting context, building sponsorship, and chartering infrastructure. The diagnosing phase covers characterizing current and desired states and developing recommendations. The establishing phase covers setting priorities, developing approach, and planning actions. The acting phase covers creating solution, piloting/testing, refining, and implementing solution. The learning phase covers analyzing/validating and proposing future actions. Useful tips on SPI are also provided.

📝 Lecture Summary

Introduction

The IDEAL model is an organizational improvement model that serves as a roadmap for initiating, planning, and implementing improvement actions. It is named for the five phases it describes: initiating, diagnosing, establishing, acting, and learning. As originally conceived, it was a life-cycle model for software process improvement based upon the Capability Maturity Model® (CMM®) for Software, but the SEI has revised it for broader application. The model provides a usable, understandable approach to continuous improvement by outlining the steps necessary to establish a successful improvement program.

💡 Why this matters: Following the phases, activities, and principles of the IDEAL model has proven beneficial in many improvement efforts, providing a disciplined engineering approach that focuses on managing the improvement program and establishes the foundation for a long-term improvement strategy.

🔑 Definition — IDEAL Model: An organizational improvement model that serves as a roadmap for initiating, planning, and implementing improvement actions, named for its five phases: Initiating, Diagnosing, Establishing, Acting, and Learning.

The Initiating Phase

Critical groundwork is completed during the initiating phase. The business reasons for undertaking the effort are clearly articulated. The effort's contributions to business goals and objectives are identified, as are its relationships with the organization's other work. The support of critical managers is secured, and resources are allocated on an order-of-magnitude basis. Finally, an infrastructure for managing implementation details is put in place.

💡 Why this matters: If the initiating phase activities are done completely and well, subsequent activities can proceed with minimal disruption. If done poorly, incompletely, or haphazardly, then time, effort, and resources will be wasted in subsequent phases.

Stimulus for Change: It is important to recognize the business reasons for changing an organization's practices. The stimulus for change could be unanticipated events or circumstances, an edict from someone higher up in the organization, or the information gained from benchmarking activities as part of a continuous improvement approach. Whatever the stimulus, it can have far-reaching influence on the effort's visibility, conduct, and ultimate success. Change for the sake of change rarely results in significant improvement. When the business reasons for change are more evident, there is greater buyin throughout the organization and greater chances for success.

Set Context: Once the reasons for initiating change have been clearly identified, the organization's management can set the context for the work. "Setting context" means being very clear about where this effort fits within the organization's business strategy. Questions to answer include: What specific business goals and objectives will be realized or supported by this change? How will it affect other initiatives and ongoing work? What benefits (such as return on investment or improved capabilities and morale) will result? Context and implications often become more evident as the effort proceeds, but it is important to be as clear as possible regarding these issues early in the effort.

Build Sponsorship: Effective sponsorship is one of the most important factors for improvement efforts. It is necessary to maintain sponsorship levels throughout an improvement effort, but because of the uncertainty and chaos facing the organization in the beginning, it is especially important to build critical management support early in the process. The commitment of essential resources is an important element of sponsorship, but effective sponsors often do much more than this. Sponsors can be most effective if they give personal attention to the effort and stick with it through difficult times.

Charter Infrastructure: Once the reason for the change and the context are understood and key sponsors are committed, the organization must set up a mechanism for managing the implementation details. The infrastructure may be temporary or permanent, and its size and complexity may vary substantially depending on the nature of the improvement. For a small effort, the infrastructure may be a single part-time employee; for a large and complex effort, such as software process improvement, it may involve 2-3% of the organization's people across a number of groups. Chartering the infrastructure involves developing explicit written agreements that document and clarify expectations and describe responsibilities.

The Diagnosing Phase

The diagnosing phase builds upon the initiating phase to develop a more complete understanding of the improvement work. During this phase, two characterizations of the organization are developed: the current state of the organization and the desired future state. These organizational states are used to develop an approach for improving business practice.

Characterize Current and Desired States: Characterizing the current and desired states is similar to identifying the origin and destination of a journey. This can be done more easily using a reference standard such as the CMM for Software. Where such a standard is not available, a good starting point is the factors identified as part of the "stimulus for change" activity. This activity should focus on elements critical to the changes being introduced, rather than every aspect of an organization's work.

Develop Recommendations: The recommendations developed as a part of this activity suggest a way of proceeding in subsequent activities. The diagnosing phase activities are most often performed by a team with experience and expertise relevant to the task at hand. Their recommendations often weigh heavily in the decisions made by key managers and sponsors.

The Establishing Phase

The purpose of the establishing phase is to develop a detailed work plan. Priorities are set that reflect the recommendations made during the diagnosing phase as well as the organization's broader operations and the constraints of its operating environment. An approach is then developed that honors and factors in the priorities. Finally, specific actions, milestones, deliverables, and responsibilities are incorporated into an action plan.

Set Priorities: The first activity of this phase is to set priorities for the change effort. These priorities must take many factors into account: resources are limited, dependencies exist between recommended activities, external factors may intervene, and the organization's more global priorities must be honored.

Develop Approach: Combining increased understanding of the scope of work (gained in the diagnosing phase) with a set of priorities leads to the development of a strategy for accomplishing the work and identifying resource availability. Technical factors might include the specifics of installing the new technology and new skills and knowledge required for using a technology. Non-technical factors, including the organization's culture, likely sources of resistance, sponsorship levels, and market forces, also must be considered.

Plan Actions: With the approach defined, a detailed implementation plan can be developed. This plan includes schedule, tasks, milestones, decision points, resources, responsibilities, measurement, tracking mechanisms, risks and mitigation strategies, and any other elements required by the organization.

The Acting Phase

The activities of the acting phase help an organization implement the work that has been conceptualized and planned in the previous three phases. These activities will typically consume more calendar time and more resources than all of the other phases combined.

Create Solution: The acting phase begins with bringing all available key elements together to create a "best guess" solution to address the previously identified organizational needs. These key elements might include existing tools, processes, knowledge, and skills, as well as new knowledge, information, and outside help. The solution, which may be quite complex and multi-faceted, is often created by a technical working group.

Pilot/Test Solution: Once a solution has been created, it must be tested, as best guess solutions rarely work exactly as planned. This is often accomplished through a pilot test, but other means may be used.

Refine Solution: Once the paper solution has been tested, it should be modified to reflect the knowledge, experience, and lessons gained from the test. Several iterations of the test-refine process may be necessary to reach a satisfactory solution. A solution should be workable before it is implemented, but waiting for a "perfect" solution may unnecessarily delay the implementation.

Implement Solution: Once the solution is workable, it can be implemented throughout the organization. Various roll-out approaches may be used for implementation, including top-down (starting at the highest level and working down) and just-in-time (implementing project-by-project at an appropriate time in its life cycle). No one roll-out approach is universally better than another; the approach should be chosen based on the nature of the improvement and organizational circumstances. For a major change, implementation may require substantial time and resources.

The Learning Phase

The learning phase completes the improvement cycle. One of the goals of the IDEAL Model is to continuously improve the ability to implement change. In the learning phase, the entire IDEAL experience is reviewed to determine what was accomplished, whether the effort accomplished the intended goals, and how the organization can implement change more effectively and/or efficiently in the future. Records must be kept throughout the IDEAL cycle with this phase in mind.

Analyze and Validate: This activity answers several questions: In what ways did the effort accomplish its intended purpose? What worked well? What could be done more effectively or efficiently? Lessons are collected, analyzed, summarized, and documented. The business needs identified during the initiating phase are reexamined to see if they have been met.

Propose Future Actions: During this activity, recommendations based on analysis and validation are developed and documented. Proposals for improving future change implementations are provided to appropriate levels of management for consideration.

Useful Tips on SPI

• Proactively identify and seek support of process-improvement champions and sponsor
• Reinforce management awareness and commitment with a strong business case for each desired process improvement
• Build an infrastructure strong enough to achieve and hold software core competence
• Measure the extent of adoption of each desired process improvement until it is effectively, efficiently, and across all appropriate parts of the organization

Conclusions

The IDEAL model provides an effective approach to adopting improved software engineering processes, methods, and tools.

⭐ Key Takeaways

The IDEAL model is a five-phase organizational improvement roadmap (Initiating, Diagnosing, Establishing, Acting, Learning) that provides a disciplined engineering approach to software process improvement. The initiating phase is critical as it lays the groundwork and builds sponsorship; if done poorly, resources will be wasted in subsequent phases. Effective sponsorship is one of the most important factors for improvement success, and sponsors must give personal attention and stick with the effort through difficult times. The acting phase consumes the most calendar time and resources, and solutions should be tested and refined before full implementation, with roll-out approaches chosen based on organizational circumstances. The learning phase completes the cycle by analyzing what was accomplished and proposing future actions, ensuring continuous improvement in the ability to implement change.

🧠 Quick Revision Questions

1. What are the five phases of the IDEAL model, and what is the primary purpose of each phase?
2. Why is the initiating phase considered so critical to the success of an improvement effort?
3. What are the two characterizations developed during the diagnosing phase, and what is their purpose?
4. What factors must be considered when setting priorities during the establishing phase?
5. What three activities are involved in the acting phase before a solution is fully implemented, and why is each important?

📘 Lecture 29 — Software Process Assessments

📖 Overview: This lecture introduces software process assessments, their purpose, and various methodologies used worldwide. It explains how assessments serve as diagnostic tools to identify strengths and weaknesses in software development practices, and outlines key process areas and personnel-related topics examined during assessments. The lecture also covers major assessment methodologies including SEI and SPR approaches, and presents a six-stage improvement program for organizations.

🗂️ Topics Covered

The lecture covers the definition and purpose of software process assessments, key process areas examined during assessments, personnel-related topics, introduction to various assessment methods including SEI and SPR approaches, and a comprehensive six-stage software improvement program covering management technologies, software processes, tools, infrastructure, reusability, and industry leadership, along with the cost, timing, and value of process improvements.

📝 Lecture Summary

Software Process Assessments

Since software applications are built by human beings, the methods, tools, and practices used for software have become subject to study and analysis. Software process assessments are normally carried out on-site by independent assessors, though in-house assessments are also common. Formal software assessments originated within the IBM corporation in the early 1970s. An assessment finds all the strengths and weaknesses associated with software.

The primary goal of software process assessments is to gather qualitative information about the practices, methods, tool suites, and organizational structures used for software. Some organizations gather additional information on topics such as office space and ergonomic factors. As software has become a worldwide business phenomenon, the need for software process assessments has become increasingly important, especially since software failures outnumber successes by a considerable margin for large systems.

Assessment data is normally gathered by means of structured interviews with managers and technical personnel. To ensure consistency among enterprises, standard questionnaires are utilized by most assessment methods, although the questions vary from method to method. Scripts and questionnaires for data collection are now utilized throughout the world.

Software Process Assessment Methodologies include:

• The SPR assessment methodology
• The SEI assessment methodology (CMMI)
• The ISO assessment methodology
• The SPICE assessment methodology
• The TickIT assessment methodology
• The Trillium assessment methodology
• The Howard Rubin Associates assessment methodology
• The Gartner Group assessment methodology

A software process assessment is somewhat equivalent to undergoing an annual medical examination. The annual physical examination is not intended to cure any specific disease — the main purpose is to find out about the health of the patient. If the physical examination finds any serious problems, the physician can prescribe a therapy program. However, the examination itself is performed for diagnostic purposes, not for therapeutic purposes.

💡 Why this matters: Companies often mistakenly regard the assessment as an end in itself, rather than a means to an end. Assessments by themselves will not improve software quality or productivity, but without the assessment, many problems would remain invisible. Assessments should be viewed as annual checkups to ensure progress is being made on curing problems found during the last assessment.

Key Process Areas

Out of a total of 250 factors that can affect the outcome of a software development project, an individual project would be affected by 15 to 20 factors. These factors are collected from projects in the domains of system software, information systems, military software, commercial software, outsourced projects, and end user applications.

The major topics covered by software process assessment can be summarized by ten key process areas common among many assessment methods. These areas are examined because they have a very significant impact on software project schedules, productivity, and quality levels.

Ten Key Process Areas:

1. Project management methods such as estimation
2. Requirements-gathering and analysis methods
3. Design and specification methods
4. Coding methods
5. Reusability methods
6. Change control methods
7. User documentation methods
8. Pretest defect removal methods such as inspections
9. Testing methods and tools
10. Maintenance methods and tools

Software process assessments attempt to identify practices that can lead to successful outcomes and seek to identify harmful practices that might lead to delays or unacceptable outcomes.

Key Personnel-Related Topics

As software projects are very labor intensive and require exceptional skills, software assessments may also examine factors concerned with hiring and supporting software personnel.

Ten Key Personnel-Related Topics:

1. Staff hiring practices
2. Staff training and education
3. Management training and education
4. Specialists and occupational groups
5. Compensation levels
6. Office ergonomics
7. Organizational structures
8. Morale surveys and results
9. Work patterns and overtime
10. Staff turnover rates

Introduction to Some Assessment Methods

Because software development is still a largely manual activity performed by skilled craftsmen, success on software projects requires very capable personnel led by capable managers. Good teams must utilize methods and practices known to produce excellent results and avoid practices that have led to failures.

SEI Assessment Approach

The SEI assessment data is collected by means of on-site interviews using standard questions, observations, and information data. Once collected, the assessment data are analyzed and used to place a software organization on one of the five maturity levels or capability levels of the SEI CMMI. Although the assessment and maturity-level concepts can be considered separately, most people regard the two as being a linked set.

Formal SEI assessments are performed by the SEI or by licensed consulting companies. Informal "SEI-style" assessments are performed by companies on their own or by consultants who use assessment approaches derived from published SEI materials. The SEI assessment approach originally dealt primarily with software processes and methodologies used by very large companies producing very large systems, but has now broadened the sets of factors included under the assessment.

💡 Why this matters: Software process improvements cannot be successful if they overlook key topics or emphasize one topic and ignore others. The broad range of topics included in SEI's programs emphasizes that success in software requires being good in many activities.

The SPR Assessment Approach

The SPR (Software Productivity Research) assessment interview approach is a group activity for each project analyzed. The SPR consultant meets with the project manager and as many as seven technical staff members at the same time. All participants have seen the questionnaires ahead of time.

The "official" assessment questionnaire is filled out by the SPR consultant using group consensus as the source of information. Sometimes there may be debate or polarized opinions among team members, and a full set of responses is recorded. These differences are then resolved.

The SPR assessment approach uses structured interviews based on a set of scripted questions. The questionnaires make use of sets of 100 or more than 250 multiple-choice questions. Some SPR questions overlap the same topics as SEI questions, although the forms are different.

Once a sample of projects has been analyzed using the questionnaires, the answers are input directly into a tool that performs statistical analysis of the results and shows the mean values and standard deviations of all factors.

💡 Why this matters: Although strengths are very laudable, the real value of a process assessment for most clients is the discovery and objective analysis of areas where the client is lagging and needs improvement. Identifying these areas of weakness is the key to a successful process improvement program.

Six-Stage Improvement Program

Once an assessment is completed, the patterns of strengths and weaknesses observed usually lead to a desire to improve software development processes. Organizations can follow a six-stage improvement program that provides an overall framework for software improvement strategies.

Stage 1: Focus on Management Technologies Stage 1 concentrates on management issues, which are critical to all downstream activities. Software process improvement is a fairly expensive undertaking that can stretch out over three to five years. Unless the management community is at state-of-the-art levels in cost estimating, value analysis, and risk analysis, funding for process improvement may be denied. Management needs training in critical technologies such as planning, sizing, estimating, tracking, measurement, risk analysis, and value analysis. Managers collect ROI (Return on Investment) and data that demonstrate progress.

Stage 2: Focus on Software Processes and Methodologies Stage 2 involves the introduction of specific process technologies, e.g., formal code and design inspections. These improvements require training and cultural adjustments but are not very expensive in terms of capital equipment or major investments. Solid approaches for dealing with requirements, design, development, and quality control are implemented. According to a study, projects that used no formal methods at all lagged in terms of productivity, schedules, defect prevention, and defect removal efficiencies against those projects which used them.

Stage 3: Focus on New Tools and Approaches Stage 3 improvements may involve fairly costly tool suites and capital equipment, such as better workstations. As tools support processes rather than the other way around, process improvements precede changes in tool suites. Organizations use various tools and explore new or advanced technologies. Studies show the difference between leading and lagging companies is the use of quality assurance tools, testing tools, and project management tools.

Stage 4: Focus on Infrastructure and Specialization Stage 4 improvements deal with corporate "infrastructures." An example is the establishment of formal quality assurance departments, formal measurement departments, and testing and maintenance departments. As infrastructure changes are both expensive and complex, they need to be deferred until the value of previous kinds of changes has started to become obvious. This stage addresses issues of organization and specialization, moving toward the establishment of specialized teams for handling testing, maintenance, integration, and configuration control, along with formal quality assurance departments and policies on continuing education.

Stage 5: Focus on Reusability Stage 5 improvements are the most valuable in terms of improving quality and productivity, but also the most challenging. Stage 5 moves toward a full and formal software reusability program. Reuse is a double-edged sword — if reusable materials are of top quality, reuse has the best positive ROI of any software technology. However, if reusable materials are error prone and filled with defects, the ROI switches from positive to sharply negative.

Successful software reuse demands state-of-the-art quality control, software measurements, and software management training.

Reusable Components include:

• Reusable architecture
• Reusable requirements
• Reusable plans
• Reusable estimates
• Reusable designs and specifications
• Reusable interfaces
• Reusable data
• Reusable screens or screen elements
• Reusable source code
• Reusable user documents
• Reusable test plans
• Reusable test cases

Stage 6: Focus on Industry Leadership Stage 6 involves using software excellence for competitive purposes, such as winning a Baldrige Award and using this award to aid in gaining market share. Characteristics include executives who understand and support software, software project managers and technical staff of exceptional ability, more specialists than average organizations, better measurements, better customer satisfaction and staff morale levels, and powerful tool suites for project management, software development, quality assurance, testing, and geriatric tools for aging software.

The Cost of Process Improvements

Every company needs to create an individualized plan and budget for its improvement strategy. Cost elements include training, consulting fees, capital equipment, software licenses, and improvement in office conditions.

The Timing of Process Improvements

Smaller companies move much more rapidly than larger corporations and government agencies. Process improvement is a multi-year undertaking. When there is polarization of opinion or political opposition, progress can be very slow or nonexistent.

The Value of Process Improvements

The lecture notes indicate this section was not filled in the provided text.

⭐ Key Takeaways

Software process assessments serve as diagnostic tools, similar to annual medical checkups, that identify strengths and weaknesses in software development practices — they are not therapeutic but diagnostic. The ten key process areas and ten personnel-related topics examined during assessments have significant impacts on project schedules, productivity, and quality. The SEI and SPR assessment methodologies use different approaches: SEI uses on-site interviews with five maturity levels of CMMI, while SPR uses group consensus with statistical analysis of multiple-choice questionnaires. The six-stage improvement program sequentially addresses management technologies, software processes, tools, infrastructure, reusability (which has the best ROI but requires state-of-the-art quality control), and industry leadership — and must be viewed as a multi-year undertaking rather than a quick fix.

🧠 Quick Revision Questions

1. What is the primary purpose of software process assessments, and how does the medical examination analogy explain their role in software improvement?

2. List the ten key process areas examined during software process assessments and explain why these specific areas are commonly assessed across different methodologies.

3. Compare and contrast the SEI assessment approach with the SPR assessment approach in terms of data collection methods, interview structure, and outcome analysis.

4. What are the six stages of the software improvement program? Explain why stage 5 (reusability) is described as both the most valuable and the most challenging.

5. Why is software reusability described as a "double-edged sword," and what conditions are necessary for achieving positive ROI from software reuse?

📘 Lecture 30 — Software Process Benchmarks

📖 Overview: This lecture defines software benchmarks and baselines, detailing their role in comparing organizational performance against industry norms and internal historical data. It explores various types of benchmarks, how they are conducted, and their critical importance as a precursor to effective software process improvement initiatives. Understanding benchmarks is essential because they provide tangible, quantitative data that executives can understand and act upon, making the case for process improvement clearer than abstract maturity models.

🗂️ Topics Covered

This lecture begins by defining the fundamental concept of a benchmark as a standard, yardstick, or point of reference. It then categorizes different types of software benchmarks, including macroeconomic, economic, corporate IT, customer satisfaction, and project-level benchmarks. The discussion extends to how benchmark studies are conducted, the goal of a software benchmark analysis, and a basic sequence for performing them. The lecture concludes by contrasting benchmarks with software baselines, explaining that baselines compare a company against its own past performance and highlighting the significant challenges in creating accurate baselines.

📝 Lecture Summary

Benchmarks

Software benchmarks are formal comparisons of a company’s methods and results against those of other organizations. The lecture begins by defining a benchmark as a standard, yardstick, target, scale, or point of reference.

🔑 Definition — Benchmark: A formal comparison of a company’s software methods and results against those of other organizations, primarily to compare against industry norms.

📌 Example: The text provides an example where moving up from an SEI CMMI level 1 to a level 3 might be abstract, but the need to improve productivity rates from five function points per staff month to ten to reach industry norms is a fairly unambiguous goal driven by benchmark data.

Macroeconomic Benchmarks

Macroeconomic benchmarks are concerned with very large-scale issues such as the impact of computers and information technology on the economic health of nations and industries. These benchmarks are usually performed by national governments and sometimes by economists or universities.

Economic Benchmarks

Economic benchmarks are concerned with topics such as whether investments in computers, software, factory automation, or process improvement benefit the profitability and market shares of companies that spend more than others, a concept related to the productivity paradox. These are performed by economists, universities, and management consulting groups.

Corporate IT Benchmarks

Corporate information technology benchmarks deal with comparisons among companies on topics like the percent of corporate employees in IT, number of users supported per IT staff member, annual corporate spending for computer hardware, and revenues per IT employee. These benchmarks are produced by universities, consulting organizations, and sometimes by software journals.

Customer Satisfaction Benchmarks

Customer satisfaction benchmarks predate the computer era. In the context of software, customer satisfaction surveys for specific products are often carried out by the vendors themselves, while comparative benchmarks between classes of similar products require more extensive data.

Project-Level Benchmarks

Project-level benchmarks for software have been carried out since the 1970s. They require a great deal of care to ensure an ‘apples-to-apples’ comparison, but can generate very useful information.

Benchmark studies

Benchmark studies can be carried out in several ways, including:

• Blind studies in which none of the participating companies are identified.
• Hybrid studies with a mix of named and unnamed participants.
• Open studies in which all participants are aware of one another.
• Targeted studies between two specific companies.
• General studies between one company and industry averages.

These studies can be conducted using mailed/e-mailed survey instruments, telephone surveys, or on-site interviews.

Software Benchmark Analysis

The goal of a software benchmark analysis is to show clients exactly where they stand in the context of their own industry. A good benchmark should uncover information on why the results are the way they are and indicate some of the steps that might be needed to improve the client’s standing.

💡 Why this matters: The lecture states that benchmarks are even more effective than assessments in causing companies to start software process improvement programs because they present tangible data that executives can understand easily, such as a need to improve productivity rates to reach industry norms.

📌 Example: Accurate benchmarking was difficult for many years due to problems with the LOC software metric, which was unreliable for studies involving multiple programming languages. The advent of the function point metric has opened a door to more accurate benchmark studies.

Most FAQs

The most frequent questions in software benchmarking are related to best-in-class results, such as:

• What are best-in-class productivity rates in our industry?
• What are best-in-class development costs per function point?
• What are best-in-class quality levels?

The lecture emphasizes that for benchmarks to be useful, the projects being compared must be similar in size and nature; it is not a normal practice to compare unlike software applications, such as a small Web applet with a large military system.

Software Baselines

A baseline is a milestone in the development of software. While benchmarks compare a company against industry norms, baselines compare a company against its own history for several years in the past.

🔑 Definition — Baseline: A starting point in a process improvement program that provides a firm quantitative basis for productivity, schedules, costs, quality, and user satisfaction to judge the rate of improvement.

📌 Example: The lecture explains that if a software organization’s actual productivity is 5.0 FPs per staff month but their cost tracking system only captures low-level design, coding, and testing costs, their apparent productivity might look like 15.0 FPs per staff month. This makes initial baselines often very unrealistic due to leakage of costs (e.g., unpaid overtime, client work, omitted activities like requirements and project management).

⭐ Key Takeaways

A student must remember that software benchmarks are formal comparisons against industry norms, used to identify best-in-class practices and gaps in performance, while software baselines are internal comparisons against an organization’s own historical data to measure improvement over time. The key distinction is that benchmarks are external and baselines are internal. Benchmarks are highly effective for motivating process improvement because they provide tangible, quantitative data like productivity and quality rates, which are more compelling to executives than abstract maturity levels. A primary challenge in creating accurate baselines is the prevalence of "leakage" — the failure to track all costs and effort, such as unpaid overtime, client work, and activities outside core development, leading to unrealistic initial data. Finally, for any benchmark or baseline to be valid and useful, the data must be collected using a reliable metric like function points, and the projects and activities being compared must be similar in nature and scope to ensure an "apples-to-apples" comparison.

🧠 Quick Revision Questions

1. What is the primary difference between a software benchmark and a software baseline?
2. What is "leakage" in the context of software baselines, and why does it make baselines challenging to create?
3. Name at least three of the seven types of benchmarks discussed in the lecture (Macroeconomic, Economic, Corporate IT, Customer Satisfaction, Project-Level, Assessment Studies, Assessment vs. Benchmarks).
4. What is the goal of a software benchmark analysis, and why is it more effective than assessments in motivating process improvement?
5. What question does the "productivity paradox" address in the context of economic benchmarks?

📘 Lecture 31 — Agile Software Process

📖 Overview: This lecture introduces the philosophy and key principles of Agile software development, contrasting it with traditional plan-driven methods like the waterfall model. It explores the Agile Manifesto, various agile methods, and critically examines how agility can be applied to Software Process Improvement (SPI), arguing for a balanced, human-centric approach that is responsive to change.

🗂️ Topics Covered

The lecture covers the philosophy and development guidelines of Agile software engineering, listing key agile process models. It details the four values of the Agile Manifesto and its twelve underlying principles. The lecture then discusses the diversity among agile methods and their key messages for SPI. It applies agility to process improvement itself, reinterpreting agile principles for SPI, and concludes by presenting a model for an agile SPI method and the factors necessary for a successful SPI program.

📝 Lecture Summary

Agile Software Process

Agile software engineering combines a philosophy and a set of development guidelines. The philosophy encourages customer satisfaction, early incremental delivery, small motivated teams, informal methods, minimal work products, and overall simplicity. The guidelines stress delivery over analysis and design, and active communication. There are several agile process models, including Adaptive Software Development, Extreme Programming (XP), Feature-Driven Development, SCRUM, Agile Modeling, and Crystal. A key feature of these methods is that they accept change (in requirements and technology) and try to work with human nature. A consequence of this is creating less documentation, actively avoiding it for its own sake.

🔑 Definition — Agile Software Engineering: A philosophy and set of development guidelines that prioritize customer satisfaction, early delivery, motivated teams, and simplicity, while actively embracing and managing change.

The Agile Manifesto

The Agile Manifesto states that the authors have come to value:

• Individuals and interactions over process and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan The manifesto explicitly notes that while there is value in the items on the right (process, documentation, contract, plan), they value the items on the left more.

Agile Principles

The Agile Manifesto is based on twelve principles. These include: highest priority is satisfying the customer through early and continuous delivery; welcoming changing requirements even late in development; delivering working software frequently (weeks to months); business people and developers working together daily; building projects around motivated individuals and trusting them; using face-to-face conversation as the most efficient method; having working software as the primary measure of progress; promoting sustainable development; continuous attention to technical excellence; simplicity (maximizing work not done); allowing the best designs to emerge from self-organizing teams; and having the team regularly reflect and adjust its behavior.

Agile Methods

Not all agile methods are the same; for example, XP focuses on design, coding, and testing, while Crystal focuses on scalability. In general, agile methods focus on overall approaches to programming, testing, documentation, planning, and team interaction rather than detailed procedures like configuration management. The key high-level messages are: don't assume the waterfall life cycle is the only approach; work with customers and users; accept and manage change; and don't be more rigorous than the situation demands. While agile methods emphasize process improvement, they are sometimes seen as the antithesis of capability maturity models like CMM/CMMI. However, the lecture argues that agile methods are entirely consistent with the process improvement requirements of ISO 9001:2000/2008.

💡 Why this matters: This reconciles the apparent conflict between agile's flexible, people-focused approach and formal, model-based SPI standards, showing that they can coexist.

Agility Applied to Process Improvement

Agility can be applied to SPI activities themselves by adapting agile principles. For example: highest priority is to satisfy the user through early and continuous delivery of valuable productivity aids; software developers and process improvers must work closely; recognize motivated individuals; simplicity is essential; and teams should regularly reflect and adjust. Process improvement should be carried out in an agile, iterative manner. An "agile process improvement" approach may need to challenge the unyielding link between measurement and improvement, instead seeing quantitative data as a valuable bonus, not an absolute prerequisite. The lecture advocates for a practical, informed approach: if people believe something is needed, do it; if an approach looks good, try it; collect data cost-effectively if possible, but don't hold your breath; justify large expenditures rationally; and don't use this as an excuse to avoid methodical estimating.

A Model of an Agile SPI Method

An agile approach to SPI should be responsive, flexible, encourage innovation, build projects around motivated people, encourage self-organizing teams, and promote sustainable process development. A model of an agile SPI method conceptualizes the features needed to support this.

Factors in Ensuring a Successful SPI Program

Several factors are critical for a successful SPI program. Changes must align with business strategy and priorities. Commitment and investment must come from the top. SPI requires consensus and ownership from all stakeholders. Current processes must be understood. Change must be continuous. An improvement infrastructure is needed to manage changes. Results should be monitored. The philosophy must shift from process stability to being adaptive and flexible. The goal should be a successful product, not just following a predefined methodology. The primary means for adjusting the process is learning from practice, which requires encouraging innovation. The Software Engineering Process Group (SEPG) can be adapted to support this emergent view, acting as a facilitating group rather than the sole identifier of process areas, supporting through training and resources and acting as a research group for boundary scanning. Finally, good people can produce good work in an "immature" organization, but good processes without good people will achieve very little.

⭐ Key Takeaways

The Agile Manifesto prioritizes individuals, working software, customer collaboration, and responding to change over rigid processes, documentation, contracts, and plans. These values are supported by twelve principles that emphasize early delivery, welcoming change, motivated teams, face-to-face communication, and simplicity. Agility can and should be applied to Software Process Improvement itself, advocating for an iterative, people-centric, and pragmatic approach where quantitative data is a bonus, not a prerequisite. The success of an agile SPI program depends on alignment with business strategy, top-down commitment, stakeholder ownership, continuous learning from practice, and a supportive infrastructure, with the SEPG acting as a facilitator. Ultimately, good people are more critical to success than good processes alone.

🧠 Quick Revision Questions

1. What are the four values of the Agile Manifesto?
2. List three of the twelve Agile Principles.
3. How does the lecture reconcile agile methods with formal process improvement models like CMMI and ISO 9001?
4. What does it mean to "challenge the inexorable link between measurement and process improvement"?
5. What are two key factors for ensuring a successful SPI program according to this lecture?

📘 Lecture 32 — Process Patterns

📖 Overview: This lecture introduces the concept of process patterns as reusable building blocks for tailoring a mature software process. It explains the three types of process patterns—task, stage, and phase—and provides detailed examples of each, including the Technical Review task pattern, Program stage pattern, and Construct phase pattern. The lecture emphasizes how these proven approaches help organizations improve software quality, maintainability, and extensibility while avoiding ineffective anti-patterns.

🗂️ Topics Covered

The lecture begins by defining the core concepts of process and patterns, then explains process patterns as collections of general techniques for developing software. It categorizes process patterns into three types: task process patterns (detailed steps for specific tasks), stage process patterns (iterative steps for a project stage), and phase process patterns (interactions between stage patterns). The lecture covers a standard format for documenting process patterns (Name, Forces, Initial Context, Solution, Resulting Context, Related Patterns, Known Uses) and provides concrete examples: the Technical Review task pattern, the Program stage pattern, and the Construct phase pattern. Finally, it introduces process anti-patterns and discusses the Object-Oriented Software Process (OOSP) framework.

📝 Lecture Summary

Process

A series of actions in which one or more inputs are used to produce one or more outputs.

🔑 Definition — Process: A series of actions where inputs produce outputs.

Patterns

A general solution to a common (recurring) problem or issue, one from which a specific solution may be derived. Patterns can exist at all scales, as noted by Christopher Alexander.

Process Patterns

A process pattern is a collection of general techniques, actions, and/or tasks (activities) for developing software. A process pattern describes what you should do but not the exact details of how you should do something. When applied together in an organized manner, process patterns can be used to construct a software process for your organization. Since process patterns do not specify how to perform a given task, they can be used as reusable building blocks from which you may tailor a software process that meets the specific needs of your organization.

Software development patterns come in many flavors, including analysis patterns, design patterns, organizational patterns, and process patterns. A process pattern describes a proven, successful approach and/or series of actions for developing software. It can be argued that process patterns and organizational patterns go hand in hand. Organizational patterns describe common management techniques or organizational structures and describe proven, successful approaches for organizing and managing people involved with the software process.

An important feature of process patterns is that it is possible to develop them for all aspects of development. The scope of a single process pattern may range from a high-level view of how applications are developed to a more-detailed view of a specific portion of the software process.

There are at least three types of process patterns:

• Task process patterns: Depict the detailed steps to perform a specific task, such as the Technical Review and Reuse First process patterns.
• Stage process patterns: Depict the steps, which are often performed iteratively, of a single project stage. A project stage is a higher-level form of process pattern, one that is often composed of several task process patterns. A stage process pattern is presented for each project stage such as the Program and Rework stages.
• Phase process patterns: Depict the interactions between the stage process patterns for a single project phase, such as Initiate and Delivery phases. A phase process pattern is a collection of two or more stage process patterns. Project phases are performed in a serial manner; phase process patterns are performed in serial order, made up of stage process patterns which are performed iteratively.

🔑 Definition — Process Pattern: A collection of general techniques, actions, and/or tasks (activities) for developing software that describes what to do but not exactly how to do it.

🔑 Definition — Task Process Pattern: A process pattern that depicts the detailed steps to perform a specific task.

🔑 Definition — Stage Process Pattern: A process pattern that depicts the iterative steps of a single project stage, often composed of several task process patterns.

🔑 Definition — Phase Process Pattern: A process pattern that depicts the interactions between stage process patterns for a single project phase, performed in serial order.

Documenting Process Patterns

We can combine the existing work in process patterns with that of the existing process improvement community. The implication is the need for a format/template that both communities understand and agree to.

Process Patterns Format

• Name: Provide a concise, strong name for the pattern, such as Program or Reuse First.
• Forces: Applicable forces motivating the process pattern.
• Initial context/Entry conditions: Indicate the situation to which the pattern solution applies, and if applicable the entry conditions that must be true before the process may begin.
• Solution: Describe in detail how to perform the steps/activities of the process pattern. You may also choose, particularly for phase and stage process patterns, to describe management, quality assurance, and risk management issues, as well as indicate potential metrics to collect when working the process.
• Resulting context/Exit conditions: Indicate the situation/context which will result from performing the process pattern solution, including if applicable the conditions that must be true for the process to be considered complete.
• Related Patterns: Indicate other patterns that this pattern is composed of, is a part of, or is associated to.
• Known Uses/Examples: Indicate where/how the process pattern has been applied in use. For example, the Technical Review task process pattern can be applied to the management of peer reviews, code reviews, model reviews, and management reviews.

💡 Why this matters: The structured format ensures consistency and clarity when documenting process patterns, making them easier to understand, reuse, and adapt across different projects and organizations.

Example of a Task Process Pattern: Technical Review

The Technical Review task process pattern describes how to organize, conduct, and follow through on the review of one or more deliverables.

Forces:

• First, the deliverables (models, prototypes, documents, source code, etc.) produced during the development process help to define the software and related products to be released to your user community, therefore you should validate that each deliverable is of sufficient quality before building on it.
• Second, because the cost of fixing defects increases the later they are detected in the development life cycle as a result of the error snowballing throughout your work, you want to detect defects as early as possible so you may fix them early (and inexpensively).
• Third, because it is difficult to review your own work you want "a second set of eyes" to review a deliverable.
• Fourth, you want to communicate your work to others, and one way to do that is to have your teammates review the deliverables that you produce.

Initial Context: There are one or more deliverables to be reviewed, the deliverables are ready to be reviewed, and the development team is ready to have the deliverables reviewed.

Solution: There are six basic steps to technical reviews process pattern (model reviews, document reviews, prototype reviews, requirement reviews, and code inspections are all specific processes that follow the Technical Review process pattern).

1. Prepare for review: The item(s) that are to be reviewed are gathered, organized appropriately, and packaged so that they may be presented to the reviewers.
2. Indicate readiness for review: The development team must inform the review manager, often a member of quality assurance, when they are ready to have their work reviewed as well as what they intend to have reviewed.
3. Perform cursory inspection: The first thing that the review manager must do is determine if the development team has produced work that is ready to be reviewed. The manager will probably discuss the development team's work with the team leader and do a quick rundown of what they have produced. The main goal is to ensure that the work to be reviewed is good enough to warrant getting a review team together.
4. Organize review: The review manager must schedule a review room and any equipment needed for the review, invite the proper people, and distribute any materials ahead of time that are needed for the review.
5. Hold review: Technical reviews should take about two hours so as not to overwhelm the people involved. The entire development team should attend, or at least the people responsible for what is being reviewed, to answer questions and to explain/clarify their work. There are typically between three to five reviewers, as well as the review manager, all of whom are responsible for doing the review. It is important that all material is reviewed. It is too easy to look at something quickly and assume that it is right. It is the job of the review facilitator to ensure that everything is looked at, that everything is questioned.
6. Act on review results: A document is produced during the review describing both the strengths and weaknesses of the work being reviewed. This document should provide both a description of any weakness, why it is a weakness, and provide an indication of what needs to be addressed to fix the weakness. This document will be given to the development team so that they can act on it, and to the review manager to be used in follow-up reviews in which the work is inspected again to verify that the weaknesses were addressed.

🔑 Definition — Error Snowballing: The phenomenon where the cost of fixing defects increases the later they are detected in the development life cycle.

🔑 Definition — Review Facilitator: The person whose job is to ensure that all material is looked at and questioned during a technical review.

Resulting Context:

• Senior management is assured that the development team has produced quality deliverables that meet the needs of their user community.
• The development team, and the reviewers, have a better understanding of the deliverables that they are building and how their work fits into the overall software project.
• Individual team members and reviewers are likely to learn new techniques during the review, either techniques applied to the deliverable itself, management techniques applied during the review, or development techniques suggested during the review to improve the deliverable.

📌 Example: The Technical Review task process pattern can be applied to model reviews, document reviews, prototype reviews, requirement reviews, and code inspections.

Example of a Stage Process Pattern: Program

An important aspect of software development is the actual development of the source code. As experienced developers know, there is far more to programming than just sitting down at a computer and typing in source code. The Program stage process pattern describes the iterative tasks/activities of programming.

Forces:

• Programmers need to develop software that meets the needs of their user communities, needs that have been reflected in the form of models and documents developed during the Model stage and the Define and Validate Initial Requirements stage.
• The developed source code should reflect the information contained in these deliverables, yet at the same time may drive changes to them as programmers gain a detailed understanding of the domain.
• Furthermore, most organizations want software to be developed in a timely and efficient manner but at the same time want it to be maintainable and extensible so that changes in the future may be made efficiently and quickly.

Initial Context/Entry Conditions: Several conditions must be met before coding may begin:

• First, your design models should be in place for the code that you intend to write.
• Second, your project infrastructure should be in place, defined during the Define Infrastructure stage of the Initiate phase. The infrastructure includes the development and supporting tools that your programmers will use as well as the standards and guidelines that they will follow.
• Third, programmers must be available to do the work.

Solution: The Program process pattern shows that there is more to this stage than simply writing source code: you need to understand the models, then seek out reusable artifacts to reduce your work load, then document what you are going to write, then write the code, then inspect and improve it, then test and fix it, and then finally package it.

Resulting Context/Exit Conditions:

• First, your code should have passed inspection.
• Second, the code should work (it passed testing).
• Third, the code should have been optimized sufficiently.
• Fourth, if applicable the software should be integrated and packaged for delivery.

Example of a Phase Process Pattern: Construct Phase

The main goal of the Construct phase is to build working software that is ready to be tested and delivered to your user community. This software will be accompanied by the models and source code that was used to develop it, a test plan to verify that the software works, any reusable artifacts that can be used on future projects, and the initial documentation and training plans supporting the software.

Forces: There are several forces applicable to the Construct Phase, including a lack of understanding of how to work the phase by both senior management and by developers; an unwarranted focus on programming to the neglect of modeling, testing, and generalization; and a penchant by everyone involved to cut corners and take shortcuts that more often than not result in poor quality software that is late and over budget anyway.

Initial Context/Existing Conditions: The Construct phase can be entered two different ways, either from the Initiate phase or from the Maintain and Support phase. Regardless, there are several conditions that must be met before the Construct phase may begin:

• First, the key project management documents (project plan, estimate, schedule, risk assessment, etc.) should be available and up-to-date.
• Second, the project infrastructure should be defined, or at least a good portion of it is defined, so that the tools, processes, and standards are available to your development team.
• Third, the high-level requirements for your software should be in place as well as the project charter for your team.
• Fourth, maintenance changes applicable to the software you are constructing should be allocated to the release that you are working on (this is applicable only for existing software that is being updated).
• Finally, your development team should be selected and made available (as best as possible) for when they are needed by your project.

Solution: An important point is that you are not starting from scratch when you enter the Construct phase—important management documents such as the project plan and initial risk assessment have been defined, the initial requirements for your application should have been defined, the project infrastructure is (mostly) defined, and the funding and charter for the project have been obtained.

The four iterative stages of the Construct phase are highly interrelated:

• The Model stage concentrates on the abstraction of the technical and/or problem domain via the use of diagrams, documents, and prototypes.
• The Program stage focuses on the development and documentation of program source code.
• The Generalize stage is critical to your organization's reuse efforts as it focuses on identifying reusable items, or items that may become reusable once modified, from a software project. This is effectively "opportunistic reuse" because you attempt to develop reusable items by harvesting your work after the fact, instead of "systematic reuse" in which you design software during modeling to be reusable.
• The goal of the Test In The Small stage is to verify and validate the deliverables developed by the other stages of the Construct Phase. In many ways this stage is the equivalent of unit testing from the structured world combined with quality assurance techniques such as code inspections and technical reviews.

🔑 Definition — Opportunistic Reuse: Developing reusable items by harvesting work after the fact, instead of designing specifically for reuse.

🔑 Definition — Systematic Reuse: Designing software during modeling to be reusable from the start.

Resulting Context/Exit Conditions: The Construct phase effectively ends when a code/development freeze has been declared. For a code/development freeze to be official, the following deliverables must be in place (when applicable): Models (Class Model, Use-Case Model, Sequence Diagrams, etc.), Requirements Allocation Matrix (RAM), Source code, Master Test/QA Plan, User Documentation, Operations Documentation, Support Documentation, the software itself, Training Plan, Release Plan, and Lessons Learned. At this point your software is ready to move on to the Deliver phase where it will be tested in the large, reworked as needed, and released to your user community.

🔑 Definition — Code/Development Freeze: The point where development stops, and deliverables must be in place before moving to the Deliver phase.

Why Process Patterns?

• They are an excellent mechanism for communicating approaches to software development that have proven to be effective in practice.
• They are reusable building blocks from which an organization may tailor a mature software process.

The Object-Oriented Software Process (OOSP)

There are four project phases within the OOSP – Initiate, Construct, Deliver, and Maintain and Support – each of which is described by a phase process pattern. There are 14 project stages in the OOSP – Justify, Define and Validate Initial Requirements, Define Initial Management Documents, Define Infrastructure, Model, Program, Test In The Small, Generalize, Test In The Large, Rework, Release, Assess, Support, and Identify Defects and Enhancements – each of which is described by a stage process pattern. Project stages are performed in an iterative manner within the scope of a single project phase. Project phases, on the other hand, are performed in a serial manner within the OOSP.

Process patterns are a key enabler for tailoring/defining a mature software process for your organization. The reality of process improvement, however, is that you cannot make all of the changes that you want to immediately; it is simply too great a change for your organization to absorb at once. Software process improvement promoters suggest that you prioritize the process improvements that your organization needs to make, expect that it will take several years to make the needed changes, and expect that you will experience difficulties while doing so. Experience shows that organizations that try to make immediate, large-scale process changes are likely to fail doing so.

Process Anti-Patterns

Process anti-patterns are approaches to software development that are proven to be ineffective in practice, such as hacking.

🔑 Definition — Process Anti-Pattern: Approaches to software development that are proven to be ineffective in practice.

⭐ Key Takeaways

Process patterns are reusable building blocks that describe proven, successful approaches to software development without specifying exact implementation details, allowing organizations to tailor a mature process to their specific needs. There are three types of process patterns—task patterns (detailed steps for specific tasks like technical reviews), stage patterns (iterative steps for a project stage like programming), and phase patterns (serial interactions between stages like the Construct phase). Each process pattern is documented using a standard format including Name, Forces, Initial Context, Solution, Resulting Context, Related Patterns, and Known Uses. The Technical Review task pattern exemplifies how to conduct reviews with six basic steps, while the Program stage pattern and Construct phase pattern show how larger processes are composed of smaller, iterative patterns. Finally, process anti-patterns (like hacking) represent proven ineffective approaches that should be avoided, and process improvement must be done incrementally rather than through large-scale changes.

🧠 Quick Revision Questions

1. What are the three types of process patterns, and how do they differ in scope and what they describe?
2. List the six components of the standard process pattern documentation format and briefly explain what each component contains.
3. What are the six basic steps of the Technical Review task process pattern, and what is the role of the review facilitator?
4. What are the four entry conditions that must be met before the Program stage process pattern can begin?
5. What is the difference between "opportunistic reuse" and "systematic reuse" as described in the Generalize stage of the Construct phase?

📘 Lecture 33 — ISO/IEC 12207:2008

📖 Overview: This lecture introduces ISO/IEC 12207:2008, the first international standard to provide a comprehensive set of life cycle processes for software. It establishes a common language and structure for all stakeholders involved in software development, based on principles of modularity and responsibility. The lecture details the three main process groups—System Context Processes, Software Specific Processes, and Agreement Processes—and explains their sub-processes.

🗂️ Topics Covered

The lecture covers the definition and purpose of ISO/IEC 12207, its principles of modularity and responsibility, the classification of processes into three types (basic, support, organizational), the various modes of using the standard, the detailed breakdown of life cycle process groups (Agreement, Organizational Project-Enabling, Project, Technical, Software-Specific) including their sub-processes, the concept of tailoring, and the conditions for conformance.

📝 Lecture Summary

ISO/IEC 12207:2008

ISO/IEC 12207 is an ISO standard for software lifecycle processes. It is the first International Standard to provide a comprehensive set of life cycle processes, activities, and tasks for software that is part of a larger system, and for standalone software products and services. It was first introduced on August 1, 1995. The ISO 12207 standard establishes a process of lifecycle for software, including processes and activities applied during the acquisition and configuration of the services of the system. Each Process has a set of outcomes associated with it. The standard has the main objective of supplying a common structure so that the buyers, suppliers, developers, maintainers, operators, managers and technicians involved with the software development use a common language. This common language is established in the form of well-defined processes. ISO/IEC 12207 also provides a process that can be employed for defining, controlling, and improving software life cycle processes. The structure of the standard was intended to be conceived in a flexible, modular way so as to be adaptable to the necessities of whoever uses it.

🔑 Definition — ISO/IEC 12207: An international standard that provides a comprehensive set of life cycle processes, activities, and tasks for software.

Modularity

Modularity means processes with minimum coupling and maximum cohesion.

🔑 Definition — Modularity: A principle where processes are designed with minimum coupling (low interdependence) and maximum cohesion (high internal focus).

Responsibility

Responsibility means to establish a responsibility for each process, facilitating the application of the standard in projects where many people can be legally involved. The set of processes, activities and tasks can be adapted according to the software project. These processes are classified in three types: basic, for support and organizational. The support and organizational processes must exist independently of the organization and the project being executed. The basic processes are instantiated according to the situation.

🔑 Definition — Responsibility: A principle that establishes clear ownership for each process, facilitating legal and operational clarity in projects.

Modes of Use

This International Standard can be used in one or more of the following modes:

• By an organization: To help establish an environment of desired processes. These processes are to be supported by an infrastructure of methods, procedures, techniques, tools and trained personnel. The organization may then employ this environment to perform and manage its projects and progresses systems through their life cycle stages. In this mode this International Standard is used to assess conformance of a declared, established set of life cycle processes to its provisions.
• By a project: To help select, structure and employ the elements of an established set of life cycle processes to provide products and services. In this mode this International Standard is used in the assessment of conformance of the project to the declared and established environment.
• By an acquirer and a supplier: To help develop an agreement concerning processes and activities. Via the agreement, the processes and activities in this International Standard are selected, negotiated, agreed to and performed. In this mode this International Standard is used for guidance in developing the agreement.
• By organizations and assessors: To perform assessments that may be used to support organizational process improvement.

Life Cycle Process Groups

The processes are organized into two major groups:

• System Context Processes
• Software Specific Processes

Each process of this standard is described in terms of the following attributes:

• The title conveys the scope of the process as a whole
• The purpose describes the goals of performing the process
• The outcomes express the observable results expected from the successful performance of the process
• The activities are a list of actions that are used to achieve the outcomes
• The tasks are requirements, recommendations, or permissible actions intended to support the achievement of the outcomes

Agreement Processes

These processes define the activities necessary to establish an agreement between two organizations. There are two agreement processes:

• Acquisition Process: If the Acquisition Process is invoked, it provides the means for conducting business with a supplier of products that are supplied for use as an operational system, of services in support of an operational system, or of elements of a system being developed by a project.
• Supply Process: If the Supply Process is invoked, it provides the means for conducting a project in which the result is a product or service that is delivered to the acquirer.

Organizational Project-Enabling Processes

There are five organizational project-enabling processes:

• Life Cycle Model Management Process
• Infrastructure Management Process
• Project Portfolio Management Process
• Human Resource Management Process
• Quality Management Process

The Organizational Project-Enabling Processes manage the organization's capability to acquire and supply products or services through the initiation, support and control of projects. They provide resources and infrastructure necessary to support projects and ensure the satisfaction of organizational objectives and established agreements. They are not intended to be a comprehensive set of business processes that enable management of the organization's business.

Project Processes

In this International Standard, the project has been chosen as the context for describing processes concerned with planning, assessment and control. The principles related to these processes can be applied in any area of an organization's management. There are two categories of Project Processes:

• Project Management Processes: Used to establish and evolve project plans, to assess actual achievement and progress against the plans and to control execution of the project through to fulfillment. Individual Project Management Processes may be invoked at any time in the life cycle and at any level in a hierarchy of projects, as required by project plans or unforeseen events. Simply put, the Project Management Processes are used to plan, execute, assess and control the progress of a project. There are two:

  • Project Planning Process
  • Project Assessment and Control Process

• Project Support Processes: Provide a specific focused set of tasks for performing a specialized management objective. They are all evident in the management of any undertaking, ranging from a complete organization down to a single life cycle process and its tasks. There are five:

  • Decision Management Process
  • Risk Management Process
  • Configuration Management Process
  • Information Management Process
  • Measurement Process

Technical Processes

There are ten technical processes:

• Stakeholder Requirements Definition Process
• System Requirements Analysis Process
• System Architectural Design Process
• Implementation Process
• System Integration Process
• System Qualification Testing Process
• Software Installation Process
• Software Acceptance Support Process
• Software Operation Process
• Software Maintenance Process
• Software Disposal Process

The Technical Processes are used to define the requirements for a system, to transform the requirements into an effective product, to permit consistent reproduction of the product where necessary, to use the product, to provide the required services, to sustain the provision of those services and to dispose of the product when it is retired from service. The Technical Processes define the activities that enable organizational and project functions to optimize the benefits and reduce the risks that arise from technical decisions and actions. These activities enable products and services to possess the timeliness and availability, the cost effectiveness, and the functionality, reliability, maintainability, usability and other qualities required by acquiring and supplying organizations. They also enable products and services to conform to the expectations or legislated requirements of society, including health, safety, security and environmental factors.

Software-Specific Processes

The Software-Specific Processes are:

• Software Implementation Processes
• Software Support Processes
• Software Reuse Processes

Software Implementation Processes

The Software Implementation Processes are used to produce a specified system element (software item) implemented in software. This process transforms specified behavior, interfaces and implementation constraints into actions that create a system element implemented as a software product or service, otherwise known as a software item. This process results in a software item that satisfies architectural design requirements through verification and stakeholder requirements through validation. For the purpose of clear description, processes are sometimes decomposed into smaller pieces. Some processes are decomposed into activities and/or lower-level processes. A lower-level process is described when the decomposed portion of the process itself satisfies the criteria to be a process.

🔑 Definition — Software Item: A system element implemented as a software product or service.

The sub-processes of the Software Implementation Process are:

• Software Requirements Analysis Process: A lower-level process of the Software Implementation Process. The purpose is to establish the requirements of the software elements of the system.
• Software Architectural Design Process: A lower-level process of the Software Implementation Process. The purpose is to provide a design for the software that implements and can be verified against the requirements.
• Software Detailed Design Process: A lower-level process of the Software Implementation Process. The purpose is to provide a design for the software that implements and can be verified against the requirements and the software architecture and is sufficiently detailed to permit coding and testing.
• Software Construction Process: A lower-level process of the Software Implementation Process. The purpose is to produce executable software units that properly reflect the software design.
• Software Integration Process: A lower-level process of the Software Implementation Process. The purpose is to combine the software units and software components, producing integrated software items, consistent with the software design, that demonstrate that the functional and non-functional software requirements are satisfied on an equivalent or complete operational platform.
• Software Qualification Testing Process: A lower-level process of the Software Implementation Process. The purpose is to confirm that the integrated software product meets its defined requirements.

Software Support Processes

The Software Support Processes provide a specific focused set of activities for performing a specialized software process. A supporting process assists the Software Implementation Process as an integral part with a distinct purpose, contributing to the success and quality of the software project. There are eight of these processes:

• Software Documentation Management Process
• Software Configuration Management Process
• Software Quality Assurance Process
• Software Verification Process
• Software Validation Process
• Software Review Process
• Software Audit Process
• Software Problem Resolution Process

Software Reuse Processes

The Software Reuse Process Group consists of three processes that support an organization's ability to reuse software items across project boundaries. These processes are unique because, by their nature, they operate outside the bounds of any particular project:

• Domain Engineering Process
• Reuse Asset Management Process
• Reuse Program Management Process

Tailoring Process

Tailoring is not a requirement for conformance to the standard. In fact, tailoring is not permitted if a claim of "full conformance" is to be made. If a claim of "tailored conformance" is made then tailoring is to be performed as required by this process. The purpose of the Tailoring Process is to adapt the processes of this International Standard to satisfy particular circumstances or factors that:

• Surround an organization that is employing this International Standard in an agreement
• Influence a project that is required to meet an agreement in which this International Standard is referenced
• Reflect the needs of an organization in order to supply products or services

It should be noted that tailoring may diminish the perceived value of a claim of conformance to this standard. This is because it is difficult for other organizations to understand the extent to which tailoring may have deleted desirable provisions. An organization asserting a single-party claim of conformance to this standard may find it advantageous to claim absolute conformance to a smaller list of processes rather than tailored conformance to a larger list of processes.

🔑 Definition — Tailoring: The process of adapting the standard's processes to satisfy particular circumstances or factors; it is not permitted for "full conformance" but is required for "tailored conformance."

💡 Why this matters: Tailoring is a critical concept because it introduces a trade-off between flexibility and the perceived value of conformance. Students must understand that while tailoring allows adaptation, it can weaken the credibility of a conformance claim, making it often better to claim full conformance to a smaller, well-defined set of processes.

⭐ Key Takeaways

The most critical takeaway from this lecture is understanding ISO/IEC 12207 as the foundational international standard for defining software life cycle processes, built on the principles of modularity (minimum coupling, maximum cohesion) and responsibility. You must be able to distinguish between the three main process types (Agreement, Organizational, and Project/Technical processes) and their specific sub-processes, especially the key groups like Agreement (Acquisition & Supply), Project Management (Planning & Assessment), and Software Implementation (Requirements, Design, Construction, Integration, Testing). A critical distinction to remember is the difference between "full conformance" and "tailored conformance" in the Tailoring Process, understanding that tailoring is not permitted for full conformance and can diminish the standard's perceived value. Finally, recognize the standard's flexible, modular nature allowing it to be used by organizations, projects, acquirers/suppliers, and assessors for different purposes, including process improvement and conformance assessment.

🧠 Quick Revision Questions

1. What are the two basic principles on which the ISO 12207 standard is based?
2. List the two Agreement Processes defined in ISO 12207 and explain the primary goal of each.
3. What is the difference between "full conformance" and "tailored conformance" to the standard?
4. Name the three main groups of Software-Specific Processes and explain the unique characteristic of the Software Reuse Processes.
5. What are the six sub-processes of the Software Implementation Process, and what is the final purpose of the Software Qualification Testing Process?

📘 Lecture 34 — SPICE – ISO/IEC 15504

📖 Overview: This lecture introduces SPICE (Software Process Improvement and Capability dEtermination), also known as ISO/IEC TR Standard 15504. It is a major international initiative for software process assessment, providing a reference model with both process and capability dimensions. The lecture covers the structure of processes, capability levels, process attributes, the assessment process, and how SPICE is used for process improvement and capability determination.

🗂️ Topics Covered

The lecture covers the original goals of the SPICE project, the ISO/IEC 15504 reference model including its process dimension (five process categories) and capability dimension (six levels from Incomplete to Optimizing), the nine process attributes, the four-point rating scale for assessment, steps in the assessment process, the assessment model and tools, assessor qualifications, the two main uses of SPICE (process improvement and capability determination), and the acceptance and limitations of ISO/IEC 15504 compared to CMM/CMMI.

📝 Lecture Summary

SPICE – ISO/IEC 15504

SPICE is a major international initiative to support the development of an International Standard for Software Process Assessment. It is also known as ISO/IEC TR Standard 15504. SPICE is an acronym for Software Process Improvement and Capability dEtermination.

🔑 Definition — SPICE: A major international initiative to develop an International Standard for software process assessment, also known as ISO/IEC TR 15504.

Principal/Original Goals of SPICE Project

The primary goals were to develop a working draft for a standard for software process assessment, to conduct industry trials of the emerging standard, and to promote the technology transfer of software process assessment into the software industry world-wide. The first goal was achieved in June 1995 with the release of Version 1 of a draft standard. The SPICE documents were published as ISO/IEC TR 15504 - Software Process Assessment. The primary impetus for assessment use came from acquirers of large, critical software-intensive systems, notably in defense and telecommunications. The increasing number of assessment approaches and their use in commercially-sensitive areas motivated the development of this International Standard. All industries now depend on software for competitive advantage, and growth requires meeting international standards and world's best practices.

ISO/IEC 15504

ISO/IEC 15504 is the reference model for maturity models, consisting of capability levels which consist of process attributes and further consist of generic practices. Assessors use this model to place the evidence they collect during assessment so they can give an overall determination of the organization's capabilities for delivering products (software, systems, and IT services). The reference model defines a process dimension and a capability dimension.

🔑 Definition — Process Dimension: Shows processes divided into five process categories.

🔑 Definition — Capability Dimension: Defines capability levels measured using process attributes.

Processes

The process dimension shows processes divided into five process categories:

• Customer-supplier
• Engineering
• Supporting
• Management
• Organization

It is expected that the process categories will expand, particularly for IT service and enterprise process categories.

Capability Levels and Process Attributes

For each process, ISO/IEC 15504 defines a capability level on the following scale:

The capability of processes is measured using process attributes. The international standard defines nine process attributes:

🔑 Definition — Process Attributes: Nine attributes used to measure the capability of processes.

• 1.1 Process Performance
• 2.1 Performance Management
• 2.2 Work Product Management
• 3.1 Process Definition
• 3.2 Process Deployment
• 4.1 Process Measurement
• 4.2 Process Control
• 5.1 Process Innovation
• 5.2 Process Optimization

Each process attribute consists of one or more generic practices, which are further elaborated into practice indicators to aid assessment performance.

Assessment of Process Attributes

Each process attribute is assessed on a four-point (N-P-L-F) rating scale:

• Not achieved (0 - 15%)
• Partially achieved (>15% - 50%)
• Largely achieved (>50% - 85%)
• Fully achieved (>85% - 100%)

The rating is based upon evidence collected against the practice indicators, which demonstrate fulfillment of the process attribute.

Assessments

ISO/IEC 15504 provides a guide for performing an assessment. This includes:

• The assessment process
• The model for assessment
• Any tools used in the assessment

Assessment Process

A conformant assessment method must be used. The actual method is not specified in the standard. The standard places requirements on qualification and competence of assessors and provides general guidance, which must be supplemented by formal training and detailed guidance during initial assessments.

Steps in Assessments:

1. Initiate an assessment (assessment sponsor)
2. Select assessor and assessment team
3. Plan the assessment, including processes and organizational unit to be assessed (lead assessor and assessment team)
4. Pre-assessment briefing
5. Data collection
6. Data validation
7. Process rating
8. Reporting the assessment result

An assessor can collect data by various means including interviews, documents and quality records, and statistical process data. The assessor validates this data to ensure it is accurate and completely covers the assessment scope. The assessor assesses this data using their expert judgment against a process's base practices and the capability dimension's generic practices in the process rating step. Process rating requires exercising expert judgment, which is why there are requirements on assessor qualifications and competency. The process rating is presented as a preliminary finding to the sponsor to ensure agreement that the assessment is accurate.

Assessment Model

The process assessment model (PAM) is the detailed model used for an actual assessment. It is an elaboration of the process reference model (PRM) provided by the process lifecycle standards. The PAM is based on the process reference model for systems: ISO/IEC 15288. The standard allows other models to be used if they meet ISO/IEC 15504's criteria, including a defined community of interest and meeting requirements for content (process purpose, process outcomes, and assessment indicators).

🔑 Definition — Process Assessment Model (PAM): The detailed model used for an actual assessment, elaborated from the Process Reference Model (PRM).

Tools Used in the Assessment

There exist several assessment tools. The simplest comprise paper-based tools that incorporate the assessment model indicators, including base practice indicators and generic practice indicators. Assessors write down assessment results and notes. There are also computer-based tools that present indicators and allow users to enter assessment judgment and notes in formatted screens, as well as automate collated assessment results (process attribute ratings) and create reports.

Assessor Qualification and Competency

For a successful assessment, the assessor must have a suitable level of relevant skills and experience. These include:

• Personal qualities such as communication skills
• Relevant education, training and experience
• Specific skills for particular categories, e.g., management skills for the management category
• ISO/IEC 15504 related training and experience in process capability assessments

Uses of SPICE

ISO/IEC 15504 can be used in two contexts:

• Process improvement
• Capability determination (= evaluation of supplier's process capability)

Process Improvement

ISO/IEC 15504 can be used to perform process improvement within a technology organization. Process improvement is always difficult, and initiatives often fail, so it is important to understand the initial baseline level (process capability level) and assess the situation after an improvement project. ISO 15504 provides a standard for assessing the organization's capacity to deliver at each stage. The reference framework provides a structure for defining objectives, which facilitates specific programs to achieve these objectives. It specifies requirements for improvement programs and provides guidance on planning and executing improvements.

🔑 Definition — Process Capability Level: The initial baseline level of an organization's process capability, used to understand the starting point for process improvement.

Capability Determination

An organization considering outsourcing software development needs to have a good understanding of the capability of potential suppliers to deliver. ISO/IEC 15504 can be used to inform supplier selection decisions. It provides a framework for assessing proposed suppliers, as assessed either by the organization itself or by an independent assessor. ISO/IEC 15504 specifies the high level requirements for covering target process profiles. Target process profiles are particularly important in contexts where the organization (e.g., a government department) is required to accept the cheapest qualifying vendor. This also enables suppliers to identify gaps between their current capability and the level required by a potential customer, and to undertake improvement to achieve contract requirements (i.e., become qualified).

🔑 Definition — Target Process Profiles: Profiles specifying the required capability level for processes, important for contexts like government procurement where the cheapest qualifying vendor must be accepted.

Acceptance of ISO/IEC TR 15504

ISO/IEC 15504 is publicly available through National Standards Bodies and has the support of the international community. Thousands of assessments have been performed to date. Major sectors like automotive, space, and medical systems lead the pace with industry relevant variants. Domain-specific models like Automotive SPICE and SPICE 4 SPACE can be derived from it. There have been many international initiatives to support take-up such as SPICE for small companies.

On the other hand, ISO/IEC 15504 has not yet been as successful as the models developed by the Software Engineering Institute. Reasons include:

• ISO/IEC 15504 is not available as free download but must be purchased from ISO; CMM and CMMI are free downloads from the SEI website
• The CMMI is actively sponsored by the US Department of Defense
• The CMM was created first and reached critical market share before ISO 15504 became available
• The CMMI has subsequently replaced the CMM and incorporates many ideas of ISO/IEC 15504 but retains benefits of the CMM
• ISO/IEC 15504 was created in a development context, making it difficult to apply in a service management context; work has started to develop an ITIL-based process reference model

You can join the SPICE users' group at www.spiceusergroup.org. Every year an international conference is held on SPICE, named the SPICE Conference.

💡 Why this matters: Understanding the differences in adoption and acceptance between SPICE/ISO 15504 and CMM/CMMI helps professionals choose the right assessment framework and recognize the competitive advantages each model offers.

⭐ Key Takeaways

SPICE (ISO/IEC 15504) is an international standard for software process assessment that defines a two-dimensional reference model with five process categories and six capability levels (0 to 5). The nine process attributes, each with generic practices, are assessed on a four-point N-P-L-F rating scale to determine capability. The assessment process involves eight steps from initiation to reporting, requiring qualified assessors with specific skills and training. SPICE is used for both process improvement (to establish baselines and track progress) and capability determination (to evaluate and select suppliers). Despite its international support and domain-specific variants like Automotive SPICE, ISO/IEC 15504 has not achieved the same adoption as CMM/CMMI due to factors including cost, sponsorship, and timing.

🧠 Quick Revision Questions

1. What are the five process categories in the ISO/IEC 15504 process dimension?
2. List the six capability levels from lowest to highest as defined in ISO/IEC 15504.
3. What are the nine process attributes used to measure process capability?
4. Describe the four-point rating scale (N-P-L-F) used for assessing process attributes, including the percentage ranges.
5. What are the two main contexts in which ISO/IEC 15504 can be used, and briefly explain each.

📘 Lecture 35 — Process Assurance

📖 Overview: This lecture defines process assurance as the collective activities carried out during product development to ensure methods and techniques are integrated, consistent, and correctly applied. It details the critical components of planning and organization necessary for successful process assurance, including project teams, standards, scheduling, and communication. The lecture concludes with an examination of the primary causes of failure in process assurance efforts, emphasizing management failures over technical ones.

🗂️ Topics Covered

The lecture begins by defining process assurance and its emphasis on cost, time, technical requirements, testing measurements, and prototyping. It then details ten key components of planning and organization for process assurance: project team, project standards, schedule monitoring, project tracking, estimation, effective communication, steering committee, project risks, measurement, and integrated technology. The lecture concludes by analyzing four major causes of failure in process assurance: lack of management support, lack of user involvement, lack of project leadership, and lack of measures of success.

📝 Lecture Summary

Process Assurance

Process assurance consists of the collective activities carried out while developing a product to ensure that the methods and techniques used are integrated, consistent, and correctly applied. Emphasis is given to cost, time, technical requirements, testing measurements, and prototyping. Process assurance involves the interrelationships of several different components, and depending on how these are managed, they can have a major positive impact on the products. Once an effective process assurance program is put in place and shown to be beneficial, then emphasis can be placed in making verification and validation strategies effective and in improving the quality of the products. Successful process assurance is based on planning and organization, which have several important components.

Components of Planning and Organization

1. Project Team

The project team is the project manager’s only means of reaching the project goals. Selection of team members is a vital step to the success of the project. The size of the team depends on the size and complexity of the project. It is important to identify the blend of the technical knowledge and the experience required for the successful completion of the project. Special attention should be paid to creating team composition that fosters mutual respect among team members and maintains good team morale.

2. Project Standards

Before the project is started, the team should establish standards for activities such as requirements gathering, developing design, and conducting unit tests. Standards or guidelines should also be established for quality control activities such as walk-throughs, reviews, and inspections. Many companies follow IEEE Software Engineering standards or have their own internally developed standards. The standards should be flexible enough to be applied to large or small projects. Any deviations from the standards should be approved by the project team, and the reason for such deviation should be noted in the minutes of the project meetings.

3. Schedule Monitoring

Stringent deadlines for the project are frequently established by management, end users, a project sponsor, or a client with no regard to the reality of achievement. For this reason, the project start date, milestones, and completion date should be negotiated upfront. If the unrealistic date is accepted and the project activities are then made to fit within this time frame, the quality of the project certainly will suffer. The key to an “on-time” completion of project lies in the ability to identify the critical path before starting the project. The critical path of a project is where problems that may affect the overall schedule are faced. To define the critical path, you should develop systematic work breakdown structures which identify task groupings, task sequences, and entrance/exit criteria for each task. They assist in selecting the correct skilled resources and ensure no activity is forgotten. Once you have defined a critical path, review the tasks and schedule with the project team members and other significantly impacted individuals, who are the stakeholders. Avoid the most common mistake of adding another resource to shorten or meet the schedule, as this usually increases overhead and reduces efficiency. Overtime can be effective for a short period, but longer periods can exhaust the team. It is possible to minimize risk by managing your resources, especially never putting your best programmers on the critical path.

🔑 Definition – Critical Path: The sequence of project activities where problems that may affect the overall schedule are faced.

4. Project Tracking

There are several project-tracking tools available on the market which give project progress information and provide a view of the total project schedule at a glance. Some of these tools have features to do task dependencies and estimations. These features allow the project manager to evaluate the impact of a delay in the completion of any individual activity within the project. Such tools are excellent vehicles in giving early warnings for project delays so there are no surprises at the end.

5. Estimation

The project manager must be capable of understanding any assumptions in the project, what is possible, and what is financially desirable. This will result in a fine-tuning of schedule estimates, thus creating more realistic estimates. Time estimates are not foolproof, as activities are many times not clearly understood or defined. Allow time for resource management and unforeseen events like illness of a team member; such contingencies should be viewed as reserved time. Revise estimates at the end of each major phase and if there are any changes in scope, cost, or schedules encountered, notify management and obtain appropriate approvals.

6. Effective Communication

Two-way communication between management and the project team is a critical interpersonal skill. An effective project manager must have the skills to listen, observe, and give guidance to the team. During a time constraint situation, the project manager should be able to delegate responsibilities but also give guidance to control stress levels. The success of team harmony and good rapport depends on the project manager's ability to encourage open communication frequently and resolve any conflict through informal negotiation.

7. Steering Committee

A steering committee is responsible for defining project policy, reviewing project milestones, and evaluating risk factors. Members should represent all impacted areas of the business and should be knowledgeable enough to make informed technological decisions. If project features are changed or new features are added, the steering committee is responsible for evaluating the impact and authorizing, rejecting, or deferring the changes. The steering committee should be in charge of system management tasks such as estimating maintenance time, deciding on support from operations, managing data, and forming a Configuration Control Board (CCB) that manages the impact of changes.

8. Project Risks

Systems projects are risky if project expectations are not controlled or if the outcome was ill-defined. While risk cannot be eliminated, many companies identify and address risk factors upfront. Project risks can be assessed in three categories: business risk, technology risk, and project size risk. Risks can be minimized by implementing controls from initiation, ensuring preestablished development standards are followed, providing project management training, and reducing scope through incremental or phased development. Technical risk is encountered when using new technology for the first time and can be controlled by appointing a qualified technical project leader, implementing a strong quality control group, and getting additional expertise from outside consultants.

🔑 Definition – Technical Risk: A project risk encountered when the project team utilizes a new technology like new hardware or a new development methodology for the first time.

9. Measurement

Establishing measurement criteria against which each phase of the project will be evaluated is vital. When the exit criteria are well-defined, it is sufficient to evaluate the outcome of each phase. If the outcome does not meet the performance criteria, the project manager should control the project by evaluating problems and implementing new processes. Preestablished quality goals can serve as measurement criteria. It is important to know that certain elements cannot be controlled by project management, such as selection of the wrong technology, proceeding without understanding business needs, underestimation of the learning curve, and a lack of strategic plans.

10. Integrated Technology

A strategy for Integrated Technology (IT) should be considered by management in relation to other business needs. This empowers management to react to operational needs and take inventory of current systems and technical staff quality. When planning a project, the system architecture with its data groups, structures, processes, and dependencies must be considered. Parts of the new system that will interface with existing systems should be identified. If the technology is new and not well understood, allowances to incorporate experiments should be made in the overall project plan and schedule.

Causes of Failure in Process Assurance

There are many reasons why process assurance efforts fail, most of which are related to management failures rather than technical failures:

1. Lack of Management Support
2. Lack of User Involvement
3. Lack of Project Leadership
4. Lack of Measures of Success

1. Lack of Management Support

Management very often assigns a project and then forgets about it or fails to allocate adequate resources. This lack of project sponsorship leads to serious morale problems and lost resources. The project sponsor is usually a member of senior management responsible for researching regulatory issues, obtaining project funding, and managing business implications. Since the sponsor acts as the main advocate, lack of support from this individual is extremely serious.

2. Lack of User Involvement

When end users are not involved in the design and development phases, many assumptions are made regarding the requirements, especially when requirements are ambiguous. This situation can be corrected by assigning a user liaison responsible for getting support, providing detailed user requirements, providing feedback, managing user expectations, and identifying user acceptance criteria.

3. Lack of Project Leadership

The single most important cause of failure is due to lack of project leadership. Consequences include IT staff not understanding business needs, project scope and goals not being defined, performance not being controlled, and improper selection of team members. Ensuring the project is in the hands of a good project manager is vital. An effective project manager must have responsibility, authority equal to responsibility, and access to necessary resources. The project manager should be flexible, responsive, effective in coordinating activities, credible, and respected by team members.

💡 Why this matters: The balance of responsibility, authority, and access to resources is a critical success factor for any project manager.

4. Lack of Measures of Success

Failure also occurs when project success is based only on development efficiency measures (e.g., on time and within budget) while effectiveness measures (e.g., technical performance and quality) are ignored. To evaluate project success, objective measurements should be developed, such as completing the project on time, within budget, and with all required features. Each element of the success criteria must be defined and agreed upon prior to starting the project.

⭐ Key Takeaways

Process assurance ensures that development methods and techniques are integrated, consistent, and correctly applied, with emphasis on cost, time, technical requirements, testing, and prototyping. Ten core components of planning and organization are essential: project team selection, project standards, schedule monitoring using critical path and work breakdown structures, project tracking tools, realistic estimation with contingencies, effective two-way communication, a steering committee, risk assessment and minimization, measurement against exit criteria, and integrated technology planning. The four major causes of process assurance failure are all management-related: lack of management support, lack of user involvement, lack of project leadership, and lack of measures of success. A project manager must have balanced responsibility, authority, and access to resources, while success must be measured by both efficiency (on time, within budget) and effectiveness (technical performance, quality) criteria.

🧠 Quick Revision Questions

1. What are the five areas of emphasis in process assurance?
2. What is the critical path and why is it important for schedule monitoring?
3. What three categories can project risks be assessed into?
4. What are the four primary causes of failure in process assurance?
5. What three attributes must be treated equally for a project manager to be effective?

📘 Lecture 36 — People Capability Maturity Model – 1

📖 Overview: This lecture introduces the People Capability Maturity Model (People CMM), a framework for managing and developing an organization's workforce. It explains how the People CMM applies the process maturity framework from the SW-CMM to workforce practices, helping organizations attract, train, deploy, and retain talent. The lecture covers the principles of the People CMM, the critical success factors for managing human capital, and the five maturity levels that guide an organization from ad hoc workforce practices to continuous improvement.

🗂️ Topics Covered

The lecture begins by introducing the People CMM as a tool for addressing critical people issues, based on best practices from human resources, knowledge management, and organizational development. It explores the philosophy behind the model through ten principles and seven principles of workforce management from Jeffrey Pfeffer. The lecture then examines four human capital cornerstones for strategic management, the process maturity framework adapted from Humphrey, and a detailed breakdown of each of the five maturity levels from Initial (Level 1) to Optimizing (Level 5). Finally, it lists types of organizations using People CMM globally.

📝 Lecture Summary

People Capability Maturity Model – 1

The People Capability Maturity Model (People CMM) is a tool that helps organizations successfully address critical people issues in managing and developing their workforce. It employs the process maturity framework of the Capability Maturity Model for Software (SW-CMM) as a foundation for best practices in fields like human resources, knowledge management, and organizational development. The People CMM helps organizations characterize the maturity of their workforce practices, establish a program of continuous workforce development, set priorities for improvement, integrate workforce development with process improvement, and establish a culture of excellence. Since its release in 1995, it has been used worldwide by organizations like IBM, Boeing, and Tata Consultancy Services. The primary objective of the People CMM is to improve the capability of the workforce.

🔑 Definition — Workforce Capability: The level of knowledge, skills, and process abilities available for performing an organization’s business activities. It indicates an organization’s readiness for performing critical business activities, likely results from these activities, and potential for benefiting from process improvement or advanced technology.

🔑 Definition — Workforce Competency: A unique integration of knowledge, skills, and process abilities acquired through specialized education or work experience. The workforce must be divided into its constituent workforce competencies to measure and improve capability.

💡 Why this matters: The People CMM provides an evolutionary improvement path from ad hoc, inconsistently performed workforce practices to a mature infrastructure for continuously elevating workforce capability, directly linking workforce capability to business performance.

Principles of People CMM – 1

The philosophy of the People CMM is summarized in ten principles:

1. In mature organizations, workforce capability is directly related to business performance.
2. Workforce capability is a competitive issue and a source of strategic advantage.
3. Workforce capability must be defined in relation to the organization’s strategic business objectives.
4. Knowledge-intense work shifts the focus from job elements to workforce competencies.
5. Capability can be measured and improved at multiple levels, including individuals, workgroups, workforce competencies, and the organization.
6. An organization should invest in improving the capability of those workforce competencies critical to its core competency.
7. Operational management is responsible for the capability of the workforce.
8. The improvement of workforce capability can be pursued as a process composed from proven practices and procedures.
9. The organization is responsible for providing improvement opportunities, and individuals are responsible for taking advantage of them.
10. Because technologies and organizational forms evolve rapidly, organizations must continually evolve their workforce practices and develop new workforce competencies.

The People CMM narrows the scope of improvement activities to the vital few practices that provide the next foundational layer for developing the organization’s workforce.

Seven Principle of Workforce Management

Jeffrey Pfeffer, in his book The Human Equation, identified seven principles of workforce management that distinguished companies with the largest percentage stock market returns:

1. Employment security
2. Selective hiring of new personnel
3. Self-managed teams and decentralization of decision making
4. Comparatively high compensation contingent on organizational performance
5. Extensive training
6. Reduced status distinctions and barriers
7. Extensive sharing of financial and performance information

These principles characterize organizations where employees act as independent centers of intelligent action coordinated toward a common purpose. Deep technical and business knowledge is required for rapid, consistent decisions.

Critical Success Factors for Managing Human Capital Strategically

Four Human Capital Cornerstones are identified:

• Leadership: Commitment to Human Capital Management; Role of the Human Capital Function.
• Strategic Human Capital Planning: Integration and Alignment; Data-Driven Human Capital Decisions.
• Acquiring, Developing, and Retaining Talent: Targeted Investments in People; Human Capital Approaches Tailored to Meet Organizational Needs.
• Results-Oriented Organizational Cultures: Empowerment and Inclusiveness; Unit and Individual Performance Linked to Organizational Goals.

Process Maturity Framework

Humphrey designed the process maturity framework to enable an organization to achieve a state of continuous process improvement in five stages. This framework is more than a process standard; it integrates improved practices into a staged model that guides an organization through a series of cultural transformations. Over a dozen years of experience demonstrates this framework is applicable to the management and improvement of workforce practices.

First Level of Maturity

At the Initial Level (Level 1), an organization has no consistent way of performing its work. Work processes are ad hoc, constantly reinvented, and frequently appear chaotic. Managers have no reliable basis for estimating effort, leading to overly aggressive deadlines, cutting corners, and mistakes that are costly to fix. Projects lose control of schedule, costs, and product quality. Results depend largely on the skills of exceptional individuals and excessive overtime.

Second Level of Maturity

At the Repeatable Level (Level 2), organizations must establish a foundation for deploying common processes. Management must first establish a stable environment for professional work, ensuring people are not constantly rushing or fighting fires. The primary objective is to enable people to repeat practices they have used successfully.

Third Level of Maturity

At the Defined Level (Level 3), the organization identifies its best practices and integrates them into a common process. Once people can perform work using practices that work, the organization identifies which practices work best. These practices are documented and integrated into a common process in which the entire organization is trained. Measures of critical practices are defined and collected into a repository for analysis.

Fourth Level of Maturity

At the Managed Level (Level 4), the organization begins managing its processes through data that describes its performance. The performance of the organization’s critical processes is characterized statistically so that historical performance can be used to predict and manage future performance.

Fifth Level of Maturity

At the Optimizing Level (Level 5), the organization uses its profound, quantitative knowledge to make continuous improvements. Based on data, the organization identifies which processes can benefit most from improvement actions. The maturity framework builds an environment where:

• Practices can be repeated.
• Best practices can be transferred rapidly across groups.
• Variations in performing best practices are reduced.
• Practices are continuously improved to enhance their capability.

Types of organizations around the world using the People CMM

Industries using the People CMM include: Business Process Outsourcing, Information Technology, Hospitality Consulting, Construction, Defense Contractors, Insurance Pharmaceuticals, Government Agencies/Defense Agencies, Energy/Utilities, Software Development, Banking/Financial Services Management, and Information Systems.

Maturity Levels of the People CMM

The People CMM applies the principles of the process maturity framework to the domain of workforce practices. Each of its five maturity levels represents a different level of organizational capability for managing and developing the workforce. Each maturity level provides a layer in the foundation for continuous improvement. When institutionalized, these workforce practices create new capabilities within the organization.

⭐ Key Takeaways

The People CMM is a staged framework based on the process maturity model, designed specifically to improve workforce capability—the level of knowledge, skills, and process abilities in an organization. Its philosophy is rooted in ten principles, including that workforce capability is a competitive issue directly linked to business performance, and that improvement can be pursued as a process. The five maturity levels guide an organization from an ad hoc, chaotic Initial Level through Repeatable, Defined, Managed, and finally to an Optimizing Level where data-driven continuous improvement occurs. A key concept is the division of the workforce into competencies, and the identification of critical success factors like leadership, strategic planning, talent management, and results-oriented culture. Finally, the lecture emphasizes that the People CMM provides an evolutionary path, not just a list of best practices, to build a mature infrastructure for workforce excellence.

🧠 Quick Revision Questions

1. What is the primary objective of the People Capability Maturity Model (People CMM)?
2. Define "workforce capability" and explain why it is important for an organization.
3. List the four human capital cornerstones that are critical for managing human capital strategically.
4. Describe the key difference in workforce management between an organization at Maturity Level 1 (Initial) and one at Maturity Level 5 (Optimizing).
5. According to Jeffrey Pfeffer's seven principles, what are two specific practices that distinguish companies with the largest stock market returns?

📘 Lecture 37 — People CMM – 2

📖 Overview: This lecture continues the exploration of the People Capability Maturity Model (People CMM) by detailing the characteristics and process areas of its five maturity levels. It explains how organizations progress from ad hoc workforce practices at the Initial Level to a culture of continuous improvement at the Optimizing Level. Understanding these levels is critical for systematically improving an organization's workforce capability and strategic alignment.

🗂️ Topics Covered

This lecture covers the five maturity levels of the People CMM: Initial Level (Level 1), Managed Level (Level 2), Defined Level (Level 3), Predictable Level (Level 4), and Optimizing Level (Level 5). For each level, the key characteristics, focus, and specific process areas are defined, with detailed explanations of how workforce practices evolve and become more sophisticated as maturity increases.

📝 Lecture Summary

The Initial Level: Maturity Level 1

Organizations at the Initial Level of maturity usually have difficulty retaining talented individuals. Even though many low-maturity organizations complain about a talent shortage, the inconsistency of their actions belies whether they actually believe it. Low-maturity organizations are poorly equipped to respond to talent shortages with anything other than slogans and exhortations. Despite the importance of talent, workforce practices in low-maturity organizations are often ad hoc and inconsistent. In some areas, the organization has not defined workforce practices, and in other areas, it has not trained responsible individuals to perform the practices that exist. Organizations at the Initial Level typically exhibit four characteristics: inconsistency in performing practices, displacement of responsibility, ritualistic practices, and an emotionally detached workforce. Generally, managers and supervisors in low-maturity organizations are ill-prepared to perform their workforce responsibilities and their management training is sparse. Low-maturity organizations implicitly assume that management skill either is innate or is acquired by observing other managers. Management capability should ultimately be defined as a competency, just like other critical skill sets that are required by the organization. Since low-maturity organizations rarely clarify the responsibilities of managers, inconsistencies are to be expected. Consequently, the way people are treated depends largely on personal orientation, experience, and the individual "people skills" of their managers, supervisors, or team leaders. Studies have consistently shown that one of the major causes for voluntary turnover is related to individuals' relationships with their managers or supervisors. The first step in changing this state of affairs is to get managers to take responsibility for the capability and development of those who report to them.

🔑 Definition — Ritualistic Practices: [Not explicitly defined in the text, but inferred as] Performing workforce activities without genuine understanding or commitment, simply as a routine or ceremony, rather than for their intended purpose.

📌 Example: An organization that requires annual performance reviews but managers complete them superficially without providing meaningful feedback or development goals.

The Managed Level: Maturity Level 2

The first step toward improving the capability of the workforce is to get managers to take workforce activities as high-priority responsibilities of their job. They must accept personal responsibility for the performance and development of those who perform the unit's work. The practices implemented at Maturity Level 2 focus a manager's attention on unit level issues such as staffing, coordinating commitments, providing resources, managing performance, developing skills, and making compensation decisions. Building a solid foundation of workforce practices in each unit provides the bedrock on which more sophisticated workforce practices can be implemented at higher levels of maturity. Maturity Level 2 focuses on establishing basic practices in units that address immediate problems and prepare managers to implement more sophisticated practices at higher levels. In a Maturity Level 2 organization, managers are vigilant for problems that hinder performance in their units. These problems include work overload, environmental distractions, unclear performance objectives or feedback, lack of relevant knowledge or skill, poor communication, and low morale. The effort to ensure that workforce practices are performed in each unit begins when executive management commits the organization to continuously improve the knowledge, skills, motivation, and performance of its workforce. Executive management manifests these commitments in policies and provides the resources needed to support unit-level implementation of basic workforce practices. Through policies and accountability, executive management communicates that managers are to accept personal responsibility for ensuring that workforce practices are implemented effectively in their units. In applying the People CMM, it is important to distinguish between management and managers. There are responsibilities that need to be managed and there are people called managers, but there is no required one-to-one mapping between them. At Maturity Level 2, units become stable environments for performing work and are able to balance their commitments with available resources. At Maturity Level 2, an organization's capability for performing work is best characterized by the capability of units to meet commitments. This capability is achieved by ensuring that people have the skills needed to perform their assigned work and that performance is regularly discussed to identify actions that can improve it. Measurements of status and performance of these workforce activities provide management with a means of monitoring and ensuring appropriate performance of workforce practices. One of the first benefits organizations experience when they implement improvements guided by the People CMM is a reduction in voluntary turnover. At Maturity Level 2, the People CMM addresses one of the most frequent causes of turnover—poor relations with the immediate supervisor. The process areas of Level 2 include:

• Staffing: to establish a formal process by which committed work is matched to unit resources and qualified individuals are recruited, selected, and transitioned into assignments.
• Communication and Coordination: to establish timely communication throughout the organization and to ensure that the workforce has the skills to share information and coordinate activities efficiently.
• Work Environment: to establish and maintain physical working conditions and to provide resources that allow individuals and workgroups to perform their tasks efficiently without unnecessary distractions.
• Performance Management: to establish objectives related to committed work against which unit and individual performance can be measured, to discuss performance against these objectives, and to continuously enhance performance.
• Training and Development: to ensure that all individuals have the skills required to perform their assignments and are provided relevant development opportunities.
• Compensation: to provide all individuals with remuneration and benefits based on their contribution and value to the organization.

🔑 Definition — Managed Level (Maturity Level 2): The maturity level where managers take personal responsibility for their unit's performance and development, and basic workforce practices like staffing, performance management, and training are established.

📌 Example: A unit manager at Level 2 ensures that each team member has clear performance objectives, receives regular feedback, and has the necessary skills and resources to meet their commitments.

The Defined Level: The Level 3

Once a foundation of basic workforce practices has been established in the units, the next step is for the organization to develop an organization-wide infrastructure building on these practices that ties the capability of the workforce to strategic business objectives. The primary objective of the Defined Level is to help an organization gain a competitive advantage by developing the various competencies that must be combined in its workforce to accomplish its business activities. These workforce competencies represent the critical pillars that support the strategic business plan; their absence poses a severe risk to strategic business objectives. In tying workforce competencies to current and future business objectives, the improved workforce practices implemented at Maturity Level 3 become critical enablers of business strategy. The concept of workforce competencies implemented in the People CMM differs from the concept of "core competency." Core competency refers to an organization's combination of technology and production skills that create its products and services and provide its competitive advantage in the marketplace. In the People CMM, workforce competencies reside one level of abstraction below an organization's core competency. Each workforce competency represents a distinct integration of the knowledge, skills, and process abilities required to perform some of the business activities that contribute to an organization's core competency. The range of workforce competencies an organization must integrate depends on the breadth and type of business activities that comprise its core competencies. Therefore, these workforce competencies are a strategic underpinning (foundation) of the organization's core competencies. By defining process abilities as a component of a workforce competency, the People CMM becomes linked with the process frameworks established in other CMMs and with other process-based methods, such as business process reengineering. A process ability is demonstrated by performing the competency-based processes appropriate for an individual's required workforce competency. To define the process abilities incorporated in each workforce competency, the organization defines the competency-based processes that an individual would be expected to perform in accomplishing his or her committed work. Within a workforce competency, a competency-based process defines how individuals apply their knowledge, perform their skills, and apply their process abilities in the context of the organization's defined work processes. At Maturity Level 3, the organization builds an organization-wide framework of workforce competencies that establishes the architecture of the organization's workforce. Each workforce competency is an element of the workforce architecture, and dependencies among competency-based processes describe how these architectural elements interact. The architecture of the organization's workforce must evolve as business conditions and technologies change. Because workforce competencies are strategic, the organization must develop strategic workforce plans for ensuring the required capability in each of its current or anticipated workforce competencies. When the processes to be performed by each workforce competency are defined, the organization has a new foundation for developing workgroups. Competency-based processes form a basis for defining workgroup roles and operating processes. Workgroups can now organize themselves by tailoring and applying standard competency-based processes. A common organizational culture typically develops as the organization achieves the Defined Level. This culture is best described as one of professionalism, since it is built from common understanding of the knowledge and skills that need to be developed to achieve superior levels of performance and a definition of the competency-based processes that such individuals perform. Since these workforce competencies are strategic to the business, the organization reinforces their importance by developing and rewarding them. As a result, the entire workforce begins to share responsibility for developing increasing levels of capability in the organization's workforce competencies. The workforce practices that were implemented at Maturity Level 2 are now standardized and adapted to encourage and reward growth in the organization's workforce competencies. The process areas of Level 3 include:

• Competency Analysis: to identify the knowledge, skills, and process abilities required to perform the organization's business activities so that they may be developed and used as a basis for workforce practices.
• Workforce Planning: to coordinate workforce activities with current and future business needs at both the organizational and unit levels.
• Competency Development: to enhance constantly the capability of the workforce to perform its assigned tasks and responsibilities.
• Career Development: to ensure that individuals are provided opportunities to develop workforce competencies that enable them to achieve career objectives.
• Competency-Based Practices: to ensure that all workforce practices are based in part on developing the competencies of the workforce.
• Workgroup Development: to organize work around competency-based process abilities.
• Participatory Culture: to enable the workforce's full capability for making decisions that affect the performance of business activities.

🔑 Definition — Workforce Competency: A distinct integration of the knowledge, skills, and process abilities required to perform business activities that contribute to an organization's core competency and strategic objectives.

📌 Example: A software company might define a "Software Design" workforce competency that includes knowledge of design patterns, skills in using UML, and process abilities in performing design reviews. This competency supports the company's core competency of developing high-quality software products.

The Predictable Level: Maturity Level 4

At the Predictable Level, the organization manages and exploits the capability created by its framework of workforce competencies. This framework is sustained through formal mentoring activities. The organization is now able to manage its capability and performance quantitatively. The organization is able to predict its capability for performing work because it can quantify the capability of its workforce and of the competency-based processes they use in performing their assignments. There are at least three ways in which the framework of workforce competencies enables the organization to more fully use the capabilities of its workforce. First, when competent people perform their assignments using proven competency-based processes, management trusts the results they produce. This trust enables the organization to preserve the results of performing competency-based processes and develop them as organizational assets to be reused by others. In essence, people trust the asset because they trust the methods through which it was produced. When these assets are created and used effectively, learning spreads rapidly through the organization and productivity rises when reuse replaces redevelopment. Second, this trust also gives managers the confidence they need to empower workgroups. Managers will transfer responsibility and authority for committed work into workgroups only if they believe the members of the workgroup are competent to perform the work and use processes that have been proven effective. In achieving Maturity Level 4, management senses less risk in empowering workgroups and is willing to delegate increasingly greater levels of authority for managing day-to-day operations and for performing some of their own workforce practices. Increasingly free of managing operational details, managers at Maturity Level 4 are able to turn their attention to more strategic issues. Third, when members of each workforce competency community have mastered their competency-based processes, the organization is able to integrate different competency-based processes into a single multidisciplinary process. An example would be the integration of software and hardware design processes into a single product design process in which different competency-based processes are interwoven at every point where they share a potential dependency. Such multidisciplinary processes have proven to accelerate business results. In addition to exploiting the possibilities enabled by the competency framework, the organization begins to manage its capability quantitatively. Within each unit or workgroup, the performance of competency-based processes most critical for accomplishing business objectives is measured. These measures are used to establish process performance baselines that can be used to manage competency-based processes and assess the need for corrective action. The combined availability of workforce capability baselines and process capability baselines for competency-based processes enables both unit and organizational performance to become more predictable. These data allow management to make more accurate predictions about performance and better decisions about tradeoffs involving workforce capability or process performance issues. The quantitative management capabilities implemented at Maturity Level 4 provide management with better input for strategic decisions, while encouraging delegation of operational details to people close to the processes. The process areas of Level 4 include:

• Competency Integration: to improve the efficiency and agility of interdependent work by integrating the process abilities of different workforce competencies.
• Empowered Workgroups: to invest workgroups with the responsibility and authority to determine how to conduct their business activities most effectively.
• Competency-Based Assets: to capture the knowledge, experience, and artifacts developed in performing competency-based processes for use in enhancing capability and performance.
• Quantitative Performance Management: to predict and manage the capability of competency-based processes for achieving measurable performance objectives.
• Organizational Capability Management: to quantify and manage the capability of the workforce and of the critical competency-based processes it performs.
• Mentoring: to transfer the lessons of greater experience in a workforce competency to improve the capability of other individuals or workgroups.

🔑 Definition — Competency-Based Assets: The knowledge, experience, and artifacts developed from performing competency-based processes that are captured for reuse to enhance capability and performance.

📌 Example: A design team creates a reusable library of software components and design documents from a successful project. This asset, created using a proven design process, is trusted and reused by other teams, accelerating future development.

The Optimizing Level: Maturity Level 5

At the Optimizing Level, the entire organization is focused on continual improvement. These improvements are made to the capability of individuals and workgroups, to the performance of competency-based processes, and to workforce practices and activities. The organization uses the results of the quantitative management activities established at Maturity Level 4 to guide improvements at Maturity Level 5. Although several individuals may be performing identical competency-based processes, they frequently exhibit individual differences in the methods and work styles they use to perform their assignments. At Maturity Level 5, individuals are encouraged to make continuous improvements to their personal work processes by analyzing their work and making necessary process enhancements. So is true for workgroups. Although individuals and workgroups continually improve their performance, the organization must be vigilant to ensure that performance at all levels remains aligned with organizational objectives. At Maturity Level 5, the process performance data collected across the organization is evaluated to detect instances of misalignment. Further, the impact of workforce practices and activities is evaluated to ensure that they encourage rather than discourage alignment. Corrective action is taken to realign performance objectives and results when necessary. The process areas of Level 5 include:

• Continuous Capability Improvement: to provide a foundation for individuals and workgroups to continuously improve their capability for performing competency-based processes.
• Organizational Performance Alignment: to enhance the alignment of performance results across individuals, workgroups, and units with organizational performance and business objectives.
• Continuous Workforce Innovation: to identify and evaluate improved or innovative workforce practices and technologies, and implement the most promising ones throughout the organization.

🔑 Definition — Organizational Performance Alignment: The process of ensuring that performance results at all levels (individual, workgroup, unit) are aligned with organizational performance and business objectives.

📌 Example: An organization collects performance data from all teams and identifies that one team's focus on speed is causing quality issues, misaligning with the organization's strategic objective of high-quality products. Corrective action is taken to realign their performance objectives.

⭐ Key Takeaways

The People CMM provides a structured path for organizations to systematically improve their workforce capability. The journey begins at the Initial Level (1), characterized by ad hoc and inconsistent practices, and progresses through the Managed Level (2) where managers take responsibility for unit-level practices like staffing and performance management. The Defined Level (3) shifts the focus to strategic alignment by identifying and developing workforce competencies that support core business objectives, creating an organization-wide infrastructure. The Predictable Level (4) enables the organization to manage capability quantitatively, using data to make performance predictions and empower workgroups, while the Optimizing Level (5) fosters a culture of continuous improvement where individuals and teams constantly enhance their processes, and all performance is kept aligned with strategic goals. Throughout this progression, each level builds upon the foundation of the previous one, transforming workforce practices from an afterthought into a strategic enabler.

🧠 Quick Revision Questions

1. What are the four typical characteristics exhibited by organizations at the Initial Level (Maturity Level 1)?
2. What is the primary focus of the Managed Level (Maturity Level 2), and what is one of its first measurable benefits?
3. How does the People CMM's concept of a "workforce competency" differ from an organization's "core competency"?
4. What three key benefits does the framework of workforce competencies enable at the Predictable Level (Maturity Level 4)?
5. What is the primary role of quantitative performance data at the Optimizing Level (Maturity Level 5)?

📘 Lecture 38 — Introduction to Measurements

📖 Overview: This lecture introduces the fundamental concepts of software measurement, explaining why it has become essential for good software engineering practice. It formally defines measurement, explores the distinction between entities and attributes, and discusses the consequences of neglecting measurement in software projects while outlining its key objectives and scope.

🗂️ Topics Covered

This lecture begins by establishing the pervasive nature of measurement in everyday life and technology, then formally defines measurement as the assignment of numbers or symbols to attributes of entities according to clearly defined rules. It explores the critical distinction between entities (objects/events) and their attributes (features/properties), raises difficult philosophical questions about what constitutes valid measurement, and distinguishes between direct measurement and indirect calculation. The lecture then addresses the neglect of measurement in software engineering, outlines objectives for software measurement from management and engineering perspectives, explains the importance of measurement for understanding, controlling, and improving processes and products, and finally surveys the broad scope of software metrics including cost estimation, productivity, quality, reliability, performance, complexity, and capability maturity assessment.

📝 Lecture Summary

Introduction to Measurements — Software measurement

Software measurement, once considered an obscure specialty, has become essential to good software engineering. Many of the best software developers measure characteristics of the software to get some sense of whether the requirements are consistent and complete, or whether the code is ready to be tested. Effective project managers measure attributes of process and product to tell when the software will be ready for delivery and whether the budget will be exceeded. Informed customers measure aspects of the final product to determine if it meets requirements and is of sufficient quality.

Measurement lies at the heart of many systems that govern our lives. Economic measurements determine price and pay increases. Measurements in radar systems enable us to detect aircraft when direct vision is obscured. Medical system measurements enable doctors to diagnose specific illnesses, and measurements in atmospheric systems form the basis for weather prediction. Without measurement, technology cannot function. Measurement is not solely the domain of professional technologists; each of us uses it in everyday life, such as using price as a measure of value, using height/weight/size for clothing, or calculating distance and speed for a journey.

In every case, some aspect of a thing is assigned a descriptor that allows us to compare it with others — we compare the price of one item with another, contrast clothing sizes, or compare distance travelled to distance remaining. The rules for assignment and comparisons are made according to a well-defined set of rules.

Measurement

Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to describe them according to clearly defined rules. Thus, measurement captures information about attributes of entities.

An entity is an object or an event in the real world, such as a person, a room, a journey, or the testing phase of a software project. We describe the entity by identifying characteristics important to us in distinguishing one entity from another. An attribute is a feature or property of an entity, such as the area or color of a room, the cost of a journey, or the elapsed time of the testing phase.

We often talk about entities and their attributes interchangeably in everyday speech ("it is cold today" actually means the air temperature is cold, or "Usman is taller than Ali" means his height is greater). This loose terminology is acceptable for everyday speech but is incorrect and unsuitable for scientific endeavors. It is wrong to say that we measure things or that we measure attributes; in fact, we measure attributes of things. It is ambiguous to say "we measure a room" since we can measure its length, area, or temperature. Similarly, it is ambiguous to say "we measure the temperature" since we measure the temperature of a specific geographical location under specific conditions.

We can describe entities by using attributes, often defining the attributes using numbers or symbols — price can be designated as Rupees or dollars, clothing size may be "small", "medium", or "large". These numbers and symbols are abstractions that reflect our perceptions of the real world. In defining them, we try to preserve certain relationships among entities, such that someone who is six feet in height is taller than someone who is five feet, and a "medium" T-shirt is smaller than a "large" T-shirt.

Measurement is a process whose definition is far from clear-cut, and many different authoritative views lead to different interpretations. To understand what measurement is, we must ask difficult questions:

1. In a room with blue walls, is "blue" a measure of the color of the room?
2. Is intelligence adequately measured by an IQ test score? Can soft drink quality be measured using expert ratings?
3. Some measures are not likely to be accurate because the measurement is imprecise or depends on judgment — is this a reason to reject them as bona fide measurements?
4. Even with reliable devices, there is a margin of error — how do we decide which error margins are acceptable?
5. Different scales measure the same attribute (meters, inches, feet for height) — when is a scale acceptable for the purpose to which it is put?
6. Why is it acceptable to say Fred is twice as tall as Joe, but not to say it is twice as hot today as yesterday? Why is it meaningful to calculate the mean of heights but not the mean of football jersey numbers?

💡 Why this matters: The concepts of time, temperature, and speed, once unmeasurable by primitive peoples, are now commonplace and easily measured. To improve the rigor of measurement in software engineering, we need not restrict the type or range of measurements we can make. Measuring the unmeasurable should improve our understanding of entities and attributes, making software engineering as powerful as other engineering disciplines.

There are two kinds of quantification: measurement and calculation. Measurement is a direct quantification of something, as in measuring the height of a tree or weight of a shipment of bricks. Calculation is indirect, where we take measurements and combine them into a quantified item reflecting some attribute whose value we are trying to understand — for example, valuation of a house for tax purposes, or in the decathlon athletics event where measures of running times and jumping distances are combined into an overall score using a complex weighting scheme.

Neglect of Measurements in SE

Measurement has been considered a luxury in software engineering. We fail to set measurable targets for our software products, promising that the product will be user-friendly, reliable, and maintainable without specifying clearly what these terms mean. As a result, when the project is complete, we cannot tell if we have met our goals. Projects without clear goals will not achieve their goals clearly.

We fail to understand and quantify the component costs of software projects — most projects cannot differentiate the cost of design from the cost of coding and testing. Since excessive cost is a frequent complaint, we cannot hope to control costs if we are not measuring the relative components of cost. We do not quantify or predict the quality of the products we produce, so we cannot tell a potential user how reliable a product will be in terms of likelihood of failure, or how much work will be needed to port it to a different environment. We allow anecdotal evidence to convince us to try new technologies without doing carefully controlled studies to determine if the technology is efficient and effective.

Objectives for Software Measurements

Even when a project is not in trouble, measurement is not only useful but necessary — how can you tell if your project is healthy if you have no measures of its health? Measurement is needed at least for assessing the status of your projects, products, processes, and resources.

From managers' perspectives, key objectives include: What does each process cost? How productive is the staff? How good is the code being developed? Will the user be satisfied with the product? How can we improve?

Importance of Measurement

Measurement is important for three basic activities:

First, there are measures that help us to understand what is happening during development and maintenance. We assess the current situation, establishing baselines that help us set goals for future behavior. The measurements make aspects of process and product more visible to us, giving us a better understanding of relationships among activities and the entities they affect.

Second, measurement allows us to control what is happening on our projects. Using our baselines, goals, and understanding of relationships, we predict what is likely to happen and make changes to processes and products that help us meet our goals. For example, we may monitor the complexity of code modules, giving thorough review only to those that exceed acceptable bounds.

Third, measurement encourages us to improve our processes and products. For instance, we may increase the number or type of design reviews based on measures of specification quality and predictions of likely design quality. You can neither predict nor control what you cannot measure.

Scope of Software Metrics

Software metrics embraces many activities involving some degree of software measurement:

• Cost and effort estimation: Managers provided original motivation for deriving and using software measures to predict costs during early phases. Many models are available to express cost and size.

• Productivity models and measures: Numerous attempts have defined measures and models for assessing staff productivity during different software processes and in different environments.

• Data collection: The quality of any measurement program depends on careful data collection. Consistent and accurate data collection is very difficult. Metrics data collection must be planned and executed in a careful and sensitive manner.

• Quality models and measures: Productivity cannot be viewed in isolation without an accompanying assessment of product quality. A number of quality models have measurements that can be combined with those of productivity models.

• Reliability models: Most quality models include reliability as a component factor, but the need to predict and measure reliability itself has led to a separate specialization in reliability modeling and prediction.

• Performance evaluation and models: Performance evaluation includes externally observable system performance characteristics such as response times and completion rates, as well as investigation of internal workings including algorithmic efficiency and complexity.

• Structural and complexity metrics: Desirable quality attributes like reliability and maintainability cannot be measured until operational code is available, yet we wish to predict which parts of the system are likely to be less reliable or more difficult to test. We measure structural attributes of software representations available in advance of execution, then try to establish empirically predictive theories for quality assurance, quality control, and quality prediction.

• Management by metrics: Measurement is becoming an important part of software project management. Customers and developers rely on measurement-based charts and graphs to decide if the project is on track. It is important to collect measures in a standard format for comparisons and contrast.

• Evaluation of methods and tools: To select a method or tool for adoption, investigations in the form of experiments, case studies, or surveys must be done before incorporation into an organization. These investigations cannot be done without careful, controlled measurement and analysis.

• Capability maturity assessment: SEI proposed capability maturity assessments to evaluate contractors, in use since the 1980s using quality models. Capability maturity assessment is based on key practices that every good contractor should be using. Other organizations are using their own capability assessments.

⭐ Key Takeaways

Measurement is the process of assigning numbers or symbols to attributes of entities according to clearly defined rules, distinguishing sharply between entities (objects/events) and their attributes (features/properties). Software engineering has historically neglected measurement by failing to set measurable targets, quantify costs, predict quality, and control processes, making measurement essential for understanding current status, controlling ongoing work, and improving future outcomes. The scope of software metrics is broad and includes cost estimation, productivity assessment, quality and reliability modeling, performance evaluation, structural complexity analysis, and capability maturity assessment. Students must remember that measurement and calculation are distinct — measurement is direct quantification while calculation is indirect combination of measures — and that the ultimate goal of software measurement is to make software engineering as rigorous and powerful as other engineering disciplines.

🧠 Quick Revision Questions

1. What is the formal definition of measurement, and why is it incorrect to say "we measure a room" or "we measure the temperature"?
2. What is the difference between an entity and an attribute, and why must this distinction be maintained in scientific measurement?
3. What are the three basic activities for which measurement is important in software engineering, and how does each contribute to process and product success?
4. What are the consequences of neglecting measurement in software engineering, particularly regarding project goals, cost control, and quality prediction?
5. List and briefly describe at least five different areas within the scope of software metrics, explaining what each area aims to measure or predict.

📘 Lecture 39 — Basics of Measurements

📖 Overview: This lecture introduces the fundamental concepts and challenges of measurement in software engineering. It explores why measuring software attributes is difficult compared to physical sciences, presents the representational theory of measurement as a formal framework, and covers empirical relations, mapping rules, measurement scales, and the distinction between direct and indirect measurement. Understanding these basics is crucial for establishing meaningful and useful software metrics.

🗂️ Topics Covered

This lecture begins by posing four fundamental questions about measuring software attributes, such as complexity and quality. It then introduces the representational theory of measurement from the physical sciences. The concept of empirical relations is explained, using comparisons like height and software product surveys. The lecture details the rules of mapping from the empirical to the mathematical world, the representation condition of measurement, and discusses measurement and models, including direct and indirect measures like programmer productivity and defect density. It concludes with an introduction to measurement scales (nominal, ordinal, interval, ratio, absolute) and prediction systems.

📝 Lecture Summary

Basics of Measurements

Ordinarily, when we measure physical attributes like length or temperature, we apply sophisticated tools and principles developed over centuries. However, we lack a comparably deep understanding of software attributes and the associated measurement tools. This leads to four difficult questions: 1) How much must we know about an attribute (e.g., complexity) before measuring it? 2) How do we know if we have measured the intended attribute? 3) What meaningful statements can we make from measurements? 4) What meaningful operations can we perform on measures (e.g., averaging productivity)? To answer these, we must establish the basics of a formal theory of measurement, starting from the physical sciences.

The Representational Theory of Measurement

In any measurement activity, rules provide consistency and a basis for interpreting data. Measurement theory codifies these rules, similar to how mathematicians define axioms for geometry. The representational theory of measurement seeks to formalize our intuition about the world: the data we obtain as measures should represent attributes of the entities we observe, and manipulation of the data should preserve the relationships we observe among the entities. Our intuition is the starting point for all measurement, leading to the concept of empirical relations.

🔑 Definition — Empirical Relation: A relation (like "taller than") for which there is a reasonable consensus about which pairs or groups of entities are in the relation.

Empirical Relations

We understand the world by comparing things. For instance, we can observe that Frankie is taller than Wonderman. This "taller than" is a binary empirical relation defined on pairs of people. Empirical relations can also be unary ("is tall") or ternary (comparing groups of three). These relations are mappings from the empirical world (entities and attributes) to the formal mathematical world. If we agree Frankie is taller than Wonderman, then any measure of height must assign a higher number to Frankie.

📌 Example: A survey of 100 users compared the functionality and user-friendliness of four word processors (A, B, C, D). The table for Functionality shows the percentage who preferred the row's program. For example, 80% of respondents preferred A over B. The table for User-Friendliness shows a different pattern, where preferences are much closer to 50%. The lecture asks if we can make judgments about greater or lesser functionality/user-friendliness from this data. Analyzing such results often clarifies the attribute itself. 💡 Why this matters: This illustrates how we can begin to understand attributes using relatively unsophisticated relationships without special tools, setting the stage for formal measurement.

Measurement is a mapping from the empirical world to the formal, relational world. A measure is the number or symbol assigned to an entity by this mapping to characterize an attribute. Rating formats include: Likert scale (Strongly Agree to Strongly Disagree), Forced ranking, Verbal frequency scale, Ordinal scale, Comparative scale, and Numerical scale.

Rules of Mapping

When mapping an attribute to a mathematical system (the range), we have choices over numbers (real, integers) or non-numeric symbols. A measure must specify the domain, range, and the rule for performing the mapping. Terminology can be imprecise (e.g., "Felix is 11" implies a mapping of age in whole years since birth). In software, measuring "lines of code" requires a clear definition of what constitutes a line.

Representation Condition of Measurement

By definition, each relation in the empirical relational system corresponds via the measurement to an element in a number system. The representation condition asserts that the mapping M must map entities into numbers and empirical relations into numerical relations such that the empirical relations preserve and are preserved by the numerical relations.

📐 Formula: A is taller than B if and only if M(A) > M(B). The empirical relation is preserved under M as a numerical relation. 📌 Example: For Frankie, Wonderman, and Peter, the representation condition would require that if "Frankie is taller than Wonderman" is true, then M(Frankie) > M(Wonderman). This must hold for all such empirical relations.

Measurement and Models

Direct measurement measures an attribute of an entity itself (e.g., length of source code, duration of testing process). Indirect measurement is derived from other direct measures and is often useful for making interactions visible.

🔑 Definition — Indirect Measure: A measure derived from two or more direct measures.

📌 Common Indirect Measures in SWE:

• Programmer productivity = LOC produced / person month effort
• Module defect density = number of defects / module size
• Defect detection efficiency = number of defects detected / total number of defects
• Requirements stability = number of initial requirements / total number of requirements
• Test effectiveness ratio = number of items covered / total number of items
• System spoilage = effort spent on fixing faults / total project effort

Where no previous measurement has been performed, direct measurement is natural. However, simple models can lead to more accurate indirect measurement later (e.g., temperature measured indirectly via mercury column length). Measurement for assessment evaluates existing entities (past events). Measurement for prediction uses models to predict attributes of entities that do not yet exist (e.g., early reliability assurance for a future system). A prediction system consists of a mathematical model with procedures for determining unknown parameters and interpreting results.

Measurement Scales and Scale Types

Not all measurement mappings are the same. The differences restrict the kind of analysis we can do. A measurement scale helps us understand which analyses are appropriate.

The lecture mentions five types of measurement scales:

• Nominal
• Ordinal
• Interval
• Ratio
• Absolute

⭐ Key Takeaways

This lecture establishes the foundational theory for software measurement. The key takeaway is that measuring software is fundamentally difficult due to our lack of deep understanding of software attributes, unlike physical sciences. The representational theory of measurement provides a formal framework where measurement is a mapping from empirical relations (like "taller than") to numerical relations, and the representation condition ensures that this mapping preserves the observed relationships. Students must understand the distinction between direct and indirect measures, the definition of key indirect measures like programmer productivity and defect density, and the crucial role of measurement scales in determining appropriate data analysis. Finally, the lecture distinguishes between measurement for assessing existing entities and measurement for predicting the attributes of future ones, setting the stage for model-based prediction.

🧠 Quick Revision Questions

1. What are the four fundamental questions about software measurement that highlight its difficulty, and how does the representational theory of measurement aim to answer them?
2. Define an empirical relation and explain the representation condition of measurement using the concept of "taller than" as an example.
3. Distinguish between a direct measure and an indirect measure, and provide two examples of common indirect measures in software engineering with their formulas.
4. What is the purpose of a measurement scale, and why is it important to know which scale type (e.g., nominal, ordinal, ratio) a measure belongs to?
5. Explain the difference between measurement for assessment and measurement for prediction. What components form a prediction system?

📘 Lecture 40 — Measurement Scales

📖 Overview: This lecture covers the fundamental types of measurement scales used in software metrics and introduces a goal-based framework for software measurement. It explains how different scales (nominal, ordinal, interval, ratio, absolute) determine what mathematical operations are permissible, and provides a structured approach to measuring processes, products, and resources in software development.

🗂️ Topics Covered

The lecture begins with a detailed classification of five measurement scales: nominal, ordinal, interval, ratio, and absolute, including their characteristics, admissible transformations, and examples from software engineering. It then explores the concept of meaningfulness in measurement, emphasizing that statements must be invariant under scale transformations. Finally, it presents a comprehensive goal-based framework for software measurement that classifies entities into processes, products, and resources, distinguishes between internal and external attributes, and introduces the Goal-Question-Metric (GQM) paradigm for selecting and implementing metrics.

📝 Lecture Summary

1 – Nominal Scale

• We define classes or categories, and then place each entity in a particular class or category, based on the value of the attribute. This is nominal measurement. Classes are not ordered. The empirical relation system consists only of different classes; there is no notion of ordering among the classes. Any distinct numbering or symbolic representation of the classes is an acceptable measure, but there is no notion of magnitude associated with the numbers or symbols.

🔑 Definition — Nominal Scale: A measurement scale where entities are placed into unordered classes or categories based on an attribute value. Any mapping that preserves distinct classes is acceptable.

📌 Example: Capturing the location of software faults (specification, code, design)

2 – Ordinal Scale

• The ordinal scale is often useful to augment the nominal scale with information about an ordering of the classes or categories. The ordering leads to analysis not possible with nominal measures. The empirical relation system consists of classes that are ordered with respect to the attribute. Any mapping that preserves the ordering (that is, any monotonic function) is acceptable. The numbers represent ranking only, so addition, subtraction, and other arithmetic operations have no meaning.

🔑 Definition — Ordinal Scale: A measurement scale where classes are ordered with respect to the attribute, but only ranking is meaningful; arithmetic operations have no meaning.

📌 Example: Complexity of software modules is described as: Trivial, Simple, Moderate, Complex, Incomprehensible

3 – Interval Scale

• The interval scale carries more information still, making it more powerful than nominal or ordinal scales. This scale captures information about the size of the intervals that separate the classes, so that we can in some sense understand the size of the jump from one class to another. An interval scale preserves order, as with an ordinal scale. An interval scale preserves differences but not ratios. That is, we know the difference between any two of the ordered classes in the range of the mapping, but computing the ratio of two classes in the range does not make sense. Addition and subtraction are acceptable on the interval scale, but not multiplication and division.

🔑 Definition — Interval Scale: A measurement scale that preserves order and differences between classes, but not ratios. Addition and subtraction are meaningful, but multiplication and division are not.

📌 Example: Complexity of software modules is described as: Trivial (0), Simple (2), Moderate (4), Complex (6), Incomprehensible (8)

4 – Ratio Scale

• We would like to be able to say that one liquid is twice as hot as another, or that one project took twice as long as another. This need for ratios gives rise to ratio scale. It is a measurement mapping that preserves ordering, the size of intervals between entities, and ratios between entities. There is a zero element, representing total lack of the attribute. The measurement mapping must start at zero and increase at equal intervals, known as units. All arithmetic can be meaningfully applied to the classes in the range of the mapping.

🔑 Definition — Ratio Scale: A measurement scale with a true zero point that preserves ordering, interval sizes, and ratios. All arithmetic operations are meaningful.

📌 Example: The length of software code is measurable on a ratio scale

5 – Absolute Scale

• There is only one way in which the measurement can be made, so M and M' must be equal. The absolute scale is the most restrictive of all. The measurement for an absolute scale is made simply by counting the number of elements in the entity set. The attribute always takes the form "number of occurrences of x in the entity". There is only one possible measurement mapping, namely the actual count. All arithmetic analysis of the resulting count is meaningful.

🔑 Definition — Absolute Scale: The most restrictive measurement scale where measurement is made by counting the number of elements; only one possible mapping exists (the actual count).

📌 Example: LOC is an absolute-scale measure of the attribute "number of lines of code" of a program. However, LOC is not an absolute-scale measure of length, because there are different ways to measure length (such as thousands of LOC, number of characters, and number of bytes).

Scales of Measurement

💡 Why this matters: The permissible transformations for each scale define what statistical operations are valid. Using inappropriate operations (like calculating the mean of ordinal data) can lead to meaningless conclusions.

Meaningfulness in Measurement

• Can we deduce meaningful statements about the entities being measured? A statement involving measurement is meaningful if its truth value is invariant of transformations of allowable scales.

📌 Example of meaningful statement: "The number of errors discovered during the integration testing of program X was at least 100" 📌 Example of meaningless statement: "The cost of fixing each error in program X is at least 100" (meaningless without reference to a particular scale) 📌 Example of meaningful statement: "A semantic error takes twice as long to fix as a syntactic error" 📌 Example of meaningless statement: "A semantic error is twice as complex as a syntactic error" (not meaningful without clarifying complexity)

A Goal-based Framework for Software Measurement

• Now, we'll talk about a conceptual framework for the diverse software-measurement activities that contribute to an organization's software practices. These practices may include not only usual development and maintenance activities, but also any experiments and case studies. This framework is based on three principles:
  • Classifying the entities to be examined
  • Determining relevant measurement tools
  • Identifying the level of maturity that your organization has reached

Classifying Software Measures

• The first obligation of any software measurement activity is identifying the entities and attributes we wish to measure. In software, there are three such classes:

  • Processes: Collections of software-related activities
  • Products: Any artifacts, deliverables, or documents that result from a process activity
  • Resources: Entities required by a process activity

• A process is usually associated with some timescale, where activities are ordered or related in a way that depends on time. Resources and products are associated with the process. Each process activity has resources and products that it uses, as well as products that are produced.

• Before discussing these entities in detail, we distinguish between internal and external attributes:

  • Internal attributes: Can be measured purely in terms of the product, process, or resource itself, separate from its behavior
  • External attributes: Can be measured only with respect to how the product, process, or resource relates to its environment

📌 Examples of Internal and External Attributes:

• Internal: software size, complexity, dependencies among modules (determined without execution)
• External: number of failures experienced by a user, difficulty in navigating screens, database search time (measured when code is executed)

Processes (3.1.1)

• We often have questions about our software development activities and processes that measurement can help us answer. Only a limited number of internal process attributes can be measured directly: the duration of the process, the effort associated with the process, and the number of incidents of a specified type arising during the process.

📌 Example: Measuring the effectiveness of a requirements review process by measuring the number of requirements errors found during specification, compared to those found during integration testing.

📐 Formula: Average Cost of each Defect Detected = Cost / Number of Errors Found → This is an indirect measure

📌 Example: Effectiveness of Software Inspections — measure the average amount of effort expended per thousand lines of code reviewed

📌 Example: Effectiveness of Software Testing Process — track the number of errors identified in each sub-process (unit testing, integration testing, system testing, acceptance testing) along with the duration and cost

• External process attributes include controllability, observability, and stability.

Products (3.1.2)

• Products are not restricted to items management commits to deliver to the customer. Any artifact or document produced during the software life cycle can be measured and assessed (prototypes, test harnesses, documents).

• External Product Attributes: Depend on both product behavior and environment.

  • 📌 Example: Measuring reliability of code must consider the machine on which the program runs and the mode of operational usage
  • 📌 Example: The understandability of a document depends on the reader's experience
  • External attributes include: maintainability, usability, integrity, efficiency, testability, reusability, portability, interoperability

• Internal Product Attributes: Sometimes easy to measure (size by pages or words). More difficult attributes include code complexity, structuredness, module coupling, and cohesiveness.

📌 Example: A software system design document consisting of module design documents

• Average module size can be derived by calculating the size of each module
• Size of one Module = No of DFD Bubbles
• Average Module Size = 1/n ∑(i=1 to n) No of DFD Bubblesᵢ

💡 Why this matters: Internal attributes (like faults found during inspection) provide insight into external attributes (like reliability based on failures), even though they measure different aspects.

• Quality is multi-dimensional, similar to GNP (Gross National Product) of a country. The GNP is a weighted combination of key goods and services; the weights reflect priorities. Similarly, we measure quality by measuring and controlling a number of internal (structural) product attributes.

Resources (3.1.3)

• Resources include any input for software production: Personnel (individuals and teams), Materials (including office supplies), Tools (both software and hardware), Methods.

• We measure resources to determine their magnitude, cost, and quality. These measures help us understand and control the process.

📐 Formula: Productivity = Amount of Output / Effort Input → A resource measure combining a process measure (input) with a product measure (output)

📌 Example: Staff attributes that can be measured include experience, age, or intelligence of a developer, which may affect design or code quality

Determining What to Measure?

• Utility is in the eye of the beholder. The Goal-Question-Metric (GQM) approach to process and metrics has proven to be a particularly effective approach to selecting and implementing metrics.

• Steps of GQM:

  1. Express the overall goals of your organization
  2. Generate questions whose answers you must know to determine if goals are being met
  3. Analyze each question in terms of what measurements you need to answer each question

• Once you have a list of possible measures, assess your organization to see whether it is capable of providing useful information. The more mature your process, the more that is visible and therefore measurable.

📌 Example: If developers cannot tell when design ends and coding begins, measuring the duration of the design activity will be impossible.

GQM Paradigm

• A measurement program can be more successful if it is designed with the goals of the project in mind. The GQM approach involves three steps:
  1. List the major goals of the development or maintenance project
  2. Derive from each goal the questions that must be answered to determine if the goals are being met
  3. Decide what must be measured in order to be able to answer the questions adequately

📌 Example Goal: Evaluate the effectiveness of using a coding standard

• Goal-derived questions: Who is using the standard? How does their productivity compare? How does quality compare?
• Metrics needed: Proportion of coders using the standard, experience profile, productivity (effort/size in LOC or FPs), number of errors found in code

💡 Why this matters: By deriving measurements from goals, it becomes clear how to use the resulting data. Several metrics might be generated from a single goal, and one metric may support several goals.

⭐ Key Takeaways

The five measurement scales (nominal, ordinal, interval, ratio, absolute) form a hierarchy where each successive scale adds more information and permits more mathematical operations, with absolute being the most restrictive. The meaningfulness of measurement statements depends on invariance under permissible scale transformations — a statement about software attributes is only meaningful if its truth remains unchanged when the measurement scale is transformed. Measurement entities in software are classified into processes (activities), products (artifacts), and resources (inputs), each with both internal attributes (measured within the entity) and external attributes (measured in relation to the environment). The Goal-Question-Metric (GQM) paradigm provides a structured approach to measurement by starting with organizational goals, deriving questions that assess goal achievement, and identifying specific metrics needed to answer those questions. Successful measurement programs require understanding organizational process maturity, as more mature processes have greater visibility and are therefore more measurable.

🧠 Quick Revision Questions

1. What are the five measurement scales, and what mathematical operations are permissible for each scale?
2. According to the GQM paradigm, what are the three steps for establishing a measurement program?
3. Distinguish between internal and external attributes, and provide one example of each for software products.
4. What does it mean for a measurement statement to be "meaningful," and why is the concept of admissible transformations important?
5. How would you classify software LOC (Lines of Code) as a measurement scale, and what attribute is it actually measuring according to the absolute scale definition?

📘 Lecture 41 — Goal-Question-Metric (GQM)

📖 Overview: This lecture introduces the Goal-Question-Metric (GQM) approach, a structured measurement framework for software process improvement. It explains how to define measurement goals, derive questions to assess those goals, and associate metrics to answer the questions quantitatively. The lecture emphasizes that GQM ensures measurement is purposeful, traceable, and linked to organizational objectives.

🗂️ Topics Covered

The GQM approach is based on the assumption that organizations must first specify goals, then trace those goals to data, and provide a framework for interpreting data. The resulting measurement model has three levels: Conceptual (Goal), Operational (Question), and Quantitative (Metric). The lecture covers templates for goal definition (purpose, perspective, environment), guidelines for defining product and process-related questions, and provides detailed examples including AT&T’s inspection process and change request processing. It also explains how GQM complements the entity-attribute measurement framework.

📝 Lecture Summary

Goal Question Metric Approach

The Goal Question Metric (GQM) approach is based upon the assumption that for an organization to measure in a purposeful way: it must first specify the goals for itself and its projects, then trace those goals to the data that are intended to define those goals operationally, and finally provide a framework for interpreting the data with respect to the stated goals. The result of applying the GQM approach is the specification of a measurement system targeting a particular set of issues and a set of rules for the interpretation of the measurement data. The resulting measurement model has three levels: Conceptual, Operational, and Quantitative.

💡 Why this matters: GQM ensures that every metric collected directly serves a defined business or project goal, preventing wasteful and meaningless data collection.

1 – Conceptual Level (Goal)

A goal is defined for an object, for a variety of reasons, with respect to various models of quality, from various points of view, relative to a particular environment. Objects of measurement are:

• Products: Artifacts, deliverables, and documents produced during the system life cycle (e.g., specifications, designs, programs, test suites).
• Processes: Software-related activities normally associated with time (e.g., specifying, designing, testing, interviewing).
• Resources: Items used by processes to produce their outputs (e.g., personnel, hardware, software, office space).

2 – Operational Level (Question)

A set of questions is used to characterize the way the assessment/achievement of a specific goal is going to be performed based on some characterizing model. Questions try to characterize the object of measurement (product, process, resource) with respect to a selected quality issue and to determine its quality from the selected viewpoint.

3 – Quantitative Level (Metric)

A set of data is associated with every question in order to answer it in a quantitative way. The data can be:

• Objective: If they depend only on the object that is being measured and not on the viewpoint from which they are taken (e.g., number of versions of a document, staff hours spent on a task, size of a program).
• Subjective: If they depend on both the object that is being measured and the viewpoint from which they are taken (e.g., readability of a text, level of user satisfaction). A number of templates are available to aid in generating the goals, questions, and metrics.

🔑 Definition — Objective Data: Data that depends only on the object being measured, not on the viewpoint. Example: staff hours spent on a task. 🔑 Definition — Subjective Data: Data that depends on both the object being measured and the viewpoint from which they are taken. Example: level of user satisfaction.

Template for Goal Definition

• Template for Purpose: To (characterize, evaluate, predict, motivate, etc.) the (process, product, model, metric, etc.) in order to (understand, assess, manage, engineer, learn, improve, etc.) it.
  • Example: To evaluate the maintenance process in order to improve it.
• Template for Perspective: Examine the (cost, effectiveness, correctness, defects, changes, product measures, etc.) from the viewpoint of the (developer, manager, customer, etc.).
  • Example: Examine the cost from the viewpoint of the manager.
• Template for Environment: The environment consists of the following: (process factors, people factors, problem factors, methods, tools, constraints, etc.).
  • Example: The maintenance staff are poorly motivated programmers who have limited access to tools.

Goal Question Metric Approach (continue)

The GQM approach also provides separate guidelines for defining product-related and process-related questions. Steps involve defining the process or product, defining the relevant attributes, and obtaining feedback related to the attributes. What constitutes a goal or question may be vague, and several levels of refinement may be required. We can relate the GQM templates to the attribute framework. A goal or a question can be associated with at least one pair of entities and attributes. The use of the measure is determined by the goals and questions, so that assessment, prediction, and motivation are tightly linked to the data analysis and reporting. Thus, the GQM complements the entity-attribute measurement framework. The results of a GQM analysis are a collection of measurements related by goal tree and overall model. However, GQM does not address issues of measurement scale, objectivity, or feasibility. So the GQM measures should be used with care, remembering the overall goal of providing useful data.

Example of GQM

AT&T used GQM to help determine which metrics were appropriate for assessing their inspection process. Their goals, questions, and metrics are shown below:

AT&T Goals, Questions, and Metrics for Inspection Process – Plan

• Goal: Plan
  • Question: How much does the inspection process cost?
    • Metrics: 1. Average effort per KLOC, 2. Percentage of reinspections, 3. Average effort per KLOC
  • Question: How much calendar time does the inspection process take?
    • Metrics: 4. Total KLOC inspected

AT&T Goals, Questions, and Metrics for Inspection Process – Monitor and Control

• Goal: Monitor and Control
  • Question: What is the quality of inspected software?
    • Metrics: 1. Average faults detected per KLOC, 2. Average inspection rate, 3. Average preparation rate
  • Question: To what degree did the staff conform to the procedures?
    • Metrics: 4. Average inspection rate, 5. Average preparation rate
  • Question: What is the status of the inspection process?
    • Metrics: 6. Average lines of code inspected, 7. Percentage of reinspections, 8. Total KLOC inspected

AT&T Goals, Questions, and Metrics for Inspection Process – Improve

• Goal: Improve
  • Question: How effective is the inspection process?
    • Metrics: 1. Defect removal efficiency, 2. Average faults detected per KLOC, 3. Average inspection rate, 4. Average preparation rate, 5. Average lines of code inspected
  • Question: What is the productivity of the inspection process?
    • Metrics: 6. Average effort per fault detected, 7. Average inspection rate, 8. Average preparation rate, 9. Average lines of code inspected

GQM-Goal

A GQM model is a hierarchical structure starting with a goal (specifying purpose of measurement, object to be measured, issue to be measured, and viewpoint from which the measure is taken). So a GQM goal has three coordinates and a purpose.

GQM-Question

The goal is refined into several questions that usually break down the issue into its major components. In general, we will ask at least three groups of questions:

• Group 1: How can we characterize the object (product, process, or resource) with respect to the overall goal of the specific GQM model?
• Group 2: How can we characterize the attributes of the object that are relevant with respect to the issue of the specific GQM model?
• Group 3: How do we evaluate the characteristics of the object that are relevant with respect to the issue of the specific GQM model?

GQM-Metric

Once the questions have been developed, we proceed to associating the question with appropriate metrics. The factors we consider are many, among them:

• Amount and quality of existing data: maximize the use of existing data sources if available and reliable.
• Maturity of the objects of measurement: apply objective measures to more mature objects, and use more subjective evaluations for informal or unstable objects.
• Learning process: GQM models need refinement and adaptation; measures must help evaluate both the object of measurement and the reliability of the model used to evaluate it.

Another Example of GQM

Goal: Improve timelines of change request processing

GQM - Goals A GQM model is a hierarchical structure starting with a goal. A GQM goal has three coordinates:

1. Issue: Timeliness
2. Object (process): Change request processing
3. Viewpoint: Project manager

GQM - Questions The goal is refined into several questions. We try to ask at least three groups of questions.

GQM – Questions-Group 1: How can we characterize the object with respect to the overall goal?

• Example: What is the current change request processing speed?
• Example: Is the (documented) change request process actually performed?

GQM – Questions-Group 2: How can we characterize the attributes of the object that are relevant?

• Example: What is the deviation of the actual change request processing time from the estimated one?
• Example: Is the performance of the process improving?

GQM – Questions-Group 3: How do we evaluate the characteristics of the object that are relevant?

• Example: Is the current performance satisfactory from the viewpoint of the project manager?
• Example: Is the performance visibly improving?

GQM – Metrics Once the questions have been developed, we associate them with appropriate metrics. Factors include: amount and quality of existing data, maturity of objects of measurement, and the learning process.

GQM – Model

⭐ Key Takeaways

The GQM approach ensures that measurement is driven by goals, not data availability, and provides a three-level structure: goals at the conceptual level, questions at the operational level, and metrics at the quantitative level. A GQM goal specifies purpose, object, issue, and viewpoint, and is refined into at least three groups of questions: characterizing the object, its relevant attributes, and evaluating those attributes. Metrics can be objective (independent of viewpoint) or subjective (dependent on viewpoint), and their selection depends on data availability, object maturity, and the need for model refinement. The approach is illustrated with two real-world examples: AT&T’s inspection process (goals of plan, monitor/control, improve) and change request processing (goal of improving timelines), showing how each goal generates specific questions and measurable metrics. GQM complements the entity-attribute measurement framework but does not address scale, objectivity, or feasibility, so measures should be used with care.

🧠 Quick Revision Questions

1. What are the three levels of a GQM measurement model and what does each level represent?
2. What are the three objects of measurement in the conceptual level of GQM? Give one example of each.
3. What is the difference between objective and subjective data in GQM? Provide one example of each.
4. What are the three coordinates of a GQM goal, and what is an example of a goal definition using them?
5. What are the three groups of questions that should be asked when refining a GQM goal? Provide an example question for each group based on the change request processing scenario.

📘 Lecture 42 — Software-Metrics Data Collection

📖 Overview: This lecture emphasizes that software measurement is only as valuable as the quality of the data collected. It explores the critical characteristics that define “good data,” outlines the questions an organization must ask before and during data collection, and distinguishes between raw and refined data in the software measurement process.

🗂️ Topics Covered

The lecture begins by stating that software measurement is only as good as the data collected, citing Moroney (1950) on the necessity of a clear purpose. It then defines what constitutes good data by examining six key qualities: correctness, accuracy, appropriate precision, consistency, association with time/activity, and replicability. Finally, it introduces the two types of data (raw and refined) and illustrates the role of data collection in the overall software measurement framework.

📝 Lecture Summary

Software-Metrics Data Collection

Software measurement is only as good as the data that are collected and analyzed. In other words, we cannot make good decisions with bad data. Moroney (1950) warned that data should be collected with a clear purpose and a clear idea of how they will be analyzed. We need to ask questions like “What constitutes good data?”

What is Good Data?

Measures should be well-defined and valid, meaning that our measures should reflect the attributes we claim they do. However, even with a well-defined measure, we need to ask several questions about the data.

• Are they Correct? Correctness means that the data were collected according to the exact rules of definition of the metric. For example, if a lines of code count is supposed to include everything but comments, then a check for correctness assures us that no comments were counted. A measure of process duration is correct if time is measured from the beginning of one specified activity and ends at the completion of another specified activity.

• Are they Accurate? Accuracy refers to the difference between the data and the actual value. For example, time measured using an analog clock may be less accurate than time measured using a digital clock.

• Are they Appropriately Precise? Precision deals with the number of decimal places needed to express the data. For instance, activity duration need not be reported in hours, minutes, and seconds; hours or days are usually sufficient. Likewise, it is not necessary to calculate the mean cyclomatic number to several decimal places, since cyclomatic number expresses one plus the number of decisions in a module; fractions of a decision to several decimal places are meaningless.

• Are they Consistent? Data should be consistent from one measuring device or person to another, without large differences in value. Thus, two evaluators should calculate the same or similar function-point values from the same requirements document. Similarly, when the same data value is computed repeatedly over time, the data should be captured in the same way. For example, we often measure the growth in size of a product over time, especially during maintenance. We want to be sure that the size measure is calculated the same way each time, so that the resulting measures are comparable.

• Are they Associated with a Particular Activity or Time Period? If so, then the data should be time-stamped, so that we know exactly when they were collected. This association of values allows us to track trends and compare activities.

• Can they be Replicated? Data are often collected to support surveys, case studies, and experiments. These investigations are frequently repeated under different circumstances, and the results compared. At the very least, project histories and study results are stored in a historical database, so that baseline measures can be established and organizational goals set. Unless the data collection from one study can be replicated on others, this amalgamation is impossible. Thus, it is very important to assess the quality of data and data collection before data collection begins. Your measurement program must specify not only what metrics to use, but also what precision is required, what activities and time periods are to be associated with data collection, and what rules govern the data collection (such as usage of a specific tool for data collection). Of critical importance is the definition of the metric. Terminology must be clear and detailed, so that all involved understand what the metric is and how to collect it.

How to Define the Data?

There are two kinds of data with which we are concerned:

• Raw data
• Refined data

Raw data results from the initial measurement of process, product, or resource. But through the refinement process, raw data are transformed into refined data by extracting essential data elements. The analysts can then derive values about attributes.

The Role of Data Collection in Software Measurement

This section appears as a figure in the original text, illustrating the transformation of raw data into refined data for analysis.

⭐ Key Takeaways

A student must remember that software data quality is paramount; bad data leads to bad decisions. The six key attributes of good data are correctness, accuracy, precision, consistency, temporal association, and replicability. Correctness demands strict adherence to the metric’s definition, while precision requires using only meaningful decimal places (e.g., not reporting cyclomatic complexity to several decimals). Raw data is the initial measurement, which is then refined into essential data elements for analysis. Finally, a measurement program must specify not just the metrics, but also the rules, precision, and terminology for consistent data collection across projects.

🧠 Quick Revision Questions

1. What does it mean for data to be “correct” according to the lecture?
2. Why is it unnecessary to report the mean cyclomatic number to several decimal places?
3. How does “consistency” apply when two evaluators calculate function points from the same requirements document?
4. What is the difference between raw data and refined data in the context of software measurement?
5. Why is it critical to assess the quality of data collection before data collection begins?

📘 Lecture 43 — Process Metrics

📖 Overview: This lecture focuses on process metrics, which are used to improve software development and maintenance processes. It explains how to define, collect, store, and analyze data for measuring software quality, defects, and process effectiveness, emphasizing the importance of consistent terminology and well-planned data collection.

🗂️ Topics Covered

The lecture begins by discussing the difference between direct and indirect measures and the importance of GQM analysis for defining what to measure. It then covers software quality terminology, including faults, failures, and errors, followed by guidelines on how and when to collect data, and how to store and extract it using a DBMS. The main focus is on process metrics such as defect density, defect arrival rate, test effectiveness, defects by phase, defect removal effectiveness, and metrics for software maintenance.

📝 Lecture Summary

Back to defining data for collection

Deciding what to measure is the first step. We must specify which direct measures are needed and which indirect measures may be derived from them. Sometimes, from a GQM analysis, we determine the indirect measures we want and then work backward to find the required direct measures. Most organizations differ in business goals, cultures, and staff skills, so a GQM analysis may result in different metrics at different companies. This is why GQM is preferable to adopting a "one-size-fits-all" standard measurement set. Nevertheless, most organizations share similar interests in software quality, cost, and schedule, so most developers collect metrics about these aspects.

💡 Why this matters: Understanding that metrics must be tailored to specific organizational goals through GQM prevents collecting irrelevant or misleading data.

The problem with problems

No software developer consistently produces perfect software the first time. It is important to measure different aspects of software quality to determine: how many problems have been found with a product; how effective are the prevention, detection, and removal processes; whether the product is ready for release; and how the current version compares in quality with previous or competing versions.

Software Quality Terminology

A fault occurs when a human error results in a mistake in some software product. The fault is the encoding of the human error. A failure is the departure of a system from its required behavior. Failures can be discovered both before and after system delivery. We need to use and understand terminology consistently, clearly defining terms like faults, errors, processing errors, anomalies, defects, bugs, failures, and crashes. We also need a clear way of describing what we do in reaction to problems. For example, if an investigation of a failure results in the detection of a fault, then we make a change (or changes) to the product to remove it. Whenever a problem is observed, we want to record its key elements: location, timing, symptom, end result, mechanism, cause, severity, and cost.

🔑 Definition — Fault: a mistake in some software product resulting from a human error. 🔑 Definition — Failure: the departure of a system from its required behavior.

How to Collect Data?

Since software development is an intellectual activity, data collection requires human observation and reporting. Managers, analysts, programmers, testers, and users must record raw data on forms, which is subject to bias, error, omission, and delay. Automatic data capture is desirable, especially for recording execution time, but often there is no alternative to manual collection. To ensure data accuracy and completeness, we must plan our collection effort. Ideally, we should: keep procedures simple; avoid unnecessary recording; train staff; provide prompt and useful results feedback; and validate all data at a central collection point. The last point is critical, as studies show half the data collected can be corrupted.

Planning for data collection involves several steps. First, decide which products to measure based on your GQM analysis. The GQM analysis suggests which attributes and measures to evaluate. Once committed, you must decide exactly which attributes to measure and how indirect measures will be derived. Direct and indirect measures may be related by a measurement model. Once the set of metrics is clear, you must devise a scheme for identifying each entity (products, versions, failures, faults). This enables you to proceed to form design. Finally, establish procedures for handling forms, analyzing data, and reporting results. Define who fills in what, when, and where, and set up a central collection point. If no one person has responsibility for a given step, the data collection process will stop.

When to Collect Data

Data-collection planning must begin when project planning begins. Actual data collection takes place during many phases of development. Some data are collected at the start of the project, while other data collection is done later. Data about faults found by inspections and tests should be recorded consistently to determine the relative effectiveness of these activities. It is essential that data-collection activities become part of the regular development process. It is helpful to compare a model of the normal development process with a list of desired measurements and map measurements directly to the process model. If something needs to be measured and there is no obvious place to collect the data, the process can be modified to enable measurement.

How to Store and Extract Data

Raw software-engineering data should be stored on a database. A DBMS (Database Management System) is an automated tool for organizing, storing, and retrieving data, and it has many advantages over manual record-keeping.

Process Metrics

Process metrics are those that can be used for improving the software development and maintenance processes. Examples include the effectiveness of defect removal during development, the pattern of testing defect arrival, and the response time of the fix process. Compared to end-product quality metrics, process quality metrics are less formally defined, and their practices vary greatly among software developers.

Defect density

Defect density never follows a uniform distribution. It is an indication of defect injection in code or other software products. If a piece of code has higher testing defects, it could be due to more effective testing or higher latent defects. Defects per KLOC (thousand lines of code) or another denominator is a simple metric that is a good indicator of quality while the software is still under testing. It is especially useful for monitoring subsequent releases of the same product within the same organization, as release-to-release comparisons are not contaminated by extraneous factors.

🔑 Definition — Defect density: the number of defects found per unit size (e.g., per KLOC) of a software product.

Defect arrival rate

The defect arrival rate is the number of defects found during testing measured at regular intervals over some period of time. A set of values is associated with this metric. When plotted on a graph, the data may rise (positive arrival rate), stay flat (constant rate), or decrease (negative arrival rate). Interpretation can be difficult. A negative rate might indicate the product is improving, but you must eliminate other possible causes, such as test effectiveness declining over time, or the test organization being understaffed.

📐 Formula: Defect arrival rate = Number of defects found during a specific time interval.

Test effectiveness

Test effectiveness measures how effective formal tests are at finding defects. It is calculated by dividing the number of defects found by formal tests (Dn) by the total number of formal tests (Tn). When calculated at regular intervals and plotted, test effectiveness can be observed over time. A rising graph may indicate improving test effectiveness, while a falling graph may indicate waning effectiveness.

📐 Formula: TE = Dn / Tn → Test effectiveness equals the number of defects found by formal tests divided by the total number of formal tests.

Defects by phase

The defects by phase metric is a variation of the defect arrival rate metric. At the conclusion of each discrete phase of the development process, a count of new defects is taken and plotted to observe a trend. It is much less expensive to eliminate defects early. If the graph is rising, it suggests that defect detection and removal methods in earlier phases are not effective. If the graph is falling, early defect detection and removal is effective. This can demonstrate the snowball effect, where defects found early prevent many more from being introduced later.

Defect removal effectiveness

Defect removal effectiveness (DRE) can be calculated by dividing the number of defects removed prior to release (Dr) by the sum of defects removed prior to release and the total number of defects that remain in the product at release (Dt). When multiplied by 100, this value can be expressed as a percentage.

📐 Formula: DRE = Dr / (Dr + Dt) → Defect removal effectiveness equals defects removed before release divided by the total defects before release (removed + remaining).

Metrics for Software Maintenance Process

When a software product is released, it enters the maintenance phase. During this phase, the defect arrivals by time interval and customer problem calls by time interval (which may or may not be defects) are the de facto metrics.

⭐ Key Takeaways

The most critical points from this lecture are: first, data collection must be carefully planned using GQM analysis to align metrics with organizational goals, and all data must be validated at a central point to avoid corruption. Second, consistent terminology (e.g., fault vs. failure) is essential for accurate measurement and communication across the organization. Third, key process metrics include defect density, defect arrival rate, test effectiveness, defects by phase, and defect removal effectiveness, each providing different insights into quality and process efficiency. Fourth, data collection should be integrated into the regular development process, with measurements mapped directly to process activities. Finally, removing defects early in the lifecycle is significantly more cost-effective, and metrics like defects by phase and DRE help evaluate this effectiveness.

🧠 Quick Revision Questions

1. What is the difference between a direct measure and an indirect measure in software measurement?
2. Why is it important to validate all data at a central collection point, and what problem does this address?
3. What does a rising defect arrival rate graph potentially indicate about test effectiveness or product quality?
4. How is Defect Removal Effectiveness (DRE) calculated, and what does a high DRE percentage signify?
5. Why is it more valuable to use defects by phase metric rather than just a total defect count at the end of a project?

📘 Lecture 44 — Software Process Improvement: Your Mileage May Vary

📖 Overview: This lecture explores the critical metrics used in software maintenance to track defects and fix quality, then transitions to the strategic application of these metrics. It emphasizes the importance of distinguishing between private and public data to avoid using metrics punitively against individuals. Finally, it introduces the concept that every organization’s process improvement journey is unique, encapsulated in the idea that "your mileage may vary."

🗂️ Topics Covered

The lecture covers key metrics in software maintenance, including defect backlog, backlog management index (BMI), fix response time, percent delinquent fixes, and defective fixes. It then discusses the strategic application of software metrics through sensitive usage of data, process improvement via defect analysis, and validation of best practices. A major focus is on the distinction between private and public data, and the roles of functional management, project management, and the project team in handling metrics. The lecture concludes with the “Defect Cube” for dissecting defects and the spiral model for process improvement adoption.

📝 Lecture Summary

Metrics in Software Maintenance

The lecture begins by stating that while the number of problem arrivals is largely determined by the development process before maintenance, what can be done during maintenance is to fix defects quickly and with excellent quality. This can improve customer satisfaction even if it doesn't improve the product's defect rate.

Key metrics in this phase include:

• Defect backlog: A count of the number of defects in the product following its release that require a repair. It is usually measured at regular intervals for trend analysis. By itself, it provides little useful information (e.g., what does a count of 128 tell you?). A more useful representation is by defect severity.

• Backlog management index (BMI): This metric tracks whether a team is gaining or losing ground on the defect backlog. If new defects exceed closed defects, the team is losing ground; if they close more than arrive, they are gaining ground.

🔑 Definition — Backlog Management Index (BMI): Calculated by dividing the number of defects closed during some period (Dc) by the number of new defects that arrived during that same period (Dn).

📐 Formula: BMI = Dc / Dn → Plain-English meaning: If the result is greater than 1, your team is closing defects faster than new ones are arriving, meaning they are "gaining ground."

📌 Example: If a team closes 30 defects in a week but 20 new defects arrive, the BMI is 30/20 = 1.5. This indicates the team is gaining ground on the backlog.

• Fix response time: The average time it takes a team to fix a defect. It can be measured as elapsed time from discovery to unverified fix or from discovery to verified fix. A better alternative is to measure fix response time by severity.

• Percent delinquent fixes: A fix is delinquent if it exceeds your established fix response time criteria (e.g., a 48-hour maximum).

🔑 Definition — Percent Delinquent Fixes (PDF): The percentage of fixes that exceed the established fix response time criteria.

📐 Formula: PDF = (Fd / Fn) * 100 → Plain-English meaning: The number of delinquent fixes (Fd) divided by the number of non-delinquent fixes (Fn), multiplied by 100.

📌 Example: If a team has 5 delinquent fixes and 20 non-delinquent fixes in a month, the PDF is (5/20) * 100 = 25%.

• Defective fixes: A fix that is later found to be defective or creates additional problems. This metric (also known as fix quality) requires tracking reopened defects and new defects caused by a fix.

Strategic Application of Software Metrics

This section covers the sensitive and strategic use of metrics data for process improvement.

• Sensitive usage of metrics data: Use the concept of public and private data to drive decisions on how to collect and analyze data.
• Process improvement through defect analysis: Analyze defects to improve processes, not to blame people.
• Validation of best practices.

Private Data

Individuals prefer to keep defect data private. The goal is to analyze defects to improve processes. Private data becomes public at well-defined handoffs or transitions, introducing accountability.

• Inspections: Inspections are such transitions. If properly run, defects found here are less embarrassing than those found later. At this point, data is no longer private, and the focus of accountability shifts from the individual to the team.
• Project's Private Data: Includes time data. The principle of "Information Hiding" is applied during project review.

Public Data

Metrics for public data include calendar times, defect rates, project costs, and measure of functionality.

The lecture provides a detailed table showing the flow of data from private to public:

💡 Why this matters: A key lesson is to apply your best management sensitivity to interpretation and use; never use metrics to measure individuals.

Functional Management

• Don't allow anyone to use metrics to measure individuals.
• Set clear goals and let staff help define metrics for success.
• Understand the data your people take pride in reporting; never use it against them, not even hint at it.
• Don't emphasize one metric to the exclusion of others.
• Support your people when their reports are backed by useful data.

Project Management

• Don't try to measure individuals.
• Gain agreement with your team on metrics to track and define them in a project plan.
• Provide regular feedback to the team about the data they help collect.
• Know the strategic focus of your organization and emphasize supporting metrics.

Project Team

• Do your best to report accurate, timely data.
• Help managers focus project data on improving processes.
• Don't use metrics data to brag, or you will encourage others to show the opposite.
• Use software failure analysis to drive process improvement decisions and measure their impact.

Dissecting Software Defects

Defects are viewed from three perspectives: the customer's view, the development engineer's view, and the support engineer's view. This is illustrated by the Defect Cube.

Key Questions to Learn from Mistakes

• What development or maintenance process failed?
• How often do such failures occur?
• How expensive is it to fix such failures?
• Which components are most subject to failures?
• What process change will detect or eliminate these failures?

All software process changes must be evaluated in terms of measurable ROI.

Spiral Model for Process-Improvement Adoption

The lecture introduces the concept that “your mileage may vary” for process improvement. “Mileage” here refers to “success rate.” The spiral model shows that software process improvement is an unending journey. While each improvement can be a tactical project, you must follow through with others to remain competitive. If unwrapped into a linear model, it shows a continuous cycle.

⭐ Key Takeaways

The most critical concepts for a student to remember are the distinction between private and public data and the ethical imperative to never use metrics to evaluate individuals. Key formulas to memorize are the Backlog Management Index (BMI = Dc / Dn) and Percent Delinquent Fixes (PDF = (Fd / Fn) * 100) , as they provide concrete ways to measure maintenance team performance. For strategic use, the “Defect Cube” highlights that defects have different meanings for customers, developers, and support engineers, and analyzing failures is for process improvement, not blame. Finally, remember that process improvement is an unending journey, and every organization’s success rate ("mileage") will vary.

🧠 Quick Revision Questions

1. What is the formula for the Backlog Management Index (BMI), and what does a value greater than 1 signify?
2. What is the primary difference between “private data” for an individual and “public data” for a project team, according to the lecture?
3. How is a “delinquent fix” defined, and what is the formula for calculating the “Percent Delinquent Fixes”?
4. According to the lecture, what is the single most important rule for Functional Management regarding the use of metrics?
5. What does the phrase "your mileage may vary" mean in the context of software process improvement?

📘 Lecture 45 — The Software-Improvement Journey

📖 Overview: This lecture provides a comprehensive review of the software-improvement journey, covering how organizations move from their current state toward a desired future state while navigating roadblocks. It also introduces Team Software Process (TSP) as a key methodology for forming high-performance software engineering teams, and discusses the fundamental role of measurement in software engineering.

🗂️ Topics Covered

The lecture covers the software-improvement journey model with eleven strategies for improving "mileage" (success probability and speed), including understanding business needs, current state assessment, cost analysis, organizational readiness, force-field analysis, and resistance to change. It then transitions to a comprehensive review of software problems (technical vs. systemic), software processes, and an in-depth introduction to TSP including team definition, team structure paradigms, high-performance teams, toxic environments, TSP principles, teambuilding, the launch process, and the importance of software measurement.

📝 Lecture Summary

The Software-Improvement Journey

We start moving toward a desired future state from a current state. Almost always, we encounter roadblocks, but gradually we move ahead with solutions. An intermediate state represents an acknowledged success. We do not stop here; the dots represent other projects or later progressions from current to intermediate states for other spiral loops.

Define Process Improvement in Terms of the Strategic Future

Your starting point is very important, which is where you want to go. It represents your organization's strategic future, so the first way to improve your mileage (chances for, and speed of, success) is to define that clearly. The most succinct definition of a future desired state is a vision.

Improving Mileage #1

• Understand Business Needs Well: A process that adds structure and commitment to your organization's vision is the core-competence planning process. Its major contribution is that it elevates process-improvement thinking to a strategic level. It also expands top management's competitive view from a traditional product focus to include key abilities the organization needs to create those products. Ideally, the core-competence planning process is as strong as your business-planning process. Just as business planning isn't always ideal, core-competence planning isn't either. Think of core-competence planning as a spiral process also.
• Get a Clear Picture of Your Current State: Once you have a good picture of your desired future, there are three key methods by which to gain a similar understanding of your current state.

Improving Mileage #2

Understand Your Software Development Costs and Their Leverage Points: The software-improvement investment model gives you a simple way to model major development, rework, and knowledge recovery costs. It helps to: give your management a better understanding of how software developers work, change people's opinions of the relative importance of proposed improvements, and set realistic expectations for proposed cost savings. You get an even better picture of improvement opportunities when you also analyze your major defect sources. This information is especially powerful when you combine it with the investment-model information. It further clarifies costs, gives you an excellent way to evaluate the potential of proposed improvements, and gives you an effective way to track and communicate the successful changes that you make.

Improving Mileage #3

• Understand Your Organization's Readiness for Change: One way to get a clear picture of your current state is to do an assessment that is well matched to your organization's needs. Pick an assessment approach that matches your business needs, level of sponsorship, scope of needed change, and your best guess of organizational readiness. An assessment does three key things: gives you a reference point of current strengths and opportunities, motivates people to change, and models how to approach improvements. It isn't easy to get a clear picture of an organization's readiness for change; assessments help us in this regard.
• Avoid and Minimize Potential Roadblocks: Process-improvement projects face complex sociological issues not normally faced in product projects. Extra analysis and planning can go a long way in helping to avoid roadblocks.

Improving Mileage #4

Understand Your Business and Organizational Forces: A force-field analysis of management commitment gives you the best snapshot of potential varying "mileage." The force-field analysis helps you steer towards simple risk-minimizing strategies.

Improving Mileage #5

Understand the Origins of Resistance to Change: People develop reasonable beliefs based on experiences. Proposed changes can be dissimilar enough to their past experiences that when they try to draw analogies, the analogies won't reflect probable outcomes. One way to avoid being judgmental about such derived beliefs is to think of their resistance as a "standard reason" not to do things. This can help you open your viewpoint so you can think about both your and their ladders of inference. Resulting discussions will help you to create ways to remove roadblocks.

Improving Mileage #6

Strengthen Plans: More complex projects require better planning and risk management, and even the simplest process improvement isn't easy. Thus, you especially need to plan to motivate people and to optimize their working environment. Process improvements also more fully use the PDCA cycle. This requires activities that many team members aren't experienced with. These added risks mandate at least a minimal, written, process-improvement plan.

Improving Mileage #7

Communicate Solutions to Maximize Success: You will need to effectively describe your project and its benefits often. A good approach is to imagine its completion early and create a storyboard that envisions how that will appear. This is a powerful way to describe your project clearly as part of the organization's future desired state. Another useful tool that ensures strong ties to business goals is the goal/question/metric (GQM) process. These ties lead to more easily accepted communications.

Improving Mileage #8

Set Reasonable Expectations for Results: The first part of setting reasonable improvement expectations is to calculate cost savings. This is a double-edged sword. If you don't do these estimates or if the way you communicate them makes them seem low, your project is unlikely to be approved. If you estimate too high, your project may be cancelled because you aren't believed. Also, if your estimates are high and you are allowed to finish, your project may still be considered a failure. Doing a cost-savings analysis early in your planning forces you to document assumptions. This analysis enables you to make your intermediate steps and improvement gains clearer. These will generally be both believable and high enough.

Improving Mileage #9

Frame Expectations for Different Audiences: A key step in gaining and holding project sponsorship is to translate your cost savings into benefits.

Improving Mileage #10

Reinforce Success with Measured Results: There is nothing like quantified intermediate results to eliminate doubts and solidify support. By imagining how your results will look during your initial planning/storyboarding, you will ensure that you collect the right data to report successes confidently.

Your Improvement Future

The spiral model emphasizes looking toward a future desired state. As we continue to mature software engineering, improvement techniques will get even better, and we will understand our sociology better.

Review from Mid-Term to Final — Software Problems

• Technical: Need a technical solution; easy to understand and solve.
• Systemic: Need a process/management solution; difficult to understand and solve.

Solution to Software Problems

Treat the software task as a process that can be controlled, measured, and improved. Relate the required tasks, tools, methods with skill, training, and motivation of people involved.

Software Processes

Software engineering, as a discipline, has many processes. These processes help in performing different software engineering activities in an organized manner. The software process is the set of tools, methods, and practices that people use to produce (develop and maintain) software and associated products (e.g., project plans, design documents, code, test cases, and user manuals). Characteristics of software processes include: requires creativity, interactions between a wide range of different people, engineering judgment, background knowledge, and experience.

Introduction to TSP — Why Projects Fail

Projects seldom fail due to technical reasons; most reasons involve people problems and excessive schedule pressure. People problems can be solved with the help of good communication and good teams. Good teams guide in handling pressure. Development of Team Software Process (TSP) is another important step in software process improvement, after the development of CMM and PSP. The TSP provides a disciplined context for engineering work. The principal motivator for the development of the TSP was the conviction that engineering teams can do extraordinary work, but only if they are properly formed, suitably trained, staffed with skilled members, and effectively led. The objective of the TSP is to build and guide such teams. A team is more than just a group of people who happen to work together. Teamwork takes practice and it involves special skills. Teams require common processes, agreed upon goals, and effective guidance and leadership.

🔑 Definition — Team Software Process (TSP): A disciplined context for engineering work that provides guidance for building and guiding effective software engineering teams.

What is a Team?

A team consists of at least two people working toward a common goal, with each person having a specific assigned role, and with some form of dependency among group members to complete the mission. Roles provide a sense of ownership and belonging; they help guide team members, prevent conflicts, duplicate work, and wasted effort; and they provide members with a degree of control over their working environment. Interdependence means each team member depends to some degree on the performance of other members; it improves individual performance because members can help and support each other. Team performance is further enhanced by the social support of membership. Constantine suggests four organizational paradigms for software engineering teams:

• Closed paradigm – traditional, good for déjà vu projects
• Random paradigm – loose, innovative projects
• Open paradigm – mix of traditional and random
• Synchronous paradigm – natural compartmentalization of problem

High-Performance Team

Team members must have trust in one another. The distribution of skills must be appropriate to the problem. Mavericks may have to be excluded from the team if team cohesiveness is to be maintained. A jelled team is a group of people so strongly knit that the whole is greater than the sum of the parts. Once a team begins to jell, the probability of success goes way up.

Factors that Foster Toxic Team Environment

• A frenzied work atmosphere where team members waste energy and lose focus
• High frustration causing friction among team members
• Fragmented or poorly coordinated procedures or a poorly defined process model
• Unclear definition of roles resulting in lack of accountability
• Continuous and repeated exposure to failure leading to loss of confidence and lowered morale

Principle Elements of TSP

Before team members can participate on a TSP team, they must know how to do disciplined work. Training in the Personal Software Process (PSP) is required to provide engineers with the knowledge and skills to use the TSP. PSP training includes learning how to make detailed plans, gathering and using process data, developing earned value plans, using earned value to track a project, measuring and managing product quality, and defining and using operational processes. The Capability Maturity Model (CMM) or CMMI provides the overall improvement framework needed for effective engineering work. The PSP provides the engineering disciplines that engineers need for consistently using a defined, planned, and measured process.

💡 Why this matters: The TSP bridges the gap between individual discipline (PSP) and organizational capability (CMM/CMMI), providing the practical guidance for teams to actually execute improvements.

Tying Process Improvement Efforts

The TSP couples the principles of integrated product teams with the PSP and CMM/CMMI methods to produce effective teams. In essence, the CMM/CMMI and PSP provide the context and skills for effective engineering while the TSP guides engineers in actually doing the work. Thus, the TSP capitalizes on the preparation provided by the PSP and CMM/CMMI while also providing explicit guidance on how to do the work.

TSP Teambuilding Principles

The team members establish common goals and defined roles. The team develops an agreed-upon strategy. The team members define a common process for their work. All team members participate in producing the plan, and each member knows his or her personal role in that plan. The team negotiates the plan with management, and management reviews and accepts the negotiated plan. The team members do the job in the way they planned. The team members communicate freely and often. The team forms a cohesive group: the members cooperate and are all committed to meeting the goal. The engineers know their status, get feedback on their work, and have leadership that sustains their motivation. Effective team formation requires that the members truly understand what they are supposed to do, agree on how to do the job, and believe that their plan is achievable. In the TSP, this demanding team-building task is a four-day planning process called the team launch.

TSP Process

TSP processes include: launching a team project, development strategy, development plan, design process, implementation process, test plan, and postmortem.

The TSP Launch Process

Teams generally need professional guidance to properly complete the launch process. This guidance is provided by a trained launch coach who leads the team through the launch process. While the TSP scripts provide essential guidance, every team has unique problems and issues, so a simple process cannot provide all the material needed to guide an inexperienced team. Unless teams are very experienced and have a team leader who has completed several TSP projects, they generally need the support of a trained launch coach.

Introduction to Measurements

Software measurement, once an obscure and esoteric specialty, has become essential to good software engineering. Effective project managers measure attributes of process and product to be able to tell when the software will be ready for delivery and whether the budget will be exceeded. Informed customers measure aspects of the final product to determine if it meets requirements and is of sufficient quality. Without measurement, technology cannot function.

📌 Examples of Measurements: Prices act as a measure of value; height, weight, and size measurements ensure clothing fits properly; when making a journey, we calculate distance, choose route, measure speed, and predict arrival time.

⭐ Key Takeaways

The software-improvement journey requires a clear vision of the strategic future and systematic approaches to improving "mileage" through understanding business needs, current state, costs, organizational readiness, and resistance to change. The TSP provides a disciplined framework for building effective engineering teams through proper team formation, role assignment, and the four-day team launch process led by a trained coach. Teams are defined by having at least two members with common goals, specific roles, and interdependence. Software measurement is essential for understanding whether requirements are met, code is ready for testing, and projects are on track. The TSP effectively bridges individual discipline (PSP) with organizational capability (CMM/CMMI) to guide practical improvement execution.

🧠 Quick Revision Questions

1. What are the four key elements that define a team according to the TSP framework?
2. What are the ten strategies (Improving Mileage #1 through #10) for improving success in the software-improvement journey?
3. What is a "jelled team" and what happens when a team begins to jell?
4. What are the five factors that foster a toxic team environment?
5. How does the TSP tie together PSP and CMM/CMMI, and what is the purpose of the team launch?