Objective Evidence vs. Typical Work Products

Would you please explain the relationship between Objective Evidence and Typical Work Products?

This is an interesting question, and one that can be easily answered by reading the CMMI Glossary, a little used part of the model. Most answers to questions can be found by first reading the Glossary definitions.

From the CMMI Glossary, the definition of Objective Evidence: As used in CMMI appraisal materials, documents or interview results used as indicators of the implementation or institutionalization of model practices. Sources of objective evidence can include instruments (questionnaires), presentations, documents, and interviews.

Typical Work Products are one of the topics contained in the Informative Component. All model components are important because the Informative Component helps you understand the Expected and Required Components. It is best to take these model components as a whole. If you understand all three types of information, then you can understand how everything fits together to form a framework that can benefit your organization.

From the CMMI Glossary, the definition of Typical Work Products: An informative model component that provides sample outputs from a specific practice. These examples are called typical work products because there are often other work products that are just as effective but are not listed.

So, in a nutshell, a Typical Work Product is an example of a tangible output that might be produced by performing a Specific Practice. The Objective Evidence are the actual work products and other items produced by performing the organization’s documented processes. So the Objective Evidence provided in an appraisal might or might not match the items listed as Typical Work Products.

What Class of SCAMPI Appraisal is Appropriate?

My organization has just started our process improvement journey. For our first appraisal, which SCAMPI class (A, B, or C) should we use?

I would strongly suggest that your organization hire either a SCAMPI Lead Appraiser or a SCAMPI B/C Team Leader to help you determine which SCAMPI appraisal A, B, or C would be best for your organization. In order for you to obtain the best information and useful results, you need to use either an SEI-trained and authorized SCAMPI B/C Team Leader or a SCAMPI A Lead Appraiser. Do not attempt to do this by yourself.

For a new client, I typically recommend an initial Gap Analysis to provide a starting point for determining the path forward. The Gap Analysis, depending upon its rigor, can be either a SCAMPI C or a SCAMPI B.

2009 SEPG Conference - Day 2

Yesterday the conference kicked off into full swing with the two Keynote Speakers. The first speaker was Scott Cook, Chairman of the Executive Committee of the Board, Intuit Inc. Scott talked about the key process factors that have led to Intuit's success: Team Software Process (TSP), Agile, and the Toyota Production System (TPS). Intuit has realized quite some important gains by using these different process improvement methods. At the conclusion of his talk, he presented an entirely different issue that is ripe for process improvement. Scott showed a slide of various causes of death each year in the US and he asked us what we thought were the top three causes. It turns out that Heart Disease and Cancer are the top two, but the third cause is Hospital-Related. Scott claimed that there are 160,000 Hospital-Related deaths each year, which is the equivalent of a fully loaded 747 crashing and killing all on-board once a day. And the majority of these deaths can be attributed to people and process problems: not enough resources available, wrong diagnoses made, wrong surgical operations performed, incorrect medication delivered, etc. So Intuit is helping the Stanford Medical Center use TPS, based on Intuit's experience, to address these people and process issues and they are already seeing benefit.

The second speaker was Jim Bampos, Vice President, Information & Quality Management, EM Corporation. Jim spoke about the process improvement challenges faced by a primarily hardware company that now has a software component. And the added complexity of EMC continually acquiring new businesses and integrating them into their corporate culture. Jim recommended Marcus Buckingham's book Put Your Strengths to Work.

2009 SEPG Conference

The 2009 SEPG Conference kicked off yesterday with a day of tutorials. This year I had the pleasure of moderating the Risk Management tutorials. There were two half-day tutorials. The morning session was "New Directions in Risk: A Success-Oriented Approach" by Audrey Dorofee, a senior member of the technical staff at the Software Engineering Institute. This was a very interesting tutorial presenting a pratical approach to managing risks based on risk drivers as opposed to focusing on the Top Ten List of Risks. There were also three exercises providing the attendees hands-on experience using this new, practical, and easy to understand technique. The presentation can be downloaded from www.sei.cmu.edu/msce/presentations.html

The afternoon session was "Avoid V-Nosediving or Spiraling Out of Control with the Incremental Commitment Model" by Barry Boehm, TRW professor of software engineering and director of the Center for Systems and Software Engineering at the University of Southern California and the originater of the Spiral Model. Barry spoke to the audience about the Incremental Commitment Model (ICM) and how it builds on the strengths of phased models (such as the V-model and the Spiral Model) and incorporates a risk-driven approach. There was a lot of information packed into the 99 slides presented over the two 1 1/2 hour sessions for this tutorial. It will take a lot of studying to understand and implement this new approach. Apparently, the software engineering students at the University of Southern California are using the ICM and there is growing evidence of its value and benefit. You can read more about the ICM at http://csse.usc.edu/ and look for Technical Report 2009-500.

Evaluating the CMMI for Services

I'm currently involved in an evaluation of different types of assessments for the area of IT Service Management (ITSM) and therefore challenging the CMMI for Services 1.2 (CMMI-SVC).

A major problem I see in CMMI-SVC, is the disregarded sufficient differentiation of practices over different maturity levels. Maybe I´m wrong with my opinion and someone can help me fixing this mess.

I´d like to give a background for a better understanding o f what I mean:

1. In my opinion many (internal) IT service units in mid-size companies don´t have any official agreements (nor contracts) with their business customer to specify service content, service levels or support. Sometimes these things are partly available and if at all, then often are informally and silently accepted.
2. In such (above) described IT service units you will find quite often an official HelpDesk (mostly official because it´s a specific function within IT) or minimum some guys necessary for service support (like handling service incidents) to keep the business process running.

Now take a look at the CMMI-SVC Process Areas and their associated Maturity Levels. You will find Service Development (SD) at Maturity Level 2 including specific practices:

SP 1.1 Analyze Existing Agreements and Service Data
SP 1.2 Establish the Service Agreement
SP 2.1 Establish the Service Delivery Approach
SP 2.2 Prepare for Service System Operations
SP 2.3 Establish a Request Management System
SP 3.1 Receive and Process Service Requests
SP 3.2 Operate the Service System
SP 3.3 Maintain the Service System

and you will find Incident Resolution and Prevention (IRP) at maturity level 3 including specific practices:

SP 1.1 Establish an Approach to Incident Resolution and Prevention
SP 1.2 Establish an Incident Management System
SP 2.1 Identify and Record Incidents
SP 2.2 Analyze Incident Data
SP 2.3 Apply Workarounds to Selected Incidents
SP 2.4 Address Underlying Causes of Selected Incidents
SP 2.5 Monitor the Status of Incidents to Closure
SP 2.6 Communicate the Status of Incidents
SP 3.1 Analyze Selected Incident Data
SP 3.2 Plan Actions to Address Underlying Causes of Selected Incidents
SP 3.3 Establish Workarounds for Selected Incidents

To come to an end I would expect that it is an essential part of any IT unit to solve service incidents, to fulfill the main goal of the company, and keep the business process working. Therefore I would assign half of the listed IRP practices to Maturity Level 2 and would other way around assign half of the SD practices to Maturity Level 3. The CMMI-SVC therefore seems for me to be not sufficient in differentiating practices over Maturity Levels, and would lead to the conclusion that CMMI-SVC is not useful for Maturity Level determination.

I appreciate any explanation if there is a misunderstandig or if there exists a grain of truth.

Your position is much the same as the kind of statements we heard regarding the engineering practices being at ML 3 in the CMMI-DEV. Just because these practices are at ML 3 does not mean that they are not important and are probably even performed at ML 1. What you have to bear in mind is that the CMMI is a set of process improvement guidelines , as well as the definition and purpose of ML 2 and ML 3. At ML 2, projects establish the foundation for an organization to become an effective service provider by institutionalizing basic project management and service establishment and delivery practices. Basically, ML 2 is about gaining control over the projects and service delivery and that is why there is only one service PA at ML 2. You have to get delivery under control before you can focus improving the other aspects of services like Incident Resolution and Prevention. And at ML 3, service providers use defined processes for managing projects. They embed tenets of project management and services best practices, such as service continuity and incident resolution and prevention, into the standard process set.

CMMI for Aquisition

I am looking for standards for aquisitions and outsourcing. I heard that there is a CMMI standard for aquisitions? Can anyone confirm this? Also is there any standard for outsourcing?

Yes, there is a CMMI for Acquisition (CMMI-ACQ) and it was released in November 2007. You can find out more information about the CMMI-ACQ and detailed answers to your questions at http://www.sei.cmu.edu/publications/documents/07.reports/07tr017.html

Generic Practice and PA Relationship

Are all Generic Practices (GPs) required for each and every Process Area (PA) when the organization is being appraised? Or there is a main GP for each PA required?

This question is explained in detail in the Intro to CMMI class. If you haven’t already taken the class, I strongly recommend that you take it. Also, if you read the Generic Goal (GG) section of the CMMI book, you read more detail about the GPs.

The short answer to your question is it depends upon the scope of the appraisal and the representation (Staged or Continuous). For Maturity Level 2 all PAs must meet GG2, for ML 3 GG 3, ML 4 GG3, ML 5 GG3. For Capability Level 1 GG 1, CL 2 GG 2, CL 3 GG 3, CL 4 GG 4, CL 5 GG5. But I strongly urge you to take the class and read the model to gain a complete understanding of the answers.

Computing a Process-Compliance Index

I am interested in building a dashboard tool for measuring compliance to the CMMI process areas for specific projects. Would you have any insight on how a process-compliance index could be computed?

What immediately comes to mind is the scoring system for a Class B appraisal. For each practice Green is Satisfied, Yellow is Partially Satisfied, and Red in Unsatisfied. You can then use these colors for your dashboard and chart the movement from red to yellow to green.

Moving to Maturity Level 4

Our company achieved Maturity Level 3 (ML 3) this year. Now we want to go for Maturity Level 4 (ML 4). Would you please give us some suggestions on what we have to do for ML 4?

I can give you a list of items and practices that you need to have in place, but that is not enough. I highly recommend that you and your company hire a High Maturity Lead Appraiser and/or consultant and have them work with you to implement OPP and QPM. I also suggest that you take the SEI’s Understanding CMMI High Maturity Concepts or equivalent. This class will greatly help your understanding of ML 4 and ML 5. You will need to have someone on your staff that has a good understanding of statistics and statistical methods to help you build your Process Performance Baselines (PPBs) and Process Performance Models (PPMs) that support your Quality and Process Performance Objectives (QPPOs). In addition, in order for all this to work, your processes have to be stable so you can perform meaningful statistical analyses. This also means having a repository of historical data from the stable processes. So not only do you need expert help, you also need a sufficient amount of historical data, which could range from months to years in order to achieve ML 4.