Excluding Supplier Agreement Management

Page 440 of CMMI-DEV 1.2 model clearly states that: "SAM process area does not directly address arrangements in which the supplier is integrated into the project team and uses the same processes and reports to the same management as the product developers (for example, integrated teams)."

This statement opens room for some Lead Appraisers to trigger the default button: "SAM is out." But the paragraph continues with the following statement: "Typically, these situations are handled by other processes or functions, possibly external to the project, though some of the specific practices of this process area may be useful in managing the formal agreement with such a supplier."

In my oppinion, not considering SAM may incur problems in the future because you may be postponing the elaboration of a mature way to handle suppliers and contracts and this will be necessary when the organization evolves to higher maturity levels. And, of course, it will be necessary to survive in a global IT world driven by strong and stronger supplier/acquirer relationships.

In my country, only 2 out of 16 organizations who published their Maturity Level 2 appraisals considered SAM in their scope. And, guess what? Many of these organizations use a high number of contractors in their development phases. So, what led them to exclude SAM?

It is important to understand the intent of SAM. If you are augmenting your staff by having a supplier provide people and these people then act, for all intents and purposes, as your employees, then SAM does not apply. Basically, they are following your processes and not managing any of the work on their own, you are managing the work. However, if you give the supplier a chunk of work that they can manage by themselves using their own processes, then SAM applies. So both the organization and the Lead Appraiser need to be aware that if the relationship changes with the supplier, then SAM may move from being N/A to in scope for an appraisal. And you raise a good point, whether or not the Lead Appraiser determines if SAM is in or out of scope, the organization should be aware of the necessary practices it should have in place to manage a supplier.

And if the organization wants to become more sophisticated in managing suppliers, they should be using the CMMI-ACQ.

PPM - Data Analysis

We are a CMMI ver 1.1 level 5 assessed software company and we are in the process of ver 1.2 assessment. As part of this, we are developing the Process Performance Models. We would like to get input on the following points.

1. What kind of statistical analysis are required on the data that is collected for the PPM development?.
2. Do we need to perform Gage R&R test on the data that is collected?. As mentioned, since we are a software company and not a manufacturing company, i am not too sure about the data collection that needs to happen at multiple instances by the same person on the same tool and different person on the same tool.

For example, to check for Repeatability, if I am considering Schedule Variance of various feature development as a measure, there might not be any difference in the schedule variance that will be measured by different people, if the operational definition is clear for the metric.
Now to check for Reproducibility, if competency of the resource for code development is considered as a measure, this could be changing from period to period, as unlike in manufacturing industry where the activities are of repetitive nature. So the reproducibility will be minimal in this scenario.
If its mandatory to perform Gage R&R analysis on the data, can you throw some ideas on the different areas where this can be applied and how the analysis can be performed. Please share your thoughts on this.

It sounds like the Process Performance Models (PPMs) were overlooked for your organization when it was appraised to CMMI v1.1 3 years ago. I would like to make several points regarding PPMs.

1. The CMMI does not specify any required statistical analysis technique for PPM development. Based on your data, your QPPOs, PPBs, the organization has to decide the proper analytic techniques to use. There is a wide variety available for use.
2. What is the reason you are considering Repeatability and Reproducibility? Are you led to these items by your Quality and Process-Performance Objectives (QPPOs)?
3. There is no CMMI requirement to use Gage R&R.
4. It sounds to me that it would be a good idea for you and your organization to have someone facilitate a Measurement and Analysis workshop for you to properly identify your measures, PPBs, and PPMs.
5. You are asking some questions that cannot be properly answered on this blog unless we are working directly with your organization and have some knowledge of your business.

PPQA Audits

Would you please distinguish the different types of audits 1) Projects, 2) Process and 3) products? Does PPQA audit the Project, Process, or Product? Or all the three? And from which area do we need to collect improvements, 1, 2, or 3? I'm confused, can you help?

You say that you are confused. I Let me try to provide an explanation for what I think you are asking about PPQA. The intent of PPQA is to act as the eyes and ears of senior management to ensure that the practitioners are following the documented processes to produce the work products. So PPQA performs two types of audits: process audits and work product audits. Now the processes being audited can be at the individual level, project level, or the organization level. And the processes being audited are not restricted to the CMMI Process Areas. The organization has to determine which processes to audit based on its business goals and objectives, so there may be processes audited in addition to the processes covered by the CMMI.

A process audit is conducted by first studying the documented process and then interviewing the practitioners to determine if they are following the process as documented.

Each process has one or more work products that are produced by following the process. These work products can be at the individual, project, or organizational level as well. The work products can be audited by sitting at a desk and reviewing the work product against the documented requirements for the work product. Is the work product produced correctly? Does it contain the proper level of information? Etc.

Both process and work product audits will identify non-compliances. By analyzing the non-compliance issues, PPQA should be able to identify the underlying causes for the issues and recommend one or more process improvement suggestions.

Estimation Rationale

I guess this seems little silly, but I needed more clarity on it. According to PP 1.2, attributes need to be estimated. Can anyone give me examples of attributes for project? I guess, when we say small, medium , large type projects they form the characteristics of project and not the attibutes. For sure, once we are clear with attributes we can surely ensure that our effort estimates (PP 1.4) are based on estimation rationale. How will project attibutes map up to estimation rationale?

What PP SP 1.2 is addressing is the Basis of Estimate for the project tasks, deliverables, etc. Basically this practice is looking for the sizing attributes that you use to ultimately determine your effort estimates. For a document, an attribute you might estimate is the number of pages for the document that have to be added, modified, or deleted. For hardware, it might be the number of drawings you have to maintain. For software, it might be the number of lines of code, function points, etc. Just using small, medium, or large doesn’t provide a Basis for Estimate. The Basis of Estimate is then your estimation model that is derived from the historical data from prior projects going through this same process. When you first begin to use the Basis of Estimate approach you may not have very good data, or no data at all. So the best you can do is make a guess as to the estimate (SWAG or engineering judgment). But over time as you collect the project data from each project, you will be able to refine this approach and have much better estimates. Then you take these sizing parameters or attributes and apply a productivity factor, again based on your historical data, to arrive at the effort and cost estimates in SP 1.4.

MA and PPQA Questions

I have the following two basic queries about CMMI ML 2:

1. While writing a Metrics and measurement process, should we address the organization level metrics data consolidation and review. As ML 2 is project specific, is it proper to also document the organization level data consolidation? Also can anyone tell me, the right site for definition of metrics like requirement stability index, schedule variance, effort variance etc.?
   
2. Similarly while documenting PPQA process, is it proper to start with defining an organization level PPQA plan? I am looking for boundaries where to limit writing processes compliant to ML 2. I know that G.P 2.1 to G.P 2.10 must be in place to achieve CMMI ML 2, but the organization specific plans/areas must not be mentioned/documented at CMMI ML 2.

You sound like you are focusing on CMMI compliance rather than on your business goals and objectives. One of the basic tenets of the model is your business objectives. That is where your focus belongs. And if done properly, you will have the side benefit of being CMMI compliant. So, to address your questions:

1. When documenting your Measurement and Analysis process, you should focus on those measures that are important to you. Remember, the first MA practice SP 1.1 states “Establish and maintain measurement objectives that are derived from identified information needs and objectives.” So whatever you have identified as information needs and objectives, that should be your MA focus. At ML 2, for many organizations that are just doing this for the first time, I recommend the org take baby steps and begin with a project focus. But you don’t have to be restricted to the project, an ML 2 org may have also identified some org level measures as well. Go to the Practical Software and Systems Measurement web site for the specific measurement information you need www.psmsc.com
   
2. There are NO CMMI-imposed restrictions on the limits of PPQA. Your organization must define its own limits for the processes you are going to audit. Since GP 2.9 applies to all Process Areas, at a minimum for ML 2, PPQA applies to all of the ML 2 Process Areas you have implemented in your organization. But, if there are other processes that are critical and/or important to the success of your business, then it makes perfect sense to have PPQA audit them as well. Again, do what is right for your business.

Integrated SCAMPI

I am faced with a problem with determining which CMMI to use (CMMI-DEV, CMMI-SVC, or CMMI-ACQ) for an in-house IT Department that performs all three types of functions for the organization. I face the following questions:

1. Is there an integrated SCAMPI for all three models held together? Or is the scope is simply determined by adding different PAs from different models? In this case, against what model will the ratings be announced?
2. What about the cost paid to the SEI? Is it calculated differently for such a SCAMPI?
3. What about exclusions if all PAs from these three models that are not fully applicable? Is there a way other than pursuing Continuous Representation?
4. Can you recommend any research work done already on integrating the three model for designing and implementing the OSSP?

I will be greatful for your help and support...

As a first step you should hire an SEI-certified Lead Appraiser, preferably in all three constellations, to provide you the proper advice as to which constellation is appropriate for your organization.

I would only recommend that an organization use the CMMI-ACQ if their primary focus was acquiring products and services from vendors. The CMMI-DEV and CMMI-SVC both have the Supplier Agreement Management Process Area, so either constellation will work if acquisition is not the primary focus of the organization.

Here are my answers to your specific questions:

1. It is possible to conduct blended SCAMPI A appraisals that cover more than one constellation. But your Lead Appraiser will have to work with your organization and the SEI on how best to perform the blended appraisal and determine your Capability or Maturity Level ratings.
2. There are NO fees paid to the SEI by the organization for any appraisal. Any appraisal costs are negotiated between you and your Lead Appraiser.
3. The determination of the appraisal scope is performed jointly with your Lead Appraiser when planning the appraisal. The appraisal scope specifies the representation and the Process Areas being evaluated.
4. I am unaware of any reported results using blended constellations. Though Mike Phillips from the SEI has said that blended SCAMPIs are permissible. I suggest that you contact the SEI and ask for this kind of information, if it exists.

Definition of SOW

I would like to know what is the content of the Statement of Work and how does this document meet requirements of the CMMI?

As a best practice with definition questions, the first place to look is the CMMI Glossary. In this case there is a definition of Statement of Work (SOW) in the Glossary. “A description of contracted work required to complete a project.” In other words, the SOW contains the requirements for what you have to delivery to the customer. And there are varying levels of detail from SOW to SOW depending on the size, criticality, and nature of the work. For large government contracts, the SOW may also contain the Work Breakdown Structure (WBS) for the program/project as well. At a minimum, the SOW will contain the customer’s requirements that the organization will further develop. So the SOW supports RD and may also support PP.