Covering SAM SP2.2 and SP2.3

I want to ask you about CMMI Supplier Agreement Management (SAM) SP 2.2 Monitor Selected Supplier Processes and SP 2.3 Evaluate Selected Supplier Work Products. I have difficulty in understanding these practices and how to apply them in my company. We have a project that needs to buy new hardware and deliver it to customer.

How can I select and monitor supplier processes in this case?

Can you please explain that to me and give me an example of a supplier process that is applicable in our situation?

The best way to answer this question is to attend the SEI’s Introduction to CMMI 3-day class. During the SAM topic the instructor will explain the difference between these two practices for you. I would also suggest that you read the informative material in the SAM Process Area, for not just these two practices, but all of the Process Areas. The informative material is extremely rich in information, hints, tips, etc. and it provides guidance for understanding the intent of each practice and goal. I would also recommend that you not use the goal and practice titles for anything other than labels. The titles are shortened statements and do not always communicate the correct meaning of the goal or practice.

If you read the informative material for SAM SP 2.2 and SP 2.3 it will be immediately clear to you the differences. And if you still have questions regarding your specific circumstances, I suggest that you ask your SEI-certified Lead Appraiser. Without having a good understanding of your company and how it conducts business, asking specific implementation questions will most likely not provide the correct advice to you.

However, here is a very brief description of the differences between these two practices.

SAM SP 2.2 “Select, monitor, and analyze processes used by the supplier.” You as the acquirer of the supplier’s products or services must define in the supplier’s contract or agreement those specific supplier processes that are critical to your success. You have to state how and when you will be monitoring these critical processes. This practice is done as a risk mitigation to avoid surprises at the end of the contract when the supplier delivers the end product or service. Think of this practice as the acquirer performing PPQA process audits of the supplier’s processes.

SAM SP 2.3 “Select and evaluate work products from the supplier of custom-made products.” If your supplier is only delivering off-the-shelf product to you without any customization, this practice does not apply. However, if your supplier is providing custom-made or modifications to your specifications, then you need to decide which of these products are critical to your success and define those in the supplier’s contract or agreement. You have to state how and when you will be evaluating the products. Think of this practice as the acquirer performing PPQA work product audits of the supplier’s work products.

Applicability of SAM

Does Supplier Agreement Management (SAM) apply to the supplier? Suppose you're an organization that is supplying to another organization , for example to develop a software package. This organization is not using suppliers. I assume that SAM is not applicable to this organization. Is my assumption correct?

Your assumption is correct. But allow me to restate your scenario to avoid any confusion by using the word organization mulitple times in one sentence. Organization A develops a software package and delivers it to Organization B. Organization A does not use any suppliers or outsource any work. In this case Organization B is the acquiring organization and Organization A is the supplier organization. Therefore, SAM would apply to Organization B, but not to Organization A.

Why SAM is excluded ?

Why is SAM removed from CMMI L3?

Supplier Agreement Management (SAM) is not excluded from the CMMI-DEV or CMMI-SVC. When you say L3, I assume you mean Maturity Level 3 and SAM is definitely NOT excluded from ML 3. If, however, the Lead Appraiser in working with the organization determines that SAM is not applicable to the work performed by the organization, SAM will be considered Not Applicable to the scope of the appraisal. And that could be at any Maturity Level. Please read previous my posts regarding SAM for more information. http://ppqc.blogspot.com/2009/04/excluding-supplier-agreement-management.html and http://ppqc.blogspot.com/2009/07/cmmi-novice-question.html

CMMI Novice Question

I would like to know if all the Maturity Level 2 Process Areas must be completed for a Maturity Level 2 appraisal? There are some process areas which are not applicable to our organization.

For the CMMI for Development (CMMI-DEV) there is only one Process Area (PA) that can be designated Not Applicable (N/A), SAM. Therefore, at a minimum, REQM, PP, PMC, MA, PPQA, and CM are required for a Maturity Level 2 (ML 2) appraisal. And if the organization has outsourced some work, then SAM is also applicable.

I find it hard to believe that you can state that one or more of these PAs are not applicable to your organization. Every project has requirements to manage from the janitor to the President. Everyone works on a project. You just have to define what a project is. And then you manage the project. Everyone can define specific measures that can be used to manage the project. Everyone has some sort of configuration items or documents that have to be managed. And everyone needs some sort of objective evaluation of the process and project compliance. For a small organization, you may have combined one or more of these PAs under one person. But that does not mean these PAs are not applicable.

Now if you said that you had problems with the engineering PAs (RD, TS, PI, VER, and VAL), then I would suggest that the CMMI-DEV may not be the appropriate model constellation for your use and you should look at the CMMI for Services (CMMI-SVC) or CMMI for Acquisition (CMMI-ACQ).

Excluding Supplier Agreement Management

Page 440 of CMMI-DEV 1.2 model clearly states that: "SAM process area does not directly address arrangements in which the supplier is integrated into the project team and uses the same processes and reports to the same management as the product developers (for example, integrated teams)."

This statement opens room for some Lead Appraisers to trigger the default button: "SAM is out." But the paragraph continues with the following statement: "Typically, these situations are handled by other processes or functions, possibly external to the project, though some of the specific practices of this process area may be useful in managing the formal agreement with such a supplier."

In my oppinion, not considering SAM may incur problems in the future because you may be postponing the elaboration of a mature way to handle suppliers and contracts and this will be necessary when the organization evolves to higher maturity levels. And, of course, it will be necessary to survive in a global IT world driven by strong and stronger supplier/acquirer relationships.

In my country, only 2 out of 16 organizations who published their Maturity Level 2 appraisals considered SAM in their scope. And, guess what? Many of these organizations use a high number of contractors in their development phases. So, what led them to exclude SAM?

It is important to understand the intent of SAM. If you are augmenting your staff by having a supplier provide people and these people then act, for all intents and purposes, as your employees, then SAM does not apply. Basically, they are following your processes and not managing any of the work on their own, you are managing the work. However, if you give the supplier a chunk of work that they can manage by themselves using their own processes, then SAM applies. So both the organization and the Lead Appraiser need to be aware that if the relationship changes with the supplier, then SAM may move from being N/A to in scope for an appraisal. And you raise a good point, whether or not the Lead Appraiser determines if SAM is in or out of scope, the organization should be aware of the necessary practices it should have in place to manage a supplier.

And if the organization wants to become more sophisticated in managing suppliers, they should be using the CMMI-ACQ.

Integrated SCAMPI

I am faced with a problem with determining which CMMI to use (CMMI-DEV, CMMI-SVC, or CMMI-ACQ) for an in-house IT Department that performs all three types of functions for the organization. I face the following questions:

1. Is there an integrated SCAMPI for all three models held together? Or is the scope is simply determined by adding different PAs from different models? In this case, against what model will the ratings be announced?
2. What about the cost paid to the SEI? Is it calculated differently for such a SCAMPI?
3. What about exclusions if all PAs from these three models that are not fully applicable? Is there a way other than pursuing Continuous Representation?
4. Can you recommend any research work done already on integrating the three model for designing and implementing the OSSP?

I will be greatful for your help and support...

As a first step you should hire an SEI-certified Lead Appraiser, preferably in all three constellations, to provide you the proper advice as to which constellation is appropriate for your organization.

I would only recommend that an organization use the CMMI-ACQ if their primary focus was acquiring products and services from vendors. The CMMI-DEV and CMMI-SVC both have the Supplier Agreement Management Process Area, so either constellation will work if acquisition is not the primary focus of the organization.

Here are my answers to your specific questions:

1. It is possible to conduct blended SCAMPI A appraisals that cover more than one constellation. But your Lead Appraiser will have to work with your organization and the SEI on how best to perform the blended appraisal and determine your Capability or Maturity Level ratings.
2. There are NO fees paid to the SEI by the organization for any appraisal. Any appraisal costs are negotiated between you and your Lead Appraiser.
3. The determination of the appraisal scope is performed jointly with your Lead Appraiser when planning the appraisal. The appraisal scope specifies the representation and the Process Areas being evaluated.
4. I am unaware of any reported results using blended constellations. Though Mike Phillips from the SEI has said that blended SCAMPIs are permissible. I suggest that you contact the SEI and ask for this kind of information, if it exists.

CMMI Implementation

I recently joined a company where there is no process and management recruited me to implement the CMMI. The organization has different business units. Though everyone sits together, they work very differently. I conducted a Gap analysis based on the CMMI Level 2 processes and here are the findings:

1. Project Planning & PMC -- they create project plans and they have the regular project team meetings and they share the minutes. Each team has their own format and templates. The projects don't really do estimations. Can we satisfy the PP & PMC PA'ss without doing any estimation? I know it can't be that way, but can it be tailored?
2. Requirements Management: Some of the business units have CCBs to discuss change requests and other business units discuss requirements changes in their project team meetings. The most significant gap I found is in Requirements Traceability. Traceability of Customer requirements to the Functional Requirements and Traceability of Use cases to the Test cases are missing. Is this reason enough to fail the RM PA? One of the Engineering directors asked me how much traceability you need to satisfy this condition. At that time I said 100% of all the requirements. Then I also read from somewhere that it is OK to define that we maintain traceability for at least the MUST BE CUSTOMER REQUIREMENTS. Traceability of other requirements can be made optional. Can it be possible like that?
3. Configuration Management: The projects thought they have a CMP which is embedded in the project plan. What I found missing are the Configuration Audits ( PCA & FCA). Is it possible to satisfy the Configuration Management PA without doing Configuration Audits to check the document status, builds, and backup strategy?
4. PPQA: One of the business units has a Software Quality Assurance plan, but it is done by one of the testers from another project. As a part of PPQA the SQA will do some spot checks based on a pre-defined checklist, which includes Project Planning, Risk Management, Project Monitoring and Control, Integration and releases. But I guess this can be improved by my role as a independent software quality engineer.
5. M&A: I am very much worried with this process area, as of now the organization status is nil with respect to metrics collection, they don't have a metrics database and no metrics have been defined yet. Is it OK to start now to form a team to do some reasearch and come up with metrics definitions, deploy them and start collecting data? My question is how much data do we need to collect to satisfy this PA? And do we need to provide evidence of analyzing these collected data and show some improvements steps taken at the time of SCAMPI apprisals? How long will it take in general to satisfy the Measurement & Analysis PA?
6. SAM: can we tailor this PA if we are not dealing with suppliers? If yes how can it be possible?
   

The directive from the Leadership team is to acheive CMMI Level 3 by end of 2009. I was baffled to hear this. Under these circumstances, what are the chances of getting CMMI Level 3 or my traget is at least CMMI Level 2? That is what my initial target. Can I acheive CMMI Level 2 by the end of 2009? If so, what are the things I need to address?

Here are some of the things I have already started:

1. CMMI Overview training to all the teams
2. Dailogue session on metrics identification
3. Looking into some Requirements tools which provide Traceability
4. Need to push the project team to have configuration audits.

In addition we already have established a process data base and the processes are defined and templates are being used from the parent organization. Since our company is a multi-site company, one of our counter parts has already acheived CMMI Level 3. We will be using the same process database and their templates. I thinking of providing their training on each Process Area as well. Is this a correct way to use the processes and templates of our parent organization? If not, do we need to establish our own local process data base?

First of all, I sympathize with you and the challenges you face. I applaud the fact that you had the foresight to conduct a Gap Analysis of the organization. You have highlighted a number of key weaknesses within the organization. However, to provide you meaningful feedback on all of your points would require working directly with you and your company.

1. What is your CMMI experience? Have you taken the SEI’s Introduction to CMMI class? If not, I strongly recommend that you and possibly those others in your company who are responsible for your processes take the three day class. The class should provide you a more thorough understanding of the CMMI, its interpretations, and material for constructing an internal Overview class.
2. You have identified some serious deficiencies within the organization in all of the ML 2 Process Areas. These need to be analyzed, addressed, corrected, solutions implemented, and then re-evaluated some months into the future before you can consider a formal SCAMPI at any Maturity Level. The length of time before the next evaluation is a function of a number of factors: number of people in the organization, number of projects, typical project duration, how much time and other resources are dedicated to process improvement, etc.
3. The organization needs to first implement Maturity Level 2 to form a firm foundation before considering moving to Maturity Level 3. The issues you have identified are fairly typical. Basically, it sounds like your organization does not perform PPQA or MA and is challenged with Project Management, Requirements Management, and Configuration Management. The first steps here should be to identify the necessary skills-based training classes you need to bring in-house and train your staff on these concepts. If you just purchase tools and push for audits, you most likely will not achieve the desired effect. You have to understand your process first and the reasons for why it is important to perform each of the steps.
4. Since another division has already achieved ML 3, it is a good idea to learn from their mistakes. But be very careful of the temptation to “clone” their processes and procedures. You have to implement the processes and procedures that match the way you conduct business today.
5. Based on what you have outlined, and given how much time and effort it could take just to address the ML 2 issues, I would say that ML 3 is out of the question for 2009. You could conduct a ML 3 SCAMPI A by the end of this year, but in all likelihood it would not be successful.
6. The best suggestion I have for you is to hire a CMMI consultant and Lead Appraiser to provide you with the proper advice and guidance. Otherwise, you could be spending a lot more time and effort than originally anticipated.

Supplier Agreement Management Question

I have a question related to Supplier Agreement Management (SAM) SP 2.2 - Monitor Selected Supplier Processes. What is the basic intent of this practice and in what scenario does it fit in? It specifies "...situations of tight alignment between processes implemented by the supplier and those of the project..." - which is normally not the case in most (small) projects (as I know). I hope here we are not including "Acceptance" and "Transition" as aligned processes. Also it seems redundant to me with SP 2.1 - Execute the Supplier Agreement because in the contract/ SOW, it is usually mentioned how the supplier needs to monitor his processes (frequency to perform process audits etc.) and the frequency/condition when the customer may ask for a process audit/assessment. So doesn't executing the Supplier agreement covers these two practices?

I can see where you might have some confusion concerning these two SAM practices. SAM SP 2.1 says “Perform activities with the supplier as specified in the supplier agreement.” And your confusion comes about from sub-practice 1 “Monitor supplier progress and performance (schedule, effort, cost, and technical performance) as defined in the supplier agreement.” In essence SAM SP 2.1 is all about performing Project Monitoring and Control (PMC) over the supplier, which should be spelled out in the supplier agreement. You are basically acting as the Project Manager for the supplier by monitoring and controlling their project and technical performance. There is no intent to perform any Process and Product Quality Assurance (PPQA) audits or activities to support this practice.

In contrast SAM SP 2.2 says “Select, monitor, and analyze processes used by the supplier.” This practice is where you perform PPQA activities (process audits and work product audits) on selected supplier processes that are critical to the success of your project and business. Again, this ability to perform PPQA on the supplier must be specified in the supplier agreement. But you want to have the freedom to select any supplier process, so don’t indicate specific processes in the supplier agreement. For example you might decide to monitor and analyze how your supplier performs peer reviews or how they manage their requirements. If you have your supplier doing small projects (less than a month in duration) you may not have many opportunities to perform PPQA on a given supplier project. This situation is the same as when you have small projects done completely in house. There is no hope or expectation that you will be able to perform PPQA activities on every small project.

Look back at my blog on PPQA Audit Frequency http://ppqc.blogspot.com/2008/05/ppqa-audit-frequency.html for a simple way to adjust the frequency of the PPQA audits based on the quality issues discovered. You can use this same approach to determine the frequency of conducting PPQA activities on your supplier, assuming that you supplier regularly performs small projects for you.