SCAMPI Document Review

I have two questions or requests for clarification:

• During the document review in SCAMPI B, wherever we do not have a doc, ppt, or xls as an artifact, we provide screenshots from tools where the planning and tracking are done.  However, our Lead Appraiser (LA) is asking for access to tools for all the ATMs.  But it is not possible to give access to certain tools as they are client specific and only the team working on the project gets access (that too after signing an NDA).  How do we handle this situation?  Even if the tool is internal, access is very much restricted based on role in the project.  Can the LA really require access to tools for all ATMs, which is not allowed as per the policy of an organization?  Is there a guideline on what process to follow in case access to certain tools or application is restricted? 

• Is there a material, which shows a linkage of all the PAs (representing interaction of PAs) and gives a holistic view of CMMI PAs when applied to an organization at Maturity L5. This is more from a training perspective. 

Have you explained these restrictions to your Lead Appraiser?  The Lead Appraiser should be flexible regarding access to restricted tools.  As a Lead Appraiser, I would find the screen shots as acceptable evidence in your situation.  And if I wanted or needed to view additional evidence, I would request that you provide a demonstration of the tool by an authorized user and have this person be directed by the ATMs or Lead Appraiser to view specific information.  I have used screen shots as evidence on numerous appraisals in the past.  The SCAMPI method allows for screen shots and tool demos for just such reasons as yours.  If your Lead Appraiser is unwilling to abide by your restrictions on tool access, then I would strongly urge you to find a new Lead Appraiser and possibly report him or her to the SEI.

Look in Chapter 4 of the CMMI-DEV book and you will find a series of diagrams that show the linkages between the PAs at a very high level.

Auditor Directing a PPM

We are trying for CMMI ML 5 Ver 1.2, and our auditor has asked us to come up with a PPM for predicting the outcome of CAR and OID. Can you help me on how to go about it?

When you say auditor, I assume that you mean your Lead Appraiser (LA) is asking you for a PPM for predicting the outcome of Causal Analysis and Resolution (CAR) and Organizational Innovation and Deployment (OID). Is your Lead Appraiser an SEI-certified High Maturity Lead Appraiser? Has your organization identified the need for a Process Performance Model (PPM) to predict the CAR and OID outcomes, or is this solely a request from your LA? Your LA is not the person to tell you which PPMs you need. Do you have Process and Product Quality Objectives (QPPOs) that require PPM(s) to predict CAR and OID outcomes? If the answer is no, then you don't need a PPM for CAR and OID.

What I find odd is that you do not mention a Process Performance Baseline (PPB) for CAR and OID. If you are going to define and develop a PPM, then you really need to develop the CAR and OID PPBs first before you can determine the PPMs. From your brief description, it sounds like your Lead Appraiser may have overstepped his boundaries in asking for the CAR and OID PPM.

Can the Same Firm Provide CMMI Consulting & SCAMPI A Services?

I have been hearing that the SEI has stringent rules stating that the consulting firm cannot perform a SCAMPI A appraisal as it results in conflict of interest and the appraisal may lose it's objectivity. Couold you please throw light on this issue and give me the correct information? It would be great if you can refer to any documents listing these rules e.g. SEI audit policies.

Yes, the same COMPANY and same LEAD APPRAISER can provide both CMMI consulting and SCAMPI A appraisal services to the same client. However, the Lead Appraiser and SEI Partner must notify the SEI if there is a Conflict of Interest. The situation to avoid is the Lead Appraiser appraising his or her own work in a SCAMPI A appraisal. The Code of Professional Conduct (CoPC) spells out the policy and the steps that must be taken if there is an indentified Conflict of Interest. http://www.sei.cmu.edu/partners/copc.html

The primary issue concerns the type of consulting provided by the Lead Appraiser. If the consulting consists of writing processes and process assets for the client, then there is a clear Conflict of Interest. But if the consulting consists of training on different ways to implement a Process Area, reviewing documents and identifying strengths and weaknesses, reviewing evidence for inclusion in the PIIDs, etc. then there is no Conflict of Interest.

So most Lead Appraisers will try their best to remain ethical and either just provide the hands-on consulting and have another Lead Appraiser lead the SCAMPI A appraisal OR lead the SCAMPI A appraisal and help the organization get prepared for the appraisal.

What Class of SCAMPI Appraisal is Appropriate?

My organization has just started our process improvement journey. For our first appraisal, which SCAMPI class (A, B, or C) should we use?

I would strongly suggest that your organization hire either a SCAMPI Lead Appraiser or a SCAMPI B/C Team Leader to help you determine which SCAMPI appraisal A, B, or C would be best for your organization. In order for you to obtain the best information and useful results, you need to use either an SEI-trained and authorized SCAMPI B/C Team Leader or a SCAMPI A Lead Appraiser. Do not attempt to do this by yourself.

For a new client, I typically recommend an initial Gap Analysis to provide a starting point for determining the path forward. The Gap Analysis, depending upon its rigor, can be either a SCAMPI C or a SCAMPI B.

CMMi 1.2 Level 3 Query

We have planned to use CMMI 1.2 model for our Organization for process improvement and we are targetting to reach maturity Level 3. The constraint here is we have a very small team in development. E.g. there are only 3 members for database management, 1- 2 team members for front end development (the same is true for middleware development), and a few for production support projects. I am not sure how to manage peer reviews and many other aspects. In all there are around 20 members in the development team. Please let me know the challenges we may face during our journey to maturity Level 3, based on your experience.

Also I am driving the CMMi initiative. Is there any mandate that I should be an SEI-Certified Assessor? I have the required experience in Quality Management System for driving the CMMi initiative, but unfortunately I have not taken any formal Certification on CMMI. Please let me know your thoughts on this point as well.

There are many challenges with process improvement and implementing the CMMI regardless of your size. However, for small organizations, the challenges can be even greater. The biggest challenge is probably budget. Small organizations usually do not have a lot of extra funds for activities not directly associated with producing a product or service. Therefore, you are faced with adding the responsibilities of mapping, defining, and documenting your processes and procedures to your normal 40+ hour/week job. Industry averages over the past two decades show that Process Improvement is roughly 3 – 5% of the organization size. Configuration Management takes another 3 – 5%, as well as PPQA. Worst case the total could be 15% of the organization. So for a development team of about 20 people, you would need 1 person full time on Process Improvement, 1 person full time for CM, and 1 person full time for PPQA. Senior Management is usually reluctant to set aside this much budget when you begin your journey. There is no visible business case for the additional people and what happens is that people have to wear multiple hats to cover these new positions until the burden gets so big that either you stop these activities or management agrees to the additional funding.

Beyond this much it is difficult to provide more information without a better understanding of your organization, management commitment, line of business, etc. There are many ways available to you to scale the CMMI to your organization. You just have to be very careful not to implement something because you think the CMMI requires it, but there is no business value to you. That is the situation that gives process improvement a bad name. Everything that you do with the CMMI must be value-added. About the only way that I know of to avoid mistakes and/or potential obstacles is to work with an experienced CMMI consultant and Lead Appraiser. Over the years we have seen what works and what doesn’t work and there is no need for you to reinvent the “process improvement” wheel one more time.

A third point is that your description implies that you are going directly to Maturity Level 3 without first establishing the foundation of Maturity Level 2. That approach can be a costly one and could jeopardize your program, unless you have a history of process improvement in the organization. The SEI and all Lead Appraisers that I know always recommend that you implement one Maturity Level at a time. The CMMI Staged Representation is a foundational model. You must first establish a firm foundation for process improvement, and that is what you achieve by attaining Maturity Level 2. Then Maturity Level 3 builds upon the foundation of ML 2. Then ML 4 builds upon ML 3. And ML 5 builds upon ML 4. Please keep in mind that I am not saying that you have to be appraised at each level with a SCAMPI A Appraisal. But you need some indication that you have implemented a Maturity Level. The big mistake many organizations make is skipping over ML 2 and going directly to ML 3. There are some fundamental differences in how Project Managers behave at ML 2 vs. ML 3. By skipping ML 2, you run the risk of having ML 3 processes documented but practiced as a ML 2 organization. Therefore, when you are appraised, the results have a high probability of being ML 2, which would NOT be a happy day.

There are no SEI requirements that the person in the organization responsible for implementing the CMMI be an SEI-authorized Lead Appraiser. There really is no benefit to you or your organization for you to become a Lead Appraiser, unless you have lots of internal opportunities to lead appraisals, or your company is in the business of providing CMMI services to other clients. Becoming a Lead Appraiser is an expensive proposition. You have to take at least three classes, which can be a total of $15,000. Before you can be accepted in the classes, you have to be an appraisal team member on at least two appraisals. You have to pass several exams and be observed leading an appraisal. Then you have to lead at least two appraisals over a three year period. In addition, a Lead Appraiser cannot lead an appraisal of his or her own organization.

My best advice to you is to hire a CMMI expert to help you on your journey.

Appraising Your Own Organization

In writing yesterday's blog entry I had included a statement that a SCAMPI A Lead Appraiser was not allowed to audit his or her organization, which I removed before posting. The SEI requires that the appraisal be done by an external third party. However, when I looked for this requirement in the ARC and MDD I could not find the reference. Through the help of other Lead Appraisers I have since located this requirement. Go to the SEI's Appraisal Policy Changes site and read the 4th bullet http://www.sei.cmu.edu/cmmi/faq/apc-faq.html.

Can a Lead Appraiser in an organization perform an v1.2 appraisal in that organization?

All Class A SCAMPI v1.2 appraisals that will become public record (e.g., announced in a press release or on an organization's Web site, or posted on a published SCAMPI appraisals results Web page) or used in a proposal in response to U.S. Department of Defense requirements must be led by an SEI-authorized SCAMPI Lead Appraiser from an external, third-party organization.

The external, third-party organization can be another SEI Partner organization or a separate business unit from the one containing the appraised organization (e.g., from corporate or from a different division, group, or other organizational business type, which is under separate management).

Lead Appraiser Authorization Steps

What are the requirements for the SEI to "approve" a lead appraiser?

The SEI is the only body allowed to authorize a SCAMPI Lead Appraiser (LA).
The process for becoming a SCAMPI LA is:

1. You must take the SEI’s Intro to CMMI 3-day class, either the current v1.2 class or the v1.2 Upgrade class if you have taken an earlier version.
   
2. You must have an SEI Partner sponsor you as a candidate Lead Appraiser
   
3. You must take the SEI’s 5-day Intermediate CMMI Class v1.2 $2750
   
4. You must participate on at least two SCAMPI A appraisals within the past 24 months
   
5. You must take the SEI’s 5-day SCAMPI LA class $4200
   
6. You must be observed by an SEI Observer leading a SCAMPI A Appraisal
7. The SCAMPI LA renewal requirements can be found here http://www.sei.cmu.edu/appraisal-program/appraiser-communications/scampi-renewal.pdf . There is a three year period in which the LA has to conduct at least one SCAMPI A and accrue at least 3 points

In addition, the prerequisites for becoming a LA are:

• at least ten years of project management and engineering experience in systems or software engineering
• a minimum of two years of experience managing technical personnel
• an advanced degree in a related technical area or equivalent experience

Since there are limited appraisal opportunities within a company, there may not be enough opportunities for the LA to accumulate the necessary 3 points in order to maintain his or her LA credentials.

CMMI Updates from the SEI

Last week I participated in the workshop at the Software Engineering Institute in Pittsburgh to develop questions for the Lead Appraiser certification exam that is planned to be administered for the first time in October 2008. To kick off the workshop, Mike Phillips/Program Manager of the CMMI, gave us the latest information on a number of topics that I want to summarize for you here.

1. The purpose of this workshop was to help increase the professional aspect of the Lead Appraiser profession and it represents a maturation of the profession.
2. The first opportunity to use the Lead Appraiser test will be at the Lead Appraiser Workshop in Vancouver, WA in October 2008. All Lead Appraisers must take and pass the exam and there will be a one-year window for taking the exam.
3. The release of CMMI v1.3 will not be very long after the release of CMMI-SVC constellation, which is currently planned for March 2009. The SEI is trying to get it out sooner, possibly as early as January 2009.
4. v1.3 will include a number of changes resulting from developing CMMI-ACQ and CMMI-SVC. There are two IPPD practices for ACQ, one in OPD and one in IPM, that are now mandatory. The SEI also wants to include the updated High Maturity material in this release. The intent is to bring all three constellations into a greater harmony. The plan is to release v1.3 by the end of 2009.
5. The strategy for the Introduction to the CMMI class will also be changing somewhat. At some unspecified time in the future, the new class will consist of a 3-day generic course applicable to any constellation, and then 1-day supplemental classes for each constellation. The generic class is expected to be for everyone and the supplemental classes will be for appraisal team members only.
6. Someone in the audience asked Mike Phillips if the SEI is going to consider hardware engineering as a separate discipline. His answer was that the SEI is not trying to differentiate hardware engineering at this time. They are backing away from discipline-specific distinctions.
7. Mike Phillips said that there are ongoing discussions on future constellations. Possibly one for manufacturing and another for operations. But these constellations, if they were to materialize, are way off into the future.
8. When v1.3 is released near December 2009, the SEI will issues three TRs, one for each constellation.
9. v1.3 was approved as an idea by the SEI Steering Group one month ago. The next steps are in work, but it is still too early in the process to be definitive.

Changes to Intro to CMMI Training and Appraisals for CMMI-ACQ

When taking the CMMI-ACQ upgrade training class from the SEI on March 21, there were a number of important changes that will impact CMMI Instructors, Lead Appraisers, and organizations that I think should be made available. No doubt over the next few months more information will be communicated by the SEI.

The FERPA form has been a pain for instructors because you have to be sure to collect them from each student and submit them to the SEI. In the past the SEI has stated that the FERPA forms were required by Carnegie Melon. In addition, the FERPA form is required so the results can be input to SAS and then made available to the Lead Appraiser when planning an appraisal.

The SEI made some small incremental additions to the core 16 Process Areas to accommodate ACQ. These changes are to the informative material. The CMMI Constellation architecture allows the SEI the freedom to tailor the core PAs at the sub-practice level. The sub-practice changes are posted on the web.

There is no difference in SCAMPI appraisals for the CMMI-ACQ. The SAS is almost ready to use for CMMI-ACQ.

On-line training for CMMI-ACQ is being developed with blended learning and will be available in May at about the same cost. The intent is that students can take the class online and then the instructor will schedule an online meeting with the class to discuss the material.

The CMMI-ACQ contains a new feature in the Process Areas. Typical Supplier Deliverables. This new feature ≠ deliverable artifacts. The Typical Supplier Deliverables are there to remind the acquirer of typical work products the acquirer might produce

The core Process Areas have some differences and this is intentional. To quote Rusty Young “Informative NOT Ignorative!” so the changes are important to note.

The CMMI-ACQ class will be a one-day add on training class. The current plan is to deliver the three-day class ( a generalist course) followed by a single day supplemental class in either DEV, ACQ, or SVC. The three-day CMMI class is for most people and the one-day supplemental is intended for Appraisal Team Members. This new structure will be available in about a year from now, March 2009.

When conducting a SCAMPI for a combined DEV and ACQ appraisal use the Continuous Representation if the overlap of the two constellations becomes a strength. You should be able to enter CMMI-DEV plus some ACQ PAs that best fit in the OTHER category in SAS.

The SEI relabled the categories in the Continuous Representation for CMMI-ACQ. REQM is included in Project Management.

And, finally, comparisons of the CMMI-ACQ to CMMI-DEV are provided at http://www.sei.cmu.edu/cmmi/models/ACQ-v12-comparetoDEV.html . Go to the bottom of the page for chapter-by-chapter detailed comparisons and to download PDF copies.
There may be a CMMI v1.2a update released after all the CMMI constellations have been released, perhaps by the end of 2009. And it may be called v1.3 instead.