How Do We Select a High Maturity Consultant?

My friend is a Quality Manager in a company who has reached CMMI Maturity Level 3. They now want to achieve Maturity Level 5. They started taking quotes from different companies. In the selection process they found that there are 3 or 4 major players in our country who have up to 3 High Maturity Lead Appraisers. Most of these companies have submitted proposals for consulting and appraisal in a single quote. Now my friend fears that:

1. Most of them already have at least 40 High Maturity clients and at least 30 Maturity Level 3 clients. Will they have the capacity do lead the appraisal on time for my friend's company ( considering 12 SCAMPI appraisals per year per LA) ?
2. Most of them deliver the consulting and training activities , which is 70 % of the contract value and sometimes they break the contracts and not deliver the SCAMPI, which is still highly profitable, since only 30 % value is lost, and no need to deal with High Maturity appraisal needs.

In order to address this issue, does the SEI publish a list of contracts for SEI Partner? Does the SEI have a specific committee or group to look in to the capacity management and availability management of their SEI partners, so that companies will not have such concerns?

All that the SEI does is maintain a list of SEI Partners and certified High Maturity Lead Appraisers. If there is indeed a problem as you have stated, then you or your friend should contact the SEI about the SEI Partner in question as this certainly sounds like unethical behavior.

Another issue is that an SEI-certified High Maturity Lead Appraiser cannot appraise the organization if he or she has provided the consulting to the organization, unless the SEI approves the potential Conflict of Interest.

The steps taken by SEI in this area are impressive. Also I understand that the control on appraisals/per year is established by SEI. Out of curiosity i would like to ask follow up questions.

I belive the critical part of this entire process is that consulting and apprisal services cannot be performed by the same Lead appriser. If there is a need of separate contract for SCAMPI A appraisal activities, and it cannot be included in a consulting contract, then it can have more credibility. Also like ISO where the certification agencies are audited annually (correct me if am wrong), will SEI do an onsite audit on SEI partners? Or do they have a databse of all the contracts established by SEI partners around the world (considering 800 to 1000 appraisals per year)? Because the user community trusts the SEI more than the SEI Partners ( for most of the users it may be the first time to contract with an SEI Partner and they might not be sure of the guidelines provided by the SEI or about the SEI Ethics commitee). All this can be prevented if the SEI takes a copy of all contracts established for SCAMPI A across the countries. What are your views on it? .

Note:The intent of the question is to increase the user communities' trust on SEI to increase, but not to reduce the credibility of SEI partners/Lead Appraisers.

The SEI does not have the time or resources to perform annual on-site audits of the SEI Partners. And as an SEI Partner, I would not welcome an on-site audit by the SEI. It would be additional expense for me.

What the SEI does provide that may help with your concerns is that they perform a QA audit of the results of every appraisal submitted by a Lead Appraiser. If the appraisal results do not meet the evaluation criteria, then a more in-depth audit occurs. What can then happen is that if the problems are serious enough, the Lead Appraiser can lose his or her CMMI credentials. This has happened to a number of Lead Appraisers since this policy was put in place.

In addition, each Lead Appraiser must be certified by the SEI, which provides another layer of credibility.

The SEI Partners provide the service and the certified-Lead Appraisers deliver the service. As a buyer of CMMI services, it is your responsibility to learn about the SEI policies regarding Ethics and Conflict of Interest, as well as the credibility of the different SEI Partners and Lead Appraisers. Otherwise, you get what you pay for. In other words, buyer beware!

And depending upon what country you are in, the SEI Partners are trusted as much or more than the SEI by the user community.

Achieving a Maturity Level Without a Consultant or Training

I was wondering if it is possible to go it alone with CMMI Maturity Level 2. I have been told by many that attempting CMMI Maturity Level 2 without a consultant or highly trained staff would be somewhat challenging. What are your thoughts?

Also, having limited resources for process improvement being a strong possibility, do you have any recommendations for online sources that can help offset the costs?

Though it is entirely possible to implement the CMMI without hiring a CMMI consultant, that is a high risk approach. Though I don’t see how you could avoid taking the Introduction to CMMI training class. That, in my opinion, would be a huge mistake. There are some areas in the CMMI that are open to interpretation when trying to go it alone and you can end up doing things in the spirit of achieving Maturity Level 2 that have no business value to you. In addition, since your organization has to provide 4 to 8 appraisal team members, each appraisal team member must take the SEI’s Introduction to CMMI class.

These are all worthwhile expenditures of your process improvement budget. And when you compare these expenses to your internal costs for process improvement, these are usually negligible in comparison.

But if you are trying to do things as cheaply as possible, that begs the question, why are you even considering implementing the CMMI and trying to achieve Maturity Level 2? Basically you get what you pay for. Going the cheap route doesn’t demonstrate management commitment to process improvement and can result in wasted effort, wasted money, and an aborted process improvement initiative.

Some Appraisal Questions

1. Why does the SEI ask for focus projects instead of all the projects done by the company?
2. Usually companies can select consultants and the Lead Appraiser (LA). Why is the SEI giving the right to chose the LA by the company or consultant?
3. Why do the appraisal results expire after 3 years?
4. Why doesn't the SEI have compliance appraisals every 6 months or 1 year similar to ISO?

1. The SCAMPI method is a sampling method to determine the degree of institutionalization of the processes on the projects. Therefore the use of focus projects. For organizations where there are only 1 or 2 projects, then all of the projects are usually included in the appraisal scope. But for organizations with many projects, it would be prohibitive to evaluate all of the projects. That is why it is the responsibility of the Lead Appraiser to select the focus projects, along with input from the organization. The principle here is that if the processes are truly institutionalized throughout the organization, then it doesn’t matter which projects are selected for the appraisal. Any set of selected projects should be representative of how all projects in the organization behave.
   
   
2. If I understand your statement, you are incorrect. The organization does select the CMMI consultant and Lead Appraiser. However, only SEI-certified Lead Appraisers are allowed to lead and report SCAMPI appraisal results. If a Lead Appraiser is NOT SEI-certified and he or she leads a SCAMPI appraisal, then the appraisal results are NOT valid.
   
   
3. The appraisal results expire after three years because in the past many organizations tended to backslide in their process maturity after having their appraisal. The three year period is long enough to address the findings from the SCAMPI A appraisal and prepare for a re-appraisal at the same or higher Maturity Level. If there is no expiration date, then there could be less motivation to continue with Process Improvement.
   
   
4. In addition, there is no such thing as a compliance appraisal at this time. There has been some discussion along these lines, but nothing has been settled. There is a fundamental difference between ISO audits and CMMI appraisals. ISO is a standard and the result of the audit is certification. CMMI is a set of guidelines for process improvement and the result is Maturity Level or Capability Level that is valid for three years.

How Do I Become A Lead Appraiser?

Would you please tell me step by step the procedure to become a SCAMPI Lead Appraiser (LA)? I am currently working with a Maturity Level 5, company in India with 5.5 years of relevant experience in Process and Quality Consulting. It would be really great if you could provide me with some references, emails, and sites as a roadmap.

Here are the steps and requirements for becoming a SCAMPI Lead Appraiser. This information is directly from the SEI’s web site.

To become an instructor or Lead Appraiser, you must successfully complete authentic SEI courses. The first prerequisite course for becoming either an instructor or Lead Appraiser is the Introduction to CMMI Version 1.2 course. This courses is available from SEI Partners (see SEI Partner Network Directory and Guide to Services) or from the SEI. The second prerequisite is the Intermediate Concepts of CMMI Version 1.2 course that is available only from the SEI.

CMMI Version 1.2 Instructor Training is the final course leading to candidacy as an Introduction to CMMI Instructor. After successfully completing this course, the candidate instructor must also be observed, by an authorized SEI Observer, teaching the SEI's Introduction to CMMI V1.2 course. Upon successful completion of the observation, the instructor is then authorized as an Introduction to CMMI Instructor.

SCAMPI Lead Appraiser Training is the final course leading to candidacy as a SCAMPI Lead Appraiser. After successfully completing this course, the candidate Lead Appraiser must be observed leading a SCAMPI A appraisal using a CMMI model and be approved by an authorized SEI Observer.

You can find more information on the SEI’s site by searching for SCAMPI Lead Appraiser. Of course you will have to either upgrade to v1.3 or take the v1.3 classes when the new versions are released later this year.

Finding a Lead Appraiser

My company needs a very inexpensive Lead Appraiser – where can I find one?

I hate to say this, but you are using the wrong criteria for contracting a Lead Appraiser. When it comes to the CMMI and Lead Appraisers, you get what you pay for. And in point of fact, if you go with the lowest price Lead Appraiser that you can find, it is quite possible that this person is either very inexperienced as a Lead Appraiser or may not have the necessary experience or background to provide credible services. The risk you run is that the SEI may not accept the appraisal results in this situation. Then you have spent the money and may need to repeat everything, which can be a very expensive proposition.

Much better criteria to use when considering to hire a Lead Appraiser include:

• number of years experience as a Lead Appraiser
• number of appraisals led (SCAMPI A, B, and C)
• experience in working with small organizations
• ability to interpret the CMMI to appropriately fit your organization's needs
• recommendations/testamonials from clients who have worked with the Lead Appraiser

You should interview the Lead Appraiser. Prepare some questions and scenarios that apply to your organization and learn how they would interpret the CMMI in your context and what they would recommend.

The basic question comes down to why do you want to be appraised? Your internal costs for implementing the CMMI will far outweigh any external costs associated with an SEI-certified Lead Appraiser. You should be implementing the CMMI because there is some business value associated with the CMMI. Lead Appraisers are experts in process improvement. As an analogy, if you are going into the hospital for major surgery, do you go to the lowest price surgeon you can find (possibly someone with questionable credentials) or do you go to a surgeon that has a reputation for quality and success?

Think of it this way. Selecting a Lead Appraiser is a very important decision. There is certainly cost factors as well as risks to consider, and possibly other considerations as well. Therefore, I suggest that you look at the Decision Analysis and Resolution (DAR) Process Area and use the practices described there to select your CMMI services provider.

Auditor Directing a PPM

We are trying for CMMI ML 5 Ver 1.2, and our auditor has asked us to come up with a PPM for predicting the outcome of CAR and OID. Can you help me on how to go about it?

When you say auditor, I assume that you mean your Lead Appraiser (LA) is asking you for a PPM for predicting the outcome of Causal Analysis and Resolution (CAR) and Organizational Innovation and Deployment (OID). Is your Lead Appraiser an SEI-certified High Maturity Lead Appraiser? Has your organization identified the need for a Process Performance Model (PPM) to predict the CAR and OID outcomes, or is this solely a request from your LA? Your LA is not the person to tell you which PPMs you need. Do you have Process and Product Quality Objectives (QPPOs) that require PPM(s) to predict CAR and OID outcomes? If the answer is no, then you don't need a PPM for CAR and OID.

What I find odd is that you do not mention a Process Performance Baseline (PPB) for CAR and OID. If you are going to define and develop a PPM, then you really need to develop the CAR and OID PPBs first before you can determine the PPMs. From your brief description, it sounds like your Lead Appraiser may have overstepped his boundaries in asking for the CAR and OID PPM.

Selection criteria for a CMMI Lead Appraiser

My organization has used the same lead appraiser for several appraisls and we are now considering using a different LA for our upcoming ML 3 appraisal. My thoughts are to identify several candidate appraisers and then evaluate them against a set of criteria. Here are some of the criteria that I've identified to date:

• Experience with organization similar to our (kinds of products, size, geographically distributed, etc.)
• Expected time for appraisal of similar scope to ensure that we have similar expectations. We want to check length of appraisal and expected number of hours per day.
• Availability of toolset to support appraisal activities (i.e., PIID collection, ratings, consensus tracking, etc.)
• Availability during dates planned for appraisals
• Costs

Are there any other evaluation criteria I should be considering?

I don’t believe that the criteria you have specified will be able to determine a best fit, some of your criteria might but not the rest.

1. Experience working with other organizations of similar size, etc. is a good choice.
2. Expected duration of the appraisal is not a good choice. If you provide sufficient information for the Lead Appraiser to scope the work, I would expect that all qualified Lead Appraisers would provide the same answer, though there may be some variations. In my experience Readiness Reviews are 5 days and SCAMPI A on-site periods rarely exceed 10 days. It might be better if you presented your expectations for the appraisal duration and see if you got agreement from the Lead Appraiser.
3. Availability of tool set would not be a good choice. Usually qualified Lead Appraisers have a tool set they prefer to use. A better criteria might be the willingness of the Lead Appraiser to use your specified tool set.
4. Availability during your planned appraisal dates is not a good choice. Anyone bidding would state that they were available. Besides, we all know that planning dates are simply planning dates and the actual dates are different.
5. Cost is a good choice and probably the only one where you will be able to discriminate between bids. But I encourage you to understand that a SCAMPI appraisal is NOT a commodity. When you purchase any service, including appraisal services, you must understand the value you are intending to obtain. Requests for Propsal should be structured to obtain the information needed to compare value, not just fees or costs.
   

I recommend that since you do have experience with the CMMI, appraisals, and Maturity Level 2 that you put together a plan for your ML 3 appraisal based on the historical data from your ML 2 efforts. Then ask for bids from several Lead Appraisers. Also ask for recommended optional services. Some Lead Appraisers will bid the job by number of days and others will bid the job by appraisal activity. That should provide you with some very good information to make your decision. And here is a radical thought, since you are talking about ML 3, why don’t you use DAR to help you make your decision. :-)

Finally, I recommend that you interview each candidate Lead Appraiser, either over the phone or in person to see how comfortable you feel with him or her and how each responds to your appraisal needs.

Interview Questions to Hire a CMMI Expert

We are implementing the CMMI within our organization and are looking to hire someone to help us achieve this goal. We don't necessarily need a certified Lead Appraiser as of yet, but would like to hire someone with CMMI experience and who may beinterested in becoming a Lead Appriaser. I have a couple of internal candidates that are not Lead Appraisers, but have had CMMI experience (according to their resumes anyhow). What would be some good questions to ask them in an interview to gauge how much experience they truly have? I appreciate any help I can get with this.

One word of caution first. It may not be in your best interests to hire someone who wants to become a Lead Appraiser. There usually aren’t enough internal appraisal opportunities for a candidate Lead Appraiser to get the minimum experience or to maintain their Lead Appraiser credentials, so the person would have to look for appraisal work outside of your organization or company.

Here are some questions that I would ask a candidate for a CMMI position:

1. Have you taken the SEI’s 3-day Introduction to CMMI class? If yes, when did you take the class? Who was your instructor?
2. Have you participated as an appraisal team member? If yes, how many times? What were your duties? What was the scope of the appraisal (Maturity Level)? Who was the Lead Appraiser? What would the Lead Appraiser say about your CMMI capabilities and performance on the appraisal team?
3. Have you helped implement the CMMI in an organization?
4. How long have you been working with the CMMI?
5. Please compare and contrast Capability Level vs. Maturity Level
6. What is the only Process Area that can be categorized as Not Applicable? SAM
7. Have you prepared a PIID? If yes, what was the most difficult task and why?
8. How many years of project management experience do you have?
9. How many years of engineering experience do you have?
10. What is your favorite Process Area and why?

Achieving CMMI Level 2

I am a junior in the field of SQA. I need help about the implementation of CMMI. I want to implement CMMI in the organization Iam working for. I need to know the names of the documents needed to achieve CMMI level 2. And it will be very beneficial for me if I could find out the templates of the documents.

The best advice that I can give you is to take the SEI’s 3-day Introduction to CMMI class. That class will provide you the basics for understanding what you need to do to achieve Maturity Level 2.

A very important concept to understand is that there ISN’T any canned set of documents or templates that you have to have in order to achieve Maturity Level 2. The specific processes you need to document and the associated process assets are a function of the work you perform and the methodology you use to produce your products. I would also suggest that you hire an SEI-authorized Lead Appraiser/consultant to help you understand how to implement the model and achieve Maturity Level 2.

An alternative is to read some of the many books that have been written on how to implement the CMMI. Just look at Amazon for some ideas.

CMMI Maturity 3 Appraisal Process

We are planning to go to CMMI 3 level certification. I recently joined in my company. Please tell me the necesary processes and steps.

1. First off, just to be clear, there is no such thing as “CMMI 3 level certification.” An organization is appraised to the CMMI using the SCAMPI A appraisal method to determine either the organization’s Maturity Level or the Capability Level of the organization’s processes. The result of the SCAMPI A is not a certification, but simply a rating of the current Maturity Level or Capability Level.
2. Has your company already achieved Maturity Level 2? Has your company hired a CMMI consultant? Has your company hired an SEI-authorized SCAMPI Lead Appraiser? Has an SEI-authorized instructor provided the SEI Introduction to CMMI class to your company?
3. If the answer to all of these questions is NO, then hire a CMMI consultant and a Lead Appraiser. The Lead Appraiser cannot provide the CMMI consulting. Most Lead Appraisers are also authorized CMMI instructors, so the next step is to train your process group and any people who might be an appraisal team member on the CMMI.
   
4. Perform a Class C appraisal (gap analysis) to identify where you need to focus your CMMI implementation efforts. Use the findings from the Class C to write a process improvement plan, and use the plan to monitor and control your CMMI implementation efforts.
   
5. Implement CMMI Maturity Level 2 FIRST. Once you have established the firm project management foundation of Maturity Level 2, THEN consider implementing Maturity Level 3. If you try to implement BOTH Maturity Level 2 and Maturity Level 3 at the same time, you will encounter difficulties. There is a huge difference between managing projects at Maturity Level 2 and managing projects at Maturity Level 3.
   
6. Once you feel comfortable that you have addressed all of the findings from the Class C and you have had several project cycles to institutionalize the documented processes, then consult with your Lead Appraiser to determine if your organization is ready to conduct a benchmarking SCAMPI A appraisal.
   
7. There will be more training (appraisal team and PIIDs) and activities leading up to the SCAMPI A, but your Lead Appraiser will tell you exactly what you will need to do to prepare for the appraisal.

Lead Appraiser Authorization Steps

What are the requirements for the SEI to "approve" a lead appraiser?

The SEI is the only body allowed to authorize a SCAMPI Lead Appraiser (LA).
The process for becoming a SCAMPI LA is:

1. You must take the SEI’s Intro to CMMI 3-day class, either the current v1.2 class or the v1.2 Upgrade class if you have taken an earlier version.
   
2. You must have an SEI Partner sponsor you as a candidate Lead Appraiser
   
3. You must take the SEI’s 5-day Intermediate CMMI Class v1.2 $2750
   
4. You must participate on at least two SCAMPI A appraisals within the past 24 months
   
5. You must take the SEI’s 5-day SCAMPI LA class $4200
   
6. You must be observed by an SEI Observer leading a SCAMPI A Appraisal
7. The SCAMPI LA renewal requirements can be found here http://www.sei.cmu.edu/appraisal-program/appraiser-communications/scampi-renewal.pdf . There is a three year period in which the LA has to conduct at least one SCAMPI A and accrue at least 3 points

In addition, the prerequisites for becoming a LA are:

• at least ten years of project management and engineering experience in systems or software engineering
• a minimum of two years of experience managing technical personnel
• an advanced degree in a related technical area or equivalent experience

Since there are limited appraisal opportunities within a company, there may not be enough opportunities for the LA to accumulate the necessary 3 points in order to maintain his or her LA credentials.

Process Improvement - A Twelve Step Process

1. We admitted we were powerless over quality/on-time delivery/estimates/project management/etc. – that our projects had become unmanageable.
2. We came to believe that a model greater than ourselves (the CMMI) could restore us to sanity.
3. We made a decision to turn our processes and procedures over to the care of Software Engineering Institute.
4. We conducted a searching and fearless gap analysis of our organization.
5. We admitted to our Lead Appraiser, to ourselves, and to our executive management the exact nature of our process weaknesses and gaps.
6. We were entirely ready to have our Lead Appraiser help us address these weaknesses and gaps.
7. We humbly asked our Lead Appraiser to help us remove our weaknesses.
8. We made a list of all projects that had suffered because of our bad practices, and became willing to take corrective actions to address the issues, as applicable.
9. We made direct modifications to our processes wherever possible, except when to do so would jeopardize the success a project.
10. We continued to appraise the organization, and when we had weaknesses we promptly admitted them.
11. We sought through the Engineering Process Group (EPG) and the Management Steering Group (MSG), to improve our direct contact with our Lead Appraiser and the SEI, asking only for their knowledge and expertise to guide us on our process improvement journey.
12. We have recognized the benefits of process improvement as the result of these twelve steps; we have tried to carry this message to other internal groups and external organizations and to practice these principles in all our affairs.