Some Appraisal Questions

1. Why does the SEI ask for focus projects instead of all the projects done by the company?
2. Usually companies can select consultants and the Lead Appraiser (LA). Why is the SEI giving the right to chose the LA by the company or consultant?
3. Why do the appraisal results expire after 3 years?
4. Why doesn't the SEI have compliance appraisals every 6 months or 1 year similar to ISO?

1. The SCAMPI method is a sampling method to determine the degree of institutionalization of the processes on the projects. Therefore the use of focus projects. For organizations where there are only 1 or 2 projects, then all of the projects are usually included in the appraisal scope. But for organizations with many projects, it would be prohibitive to evaluate all of the projects. That is why it is the responsibility of the Lead Appraiser to select the focus projects, along with input from the organization. The principle here is that if the processes are truly institutionalized throughout the organization, then it doesn’t matter which projects are selected for the appraisal. Any set of selected projects should be representative of how all projects in the organization behave.
   
   
2. If I understand your statement, you are incorrect. The organization does select the CMMI consultant and Lead Appraiser. However, only SEI-certified Lead Appraisers are allowed to lead and report SCAMPI appraisal results. If a Lead Appraiser is NOT SEI-certified and he or she leads a SCAMPI appraisal, then the appraisal results are NOT valid.
   
   
3. The appraisal results expire after three years because in the past many organizations tended to backslide in their process maturity after having their appraisal. The three year period is long enough to address the findings from the SCAMPI A appraisal and prepare for a re-appraisal at the same or higher Maturity Level. If there is no expiration date, then there could be less motivation to continue with Process Improvement.
   
   
4. In addition, there is no such thing as a compliance appraisal at this time. There has been some discussion along these lines, but nothing has been settled. There is a fundamental difference between ISO audits and CMMI appraisals. ISO is a standard and the result of the audit is certification. CMMI is a set of guidelines for process improvement and the result is Maturity Level or Capability Level that is valid for three years.

Is Going Directly for a CMMI ML 5 Appraisal Allowed?

Is a CMMI v1.2 ML 5 appraisal allowed in the following situation?

One of my company's divisions was successfully appraised to CMMI v1.1 ML 5 but the appraisal results expired in 2009. Now my company wants all three divisions, which are located in different cities, appraised to ML 5. Though skipping levels is not recommended, however, is it allowed to go for the appraisal?

There is nothing from the SEI that prevents a company from being appraised to whatever Capability Level or Maturity Level that it chooses. That being said, what does drive the CL or ML is the organization being able to collect, analyze, and correctly use data for statistical and quantitative analysis, as well as being able to institutionalize the behaviors and practices. You should hire an SEI-Certified High Maturity Lead Appraiser and have him or her perform a Class C or Class B appraisal to determine the risks with your current approach and implementation of ML 5. The outcome of this exercise will determine if it is feasible to achieve ML 5 at all three sites.

Product Planning and Configuration

It is common knowledge that Maturity Level 2 is project specific, and still I find at times Lead Appraisers asking funny questions during SCAMPI A appraisals. Quite recently, one of my friends told me that his Lead Appraiser is looking for planning at the product level, as well as Configuratuion Management at the product level. I was a bit amazed, thinking that Maturity Level 2 focuses on Project Planning, not product planning. What do you think about this situation? Have you been faced with this situation before? Is there a workaround for it?

From what you describe, it sounds like this Lead Appraiser could be misinterpreting the CMMI and possibly misleading the organization. The CMMI is quite clear that the Project Planning (PP) Process Area (PA) is for project planning purposes, not product planning.

"The purpose of PP is to establish and maintain plans that define project activities."

However, sometimes the difference between project and product can be blurred. By not knowing the context of the situation you described, the Lead Appraiser may have been trying a different approach to draw project planning information out in the interview sessions.

In one respect, it really doesn’t matter the line of questioning in a SCAMPI interview session. The Lead Appraiser could really ask about any topic. However, once he or she starts deviating from the CMMI, they are on shaky ground and could lose credibility. What does matter however, is the set of findings produced by the Lead Appraiser and the Appraisal Team. If there are findings associated with product planning that cannot be tied to the satisfaction of a CMMI Specific Goal or a Specific Practice, then these would be non-model findings and should have no impact on the resulting appraisal rating. However, if these non-model findings do impact the appraisal rating and the Lead Appraiser and Appraisal Team fail to demonstrate the linkage to Goal and Practice satisfaction/implementation, then the Lead Appraiser has not correctly performed his or her Lead Appraiser duties and the SEI should be informed about this issue so it can be investigated.

Need Some Clarification About Process Improvement

What types of activities should be considered process improvement? If we modify the templates (any major changes), do we categorize these changes as process improvement?

Merely modifying a template may or may not constitute process improvement. Your template may have changed because of external reasons (your customer wants you to use a different template) that have nothing to do with process improvement. You should explain the rationale for modification, then you would have a stronger case to demonstrate process improvement.

Process improvement suggestions can come from any number of sources:

1. Appraisal findings
2. PPQA audit findings
3. Lessons learned
4. Employee suggestions

If you couple the source of the process improvement suggestion with the actual change, usually spelled out in the Process Improvement Plan, then you have the information you need.

Does the SEI issue any certificates?

Does the SEI issue any kind of certificate to the organization who achieves a Maturity Level in a SCAMPI A? Please share any information about it.

The SEI does not issue certificates or certifications. The closest they come is certifying the High Maturity Lead Appraisers.

Since most organizations want to receive some sort of certificate documenting the results of a SCAMPI A appraisal, Lead Appraisers will present something that looks like a certificate to the company indicating the organization being appraised, the date the SCAMPI A concluded, and the appraisal team’s results (Maturity or Capability Level), with the caveat that the results are contingent upon the results of the SEI’s Quality Assurance audit. But nowhere on this form will it indicate certificate. This "certificate" is the unofficial document of the resulting Maturity or Capability Level.

The Appraisal Disclosure Statement is the only valid document of the appraisal results, and if approved for posting by the Appraisal Sponsor and if it passes the SEI’s QA audit, then this is the document that is posted on the SEI’s Appraisal Results web site http://sas.sei.cmu.edu/pars/pars.aspx. And keep in mind, these results are only valid for three years from the date the SCAMPI A concluded.

Stakeholder Invovlement Question

Our organization has project stakeholders whose only project involvement is as a decision maker in an end-of-phase meeting. What is expected of the stakeholders (e.g., training and project involvement) if they are outside of our process, as far as the appraisal interviews are concerned?

In the case described, the stakeholder is NOT outside the process.

The “plan for stakeholder involvement” defines the expected role of the stakeholder in the project, or process. This, in turn, defines the scope of training required. In the case described, the stakeholder would minimally need training in the conduct of end-of-phase meetings. However, if decision making required knowledge of work products produced by the project team, orientation to those products might also be required. Training in Decision Analysis and Resolution (DAR) may also be required, depending upon whether or not the decisions meet the criteria for initiating the DAR process.

Note that training may take many forms, including orientation or briefings on required topics. The expectation is that the stakeholder knows enough to credibly perform the responsibilities described in the project plan or process description.

In the case described, the stakeholder might be selected for an appraisal interview.

What is the cost of a CMMI v1.2 L3 certification?

Our company is interested to go for CMMI v1.2 ML3 certification. Can you answer the following questions:

1. What are the costs involved (Internal & External)?
2. Are there any SEI cerfication bodies in India?

First of all let me make one clarification to your query. There is no such thing as a CMMI certification. What an organization receives are the results of a SCAMPI A appraisal that indicate the Maturity Level of the organization on the day the appraisal concludes. The organization is not certified. And there is no such thing as an SEI certification body anywhere in the world. What do exist are authorized SEI partners that are allowed to provide CMMI consulting, training, and appraisal services. Visit this web site http://partner-directory.sei.cmu.edu/ and you will be able to find the SEI-authorized partners in India.

The answers to your questions are highly variable depending on the size and scope of your organization and your geographical location. The best place to obtain realistic estimates is to ask several local SEI-authorized CMMI consulting and appraisal providers for their cost proposals, then you will have a handle on the external costs. Internal costs really cannot be determined until you figure out how much work you have to do in order to implement the CMMI and prepare for an appraisal. Suffice it to say, your internal costs will most likely be greater than your external costs.

CMMi 1.2 Level 3 Query

We have planned to use CMMI 1.2 model for our Organization for process improvement and we are targetting to reach maturity Level 3. The constraint here is we have a very small team in development. E.g. there are only 3 members for database management, 1- 2 team members for front end development (the same is true for middleware development), and a few for production support projects. I am not sure how to manage peer reviews and many other aspects. In all there are around 20 members in the development team. Please let me know the challenges we may face during our journey to maturity Level 3, based on your experience.

Also I am driving the CMMi initiative. Is there any mandate that I should be an SEI-Certified Assessor? I have the required experience in Quality Management System for driving the CMMi initiative, but unfortunately I have not taken any formal Certification on CMMI. Please let me know your thoughts on this point as well.

There are many challenges with process improvement and implementing the CMMI regardless of your size. However, for small organizations, the challenges can be even greater. The biggest challenge is probably budget. Small organizations usually do not have a lot of extra funds for activities not directly associated with producing a product or service. Therefore, you are faced with adding the responsibilities of mapping, defining, and documenting your processes and procedures to your normal 40+ hour/week job. Industry averages over the past two decades show that Process Improvement is roughly 3 – 5% of the organization size. Configuration Management takes another 3 – 5%, as well as PPQA. Worst case the total could be 15% of the organization. So for a development team of about 20 people, you would need 1 person full time on Process Improvement, 1 person full time for CM, and 1 person full time for PPQA. Senior Management is usually reluctant to set aside this much budget when you begin your journey. There is no visible business case for the additional people and what happens is that people have to wear multiple hats to cover these new positions until the burden gets so big that either you stop these activities or management agrees to the additional funding.

Beyond this much it is difficult to provide more information without a better understanding of your organization, management commitment, line of business, etc. There are many ways available to you to scale the CMMI to your organization. You just have to be very careful not to implement something because you think the CMMI requires it, but there is no business value to you. That is the situation that gives process improvement a bad name. Everything that you do with the CMMI must be value-added. About the only way that I know of to avoid mistakes and/or potential obstacles is to work with an experienced CMMI consultant and Lead Appraiser. Over the years we have seen what works and what doesn’t work and there is no need for you to reinvent the “process improvement” wheel one more time.

A third point is that your description implies that you are going directly to Maturity Level 3 without first establishing the foundation of Maturity Level 2. That approach can be a costly one and could jeopardize your program, unless you have a history of process improvement in the organization. The SEI and all Lead Appraisers that I know always recommend that you implement one Maturity Level at a time. The CMMI Staged Representation is a foundational model. You must first establish a firm foundation for process improvement, and that is what you achieve by attaining Maturity Level 2. Then Maturity Level 3 builds upon the foundation of ML 2. Then ML 4 builds upon ML 3. And ML 5 builds upon ML 4. Please keep in mind that I am not saying that you have to be appraised at each level with a SCAMPI A Appraisal. But you need some indication that you have implemented a Maturity Level. The big mistake many organizations make is skipping over ML 2 and going directly to ML 3. There are some fundamental differences in how Project Managers behave at ML 2 vs. ML 3. By skipping ML 2, you run the risk of having ML 3 processes documented but practiced as a ML 2 organization. Therefore, when you are appraised, the results have a high probability of being ML 2, which would NOT be a happy day.

There are no SEI requirements that the person in the organization responsible for implementing the CMMI be an SEI-authorized Lead Appraiser. There really is no benefit to you or your organization for you to become a Lead Appraiser, unless you have lots of internal opportunities to lead appraisals, or your company is in the business of providing CMMI services to other clients. Becoming a Lead Appraiser is an expensive proposition. You have to take at least three classes, which can be a total of $15,000. Before you can be accepted in the classes, you have to be an appraisal team member on at least two appraisals. You have to pass several exams and be observed leading an appraisal. Then you have to lead at least two appraisals over a three year period. In addition, a Lead Appraiser cannot lead an appraisal of his or her own organization.

My best advice to you is to hire a CMMI expert to help you on your journey.

Interpreting OPF SP 1.2

While reading and then interpreting SP 1.2 of OPF "Appraise the Organization's Processes" it seems that it is a mini SCAMPI (correct me if I am wrong). Now in this context I have few questions and would like answered.

Organizational Process Focus (OPF) SP 1.2 states “Appraise the organization’s processes periodically and as needed to maintain an understanding of their strengths and weaknesses.”

1. These internal appraisals to satisfy this practice will be performed by the PEG (Process Engineering Group), how much rigor is required? Can we employ the SCAMPI Class C method to satisfy this practice?
SCAMPI A, B, and C appraisals all satisfy this practice as well as any other type of appraisals or assessments that provide an understanding of the organization’s process strengths and weaknesses. There is no level of rigor implied by this practice.

2. Should we check the evidences and/or satisfaction at processes' implementation at project level or its better to keep the scope of these appraisals at OU level only?
OPF is an organizational level process. And the practice states to appraise the organization’s processes. Some of them are performed at the project level and others at the organizational level. But the focus should be at the organizational level.

3. Should we formally present the findings as normally done in SCAMPI Class A appraisals by the LA or that much rigor is not required (as it required lot of stakeholders presence like CEO if he is the sponsor)?
The rigor of the Findings Presentation is up to you. However, you should determine the level of rigor when you are in the planning phase for these evaluations. As a Lead Appraiser, I create an appraisal plan for any appraisal I conduct and I specify in the plan how the practices, projects, organization, etc. will be scored and how the results will be presented.

4. If we are appraising the organizational processes, then should we appraise all the ML 2 and ML 3 process area or we can make selection?
Since this practice covers all process appraisals conducted on the organization’s processes, depending on the appraisal, you have a lot of flexibility on the scope and conduct. It all needs to be specified in the appraisal plan. Of course a SCAMPI A appraisal using the staged representation will dictate which PAs to include depending on the Maturity Level. Other than the SCAMPI A, you have the freedom to pick and choose what you want to appraise. But keep in mind, you do need to have an overall strategy and plan for these appraisals. As a Lead Appraiser appraising this practice, I expect to see a periodic plan for your appraisals, not just the SCAMPI A appraisals I conducted.

5. If we make a checklist for all process areas, then is it a good idea that we include the Subpractices as a questions (may be not all subpractices) in checklist for all ML 2 and ML 3 PAs?
Again, you can use whatever checklists you want for your internal appraisals. In fact, it might work to your advantage to be very rigorous in an internal appraisal. But remember, when it comes time for the SCAMPI A appraisal, you will not be evaluated against the sub-practices.

CMMI Certifications & Career Development

I'm a graduate student and starting my degree in 2009. I have over 2 years experience (3 mini assessments) in Process Improvement, CMMI ML 3 and ML 5 evidence collection, and verification. I will be taking the Intro to CMMI class in December 2008.

As I am interested in researching the CMMI and Quality Assurance for my thesis and to shape up my career to be a CMMI consultant position; I contacted SEI. The SEI suggested that I seek the advice and guidance from a few registered Lead Appraisers in different parts of the world on how I should plan my future studies.

My questions are:

1. In the professional world job market is there a value for such research?
2. What other professional qualifications should I gather to be a CMMI consultant?
3. From your experience do you think its worth it (career development wise) to conduct appraisals externally rather than waiting for your organization to provide them free for you?
4. Any other advice you want to provide me in my career planning?

Your guidance in these questions are highly appreciated.

Here is my advice to you to help further your interests and your career.

1. There is always room in the professional arena for research. Research results are regularly presented at professional symposia and conferences. I am sure that the SEI would encourage you to submit an abstract for consideration at an upcoming SEPG Conference. If your research is credible, and you gain visibility in the community, that can only strengthen your credentials as a consultant. It also demonstrates that you are staying current in the field of process improvement.
2. To become a CMMI consultant, you would need practical experience, not necessarily additional professional qualifications. Take a look at the criteria for being on an appraisal team. That criteria also makes for a credible CMMI consultant. So I would recommend that you gain some experience in Project Management, as well as engineering experience, at least 3 to 5 years worth. If you were to start consulting immediately upon graduation, you may be looked upon as providing an academic approach to CMMI implementation.
3. Since implementing the CMMI can take several years and the expenses for appraising an organization are fairly high, there will not be many internal opportunities for appraisals. Especially if you want to become a Lead Appraiser. You may be able to satisfy the minimum requirements to become a Lead Appraiser with internal appraisals, but most likely you won’t be able to conduct enough internal appraisals to maintain your credentials. Therefore, I recommend that you conduct external appraisals. If you were to work for a consulting company specializing in the CMMI, then the company would most likely pay your expenses. Otherwise, you will be faced with financing them yourself.