Is Going Directly for a CMMI ML 5 Appraisal Allowed?

Is a CMMI v1.2 ML 5 appraisal allowed in the following situation?

One of my company's divisions was successfully appraised to CMMI v1.1 ML 5 but the appraisal results expired in 2009. Now my company wants all three divisions, which are located in different cities, appraised to ML 5. Though skipping levels is not recommended, however, is it allowed to go for the appraisal?

There is nothing from the SEI that prevents a company from being appraised to whatever Capability Level or Maturity Level that it chooses. That being said, what does drive the CL or ML is the organization being able to collect, analyze, and correctly use data for statistical and quantitative analysis, as well as being able to institutionalize the behaviors and practices. You should hire an SEI-Certified High Maturity Lead Appraiser and have him or her perform a Class C or Class B appraisal to determine the risks with your current approach and implementation of ML 5. The outcome of this exercise will determine if it is feasible to achieve ML 5 at all three sites.

Project Planning SP 1.2 - Task Attribute: Effort or Size?

Is it possible to establish estimates of work product and task attributes by means of task time estimates? Can the task effort be similar to the size of a task?

By going directly to task time estimates you have effectively skipped performing Project Planning Specific Practice 1.2. The intent of SP 1.2 is for you to perform some sort of basis of estimate for the project’s tasks and activities. This is a bottom-up approach. If you are not used to this approach, it can be a struggle at first to take a step backwards from task time estimates and really understand the underlying assumptions that people are making in their heads about the factors that are driving the task time estimates. Some very basic task attributes include estimating the number of pages in a document that is being produced or updated, the number of technical drawings being produced or updated for a hardware item, the number of new or modified interfaces, the number of new or modified screens , etc. Then based on your historical data from previous projects, it is possible for you to empirically determine a set of productivity factors that will convert these sizing parameters into effort and arrive at the task time estimates. The bottom line is effort of a task is not the same as the size of a task.

Do you think this practice would be Fully, Largely, Partially or Not Implemented? Would this be this a problem in a SCAMPI A appraisal? What do you think about that?

Taking this example out of context with everything else your organization is doing makes this a difficult question to answer. The appraisal team is the only group that would be qualified to make that judgment based on documented evidence and the interviews. However, as a Lead Appraiser, I would have to say that you have a problem that needs to be addressed before you conduct a SCAMPI. The SCAMPI rules state that if a Process Area is in the scope of the appraisal, then all of its Specific and Generic practices are applicable. And if you are not performing a practice, which may or may not be the case, then there could be issues in Project Planning that impact Goal Satisfaction and result in a Maturity Level 1 rating.

To provide you the best answer, you should be talking to your Lead Appraiser and have him or her give you the proper guidance on this issue. As a risk mitigation, I would recommend that you put a process in place to estimate sizing parameters that are then used to calculate effort. Your estimators are already doing this, but it sounds like they are doing it in their heads. You just have to break the process down into smaller steps to allow the sizing estimates to be captured first. There is benefit to doing this.

How to Start With the CMMI

If a private software company wants to do get a certificate of quality for CMMI Level 2, what must this company do in order to obtain Level 2? And also what are the steps that must be followed from the start (initial) to achieve CMMI Level 2 (Managed)?

First let me state that there is no certification of any type for the CMMI. Individual Lead Appraisers will issue something looks like a certificate to the organization that was appraised indicating the SCAMPI A results. But this is not a certification. The only things that the SEI certifies are the CMMI instructors and Lead Appraisers.

Here are the necessary steps for achieving Maturity Level 2 for the CMMI-DEV. These steps are not necessarily sequential, some can occur concurrently.

1. Hire an SEI-certified Lead Appraiser to conduct a Gap Analysis of the company to determine the current process strengths and weaknesses and help the company construct a Process Improvement Plan (PIP).
2. Obtain executive management sponsorship for the process improvement effort.
3. Train the people responsible for the company’s processes and for addressing the action items in the PIP on the 3-day SEI Introduction to CMMI v1.2 class.
4. Address the issues from the Gap Analysis, document the necessary processes and procedures, and begin conducting the PPQA process and work product audits.
5. Allow time for the new and/or modified processes to get some use on various projects.
6. Conduct a SCAMPI B appraisal as a dress rehearsal for the SCAMPI A. Identify any issues and weaknesses that are potential risks to achieving Maturity Level 2.
7. Create a new PIP to address the SCAMPI B identified weaknesses and risks.
8. Address these issues
9. Conduct the Maturity Level 2 SCAMPI A appraisal.

SCAMPI A Appraisal Questions for FAR Groups

Can you share the Questionnaires for a ML 3 SCAMPI A appraisal? Currently my organization is preparing for SCAMPI A and I want to trained the interviewees as this is their first experience an appraisal.

I am looking for some questions based on each Process Area, so the FAR Group Members will be prepared to answer them.

I have searched on Google but not able to find any information. :(

I am not surprised that you have not been able to find what you are looking for on the internet. A Lead Appraiser will not share his or her questions with you. Simply because the Lead Appraiser does not want the interviewees to be trained on what answers to provide, thereby biasing the results of any SCAMPI appraisal. If you and the organization are nervous about the appraisal, don’t be. There is no preparation for the interviews necessary. All that is expected by the Lead Appraiser is that any interviewee should be able to talk about HOW they perform their job duties. If people cannot do that, then the organization is not ready for an appraisal.

If you still feel uneasy, talk to your Lead Appraiser and have him or her conduct a Class B appraisal with interviews. A Class B or SCAMPI B appraisal can function as a dress rehearsal for a SCAMPI A without having the organization and the people worry about “flunking” or “passing” the appraisal. The interviewees will have the experience of being interviewed.

If you still want to perform an internal set of interviews, simply take the CMMI and step through the applicable practices with different groups of people (project managers, configuration managers, developers, testers, etc.) and ask them HOW they perform the Specific and Generic Practices. There is no set of standard questions. Each Lead Appraiser has their own style of questioning interviewees.

CMMI Practices for Documentation Teams

Our organization is CMMI Maturity Level 3 Ver 1.2 certified. Our delivery teams are going to implement CMMI practices soon. I did see the processes of the organization and though all the roles appeared from project leader to developer to manager, except for documentation teams. In the same context, I am very curious to see if there are any set of practices to be followed for the documentation department in CMMI as all the processes at CMMI ML2, ML3 are specific to project management, engineering, support, organization areas.

Your question actually raises some other questions:

1. What are the roles and responsibilities of your documentation teams and documentation department?
2. If this group of people is responsible for the technical documentation (e.g. requirements, design, etc.) and the user documentation, how and why were they excluded from your ML 3 SCAMPI A?
3. How are the delivery teams different from the organization that achieved Maturity Level 3?

What is puzzling with your question is that REQM, RD, TS, PI, and CM cover the different aspects of writing and controlling the various documents associated with designing, developing, maintaining, operating, using, and deploying products. And then VER covers the inspection/review of the documents before placing them in the baseline and controlling them with CM.

It does appear from your description that your organization omitted the documentation people as a process role from your process documentation. In my opinion, at a minimum, your PPQA audits should have identified this omission long before your SCAMPI A appraisal. Then your Lead Appraiser should have identified this gap during the appraisal planning process before the SCAMPI A and should have taken steps to address the gap or postponed your appraisal until the documentation group was included in the scope. Since your documentation group was apparently not included in the scope of your appraisal, this oversight also calls into question your Lead Appraiser’s credentials and quite possibly the validity of your SCAMPI A results.

The bottom line, in my opinion, and based on only what you stated, your documentation group should have been included in the scope of your ML 3 SCAMPI A. Even if all they do is Document Configuration Control (which would be covered under CM) or Document Quality Assurance (which would be covered under PPQA).

The answers to my above questions could provide additional information that would change my opinion.

Project Specific Findings

While reading the SCAMPI A Method Definition Document (MDD), I discovered that providing project specific findings is an option that can be requested by the project sponsor. I always believed that the Findings Presentation could only include findings at organizational unit level. So if the sponsor requests project specific findigs as part of the appraisal output, how are they communicated? Included in the final findings? If this is the case, what about the non-attribution of findings? Or via a separate document?

The SCAMPI Method does indeed allow for project specific findings if the Appraisal Sponsor requests them. If requested, then project specific findings could be communicated in the Final Findings Presentation along with the other information, or they could be communicated separately. Reporting will be negotiated and documented in the Appraisal Plan. Keep in mind that even if the Appraisal Sponsor doesn’t request project specific findings, the PIIDs will contain project specific observations, so if someone were interested in finding out about project specific information, all they would have to do would be to read the PIIDs. The organization is required to retain the PIIDs used for the appraisal for three years, the same length of time that the appraisal results are valid.

The non-attribution issue concerns identifying any individual, project, or group as the SOURCE of the information. So by reporting project specific findings, the Lead Appraiser and Appraisal Team are not violating the non-attribution rules unless they reveal the source of the finding(s). When I conduct an appraisal, the final Appraisal Team activity is to scrub the PIIDs of all attribution information (names, interview sessions, etc.) and deleting all previous versions so the organization only retains the scrubbed PIIDs.

Selecting Projects for a SCAMPI Appraisal

One of my client organisations works on hardware and software design projects related to locomotive design for a manufacturer. They want to adopt the CMMI-DEV and scope their SCAMPI appraisals only for their software projects. Both, hardware and software design projects are undertaken by the same organisation, under the same management, same company name and at the same location. Is it be proper for the organization to exclude the hardware projects from their CMMI journey and appraisals? I feel it is not proper and it violates the principle of institutionalization of processes across the organization. Moreover, if the Lead Appraiser agrees to conduct an appraisal for software projects only, he or she will violate the principle of randomly selecting the projects for the appraisal.

The way I look at this situation depends upon on how the company is organized. If there are separate hardware and software development groups, departments, or divisions that deliver products to a program office (for example), then the software group could be appraised on its own. The same could be true for the hardware group. In fact, this situation occurs quite frequently in my experience here in the United States.

However, if the hardware and software groups are tightly intertwined in building and delivering a product (meaning you cannot separate the two), then I would say that both the hardware and software groups had to be appraised together.

The correct decision requires the Lead Appraiser to have a very good understanding of the organization and its business, as well as being a function of how the organization defines a project and what its process documentation states.

Product Planning and Configuration

It is common knowledge that Maturity Level 2 is project specific, and still I find at times Lead Appraisers asking funny questions during SCAMPI A appraisals. Quite recently, one of my friends told me that his Lead Appraiser is looking for planning at the product level, as well as Configuratuion Management at the product level. I was a bit amazed, thinking that Maturity Level 2 focuses on Project Planning, not product planning. What do you think about this situation? Have you been faced with this situation before? Is there a workaround for it?

From what you describe, it sounds like this Lead Appraiser could be misinterpreting the CMMI and possibly misleading the organization. The CMMI is quite clear that the Project Planning (PP) Process Area (PA) is for project planning purposes, not product planning.

"The purpose of PP is to establish and maintain plans that define project activities."

However, sometimes the difference between project and product can be blurred. By not knowing the context of the situation you described, the Lead Appraiser may have been trying a different approach to draw project planning information out in the interview sessions.

In one respect, it really doesn’t matter the line of questioning in a SCAMPI interview session. The Lead Appraiser could really ask about any topic. However, once he or she starts deviating from the CMMI, they are on shaky ground and could lose credibility. What does matter however, is the set of findings produced by the Lead Appraiser and the Appraisal Team. If there are findings associated with product planning that cannot be tied to the satisfaction of a CMMI Specific Goal or a Specific Practice, then these would be non-model findings and should have no impact on the resulting appraisal rating. However, if these non-model findings do impact the appraisal rating and the Lead Appraiser and Appraisal Team fail to demonstrate the linkage to Goal and Practice satisfaction/implementation, then the Lead Appraiser has not correctly performed his or her Lead Appraiser duties and the SEI should be informed about this issue so it can be investigated.

Importance of the Sub-Practices

I am trying to explain to a organization that they need to look at the sub-practice areas and make sure that the artifacts for the SCAMPI A also answer the subpractice areas. I understand that the subpractice area are a detailed description that provides guidance.

Has the organization been trained on the SEI’s 3-day Intro to CMMI class? The CMMI instructor should have explained and emphasized the role of the informative material (e. g., sub-practices). And in the words of Rusty Young, this material is “informative” NOT “ignorative.” Another way to look at the informative material is if it has no value to the model, there is no point in including it. Then the model would only consist of goal and practice statements, which would only take about 10 pages to document. The sub-practices are provided to help the reader understand the intent of the practice and goal statements.

HOWEVER, in a SCAMPI A appraisal the appraisal team will only be evaluating the required (goals) and expected (practices) components of the model, NOT the informative material (sub-practices et. al.) So you would be mistaken if you required the organization to provide evidence (Direct and Indirect) for the sub-practices. The organization only provides evidence for the goals and practices in a SCAMPI A.

Cut Off Time for Updating Documents Prior to a SCAMPI

What is the normal time-frame an organization is allowed to continue making changes to their documentation prior to a SCAMPI?

There is no hard and fast rule for this practice. You need to work this time frame out with your Lead Appraiser to see what he or she is comfortable with. But, think about what you are asking for a minute. Are you talking about process changes? Changes to artifacts? Or both?

If you are talking about process changes, then you need to consider the purpose of the SCAMPI. One of the jobs of the appraisal team is to determine the amount of institutionalization. In order to determine the degree of institutionalization (GGs and GPs), changes to the processes and procedures need to be minimized so there is sufficient time for institutionalization and to collect and present the proper Direct and Indirect Evidence. To be on the safe side and mitigate this risk, organizations may decide to have no process changes for six months before the SCAMPI.

If you are talking about the artifacts, then you need to keep in mind the definition of Focus and Non-Focus Projects and work with your Lead Appraiser to determine your evidence needs. In my experience, my clients have taken the risk mitigation approach of “freezing” the evidence about a month before the Readiness Review to build the PIIDs and then only allow changes after that point if there are weaknesses in the PIIDs that need to be addressed before the SCAMPI.

Please keep in mind that I am not advocating freezing the processes 6 months before an appraisal, it just has been my experience that as a risk mitigation some clients have held off making changes until after their appraisal. This behavior is typical for a first time SCAMPI A in a risk averse organization who wants to do everything possible to have a successful SCAMPI A. After all the CMMI is a set of process improvement guidelines, so I as a Lead Appraiser would expect to see evidence of continuous process improvement. But the org has to take an intelligent approach when rolling out new changes. The workforce gets frustrated with chasing a moving target if the processes and assets are frequently changing, i.e. major updates.

Can the Same Firm Provide CMMI Consulting & SCAMPI A Services?

I have been hearing that the SEI has stringent rules stating that the consulting firm cannot perform a SCAMPI A appraisal as it results in conflict of interest and the appraisal may lose it's objectivity. Couold you please throw light on this issue and give me the correct information? It would be great if you can refer to any documents listing these rules e.g. SEI audit policies.

Yes, the same COMPANY and same LEAD APPRAISER can provide both CMMI consulting and SCAMPI A appraisal services to the same client. However, the Lead Appraiser and SEI Partner must notify the SEI if there is a Conflict of Interest. The situation to avoid is the Lead Appraiser appraising his or her own work in a SCAMPI A appraisal. The Code of Professional Conduct (CoPC) spells out the policy and the steps that must be taken if there is an indentified Conflict of Interest. http://www.sei.cmu.edu/partners/copc.html

The primary issue concerns the type of consulting provided by the Lead Appraiser. If the consulting consists of writing processes and process assets for the client, then there is a clear Conflict of Interest. But if the consulting consists of training on different ways to implement a Process Area, reviewing documents and identifying strengths and weaknesses, reviewing evidence for inclusion in the PIIDs, etc. then there is no Conflict of Interest.

So most Lead Appraisers will try their best to remain ethical and either just provide the hands-on consulting and have another Lead Appraiser lead the SCAMPI A appraisal OR lead the SCAMPI A appraisal and help the organization get prepared for the appraisal.

What Class of SCAMPI Appraisal is Appropriate?

My organization has just started our process improvement journey. For our first appraisal, which SCAMPI class (A, B, or C) should we use?

I would strongly suggest that your organization hire either a SCAMPI Lead Appraiser or a SCAMPI B/C Team Leader to help you determine which SCAMPI appraisal A, B, or C would be best for your organization. In order for you to obtain the best information and useful results, you need to use either an SEI-trained and authorized SCAMPI B/C Team Leader or a SCAMPI A Lead Appraiser. Do not attempt to do this by yourself.

For a new client, I typically recommend an initial Gap Analysis to provide a starting point for determining the path forward. The Gap Analysis, depending upon its rigor, can be either a SCAMPI C or a SCAMPI B.

Appraisal Team Size Redux

For a continuous appraisal of lets say 1 to 2 PAs, does the minimum 4 ATM for SCAMPI A still apply? Seems unreasonable to me. The MDD does not appear to address any tailoring option on this for continuous appraisals.

The SCAMPI Method Description Document (MDD) is clear that the minimum appraisal team size is 4 people. You raise an interesting point. Forgetting the team size for the moment, why would an organization go through the expense of a SCAMPI A just for one or two Process Areas? To me that just doesn’t appear to be a good use of the organization’s money. Seems to me that the organization would be better served by a Continuous SCAMPI A evaluating 4 or more PAs that would provide a better indication to the organization. Now you may want to perform a Class C or Class B on one or two PAs. That makes sense and there are no team size limitations on Bs and Cs. In fact, the Lead Appraiser could be the only person on the team then.

Appraisal Team Size

We are a small IT company (40 members) that wants to get Maturity Level 3 as a first effort. We are thinking about forming an appraisal team (for a SCAMPI A appraisal) composed of 3 internal members and 1 external team leader. Is there any problem or recommendation about this approach?

The SCAMPI method requires a minimum appraisal team size of four people including the Lead Appraiser. And it is a good practice to have at least two appraisal team members be from the organization being appraised. So having three internal team members plus the Lead Appraiser meets the minimum SCAMPI requirements for an appraisal. From my experience, I would suggest adding some external team members. Having additional people will ease the burden on the team for evaluating the evidence and will also provide some outside perspectives on your processes. Your Lead Appraiser should be able to suggest the proper team size and recommend some external team members. And many times, external team members are willing to participate for free since they may be looking for appraisals in order to maintain their credentials, etc.

Mock CMMI Appraisal

My Organisation is planning to conduct a CMMi mock assessment soon. Googling for the same has not helped me out so I am asking if you could share the mock assessment questions if possible.

I would assume the reason that your organization would be conducting a CMMI mock appraisal would be to prepare you for the formal SCAMPI A appraisal. Basically a dress rehearsal, which should be led by your Lead Appraiser. What you are asking for is a SCAMPI B appraisal. Do not attempt to conduct this internally without a Lead Appraiser, unless you have someone internally who is a SCAMPI B or C Team Lead.

A dress rehearsal is an excellent exercise to conduct. It helps put everyone at ease and provides practical experience in what to expect for the formal SCAMPI A. Therefore, the interview questions for the mock appraisal should be the same as those asked in a SCAMPI A. But again, these questions should be asked by the Lead Appraiser, otherwise you run the risk of misinterpreting the model, the evidence, or asking inappropriate questions.