Some Appraisal Questions

1. Why does the SEI ask for focus projects instead of all the projects done by the company?
2. Usually companies can select consultants and the Lead Appraiser (LA). Why is the SEI giving the right to chose the LA by the company or consultant?
3. Why do the appraisal results expire after 3 years?
4. Why doesn't the SEI have compliance appraisals every 6 months or 1 year similar to ISO?

1. The SCAMPI method is a sampling method to determine the degree of institutionalization of the processes on the projects. Therefore the use of focus projects. For organizations where there are only 1 or 2 projects, then all of the projects are usually included in the appraisal scope. But for organizations with many projects, it would be prohibitive to evaluate all of the projects. That is why it is the responsibility of the Lead Appraiser to select the focus projects, along with input from the organization. The principle here is that if the processes are truly institutionalized throughout the organization, then it doesn’t matter which projects are selected for the appraisal. Any set of selected projects should be representative of how all projects in the organization behave.
   
   
2. If I understand your statement, you are incorrect. The organization does select the CMMI consultant and Lead Appraiser. However, only SEI-certified Lead Appraisers are allowed to lead and report SCAMPI appraisal results. If a Lead Appraiser is NOT SEI-certified and he or she leads a SCAMPI appraisal, then the appraisal results are NOT valid.
   
   
3. The appraisal results expire after three years because in the past many organizations tended to backslide in their process maturity after having their appraisal. The three year period is long enough to address the findings from the SCAMPI A appraisal and prepare for a re-appraisal at the same or higher Maturity Level. If there is no expiration date, then there could be less motivation to continue with Process Improvement.
   
   
4. In addition, there is no such thing as a compliance appraisal at this time. There has been some discussion along these lines, but nothing has been settled. There is a fundamental difference between ISO audits and CMMI appraisals. ISO is a standard and the result of the audit is certification. CMMI is a set of guidelines for process improvement and the result is Maturity Level or Capability Level that is valid for three years.

Selecting Projects for a SCAMPI Appraisal

One of my client organisations works on hardware and software design projects related to locomotive design for a manufacturer. They want to adopt the CMMI-DEV and scope their SCAMPI appraisals only for their software projects. Both, hardware and software design projects are undertaken by the same organisation, under the same management, same company name and at the same location. Is it be proper for the organization to exclude the hardware projects from their CMMI journey and appraisals? I feel it is not proper and it violates the principle of institutionalization of processes across the organization. Moreover, if the Lead Appraiser agrees to conduct an appraisal for software projects only, he or she will violate the principle of randomly selecting the projects for the appraisal.

The way I look at this situation depends upon on how the company is organized. If there are separate hardware and software development groups, departments, or divisions that deliver products to a program office (for example), then the software group could be appraised on its own. The same could be true for the hardware group. In fact, this situation occurs quite frequently in my experience here in the United States.

However, if the hardware and software groups are tightly intertwined in building and delivering a product (meaning you cannot separate the two), then I would say that both the hardware and software groups had to be appraised together.

The correct decision requires the Lead Appraiser to have a very good understanding of the organization and its business, as well as being a function of how the organization defines a project and what its process documentation states.

Criteria for selection of Focus and Non Focus projects for CMMI 1.2 Assessment

What are the criteria used to determine project selection for CMMI 1.2 Appraisals? I understand they are normally termed as Focus and Non Focus projects.

1. Based on size of organization ( count of projects + resources ) how many projects need to be selected? Can someone specify this as a number of projects or percentage of projects required?
2. We have categorised projects as Development + Maintenance + Testing - so ideally how many projects count or percentage wise under each Project Type would need to be selected?
3. What is the criteria for selecting a project as a focus project? I have read and heard that one of the factors could be presence of all Software Development Lifecycle (SDLC) phases in that project, contribution of data points, it should be an on-going project at time of assessment etc.

Here is the criteria for project selection extracted from section 1.1.3 of the Method Description Document (MDD) v1.2:

Sample projects and support groups selected to form the organizational scope (i.e., the combination of focus and non-focus projects and support functions) must represent all critical factors identified for the organizational unit to which the results will be attributed. The coverage of the organizational critical factors provided by these sample projects and support groups in the organizational scope in relation to the organizational unit must be documented, in quantitative terms, in the appraisal input and ADS.

Each sample project or support group in the planned organizational scope of the appraisal must be one of the three types listed below:

• Focus projects must provide objective evidence for every PA within the model scope of the appraisal which addresses model practices applicable to those projects.
• Non-focus projects must provide objective evidence for one or more PAs within the model scope of the appraisal which address practices performed on projects.
• Support functions must provide objective evidence for practices within the model scope of the appraisal that address organizational infrastructure or functions.

In appraisals where the reference model scope includes any project-related PA, the organizational scope must include at least one focus project. If the organizational unit includes more than 3 projects, then the organizational scope must include sufficient focus projects and non-focus projects to generate at least 3 instances of each practice in each project-related PA in the model scope of the appraisal.

Projects, categories, or groups/functions that are specifically excluded from the appraisal must be identified in the appraisal input and in the ADS as well as the justification for their exclusion. This identification includes legacy projects not using current organizational processes and projects waived from using current organizational processes. As needed, the appraisal team may seek clarification or data from other projects or support functions within the organizational unit. These projects or support functions must also be identified in the ADS.

What this means is that you must have at least one focus project. A focus project is a project that has evidence for every practice and Process Area (PA) in the appraisal scope. In order for that to be the case, that means the project is either already completed or very close to completion. A non-focus project is a project that does NOT have evidence for every practice and PA in the appraisal scope. Meaning a non-focus project is either a project that has recently started or a project that began before the processes were deployed and only has evidence for a subset of the practices.

Keep in mind that some PAs are organization level PAs while others are project-related. For the Organization PAs (e.g., OPF, OPD, OT, etc.) you only have to provide one instance of evidence. So the issue of number of projects only applies to the project-related PAs (e.g., PP, PMC, IPM, REQM, RD, etc.). Therefore, according to the MDD, if the organization has more than three projects, you have provide three instances of evidence for each practice. You could choose to supply three focus projects, two focus projects and multiple non-focus projects, or one focus project and multiple non-focus projects.

Another aspect of evidence that you must comply with is that you have to provide Direct Evidence for every practice and PA in the scope of the appraisal. But you don’t not have to supply Indirect Evidence for every practice and PA. Your Lead Appraiser can explain to you the one row, one column and 50% rules. As a risk mitigation, most of my clients choose to provide 100% coverage for both Direct and Indirect Evidence.