SEI Published Appraisal Results

I have a question related to the published data in SEI web site: http://www.sei.cmu.edu/appraisal-program/profile/pdf/CMMI/2008MarCMMI.pdf Process Maturity Profile by All Reporting Organizations (Page 5):

1. 1.5% or the reporting organizations are (Initial - Maturity Level 1) rating. How is a Maturity Level 1 rating achieved?
2. Also, 8% of the reporting organizations indicate that a Maturity Level is Not Given. What does ‘Not Given’ mean…? Level 0 ?

In my understanding, Maturity Level is related to the Staged representation and can have
rating from ML 2 - 5.

It is great to see and hear that people are actually looking at this information and asking questions about how to interpret the data. You are on your way to understanding Measurement and Analysis!

To answer your first question, how is Maturity Level 1 provided? Obviously, an organization does not conduct a SCAMPI A appraisal to confirm that they are Maturity Level 1. So what happens in this case is the scope of the SCAMPI A appraisal is Maturity Level 2, 3, 4, or 5. But what happens is during the appraisal a serious goal impacting weakness in a Maturity Level 2 Process Area is found by the appraisal team. By the rules of the SCAMPI method then the resulting Maturity Level is 1. Since the SCAMPI A results must be submitted to the SEI, a rating of Maturity Level 1 appears. It doesn’t happen that often because most organizations take the appropriate steps to be prepared to achieve their target Maturity Level. That is why 1.5% of the reported results are for Maturity Level 1.

The Not Given category means that the organization did not report its resulting Maturity Level rating. I would hazard a guess that it probably means that they had successfully achieved Maturity Level 1. There is no Maturity Level 0, despite what many chaotic organizations say they are when first encountering the CMMI.

A Question About Estimation

I'm a bit confused about what the CMMI requires for estimation. Is it valid for us to define our process as simple as guessing? Could we use something similar to Planning Poker or other similar agile techniques? The problem is that we've been told we need to have a quantitative and repeatable process for estimating in order to meet Project Planning (PP) SP 1.4 "Estimate the project effort and cost for the work products and tasks based on estimation rational." But in reality we don't have historical data and many of us think estimating is subjective anyway.

When you start implementing Maturity Level 2 there is no expectation that you have any historical data for estimating. The best you can do at that point is guess, SWAG, or use engineering judgment. One of the characteristics of being a Maturity Level 2 organization is that you are learning. You are learning what it means to estimate a project. You are learning how to improve your estimates so that over time you will be able to produce accurate estimates to build the foundation to allow the organization to move to Maturity Level 3. So, by the time you are ready for a Maturity Level 2 SCAMPI A appraisal, the expectation is that you have a repeatable estimation process (repeatability is at the core of Maturity Level 2) that is based on historical project data. Estimation is not difficult, it just takes a little bit of brain power and being systematic in your approach. One of the purposes of achieving Maturity Level 2 is moving from being subjective in estimation to being objective. If you have people that believe estimation is subjective, then I am afraid that you are still operating as a Maturity Level 1 organization.