REQM and RD in the CMMI

Why is REQM Management at Maturity Level 2 and Requirement Development at Maturity Level 3? We develop the requirements first and then manage them in the project.

There reason for the placement is due to the meaning of ML 2 vs. ML 3. ML 2 is all about stabilizing projects and gaining control over project estimates. Once the organization has achieved this, then it can begin to evaluate how to improve the engineering areas.

Since you need to have a baseline upon which to plan a project and the other ML 2 Process Areas, that is why REQM is the first Process Area in ML 2. The intent is to manage the collection of project requirements: good, bad, or indifferent. And use this collection to plan the project, etc. Then when you have achieved ML 2 and move to ML 3, then you can address how to improve the Requirements Elicitation to obtain better requirements.

Please keep in mind that the CMMI is a collection of guidelines and best practices for doing process improvement. The CMMI is not a roadmap for how to do software engineering.

Implementing CMMI Along With ISO 9001

Suppose a company already has an ISO 9001 certification. Then they decide to achieve an appraisal at level 3. Suppose further that their plan is to add additional process assets according to their gap analysis. But then they discover that certain ISO 9001 work instructions or templates require changes to meet the requirements of CMMI level 3. If these assets are changed, would that necessarily invalidate the ISO 9001 certification? Under what conditions would the certification be invalidated, and what needs to be done under those circumstances? Is there a way to avoid this issue?

Without having any details as to your situation, I find it hard to believe that if you are ISO 9001 certified that the CMMI is causing you to make changes to your quality system that would invalidate your ISO 9001 certification. The ISO 9001 standard and the CMMI-DEV model are compatible. Are you working with an SEI-certified Lead Appraiser/consultant? If not, you may be making some decisions to change processes and process assets that are not necessary.

To properly address your concerns, you should have a Lead Appraiser conduct a gap analysis of your organization to determine what you currently have in place that is compatible with the CMMI and identify those gaps that need to be addressed in order for your organization to achieve Maturity Level 3. And any updates to existing processes and process assets should be compatible with both the ISO standard and the CMMI-DEV.

PPQA After Maturity Level 2

I work in an IT organization that achieved CMMI Maturity Level 2 several years ago (we let the rating lapse) and I was wondering if you had some ideas on the following two questions:

1) What types of activities would PPQA engage in if the org had been Maturity Level 2 (I think they could have pursued Maturity Level 3 and been close)? Please also consider that the company is pursuing other types of improvement methods and models such as lean/6-sigma and ITIL.

2) What strategies should we pursue to show the worth of PPQA? Even in the good old CMM days and SQA one of the issues I had was that it was difficult to show the practical monetary worth of these support functions; one generally had to take it on faith that PPQA/SQA delivered some degree of worth to the company. Any thoughts?

The answer to question 1 is simple. Just read the PPQA Process Area and GP 2.9. The PPQA activities include performing both process and work product audits of the project and organization processes. For Maturity Level 2 that would mean auditing your REQM, PP, PMC, SAM, MA, PPQA, and CM processes.

The answer to question 2 is a bit more difficult. Basically you are asking, what is the cost of quality? One method you can use is to look at the total cost for the project and analyze it using Crosby’s Cost of Quality Model. The total costs break down into two categories: the Cost of Quality and the Cost of Performance.

The Cost of Performance includes such things as: generating plans, documentation, and developing requirements, design, code, and integration.

The Cost of Quality breaks down further into two categories: Cost of Conformance and Cost of Non-Conformance.

The Cost of Non-Conformance includes fixing defects, reworking documents, updating source code, re-reviews, re-tests, patches, engineering changes, CCBs, external failures and fines, Customer Support, and Help Desk.

The Cost of Conformance breaks down to two more categories: Cost of Appraisal and Cost of Prevention.

The Cost of Appraisal includes reviews, walkthroughs, testing (first time), independent V&V, and Audits.

The Cost of Prevention includes training, policies, procedures, tools, planning, quality improvement, data gathering and analysis, root cause analysis, and quality reporting.

The cost of PPQA is included in the Cost of Prevention.

When you consider these definitions and cost break down, the only category that will be affected by PPQA is the Cost of Non-Conformance. When PPQA audits the processes and work products, the audits will reveal non-conformances with people following the documented processes and procedures, which lead to re-work. By addressing these non-conformances, the goal is to reduce or effectively eliminate the rework and that is where you can demonstrate the value of PPQA.

Hope this helps.

Rationale for Maturity Level 5

Our organization is currently at Maturity Level 3 and is now planning to go for Maturity Level 5. Our CEO is asking for the short term and long term benefits and challenges for implementing Maturity Level 5. How do we provide this information?

The question that needs to be asked is why does your organization want to achieve Maturity Level 5 (ML 5)? If the organization has already achieved Maturity Level 3 (ML 3), what is the motivation for achieveing ML 5? Most likely it is not mandated in a contract or by its customers. But achieving ML 5 is desirable to be competitive in the marketplace. Therefore, the reasons for achieving ML 5 should provide some indication of the short term and long term benefits.

The first step in becoming a High Maturity organization is defining your Quality and Process Performance Objectives (QPPOs) that are based on your business goals and objectives as well as your customer needs. The QPPOs should be stated in a form such that they specify a timeframe. And that information will provide some ideas of the short term and long term benefits.

However, the best advice that I can provide is to hire an SEI-certified High Maturity Lead Appraiser (HMLA) who will work with you to help explain the benefits of ML 5 to the CEO. The HMLA should have experience working with many different organizations at various Maturity Levels and be able to talk about the different challanges that have been faced by other organizations, as well as the challenges and risks within your organization.

At this point, it sounds like your organization is just beginning its journey to ML 5, so I would have to be convinced that your processes are stable enough to provide the data needed to quantify any short term and long term benefits. I think that the best you could do at this point is communicate this information in qualitative terms. Any quantitative information may not be accurate until you have implemented High Maturity.

Software Sizing

We are trying to achieve CMMI Maturity Level (ML) 3 in my company and we have decided to skip ML 2. So, now, one of our problems is related to software size estimation. We defined a proprietary method, based on Use Case Points and Function Points, but the practitioners are struggling with it. From your experience, what other methods have you seen or implemented in the companies with this same problem? Or, if a proprietary method was defined, what were the main aspects to take in account?

I would strongly urge you to forget the ML 3 Process Areas until you have mastered ML 2. There is a fundamental difference between how a ML 2 Project Manager approaches Project Planning (PP) and Project Monitoring and Control (PMC) vs. a ML 3 Project Manager. Estimation being one of the differences. Use Case Points and Functions Points are fairly sophisticated concepts and there are challenges with getting consistency in determining what each of these things are. I would recommend that you take a step back from the model and the projects and look at your historical project data. Use the actual effort, costs, etc. from previous projects to estimate a new project. Forget about Use Case Points and Function Points for now. Once you have mastered being able to use historical information to build an empirical estimation model, then it might make sense to add a layer of sophistication by considering Use Case Points or Function Points.

Another recommendation is let the Project Manager create the project estimates and then review them with the practitioners as a sanity check rather than ask the practitioners to create the estimates. Over time as the organization gains experience estimating projects etc., then it makes sense to involve the practitioners up front in the estimation process. You have to learn to crawl first with estimation before you can sprint with the big boys.

SEI Published Appraisal Results

I have a question related to the published data in SEI web site: http://www.sei.cmu.edu/appraisal-program/profile/pdf/CMMI/2008MarCMMI.pdf Process Maturity Profile by All Reporting Organizations (Page 5):

1. 1.5% or the reporting organizations are (Initial - Maturity Level 1) rating. How is a Maturity Level 1 rating achieved?
2. Also, 8% of the reporting organizations indicate that a Maturity Level is Not Given. What does ‘Not Given’ mean…? Level 0 ?

In my understanding, Maturity Level is related to the Staged representation and can have
rating from ML 2 - 5.

It is great to see and hear that people are actually looking at this information and asking questions about how to interpret the data. You are on your way to understanding Measurement and Analysis!

To answer your first question, how is Maturity Level 1 provided? Obviously, an organization does not conduct a SCAMPI A appraisal to confirm that they are Maturity Level 1. So what happens in this case is the scope of the SCAMPI A appraisal is Maturity Level 2, 3, 4, or 5. But what happens is during the appraisal a serious goal impacting weakness in a Maturity Level 2 Process Area is found by the appraisal team. By the rules of the SCAMPI method then the resulting Maturity Level is 1. Since the SCAMPI A results must be submitted to the SEI, a rating of Maturity Level 1 appears. It doesn’t happen that often because most organizations take the appropriate steps to be prepared to achieve their target Maturity Level. That is why 1.5% of the reported results are for Maturity Level 1.

The Not Given category means that the organization did not report its resulting Maturity Level rating. I would hazard a guess that it probably means that they had successfully achieved Maturity Level 1. There is no Maturity Level 0, despite what many chaotic organizations say they are when first encountering the CMMI.

A Question About Estimation

I'm a bit confused about what the CMMI requires for estimation. Is it valid for us to define our process as simple as guessing? Could we use something similar to Planning Poker or other similar agile techniques? The problem is that we've been told we need to have a quantitative and repeatable process for estimating in order to meet Project Planning (PP) SP 1.4 "Estimate the project effort and cost for the work products and tasks based on estimation rational." But in reality we don't have historical data and many of us think estimating is subjective anyway.

When you start implementing Maturity Level 2 there is no expectation that you have any historical data for estimating. The best you can do at that point is guess, SWAG, or use engineering judgment. One of the characteristics of being a Maturity Level 2 organization is that you are learning. You are learning what it means to estimate a project. You are learning how to improve your estimates so that over time you will be able to produce accurate estimates to build the foundation to allow the organization to move to Maturity Level 3. So, by the time you are ready for a Maturity Level 2 SCAMPI A appraisal, the expectation is that you have a repeatable estimation process (repeatability is at the core of Maturity Level 2) that is based on historical project data. Estimation is not difficult, it just takes a little bit of brain power and being systematic in your approach. One of the purposes of achieving Maturity Level 2 is moving from being subjective in estimation to being objective. If you have people that believe estimation is subjective, then I am afraid that you are still operating as a Maturity Level 1 organization.

CMMI Maturity 3 Appraisal Process

We are planning to go to CMMI 3 level certification. I recently joined in my company. Please tell me the necesary processes and steps.

1. First off, just to be clear, there is no such thing as “CMMI 3 level certification.” An organization is appraised to the CMMI using the SCAMPI A appraisal method to determine either the organization’s Maturity Level or the Capability Level of the organization’s processes. The result of the SCAMPI A is not a certification, but simply a rating of the current Maturity Level or Capability Level.
2. Has your company already achieved Maturity Level 2? Has your company hired a CMMI consultant? Has your company hired an SEI-authorized SCAMPI Lead Appraiser? Has an SEI-authorized instructor provided the SEI Introduction to CMMI class to your company?
3. If the answer to all of these questions is NO, then hire a CMMI consultant and a Lead Appraiser. The Lead Appraiser cannot provide the CMMI consulting. Most Lead Appraisers are also authorized CMMI instructors, so the next step is to train your process group and any people who might be an appraisal team member on the CMMI.
   
4. Perform a Class C appraisal (gap analysis) to identify where you need to focus your CMMI implementation efforts. Use the findings from the Class C to write a process improvement plan, and use the plan to monitor and control your CMMI implementation efforts.
   
5. Implement CMMI Maturity Level 2 FIRST. Once you have established the firm project management foundation of Maturity Level 2, THEN consider implementing Maturity Level 3. If you try to implement BOTH Maturity Level 2 and Maturity Level 3 at the same time, you will encounter difficulties. There is a huge difference between managing projects at Maturity Level 2 and managing projects at Maturity Level 3.
   
6. Once you feel comfortable that you have addressed all of the findings from the Class C and you have had several project cycles to institutionalize the documented processes, then consult with your Lead Appraiser to determine if your organization is ready to conduct a benchmarking SCAMPI A appraisal.
   
7. There will be more training (appraisal team and PIIDs) and activities leading up to the SCAMPI A, but your Lead Appraiser will tell you exactly what you will need to do to prepare for the appraisal.

ML 2/ML 3 Measurement Objectives

For a Maturity Level 3 organization , is it acceptable if the organization sets its process performance objectives qualitatively? For example reduce testing cycle time , improve productivity, etc. and in OPF SG1, one of sub-practices states that process performance objectives may be expressed either quantitatively or qualitatively. Also would you please explain this clarification linking to ML2- MA - SP 1.1 - Establish measurement objective?

These are good questions, but the best way to receive a decent understanding of the concepts it to attend the SEI’s Introduction to the CMMI class. Intro to CMMI course description Brief answers, such as mine in this blog, will probably only raise more questions that would be better addressed in a classroom setting.

1. Only for High Maturity organizations (ML 4 and 5) is there an expectation for quantitatively setting process performance objectives. It is acceptable at ML 3 for qualitative objectives. But to fully understand this distinction you really require a good discussion on the expectations for each Maturity Level, which you would receive in the Intro to CMMI class.
   
2. A measurement objective is different from a process performance objective. The measurement objectives are the purposes for which measurement and analysis is performed and also specify what kinds of actions may be taken as a result of data analyses. In order to determine the measurement objectives, you first have to describe the information needs. An information need is an insight necessary to manage objectives, goals, issues, risks, and problems. The measurement objective is derived from the information need and it is a statement about what should be measured in order to satisfy the information need. For example, the project manager or other decision maker concerned with allocating budget and associated resources to a task may believe that productivity is related to the type of task being performed. Increasing productivity is then the measurement objective that addresses the defined information need. Determining productivity then requires that entities such as the product and the process be measured. There are many ways that productivity can be computed, but the measurement objective is unaffected by the different methods.

To fully understand these concepts you should take a class on Measurement and Analysis or Practical Software Measurement. Trying to explain these concepts in a paragraph or two does not do the subject justice.

Query on ML 3 and ML 4

What is the significance of Maturity Level 3 and Maturity Level 4? And can you explain to me what is Integrated Project Management?

What broad questions! These questions really need a long in depth answer and are addressed very well in the Introduction to CMMI class. So first off I would suggest that you find an opportunity to take this class. Please visit http://www.ppqc.net/training/training.htm for more information about the class.

To briefly answer these two questions, the answer needs to address ML 2 as well. So I will start with some definitions from the CMMI book.
Process Area (PA) – a cluster of related practices in an area that, when implemented collectively, satisfy a set of goals considered important for making improvement in that area.
Maturity Level (ML) – degree of process improvement across a predefined set of process areas in which all goals in the set are attained. An ML is a defined evolutionary plateau for organization process improvement. Each ML matures an important subset of the organization’s processes, preparing it to move to the next ML.
Maturity Level 1: Initial – processes are usually ad hoc and chaotic. The organization usually does not provide a stable environment to support the process. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes.
Maturity Level 2: Managed – projects of the organization have ensured that processes are planned and executed in accordance with policy; the projects employ skilled people who have adequate resources to produce controlled outputs; involve relevant stakeholders; are monitored, controlled, and reviewed; and are evaluated for adherence to their process descriptions, The process discipline reflected by ML 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.
Maturity Level 3: Defined – processes are well characterized and understood, and are described in standards, procedures, tools, and methods. The organization’s set of standard processes, which is the basis for ML 3, is established and improved over time. These standard processes are used to establish consistency across the organization. Projects establish their defined process by tailoring the organization’s set of standard processes according to tailoring guidelines.
Maturity Level 4: Quantitatively Managed – the organization and projects establish quantitative objectives for quality and process performance and use them as criteria in managing processes. Quantitative objectives are based on the needs of the customer, end users, organization, and process implementers. Quality and process performance is understood in statistical terms and is managed throughout the life of the processes.
Maturity Level 5: Optimizing – an organization continually improves its processes based on a quantitative understanding of the common causes of variation inherent in processes.

Given these definitions and explanations, one of the fundamental differences between ML 3 and ML 4 is that at ML 3 the organization is learning how to use a standard set of processes, tailoring them to the individual project needs, and collecting enough process data such that Process Performance Baselines and Process Performance Models can be built and used at ML 4 to quantitatively manage projects and statistically manage sub-processes to achieve the organization’s quality and process performance objectives.

To answer the second question, you first need to understand the Project Planning (PP), Project Monitoring and Control (PMC), and Integrated Project Management (IPM) PAs. PP and PMC are ML 2 PAs that address the basic project management practices of planning a project, creating a project plan, and using that project plan to track and monitor the project. At ML 2, the organization typically is learning how to create accurate and realistic estimates by building estimation models. It takes time to refine these estimation models, so an ML 2 organization is expected to frequently revise and re-baseline the project plan as the projects get smarter about estimation. At ML 3, one of the project management expectations is that the project estimates are now accurate and realistic. So, rather than constantly update the estimates to match the actuals as done at ML 2, the Project Manager now manages the project to the estimates, meaning that the PM can now fairly accurately predict early on in the lifecycle whether or not the project will hit its downstream targets and take appropriate corrective action to mitigate these risks. The other differences between IPM and PP/PMC include establishing the project’s defined process by applying appropriate tailoring criteria to the organization’s standard processes, establishing the project’s work environment, integrating the various plans that comprise the project plan, managing the project using the integrated plans, and managing the project’s relevant stakeholders. In other words, IPM builds on the project management foundation established by PP and PMC.

This a lengthy explanation but only a surface treatment on these subjects. Again I strongly recommend to anyone interested in this topic that you attend an offering of the Introduction to CMMI v1.2 class. You will go into these concepts in much greater detail and you will come out with a much better understanding of the model, PAs, and MLs than I can convey in this blog.

Can I Update My Appraisal Results After Submission to the SEI?

I had an interesting call from a client the other day. He wanted to know if is was possible to change the wording in the Appraisal Disclosure Statement after it has been posted by the SEI on the Published Appraisal Results web site. I told him no, after the appraisal sponsor and Lead Appraiser have signed the ADS and submitted it to the SEI, there can be no changes. I then asked him what he wanted to change on the ADS and he said it was the wording of the project descriptions. Apparently they had responded to an RFP that required the bidders have a Maturity Level 3 rating for the type of product development required by the contract. Unfortunately for the client, their appraisal covered a different type of product development and they got downgraded for this reason. There are several lessons to be learned from this situation: 1. appraisals should cover projects that are the core business of the organization and 2. there should be a section in the proposal that explains how the Maturity Level 3 capability and the processes and procedures from an unrelated project will be used on the contract to achieve the same results.