Review Activity for a Short Term Project

Our organization will be going through CMMI Maturity Level 2 Appraisal in a couple of months. I have a PPQA question. As per the PPQA Process Area (PA), we require a review of the work products (content/template) and procedures required at Maturity Level2 during the project life cycle. We have one project that is 3 months long. There are many work products that will be produced during the project development life cycle.

• Requirement documents such as SRS, Use cases, Bidirectionally traceability matrix document, change log, etc;
• Plans for all the PAs, e.g. requirements management plan, project plan, configuration plan, etc;
• Development artifacts, such as ERD, Code, UML diagrams, etc;
• QC artifacts, such as test cases, test reports, etc.
• Monitoring/controlling artifacts, such as Issue list, MoMs, Risks, etc.

How is it possible to review the work products for a 3 month project when we don't have a separate QA department and the stakeholders involved in development do the work product reviews one way or the other.

This same question holds true for reviewing procedures.

Of course, we review high priority documents, such as Project Plan, Use Cases, ERD, Application; but not all of them.

Can you help me understand what should be done for a short duration project, such that the PPQA PA requirements are met and we don't have to hire separate people just to fulfill the requirement?

The first thing that I would do is postpone your ML 2 SCAMPI A appraisal as apparently you have a major risk to achieving ML 2 since PPQA does not appear to be in place in your organization. And even if you could put PPQA in place for a 3 month project between now and your appraisals, that may still not be enough time to demonstrate institutionalization, meaning that you have a repeatable process. Essentially you will have one project using PPQA, which is one data point. And it is not possible to determine institutionalization from one data point. Your organization will be at serious risk of not achieving ML 2.

Industry average shows that PPQA is 3 – 5% of your organization. You haven’t told me how large your organization is. But if your organization is 25 people, than 1 person should be assigned to perform the PPQA practices.

I think that you are misunderstanding the differences between reviewing a work product and objectively evaluating a work product. It sounds like your project teams are already reviewing the work products. The role of PPQA is not to review the work products, but to audit the work products and processes to ensure that the work products follow the specific standards and are products according to your documented processes.

I highly recommend that you, or someone you select in your organization, take a training class on how to perform PPQA. I cannot adequately explain how to perform PPQA and answer your specific questions in this blog. The person you select for the training needs to be taught how to conduct a work product audit, how to conduct a process audit, how to plan PPQA audits, how to communicate audit results, and how to track audit non-compliances to resolution. If you don’t already have this capability in house, it will take some time to develop it internally. And I strongly advise against using an external consultant to provide this service. PPQA is for the benefit of your organization and management. It is essentially the eyes and ears of your senior management. And an external consultant may be motivated by other considerations than your best business interests if asked to provide PPQA services.

Achieving a Maturity Level Without a Consultant or Training

I was wondering if it is possible to go it alone with CMMI Maturity Level 2. I have been told by many that attempting CMMI Maturity Level 2 without a consultant or highly trained staff would be somewhat challenging. What are your thoughts?

Also, having limited resources for process improvement being a strong possibility, do you have any recommendations for online sources that can help offset the costs?

Though it is entirely possible to implement the CMMI without hiring a CMMI consultant, that is a high risk approach. Though I don’t see how you could avoid taking the Introduction to CMMI training class. That, in my opinion, would be a huge mistake. There are some areas in the CMMI that are open to interpretation when trying to go it alone and you can end up doing things in the spirit of achieving Maturity Level 2 that have no business value to you. In addition, since your organization has to provide 4 to 8 appraisal team members, each appraisal team member must take the SEI’s Introduction to CMMI class.

These are all worthwhile expenditures of your process improvement budget. And when you compare these expenses to your internal costs for process improvement, these are usually negligible in comparison.

But if you are trying to do things as cheaply as possible, that begs the question, why are you even considering implementing the CMMI and trying to achieve Maturity Level 2? Basically you get what you pay for. Going the cheap route doesn’t demonstrate management commitment to process improvement and can result in wasted effort, wasted money, and an aborted process improvement initiative.

Bi-directional Traceability

Our organization is in the process of preparing for a CMMI Maturity Level 2 SCAMPI A appraisal. We are concerned about our approach for bi-directional traceability REQM SP 1.4. We maintain traceability is follows:

1. High Level Requirements <--> Use Cases (Includes GUIs and Database Interactions) <--> Test Cases

2. Use Cases <--> Source Code

Note: One can trace from Test Cases to Source Code through the Use Cases and Vice Versa, but the traceability is not direct. The reason behind this is, test cases are generated from use cases and are tested against the application (black box testing). Source code does not have associated test cases.

Is this kind of traceability considered bi-directional and is satisfactory for Maturity Level 2?

What you describe is one of many ways to implement bi-directional traceability and meet the intent of the CMMI. If your method supports your business goals and objectives and there are no quality issues, your approach should be acceptable for a Maturity Level 2 appraisal.

It is interesting that you think tracing from Test Cases to Source Code via Use Cases may not be acceptable. Traceability is a multi-dimensional mapping that can have one-to-many and many-to-one relationships. As long as you trace from the top all the way to the bottom and vice versa, you should be fine no matter how many links there are in the chain, and the chain can have branches as well.

Please note that bi-directional traceability does not mean tracing one whole document to another whole document. What it means is that a given item in one document (a specific requirement for example) can trace to multiple items in another document, multiple items in one document can trace to one item in another document, one item can trace to one item, etc.

REQM and RD in the CMMI

Why is REQM Management at Maturity Level 2 and Requirement Development at Maturity Level 3? We develop the requirements first and then manage them in the project.

There reason for the placement is due to the meaning of ML 2 vs. ML 3. ML 2 is all about stabilizing projects and gaining control over project estimates. Once the organization has achieved this, then it can begin to evaluate how to improve the engineering areas.

Since you need to have a baseline upon which to plan a project and the other ML 2 Process Areas, that is why REQM is the first Process Area in ML 2. The intent is to manage the collection of project requirements: good, bad, or indifferent. And use this collection to plan the project, etc. Then when you have achieved ML 2 and move to ML 3, then you can address how to improve the Requirements Elicitation to obtain better requirements.

Please keep in mind that the CMMI is a collection of guidelines and best practices for doing process improvement. The CMMI is not a roadmap for how to do software engineering.

How to Start With the CMMI

If a private software company wants to do get a certificate of quality for CMMI Level 2, what must this company do in order to obtain Level 2? And also what are the steps that must be followed from the start (initial) to achieve CMMI Level 2 (Managed)?

First let me state that there is no certification of any type for the CMMI. Individual Lead Appraisers will issue something looks like a certificate to the organization that was appraised indicating the SCAMPI A results. But this is not a certification. The only things that the SEI certifies are the CMMI instructors and Lead Appraisers.

Here are the necessary steps for achieving Maturity Level 2 for the CMMI-DEV. These steps are not necessarily sequential, some can occur concurrently.

1. Hire an SEI-certified Lead Appraiser to conduct a Gap Analysis of the company to determine the current process strengths and weaknesses and help the company construct a Process Improvement Plan (PIP).
2. Obtain executive management sponsorship for the process improvement effort.
3. Train the people responsible for the company’s processes and for addressing the action items in the PIP on the 3-day SEI Introduction to CMMI v1.2 class.
4. Address the issues from the Gap Analysis, document the necessary processes and procedures, and begin conducting the PPQA process and work product audits.
5. Allow time for the new and/or modified processes to get some use on various projects.
6. Conduct a SCAMPI B appraisal as a dress rehearsal for the SCAMPI A. Identify any issues and weaknesses that are potential risks to achieving Maturity Level 2.
7. Create a new PIP to address the SCAMPI B identified weaknesses and risks.
8. Address these issues
9. Conduct the Maturity Level 2 SCAMPI A appraisal.

PPQA After Maturity Level 2

I work in an IT organization that achieved CMMI Maturity Level 2 several years ago (we let the rating lapse) and I was wondering if you had some ideas on the following two questions:

1) What types of activities would PPQA engage in if the org had been Maturity Level 2 (I think they could have pursued Maturity Level 3 and been close)? Please also consider that the company is pursuing other types of improvement methods and models such as lean/6-sigma and ITIL.

2) What strategies should we pursue to show the worth of PPQA? Even in the good old CMM days and SQA one of the issues I had was that it was difficult to show the practical monetary worth of these support functions; one generally had to take it on faith that PPQA/SQA delivered some degree of worth to the company. Any thoughts?

The answer to question 1 is simple. Just read the PPQA Process Area and GP 2.9. The PPQA activities include performing both process and work product audits of the project and organization processes. For Maturity Level 2 that would mean auditing your REQM, PP, PMC, SAM, MA, PPQA, and CM processes.

The answer to question 2 is a bit more difficult. Basically you are asking, what is the cost of quality? One method you can use is to look at the total cost for the project and analyze it using Crosby’s Cost of Quality Model. The total costs break down into two categories: the Cost of Quality and the Cost of Performance.

The Cost of Performance includes such things as: generating plans, documentation, and developing requirements, design, code, and integration.

The Cost of Quality breaks down further into two categories: Cost of Conformance and Cost of Non-Conformance.

The Cost of Non-Conformance includes fixing defects, reworking documents, updating source code, re-reviews, re-tests, patches, engineering changes, CCBs, external failures and fines, Customer Support, and Help Desk.

The Cost of Conformance breaks down to two more categories: Cost of Appraisal and Cost of Prevention.

The Cost of Appraisal includes reviews, walkthroughs, testing (first time), independent V&V, and Audits.

The Cost of Prevention includes training, policies, procedures, tools, planning, quality improvement, data gathering and analysis, root cause analysis, and quality reporting.

The cost of PPQA is included in the Cost of Prevention.

When you consider these definitions and cost break down, the only category that will be affected by PPQA is the Cost of Non-Conformance. When PPQA audits the processes and work products, the audits will reveal non-conformances with people following the documented processes and procedures, which lead to re-work. By addressing these non-conformances, the goal is to reduce or effectively eliminate the rework and that is where you can demonstrate the value of PPQA.

Hope this helps.

CMMI Practices for Documentation Teams

Our organization is CMMI Maturity Level 3 Ver 1.2 certified. Our delivery teams are going to implement CMMI practices soon. I did see the processes of the organization and though all the roles appeared from project leader to developer to manager, except for documentation teams. In the same context, I am very curious to see if there are any set of practices to be followed for the documentation department in CMMI as all the processes at CMMI ML2, ML3 are specific to project management, engineering, support, organization areas.

Your question actually raises some other questions:

1. What are the roles and responsibilities of your documentation teams and documentation department?
2. If this group of people is responsible for the technical documentation (e.g. requirements, design, etc.) and the user documentation, how and why were they excluded from your ML 3 SCAMPI A?
3. How are the delivery teams different from the organization that achieved Maturity Level 3?

What is puzzling with your question is that REQM, RD, TS, PI, and CM cover the different aspects of writing and controlling the various documents associated with designing, developing, maintaining, operating, using, and deploying products. And then VER covers the inspection/review of the documents before placing them in the baseline and controlling them with CM.

It does appear from your description that your organization omitted the documentation people as a process role from your process documentation. In my opinion, at a minimum, your PPQA audits should have identified this omission long before your SCAMPI A appraisal. Then your Lead Appraiser should have identified this gap during the appraisal planning process before the SCAMPI A and should have taken steps to address the gap or postponed your appraisal until the documentation group was included in the scope. Since your documentation group was apparently not included in the scope of your appraisal, this oversight also calls into question your Lead Appraiser’s credentials and quite possibly the validity of your SCAMPI A results.

The bottom line, in my opinion, and based on only what you stated, your documentation group should have been included in the scope of your ML 3 SCAMPI A. Even if all they do is Document Configuration Control (which would be covered under CM) or Document Quality Assurance (which would be covered under PPQA).

The answers to my above questions could provide additional information that would change my opinion.

Identifying Risks

What is the difference between PP SP 2.2 Identify Project Risk and RSKM SP 2.1 Identify Risks?

What you are asking about is one of the basic differences between Maturity Level 2 (ML 2) and Maturity Level 3 (ML 3). Project Planning (PP) is a ML 2 Process Area (PA) and Risk Management (RSKM) is a ML 3 PA. At ML 2, the project only needs to be able to identify risks and that is what PP Specific Practice (SP) 2.2 addresses. At ML 3, RSKM builds upon the foundation of identifying and tracking risks put in place by PP and Project Monitoring and Control (PMC). RSKM SP 2.1 therefore builds upon PP SP 2.2 by adding more rigor for risk identification. Just read the informative material and sub-practices for both SPs and you will immediately see and understand the difference.

Software Sizing

We are trying to achieve CMMI Maturity Level (ML) 3 in my company and we have decided to skip ML 2. So, now, one of our problems is related to software size estimation. We defined a proprietary method, based on Use Case Points and Function Points, but the practitioners are struggling with it. From your experience, what other methods have you seen or implemented in the companies with this same problem? Or, if a proprietary method was defined, what were the main aspects to take in account?

I would strongly urge you to forget the ML 3 Process Areas until you have mastered ML 2. There is a fundamental difference between how a ML 2 Project Manager approaches Project Planning (PP) and Project Monitoring and Control (PMC) vs. a ML 3 Project Manager. Estimation being one of the differences. Use Case Points and Functions Points are fairly sophisticated concepts and there are challenges with getting consistency in determining what each of these things are. I would recommend that you take a step back from the model and the projects and look at your historical project data. Use the actual effort, costs, etc. from previous projects to estimate a new project. Forget about Use Case Points and Function Points for now. Once you have mastered being able to use historical information to build an empirical estimation model, then it might make sense to add a layer of sophistication by considering Use Case Points or Function Points.

Another recommendation is let the Project Manager create the project estimates and then review them with the practitioners as a sanity check rather than ask the practitioners to create the estimates. Over time as the organization gains experience estimating projects etc., then it makes sense to involve the practitioners up front in the estimation process. You have to learn to crawl first with estimation before you can sprint with the big boys.

Product Planning and Configuration

It is common knowledge that Maturity Level 2 is project specific, and still I find at times Lead Appraisers asking funny questions during SCAMPI A appraisals. Quite recently, one of my friends told me that his Lead Appraiser is looking for planning at the product level, as well as Configuratuion Management at the product level. I was a bit amazed, thinking that Maturity Level 2 focuses on Project Planning, not product planning. What do you think about this situation? Have you been faced with this situation before? Is there a workaround for it?

From what you describe, it sounds like this Lead Appraiser could be misinterpreting the CMMI and possibly misleading the organization. The CMMI is quite clear that the Project Planning (PP) Process Area (PA) is for project planning purposes, not product planning.

"The purpose of PP is to establish and maintain plans that define project activities."

However, sometimes the difference between project and product can be blurred. By not knowing the context of the situation you described, the Lead Appraiser may have been trying a different approach to draw project planning information out in the interview sessions.

In one respect, it really doesn’t matter the line of questioning in a SCAMPI interview session. The Lead Appraiser could really ask about any topic. However, once he or she starts deviating from the CMMI, they are on shaky ground and could lose credibility. What does matter however, is the set of findings produced by the Lead Appraiser and the Appraisal Team. If there are findings associated with product planning that cannot be tied to the satisfaction of a CMMI Specific Goal or a Specific Practice, then these would be non-model findings and should have no impact on the resulting appraisal rating. However, if these non-model findings do impact the appraisal rating and the Lead Appraiser and Appraisal Team fail to demonstrate the linkage to Goal and Practice satisfaction/implementation, then the Lead Appraiser has not correctly performed his or her Lead Appraiser duties and the SEI should be informed about this issue so it can be investigated.

Measurement and Analysis vs. Generic Practice 2.8

Would you expect that measures being reported for Generic Practice (GP) 2.8 on all Maturity Level (ML) 2 and ML 3 Process Areas (PAs) would follow the complete Measurement and Analysis (MA) PA? That is these GP 2.8 measures would have written objectives, operational definitions, storage, collection and analysis procedures in line with MA Specific Goal (SG) 1 and SG 2. Or can the PAs be monitored (measured) and controlled without the full breath of the MA PA?

This is a good question. So let’s take a step backwards and look at the CMMI and Generic Practice – PA Relationships. The summary table in the Generic Practice section of the model clearly states that Project Monitoring and Control (PMC) can implement GP 2.8 for all project-related processes. And MA provides general guidance about measuring, analyzing, and recording information that can be used in establishing measures for monitoring actual performance of the process. Please note that this information is GUIDANCE and part of the INFORMATIVE material. Therefore, it is not required that the org use MA for GP 2.8. HOWEVER, from a practical point of view, why would MA be part of the model if there wasn’t a requirement and expectation that it would be implemented? Since it is a ML 2 PA and you are asking about ML 3, as a Lead Appraiser I would expect to see that MA was used for defining, collecting, analyzing, and reporting both the project and process measures. Without implementing MA for the process measures, the org would be receiving little to no benefit from GP 2.8. And as I have seen GP 2.8 implemented, sometimes the process measures are embedded in the project measures that have been defined using MA.

CMMI Novice Question

I would like to know if all the Maturity Level 2 Process Areas must be completed for a Maturity Level 2 appraisal? There are some process areas which are not applicable to our organization.

For the CMMI for Development (CMMI-DEV) there is only one Process Area (PA) that can be designated Not Applicable (N/A), SAM. Therefore, at a minimum, REQM, PP, PMC, MA, PPQA, and CM are required for a Maturity Level 2 (ML 2) appraisal. And if the organization has outsourced some work, then SAM is also applicable.

I find it hard to believe that you can state that one or more of these PAs are not applicable to your organization. Every project has requirements to manage from the janitor to the President. Everyone works on a project. You just have to define what a project is. And then you manage the project. Everyone can define specific measures that can be used to manage the project. Everyone has some sort of configuration items or documents that have to be managed. And everyone needs some sort of objective evaluation of the process and project compliance. For a small organization, you may have combined one or more of these PAs under one person. But that does not mean these PAs are not applicable.

Now if you said that you had problems with the engineering PAs (RD, TS, PI, VER, and VAL), then I would suggest that the CMMI-DEV may not be the appropriate model constellation for your use and you should look at the CMMI for Services (CMMI-SVC) or CMMI for Acquisition (CMMI-ACQ).

Levying CMMI Requirements on Your Suppliers

I would like to know whether or not an acquirer can specify a CMMI Maturity Level (ML) as a requirement in a call for tenders or invitation to public bidding. What the ML is required? What is the domain ? and for what kind of projects?

The answer is yes. The acquirer can specify any requirements they want the vendors to meet. In my experience, I have seen acquirers specifying that the bidders be either at ML 2 or ML 3. What this means though is that the acquirer has done its homework and appropriately determined the necessary Maturity Level for the vendor to support the acquirer’s business and quality goals and objectives. There should be a good match between the ML of the acquirer and the ML of the vendor, to work well it may be best if both organizations are at the same ML. Otherwise, there can be problems.

As the acquirer, you probably would find benefit from implementing the CMMI for Acquisition (CMMI-ACQ). The CMMI-ACQ provides a lot of guidance for tenders and contracts that meet the acquirer’s needs.

Procedures Specify HOW to Do Something, Not WHAT to Do

A common problem I notice when reviewing an organization's processes and procedures is that the organization doesn't understand how to document a procedure. Rather than simply telling someone HOW to perform the process, the document states a set of requirements that must be met by the people performing the procedure, which then allows multiple ways of performing the procedure by any person attempting to use the document. There seems to be a general misunderstanding that commands or directives tell someone HOW to perform a process, when in actuality all that is being communicated is WHAT to do.

The CMMI Glossary defines process as activities that can be recognized as implementations of practices in a CMMI model. These activities can be mapped to one or more practices in CMMI process areas to allow a model to be useful for process improvement and process appraisal.

The Glossary also defines a process element as the fundamental unit of a process. A process can be defined in terms of sub-processes and/or process elements. A sub-process can be further decomposed into sub-processes and/or process elements; a process element cannot. Each process element covers a closely related set of activities (e.g. estimating element, peer review element). Process elements can be portrayed using templates to be completed, abstractions to be refined, or descriptions to be modified or used. A process element can be an activity or task.

Page 53 of the CMMI book provides the following description of Maturity Level 2:

At Maturity Level 2, the projects of the organization have ensured that processes are planned and executed in accordance with policy; the projects employ skilled people who have adequate resources to produce controlled outputs; involve relevant stakeholders; are monitored, controlled, and reviewed; and are evaluated for adherence to their process descriptions. The process discipline reflected by Maturity Level 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.

At Maturity Level 2, the status of work products and the delivery of services are visible to management at defined points (e.g., at major milestones and at the completion of major tasks). Commitments are established among relevant stakeholders and are revised as needed. Work products are appropriately controlled. The work products and services satisfy their specified process descriptions, standards, and procedures.

Page 54 of the CMMI book provides the following description of Maturity Level 3:

At Maturity Level 3, processes are well characterized and understood, and are described in standards, procedures, tools, and methods. The organization’s set of standard processes, which is the basis for Maturity Level 3, is established and improved over time. These standard processes are used to establish consistency across the organization. Projects establish their defined processes by tailoring the organization’s set of standard processes according to tailoring guidelines.

A critical distinction between Maturity Levels 2 and 3 is the scope of standards, process descriptions, and procedures. At Maturity Level 2, the standards, process descriptions, and procedures may be quite different in each specific instance of the process (e.g., on a particular project). At Maturity Level 3, the standards, process descriptions, and procedures for a project are tailored from the organization’s set of standard processes to suit a particular project or organizational unit and therefore are more consistent, except for differences allowed by the tailoring guidelines.

Another critical distinction is that at Maturity Level 3, processes are typically described more rigorously than at Maturity Level 2. A defined process clearly states the purpose, inputs, entry criteria, activities, roles, measures, verification steps, outputs, and exit criteria. At Maturity Level 3, processes are managed more proactively using an understanding of the interrelationships of the process activities and detailed measures of the process, its work products, and its services.

Page 295 in Organizational Process Definition provides some Tips that relate to processes and process assets:

Organizational process assets support a fundamental change in behavior. Projects no longer create their processes from scratch but instead use the best practices of the organization, thus improving quality and saving time and money.

Standard processes define the key activities performed in an organization. Some examples of standard processes include requirements elicitation, design, and testing; planning, estimating, monitoring, and control; and product delivery and support.

The objective is to decompose and define the process so that it can be performed consistently across projects but will allow enough flexibility to meet the unique requirements of each project.

And from the book Interpreting the CMMI by Margaret Kulpa and Kent Johnson, second edition on page 198 there is a section on Defining Procedures that supports the CMMI material quoted above. In the context of the CMMI, procedures are equivalent to sub-processes and process elements.

Procedures are step-by-step instructions on how to perform a task. To be repeatable, the steps need to be broken down to a level that anyone who needs to perform the task, with a general understanding of the work to be done, can perform the work adequately by following the instructions. Procedures are a subset of processes. The process is what to do; the procedures are how to do the steps of the process.

Procedures are step-by-step instructions of how your processes are performed. They include:

• Sequence of activities
• Deliverables
• Controls
• Inspections and reviews
• Guidelines and standards used

In addition, in the CMMI book on pages 99 and 100, Bill Curtis, an author of the Software CMM, talks about policies, PPQA, and process improvement:

Polices that merely regurgitate goals from CMMI process areas represent a lost opportunity for executives to communicate their expectations for behavior in their organizations. Once policies are established, executives need visibility into compliance. Assurance groups have influence only to the extent that executives attend to their reports and address noncompliance. However, the greatest value of assurance groups, and this is subtle in Process and Product Quality Assurance (PPQA), is when they serve as mentors to project managers and technical staff on practices that support compliance. Consequently, assurance groups need to be staffed with competent developers and managers so that they are credible in transferring knowledge of best practices across the organization.

Process improvement must be conducted as a project. Executives must assign responsibility for managing the project, provide funding and resources, expect periodic status reports, and measure results. The person assigned to lead the improvement project must be a good role model for other project managers. Executives should ask frequent questions about project plans and the assumptions underlying them. The guidebooks, defined processes, measures, checklists, and other artifacts produced through process improvements are organizational assets. They should be treated as products, albeit for internal use, and be produced with the same discipline used in producing any other product.

Consequences and Summary
By not specifying how to perform the process steps it gives carte blanche to whoever is performing the process to do whatever they want to in order to satisfy the step. In addition, by not specifying how to perform the process steps, it makes it impossible to perform a PPQA process audit. The person performing the objective evaluation of the process (process audit) needs to know how the process is supposed to be performed in order to create a checklist so he or she can objectively evaluate whether or not the process is being followed the way it is supposed to be followed.

Essentially, by not supplying the information of how to perform a process, you do not have a repeatable process, which is at the core of Maturity Level 2. A repeatable process means that every time someone uses the same documented process description, he or she will perform it the same way. The only way that you can ensure this outcome is to document how the process is performed. At Maturity Level 2 it is permissible to have multiple documented processes for the same process. At Maturity Level 3 the organization is supposed to examine the multiple documented processes and look for exemplar practices to elevate to the organization level to become the standard process for the organization.

The challenge facing the process writers is to document the process such that it is neither too detailed nor too general to follow. If it is too detailed, then you have painted yourself into a corner and the process may be too detailed to correctly follow. If too general, then it basically allows you to do anything you want. Either way, there is no benefit to the organization.

This issue also points out a fundamental difference between ISO 9000 and the CMMI. At its core, ISO 9000 is a set of standards for a quality system. It requires the existence of processes good, bad, or indifferent. In contrast, the CMMI is a set of guidelines for process integration and product improvement. The whole focus of the CMMI is process improvement. And it becomes very difficult to do process improvement if you don’t spell out how a process is performed.

To illustrate the issue allow me to cite an example of incorrectly worded procedure step.

• The Project Manager derives estimates for the size of the software work products, or changes to the size of the software work products.

There is more than one way to estimate size if you don’t specify how to perform this step. By not specifying how, you cripple your ability to analyze the process and identify process improvements.

If I were to ask the Project Manager how he or she peforms this step, I might find out that they do this by using the project requirements and historical data (notes and reports from past projects and lessons learned) from similar past projects to determine size factors (SLOCs, number of files, database size, number of screens) and applying appropriate scaling factors using engineering judgment. The Project Manager then reviews the size estimates with the Team Leads for reasonableness. This is an excellent explanation and practice.

By providing this level of detail, now there is enough information for PPQA to perform a process audit and also enough information to perform process improvement and determine if an estimation model can be built to improve the accuracy of project estimates.

MA and PPQA Questions

I have the following two basic queries about CMMI ML 2:

1. While writing a Metrics and measurement process, should we address the organization level metrics data consolidation and review. As ML 2 is project specific, is it proper to also document the organization level data consolidation? Also can anyone tell me, the right site for definition of metrics like requirement stability index, schedule variance, effort variance etc.?
   
2. Similarly while documenting PPQA process, is it proper to start with defining an organization level PPQA plan? I am looking for boundaries where to limit writing processes compliant to ML 2. I know that G.P 2.1 to G.P 2.10 must be in place to achieve CMMI ML 2, but the organization specific plans/areas must not be mentioned/documented at CMMI ML 2.

You sound like you are focusing on CMMI compliance rather than on your business goals and objectives. One of the basic tenets of the model is your business objectives. That is where your focus belongs. And if done properly, you will have the side benefit of being CMMI compliant. So, to address your questions:

1. When documenting your Measurement and Analysis process, you should focus on those measures that are important to you. Remember, the first MA practice SP 1.1 states “Establish and maintain measurement objectives that are derived from identified information needs and objectives.” So whatever you have identified as information needs and objectives, that should be your MA focus. At ML 2, for many organizations that are just doing this for the first time, I recommend the org take baby steps and begin with a project focus. But you don’t have to be restricted to the project, an ML 2 org may have also identified some org level measures as well. Go to the Practical Software and Systems Measurement web site for the specific measurement information you need www.psmsc.com
   
2. There are NO CMMI-imposed restrictions on the limits of PPQA. Your organization must define its own limits for the processes you are going to audit. Since GP 2.9 applies to all Process Areas, at a minimum for ML 2, PPQA applies to all of the ML 2 Process Areas you have implemented in your organization. But, if there are other processes that are critical and/or important to the success of your business, then it makes perfect sense to have PPQA audit them as well. Again, do what is right for your business.

Evaluating the CMMI for Services

I'm currently involved in an evaluation of different types of assessments for the area of IT Service Management (ITSM) and therefore challenging the CMMI for Services 1.2 (CMMI-SVC).

A major problem I see in CMMI-SVC, is the disregarded sufficient differentiation of practices over different maturity levels. Maybe I´m wrong with my opinion and someone can help me fixing this mess.

I´d like to give a background for a better understanding o f what I mean:

1. In my opinion many (internal) IT service units in mid-size companies don´t have any official agreements (nor contracts) with their business customer to specify service content, service levels or support. Sometimes these things are partly available and if at all, then often are informally and silently accepted.
2. In such (above) described IT service units you will find quite often an official HelpDesk (mostly official because it´s a specific function within IT) or minimum some guys necessary for service support (like handling service incidents) to keep the business process running.

Now take a look at the CMMI-SVC Process Areas and their associated Maturity Levels. You will find Service Development (SD) at Maturity Level 2 including specific practices:

SP 1.1 Analyze Existing Agreements and Service Data
SP 1.2 Establish the Service Agreement
SP 2.1 Establish the Service Delivery Approach
SP 2.2 Prepare for Service System Operations
SP 2.3 Establish a Request Management System
SP 3.1 Receive and Process Service Requests
SP 3.2 Operate the Service System
SP 3.3 Maintain the Service System

and you will find Incident Resolution and Prevention (IRP) at maturity level 3 including specific practices:

SP 1.1 Establish an Approach to Incident Resolution and Prevention
SP 1.2 Establish an Incident Management System
SP 2.1 Identify and Record Incidents
SP 2.2 Analyze Incident Data
SP 2.3 Apply Workarounds to Selected Incidents
SP 2.4 Address Underlying Causes of Selected Incidents
SP 2.5 Monitor the Status of Incidents to Closure
SP 2.6 Communicate the Status of Incidents
SP 3.1 Analyze Selected Incident Data
SP 3.2 Plan Actions to Address Underlying Causes of Selected Incidents
SP 3.3 Establish Workarounds for Selected Incidents

To come to an end I would expect that it is an essential part of any IT unit to solve service incidents, to fulfill the main goal of the company, and keep the business process working. Therefore I would assign half of the listed IRP practices to Maturity Level 2 and would other way around assign half of the SD practices to Maturity Level 3. The CMMI-SVC therefore seems for me to be not sufficient in differentiating practices over Maturity Levels, and would lead to the conclusion that CMMI-SVC is not useful for Maturity Level determination.

I appreciate any explanation if there is a misunderstandig or if there exists a grain of truth.

Your position is much the same as the kind of statements we heard regarding the engineering practices being at ML 3 in the CMMI-DEV. Just because these practices are at ML 3 does not mean that they are not important and are probably even performed at ML 1. What you have to bear in mind is that the CMMI is a set of process improvement guidelines , as well as the definition and purpose of ML 2 and ML 3. At ML 2, projects establish the foundation for an organization to become an effective service provider by institutionalizing basic project management and service establishment and delivery practices. Basically, ML 2 is about gaining control over the projects and service delivery and that is why there is only one service PA at ML 2. You have to get delivery under control before you can focus improving the other aspects of services like Incident Resolution and Prevention. And at ML 3, service providers use defined processes for managing projects. They embed tenets of project management and services best practices, such as service continuity and incident resolution and prevention, into the standard process set.

Query on Process Change Management

Processes are meant for Doing things Right, First time, Every time. This is one of the core concepts I have seen people referring to during process formulation exercises. I presume the whole idea behind this is to make an organisation process driven instead of people driven.In my limited experience, (esp. in small to medium size organisations) I have seen new unit/department heads within a short span of time after their recruitment, devise a series of Continuous Improvement Requests (CIRs) and posted it to the EPG, who on most occasions weigh the CIRs against department and compliance goals, and if they don't see any adverse effect due to these changes, they approve it.One of the premise for approving the changes is to promote positive initiatives in the organisation.

Now, my questions are:

1. Will adopting the practices prescribed by a new department head, which is based on his experiences, affect his subordinates' work( who are the actual practitioners) in an adverse manner?
2. Will organic growth of existing practices (due to collective experience of the practitioners) be ruined due to implementing new practices?
3. Will allowing practice changes from new department heads make an organisation fall behind from process to people driven? (even though this may last until the organisation get accustomed to the new practices) And is this permissible, if we consider the organisation's overall development?

First off, the impacts depend upon the organization’s Maturity Level. A Maturity Level 2 organization doesn’t necessarily have standard procedures for all projects to follow, though I have seen many ML 2 organizations take this approach. Therefore at Maturity Level 2 you can have multiple ways of doing the same thing, from project to project and from manager to manager.

When the organization matures to Maturity Level 3, the premise is that the organization has examined the multiple ways of performing a given practice and determined the Best Practice for the organization and then documents these Best Practices as the set of standard processes for the organization. This examination, coordination, and distribution of the standard processes is typically the responsibility of the Process Group. The Process Group manages the processes and is responsible for coordinating all process changes.

Now having said this, a new department head can make any process changes he or she wants to make. At ML 2 these changes could provide a Best Practice for consideration or additional information on things not to do. However, at ML 3 the organization should have established OPF and OPD processes for making process changes. The new department head would have to follow these change processes to propose his new processes. The Process Group would evaluate his proposals and pilot them as appropriate so the changes could be evaluated in a controlled manner. The outcome of the pilot(s) would determine whether or not the changes are made.

By following these steps, there shouldn’t be any of the problems you allude to in your note. However, if your organization does not have processes in place for making changes to the organization’s processes or the new department head mandates process changes without following the process, then you do have some serious issues to address.

1. Making uncontrolled process changes will impact the practitioners, most likely adversely. People most likely will be frustrated because of the changes.
   
2. Replacing existing procedures with new ones in an uncontrolled manner will adversely disrupt any process improvements and process evolution you have already made.
   
3. Making uncontrolled process changes can cause the organization to regress in Maturity Levels, probably drop from whatever ML you are currently at to ML 1.

Full Time Resources for Implementing the CMMI

I have been asked to estimate the number of full time resources required by my company to facilitate its drive to Maturity Level 3 PM, SYS, SW, HW and ACQ. Is there any documentation or published information that will help in putting together a robust estimate of resources required?

First and foremost the driving factor for estimating the number of full time resources needed to implement the CMMI is the size of the organization. Over time we have seen that it takes 3 – 5% of the organization to perform PPQA, 3- 5 % of the organization to perform CM, and 3 – 5% of the organization to perform the necessary CMMI implementation activities.

So, if your organization is about 20 – 30 people, then you may only need one full time resource. However, if your organization is about 100 people, then you may need 3 to 5 full time resources.

Other factors contributing to this estimate is how strong a ML 2 foundation is already in place and how much of what you currently have in place is at ML 3. Documenting the processes and procedures is the easy part, and it can be done by a small core group. The larger task is deploying the new processes and process assets and having people use them to change how they approach their jobs.