PPQA Audits

Would you please distinguish the different types of audits 1) Projects, 2) Process and 3) products? Does PPQA audit the Project, Process, or Product? Or all the three? And from which area do we need to collect improvements, 1, 2, or 3? I'm confused, can you help?

You say that you are confused. I Let me try to provide an explanation for what I think you are asking about PPQA. The intent of PPQA is to act as the eyes and ears of senior management to ensure that the practitioners are following the documented processes to produce the work products. So PPQA performs two types of audits: process audits and work product audits. Now the processes being audited can be at the individual level, project level, or the organization level. And the processes being audited are not restricted to the CMMI Process Areas. The organization has to determine which processes to audit based on its business goals and objectives, so there may be processes audited in addition to the processes covered by the CMMI.

A process audit is conducted by first studying the documented process and then interviewing the practitioners to determine if they are following the process as documented.

Each process has one or more work products that are produced by following the process. These work products can be at the individual, project, or organizational level as well. The work products can be audited by sitting at a desk and reviewing the work product against the documented requirements for the work product. Is the work product produced correctly? Does it contain the proper level of information? Etc.

Both process and work product audits will identify non-compliances. By analyzing the non-compliance issues, PPQA should be able to identify the underlying causes for the issues and recommend one or more process improvement suggestions.

Reviewing Process Status with Senior Management

When a manager reviews the work products of a process, is it not effectively the same as reviewing the process? The model can't expect status reports to senior management to have a list of 18 or 22 process areas to be reviewed with senior management on a regular basis. Project managers I work with could see no possible value in this and their managers/supervisors won't listen to it. And it's not because they don't want to support process improvement.

It sounds like you are having difficulty understanding the intent of GP 2.10 “Review the activities, status, and results of the process with higher level management and resolve issues.”

First off, I need to ask you why is a manager reviewing the process work products? The model doesn’t require that managers review process work products. However, it sounds like your processes may have this requirement.

GP 2.9 “Objectively evaluate adherence of the process against its process description, standards, and procedures, and address noncompliance” does call for objectively evaluating selected work products, but this is a PPQA responsibility. And the intent is to determine if the work product meets the requirements of your documented processes and applicable standards.
There is a huge difference between reviewing a process work product and reviewing the process. Just because a work product exists does not mean that the documented process was followed for producing the work product. For example, your design process states that you document the customer requirements first, then you derive the functional requirements from the customer requirements and create a functional requirements spec. Then you create a design based on the functional requirements and write a design specification. So you would expect to find these three documents in the project folder at the end of the project.

Scenario 1) The project follows the documented procedures and produces the documents in the order described and per the procedures.
Scenario 2) The project doesn’t follow the documented procedures and launches immediately into design and when preparing to close out the project, they rapidly create these three documents .

The work products exist for both scenarios, but they have value in Scenario 1 and no added no value in Scenario 2 and could be viewed as a waste of time to create. In addition for Scenario 2, the product documentation was written per the design, when in fact the product was supposed to be designed per the written documentation.

So, simply reviewing a document will NOT be the same as reviewing the process.

Now to get back to GP 2.10, the intent of this GP is to provide higher management with appropriate visibility into the process. The reviews called for by GP 2.10 are for managers who provide the policy and overall guidance for the process, not for those who perform the direct day-to-day monitoring and controlling of the process (which is the intent of GP 2.8). The intent of GP 2.10 is review the process results, which is typically an aggregate of the results across projects, not the detailed process results on each project. And yes, the expectation is that higher management reviews the process results for ALL Process Areas (PAs) in scope; 7 PAs for Maturity Level 2, 18 PAs for Maturity Level 3, 20 PAs for Maturity Level 4, and 22 PAs for Maturity Level 5. Why else would this be a Generic Practice? But keep in mind how you have implemented these PAs. Most likely your implementation has combined some PAs into a single process and you may also have multiple processes for a single PA. The intent of GP 2.10 is to review your processes. When you prepare for an appraisal you simply have to demonstrate that you have covered each PA with your process reviews.

From a practical standpoint, it is tedious to review the complete set of PAs in scope at a single setting. Especially those processes that do not occur often and there may be nothing new to review. It also depends on the frequency of your process reviews. The model does not specify a frequency, it has to be set by the organization. You can also have these reviews be event-driven. One approach that I have seen work in several clients is to hold quarterly process review meetings with senior management and only review a subset of the PAs at each review. There is an overall plan for reviewing the PAs, for example it may take a year to complete the PA set, but each quarterly review only examines ¼ of the PAs. That eases the burden on everyone. Then if a process issue arises and that PA is not due to be reviewed for 9 months, a special event-driven review for that PA is held.

Management Commitment

I am continually struck by the misunderstanding of management's role in process improvement in any size organization. Many times the all the senior manager feels that they have to do is give a directive to achieve a Maturity Level by some date. And that is the extent of their involvement, expecting that the Maturity Level will simply happen because they said so. And then to make matters worse, this same senior manager does not understand why the people they put in charge of implementing the process improvement program can't make things happen faster, especially when they are perceived as setting capricious and arbitrary deadlines without asking the process team if the new dates are achievable.

Managers who are committed to correctly performing process improvement both "talk the talk" and "walk the walk". They understand the importance of:

1. setting realistic process improvement goals
2. providing the necessary support to the process team
3. addressing process improvement challenges and removing obstacles
4. instilling the process improvement mindset throughout the management structure
5. encouraging process improvement suggestions
6. being a process improvement advocate

Without being actively supportive of their process improvement goals and emphasizing this same expected behavior from the rest of the management, their process improvement goals are at serious risk. Management does have to become engaged and not simply expect that things will happen just because they spoke.