PI SP 3.1 Confirm Readiness of Product Components for Integration

I need help with understanding this practice. Here is the situation: In our organization we have implemented MS Team System. This tool allows us to analyze the code from different perspectives. We have implemented peer reviews. The code reviews allow us to verify if the complies with the design specification. We have also implemented CM audits to check the identification of every configuration item. Consequently, I´m not certain we are fully aligned with this practice.

The purpose of PI SP 3.1 is to ensure that all of the components that you will be assembling are ready for assembly. For purely a software project, this practice is pretty easy and straightforward. At a minimum, you want to be certain that every module has been properly checked into your CM system, that every configuration unit has been unit tested, and that the external and internal interfaces have been examined to verify that they comply with the documented interface descriptions. It sounds like you might have most of these activities covered by MS Team System and your peer reviews. What I don’t see in your description is any activity associated with checking the interfaces against their descriptions. When you are integrating hardware and software, or have a large and complex software project with many different systems, this practice becomes more complicated.
Hope this short explanation helps.

I have one more question. Should we run unit tests for every configuration unit? Is it possible to implement actions other than unit testing to comply with this best practice? I think that the Static Code Analysis in VS Team System checks the interfaces betwen components. In the peer reviews of code we check the interfaces against their descriptions documented in design specifications.

You are actually focusing on the wrong topic. Instead you should be seeking answers to these types of questions.

1. What do your business goals and objectives tell you about the required quality level of products?
2. What is the reason for performing unit tests? Or what are you trying to achieve by unit testing the code?
3. Do your customer requirements and your business goals and objectives require a quality level that demands that you perform unit tests before creating a product build?
4. What are your requirements for each configuration item before creating a build?

Answers to these questions will provide the answers to your questions.

Basically, your configuration audits are there in order for you to determine if all of the configuration items are ready to be assembled. Perhaps the Static Code Analysis in VS Team System is satisfactory, perhaps it is not. That is for you to decide based on the quality requirements for your product.

Hope this explanation helps, but there is no clear answer to your question without being able to spend some time with you and your organization to perform an in-depth analysis of your processes and procedures.

Traceability in Pure Testing Projects

I have a question about addressing requirements traceability for pure testing projects (Understanding requirements->Writing Manual Test cases->executing them). If the application is not developed by us, what information other than Module name, Requirement ID, description, and Manual Test Case ID needs to be mapped?

There is no definitive answer to your question. The actual answer is up to you and your organization to decide what is necessary for your traceability. What does each testing project need to know about traceability? If you can answer that question, then you have the answer to your question as well. What you have listed sounds reasonable, but only you can determine if it is complete or that you need to add other elements.

Review Activity for a Short Term Project

Our organization will be going through CMMI Maturity Level 2 Appraisal in a couple of months. I have a PPQA question. As per the PPQA Process Area (PA), we require a review of the work products (content/template) and procedures required at Maturity Level2 during the project life cycle. We have one project that is 3 months long. There are many work products that will be produced during the project development life cycle.

• Requirement documents such as SRS, Use cases, Bidirectionally traceability matrix document, change log, etc;
• Plans for all the PAs, e.g. requirements management plan, project plan, configuration plan, etc;
• Development artifacts, such as ERD, Code, UML diagrams, etc;
• QC artifacts, such as test cases, test reports, etc.
• Monitoring/controlling artifacts, such as Issue list, MoMs, Risks, etc.

How is it possible to review the work products for a 3 month project when we don't have a separate QA department and the stakeholders involved in development do the work product reviews one way or the other.

This same question holds true for reviewing procedures.

Of course, we review high priority documents, such as Project Plan, Use Cases, ERD, Application; but not all of them.

Can you help me understand what should be done for a short duration project, such that the PPQA PA requirements are met and we don't have to hire separate people just to fulfill the requirement?

The first thing that I would do is postpone your ML 2 SCAMPI A appraisal as apparently you have a major risk to achieving ML 2 since PPQA does not appear to be in place in your organization. And even if you could put PPQA in place for a 3 month project between now and your appraisals, that may still not be enough time to demonstrate institutionalization, meaning that you have a repeatable process. Essentially you will have one project using PPQA, which is one data point. And it is not possible to determine institutionalization from one data point. Your organization will be at serious risk of not achieving ML 2.

Industry average shows that PPQA is 3 – 5% of your organization. You haven’t told me how large your organization is. But if your organization is 25 people, than 1 person should be assigned to perform the PPQA practices.

I think that you are misunderstanding the differences between reviewing a work product and objectively evaluating a work product. It sounds like your project teams are already reviewing the work products. The role of PPQA is not to review the work products, but to audit the work products and processes to ensure that the work products follow the specific standards and are products according to your documented processes.

I highly recommend that you, or someone you select in your organization, take a training class on how to perform PPQA. I cannot adequately explain how to perform PPQA and answer your specific questions in this blog. The person you select for the training needs to be taught how to conduct a work product audit, how to conduct a process audit, how to plan PPQA audits, how to communicate audit results, and how to track audit non-compliances to resolution. If you don’t already have this capability in house, it will take some time to develop it internally. And I strongly advise against using an external consultant to provide this service. PPQA is for the benefit of your organization and management. It is essentially the eyes and ears of your senior management. And an external consultant may be motivated by other considerations than your best business interests if asked to provide PPQA services.

Why Isn't the SEI Implementing the CMMI for Itself?

Why doesn't the SEI use its own model- CMMI for all its different product development and services? Even for SEI projects and program management it is crucial, and they have customers the world over. If the SEI goes for CMMI ML3 Appraisal it will be great for the user community and they can achieve their mission in a planned manner, right?

Do Lead Appraisers & SEI Partners feel that they can benefit if the SEI gets CMMI ML3 (defined Process)?

In such a case, who will appraise the SEI? (sorry for such a hypothetical Question)

As the SEI does not develop software, but delivers services, the CMMI-DEV doesn’t apply. That is why the SEI has not been previously appraised to the CMMI. However, the SEI is now implementing the CMMI-SVC for the services it delivers. This is a good thing and the SEI Partners are noticing some of the improvements. Obviously, by the SEI’s Conflict of Interest policy, a CMMI-SVC Lead Appraiser external to the SEI organization being appraised would have to lead the appraisal team.

Appraisals: Practice or Subpractice level?

For successful SCAMPI appraisals, is there any reason to prepare process compliance at the sub-practice level? Would appraisers be looking for evidence at that level?

This is a question answered by taking the 3-day Introduction to CMMI class and also by your Lead Appraiser. There are three CMMI components: Required, Expected, and Informative. An appraisal only covers the Required (Goals) and Expected (Practices) components. Your Lead Appraiser should also be providing some training or guidance on how to build the PIIDs, which contain the objective evidence for an appraisal. And the whole appraisal team is involved in reviewing the PIIDs during the Readiness Review to determine if the evidence is proper for a SCAMPI appraisal.

If a Lead Appraiser or the appraisal team is appraising you to the sub-practice level, they have gone too far. The SCAMPI method is only concerned with appraising the organization to the Goals and Practices.

How Do We Select a High Maturity Consultant?

My friend is a Quality Manager in a company who has reached CMMI Maturity Level 3. They now want to achieve Maturity Level 5. They started taking quotes from different companies. In the selection process they found that there are 3 or 4 major players in our country who have up to 3 High Maturity Lead Appraisers. Most of these companies have submitted proposals for consulting and appraisal in a single quote. Now my friend fears that:

1. Most of them already have at least 40 High Maturity clients and at least 30 Maturity Level 3 clients. Will they have the capacity do lead the appraisal on time for my friend's company ( considering 12 SCAMPI appraisals per year per LA) ?
2. Most of them deliver the consulting and training activities , which is 70 % of the contract value and sometimes they break the contracts and not deliver the SCAMPI, which is still highly profitable, since only 30 % value is lost, and no need to deal with High Maturity appraisal needs.

In order to address this issue, does the SEI publish a list of contracts for SEI Partner? Does the SEI have a specific committee or group to look in to the capacity management and availability management of their SEI partners, so that companies will not have such concerns?

All that the SEI does is maintain a list of SEI Partners and certified High Maturity Lead Appraisers. If there is indeed a problem as you have stated, then you or your friend should contact the SEI about the SEI Partner in question as this certainly sounds like unethical behavior.

Another issue is that an SEI-certified High Maturity Lead Appraiser cannot appraise the organization if he or she has provided the consulting to the organization, unless the SEI approves the potential Conflict of Interest.

The steps taken by SEI in this area are impressive. Also I understand that the control on appraisals/per year is established by SEI. Out of curiosity i would like to ask follow up questions.

I belive the critical part of this entire process is that consulting and apprisal services cannot be performed by the same Lead appriser. If there is a need of separate contract for SCAMPI A appraisal activities, and it cannot be included in a consulting contract, then it can have more credibility. Also like ISO where the certification agencies are audited annually (correct me if am wrong), will SEI do an onsite audit on SEI partners? Or do they have a databse of all the contracts established by SEI partners around the world (considering 800 to 1000 appraisals per year)? Because the user community trusts the SEI more than the SEI Partners ( for most of the users it may be the first time to contract with an SEI Partner and they might not be sure of the guidelines provided by the SEI or about the SEI Ethics commitee). All this can be prevented if the SEI takes a copy of all contracts established for SCAMPI A across the countries. What are your views on it? .

Note:The intent of the question is to increase the user communities' trust on SEI to increase, but not to reduce the credibility of SEI partners/Lead Appraisers.

The SEI does not have the time or resources to perform annual on-site audits of the SEI Partners. And as an SEI Partner, I would not welcome an on-site audit by the SEI. It would be additional expense for me.

What the SEI does provide that may help with your concerns is that they perform a QA audit of the results of every appraisal submitted by a Lead Appraiser. If the appraisal results do not meet the evaluation criteria, then a more in-depth audit occurs. What can then happen is that if the problems are serious enough, the Lead Appraiser can lose his or her CMMI credentials. This has happened to a number of Lead Appraisers since this policy was put in place.

In addition, each Lead Appraiser must be certified by the SEI, which provides another layer of credibility.

The SEI Partners provide the service and the certified-Lead Appraisers deliver the service. As a buyer of CMMI services, it is your responsibility to learn about the SEI policies regarding Ethics and Conflict of Interest, as well as the credibility of the different SEI Partners and Lead Appraisers. Otherwise, you get what you pay for. In other words, buyer beware!

And depending upon what country you are in, the SEI Partners are trusted as much or more than the SEI by the user community.

Achieving a Maturity Level Without a Consultant or Training

I was wondering if it is possible to go it alone with CMMI Maturity Level 2. I have been told by many that attempting CMMI Maturity Level 2 without a consultant or highly trained staff would be somewhat challenging. What are your thoughts?

Also, having limited resources for process improvement being a strong possibility, do you have any recommendations for online sources that can help offset the costs?

Though it is entirely possible to implement the CMMI without hiring a CMMI consultant, that is a high risk approach. Though I don’t see how you could avoid taking the Introduction to CMMI training class. That, in my opinion, would be a huge mistake. There are some areas in the CMMI that are open to interpretation when trying to go it alone and you can end up doing things in the spirit of achieving Maturity Level 2 that have no business value to you. In addition, since your organization has to provide 4 to 8 appraisal team members, each appraisal team member must take the SEI’s Introduction to CMMI class.

These are all worthwhile expenditures of your process improvement budget. And when you compare these expenses to your internal costs for process improvement, these are usually negligible in comparison.

But if you are trying to do things as cheaply as possible, that begs the question, why are you even considering implementing the CMMI and trying to achieve Maturity Level 2? Basically you get what you pay for. Going the cheap route doesn’t demonstrate management commitment to process improvement and can result in wasted effort, wasted money, and an aborted process improvement initiative.