SCAMPI A Appraisal Questions for FAR Groups

Can you share the Questionnaires for a ML 3 SCAMPI A appraisal? Currently my organization is preparing for SCAMPI A and I want to trained the interviewees as this is their first experience an appraisal.

I am looking for some questions based on each Process Area, so the FAR Group Members will be prepared to answer them.

I have searched on Google but not able to find any information. :(

I am not surprised that you have not been able to find what you are looking for on the internet. A Lead Appraiser will not share his or her questions with you. Simply because the Lead Appraiser does not want the interviewees to be trained on what answers to provide, thereby biasing the results of any SCAMPI appraisal. If you and the organization are nervous about the appraisal, don’t be. There is no preparation for the interviews necessary. All that is expected by the Lead Appraiser is that any interviewee should be able to talk about HOW they perform their job duties. If people cannot do that, then the organization is not ready for an appraisal.

If you still feel uneasy, talk to your Lead Appraiser and have him or her conduct a Class B appraisal with interviews. A Class B or SCAMPI B appraisal can function as a dress rehearsal for a SCAMPI A without having the organization and the people worry about “flunking” or “passing” the appraisal. The interviewees will have the experience of being interviewed.

If you still want to perform an internal set of interviews, simply take the CMMI and step through the applicable practices with different groups of people (project managers, configuration managers, developers, testers, etc.) and ask them HOW they perform the Specific and Generic Practices. There is no set of standard questions. Each Lead Appraiser has their own style of questioning interviewees.

REQM SP 1.2 Question

Can you please help me?

I am developing a study based on CMMI-DEV and I am having some difficulty with the interpretation of Specific Practice 1.2. My concern is how to apply SP 1.2 in practice. Who are the members of the project that must be committed to the requirements (business analysts, systems analysts, test analysts, developers, systems architects, project manager, and project leader)?

How should their commitment be documented? For example, for each new or changed requirement the members must sign a document that means that they are aware of the requirement and are committed to it.

Do you have a copy of the Addison Wesley published CMMI-DEV version 1.2? If so, there are some very helpful tips in the margins for REQM SP 1.2.

When you have a set of requirements for a project team, everyone on the team impacted by the requirements needs to share a common understanding of the requirements. In addition, since you typically do not have the full set of requirements available at project start, the project team needs to continually evaluate and reevaluate their ability to meet the requirements as the requirements evolve over the life of the project. One way of evaluating requirements is an impact assessment on the existing requirements, design, documents, test cases, current and downstream tasks and activities, cost, schedule, etc. for a new or changed requirement. By going through an impact analysis, that will provide a way for the project participants to communicate their ability to meet the requirements and associated commitments. The resulting impact analysis report can be one way of documenting the commitment. Other methods for documenting commitments include meeting minutes, document signatures, or email. However, just signing a document indicating that the participants are aware of a requirement really does not indicate that they are committed to the requirements and you might run into difficulties later on in the lifecycle.

Please keep in mind that commitments include both the resources involved (people, tools, and facilities) and the schedule for completion.

Covering SAM SP2.2 and SP2.3

I want to ask you about CMMI Supplier Agreement Management (SAM) SP 2.2 Monitor Selected Supplier Processes and SP 2.3 Evaluate Selected Supplier Work Products. I have difficulty in understanding these practices and how to apply them in my company. We have a project that needs to buy new hardware and deliver it to customer.

How can I select and monitor supplier processes in this case?

Can you please explain that to me and give me an example of a supplier process that is applicable in our situation?

The best way to answer this question is to attend the SEI’s Introduction to CMMI 3-day class. During the SAM topic the instructor will explain the difference between these two practices for you. I would also suggest that you read the informative material in the SAM Process Area, for not just these two practices, but all of the Process Areas. The informative material is extremely rich in information, hints, tips, etc. and it provides guidance for understanding the intent of each practice and goal. I would also recommend that you not use the goal and practice titles for anything other than labels. The titles are shortened statements and do not always communicate the correct meaning of the goal or practice.

If you read the informative material for SAM SP 2.2 and SP 2.3 it will be immediately clear to you the differences. And if you still have questions regarding your specific circumstances, I suggest that you ask your SEI-certified Lead Appraiser. Without having a good understanding of your company and how it conducts business, asking specific implementation questions will most likely not provide the correct advice to you.

However, here is a very brief description of the differences between these two practices.

SAM SP 2.2 “Select, monitor, and analyze processes used by the supplier.” You as the acquirer of the supplier’s products or services must define in the supplier’s contract or agreement those specific supplier processes that are critical to your success. You have to state how and when you will be monitoring these critical processes. This practice is done as a risk mitigation to avoid surprises at the end of the contract when the supplier delivers the end product or service. Think of this practice as the acquirer performing PPQA process audits of the supplier’s processes.

SAM SP 2.3 “Select and evaluate work products from the supplier of custom-made products.” If your supplier is only delivering off-the-shelf product to you without any customization, this practice does not apply. However, if your supplier is providing custom-made or modifications to your specifications, then you need to decide which of these products are critical to your success and define those in the supplier’s contract or agreement. You have to state how and when you will be evaluating the products. Think of this practice as the acquirer performing PPQA work product audits of the supplier’s work products.

CMMI Practices for Documentation Teams

Our organization is CMMI Maturity Level 3 Ver 1.2 certified. Our delivery teams are going to implement CMMI practices soon. I did see the processes of the organization and though all the roles appeared from project leader to developer to manager, except for documentation teams. In the same context, I am very curious to see if there are any set of practices to be followed for the documentation department in CMMI as all the processes at CMMI ML2, ML3 are specific to project management, engineering, support, organization areas.

Your question actually raises some other questions:

1. What are the roles and responsibilities of your documentation teams and documentation department?
2. If this group of people is responsible for the technical documentation (e.g. requirements, design, etc.) and the user documentation, how and why were they excluded from your ML 3 SCAMPI A?
3. How are the delivery teams different from the organization that achieved Maturity Level 3?

What is puzzling with your question is that REQM, RD, TS, PI, and CM cover the different aspects of writing and controlling the various documents associated with designing, developing, maintaining, operating, using, and deploying products. And then VER covers the inspection/review of the documents before placing them in the baseline and controlling them with CM.

It does appear from your description that your organization omitted the documentation people as a process role from your process documentation. In my opinion, at a minimum, your PPQA audits should have identified this omission long before your SCAMPI A appraisal. Then your Lead Appraiser should have identified this gap during the appraisal planning process before the SCAMPI A and should have taken steps to address the gap or postponed your appraisal until the documentation group was included in the scope. Since your documentation group was apparently not included in the scope of your appraisal, this oversight also calls into question your Lead Appraiser’s credentials and quite possibly the validity of your SCAMPI A results.

The bottom line, in my opinion, and based on only what you stated, your documentation group should have been included in the scope of your ML 3 SCAMPI A. Even if all they do is Document Configuration Control (which would be covered under CM) or Document Quality Assurance (which would be covered under PPQA).

The answers to my above questions could provide additional information that would change my opinion.