CMMI for Modeling and Simulation Efforts

I am interested in applying CMMI techniques to modeling and simulation efforts. I do not mean the modeling and simulation of the software engineering process (I have plenty of information, models etc. on that).

We do modeling and simulation work (e.g. growth of the biofuel industry, migration issues, watershed management) to inform policy makers. We have a grassroots 'best practices' movement and are looking for suggestions, ideas, etc. to improve our process. We do not use 'popular' tools and the niche software we do use is not conducive to configuration management tools etc. In addition, we rarely have a requirements document. We are essentially doing research.

Do you have any suggestions?

In my experience, a pure R&D shop has difficulty implementing the CMMI-DEV because of the free form nature of the environment and work. However, what makes more sense for you to investigate is the CMMI-SVC. It sounds like you are performing engineering or research and analysis services for policy makers.

As you may not have a requirements document for your niche software, you most surely have requirements for the research you are performing for your clients. If not, then you run the risk of having your results called into question.

Therefore, I think that you would be better served by the CMMI-SVC than the CMMI-DEV as it sounds like you may not be doing a lot of development, but instead providing research services.

Project Tracking Through Milestones

We have a Project management process defined for covering Project Planning (PP) and Project Monitoring and Control (PMC). It includes a Work Breakdown Structure (WBS) (task effort size must be about 20 hours), estimation process, assigned resources, etc. We are using MS-Project for tracking the schedule, and each month there is an Excel report with a project status summary.

People think it is very heavy to track the all of the fine-grained tasks in MS-Project. They would prefer tracking the project through milestones in Excel.

I’ve heard about using the burn down chart in Scrum, and I know some organizations use agility within CMMI model.

It is possible to do that? Could we be more “agile”?

The short answer is yes. The CMMI does NOT prescribe any project management tool or level of tracking. These decisions are left up to the organization to make. Sounds like what you need is a process that fits your organization. If people are complaining that the tracking process is too cumbersome to use, then you should definitely examine other methods. Take a top down approach from the business goals and objectives. What project tracking information does a project manager need in order to determine if the business goals and objectives are being met? Once you answer this question, that will help you decide the proper level of project tracking and monitoring. If you are still having difficulty figuring out the best method, work with a CMMI consultant to help you define a process that will be the best fit for your organization.

How Do I Determine Estimates of the Size Attributes?

CMMI-DEV Project Planning (PP) Specific Practice (SP) 1.2 states 'Establish and maintain estimates of the attributes of the work products and tasks.'

During appraisals it has been noted that most of the time we are relying on expert knowledge to determine effort and cost rather than estimating the sizing attributes. Just using expert knowledge makes it difficult to improve our effort estimation, hence having a dedicated set of attributes is essential in order for us to improve our estimations. We have evaluated some comprehensive methods (e.g. Function Points), but these are not considered as beneficial to our projects. We are now looking for an approach that allows us to improve our estimates with a simpler method which could then be refined based on our needs. Would you please give me advice how to tackle this challenge?

What PP SP 1.2 is looking for is the Basis of Estimate for the work product. If you are estimating a document for example, what aspects or attributes of the document can you estimate that when combined with a productivity factor will yield effort and cost? You determine these attributes by analyzing the historical data from your organization’s past projects. So based on the project’s requirements, you could estimate the number of pages that would be written for a document, the number of figures or drawings that need to be created or modified, etc. These items are then sizing parameters.

For source code, you need to analyze historical data from past projects to give an empirical estimate. As you say, function points are comprehensive. A simpler method may be to classify your software requirements in categories such as interface requirements, display requirements, processing requirements, reporting requirements, etc. and then using your historical data determine the correlations between these requirements categories and code size. Many organizations start with this simple kind of estimation/prediction model and continually refine it by incorporating data from each new project until a fairly accurate estimation model emerges. This estimation model is then very specific to the organization and type of software development it performs.

Hope these ideas help.

GP 2.9 in PPQA process area - what do I need?

I'm responsible for coordinating CMMI ML2 implementation in my organization. For PPQA GP 2.9 "Objectively evaluate adherence of the process against its process description, standards, and procedures, and address noncompliance" who can be an internal auditor of my organization?

And about tools and forms for this evaluation: Could I elaborate a checklist for this evaluation? Do you have any examples of this?

An objective evaluation implies some independence from the people performing the process activities. That means the people who perform the PPQA activities do not evaluate their own work. Organizations perform GP 2.9 of PPQA in a variety of ways.

1. Someone else in the organization who is not performing the PPQA audits of REQM, PP, PMC, etc. audits the PPQA activities

2. If the company is large and has several divisions, someone who performs the PPQA activities in another division audits the PPQA activities in your division

3. An external auditor (e.g., ISO 9000) audits the PPQA activities

4. An external consultant audits the PPQA activities

5. If your company is a government supplier, then the government may be auditing the PPQA activities

6. Etc.

So in your case, you could use an internal auditor to audit the PPQA activities as long as that person is not auditing his or her own work.

As far as a checklist, that needs to be developed by the person who is auditing the PPQA activities, just like the PPQA audits develop the checklists for the other Process Areas. The checklists need to cover both a process audit and a work product audit, and it may be easier to have two checklists instead of one. The checklist needs to be based on your documented PPQA process and process assets, not the CMMI.

If you are having difficulty understanding how to write a checklist, then I strongly encourage you and your organization to have someone come and train you how to perform PPQA audits. It is very important to conduct these properly; otherwise you could face difficulties with your process improvement efforts.

SCAMPI A Appraisal Questions for FAR Groups

Can you share the Questionnaires for a ML 3 SCAMPI A appraisal? Currently my organization is preparing for SCAMPI A and I want to trained the interviewees as this is their first experience an appraisal.

I am looking for some questions based on each Process Area, so the FAR Group Members will be prepared to answer them.

I have searched on Google but not able to find any information. :(

I am not surprised that you have not been able to find what you are looking for on the internet. A Lead Appraiser will not share his or her questions with you. Simply because the Lead Appraiser does not want the interviewees to be trained on what answers to provide, thereby biasing the results of any SCAMPI appraisal. If you and the organization are nervous about the appraisal, don’t be. There is no preparation for the interviews necessary. All that is expected by the Lead Appraiser is that any interviewee should be able to talk about HOW they perform their job duties. If people cannot do that, then the organization is not ready for an appraisal.

If you still feel uneasy, talk to your Lead Appraiser and have him or her conduct a Class B appraisal with interviews. A Class B or SCAMPI B appraisal can function as a dress rehearsal for a SCAMPI A without having the organization and the people worry about “flunking” or “passing” the appraisal. The interviewees will have the experience of being interviewed.

If you still want to perform an internal set of interviews, simply take the CMMI and step through the applicable practices with different groups of people (project managers, configuration managers, developers, testers, etc.) and ask them HOW they perform the Specific and Generic Practices. There is no set of standard questions. Each Lead Appraiser has their own style of questioning interviewees.

REQM SP 1.2 Question

Can you please help me?

I am developing a study based on CMMI-DEV and I am having some difficulty with the interpretation of Specific Practice 1.2. My concern is how to apply SP 1.2 in practice. Who are the members of the project that must be committed to the requirements (business analysts, systems analysts, test analysts, developers, systems architects, project manager, and project leader)?

How should their commitment be documented? For example, for each new or changed requirement the members must sign a document that means that they are aware of the requirement and are committed to it.

Do you have a copy of the Addison Wesley published CMMI-DEV version 1.2? If so, there are some very helpful tips in the margins for REQM SP 1.2.

When you have a set of requirements for a project team, everyone on the team impacted by the requirements needs to share a common understanding of the requirements. In addition, since you typically do not have the full set of requirements available at project start, the project team needs to continually evaluate and reevaluate their ability to meet the requirements as the requirements evolve over the life of the project. One way of evaluating requirements is an impact assessment on the existing requirements, design, documents, test cases, current and downstream tasks and activities, cost, schedule, etc. for a new or changed requirement. By going through an impact analysis, that will provide a way for the project participants to communicate their ability to meet the requirements and associated commitments. The resulting impact analysis report can be one way of documenting the commitment. Other methods for documenting commitments include meeting minutes, document signatures, or email. However, just signing a document indicating that the participants are aware of a requirement really does not indicate that they are committed to the requirements and you might run into difficulties later on in the lifecycle.

Please keep in mind that commitments include both the resources involved (people, tools, and facilities) and the schedule for completion.

Covering SAM SP2.2 and SP2.3

I want to ask you about CMMI Supplier Agreement Management (SAM) SP 2.2 Monitor Selected Supplier Processes and SP 2.3 Evaluate Selected Supplier Work Products. I have difficulty in understanding these practices and how to apply them in my company. We have a project that needs to buy new hardware and deliver it to customer.

How can I select and monitor supplier processes in this case?

Can you please explain that to me and give me an example of a supplier process that is applicable in our situation?

The best way to answer this question is to attend the SEI’s Introduction to CMMI 3-day class. During the SAM topic the instructor will explain the difference between these two practices for you. I would also suggest that you read the informative material in the SAM Process Area, for not just these two practices, but all of the Process Areas. The informative material is extremely rich in information, hints, tips, etc. and it provides guidance for understanding the intent of each practice and goal. I would also recommend that you not use the goal and practice titles for anything other than labels. The titles are shortened statements and do not always communicate the correct meaning of the goal or practice.

If you read the informative material for SAM SP 2.2 and SP 2.3 it will be immediately clear to you the differences. And if you still have questions regarding your specific circumstances, I suggest that you ask your SEI-certified Lead Appraiser. Without having a good understanding of your company and how it conducts business, asking specific implementation questions will most likely not provide the correct advice to you.

However, here is a very brief description of the differences between these two practices.

SAM SP 2.2 “Select, monitor, and analyze processes used by the supplier.” You as the acquirer of the supplier’s products or services must define in the supplier’s contract or agreement those specific supplier processes that are critical to your success. You have to state how and when you will be monitoring these critical processes. This practice is done as a risk mitigation to avoid surprises at the end of the contract when the supplier delivers the end product or service. Think of this practice as the acquirer performing PPQA process audits of the supplier’s processes.

SAM SP 2.3 “Select and evaluate work products from the supplier of custom-made products.” If your supplier is only delivering off-the-shelf product to you without any customization, this practice does not apply. However, if your supplier is providing custom-made or modifications to your specifications, then you need to decide which of these products are critical to your success and define those in the supplier’s contract or agreement. You have to state how and when you will be evaluating the products. Think of this practice as the acquirer performing PPQA work product audits of the supplier’s work products.