Bi-directional Traceability

Our organization is in the process of preparing for a CMMI Maturity Level 2 SCAMPI A appraisal. We are concerned about our approach for bi-directional traceability REQM SP 1.4. We maintain traceability is follows:

1. High Level Requirements <--> Use Cases (Includes GUIs and Database Interactions) <--> Test Cases

2. Use Cases <--> Source Code

Note: One can trace from Test Cases to Source Code through the Use Cases and Vice Versa, but the traceability is not direct. The reason behind this is, test cases are generated from use cases and are tested against the application (black box testing). Source code does not have associated test cases.

Is this kind of traceability considered bi-directional and is satisfactory for Maturity Level 2?

What you describe is one of many ways to implement bi-directional traceability and meet the intent of the CMMI. If your method supports your business goals and objectives and there are no quality issues, your approach should be acceptable for a Maturity Level 2 appraisal.

It is interesting that you think tracing from Test Cases to Source Code via Use Cases may not be acceptable. Traceability is a multi-dimensional mapping that can have one-to-many and many-to-one relationships. As long as you trace from the top all the way to the bottom and vice versa, you should be fine no matter how many links there are in the chain, and the chain can have branches as well.

Please note that bi-directional traceability does not mean tracing one whole document to another whole document. What it means is that a given item in one document (a specific requirement for example) can trace to multiple items in another document, multiple items in one document can trace to one item in another document, one item can trace to one item, etc.

REQM and RD in the CMMI

Why is REQM Management at Maturity Level 2 and Requirement Development at Maturity Level 3? We develop the requirements first and then manage them in the project.

There reason for the placement is due to the meaning of ML 2 vs. ML 3. ML 2 is all about stabilizing projects and gaining control over project estimates. Once the organization has achieved this, then it can begin to evaluate how to improve the engineering areas.

Since you need to have a baseline upon which to plan a project and the other ML 2 Process Areas, that is why REQM is the first Process Area in ML 2. The intent is to manage the collection of project requirements: good, bad, or indifferent. And use this collection to plan the project, etc. Then when you have achieved ML 2 and move to ML 3, then you can address how to improve the Requirements Elicitation to obtain better requirements.

Please keep in mind that the CMMI is a collection of guidelines and best practices for doing process improvement. The CMMI is not a roadmap for how to do software engineering.

REQM SP 1.2 Question

Can you please help me?

I am developing a study based on CMMI-DEV and I am having some difficulty with the interpretation of Specific Practice 1.2. My concern is how to apply SP 1.2 in practice. Who are the members of the project that must be committed to the requirements (business analysts, systems analysts, test analysts, developers, systems architects, project manager, and project leader)?

How should their commitment be documented? For example, for each new or changed requirement the members must sign a document that means that they are aware of the requirement and are committed to it.

Do you have a copy of the Addison Wesley published CMMI-DEV version 1.2? If so, there are some very helpful tips in the margins for REQM SP 1.2.

When you have a set of requirements for a project team, everyone on the team impacted by the requirements needs to share a common understanding of the requirements. In addition, since you typically do not have the full set of requirements available at project start, the project team needs to continually evaluate and reevaluate their ability to meet the requirements as the requirements evolve over the life of the project. One way of evaluating requirements is an impact assessment on the existing requirements, design, documents, test cases, current and downstream tasks and activities, cost, schedule, etc. for a new or changed requirement. By going through an impact analysis, that will provide a way for the project participants to communicate their ability to meet the requirements and associated commitments. The resulting impact analysis report can be one way of documenting the commitment. Other methods for documenting commitments include meeting minutes, document signatures, or email. However, just signing a document indicating that the participants are aware of a requirement really does not indicate that they are committed to the requirements and you might run into difficulties later on in the lifecycle.

Please keep in mind that commitments include both the resources involved (people, tools, and facilities) and the schedule for completion.

CMMI Practices for Documentation Teams

Our organization is CMMI Maturity Level 3 Ver 1.2 certified. Our delivery teams are going to implement CMMI practices soon. I did see the processes of the organization and though all the roles appeared from project leader to developer to manager, except for documentation teams. In the same context, I am very curious to see if there are any set of practices to be followed for the documentation department in CMMI as all the processes at CMMI ML2, ML3 are specific to project management, engineering, support, organization areas.

Your question actually raises some other questions:

1. What are the roles and responsibilities of your documentation teams and documentation department?
2. If this group of people is responsible for the technical documentation (e.g. requirements, design, etc.) and the user documentation, how and why were they excluded from your ML 3 SCAMPI A?
3. How are the delivery teams different from the organization that achieved Maturity Level 3?

What is puzzling with your question is that REQM, RD, TS, PI, and CM cover the different aspects of writing and controlling the various documents associated with designing, developing, maintaining, operating, using, and deploying products. And then VER covers the inspection/review of the documents before placing them in the baseline and controlling them with CM.

It does appear from your description that your organization omitted the documentation people as a process role from your process documentation. In my opinion, at a minimum, your PPQA audits should have identified this omission long before your SCAMPI A appraisal. Then your Lead Appraiser should have identified this gap during the appraisal planning process before the SCAMPI A and should have taken steps to address the gap or postponed your appraisal until the documentation group was included in the scope. Since your documentation group was apparently not included in the scope of your appraisal, this oversight also calls into question your Lead Appraiser’s credentials and quite possibly the validity of your SCAMPI A results.

The bottom line, in my opinion, and based on only what you stated, your documentation group should have been included in the scope of your ML 3 SCAMPI A. Even if all they do is Document Configuration Control (which would be covered under CM) or Document Quality Assurance (which would be covered under PPQA).

The answers to my above questions could provide additional information that would change my opinion.

REQM - SP1.1 Query

In REQM – SP1.1 calls for “Criteria for evaluation and acceptance.” Can you tell me what this expected “criteria” would be?

I suggest looking at Requirements Management (REQM) Process Area in your copy of the CMMI. On page 490 in REQM Sub-practice 2 there is a list of example evaluation and acceptance criteria.

REQM does not expect the organization to have evaluation and acceptance criteria. What you are expected to provide is evidence that support the required components (Specific Goal 1 in this case; evidence that demonstrates that your requirements are managed and inconsistencies with project plans and work products are identified) and evidence that supports the expected components (Specific Practice 1.1 in this case; evidence that demonstrates that you have developed an understanding with the requirements providers on the meaning of the requirements). If your Lead Appraiser is telling you that you have to provide these criteria, then he or she should be challenged as to why they are telling you that you need to provide this evidence.

The evaluation and acceptance criteria are one way of developing this understanding. Please bear in mind the section title is TYPICAL Work Products, not EXPECTED Work Products, or REQUIRED Work Products. The list of Typical Work Products is merely a list of example work products.

Requirements Development/Management Question

I was slightly confused after reading this statement in the Requirements Development (RD) Process Area (PA), Specific Practice (SP) 2.1 states "Establish and maintain product and product component requirements, which are based on the customer requirements"

The informative material goes on to say "The modification of requirements due to approved requirement changes is covered by the "maintain" function of this specific practice; whereas, the administration of requirement changes is covered by the Requirements Management process area."

I do not understand the difference between maintenance of requirements in RD and administration of requirement changes in REQM. How are they different?

The Requirements Management (REQM) Process Area (PA) is there to help an organization manage changes to an agreed to set of requirements (baseline). So, no one makes any changes to the requirements (add/modify/delete) without following the documented requirements change control process, which should include review and approval by the relevant stakeholders of the requirements.

And actually, REQM is a specific instance of Configuration Management (CM). But REQM is such an important and vital practice that it is its own Process Area. So it is entirely possible for an organization to have one Change Management process that it uses to manage changes to requirements, documents, plans, test scripts, code, etc.

In contrast the intent of Requirements Development (RD) is to actively elicit and further refine the requirements that are then agreed to and incorporated into the requirements baseline. Specific Practice 2.1 contains the phrase “establish and maintain,” which has a special meaning within the context of the CMMI. If you have taken the Introduction to CMMI class you would have learned what this phrase means. This phrase is also defined in the CMMI Glossary and it means formulate, document, and use. For SP 2.1, this means that the product and product component requirements are formulated (elicited, discussed, refined, reviewed, etc.), documented (written down in a requirements specification, requirements management tool, etc.), and used (develop the project plan, design, code, tests, etc.).

So the connection between REQM and RD is that proposed requirements changes are reviewed, analyzed, approved, and processed by REQM and the authorized and approved changes are propagated through the baselined requirements by RD.

I would not get too focused on trying to understand the separation between REQM and RD. You have identified one of the many areas in the CMMI where there is a tight coupling between PAs, if not overlap. What is important is that you

1. create a requirements baseline that does not change unless there is an approved change request,
2. investigate, impact, review, and disposition all proposed requirements changes with the relevant stakeholders (approved or rejected), and
3. only make approved changes to the baselined requirements and appropriately update the traceability

CMMI Implementation

I recently joined a company where there is no process and management recruited me to implement the CMMI. The organization has different business units. Though everyone sits together, they work very differently. I conducted a Gap analysis based on the CMMI Level 2 processes and here are the findings:

1. Project Planning & PMC -- they create project plans and they have the regular project team meetings and they share the minutes. Each team has their own format and templates. The projects don't really do estimations. Can we satisfy the PP & PMC PA'ss without doing any estimation? I know it can't be that way, but can it be tailored?
2. Requirements Management: Some of the business units have CCBs to discuss change requests and other business units discuss requirements changes in their project team meetings. The most significant gap I found is in Requirements Traceability. Traceability of Customer requirements to the Functional Requirements and Traceability of Use cases to the Test cases are missing. Is this reason enough to fail the RM PA? One of the Engineering directors asked me how much traceability you need to satisfy this condition. At that time I said 100% of all the requirements. Then I also read from somewhere that it is OK to define that we maintain traceability for at least the MUST BE CUSTOMER REQUIREMENTS. Traceability of other requirements can be made optional. Can it be possible like that?
3. Configuration Management: The projects thought they have a CMP which is embedded in the project plan. What I found missing are the Configuration Audits ( PCA & FCA). Is it possible to satisfy the Configuration Management PA without doing Configuration Audits to check the document status, builds, and backup strategy?
4. PPQA: One of the business units has a Software Quality Assurance plan, but it is done by one of the testers from another project. As a part of PPQA the SQA will do some spot checks based on a pre-defined checklist, which includes Project Planning, Risk Management, Project Monitoring and Control, Integration and releases. But I guess this can be improved by my role as a independent software quality engineer.
5. M&A: I am very much worried with this process area, as of now the organization status is nil with respect to metrics collection, they don't have a metrics database and no metrics have been defined yet. Is it OK to start now to form a team to do some reasearch and come up with metrics definitions, deploy them and start collecting data? My question is how much data do we need to collect to satisfy this PA? And do we need to provide evidence of analyzing these collected data and show some improvements steps taken at the time of SCAMPI apprisals? How long will it take in general to satisfy the Measurement & Analysis PA?
6. SAM: can we tailor this PA if we are not dealing with suppliers? If yes how can it be possible?
   

The directive from the Leadership team is to acheive CMMI Level 3 by end of 2009. I was baffled to hear this. Under these circumstances, what are the chances of getting CMMI Level 3 or my traget is at least CMMI Level 2? That is what my initial target. Can I acheive CMMI Level 2 by the end of 2009? If so, what are the things I need to address?

Here are some of the things I have already started:

1. CMMI Overview training to all the teams
2. Dailogue session on metrics identification
3. Looking into some Requirements tools which provide Traceability
4. Need to push the project team to have configuration audits.

In addition we already have established a process data base and the processes are defined and templates are being used from the parent organization. Since our company is a multi-site company, one of our counter parts has already acheived CMMI Level 3. We will be using the same process database and their templates. I thinking of providing their training on each Process Area as well. Is this a correct way to use the processes and templates of our parent organization? If not, do we need to establish our own local process data base?

First of all, I sympathize with you and the challenges you face. I applaud the fact that you had the foresight to conduct a Gap Analysis of the organization. You have highlighted a number of key weaknesses within the organization. However, to provide you meaningful feedback on all of your points would require working directly with you and your company.

1. What is your CMMI experience? Have you taken the SEI’s Introduction to CMMI class? If not, I strongly recommend that you and possibly those others in your company who are responsible for your processes take the three day class. The class should provide you a more thorough understanding of the CMMI, its interpretations, and material for constructing an internal Overview class.
2. You have identified some serious deficiencies within the organization in all of the ML 2 Process Areas. These need to be analyzed, addressed, corrected, solutions implemented, and then re-evaluated some months into the future before you can consider a formal SCAMPI at any Maturity Level. The length of time before the next evaluation is a function of a number of factors: number of people in the organization, number of projects, typical project duration, how much time and other resources are dedicated to process improvement, etc.
3. The organization needs to first implement Maturity Level 2 to form a firm foundation before considering moving to Maturity Level 3. The issues you have identified are fairly typical. Basically, it sounds like your organization does not perform PPQA or MA and is challenged with Project Management, Requirements Management, and Configuration Management. The first steps here should be to identify the necessary skills-based training classes you need to bring in-house and train your staff on these concepts. If you just purchase tools and push for audits, you most likely will not achieve the desired effect. You have to understand your process first and the reasons for why it is important to perform each of the steps.
4. Since another division has already achieved ML 3, it is a good idea to learn from their mistakes. But be very careful of the temptation to “clone” their processes and procedures. You have to implement the processes and procedures that match the way you conduct business today.
5. Based on what you have outlined, and given how much time and effort it could take just to address the ML 2 issues, I would say that ML 3 is out of the question for 2009. You could conduct a ML 3 SCAMPI A by the end of this year, but in all likelihood it would not be successful.
6. The best suggestion I have for you is to hire a CMMI consultant and Lead Appraiser to provide you with the proper advice and guidance. Otherwise, you could be spending a lot more time and effort than originally anticipated.

Role of the System Architect

Someone sent me the link to this YouTube video that explains the role of the systems architect.

The message given in this video also addresses the need for proper documentation of requirements, Requirements Management, and Requirements Engineering. The video uses simple roles and examples to clearly illustrate these concepts. Plus it is cute and fun to watch. Enjoy!

Frequency for REQM SP 1.5

A system analyst asked me how often he should be checking for inconsistencises between the project plan, work products, and requirements. I told him that he should be checking at least at each milestone, ideally at each week, to verify that the interim work products satisfy the baselined requirements; if he finds any gaps, document them in the defect log, and send it to PM for resolution. If he thinks that the project plan needs to be revised, document that in the defect log too. My answer is based on the fact that the PM reviews the project activities/schedules weekly. Could anyone provide more practical way to do this practice?

In my opinion checking for inconsistencies between the requirements, work products, and project plan each week is overkill. It would be proper to check for inconsistencies at each project milestone and probably each time the requirements, work products, and/or the project plan are updated. By updating the project plan, I mean re-planning or re-baselining the project plan, NOT the regular updating of the project status (e.g. % complete, etc.). If you checked at a higher frequency, most likely you won’t find any inconsistencies and whoever is performing this check will begin to question its business value.

Once an inconsistency is found, the process you described seems reasonable to me.

Developing Questions for the CMMI Specific and Generic Goals Would Lead to Implementing Alternative Practices

I would be very CAUTIOUS about taking this approach to implementing the CMMI. The CMMI authors did not go to the lengths they did with defining the Expected components (Specific and Generic Practices) if all they really wanted an organization to do was start at the Goals and identify the supporting practices. In my experience as a Lead Appraiser, it is only on the rare occasion that I have identified and documented an alternative practice in a SCAMPI. 99.999% of the time, the organization’s practices are aligned with the Expected components. And I assert that the only time an alternative practice comes up is with legacy projects that are years in duration using practices that are no longer state of the art and there is no business value in changing them. For example one of the underlying assumptions of REQM is that an organization is using tools to help manage and control the project’s requirements. But if the original requirements were established before the advent of requirements tools and the organization’s documented process for reviewing and managing requirements was to manually review, trace, and track requirements through meetings and discussions re-deriving the traceability each time and the process works, it is an alternative practice. There is no requirements database, just a requirements document. There is no requirements traceability matrix or requirements tracking system, just the documented requirements and the expertise of the requirements analysts who are highly trained in how to procedurally review, maintain, manage, and control requirements. So it is difficult to evaluate SP 1.1 – 1.5 as FI, but REQM SG 1 is clearly FI.

GP 2.1 Establish an Organizational Policy

This Generic Practice is fairly simple and straightforward, but one that many organizations have difficulties implementing the first time around. The practice states "Establish and maintain an organizational policy for planning and performing the process." What I see many organizations do is write a high level process or procedure as the policy instead of providing the necessary organizational expectations for the process.

One of the underlying principles of the CMMI is the principle of setting goals and objectives. Senior/Executive Management should be setting and clearly communicating the organization's business goals and objectives. The policy required by GP 2.1 is then the place for aligning the quality system with the organization's business goals and objectives.

Another problem I have encountered is that organizations tend to write huge policies when in point of fact, all that is necessary to clearly communicate the organization's policy is a few sentences or a paragraph.

And in an extreme case, I have seen one organization declare that it is their policy for each organizational unit to achieve a specific Maturity Level. This odd policy is very similar to the blog I wrote the other day titled Setting Proper Goals and Objectives. The organization's core business is NOT to achieve a Maturity Level, implement a specific model, or comply with with a specific standard. The organization may need to achieve a specific Maturity Level or comply with a standard in order retain existing business or win new business, but that is not why the organization is in business. The organizational policy writer(s) need to dig a bit deeper and write the policies to be aligned with the organization's actual business goals.

For example, if one of the organization's core business goals and objectives is to deliver high quality products and services to their customers, then each organizational policy should have that focus. Keeping this focus in mind, then an example Requirements Management Policy might look like this:

Organization XYZ is committed to delivering high quality products to its customers by establishing a common understanding between the customer and the project of the customer’s requirements. This policy applies to all XYZ projects that meet one or more of the following criteria:
1. [Project effort is expected to exceed 1000 hours]; OR
2. [Project budget is expected to exceed $250,000]; OR
3. [Project schedule is expected to exceed 6 months].

All projects that satisfy these criteria must follow the Requirements Management Process and ensure that:
1. The requirements are documented.
2. The requirements are reviewed by the managers, and other affected groups.
3. The plans, work products, and activities are changed to be consistent with changes to the requirements.

Senior Management is responsible for:
1. Establishing and ensuring conformance to this policy.
2. Establishing responsibility for analyzing the system requirements and allocating them to hardware, software, and other system components.
3. Providing adequate resources and funding for managing the allocated requirements.
4. Reviewing the activities for managing the allocated requirements on a periodic basis.