Appraisal Scope Question for Agile Development

I am preparing for an appraisal coming up in a few months and I have run into an interesting question that I would like to ask you about.

Some of the projects in the Organizational Unit (OU) follow an Agile method for development. It is really a modified combination of SCRUM and XP. One of the questions they asked me was whether they would have to provide evidence for all of the Process Areas (PAs) in the scope of the appraisal? I was perplexed by this question because it seemed rather obvious to me, so I replied "of course." But this question stuck in the back of my mind so I dug into the SCAMPI Method Definition Document (MDD) for more clarification.

From the MDD, Method Assumptions and Design Principles, page I-19 and I-20: "The extent to which an organizational unit has implemented appraisal reference model practices can be determined only by considering, in aggregate, the extent to which those practices are implemented within the organizational unit by project and support groups. This process, in turn, necessitates the consideration of objective evidence for each instantiation, for each model practice within the appraisal scope."

This seems fairly clear to me, we must examine objective evidence for all practices of all instantiations within the scope of the appraisal. However, under the process definition in part 2 of the MDD you can find less convincing information under the Parameters and Limits section of 1.1.3 Determine Appraisal Scope. "Sample projects and support groups selected to form the organizational scope (i.e., the combination of focus and non-focus projects and support functions) must represent all critical factors identified for the organizational unit to which the results will be attributed." So it seems that as long as the combination of all focus and non-focus projects represent all the critical factors it should be sufficient. I know, critical factors are different from the PAs in the model scope. :-)

Also, in Parameters and Limits it clearly states: "Focus projects must provide objective evidence for every PA within the model scope of the appraisal..." "Non-focus projects must provide objective evidence for one or more PAs within the model scope of the appraisal..." So it seems that if an Agile project was a non-focus project they would not have to provide evidence for all PAs. Do you think this is correct?

The MDD goes on to further say in the Parameters and Limits: "In appraisals where the reference model scope includes any project-related PA, the organizational scope must include at least one focus project. If the organizational unit includes more than 3 projects, then the organizational scope must include sufficient focus projects and non-focus projects to generate at least 3 instances of each practice in each project-related PA in the model scope of the appraisal."

So, conceivably, you could have only 1 focus project, and say maybe 6 non-focus projects with each only having a few PAs to "generate at least 3 instances" of the practices that need to be covered in the scope of the appraisal.

I know this may seem a bit extreme, but believe me, I have run into stranger scenarios than this one. lol :-)

Ideally, all projects would cover all PAs. But the role and function of non-focus projects really seems to muddy the waters in my opinion.

What are your thoughts?

There is an SEI Technical Note CMU/SEI-2008-TN-003 published in November 2008 titled CMMI® or Agile: Why Not Embrace Both! by Hillel Glazer, Jeff Dalton, David Anderson, Mike Konrad, and Sandy Shrum that contains a lot of pertinent information about Agile and the CMMI, as well as Hillel's recently published companion article in Crosstalk Love and Marriage: CMMI and Agile Need Each Other http://www.stsc.hill.af.mil/crosstalk/2010/01/1001Glazer.html

In my opinion, if Agile is one of many different development methods in use by the organization and the Agile projects do not cover all of the PAs in scope of the appraisal, then it is not in the spirit of the MDD and non-focus projects to mix and match a number of Agile projects just so there is evidence for all the PAs. The way that I look at non-focus projects is that there may be some projects in the appraisal scope that do not have evidence because the projects have not reached that part of the lifecycle yet, NOT that they are not performing the practice.

And, just because a project is using Agile in a Maturity Level (ML) 3 organization does not mean that they don’t have to perform all of the PAs. There should be evidence available though they may be using terminology that is not obviously mapping to the CMMI. Someone with deep CMMI and Agile knowledge should work with the organization to show them how what they are doing does comply with the CMMI.

Project Tracking Through Milestones

We have a Project management process defined for covering Project Planning (PP) and Project Monitoring and Control (PMC). It includes a Work Breakdown Structure (WBS) (task effort size must be about 20 hours), estimation process, assigned resources, etc. We are using MS-Project for tracking the schedule, and each month there is an Excel report with a project status summary.

People think it is very heavy to track the all of the fine-grained tasks in MS-Project. They would prefer tracking the project through milestones in Excel.

I’ve heard about using the burn down chart in Scrum, and I know some organizations use agility within CMMI model.

It is possible to do that? Could we be more “agile”?

The short answer is yes. The CMMI does NOT prescribe any project management tool or level of tracking. These decisions are left up to the organization to make. Sounds like what you need is a process that fits your organization. If people are complaining that the tracking process is too cumbersome to use, then you should definitely examine other methods. Take a top down approach from the business goals and objectives. What project tracking information does a project manager need in order to determine if the business goals and objectives are being met? Once you answer this question, that will help you decide the proper level of project tracking and monitoring. If you are still having difficulty figuring out the best method, work with a CMMI consultant to help you define a process that will be the best fit for your organization.

2009 SEPG Conference - Day 2

Yesterday the conference kicked off into full swing with the two Keynote Speakers. The first speaker was Scott Cook, Chairman of the Executive Committee of the Board, Intuit Inc. Scott talked about the key process factors that have led to Intuit's success: Team Software Process (TSP), Agile, and the Toyota Production System (TPS). Intuit has realized quite some important gains by using these different process improvement methods. At the conclusion of his talk, he presented an entirely different issue that is ripe for process improvement. Scott showed a slide of various causes of death each year in the US and he asked us what we thought were the top three causes. It turns out that Heart Disease and Cancer are the top two, but the third cause is Hospital-Related. Scott claimed that there are 160,000 Hospital-Related deaths each year, which is the equivalent of a fully loaded 747 crashing and killing all on-board once a day. And the majority of these deaths can be attributed to people and process problems: not enough resources available, wrong diagnoses made, wrong surgical operations performed, incorrect medication delivered, etc. So Intuit is helping the Stanford Medical Center use TPS, based on Intuit's experience, to address these people and process issues and they are already seeing benefit.

The second speaker was Jim Bampos, Vice President, Information & Quality Management, EM Corporation. Jim spoke about the process improvement challenges faced by a primarily hardware company that now has a software component. And the added complexity of EMC continually acquiring new businesses and integrating them into their corporate culture. Jim recommended Marcus Buckingham's book Put Your Strengths to Work.