Selection criteria for a CMMI Lead Appraiser

My organization has used the same lead appraiser for several appraisls and we are now considering using a different LA for our upcoming ML 3 appraisal. My thoughts are to identify several candidate appraisers and then evaluate them against a set of criteria. Here are some of the criteria that I've identified to date:

• Experience with organization similar to our (kinds of products, size, geographically distributed, etc.)
• Expected time for appraisal of similar scope to ensure that we have similar expectations. We want to check length of appraisal and expected number of hours per day.
• Availability of toolset to support appraisal activities (i.e., PIID collection, ratings, consensus tracking, etc.)
• Availability during dates planned for appraisals
• Costs

Are there any other evaluation criteria I should be considering?

I don’t believe that the criteria you have specified will be able to determine a best fit, some of your criteria might but not the rest.

1. Experience working with other organizations of similar size, etc. is a good choice.
2. Expected duration of the appraisal is not a good choice. If you provide sufficient information for the Lead Appraiser to scope the work, I would expect that all qualified Lead Appraisers would provide the same answer, though there may be some variations. In my experience Readiness Reviews are 5 days and SCAMPI A on-site periods rarely exceed 10 days. It might be better if you presented your expectations for the appraisal duration and see if you got agreement from the Lead Appraiser.
3. Availability of tool set would not be a good choice. Usually qualified Lead Appraisers have a tool set they prefer to use. A better criteria might be the willingness of the Lead Appraiser to use your specified tool set.
4. Availability during your planned appraisal dates is not a good choice. Anyone bidding would state that they were available. Besides, we all know that planning dates are simply planning dates and the actual dates are different.
5. Cost is a good choice and probably the only one where you will be able to discriminate between bids. But I encourage you to understand that a SCAMPI appraisal is NOT a commodity. When you purchase any service, including appraisal services, you must understand the value you are intending to obtain. Requests for Propsal should be structured to obtain the information needed to compare value, not just fees or costs.
   

I recommend that since you do have experience with the CMMI, appraisals, and Maturity Level 2 that you put together a plan for your ML 3 appraisal based on the historical data from your ML 2 efforts. Then ask for bids from several Lead Appraisers. Also ask for recommended optional services. Some Lead Appraisers will bid the job by number of days and others will bid the job by appraisal activity. That should provide you with some very good information to make your decision. And here is a radical thought, since you are talking about ML 3, why don’t you use DAR to help you make your decision. :-)

Finally, I recommend that you interview each candidate Lead Appraiser, either over the phone or in person to see how comfortable you feel with him or her and how each responds to your appraisal needs.