- Experience with organization similar to our (kinds of products, size, geographically distributed, etc.)
- Expected time for appraisal of similar scope to ensure that we have similar expectations. We want to check length of appraisal and expected number of hours per day.
- Availability of toolset to support appraisal activities (i.e., PIID collection, ratings, consensus tracking, etc.)
- Availability during dates planned for appraisals
I don’t believe that the criteria you have specified will be able to determine a best fit, some of your criteria might but not the rest.
- Experience working with other organizations of similar size, etc. is a good choice.
I recommend that since you do have experience with the CMMI, appraisals, and Maturity Level 2 that you put together a plan for your ML 3 appraisal based on the historical data from your ML 2 efforts. Then ask for bids from several Lead Appraisers. Also ask for recommended optional services. Some Lead Appraisers will bid the job by number of days and others will bid the job by appraisal activity. That should provide you with some very good information to make your decision. And here is a radical thought, since you are talking about ML 3, why don’t you use DAR to help you make your decision. :-)
Finally, I recommend that you interview each candidate Lead Appraiser, either over the phone or in person to see how comfortable you feel with him or her and how each responds to your appraisal needs.