In REQM – SP1.1 calls for “Criteria for evaluation and acceptance.” Can you tell me what this expected “criteria” would be?
I suggest looking at Requirements Management (REQM) Process Area in your copy of the CMMI. On page 490 in REQM Sub-practice 2 there is a list of example evaluation and acceptance criteria.
REQM does not expect the organization to have evaluation and acceptance criteria. What you are expected to provide is evidence that support the required components (Specific Goal 1 in this case; evidence that demonstrates that your requirements are managed and inconsistencies with project plans and work products are identified) and evidence that supports the expected components (Specific Practice 1.1 in this case; evidence that demonstrates that you have developed an understanding with the requirements providers on the meaning of the requirements). If your Lead Appraiser is telling you that you have to provide these criteria, then he or she should be challenged as to why they are telling you that you need to provide this evidence.
The evaluation and acceptance criteria are one way of developing this understanding. Please bear in mind the section title is TYPICAL Work Products, not EXPECTED Work Products, or REQUIRED Work Products. The list of Typical Work Products is merely a list of example work products.
Showing posts with label evaluation criteria. Show all posts
Showing posts with label evaluation criteria. Show all posts
Saturday, October 17, 2009
Tuesday, November 11, 2008
A Question About DAR Calculations
I have a question regarding DAR calculations. When assigning wieghts to the selected criteria, do I have to relate them to each other? Meaning, do I have to refer to a number that represents my 100% need and relate all criteria to it? For example, I have three Criteria A, B, and C, then I'll assign 3 (30%) to A, 5 (50%) to B, and 2 (20%) to C, and in case one of them was decreased the others will increase. Currently, I define a range for criteria weight (for example from 1 to 5), and I select a number subjectively based on the weight of the criteria.
I think that you may have mixed things up a bit here. The evaluation criteria should be independent and not assigned percentages, the percentages apply to the weighting factors. For each criterion you assign scores between 1 (low) and 5 (high). The weights are derived from team consensus and total to 100% or 1. 20%, 30%, and 50% are typical weights. It is best not to have more than three evaluation criteria because if you have more, then there is the potential for the resulting scores to be too close to each other and it will be very difficult to determine which is the best solution.
With three criteria it is easy to assign 50%, 30%, and 20%. With more criteria you will be forced to assign weights that are closer together. So if you go strictly by the numerical calculations, then there is the potential for lots of ties or very close results thereby making it a challenge to use this technique for selecting an answer. For example, if you had five criteria, then one possible approach, though wrong one, would be to assign 20% to each criteria. Other options would be to assign weights of 30%, 25%, 20%, 15%, and 10% or 50%, 20%, 15%, 10%, and 5%. But you might find it difficult to reach consensus among the team for these values and how to apply them to the criteria. With these assignments, you might calculate the same score for one or more alternative solutions. So with more than three criteria the resulting unique weights could be very close together and not very useful for using this method to calculate scores and use the results to make a decision.
My premise is to limit the number of evaluation criteria to no more than three, not the number of alternative solutions. You can have as many alternative solutions as you want.
And another point is to try to select evaluation criteria that are independent, otherwise you will face challenges as well. For example if you are buying an automobile the independent evaluation criteria might be: 1. Features and price 2. Distance from your home to the dealer and 3. Service reputation.
However, you always have to use the sanity test to determine if the results of the calculation make any sense. You could have chosen the wrong criteria, assigned the wrong weights, and/or assigned the wrong values to each criteria. And yes, management always has the prerogative of making a decision without using the results of the process.
I think that you may have mixed things up a bit here. The evaluation criteria should be independent and not assigned percentages, the percentages apply to the weighting factors. For each criterion you assign scores between 1 (low) and 5 (high). The weights are derived from team consensus and total to 100% or 1. 20%, 30%, and 50% are typical weights. It is best not to have more than three evaluation criteria because if you have more, then there is the potential for the resulting scores to be too close to each other and it will be very difficult to determine which is the best solution.
With three criteria it is easy to assign 50%, 30%, and 20%. With more criteria you will be forced to assign weights that are closer together. So if you go strictly by the numerical calculations, then there is the potential for lots of ties or very close results thereby making it a challenge to use this technique for selecting an answer. For example, if you had five criteria, then one possible approach, though wrong one, would be to assign 20% to each criteria. Other options would be to assign weights of 30%, 25%, 20%, 15%, and 10% or 50%, 20%, 15%, 10%, and 5%. But you might find it difficult to reach consensus among the team for these values and how to apply them to the criteria. With these assignments, you might calculate the same score for one or more alternative solutions. So with more than three criteria the resulting unique weights could be very close together and not very useful for using this method to calculate scores and use the results to make a decision.
My premise is to limit the number of evaluation criteria to no more than three, not the number of alternative solutions. You can have as many alternative solutions as you want.
And another point is to try to select evaluation criteria that are independent, otherwise you will face challenges as well. For example if you are buying an automobile the independent evaluation criteria might be: 1. Features and price 2. Distance from your home to the dealer and 3. Service reputation.
However, you always have to use the sanity test to determine if the results of the calculation make any sense. You could have chosen the wrong criteria, assigned the wrong weights, and/or assigned the wrong values to each criteria. And yes, management always has the prerogative of making a decision without using the results of the process.
Subscribe to:
Posts (Atom)