I have a question related to Supplier Agreement Management (SAM) SP 2.2 - Monitor Selected Supplier Processes. What is the basic intent of this practice and in what scenario does it fit in? It specifies "...situations of tight alignment between processes implemented by the supplier and those of the project..." - which is normally not the case in most (small) projects (as I know). I hope here we are not including "Acceptance" and "Transition" as aligned processes. Also it seems redundant to me with SP 2.1 - Execute the Supplier Agreement because in the contract/ SOW, it is usually mentioned how the supplier needs to monitor his processes (frequency to perform process audits etc.) and the frequency/condition when the customer may ask for a process audit/assessment. So doesn't executing the Supplier agreement covers these two practices?
I can see where you might have some confusion concerning these two SAM practices. SAM SP 2.1 says “Perform activities with the supplier as specified in the supplier agreement.” And your confusion comes about from sub-practice 1 “Monitor supplier progress and performance (schedule, effort, cost, and technical performance) as defined in the supplier agreement.” In essence SAM SP 2.1 is all about performing Project Monitoring and Control (PMC) over the supplier, which should be spelled out in the supplier agreement. You are basically acting as the Project Manager for the supplier by monitoring and controlling their project and technical performance. There is no intent to perform any Process and Product Quality Assurance (PPQA) audits or activities to support this practice.
In contrast SAM SP 2.2 says “Select, monitor, and analyze processes used by the supplier.” This practice is where you perform PPQA activities (process audits and work product audits) on selected supplier processes that are critical to the success of your project and business. Again, this ability to perform PPQA on the supplier must be specified in the supplier agreement. But you want to have the freedom to select any supplier process, so don’t indicate specific processes in the supplier agreement. For example you might decide to monitor and analyze how your supplier performs peer reviews or how they manage their requirements. If you have your supplier doing small projects (less than a month in duration) you may not have many opportunities to perform PPQA on a given supplier project. This situation is the same as when you have small projects done completely in house. There is no hope or expectation that you will be able to perform PPQA activities on every small project.
Look back at my blog on PPQA Audit Frequency http://ppqc.blogspot.com/2008/05/ppqa-audit-frequency.html for a simple way to adjust the frequency of the PPQA audits based on the quality issues discovered. You can use this same approach to determine the frequency of conducting PPQA activities on your supplier, assuming that you supplier regularly performs small projects for you.
Showing posts with label audit frequency. Show all posts
Showing posts with label audit frequency. Show all posts
Wednesday, August 13, 2008
Wednesday, May 14, 2008
PPQA Audit Frequency
Organizations that have no history with peforming Process and Product Quality Assurance (PPQA) audits usually ask me how often should they be auditing their processes and work products. And the correct answer is "it depends", but that is usually not satisfactory. The frequency of audits depends upon the nature and severity of the quality issues associated with following the organization's processes. If there are minor findings or quality issues, then the audits don't have to occur very often, may be only once a year. But if there are major findings or issues, then they should be occuring at a higher rate until the issues go away and the processes stabilize.
Yesterday Pat O'Toole posted a message on the CMMI discussion group that take this approach one step further.
When consulting with a client on PPQA Pat suggests that PPQA use a "compliance scale" similar to that used in a SCAMPI appraisal: Fully Compliant, Largely Compliant, Partially Compliant, and Not Compliant.
This approach avoids the game playing of "just doing enough to get a 'Yes' in the audit." It also allows for a finer grading of compliance metrics and trends. And turns the audit feedback sessions into more of an internal consulting discussion than merely a "did we pass or not" exercise.
To "score" an audit, award 100 points for Fully Compliant, 75 points for Largely Compliant, 25 points for Partially Compliant, and 0 points for Not Compliant. Average the score over all of the audit items and you get the score for that particular audit.
You can average the scores of all PPQA audits conducted on a particular project to get the project-level compliance score. Hopefully you find that there is a positive correlation between projects with high compliance scores and the "success" of the project. (If there is a negative correlation you have serious cause of concern!)
I also recommend that you maintain a database (or Excel spreadsheet) with the audit items and their scores across projects and time. You can use the same scoring mechanism described above to show the average score for each audit item.
Audit items that average 90+ for 3 months are candidates for sampling - people appear to "get it" for these items. Audit items that average below some minimum threshold (60?) are probably candidates for reworking the process infrastructure - whatever you've provided isn't being used anyway, so perhaps it's time to give them something that they CAN use (and/or DO find value added).
Pat's quantitative approach makes it very clear which processes and/or projects need to be audited more frequently than others. So when a process or project scores above 90% (or so), you can reduce the audit frequency for that process or project. The default audit frequency needs to be set by the organization. Auditing once a month may be too frequent for some organizations and just right for others. The frequency should match the normal durations of your project lifecycles. Assuming a monthly frequency, if the audit score is 90+% then the frequency for that audit can go to a bi-monthly frequency. If the next time that particular audit again achieves 90+%, the audit can go to on a six month cycle. If on the other hand the score drops below 90, then audit frequency should drop back to the previous frequency. Now you have a variable audit frequency that you can tie directly to the audit results. Pretty cool!
Yesterday Pat O'Toole posted a message on the CMMI discussion group that take this approach one step further.
When consulting with a client on PPQA Pat suggests that PPQA use a "compliance scale" similar to that used in a SCAMPI appraisal: Fully Compliant, Largely Compliant, Partially Compliant, and Not Compliant.
This approach avoids the game playing of "just doing enough to get a 'Yes' in the audit." It also allows for a finer grading of compliance metrics and trends. And turns the audit feedback sessions into more of an internal consulting discussion than merely a "did we pass or not" exercise.
To "score" an audit, award 100 points for Fully Compliant, 75 points for Largely Compliant, 25 points for Partially Compliant, and 0 points for Not Compliant. Average the score over all of the audit items and you get the score for that particular audit.
You can average the scores of all PPQA audits conducted on a particular project to get the project-level compliance score. Hopefully you find that there is a positive correlation between projects with high compliance scores and the "success" of the project. (If there is a negative correlation you have serious cause of concern!)
I also recommend that you maintain a database (or Excel spreadsheet) with the audit items and their scores across projects and time. You can use the same scoring mechanism described above to show the average score for each audit item.
Audit items that average 90+ for 3 months are candidates for sampling - people appear to "get it" for these items. Audit items that average below some minimum threshold (60?) are probably candidates for reworking the process infrastructure - whatever you've provided isn't being used anyway, so perhaps it's time to give them something that they CAN use (and/or DO find value added).
Pat's quantitative approach makes it very clear which processes and/or projects need to be audited more frequently than others. So when a process or project scores above 90% (or so), you can reduce the audit frequency for that process or project. The default audit frequency needs to be set by the organization. Auditing once a month may be too frequent for some organizations and just right for others. The frequency should match the normal durations of your project lifecycles. Assuming a monthly frequency, if the audit score is 90+% then the frequency for that audit can go to a bi-monthly frequency. If the next time that particular audit again achieves 90+%, the audit can go to on a six month cycle. If on the other hand the score drops below 90, then audit frequency should drop back to the previous frequency. Now you have a variable audit frequency that you can tie directly to the audit results. Pretty cool!
Subscribe to:
Comments (Atom)