|
||
|
||
|
||
|
This snapshot is taken from our new 400-page reference text, The Request For Proposal Handbook (Third Edition). This book focuses on best RFP practices that will help you be successful. Chapter 8 of The Request For Proposal Handbook (Third Edition) discusses the nine building blocks of the evaluation process.
Hear what the President of NASPO says about this book . . . “Michael Asner's book removes a lot of the RFP process mystery. It is one of very few RFP books that address the negotiation process. I refer to Mr. Asner's book often and it is recommended reading for public procurement officers.” In reviewing a large number of RFPs, we identified nine different components of the process, such as reviewing a proposal for compliance with mandatory requirements, or interviewing the suppliers. We refer to each of these components as a building block: Establishing Compliance with Mandatory Requirements Scoring the Proposals http://www.rfpmentor.com/buildingblock2.html http://www.rfpmentor.com/buildingblock2b.html Imposing Upset Levels Evaluating the Cost Developing a Short-List Interviewing Suppliers Negotiating the Contract Requesting Best and Final Offers Checking References Scoring Systems Scoring systems are used to establish a numerical value indicating how well the proposal satisfies each selection criterion. Numerical scoring systems are easier to understand and explain than those based on only descriptive phrases. However, there are many different ways to evaluate proposals and establish the best value. These methods include color coding, adjectival ratings, and rankings. None of these methods is without problems and each method has its proponents and its critics. There is no requirement in many jurisdictions to use a numerical scoring system. Whatever method is used, including the common approach described in this text, the method is successful if it leads to an understanding of the deficiencies, weaknesses, strengths and risks of each proposal. The Louisiana RFP manual[i] provides a good example of adjectival scoring: 12. Evaluating Risk: evaluates how risky a proposer’s approach is in relation to cost and schedule. Risk Scale: Serious - Expected to cause serious disruption of schedule or increase in cost. Will require a significant level of contractor emphasis and government monitoring to overcome difficulties Moderate - Expected to cause moderate disruption of schedule or increase in cost. Will require average level of contractor emphasis and government monitoring to overcome difficulties. Minor - Expected to cause minor disruption of schedule or increase in cost. Will require a low level of contractor emphasis and government monitoring to overcome difficulties. Minimal - Expected to cause minimal disruption of schedule or increase in cost. Will require little or no contractor emphasis and government monitoring to overcome difficulties. While no method is perfect, some are better than others. One method that has been found satisfactory in many jurisdictions is a weighted point system in which points are awarded for each proposal's ability to meet predetermined criteria. Many organizations have found, through bitter experience, that it is harder to defend an evaluation based solely on words than on numbers. Scores seem easier to justify and to defend as being objective. Part of this is reality - often the detailed process for assigning scores is based on well-defined measurable factors. For example: "The proposal will receive one point (to a maximum of 4) for each full-time member of the project team with more than 5 years of directly related experience on similar projects." Part of the attraction of a numerical score is fiction: saying that a proposal received 230 points out of a maximum score of 400 is, to many people, much more concrete than saying that the proposal was "poor". Numbers imply objectivity and fairness, sometimes more than warranted. Evaluators sometimes have a difficult time deciding on the specific score of a factor. Suppose you were evaluating the plan for a project, or the project manager's experience. When is it worth 5 out of 10? Why isn't it worth a score of 6? Or 4? The more general question is how do we eliminate personal bias and take some of the arbitrariness out of scoring? Can we ensure that each of the evaluators is using the same scheme? There are two major types of scoring systems. The first is generic; it=s the system used for the last 100 years in public schools to grade students= compositions. Ten out of 10 is excellent. Five is acceptable. Most evaluations use some variation of this method. The second type of system is based on the specific characteristics of the procurement and requires a lot more work to be done prior to issuing the RFP. In this system, positive and negative indicators are developed for each factor. For example, in evaluating the strength of the project team, a positive factor would be the proposed manager=s experience with a similar system. A negative factor would be the use of a part-time project manager. After reviewing the proposals and these indicators, the evaluators would assign a score, usually using the zero-to-ten scale. The use of these indicators is a powerful tool in performing an objective evaluation, and one that is easy to justify should it come under public scrutiny. If these indicators are not formally developed and written down when the RFP is being constructed, they always emerge when evaluators are discussing their score for a specific proposal. These indicators simply reflect the concerns that an informed person would have related to each evaluation criterion. For example: Is the project manager experienced? Is the project manager full-time? The remainder of this section contains three examples of scoring systems. A scoring system published in an RFP should help the supplier understand the process and create a better proposal. Scoring systems are intended to assist the evaluators to identify the merits or the deficiencies in a proposal in an unbiased, objective way. They should be easily understood and simple to use. This first example is poor. It fails to promote an objective evaluation. It doesn’t provide any assistance in differentiating a “good” from a “satisfactory”. Unfortunately, tables such as this are used in many jurisdictions.
This second example is excellent. It helps the evaluators by providing precise definitions and boundaries on the scoring. It sets the agenda for discussions among the evaluators. The following rules were taken from a 1. If a requirement (objective) is particularly difficult to meet and the proposal offers an approach which, with little or no risk, will yield a result which exceeds requirements qualitatively, the item should score "8,", "9," or "10,", dependent upon the level of exceptional features offered. 2. If the requirement (objective) is relatively difficult to meet, the majority of the factors are acceptable, no major deficiencies or risks exist therein, and the collective approach yields a qualitative benefit beyond that which is minimal, a score of "6" or "7" should be assigned, dependent upon the benefits to be attained. 3. If the majority of the factors meet standards, the requirement is not overly difficult to meet, and the factors which are deficient are of a very minor nature or are susceptible to easy correction, the item should be scored "5". 4. If the major number of important factors are acceptable but one or more factors is deficient and some minor risk is involved in the correction thereof, the score for the item should be "4". 5. If a majority of the factors for the item are deficient and their correction, either collectively or individually, poses a serious problem in correction or has a "domino" effect on the other design features, or the approach poses a high risk without means for correction, or if the approach fails, a score of "3", "2,", or "1" should be assigned, with the lower score indicating a serious or severe condition. 6. If the major factors of the item are deficient to the extent that a major reorientation of the proposal is necessary, of if the approach taken is undesirable and correction would require a major and material change in the proposal, the item should be scored "0". This final example was developed for a specific RFP. While, at first glance, it seems to require a lot of up-front work, this might be a solid investment. The first time this type of scheme is used, it requires the evaluators to identify their specific requirements. However, in subsequent RFPs, this scheme can simply be revised to suit the particulars. For example, the characteristics of the Project Team in subsequent RFPs can be readily established by beginning with these definitions and then editing the text. A few years ago, the Ontario Government used the guidelines described in this section to identify the winner in an RFP[ii] for re-engineering the procurement function. Each of ten different factors was to be scored by each evaluator. The scores were then multiplied by the corresponding weights to determine the overall total score for each proposal. The evaluation guidelines contained both positive and negative indicators for each factor. This material was included in the RFP. Many organizations do not specifically identify these negative factors. The rest of this section contains the factors, the indicators and the weights assigned. Here are two of the factors:
The entire 400-page book focuses on how to create effective, low-risk RFPs. You will learn about best practices that will help you be successful.
|
|
||||||||||||||||||||||||||||||||||||
|
The Institute
Home | Order | SOW Training CD | Demystified CD | Six-Pack CD | Licenses | Biographies | Evaluation CD Michael Asner Consulting
Home | Order | RFP Report | Products | Contact | About Michael Asner Fairness Officer
Home | Fairness Articles | Fairness Scandals | RFP's/RFQ's | Fairness Consultants | Ethics | Elected Officials Guide
|
|||||||||||||||||||||||||||||||||||||
