This extract deals with the fourth building block - Evaluating the Cost.

EVALUATING THE COST

Cost is a significant, critical, and sensitive issue. In RFPs, the award is made on the basis of best score, that is, best fit with all the requirements including cost. It is never made solely on the basis of cost. So, the winner is invariably not the least cost proposal. Hence, the value of the contract is an easy target for the disgruntled. In these times of budget constraint and cutbacks, it=s easy to politicize the process. In developing the RFP, always assume that your decision will be challenged, and prepare to answer questions such as: 'Why did you select that proposal when the second place one was almost as good and cost $200,000 less?'

There are several different approaches for incorporating cost into an evaluation. Whichever approach is used must reflect the priorities and the business case related to the project. Cost is almost always isolated from the technical and management parts of the proposal and submitted as a separate document. In many jurisdictions, the inclusion of any cost figures in the technical/management proposal is grounds for declaring the proposal non-compliant and eliminating it from further consideration. In this way, the Evaluation Team, which has been formed to deal with functionality and other issues, is not tainted by knowing the costs of various proposals.

While cost is usually analyzed separately, there is communication between the Evaluation Team and the Financial Team to ensure that the tasks underlying the costs are reasonable. Often, the Financial Officer will attend meetings of the Evaluation Team to obtain a better understanding of each proposal's approach and to ensure that all cost items have been identified.

Cost usually means cumulative cost, a total cost of all related activities, goods and services. In some jurisdictions, they use life cycle costing based on a nominal period of five years. Life cycle costs may included the start-up costs associated with a particular approach as well as the “off-ramp” costs (the costs of leaving) at the end of the contract. In other jurisdictions, they determine the costs over the contract period. In still others, they use an 'evaluated cost' based on features and requirements.

Usually, RFPs provide detailed directions in terms of the cost proposal. Increasingly, they provide forms or spread sheets to be completed and submitted both in hard copy and on a disk.

Many organizations do not even open the cost proposal until an analysis of the corresponding technical/management proposal has been completed. It is becoming a common practice to review the cost proposals only for those suppliers whose technical/management proposals have been reviewed and found capable of potentially providing an acceptable solution. Here is the wording for this practice from Yukon=s RFP guidelines:[i]

When the technical evaluation is complete, those proposals which meet or exceed the minimum acceptable score identified, will have the white, price envelope opened. Price will then be scored according to the evaluation criteria.

Price envelopes, for proponents who do not meet the minimum acceptable score, will be returned unopened.

Some organizations first ensure that the Technical/Management proposal has satisfied the mandatory requirements of the RFP. In other organizations, they evaluate the proposals and eliminate those which failed to achieve a pre-defined minimum technical/management score. AThere are 700 points for technical/management factors. Those proposals scoring less than 700 will be deemed unable to satisfy our minimum requirements and will be eliminated from further consideration. For those proposals, the cost proposal will be returned to the supplier unopened. For those proposals scoring at least 700 points, the cost proposal will be opened and evaluated.@

Different Ways of Handling Costs

There are many different approaches to handling costs.  Cost, as used in this section, means life-cycle costs: the total value of all costs associated with a proposal over the life of the contract or the life of the solution. Each different approach could theoretically yield a different "winner" from the same set of proposals.

The same proposal can "win" in one process, and not even be a serious finalist in another.

   Approach 1: The Best Solution Within Budget. If you are looking for the proposal which provides the "best solution" within budget, determine the score for all the non-cost factors. Then select the proposal with the highest point score that doesn't exceed the budget.

  Approach 2: Cost is Just Another Evaluation Criterion. In this method, cost is simply another factor which is included in the scoring scheme. For example, cost could be assigned 20 points. Based on the particular scoring scheme, points would be assigned to cost for each proposal.

There is a significant argument raised by many jurisdictions against assigning points to cost. These entities argue that it is inappropriate and misleading to rely on a mathematical formula dealing with costs rather than a well-reasoned analysis. Here is how the Federal Transit Authority views this issue:[ii]

The difficulties in trying to assign a predetermined weight to price and then scoring price proposals is that no one is smart enough to predict in advance how much more should be paid for certain incremental improvements in technical scores or rankings (depending on what scoring method is used). For example, no one can predict the nature of what will be offered in the technical proposals until those proposals are opened and evaluated. Only then can the nature of what is offered be ascertained and the value of the different approaches proposed be measured. It is against the actual technical offers made that the prices must be compared in a tradeoff process. Agencies cannot predict in advance whether a rating of “Excellent” for a technical proposal will be worth X$ more that a rating of “Good,” or whether a score of 95 is worth considerably more or only marginally more than a score of 87. It is what is underneath the “Excellent” and the “Good” ratings, or what has caused a score of 95 vs. a score of 87, that is critical. The goal is to determine if more dollars should be paid to buy the improvement, and equally important, how many more dollars those improvements are perceived to be worth. It could well be that the improvements reflected in the higher ratings are worth little in terms of perceived benefits to the agency. In this case the grantee does not want to get “locked in” to a mathematically derived source selection decision. This may very well happen when price has been assigned a numerical score and the selection is based on a mathematical formula instead of a well-reasoned analysis of the relative benefits of the competing proposals.

The calculation of a proposal’s points for cost requires two components: as assigned weight to reflect the importance of cost, and an approach for calculating cost.

Assigning Weights. The importance of cost is reflected in the number of points or percentage of the total points assigned to cost. Clearly, the larger the percentage of points given to cost, the more it influences the decision. (When all of the requirements are mandatory, and cost is 100%, the RFP becomes an Invitation To Quote.) Cost seems to range from 25% to 60% in most RFPs. In some jurisdictions, the minimum weight is determined by Regulation. It others, it is determined by the Project Team. In Alaska, the minimum weight is sixty percent:[iii]

Agencies are required to give a minimum weight of 60% for professional and non-professional services contracts, 75% for supply contracts, and between 60% and 75% for procurements involving a combination of both.

Determining the Score. There are several ways of determining the points or the score based on the costs of each proposal. If cost is included as one of the evaluation criteria, then we require some way of translating the dollar amount into a score. Suppose cost has been assigned 50 evaluation points out of a total possible score of 200. How many points does each proposal get? How are they calculated?

Here are three techniques that are commonly used. The first is based on the ratio of  costs of each proposal to the least expensive one. The second is based on the relative differences in costs among the proposals; the third, on an interval scale. In establishing a costing procedure care must be taken to ensure that an artificially low price can be accommodated as some bona fide suppliers may submit a low bid to obtain the work.

For each of these examples, let=s assume we have three proposals each with a different cost: A costs $300,000; B costs $250,000; and C costs $275,000. Let us also assume that cost is worth 100 points.

Ratio of Costs

Using the first method, the vendor with the lowest cost proposal receives all 100 available points.  All other vendors would receive a smaller number of points as determined by the ratio of their costs to the least expensive proposal.

Differences in Costs

The points are based on the differences in costs. Using the same data, we first determine the difference in cost between the least cost proposal and the one under consideration. We then express this difference as a percentage of the lowest cost proposal.

Points per Interval

In this method, all proposals within the same range of costs receive the same number of points. For example, those within 10% of the lowest price, receive 100% of the points. Those proposals whose costs are between 10% and 15% greater than the lowest cost receive 80% of the points. Between 16% and 30% greater, 60% of the points.

  Approach 3: ‘Bang for the Buck’. In this approach, we use the concept of value – points per dollars. Each proposal is evaluated and a score established for it. The score excludes any considerations of cost. Once this has been completed, the Total Score for each proposal is divided by the Total Cost to obtain a "points per dollar" measurement of the proposal. The Proposal with the greatest "points per dollar" represents the greatest value and is selected. Cost is usually the life-cycle or total contract cost.

Let=s assume that for Proposals A, B, and C, the scores for the technical/management parts were 650, 730, and 800 respectively. Now let=s look at the calculation:

Using this approach, proposal B would be selected as it provides the greatest value.

  Approach 4: Two Steps - First Merit, Then Cost. This approach represents a workable trade-off between the often divided members of the Evaluation Committee. Typically, the technical people want to select the proposal with the strongest technical appeal, regardless of the cost. The finance people, on the other hand, are not terribly concerned with the technical issues and simply want to spend as little as possible. They often think “least cost” and forget that this is an RFP process.

Also, senior management often focuses on budget. They question selecting the most expensive proposal. It is awkward and difficult to explain to a senior manager or a politician why you didn’t select Proposal D over proposal A. Proposal A got 86 points and costs $250,000. Proposal D got 82 points and costs $200,000. What did you get for the extra 4 points? Is it worth $50,000 which could be applied to a currently unfunded high visibility project?

Identify Acceptable Proposal

In this two step approach, we first evaluate the merits of each proposal (but not cost). The Evaluators eliminate any proposals which do not satisfy the organization's mandatory requirements. A mandatory requirement may be the ability to service 500 user terminals concurrently, or the ability to provide a particular set of applications programs. Each of the remaining proposals is evaluated and a score established for the technical and management parts. We now have a table of proposals and their technical and management scores:

In discussions with the technical and management stakeholders, they agreed that any proposal scoring more than 72 points was capable of satisfying their requirements. Now that knew that a 90 point proposal would be superior to a 75-point proposal, but they were prepared to accept anything above a 72. This agreement was reflected in the RFP which stated that any proposals scoring less than 72 points would be judged as not capable of providing an acceptable solution and eliminated from further consideration.

On this basis, Company C and E were eliminated from further consideration. The remaining three proposals were deemed as potentially acceptable.

Select the Least Cost Proposal

The cost for each of the remaining proposals was then established.

The selection is made on the basis of least cost. That is, select the proposal which was acceptable based on the technical & management score and costs less than the others as determined by the life-cycle cost.

In this analysis, D is the winner. It was acceptable based on the evaluation of the technical and management factors. And it costs the least of all those that were acceptable.

 _________________