Cost-benefit analysis guide

Decision-making is huge part of our everyday lives. Some decisions feel almost automaticothers require immense consideration. We understand how challenging decision-making can be. 

NineSquared’s mission is to help our clients make great decisions that make people and places better. We do this by using economics and data to give them the knowledge and insights they need to make decisions with confidence 

We have created a short non-technical guide that explains how cost-benefit analysis (CBA) can be used for decision making. The guide explains what CBA ishow it can be used and the key components of undertaking a CBA. It explains how to interpret CBA results, and some pitfalls you should look out for when undertaking a CBA. The guide also briefly outlines other economic analysis techniques such as Cost-Effectiveness Analysis (CEA).  

Not everyone is an economist. The guide is aimed at anyone who needs to know more about tools and techniques for comparing options and making decisions about projects and policies. It can be read by anyone from a senior executive with limited background in economics or direct experience in undertaking a CBA, to a junior team member who is new to project planningWe hope the guide gives you a good understanding of how CBA works, and helps you to make great decisions. 

Economics can help make great decisions.


Cost-benefit analysis guide

Download the NineSquared cost-benefit analysis guide.

Overview

What is it?

At its simplest, Cost-Benefit Analysis (or CBA) is based on the idea that any new project or policy by the public sector should contribute more to society than it takes away. To assess this, CBA attempts to sum up all of the benefits and all of the costs associated with a new project or policy to see if the benefits are greater than the costs over the life of the project or policy.

Simple enough. However, CBA attempts to sum up not only the financial costs and benefits, things like construction costs and savings that a person might gain from a specific policy setting, but also the social costs and benefits to arrive at an evaluation of the overall impact on social welfare. To do so, CBA considers costs like the reduction in air quality that might result from a project that creates pollution, or perhaps the social benefits of reducing deaths and injuries because a road has been made safer. These costs and benefits cannot typically be observed in the market place. These social costs have to be estimated using techniques that have been developed by economists over many years. We outline some of these techniques within the guide.

Why use it?

Policy, program and project evaluation
CBA is useful when considering whether or not a particular course of action will have an overall positive or negative economic impact on the community. A course of action might be a new policy or program that is being developed, or perhaps a new infrastructure project. An evidence based evaluation, founded on a consistent and methodologically sound approach, means that decision-makers can approve or reject proposed changes with greater confidence that their decision will enhance social welfare rather than reduce it.
Post completion reviews
CBA is also a useful tool for evaluating the impact of an existing policy, program or infrastructure project. Post Completion Reviews or ‘ex postanalysis of a policy or program assesses the actual costs and benefits that occurred as a result of the change compared with the costs and benefits that would have occurred without the change being implemented. Such reviews can provide decision- makers with a better understanding of project outcomes and assist in the evaluation of future projects.
Project prioritisation and selection
Because CBA provides an evaluation methodology and reporting metrics that are applicable to a variety of different policies, programs and projects, it can be used to prioritise projects in relation to each other. This enables decision-makers to select the highest value policy, program or project from those under consideration.
Early project option assessment
Often in seeking to address a specific issue or problem, a project team may develop a substantial number of options. These may address the problem in very different ways. Detailed development of every option may be prohibitively expensive. Some options, however, may be able to be excluded early in the option development process by applying the principles of CBA in a consistent but shortened version of the technique, known as Rapid CBA. Rapid CBA can allow project teams to determine those options that are unlikely to have a positive impact on social welfare early in the option development process allowing project teams to focus more time on viable options.

Chapters

Conceptually simple, analytically complex

While conceptually simple, preparation of a CBA can be analytically complex. It may, for example, require the forecasting of future demand for a product or service or the valuation of costs and benefits that are not traded in the marketplace and for which, as a consequence, no market price is observable. It may require consideration of external costs and benefits, and how these should be treated and valued.

Then there are considerations about what the outcomes of a project should be compared to, how to treat costs and benefits in the future to ensure that double counting is avoided, and how risk should be incorporated into the analysis.

This section outlines the steps we take when undertaking a CBA. This overview is not intended to provide a detailed coverage of these issues. For those who would like more detailed guidance, there are several good overviews of the matters outlined above, particularly in the context of major project evaluation in Australia.

Starting the CBA – Specifying the base case

CBA compares an expected (or actual in the case of a Post Completion Review) project outcome with what would have happened if the project, policy or program was not progressed. This counter-factual is typically referred to as the ‘base case’.

The base case is the benchmark against which all other options are compared. The definition or specification of the base case underpins the accuracy of the remaining analysis. In effect, the base case maintains the existing level of service.

Importantly, the base case is not a ‘do nothing’ option. Rather it is a statement of the expected outcome of the current course of action, together with any announced or required spending to maintain that course of action into the future. Such spending might, for example, include requirement maintenance spending to ensure that a facility is maintained to current levels of service. Mis-specifying the base case will cause the CBA to produce misleading results.

Identifying costs and benefits

CBA considers the social costs and benefits of a proposed policy, program or project. As a result, many of the costs and benefits considered in the CBA that apply to producers, consumers, and the community more generally, are not typically considered in the financial assessment.

Different policies, programs and projects will result in different costs and benefits so each CBA will necessarily consider a different set of costs and benefits. The first step in valuing these is to identify and categorise each impact that is material to the analysis. Again, while conceptually simple, there are adjustments that need to be made. For example, CBA does not typically consider taxes as a cost or a benefit as they are simply a transfer between taxpayers (either producers or consumers) and the Government. However, there are some inputs that might be important in a CBA that include taxes – fuel for example includes fuel excise, and other taxes and adjustments will need to be made.

Estimating demand

Once the categories of costs and benefits have been identified, it is necessary to estimate the change in the cost or benefit that will result from the proposed policy, program or project. In many cases, this requires forecasting changes in the demand for something.

Forecasting demand requires estimating the change in consumer behaviour (and sometimes producers behaviour) over the analysis period. In some cases, the expected behavioural response from a change, in some parameter, has already been measured and codified in widely accepted guidelines, such as those that exist in Australia for transport project evaluation. In other cases, it will be necessary to undertake statistical and econometric analysis. This may include a consideration of demographic factors, the likely behavioural response to a change in service level or price (or both) and the range of alternatives available to consumers and producers (for example, people may decided to drive when bus fares increase).

Monetising costs and benefits

Once there is an understanding of the categories of costs and benefits, and the change in project impact (for example as a result of the future changes in demand), it is necessary to monetise the costs and benefits. In some instances, this is a relatively simple process. Project capital costs, for example, are usually already expressed in dollar terms. In many instances, however, costs are not expressed in dollar terms.

The value of a person’s life, the benefit of improved air quality, and the cost associated with traffic congestion, are all examples of intangible costs and benefits that require some consideration as to their dollar value if they are to be included in a CBA. To monetise these, economists have developed a number of techniques. These include estimating the value of something based on the value of a related thing. For example, one approach to the valuation of a human life is to link its value to a person’s lifetime earning potential, known as the Human Capital Approach.

The Human Capital Approach is one type of valuation methodology known generically as Revealed Preference (RP). In this case, people reveal their preferences by requiring more money for more dangerous jobs. Similar approaches can be used to estimate the value of, say, national parks, through consumers revealing how much time, effort and money they are willing to spend to travel to a park in the first place.

In some cases, it is not possible to examine consumers revealed preferences. In this case, monetising costs may require using a Stated Preference (SP) Methodology. In the SP approach, consumers are surveyed and asked about their willingness to pay for something (say, a park or improved health outcome). Alternatively, they may be asked about how much they would need to be paid in order to accept something (for example, how much would they need to be paid to accept increased congestion on their commute to work).

These approaches can be expensive and complex to undertake. There is a substantial body of work that has been undertaken in how to structure and conduct RP and SP surveys, and how they should be analysed and interpreted.

Because of the complexity and cost associated with RP and SP methods, they may not be feasible or financially viable options for smaller CBAs. In these cases, consideration may be given to methods that provide an indicative and indirect valuation of non-monetary costs and benefits. One approach is to estimate the replacement cost of an unpriced asset. What would it cost, for example, to replace a playground or a park. Another method known as Benefit Transfer is to use valuation estimates from a different study or analysis and transfer those estimates to the CBA being undertaken. This approach is appropriate only when the characteristics between the two studies are sufficiently similar to allow comparison.

Determining the analysis period

Choosing an appropriate analysis period is a important consideration in the CBA design. In theory, the analysis period should be consistent with the expected life of a project asset, policy or program.  For infrastructure assets, the expected life is typically assumed to be the operating life of the asset measured from the first year in which the asset starts to produce benefits. Different assets will have different operating life spans, there are some guidelines that outline the lives of specific asset classes.

The appropriate analysis period for policy and program evaluations is less clear. In theory, the analysis period should be the life of the policy. However if the proposed policy settings provide for a sunset clause, the ‘operating life span’ of the policy may be unclear.

In both cases judgement needs to be applied to ensure that the analysis period is not too short, so that it misses material benefits beyond the analysis period, nor too long such that accurately forecasting benefits and costs becomes impossible.

The discount rate

Another important analysis design question is the selection of an appropriate discount rate. Discount rates are used to ‘discount’ future costs and benefits. This allows the CBA to take into account the time preference for money (that is the concept that people would value a dollar today more than they would value a dollar in a year’s time) and risk.

It is important to ensure that the correct approach to the discount rate is followed. Typically, the discount rate used in economic CBA is different to that used in a financial appraisal, which is often more focussed on the cost of capital. Discount rates used in economic CBA is also usually a real discount rate reflecting the fact that the appraisal is often undertaken in real rather than nominal dollars (real dollars are those that have been adjusted for inflation, while nominal dollars have not been adjusted and represent prices at the current time).

Discounting future costs and benefits means that, at high discount rates, future benefit (and cost) streams will become less important as they get further away from the present day. This can have implications for project appraisal when there are large upfront costs and a long stream of benefits into the future.

Sensitivity testing

Testing the results of the CBA to key inputs is an important part of the CBA appraisal. Sensitivity testing is the ‘flexing’ of inputs to determine the sensitivity of the results to those inputs. The strength of the result of the CBA can then be assessed by decision makers against how sensitive it is to risk and uncertainty surrounding critical inputs.

Typically, sensitivity testing might be undertaken on a range of inputs and parameters. These can include the discount rate, the costs and benefits used in the analysis and the analysis period. Additional sensitivity test may be undertaken in relation to different components of the CBA depending on the subject matter – for example, a transport CBA may assess the sensitivity to traffic demand forecasts.

Sensitivity testing typically involves inputs being flexed individually, so that the impact of each individual input can be assessed. It is also possible to combine the flexing of inputs to provide decision makers with a ‘worst case’ and ‘best case’ scenario.

Costs and benefits can be thought of as impacts on producers, consumers or third parties. Different projects and policies will have differing impacts across these groups and each CBA needs to consider whether specific costs and benefits are material to the analysis being undertaken.

Private costs

  • Initial project capital costs
  • Project operating costs – operating and maintenance costs over the analysis period
  • Capital replacement costs – cost of replacing assets during the analysis period, for example, replacement of IT systems during the analysis period
  • Decommissioning and rehabilitation costs – costs associated with the decommissioning of existing assets and services and rehabilitation of the environment
  • Other costs incurred directly or indirectly by the project, such as by other government agencies

Private producer benefits

  • Increased operating revenue – the economic value from changes in revenue to the owner or operator
  • Increased ancillary revenue – the increase in revenue from other activities that may accrue as a result of the project. For example, revenue from airport retail concessions or advertising revenue
  • Avoided capital costs – costs avoided as a result of the project. For example, capital acquisitions that are no longer required
  • Avoided operating cost – for example, savings in maintenance, compliance and investment costs

Private consumer benefits

  • Improved accessibility – for example, lower cost of accessing essential facilities such as hospitals or improvements in access to services and infrastructure
  • Travel time savings – the value of reduced journey time
  • Savings in vehicle operating costs
  • Service reliability – the value of improvements in reliability
  • Service improvement – the value of greater amenity from better services
  • Health and safety – the value of a reduction in crashes, deaths, injury, disease and security incidents
  • Resilience – the benefit of improved resilience to adverse events
  • Residual value of assets – the residual or terminal values as a proxy for future user benefits beyond the analysis period

External costs and benefits

  • Environmental externalities – positive and negative impacts on the environment as a result of the project. This may include reductions or improvements in air quality, carbon emissions, water pollution, noise
  • Network externalities – changes in user behaviour may have broader impacts and result in congestion and health and safety issues elsewhere in a network
  • Land use impacts – benefits and costs associated with changes in land use as a result of a project
  • Health and safety externalities – costs or benefits accruing to third parties as a result of a policy or project. For example, governments may experience a reduction in health expenditure due to a reduction in accidents as a result of a project
  • Other social impacts – for example, a freeway project bisecting a community may reduce mobility within that community and impose costs that are not captured elsewhere

Cost-benefit analyses typically provide a number of key measures for decision makers to assess the economic merit of the policy, program of project under consideration. These measures include:

  • Net present value (NPV)
  • Benefit-cost ratio (BCR)
  • Incremental Benefit Cost Ratio (IBCR)
  • Net present value per dollar of capital invested (NPVI)
  • First year rate of return (FYRR)
Net present value

The net present value (NPV) is the sum of the discounted project benefits less the discounted project costs. When the benefits exceed the costs, the net present value will be positive. This indicates that the project has economic merit.

Benefit-cost ratio

The benefit-cost ratio (BCR) is the ratio of the present value of the benefits to the present value of the costs. Where the NPV provides an indication of the total amount of the net benefit, the BCR provides an indication of the benefit for each dollar spent on the project. A project, policy or program is potentially worth pursuing if its BCR is above 1.

Incremental benefit-cost ratio (IBCR)

The incremental benefit-cost ratio (IBCR) can be used to compare the base case of two or more mutually exclusive alternative projects. The IBCR is calculated as the ratio between the difference in cost between the incremental costs and the incremental benefits of one project option with the base case or another project option. The IBCR measures the return on each additional dollar spent on a project option compared to the base case or another project option. The option with the highest IBCR is, conceptually, the economically preferred option.

Net present value per dollar of capital invested

The NPV per dollar of capital invested (NPVI) is a measure of the overall economic return of a project in relation to its requirement for fixed capital. It is generally used to rank projects in a budget constrained environment.

First year rate of return

The first year rate of return (FYRR) is a measure of the value created by the project in its first year of operation. It can be used to determine whether a project should start as planned, be deferred or brought forward. A FYRR that is below the discount rate may suggest that a project should be deferred. Conversely, if the FYRR is substantially higher than the discount rate, then it may be an indication that it should be brought forward.

 

 

 

 

 

 

 

Double counting

Double counting (particularly the double counting of benefits) can often arise in the development of CBAs. Double counting can occur in a number of ways:

  • Firstly, there can be double counting when an SP survey has been undertaken which has been designed to capture participants’ willingness to pay for something which has several component characteristics. The error occurs if one of the components is counted again separately in the analysis.
  • Secondly, double counting can occur when a value is based on the Benefit Transfer Method and there is a lack of understanding of the components that were included in the original valuation. For example, a disability adjusted life year may include a reduction in labour productivity so it is not appropriate to include lost productivity again in the analysis.
  • Thirdly, the economic value may be embodied in the price of an asset but may be double counted as a separate benefit. For example, if a new road project resulted in an increase in house values due to the lower commuting cost. The lower commuting cost is already reflected in the house price and should not be counted again.
  • Finally, some things may look like benefits but simply reflect activity transferred from somewhere else. For example, a new dam may lead to an increase in recreational activities at the dam but this activity may have occurred elsewhere anyway.

Not excluding an irrelevant item

CBAs represent an economic appraisal of a project, policy or program. As such, they exclude a number of items that might otherwise be included in other appraisals (such as economic impact assessments or a financial appraisal). Items that should be excluded from a CBA include:

  • Sunk costs – such costs are past costs which are irrecoverable. CBAs relate to new expenditure and as a result all past costs should be excluded.
  • Depreciation – depreciation is an accounting treatment for the allocation of the cost of a capital asset over its estimated useful life. Including depreciation in the CBA would double count the capital cost of the project, policy or program.
  • Interest – Interest costs are implicitly included in the discount rate used to discount future benefits and costs to the present day. Including them explicitly in the CBA would double count the impact of interest rates.
  • Transfer payments – transfer payments are payments between groups that do not involve any transfer of economic resources. Taxes are an example of transfer payments. Because the CBA is an economic appraisal, these payments should be excluded from the CBA analysis.

Confusing costs as benefits

While allocating costs and benefits sounds like it should be simple, in practice this is not always the case. A common mistake is to allocate increases in employment as a result of a project as a benefit. Employment is generally a cost to a project, in much the same way that the capital required to be invested in a project is a cost.

Decision bias

Decision bias can occur when a decision has effectively been made before properly considering all options. In this situation, it is common to find overly optimistic benefits and/or under estimated costs. An economic appraisal should not be a business case, which typically promotes a preferred approach, but it may be included in a business case to explain how a preferred approach came to be selected. Decision bias can be somewhat mitigated by ensuring that cost and benefit estimates are from a credible, informed and recent sources. Sensitivity testing can play a useful role in determining whether a proposal is overly optimistic and the risk of a project being a poor investment.

Benefit bias

More effort can sometimes be spent on considering all possible benefits with less attention paid to identifying all the costs of a proposal. This will bias the analysis of project options as it will alter the net benefit of the project. This can be linked to decision bias, as some costs may be purposely omitted in order to make a project look more attractive.

Adapted from the Infrastructure Australia Assessment Framework, March 2018

Cost-effectiveness analysis
Cost Effectiveness Analysis (CEA) shows the costs of achieving a given outcome. That is, CEA compares projects with the same or similar outputs to the costs associated with achieving these outputs. As a result, CEA finds the most cost-effective way that an outcome can be achieved.

CEAs are useful when it is not possible to assign monetary value to all costs and benefits. It is an effective tool for agencies with strong social welfare objectives, particularly for those that operate in health, education, or other areas where benefits cannot easily be quantified.

CEAs are also useful where the need for the project itself is pre-determined but there are different options for delivering it. For example a replacing a deteriorated  wharf that supports essential transport services that cannot be delivered in any other practical or cost effective way.

This technique requires projects or options to have the same or similar benefits to properly compare them, and as a result, cannot be used to compare projects with different objectives.

CBA or CEA?

CBA and CEA are two of the most commonly used tools to perform economic analysis and are essential to performing an economic appraisal.

CBA quantifies all of the costs and benefits associated with a project into monetary terms. By monetising the costs and benefits, all projects can be assessed in a consistent manner. This is more straight-forward when there are significant financial transactions in a project, however it can be difficult when a project delivers economic values that are non-monetary. CBA provides standard outputs and measures, such as a benefit cost ratio (BCR) or net present value (NPV), that indicate the economic viability of a project and can be easily compared with other project options.

In some Australian jurisdictions, a CEA should only be used as a supplementary approach to CBA because it does not assess the net impact on social welfare.

CBA can compare a wider variety of projects in a consistent manner, and is therefore generally preferred over CEA. However, CEA is used when there are highly substantial non-quantifiable benefits that would make a CBA too complex or time-consuming. Most projects will have a range of benefits, many of which may not be quantifiable. A decision must be made as to whether these benefits are critical to the project, and therefore a CEA should be performed, or if they make up a smaller portion of the benefits and can therefore be qualitatively analysed within a CBA.

The decision to perform a CBA or CEA will also depend on any further analysis being performed. CEAs cannot be easily compared to other projects unless they have similar benefits, and hence may not be well suited to an economic appraisal that compares different projects to find the most efficient use of funds.

Incidence analysis
Incidence analysis disaggregates the overall impacts of a project or policy and considers the impact on certain individuals or groups. Typical techniques such as a CBA consider only the overall impact of a project which can make certain groups worse off to make the majority of people better off. Incidence analysis considers those who are most impacted by a project and provides information to decision makers about the distribution of benefits and costs. Factors such as the socio-economic status of these groups are often considered to determine whether actions are needed to compensate for any negative impacts of a project or policy.
Input-output (multiplier) analysis
Input-output analysis is used to assess the regional impacts of a project or policy. This involves multipliers being applied to measures of direct impact to estimate the flow-on effects of a project, such as income and employment. Multipliers measure overall linkages between an industry and the remainder of the economy. Input-output analysis measures the potential economic activity resulting from a project, however it is not a measure for project evaluation. It can return a positive result regardless of whether the project has a net benefit and is successful in meeting its objectives because even a poor investment can stimulate economic activity. Concerns can sometimes be raised with this technique because it relies heavily on estimation, and can easily be based on poor quality data. However, Input-Output Analysis can still be useful to get an approximate idea of a project’s flow-on effects.
Economic impact assessments
Economic impact analysis estimates the effect that a project will have on the structure of the economy, or on the welfare of groups of people or organisations. This is usually expressed in terms of employment and income effects, broken down by economic sector and/or region. All projects have economic impacts, primarily because they generate employment, however this is usually not the primary objective of a project. Economic impact analysis does not relate the expected benefits of a project to the costs involved. This technique has many similarities to input-output analysis and input-output analysis may be used as a tool to complete an economic impact assessment.
Multiple objective programming
Multiple objective programming uses mathematical programming techniques to select projects or policies based on explicit objectives. This tool is particularly valuable in the assessment of project with multiple objectives that cannot be monetarily quantified. When projects have multiple, sometimes conflicting objectives, there often isn’t a single solution that satisfies all requirements. Multiple objective programming assesses projects and project objectives, and presents the solution that will most optimally satisfy objectives.
Economic impact assessments
Economic impact analysis estimates the effect that a project will have on the structure of the economy, or on the welfare of groups of people or organisations. This is usually expressed in terms of employment and income effects, broken down by economic sector and/or region. All projects have economic impacts, primarily because they generate employment, however this is usually not the primary objective of a project. Economic impact analysis does not relate the expected benefits of a project to the costs involved. This technique has many similarities to input-output analysis and input-output analysis may be used as a tool to complete an economic impact assessment.

Economic analysis is commonly confused with financial analysis. While the two share many of the same principles, there are some key differences:

–    A financial analysis is usually performed from just the perspective of the entity undertaking the project or delivering the service, and does not consider the wider economic implications the project has on society. Almost any major project undertaken will have some degree of impact outside the scope of the operating entity – it is this impact on the wider community that an economic analysis aims to measure.

One example may be a mining firm exploring a new site. The firm itself will consider internal costs and benefits such as the required investment, ongoing costs, predicted revenues and taxes; whereas in reality the scope of costs and benefits extends beyond the firm and has implications on the environment, employment and nearby residents. Furthermore, a financial analysis does not consider the opportunity costs of a project. Economic analysis considers the costs and benefits to everyone in society, including implicit costs.

–    Financial analysis uses a different discounting factor (i.e. discount rate) to quantify investments at present value. The financial discounting factor is generally based on the cost of capital that accounts for the specific risks of the project owner and uses nominal values (i.e. includes inflation). Economists use a different discounting rate that reflects the investment risks to the wider community and is also presented in real terms (i.e. excluding inflation).

–    A financial analysis uses capital and recurrent costs that are in nominal terms (i.e. includes inflation) whereas an economic analysis uses costs that are in real terms (i.e. excludes inflation).

Copyright 2017 NineSquared | All Rights Reserved