The key decisions to make at this point are:
- At what level of the pathway (i.e. outcome or impact) to collect data; and
- The most appropriate indicators and metrics for measurement at the chosen levels of the pathway.
These decisions will be driven by a combination of:
- The objective of the analysis (as determined in Steps 2 and 4);
- The impact or dependency pathway (as identified in Step 6); and
- The informational requirements of the valuation approach (as identified in Step 7).
For example, if the objective of the analysis is monetary valuation of several different impacts so that their relative size can be compared or so that they can be aggregated, it will be important to define and measure indicators at the same level of the pathway (i.e. all at the impact level). The nature of these indicators will be determined by the monetary valuation technique chosen to value them and it is important that the indicators will support comparable monetary valuation techniques (i.e. they should not represent a mix of value perspectives such as one value to society and one cost/benefit to the business).
When choosing the most appropriate indicator to
measure changes in social capital, companies should consider whether:
- An indicator is good quality;
- To use proxy measures or not; and
- To use baselines or counter factual scenarios, or neither.
Identify a good quality indicator
Indicators consist of information that signals change. An indicator can be quantitative or qualitative. Ideally, it should provide a simple and reliable means to reflect the changes connected to an activity or intervention. This means that it must be relevant to the changes that are expected, and sensitive or granular enough to reflect the expected magnitude of these changes.
There are various sources of detailed guidance about what good quality indicators and metrics look like40 but, put simply, effective indicators can be thought of as having five ‘SMART’ characteristics:41, 42, 43
- Specific. Indicators should reflect simple information about what is being measured, without being affected by other factors, and should be easily understood and easy to communicate. Is it clear exactly what is being measured? Has the appropriate level of disaggregation been specified?
- Measurable. They should be objectively verifiable. Are the indicators objective? Are the indicators verifiable? Are they reliable and clear measures?
- Attainable. Indicators and their measurement units must be attainable: not impractically time-consuming or expensive to collect, and able to withstand/be sensitive to changes in context. What changes are anticipated as a result of the activity? Are the changes of a scale that is measurable by these indicators?
- Relevant. Indicators should reflect information that is meaningful. Does the indicator capture the essence of the desired result? Is it relevant to the intended outcome and impact?
- Time bound. Progress can be tracked at a desired frequency for a set period of time. Is there a clear timeframe for the indicator?44
Applying the SMART characteristics sounds simple, but many social indicators that are commonly used by companies lack alignment with these characteristics, particularly with regards to specificity.
In many cases, indicators are too vague or too subjective and hence open to discretionary use. Sometimes this is an issue around definitions, for example clarifying what is considered a ‘minor’ or ‘serious’ health and safety incident. In other cases it is the use of ratios or percentages that lack details - for example, total number of incidents vs. total number of incidents per 10,000 people.
Decide whether to use proxy measures
Measuring can be a challenging and costly endeavour. Measuring “impacts” in the technical sense is difficult to do, due to the length of time it can take for impacts to materialize, influences beyond business activities that affect the impacts measured, and the need for data outside of the scope of business operations. Companies often focus on measurement at the outcome or output level as a proxy for impact, and use data modelling techniques to understand what their longer-term impacts might be.45 Organizations must be judicious in their use of proxy indicators.46 Outcomes are stronger proxies for impacts because they are one link closer in the results chain, but neither output nor outcome proxies are guaranteed.
Decide whether to use baselines and counterfactual scenarios
At this point when developing indicators, companies should consider whether the change in social capital they wish to measure can be meaningfully measured ‘by itself,’ or whether it needs to be measured with reference to some kind fixed point: either a baseline or an alternative scenario (also known as a ‘counter factual’).
In some cases, it may be sufficient to measure a change in social capital without reference to a fixed point. For example, a company can measure its impact on employment in terms of the number of direct jobs it supports, and the data is likely to be readily available from the HR department.
But this does not measure whether that employment has changed over time, or whether those individuals are likely to have gained employment anyway. To measure these kinds of changes in social capital, comparison to a baseline or counter factual scenario is necessary.
These two terms are used widely in the field of impact measurement and Life Cycle Analysis (LCA), and can be defined in different ways. For the sake of simplicity and comparability, we base the following explanations on those set out in the Natural Capital Protocol.
- Baselines: A baseline is the starting point or benchmark against which changes in social capital can be compared. For most assessments, an explicit baseline is required to enable meaningful conclusions to be drawn. The type of baseline will vary depending on the nature of the assessment, and can include:47
- The historical situation during a specific period of time, such as a comparison of this year’s number of health and safety incidents relative to last year’s (the ‘baseline year’).
- The state of social capital (e.g. employment level) at a point in time, for example, the state immediately before a project began.
- A sector-wide or economy-wide average level of a given social capital impact or dependency (i.e. an industry benchmark for salary level).
- Counter factual scenarios: A counter factual describes a plausible alternative state that would have resulted if the business activity or intervention had not occurred. This may be achieved by measuring, or estimating, consecutive change over the same time period in a comparable population or control group who did not benefit from the intervention. If a suitable counter factual is available, this can add significant credibility to the measurement results, in particular to help justify a causal relationship between an activity or intervention and the change in social capital that is being measured. However, these techniques can add to the time and cost required for measurement and are not always feasible for a company to conduct.
When undertaking an assessment that covers an extended period (e.g. to assess the impacts of a project over 20 years), the assessment will need to take into account how the baseline social capital would have changed anyway in the absence of the company’s activities. This is the challenge of proving the ‘net’ benefit of an intervention and is known as ‘additionality.’48 Additionality is calculated by the gross benefits less the benefits that would have occurred in the absence of the intervention (the ‘deadweight’49) less the negative impacts elsewhere (including ‘displacement’50 of activity).
Choose balanced and transparent metrics
Defining standardized social metrics and gaining access to appropriate and reliable data, in a way that is practical, affordable, and pragmatic for business, remains a huge challenge. The following are recommendations for strengthening existing indicators and metrics used by companies and moving towards standardization:
- Take a balanced view of positive and negative impacts: There is often an assumed positive impact in the way many social impact indicators are designed and often, negative consequences are not taken into account. Many indicators use “increases” or “improvements” but ignore some of the risks of negative direct or indirect outcomes for different stakeholder groups.
- Take a view on the attribution of impacts (where relevant): For some social capital issues, there may be many different actors who have contributed to a change in the state of the social capital (e.g. a training program funded by multiple parties). In this case it is important to acknowledge that not all the social capital impact is directly attributable to the company. In some instances, acknowledgment of the attribution issue might be sufficient; however, in other instances it may be possible to use some method to attribute between parties - e.g. the percentage split of financial investment provided to the training course by each party.
- Disclose calculation methodologies: Ideally, companies should provide information about the methodology employed to calculate an indicator. This can be a major contributor in the drive towards increased standardization and comparability. Not only does it increase accountability and transparency, it also supports the potential for increased awareness of best practices and their wider adoption among companies.
- Disclose assumptions: It is important to disclose the assumptions that were used throughout the analysis and therefore the limitations of application of the analysis. Being open about the limitations is likely to increase credibility among stakeholders and facilitate learning and collaboration.
The following illustrative impact pathways for the subjects of skills, employment and safety include examples of qualitative and quantitative indicators at different levels of the pathway, along with the relevant ‘value perspective’ and illustrative data sources.51
40 For example, see the UNEP/SETAC, Guidelines for Social Life Cycle Assessment of Products or the Roundtable for Product Social Metrics, Handbook for Product Social Impact Assessment version 3.0
41 Search for Common Ground, Designing for Results: Indicators.
42 UNDP, Selecting indicators for impact evaluation
43 World Bank, Designing a Results Framework for Achieving Results (p.33).
44 These explanations also benefited from research conducted by MBA students as the European School of Management and Technology
45 WBCSD, Measuring socio-economic impact: A guide for business.
47 The Natural Capital Coalition, The Natural Capital Protocol
48 ‘Additionality’ is the extent to which something happens as a result of an intervention that would not have occurred in the absence of the intervention (UK Treasury Green Book)
49 ‘Deadweight’ refers to outcomes which would have occurred without intervention (UK Treasury Green Book)
50 ‘Displacement’ measures the extent to which the benefits of a project are offset by reductions in benefits elsewhere (UK Treasury Green Book)
51 Taken from WBCSD, Building the Social Capital Protocol: Insights into employment, skills and safety.
* Asian Agri, is a leading Asian palm oil producer, and like APRIL it belongs to the Royal Golden Eagle (RGE) group of companies