About

I have been involved in forecasting for over 35 years and have seen what I can only describe as “crimes against statistics” over the course of my professional career. To give the reader an understanding of what I mean, consider the following example. The figure is a reasonably accurate facsimile of a rail freight demand forecast I came across nearly thirty years ago.

 

The report was written in 1990 and it is clear that, up until that point, railway traffic was in steady decline. However, demand was forecast to change direction in a way which I could only consider to be “miraculous”!

There is a long list of factors which will have influenced the decline in demand:

• • • Deteriorating economy;
    • Government trade policy (most of the traffic was international or transit rather than domestic);
    • Weak management;
    • Competition from substitutes (road in this case);
    • Quality of the infrastructure;
    • Price; and
    • Quality of service

 

Were the authors of the report really suggesting that many or most of these factors could turn positive in the space of a year or two and so reverse the declining trend? The answer is almost certainly “no”.

The list above contains factors which are beyond the control of the forecasters, but what about the forecasters themselves? A common error in forecasting is to introduce “optimism bias” and forecasts undertaken by the UK’s Office for Budget Responsibility (OBR) neatly illustrates this point.

The figure shows that every forecast undertaken by OBR has been too optimistic with regards to the future trajectory of productivity in the UK.  Such optimism bias is a widespread phenomenon.

Short-Term

As we have just seen, forecasting is an exercise prone to error, so why go to all the trouble? Indeed, one of my abiding memories from university is one of the lecturers saying that “you can’t be right but you can be wrong”. The African rail figure is a great example of being wrong!

First, many people assume that the core purpose of forecasting is to provide information which is as accurate as possible about future events. This is certainly true of some forecasts; if we see that tomorrow’s weather forecast shows a significant increase in temperature and brighter, sunnier weather than today, then the way we dress will be different from a forecast which predicts cooler and wetter weather.

In a business environment, short-term forecasting is necessary to schedule staff to meet expected customer demand and weather forecasts are critical in certain sectors, such as transport, just-in-time inventory management and assessing demand for weather-sensitive products and services.

As a rule, it is a reasonable expectation that short-term forecasting should be accurate. If patterns of trend and seasonality in the data have been established, then developing accurate forecasts ought to be fairly straightforward. However, short-term forecasts are subject to what can be termed “shocks” and, for any forecast, there is always going to be an element of randomness. Indeed, the philosophy behind forecasting techniques is to develop models which explain historical data as much as possible and hence minimise the proportion of variation which is attributable to the random behaviour of sales, traffic density, temperature, etc.

Long-term Forecasting

Long-term forecasting, by contrast, is important when undertaking investment. If a retail company is considering opening a new outlet, for example, it will want to ensure that the asset is profitable over its life. It will need to forecast demand, utility and HR costs over several years. Of course, all of these figures may turn out to be inaccurate – demand in particular is usually the most challenging element to forecast – but, nevertheless, it is the basis on which investment is appraised.

Forecasting for infrastructure is particularly challenging. The life of an airport, sewage system or power station is measured in decades rather than years and the further into the future forecasts are, the less accurate they will be.

Similar considerations apply to the development of a new product or service. A new prescription drug has research costs in excess of $2 billion and the Airbus-A350 series has reportedly booked some $15 billion to development costs. Of course, not all new products cost this sort of money and both big pharma and aircraft manufacturers are very large companies. So, a small company wishing to risk $1 million on a new product could face similar risks of financial failure as they have much less access to capital than larger firms.

Long-term forecasting is also required when there is a response lag; in other words, when an external factor can take time to makes its impact felt on a business. For example, a change in interest rates can affect the economy over a period of up to three years as the consequences of the change work their way through business decisions on whether to change their level of borrowing or mortgage holders seek a better deal.

Focus on Food

The extended example in this book is based on a fictitious supermarket chain. The reason for selecting this business sector is that it is one with which just about everybody is familiar and comes into contact with on a regular basis.

Many years ago (1995, since you ask!) I was in conversation with two shipping sector experts who told me that shipping is “special”. My response now would be the same as I gave them then, which was “every industry is the same as and every industry in different from others”.

Allow me to explain what I meant. Disparate industries will follow various business cycles; degrees of competition will differ, as will volumes traded internationally; extent of government intervention; speed of innovation, and so on. But it is precisely these factors which make all industries similar, at a high level at least: all businesses within a particular industry need to be aware of the environment in which they operate and the extent to which individual firms can influence or at least react to those macro-environmental factors.

Therefore, although one industry in particular has been highlighted in this book, the guidance provided can be applied to any firm or industry.

Readers familiar with the subject of business strategy will notice how many topics in this book come from that particular discipline. Having combined consultancy with academia for the better part of 20 years or so, I am particularly keen on the way that academic frameworks can be applied in the commercial world. Even when quantitative methods, statistics or econometrics are taught within a business school alongside non-quantitative subjects, the two subject areas are not particularly well linked.

To provide an example, I selected a plastic waste recycling company one of whose strengths was that “the company has excellent relationships with firms that collect and distribute PET bottles”. Assuming that these relationships are superior to those of competitors, how does this strength translate into a forecast? Will these superior relationships affect costs, revenues or both?

These are anything but easy questions to address but it would be unwise to ignore how softer issues affect future demand and profitability. After all, a company undertakes a SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis not as an academic exercise but as a means to build on its strengths relative to competitors, eliminate (or at least mitigate) relative weaknesses and assessing factors in the macro-environment which the firm can use to its advantage or against which it can build defences. The ‘missing link’ mentioned in the title of this section is how all these factors are related to forecasting.

The book covers both long- and short-term forecasting. Although the two timeframes have various quantitative techniques in common, the objectives of long-term forecasting are very different from short term variants.

This book is an attempt to bridge these divides.

Objectives of a Model

A model should ideally be simple yet comprehensive, capturing as many aspects of the real world which the model is intended to mimic. A model should also be accurate; clearly, accuracy cannot be promised for the future, but a model which is able to simulate the present and past as closely as possible will give confidence that the projections generated by the model will be acceptable. But what is meant by ‘acceptable’?

 Acceptability

A model which forecasts sales may well be delegated to a sales or marketing function within an organisation but developing and verifying a model are usually only the first steps in a process to get the forecasts adopted.

Depending on the size of a company, a model will normally require input from other functional areas of the business such as operations, purchasing, finance and so on; although marketing may take the lead in developing a model, the co-operation of other functions is often essential.

Once the model has been completed, the next stage of the process is to submit the model to a CEO or Board of Directors for their approval. It is unlikely that the model itself would be considered in detail at such a high level of the company, so a model also needs documentation.

A recent model developed by the author contained 85 separate sheets in an Excel file and the following figure provides a summary of how the model works, from inputs, via calculations, to outputs.

Model Flow Diagramme

This flow diagramme is part of a 70 page, 14,000 word document called the ‘record of assumptions’ (RoA). As the name suggests, the RoA contains the assumptions used in developing the model and forecasts but also a user guide to explain how the model can be used as well as key outputs from the model.  Documenting the model can be viewed as being as important as developing the model itself, since the documentation acts as a tool to persuade and convince stakeholders to “sign on” to the forecasting process proposed.