The Bank of Canada’s Business Outlook Survey (BOS) asks participating firms several questions about their sales volumes.1 These include questions about the evolution of firms’ recent sales and their expectations for future sales. Firms’ responses, combined with their related comments, give a picture of the state of demand in the Canadian economy.
In the first quarter of 2023, Bank staff removed the question on past sales growth from the BOS questionnaire. To calculate the BOS indicator, they replaced the results of that question with those of a question on whether past sales have declined over the past 12 months. This backgrounder outlines the reasons for the change and discusses its implications.
Questions about past sales
The question on past sales growth that was removed from the survey was the longest-standing question on past sales. It asked:
(1) Over the past 12 months, did your firm’s sales volume increase at a lesser rate, the same rate or a greater rate as over the previous 12 months?
This question was intended to gather context that is helpful in interpreting questions about firms’ sales outlooks. In general, growth rate questions in the BOS are designed to capture whether changes in a particular variable are accelerating or decelerating. During extended periods of stable economic growth, these questions generate more variation in the series than questions about sales levels do.2
Question 1 above was difficult for some firms to answer because it required them to remember sales levels from previous years and estimate their historical sales growth. This exposed the survey to possible recency bias caused by respondents’ incorrect or incomplete recollection of the changes in economic variables from an earlier period. In a fast-changing economic environment, the risk of recency bias is greater, and it makes identifying the true economic signal challenging. In addition, this difficult question was asked at the beginning of the survey. Occasionally, firms misunderstood whether they were being asked about changes in growth rates or levels, and the interviewer had to probe further. This made it harder for the interviewers to establish a rapport with respondents. It also contributed to survey fatigue and left less time for other questions.
The BOS also asks about declines in sales over the past 12 months with a yes or no question:
(2) Did the overall level of the firm’s sales volume decline over the past 12 months, compared with the previous 12 months?
And since the first quarter of 2016, the survey has asked respondents about the magnitude of the change in their sales levels:
(3) Over the past 12 months, did your firm’s sales volume increase significantly, increase slightly, remain the same, decrease slightly or decrease significantly compared with the previous 12 months?
Question 3 on the magnitude of changes in past sales levels is easier than question 1 for firms to answer. When we compare responses to this question about the change in a firm’s level of sales with responses to the growth rate question, we find that the results provide similar signals (Chart 1).
Question 1 on past sales growth therefore added complexity to the BOS interview without adding more value than question 3 on the level of past sales. Together with the narratives provided by firms, question 3 can provide all the context necessary for understanding firms’ sales outlooks.
Implications for the BOS indicator
Until question 1 on past sales growth was removed from the survey, Bank staff used it in the calculation of the BOS indicator—the summary indicator that uses several core BOS questions and principal component analysis to extract the common underlying variations among the survey series.3 However, our analysis indicates that removing this question has only a minor effect on the BOS indicator (Chart 2).
The change from the original BOS indicator can be further minimized by replacing question 1 with a different past sales question. Because question 3 on the magnitude of changes in past sales levels has been asked only since the first quarter of 2016, we evaluated various options for filling the time series before 2016 (back to the third quarter of 2003) to find the one that yields the smallest root mean squared error (RMSE) compared with the original BOS indicator (Table 1).
| Scenario | RMSE |
| Regressed the past sales level data from question 3 against the data from the yes/no decline question (question 2) to backcast past sales level data. Used these results and the past sales level question after 2016Q1. | 0.119 |
| Used the yes/no decline question for the entire sample. | 0.120 |
| Used the past sales level question, while using the yes/no decline question before 2016Q1. | 0.131 |
| Used the past sales level question with values of 0 before 2016Q1. | 0.309 |
| Did not use any replacement past sales question. | 0.321 |
The technique that yields the smallest RMSE is to backcast the data on past sales levels from question 3 by running a simple linear regression with the data on sales declines from question 2. However, using the yes/no decline question (question 2) for the entire time series is much simpler and produces almost the same results. This, therefore, is the option we chose. Chart 3 compares the new BOS indicator using the yes/no decline question with the original BOS indicator.
Endnotes
- 1. All mentions of sales in this backgrounder refer to sales volumes, not dollars.[←]
- 2. See D. Amirault, N. Rai and L. Martin, “A Reference Guide for the Business Outlook Survey,” Bank of Canada Staff Discussion Paper No. 2020-15 (December 2020).[←]
- 3. For details on the BOS indicator, see L. Pichette and L. Rennison, “Extracting Information from the Business Outlook Survey: A Principal-Component Approach,” Bank of Canada Review (Autumn 2011): 21–28.[←]