Different survey methods, different results:
What do Australians really think about immigration numbers?
Katharine Betts, 25 November 2021
This piece was originally published as a briefing note on the website of Sustainable Population Australia
Summary
Two recent opinion surveys about Australians’ attitudes to immigration numbers yielded markedly differing results. This Briefing Note analyses the reasons for this. There are important implications for survey design and interpretation of results.
The Australian Population Research Institute (TAPRI) released the results of our fourth national survey of voters’ attitudes to immigration-fuelled population growth on 14 October 2021. The sample of 2516 voters found that 69 per cent believe that Australia does not need more people and that only 19 per cent want a return to the pre-Covid levels of net overseas migration (NOM). That is, only 19 per cent want to return to a NOM of around 240,000 a year and, as the table below shows, 28 per cent would prefer nil net migration.
On 26 October 2021 Essential Research (ER) released their fortnightly survey of 1200 residents aged 18 plus. This included a question on immigration levels. They asked: ‘Do you think the levels of immigration into Australia over the past ten years have been…? Much too low; A little too low; About right; A little too high; Much too high; Don’t know.’
Overall 16 per cent said the levels had been either much too low or a little too low, and 37 per cent said a little too high or much too high. This latter percentage was a lot smaller than ER had found in previous years. (In January 2019 56 per cent had said too high, in April 2018 64 per cent, and in October 2016 50 per cent.)
Clearly the 2021 ER results differ from TAPRI’s. TAPRI found, in effect, that 70 per cent thought that a return to immigration levels of the recent past would be too high, but only 37 per cent of ER respondents can be taken to be of that opinion. Can this discrepancy be explained?
There are three key points of difference between the two surveys.
- Different questions get different answers
First, the questions are different. The ER was asking respondents how they felt about what had happened in the past, TAPRI was asking what they wanted for the future.
Beyond this point, the ER question provided no background information. TAPRI’s full question read as follows:
‘Immigration increased sharply over the 10 years to December 2019. Over this decade Australia added 4 million extra people, more than the current population of Brisbane and Adelaide together (3.6 million).
‘Over 64% of this growth was due to net overseas migration. Up until 2020 this has been around 240,000 migrants per year.
‘When our borders reopen which of the following would be closest to your views?’
Table A.1: When our borders reopen which of the following would be closest to your views?
| Responses offered: | % |
| We should return to net migration of around 240,000 a year or higher | 19 |
| We should return to net migration at somewhat lower levels | 22 |
| We should return to net migration at much lower levels | 20 |
| We should keep migration low enough so that new arrivals just balance out departures | 28 |
| Don’t know | 11 |
| Total % | 100 |
| Total N | 2516 |
The ER question ‘Do you think the levels of immigration into Australia over the past ten years have been…?’ was more laconic. While it did say ‘over the past ten years’ the October 2021 respondents might have had only a hazy impression of the numbers over that period. More than this, they could have been focussed on the last two years when, because of Covid, NOM has been negligible. Australian Bureau of Statistics (ABS) data show that from 2009 to 2019 NOM averaged 212,252 per year. But in the year to March 2021 it was minus 95,300.
The media were relatively silent when net migration numbers were large, but have been thrown into high alert when they sank into negative territory. Nevertheless, the population grew by 35,700 in the year to March 2021. A natural increase of 131,100 had more than compensated for the negative net migration. (See ABS, Population and components of change—national, March 2021.)
Because of inaccurate media reporting, the public could be forgiven for believing that, without restoring immigration, Australia’s population would fall. For example, the ABC ignored natural increase and claimed that Australia’s population was actually shrinking, as did Your Investment Property and The Australian.
- Groups with different composition can have different views
Second, the scope of the two surveys was different. TAPRI’s sample was restricted to voters. The ER sample was drawn from all adults aged 18 plus. This distinction matters.
At the 2016 census there were 18,193,865 people aged 18 plus. Of these 3,505,896 or 19.27 per cent were non-citizens and therefore not voters. If we are measuring the attitudes of Australians, attitudes which may influence elections, samples where close to one fifth of respondents are non-voters will be unreliable.
The 2016 census is five years old, but there is no reason to believe that this proportion of non-citizens has shrunk. Indeed, given the high levels of NOM from 2016 to 2019 it may well have increased, despite the exodus of many temporary migrants post-Covid.
This can be especially relevant where survey questions about immigration are concerned. Many non-citizens are anxious to acquire citizenship and thus would look favourably on the idea of a larger intake. This is especially likely for those still in Australia on temporary visas, many of whom would want to achieve permanent residency as a first step. In September 2021 there were 1,643,037 people in Australia on temporary visas of one kind or another. People keen to obtain permanent visas would be only too happy to give a big ‘yes’ to high immigration.
- Weighting should correct for differences between the sample and general population
Third, the ER note on method explains that their data are weighted for age, gender, location and party ID, but makes no mention of weighting for education, that is weighting for graduate/non-graduate status.
This also matters. For example, graduates are more likely to respond to surveys than are non-graduates and, if these two groups have different views, an unweighted survey will over-represent the views of graduates. Typically, graduates are rather more likely to vote Labor or Greens than are non-graduates, and they also tend to have different views on topics such as immigration. TAPRI has found them to be more likely to favour higher numbers than non-graduates. In the 2021 survey 27% of graduates favoured a return to a NOM of 240,000 a year as opposed to only 16% of non-graduates.
Their different values combined with their stronger propensity to answer surveys were a key reason for the failure of polls to predict the outcome of the 2019 election.
An inquiry into this failure found that failing to weight for education was a big part of the problem. The report’s authors advised all pollsters to do so in future. There is little evidence that they have been taking this advice.
For example, on 27 September 2021 The Australian published a combined set of data from all of the Newspolls taken from July to September 2021. While Newspoll’s samples appear to be restricted to voters, the combined data show that, of the pooled sample of 6705 respondents, 2741 or 40.9 per cent were graduates.
At the 2016 census 22.8 per cent of voters were graduates. True, this proportion has probably increased in the five years to 2021. TAPRI’s weightings for graduate status were adjusted (using ABS survey data) to account for this increase. We arrived at an overall figure for the proportion of voters who were graduates in 2021 of 28.5 per cent. Even if this figure is out by one or two percentage points it is a long way short of 40.9 per cent.
If Newspoll is still oversampling graduates other polls are likely to still be making the same mistake.
Conclusion
The value of surveys depends on whether they represent the population of interest and whether they capture respondents’ real views. Questions should not be ambiguous and should provide essential facts that give a context to the question. On several grounds the TAPRI survey seems more likely to represent the views of Australian voters on their preferences for future levels of immigration than does the Essential Research survey.
First, the wording and presentation of the two questions were different. ER asked about the past but provided no background information, while TAPRI asked about preferences for the near future and made sure that respondents had relevant information.
Second, nearly one in five of the ER sample would have been non-voters while all of the TAPRI sample were voters.
And third, unlike the TAPRI data, the ER data do not appear to have been weighted for graduate/non-graduate status.
Dr Katharine Betts is deputy head of The Australian Population Research Institute and a patron of Sustainable Population Australia.