6 Response rate and interview length

The response rate in the main sample was 59%. For the Māori booster sample, it was 56%. The corresponding figures were 65% and 57% in the 2001 survey and 56% and 66% in the 1996 survey. Response rates have thus varied between each of these surveys. To put this in context, a decline in response rates has been generally evident over this period across a number of surveys, and in a large number of countries. For instance, the response rate for the National Crime Victimization Survey (NCVS) in the United States has dropped by approximately one percentage point per year over the last ten years.[13]

Table 6.1 shows an analysis of response rates. Of the issued sample of 7,614 addresses in the main sample, 392 were unoccupied, and for 590 there was no contact made. This gave a contact rate of 92%. Of those households contacted, the main reasons for non-response were unavailability during the survey period (11%) or refusal - either by the selected person (11%) or by the household before a respondent could be selected (10%). Non-contact accounted for most of the other failures.

Table 6.1 Analysis of response rates

A corrected response rate is shown in Table 6.1 for the Māori booster sample. The corrected figure incorporates an estimate of the proportion of ineligible dwellings (i.e., those with no resident Māori person aged 15+) among those dwellings at which no contact was made after the call regime had been completed, or where the household’s eligibility might otherwise have not been established. The corrected figure is thus an estimate, but is a better estimate of the actual response rate than the pessimistic figure of 40% obtained by simply dividing the number of complete interviews by the total number of occupied dwellings approached.

There was a large difference in the raw household refusal rate between the main and booster samples (9% versus 2% respectively), but this is easily explained. First, the booster contact sheet (in Appendix A7) shows that the determination of whether there is anyone ethnically eligible was done early on, before consent to a full interview was explicitly sought. Hence the appropriate base for comparisons is not all dwellings approached, but eligible dwellings contacted. (For the booster sample, this excludes the majority of dwellings approached.) The household refusal rates are then similar for both samples (10-11%). Second, and more subtly, it is possible that some households with Māori effectively refused by not admitting to having Māori in the household, and were therefore recorded as ineligible rather than refusals. This would be consistent with the lower than expected incidence of Māori households noted below.

The rate of unoccupied dwellings also differed somewhat between the two samples. Holiday homes make up a large proportion of this category, and are tightly clustered. For example, just 4 area units accounted for 40% of the holiday homes encountered in the main sample. This clustering may increase sampling error enough to account for the difference. Also, many areas dense in holiday homes were less likely to be selected for the Māori booster sample. For example, three of the four area units mentioned above were in the South Island and had a low density of Māori households.

While the drop in the response rate was somewhat disappointing, there were nonetheless a number of initiatives adopted to encourage participation in the survey:

• A Ministry of Justice pre-notification card was dropped into the mailbox of all the nine homes to be contacted in each cluster in the main sample. Households in the booster sample areas did not receive pre-notification cards in advance (due to the unknown eligibility of households), but got them when the interviewer made personal contact at the door.

• Households contacted were given a letter signed by the Secretary for Justice with details about the survey. This letter was also available in Māori, Cook Island Maori, Samoan, Tongan, Chinese (both full form and simplified), Japanese and Korean translations.

• New promotional material for the interviewers and respondents was designed to give the NZCASS an attractive ‘image brand’.

• Promotional activity took place in newspapers, websites, and on the radio to increase awareness of the survey and provide evidence of authenticity.

• Interviewers with low response rates were identified and re-trained and monitored.

• A prize draw for interviewers who achieved excellent response rates took place each quarter.

• Respondents were offered a summary of the results of the survey, due to be provided in 2007.

• Considerable effort was made to recruit Māori interviewers to work in the Māori booster sample areas.

The response rate was highest during the first quarter of fieldwork, and lowest during the third quarter. It was also lower in the most deprived areas (the lowest quintile), in the upper North Island and particularly in Auckland. Response rates did not differ markedly based on interviewer experience, which was surprising. Details of this are given in Appendix A1.

The final sample of 5,416 was reasonably close to the 5,600 originally expected.[15] There were more Māori in the main sample than expected, whereas fewer interviews were obtained with the Māori booster sample (1,187) than hoped for (1,600).

Although the response rate for the booster sample was slightly lower than for the main sample, this was not the largest contributor to the shortfall. A lower than expected incidence of Māori households was the main cause. The proportion of booster sample households approached that reported being eligible (i.e. that a Māori person aged 15 or more lived there) was only 22.6% (=2967/13137). This was substantially lower than the eligibility rate in the 2001 survey of 27.0% (=1115/4132). The relatively blunt screener question may have contributed to this.[16] Interviewers reported that some households that clearly appeared to have Māori residents claimed to be not eligible. In all, 1,698 Māori interviews were completed, adding those from the main sample to those from the booster sample.

Comparing the response rate data for the Māori booster sample in 2001 and 2006 revealed three differences large enough to possibly be of practical interest:

• A higher proportion of dwellings reported no eligible adult, as discussed above.

• Fewer unoccupied dwellings were encountered in 2001 (2.9% of dwellings versus 4.2%). Although the difference in holiday homes mentioned above partly explains why this is lower than the rate in the main sample, we see no obvious explanation for the difference between the rates for the 2001 and 2006 Māori booster samples. (The rate of unoccupied dwellings in the main sample was 5.1% in both surveys.)

• A higher proportion of (estimated) eligible dwellings were classified as unapproachable. This may have resulted from increased emphasis being placed on interviewer safety in more recent years.

Response to the Self-Completion components

Of those who completed the interview in other respects, 6.4% (=349/5416) refused to complete the Self-Completion component.[17] This compares with around 4% in the 2001 survey. Refusals were disproportionately from older respondents (11% for those aged 60 years or older compared with 5% for those aged 15–59). The longer interview length in the 2006 survey before the Self-Completion components started, compared with 2001, may have been a factor in the higher Self-Completion refusal rate. Another may have been the stronger emphasis put on the consent process at the beginning of the survey informing respondents they can stop the interview at any time. When respondents were asked why they refused, the most common reasons given were computer concerns, not wanting to answer, and being too busy or too tired.

Interview length

The average interview length was 52 minutes, five minutes longer than in 2001. There was a wide variation, ranging from less than 30 minutes for some interviews to more than 70 minutes for others. A quarter of the interviews took longer than an hour. Interview length is derived from questionnaire length (recorded on the laptop), plus ten minutes added as the average time for getting settled and disengaging with the respondent at the close of the interview.[18] Table 6.2 shows details.

Table 6.2 Interview duration

The average questionnaire length for non-victims was 34 minutes, and 53 minutes for victims (the longer time for victims being entirely expected). Table 6.3 shows details.

Table 6.3 Average interview duration by number of Victim Forms completed

Footnotes

13 The response rate for individuals in the 2005 NCVS was 84%, down from 90% in 2000, on top of household response rates of 91% and 93% respectively.

14 Respondents who only refused to complete the self-completion section were counted as complete interviews.

15 There were rather more interviews completed (5,476), but about 1% were deleted from the dataset due to irregularities in procedure, incomplete data etc.

16 The wording of the question in 2006 was “Is there anyone usually living here aged 15 years or older who might consider themselves Māori? That is, if asked which ethnic group or groups they belong to, they would include Māori.” The 2001 wording was “Is there anyone in this household who is Māori and aged 15 years or older?" This was changed to better match the official ethnicity standard.

17 By way of comparison, 5% of people in the 2004-05 British Crime Survey refused similar self-completion components. The New Zealand rate therefore is not unusual, especially as the BCS only requested self-completion from those aged under 60 years (Grant et al., 2006: 53).

18 The 5,274 cases used to calculate interview length is less than the total number of interviews because some interviews (those where one section of the questionnaire was recorded as taking 60 minutes or longer) were excluded. This most likely occurred because of a clear break in interviewing, such as finishing the interview on a different day. Thus, the average interview length is probably a little understated since a break is more likely to have been taken in unusually long interviews.