1 Introduction | 2 Sampling methodology | 3 Questionnaire development and testing | 4 Fieldwork methods and interviewers | 5 Checks and audits | 6 Response rate and interview length | 7 Classifications and coding | 8 Survey weights | 9 Imputation | 10 Variance estimation and significance tests | References | A1 Response rate by interviewer experience | A2 Sample and population profiles | A3 ACNeilsen area sampling frame | A4 Effect of area unit population changes | A5 Derivation of eligibility probability estimate | A6 Investigation of incident dates | A7 Contact sheets | A8 Showcards | A9 Selected CAPI screenshots
Fieldwork period
Fieldwork procedures
Interview mode
General interviewer management
General training
Fieldwork for the NZCASS took place between 9 February and 25 June 2006. This period was divided into four fairly equal time blocks (or quarters), with sample areas spread evenly across blocks. This was so that response rates could be calculated as data collection progressed.
To maximise the chances of obtaining interviews at the designated dwellings in each cluster, it was necessary to institute a ‘call-back’ regime whereby interviewers returned to dwellings where no contact was made at the first call.
In urban areas, interviewers made an initial trip to the nine homes to be contacted in each cluster in the main sample. This was to post a Ministry of Justice pre-notification card in the mailbox. They then made up to three interviewing trips in each area. This meant a minimum of three knocks on every door. However because dwellings could be approached more than once at each visit, there was effectively the opportunity for up to six door knocks (two per visit), or perhaps even more. No prenotification trip was made in rural areas. This call-back regime achieved a contact rate of over 90% in both the main and booster samples, and increasing the number of call-backs further would have produced rapidly diminishing returns from the resources expended.
Households in the booster sample areas did not receive pre-notification cards in advance (due to the unknown eligibility of households). However, they were given them when the interviewer made personal contact at the door. Four interviewing trips were made to Māori booster areas.
With the booster sample, the lack of prenotification may have reduced cooperation, but we do not have solid evidence of the effect locally (e.g. a large-scale split-sample experiment). Nor are we aware of as much evidence on the prenotification effect internationally for face-to-face interviews as there is for self-completion or even telephone. Anecdotally, there was no strong interviewer feedback on this issue in the NZCASS, but this should be seen in context: there seems no particular reason to expect much feedback about this issue (because with booster sample respondents, we do not expect them to mention something they hadn't seen; with main sample respondents, we expect the effect more to be on the probability of refusal rather than on them explicitly saying much about prenotification).
Most fieldwork was conducted after 2pm on weekdays or at weekends to maximise the chances of finding the selected resident at home at time of call. This procedure minimised the number of calls required to a particular dwelling to obtain the interview. It thus enhanced the chances of interviewing the selected respondent within the number of calls allowable to each dwelling. Interviewers working in rural locations or in lower socio-economic areas were encouraged to visit earlier in the day, as this is when they would be more likely to find people at home.
In addition, one of ACNielsen’s field rules is that a return visit to an area must be on a weekend day if the initial visit was on a weekday, and vice versa. Typically, trips were spread over two or more weeks during the interviewing month, with the exception of remote rural areas (where the interviewer would stay 3-4 nights and complete all interviewing in this time, to save travel costs) and areas where all selected households were resolved in a shorter period, so further trips were not necessary.
A small thank you gift was given to all respondents. The Ministry of Justice decided that there would be no differentiation of practice according to the ethnicity of participants, in this regard.
As in the 2001 survey, the interview was conducted using CAPI (Computer Assisted Personal Interview); and CASI (Computer Assisted Self Interviewing) whereby the interviewer turns the laptop over to the respondent to complete sensitive sections of questioning. Computer Assisted Interviewing also eliminates data entry costs and ensures survey logic is adhered to. The Confirmit CAPI software used in 2006 was new browser-style software. (The CAPI software used in 2001 was extremely problematic and seen as inappropriate for re-use.)
One important advantage of the new software was that the selection of Victim Forms could be computerised, based on answers from the screener questions. In 2001, an awkward paper-based system was used to select offences for Victim Forms when the screeners showed more than three eligible incidents.
General interviewer management
Given the complexity of this survey, interviewers working on the project were given substantial project-specific training and briefing. The company’s standard interviewer selection and training procedures are outlined below, together with the nature of specific training and briefing required for the 2006 NZCASS.
As with the 1996 and 2001 surveys, close attention was paid to interviewer selection, bearing in mind some of the sensitive nature of the questionnaire and the stringent ethical considerations which have to apply. ACNielsen’s policy was again to use female interviewers wherever possible, and to use Māori interviewers as much as possible for the booster sample. Of the 77 interviewers who worked on the project, 9 were male and 20 identified as Māori.
Recruitment
ACNielsen has a core of trained and experienced interviewers available for this type of project. Extra interviewers were also needed due to the large size of the NZCASS.
All applicants are initially interviewed by a field manager. At this time, they are required to provide personal references and sit an assessment test. If reference checks are satisfactory, they are then formally inducted. This process includes being familiarised with and signing both an employment agreement and a performance agreement. Interviewers are also familiarised with a job description, the company’s Health and Safety Handbook, the Field Handbook, and the Field Policy Document.
Recruitment of interviewers experienced in interviewing Māori presented difficulties at the start of fieldwork. One cause was the delay from the originally planned date for fieldwork starting in mid-2005. (Potential Māori interviewers were first approached by LLE Research around the mid-2005 date.) Also, many potential recruits were not able to accommodate the near full-time workload required. In the end, very few of the candidates from LLE Research proved suitable. Thus, ACNielsen had to recruit several Māori interviewers in early 2006 just before fieldwork began. There was a particular shortage of Māori booster interviewers in the Auckland area at the start of fieldwork.
Initial general training for interviewers comprises a minimum of six hours basic training.[11] This is generally carried out in three two hour blocks covering four modules:
A handbook was provided with each module. When basic training was completed, an Interviewer Induction Training Record Form was completed and signed both by the Trainer and the Trainee.
Interviewers are also briefed about every survey on which they work. At the time trainee interviewers are briefed on their first project, a "buddy" and "observer" system is used. This means that trainee interviewers are accompanied on their first foray into the field by a buddy, who observes and provides on-the-spot encouragement, advice and assistance. Briefing and training specifically for the NZCASS is discussed below.
NZCASS training and briefing
The training and briefing for this project were split into two days. During the first day, interviewers were familiarised with the laptops and the questionnaire. These briefings were conducted by ACNielsen regional supervisors some days before the second seminar (except in Christchurch, where they had to be on consecutive days due to larger travel costs). After this session, interviewers could complete practice interviews with family/friends to build up familiarity with the very lengthy questionnaire and use of CAPI.
The second day covered:
The turning around refusals component was Refusal Aversion Training (RATS). This training includes the following:
Interviewers could not start interviewing until both days’ training was completed. Hence, interviewing started in some regions before others. The second day’s training was conducted in Wellington (7 February), Auckland (9 February), Christchurch (14 February), and Rotorua (16 February). Each briefing was attended by about 20 interviewers.
Overall, a total of 28 hours of training was provided for new interviewers in the 2006 NZCASS, and 22 hours for experienced ACNielsen interviewers.
Footnote
11 ACNielsen has also developed a re-training module, designed as a refresher, or where specific circumstances demand. This include issues of poor response rates, where it has become apparent that procedures and methodology are not being correctly followed, or where there is a special requirement for a particular project.