1 Introduction | 2 Sampling methodology | 3 Questionnaire development and testing | 4 Fieldwork methods and interviewers | 5 Checks and audits | 6 Response rate and interview length | 7 Classifications and coding | 8 Survey weights | 9 Imputation | 10 Variance estimation and significance tests | References | A1 Response rate by interviewer experience | A2 Sample and population profiles | A3 ACNeilsen area sampling frame | A4 Effect of area unit population changes | A5 Derivation of eligibility probability estimate | A6 Investigation of incident dates | A7 Contact sheets | A8 Showcards | A9 Selected CAPI screenshots
Household figures from the 2001 Census have been used in the selection of area units for the 2006 NZCASS, but the populations of some areas would have changed significantly by the time fieldwork for the survey was conducted. As a result, the sample design would not have been completely self-weighting, while the weighting method assumes that it was. This introduces some potential for bias in the results. However, based on the analysis described below, this bias appears to be small enough to be of little concern.
Approximate estimates of the bias have been calculated as the difference between the survey result and a "corrected" figure. The "corrected" figures were calculated by adjusting the survey weights based on preliminary 2006 Census data released by Statistics New Zealand at area unit level, and recalculating the survey results using these adjusted weights. The original survey weights were multiplied by a factor reflecting the relative increase or decrease in the number of occupied dwellings in that area unit between 2001 and 2006.
The results of these calculations are reassuring. For instance, the prevalence of any victimisation reported in the Key Findings is 38.7%, and so is the "corrected" estimate, at least to 1 decimal place. More precisely, the reported prevalence was 38.7089%, while the corrected prevalence was 38.7183, giving an estimated bias of 0.009 percentage points. For comparison, the standard error for this prevalence rate was much higher at 0.834 percentage points.
Approximate bias estimates have also been calculated for the prevalence and incidence of all household offences, personal offences, confrontational offences and burglaries. The estimated bias for incidence figures is generally a little higher than for prevalence rates (because incidence figures are higher than the corresponding prevalence figure), but even the largest bias estimate of -0.019 offences per 100 households (for the incidence of household offences) is fairly small. For comparison, the standard error for this incidence figure is 2.676 offences per 100 people.
To put this in context, one rule of thumb is that biases should generally be taken seriously if they are over one quarter the size of the standard error. This bias seems to be much smaller, being roughly one hundredth the size of sampling error.
A few minor caveats are appropriate. The adjustment factor is based on changes for Statistics New Zealand's area units, not Nielsen Area Units, and the true factors at NAU level would probably be more variable since NAUs are only one third as large. The 2006 Census figures used are only preliminary. Also, the 2006 figures are available for 2006 area units, not 2001 area units. While details of which 2001 area unit each NAU falls into were available, the concordance between 2006 area units and NAUs was not. Although a concordance between 2001 and 2006 area units is available, this left a few ambiguous cases where the 2001 area units have been split. These were dealt with by combining area units to remove the ambiguity, which may also have made the adjustment factors slightly less variable than they should be. Finally, the adjustment factors were applied directly to the final post-stratified weights, but ideally they should be applied to the inverse probability weights, and later stages of the weighting process should be rerun using these adjusted weights. However, none of these issues seem serious enough to affect the general conclusion here, which is that the bias from growth suburbs appears too small to be of real concern.
The bias is small primarily because there was virtually no correlation between victimisation and area unit growth. The correlation coefficient between the number of offences experienced by a respondent and the relative increase in the number of dwellings within the area unit where they reside was only 0.0003. Also, most area units did not change drastically in size between 2001 and 2006. The median relative increase in the number of dwellings was 3.9%, the lower and upper quartiles were 0.4% and 10.5% respectively, and the 10th and 90th percentiles were -4% and 22% respectively.