Wednesday, August 31, 2016

CCHS - weighting of optional questions

Question

I am looking at the CCHS survey questions. I have a good understanding about the sampling technique followed and the weighting process undertaken for the responses achieved off of the core content questions.

But I am not sure about the optional content questions. These questions are optional and by optional it means that they have been selectively chosen by some provinces interested. So for example, the question about insurance covering part or all prescription medication, was an optional content question, that was included only in the surveys for the Ontario, New Brunswick and Yukon in cycles 2013 and 2014.

I understand how the respondents were chosen in these provinces. But I don’t know how I can generalize these responses i.e weigh them Canada wide or at lease provincial wide.

Answer

The CCHS 2013-2014 User Guide, on p. 8, it states:
It should be noted that, unlike the modules included in the common content, the resulting data from the optional content modules is not easily generalized across Canada.
Any analysis on optional content should be done using the master files. Optional content is not included on sub-sample or rapid response files (with the exception of when it is used as theme). Even if it were, the records would be a subset from the master anyways, so it would limit analysis to look at anything but a master.

Optional content is not the best name. We should call it sub-national content, meaning that it is not available for producing estimates at the Canada level. For example, HUI was only selected in Manitoba in 2012. So with 2012 data, you can only produce estimates for Manitoba. In 2013-2014, HUI was used as theme content, meaning that it is collected nationally and thus can be used to produce Canada level estimates. For 2012 though, there is nothing that can be done to make inference about HUI estimates in Canada based solely on the data from Manitoba. In some years you can get close with some of the content – food security, for example, is an optional module that gets selected by most of the provinces in the years it is not theme. For those years, you can be close to having national estimates, but it would still be underrepresented by the provinces that did not select the content.

If the client wants to send us the content they are interested in, we can write up a list of which years and which provinces they can create estimates for. A good number of the optional modules have been used as theme at some point, so there is likely that there will be a year for which national estimates can be produced.