Problem with Postal Code Conversion File+ 7B

Question:

A researcher here is working with PCCF+ 7B and has encountered a problem, and wondered if someone at StatCan could assist?

Text below forwarded from the researcher:

The main problem I am facing is the DB-level GAF linking, but I also found two other issues that could be affecting the linking as a whole.

I am working on a project where we are trying to use postal codes reported in a survey to identify rural/urban status (planning to use the CSize variable). There was an option to provide full postal code, or to provide only the first three digits if the respondent would prefer. The records with full postal code seem to be linking properly, but any record with only FSA appears in the problem file with most of the variables set to missing. When I went through the code closely I noticed that when the records are linked to the Geographic Attribute File (GAF) it requires records to have a dissemination block. The comment in the code above this merge says “Merge back with Geographic Attribute File to get the remainder of the codes - Start at the Dissemination Block level, then move up to the DA, CSD, etc...”, but then the files are only linked at the DB level and any record without DB is not merged. From what I understand, dissemination block cannot be assigned to records linked on less than 4 characters so our records with only FSA do not get merged to the GAF and therefore have all the GAF variables set to missing later in the cleaning up stage of the files. When I looked at the datasets right before this merge, all the records are assigned a CMA from the linking based on FSA but this variable gets dropped before merging to the GAF. I know that the GAF is a DB level file but from my understanding there are variables at higher levels, such as CMA (and therefore CSize), that could be assigned for records without DB. Would it be valid to use the CMA variable assigned in the earlier linking to merge to the GAF for variables at the CMA level?

I also found that in the wc6dups file, the SAC id variable is only one digit instead of three. The input dataset only has one digit where SAC should be, and there is a comment beside this dataset in the input file SAS code saying “Check the input file”. This doesn’t actually make a difference in the files right now because this variable is overwritten by the GAF SAC variable, but I’m not sure if it will have an impact if the files are merged using SAC/CMA. Another thing I noticed is that DMT is used frequently through the code to assign missing values to records with specific DMTs, but DMT is set to 9 after the 6 character linking, and doesn’t exist in any of the datasets used later in the linking, so any records not linked using 6 characters has a DMT=9. I’m not sure the impact that this has on the validity of the linking, but I don’t think this is what was intended.

Answer:

In developing more recent versions of the PCCF+, we have removed most of the geographic links at the 3-digit FSA level, because we consider these to be largely inaccurate in most cases – this is why your client has most of these as missing. This happens during the cleaning phase. In general, we do not recommend off-label uses of the PCCF+, such as pulling out preliminary datasets to be used in geocoding.

In regards to specific questions:

but I’m not sure if it will have an impact if the files are merged using SAC/CMA.

They are not merged  using these variables.

Another thing I noticed is that DMT is used frequently through the code to assign missing values to records with specific DMTs, but DMT is set to 9 after the 6 character linking, and doesn’t exist in any of the datasets used later in the linking, so any records not linked using 6 characters has a DMT=9. I’m not sure the impact that this has on the validity of the linking, but I don’t think this is what was intended.

We cannot assign DMT to partial postal codes – they only exist for full postal codes. We perform different geocoding processes using DMT as an indicator of positional accuracy.

PCCF/PCCF+ LibGuide

Queen's University Library has developed a PCCF/PCCF+ LibGuide, available at: https://guides.library.queensu.ca/PCCF

PCCF+ Header File

Question:
I would like so information on how to massage a pccf+ output (2006 census geo, version 5k). The preview in SAS is clear, but I generate a .csv which contains more variables (without any header) than the one that is being displayed in SAS (first 500 records). Could someone help me with this?

Answer:
I was given the following response:

“The CSV is output automatically when you run PCCF+. The output variables are indeed provided with variable names (at the top of the column), and the variables are all listed and explained in the documentation.

If the user wishes, they can import the CSV back into SAS and eliminate some of the columns, or rename their variables. This can also be done easily in Excel.

Alternatively, if they prefer the version that pops up automatically in SAS (which in this case seems to lack problems), they can hard-save that output as a SAS file using a DATA step, or output it as another CSV.

Without having more details, it’s hard to provide more feedback than that.

Please let the user know that there is a newer version of the PCCF+ available.”

PCCF+ - Neighbourhood Income Variables

Question:
In PCCF+ 7B, ten neighbourhood income quintile and deciles variables are available - BTIPPE, ATIPPE, QABTIPPE, QNBTIPPE, DABTIPPE, DNBTIPPE, QAATIPPE, QNATIPPE, DAATIPPE, and DNATIPPE ( see PCCF+ 7B Reference Guide, page 17). Prior to PCCF+ 7B, four neighbourhood income (within CMA/CA and national) quintile and decile variables are available – QAIPPE, QNIPPE, DAIPPE, and QDNIPPE (see for example PCCF+ 6D Reference Guide, page 15).

My question is which of the ten PCCF+7B correspond to the earlier definition of the neighbourhood income quintile and decile?

Answer:
We’ve received the following response from subject matter:

In previous versions, the quintiles were based only on After-Tax (AT) income.

So:

QAIPPE = QAATIPPE quintiles, area-based
QNIPPE = QNATIPPE quintiles, national
DAIPPE = DAATIPPE : deciles, area-based
DNIPPE = DNATIPPE : deciles, national

PCCF Reference Guide

Question:
Where on the EFT is the 2019 February version of the Reference Guide? 

Answer:
It is currently being stored here: /MAD_PCCF_FCCP_DAM/Root/EnglishSASInstructionsPCCF+

How to Figure out which PCCFs I need?

Question:
I have a researcher looking for the 2001, 2006, and 2011 census and corresponding PCCFs. I’ve gotten all the census data, but I’m having a heck of time figuring out which of the PCCFs I need to use (for example, in the PCCF+ folder for 2001, there is PCCF4F, PCCF4G, PCCF4H, and pccf4j). I have no idea what I’m looking at. Can anyone point me in the right direction?

Answer:
I believe the deal is that the highest alphanumeric version in any given census folder is the last version for that census. So, the 7s are for 2016, the 6s are for 2011, the 5s are for 2006, etc. and the lettering just restarts to indicate which versions have been released.

I think The odd case on the EFT might be 6D, which is still in the 2017 folder, but was created in 2015 and based on the 2011 census geo, but includes 2016 DAs.  That's a weird one.  The first official 2016 version seems to be 7A, or at least 7A's documentation says "The reference date for these geographic areas is January 1, 2016. The previous version linked to the 2011 Census of Population geographic areas."

I think the deal with the PCCF+ over the PCCF is that you can use the PCCF+ for data spanning multiple census, but I don't think you have to grab separate versions.  Someone who actually knows what they're talking about can confirm or deny, but in your case, Maggie, I think you only need to grab the latest version for 2011, which would be 6C or 6D, depending on whether or not it matters that the documentation for 6D says "Although version 6D of the PCCF+ is still based on 2011 census geography, the output includes the 2016 dissemination area (DA) based on a released intercensal correspondence file (DA16uid)."  

PCCF+ Licensing

Question:
We are working with a Master’s student that would like to use the PCCF+ to run postal codes to match to DAs. She will be collecting the postal codes from a survey that is being run as part of a larger project that involves Queen’s University, the City and a consulting firm in town. She is a co-investigator and the work she will be doing will contribute to her Master’s thesis and results will be shared with the City and the consulting firm.

While the survey data will be accessible to non-Queen’s members of the project team; the PCCF+ would only be available to the student researcher.  If the student uses the PCCF+ to run the collected postal codes and get associated DAs, can just the DAs be shared as a result with non-Queen’s members of the project team?

The composition of the project team (with the municipality and consultants) partnering with the University is throwing us off a little bit.

Answer:
We’ve received the following response from the PCCF team:

“Yes, just the DAs (geography) can be shared as a result with non-Queen’s members of the project team as long no postal codes are displayed in the student’s results or reports.

All end users receiving these products must abide by the requirements of the End-use Licence Agreement for these products.  We would like to point out some specific constraints on the use of these data products: 

• to be used for the purpose of matching postal codeOM data to geography (Clause 4);
• not to be used for mail preparation purposes (Clause 5);
• acknowledgment of data licensed from Canada Post (Clause 7);
• reproduction and publication of up to 1% of content (Clause 8); and
• a disclaimer for search engine applications (Clause 10).”

PCCF in csv Format

Question:
Until we complete hiring a new data librarian, I’m the contact for anyone in our institution needing the PCCF and PCCF+ files and my knowledge is very limited. Usually I just provide the files and all is good, but now a grad student who has received the files sent me the following: “I am not familiar with SAS. Could you please send me a csv or Excel file which contains the postal codes and corresponding regions”.

Answer:
I’ve received the following response from the PCCF team:

“We do not provide the PCCF or PCFRF files in csv or Excel format. The files are quite large, and would take up even more room if they were provided is some application format. They are provided as a flat, fixed width, ASCII text file (.txt), which is readable/importable into a wide variety of applications. This is not a format specific to SAS. The client should be able to use it in whatever data application they are familiar with.

The record layout is provided in the Reference Guide (page 11 in the 2019 February version). That explains the format of the text file, and how the client needs to define the file in their application.

It’s not the most convenient way to provide the files. However, it is the format that is usable in the most applications, and has the smallest file size.”

Postal CodeOM Conversion File Plus (PCCF+) version 7A: Revised SAS Program

Notice

Postal CodeOM Conversion File Plus (PCCF+) version 7A: Revised SAS Program / Version 7A

Two errors in the PCCF+ 7A SAS code were identified as follows:
The first error resulted in some income quintiles being misclassified. The income information was being merged using the 2006 DAuid instead of the 2016 DAuid.  This error does not impact the majority of dissemination areas (DAs) since DAs have a high correspondence from one census to the next (Statistics Canada, Correspondence Files, Reference Guide – Census year 2011; Census year 2016;  Catalogue no. 92-156-G). 
The second error resulted in some variables (FED16uid, Fed16name and CSDtype) not being initialized within the SAS code due to name changes. This resulted in the removal of these variables from the final geocoded output file.

Since the SAS input data files in PCCF+ version 7A were not affected by this error, a new version of the full product was not created. Instead, the SAS program has been revised to correct these errors.  Users should replace PCCFplus_7A_v1.sas file with the PCCFplus_7A_v2.sas in order to resolve these errors. 

EFT: /MAD_PCCF_FCCP_DAM/Root/2017/pccf-fccp-plus/pccf+fccp+7A/data

Census 2016 PCCF+ Release Date

Question:
Is there an update for the release date for the PCCF+ based on census 2016?  I have researchers waiting for this product and would like to provide an answer.

Answer:
The PCCF+ version 7A is already indexed to the 2016 Census and available through the DLI. It can be found in the following subdirectory:

EFT: /MAD_PCCF_FCCP_DAM/Root/2017/PCCF7A

Postal Code Conversion File (PCCF) - available from Canada Post in the future?

Question:
I'm in the process of replying to a non-UNB person to let her know that she cannot obtain a copy of the PCCF from us; however, when I went to the PCCF catalogue page, I see that the June 2017 version, issued on Dec. 13, 2017, is listed as the last one available from Statistics Canada.  The status is listed as discontinued, and the note says [their bolding]: This product has been discontinued as of February 2018. The last issue of this publication was a December 2017 update.  Canada Post Corporation will assume the distribution of this product, and its guide, in the future.

So my questions are what are the implications for DLI access to this product, and was there some announcement that I missed?

Further to my message, I notice that the note for the PCCF+ indicates that it will be available from Canada Post and through the DLI.  While I do like the fact that I now know we'll still be getting licensed copies, I'm wondering if there shouldn't be the hyper link on Canada Post on the PCCF+ page like there is on the PCCF page, and if there shouldn't be an additional comment preceding DLI.  For example (emphasis mine): "PCCF+ will be available via Canada Post and the Data Liberation Initiative (for authorized users within DLI institutions)."  Or some such thing.

Answer:

Yes, you are correct. Although Canada Post has assumed the distribution of this product, a copy of the PCCF will continue to be distributed by the DLI for researchers that fall under the parameters of our End-User Licence Agreement found in Appendix II of the Licence Agreement. We will pass your website recommendation on to our management. 

Follow Up Questions:

1. One more question I have is whether my researcher can contact StatCan for the June 2017 version (issued Dec. 2017), as seems to be indicated on the PCCF page, or if she'd need to contact Canada Post.  According to the note I mentioned in my first (of accidentally many) messages, versions of the PCCF after Feb. 2018 must be obtained from Canada Post; however, when you click on the "More information" link beside the three more recent versions listed on the page, there's no "contact us" link (or any further information, for that matter).

Could you check and let me know what the situation is with this?  I can give the researcher the general contact info for StatCan as needed, but I wanted to double check to see if the above situation was just a typo or intentional.

2. We just got a request for PCCF plus. It says on this page that the product has been discontinued as of Feb. 2018. Is this correct?

https://www150.statcan.gc.ca/n1/en/catalogue/82F0086X

Answer:

We have confirmed that the DLI will continue to be provided with the PCCF, PCFRF and PCCF+ files for usage under the parameters of our End-User Licence Agreement found in Appendix II of the Licence Agreement. 

Please note that the Statistics Canada catalogue (the PCCF and PCCF+ webpages referenced below) will no longer provide information on the latest release. As such, the DLI program will work towards including the PCCF, PCFRF and PCCF+ in their list of available products for your referencing. 

Follow Up Question:

Thank you for your updates about distribution. I noticed that the statement for the PCCF and the PCRFR in the STC catalogue simply states:

Canada Post Corporation will assume the distribution of this product, and its guide, in the future. 

Whereas the note for the PCCF Plus is:

PCCF+ will be available via Canada Post and the Data Liberation Initiative.

Do you have a sense of what this will mean for user support for DLI contacts?  Will Health Geography and the Geography Division at Statistics Canada still be involved for the three postal code products? 

Just a last question, is it desirable for the statements for the three products to be more consistent re: DLI?

Answer:

We have made subject matter aware of the confusion caused by the wording on the PCCF, PCRFR and PCCF+ Statistics Canada catalogue pages. I agree that is it currently a little confusing. However, since the PCCF, PCFRF and PCCF+ files are separate from the Statistics Canada Open Licence Agreement and since the DLI grants access to these files through its own licence agreement, please reference the DLI collection webpage for the most up to date information on the availability of these products. We are actively working towards listing these files on our all DLI products page.

The subject matter area internally responsible for these files has an agreement with Canada Post and will continue to be involved in the production of these files. For the purposes of the DLI, the PCCF, PCFRF and PCCF+ files, and any associated support we provide will continue to be unaffected. 

PCCF+ version 6D official release

Postal CodeOM Conversion File Plus version 6D (internal release)

This is the official release of the Postal CodeOM Conversion File Plus (PCCF+) version 6D based on the 2011 Census. This file reflects postal code data from the Canada Post Corporation up to and including August 2015.

The Postal CodeOM Conversion File Plus (PCCF+) is a SAS© control program and set of associated datasets derived from the Postal CodeOM Conversion File (PCCF), a postal code population weight file, the Geographic Attribute File, Health Region boundary files, and other supplementary data. PCCF+ automatically assigns a range of Statistics Canada’s standard geographic areas and other geographic identifiers based on postal codes. The PCCF+ differs from the PCCF in that it uses population-weighted random allocation for postal codes that link to more than one geographic area.
What’s new?

The Postal CodeOM reference date for the Postal CodeOM Conversion File (PCCF) and the Postal CodeOM Conversion File Plus (PCCF+) is still August 2015.

This release has been updated to include the 2013 Representation Order of the Federal Electoral Districts (FED16uid) and the January 2017 air stage offices.

Although version 6D of the PCCF+ is still based on 2011 census geography, the output includes the 2016 dissemination area (DA) based on a released intercensal correspondence file (DA16uid).

EFT: /MAD_PCCF_FCCP_DAM/root/2011/pccf-fccp-plus/pccf6D-fccp6D/

Postal CodeOM Conversion File Plus (PCCF+) version 6C: Revised SAS Program

Postal CodeOM Conversion File Plus (PCCF+) version 6C: Revised SAS Program

An error in the PCCF+ 6C SAS code was identified which resulted in the overwriting of 2011 Census dissemination areas (DAs) with 2006 Census DAs. This error was introduced during our process of standardizing the coding for the input data files for version 6C of PCCF+. This error does not impact the majority of dissemination areas since 92% of DAs have a one-to-one match between 2006 and 2011 censuses (Statistics Canada, Correspondence Files, Reference Guide – Census year 2011; Catalogue no. 92-156-G). For the remaining 8% of DAs, users may find their data are assigned to a decommissioned 2006 Census DA unique identifier code.

An error in the processing of the institutional routine which resulted in rural postal codes erroneously being assigned via population weighted random allocation instead of the rural post office (RPO) has been corrected in the PCCF+ SAS program.

Since the SAS input data files in PCCF+ version 6C were not affected by these errors, a new version of the full product was not created. Instead, the SAS program has been revised to correct this error. Users should replace PCCFplus_6C_revised.sas with the attached version (PCCFplus_6C_revised2.sas) in order to resolve these errors.

Please contact Health Analysis Division by e-mail (HAD-DAS@canada.ca) should you have any questions.

EFT: /MAD_PCCF_FCCP_DAM/root/2011/pccf-fccp-plus/pccf6c-fccp6c/doc/PCCFplus_6C_revised2_SAS/

PCCF and PCCF+ Analysis

Question
Does anyone know if the PCCF and PCCF+ files can be analyzed using R?

Answer
PCCF is an ASCII text file and can be read by any statistical program environment, including R.

PCCF+ consists of many files processed by a SAS program that would require a major re-write to use in R, although the output from the SAS program includes ASCII text files that could be used directly in R.

PCCF/PCCF+ Usability Question

Question
We have a researcher that has collected her own data where each entry has either a Postal Code or Full Address. This collected data spans from 2006 to 2013. They would like to use the NHS 2011, to get demographic information on a CSD level for this data. We know we need to use a PCCF file but, are unsure which one release to use due to the time span of the data (2006 to 2011).

Answer

Have you considered using the PCCF+ file?

I consulted a presentation, available on our Training Repository here, delivered by Health Analysis Division in 2015 regarding the differences of PCCF and PCCF+, and the presentation highlighted that:

“Consider using PCCF+ rather than PCCF-SLI if any of the following apply
…
4. The “vintage” of the postal codes on your file spans more than one census”
4: “Vintage” of postal codes

• Postal codes on your file spans more than one census
• PCCF+ assigns DA or EA from each census from 1981 through 2011
• Useful for time-varying variables

The presentation also presents several case scenarios and which product to use – example of one of the case scenarios presented

Potential case strategies
Several geocoding scenarios are possible
1.Only postal codes available

• Use PCCF-SLI or PCCF+ to assign geographic codes, etc…

2.Full street address available

• Use address geocoding software (GIS)
• Use PCCF-SLI or PCCF+ on postal code portion of address

3.Telephone numbers available

• Reverse lookup to get postal code or address
• Use 911 system maps to get location from address

The most recent PCCF+ is found on the EFT here
/MAD_PCCF_FCCP_DAM/Root/2011/pccf-fccp-plus/pccf6c-fccp6c

There is a read me file at the root of the safe that indicates the release date of the versions. Version 6C is the most recent.

Updated Products - PCCF+ Version 5H - Revised Files

Postal Code Conversion File Plus (PCCF+) - version 5H - Revised files

Revised files for PCCF+ Version 5H have been added to the DLI FTP and website. Please delete the previous versions of the files and replace them with the revised files.

The revised files contain corrections for errors in PCCF+ Version 5H. Our contacts for this product indicated that the errors "mainly affected coding in British Columbia, and rarely affected coding in Ontario and Quebec. The four main SAS programs (GEORES5H.SAS, GEOINS5H.SAS, R5HOLD.SAS, I5HOLD.SAS) were modified, and the modified versions of the postal code population weight files (WCFUDUPS.CAN and WCFPOINT.CAN) were also corrected."

- pccf5h-fccp5h-can.zip replaced by pccf5h-fccp5h-can-ver2.zip
- geoins5h.sas replaced by geoins5h-ver2.sas
- geores5h.sas replaced by geores5h-ver2.sas
- i5hold.sas replaced by i5hold-ver2.sas
- r5hold.sas replaced by r5hold-ver2.sas
- pccf1010.wcfpoint.can replaced by pccf1010.wcfpoint-ver2.can
- pccf1010.wcfudups.can replaced by pccf1010.wcfudups-ver2.can

WEB: http://www.statcan.gc.ca/dli-ild/data-donnees/ftp/health-sante-eng.htm

FTP: /dli/DLI Coll._Other prod/Health - PCCF+/pccf5h-fccp5h-can/pccf5h-fccp5h-can-ver2.zip

Thank you.

Updated Products - PCCF+ Version 5H

The following product is now available on the DLI FTP and website.

Postal Code Conversion File Plus (PCCF+) - version 5H

"The Postal Code Conversion File Plus (PCCF+) is a complementary product to the Postal Code Conversion File (PCCF). It is an automated system that uses postal codes to assign census geography. PCCF+ is based on the latest Postal Code Conversion File and the Postal Code Population Weight File produced by the Geography Division of Statistics Canada. PCCF+ uses weights to allocate postal codes linked to multiple dissemination areas according to the distribution of population using a given postal code." (http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?catno=82F0086X&lang=eng)

WEB: http://www.statcan.gc.ca/dli-ild/data-donnees/ftp/health-sante-eng.htm

FTP: /DissFTP/dli/DLI Coll._Other prod/Health - PCCF+/pccf5h-fccp5h-can

Thank you.