Showing posts with label Beyond 20/20. Show all posts
Showing posts with label Beyond 20/20. Show all posts

Monday, March 15, 2021

Census 2006 Data Question

January 6, 2021


Question

II've pulled data for each CSD from the 2006 census using Beyond 20/20 and my advisor wanted me to ask , based on the info for StatsCan that I've copied below - how can I know if zeroes in my dataset represent data suppression or are true zeroes? He is thinking that those that were suppressed will likely need to be treated as missing for my analyses, since they aren't true zeroes, but I'm not sure how to accurately differentiate these. He's thinking that I can likely do this by looking at CSD population size and the number of total private households, and if neither of these thresholds is exceeded (as outlined below) then the zero is likely a true zero (likely not many of these), otherwise I can replace the other zeroes as missing values. Does that make sense?


Census Info 

Area suppression for income characteristic data 

Area suppression, when applied for data quality purposes, is used to replace all income characteristic data with zeroes for geographic areas with populations and/or number of households below a specific threshold. 


If a census tabulation contains any data showing income characteristics for individuals, families or households, then the following rule applies. Income characteristic data are zeroed out for areas where the population is less than 250 or where the number of private households is less than 40. These thresholds are applied to 2006 Census data as well as all previous census data. The threshold of 40 private households is based upon the fact that weighted data are being used. With the weighting factor for each household being 5, setting a threshold of 40 ensures that there will be at least 8 households used in the calculation. The private household threshold does not apply for tabulations based on place of work geographies. 


This seems to be what was happening in my data, as some variables for a single CSD have ‘.’ And others zeroes. Those with zeroes typically seem to relate to income, proportion of household spend etc.



Answer

Statistics Canada places the highest priority on maintaining the privacy and confidentiality of respondents. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.   Because of this and the data quality measures in place, your client will not be able to distinguish between all “true zeros” and these suppression zeros. Area suppression is one type of suppression which involves removing all characteristic data for geographic areas with populations below a specified size.


Having counts based on the geography should let them filter most of those out.

250 people, if the table contains income data, and if the table also contains place-of-residence data, at least 40 private households 100 people, if it is a six-character postal code area, that is, a local delivery unit (LDU), or if it is a custom area 40 people, in all other cases.


In regards to your client’s question on individual cell suppression please see the following paragraph from Chapter One of the 2006 Overview of the Census: Dissemination Rules for Statistics:

“Tables are sometimes accompanied by statistics such as averages, totals and standard deviations. There are various ways of ensuring that these statistics do not reveal sensitive information; for instance, they may be suppressed or made less precise. Some statistics, such as totals, ratios and percentages, are based on the rounded values in the tables to which they apply. A statistic will be suppressed if there are too few data to compute it. In cases of data items expressed in dollars, if the statistic must be calculated from data where the values are too close or if a value is too high compared to the others, then the statistic will be suppressed.”

 

Depending on the income source variable, income medians and averages are most always never true 0. When there is a zero for most things it is a suppression. As for counts that have been rounded to zero, it is a feature of the confidentiality system and you cannot distinguish those rounded down from the true zeros.

Friday, April 21, 2017

DA suppression list for 2001? EA suppression lists?

Question 1Does anyone know if there is a list of DAs from which data was suppressed or unavailable in 2001? Are there similar lists for enumeration areas for other years (1991, 1996)? If they are on the EFT, where are they all hiding? J

A student is doing historical research with census data, and she found a few DAs for which we can't find any data through the archived census profile on the EFT or from other products that share census data such as the Census Analyser. Sometimes with historical research the students fail to realize that the boundaries have changed, which requires the help of the correspondence files, but these DAs definitely exist in 2001 as shown in boundary files or reference maps.

For example, DA 12090310 is located right in Halifax. But we can't find any data on it! In the 2001 Beyond 20/20 profiles, we jump straight from 12090309 to 12090311. So where can we find some sort of explanation about why data wasn't released for that DA and other DAs, preferably with lists of suppressed DAs for 2001 and EAs for older years?

Question 2
In some cases, only some subsets of data are available for select DAs. For example, in 2006, 12090292 has the total number of census families and many other (but not all) variables, but age counts (?!) seem to be unavailable. The student would love some sort of explanation that she can quote in her thesis. When looking at the notes for the profile in Beyond 20/20, only the criteria for suppressing data about income is explained.

Answer
We have received the following response from subject matter regarding your questions:

“Unfortunately, the DA suppression list is nowhere on our website…we do have the CSD suppression list in the notes for the data quality and confidentiality of the community profiles. I compiled a list of suppressed DAs (approximately 8,909 DAs) in a spreadsheet for you (please see attached spreadsheet). This suppression list is not very accessible for previous Censuses.

To answer your questions: In your example below (question 1) of DA 12090310, this DA in Halifax is suppressed. You can see from Geosuite 2001 that this DA does not meet the population threshold for the data quality and confidentiality rules.

For question 2, the specific DA 12090292 is not suppressed for 100% data but it is suppressed for 20% data (the non-response rate for the 20% was over or equal to 25%). You can find an explanation in the Beyond 20/20 notes in the data quality flag section.

For example, in the Beyond 20/20 file in the geography row you will see the following type of description for a particular DA. There is a 5 digit code containing the data quality flags which indicate whether suppression at the data quality level has been applied. See example below with the description of the DQ flags:


2006 Census – Data quality flags and descriptions
Data quality flags for place of residence

Digit Description
1st (0XXXX) Incomplete enumeration flag
2nd (X0XXX) 100% data quality flag
3rd (XX0XX) Population and dwelling counts error flag
4th (XXX0X) 20% sample data quality flag
5th (XXXX0) 2001 adjusted population flag

Flag description
Incomplete enumeration flag
0 = Default.
1 = Incompletely enumerated Indian reserve or Indian settlement (suppressed).
2 = Excludes census data for one or more incompletely enumerated Indian reserves or Indian settlements.

100% data quality flag
0 = Default.
1 = Data quality index showing, for the short census questionnaire (100% data), a global non-response rate higher than or equal to 5% but lower than 10%.
2 = Data quality index showing, for the short census questionnaire (100% data), a global non-response rate higher than or equal to 10% but lower than 25%.
3 = Data quality index showing, for the short census questionnaire (100% data), a global non-response rate higher than or equal to 25% (suppressed).

Population and dwelling counts error flag
0 = Default.
1 = An error exists in the 2006 population and dwelling counts for this area. For further details, please refer to the population and dwelling counts data section of the 'Notes' file.
2 = In 2001, the population and/or dwelling counts for this census subdivision were found to be incorrect. Since it is not possible to make changes to the 2001 Census data presented in these tables, the 2001 data should be used with caution. For further details, please refer to the population and dwelling counts data section of the 'Notes' file.
3 = Both the 2006 and 2001 population and/or dwelling counts for this area were found to be incorrect. Since it is not possible to make changes to the census data presented in these tables, these counts should be used with caution. For further details, please refer to the population and dwelling counts data section of the 'Notes' file.

20% sample data quality flag
0 = Default.
1 = Data quality index showing, for the long census questionnaire (20% sample data), a global non-response rate higher than or equal to 5% but lower than 10%.
2 = Data quality index showing, for the long census questionnaire (20% sample data), a global non-response rate higher than or equal to 10% but lower than 25%.
3 = Data quality index showing, for the long census questionnaire (20% sample data) or the short census questionnaire (100% data), a global non-response rate higher than or equal to 25% (suppressed).

2001 adjusted population flag
0 = Default.
1 = 2001 adjusted count; most of these are the result of boundary changes.

Note:
1. The 100% and 20% sample data quality flags do not apply to the population and dwelling counts.

Data quality flag for place of work

Digit Description
4th (XXX0X) 20% sample data quality flag

Flag description
20% sample data quality flag
3 = Data quality index showing, for the long census questionnaire (20% sample data) or the short census questionnaire (100% data), a global non-response rate higher than or equal to 25% (suppressed).

More detailed information can be found here: Data quality and confidentiality standards and guidelines (public): Introduction.”

Wednesday, March 1, 2017

B2020 and MAC

Question
I have a student with a Mac trying to access data in B2020 format.
Does anyone know if there is a work around for this student?


Answer
Re: Beyond 20/20 -- Wed, 30 Jan 2013 12:33:59 -0500 (EST)line 1:
I am using VMware Fusion (http://www.vmware.com/products/fusion/overview.html) on my Mac, which allows me to run an incidence of Windows. Hence, I am able to run Beyond 20/20, GeoSuite, and other Windows-based software. Works well.

RE: Beyond 20/20 --
Wed, 30 Jan 2013 11:14:09 -0500 (EST)

It is possible to turn the Windows B2020 app into a pseudo-native mac app using Wineskin, which is the way UBC distributes the Mac version of Beyond 20/20:
http://hdl.handle.net/11272/YTLIM

Answer 2 
This has the advantage of not requiring drive partitioning or a full installation of a target OS. Using a similar technique, you can get B2020 to run on linux OSs.

You also don't technically need wineskin; you can just have a local installation of WINE and run it that way.

Friday, January 21, 2011

Data by Dissemination Area in Beyond 2020 Files

Question

The researcher was using the 2006 Census Complete Cumulative Profile at the DA level and the PCCF to map socio-economic indicators by postcode - something he´s done before for the 2001 census. He had no problem for most of the country, but for the Saskatchewan he came across an abnormally large number of DAs (740/2300) that don´t appear in the B2020 file. I had a look at a couple of them and there is data for the CSD (the 7 digit number, i.e., 4701001) but not for the DA (8 digits). It doesn´t seem random enough to be a case of data suppression, what am I/he missing?

I can provide a full list of missing DAs if helpful.

Answer

The following 2006 Census dissemination area (DA) level suppression lists describe the DAs for which data have been suppressed and could be used to confirm if data for DAs in the Beyond 20/20 file have been suppressed.

Dissemination area suppression list including census subdivisions in which they are located - 100% data
http://www12.statcan.gc.ca/census-recensement/2006/ref/notes/sup_DA2A-AD2A-eng.cfm

Dissemination area suppression list including census subdivisions in which they are located - 20% sample data
http://www12.statcan.gc.ca/census-recensement/2006/ref/notes/sup_DA2B-AD2B-eng.cfm

If you have any questions about specific DAs, please don't hesitate let me know and I will be happy to ask our census contacts to advise.

I hope that this is helpful.

Thursday, September 16, 2010

Beyond 20/20 & Windows 7

Question

We haven't been able to install Beyond 20/20 on a laptop running Windows 7. We have tried compatibility modes but no luck so far. Any solutions would be greatly appreciated.

Answer

Our contact for Beyond 20/20 here at StatCan has confirmed that Beyond 20/20 is compatible with Windows 7. He is aware of an installation issue however that has to do with difference between the Beyond 20/20 installers for 32 bit and 64 bit operating systems. His description of this issue and suggestion follow:

"There are 2 different installers for B2020 currently available on our website, one for 32 bit Operating Systems and one for 64 bit Operating Systems. It is possible that the user has tried to install the 32 bit version on the 64 bit Operating System so I would look to that as the first possible solution.

I believe both install packages are available on the DLI FTP site (note: we will make sure that both install packages are on the DLI FTP site) but I am providing the link to the 64 bit below. Please have your client try the .zip file (unzip the package and run 'setup.exe'). If the problem still occurs we would probably need a little more detail as to the error message being generated so that we can pass that along to the company.

64 bit installer (zip)
http://www2.beyond2020.com/SC/probrowser.zip"

I hope that this resolves the issue you are experiencing. Please let us know if this works.

Friday, January 23, 2009

Beyond 20/20 version 7.0 vs. version 7.3 SP3

Question
No doubt this should be obvious to me, but I don't know what the
difference is between the two versions.  I've looked at guides for both
of them and they are almost identical.  What does SP3 mean?  I assume it
is a verson reference but then it should say version...

Answer

1) SP3 = Service Pack 3

It's a slightly more up-to-date version of 7.0 that corrects some
bugs without affecting functionality.

2) From the "what's new" document that comes with the program, it
seems that SP3 accumulates the following changes to version 7.0:

- new feature: Cell coloring for footnotes
- bug fixes:
-- Under certain conditions, tables containing footnotes
may display incorrect data.
-- Errors when creating and opening extracts with more
than 999 fields.

That's it!

Beyond 20/20 user tutorials and guides

Question

We can no longer link to these guides (1) or our copies (2) are out of date. In follow-up, I was unable


to find the following user tutorials and guides from Statistics Canada.



Would it still be possible to access current versions?



(1) Formerly on the StatCan Web site:



Initiation aux tableaux de Beyond 20/20


http://www.statcan.ca/english/agcensus2001/tutorials/viewlets/browsing_f_viewlet.html OR


http://www.statcan.gc.ca/english/agcensus2001/tutorials/viewlets/browsing_f_viewlet.html


Getting Started with Beyond 20/20 Tables


http://www.statcan.ca/english/agcensus2001/tutorials/viewlets/browsing_viewlet.html OR


http://www.statcan.gc.ca/english/agcensus2001/tutorials/viewlets/browsing_viewlet.html



Manipuler des dimensions en Beyond 20/20


http://www.statcan.ca/english/agcensus2001/tutorials/viewlets/nesting_f_viewlet.html OR


http://www.statcan.gc.ca/english/agcensus2001/tutorials/viewlets/nesting_f_viewlet.html


Manipulating Dimensions in Beyond 20/20


http://www.statcan.ca/english/agcensus2001/tutorials/viewlets/nesting_viewlet.html OR


http://www.statcan.gc.ca/english/agcensus2001/tutorials/viewlets/nesting_viewlet.html



(2) Formerly available from the DLI FTP site as I recall...



Beyond 20/20 «Browser» : Guide d'initiation rapide (InitiationRapide.pdf)


Beyond 20/20 Browser: QuickStart Guide(QuickStart.pdf)



Beyond 20/20 «Browser» : Guide d'initiation (BrowserFra.pdf)


Beyond 20/20 Browser: User Guide (BrowserEng.pdf)


Answer


For question 2.



The Following guides should be in your b2020 browser dir

c:\Program Files\Beyond 2020\Professional Browser\Document

I have also place a copy of the files below on the ftp site

/dli/util/b2020_doc

(2) Formerly available from the DLI FTP site as I recall...

Beyond 20/20 «Browser» : Guide d'initiation rapide (InitiationRapide.pdf)
Beyond 20/20 Browser: QuickStart Guide(QuickStart.pdf)
Beyond 20/20 «Browser» : Guide d'initiation (BrowserFra.pdf)
Beyond 20/20 Browser: User Guide (BrowserEng.pdf)

For question (1) I will forward it to the division.

Wednesday, October 15, 2008

Access to Beyond 20/20

Question
I am a little unclear about who can have access to B 20/20. I have a request from a professor who would like his students to have access to B 20/20 available to load on their personal (i.e. laptop) computers. I suspect if this were possible we would not have it on our FTP server but have it more freely available elsewhere, but I want to be sure.

Answer
It's available for students via the ESTAT page on the Statistics Canada site. It's also available at the bottom of every 2006 census data product (TBT or Profile) page, if viewed from a computer authenticated to DSP. Outside the authentication area, the links do not appear.

Friday, February 15, 2008

Geographic codes in justice files

Question

I have a researcher who would like to link the crime statistics data in the B20/20 justice files to over 500 corresponding census subdivisions. I know that the data in the justice tables is at the detachment level; however, there is a geographic code attached to each detachment which looks census-like. Is there a relationship between the geographic codes in the justice tables and census geography below the provincial level? I suspect that the answer is no but if there is a correspondence, it would make merging the researcher's census data with the crime data a whole lot easier. Any assistance with this would be very much appreciated.

Answer

You were correct, the answer is no. I asked CCJS and the UCR survey managers and analysts have told us that the first two digits are the standard province code in which the detachment resides. The other three digits are sequentially assigned and do not reflect any geographical boundaries census or otherwise. Your researcher should therefore not link these codes to census subdivisions or any other geographic level below the province.

Monday, January 14, 2008

Publication of Beyond 20/20 Tables in Textbook

Question

I know part of the answer to this question, but I was hoping for a bit more clarification. A professor here is writing a textbook and wants to generate tables in Beyond 20/20 (I think using homicide/victim/crime data) and wants to know if she can use them in her book. Now, I'm pretty sure that if she is getting the B2020 tables from DLI, the answer is NO and that she would need to seek permission directly. However, what if she is getting them from E-Stat? I suspect the answer is still no because E-Stat is for educational use? AND, what if she generates them on the Stat-Can website and then pays for the time series that way? Any help/clarification you can provide is much appreciated.

Answer

Publishing Statistics Canada products in textbooks falls outside the DLI license. The Regional Office should help your professor get the approval she needs.

The DLI Licensing database on the DLI website says, "Faculty are referred to the Marketing Division of Statistics Canada for discussions on textbook licensing."

Thursday, November 15, 2007

Benefits of Beyond 20/20

Question

I am interested in descriptions how Data Librarians offer and when they used Beyond 20/20 data.

Answers

1) The B2020 files have a great deal of value. The first thing is the interface. It is extremely intuitive and powerful - students pick it up very quickly. (the intuitiveness of B2020 has been challenged by some)

The second is the type of data and how it is indexed/presented. I find it easy to wade through census tables using B2020.

If you are looking to introduce data to non-data librarians this is a good door in.

2) Since so many commonly used resources are only available in Beyond 20/20 format, I don't see how one can avoid using it:
- Annual estimates of employment earnings and hours
- Canadian business patterns
- Labour force historical review
- Income trends in Canada
- Canada's retirement income programs
- CCJS statistics
- not to mention the cd-rom versions of some UN and pretty much all OECD products

Wednesday, November 14, 2007

Beyond 20/20 Sorting Question

Question

With regard to census data, if I only want the top Province level data in any given file - how would I do that without selecting each line separately? Is there a trick here? Right now - as far as I'm aware the only way to obtain this would be to hold my Ctrl button done, scroll through the file and select each province.

Answer

1. Make the "Geography" dimension active.
2. Click on that double-arrowed "Sort" icon above.
3. Choose "Code" & "Increasing Numeric" and "Ok".

The result should be Canada, then all provinces, then all territories.

You can also find a B2020 Help Sheet at
http://www.library.carleton.ca/ssdata/surveys/b2020.html along with links to the B2020 User Guides.

Wednesday, January 31, 2007

Mac Version of Beyond 20/20

Question

Does anyone know if there is a Mac version of the Beyond 20/20 browser?

Answer

Beyond 20/20 was never made for Macs, but there is an alternative available:

a) Using a newer MAC, you can install Bootcamp software which will allow you to install a version of Windows, which in turn will allow you to run any software on the windows platform (Beyond 20/20, Microsoft Office, etc).

b) Using older MACs, you can install Virtual PC software which will perform the same function.

Thursday, October 6, 2005

New Version of Beyond 20/20

Question

Is the Version 7.0 of the Beyond 20/20 Browser available?

Answer

CD-ROM products started using version 7.0 at the beginning of this calendar year. I am not sure of the exact number of products, but possibly 3 or 4 so far.

Version 7.0 of Beyond 20/20 is now available on the FTP site at the following address: /ftp/dli/util/b2020-70.exe

Installing Beyond 20/20

Question

I have been trying to get Beyond 20/20 installed on computers at the University -- particularly in the computer labs. I was able to get our systems people to do this in the past, but now they seem to have an issue with this. I received the following response as a result of a recent request:

"The Beyond 20/20 browser does not obey group policy, it cannot be installed on ADM machines. If there is a new version of the application available please let me know and I can test that to see if they have fixed the application. The easiest way to fix the problem would be to use standard Windows Open dialog boxes."

Has anyone experienced this at their institution? Does anyone know if the problem this systems person is describing is being addressed? I would very much like students to be able to open and work with IVT files at any computer in the university, particularly those in the labs.

Answer

2005: As far as I know there are no restrictions to the number of desktops that can have the Browser installed within the DLI community.

From the Past: Several years ago I requested the installation of the B2020 browser in our campus computing labs to increase access to the 1996 Census tables (like i said, the request was several years ago.) The Computing Centre would not install the browser until they had a license confirming that the copy they were installing was legitimate. Lynda Richardson, who was the DLI Liaison at the time, provided me with a document stating that the U of A was licensed through DLI to installed B2020 on our
campus.

I'm passing along this "old" news in the event that your computing centre raises the same concern. The DLI Unit will provide you with a document that you can give the software license administrator on your campus.

Wednesday, February 16, 2005

Using Beyond 20/20 to Convert to SPSS

Question

I'm looking for highest level of education by CSD for Newfoundland. It needs to be in raw data format (It's for an SPSS based stats course). Is this data, from either from 1996 or 2001 census, available as a PUMF? I can only find B2020 tables.

Answers

1) Your initial question sounded like you want CSD to be the unit of analysis: "I'm looking for highest level of education by CSD for Newfoundland." If this is the case, you were very close to solving your own question using the 2001 Census table, 95F0491XCB01002.IVT.

Here is an example of what a raw data format of this table for just CSD's from Newfoundland would look like using only the
first eight CSD's and first four Census characteristics:

Portugal Cove South (1001105) T 00000 60 20 35 0
Biscay Bay (1001109) T 00000 0 0 0 0
Trepassey (1001113) T 00000 135 25105 0
St. Shott's (1001120) T 00000 40 0 35 0
Division No. 1, Subd. U (1001124) SUN 190 40145 0
Cape Broyle (1001126) T 00000 70 60 10 0
Renews (1001131) T 00000 80 65 20 10
Fermeuse (1001136) T 00000 50 10 25 15

The columns of numbers represent, respectively, Total population 15 to 24 years by school attendance, Not attending school,
Attending school full time, and Attending school part time. These are just the first four Census characteristics of 42 contained in 95F0491XCB01002.IVT.

How to get this into SPSS quickly? Use B2020 to create the data structure by reorganizing the table so that Geography is
in the rows, Census characteristics are in the columns, and you've selected just CSD's in Newfoundland. Then do a File / Save As and output the data using the Excel Worksheet format.

Open this file in Excel and remove the first four rows in the spreadsheet, which contain stub headings from the B2020 table,
and re-save this file.

Next, start an SPSS session and use File / Open / Data and select Excel as the file type to open. In the "Opening Excel Data Source" dialogue box, uncheck the box for "Read variable names from the first row of data."

2) If I understand what you are looking for correctly, you are looking for the ascii version of table 95f0418xcb01006.ivt, yes?
That's the BST for highest degree, certificate or diploma, by age groups and sex, for census subdivisions.

You will find it in ascii format on the DLI ftp site, in
census/2001/Ascii/Topic-Based-Tabulations/education-in-canada-attendance-ascii/

There are no SPSS commands, but they could be created from the accompanying documentation. You will need to write a data list statement for a number of records per case (ie CSD).

The alternative to using the ascii file would be to use the Beyond 20/20 file to structure the rows the way you want them, then write the file out as a .csv or .xls file, and read that in to SPSS. It would be rather easier than working with the ascii file, at least while we can still read .ivt files.