Skip to content

Assignment 2 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cvasoff merged 2 commits into from
Jan 20, 2026
Merged

Assignment 2 #2

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0803f2e
Assignment 2 submission
cvasoff Jan 14, 2026
8b049f2
Merge branch 'main' into assignment-2
cvasoff Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 108 additions & 14 deletions 02_activities/assignments/a2_survey_design_and_evaluation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,38 +40,132 @@ For the **Canadian General Social Survey on Giving, Volunteering, and Participat

## Part A - Survey Design:

The number of your chosen topic: `#`
The number of your chosen topic: `1`

Describe the purpose of your survey:
```
write your answer here...
The purpose of the survey is to:
1. Explore reasons why employees at our large tech company is experiencing a high turnover across many of its departments, with a specific focus on two employee levels: entry-level and lower-level positions.
2. Propose possible changes to improve employee satisfaction at the company based on the reasons for high turnover uncovered by the survey.
```

Describe your target population, sampling frame, sampling units, and observational units:
```
write your answer here...
Target population: All of the large tech company employees who occupy entry- or lower-level company positions.

Frame population: All entry- or lower-level company employees at the time the survey is conducted.

Sampling frames:
1. First stage sampling frame is the list of all company departments. A simple, random sample of departments is selected from this list. The number of departments selected of will depend on the size of the company and the desired precision and power of the survey results. The first stage sampling units are company departments.
2. Second stage sampling frame is the complete list of all entry- and lower-level position employees within the selected departments. A simple, random sample of entry- or lower-level position employees is selected from within each department selected in stage one. The second stage sampling units are individual employees.

Observational units: Individuals currently employed by the tech company for whom their employee position level is clearly defined using ordinal, company-defined employee levels.

Overall sampling strategy is two-stage, cluster sampling, where:
1. The company is clustered into groups by department.
2. A simple random sample of departments is selected from these clusters.
3. Within each selected department, obtain a complete list of all entry- and lower-level position employees.
4. A simple random sample of individual employees is selected from this list, and the the survey questionnaire is distributed to them.

Sampled population: All selected individual employees from Step 4. above, who complete the survey.
```

Your 5-10 question survey:
```
1. write your question here...
2. write your question here...
3. write your question here...
4. write your question here...
5. write your question here...
6. write your question here... (optional)
7. write your question here... (optional)
8. write your question here... (optional)
9. write your question here... (optional)
10. write your question here... (optional)
1. Excluding any approved absence(s), how long have you been continuously employed with (insert company name)? Response categories: (less than 1 year)/(1 to less than 2 year)/(2 to less than 3 years)/(3 to less than 4 years)/(4 to less than 5 years)/(5 or more years).
2. How long have you worked at your current position? Response categories: (less than 1 year)/(1 to less than 2 year)/(2 to less than 3 years)/(3 to less than 4 years)/(4 to less than 5 years)/(5 or more years)
3. Are you satisfied that your current level of salary is competitive compared to other employment opportunities available outside (insert company name)? (Scaled response, 1-10)
4. Are you satisfied that you current total benefits package, including vacation time, paid personal time, and health benefits, is competitive compared to other employment opportunities available outside (insert company name)? (Scaled response, 1-10)
5. Does your current position offer you opportunities for advancement to a higher-level position within a reasonable time frame, assuming that is something you are currently seeking? (Yes/No)
6. Are you currently applying or interviewing for work opportunities external to (insert company name)? (Yes/No)
7. (If answer to Q6 is "Yes"): What is your primary motivation for seeking external work opportunities at this time? Response categories: Higher salary/Opportunities for advancement/Hybrid work placement/Flexible working hours/Vacation plus paid time away
8. Within you current position, are you actively working towards a career advancement opportunity at (Insert company name)? (Yes/No)
9. (If answer to Q8 is "No"): Does (insert company name) provide the support you require to remain satisfied in your current role? (Yes/No)
10. Please describe how (insert company name) could improve your work satisfaction with in your current role. (200 characters or less)
```

## Part B - Survey Evaluation:

Identify and describe survey features:

```
write your answer here
Survey desription link: https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&Id=796234

1. Sample type
"This is a sample survey with a cross-sectional design." It is "based on a stratified design employing probability sampling". It also used "an approach called 'rejective sampling'", such that the sample of survey respondents who indicate that they are not volunteers are randomly divided into two groups: (a) one group is asked to complete the same long interview as respondents who are classified as a volunteer, and (b) the other group is asked to complete a short interview.

2. Sample size
"A field sample of approximatively 50,000 units was used. Among them, about 40,000 invitation letters to the electronic questionnaire were sent to selected households across Canada. A completion of 24,000 questionnaires was expected."

3. Target population
"The target population for the GSS Giving, volunteering and participating includes all persons 15 years of age and older living in the ten provinces of Canada. It excludes full-time (residing for more than six months) residents of institutions."

4. Sampling frame (Is the list of sampling units)
Sampling frame is a list comprised of each randomly selected single, eligible individual within single households, which were identified using the survey's two-stage sampling design. The two sampling design stages are: 1. stratify Canada's ten provinces into 27 geographic areas (i.e.., CMAs or CMA size), representing the survey's target population, and 2. group telephone numbers (landline and/or cellular) associated with a single household address.

5. Survey mode(s)
"Data are collected directly from survey respondents either through an electronic questionnaire or through CATI (computer assisted telephone interviewing). No proxy reporting is allowed. The respondents has the choice between French and English."

6. Timeline
"Reference period: Past 12 months preceding interview date
Collection period: Every 5 years, from September to December"

7. Response rate
"The overall response rate is 41.9%."

8. Weights
- The weighting process is complex, and includes several types of weighting:
- A person weight is applied to each respondent so "that each person selected in the sample represents (in addition to himself/herself) several other persons not in the sample". In this survey, the person weighting also adjusts "for the 'rejecting' of a proportion of respondents that are not volunteers, the person weight for respondents that are not 'rejected' and are not volunteers" (i.e., non-volunteer respondent weighting is increased to account theri survey "rejection")
- Also reweighting individual respondent units to match "the weighted income distribution of GVP" with the "2017 CIS distribution" (Canadian Income Survey) for the respondent's province. See https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&Id=424633
- Bootstrap weighting of the survey estimates to calculate survey design-based variance.
- Survey data estimates are also weighted to reflect the survey target population characteristics, including "various age-sex groups by province"

7. Data processing
- "Data are collected directly from survey respondents" using either an electronic questionnaire or CATI.
- The survey information collected is "linked to the personal tax records (T1, T1FF or T4) of respondents, and tax records of all household members. Household information (address, postal code, and telephone number), respondent's information (social insurance number, surname, name, date of birth/age, sex) and information on other members of the household (surname, name, age, sex and relationship to respondent) are key variables for the linkage."
- The survey used Statistics Canada's SSPE (Social Survey Processing Environment), which "set of generalized processing steps and utilities to allow subject matter and survey support staff to specify and run the processing of the survey in a timely fashion with high quality outputs. " (See https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=5024 for the SSPE acronym name.)

8. Cleaning, imputation, etc.
- "Edits were performed automatically and manually at various stages of processing at macro and micro levels."
- "A series of checks were done to ensure the consistency of survey data."
- "Flow edits were used to ensure respondents followed the correct path and fix off-path situations."
- "Error detection was done through edits programmed into the CATI system."
- "All survey records were subjected to computer edits throughout the course of the interview."
- "Head office performed the same checks as the CATI system as well as the more detailed edits discussed previously."
- Imputation was performed in nine steps: 1. "personal income and family income", 2. to 4. involved "formal volunteering variables in the master file", 5. to 6. involved "informal volunteering variables in the master file", and 7. to 9. involved "variables in the donation file and the solicitation methods in the master file".
- Quality evaluation, including quality assurance mechanisms, and "validation and scrutiny of the data by statisticians" including (a) "Analysis of changes over time", (b) "Verification of estimates through cross-tabulations", and (c) "Confrontation with other similar sources of data".

9. Sources of error
- Household family relationships determination
- Other reported respondent characteristics, which could be checked against household members' personal tax records (e.g., respondents' age, as compared to their birthdate)
- Questionnaire response flows by respondents completing the CATI survey method
- For all survey records, computer and head office editing of "the course of the interview"
- Respondent characteristic variables' imputation accuracy
- Non-sampling error:
- Non-response bias:
- At either level of the two=stage survey information collection: "at the household level and at the individual level"
- Coverage error:
- "Households without telephones, as well as households with telephone services not covered by the current frame, represent a part of the target population that was excluded from the surveyed population."
- Dependance on linked files: "The frame for GSS was created using several linked sources, such as the Census, administrative data and billing files."
- Reliance on weighting methods: "Survey estimates were adjusted (weighted) to represent all persons in the target population, including those not covered by the survey frame."
- "Other types of non-sampling errors can include response errors and processing errors."

10. Limitations, known biases, etc
- Survey estimates comparability with previous General Social Survey on Giving, Volunteering and Participating (GSS GVP) iterations may have been limited by the following 2018 survey changes and updating:
- The 2018 survey introduced an internet (i.e., electronic) option to survey respondents. Statistics Canada comments: "It is impossible to determine with certainty whether, and to what extent, differences in a variable are attributable to an actual change in the population or to changes in the survey methodology. However, there is reason to believe that the use of an electronic questionnaire had an impact on the estimations. For this reason, it is not appropriate to compare results from 2018 GSS GVP with previous iterations."
- Consistent with "changing international standards regarding the definition of volunteering," the 2018 survey was "reworked."
- Other changes included updating online technology and social media references.
- For the electronic option, (new) "thresholds and limit values associated with volunteer hours and donation amounts were put in place"

11. Link to documentation and any additional sources used
- https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&Id=796234
- https://www23.statcan.gc.ca/imdb/p3Instr.pl?Function=assembleInstr&a=1&&lang=en&Item_Id=1183690
- https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getMainChange&Id=796234
- Wu, C., & Thompson, M. E. (2020). _Sampling Theory and Practice_. Springer International Publishing. [https://doi.org/10.1007/978-3-030-44246-0]
- Course slides, especially Slide Deck 3 (weights)
- From: https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=5024
"Processing used the Social Survey Processing Environment (SSPE) set of generalized processing steps and utilities to allow subject matter and survey support staff to specify and run the processing of the survey in a timely fashion with high quality outputs.
It used a structured environment to monitor the processing of data ensuring best practices and harmonized business processes were followed."write your answer here
```

## Rubric
Expand Down