Describe each survey applying key concepts from the textbook
Introduction
This assignment examines and applies concepts in the Survey Methods chapter and gives you the opportunity to evaluate surveys.
Instructions
- Using the principles in Chapter 7 of surveys, locate 2 surveys (can be related to your area of interest). One of these should be a survey/ questionnaire that you will justify as high quality. The second survey should be one that you will justify as poor or deficient.
- In an APA Style paper, present your responses to the following prompts:
- Describe each survey applying key concepts from the textbook [be sure to describe the:
- Purpose or goal of the surveys
- IV’s and DV’s ****the items (IVs) that are presented and the ways participants can respond (DVs),
- Critique of the surveys along with a justification of whether you judged it as high quality or poor quality.
- A statement about improving upon the surveys.
- Be sure to include internal citations and references for both surveys.
Your paper must be presented in proper APA 7e format.
Your paper should have a cover page, text body section, and a references. An abstract is not necessary – do not put an abstract into your paper.
Book: Making Sense of the Social World Methods of Investigation Sixth Edition Daniel F. Chambliss
Survey Research
Chapter 7
1
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Learning Objectives
Explain the strengths and weaknesses of omnibus surveys.
Explain the problem of sampling on the dependent variable.
Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions.
2
2
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Learning Objectives
List the different methods for improving survey questions.
Outline a cover letter for a survey that contains each of the required elements.
List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Discuss the key ethical issues in survey research.
3
3
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
Why Is Survey Research So Popular?
Survey research
Popularity due to three advantages:
Versatility
Efficiency
Generalizability
Omnibus survey
4
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Survey research: Research in which information is collected from a sample of individuals through their responses to a set of standardized questions. Omnibus survey: A survey that covers a range of topics of interest to different social scientists.
4
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Be Clear; Avoid Confusing Phrasing
Simple, direct approach
Avoid negative phrases or words
Especially double negatives
Avoid double-barreled questions
Identify what information each question is to obtain
5
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
Double negatives: A question or statement that contains two negatives, which can muddy the meaning of the question.
Double-barreled question: A single survey question that actually asks two questions but allows only one answer.
5
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Minimize Bias
Words should not trigger biases (unless this is the intent)
Must test reactions to phrasing of a question.
6
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
6
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Allow for Disagreement
Some respondents “agree” with a statement just to avoid disagreeing.
Present both sides of attitude scales in question itself.
Phrase all response choices in socially approved ways.
7
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
7
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Don’t Ask Questions They Can’t Answer
Respondents should be competent to answer questions.
Kinds of questions to mitigate this:
Filter questions
Skip pattern
Contingent question
8
Learning Objective: 7-1: Explain the strengths and weaknesses of omnibus surveys.
Learning Objective: 7-4: List the different methods for improving survey questions.
Filter questions: A survey question used to identify a subset of respondents who then are asked other questions.
Skip pattern: The unique combination of questions created in a survey by filter questions and contingent questions.
Contingent question: A question that is asked of only a subset of survey respondents.
8
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Allow for Uncertainty
Different types of respondents:
Floaters
Fence-sitters
Can be managed by . . .
Including “no opinion” category
Including neutral category
Adding open-ended questions
9
Learning Objective: 7-3: Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions.
Learning Objective: 7-4: List the different methods for improving survey questions.
Floaters: Survey respondents who provide an opinion on a topic in response to a closed-ended question that does not include a “Don’t know” option but who will choose “Don’t know” if it is available.
Fence-sitters: Survey respondents who see themselves as being neutral on an issue and choose a middle (neutral) response that is offered.
9
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should We Write Survey Questions?
Make Response Categories Exhaustive and Mutually Exclusive
Must be one and only one possible response for everyone who is asked the question.
Two exceptions:
Filter questions
“Check all that apply” questions
10
Learning Objective: 7-3: Discuss the advantages and disadvantages of including “don’t know” and neutral responses among response choices and of using open-ended questions.
Learning Objective: 7-4: List the different methods for improving survey questions.
10
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Questionnaire
Interview schedule
11
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Questionnaire: A survey instrument containing the questions in a self-administered survey.
Interview schedule: A survey instrument containing the questions asked by the interviewer in an in-person or phone survey.
11
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Build on Existing Instruments
Use an existing instrument if it measures the concept of concern.
Consider appropriateness for your survey population.
12
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
12
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Refine and Test Questions
Methods for testing questions:
Cognitive interview
Conduct a pilot study
Interpretive questions
13
Learning Objective: 7-4: List the different methods for improving survey questions.
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Cognitive interview: A technique for evaluating questions in which researchers ask people test questions, and then probe with follow-up questions to learn how they understood the question and what their answers mean.
Interpretive questions: Questions included in a questionnaire or interview schedule to help explain answers to other important questions.
13
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Maintain Consistent Focus
Have measures of all independent and dependent variables you plan to use.
Long lists of redundant or unimportant questions bore respondents.
Omnibus surveys are an exception.
14
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
14
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Order the Questions
Sort questions into thematic categories.
First question should . . .
Interest the respondent
Be easy to answer
Apply to everyone
NOT be sensitive
Context effects
15
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Context effects: In survey research, refers to the influence that earlier questions may have on how subsequent questions are answered.
15
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
How Should Questionnaires Be Designed?
Make the Questionnaire Attractive
Plenty of white space between questions.
List response choices vertically.
Distinguish response choices clearly.
Indicate skip patterns with graphics.
Use distinctive formatting for instructions.
Booklets are better than multi-pagers.
16
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
16
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Surveys can be administered in five ways
Each approach differs in one or more way:
Manner of administration
Questionnaire structure
Setting
Cost
17
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Learning Objective: 7-5: Outline a cover letter for a survey that contains each of the required elements.
17
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
Mailed, Self-Administered Surveys
Low response rate hurts representativeness.
Response rates can be improved with a cover letter.
18
Learning Objective: 7-6: List the strengths and weaknesses of each mode of survey design, giving particular attention to response rates.
Learning Objective: 7-5: Outline a cover letter for a survey that contains each of the required elements.
Mailed (self-administered) survey: A survey involving a mailed questionnaire to be completed by the respondent.
Cover letter: The letter sent with a mailed questionnaire that explains the survey’s purpose and auspices and encourages the respondent to participate.
18
Chambliss, Making Sense of the Social World, 6e. © SAGE Publishing, 2020
What Are the Alternatives for Administering Surveys?
19
