These are examples of questions that I presented in class on Tuesday, September 22.  There are some additional examples here that I did not get to in class

 

 
A GOOD SURVEY QUESTION SHOULD:
    

 
 
BREVITY AND SIMPLICITY ARE VIRTUES IN SURVEY QUESTIONS. 

HOWEVER, SOMETIMES IT IS NOT POSSIBLE TO CLEARLY CONVEY THE ISSUE YOU ARE ASKING ABOUT IN A FEW WORDS.  HERE ARE THE  QUESTIONS THAT WE DEVELOPED TO ASK OPINIONS ABOUT "SCHOOL CHOICE".  

We begin with a preamble that defines the issue, and then we make the distinction between public and private schools.  

We then asked separate questions about church related and non-church related schools. 

We made a great effort to avoid leading the respondent, and to avoid accusations of bias. 
 

There has been a lot of discussion of the pros and cons of providing assistance to families who choose to send their children to parochial or other private schools. Some people feel that tax money should be used only for the public schools. Others feel that it is all right to provide tuition support to parents who send their children to non-religious private schools, while others feel that such support should be available to parents with children in parochial schools as well.

Do you favor or oppose providing tuition assistance to families who chose to send their children to church-related schools? Are you strongly in favor, somewhat in favor, somewhat opposed, or strongly opposed?

What about providing tuition assistance to families who choose to send their children to private schools that are not church-related? Are you strongly in favor, somewhat in favor, somewhat opposed, or strongly opposed?
 
 

Because the real policy issue is subsidizing the education of low-income families, we originally had "low and moderate income" before "families" in the question.  We eliminated these words because we thought that we might introduce bias.  

In retrospect, we are not sure that this was a good decision.

 
 
WHILE WE ARE ON THE SUBJECT OF EDUCATION, HERE ARE A COUPLE OF QUESTIONS FROM A SERIES ON EDUCATION THAT APPEARED IN A RECENT SURVEY.
THE FIRST IS A CLASSIC DOUBLE-BARREL
AND THE SECOND IS LACKING CLARITY.
 

There has been a lot of discussion of things that might be done to improve the quality of education that our children receive in the public schools. For each idea, I’d like you to tell me if you think it would improve kid’s academic achievement. Use a 5 point scale where 5 means it definitely would improve academic achievement and 1 means it would not improve academic achievement at all.

Freeing up teachers from clerical chores they have to do, and paying them as real professionals
 
Making what happens in the classroom between the teacher and the student most important in education
 

 
 
 
WE NORMALLY TRY TO AVOID LOADED LANGUAGE IN SURVEY QUESTIONS.   HERE IS AN EXAMPLE OF A QUESTION THAT USES SOMEWHAT LOADED LANGUAGE ON PURPOSE, PRIMARILY BECAUSE WE WERE UNABLE TO COME UP WITH WORDING THAT CONVEYS THE MEANING WE WERE LOOKING FOR WITHOUT DOING SO. 

NOTE ALSO THE INCLUSION OF SPECIFIC TEMPORAL REFERENCE
 
>D5< Since the beginning of the current academic year, how many times, if any, have you felt that you have been "given the runaround" by the University's administrative bureaucracy?
  

 
 
 
Example of Wordy Introduction to Set Frame of Reference and Define Exactly What is Being Asked About in Current and Subsequent Questions – make distinctions that we want R to make in answering question.
 

>D1y< Sometimes students have difficulty getting into courses that they wish to take. Sometimes space is available in a course, but not at convenient times, and sometimes students are unable to get into a course at all because there are no spaces available. First we want to ask about your experience of not being able to get into a course at all.

Since the beginning of the current academic year, how many times, if any, have you been unable to get into courses required for your major, or courses which are prerequisites for courses required for a major?

 

>D2< (Since the beginning of the current academic year, how many times, if any, have you) been unable to get into other introductory courses that you really wanted to take?

 

>D3 < (Since the beginning of the current academic year, how many times, if any, have you) been unable to get into courses you wanted to take in order to explore interest in a possible major?

 

>D4< (Since the beginning of the current academic year, how many times, if any, have you) not been able) to take courses that you wanted to take because there was not space available at times that were convenient to your schedule?


 
 
 
TEMPORAL SPECIFICITY 
 

>G1< Next, we have some questions about course work and instructors. In answering this set of questions,  please consider all of the courses you have taken since the beginning of the current academic year.

How do you rate the overall quality of instruction? Would you rate it as excellent,
very good, good, fair, or poor?  

 
We were interested in measuring respondents’ participation in organizations. We decided that we would get better measurement if we listed specific types of organizations, rather than just asking about organizations in general.  

Note: this is from a self-administered questionnaire 
 

Here is a list of various kinds of organizations. How often, if at all, do you participate in each type of organization? 


 
 
USE OF EXAMPLES IN QUESTIONS:  LOOK AT PART d OF THIS QUESTION.  WE WERE LOOKING FOR EXAMPLES OF GROUP RECREATIONAL ACTIVITIES THAT WOULD COVER A BROAD RANGE OF AGES, SOCIAL AND ECONOMIC GROUPS, AND REGIONS OF COUNTRY.  THIS IS WHAT WE CAME UP WITH. 

WE WERE ALSO INTERESTED IN PUTTING SOCIABILITY WITH KIN IN THE CONTEXT OF SOCIALBILITY WITH OTHERS.   
 
 

16. About how often do you do the following things:

      a.     Spend a social evening with:

 
 
 


 
We wanted to find out which spouse "initiated" a recent marital separation. Rather than focusing on the precipitating event or crises before the separation, we asked about states of mind – who wanted most for the marriage to end.
 
 
Note: this is from self-administered questionnaire.  If this were interviewer-administered, we would have had to pose the question differently. 
 

Sometimes both partners equally want a marriage to end, other times one partner wants it to end much more than the other. Circle the answer that best describes how it was in your case.

    1    I wanted the marriage to end BUT my husband/wife did not.
    2    I wanted it to end MORE THAN my husband/wife did.
    3    We both wanted it to end
    4    My husband/wife wanted it to end MORE THAN I did.
    5    My husband/wife wanted it to end BUT I did not.
     
    

 
 
This is an example of temporal specificity – the question came in the context of a lot of retrospective questions about the separation, so we specified "current" relationship. We chose to use a "friendly – unfriendly" scale. 
 

 
Here is a question where we asked respondents to speculate about the permanence of their relationship. Note the preamble clause, and the use of the word "realistically". 
   

 
 

This is an example of a question on a sensitive topic- relationship violence. Note the preamble "Sometimes . . ." 

Note also the definition of violent acts so that they know that we want to include things like shoving and throwing things. Similarly, we ask first about the occurrence of violence, and then about injuries.
 

8. Sometimes arguments between partners become physical. During the lastyear has this happened in arguments between you and your husband/wife? 

IF YES

  

 
 
This is an example of asking hypothetical questions – we are trying to get people to assess the effect of a possible separation on their own and their partner’s lives. Again, we use an introductory clause so that the respondent to get the respondent to think hypothetically.
 
13. Even though it may be very unlikely, think for a moment about how various areas of your life might be different if you separated. For each of the following areas, how do you think things would change?  
THE SAME QUESTION SERIES WAS REPEATED WITH R REPORTING FOR SPOUSE PARTNER.  THIS IS NOT REALLY A PROXY REPORT BECAUSE WE ARE ASKING FOR R'S PERCEPTION OF THE EFFECT ON THE SPOUSE/PARTNER.  WE ARE NOT ASSUMING THAT R'S REPORT IS WHAT THE SPOUSE/PARTNER WOULD REPORT FOR THEMSELVES.
14. How about your husband/wife? How do you think these various areas of life might be different for him/her if you separated?
 


 
 
WE USED AN INTRODUCTORY SENTENCE TO DEFINE WHAT WE WERE COVERING IN THE QUESTION - A SECOND MAJOR OR CERTIFICATE
 

>A1g< Some students complete more than one major, or complete the requirements of a certificate program in addition to a major. How about you, do you think that you will graduate with more than one major or with a certificate in addition to a major?
  


 
HERE IS AN EXAMPLE OF THE NEED TO ANTICIPATE THE RANGE OF RESPONSES THAT MIGHT BE EXPECTED SO THAT EVERY ANSWER HAS A PLACE TO GO.
 

  A5< What is the highest degree you ever expect to earn?

 
 
  

 
ANOTHER EXAMPLE OF THE USE OF A PREAMBLE TO SET THE FRAME OF REFERENCE OF THE QUESTION, AND TO SHOW RESPONDENTS THAT WE RECOGNIZE THAT PEOPLE VARY IN LEVEL OF COMFORT AND ARE UNCOMFORTABLE FOR A VARIETY OF REASONS.
 

>B4< There are many factors that affect how comfortable students feel as a part of the UW-Madison community. Some students feel quite comfortable, while others are quite uncomfortable. How about you, would you say that you are very comfortable, somewhat comfortable, somewhat uncomfortable, or very uncomfortable as a part of the UW-Madison community?

  


 
 
EXAMPLE OF A SERIES WITH COMMON RESPONSE OPTIONS

 NOTE THAT WE EXPLICITY MENTION WHAT TO DO IF R HAS NO KNOWLEDGE OR CONTACT  

NOT ALSO THE TEMPORAL SPECIFICATION - "the current academic year" 

LATER WE WILL LOOK AT HOW THESE QUESTIONS ARE ADMINISTERED IN AN INTERVIEW SITUATION.  WE WILL SEE HOW IN SOME CASES THE STEM OF QUESTION  ("How do you rate") AND THE RESPONSE OPTIONS IN A SEQUENCE LIKE THIS ONE DO NOT NEED TO BE REPEATED.  
 
 

>E1< Based on your experiences during the current academic year, how do you rate the following services and facilities on campus? If you have had no contact with or knowledge of the service or facility, just tell me that.
How do you rate the libraries? Would you rate them as excellent, very good, good, fair, or poor?

 

  


 
 
WE WANTED TO KNOW HOW OFTEN RESPONDENTS SPANK THEIR CHILDREN.  SPANKING IS SOMETHING THAT PARENTS MAY NOT WANT TO ADMIT DOING, LEST THEY BE THOUGHT OF AS CHILD ABUSERS.  

WE TRIED TO PHRASE THE QUESTION IN A WAY THAT PARENTS WHO SPANK THEIR CHILDREN WOULD REPORT IT.  IN THE PREAMBLE WE NOTED THAT SOMETIMES CHILDREN MISBEHAVE; THEN WE USED THE VERB FORM "have you had to spank".
 
Sometimes children behave well and sometimes they don’t. Have you had to spank (Johnny) when (he/she) behaved badly in the past week?

If yes,

About how many times have you had to spank (Johnny) in the past week?


 
 
These are examples of a common way of asking about issues of policy etc. Respondents are presented with two statements, and asked which comes closer to their opinion.  This question is quite a mouthful for an interviewer.  This is quite a complex cognitive task for the respondent.  

Note that the question is not do you agree entirely with the statement; it is which comes closest to your opinon.
 
Still thinking about the issue of education, I’d like to read you two viewpoints. Please tell me which one comes closest to your own.

Some people say that education spending decisions should be made at the local level, and we should send federal money to states and local communities to hire additional teachers, reduce classroom size, crack down on undisciplined students, wire schools with computers, or any other priorities that parents and communities deem for their education dollars.

Other people say we need a national commitment to quality education for every child in America, as well as state and local involvement. That means establishing tough national standards, including zero tolerance for violence in the schools, and a federal priority to invest in reducing class size, modernizing our schools, and increasing the number of qualified teachers.

 

Which viewpoint comes closest to your own?

 


 
HERE  IS ANOTHER QUESTION FROM THE SAME SURVEY: 

IS IT CLEAR WHAT THEY ARE TRYING TO GET AT?   COULD YOU COME UP WITH LESS AMBIGUOUS WORDING?
 

And, thinking specifically about the education issue for a moment - -

Overall, when you think about education reform in American public schools today, which are you more worried about: keeping things as they are or changing things too much?

 

  


 
IN THE CONTEXT OF ASKING QUESTIONS ABOUT ABSENT PARENTS’ INVOLVEMENT IN THE LIVES OF THE CHILDREN, WE ASKED THE FOLLOWING QUESTION. THIS ILLUSTRATES THE NEED TO DEFINE THE TERMS USED IN THE QUESTION PARTICULARLY WHEN THERE ARE MULTIPLE MEANINGS. THIS WAS PARTICULARLY DIFFICULT BECAUSE THE LEGAL SYSTEM USES DIFFERENT LANGUAGE IN DIFFERENT STATES. 

THIS IS THE VERSION OF THE QUESTION ASKED OF THE CO-RESIDENT PARENT ABOUT THE ABSENT PARENT.
Today, many states allow for joint legal custody, which means that regardless of where the child lives, both parents have equal decision-making rights. Does your legal agreement give (Henry’s) (father/mother) an equal say in making decisions about (him/her)?

 

 

  

 

IN A SERIES OF QUESTIONS ABOUT PARENTING BEHAVIORS, WE ASKED ABOUT BEDTIMES. 

NOTE: 

  • QUESTION WAS CHILD-SPECIFIC – NOT ABOUT ALL THE CHILDREN
  • THE QUESTION ASKED ABOUT "PUTTING CHILD TO BED, NOT ABOUT GOING TO SLEEP 
  • REPONSE OPTIONS INCLUDE "NO FIXED BED TIME" 
  • THIS IS A USUAL BEHAVIOR QUESTION – THIS IS SOMETIMES PROBLEMATIC; IN THIS CASE WE FOLLOWED UP WITH HOW OFTEN THE CHILD STAYS UP LATER.
 
 

 What time do you usually put (Henry) to bed at night?

How often does (he/she) stay up later than that?  Would you say it is never, less than once a week,
about once a week, or more than once a week?

  


 
 
 
We were trying to find out when young adults left the parental household to live on their own. Those who were currently living in the parental household were asked the following quesion. 

 We specified four months or more to eliminate summer absence; we added the second sentence because in earlier work we discovered that people often do not include leaving for these reasons. 
 
 

Have you ever lived on your own, away from your parents’ household for four months or longer? Include any times you were away at school or in the armed forces.

If yes:


 
 
 
 
We also asked about parents living in the household of our respondents.  We tried to specify precisely who we were counted - parents and parents-in-law; and what was covered - parents living in R's household when they had no other household.  
 
In the time that you have been on your own, have any of your parents (or parents-in-law) ever lived with you in your household? This does not include visits when your parent still had a household somewhere else.