Professor Pamela Oliver Department of Sociology

FIELD OBSERVATION EXERCISE

DUE DATE:

For this exercise, you plan and carry out a structured field observation. You will select a hypothesis, operationalize the independent and dependent variables, and do structured observation to test your hypothesis. Preceding this formal observation, you will do some unstructured observation to train yourself to see concrete behaviors without filtering them through any interpretive filter.

Reading:

For assignment, Singleton, Chapter 11; also review Chapters 4 and 5, paying special attention this time to the discussions of reliability (pp. 114-122). For writing a research report, Singleton Chapter 17.

You may use any appropriate hypothesis for this research, but if you cannot think of anything you like better, I suggest that you focus your research on testing the hypothesis that women pause and scan or stop and groom themselves just before entering a room more often than men do. As you will read in your text, Lyn Lofland has found a "grooming" pattern in her research, so we can be pretty sure there will be something to see, but this particular hypothesis has not been tested in any research I know of. If a number of people test the same general hypothesis in different settings with different specific operationalizations, we should find it interesting to compare our results. (NOTE: If you have a different idea, discuss it in class. You always get better information if several teams work on the same general idea than if each team tries to be unique. This is because replication is essential before any finding can be treated as trustworthy.)

I suggest that you observe someplace where people are entering a room, building, or other well-defined space. Your dependent variable will be some dimension along which their behavior varies as they enter, "grooming," if you go with my example. (Another commonly observed behavior is pausing and scanning a room before entry.) You may use either of these ideas, or some other idea suggested by your own observations. I suggest focusing on entering because it eliminates a lot of the sampling problems in deciding who to focus on when. Any other locale would work if people flow through fast enough to give you enough subjects but slow enough to permit accurate observation. (An average of one a minute is a good rate. The acceptable range is as fast as three a minute or as slow as one every three minutes, on average.) People buying things at some kind of sales counter might also give a good flow of people, as might people going through some sort of exhibit. I suggest "grooming" or "pausing and scanning" because they have enough variability that you should get interesting data, and they require some effort to operationalize properly. Any other dependent variable that your preliminary observation tells you varies, and that is of comparable (or greater) difficulty to operationalize is also acceptable. The central point of this assignment is to show that you can operationalize an observable variable, so do not attempt to avoid the problem by picking a variable that is so obvious you cannot imagine there being any problem in operationalizing it.

About Teams.

You are strongly encouraged but not required to do this exercise with another class member. Teams have two options: (1) write a joint report, or (2) each person write the whole report individually. Option 1 is appropriate when team members are truly working and learning together. It is unfair and unethical for one student to do most of the studying and writing while another "free rides" under the guise of option 1. Those doing the work must refuse to "give" partners papers they did not help write. If you choose option 2, you work together until the data are collected and, if you wish, put into a statistical table, but you do not collaborate in writing your separate reports. Option 2 is a good choice if you have busy schedules which make it difficult to meet with others, but want the advantage of having a partner in thinking up the original idea and collecting data. If you find yourself in an ambiguous position about these options because of unforeseen problems, speak to me and I will help you to determine the fairest thing to do.

Steps in Execution

Preliminary Unstructured Observation

1. With your partner, select a place and time to observe for a minimum of 30 minutes. Select the same place (or type of place) and same general time of day as you plan to use for your structured observation. If as you observe it becomes clear that people move through this place too quickly or too slowly, look around for another place.
2. Your task is to force yourself to see the concrete details of people's actual behavior, not your interpretations of them. Watch what people actually do; suppress the normal instinct to evaluate people or to presume motives. Look at behavioral details like patterns of movement through space, hand gestures, posture, positions of legs or arms, ways of eating or drinking, eye or head movements, amount or volume of talking. As you observe, take brief "jotted notes" of specific behaviors worth observing.

It is OK to be thinking about the kind of behavior you might want to do your structured observation on, but do NOT focus narrowly on ONLY that kind of behavior. If you are thinking about grooming or pausing and scanning, train yourself to look for the variations and differences among people in these behaviors. Also keep your eyes and mind open to other kinds of behaviors that might be more interesting to you to study.

3. When the time is up, stop and write down what you can recall of the behaviors you saw. Your jotted notes may remind you of things, and there may be others that did not make it into your notes. Make a special note of details you saw that seem worth remembering for your structured research. Then write a short paragraph explaining when and where you observed and attach it to these observation notes. List 1-3 possible hypotheses that arise from this observation. ATTACH THIS AS AN APPENDIX TO YOUR OBSERVATION ASSIGNMENT. It is perfectly OK if this is in your normal illegible handwriting with spelling and grammar errors. I just want to know that you did this step. It is NOT worth recopying or typing it.
4. If you have a partner at this stage, compare notes on what you saw and talk over your ideas for hypotheses for your structured observation.

Some Variations

The above assignment is all that is required, but there is a great deal more that can be learned from unstructured observation if you have the time. Try observing people in different settings to see if you can notice how their behavior changes with the setting. Try observing people on TV or in a movie and then observing a similar setting in real life. Try sitting for a very long time or repeatedly in the same setting or type of setting to see if you begin to notice things that were initially invisible to you. Try behaving in some slightly "odd" way and notice the reactions. The hardest thing to do is to train yourself to see people's actual behavior, and not your highly abstracted interpretations of it. "Friendly smile," "in a hurry," "flirting," and "nervous," are interpretations, not actions.

Structured Observation

Plan for Structured Observation

1. Pick your dependent variable, the behavior you will observe. You need to operationalize this variable. This means choosing a level of measurement, deciding on the categories, and then carefully spelling out what observable cues you will use for categorizing people's behavior. Basically you will either count how many times (or how long) a person does something, or you will categorize their behavior. If you categorize, your operationalization is focused on defining the differences among the categories. If you count, your operationalization is focused on defining exactly when the behavior you are counting or timing begins and ends. We will discuss in class.

You need to understand what operationalization of a variable means so that you can carefully define the observational rules for distinguishing the exhaustive and mutually exclusive categories of this dependent variable. This operationalization should be so complete that another person could use only your written instructions and record the variable the same way you would. You will operationalize only one dependent variable. Remember to plan to have an "other" category in case something unexpected comes up in the field.

2. Pick your independent variable and explain its operationalization. It will usually be gender (male, female) or some other obvious physical characteristic. Your operationalization will usually be your subjective judgment based on common cultural standards, but you should keep in mind that what you can see is not always the truth. (You may use a more "difficult" independent variable if you wish; just be sure to explain how you operationalized it.)
3. Create a recording sheet for your structured observations. PLEASE NOTE: This format is almost always the best one; students often invent their own formats which are more error-prone and unreliable than this one. USE THIS FORMAT UNLESS I TELL YOU THAT ANOTHER IDEA IS BETTER FOR YOUR PARTICULAR PROJECT. Let each line be a different subject (unit of analysis). Select consistent shorthand symbols for each category of the independent and dependent variables. Suppose you agreed to use M and F for male and female, and to use L for licked ice cream cone, B for bit it, and O for anything else. Then your recording sheet might look like this:
   
   
   

   IndVar Sex	Dep Var Eating	Identifier	Comment
   M	B	red shirt
   F	L	blond, pink dress
   M	O	red hair green shirt	some of each
   M	L	asian, yellow shirt
   F	B	AfAm, white jacket used lips	used lips

   
   

Regardless of how you measure your dependent variable, you will need "identifiers" for the individuals observed. These identifiers are so you and your partner(s) can go back over your individual data later to check your reliability. Generally, use hair, skin, and shirt/blouse colors as identifiers.

Before the observation, agree upon the symbols and set up the columns, being sure to have a few extra made up in case you get more subjects than you expect. Comment whenever it is difficult to decide how to categorize someone on the independent or dependent variable, or to explain "other" codes.

4. Plan your sampling procedure. That is, explicitly decide who will count as a subject. In this study, you are "sampling" one period of time, and you should try to study everyone who comes into the setting during that time who is an appropriate subject. However, you should define who "counts" as a subject, both in terms of being in a position for your dependent variable to be relevant, and in terms of their personal characteristics. For example, will you consider children, or only adults? Will people have to enter a certain space or spend a certain amount of time in the setting to count as subjects? Will you exclude certain people (e.g. those wearing employee uniforms)? Will you include only those people who enter a certain physical area, or spend a minimum amount of time in the area? What will you do if too many people come in at once for you to observe? (Don't use a setting where this is the normal problem, but even when the flow is usually OK, you might have occasional problems.) NOTE: You do not have to worry about random or representative sampling at this point.
5. Consider the problem of subjectivity in your sampling criteria and the need to operationalize the variables which define your sample. For example, if you are studying only "adults," what are the criteria for "adult"? Or only people who are "alone," how do you tell if they are alone? Spell that out.

Carry Out Your Research

Using the form you have developed, you and your partner observe for a minimum of 30 minutes until you obtain a minimum of 30 observations. If it turns out that the pace is so slow that you cannot observe at least 30 people in the time you have available, move to a different setting, or come back at a different time. If it turns out that people are coming in so fast that you cannot possibly observe them, stop and move to a better location or come back at a slower time.

If you realize part way through that there is a problem with your operationalization or sampling, use the comments column to note the details for difficult cases and keep going. If you add a decision rule part way through, record it on your sheet and keep observing. Quit observing early only if the problems are so bad that you just cannot use your scheme at all, in which case you need to fix the problem and start over.

Calculate Your Reliability

Even when two people observe the same thing and follow the same rules, there will usually be some disagreement in the observation. This is more likely when you have done a "hard" variable to observe, rather than a trivially easy one. Inter-coder reliability is an excellent way to assess the accuracy with which it is possible to record data. This is somewhat tedious to do, but has the potential of being a valuable learning experience.

1. Compare your sheets and match up people, using the identifiers and the person's sex (which is almost always accurately recorded unless it was ambiguous). Mark the people who do not match up people on the other sheet. For each person who does match up, check whether the independent and dependent variables are the same; mark everyone for whom you disagree about a variable. (Note: if you are counting or timing something, give yourselves a reasonable margin of error such as being within 1 or 2 seconds still counts as "the same," so you do not have to include too many errors. See me if you need help with this.)
2. Count up the following numbers:
   
• A = number you agree on, same person, same dependent variable code.
  
• C = coding difference: number of times you saw same person but coded dependent variable differently.
  
• S1 = sample difference 1: number of times partner 1 saw someone partner 2 did not see.
  
• S2 = sample difference 2: number of times partner 2 saw someone partner 1 did not see.
  
• N = total number of distinct people seen by either partner 1 or partner 2 (or both).
3. Reliability computations:
   
• SE = (S1 + S2)/N = sample selection error (proportion of total cases that one person saw but not the other).
  
• CE = C/(A+C) = coding error (proportion you both saw that you disagree about in the dependent variable).

NOTE: If you have a three-person team, there are three possible pairs for which you can do this analysis. Either do the above for all three possible pairs (if it does not make you feel too oppressed) or do it for two of the three pairs, the two partners that seem most alike and the two partners that seem most different. SEE ME if you do not understand this or want to negotiate.


WRITTEN REPORT

PLEASE FOLLOW THIS FORMAT EXACTLY. This is based on Chapter 17 of the Singleton book, but includes some specifics for this class.

About Truthfulness. Science depends on researchers telling the truth about what really happened in their research, not what they wish had happened. At the same time, students worry that they will be graded down if they tell the truth. So, for each question in your report, I insist that you tell the truth about what really happened in the research, but then follow it with an opportunity to explain what you now think you should have done. If there was a mistake and your self-criticism gives a correct statement about what you should have done, you will receive full credit as if you had done things right in the first place.

1. Title page. Title of report, author(s), date. Put partner's name in parentheses at the bottom of the page if you worked with someone but wrote reports separately. (Give your project a real title as if you were taking it seriously, not just "assignment 1" or some such.)
2. Abstract. Write one paragraph which summarizes your research methods, hypotheses, and findings. You may include this on the title page if you wish.
3. Body of paper.
1. Introduction. Write a paragraph stating your topic and why it is worth researching. Summarize observations relevant to the topic that came up in your unstructured research. Explicitly state your bivariate hypothesis and why you believe it is true. (If you are just guessing about the hypothesis, or team members disagree, just write about the reasoning in your guess, or what the disagreement is.) (Note: we will NOT normally do literature reviews in our course assignments, but this is where it would go, and if something you read went into your thinking on this project, this is the appropriate place to mention it.)
2. Methods of research. (Note: We will write this section in a more closely structured format than the usual research article. This is so I can more easily grade your paper. Number each section of this discussion as it is numbered here, e.g. 2b for operationalization of dependent variable; this is essential for grading, as part of the grade is putting the correct information under the correct heading!)
1. Sampling.
1. Describe the setting of your research, the time of day you conducted it, and any details relevant to understanding your data.
2. Describe your sampling procedures, including any restrictions placed on eligible subjects, or other procedures for deciding whom to study within the setting.
3. Discuss the problem of possible subjectivity in the sampling criteria; how did you operationalize the sample selection variables?
4. Evaluation: why you think these procedures were good, or what you now believe should have been done differently.
2. Dependent variable.
1. Why you chose your particular operationalization
2. Complete details on your operationalization as you planned it. (Note: This is never correctly a one-sentence answer. I'm looking for details of wording that tell me you know what is important about operationalization, as well as for indications that you did things properly. Many people do it right but cannot explain it right.)
3. How the operationalization actually worked out. d) Evaluation: why you think these procedures were good, or what you now believe should have been done differently.
3. Independent variable.
1. Why you chose your particular operationalization.
2. Complete details on your operationalization as you planned it. (Note: This answer is typically short, but it must be worded to show that you know what an operationalization is.)
3. How the operationalization actually worked out. d) Evaluation: why you think these procedures were good, or what you now believe should have been done differently.
4. Ethics. Discuss your evaluation of the ethics of doing this research. Do you feel you invaded anyone's privacy? How did you feel about doing covert observation?
3. Results. (Attach the original messy data collection sheet to the back of your paper as an appendix. You will be graded down if this is missing.)
1. Sampling Reliability Analysis.
1. Present the results of your reliability calculations. (Include the computations in an appendix so that I can check your work.)
2. Discuss these results. Did you have more than trivial disagreements (more than 1 case) over sampling? Can you figure out what caused the problem(s)? Discuss in some detail.
3. Evaluation: why you conclude your procedure were good, or what you now believe you should have done differently.
2. Measurement Reliability Analysis.
1. Present the results of your reliability calculations. (Include the computations in an appendix so that I can check your work.)
2. Discuss these results. Did you have more than trivial disagreements (more than 1 case) over coding? Can you figure out what caused the problem(s)? Discuss in some detail.
3. Evaluation: why you conclude your procedure were good, or what you now believe you should have done differently.
3. Explain how you chose/constructed the data you analyzed for your hypothesis test.
4. Test of hypothesis. a) Prepare a bivariate statistical table to show the relationship between your independent variable and dependent variable. Do either a contingency table or a difference of means table. (Refer to "bivariate association" in the statistics part of this course. Or ask for help, if necessary.)
5. Write a paragraph discussing your statistical results saying what they show and whether your hypothesis is confirmed or disconfirmed.
6. Discuss anything else worth mentioning that you learned in your research, including unexpected events or surprising findings.
4. Conclusions and interpretations. This is where you talk about the larger issues your research raises, whether you feel that your findings are likely to be more generally true, and what research, if any, you would like to see pursued by yourself or others as a consequence of your research. For this class, it is also a chance to talk informally about what you liked or didn't like about the assignment or the way you did your research.
4. Appendices
   
1. Your unstructured observation notes and hypotheses. (The original messy version.)
   
2. Your original structured observation data collection sheet, the one you actually used in the field to get the data. DO NOT recopy or retype this sheet. I want to see the real data.
   
3. The work showing the computations for your reliability analysis and test of hypothesis. This can be an unrecopied "rough draft" (I certainly would NOT type it!!), but I DO need to be able to follow your computations and check them against your data sheet to be sure you did not make an error.
5. Group process report. Pick the category that applies to you and answer the relevant questions. PARTNERS MUST HAND THESE IN SEPARATELY SO THEY CANNOT POSSIBLY HAVE ACCESS TO EACH OTHER'S ANSWERS. I WILL NORMALLY NOT RETURN THESE STATEMENTS, BUT WILL KEEP THEM FOR MY RECORDS.
1. No partner.
1. How did you feel about working alone? Would you do it again, or would you prefer a group?
2. How much effort did you have to put into this project
3. How well prepared did you feel in terms of course materials and understanding what to do.
4. Tell me if there is anything I should know about you or your life that you want me to know, especially if it might affect your grade or my ability to be fair in grading your work.
2. Had partner, wrote separate papers
1. Compare you and your partner in the effort you put into the project.
2. Compare you and your partner in the extent to which you studied course materials and knew what to do for the assignment.
3. Who did your statistical analysis?
4. Did you start trying to work together before deciding to write separate papers? How far did you get?
5. Were there some things you found necessary to discuss in preparation for writing your papers? What?
6. How did the group process work out? Was it a positive or negative experience? Would you do things differently in the future?
7. Tell me anything else I should know that might affect your grade or your partner's, or that I should know to be fair in grading your work, or that you would like me to know even if it is not relevant to your grade.
3. Wrote joint paper.
1. Do you stand by the paper as written, or is there something you feel should have been said differently? Any corrections you offer at this point will be factored into your grade. This answer may be as long or short as you feel is appropriate.
2. Compare you and your partner in the effort you put into the project.
3. Compare you and your partner in the extent to which you studied course materials and knew what to do for the assignment.
4. Who did your statistical analysis?
5. How did you go about getting the writing done?
6. How did the group process work out? Was it a positive or negative experience? Would you do things differently in the future?
7. Tell me anything else I should know that might affect your grade or your partner's, or that I should know to be fair in grading your work, or that you would like me to know even if it is not relevant to your grade.
   

Top

Questions or Comments? Email Oliver -at- ssc -dot- wisc -dot- edu. Last updated December 25, 2004 © University of Wisconsin.