| Sociology 357 Piliavin
ARTICLE ANALYSIS ASSIGNMENT
DUE DATES: See summary sheet
READ THIS HANDOUT CAREFULLY! You must do this analysis by answering
the specific questions listed. Keep your answers as brief as possible
using an "outline" style rather than an elaborate writing style
Criteria for Article Selection
The articles reviewed for this assignment must report the results of
someone's research in an area of social research. The research should
have been carried out by the author(s). The article must be directed at
a scholarly audience.
Your review must be on an article reporting structured research, that
is, one with variables, statistical analyses, relationships among variables,
etc. The article may be about any social science topic you choose.
Check with me if you have any doubts about your topic. Research in sociology,
political science, psychology, education, or social work are fine. (But
remember you need research articles; not all articles in any field are
The following types of articles may NOT be used:
- Purely theoretical papers which discuss concepts and propositions,
but report no empirical research;
- Statistical or methodological papers where data may be analyzed but
the bulk of the work is on the refinement of some new measurement, statistical
or modelling technique;
- Review articles, which summarize the research of many different past
researchers, but report no original research by the author;
- Popularizations or abridged reports, commonly found in popular newsstand
magazines such as Psychology Today or books of readings designed
for use by undergraduates;
- Extremely short reports with less than four pages devoted to methods
Most research reports begin with sections on theory and reviews
of others' research, so skim the whole article or read the abstract, if
there is one, to determine whether the author reports actual research
he or she has done. Sociology, as is true of all scientific fields, is
becoming increasingly complex in its statistical analyses. I therefore
strongly suggest that you use articles no more recent than the 1970's.
A working rule is: if you can't understand the statistical analyses presented in the
results section, don't choose the article.
All articles must receive my OK. No two students may review the same
article. It is OK to use articles you have to read for another class,
if they meet all of the above criteria, but you may not use the articles
Where and How to Find an Article
You must use scholarly articles for this assignment; these are found
in professional journals, not general circulation magazines. The University
of Wisconsin subscribes to a large number of such journals,in both physical and electronic form. Recent issues
of most of the physical journals are kept in the periodicals room of Memorial Library.
Past issues are bound in hardcover by volume and kept on the first and
second floors of the south stacks of Memorial Library. Bound volumes of
some journals are in the reserve room of Helen C. White library and in
the Social Science Library. To find the call number of a specific journal,
look up the journal's title in MADCAT,
or in the list of journals in the periodicals room.
If you want to find articles about a particular topic, use the data
bases available through the Library home page. Another place to get citations
of articles in a topic area is in the bibliographies of other books or
articles in the topic area. If you are having trouble finding an article,
go to the second floor of Memorial Library and ask a librarian for help
or come see me.
I suspect that most of you will go first to full text databases. If you get an article from one of these, choose the PDF format if it is available. If it is not, MAKE
SURE to print out all of the tables and figures. You sometimes have to do
this separately in non-PDF files.
If your interests are wide, general, eclectic, or uncertain, you may
prefer to locate a supply of journals in the stacks or the reserve room
and flip through them until you spot an article that looks interesting
to you. The major general sociology journals are American Journal of Sociology,
American Sociological Review, and Social Forces. Some other journals in
sociology are: Journal of Marriage and the Family,Criminology,
Crime and Delinquency, Social Psychology Quarterly, Sociology
and Social Research, Social Problems, Journal of Political
and Military Sociology, Journal of Sport and Social Issues,
Sociology of Sport Journal. There are dozens of other specialized
Final approval will be given only on the basis of the photocopy or printout
of the whole article; I will write approval on the copy itself. When you
have found the article(s) you want, photocopy it, and write right on the
photocopy the journal name, volume number, issue number, month, year of
publication, and pages. The author's name and the article's title should
be on the first page; if they are not, copy these down too. (You should
get into the habit of writing the full citation on everything you photocopy.
This saves having to return to the library for the information when you
later decide to use the material in a term paper or, worse, not being
able to find it.) Don't save ten or twelve cents by omitting the last page of
the references. Do write your own name on copies you turn in to me. If
you wish to save money, check out the journal(s) themselves and bring them to me.
Outline for Your Article Review
PLEASE NUMBER THE SECTIONS OF YOUR REVIEW TO CORRESPOND TO THE NUMBER
OF MY QUESTIONS. It is not in your interest for me to have to guess
what you're writing about. Answer the questions as briefly as possible.
This is not a literary essay. An "outline" style, tables, and
other devices to keep your answers brief while complete are all acceptable.
NOTE: Make sure that the full citation is either printed or written
on your photocopy or you will not get credit for the review. Attach
the photocopy to your review. I simply cannot grade your review
without the photocopy.
- What is the problem or question(s) this research
concerns? You should be able to identify the central focus. If there
are additional secondary problems, identify these too. (1-4 sentences)
- What is the source of the data? (That is, questionnaire,
intensive interview, documents, existing statistical information,
observations, laboratory manipulations, field manipulations, etc.)
In some studies there are two or more sources of data. Give a brief
overview of how the data were acquired. (2-5 sentences)
- Briefly, what do the key findings turn out to be?
- External Validity
- Give the following information about the
sampling procedures in outline form, saying "not given,"
if it is not:
- Definition of the population of theoretical
or substantive interest; a) What is the population of theoretical
or substantive interest; that is, to whom does the author seem
to want to be able to generalize? Your answer to this should
be based on what the author says in the introduction to the article,
not in the methods section.
- Geographic areas, organizational units
(e.g. what state, University, county), or other primary sampling
units and how these were chosen. Was the sampling of these
units probability or nonprobability?;
- The sampling units (e.g. people, organizations,
sentences). These may or may not be the same as the units of analysis;
- Sampling frame, or operationalization
of the actual population studied; by what rule or list
were units of analysis located?
- Method of selecting the units of analysis
from the sampling frame. Was the sampling of these units
probability or nonprobability?;
- What kind of sample (e.g., convenience,
stratified random, etc.) does this seem to be?
- Response rate (e.g., to a mailed survey)
and sample size; if analyzed sample size is different from initial
sample size (e.g., cases were dropped for missing data) explain
- Does the author discuss any shortcoming
in the sample or the sampling procedures? If so, what does s/he
If you feel that this outline does not adequately
demonstrate your understanding of the sampling, or that there is
something important about the sampling that does not fit in this
outline, write an additional paragraph that provides any necessary
extensions or clarifications. (Do not, however, omit the outline.)
Often articles that use one of the well-known
large national probability samples do not give much information
about the sample because they assume that professionals will recognize
the sample title and already know the basic information. Check with
me if you suspect this is the situation with your article. You may
need to track down an earlier article to get the details.
- Evaluate the sampling procedures.
- Do the geographic or other restrictions imposed
on the actual population (b, d above) seem justified in light of
the purposes of the research and practical constraints?
- Were the units of analysis selected in a
way that allows generalization to the desired population? Why or
- Are you aware of anything in the research
procedures that added any implicit restrictions to the sample (e.g.
interviewing only during the day)?
- Does information available in the article
(e.g. frequency distributions) suggest that the sample is reasonably
representative, or does it point to problems or biases? e) Overall,
how good do you feel the sampling was?
- Strictly speaking, to what population
can the results of this research be generalized?
- To what population would you feel reasonably
confident the results probably apply? Why?
- At what point would you be very hesitant
to apply these results?
- Construct Validity of Measures
- Preferably using a chart, list ALL of the
operationalized variables in this research and the concepts or variables
of theoretical or substantive interest they are intended to represent.
You should discuss all the operationalized variables, but it will
be often easiest to write your answer by starting with the concepts,
and explaining how each is measured. Sometimes there are several measures
for one concept or variable. Do NOT "dump" all the measurement
details here. This is just a summary that lists all the measured variables
and what their logical relation is to the purposes of the research.
DO NOT talk about how one variable relates to other variables here.
It is hard to explain this question clearly, because how to do it
depends very much on what your article is like. Probably the best
explanation is an example. For the horn-honking article, the answer
would be: The independent variable is status of frustrator. This is
operationalized as the type of car and the driver's clothes. The dependent
variable is aggression, which is operationalized as latency of honking
and number of honks. The frustrating situation is operationalized
with a car being blocked at a green light. Sex of subject was a control
Different articles have different logical structures, and the best
way to do your article is to describe what is happening in it. Some
have no distinction between the concepts and the operationalizations;
everything is just operationalization. Others have complicated and
convoluted steps getting from the original concepts to the measured
This is where you should tell me if the units of analysis are different
from the sampling units. Sometimes there are several different units
of analysis in one article. Measurement is dependent on these units.
- Select the two (2) most complicated or difficult
variables in the article. That is, choose variables that must have
been hard for the author(s) to figure out how to measure, or how to
make the conceptual-operational link. Call these variables "a"
and "b" in your outline. Then do the following detailed
analysis for each. (You will then have a) 1, 2a & b, 3, 4 and
b) 1, 2a & b, 3, 4 under this section C) 2)
1) State the concept and give a brief summary of what (if anything)
the author says about issues or problems in measuring this, how others
have measured it, why s/he is measuring it this way, etc. (NOT the
measurement details themselves.)
2)What is the measure of this concept that
is used in the data analysis? (E.g., in Ransford there is a scale
for racial dissatisfaction.) If there are several, state all of
them. Then explain the following:
3) Summarize any discussion by the author of why this is a good
measure or of what its problems are, including statistics like factor
analyses or reliability coefficients, but NOT material on the bivariate
relations with other variables.
- How the relevant variables were originally measured on the
units of analysis. That is, what were the initial items of information
obtained and what were their attributes? (In the Ransford article,
this would be the questions and answer formats that make up the
- Explain how the original measured variables were combined or
modified to create the specific operational variable that was
used in the statistical analysis. (In the Ransford article, this
would be that the original questions were summed and then dichotomized,
using a conceptual split.) This is where you describe index
or scale construction. (Often the original measured variable was
not modified; if this is true, just say "does not apply.")
Discussions of how cases were grouped or regrouped belongs here,
4) Give your own evaluation of how good you think this measurement
is, explaining your reasons.
NOTE: The format of the above questions works best when the variable
that gets into the statistics is a composite of several original measured
variables. In some articles, what is more interesting is to start with
a concept that has several related measures (each of which might be
fairly simple) which are then analyzed to see which is "best,"
in which case you might want to discuss them as a group and treat the
matter of choosing among them in d). I suggest you ask me if there is
any doubt in your mind about which two variables would be good choices.
I should note that in some articles, all of the variables are pretty
straightforward. In this case, just pick any two of them. You will not
be graded down because your article is less complicated. However, I
do expect people with very uncomplicated variables to analyze them perfectly,
while I might decide that a mistake in analyzing some complicated variable
is not that bad. (If there are both simple and complicated variables
in your article and you choose to talk about the simple ones, I will
assume you do not understand operationalization, which is not in your
- Identify two of the most important bivariate
hypotheses (explicit or implicit) or questions of the research. For
each hypothesis or question, list those findings which are most centrally
relevant to it. If there are only a few relevant findings, list all
of them, but if there are many, list only the few that you think are
NOTE: A finding is the actual number(s) from the statistics, not
just the author's word summary. Often a particular hypothesis is supported
by several different findings which show that the bivariate relation
holds true after other variables have been statistically controlled,
or when the research design is altered, or when the variables are
measured in different ways. If so, you would list several different
findings as relating to the same hypothesis or question, but if there
are many different relevant numbers for the same hypothesis, you would
pick out only the few most important ones.
When articles list more than two hypotheses or goals, it can be
difficult to decide which is most important. Think about the central
purpose or argument of the article (usually found in the introduction).
Four common approaches lead to long lists of implicit or explicit
hypotheses or questions.
- They are all variations on the same general idea. In this case,
pick the two variations that seem most central in the discussion.
- The author actually believes in only one or two of the hypotheses,
and the others are set up as alternates to be proved wrong. In this
case, pick the ones the author seems to believe in.
- The argument has a series of logical steps and there are hypotheses
about each step. In this case, all the steps do matter, but pick
out the ones that seem to you or the author to be most central in
- the article does not really have a central point and there is
just a laundry list of hypotheses, questions or topics. In this
case, pick out the ones that you or the author think are most interesting.
- If there are additional findings that you or the
author found interesting or surprising, list them here. (Again, a
finding is not just the verbal summary, but the number that backs
it up.) If you already wrote a lot for (1), you may just say "no
additional findings" here.
NOTE: If your article has only a few statistics, you may end up
writing about all of them, but if your article has a lot of statistics,
do NOT write about everything. Instead, try to figure out what is
really important. I do want you to learn to read the numbers,
and you may ask me for help translating them.
- In this section, you will evaluate the internal
validity of the data. It is OK to make summary statements that are
true for all findings, where appropriate, but be very sure to discuss
the findings separately where necessary.
- Is the conclusion supported by an appropriate bivariate statistical
result? That is, look at the statistic copied above to be sure it
is actually relevant to the hypothesis it is supposed to be related
to. Sometimes in a bad article, the relevant finding is actually
not reported! (Remember that a bivariate association of zero supports
a hypothesis of no effect.)
- Is there adequate justification given or implied for the presumed
direction of causality, i.e. for why A causes B instead of B causing
A? If yes, say why in one sentence. If no, say in one sentence what
you think the problem is.
- List the potential extraneous variables that have been controlled
for in any multivariate statistical tests. (This is simply a matter
of being able to read your tables.) If multivariate statistical
tests (e.g. regression) were not done, just say so. ) Ask if you
have a question.
- What kinds of extraneous variables are simply irrelevant for
this finding and could not possibly be a problem? (Examples: on
stage effects for research on historical documents, maturation or
other time-tied variables for research that is conducted in one
short period.) Just list general classes of variables.
- Which potentially significant extraneous variables have been
controlled in the design of the research, by holding constant, by
randomization, or by some other method? Just list general classes
of variables, mentioning specifically only those which would otherwise
be a special problem (e.g. organismic variables in a within-subjects
- Are there any other possible problems or extraneous variables
that the author discusses, giving reasons why they should not be
problems? Summarize the discussion.
- Are there other possible problems or extraneous variables that
the author thinks have not been adequately eliminated? Summarize
- Are there any remaining possible problems or extraneous variables
that you can see that have not already been discussed above? Are
there variables that should be controlled that were not? Could a
different designed have eliminated problems? Are there things you
can see as problems that you wouldn't know how to fix? If yes to
any of these, discuss your concerns. I am referring to
simple random error here; you need to identify variables that are
potential threats to internal validity.
- Overall, how much internal validity do you attach to the findings?
Why? (Be sure to say whether your answer varies from finding to
- Give your overall evaluation of the methods used in this article:
what things were done well? what were done poorly? How much trust
do you put in the findings?
- Look at this article's "packaging," that is, the theoretical
introduction and the discussion or interpretation at the end. Do you
feel that the actual methods and results support the theoretical and
interpretive claims of the author? Why?
- What possible ethical issues might have arisen in the process of
doing this research? Do you think the researcher's ethical decisions
were all justified, or are some questionable? Why?
- To sum up, what do you feel you've learned worth knowing from this
article? (If your answer is "nothing", explain why.) (Please
note: this question is about the article and refers to the quality
of information it contains.)
- Tell me anything you would like me to know about your experiences
doing this analysis, or any suggestions you have for future revisions
of this assignment.
*** END OF REVIEW ***
Some Remarks on Grading Standards
- The key to this assignment is to apply the methodological concepts
you have learned in this course to the evaluation of a research article.
You demonstrate your ability by specifically linking the procedures
discussed in the article to the concepts. Think of it as a take-home
final, not as an opinion essay. You have the burden of proof to demonstrate
that you know what you are doing. In particular:
- Never answer just "yes" or "no"; always explain
- Never state some general methodological term or principle without
linking it up specifically to something in the article (or to something
missing in the article).
- Never give a vague or evasive answer in which you avoid sticking
your neck out (hoping you won't be marked "wrong"); if you
don't commit yourself to a specific answer, I will assume you do not
know what it is. But try to say what is needed as briefly as possible.
Long-winded, rambling answers are evidence that you do not know precisely
what is important.
- Questions of "fact" will be graded by comparing what the
article says with what you said it said, along with your ability correctly
to use the relevant methodological terms. Questions requiring evaluation
will be graded according to these criteria:
- you take some position
- you defend your position by talking about your article in ways
that raise issues that we discussed in class.
- If the article fails to give some information the review asks for,
you get credit by saying that the article fails to give the information.
Note that this failure should then become part of your evaluation of
the relevant section. (I will try to avoid approving articles that are
missing too much of the relevant information.)
- If the article is unclear or ambiguous, or if you are ambivalent
in your evaluation of something, it is fine to give an answer that expresses
- Don't blindly assume the author is using the correct methodological
terms for what s/he did.For example, Ransford describes his sample as
"disproportional stratified" (p. 298 of Golden reader). But
if you carefully read the paragraph on p. 298 and the extended description
of the sample on pp. 309-310, you will discover that the sample was
not stratified at all: three clusters (Watts, South Central, Crenshaw)
were chosen purposively; blocks were chosen randomly within clusters;
and households were chosen purposively within blocks, after a random
start on block corner and an overall quota of 8 households per block.
The use of the term "random methods," rather than "random
sample," is the sort of thing you'll see when the procedures are
less than ideal.
Questions? Comments? Please contact firstname.lastname@example.org