A new experiment at Harvard’s Student Social Support R&D Lab indicates that students in a summer program did better when their parents received a weekly one-sentence about their children’s performance. The study used a randomize experimental design with students in an urban school district.
What do you think the causal mechanism for this effect was?
Do you believe it is ethical to experiment on students in this way? Why or why not?
Posted in Chapter 1, Chapter 12, Chapter 2, Chapter 3, Chapter 6, Chapter 7, Chapter 9
Tagged causation, Evaluation Research, Experimental Design, Measurement, Quantitative Data Analysis, Research Ethics, Statistics
Is a walk in the woods good for you? It used to be a part of everyday life for people all over the world, but as civilization has progressed and people mostly live in urban areas, contact with nature has become much less common. Prior research has indicated that visiting natural environments can lower stress hormones, while a lack of green space is associated with more psychological problems. A recent study by a graduate student at Stanford University designed a novel experiment to test the consequences of exposure to nature on brain functioning. Gregory Bratman decided to test the effect of a walk in the woods on brain activity and feelings found that it calmed their brains and improved their feelings of mental health, compared to those who walked along a highway.
Read more about it and watch the video at:
How would you diagram the design of Bratman’s experiment? Does it meet the criteria for a true experiment?
Do you think a measure of brain activity helps to understand the causal mechanism by which the walk in the woods had an apparent effect?
After watching the video, consider how you could extend this research with a mixed methods design? What might you be able to pick up with qualitative methods that might be missed with the quantitative methods in the original study?
Posted in Chapter 10, Chapter 12, Chapter 4, Chapter 6, Chapter 7, Chapter 8
Tagged causation, Ethnography, Evaluation Research, Experimental Design, Health Care, Measurement, Mixed Methods, Visual Sociology
Does it surprise you to learn that a large longitudinal survey of parents has led to the conclusion that the rate of severe mental illness among children and adolescents has dropped considerably in the last generation? The study involved ratings by parents of impairments in household surveys involving 53,622 children aged 6 to 17, in 1996 and 2012. The surveys were sponsored by the federal Agency for Healthcare Research and Quality.
Two explanations proposed for the drop are improvements in referrals to treatment and more effective parenting. But the decline also contradicts other surveys, including by the Centers for Disease Control and Prevention, that have pointed to increased serious mental disorders among children and teenagers. The discrepancies may be related to differences in measurement, with the CDC approach including more children with mild impairments, or maybe to the AHRQ approach missing problems like substance abuse that parents may not be aware of.
More details are available at:
What have been your own beliefs about changes in the prevalence of severe mental illness among children and adolescents? What have these beliefs been based on? Evidence? Assumptions?
What advantages and disadvantages can you identify in using parents to report on youth behavior? What about youth self-reports of behavior?
Posted in Chapter 1, Chapter 16, Chapter 2, Chapter 4, Chapter 5, Chapter 8
Tagged Conceptualization, Evaluation Research, Health Care, Measurement, Quantitative Data Analysis, Survey Research
Who can best determine what people in a community need to improve their lives? Asking them seems to be the most helpful part of a good needs assessment. In Houston, a nonprofit named Neighborhood Centers assesses needs by “spending hundreds of hours conducting one-on-one interviews and community meetings, inviting residents to specify their priorities, identifying the community’s natural leaders and then going after the needed funds.” They seek funds from multiple sources to design efforts to meet those needs.
You can read more about the results of this and related efforts at:
What needs do you think would be identified in your community through such an approach?
How does this approach to needs assessment compare to a more standard social science approach?
Medical research may identify a potentially valuable treatment that must be tested in a rigorous experiment. Usually such experiments are funded by federal agencies like the National Institutes of Health, after a careful review, or by the company that has developed the treatment. But what if a researcher has identified a promising treatment but can’t secure funding for an experiment? Could patients–potential research patients–be asked to pay to participate? Many seriously ill persons are desperate for a new approach that might restore themselves or a loved one to good health.
Should such patients be allowed to “pay to play” if they would like to? Would it help to advance science?
What types of social science experiments might some people be willing to pay to participate in? What are the potential advantages and disadvantages of such a funding approach?
Do you every watch a basketball game and think that a player has a “hot hand,” by making a lot of baskets consecutively? Do you ever play on slot machines and think that you are having a “streak” of good, or bad, luck? People often tend to see patterns in random events that are not really there.
But there’s a bit more to it than that. Are the odds of heads after flipping a coin always 50/50? Sure. That’s the basis for knowing that random assignment leads to equal probabilities of selection for each condition in an experiment. But what is the answer to the question: What are the odds of heads after flipping a coin when you have just obtained a head on the prior toss?
Read more about randomness at,
Does the logic of sampling on the basis of chance make sense to you as a way to achieve a representative sample?
Is it possible to do “better than chance” when choosing individuals in order to create two or more equivalent groups in an experiment?
With concerns about reproducibility of results and exposure of instances of fraud–not to mention conflicting research results about what you should eat and how often you should be tested, it is easy to become cynical about the value of scientific research methods (see earlier blogs). But a wide gulf separates investigations of the social world using systematic methods and publication of results from what occurs when social science principles are not considered.
An extreme contrast occurs with fortunetellers–“psychics–whose business is to convince patrons that they know things or can predict events on the basis of some mysterious powers. One storefront psychic in Times Square was charged in court with bilking a man out of $713.975 for promising to reunite him with a dead woman who he had loved. Another was paid $14,500 by a vulnerable woman for a rock the psychic claimed was from a meteorite.
You can read more about psychic pseudoscience in court at:
Why do you think many people believe in paranormal phenomena?
How would suggest testing the assumptions behind paranormal beliefs?
Posted in Chapter 1, Chapter 12, Chapter 2, Chapter 3, Chapter 4, Chapter 6, Chapter 7
Tagged Conceptualization, Ethnography, Evaluation Research, Experimental Design, Homelessness, Measurement, Research Ethics