Abstract: Many people have suspicions about laboratory investigation of human behavior in economics. Most of the time, those suspicions are unfounded and misguided. I discuss the basics of an economic experiment, some of the issues, and how we handle them.[1]
A little about
experimental econ:
It is strange to think about experimenting on human subjects.
Sounds weird… like, do experimental economists just roll up in a large white
cargo van with “free candy” painted on the door, lure subjects inside, poke
them and see how they squirm? In so many words, yes! Then we write long, boring
papers about it that other pure economists and scientists put with their stash
of parking/losing lottery tickets for “toilet use” (sort of kidding). In fact,
economists have been bringing people into labs (not vans) since around the late
1940’s, but only in the last decade has it become popular.[2]
First, what is an
experiment… skip if you think you know, but feel inclined to click on the
hyperlinks:
In the hard sciences, you poke a beast and observe
how it squirms. It’s important to control for the characteristics of the beast
so that you can compare it with other beasts with those same characteristics that
were not poked (see The ScientificMethod). For example, if I wanted to see how a banana ripens when
placed in a container of a certain gas (ethylene), I’d want to have a banana to
compare it to. So, I’d place a banana in a container with “normal” air and make
sure it was as close to the exact same container and banana as possible (size, “greenness”,
etc). I then observe the relative differences in ripening for the two bananas.
Since everything between the two experiments is EXACTLY the same except the gas, I can confidently attribute
any differences in ripening to the presence of the gas.(Note: example inspired by the banana I'm eating...)
So, what’s the problem
with the lab?
In economics, the process is exactly the same except the “beast” can get
curious and mess up the results. The problem is that humans are not used to being caged and probed and will respond in
weird ways! We poke them, and then they think questions like, “Why is it poking
me?”, “How am I supposed to squirm?”, “If I squirm, what will the other beasts
in the room think of me?”, “Okay, A-hole, poke me again…”, etc. They just don’t
act naturally when the environment changes, which is why many people rule out economics
in the laboratory.
Example: DictatorGame. In economics we like to study altruism or “other regarding preferences”.
The dictator game is simple: I give you $10, you decide how much to split
between you and another person, done. The other person just sits there and receives
what you give them. We find that only about 18% of people take the whole $10.
There’s hope for the world! This simple game helps us understand humanity’s
basic regard for other people. HOW ON EARTH CAN HUMANS SCREW THIS UP!?!?
Several ways, illustrated by YOU! Yes, YOU, the person who’s
actually gotten this far into my blog. Don’t turn back now, it’s not a sunk
cost, don’t ignore it, you must continue….
Some example problems
(there are more)[3]:
1)
Scrutiny- Since your decisions are being monitored,
you don’t split it truthfully. For some silly reason you think that I, the
experimenter, care that you screw the other person over and give them nothing
(I don’t). But, since you think I care, you give more than you normally would… so, YOU
SCREW UP MY RESULTS!
Solution: I don’t interact with you.
You go to a cubicle, put the money in an envelope, I don’t touch the envelope,
I don’t distribute the money, someone else does. No pressure.[4]
2)
Framing and context- I tell you that you’re a
trader on Wall Street that has been given stocks to split between you and a
partner. You go, “Hmm… Wall Street. People are cut-throat on Wall Street. I
have permission to be a jerk!” and you leave with $10 dollars instead of giving
what you normally would…. So, YOU SCREW UP MY RESULTS!
Solution: Use neutral language. Provide
no contextual framework whatsoever. Don’t call the person you’re trading with “friend”
or “partner”, call them subject Z. Don’t tell them that they are “splitting” the
money, tell them they get to allocate the funds to themselves and the other
person.
3)
Selection- If you in fact ARE a trader on Wall
Street and I get a group of you and all of your fellow traders to do the
experiment. You all take most of the money for yourself because you are
cut-throat by nature. This result will not be representative of altruism for
the rest of the human race… so, you have SCREWED UP MY RESULTS!
Solution: Have a random sample of
individuals.
4)
Artificial restrictions on choice sets- I give
you $12 dollars, you tithe exactly 10% of all money you get, so you’re going to count
this as your tithe this week. You want to give $1.20 to your partner but you
can’t because I've only given you dollar bills, so you give $1.00. If there are many of you, I will underestimate
altruism. YOU and I HAVE SCREWED UP MY RESULTS!
Solution: As well as I possibly
can, allow for a full range of choice within the lab.
The important theme in ALL of these is that they are real
issues in the laboratory, but hardly any of them go unaddressed. Experimental
economics has evolved (and continues to evolve) to be aware of the pitfalls that
exist by bringing humans into a lab. The issues are NOT stylized facts of the laboratory,
but rather frequent issues that we must address before running an experiment (not unlike any other form of research). In
a way, our job is easier than the hard sciences because getting two “similar”
beasts can be as simple as having individuals fill out a survey and comparing the "like" ones, but the other
sciences can’t ask a banana for demographic information and their potassium content. On the other hand, our life is more
difficult for all of the reasons described above, but we can account for them.
So, rest assured, experimental economists are practicing science! Just as pure,
unfiltered, and “petri dish-like” as we can get it!
[1] This entry was inspired by a lecture on experimental demand effects from Justin Rao, PhD. (Microsoft).
[2] Falk, Armin, and James Heckman, “Lab Experiments Are a Major Source of Knowledge in the Social Sciences” Science, 2009.326, pg. 539.
[2] Falk, Armin, and James Heckman, “Lab Experiments Are a Major Source of Knowledge in the Social Sciences” Science, 2009.326, pg. 539.
[3]
For more, see Levitt and List (2007), “What do Laboratory Experiments Measuring
Social Preferences Reveal about the Real World?”.
[4]
Hoffman, David, Kevin McCabe and Vernon L. Smith. “Social Distance and
Other-regarding Behavior in Dictator Games” The
American Economic Review, 1996. 86(3). Pp.653-660.
Great post, Justin. Recently read an interview on a similiar topic with John List from Chicago. Check it out here: http://www.richmondfed.org/publications/research/region_focus/2012/q2-3/pdf/interview.pdf
ReplyDeleteI think you'll find it valuable as you begin to consider structuring field experiments.
Yes! He mentions many of those ideas regarding field experiments in his 2007 piece with Levitt. They mention a previous study (List, 2006) where they study sports card traders in a lab situation and in the field and find that they are more likely to take advantage of inexperienced buyers when in their natural setting, giving them less quality for their offered price. Somehow, being observed must make them feel like they need to "act right", so they tend to match quality with the buyer's offer.
ReplyDeleteAlso, I notice in the interview he gives a shout out to Mike Price, whom I got to work with briefly here at Tennessee. His studies about charitable giving are awesome. Look him up if you're interested;he's at Georgia State now.