Friday, October 26, 2007
Sunday, April 1, 2007
The Difference Between Statistically Significant and Not Statistically Significant is Not Statistically Significant
By Andrew Gelman and Hal Stern.
The difference between 25+-10 (statistically significantly different from zero) and 10+-10 (not statistically significantly different from zero) is 15+-14 -- not statistically significant.
A good example was the research linking homosexuality to the number of older siblings -- number of older brothers was statistically significant, number of older sisters was not, but the difference between the two effects was not statistically significant!
Gerd Gigerenzer has an article "Mindless Statistics" criticizing the use of statistics in the field of psychology. Psychologists use the "null ritual" to publish papers and advance their careers, but don't take statistics seriously. Practicioners suffer from cognitive illusions which emphasize the "usefulness" of finding a statistically significant result.
If an effect with p<0.01 is found, the probability of the null hypothesis being true is not known, nor is the probability that the effect will be found in future experiments.
The difference between 25+-10 (statistically significantly different from zero) and 10+-10 (not statistically significantly different from zero) is 15+-14 -- not statistically significant.
A good example was the research linking homosexuality to the number of older siblings -- number of older brothers was statistically significant, number of older sisters was not, but the difference between the two effects was not statistically significant!
Gerd Gigerenzer has an article "Mindless Statistics" criticizing the use of statistics in the field of psychology. Psychologists use the "null ritual" to publish papers and advance their careers, but don't take statistics seriously. Practicioners suffer from cognitive illusions which emphasize the "usefulness" of finding a statistically significant result.
If an effect with p<0.01 is found, the probability of the null hypothesis being true is not known, nor is the probability that the effect will be found in future experiments.
Marketing Models
Yonay and Breslau, "Marketing Models: The Culture of Mathematical Economics"
Sociologists often criticize economists for using unrealistic models and for focusing on formal models instead of empirical research. However, the authors look to approach economists as anthropologists approach foreign societies, without using their own standards to evaluate what they see. They focus on "mainstream economics" and in particular on "model-building" which is the "keystone of the economic discipline."
Models in economics are not meant to be realistic but insightful -- capable of highlighting important mechanisms that can explain important phenomena. Economists advance their careers by finding holes in widely accepted theories; they are interested in empirical phenomena to the extent that there is a blind spot in the literature regarding it.
Even as existing theories are revised, there are a set of standard assumptions that are culturally dictated (some assumptions are easier to make than others, because everyone else is also making them). Mainstream economics is committed to methodological individualism (modeled agents are individuals) but not necessarily rationality (although properly modelling deviations from rationality is largely a matter of gut feel and intuition). However, nonrational behavior in economics must still be formalized -- "you must say something." Models must be "close enough" to the real world but also need to be tractable, so that an equilibrium solution exists. Equilibrium solutions are emphasized not because economists believe that the world is static, but because they impose a consistency test on models, meaning that the model is a possible explanation for whatever needs to be explained. As models become more complex, it becomes harder and harder (perhaps impossible) to find an equilibrium solution.
It would be interesting to think about the strengths and limitations of relying on formal models requiring equilibrium.
Sociologists often criticize economists for using unrealistic models and for focusing on formal models instead of empirical research. However, the authors look to approach economists as anthropologists approach foreign societies, without using their own standards to evaluate what they see. They focus on "mainstream economics" and in particular on "model-building" which is the "keystone of the economic discipline."
Models in economics are not meant to be realistic but insightful -- capable of highlighting important mechanisms that can explain important phenomena. Economists advance their careers by finding holes in widely accepted theories; they are interested in empirical phenomena to the extent that there is a blind spot in the literature regarding it.
Even as existing theories are revised, there are a set of standard assumptions that are culturally dictated (some assumptions are easier to make than others, because everyone else is also making them). Mainstream economics is committed to methodological individualism (modeled agents are individuals) but not necessarily rationality (although properly modelling deviations from rationality is largely a matter of gut feel and intuition). However, nonrational behavior in economics must still be formalized -- "you must say something." Models must be "close enough" to the real world but also need to be tractable, so that an equilibrium solution exists. Equilibrium solutions are emphasized not because economists believe that the world is static, but because they impose a consistency test on models, meaning that the model is a possible explanation for whatever needs to be explained. As models become more complex, it becomes harder and harder (perhaps impossible) to find an equilibrium solution.
It would be interesting to think about the strengths and limitations of relying on formal models requiring equilibrium.
Thursday, March 29, 2007
Shortages
In the 1950's, economists Alchian, Arrow, and Capron wrote a paper discussing the existence of a shortage of scientist/engineers in America.
What exactly does a shortage mean? For economists, it's essentially a matter of price not being able to adjust to clear the market. If apples cost 10 cents each but not everyone wanting to buy apples at this price is able to do so, then there is a shortage of apples. In a properly functioning market, this should not happen -- the price of apples should rise until the market clears. Only price controls can prevent this from happening! The government can cure shortages by not interfering with the market.
The layman usually has another meaning in mind. A dramatic increase in the price of apples might lead to a shortage of apples at the old "normal" price. While people are concerned with the "proper" price of apples, economists understand prices as simple market-clearing mechanisms and therefore pretend not to share the intuition that there are proper prices and improper prices. For example, the authors mock the idea of a proper price by discussing those who claim that there is a shortage of servants (instead of realizing that servants are now able to command much higher wages on the market -- hurray!) Such an example is only partially useful rhetorically, because it avoids confrontation with the idea that a dramatic increase in the price of food might be morally problematic.
People who refer to shortages may also believe that engineers are socially undervalued. Not surprisingly the authors find this claim dubious. They are responsive to the notion that competition with the Soviet Union dictates that we increase our supply of engineers, but do a good job pointing out the limits of that logic.
The authors do allow for "dynamic shortages," where institutional factors and market inefficiencies lead to prices adjusting slowly (but in the right direction). They acknowledge possible market inefficiencies due to uncertainty, myopia, monopsonies (the government has considerable market power as a major employer of engineers), and externalities. The monopsony effect is probably particularly relevant -- a large employer of engineers is more exposed to wage increases and therefore more reluctant to provide them. There is also the fact that while greater demand for engineers drives up their wages and serves as a signal that more workers should become engineers, there is a considerable time lag due to the long training period.
There is a discussion of how engineers decide how much education to puruse. It seems misguided to me -- at one point the authors suggest that wages for MSc engineers should be the same as wages for PhD engineers, accounting for education costs (including time), but this would only be true for marginal engineers and even then it would only be true if the PhD didn't serve a useful signalling function (not everyone might have the "choice" to pursue a PhD, constrained only by earnings).
This paper is very long (over 100 pages) and it would be possible to go into a lot more detail, but that doesn't seem like a particularly good use of time. I note the authors discomfort with actively suggesting to children that engineering is a good career (clearly indicating an "inaction is morally more excusable than action" worldview) although the points they make regarding the hidden costs of more engineers are good (more engineers = fewer doctors, all things being equal).
The authors emphasize two reasons why prices should be allowed to fluctuate in a free market. (1) Prices serve as a signal to encourage greater production. (2) Fluctuating prices ensure that resources are allocated efficiently (put to their most produtive use).
What exactly does a shortage mean? For economists, it's essentially a matter of price not being able to adjust to clear the market. If apples cost 10 cents each but not everyone wanting to buy apples at this price is able to do so, then there is a shortage of apples. In a properly functioning market, this should not happen -- the price of apples should rise until the market clears. Only price controls can prevent this from happening! The government can cure shortages by not interfering with the market.
The layman usually has another meaning in mind. A dramatic increase in the price of apples might lead to a shortage of apples at the old "normal" price. While people are concerned with the "proper" price of apples, economists understand prices as simple market-clearing mechanisms and therefore pretend not to share the intuition that there are proper prices and improper prices. For example, the authors mock the idea of a proper price by discussing those who claim that there is a shortage of servants (instead of realizing that servants are now able to command much higher wages on the market -- hurray!) Such an example is only partially useful rhetorically, because it avoids confrontation with the idea that a dramatic increase in the price of food might be morally problematic.
People who refer to shortages may also believe that engineers are socially undervalued. Not surprisingly the authors find this claim dubious. They are responsive to the notion that competition with the Soviet Union dictates that we increase our supply of engineers, but do a good job pointing out the limits of that logic.
The authors do allow for "dynamic shortages," where institutional factors and market inefficiencies lead to prices adjusting slowly (but in the right direction). They acknowledge possible market inefficiencies due to uncertainty, myopia, monopsonies (the government has considerable market power as a major employer of engineers), and externalities. The monopsony effect is probably particularly relevant -- a large employer of engineers is more exposed to wage increases and therefore more reluctant to provide them. There is also the fact that while greater demand for engineers drives up their wages and serves as a signal that more workers should become engineers, there is a considerable time lag due to the long training period.
There is a discussion of how engineers decide how much education to puruse. It seems misguided to me -- at one point the authors suggest that wages for MSc engineers should be the same as wages for PhD engineers, accounting for education costs (including time), but this would only be true for marginal engineers and even then it would only be true if the PhD didn't serve a useful signalling function (not everyone might have the "choice" to pursue a PhD, constrained only by earnings).
This paper is very long (over 100 pages) and it would be possible to go into a lot more detail, but that doesn't seem like a particularly good use of time. I note the authors discomfort with actively suggesting to children that engineering is a good career (clearly indicating an "inaction is morally more excusable than action" worldview) although the points they make regarding the hidden costs of more engineers are good (more engineers = fewer doctors, all things being equal).
The authors emphasize two reasons why prices should be allowed to fluctuate in a free market. (1) Prices serve as a signal to encourage greater production. (2) Fluctuating prices ensure that resources are allocated efficiently (put to their most produtive use).
Game Theory and "Commitment"
Jack Hirshleifer, "Game-Theoretic Interpretations of Commitment"
Where to begin? We can start by looking at the examples of game situations that Hirshleifer uses to illustrate his points.
1. Chicken. Two people drive cars directly at each other. If one of them bails out, the other wins; if neither bail out, they both get killed.
2. Land or Sea. The British choose to attack by land or by sea. The Americans choose to defend either the land or the sea. The Americans win if they defend where the British attack, otherwise the British win.
3. Prisoner's Dilemma. Two prisoners have to choose whether or not to rat out their co-conspirator. If both prisoners keep quiet, they each face a small punishment. But if one of them sells out, he gets off free while the other gets screwed.
4. Battle of the Sexes. A boy and a girl try to arrange a date. The boy prefers to attend a sporting match while the girl prefers to attend the opera. However, each prefers going on a date to not going on a date.
The games all involve two self-interested parties that must each be concerned with the other's actions. In Battle of the Sexes, interests are largely aligned; in Land or Sea, interests are entirely opposed. In Prisoner's Dilemma, the optimal outcome involves cooperation but each party has a selfish interest in defecting. Chicken has a similar structure to Prisoner's Dilemma...
The alignment of interests in Battle of the Sexes means that it is advantageous to have the first move (in this case, decide where you want to go for the evening). The opposite is true in Land or Sea (it is better to be able to react to your opponent). Flexibility is a crucial advantage in Land or Sea but is somewhat of a hindrance in Battle of the Sexes. We can view "commitment" as the opposite of the flexibility -- it's the ability to lock in your move, to tie your hands. Often tying your hands can be an advantage. A child who throws an uncontrollable temper-tantrum gains an advantage over a responsive parent.
If the Prisoner's Dilemma is not an anonymous one-off, parties may reach an optimal outcome through the use of promises and threats. For example, consider the announcement that "like will be met by like." This is essentially a promise that cooperation will be matched by cooperation (because pure cold calculation implies that defection will be matched by defection). If the promise is sufficiently credible, then it will be in the other player's interest to cooperate. (It is not necessary for the promise to be 100% credible.)
Similarly, in Chicken, the second player might announce that "I will defect." This is a threat to crash the car if the other player stays on the road. If the threat is sufficiently credible, then it will be in the other player's interest to bail out. One way to make the threat credible would be to remove the steering wheel and conspicuously throw it out of the window.
It is also possible to combine a threat with a promise. For example, in Chicken a player might announce that "like will be met by like." The advantage in combining a threat with a promise is that the credibility of the promise can make up for a deficiency in the credibility of the threat.
It seems likely that this sort of analysis can provide insights into human passions such as anger and gratitude.
Where to begin? We can start by looking at the examples of game situations that Hirshleifer uses to illustrate his points.
1. Chicken. Two people drive cars directly at each other. If one of them bails out, the other wins; if neither bail out, they both get killed.
2. Land or Sea. The British choose to attack by land or by sea. The Americans choose to defend either the land or the sea. The Americans win if they defend where the British attack, otherwise the British win.
3. Prisoner's Dilemma. Two prisoners have to choose whether or not to rat out their co-conspirator. If both prisoners keep quiet, they each face a small punishment. But if one of them sells out, he gets off free while the other gets screwed.
4. Battle of the Sexes. A boy and a girl try to arrange a date. The boy prefers to attend a sporting match while the girl prefers to attend the opera. However, each prefers going on a date to not going on a date.
The games all involve two self-interested parties that must each be concerned with the other's actions. In Battle of the Sexes, interests are largely aligned; in Land or Sea, interests are entirely opposed. In Prisoner's Dilemma, the optimal outcome involves cooperation but each party has a selfish interest in defecting. Chicken has a similar structure to Prisoner's Dilemma...
The alignment of interests in Battle of the Sexes means that it is advantageous to have the first move (in this case, decide where you want to go for the evening). The opposite is true in Land or Sea (it is better to be able to react to your opponent). Flexibility is a crucial advantage in Land or Sea but is somewhat of a hindrance in Battle of the Sexes. We can view "commitment" as the opposite of the flexibility -- it's the ability to lock in your move, to tie your hands. Often tying your hands can be an advantage. A child who throws an uncontrollable temper-tantrum gains an advantage over a responsive parent.
If the Prisoner's Dilemma is not an anonymous one-off, parties may reach an optimal outcome through the use of promises and threats. For example, consider the announcement that "like will be met by like." This is essentially a promise that cooperation will be matched by cooperation (because pure cold calculation implies that defection will be matched by defection). If the promise is sufficiently credible, then it will be in the other player's interest to cooperate. (It is not necessary for the promise to be 100% credible.)
Similarly, in Chicken, the second player might announce that "I will defect." This is a threat to crash the car if the other player stays on the road. If the threat is sufficiently credible, then it will be in the other player's interest to bail out. One way to make the threat credible would be to remove the steering wheel and conspicuously throw it out of the window.
It is also possible to combine a threat with a promise. For example, in Chicken a player might announce that "like will be met by like." The advantage in combining a threat with a promise is that the credibility of the promise can make up for a deficiency in the credibility of the threat.
It seems likely that this sort of analysis can provide insights into human passions such as anger and gratitude.
Wednesday, February 21, 2007
Daniel Wegner - "Self is Magic"
Wegner suggests that consciousness and self-control is a type of magic -- it is remarkable that we can wish for something and then have it happen! "We are enchanted by the operation of our minds and bodies into believing that we are 'uncaused causes,' the origins of our own behavior."
Since we don't perceive the true physical causal links in our behavior -- neurons firing in our brain -- we have incomplete self-knowledge. Although it seems that we understand ourselves, this sense of understanding is an illusion. Wegner stresses that we are amazed by our own behavior, but this does not seem quite correct to me -- we often take ourselves completely for granted. His parallel with watching a magic trick seems strained. It is true that humans often resist attempts to "pull back the curtain" on our mental operations, however.
There are three key sources of the experience of conscious will -- consistency, priority, and exclusivity.
Consistency. Participants in an experiment were instructed to perform a voodoo curse on an experimental confederate, who later feigned a headache. The participants did believe that they had exerted some influence on the victim (even if the headache was only psychosomatic). But the feeling of responsibility was significantly strengthened among participants who were led to dislike the confederate before applying the curse. This consistency between desire and outcome led to a sense of conscious control (over something that in fact had not been caused at all).
Surely our interest in following sports is related to this effect. Why do people root for sports teams? Furthermore, many people would not want to watch a tape of a sporting event after its completion, even if they were unaware of the outcome. Somehow the unpredictability of a "live" game is more compelling than the unpredictability of a tape of the same game.
Consistency can be undermined by distractions that separate thought from action. This reduces the feeling of voluntariness for actions. Wegner suggests that this effect may underlie hypnosis.
Priority. "The matter of timing is crucial in the perception of willed action, so crucial that even happenstance events may be perceived as under one's control when they occur just after one has happened to think about them."
Exclusivity. "People see their thoughts as causing events to the degree that there are no other plausible candidate causes." If you are thinking of ordering a dish at a restaurant and someone at your table strikes first, you may be tempted to change your order. Milgram's famous torture studies can be interpreted to reveal that humans are quick to experience "agentic shift" -- to view themselves as instruments of others. The experience of conscious will is therefore very dependent on there not being other possible causes (in particular, other social agents).
Wegner moves on to discussing "Breaking the Spell." Lovers who explain why they feel love experience a diminishing feeling. At the same time, people feel that understanding and explaining evil is very close to condoning it. To retain our appreciation of evil, we need to resist understanding it. Furthemore, "explained" beliefs lose importance, but beliefs that are used to provide explanations gain in value (to the believer).
Wegner believes the illusion of self is so persistent and strong that it will not fade, not even after being explained. For example, in one experiment people were asked to judge the moral responsibility of a muderer. If the murderer had shot his victim under overwhelming duress, participants judged him less responsible, as expected. But if the murderer had wanted to do it, he was judged as more responsible, even if his want had not been the actual causal factor. (Red flags: Wegner's conclusion -- that moral responsibility is not a matter of simple causal logic -- is not as straightforward as it might seem.)
The next question is why the feeling is so persistent. Going back to biological foundations (natural selection), we ask: why would natural selection have designed minds that experience the self illusion? Why can't people just act -- why do they need to feel aware? Why do they need to feel in control?
Wegner suggests three possible avenues for the evolution of self-magic:
Social signalling. The experience of conscious will is a signal that the self is "authoring" an act; this in turn can be communicated to others. I am afraid that there seems too much taken for granted in Wegner's account, which really should start at the very beginning.
Social task allocation. Conscious will allows people to know their limitations. Once again, this account seems to presume too much.
Social control. Conscious will readies individuals to accept responsibility. The inner feeling of doing provides a conscience. This definitely hits on something fundamental but I need to spend more time thinking about it.
FURTHER READING:
Baumeister, R.F. (1997) Evil: Inside human cruelty and violence. W.H. Freeman.
Woolfolk, R.L. et al. (2006) . Identification, situational constraint, and social cognition: Studies in the attribution of moral responsibility. Cognition, 100, 283-301.
Since we don't perceive the true physical causal links in our behavior -- neurons firing in our brain -- we have incomplete self-knowledge. Although it seems that we understand ourselves, this sense of understanding is an illusion. Wegner stresses that we are amazed by our own behavior, but this does not seem quite correct to me -- we often take ourselves completely for granted. His parallel with watching a magic trick seems strained. It is true that humans often resist attempts to "pull back the curtain" on our mental operations, however.
There are three key sources of the experience of conscious will -- consistency, priority, and exclusivity.
Consistency. Participants in an experiment were instructed to perform a voodoo curse on an experimental confederate, who later feigned a headache. The participants did believe that they had exerted some influence on the victim (even if the headache was only psychosomatic). But the feeling of responsibility was significantly strengthened among participants who were led to dislike the confederate before applying the curse. This consistency between desire and outcome led to a sense of conscious control (over something that in fact had not been caused at all).
Surely our interest in following sports is related to this effect. Why do people root for sports teams? Furthermore, many people would not want to watch a tape of a sporting event after its completion, even if they were unaware of the outcome. Somehow the unpredictability of a "live" game is more compelling than the unpredictability of a tape of the same game.
Consistency can be undermined by distractions that separate thought from action. This reduces the feeling of voluntariness for actions. Wegner suggests that this effect may underlie hypnosis.
Priority. "The matter of timing is crucial in the perception of willed action, so crucial that even happenstance events may be perceived as under one's control when they occur just after one has happened to think about them."
Exclusivity. "People see their thoughts as causing events to the degree that there are no other plausible candidate causes." If you are thinking of ordering a dish at a restaurant and someone at your table strikes first, you may be tempted to change your order. Milgram's famous torture studies can be interpreted to reveal that humans are quick to experience "agentic shift" -- to view themselves as instruments of others. The experience of conscious will is therefore very dependent on there not being other possible causes (in particular, other social agents).
Wegner moves on to discussing "Breaking the Spell." Lovers who explain why they feel love experience a diminishing feeling. At the same time, people feel that understanding and explaining evil is very close to condoning it. To retain our appreciation of evil, we need to resist understanding it. Furthemore, "explained" beliefs lose importance, but beliefs that are used to provide explanations gain in value (to the believer).
Wegner believes the illusion of self is so persistent and strong that it will not fade, not even after being explained. For example, in one experiment people were asked to judge the moral responsibility of a muderer. If the murderer had shot his victim under overwhelming duress, participants judged him less responsible, as expected. But if the murderer had wanted to do it, he was judged as more responsible, even if his want had not been the actual causal factor. (Red flags: Wegner's conclusion -- that moral responsibility is not a matter of simple causal logic -- is not as straightforward as it might seem.)
The next question is why the feeling is so persistent. Going back to biological foundations (natural selection), we ask: why would natural selection have designed minds that experience the self illusion? Why can't people just act -- why do they need to feel aware? Why do they need to feel in control?
Wegner suggests three possible avenues for the evolution of self-magic:
Social signalling. The experience of conscious will is a signal that the self is "authoring" an act; this in turn can be communicated to others. I am afraid that there seems too much taken for granted in Wegner's account, which really should start at the very beginning.
Social task allocation. Conscious will allows people to know their limitations. Once again, this account seems to presume too much.
Social control. Conscious will readies individuals to accept responsibility. The inner feeling of doing provides a conscience. This definitely hits on something fundamental but I need to spend more time thinking about it.
FURTHER READING:
Baumeister, R.F. (1997) Evil: Inside human cruelty and violence. W.H. Freeman.
Woolfolk, R.L. et al. (2006) . Identification, situational constraint, and social cognition: Studies in the attribution of moral responsibility. Cognition, 100, 283-301.
Subscribe to:
Posts (Atom)
