banner
You are not using a standards compliant browser. Because of this you may notice minor glitches in the rendering of this page. Please upgrade to a compliant browser for optimal viewing:
Firefox
Internet Explorer 7
Safari (Mac and PC)
Post Archive
2018 (0)2012 (3)2011 (73)
December (1)

Robot Insects
Tuesday, December 6, 2011
November (6)October (5)

BRB
Tuesday, October 25, 2011

Women's Intuition
Friday, October 21, 2011

Thinking with your Fingers
Monday, October 17, 2011

Choice
Tuesday, October 11, 2011

On Souls and Confections...?
Tuesday, October 4, 2011
September (7)

On Souls...?
Thursday, September 29, 2011

Beware the False Consensus Effect!
Saturday, September 24, 2011

Your Preferences - Preliminary Results
Thursday, September 22, 2011

Popularity Survey - DO IT FOR SCIENCE!
Tuesday, September 20, 2011

Zietsch's Response to PZ, Laden and Scicurious.
Tuesday, September 13, 2011

The Personality of Cities
Tuesday, September 6, 2011

Discussion #1 - Walking Speed and City Size
Sunday, September 4, 2011
August (6)

People who Doodle Learn Faster = Bullshit
Thursday, August 25, 2011

Good News Everyone!
Tuesday, August 23, 2011

Got the Time? Part II
Saturday, August 20, 2011

Got the time?
Sunday, August 14, 2011

Can Randomness Predict the Future?
Tuesday, August 9, 2011

The House of Psycasm
Sunday, August 7, 2011
July (7)June (6)May (8)

Part 1: Do We Have Freewill?
Tuesday, May 31, 2011

The Open Letter: A follow-up
Tuesday, May 31, 2011

This is a lie, she said.
Sunday, May 22, 2011

MSPaint is mightier than the Sword
Saturday, May 21, 2011

The Art of Indecision
Wednesday, May 18, 2011

Skeptical Checklist 1.1
Friday, May 13, 2011

The Skeptical Checklist 1.0
Sunday, May 8, 2011

Of Chimps, Children and Post-Grads...
Monday, May 2, 2011
April (5)March (5)February (7)January (10)

Magical Thinking: Voodoo, Prayer, Black Cats, and You
Thursday, January 27, 2011

The Art of Character Creation
Tuesday, January 25, 2011

Video Game Morality: Actions inside the box?
Friday, January 21, 2011

Psychobabble goes live!
Tuesday, January 18, 2011

The Trolley Problem: Who cares?
Sunday, January 16, 2011

Podcast delay and misc. Drugs!?
Saturday, January 15, 2011

My very own Natural Disaster
Wednesday, January 12, 2011

A window into your Genetics and Mate Preference?
Sunday, January 9, 2011

Oh sorry, I totally phased out there...
Thursday, January 6, 2011

Porn: A force of Mutual Benefits
Sunday, January 2, 2011
2010 (35)
December (7)

Statistical Pwnage
Tuesday, December 21, 2010

Why you should care...
Thursday, December 16, 2010

The paper of Influence
Wednesday, December 15, 2010

Sharing: Part I - Emotions
Sunday, December 12, 2010

No-one cuts deeper than a Science Blogger.
Thursday, December 9, 2010

Me Meme [Ohh, links now]
Tuesday, December 7, 2010

Half Full, or Half Empty? Well, That Depends on the Shape of the Glass.
Sunday, December 5, 2010
November (11)

What Your Voice Says About You
Tuesday, November 30, 2010

Babushka Blog: A Meta-Blog on ResearchBlogging.
Sunday, November 28, 2010

An Announcement:
Tuesday, November 23, 2010

Willful Self-Deception is Bliss
Sunday, November 21, 2010

Dance, Blogger, Dance!
Friday, November 19, 2010

The Science of Marriage
Sunday, November 14, 2010

The Moon and Antarctica
Thursday, November 11, 2010

Available: One Mentee. Good Condition.
Tuesday, November 9, 2010

Again we are limited by our puny human-ness
Sunday, November 7, 2010

Blogging Carnival - What is Psychopathology?: Origins
Friday, November 5, 2010

*sigh* Psi: A Rebuttal
Wednesday, November 3, 2010
October (12)

The Science of Mind-Reading
Thursday, October 28, 2010

How not to think yourself smart...
Tuesday, October 26, 2010

Welcome to Assassins' League
Sunday, October 24, 2010

TODO LATER. A story of procrastination and forgiveness.
Thursday, October 21, 2010

A man and his words.
Tuesday, October 19, 2010

Little kids, little minds...
Sunday, October 17, 2010

Smoking (maybemightcould) is Good.
Thursday, October 14, 2010

How to stop the Apocalpyse
Tuesday, October 12, 2010

How to trick yourself creative
Sunday, October 10, 2010

Siesta - It sounds like Fiesta, but isn't.
Thursday, October 7, 2010

The Dread Pirate Rift
Tuesday, October 5, 2010

Being Blonde. Natural or otherwise...
Sunday, October 3, 2010
September (5)
Rate This Post
Total votes: 2
Blogger Profile

Psycasm

Psycasm is the exploration of the world psychological. Every day phenomenon explained and manipulated to one's own advantage. Written by a slightly overambitious undergrad, Psycasm aims at exploring a whole range of social and cognitive processes in order to best understand how our minds, and those mechanisms that drive them, work.

My posts are presented as opinion and commentary and do not represent the views of LabSpaces Productions, LLC, my employer, or my educational institution.

Post Tags

m
Blog RSS Feed
RSS Add to My Yahoo Add to Google
Saturday, July 16, 2011

A little while ago a friend pointed out I had made a mistake in one of my previous posts.... it was the post What You Might Not Know About Psychology: A Student's Perspective. It was basically me, with my student hat on, attempting to explain some of the nuances and pitfalls of interpreting news and blogs about psychology. Having re-read it, it's a bit of a brass-tacks approach, something I would like to think I would soften if I wrote it again (if only intone, but not content)

In one section of this post I wrote the following:

All Psychology studies are confounded and unreliable because they use university students to test participants!

That's true. The exceeding majority of results are based on populations of students and standardized onto everyone, everywhere (particularly by the media). Additionally, students tend to be WEIRD (White, Educated, from an Industrialized nation, Rich and Democratic). That looks really bad on paper, but most psychologists tend to interpret these finding in a WEIRD context. Additionally, every person I've encountered that objects to a finding because it's WEIRD are WEIRD themselves....

That was a pretty short explanation of a much bigger problem in communicating psychology (particularly things like evolutionary psychology).

So my friend wrote me:

You are wrong. Sorry dude. You absolutely need to read this well known, and short, article:

www.uoguelph.ca/~psystats/readings_3380/mook%20article.pdf

And so I did. I accept that I misrepresented the invalidity point, either through lack of depth in communication, or ignorance (and probably both).

First off, good psychology is not about making generalizations based on small samples. It's about testing generalizations within small samples. It's a nuanced difference, but clear with a little explaining. Psychology (like all science) is based on hypotheses. That is, we make predictions which we hope to be found true, prior to running the experiment. Some kind of theoretical framework exists which psych's do not accept at face value. They seek to scrutinize and criticize it, or seek the limits of its explanatory power. The reasoning follows:

If xyz theory is true, then this, by extension, is also true. Thus, if we find it we have another bit of evidence to support the case of xyz theory. If we find a lack of evidence, or contradictory evidence, then we must consider alternatives.

xyz theory is the generalization that is being tested on a smaller sample. The findings need not necessarily be generalizable to the world at large, as I both explicitly and implicitly stated.

Mook (the author of the article I was referred too), has the following to say on

The distinction between generality of findings and generality of theoretical conclusions underscores what seems to me the most important source of confusion in all this, which is the assumption that the purpose of collecting data in the laboratory is to predict real-life behaviour in the real world. Of course, there are times when that is what we are 'trying to do', and there are times when it is not. When it is, then the problem of EV [external validity] confronts us, full force. When it is not, then the problem of EV is either meaningless or trivial, and a misplaced preoccupation with it can seriously distort our evaluation of the research.

Second, some things identified or described in the lab cannot be generalized into the real world. As above, that may not be the point of the exercise; identifying some effect that would be trivial in the real world (if applied) still may have value.

Take, for example, the Macbeth Effect - the act of washing ourselves tend to influence the manner in which we make moral decisions. I argue the effect is small, based on artificial conditions, and trivial in the real-world. I would be right to say such things, but I may have missed the point of the research. The Macbeth Effect, however small it may be, and however useless it may be in reality, still asks a very interesting question, describes a fascinating phenomenon, and raises further questions that would otherwise have been missed if we only focused 'things we see in the real world'. Washing our hands probably does nothing for us when we're thinking about tax evasion, not tipping the waiter, or using our neighbours garbage bin for our most foul refuse - let's pretend evidence exists that shows this is the case. ... So what? The effect does occur in a lab, and may yet reveal some very subtle cognitive characteristics that are present under certain situations. My inclination to devalue it because it isn't 'real world' is misled; that's not the point and the research still has value.

Third, sometimes it's not about representativeness at all. It's about logic.

For instance, if a theory predicts that x is true when y, then, if y then x. Let's take an example with a terrible sample.

An Hypothesis might follow that Apes, by virtue of being so like humans, are capable of learning language.

So we take a few chimps, a few bonobos and a smattering of gorilla's and orangs, and try to teach them to sign, or to use lexigrams, or to play with a ouija board.

Some can eke out a meaningful passage but have no syntax, some fail, some manage only nouns. Now these Apes are not representative of all Apes. They're orphaned; they're hand-raised; They eat hot-dogs and watch TV. No-one's claiming that these Apes are not representative and so what has been observed is moot. No, it demonstrates the limits (both practical and logical) of theory.

If x is true when y, then, if y then x.

Language is possible if one is near-human. Thus, if Apes are near-human, they should be able to learn aspects of language.

It's a bare-bones account. They hypothesis is not: Perfect English is possible if one is near-human, therefore all Apes should be able to recite poetry.

My initial claim was the WEIRD is always a bad thing, but we deal with it. This is true, but there's much more to it that I was both ignorant of and which I did not communicate fully. But I was called up on it, and now I know so much more about it. The article linked to - here - by Douglas Mook, is clear and concise, and accounts for other common criticisms that are levelled at psychology which I did not cover here (such as the artificiality of the lab setting).

It's a quick, fascinating read on how to understand experimental psychology, and should be requirement for all first-year students. Here, hopefully, I've corrected myself appropriately. I've certainly come out more informed for trying to do so.

 

---

ResearchBlogging.org

Mook, D. (1983). In defense of external invalidity. American Psychologist, 38 (4), 379-387 DOI: 10.1037/0003-066X.38.4.379

This post has been viewed: 5558 time(s)

Blog Comments
Happy Quotes

Guest Comment

Well,I think what you wrote in your last post is actually true to some extent.I'm saying this because I'm from an asian country and some researches and results I see on the internet is not applicable to our lives.Even relationship advises and stuff.May be it's the cultural difference,I don't know for sure.Anyway thank you very much for trying to explain this :)

Add Comment?
Comments are closed 2 weeks after initial post.
Friends