How to read a scientific paper

[Apostasius]

Due to the creationist penchant for quote-mining and egregious misinterpretation of information contained in scientific papers, I thought I would provide a few links that might help reduce such poor attempts to engage scientific scholarship. Perhaps this post will also generate discussion about such things or include additional links or sources of information.

http://www.biochem.arizona.edu/classes/bioc568/papers.htm

http://www.zoo.utoronto.ca/zoo357h1f/Resources/How%20to%20read%20a%20scientific%20paper%20critica lly.pdf

http://www.fiu.edu/~collinsl/Article%20reading%20tips.htm

[ck1]

Due to the creationist penchant for quote-mining and egregious misinterpretation of information contained in scientific papers, I thought I would provide a few links that might help reduce such poor attempts to engage scientific scholarship. Perhaps this post will also generate discussion about such things or include additional links or sources of information.

http://www.biochem.arizona.edu/classes/bioc568/papers.htm

http://www.zoo.utoronto.ca/zoo357h1f/Resources/How%20to%20read%20a%20scientific%20paper%20critica lly.pdf

http://www.fiu.edu/~collinsl/Article%20reading%20tips.htm

The links you provided include information on how scientific papers are formatted. I was immediately reminded of PZ Myers' discussion yesterday of a new paper published by ICR:

COMPLEX LIFE CYCLES IN HETEROPHYID TREMATODES: STRUCTURAL AND DEVELOPMENTAL DESIGN IN THE ASCOCOTYLE COMPLEX OF SPECIES
MARK H. ARMITAGE, M.S.
ICR Graduate School, 10946 Woodside Ave. N., Santee, CA 92071
Voice: (619) 448-0900 Fax: (619) 448-3469

Sounds very technical. Turns out this paper has all but one of the usual parts of a scientific paper: Abstract, Introduction, Materials and Methods, Results, Discussion/Conclusion.

No results section!

http://scienceblogs.com/pharyngula/2008/04/they_call_this_science.php

[VoxRat]

...The links you provided include information on how scientific papers are formatted. I was immediately reminded of PZ Myers' discussion yesterday of a new paper published by ICR:

COMPLEX LIFE CYCLES IN HETEROPHYID TREMATODES: STRUCTURAL AND DEVELOPMENTAL DESIGN IN THE ASCOCOTYLE COMPLEX OF SPECIES
MARK H. ARMITAGE, M.S.
ICR Graduate School, 10946 Woodside Ave. N., Santee, CA 92071
Voice: (619) 448-0900 Fax: (619) 448-3469

Sounds very technical. Turns out this paper has all but one of the usual parts of a scientific paper: Abstract, Introduction, Materials and Methods, Results, Discussion/Conclusion.

No results section!

http://scienceblogs.com/pharyngula/2008/04/they_call_this_science.phpI would love to see Dave Hawkins, of Kids4Truth, Truthmatters.info, tricity ministries, etc - who has claimed that "creationist scholarship" [sic] is every bit as rigorous and respectable as real world scientific scholarship - defend this.

I would also really like to see pigs fly.

[SteveF]

That ICR article was hilarious and his response even more mirth-making. Armitage said in reply:

I am SO THANKFUL and indebted to Paul Myers for the carefully crafted and dispassionate published critique (above) of my work on the complexity of certain trematodes. I especially appreciate the way it was pubished in a peer-reviewed journal such as Parasitology Research or the Journal of Parasitology (oops - I got that wrong didn't I, a blog is just a rag isn't it?).

My irony meter went out of the roof when reading this. Armitage publishes what is essentially a not very interesting 1st year undergraduate essay (with a smart sounding title) at a YEC conference and then has the gall to whine at PZ for not submitting his response to a major journal. I genunely think this is delusion to the point of mental illness.

[ericmurphy]

Due to the creationist penchant for quote-mining and egregious misinterpretation of information contained in scientific papers, I thought I would provide a few links that might help reduce such poor attempts to engage scientific scholarship. Perhaps this post will also generate discussion about such things or include additional links or sources of information.

http://www.biochem.arizona.edu/classes/bioc568/papers.htm

http://www.zoo.utoronto.ca/zoo357h1f/Resources/How%20to%20read%20a%20scientific%20paper%20critica lly.pdf

http://www.fiu.edu/~collinsl/Article%20reading%20tips.htm

The links you provided include information on how scientific papers are formatted. I was immediately reminded of PZ Myers' discussion yesterday of a new paper published by ICR:

COMPLEX LIFE CYCLES IN HETEROPHYID TREMATODES: STRUCTURAL AND DEVELOPMENTAL DESIGN IN THE ASCOCOTYLE COMPLEX OF SPECIES
MARK H. ARMITAGE, M.S.
ICR Graduate School, 10946 Woodside Ave. N., Santee, CA 92071
Voice: (619) 448-0900 Fax: (619) 448-3469

Sounds very technical. Turns out this paper has all but one of the usual parts of a scientific paper: Abstract, Introduction, Materials and Methods, Results, Discussion/Conclusion.

No results section!

http://scienceblogs.com/pharyngula/2008/04/they_call_this_science.php

Creationist argument against evolution in a nutshell:

1) describe some organism or organ system or biochemical pathway that is complex.

2) assert that such an organism or organ system or biochemical pathway is too complex to have evolved.

3) assert that, therefore, evolutionary theory must be wrong.

Have I left out any steps?

[SteveF]

Creationist argument against evolution in a nutshell:

1) describe some organism or organ system or biochemical pathway that is complex.

2) assert that such an organism or organ system or biochemical pathway is too complex to have evolved.

3) assert that, therefore, evolutionary theory must be wrong.

Have I left out any steps?

The overwhelming sense of smugness and superiority. Not a step as such, but it's usually in there.

[Dlx2]

I read through, and I feel like I should add a few things:

First, the most important sections are the least interesting: Materials and Methods (M&M) and Results. M&M tells you how they acquired their data. Results tells you what that data was. Ideally, one can read just those two sections without the rest of the paper, and you'll know everything there is to know.

Anyways, I suggest the following approach.

Read the abstract. This doesn't really inform you of much, but it tells you what the general subject of the paper is, and the questions the paper attempts to address. Generally, this lets you know whether the paper is something you want to waste your time reading. It's not going to tell you a lot about the data, though.

Next, skim the introduction. Most of this is going to be the authors trying to sell you on the idea that this is an important line of inquiry, but the last paragraph or so will give you necessary background on the project and the goal of the analysis. When they present the problem, picture in your mind how ideally you'd go about testing the problem.

Now read M&M. Chances are, you're going to be upset if you know the first thing about experimental design. I know I normally am. Why? Because a lot of studies have significant sampling error problems, or use inappropriate methodology. Recognize the experimental and null hypothesis of the analysis, and note how one would go about differentiating the two. It might help to diagram the experimental design (you can do this more or less like one would diagram a flowchart). This will let you know exactly what can and can't be said from various kinds of data, and what the experiment/study won't say at all.

Now read the results. Use the author's data to navigate the flowchart you made (in your head or on paper) and determine what types of conclusions can be drawn. Look at the figures associated with that section, as well. If large swaths of data are in supplementary information, look at that, too, if you don't understand where they got their data. Think of a few factors which could be potentially hiding signal that should be there, or which could be producing signal that shouldn't. Now that you know what data they got, what they can conclude, and what potential confounding factors could have been messing with their data, you can move on.

Now read their discussion. Does it actually follow the results, or does it pull something out of left field? Do they use words like 'marginally significant' (which means, insignificant, but we really wanted it to be significant, and look! it's so close!)? Do they make interpretations that are completely off-the-wall? Make note of their proposals for new directions in research. These are either things they plan on analyzing, or good topics to consider researching, yourself, so long as the data suggests that they're not pulling shit out their asses.

Finally, read their acknowledgments. Figure out who they're corresponding with on the subject, and where their funding is coming from. If you're researching something related, these might be people/funding agencies to seek out later.

[ck1]

I read through, and I feel like I should add a few things:

First, the most important sections are the least interesting: Materials and Methods (M&M) and Results.

I agree, but the journals apparently do not emphasize this importance. The trend among the newer journals is to pull out the M&M and put them at the very end of the paper, almost as an afterthought. They must do this because they think readers just skip over that stuff.

[Apostasius]

In the third link I provided, this statement stood out: "Never cite an article after having read only the abstract!" This would appear to be good advice to certain creationists.

[SteveF]

I agree pretty much 100% with Dlx2, except in some circumstances. For example, my research is in palaeecology and fire history (with some palaeoclimatology and archaeology thrown in). A lot of papers that I read are essentially the same study, done somewhere else. For example, it might be a late glacial pollen diagram from another locality in the UK. In such an instance, I don't particularly need to pay too much attention to M&M because I know exactly what they will have done. Also, depending on what I want out from the paper, I might not really need to read the results in a huge amount of depth. They will often consistent of essentially a tedious up-core description of species changes. In some instances this is useful, in other instances less so.

So, whilst Dlx2's scheme is pretty much spot on in principal, there are instances where it does not need to be so rigorously followed.

[Febble]

I read through, and I feel like I should add a few things:

First, the most important sections are the least interesting: Materials and Methods (M&M) and Results. M&M tells you how they acquired their data. Results tells you what that data was. Ideally, one can read just those two sections without the rest of the paper, and you'll know everything there is to know.

Yeah, me too. I even had a post half written but abandoned it, and in any case yours is better. It's always tempting to skip the Methods when you just want the gist, but in effect it means you are relying on the peer-review process, which is by no means foolproof.

Not all papers say what you (or they) think they are saying once you've ploughed through the boring bit. Occasionally they say the opposite! But that's the beauty of the format of scientific papers. You have to report what you did and what you found, so that even if the authors think it means one thing, enough information is available to make an alternative inference possible.

When I was doing programs in methodology, one of the exercises we were set was to write an abstract, introduction and discussion sections for a paper for which we were only given the Methods and Results. Then to compare our versions with the authors'. It was very salutary.

[Roy]

Anyways, I suggest the following approach. Or, if you're a creationist:

Skim the abstract. This doesn't really inform you of much, but it lets you pick up some key words, and see whether the authors use words related to the subject you want to address. Generally, this lets you know whether the paper is something you want to cite without reading. It's not going to tell you a lot about the content, though.

Next, read the introduction. Most of this is going to be irrelevant, but it will tell you whether the authors are likely to be careless in their wording, and provide something mineable. When they present the problem, see if they overemphasise it, or fail to mention the scope in the same part of the sentence.

Now eat some M&Ms. Chances are, you are clueless as to what the paper is about. Why? Because a lot of studies use technical terms, include significant sampling statistics, or use scientific methodology. Learn to recognize the questions and the answers, and note how one would go about separating the two. It might help to delete the experimental design, the data and the results in order to focus on any opinions presented. This will let you know exactly what can be safely quoted from, such as what the experimenters don't say at all.

Now read the words. Use the author's text to highlight the argument you made (in your head or on paper) and determine what types of words can be used. Look at the punctuation, as well. If several useful words are in separate sentences, look at that, too, if you think you can construct something from them. Think of a few ways of constructing sentences that should be there, or excising awkward clarifications that shouldn't. Now that you know what text they wrote, what parts you can mine, and what potential there is for fabricating text that supports your preconceived ideas.

Use their discussion; ignore their results. Pull something out of left field. Do they use phrases like 'problem is solved' (which means there's a solution, but we really want there to be a problem, and look! it's so close!)? Make interpretations that are completely off-the-wall. Make note of their proposals for new directions in research. These are usually things that haven't been solved yet, hence fruitful quote-mining areas.

Finally, read their acknowledgments. Figure out who they're corresponding with on the subject, and where their funding is coming from. The more prestigious their institution is, the more authoritative they will seem. Conversely, the more obscure they are, the less chance there is of your mendacity being uncovered.

Roy

[supersport]

Due to the creationist penchant for quote-mining and egregious misinterpretation of information contained in scientific papers, I thought I would provide a few links that might help reduce such poor attempts to engage scientific scholarship. Perhaps this post will also generate discussion about such things or include additional links or sources of information.

http://www.biochem.arizona.edu/classes/bioc568/papers.htm

http://www.zoo.utoronto.ca/zoo357h1f/Resources/How%20to%20read%20a%20scientific%20paper%20critica lly.pdf

http://www.fiu.edu/~collinsl/Article%20reading%20tips.htm

The links you provided include information on how scientific papers are formatted. I was immediately reminded of PZ Myers' discussion yesterday of a new paper published by ICR:

COMPLEX LIFE CYCLES IN HETEROPHYID TREMATODES: STRUCTURAL AND DEVELOPMENTAL DESIGN IN THE ASCOCOTYLE COMPLEX OF SPECIES
MARK H. ARMITAGE, M.S.
ICR Graduate School, 10946 Woodside Ave. N., Santee, CA 92071
Voice: (619) 448-0900 Fax: (619) 448-3469

Sounds very technical. Turns out this paper has all but one of the usual parts of a scientific paper: Abstract, Introduction, Materials and Methods, Results, Discussion/Conclusion.

No results section!

http://scienceblogs.com/pharyngula/2008/04/they_call_this_science.php

Creationist argument against evolution in a nutshell:

1) describe some organism or organ system or biochemical pathway that is complex.

2) assert that such an organism or organ system or biochemical pathway is too complex to have evolved.

3) assert that, therefore, evolutionary theory must be wrong.

Have I left out any steps?

Evolutionist argument in a nutshell:

1) observe some complex organ or trait.

2) assert that it appeared for no particular reason at some point in history via random mutation, yet have no evidence that a random mutation could produce such a thing.

3) call creationists crazies.

[Dlx2]

Evolutionist argument in a nutshell:

1) observe some complex organ or trait.

2) assert that it appeared for no particular reason at some point in history via random mutation, yet have no evidence that a random mutation could produce such a thing.

3) call creationists crazies.

Have you ever read a scientific paper with an intention of learning what the scientist is saying rather than supporting your worldview?

This is a serious question, Guzman.

[Febble]

Can we keep this thread on the topic of how to read a scientific paper? It's potentially a very interesting one, and I don't want to spend another evening doing multiple splits.

How about someone finds a reasonably short interesting paper on the evolution of some trait, we all read it, and do a journal club-type dissection of it. Any suggestions?

[Lucretius III]

Due to the creationist penchant for quote-mining and egregious misinterpretation of information contained in scientific papers, I thought I would provide a few links that might help reduce such poor attempts to engage scientific scholarship. Perhaps this post will also generate discussion about such things or include additional links or sources of information.

http://www.biochem.arizona.edu/classes/bioc568/papers.htm

http://www.zoo.utoronto.ca/zoo357h1f/Resources/How%20to%20read%20a%20scientific%20paper%20critica lly.pdf

http://www.fiu.edu/~collinsl/Article%20reading%20tips.htm

The links you provided include information on how scientific papers are formatted. I was immediately reminded of PZ Myers' discussion yesterday of a new paper published by ICR:

COMPLEX LIFE CYCLES IN HETEROPHYID TREMATODES: STRUCTURAL AND DEVELOPMENTAL DESIGN IN THE ASCOCOTYLE COMPLEX OF SPECIES
MARK H. ARMITAGE, M.S.
ICR Graduate School, 10946 Woodside Ave. N., Santee, CA 92071
Voice: (619) 448-0900 Fax: (619) 448-3469

Sounds very technical. Turns out this paper has all but one of the usual parts of a scientific paper: Abstract, Introduction, Materials and Methods, Results, Discussion/Conclusion.

No results section!

http://scienceblogs.com/pharyngula/2008/04/they_call_this_science.php

Creationist argument against evolution in a nutshell:

1) describe some organism or organ system or biochemical pathway that is complex.

2) assert that such an organism or organ system or biochemical pathway is too complex to have evolved.

3) assert that, therefore, evolutionary theory must be wrong.

Have I left out any steps?

4) Haleluah ! God be praised

5)Buy our collection of Creationist books and DVDs for $119.95

[Mike PSS]

In the third link I provided, this statement stood out: "Never cite an article after having read only the abstract!" This would appear to be good advice to certain creationists.
Unless your on a message board, the abstract can be quote-mined to LOOK like it supports your point, and your too cheap to purchase access to the publisher. :D

[Martin B]

I read through, and I feel like I should add a few things:

First, the most important sections are the least interesting: Materials and Methods (M&M) and Results. M&M tells you how they acquired their data. Results tells you what that data was. Ideally, one can read just those two sections without the rest of the paper, and you'll know everything there is to know.

Anyways, I suggest the following approach.

Read the abstract. This doesn't really inform you of much, but it tells you what the general subject of the paper is, and the questions the paper attempts to address. Generally, this lets you know whether the paper is something you want to waste your time reading. It's not going to tell you a lot about the data, though.

Next, skim the introduction. Most of this is going to be the authors trying to sell you on the idea that this is an important line of inquiry, but the last paragraph or so will give you necessary background on the project and the goal of the analysis. When they present the problem, picture in your mind how ideally you'd go about testing the problem.

Now read M&M. Chances are, you're going to be upset if you know the first thing about experimental design. I know I normally am. Why? Because a lot of studies have significant sampling error problems, or use inappropriate methodology. Recognize the experimental and null hypothesis of the analysis, and note how one would go about differentiating the two. It might help to diagram the experimental design (you can do this more or less like one would diagram a flowchart). This will let you know exactly what can and can't be said from various kinds of data, and what the experiment/study won't say at all.

Now read the results. Use the author's data to navigate the flowchart you made (in your head or on paper) and determine what types of conclusions can be drawn. Look at the figures associated with that section, as well. If large swaths of data are in supplementary information, look at that, too, if you don't understand where they got their data. Think of a few factors which could be potentially hiding signal that should be there, or which could be producing signal that shouldn't. Now that you know what data they got, what they can conclude, and what potential confounding factors could have been messing with their data, you can move on.

Now read their discussion. Does it actually follow the results, or does it pull something out of left field? Do they use words like 'marginally significant' (which means, insignificant, but we really wanted it to be significant, and look! it's so close!)? Do they make interpretations that are completely off-the-wall? Make note of their proposals for new directions in research. These are either things they plan on analyzing, or good topics to consider researching, yourself, so long as the data suggests that they're not pulling shit out their asses.

Finally, read their acknowledgments. Figure out who they're corresponding with on the subject, and where their funding is coming from. If you're researching something related, these might be people/funding agencies to seek out later.

Oh bullshit. I should remind you that you share this board with other paleontologists. We know you just look at the pictures! ;)

[Dlx2]

Oh bullshit. I should remind you that you share this board with other paleontologists. We know you just look at the pictures! ;)

Only in species descriptions ;)