In a series of interviews with New York Times science writer Gary Taubes on scientificblogging, psychology professor Seth Roberts turns to the question of how do you go about making the judgment as to whether a scientist is trustworthy, especially when the topic is controversial. Taubes responds:
I’m a stickler about the use of words like “evidence” and “proof”. So if someone tells you there’s no evidence for some controversial belief, you can be fairly confident that they’re a bad scientist. There’s always evidence, or there wouldn’t be a controversy. If somebody says that “we proved that this was true” or “we set out to prove that this was true” that’s another bad sign. The point here, as [Karl] Popper noted, among others, is that you can never prove anything is true; you can only refute it. So researchers who talk about proving a hypothesis is true rather than testing it make me worried.
SETH: Yeah, I see what you’re saying. They overstate; they twist things around to make it come out the way they want. They are way too sure of what they…
TAUBES: Yes, and the really good scientists are the ones, almost by definition, who are most skeptical of evidence that seems to support their beliefs. They’re most aware of how they could have been fooled, how they could have screwed up, or how they might have missed artifacts in their experiment that could have explained what they observed. They’re very careful about what they say. If you ask them to do play devil’s advocate, and tell you how they could have screwed up, then at the very least, they’ll say “Well, if I knew how I could have done it, I would have checked it before I made the claim”. So when I’m talking about discerning the difference between a good scientist and a bad scientist, I’m talking about how they speak about their research, the evidence itself, it’s presence or absence.
Worth bearing in mind when you hear something which appears to overturn consensus expressed in strident terms: Where all the other possible explanations for the phenomenon considered? How did the researchers test their theory and data against the best possible countervailing research? Why do their conclusions offer better explanatory power?
January 31, 2008 at 11:53 pm |
Wise. Thank you for that.
February 1, 2008 at 11:57 am |
[...] an article from TheStatsBlog that I found interesting. Here is what they say about the organization behind it. Since its [...]
February 1, 2008 at 2:17 pm |
Pfff, I disagree with Taubes. Sure, he’s right that people preferably shouldn’t say they proved stuff when they didn’t really – but that implies that all good scientists would bother about the philosophy behind that choice of words.
Some don’t. Most haven’t read Popper. They may not be versed in philosophy of science at all. Does that invalidate their work?
February 1, 2008 at 5:47 pm |
Hi Yungchin,
I don’t think one needs to have absorbed Popper in order to have an intellectually scrupulous approach to one’s work: it’s more a matter of asking, have I honestly accounted for the strongest arguments against my hypothesis or theory? Can I address them with better data and explanatory power?
I think the problem is when scientists start believing in their research at the expense of testing their fundamental premises.
As for Popper, his theory of falsifiability is very useful in that it allows us a quick way of deciding what is scientific and what isn’t: if there is no possible way to disprove, say, certain tenets of Freudianism, then there is no possible way of proving them either. Popper’s real problem is in his logical absolutism on inference, which, ironically, makes him more of a mystic than a hard-nosed rationalist.
February 1, 2008 at 7:33 pm |
Hi, thanks for your reply. I see what you mean, and agree. I guess what I dislike about the strong position taken above is just the choice of words: “bad scientist”. I’d rather say this would make an immature scientist.
February 1, 2008 at 10:46 pm |
I’m afraid that some of those this might apply to are quite mature scientists. What bothers me most is the unavailability of raw data, unverified methodology, use of old out of date and sometimes disproved proxies. A good scientist would be searching for the truth and would appreciate learning of any error or incorrect data. That is how science is supposed to be self correcting.
just my 2 cents
Bill
February 7, 2008 at 4:27 pm |
Yeah, I think Taubes is wrong here as well because there are many things where there really *is* no evidence. By that, scientists mean “no good evidence.”
For example, the autism/MMR link: if someone said there was no evidence supporting it, they would be right! If someone said there’s no evidence supporting boot camps for kids, they would be right: all of the existing data does not show that they hold any advantage over alternatives. The only evidence favoring them– and creating ‘controversy’– is anecdote and commercial promotion. I think it’s fair for a scientist to not count that as evidence!
August 19, 2008 at 5:08 am |
i love the explanation and now i can tell a good scientist from a bad scientist
November 10, 2008 at 9:20 pm |
Hey there,
A good scientist is one who is a better philosopher, Science becomes interesting when you compare the fact you are studying with the normal/ behavioural science in humans and nature for example, the bond between covalent can be compared to the shared friendship among true friends, Besides a TRUE SCIENTIST IS one, who always tells, WE HAVE PROVED THIS, But this DOESNT mean that the other is wrong, To prove it to be Wrong you actually have to prove it.