Tag Archives: Data

Commentary

The Economist: some problems with the peer-review process

1 Comment

From The Economist, of all places.

Excerpt:

The idea that the same experiments always get the same results, no matter who performs them, is one of the cornerstones of science’s claim to objective truth. If a systematic campaign of replication does not lead to the same results, then either the original research is flawed (as the replicators claim) or the replications are (as many of the original researchers on priming contend). Either way, something is awry.

It is tempting to see the priming fracas as an isolated case in an area of science—psychology—easily marginalised as soft and wayward. But irreproducibility is much more widespread. A few years ago scientists at Amgen, an American drug company, tried to replicate 53 studies that they considered landmarks in the basic science of cancer, often co-operating closely with the original researchers to ensure that their experimental technique matched the one used first time round. According to a piece they wrote last year in Nature, a leading scientific journal, they were able to reproduce the original results in just six. Months earlier Florian Prinz and his colleagues at Bayer HealthCare, a German pharmaceutical giant, reported in Nature Reviews Drug Discovery, a sister journal, that they had successfully reproduced the published results in just a quarter of 67 seminal studies.

Let’s take a look at some of the problems from the article.

Problems with researcher bias:

Other data-heavy disciplines face similar challenges. Models which can be “tuned” in many different ways give researchers more scope to perceive a pattern where none exists. According to some estimates, three-quarters of published scientific papers in the field of machine learning are bunk because of this “overfitting”, says Sandy Pentland, a computer scientist at the Massachusetts Institute of Technology.

Problems with journal referees:

Another experiment at the BMJ showed that reviewers did no better when more clearly instructed on the problems they might encounter. They also seem to get worse with experience. Charles McCulloch and Michael Callaham, of the University of California, San Francisco, looked at how 1,500 referees were rated by editors at leading journals over a 14-year period and found that 92% showed a slow but steady drop in their scores.

As well as not spotting things they ought to spot, there is a lot that peer reviewers do not even try to check. They do not typically re-analyse the data presented from scratch, contenting themselves with a sense that the authors’ analysis is properly conceived. And they cannot be expected to spot deliberate falsifications if they are carried out with a modicum of subtlety.

Problems with fraud:

Fraud is very likely second to incompetence in generating erroneous results, though it is hard to tell for certain. Dr Fanelli has looked at 21 different surveys of academics (mostly in the biomedical sciences but also in civil engineering, chemistry and economics) carried out between 1987 and 2008. Only 2% of respondents admitted falsifying or fabricating data, but 28% of respondents claimed to know of colleagues who engaged in questionable research practices.

Problems releasing data:

Reproducing research done by others often requires access to their original methods and data. A study published last month inPeerJ by Melissa Haendel, of the Oregon Health and Science University, and colleagues found that more than half of 238 biomedical papers published in 84 journals failed to identify all the resources (such as chemical reagents) necessary to reproduce the results. On data, Christine Laine, the editor of the Annals of Internal Medicine, told the peer-review congress in Chicago that five years ago about 60% of researchers said they would share their raw data if asked; now just 45% do. Journals’ growing insistence that at least some raw data be made available seems to count for little: a recent review by Dr Ioannidis which showed that only 143 of 351 randomly selected papers published in the world’s 50 leading journals and covered by some data-sharing policy actually complied.

Critics of global warming have had problems getting at data before, as Nature reported here:

Since 2002, McIntyre has repeatedly asked Phil Jones, director of CRU, for access to the HadCRU data. Although the data are made available in a processed gridded format that shows the global temperature trend, the raw station data are currently restricted to academics. While Jones has made data available to some academics, he has refused to supply McIntyre with the data. Between 24 July and 29 July of this year, CRUreceived 58 freedom of information act requests from McIntyre and people affiliated with Climate Audit. In the past month, the UK Met Office, which receives a cleaned-up version of the raw data from CRU, has received ten requests of its own.

Why would scientists hide their data? Well, recall that the Climategate scandal resulted from unauthorized release of the code used to generate the data used to promote global warming alarmism. The leaked code showed that the scientists had been generating faked data using a “fudge factor”.

Elsewhere, leaked e-mailed from global warmists revealed that they do indeed suppress articles that are critical of global warming alarmism:

As noted previously, the Climategate letters and documents show Jones and the Team using the peer review process to prevent publication of adverse papers, while giving softball reviews to friends and associates in situations fraught with conflict of interest. Today I’ll report on the spectacle of Jones reviewing a submission by Mann et al.

Let’s recall some of the reviews of articles daring to criticize CRU or dendro:

I am really sorry but I have to nag about that review – Confidentially I now need a hard and if required extensive case for rejecting (Briffa to Cook)
…
If published as is, this paper could really do some damage. It is also an ugly paper to review because it is rather mathematical, with a lot of Box-Jenkins stuff in it. It won’t be easy to dismiss out of hand as the math appears to be correct theoretically, (Cook to Briffa)
…
Recently rejected two papers (one for JGR and for GRL) from people saying CRU has it wrong over Siberia. Went to town in both reviews, hopefully successfully. (Jones to Mann)

One last quote from the Economist article. One researcher submitted a completely bogus paper to many journals, and many of them accepted it:

John Bohannon, a biologist at Harvard, recently submitted a pseudonymous paper on the effects of a chemical derived from lichen on cancer cells to 304 journals describing themselves as using peer review. An unusual move; but it was an unusual paper, concocted wholesale and stuffed with clangers in study design, analysis and interpretation of results. Receiving this dog’s dinner from a fictitious researcher at a made up university, 157 of the journals accepted it for publication.

Dr Bohannon’s sting was directed at the lower tier of academic journals. But in a classic 1998 study Fiona Godlee, editor of the prestigious British Medical Journal, sent an article containing eight deliberate mistakes in study design, analysis and interpretation to more than 200 of the BMJ’s regular reviewers. Not one picked out all the mistakes. On average, they reported fewer than two; some did not spot any.

The Economist article did not go into the problem of bias due to worldview presuppositions, though. So let me say something about that.

A while back Casey Luskin posted a list of problems with peer review.

Here was one that stuck out to me:

Point 5: The peer-review system is often biased against non-majority viewpoints.
The peer-review system is largely devoted to maintaining the status quo. As a new scientific theory that challenges much conventional wisdom, intelligent design faces political opposition that has nothing to do with the evidence. In one case, pro-ID biochemist Michael Behe submitted an article for publication in a scientific journal but was told it could not be published because “your unorthodox theory would have to displace something that would be extending the current paradigm.” Denyse O’Leary puts it this way: “The overwhelming flaw in the traditional peer review system is that it listed so heavily toward consensus that it showed little tolerance for genuinely new findings and interpretations.”

Recently, I summarized a podcast on the reviewer bias problem featuring physcist Frank Tipler. His concern in that podcast was that peer-review would suppress new ideas, even if they were correct. He gave examples of this happening. Even a paper by Albert Einstein was rejected by a peer-reviewed journal. Elsewhere, Tipler was explicitly told to remove positive references to intelligent design in order to get his papers published. Tipler’s advice was for people with new ideas to bypass the peer-reviewed journal system entirely.

Speaking about the need to bypass peer-review, you might remember that the Darwinian hierarchy is not afraid to have people sanctioned if they criticize Darwinism in peer-reviewed literature.

Recall the case of Richard Sternberg.

Excerpt:

In 2004, in my capacity as editor of The Proceedings of the Biological Society of Washington, I authorized “The Origin of Biological Information and the Higher Taxonomic Categories” by Dr. Stephen Meyer to be published in the journal after passing peer-review. Because Dr. Meyer’s article presented scientific evidence for intelligent design in biology, I faced retaliation, defamation, harassment, and a hostile work environment at the Smithsonian’s National Museum of Natural History that was designed to force me out as a Research Associate there. These actions were taken by federal government employees acting in concert with an outside advocacy group, the National Center for Science Education. Efforts were also made to get me fired from my job as a staff scientist at the National Center for Biotechnology Information.

So those are some of the issues to consider when thinking about the peer-review process. My view is that peer-reviewed evidence does count for something in a debate situation, but as you can see from the Economist article, it may not count for as much as it used to. I think my view of science in general has been harmed by what I saw from physicist Lawrence Krauss in his third debate with William Lane Craig. If a scientist can misrepresent another scientist and not get fired by his employer, then I think we really need to be careful about the level of honesty in the academy.

News

New study: domestic violence is twice as likely for two-income couples

4 Comments

From Psych Central. (H/T Stuart Schneiderman)

Excerpt:

Intimate partner violence is two times more likely to occur in two income households, compared to those where only one partner works, according to a new study.

Conducted by Sam Houston State University researchers Cortney A. Franklin, Ph.D., and doctoral student Tasha A. Menaker and supported by the Crime Victims’ Institute, the study looked at the impact of education levels and employment among heterosexual partners as it relates to domestic violence.

While the researchers found that differences in education levels appeared to have little influence, when both partners were working, intimate partner violence increased.

“When both male and females were employed, the odds of victimization were more than two times higher than when the male was the only breadwinner in the partnership, lending support to the idea that female employment may challenge male authority and power in a relationship,” said the researchers.

The study was based on telephone interviews with 303 women who identified themselves as either currently or recently in a serious romantic relationship.

[…]The study found that more than 60 percent of women in two-income couples reported victimization, while only 30 percent of women reported victimization in cases when only the male partner was employed.

[…]The study is scheduled to be published in the journal Violence Against Women.

Dr. Schneiderman comments:

To the best of my knowledge the research does not show whether the wives in question were  using their income as a way to diminish and disrespect their husbands.

Feminism has long been claiming that the male role of provider or breadwinner is a social construct designed to oppress women. If, however, the role is instinctive, and if it is intrinsic to male pride, then the feminist derogation of it is a losing fight.

In my own case, I would never, ever ever marry a woman who expected to work at all if there were children under 5 in the home. What a woman does in marriage is care for her husband and her children. If she is dismissive of the needs of men and children, then marriage is not for her. That’s why it is so important to talk to women about what they believe marriage is, why they want to get married, and why they want to have children. Ask them what the needs of men are in a marriage. Ask them what the needs of children are through their development. What is her plan for her husband and children? How does she intend to achieve those plans? What decisions has she made to prepare? What actions has she performed to show where her priorities lie?

Marriage is not just whatever people decide marriage is. It’s the joining together of two opposite sexual natures, and there are rules and guidelines about how to do that. It is a tense, close-quarters situation that requires that both parties understand that the sexes are different and have different needs. A man has to make certain choices and perform certain actions to fuel his wife and keep her engaged. And a woman has to make certain choices and perform certain actions to fuel her husband and keep him engaged. You can’t have a real marriage with a feminist who repudiates sex differences and the obligations that natural marriage imposes on each partner. It’s fine if a woman says things like “I want to keep working after I get married” or “I will put my children in day care a few weeks after they are born”. All that means is that she isn’t qualified for marriage. Cohabitation is a better option, or maybe just hook-up sex, divorce, single motherhood and spinsterhood. Those are the options – either marriage or feminism.

By the way, please note that research shows that women are as likely to commit domestic violence as men. That’s not my opinion, that’s what studies by the Canadian and UK governments show.

Polemics

Four reasons why positing the resurrection best explains the historical data

2 Comments

From Ratio Christi at Ohio State University.

Introduction:

When it comes to the Christian faith, there is no doctrine more important than the resurrection of Jesus. Biblical faith is not simply centered in ethical and religious teachings. Instead, it is founded on the person and work of Jesus. From a soteriological perspective, if Jesus was not raised from the dead, we as His followers are still dead in our sins (1Cor.15:7).

He lists four pieces of “historical bedrock”. These are the facts that even really skeptical atheists like Bart Ehrman and James Crossley will give you in a debate.

  1. The post-mortem appearances
  2. Paul’s use of the Greek word “soma”, which means body
  3. Identification of Jesus as divine by the earliest Christian community
  4. The rapid growth of early Christianity even after its founder was dead

This is a very very minimal set. He did not even use the burial, much less the empty tomb, which is a harder sell.

Here’s a closer look at Reason #2 of 4:

2.The Bodily Resurrection of Jesus Explains Paul’s Use of the Word “Soma”

Whenever the New Testament mentions the word body, in the context of referring to an individual human being, the Greek word (soma, always refers to a literal, physical body). This is significant because Paul uses the word soma to describe the resurrection body of Jesus (1 Cor.15:42-44).Greek specialist Robert Gundry says “the consistent and exclusive use of soma for the physical body in anthropological contexts resists dematerialization of the resurrection, whether by idealism or by existentialism.” (1) Furthermore, N.T. Wright’s The Resurrection of the Son of God shows that the Greek word for resurrection which is “anastasis” was used by ancient Jews, pagans, and Christians as bodily in nature, with this being the case until much later(A.D. 200).

The only explanation that can be given to the emphatic insistence on the early proclamation of the bodily resurrection of Jesus, rather than translation or even a spiritual body is the fact that the apostles did in fact actually witness a material resurrection.

And one Bible passage from his Reason #3: 1 Corinthians 8:4-6.

4So then, about eating food sacrificed to idols: We know that an idol is nothing at all in the world and that there is no God but one.

5For even if there are so-called gods, whether in heaven or on earth (as indeed there are many “gods” and many “lords”),

6yet for us there is but one God, the Father, from whom all things came and for whom we live; and there is but one Lord, Jesus Christ, through whom all things came and through whom we live.

1 Corinthians is unanimously viewed by historians across the ideological spectrum – from evangelical to atheist – as a genuine epistle written by Paul, around 55 AD. I think this passage argues strongly that the earliest Christians thought of Jesus as other than an ordinary man. And the Ratio Christi post has many more passages to support Reason #3 as well.

Read the whole thing. If you want to see a great debate on the historicity of the resurrection of Jesus, you should watch William Lane Craig debate James Crossley.