Economist Magazine Backs Secret Peer-Review Blacklists
SPOTLIGHT: A few weeks ago, The Economist ran an article titled Some science journals that claim to peer review papers do not do so. It not only missed the elephant in the room, it advocated unjust hiring practices.
BIG PICTURE: Peer review is a tool. Used by the scholarly publishing industry, it serves their needs. A couple of knowledgeable individuals are invited to critique a piece of research. Taking into account the feedback from those individuals, the journal then decides to publish the research or not.
In a report written two years ago, I referred to peer review as cargo cult science. Exhaustive attempts to prove it raises research quality have come up empty. On the other hand, its shortcomings (inconsistency, superficiality, subjectivity, inability to detect fraud, tribalism) are legion.
Over the years, academia has mythologized peer review. In that universe, it isn’t a mundane editorial tool, but a sacred process by which unpublished research gets elevated and anointed and transformed into ‘real’ work you can point to when applying for grants and promotions. If a series of pre-publication steps are followed, goes this cargo cult thinking, sound science is the end result.
One of the dirty little secrets of this mythology is that journals have always been at liberty to define peer review however they please. External oversight, enforcement, and minimum standards have never been part of the picture.
Recently, certain individuals and organizations have begun compiling blacklists of journals they suspect are only pretending to conduct peer review. Astonishingly, The Economist thinks this is a good idea. Even worse, it thinks universities should be “vigilant about checking candidates’ publication histories” against these blacklists when considering who to hire.
That would be profoundly unjust. A blacklist compiled in secret by anonymous people should not determine anyone’s job prospects.
There’s nothing open or transparent about these lists. We’re told one organization “employs 65 criteria to determine whether a journal” should be added to those “it believes are guilty” (my italics). However, a spokesperson is “reluctant to go into details.”
The rest of us have no way of evaluating the competence of these blacklisting efforts, or the fairness of their decision-making processes. Are journals given a chance to defend themselves? Who knows. The scope for errors and abuse is mind-boggling.
TOP TAKEAWAY: The Economist thinks people should be denied a job because journals that published their work have landed on a shadowy blacklist. Whether the job applicant has even heard of the blacklist, and whether the journals are guilty or innocent is apparently immaterial.
Read more at Big Pic News
Trackback from your site.
jerry krause
| #
Hi Donna,
Very good article but here are some additional thoughts you might consider. You omitted Galileo’s book which is considered to have gotten the ball rolling. But we know that Copernicus had some radical thoughts which he put down on paper which I have never seen any translation of all though I read a biography by David Brewster–The Martyrs of Science–in which he includes a direct translated quote of Galileo describing how Galileo himself though Copernicus’s ideas were nonsense and how Galileo himself moved more thoughtfully consider what Copernicus had written.
I consider you are wrong when you state the Newton’s–The Principia–was not peer-reviewed. For I read its publication was financed by the members of the Royal Society. For I considered when books were ‘rare’ the publishers of books judged whether the reading of an author justified the expense of the publication. Some authors, like Robert Boyle, were independently wealthy and could publish their own writings.
But I do not know when, the publication of books began so inexpensive and enough readers could afford these books that publishing became a profitable business so they invited potential authors to write books for them because of the acquired expertise of the author. Again, this is a form of peer review.
Just some thoughts.
Have a good day, Jerry
Reply
Xiaowei
| #
When did books become worth it? around the French Revolution.
George Greene (1793-1841) is one of many commoners of that era to publish something using his own money that nobody read or realized the importance of until much later. Principia was written a century before this and not intended for the masses. Keep in mind that natural philosophy in that era was not something you shared with others for their benefit but something you kept in hidden journals to prove that you came up with it first (think Gauss).
Reply
jerry krause
| #
Hi Xiaowei,
But Galileo’s Dialogues Conserning Two New Sciences was published in Italian and he regularly referred to the knowledge (experiences) of the artificers. And as I read Motte’s translation of The Principia I do not find Newton recognizing Galileo who clearly paved the way for Newton. And if you read the first two books of the Principlia is plain their content was not intended for the masses whose educational background certainly had not prepared the to possibly understand what Newton wrote; regardless of the language it was written.
Newton was writing for the intellectual community whether it be English, French, Italian, Greek, German, etc. Just as today most scientific articles are written in English regardless of the native language of the author.
When you write: “Keep in mind that natural philosophy in that era was not something you shared with others for their benefit.”, I have to assume ‘that era’ was that of Newton. So, how was it that almost immediately after he had finished Book iii, the it was published so that a good portion of the world’s intellectual community (natural philosophers) could benefit from it?
Have a good day, Jerry
Reply
jerry krause
| #
No, I do not purposefully omit words to challenge you and any reader. It’s one of my faults which I am not disciplined enough to eliminate. Sorry, Jerry
Reply