The strain on scientific publishing \ stacker news ~science

pull down to refresh

The strain on scientific publishing direct.mit.edu/qss/article/5/4/823/124269/The-strain-on-scientific-publishing

2619 sats \ 17 comments \ @south_korea_ln 17 Jan science

Scientists are increasingly overwhelmed by the volume of articles being published. The total number of articles indexed in Scopus and Web of Science has grown exponentially in recent years; in 2022 the article total was ∼47% higher than in 2016, which has outpaced the limited growth—if any—in the number of practicing scientists. Thus, publication workload per scientist has increased dramatically. We define this problem as “the strain on scientific publishing.” To analyze this strain, we present five data-driven metrics showing publisher growth, processing times, and citation behaviors. We draw these data from web scrapes, and from publishers through their websites or upon request. Specific groups have disproportionately grown in their articles published per year, contributing to this strain. Some publishers enabled this growth by hosting “special issues” with reduced turnaround times. Given pressures on researchers to “publish or perish” to compete for funding, this strain was likely amplified by these offers to publish more articles. We also observed widespread year-over-year inflation of journal impact factors coinciding with this strain, which risks confusing quality signals. Such exponential growth cannot be sustained. The metrics we define here should enable this evolving conversation to reach actionable solutions to address the strain on scientific publishing.

Here, an interesting paper suggests possible practical ways to tackle it. I recommend reading the introduction too, as it really sets the problem quite well.

Here some suggestions from the paper:

First, we can make it easy to track scientific progress and reduce overpublishing by moving to open-ended and stackable publications instead of publishing multiple papers for each research direction. For example, instead of ten papers published on one line of research, a scientist can prepare a single study where each piece (‘chapter’) can be stacked with or inserted into the previous piece. A similar approach is implemented on Github where codes can be updated and expanded; or on Jupyter where the data, analysis and text can be published on a single page (with more chapters being added as the study develops further). Importantly, Jupyter notebooks are free and do not charge for open access as most publishers do, pointing towards a possible solution for reduced publishing fees. Existing examples include a Jupyter paper published by Ziatdinov et al.8 and an interactive book Deep Learning for Molecules & Materials created by Andrew White9. The book has a relevant repository on Github and incorporates contributions and feedback from community members.

As well as (@Rothbardian_fanatic, this one relates to your observation regarding the weakness of the peer-review process, as opposed to the scientific method in itself):

Second, we should move to community-based reviewing. For each open-ended study, the authors may choose the referees for the initial review and criticism. However, the work will also remain open to community members for constructive criticism and feedback. Changes can be implemented by authors directly in the published study, with the referees (community members) confirming that the issues have been resolved. Importantly, clearly visible professional feedback from the community will make it easy for the readers to evaluate the study and learn about open questions. Some initial steps in this direction have already been made by eLife10. Furthermore, as such open-ended studies can be published without the initial peer review, the careers of authors will not depend on the reviewing time, journal selection, or preparation of rebuttals. Recently, the European Molecular Biology Organization (EMBO) has adopted new criteria for postdoc evaluation that discourage the accumulation of publications, whereby refereed preprints will be viewed as publications in the assessment process11. This signifies an important paradigm shift in publishing that is likely to become widespread in the future.

As well as

Third, community-based reviewing will help properly document and recognize such activities. A recent study suggests that researchers spend more than 130 million hours reviewing papers each year, with a monetary value of US$1.5 billion in 2020 in the United States alone12. This represents a substantial contribution to science that remains overlooked13. Online open-ended publishing systems can track reviewing activities and provide relevant information (or metrics) for contribution assessment (ORCiD has already implemented a similar system).

I wonder if this could relate to payment of referees' time, see #805976.

Two more suggestions:

Fourth, a specific contribution from each author can become clearly visible when needed. Online open-ended publishing will allow authors to precisely assign different parts of the study as contributions from specific people.

and

Fifth, advanced machine learning algorithms will be able to compile ‘scientific reviews’ for any question asked by the user. This is likely to eliminate papers and reviews as we know them now. Writing and perfecting the text of the manuscript will no longer be needed. Computer algorithms will be able to do it for us, delivering easy-to-understand stacks of information with assigned ‘trustworthy/reproducibility scores’ and providing a clear time-stamped overview of the progress within any field of science.

For this last one, I need to process it a bit more. The writer of the piece is into AI if I'm not mistaken, so he might be a bit biased on that part.

view all related items

164 sats \ 3 replies \ @SimpleStacker 17 Jan

I pretty much agree with all these suggestions except #5. I don't trust AI to offer proper summaries, and I don't think the writing is wasteful at all. Oftentimes, as I try to explain my ideas in writing, I realize holes in my ideas that I didn't notice before.

The thing is, we have the tools necessary already for productive scientific collaboration, like github. Moving to a more open source model would be super fruitful, I think. Competition should be between projects, and hypotheses, not between people and labs.

The reason I don't see it happening (in my field at least) is that it is asking the people at the top to give up a lot of power, as well as exposing themselves to a lot of risk. When their work moves to a more open source model, they won't be able to manage their reputations or the narratives about their work as easily. Why subject yourself to that risk when you are already at the top of your profession?

30 sats \ 0 replies \ @south_korea_ln OP 17 Jan

Why subject yourself to that risk when you are already at the top of your profession?

Yes, case in point, I got to know about this paper through a fellow academic who left academia a few years back. He could not reach the top. He hence has a clearer view of what is wrong and how it disadvantaged him in securing a permanent position in academia. Yet, he's likely a better scientist, both in coding and physics, than many people that I know at the top.

We're trying to move to a more open-source approach in the lab I am in now. The problem that I see in our case is that academics are notoriously messy and lack proper documentation of code. Also, time. Why would one bother doing this extra work when so much is already asked outside of actual research?

The whole system needs to change. There are fundamental flaws that make it extremely hard to change things, even incrementally.

0 sats \ 1 reply \ @000w2 17 Jan

The incentives won't change until the funding model changes.

0 sats \ 0 replies \ @south_korea_ln OP 17 Jan

Some funding agencies, especially in Europe, are pushing people to move away from the opaque and closed-source editing houses.

I am applying for a job there, and I can really feel how in Korea, my number of publications matters most, while in Europe, they try to assess many other metrics.

Of course, I'm not naive to think that my papers with Springer journals won't carry a higher weight than the ones in other open-source journals in the final grading. But at least, there is a clear intent from the EU to change things, for the better.

106 sats \ 4 replies \ @Rothbardian_fanatic 17 Jan

Community based reviews are a good idea as long as you can get a diverse (in opinion and method) population of reviewers, not just paradigm followers. That is the danger of community review, more groupthink. I still think just putting out the details of the experiment and let people interested in it try to replicate the results or falsify them.

On the fifth point, I am not trusting AI at all. Not one bit. It is nothing but a statistical correlator of words. One word that statistically follows another is not intelligence and as the scientists like to quote quite often; correlation does not equal causation. I will still trust people doing the work and correlations more than the machines that have the biases due to their training.

20 sats \ 3 replies \ @south_korea_ln OP 17 Jan

On the first point, some kind of non-binding review by fellow experts might still be useful to guide the reader in what is considered correct or not, from a methodological point of view. But indeed, the risk of groupthink is always lurking.

I agree with your assessment of the 5th point. AI really hasn't convinced me yet for me blindly trust it.

123 sats \ 2 replies \ @Rothbardian_fanatic 17 Jan

The real problem comes in when you follow the money. If the non-binding reviewers are paid shills for someone else, their reviews are just as possibly bogus as a groupthink review. I think that the only valid review would be re-running the experiment under the same conditions with the same methods to see the results and then report those results. AI has a very, very long way to go before I could consider trusting it. I do not like the bias in training, either.

35 sats \ 1 reply \ @south_korea_ln OP 17 Jan

I think that the only valid review would be re-running the experiment under the same conditions with the same methods to see the results and then report those results.

That's indeed the only valid review. That's how science is ultimately self-correcting. In theory.

The problem is that this process can take a lot of time. Some of the experiments in my field require such specialized and expensive equipment, that only a few groups in the world have the expertise to carry them out. Same with the theory and simulations, it requires sometimes very specific math and codes, that again, only few people have the skills to repeat the previous results.

And all of this comes back to the discussion of financial or reputation incentives. Why would I spend time repeating work that has already been done?

0 sats \ 0 replies \ @Rothbardian_fanatic 17 Jan

You would spend time repeating work that has already been done to validate or falsify it. As you say, it is the only valid review. How many times have we seen falsified data, bogus results and other scientific hijinks in the peer-reviewed but not replicated published papers? I realize that sometimes you cannot replicate because of expense or lack of equipment and talent, but what else can you do?

10 sats \ 7 replies \ @cryotosensei 17 Jan

I came here to express similar sentiments stated by @Rothbardian_fanatic. What qualifies one to be a community reviewer, and would he or she have the requisite expertise to critique papers? And would a can of worms be opened as relationships come into play? How can we ensure the impartiality of the reviewers involved?

10 sats \ 3 replies \ @south_korea_ln OP 17 Jan

What qualifies one to be a community reviewer, and would he or she have the requisite expertise to critique papers?

That brings us back to the positive side of peer review. In theory, this is what makes sure that the reviewer is qualified, in one way or another. I only get to referee papers where the editor knows I'm capable of carrying out the specific equations or simulations used in the paper. And these days, many journals show the referee reports along with the papers.

Making this process more open could lead to some kind of review by the community where the opaqueness and secrecy of the current review system gets addressed. At least with it being open, one can better see if relationships (guanxi) have corrupted the process...

0 sats \ 2 replies \ @Rothbardian_fanatic 17 Jan

I think that removing some of the money from the process may also provide less problems than we have now..

0 sats \ 1 reply \ @south_korea_ln OP 18 Jan

Fully agree that the way openaccess journals have basically "pay-to-publish" journals has corrupted a big part of the field. That's why, sadly, the "reader-pays-to-read" model is still the more reliable than the "pay-to-publish" model. All of this is because of the middle man (the editor) that is profiting all the way, without adding much value to the equation. My research should be open for everyone to read, as my research was paid with tax money, so the taxpayer should have free access to it.

All of this is valid within the current paradigm. But for reasons mentioned in other messages, the current paradigm needs a drastic overhaul to actually solve the issues.

0 sats \ 0 replies \ @Rothbardian_fanatic 18 Jan

Your observation on who owns your research is one that is denied in most cases. If a company pays for research it is their property to dispose of as they like, including publication or non-publication. If a taxpayer funded research project is owned by the taxpayers, then there should only be a publish option and no other. The state could put a small fee for reading, just to keep the trolls out, but not large enough to keep the taxpayers out. Trolls seem to go for the free stuff the most. BTW, that is also why I like stacker news, not many trolls.

0 sats \ 2 replies \ @Rothbardian_fanatic 17 Jan

I also have another caveat, this only works for physical science, not sciences that involve humans. Sciences like psychology, sociology praxeology, and others will not work with he same scientific method because humans learn and the variables cannot be controlled. Trying to work the “scientific method” from the physical world in the humanities world, looks to me like scientism, not science.

10 sats \ 1 reply \ @south_korea_ln OP 18 Jan

Right, most of what I wrote was based on my experience as a physicist where the human aspect is minimized. I'm not sure how much of that translates to the world of humanities where I don't have any personal experience. Indeed, the problem of confounding variables, etc are much harder to assess and control. That's why I am also always much more skeptical on conclusions made on human behavior. Remember the reproducibility crisis in the field of psychology...

0 sats \ 0 replies \ @Rothbardian_fanatic 18 Jan

The problem in the human studies areas with reproducibility are legend. The whole point there is that they are applying the wrong methods to study their topics. They should be using praxeology rather than other methods. When they use it en economics the results are amazingly reproducible! Some economists do not like to admit that because they are applying physical science methods to economics and getting results that are all fantasyland.