HackerTubewatch
Surprise me
Frame from Is Most Published Research Wrong? Plays via YouTube's privacy-enhanced embed

Is Most Published Research Wrong?

92 videos recommended on HN
YouTube
8 HN users recommended this explainer8 months2017–2025 score 10
Years with an HN recommendation, 2007–2026

HN commenters link it whenever a shaky study or the p<0.05 threshold comes up, as a short primer on false positives in science.

Veritasium explains why a large share of published research findings may be false, covering p-values, p-hacking, publication bias and low replication rates. It draws on John Ioannidis's well-known paper and describes reforms such as preregistration.

“This video is a good primer” (HN, 2018)

4 strong recommendations · 4 recommendations · 11 citations · Explainers

19 Hacker News comments

From the HN archive; up to 20 kept per video, favoring different commenters. Recommendations first.

@sillysaurusx Oct 2020 Strongly recommends
on HN thread Actively speaking two languages protects against cognitive impairment
Cue “p = 0.05 is a random choice” and a big thread about p-hacking. Any takers? It feels like everything to say has already been said about the bias of small sample sizes and the dangers of p-hacking. But it also seems like the discussion is inevitable here. I wonder if there’s a new insight to glean, somehow... For a background on why p-values are misleading, see [this video] which is excellent. He points out that 0.05 was selected by Ronald Fischer in 1925, and people have gone with it since.
Background on why p-values mislead and origin of 0.05.
@scotty79 Aug 2021 Strongly recommends
on HN thread The Coronavirus Is Here Forever
Please watch this video. It neatly shows why should you treat small and less rigorous studies with suspicion, even if there are many giving some results: [this video]
Why to distrust small, less rigorous studies.
@KK7NIL Dec 2025 Strongly recommends
on HN thread I don't think Lindley's paradox supports p-circling
Wikipedia is infamously bad at teaching math. This Veritasium video does a great job at explaining how such skewed priors can easily appear in our current academic system and the paradox in general: [this video]
Clearly explains skewed priors and the paradox.
@burntrelish1273 Apr 2017 Recommends
on HN thread Receding glacier causes Canadian river to vanish in four days
Definitely. Another good idea is to show the difference between reality and data can be overwhelmingly significant. Videos demonstrating this with examples: - "The Bayesian Trap" [link] - "Is Most Research Wrong?" [this video]
Shows reality and data can differ significantly.
@ReligiousFlames May 2017 Recommends
on HN thread Science to Retract Study on Fish and Microplastics
There's an interesting video by Veratasium "Is Most Published Research Wrong?" (Clickbaity title to be sure.) It shows, on the overall statistics of valid/invalid vs. published/unpublished papers, the arbitrary p-value threshold is wholy inadequate to prevent a sizable fraction of published papers from being false positives (invalid but published). [this video]
Shows p-value threshold lets many false positives be published.
@wolfram74 Jul 2017 Recommends
on HN thread The US government is removing scientific data from the Internet
It's not 'these days' it's the intrinsic nature of false hypotheses outnumbering true hypotheses by an enormous ratio. Even if everything were working perfectly you'd expect false results to outnumber true results. It's like the common example of disease screening, where even if you only get a false positive 1% of the time, but there are only 1% of true positives in your sample population, you're going get many more inaccurate results than accurate results. This video walks through an example calculation and what kinds of steps are being taken to improve on things. [this video]
Walks through example calculation and improvement steps.
@hannasanarion Nov 2018 Recommends
on HN thread Dead sperm whale found in Indonesia had ingested '6kg of plastic'
Read more
My dude. Please go get yourself some education on how science works. I feel embarrassed for you. The reproducibility crisis is about the prevalence of false positive results in the scientific literature. This video is a good primer: [this video] A false positive is when a paper says "this works" when it actually doesn't, because of a statistical fluke in their sample. Even if citing the reproducibility crisis as an excuse to believe something that science has repeatedly found to be false isn't a really bad reason for belief (which it definitely is)... it doesn't even apply because there can't be false positive results regarding neurolinguistic programming because there aren't any positive results regarding neurolinguistic programming. Even the paper put out by neurolinguistic programming practicioners themselves wasn't able to produce a detectable effect over a placebo. I'm sorry, my friend, but this is an intervention. You're in a cult. You're making the exact same arguments as homeopaths and faith healers and fortune tellers.
Primer on false positives in the literature.

Citations, criticism and jokes

@DiThi Jul 2017 Cites it
on HN thread High refined sugar intake linked to a 23% higher risk of mental disorders
Read more
The vast majority of the studies mentioned in those videos are old, where most researchers could get away with hacking the P values [0]. A lot of studies done in the last decade contradict the older ones (the article I linked earlier is full of sources). Also correlation is not causation. In how many of those studies controlled the amount of carbs? Because the combination is very important [1] [0] [this video] [1] [link] Edit: Looking at the last link of your first video, it acknowledges low carb diets are beneficial short and medium term, but there's very little literature for long term. It concludes low carbs are bad long term... but you must see the table on page 13 to see how they reached that conclusion. The "low carb score" group still had a crazily high 37-42% of carbs (compared to LCHF diets which are usually 5% carbs), it had more smokers and all of them had trans fats (very dangerous but can be easily avoided altogether).
@trapperkeeper74 Oct 2017 Cites it
on HN thread “Peer review” is younger than we think
There are many, many BS papers because of the mass-production death-march researchers are told to produce, low p-value and sample size standards and ”Is Most Published Reseach Wrong?” (Veritasium, 2016) [this video] Which is about an essay from 2005: [link] This doesn’t mean adversarial and critical feedback are bad, it means dis/incentives, politics and standards need adjustments.
Cited on bad papers and low p-value standards.
@sillysaurusx Dec 2020 Cites it
on HN thread 'I am mine': Alzheimer's at 41 (2018)
Yeah, sure. Just don't eat, and that'll prevent alzheimer's. If you study history, you have to reconcile these views with the fact that we've been wrong about every medical development until recent history. At one point we were killing cats to curtail the black plague, when in fact it was the fleas that were the cause. We just don't know. And it's fine not to know. [this video]
@dharmab Nov 2023 Cites it
on HN thread Ask HN: Where do you get your health news?
Most published research is wrong: [this video] I usually check the UK NHS website for general information that has been reproduced consistently, and talk to a doctor for anything more specific. This about how poorly technology and law news is reported. Health news is _even worse_.
Claims most published research is wrong.
@d0mine Jan 2024 Cites it
on HN thread Lancet journal retracts article on hearing aids and dementia after prodding
> most science withstands the scrutiny If "most" means "most published papers" and "scrutiny" means "can replicate" then no, it doesn't withstand scrutiny: most published research is wrong. [this video] (2016) (the exact percentages of non-replicatable articles may be different in different fields)
Cited for claim that most published research fails replication.
@WhitneyLand Jan 2024 Cites it
on HN thread The problem is not plagiarism, but cargo cult science
Possibly you and the OP would be better served to say “ I wouldn’t be surprised if some claim about x%” is true. Or, “I would guess it’s true that”. Back to the main subject Vertasium has a related video: Is Most Published Research Wrong [this video]
Related Veritasium video.
@iLoveOncall Mar 2024 Cites it
on HN thread Too many labs run like this
It's really starting to get hard to believe ANYTHING that comes out in a research paper this days. 7 years ago Veritasium published a video showing that most published research is wrong. News since then has shown that it's not getting better. [this video]
Showed most published research is wrong.
@authorfly Aug 2024 Cites it
on HN thread GPT-4 LLM simulates people well enough to replicate social science experiments
Read more
Fair enough, you might have indeed rejected those authors - however, vast swathes, for Baumeister the majority, did not at the time. It's almost certainly true now for existing authors we are yet to identify, or maybe never will. I admit the point on TESS, I didn't research that enough. I'll look into that at a later point as I have an interest in learning more. To address your studies regarding expert / study forecasting - thank you for sharing some papers. I had time and knew some papers in the area so I have formulated a response because, as you allude to later regarding cultural predictions, there is debate in the question of the usefulness of expert vs non-expert forecasts (and e.g. there is a wide base of research on recession/war predictions showing the error rate is essentially random at a certain number of years out). I have not fully comprehended the first paper but I understand the gift of it. Economics bridges sociology and the harder science of mathematics, and I do think it makes sense for it to be more predictable than psychology studies by experts(and note the studies being predicted were not survey-response like most are in psychology), but even this one paper does not particularly support your point. Critically, one conclusion in the paper you cite is that "Forecasters with higher vertical, horizontal, or contextual expertise do not make more accurate forecasts.", "If forecasts are used just to rank treatments, non-experts, including even an easy-to-recruit
@r_singh Aug 2025 Cites it
on HN thread New research reveals longevity gains slowing, life expectancy of 100 unlikely
Read more
I can understand the sarcasm and respect your take Science has helped reduce mortality at birth and manage or even eliminate a lot of diseases like malaria, HIV, small pox, polio and many more. Also for burns, broken bones and accidents. And continues to. I advocate vaccines. I myself took the Anti Covid shot 3 times. And use medicines and supplements regularly. There’s another side to it though. At what point do I just take an Adderall that’s prescribed to me? Or a pain killer? Rather than trying preventive lifestyle cures? We need a way to tell when to apply what but IMHO in practical life what gains ground is what serves the providers more rather than the beneficiaries. An interesting example is that Anaesthesia was a more quickly adopted invention than Anti-Septic. Even though the latter is more important for the safety of patients. But the former makes life easier for docs, hence it was adopted much faster. There is a place for both and knowing when to apply which is key. You cannot trust HCPs using allopathy aka science backed by academia in every case cause they’re motivated by self interest and aren’t perfect! Even people like Gabriel Weinberg have acknowledged this in his book Super Thinking when touching upon inefficiencies in healthcare and academia. I believe that to be able to trust science more we need to save science from p-value manipulation by self-interest groups… This video by Veritasium also talks about how the accuracy of most published research is contes

Video list and Hacker News comment labels from HackerTube (prompted by @karpathy), released under ODC-By 1.0, built on the open-index/hacker-news archive. Hacker News data is provided by Y Combinator; comments remain the work of their authors. Video titles, channel names and thumbnails belong to their creators and are shown via YouTube. HackerTube Watch is an unofficial edition. Data export of 2026-10-31.

Keyboard shortcuts

/
Search
j k
Next / previous video
r
Surprise me
l
Like (video page)
w
Watch later (video page)
?
This help
Esc
Close