There's another way to look at small studies.
When I was younger I had a temporary interest in ESP (say, reading hidden cards), from the skeptic's point of view. One of the things I learned was that, when you have a small study, you typically have a threshold of statistical significance of maybe 5%. That means that, if you were to run a study 20 times, in one of those you'd get a statistically significant result supporting your hypothesis EVEN WHEN the underlying facts/mechanism were contrary to your hypothesis.
So the way the ESP guys do it, they just keep trying, maybe with different volunteers, doesn't matter. Roughly one time in 20 they'll get a "statistically significant" result, and THAT'S what they report to the uncritical press. Just don't mention the other 19 tries.
Now for PC drugs, maybe they re-jigger the formulation, or the dosage, or whatever. I'm sure it's all done in good faith. I'm sure they're convinced that THIS time, they've got it right and they really have achieved a cure.
But maybe it's just one in 20.
Larry ShickPersonal homepage incl. PCa story: www.sv-moira.com
(age 60) biopsy PSA 4.4, free PSA 9%, T2c stage, Gleason 7 (3+4), 7 of 14 cores; 6'2", 200 lbs.03/09: Robotic surgery
(Dr. Kawachi, City of Hope) 47 gms, 10% involved, staging/Gleason unchanged (pT2cNXMX), margins clear, no ECE/sem ves involvement, fully continent from day 1, some success w/Viagra 50mg/day.Followup: 05/09 0.006
Post Edited (CapnLarry) : 6/24/2009 6:23:38 PM (GMT-6)