Here’s a fun modified version of the trolley problem: You are cruising along in the Trolley, as one does, and look ahead to see 2.6 million people on the tracks; the Trolley is set to roll over all of them.
Anyway, to step back remember the function of a bureaucracy is to maintain the bureaucracy. And the fact that it is populated by scientists and not clerks doesn't change this. The head of the organization is there because someone thought they would be good for that organization. This is not the same as achieving the goals you think that organization should.
Normally there will be some alignment with an orgs stated goals because it's authority is based on this perception. But in a crisis executive decision making is required which may put the org at risk, this is the conflict.
So the result is the political response of mitigation (mask, social distancing...) and the vaccine decision can be made by existing protocol. The rest is just power networking which your dept heads know how to do.
I would ditch the trolley intro or keep riding. The way you changed gears was kind of harsh.
I didn't realize I liked my own comment! I removed it, but I like the sheer egomania it implies - the "people need to know how great I think I am" of it.
I'm not sure I understand what you are saying here well enough to give it a fair response - is it basically "bureaucrats gonna bureaucrat, you can't really blame them for this"? Or something different?
I don't have a ton of time this morning but I'll try to say something that I hope makes sense:
The really short version of this is I think the people at the FDA had a clear option to save a lot of lives that out of mental sloth or self interest they didn't choose; I think they were able to choose it and just didn't. So regardless of them crying about their jobs or the unchangeability of bureaucracy, I really think this makes them something close to mass murderers.
You seem to be arguing that they are a machine and can't be blamed morally for anything they do even where it kills a ton of people. But if this is true, then we have to treat them that way and give them the effective inputs to get the change we want. And the inputs are the same - screaming about being murdered and how much we don't like it and how bad it is that we have a giant, mindless murder-machine that kills us for no reason besides self-preservation until we get some change.
I disagree with your premise that there's simply nothing they could have done - it would have been within their power to do the right thing, they either knew what the right thing was or should have, and careers shouldn't be an all-important terminal value when your career is having fiat control over whether people live or die. They had a good choice to make and didn't make it. But even if they didn't have a choice in the way you describe, the right thing to do here if you want them to stop threshing you like wheat in this kind of situation is to scream and cry and point out the mindless death-reaping as loud as you can until the operator takes notice and hits the override.
I'm old enough that the thalidomide babies were children at the same time as me. I vaguely recall attending school with one of them, and met another as an adult.
Obviously this is not a case of vaccines gone wrong - it was a drug given to pregnant women, prescribed for morning sickness among other things.
I also don't have to go far to find all kinds of (ahem) snake oil - products sold with carefully written non-claims of effects they don't actually have, plus the (true) claim that they can't just say explicitly what the product supposedly does, because of FDA rules.
That doesn't mean that the FDA was right in this case - and they do seem to have confused "we didn't personally develop/vet it" with "this shouldn't be used" during this epidemic (e.g. with regard to covid tests, early on).
And bureaucracy tends to grow beyond what's actually needed, as well as tending to impose additional costs. But I'm nonetheless grateful that the FDA was involved in approving treatments I'm receiving (not related to covid).
I also don't especially trust big pharma. The incentives for almost any executive insulate them from significant bad consequences if they make decisions that kill people; relatively recent PG&E behaviour comes to mind. (They saved so much money not doing maintenance that it was a great business decision, increasing the stock price - until there was a fire, and deaths, and lawsuits, and bankruptcy - none of which affected any profits the executives had already taken - and quite likely also didn't affect their future employment prospects.)
Of course the other thing about "challenge trials" is they presume that vaccines are binary - they either protect or don't. You presumably can't (ethically)/won't challenge unvaccinated controls, so you don't know how many unvaccinated people would (not) have caught covid with the same challenge procedure. If any vaccinated person winds up with covid - and I presume some will, even with 95% effectiveness - you're left guessing how many is too many, in judging the vaccine to be (in)effective.
It's the seen vs. the unseen problem. People see Thalidomide, but they don't see the millions of people who have died or lived much worse lives because the FDA didn't approve something in a timely manner.
When the FDA puts out a press release bragging that they approved a drug which will save a million people a year and you notice they took ten years to do it, that means they prevented ten million people from being saved by that drug.
The other thing we don't see are the drugs that get rejected with good reason - either by the FDA itself, or by potential manufacturers who decide "this will never get approved".
I should clarify that I'm not advocating we throw out all regulation entirely, or even switch to my "fantasy model" as a normal course of action. Of course some level of regulation is fine - we want that. But that's not the same thing as "Let's test everything exactly the same, even when it's clear in some cases that doing so will kill hundreds of thousands or millions of people. It's caution on one hand and insanity on the other; there has to be some middle ground between "We let people sell whatever snake oil they want" and "We always take a year to get out drugs - if there was a disease that would wipe out half of us in a year, we'd let it - after all, we always take a year.".
We absolutely could challenge with unvaccinated controls, and do so ethically (by most normal-people standards) if we had people who were willing to volunteer. We wouldn't consider it, so to save whatever amount of people die out of a hundred or a thousand young, strong people with top-notch monitoring and medical care, we made a choice that we knew would lead to several hundred thousand people dying. This isn't a sane form of ethics - this is us saying "A few of the strong strong want to voluntarily take a small risk - we'd prefer if many, many weak people involuntarily took a much larger risk".
The part I really want to emphasize is that's the choice we've decided is OK - that it's somehow the right moral choice to doom multitudes of people to choking to death in hospital beds with no choice whatsoever to protect a few people who know and want a risk and who, by and large, are much less likely to be injured in any way by the experiment. I think at the high point there were 25,000 or so volunteers of this kind in the US - we just ignored them and opted for maximal death counts.
Yeah, I'm being unfair by to an extent reacting to a straw supporter of challenge trials, such as individuals I've encountered online who were obviously coming from an _extreme_ libertarian perspective.
I don't think it was that unfair, or at least even if it was that it wasn't unfair in a productive way. It at least gets me on record saying "Hey, I'm not saying there should be no regulation at all" than have it open to being misread in that way - I'm not always the clearest in my writing, so it's useful.
We let people volunteer to climb Mt Everest or sail across the Atlantic solo, which is many times as risky as subjecting a healthy 25-year-old to a challenge trial of Covid, so intuitively, it's hard for me to see why letting people sign up for such a trial would be unethical, but letting them do those other risky things is okay.
You say that the current mRNA vaccines were both: 1, pretty much sure to work, and 2. had only a small chance of not working. What are you basing those assumptions on?
For 1, we've never had a mRNA vaccine used successfully on humans. Ever. In the history of the world. To claim that they were "pretty much sure to work" seems like hindsight bias.
For 2, here's a paper in Nature that shows that only about 20% of vaccine candidates make it through all phases of testing.
Now, you chose to ask only for failed vaccine launches, and of course there aren't many! That's what all of the phases are for, the ones that weed out the 4 out of every 5 vaccine candidates that just don't work! Again, thinking that we should have had more than a ~20% expectation of these vaccines working is hindsight bias.
Could the FDA have moved faster? Absolutely, and hopefully they will in the future with the information that we now know. But transposing our current understanding of mRNA vaccines onto the world as it was January 2020 is a recipe for bias.
Your first premise is untrue - we do have working mRNA vaccines that preceded Moderna's Covid vaccine; two passed basic safety/efficacy testing last year. And we've had the knowledge regarding how to make them for longer; they've been delayed because they are expensive to develop with very little upside.
For a little bit more background: the primary difficulty with mRNA treatments isn't the mechanism by which they work - it's that in a large enough dose they are toxic enough that the trade-offs don't work, especially used over time. But that's why the Moderna shifted to vaccines in the first place after Crigler-Najjar failed - they are small enough dose that they were a pretty sure bet. Note that they didn't expect to make any real money on those vaccines - they were a fallback to try to save face and market valuation when they couldn't get longer-term drugs to work.
That "vaccines don't make a lot of money" wrinkle plays into point 2 - that article is 20 years old. In and of itself that's not a huge problem, but that's right smack dab in the middle of an era where pharmaceutical companies were seriously backing of vaccines due to unprofitability (this is a fair summary - https://www.theatlantic.com/business/archive/2015/02/vaccines-are-profitable-so-what/385214/). In that situation you expect to see what that article shows - that drugs fail quite a bit in the preclinical portion of the process before any money is devoted to them.
But note that we aren't really talking about the preclinical portion in any meaningful sense - the story here starts with a complete, developed drug that the manufacturer has already committed fully to taking to phase 1; at that point, even judging by 80's and early 90's technology as shown in the paper we'd expect a 40% success rate. And that's before we factor in massive profitability - remember, the lack of successfully registered drugs in that period attributed by most to unprofitability, not any particular difficulty/danger.
But let's assume I'm wrong on both points. It's possible - I'm at best some internet jackass. What does that do to my argument?
In the first scenario, the FDA had a choice between a drug that was likely to work and be safe now, or a drug that was likely to work and be safe later; i.e. I was arguing the FDA believed the drugs were likely to be safe and likely to be work, and were grinding a very small area of uncertainty to protect small volunteer test groups at the expense of a couple million human lives. Essentially I was saying, hey, we might lose a couple people here in a worst case scenario - that's better than a million or two people choking to death in agony, which is what we got.
Let's say I was willing to sacrifice as many as a thousand people for this improved outcome. I think that's a reasonable "outrageously high" ceiling for this - I was talking about running it through a couple hundred of people at first to make sure they didn't just drop dead, and presumably we'd stop if they did. But let's say the risk is 1000 people for a drug we are basically sure will work.
Now let's run it through your scenario: We now drop from being near-100% the vaccine will work to 20%, which is the rock-bottom-vaccine-famine-era number before we consider profitability and what "preclinical" encompasses. In that worst case scenario, we'd expect to have to run five trials like this side-by-side. Now we are talking an overall risk of 5000 people, right? Something like that?
The thing is that the numbers are so very lopsided - the FDA's stance makes so little sense in any rational universe - that the choice is still between 5000 people maybe dying if everything goes perfectly, exquisitely and improbably wrong and a MILLION OR TWO PEOPLE DYING FOR SURE. That's the choice that's being made here - we knowingly let millions of people die, because that's how we do things. We don't do challenge trials - they are dangerous and a few people might get hurt - instead, let's let hundreds of thousands or millions of people die to avoid that.
I'm sorry for being strident on this; I feel very strongly about it, because I still haven't heard any great argument why it's a good idea to condemn the population of a small country to death instead of letting a handful of people take a small risk that's almost completely sure to save them.
ISTR from the In The Pipeline blog that about 50% of vaccine trials fail, and there have been several Covid vaccines that have failed. You have to do the trials to be sure they'll work and that they won't have some bad side-effect. But I think you're right that this could have been done with a set of available trial vaccines *way* earlier than actually happened, and doing this would likely have saved a lot of lives even if one of the trial vaccines had caused terrible side-effects and killed some of the volunteers who took it.
An important thing to remember here is that most of the time, the FDA is approving something that's not so urgent--yet another antidepressant that will maybe work a little better than the existing ones for some patients, a new statin that might be a good choice for folks who get bad side effects on the existing ones, etc. The super-cautious process might be pretty reasonable for those drugs, where the available benefit is really small. It's just not very reasonable for a once-every-few-decades global pandemic that's set to kill millions of people.
Great post. The trade off seems even worse for the case of the FDA and rapid testing. I don't think there is any risk of harm from a paper strip test. Just a few days ago the FDA announced that they're taking steps to "streamline" the path for screening tests. They say this over a year into the pandemic. We had a dozen of these tests developed and ready for production last May. How can they say "streamline" with a straight face?!
Unrelated to covid - In a recent seminar class on the epidemiology of cardiovascular disease, a guest lecturer discussed the role of the FDA in the development of CVD treatments. He talked about type 2 diabetes and the current rule that states that even after a drug is approved as safe and efficacious for treating T2DM and enters the market, the drug company must continue to do trials to assess risk of CVD outcomes. The speaker claimed that this was a great step because "pharmaceutical companies clearly wouldn't do this on their own." Aside from the fact that this sounds like an empirical question, I get the sense that many proponents of the FDA share this belief. Why do so many presume that in a world without an FDA there would be no clinical trials, or that every company would just start selling snake oil? What could be worse for the stock price of a pharma company than news that a drug harms people and that executives knew about it? Once Moderna and Pfizer developed the vaccine, why would they sell it to anyone if they weren't extremely confident that it wouldn't seriously harm anyone? Just as consumers demand safer cars, wouldn't consumers demand safer drugs? Why wouldn't drug companies have a third party conduct clinical trials and make public the results? Why would consumers want to take a drug that hasn't released results of clinical trials? And wouldn't competition and lower barriers to entry reinforce this? There are many other variables including the patent system and the fact that the patient isn't always the one who directly pays for the drug. Perhaps, I don't understand the full argument, but I've never understood why people believe that in a world without an FDA there would be no rigorous trials. Do we have evidence for this? Any thoughts?
There's a whole industry of nutritional supplements that claim various medicinal properties and are sold and used widely. Most have never been subjected to a clinical trial--it wouldn't pay to do so, since they're not patentable. It sure looks to me like a lot of that industry is selling snake-oil (perhaps literally in some cases).
Similarly, there's a whole industry of alternative medicine. Most of that stuff is pure fluff, and very little of it has been subjected to any kind of careful trial (for good reason!). Somewhere around the edges you get stuff that may do some good (maybe acupuncture and chiropractic help with some stuff?), but mostly it's just snake oil.
That's a fair point, and I would expand it to include the whole gamut of FDA approved drugs and devices that "lower risk." A great many such things are shown to lower the risk of disease or problem X, but efficacy is judged based on very sketchy statistical estimates, not something so reliable as "Does this person suffer from X and then get better if we give them the drug?"
That is to say, the entire shift into preventative medicine has resulted in lots of highly questionable products being sold as beneficial. Many are approved by the FDA.
That doesn't give me a lot of warm and fuzzy feelings regarding the FDA, however.
What I'd really like to see is this argument (or a response to this argument) (or really anything on this subject) from someone who works at the FDA, or a similar regulatory agency. Because then we could get more informed criticism, and a more detailed accounting of how FDA made the decisions it did and what sort of institutional incentives need to be tweaked to change how it operates in the future. I don't have a good understanding of that.
Also, it's not as simple as arguing that FDA should approve drugs faster. FDA has at times been much too quick to approve drugs with serious side effects or drugs that have no real benefit (Vioxx, Sarafem). I'd like to see them do better cost-benefit analysis, and keep the interests of patients at the center—i.e., our real interests, not what the medical ethicists say our interests are.
I think Vioxx at least is a bad example here - I think the overall evidence these days is that the harm related to Vioxx was an illusion. I'm also a bit confused - was Vioxx rushed through in some way I'm not aware of? There's always going to be occasional drugs which are harmful in some way that's not apparent until they are being prescribed at a population level - that's just the nature of the beast.
As for the FDA's side of things, I'd be open to hear it but it's going to be tricky for them to convince me. The starting point is that they protected a small group of willing risk-takers at the expense of millions of people worldwide - that's not really in dispute as far as I can tell. If they can't disprove that in some way, then all that's left is for them to convince me that 1,000,000 is a smaller number than 10,000,. I'm not sure they can do that.
The cost/benefit analysis with Vioxx was off in part because it turned out to be harmful, but also because it had no real benefit compared to existing, cheaper drugs. So there was no real urgency to approve it, no reason to rush the safety analysis. Of course FDA made the opposite mistake with COVID vaccines, when there was every reason to approve them quickly.
I'm not so much arguing that we should hear the FDA's side of things, but rather we should hear from someone who actually knows, at a detailed level, why they did what they did. Perhaps there'll be a book in a few years? Right now all the critiques I'm seeing are from outsiders judging the FDA by its results, and while I'm convinced by a lot of the critiques, I still don't grasp what actually happened.
RC, I like that you liked your own comment :).
Anyway, to step back remember the function of a bureaucracy is to maintain the bureaucracy. And the fact that it is populated by scientists and not clerks doesn't change this. The head of the organization is there because someone thought they would be good for that organization. This is not the same as achieving the goals you think that organization should.
Normally there will be some alignment with an orgs stated goals because it's authority is based on this perception. But in a crisis executive decision making is required which may put the org at risk, this is the conflict.
So the result is the political response of mitigation (mask, social distancing...) and the vaccine decision can be made by existing protocol. The rest is just power networking which your dept heads know how to do.
I would ditch the trolley intro or keep riding. The way you changed gears was kind of harsh.
I didn't realize I liked my own comment! I removed it, but I like the sheer egomania it implies - the "people need to know how great I think I am" of it.
I'm not sure I understand what you are saying here well enough to give it a fair response - is it basically "bureaucrats gonna bureaucrat, you can't really blame them for this"? Or something different?
Pretty much. I think you expect something impossible.
I don't have a ton of time this morning but I'll try to say something that I hope makes sense:
The really short version of this is I think the people at the FDA had a clear option to save a lot of lives that out of mental sloth or self interest they didn't choose; I think they were able to choose it and just didn't. So regardless of them crying about their jobs or the unchangeability of bureaucracy, I really think this makes them something close to mass murderers.
You seem to be arguing that they are a machine and can't be blamed morally for anything they do even where it kills a ton of people. But if this is true, then we have to treat them that way and give them the effective inputs to get the change we want. And the inputs are the same - screaming about being murdered and how much we don't like it and how bad it is that we have a giant, mindless murder-machine that kills us for no reason besides self-preservation until we get some change.
I disagree with your premise that there's simply nothing they could have done - it would have been within their power to do the right thing, they either knew what the right thing was or should have, and careers shouldn't be an all-important terminal value when your career is having fiat control over whether people live or die. They had a good choice to make and didn't make it. But even if they didn't have a choice in the way you describe, the right thing to do here if you want them to stop threshing you like wheat in this kind of situation is to scream and cry and point out the mindless death-reaping as loud as you can until the operator takes notice and hits the override.
I'm old enough that the thalidomide babies were children at the same time as me. I vaguely recall attending school with one of them, and met another as an adult.
Obviously this is not a case of vaccines gone wrong - it was a drug given to pregnant women, prescribed for morning sickness among other things.
I also don't have to go far to find all kinds of (ahem) snake oil - products sold with carefully written non-claims of effects they don't actually have, plus the (true) claim that they can't just say explicitly what the product supposedly does, because of FDA rules.
That doesn't mean that the FDA was right in this case - and they do seem to have confused "we didn't personally develop/vet it" with "this shouldn't be used" during this epidemic (e.g. with regard to covid tests, early on).
And bureaucracy tends to grow beyond what's actually needed, as well as tending to impose additional costs. But I'm nonetheless grateful that the FDA was involved in approving treatments I'm receiving (not related to covid).
I also don't especially trust big pharma. The incentives for almost any executive insulate them from significant bad consequences if they make decisions that kill people; relatively recent PG&E behaviour comes to mind. (They saved so much money not doing maintenance that it was a great business decision, increasing the stock price - until there was a fire, and deaths, and lawsuits, and bankruptcy - none of which affected any profits the executives had already taken - and quite likely also didn't affect their future employment prospects.)
Of course the other thing about "challenge trials" is they presume that vaccines are binary - they either protect or don't. You presumably can't (ethically)/won't challenge unvaccinated controls, so you don't know how many unvaccinated people would (not) have caught covid with the same challenge procedure. If any vaccinated person winds up with covid - and I presume some will, even with 95% effectiveness - you're left guessing how many is too many, in judging the vaccine to be (in)effective.
It's the seen vs. the unseen problem. People see Thalidomide, but they don't see the millions of people who have died or lived much worse lives because the FDA didn't approve something in a timely manner.
When the FDA puts out a press release bragging that they approved a drug which will save a million people a year and you notice they took ten years to do it, that means they prevented ten million people from being saved by that drug.
The other thing we don't see are the drugs that get rejected with good reason - either by the FDA itself, or by potential manufacturers who decide "this will never get approved".
Just saw this. Sorry for the late reply.
Whether or not a drug is approved by the FDA doesn't prevent victims and their families from suing a drug maker (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2697102/).
Meanwhile, Thalidomide's maker literally uses the FDA to prevent generic competition in the drug, thus keeping it's drug prices high (https://mlexmarketinsight.com/news-hub/editors-picks/area-of-expertise/antitrust/how-celgene-uses-safety-concerns-to-keep-its-$7-billion-cancer-drug-on-top) and thus continuing on the margin to help kill the people who would otherwise benefit from it.
I should clarify that I'm not advocating we throw out all regulation entirely, or even switch to my "fantasy model" as a normal course of action. Of course some level of regulation is fine - we want that. But that's not the same thing as "Let's test everything exactly the same, even when it's clear in some cases that doing so will kill hundreds of thousands or millions of people. It's caution on one hand and insanity on the other; there has to be some middle ground between "We let people sell whatever snake oil they want" and "We always take a year to get out drugs - if there was a disease that would wipe out half of us in a year, we'd let it - after all, we always take a year.".
We absolutely could challenge with unvaccinated controls, and do so ethically (by most normal-people standards) if we had people who were willing to volunteer. We wouldn't consider it, so to save whatever amount of people die out of a hundred or a thousand young, strong people with top-notch monitoring and medical care, we made a choice that we knew would lead to several hundred thousand people dying. This isn't a sane form of ethics - this is us saying "A few of the strong strong want to voluntarily take a small risk - we'd prefer if many, many weak people involuntarily took a much larger risk".
The part I really want to emphasize is that's the choice we've decided is OK - that it's somehow the right moral choice to doom multitudes of people to choking to death in hospital beds with no choice whatsoever to protect a few people who know and want a risk and who, by and large, are much less likely to be injured in any way by the experiment. I think at the high point there were 25,000 or so volunteers of this kind in the US - we just ignored them and opted for maximal death counts.
Yeah, I'm being unfair by to an extent reacting to a straw supporter of challenge trials, such as individuals I've encountered online who were obviously coming from an _extreme_ libertarian perspective.
I don't think it was that unfair, or at least even if it was that it wasn't unfair in a productive way. It at least gets me on record saying "Hey, I'm not saying there should be no regulation at all" than have it open to being misread in that way - I'm not always the clearest in my writing, so it's useful.
We let people volunteer to climb Mt Everest or sail across the Atlantic solo, which is many times as risky as subjecting a healthy 25-year-old to a challenge trial of Covid, so intuitively, it's hard for me to see why letting people sign up for such a trial would be unethical, but letting them do those other risky things is okay.
You say that the current mRNA vaccines were both: 1, pretty much sure to work, and 2. had only a small chance of not working. What are you basing those assumptions on?
For 1, we've never had a mRNA vaccine used successfully on humans. Ever. In the history of the world. To claim that they were "pretty much sure to work" seems like hindsight bias.
For 2, here's a paper in Nature that shows that only about 20% of vaccine candidates make it through all phases of testing.
https://www.nature.com/articles/nbt0596-591.pdf?origin=ppub
Now, you chose to ask only for failed vaccine launches, and of course there aren't many! That's what all of the phases are for, the ones that weed out the 4 out of every 5 vaccine candidates that just don't work! Again, thinking that we should have had more than a ~20% expectation of these vaccines working is hindsight bias.
Could the FDA have moved faster? Absolutely, and hopefully they will in the future with the information that we now know. But transposing our current understanding of mRNA vaccines onto the world as it was January 2020 is a recipe for bias.
Your first premise is untrue - we do have working mRNA vaccines that preceded Moderna's Covid vaccine; two passed basic safety/efficacy testing last year. And we've had the knowledge regarding how to make them for longer; they've been delayed because they are expensive to develop with very little upside.
For a little bit more background: the primary difficulty with mRNA treatments isn't the mechanism by which they work - it's that in a large enough dose they are toxic enough that the trade-offs don't work, especially used over time. But that's why the Moderna shifted to vaccines in the first place after Crigler-Najjar failed - they are small enough dose that they were a pretty sure bet. Note that they didn't expect to make any real money on those vaccines - they were a fallback to try to save face and market valuation when they couldn't get longer-term drugs to work.
That "vaccines don't make a lot of money" wrinkle plays into point 2 - that article is 20 years old. In and of itself that's not a huge problem, but that's right smack dab in the middle of an era where pharmaceutical companies were seriously backing of vaccines due to unprofitability (this is a fair summary - https://www.theatlantic.com/business/archive/2015/02/vaccines-are-profitable-so-what/385214/). In that situation you expect to see what that article shows - that drugs fail quite a bit in the preclinical portion of the process before any money is devoted to them.
But note that we aren't really talking about the preclinical portion in any meaningful sense - the story here starts with a complete, developed drug that the manufacturer has already committed fully to taking to phase 1; at that point, even judging by 80's and early 90's technology as shown in the paper we'd expect a 40% success rate. And that's before we factor in massive profitability - remember, the lack of successfully registered drugs in that period attributed by most to unprofitability, not any particular difficulty/danger.
But let's assume I'm wrong on both points. It's possible - I'm at best some internet jackass. What does that do to my argument?
In the first scenario, the FDA had a choice between a drug that was likely to work and be safe now, or a drug that was likely to work and be safe later; i.e. I was arguing the FDA believed the drugs were likely to be safe and likely to be work, and were grinding a very small area of uncertainty to protect small volunteer test groups at the expense of a couple million human lives. Essentially I was saying, hey, we might lose a couple people here in a worst case scenario - that's better than a million or two people choking to death in agony, which is what we got.
Let's say I was willing to sacrifice as many as a thousand people for this improved outcome. I think that's a reasonable "outrageously high" ceiling for this - I was talking about running it through a couple hundred of people at first to make sure they didn't just drop dead, and presumably we'd stop if they did. But let's say the risk is 1000 people for a drug we are basically sure will work.
Now let's run it through your scenario: We now drop from being near-100% the vaccine will work to 20%, which is the rock-bottom-vaccine-famine-era number before we consider profitability and what "preclinical" encompasses. In that worst case scenario, we'd expect to have to run five trials like this side-by-side. Now we are talking an overall risk of 5000 people, right? Something like that?
The thing is that the numbers are so very lopsided - the FDA's stance makes so little sense in any rational universe - that the choice is still between 5000 people maybe dying if everything goes perfectly, exquisitely and improbably wrong and a MILLION OR TWO PEOPLE DYING FOR SURE. That's the choice that's being made here - we knowingly let millions of people die, because that's how we do things. We don't do challenge trials - they are dangerous and a few people might get hurt - instead, let's let hundreds of thousands or millions of people die to avoid that.
I'm sorry for being strident on this; I feel very strongly about it, because I still haven't heard any great argument why it's a good idea to condemn the population of a small country to death instead of letting a handful of people take a small risk that's almost completely sure to save them.
ISTR from the In The Pipeline blog that about 50% of vaccine trials fail, and there have been several Covid vaccines that have failed. You have to do the trials to be sure they'll work and that they won't have some bad side-effect. But I think you're right that this could have been done with a set of available trial vaccines *way* earlier than actually happened, and doing this would likely have saved a lot of lives even if one of the trial vaccines had caused terrible side-effects and killed some of the volunteers who took it.
An important thing to remember here is that most of the time, the FDA is approving something that's not so urgent--yet another antidepressant that will maybe work a little better than the existing ones for some patients, a new statin that might be a good choice for folks who get bad side effects on the existing ones, etc. The super-cautious process might be pretty reasonable for those drugs, where the available benefit is really small. It's just not very reasonable for a once-every-few-decades global pandemic that's set to kill millions of people.
Great post. The trade off seems even worse for the case of the FDA and rapid testing. I don't think there is any risk of harm from a paper strip test. Just a few days ago the FDA announced that they're taking steps to "streamline" the path for screening tests. They say this over a year into the pandemic. We had a dozen of these tests developed and ready for production last May. How can they say "streamline" with a straight face?!
Unrelated to covid - In a recent seminar class on the epidemiology of cardiovascular disease, a guest lecturer discussed the role of the FDA in the development of CVD treatments. He talked about type 2 diabetes and the current rule that states that even after a drug is approved as safe and efficacious for treating T2DM and enters the market, the drug company must continue to do trials to assess risk of CVD outcomes. The speaker claimed that this was a great step because "pharmaceutical companies clearly wouldn't do this on their own." Aside from the fact that this sounds like an empirical question, I get the sense that many proponents of the FDA share this belief. Why do so many presume that in a world without an FDA there would be no clinical trials, or that every company would just start selling snake oil? What could be worse for the stock price of a pharma company than news that a drug harms people and that executives knew about it? Once Moderna and Pfizer developed the vaccine, why would they sell it to anyone if they weren't extremely confident that it wouldn't seriously harm anyone? Just as consumers demand safer cars, wouldn't consumers demand safer drugs? Why wouldn't drug companies have a third party conduct clinical trials and make public the results? Why would consumers want to take a drug that hasn't released results of clinical trials? And wouldn't competition and lower barriers to entry reinforce this? There are many other variables including the patent system and the fact that the patient isn't always the one who directly pays for the drug. Perhaps, I don't understand the full argument, but I've never understood why people believe that in a world without an FDA there would be no rigorous trials. Do we have evidence for this? Any thoughts?
There's a whole industry of nutritional supplements that claim various medicinal properties and are sold and used widely. Most have never been subjected to a clinical trial--it wouldn't pay to do so, since they're not patentable. It sure looks to me like a lot of that industry is selling snake-oil (perhaps literally in some cases).
Similarly, there's a whole industry of alternative medicine. Most of that stuff is pure fluff, and very little of it has been subjected to any kind of careful trial (for good reason!). Somewhere around the edges you get stuff that may do some good (maybe acupuncture and chiropractic help with some stuff?), but mostly it's just snake oil.
That's a fair point, and I would expand it to include the whole gamut of FDA approved drugs and devices that "lower risk." A great many such things are shown to lower the risk of disease or problem X, but efficacy is judged based on very sketchy statistical estimates, not something so reliable as "Does this person suffer from X and then get better if we give them the drug?"
That is to say, the entire shift into preventative medicine has resulted in lots of highly questionable products being sold as beneficial. Many are approved by the FDA.
That doesn't give me a lot of warm and fuzzy feelings regarding the FDA, however.
What I'd really like to see is this argument (or a response to this argument) (or really anything on this subject) from someone who works at the FDA, or a similar regulatory agency. Because then we could get more informed criticism, and a more detailed accounting of how FDA made the decisions it did and what sort of institutional incentives need to be tweaked to change how it operates in the future. I don't have a good understanding of that.
Also, it's not as simple as arguing that FDA should approve drugs faster. FDA has at times been much too quick to approve drugs with serious side effects or drugs that have no real benefit (Vioxx, Sarafem). I'd like to see them do better cost-benefit analysis, and keep the interests of patients at the center—i.e., our real interests, not what the medical ethicists say our interests are.
I think Vioxx at least is a bad example here - I think the overall evidence these days is that the harm related to Vioxx was an illusion. I'm also a bit confused - was Vioxx rushed through in some way I'm not aware of? There's always going to be occasional drugs which are harmful in some way that's not apparent until they are being prescribed at a population level - that's just the nature of the beast.
As for the FDA's side of things, I'd be open to hear it but it's going to be tricky for them to convince me. The starting point is that they protected a small group of willing risk-takers at the expense of millions of people worldwide - that's not really in dispute as far as I can tell. If they can't disprove that in some way, then all that's left is for them to convince me that 1,000,000 is a smaller number than 10,000,. I'm not sure they can do that.
The cost/benefit analysis with Vioxx was off in part because it turned out to be harmful, but also because it had no real benefit compared to existing, cheaper drugs. So there was no real urgency to approve it, no reason to rush the safety analysis. Of course FDA made the opposite mistake with COVID vaccines, when there was every reason to approve them quickly.
I'm not so much arguing that we should hear the FDA's side of things, but rather we should hear from someone who actually knows, at a detailed level, why they did what they did. Perhaps there'll be a book in a few years? Right now all the critiques I'm seeing are from outsiders judging the FDA by its results, and while I'm convinced by a lot of the critiques, I still don't grasp what actually happened.