John Lantos - How many babies will die because of false positive genomic analysis?

Mixed event-introduction and discussion fragments

the Director of Pediatric Ethics at Mercy Hospital in Kansas City, and today, John, I don't know what you're talking about. Genomics. Genomics, okay, and what happens when genomics leads to a false positive result. Exactly. Thank you very much. Thank you.

Dr. John Lantos

And thanks, it's great to be back and always humbling to hear so many great talks. It's an astounding record of success here at the McLean Center. Those of you who live here may think that this kind of stuff happens everywhere else. It doesn't. It's an amazing place.

Dr. John Lantos

My talk will really be focusing on this. We have a solution, but now we're trying to figure out the problem. How many babies will die because of false positive genomic tests? I have no conflicts, but we'll be discussing off-label use of things that the FDA calls devices, but I'll explain that in a little more detail soon. So in 2010, my hospital in Kansas City, Children's Mercy, started a center for pediatric genomic medicine. And the next year, the NHGRI had a call for proposals to look at ethical issues in newborn genomics. Like all good ethicists, if grant money was available, I became very interested in the topic.

Dr. John Lantos

We had a director with an audacious idea, Stephen Kingsmore, whose idea was to sequence genes in 20 hours, whole genomes, provide interpretations to doctors within 48 hours for sick babies in the NICU, and try to show that such results would actually improve the care in real time of sick babies in the NICU. At the time, whole genomes usually took about three months to go through this process. The problem was that sequencing was getting easier. In fact, there were machines that could churn out whole genomes in 12 or 18 hours if you had enough money to buy them. But interpretation is pretty hard. Leslie Biesecker wrote in a paper about opportunities and challenges. You get the result but it's overwhelming because the variants you see range from those that are extremely likely to cause disease to those that are nearly certain to be benign and everything in between. So how to speed up interpretation. We did What most places have started to do, which was to develop our own proprietary software or informatic devices or interpretive pipelines. These things have lots of names. And we, like everybody else, gave them cute acronyms. Ours had sort of a

Dr. John Lantos

Viking theme. S-Saga, Runes, and Viking. None are validated or FDA approved because all are constantly changing and being updated with every new interpretive discovery or report. So how does it work? What does S-Saga do? It's software by which clinicians can describe the signs or symptoms that the baby has and then the software matches those symptoms with previously reported candidate genes. So clinicians would go to a software interface that looked like this. You'd enter a new patient. You'd get drop-down menus for cardiovascular, endo, ID, constitutional, metabolic, etc. You'd put in the signs and symptoms that the baby had, and the software would spit out a list of genes that had been previously associated with that. then the whole genome would be sequenced but only those genes that Saga said were likely to be associated with the baby's symptoms would be analyzed and this shortened the time necessary for analysis and also importantly reduced the likelihood of incidental findings like BRCA or genes associated with Alzheimer's because although we had the sequence, we wouldn't be looking at it. It would, so to speak, end up on the cutting room floor

Dr. John Lantos

Then to figure out whether the variants identified were pathogenic, we had another software program, RUNES, Rapid Understanding of Nucleotide Variant Effect software. Good, huh? Took these variants and assigned them to existing American College of Medical Genetics categories for pathogenicity. It did this by running each of the variants through dozens of existing databases which are used to categorize things according to this

Dr. John Lantos

framework that's probably familiar to many of you, category one, previously reported to cause the disorder, novel or expected to cause the disorder, novel may or may not be causative or sometimes called a variant of unknown significance, novel but unlikely to cause the disease and the docs would get complicated reports like this where at some point on the report it put it into that category. This is a category one likely to cause the disease. And then Viking would do variant integration and knowledge interpretation which integrates Saga and Runes, prioritizes variants and generates a report for the doctor in understandable lay language about whether this process led to what was called a molecular diagnosis, a genetic finding likely to explain the baby's symptoms. So taken together, these pipelines suggest whether genetic variants are likely to be associated with the patient's symptoms. Does this without the need for much human thought or interpretation? That is, once the doc filled out the S-SAGA list of signs and symptoms, the rest was pretty much automated. And it led to lots of molecular diagnoses. Not just at our place, but at most places, people using similarly

Dr. John Lantos

off-label and proprietary interpretive pipelines that are all a little different report high rates of molecular diagnoses in a variety of different populations. Of note, all these populations are what I would call highly enriched. They're all populations of people thought to have some sort of genetic disease. But these are four recent papers. Somebody said 40%, somebody said 57%, that was us, 50, 58%. And that, I would say, is very cool science. Wouldn't you agree? Yeah. Yeah. In fact, Stephen Kingsmore got into the Guinness Book of World Records for the fastest interpretation of the human genome. How cool is that? Lots of people thought this was the way of the future. This guy started the Precision Medicine Initiative. Francis Collins, the director of the NIH, said DNA sequencing will lead to a medical landscape in which each baby's genome is sequenced and that information is used to shape a lifetime of personalized strategies for disease prevention, detection, and treatment. So what could go wrong? I

Dr. John Lantos

think the problem is with this term, molecular diagnosis, because the question is, what does it mean to find a gene that's associated with a pathologic finding, especially because association, as we know, is not causation. Yang, one of the first papers looking at this, published in JAMA just about three years ago, defined molecular diagnosis. A whole exome sequencing case was classified as molecularly diagnosed if the pathogenic, so the result is in the definition of pathogenic or likely pathogenic variants, were detected in Mendelian disease genes that overlapped with described phenotypes of the patient. So a gene that had been previously reported, even if it's a novel variation that overlaps, counts as a molecular diagnosis. But do these molecular diagnoses mean anything? Do they lead to beneficial changes in treatment? It turns out that there are very few reports of beneficial changes in treatment and most of those reports are not in peer-reviewed medical literature. In fact, they appear in places like Time magazine. Genetic screening saved this baby's life. Time magazine reported this was actually a case from our place. Six month old, had abnormal development, was in the PICU on a vent, got molecular diagnosis that according to Time led to beneficial treatment. The variant was associated with a defect in citrate transport. He was started on citrate and reported by Time to be doing better.

Dr. John Lantos

There was no follow-up in a peer-reviewed journal. If this is true, this would be a huge breakthrough and certainly worth reporting, but it's now three years later and there's no peer-reviewed report. So that was part of what this NHGRI proposal was meant to stimulate, to get people to start doing more rigorous studies, and we designed a study to try to figure out whether these tests were actually beneficial. whether they were harmful or whether they were just irrelevant. Here's what we did. We had neonates who the doctors thought might have a genetic syndrome. We randomized them either to current standard of care, whatever that was, we didn't define it, or current standard of care plus a whole genome. So some of the babies got the whole genome, some of them didn't. And the main study question was, if you get a molecular diagnosis, Do the doctors and the parents perceive it as helpful, harmful, or acute but irrelevant? Here were the preliminary results. It was a level four NICU. We only enrolled babies less than four months of age. We randomized 65 patients. About half got the standard tests, half got rapid genomes. Note that there were five crossovers because even at the beginning, doctors had some trouble with equipoise. And if babies were randomized to standard care and the doctors really, really thought they needed a whole genome, they could cross over to the whole genome. And what they found was, what we found with whole genome, there were more molecular diagnoses, 41 versus 15%. And they were made quicker, 16 days versus 65 days. This is sort of a stupid result to report, that is. If you're doing a test that's being done faster, the fact that it gets done faster is not a surprising finding.

Dr. John Lantos

And in 57%, we got a diagnosis. The doctors felt that the diagnosis was clinically actionable in 37%, but here's what I want you to focus on. Of those 37%, about half of them, the action was what we called palliative care guidance, which essentially was a recommendation for palliative care. And in the published report in Lancet, genetic diagnosis that confers a dire prognosis can empower early discussions with the infant's parents about palliative care to minimize suffering. And palliative care was given to 29% of infants with a genetic diagnosis. And I add this, made up about half of the cases that were reported to be clinically actionable, 17 out of 37%.

Dr. John Lantos

So let's look at one case just to see what I see as the problem with this sort of thing. This was Children's Mercy Hospital case 545, a baby with bilateral chylus effusions, VSD, and unusual faces. He was enrolled in the study and found to have a gene associated with Noonan syndrome. For those who like this genomic language, there are the details, and was reported as having a molecular diagnosis of Noonan's. The diagnosis was made at 69 days of age, palliative care was initiated, and he died at 88 days of age. So what is Noonan syndrome? Features are short stature, congenital heart defects, developmental delay. A lot of babies have congenital heart disease and some mild intellectual disability. Not in every case, but in some cases. heart disease is treatable and usually the developmental disabilities are treated with early intervention and individualized education strategies. There's many genes associated with Noonans, although again these are associations and causation is difficult to prove, but there are no population studies to show how often these genes occur in asymptomatic or people without clinical features of Noonans. So it's hard to know whether these genetic findings are true positives or false positives. So it seems there's many genetic variants. Each may or may not be diagnostic. The disease itself is not fatal. The molecular diagnosis may be a false positive, but even if it's a true positive, it doesn't justify withdrawal of life support. So some key questions here. Did they really withdraw life support based on the molecular diagnosis of Noonan's? If so, I'd say that's really bad clinical care. If not, then it's very bad science to report it as a clinically actionable finding. And the way it was reported, we just don't know which is which. And this is a common feature of every report you read or that I've read about molecular diagnoses. There are exaggerated claims of clinical utility, no scientific standards in making these claims, little documentation of follow-up or outcomes. And it seems that peer-reviewed journals lower their standards in their awe and wonder over molecular diagnoses. Or it may be that the papers are sent to other experts. That would be other genomicists who have an interest in showing that the work that they're doing is valuable.

Dr. John Lantos

One more cautionary tale, because this is one example where, unlike most tests that come back, we actually know what the prevalence of genomic variance is in the general population. Crab A disease is a neurodegenerative disease. Onset at infancy usually leads to death before age two. And according to the Mayo Clinic website and many other sources, affects about 1 in 100,000 live births. Remember that 1 in 100,000 number.

Dr. John Lantos

So New York State and many other states, including now Illinois, started a statewide screening program in 2006. It was a very rigorous program. They first measured the enzyme activity, which was the traditional test, and then they added, for specimens with low enzyme, genomic sequencing of 17 genes associated with Crab A. And infants with low enzyme and a confirmatory molecular diagnosis were then sent for clinical confirmatory testing, which included a detailed history physical exam by a neurologist, repeat of the enzyme activity and the genotype, MRIs, LPs, and nerve conduction studies. So this was an extremely conservative approach to diagnosis. What were their results? This was as of about a year ago. They'd tested two million babies. 99.9% were negative. But 620 had low enzyme activity. That's one in about 3,000 babies. Of those, about half had a known or potentially pathogenic mutation in the gene. That's a molecular diagnosis. That's one in about 6,000 babies.

Dr. John Lantos

And those were further evaluated, about two thirds were thought to be at low risk, but about a third were, or no risk, about a third were low, moderate, or high risk. If you put those together, that's one in 14,000, so about six times what was thought to be the population prevalence before this was done.

Dr. John Lantos

And of the 14 highest risk babies, who were sent for confirmatory exams, only five had an exam consistent with Crab A, four underwent a stem cell transplant, which is the only known treatment, two survived with severe developmental delays, and two died within three months after the transplant. But most importantly here, nine, or about 70%, remain asymptomatic and follow up is now between one and nine years. So 70% of these tests were false positive, and we found that the population prevalence of low enzyme and confirmatory molecular diagnosis is much, much higher than we thought, although most not associated with the development of the disease. So given these numbers, what would happen if somebody did a test for the gene of a sick baby in the NICU and found they had the genomic variant associated with Crab A? How many parents would be counseled to withdraw life support? How many would agree to a bone marrow transplant? And how many of the tests are false positive? In Crab A, we now know 70%. For all the other genomic findings we're getting, we have no idea.

Dr. John Lantos

Quick real-world example, even though my time's up, as part of this study, we also did a bunch of interviews with parents. We talked to one parent who had a set of twins. Both were normal until 11 months. Then one started losing developmental skills and at 13 months had seizures. And after an eight-month diagnostic, Odyssey was diagnosed with Krabbe at nearly two years of age. Both twins are still alive at 10 years of age, and the twin with Krabbe seems to have leveled out developmentally. That is, still impaired but not getting worse. And the parents said nothing they read or heard from any doctor, any website, or any support group was accurate as to what the clinical course would be. And the natural history for the kid with Krabbe was very similar to the natural history of kids who get bone marrow transplants. And there's a claim that the bone marrow transplant was successful. It's illustrated by this study that was published in Neurology Just a few months ago, developmental outcomes after cord blood transplantation for Crab A's, cognitive development below the third percentile and gross motor skills not developing at all. What seems like claims of success mirror the natural history of untreated disease.

Expert audience discussant on newborn screening

So what are future

Dr. John Lantos

needs here? I think we need standardization of the pipelines for interpreting variants, more careful reports of clinical utility, tougher peer review, and all in the service of learning more about population prevalence of genomic variants, penetrance, expressivity, natural history of disease or non-disease in the full range of patients with particular molecular findings. In spite of this, our doctors love whole genome sequencing. They're no longer in equipoise. In fact, we stopped the study because they're unwilling to enroll people to get them randomized. And instead, it's a complicated slide that basically shows the total enrolled in the study has dropped to zero, in part because we're now offering this test clinically, so you don't have to enroll in a study to get it. So the key question may be, where are we in the hype cycle? It seems that we're now here at the peak of inflated expectations. I think we're headed to here because we'll soon be able to measure just how many babies die because of inaccurate tests or inappropriate decisions based on accurate but misleading tests. Thanks.

Dr. John Lantos

That's Children's Mercy. It looks a little like Disneyland. Isn't that nice?

Audience member 2

John, this is great work. And I think what you're doing is really pointing out what happens with new technologies in general. I don't think that genetics is, although it's offering a lot, I think new technologies in general are approached like this. I think that applies to many. So, I mean, obviously there are lots of conflicts of interest of people who are involved with this. I mean, one is just that's their field and they want it to be good. There's also the idea that these tests are sold for money and that's a profit center in hospitals. How do you see us making this a better process based on what you've found?

Dr. John Lantos

I mean, it seems only by careful study and the integrity of researchers to accurately report their results and report them with more rigor. As you say, it's not a unique story to genomics.

Dr. John Lantos

Adam Sifu here has the best book on medical reversals, which would be situations where, you know, we've gone from the peak of inflated expectations, you know, everything from pulmonary artery catheters to antiarrhythmics to postmenopausal hormone therapy, all of which have similar things, but that eventually through careful and courageous research are eventually shown either to be useless or the hope would be after the expose you do find sort of the situations where it's most useful and then tailor the treatment or the test for those situations.

Ben Brown

So, thank you. I really appreciated your talk. Tell

Dr. John Lantos

everybody who

Ben Brown

you are. Say that again? Tell everybody who you are. Oh, sure. I'm Ben Brown. I'm one of the current fellows. I'm an OB-GYN and a family planning fellow here. I agree with you wholeheartedly about the rush to adopt new technology. I was just having a conversation earlier about robotics in surgery. We could say that about many, many new technologies, and I take your point very much. In particular, the correlation is not causation aspect of it. I want to push back a little bit on the primary ethical argument you make, or at least the one that you highlight in your title, which is the concern about death, and are we having reasonable deaths, unwarranted deaths for pediatric patients. And I actually want to, I don't know if she would agree with me, but I'm gonna borrow a little bit from Dr. Ross's framework of pediatric ethics and say,

Ben Brown

while the certain knowledge of uncertainty is a valid part of decision making, And so I think that perhaps our biggest problem is the way that physicians are understanding these tests, the way that they are incorporating them into the clinical picture, and the way in which they are counseling, not the fact that this is actually being incorporated potentially into the counseling, and in fact that someone might choose to withdraw life support as a result of it, as long as it is understood in the proper context of its uncertainty, that that to me strikes me as an appropriate respect for the constrained authority of the parent in deciding what is going to be the ongoing course for that child's care, in particular because my assumption based on the cases that you've described here is that these tests are being done for patients who do have obvious and significant clinical findings. What are your thoughts on that?

Dr. John Lantos

I mean, it sounds a little like the guns don't kill, people kill argument, but, you know, it's not the test, which is, of course, morally neutral. It's how the test is used. But if what we learn from doing studies is that it's used in ways like this, yeah, there would be two solutions. One would be use it better. And that could be either figuring out where it shouldn't be used at all or, as you suggest, figure out how to better convey both to the doctors and the parents the uncertainties associated with it. So that may be an answer to Dan's question. I mean, to the extent that we could do that, we would avoid many of these harms. It's difficult to do, though. Thank you.

Audience member 1

Thank you, John. Excellent as usual. I'd have a comment and a question. The comment is, as was mentioned earlier, this is something that is not new about genetics. Jerome Lejeune, 50 years ago, when he described in trisomy 21 causing Down syndrome predicted just this kind of thing. And I would tell you those of us who work with... And he was right. And he was right. And those of us who work with disabilities, we're still in the trough of disillusionment. There's no upward slope of enlightenment. Individuals with Down syndrome now live a full life, and yet we're still screening for it and spending tons of money over it. And so the question I have for you is, if it's moved to a clinical, who's paying for that? Are the citizens of Missouri paying for that if they're on Medicaid?

Dr. John Lantos

Yeah.

Audience member 1

That's a moral outrage. When we can't pay for other things, and we're paying for this, which is causing more death, that's terrible.

Dr. John Lantos

Right. I guess the question is, if

Dr. John Lantos

you have reported, I mean, multiple reports in peer-reviewed journals showing that it's useful, many of the reports show that it's not just making diagnoses, but they're clinically actionable. if the doctors who are caring for the babies believe that it's useful and want to order the tests and the parents want the test as well, where would you locate the moral outrage?

Dr. John Lantos

But if everybody wants it and we live in a democracy.

Audience member 1

Yeah.

Dr. John Lantos

Yeah.

Dr. John Lantos

but I'm not sure how to harness my moral outrage.

Expert audience discussant on newborn screening

John, it's even more complex because we're... I only had 20 minutes. Something like crab bays was put in newborn screening panels under the assumption that we would find these children and convert a lethal disease, which was a rapidly progressive dysmyelination with no responsiveness in the first six months of life to a kid who walked and talked. And what they did is change it to a slowly progressive complex disorder. At the same time, to parents who had the condition, that was much better than being minimally responsive. Again,

Dr. John Lantos

to Peter's point, motivated by parents who had a kid with Krabbe who lobbied legislators to get it done.

Expert audience discussant on newborn screening

And thereby had hope. But the individuals who had those genetic variants, which outnumbered by more than tenfold, they were told, You can never know when you will become the patient in waiting for the rapidly progressive type

Dr. John Lantos

and

Expert audience discussant on newborn screening

their children.

Dr. John Lantos

And I know that you and I were talking about this yesterday. Mike is a developmental pediatrician who in Illinois has to evaluate all the kids to do that detailed evaluation. neurologic evaluation so sees all the false positives whose parents are coming back. Let me just tell you one story though that relates to all these points and then I know we have to stop. I gave this talk at Vanderbilt or a longer but more complex version and at the end of it to your point that it's substituting another disease and these kids are not going to get up and walk. One of the people who asked a question was a pediatric neurologist and he said, now the problem is we're just not transplanting them soon enough. If we transplanted them sooner, it would be completely curative. And I said, well then you're going to have to transplant a lot of kids who never would have developed disease. And he said, no we're not.

Mixed event-introduction and discussion fragments

Actually, the data would suggest you have to be transplanting them in utero because many of them are born with the symptoms.