It is challenging for any scientist, clinician, or stakeholder to keep abreast of the peer-reviewed published observations in their respective field. ALS is no different. It is even more challenging and critical to filter through all the available daily content from additional sources and focus on the pivotal advances. The breakthroughs at any point in the drug development value chain that could lead to the “Ah ha” moment and eventually a treatment for ALS.
Looking back, it’s not very surprising how little mental energy I put into the announcement of Genervon successfully completing a phase IIA trial of GM604 in ALS back in October of 2014. After all it was only 12 patients and 12 weeks of exposure to GM604. Typically that would make it just the first check box in the long and arduous process of testing if a drug is safe and provides any measureable benefit to ALS patients or any other disease.
I remember going to ClinicalTrials.gov and looking at the design and outcomes and a doing a quick pubmed search of GM604. The summary on clinicaltrials.gov confirmed the design and suggested it was really just a safety trial. With so few patients, the clinical outcomes and biomarker arms looked too underpowered. When I say that something is underpowered, I mean that there were not enough people enrolled in the trial to truly tell whether the drug is being helpful or harmful. As far as the PubMed search, a search for GM604 returned very little. A couple of publications of a five amino acid fragment of Motorneuron trophic factor (MNTF) having some effect in preclinical models of ischemia.
ALS TDI covered the phase II results and our interpretation of the results on October 14 of 2014 publishing our conclusion on our website, several days after the announcement and I left it at that.
The one paragraph I should have noted in the Genervon announcement dated October 19th 2014 was as follows:
“ALS Compassionate Use: An ALS patient was first diagnosed with the disease in Q1 2005 and by Q3 2008 was quadriplegic and on a ventilator. After a six-dose treatment of GM604 under an approved single patient compassionate-use IND, the patient’s swallowing volume increased in two weeks to 20cc from a baseline of 10cc. Five weeks after treatment, the patient consumed 240cc of water in 20-25cc bursts without leakage.”
Kind of like the Ice Bucket Challenge from the Summer of 2014, I don’t think anyone could have predicted what ensued. The ALS patient community became galvanized over the announcement, convinced that GM604 should have rapid approval by the FDA for all ALS patients. Everyone keyed in on the seemingly amazing results of the Phase IIA safety trial and perhaps even more so to the single patient compassionate use trial.
Everyone wanted access. And they wanted it now.
Genervon followed up in January with more detailed results of the compassionate use trial. Most doctors call observations from a single patient a clinical case study, but I guess Genervon preferred to call it a clinical trial. The tsunami of appeals from patients wanting access continued to escalate.
A petition was generated by an ALS patient Nicholas Grillo appealing to the FDA for accelerated approval of GM604 for all ALS patients based on the data from the 12 patients in the phase II safety study ALS.
Tens, no hundreds of thousands of people have signed the petition and most recently a peaceful rally at the FDA was held in Washington demanding that the FDA approve GM604 for accelerated approval.
Here I am today trying to understand why the community has rallied around GM604?
Why not NP001? It is a drug that went through a much larger clinical trial and showed efficacy!
Why not Tirasemtiv? A compound that went through several much larger trials in ALS and also showed some signs of efficacy…The data are certainly as good as those from GM604.
Why not Retigabine, Memantine, Rasagaline, TUDCA, NurOwn, or Ozanezumab? You get the picture.
Of course my scientific brain immediately took over trying to rationalize the situation and I asked the basic question…
Is there really any way to justify all of the attention being given to GM604? Given the heterogeneity of disease onset and progression in ALS, how could one measure a drug effect in only a 12 patient study over 12 weeks?
Is there data out there that could support or refute the claims of the phase IIA results in any way?
The trial reported outcomes for ALS-FRS, Forced Vital Capacity (FVC), and the levels of several protein markers in cerebrospinal fluid. The raw data from these outcomes have not been published. However Genervon did provide some high level statistical data on median change from baseline as well as bounds (minimum and maximum change) for the FVC. So I chose to focus some effort here.
There is a data repository of ALS clinical data that should be able to shed some light on this issue. It’s called the PRO-ACT Database and it was designed and assembled by the non-profit Prize4Life and the Neurological Clinical Research Institute (NCRI) at Massachusetts General Hospital.
Forced vital capacity or FVC is measured in the clinic with a spirometer. The device measures the amount of air a person can forcefully dispense from the lungs measured in liters. Often FVC from a subject is compared to a percentage of what a normal healthy subject of similar age and height would be.
In clinical trials the decline is often measured at set intervals and percent decline from trial initiation is calculated
The PRO-ACT database has clinical measurements from 8,600 subjects with ALS. The database contains 45,000 FVC measurements from 7,854 subjects. That’s a lot of data! The graph here shows all the measurements in the database and clearly demonstrates a correlation between days since diagnosis (x axis) and FVC measurements in liters (Y axis).
Here are the enrollment characteristics and endpoint assessments for FVC from the Genervon trial:
- 24 months or less since symptom onset
- FVC at baseline greater than 65%
- 18 years or older
- Endpoint for FVC
- FVC measured at baseline, 2, weeks, 6 weeks, and end of study 12 weeks
What I did here isn’t much different from what sports statisticians do when they use a computer to project outcomes of a game between two teams. They enter key attributes of the teams into the computer and ask the computer to predict who is most likely to win. Here, I entered key attributes of the patient population enrolled and asked about the predicted outcomes.
Specifically, to build this population group from the PRO-ACT database, I filtered out subjects who had presented with ALS symptoms for more than 24 months prior to enrollment in a trial and those who were less than 18 years of age and who had an FVC < 65% at baseline. Since this was a longitudinal study, I also filtered out subjects who had fewer than 3 FVC measurements including the baseline visit and those who had an FVC measurement out to at least 70 days to simulate a 12 week study.
The filtering reduced the number of “qualified subjects” to 777 subjects and reduced the total number of FVC measurements from 45,000 measurements to 5,439 measurements.
It was still an incredibly large sample set to build a model of FVC function and to compare the FVC measurements from the Genervon trial, especially when one considers that the entire Genervon phase IIA trial would only have 48 FVC measurements (12 subjects with 4 measurements).
Genervon reported the changes in FVC from the placebo groups and the group treated with GM604 as a percentage of change from the baseline measurement at time 0 to the endpoint time of 12 weeks.
I calculated a percent change in FVC of subjects from their baseline measurement at time 0 to the closest point to 84 days. No one included in the calculation had a duration shorter than 70 days and in some cases there were patients included with 120 days of data.
A plot of the distribution of the percent change in FVC from this cohort shows an almost normal distribution. A normal distribution forms an inverted “U”. The Y axis in the distribution plot shows a count of the number of subjects in that bin and the X axis is the percent change from baseline into bins of 2% changes. The numbers above each bin are the numbers of subjects in that bin.
The distribution is slightly skewed to the left. Not surprising since patients with ALS do have decreased respiratory function over time and would be reflected in a decrease in FVC. It is interesting to note though that 50% of the subjects change less than 5% in a positive or negative direction in their FVC from baseline over 12 weeks. So with a bunch of patients, you see a hint of worsening, but most patients don’t change very much in this time frame.
So what does all of this statistical jargon mean in terms of the Genervon’s phase II results?
Here are their conclusions from their press release:
Mean FVC -22 -5.6
STD FVC 15.6 7.7
Median FVC -25 -8.9
Minimum FVC -36 -14.9
Maximum FVC -3.7 5.8
pValue from T-TEST 0.0359
pValue from Wilcoxon Sum Rank 0.0726
If we focus on the median decline in FVC that they reported in their two cohorts, they claim a dramatic therapeutic benefit because FVC declines by a median of 22% in the placebo group and only declines by 5.5% in the GM604 treatment group.
If we now go back and compare these numbers to our much larger sample population from the PRO-ACT database, let’s see what we might conclude.
We can calculate some general statistics from these data and calculate some confidence intervals to our distribution that is calculated in this table.
- Maximum decline: -40.09%
- Maximum improvement: 61%
- Median change from baseline: -4.33%
- Mean change from baseline: -4.02%
- Standard deviation: 102
- Sample size: 778
- Sqrt sample size: 89
- Standard error: 007
- 95 confidence interval
- -4.33 +/- 0.007 (shown as redline)
Ultimately we can show that the median decline in FVC from this very large cohort of patients collected randomly from multiple trials over several years is a decline in FVC of only 4.33% over 12 weeks (Shown in red in the distribution curve).
Should we really believe that the placebo group from the Genervon trial, with its 22% decline over the same duration, can fairly represent the overall ALS patient population?
If we now color the median declines for the two cohorts in the Genervon trial onto our distribution you can see that the GM604 treated subjects are still several standard deviations below what one would predict could be possible in this study over this time (Shown in yellow in the distribution curve). More importantly the placebo group in this study are way out on the distribution many standard deviations away from the predicted value (Shown in magenta on the distribution curve).
How can this happen?
So I can only conclude several possibilities.
One, the FVC data could have been collected improperly. Highly unlikely.
Two, fast progressing patients were enrolled in both cohorts and enriched for really REALLY fast progressing patients in the placebo cohort. The data could be very skewed by a couple of fast progressing patients in each cohort.
Or could there have been an issue with the prescreen FVC of 65% of greater?
Regardless, it makes me wonder whether any real conclusions can be drawn from such a small trial. Based on the modeling demonstrated here from a very large patient cohort it does not appear plausible that the cohort of patients enrolled in the Genervon trial would represent a “normal” cohort of ALS patients. Therefor based on the model I continue to be very skeptical that there is any way that GM604 could have provided any therapeutic benefit in this small of a trial with this short of an exposure.
I am sympathetic to the needs of the ALS patient community to have effective treatments for this disease.
I agree that patients should have access to promising treatments as soon as possible. I am a passionate supporter of providing hope to ALS patients. Of turning over any stone and determining if it could provide any level of benefit to slow down disease progression in ALS. These are not just words. Anyone who knows me will validate its backed up by actions.
However in the early stages of drug development these types of “false positive” read outs are all too common and we have seen them many times in the ALS community. Think no further back than lithium, ceftriaxone, dexepramipexole, creatine and we could go on. The only logical path forward with the current data is that GM604 did not have an acute safety issue and warrants further testing in larger scale clinical trials to continue to monitor safety and with a well designed and adequately powered trial hopefully a positive outcome on disease progression and life expectancy.
The ALS community may not appreciate this opinion but each and every patient should consider that GM604 has not been adequately tested in patients as of today. There is still evidence to be collected on long term safety. There is still data to collected on if it actually makes disease progression worse and does indeed provide therapeutic benefit. Each of these is a risk for every patient who chooses to demand and ultimately take an experimental treatment. In addition a patient taking an experimental treatment like GM604 could make that patient be excluded for the next clinical trial of a potential treatment. A potential treatment that might actually slow down disease in ALS.