Seasonal influenza epidemics cause tens of millions of respiratory illnesses and nearly half a million fatalities worldwide per year, including about 35,000 in the US alone. And, as last winter’s near-miss with the H1N1 virus reminded us, new strains of influenza virus have the potential to create a pandemic which could cause a million deaths or more.
If public health officials can detect influenza outbreaks early and respond aggressively, the impact of both seasonal and pandemic influenza can be reduced significantly. Early detection is difficult however since even countries with the advanced health systems rely on old-school detection systems. Traditional detection systems involve manual reports by doctors of cases they have seen, time-consuming lab tests and archaic communication links to public health agencies. What is more, these methods cannot detect influenza in people who haven’t visited their doctor.
Thus it was with great anticipation 18 months ago that Google released Flu Trends, a free tool from the company’s philanthropic division that purported to detect regional influenza outbreaks 7-10 days faster than old-school methods.
Flu Trends relies on the fact that many people enter phrases like “do I have the flu?” into search engines long before calling their physicians. Working with the Centers for Disease Control, Google created a basket of search phrases that suggest influenza and then aggregated all the hits by location. Terms like thermometer, muscle aches, and flu symptoms made the list.
The buzz became deafening after a subsequent article in Nature suggested Google Flu Trends was a highly sensitive tool for detecting regional influenza outbreaks.
Alas, a new study presented on Monday at the American Thoracic Society meeting in New Orleans suggests that Google Flu Trends has a specificity problem. According to Justin Ortiz and colleagues at the University of Washington, Flu Trends was about 72% accurate in predicting confirmed cases of influenza between 2003-2008, as compared with an 85% accuracy rate for the CDC’s gold-standard (though slow) flu surveillance network.
“We knew from the Google Flu Trends validation study that it is highly correlated with surveillance for the non-specific syndrome of influenza-like illness,” Ortiz said in a press release. “However, it has never been evaluated against a gold standard of actual laboratory tests positive for influenza virus infection. When we compared Google Flu Trends data to CDC’s national surveillance for influenza laboratory tests positive, we found that Google Flu Trends was 25% less accurate at estimating rates of laboratory confirmed influenza virus infection.”
The problem with Flu Trends is that the influenza virus causes only 20-70% of influenza-like illnesses, even during flu season. Many other less deadly respiratory viruses produce flu-like illnesses.
Among the 6 flu seasons studied by Ortiz’s group, Google Flu Trends performed most poorly during the 2003-04 influenza season, which was characterized by early, intense influenza activity, many childhood deaths and a requisite amount of media attention.
“Internet search behavior is likely different during anomalous seasons like 2003-4,” Ortiz explained. “During periods of intense media interest or unexpected influenza activity such as the 2009 H1N1 influenza pandemic, Google Flu Trends may be least accurate at estimating influenza activity.”
Still, most people believe Google Flu Trends remains an excellent public health tool. Its results are provided instantly and at low cost, for example. It also can sometimes provide better location-specific information than the traditional CDC method. In addition, it can capture information from people who do not to see a doctor when they develop flu-like symptoms.
Each flu surveillance tool “tells a slightly different story,” Matt Mohebbi, Google Flu Trends’ lead engineer told the Wall Street Journal Health Blog. “It’s important to look at them [together] in order to get a real sense of the situation.”
Glenn Laffel, MD, PhD
Sr. VP Clinical Affairs, Practice Fusion
