NYC Health Department Identifies 10 Outbreaks of Foodborne Illness Using Yelp Reviews Since 2012

A computer system developed by Columbia University with New York City Health Department epidemiologists detects foodborne illness and outbreaks in NYC restaurants based on keywords in Yelp reviews

Using Yelp, 311, and reports from health care providers, the Health Department has identified and investigated approximately 28,000 complaints of suspected foodborne illness overall since 2012

The NYC Health Department has used a computer system created by the Department of Computer Science to track foodborne illnesses via Yelp reviews.

The New York City Health Department today announced that since 2012, 10 outbreaks of foodborne illness were identified solely through a computer system jointly created with Columbia University’s Department of Computer Science. Launched in 2012, the computer system tracks foodborne illnesses based on certain keywords that appear in Yelp restaurant reviews. This strategy has helped Health Department staff identify approximately 1,500 complaints of foodborne illness in New York City each year, for a total of 8,523 since July 2012.

Improvements to the computer system are the subject of a joint study published this week by the Journal of the American Medical Informatics Association. The Health Department and Columbia continue to expand the system to include other social media sources, such as Twitter, which was added to the system in November 2016. The computer system allows the Health Department to investigate incidents and outbreaks that might otherwise go undetected. New Yorkers are encouraged to call 311 to report any suspected foodborne illness.

“Working with our partners at Columbia University, the Health Department continues to expand its foodborne illness surveillance capabilities,” said Health Commissioner Dr. Mary T. Bassett. “Today we not only look at complaints from 311, but we also monitor online sites and social media. I look forward to working with Columbia University on future efforts to build on this system. The Health Department follows up on all reports of foodborne illness – whether it is reported to 311 or Yelp.” 

Each year, thousands of New York City residents become sick from consuming foods or drinks that are contaminated with harmful bacteria, viruses or parasites. The most common sources of food poisoning include raw or undercooked meat, poultry, eggs, shellfish and unpasteurized milk. Fruits and vegetables may become contaminated if they are handled or processed in facilities that are not kept clean, if they come into contact with contaminated fertilizer, or if they are watered or washed with contaminated water. Contamination may also occur if food is incorrectly handled by an infected food worker or if it touches other contaminated food.

“Effective information extraction regarding foodborne illness from social media is of high importance—online restaurant review sites are popular and many people are more likely to discuss food poisoning incidents in such sites than on official government channels,” said Luis Gravano and Daniel Hsu, who are coauthors of the study and professors of Computer Science at Columbia Engineering. “Using machine learning has already had a significant impact on the detection of outbreaks of foodborne illnesses.”

“The collaboration with Columbia University to identify reports of food poisoning in social media is crucial to improve foodborne illness outbreak detection efforts in New York City,” said Health Department epidemiologists Vasudha Reddy and Katelynn Devinney, who are coauthors of the publication. “The incorporation of new data sources allows us to detect outbreaks that may not have been reported and for the earlier identification of outbreaks to prevent more New Yorkers from becoming sick.”

“I applaud DOHMH Commissioner Bassett for embracing the role that crowdsourcing technology can play in identifying outbreaks of foodborne illness. Public health must be forward-thinking in its approach to triaging both everyday and acute medical concerns,” said Brooklyn Borough President Eric Adams.

Most restaurant-associated outbreaks are identified through the Health Department’s complaint system, which includes 311, Yelp, and reports from health care providers. Since 2012, the Department has identified and investigated approximately 28,000 suspected complaints of foodborne illness overall. The Health Department reviews and investigates all complaints of suspected foodborne illness in New York City.

The symptoms, onset and length of illness depend on the type of microbe, and how much of it is swallowed. Symptoms usually include vomiting, diarrhea and stomach cramps. If you suspect you became sick after eating or drinking a contaminated item, call 311 or submit an online complaint form.

New Yorkers should call their doctor if they experience a high fever (over 101.5°F), blood in the stool, prolonged vomiting, dehydration, or diarrhea for more than three days.

The Centers for Disease Control and Prevention estimates that there are 48 million illnesses and 3000 deaths caused by the consumption of contaminated food in the United States each year.

About the Study

The study is titled “Discovering Foodborne Illness in Online Restaurant Reviews.”

Authors are: Thomas Effland, Anna Lawson, Luis Gravano, and Daniel Hsu (Department of Computer Science, Columbia Engineering); Sharon Balter, Katelynn Devinney, Vasudha Reddy, HaeNa Waechter (Bureau of Communicable Disease, New York City Department of Health and Mental Hygiene).

This work was supported by National Science Foundation grant IIS-15- 63785, Google Research Award “Information Extraction from Social Media: Detecting Disease Outbreaks,” Alfred P. Sloan Foundation grant G-2015- 95 14017, the Centers for Disease Control and Prevention PHEP grant NU90TP000546, and CDC/ELC grant NU50CK000407-03.

This material is based on work supported in part by a Google Research Award. In accordance with Columbia University reporting requirements, LG acknowledges ownership of Google stock as of the writing of this paper.

500 W. 120th St., Mudd 510, New York, NY 10027    212-854-2993