How Search Engine Data Enhance the Understanding of Determinants of Suicide in India and Inform Prevention: Observational Study

Natalia Adler, Ciro Cattuto, Kyriaki Kalimeri, Daniela Paolotti, Michele Tizzoni, Stefaan Verhulst, Elad Yom-Tov, Andrew YoungJournal of Medical Internet Research 2018 · OpenReview

Abstract

India accounts for approximately 20% of global suicide deaths, yet official statistics are often hampered by social stigma and historical legal barriers. This observational study leverages private-sector search engine data from Bing to gain a fuller, more accurate picture of the suicide crisis in India. By combining official demographic statistics with search query data, the researchers aimed to determine if digital footprints can serve as a proxy for hidden factors contributing to suicide and inform future prevention efforts.

Introduction: The Data Gap in Public Health

The scale of the suicide crisis in India is immense, yet the data available to public health officials is often incomplete. Until recently, suicide was a criminal offense in India, and the deep-rooted social stigma surrounding mental health continues to lead to significant underreporting. Traditional datasets, such as those from the National Crimes Record Bureau (NCRB), often lack the real-time granularity needed for effective intervention. This study explores the potential of "data collaboratives"—partnerships between the private sector and public health agencies—to fill these gaps using search engine query data.

Methods: Modeling Digital Footprints

The researchers analyzed English-language queries from Bing in India over a four-month period (November 2016 – February 2017). Queries were categorized into five primary topics: suicide, depression, hanging, pesticide, and poison. These digital data were then integrated with state-level demographic information, including urbanization rates, population growth, sex ratios, and internet penetration levels.

The team employed a stepwise linear modeling approach to predict suicide rates per state. A key component of their methodology was a novel iterative outlier rejection process. This allowed the researchers to identify states where the relationship between search behavior and reported suicide rates deviated significantly from national trends, highlighting potential areas of extreme underreporting or localized crises.

Results and Key Insights

The study revealed that search behavior is a powerful, albeit complex, indicator of suicide risk. Initial models using only query data showed a correlation ($R^2$) of approximately 0.5. However, the predictive power increased dramatically when specific "method-based" queries were prioritized.

The "Method" is the Message

Interestingly, queries about specific suicide methods—such as pesticides, poison, or hanging—were often better predictors of actual suicide rates than general searches for "depression" or "suicide." This suggests that search data captures individuals who may be further along the path of suicidal ideation and planning. When outliers were removed, the model for pesticide-related queries saw its $R^2$ jump from 0.16 to 0.71, eventually outperforming models based solely on demographic data.

Temporal Peaks in Search Volume

Analysis of the search logs uncovered distinct temporal patterns in how users look for suicide-related information. Search volume follows a clear diurnal cycle, peaking significantly during the evening and late-night hours. There is also a weekly variation, with certain days showing higher engagement with these topics.

Diurnal and weekly patterns of suicide-related search queries compared to a baseline.

These temporal insights have immediate practical applications. For instance, crisis helplines and online support services could optimize their staffing levels to match these peak periods of vulnerability.

Discussion: Regional Variance and Outliers

The use of outlier rejection served as a diagnostic tool rather than just a data-cleaning step. The researchers identified a "South India" cluster with high reported suicide rates, which correlated with higher urbanization. Conversely, states like Jammu & Kashmir were identified as negative outliers, showing high search volume relative to low reported deaths, which strongly suggests significant underreporting in those regions. In contrast, states like Telangana appeared as positive outliers, where reported rates exceeded search-based predictions, likely due to specific local socio-economic crises such as those affecting rural farmers.

Limitations and Future Directions

While promising, the study notes several limitations. The reliance on English-language queries likely excludes a large portion of the population that uses regional languages. Furthermore, the "digital divide" means that the most vulnerable populations, such as rural farmers without internet access, are not captured in this data. Future research should focus on integrating multi-lingual data and exploring how these models can trigger real-time public health alerts or localized awareness campaigns.

Conclusion

This work demonstrates that search engine data can serve as a vital proxy for unmeasured factors corresponding to suicide rates. By identifying regional discrepancies and temporal peaks, data collaboratives can provide public health officials with the tools needed to design more targeted and timely suicide prevention strategies in India and beyond.

Summary updated

Abstract

Background: India is home to 20% of the world’s suicide deaths. Although statistics regarding suicide in India are distressingly high, data and cultural issues likely contribute to a widespread underreporting of the problem. Social stigma and only recent decriminalization of suicide are among the factors hampering official agencies’ collection and reporting of suicide rates.Objective: As the product of a data collaborative, this paper leverages private-sector search engine data toward gaining a fuller, more accurate picture of the suicide issue among young people in India. By combining official statistics on suicide with data generated through search queries, this paper seeks to: add an additional layer of information to more accurately represent the magnitude of the problem, determine whether search query data can serve as an effective proxy for factors contributing to suicide that are not represented in traditional datasets, and consider how data collaboratives built on search query data could inform future suicide prevention efforts in India and beyond.Methods: We combined official statistics on demographic information with data generated through search queries from Bing to gain insight into suicide rates per state in India as reported by the National Crimes Record Bureau of India. We extracted English language queries on “suicide,” “depression,” “hanging,” “pesticide,” and “poison”. We also collected data on demographic information at the state level in India, including urbanization, growth rate, sex ratio, internet penetration, and population. We modeled the suicide rate per state as a function of the queries on each of the 5 topics considered as linear independent variables. A second model was built by integrating the demographic information as additional linear independent variables.Results: Results of the first model fit (R2) when modeling the suicide rates from the fraction of queries in each of the 5 topics, as well as the fraction of all suicide methods, show a correlation of about 0.5. This increases significantly with the removal of 3 outliers and improves slightly when 5 outliers are removed. Results for the second model fit using both query and demographic data show that for all categories, if no outliers are removed, demographic data can model suicide rates better than query data. However, when 3 outliers are removed, query data about pesticides or poisons improves the model over using demographic data.Conclusions: In this work, we used search data and demographics to model suicide rates. In this way, search data serve as a proxy for unmeasured (hidden) factors corresponding to suicide rates. Moreover, our procedure for outlier rejection serves to single out states where the suicide rates have substantially different correlations with demographic factors and query rates.J Med Internet Res 2019;21(1):e10179doi:10.2196/10179
BibTeX
@article{adler2018how,
  author = {Natalia Adler and Ciro Cattuto and Kyriaki Kalimeri and Daniela Paolotti and Michele Tizzoni and Stefaan Verhulst and Elad Yom-Tov and Andrew Young},
  title = {{How Search Engine Data Enhance the Understanding of Determinants of Suicide in India and Inform Prevention: Observational Study}},
  journal = {Journal of Medical Internet Research},
  year = {2018},
  url = {https://openreview.net/forum?id=bRqZzUctZj},
}
Figure