Some features of statistical analysis of spontaneous adverse drug reporting data

Introduction. Risk of adverse drug reactions (ADRs) is a serious issue in pharmacotherapy and a major public health concern. Safety signal detection during the post-marketing phase is one of the most important goals of drug safety surveillance. Spontaneous reporting systems (SRS) are still widely used to identify safety signals based on real-world data. Various data mining statistical methods have been developed for this purpose, and they are classified into frequentist and Bayesian approaches. Statistical methods can also be used for the analysis of patient-related risk factors (demographic characteristics, concomitant diseases or medications). Identification of patients at high ADR risk is important for personalized pharmacotherapy.

Objective. To present and review issues and features of the statistical methods for SRS data, developed by other authors and published in the literature, this tool may be useful for appropriate statistical analysis and accurate interpretation of passive surveillance data.

Methods. In this paper, we present the known and commonly used frequentist or classical methods for correct statistical analysis of spontaneous reports. These methods for signal detection and their modification for drug-host factor interaction analysis are relatively easy to understand, interpret, and compute based on the contingency 2x2 tables: reporting odds ratio (ROR), proportional reporting ratio (PRR), and normal approximation test. Different approaches to the multiple comparison problem in passive safety surveillance settings were also discussed.

Results. As an example, the aforementioned methods were applied to analyze sex disparities in liver toxicity based on the spontaneous reports extracted from the Russian National Pharmacovigilance database. The tests identified drugs for which liver toxicity demonstrates significant disproportionality regarding sex compared with other AEs. The results of all statistical methods were similar.

Conclusions. Although spontaneous report databases are subject to numerous potential sources of bias and well-known limitations, these large-scale databases remain a widely used, effective, and relatively inexpensive approach for post-marketed drug surveillance. With the use of correct statistical methods, spontaneous reporting databases can provide valuable information for hypothesis generation, which should be investigated further, as well as essential data on the evaluation of risk factors and risk populations.

1. Huang L, Guo T, Zalkikar JN, Tiwari RC. A review of statistical methods for safety surveillance. Ther Innov Regul Sci. 2014;48(1):98–108. https://doi.org/10.1177/2168479013514236

2. Arnaud M, Bégaud B, Thurin N, Moore N, Pariente A, Salvo F. Methods for safety signal detection in healthcare databases: a literature review. Expert Opin Drug Saf. 2017;16(6):721–732. https://doi.org/10.1080/14740338.2017.1325463

3. Park G, Jung H, Heo SJ, Jung I. Comparison of data mining methods for the signal detection of adverse drug events with a hierarchical structure in postmarketing surveillance. Life (Basel). 2020;10(8):138. https://doi.org/10.3390/life10080138

4. Motrinchuk ASh, Loginovskaya OA, Kolbatov VP. Methods for drug safety signal detection using routinely collected observational electronic health care data: a systematic review. Real-World Data & Evidence. 2023;3 (2):42–55. (In Russ.). https://doi.org/10.37489/2782–3784-myrwd-35

5. Rothman KJ, Lanes S, Sacks ST. The reporting odds ratio and its advantages over the proportional reporting ratio. Pharmacoepidemiol Drug Saf. 2004;13(8):519–523. https://doi.org/10.1002/pds.1001

6. Yates F. Contingency table involving small numbers and the χ2 test. J R Stat Soc. 1934;1(2): 217–235. Supplement to the Journal of the Royal Statistical Society, https://doi.org/10.2307/2983604

7. Huang L, Zalkikar J, Tiwari RC. Likelihood ratio test-based method for signal detection in drug classes using FDA's AERS database. J Biopharm Stat. 2013;23(1):178–200. https://doi.org/10.1080/10543406.2013.736810

8. Caster O, Aoki Y, Gattepaille LM, Grundmark B. Disproportionality analysis for pharmacovigilance signal detection in small databases or subsets: recommendations for limiting false-positive associations. Drug Saf. 2020;43:479–487. https://doi.org/10.1007/s40264-020-00911-w

9. DuMouchel W. Bayesian data mining in large frequency tables, with an application to the FRA spontaneous reporting system. The American Statistician, 1999;53:177–190. https://doi.org/10.1080/00031305.1999.10474456

10. Northardt  T.  A Bayesian generating function approach to adverse drug reaction screening. PLoS One. 2024;19(1):e0297189. https://doi.org/10.1371/journal.pone.0297189

11. Bunchuailua W, Zuckerman I, Kulsomboon V Suwankesawong W, Singhasivanon P, Kaewkungwal J. Detection of adverse drug reaction signals in the thai FDA database: comparison between reporting odds ratio and Bayesian confidence propagation neural network methods. Ther Innov Regul Sci. 2010;44:393–403. https://doi.org/10.1177/009286151004400404

12. Kulldorff M, Dashevsky I, Avery TR, Chan AK, Davis RL, Graham D, at al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol Drug Saf. 2013;22(5):517–523. https://doi.org/10.1002/pds.3423

13. Moore N, Kreft-Jais C, Haramburu F, Noblet C, Andrejak M, Ollagnier M, et al. Reports of hypoglycaemia associated with the use of ACE inhibitors and other drugs: a case/non-case study in the French pharmacovigilance system database. Br J Clin Pharmacol. 1997;44(5):513–518. https://doi.org/10.1046/j.1365–2125.1997.00615.x

14. Lu Z, Suzuki A, Wang D. Statistical methods for exploring spontaneous adverse event reporting databases for drug-host factor interactions. BMC Med Res Methodol 2023;71. https://doi.org/10.1186/s12874-023-01885-w

15. Zheng S, Smith MG. Comparing two quantities by using a ratio. Biometrics & Biostatistics International Journal. 2020;9(5):186–187. https://doi.org/10.15406/bbij.2020.09.00318

16. Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10(6):483–486. https://doi.org/10.1002/pds.677

17. Norén GN, Bate A, Orre R, Edwards IR. Extending the methods used to screen the WHO drug safety database towards analysis of complex associations and improved accuracy for rare events. Stat  Med. 2006;25 (21):3740–3757. https://doi.org/10.1002/sim.2473

18. Caster O, Norén GN, Madigan D, Bate  A.  Large scale regression based pattern discovery: the example of screening the WHO global drug safety database. Stat Anal Data Min ASA Data Sci J. 2010;3:197–208. https://doi.org/10.1002/sam.10078

19. Sandberg L, Taavola H, Aoki Y, Chandler R, Norén GN. Risk factor considerations in statistical signal detection: using subgroup disproportionality to uncover risk groups for adverse drug reactions in VigiBase. Drug Saf. 2020;43:999–1009. https://doi.org/10.1007/s40264-020-00957-w

20. Souyri C, Olivier P, Grolleau S, Lapeyre-Mestre  M.  Severe necrotizing soft-tissue infections and nonsteroidal anti-inflammatory drugs. Clin Exp Dermatol. 2008;33(3):249–255. https://doi.org/10.1111/j.1365–2230.2007.02652.x

21. Seabroke S, Candore G, Juhlin K, Quarcoo N, Wiśniewski AF, Arani RB, et al. Performance of stratified and subgrouped disproportionality analyses in spontaneous databases. Drug Saf. 2016;39(4):355–364. https://doi.org/10.1007/s40264-015-0388-3

22. Hopstadius J, Norén GN, Bate A, Edwards IR. Impact of stratification on adverse drug reaction surveillance. Drug  Saf. 2008;31(11):1035–1048. https://doi.org/10.2165/00002018-200831110-00008

23. Woo EJ, Ball R, Burwen DR, Braun MM. Effects of stratification on data mining in the US Vaccine Adverse Event Reporting System (VAERS). Drug  Saf. 2008;31(8):667–674. https://doi.org/10.2165/00002018-200831080-00003

24. Evans SJ. Stratification for spontaneous report databases. Drug Saf. 2008;31 (11):1049–1052. https://doi.org/10.2165/00002018-200831110-00009

25. Wisniewski AF, Bate A, Bousquet C, Brueckner A, Candore G, Juhlin K, et al. Good signal detection practices: Evidence from IMI PROTECT. Drug Saf. 2016;39(6):469–490. https://doi.org/10.1007/s40264-016-0405-1

26. Dmitrienko A, D'Agostino R Sr. Traditional multiplicity adjustment methods in clinical trials. Stat in Med. 2013;32(29):5172–218. https://doi.org/10.1002/sim.5990

27. Dudoit S, Shaffer JP, Boldrick JC. Multiple hypothesis testing in microarray experiments. Statistical Science. 2003;18(1):71–103. https://doi.org/10.1214/ss/1056397487

28. Westfall PH, Young SS. Resampling-based multiple testing: Examples and methods for p-value adjustment. Third Avenue. New York: John Wiley & Sons; 1993.

29. Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J R Stat Soc Ser B. 1995;57:289–300. https://doi.org/10.2307/2346101

30. Gould AL. Accounting for multiplicity in the evaluation of «signals» obtained by data mining from spontaneous report adverse event databases. Biom J. 2007;49(1):151–165. https://doi.org/10.1002/bimj.200610296

31. Mennecozzi M, Landesmann B, Palosaari T, Harris G, Whelan M. Sex differences in liver toxicity-do female and male human primary hepatocytes react differently to toxicants in vitro? PLoS ONE. 2015;10(4):e0122786. https://doi.org/10.1371/journal.pone.0122786

32. Ingawale DK, Mandlik SK, Naik SR. Models of hepatotoxicity and the underlying cellular, biochemical and immunological mechanism (s): a critical discussion. Environ Toxicol Pharmacol. 2014;37(1):118–133. https://doi.org/10.1016/j.etap.2013.08.015

33. Amacher DE. Female gender as a susceptibility factor for drug-induced liver injury. Hum Exp Toxicol. 2014;33(9):928–39. https://doi.org/10.1177/0960327113512860

34. Li X, Tang J, Mao Y. Incidence and risk factors of drug-induced liver injury. Liver Int. 2022;42 (9):1999–2014. https://doi.org/10.1111/liv.15262

35. Farkouh A, Riedl T, Gottardi R, Czejka M, Kautzky-Willer A. Sex-related differences in pharmacokinetics and pharmacodynamics of frequently prescribed drugs: A review of the literature. Adv Ther. 2020;37(2):644–655. https://doi.org/10.1007/s12325-019-01201-3

36. Farkouh A, Baumgärtel C, Gottardi R, Hemetsberger M, Czejka M, Kautzky-Willer  A.  Sex-related differences in drugs with anti-inflammatory properties. Journal of Clinical Medicine. 2021;10(7):1441. https://doi.org/10.3390/jcm10071441

37. Rubin JB, Hameed B, Gottfried M, Lee WM, Sarkar M. Acute liver failure study group. acetaminophen-induced acute liver failure is more common and more severe in women. Clin Gastroenterol Hepatol. 2018;16(6):936– 946. https://doi.org/10.1016/j.cgh.2017.11.042

38. Torgersen J, Mezochow AK, Newcomb CW, Carbonari DM, Hennessy S, Rentsch CT, et al. Severe acute liver injury after hepatotoxic medication initiation in real-world data. JAMA Intern  Med. 2024. https://doi.org/10.1001/jamainternmed.2024.1836

39. Matsuda S, Aoki K, Kawamata T, Kimotsuki T, Kobayashi T, Kuriki H, et al. Bias in Spontaneous Reporting of Adverse Drug Reactions in Japan. PLoS ONE. 2015;10(5):e0126413. https://doi.org/10.1371/journal.pone.0126413

40. Fletcher AP. Spontaneous adverse drug reaction reporting vs event monitoring: a comparison. J R Soc  Med. 1991;84(6):341–344. https://doi.org/10.1177/014107689108400612

41. Begaud B, Moride Y, Tubert-Bitter P, Chaslerie A, Haramburu  F.  Falsepositives in spontaneous reporting: Should we worry about them? Br J Clin Pharmacol. 1994;38(5):401–404. https://doi.org/10.1111/j.1365–2125.1994.tb04373.x

42. Kazakov AS, Darmostukova MA, Bukatina TM, Velts NYu, Alyautdin RN. Comparative Analysis of International Databases of Adverse Drug Reactions. Safety and Risk of Pharmacotherapy. 2020;8(3):134–140. (In Russ.) https://doi.org/10.30895/2312-7821-2020-8-3-134-140

43. Montastruc JL, Sommet A, Bagheri H, Lapeyre-Mestre M. Benefits and strengths of the disproportionality analysis for identification of adverse drug reactions in a pharmacovigilance database. Br J Clin Pharmacol. 2011;72(6):905–908. https://doi.org/10.1111/j.1365–2125.2011.04037.x

44. Poluzzi E, Raschi E, Piccinni C, Ponti F. Data mining techniques in pharmacovigilance: analysis of the publicly accessible FDA adverse event reporting system (AERS). In: Karahoca A, ed. Data mining applications in engineering and medicine. InTechOpen; 2012. P. 265–302.

45. Raschi E, Piccinni C, Poluzzi E, Marchesini G, De Ponti F. The association of pancreatitis with antidiabetic drug use: gaining insight through the FDA pharmacovigilance database. Acta Diabetol. 2013;50(4):569–577. https://doi.org/10.1007/s00592-011-0340-7

46. Grégoire F, Pariente A, Fourrier-Reglat A, Haramburu F, Bégaud B, Moore N. A signal of increased risk of hypoglycaemia with angiotensin receptor blockers caused by confounding. Br J Clin Pharmacol. 2008;66 (1):142–145. https://doi.org/10.1111/j.1365–2125.2008.03176.x

47. Pariente A, Didailler M, Avillach P, Miremont-Salamé G, Fourrier-Reglat A, Haramburu F, et al. A potential competition bias in the detection of safety signals from spontaneous reporting databases. Pharmacoepidemiol Drug Saf. 2010;19(11):1166–1171. https://doi.org/10.1002/pds.2022

48. Cutroneo PM, Sartori D, Tuccori M, Crisafulli S, Battini V, Carnovale C, et al. Conducting and interpreting disproportionality analyses derived from spontaneous reporting systems. Front. Drug. Saf. Regul. 2024;3:1323057. https://doi.org/10.3389/fdsfr.2023.1323057