April 7, 2023 | Blog | 7 minutes

Real world data is getting increasingly complex

Balaji Krishnan

Chief of Customer Success

Importance of real-world data for Pharmacovigilance 

Drug safety is of utmost importance when it comes to patient care. Even though clinical trials are carefully designed and conducted, they still have limitations. Patients with comorbidity, pregnant women and children are often excluded from clinical trials, which can limit the generalizability of the results. The clinical trials do not reach the power to identify all the possible adverse reactions (rare & very rare) with the drug. Hence there is limited information on the safety and efficacy of a medicinal product at the time of receiving marketing approval. Additionally, drug-drug interactions (DDIs) may not be fully known or understood during clinical trials.  

This is where real world data comes in.  

 Real world data can provide valuable insights into the safety and effectiveness of drugs in the general population, including those with comorbidities and potential DDIs. By analysing real world data, we can better understand the risks and benefits of drugs in real-world settings, leading to better patient care. Real world data can be collected from various sources, such as electronic health records, claims and billing data, and patient-generated data. This data can be used to identify trends and patterns in drug safety and effectiveness, as well as inform regulatory decisions. While clinical trials are still important for drug development and approval, real world data can complement clinical trial data and provide a more complete picture of a drug's safety and efficacy. Therefore, it is crucial to continue to prioritize the collection and analysis of real-world data for drug safety. 

Ever changing sources of data and complexity 

The current challenge of pharmacovigilance is two-fold. First, there is a lack of consistency in how safety and efficacy data is collected, analysed, and reported. This leads to confusion and can lead to medical errors. Second, the sheer volume and complexity of data available makes it difficult to separate out the signals that indicate potential health issues. In the future, pharmacovigilance will become increasingly complex as more data sources are available, from electronic medical records to mobile health tools. With this complexity comes a need to develop better methods of integrating and analysing data, as well as developing systems that enable predictive analytics. As such, it is essential that healthcare providers and regulatory bodies remain abreast of the current and future needs of pharmacovigilance. 

Healthcare information systems are generating more patient-level data than ever before with significant rise in prescription drug consumption and number of patients taking prescription drugs. This high influx of data must be managed and analysed if drug safety is to be monitored effectively. The use of non-traditional data sources such as digital and social media for pharmacovigilance is on the rise with advancement in technology and ease with which the population is accessing these technology tools. Data is getting generated from sites such as Twitter, Reddit, Facebook, YouTube, and Instagram, among other sources. Social media platforms like Twitter and Facebook have become a popular forum for patients and consumers to share their experiences with products and services. This includes sharing of information and online conversations about adverse events, and as a result, social media has become an important source of data for pharmacovigilance and drug safety. These data sources can provide valuable insights about patient safety and the potential risks associated with certain medicines. New data sources can be accessed to capture safety signals more accurately and timely that may otherwise be missed. New sources of patient-level data, such as electronic health records, mobile health applications, and wearables, provide a more holistic view of drug safety and patient demographics.  

The emergence of digital and social media has presented the pharmacovigilance function with a range of opportunities as well as challenges. As technology advances, so does the complexity and volume of available data sources which is making it difficult to fully understand how they can be best leveraged to support patient safety. Existing data mining and data analysis techniques must be upgraded to be able to handle the new types of data. Some of the challenges associated with social media data on adverse events include: 

  • Difficulty in collecting relevant data from disparate sources  
  • Identify and separate safety signals from the vast amount of data  
  • Ensuring data accuracy and quality 
  • Developing appropriate analytical tools to interpret  
  • Identifying trained resources to manage this level of volume and data complexity  
  • Ensuring patient privacy and confidentiality 
  • Ensuring compliance to regulatory requirements  

Despite these challenges, social media data remains a valuable resource for monitoring adverse events and identifying potential safety issues. It is important for regulators and pharmaceutical companies to continue to explore ways to leverage this data while also addressing the challenges it presents. 

Additionally, these data sources often include information that is changing very rapidly. Pharmacovigilance teams must be equipped with the necessary algorithms and tools to ensure the data is both accurate and up to date. At the same time, they must also ensure they continue to comply and adhere to local and international regulations with respect to data security and privacy. 

The increased availability and complexity of data is driving the need for better processes and infrastructure to support the efficient and timely use of data sources. The development of an integrated approach, with the focus on safety intelligence, data integration, and process automation, is essential for pharmacovigilance to be as effective as possible. Companies must have the right technology and processes in place to ensure data accuracy and up-to-date information. Additionally, data privacy and security must be a priority, to abide by the respective regulations. By understanding and effectively leveraging the data sources available, pharmacovigilance teams can best support patient safety.  

Pharmacovigilance in the Big Data Era  

Traditional methods of pharmacovigilance are no longer sufficient to handle this unprecedented growth in data volume. With millions of drug products now available in the market, the sheer volume of data coming from disparate and heterogeneous sources requires more advanced approaches to pharmacovigilance. The data needs to be converted from unstructured to structured form before subjecting it to the different data mining and analysis techniques.  

big data in pharmacovigilance


Recent technological advances and data mining techniques have enabled the development of innovative pharmacovigilance strategies to identify potential safety signals. To ensure the safety of drug products, the pharmacovigilance field has embraced the use of big data techniques to analyse large volumes of information. AI (Artificial Intelligence) and machine learning technology is widely employed to collect and analyses large and complex datasets. By leveraging advances in artificial intelligence (AI) and machine learning (ML), pharmacovigilance teams can now detect safety signals much sooner, more efficiently and more accurately than before. This allows much quicker action on the usage of drugs and prevents harm to the larger population. Additionally, the use of predictive analytics can help anticipate adverse events before they occur and provide insight into how drugs interact with other treatments.  

The use of Artificial Intelligence (AI) and Machine Learning (ML) can bring major transformation in pharmacovigilance and drug safety signal management. AI and ML can be used to reduce the manual efforts spent on a wide range of drug safety activities and improve the results on signal detection and management. AI-driven systems can be used to automate and improve the signal detection process by identifying information from large sources including spontaneous reporting system (SRS) databases, Electronic Health Records (EHRs), and other structured and unstructured data sources. This data can provide a more detailed picture than what is provided by manual methods, drastically improving the safety signal detection process. AI and ML can also be used to alert medical professionals to potential safety issues faster, while also providing greater insights into the effectiveness of treatments. Additionally, AI can be used to identify potential drug-drug interactions and provide personalized predictive analytics. AI-driven systems can even be used to monitor safety over time and quickly identify potential safety issues in future. All these advances will be invaluable in ensuring the safety of patients and efficiency of the drug.