Part 2 of A short series offering actionable insights from a Health Affairs article
Part 2: How did they review the programs?
In an April 2017, Health Affairs article, “Industry-Led Access-To-Medicines Initiatives in Low- And Middle-Income Countries: Strategies and Evidence,” various health initiatives were reviewed. The first part of this series provided the facts and figures associated with the analysis. Part 2 shows how they reviewed the programs…
So, let’s start…
I have provided insights on various subjects to my clients over the years and the insights are specific and unique to their needs. The only thing I pull off the shelf is a general approach to do the job.
As you go through the analysis process, you must continue to keep in mind what the client is looking for because there’s always interesting things that pop up. “Parking lot” those interesting tidbits but unless they show up so much so that they color the analysis, don’t include them in your report.
There always are subjective and objective aspects to insight development. For example, the categories by which things are classified may be objective – but what determines the category in which you put something can be subjective.
An analysis I did for one client years ago required reading approximately 750 articles; fortunately, most were short. While reviewing them, I had to categorize them and ultimately identify topics in which to put them. This was before evaluating them and identifying insights. It actually combined subjective and objective elements. After ongoing grouping and “narrowing” the categories, you pull out the parts that clients can act on – i.e., the Insights. All of this process requires a combination of objective and subjective analysis; often that just comes with experience (i.e., years in the business).
And that leads me to here because no clear analytical method existed so the authors created one. It can be challenged but another analysis doesn’t exist. You can glean insights from this which is very helpful – that is, you can set a template for how to evaluate and gain recognition for the sustainable work you do.
Let’s take a look at what they did and we can break it down into Organize, Review and Analyze.
Step 1: Which companies to evaluate? (Identify companies)
a) They started with several sources – Source 1: the 2015 Access to Medicines (AtM) index and a list of 21 companies in the index
- The index is developed by the Access to Medicine Foundation, whose mission is to stimulate and guide pharmaceutical and vaccine companies to do more for the people living in low- and middle-income countries without access to medicine or vaccines. They determine which actions pharmaceutical and vaccine companies can and should be taking in this regard, analyze what specific companies are doing, and use the findings to drive change. The Foundation is an independent, non-profit organization. It was founded in 2003 by Dutch entrepreneur Wim Leereveld and has been issued every two years since 2008. Now funded by the Bill & Melinda Gates Foundation and the UK and Dutch governments, the index has since come out with updated company rankings every two years and appears to have created an incentive for expanding AtM initiatives, as well as for more transparent reporting.
b) Source 2: Looked at the International Federation of Pharmaceutical & Manufacturer’s Association (IFPMA) Health Partnerships Directory – Companies’ list of Health Initiatives
- IFPMA represents research-based biopharmaceutical companies, and regional and national associations across the world. It brings the industry and broader health community together to foster innovation, promote resilient regulatory systems and high standards of quality, uphold ethical practices, and advocate sustainable health policies to meet global needs.
c) Source 3: Supplemented with information from reports published by the (AtM) Index and annual and corporate social responsibility reports for those organization listed in the AtM Index but not members of IFPMA.
Step 2: Which programs to consider for evaluation? (Identify programs)
a) Identified initiatives that aimed to improve the health of patients and populations through efforts that go beyond traditional drug production and sales and that were implemented in low- and middle-income countries (According to the World Bank definition of low- and middle-income countries)
- World Bank is comprised of 189 member countries and is a vital source of financial and technical assistance to developing countries around the world. It represents unique partnerships to reduce poverty and support development. (http://www.worldbank.org/) Established in 1944, the World Bank Group is headquartered in Washington, D.C. We have more than 10,000 employees in more than 120 offices worldwide.
- In this case it was under Gross National Income [GNI] of $4,035
b) Outlined key information for each initiative, including
- company or companies involved
- disease focus
- target population
- program countries
- years of operation
Step 3: Identify evaluation categories/strategies (how to evaluate programs)
The authors identified categories in which the big picture of access initiatives could be placed. Some of these programs – or health initiatives – were listed in several strategies. Their breakdown is as follows:
#1 Access to Medicines (Strategies aimed at reducing access barriers to medicine directly)
- Price reduction: Activities that lower the price of medicines at the point of original sale by the pharmaceutical company
- Medicine donation: Activities that supply medicines at no cost at the point of original sale by the pharmaceutical company
- Licensing agreement: Activities that allow local manufacturers to produce generic forms of patented medicines
- Supply chain strengthening: Activities that improve the process of getting drugs from the point of manufacturer through the distribution system all the way to patients
#2 Health Services (Strategies aimed at improving service delivery)
- Service sponsorship: Activities that provide health services directly to populations, for example mobile clinics or community health fairs
- Infrastructure investment: Activities that improve the infrastructure needed to delivery health services, for example building clinics or donating equipment or supplies
- Provider training: Activities to train existing or future health care providers, for example to improve diagnosis and prescribing practices or counseling services
- Awareness campaign: Activities that provide patients with health information, for example radio spots or community information sessions
#3 Indirect support strategies (strategy aimed at improving population health indirectly)
- Financial support: Activities that provide direct funding to other health organizations, for example money donations to the Red Cross
- Community development: Activities that promote general community development, for example support for micro-finance groups
- Research and development: Activities meant to identify new molecules for the treatment of disease
Step 4: Prioritize based on categories: Had to have at least one access-to-medicines strategy (price reduction, medicine donation, licensing agreement, or supply-chain strengthening)
Step 1: Collect all the available evidence on the initiatives’ impact.
a) Find them: Searched several sources for published reports and peer-reviewed journal articles
- Search PubMed for articles related to each initiative using the initiative name, company name, disease focus and country focus
- Search Google Web and Google Scholar using the same search terms used with PubMed
- Review each initiative’s webpage on the IFPMA Health Partnerships Directory. Links to publications are also frequently included
Step 2: Read the abstracts of the publications collected
a) Confirmed that the article provided an “evaluation” of the initiatives, defining evaluation as any article that attempted to measure outcomes or impacts using a clear research design. Excluded were the following:
- Describing initiatives or reporting inputs or outputs (e.g., money spent or pills distributed) were excluded
- Efficacy trials that tested the impact of a medicine used in an initiative rather than the impact of the initiative itself
Stage 3: Compare claims made related to impacts
a) Compared sources to evaluate claims made of the initiatives’ impacts. For this they read the articles and compared to other sources, such as the initiative web pages on the IFPMA Health Partnerships Directory; the Annual reports and corporate social responsibility reports non-IFPMA members
a) Now that there is a respectable list of companies and initiatives to evaluate, the authors began the process to analyze. They used the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system
- This is a system developed by the Cochrane Collaboration and used by approximately 25 organizations (including WHO). Cochrane is an international not-for-profit and independent organization, dedicated to making up-to-date, accurate information about the effects of healthcare readily available worldwide. It produces and disseminates systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions. Cochrane was founded in 1993 and named after the British epidemiologist, Archie Cochrane. The major product of Cochrane is the Cochrane Database of Systematic Reviews which is published quarterly as part of the Cochrane Library – and considers the following characteristics:
- Quality of Evidence
- Uncertainty about the balance between desirable and undesirable effects
- Uncertainty of variability in values and preferences
- Uncertainty about whether the intervention represents a wise use of resources
b) Based on the GRADE system, they assigned an overall quality rating of high, moderate, low, or very low to each article based on consensus or a majority among the authors’ ratings. Descriptive information was also extracted from each publication, including the aim of the evaluation and the study design.
The four key findings.
1. Number of AtM initiatives has grown substantially since 2000, when the Millennium Development Goals stated for the first time that pharmaceutical companies:
a) Must play a leading role in improving access to medicines in low- and middle-income countries
b) Have adopted reduced pricing and medicine donation as the primary strategies for their initiatives
2. Many claims have been made as to the impact of AtM initiatives, but very few rigorous evaluations have been conducted and published
3) Evaluations that have been published are generally of low quality, though there were three particularly good studies generating evidence of moderate quality. In addition, Nearly all AtM initiatives in low- and middle-income countries included a focus on infectious diseases, while only half included noncommunicable diseases. Given the growing burden of such diseases in these countries, an increased focus on access to medicines for them may be desirable.
4) There is at least one AtM initiative in every low- and middle income country. However, the processes that companies use to determine initiative strategies and set priorities are not transparent, which raises questions about whether the resources involved are being allocated in line with the countries’ national priorities, For example, nearly half of the AtM initiatives we analyzed used medicine donation as their main strategy. It has previously been argued that such donations are unsustainable and might cause harm in the long term by distorting local market incentives for competition. Therefore, local stakeholders might prefer alternative strategies to donations in some instances. What is clearly desirable is a more transparent and inclusive strategy and priority setting process that holds the industry more accountable to local stakeholders. As many low- and middle income countries pursue efforts to achieve universal health coverage, assessing the contributions of AtM initiatives to these efforts— particularly those focused on coverage for populations in poverty—becomes more important –
Finally — Key Factors in Lack of Evaluation
Very few companies have evaluated initiatives in a substantive manner, which could be attributable to cost, skill sets, and potential for a reputational risk if there is no impact. However, quasi-experimental methodologies and partnerships with global health funders and/or academic institutions could help keep costs down and identify skill sets needed for evaluations.
Finally, the authors suggest that either pharmaceutical companies could band together to develop evaluation standards and/or the Access to Medicine Index could identify greater reputational incentives.
Next — Part 3 — the Template