The value proposition for research in integrated delivery systems
In the last few months, executive leaders from several integrated health care delivery systems have asked, “What is the return on investment (ROI) from research in our organization?” This question typically arises when we are promoting the paradigm of the learning health care system in these organizations. We all agree that ongoing collaboration between researchers and clinical/operational leaders can improve the relevance of research. But our leaders are asking a searching question- whether such gains are sufficient to justify ongoing support of the research enterprise as a whole.  

According to Wikipedia, ROI “…measures, per period, rates of return on money invested in an economic entity in order to decide whether or not to undertake an investment.” Similar to a cost-benefit analysis in health care, ROI assesses both inputs and outputs in financial terms. Asking research to demonstrate ROI holds us to a very stringent standard and a very tight timeline, since we know that very few health care interventions actually save money and that successful interventions are generally built on years of observational and interventional studies. In a few special cases, research in Kaiser Permanente Colorado and elsewhere does point the way toward organizational savings, but I think that we, as researchers, need to make a broader case for the value of what we do. We need to change the terms of the discussion.

A “value proposition” for research should emphasize four broad areas: development of new tools and approaches; information to support clinical and operational decisions; organizational communication; and (not to overlook it) cost savings.

Developing new tools and approaches is critical for innovative research. Along with clinical and operational colleagues and research partners throughout Kaiser Permanente and the HMO Research Network, the Institute for Health Research has developed new data resources, such as our virtual data warehouse (VDW) and new analytic methods for predicting clinical outcomes using electronic health data.  We have tested new approaches to data collection, such as interactive voice response (IVR) telephone surveys and on-line surveys for KP members. We are exploring the potential of social media interventions. We have developed tools to assess the outcomes of behavioral health care and important risk factors in the elderly. All these tools have uses in day-to-day operations.

We provide information to support clinical and operational decisions in several ways. When our investigators conduct research that can improve care, we work with operational and clinical colleagues to disseminate and implement those findings. Often, we are approached by operational leaders to evaluate new programs and initiatives. Our ongoing evaluation of the health needs and utilization of new KP members through the Affordable Care Act is the most recent example of this type of operationally-driven research. At times, we can support operational decisions by showing that a popular program is not as effective as initially anticipated, or that its impact is complex or nuanced. One example of this is our study showing that (contrary to our expectations), utilization of in-person care actually increased in the year after KP members signed up for our electronic portal. This surprising finding gave rise to an exciting partnership between KP researchers and KP IT nationally, through which all of us are learning about the potential and limitations of e-health care for KP members.

Research is a surprisingly important component of organizational communication. Communications leaders of many integrated delivery systems have told me that research findings comprise over 50% of the positive media mentions for their organizations.  Since research budgets are a small portion (typically < 0.1%) of the overall budget for those systems, research provides substantial leverage for our “brand” in the community.

And yes, at times we do conduct studies that help to control costs. A recently-completed study (which began with a request from the Population and Preventive Services Department in KPCO), showed that IVR reminders for influenza vaccination in adults with chronic lung disease were as effective, and less expensive when taken to scale, than traditional postcard reminders. Individually, such findings may not have a big effect on the “bottom line”, but every research finding that becomes a standard procedure helps us become a more efficient and evidence-based organization.

To use a metaphor from biology, two “organisms”-- like a research department in a delivery system-- can be related through parasitism, commensalism, or mutualism.  A parasitic relationship harms the host, a commensal relationship benefits the one without harming the other, and a mutualist relationship provides benefit to both. When organizational leaders ask us hard questions about ROI, they challenge us to make the case that we are more than commensals in our delivery systems. (I don’t think any of them are concerned that we are parasites). I believe that the argument for mutualism is a strong one. To demonstrate this, we need to do what researchers do best – specify the question and the appropriate measures to assess it, and then muster evidence to inform the answer. A broader discussion of all the areas where research can add value to its organizational “host” will benefit us all.

Warm Regards,
John F. Steiner, MD, MPH
Senior Director

Arne L. Beck, PhD, is the Director of Quality Improvement and Strategic Research in the Institute for Health Research. He also is an Associate Professor of Family Medicine at the University of Colorado Denver Health Sciences Center and Chair Emeritus for Board of Directors of the Jefferson Center for Mental Health, a community mental health center serving the Colorado counties of Jefferson, Gilpin and Clear Creek. His broad research focus is in mental health services, including:

• integration of behavioral health and medical care for patients with chronic illness
• brief psychotherapy for perinatal depression
• online mindfulness-based cognitive therapy for depression relapse prevention
• and suicide prevention programs

Dr. Beck's current roles include KPCO site investigator for the NIMH-funded Mental Health Research Network, Co-Director for the Center for Health Education, Dissemination, and Implementation (CHEDIR) research, and expert consultant to KPCO’s Depression Governance Council and Behavioral Health Quality Outcomes Committee. 

The Institute for Health Research hosted the 20th Annual HMO Research Network (HMORN) conference from April 1-3 in Phoenix, AZ. IHR Senior Director, John F. Steiner, MD, MPH, served as the Conference Chair and IHR Investigator, Heather Spencer Feigelson, PhD, served as the conference Scientific Chair. The theme for the conference this year was Embedded Research to Improve Health, highlighting research that has improved the health and well-being of the patients and communities the HMORN delivery systems serve. This year's conference featured many "firsts" including awards for “Paper of the Year,” “Poster of the Year,” and “Biggest Reject.” The "Biggest Reject" competition asked conference attendees to submit their wounding stories of rejected manuscripts and proposals. With 16 conference tracks, 3 days of 4 concurrent sessions, ancillary meetings, 3 plenaries, 2 luncheon presentations by representatives from PCORI, a Fun Run, and a field trip to the Desert Botanical Garden, it was a packed program for all participants and a very successful meeting.

Jason Glanz, PhD, co- authored a study published in the New England Journal of Medicine that looked at two types of rotavirus vaccines and compared each of their risk of causing intussusception. Findings showed when Rotarix was compared with RotaTeq, Rotarix had a much higher risk for intussusception. Based on this data, the RotaTeq vaccine appears to be a lot safer than the Rotarix vaccine with regards to intussusception. (PM: 24422678)

Debra Ritzwoller, PhD, led a study demonstrating the use of bevacizumab plus carboplatin-paclitaxel (BCP) chemotherapy versus carboplatin-paclitaxel (CP) only in patients under 21 years of age with nonsquamous, non-small-cell lung cancer. The comparative effectiveness analysis showed that patients receving BCP therapy had a higher survival rate than those receiving just CP across cohorts. (PM: 24633407)

Matt Daley, MD, Simon Hambidge, MD, PhD, and Jason Glanz, PhD, studied the safety of the DTaP-IPV vaccine using over 200,000 patients in the Vaccine Safety Datalink (VSD). They examined the risk of serious adverse events such as seizure, stroke, Guillain-Barre syndrome, and encehpahlitis, and found that DTaP-IPV recipients were not at an increased risk for these events following vaccination. (PM: 24699471)

Arne Beck, PhD, co-authored a study published in the Journal of the American Board of Family Practice investigating the frequency of shared decision making (SDM) processes in depression treatment across 88 primary care clinics across Minnesota. The study found that patients who were older and patients who had been seeking treatment for longer periods of time reported less SDM in their care. (PM: 24610182)

David Magid, MD, MPH, co-authored a study examining the relationship between obesity and high blood pressure in over 115,000 children between the ages of 6-17. The study found that children in the highest obesity percentiles were at a nearly three-fold risk of high blood pressure compared to children with less severe obesity. (PM: 24580759)

David Tabano, MA, presented at the 19th Annual International Meeting of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) conference held in Montreal in June. Co-authored by IHR Investigator Matt Daley, MD, and IHR Project Manager Jennifer Barrow, MSPH, David presented the paper, “A Spatial Distribution of Adult Obesity Prevalence in Denver County, Colorado: An Empirical Bayes Approach.” As a PhD student, David also won the Best Student Podium Award. His presentation can be found here.

Clinical Investigator Ted Palen, PhD, MD, MSPH, presented at several recent conferences, including:

• April 30: National Quality Conference Leading the Challenge: Making Great Care Affordable, Universal City, CA. "Interregional Specialty Care Teams providing the Best Affordable Care Everywhere."
• May 1: National Quality Conference Leading the Challenge: Making Great Care Affordable, Universal City, CA. "The KP SmartBook's Top 10 for 2013."
• May 18: American Telemedicine Association Annual Meeting, Baltimore, MD. "How to Find Data - Mining for TeleHealth Gold." 
• May 20: American Telemedicine Association Annual Meeting, Baltimore, MD. Abstract accepted for presentation by co-author Jennifer Cunha, "Implementing Mobile Telemedicine Virtual Rounds in the ICU."
• May 22: eHealth Initiatives: National Forum on Data and Analytics, Washington, DC. "Deep Dive: Provider Analytics to Improve Patient Outcomes."

The Colorado Medical Society Committee on Professional Education and Accreditation has awarded the highest accreditation level to the Colorado Permanente Medical Group (CPMG) for continuing medical education (CME) to the physicians of KPCO. As Director of Medical Education for the Colorado region, Dr. David Price and his team, Elaine Miller, MS, and Debbie Horner have earned the group a six-year accreditation. To receive commendation, organizations must also demonstrate that they use CME as a tool to improve quality performance and health outcomes, and that they collaborate with internal or external stakeholders to further improve quality.  

Senior Data Specialist and SAS Programmer Heather Tavel, MS, was awarded the Colorado School of Public Health's Best Presentation Award in 2014 for "Using a Validated Risk Score to Focus Care Interventions: Can We Identify Intervenable Populations Within the Highest Risk Score Segment." She was highlighted in the school's Public Health Forum. You can find the article here.

Susan Shetterly, MS, a Senior Biostatistician in the IHR, was recently nominated for a Teacher of the Year award, which recognizes and rewards excellence in professional teaching across KPCO. Susan teaches in both formal and informal settings and is a leader in teaching statistical and analytic techniques to IHR colleagues. She has revamped IHR monthly biostatisticians' meetings, focusing on education of statistical techniques. She facilitates an open and engaging learning environment, excelling at one-on-one teaching, and effectively helping junior analysts master new techniques.

After 14 years as Director of the Human Research Participant Protection Program (HRPPP), Jared Rowe, PharmD, is moving on to a new job opportunity at the Catholic Health Initiatives, working to develop their research infrastructure.  Good luck, Jared. We will miss you!

Project Coordinator Jennifer Boggs, MSW, was accepted into a PhD program in Health Services Research at the University of Colorado Denver Anschutz. The program is a collaboration between the Clinical Science Program and the Colorado School of Public Health. She will begin classes in August 2014. 

Project Manager Nat Jackson, MS, is leaving the IHR to move to KPCO's Claims Department. As a Senior Business Specialist, Nat will be working with his new team to improve claims processes. We wish him the very best in his new adventure!

After nearly 25 years at Kaiser Permanente, Senior Research Specialist Marilyn Pearson is retiring. Marilyn first joined Kaiser in 1989 in the Quality Assurance/ Utilization Management department, eventually joining the IHR in 2000. We wish you the best on your new journey, Marilyn!

To see the IHR's publications for 2014 please visit our website.