PAME-AI: Patient Messaging Creation and Optimization using Agentic AI

Junjie Luo, Yihong Guo, Anqi Liu, Ritu Agarwal, Gordon Gao

TL;DR

PAME-AI introduces a multi-agent AI framework structured around the Data-Information-Knowledge-Wisdom (DIKW) hierarchy to solve the "curse of dimensionality" in healthcare message design. By progressively transforming raw clinical trial data into generalizable behavioral insights, the system automatically designs optimized patient reminders, achieving a 12.2% relative improvement in engagement over standard baselines.

What Problem Does This Paper Address?

Effective patient communication (like text message reminders for prescription pickups) directly improves medication adherence and healthcare efficiency. However, designing the perfect message is incredibly difficult due to the "curse of dimensionality." If you vary just 10 message attributes (such as tone, urgency, or length) across 3 different levels, you end up with nearly 60,000 possible message variations—far too many to test via traditional A/B testing.

Furthermore, human healthcare experts are notoriously poor at predicting which phrasing will actually motivate patients, and standard machine learning approaches treat experimental data as static findings rather than building an iterative, reusable knowledge base.

Methods

The authors tackle this challenge by developing PAME-AI, an agentic architecture built on the classic Data-Information-Knowledge-Wisdom (DIKW) hierarchy. Instead of relying on a single monolithic model, PAME-AI orchestrates a team of specialized LLM agents (powered by Claude-4 Sonnet) that operate at different layers of abstraction:

  • Data Agents: Handle the foundational layer by writing, validating, and executing code to clean raw experimental datasets and ensure data integrity.
  • Information Agents: Compute objective, deterministic statistical facts, distributions, and correlations from the structured data (e.g., calculating exact click-through rates per demographic).
  • Knowledge Agents: Move beyond the immediate dataset to test generalizable hypotheses, combining the statistical facts with behavioral science theories to assign confidence scores to specific communication principles.
  • Wisdom Agents: Translate validated knowledge into actionable strategies. They generate new message variants using a dual strategy: exploitation (doubling down on high-confidence patterns like authority framing) and exploration (testing novel psychological frameworks).
Figure 1: Multi-agent DIKW architecture for healthcare message optimization

Datasets

The framework was evaluated using a rigorous two-stage experimental approach:

  • Stage 1 (Baseline Data): PAME-AI ingested a large-scale randomized controlled trial (RCT) dataset containing 444,691 patient encounters across 13 human-designed psychological message variants.
  • Stage 2 (Validation Trial): The framework generated 20 new message designs. After expert filtering, 74,908 patient encounters were tested in a follow-up RCT evaluating 17 AI-generated variants alongside 3 top baselines.

Key Insights and Ideas

  • Healthcare is Not Standard Marketing: Standard e-commerce tactics like "social proof" ("Most patients find this useful...") consistently underperformed in a medical context. Instead, healthcare communication thrives on urgency, authority, and task-completion framing.
  • The Power of Context Integration: Framing the message as a natural extension of the clinical workflow ("Final step from your visit - review prescription") successfully leverages a patient's natural desire for goal completion.
  • Age is the Dominant Demographic Factor: Age heavily dictated message responsiveness, with older patients showing a 12% higher baseline engagement. However, the severity of the medical condition acted as a powerful moderator; acute conditions triggered high engagement regardless of demographics.
  • Linguistic Precision: Concise messages under 65 characters that established authority right at the opening ("Dr. Johnson's office:...") significantly outperformed passive, informational framing.

Key Results

  • Significant Engagement Boost: The top-performing AI-generated message (efficiencyTech: "Dr. Kristen Johnson: New Rx info needs quick review") achieved a 68.76% engagement rate, compared to the 61.27% default baseline—representing a 12.2% relative improvement in click-through rates.
  • High Performance Spread: The experiment revealed a 28.9% relative improvement between the best and worst-performing messages, highlighting how drastically subtle linguistic choices impact patient behavior.
Figure 2: Message Performance Analysis: Click rates across 20 variants ranked by effectiveness. Colors indicate generation strategy: Exploitation (blue), Exploration (purple), and Last Round baseline (orange). The framework achieved a 28.9% relative improvement between best and worst performers, with a 7.5 percentage point improvement over the default baseline.

Limitations and Future Work

While highly effective, the authors note a few limitations:

  • Predefined Hypotheses: The Knowledge agents currently operate on predefined hypothesis templates, which limits their ability to organically discover entirely unexpected behavioral phenomena.
  • Short-Term Metrics: The evaluation focused primarily on immediate click-through rates rather than tracking long-term, sustained behavioral changes or actual medication adherence over months.
  • Future Directions: The team plans to explore cross-domain validation (moving beyond prescription notifications) and integrating reinforcement learning to allow the system to dynamically adapt messages in real-time.

Summary updated

Abstract

Messaging patients is a critical part of healthcare communication, helping to improve things like medication adherence and healthy behaviors. However, traditional mobile message design has significant limitations due to its inability to explore the high-dimensional design space. We develop PAME-AI, a novel approach for Patient Messaging Creation and Optimization using Agentic AI. Built on the Data-Information-Knowledge-Wisdom (DIKW) hierarchy, PAME-AI offers a structured framework to move from raw data to actionable insights for high-performance messaging design. PAME-AI is composed of a system of specialized computational agents that progressively transform raw experimental data into actionable message design strategies. We demonstrate our approach's effectiveness through a two-stage experiment, comprising of 444,691 patient encounters in Stage 1 and 74,908 in Stage 2. The best-performing generated message achieved 68.76% engagement compared to the 61.27% baseline, representing a 12.2% relative improvement in click-through rates. This agentic architecture enables parallel processing, hypothesis validation, and continuous learning, making it particularly suitable for large-scale healthcare communication optimization.
BibTeX
@misc{luo2025pame,
  author = {Junjie Luo and Yihong Guo and Anqi Liu and Ritu Agarwal and Gordon Gao},
  title = {{PAME-AI: Patient Messaging Creation and Optimization using Agentic AI}},
  year = {2025},
  eprint = {2509.24263},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.24263},
}
Figure