TrackMem: Tracking Evolving Beliefs in Long-Horizon Financial Dialogue

Austin M. Y. Cheung, Harsh Vardhan Gupta, Go Nam Lui, Lok Kan Chan, Ginny Y. Wong, Simon SeearXiv 2026 · arXiv

TL;DR

TrackMem is a novel graph-based memory system designed for the "messy" reality of financial dialogue, where facts are frequently revised and stances evolve over time. By tracking information in "event time" rather than just "conversation time," it significantly outperforms existing memory architectures on complex, long-horizon investment queries, particularly when handling updated figures and shifting perspectives.

What Problem Does This Paper Address?

Most LLM memory systems (like RAG or MemGPT) are evaluated on social dialogues where facts are "append-only"—once something is said, it stays true. Financial dialogue breaks this assumption. In an investment context:

  • Figures are revised: A price target set in January is superseded by a new one in March.
  • Stances evolve: A user’s "bullish" outlook might shift to "neutral" based on a series of market events across multiple sessions.
  • Constraints persist: A rule stated once (e.g., "never hold more than 5% in one stock") governs all future advice, even if it isn't mentioned again.
  • Lenses matter: The same company might be discussed through different analytical lenses (valuation, regulatory risk, etc.) that need to remain distinct.

Standard retrieval systems often fail here because they grab the most "similar" or "recent" text, which might be an outdated figure, a hypothetical scenario, or a different speaker's opinion.

Methods

TrackMem moves away from flat logs and toward a structured, per-speaker property graph ($G_p$). The system categorizes information into four distinct node types, as illustrated in the design overview:

Figure 3: Design of TRACKMEM
  1. Views: Summaries of a user's understanding of an entity (like a stock) under a specific topic or "lens."
  2. Anchors: Typed numeric values (e.g., price targets, yields) that maintain a versioned history.
  3. Claims: Immutable assertions from a single turn.
  4. Attributes: Stable, long-term user properties like risk tolerance or global portfolio constraints.

The core innovation is event-time tracking. Instead of just remembering when a turn occurred, TrackMem records the interval for which a fact is "in force." When a new session is ingested, a schema-constrained extraction process updates the graph. A "quantity resolver" decides if a new figure is a revision of an existing one or a separate data point.

Datasets

The authors introduce FinLoCoMo (Financial Long-Horizon Conversational Memory), a benchmark consisting of 15 conversations spanning 191–612 days. It includes 765 questions across two splits:

  • Standard Split: Follows existing benchmarks (LoCoMo) with single-hop, multi-hop, and temporal questions.
  • Modes Split: Specifically targets the four failure modes identified: Revision, Stance, Constraint, and Aspect.

Key Insights and Ideas

  • Structure Beats Retrieval: The authors found that even when a flat RAG system is given the same "retrieval budget" (number of turns), TrackMem's structured approach is more accurate. This is because the graph explicitly tells the reader model which value is currently valid, whereas flat retrieval leaves the model to guess between conflicting mentions.
  • The "Event Time" Advantage: By treating memory as a versioned history of state, the system avoids the "recency trap." It can distinguish between what was true at a specific point in the past versus what is true "now."
  • Channel-Specific Retrieval: Different questions require different data. TrackMem uses a "Router" to decide which channels (Anchors, Trajectories, Views) to query based on the intent of the question.

Key Results

TrackMem consistently outperformed five major baselines (MemGPT, Mem0, A-Mem, Zep, and O-Mem) across both splits.

Figure 1: Evidence recall against context length.
  • Revision Mastery: On questions about revised figures, TrackMem reached 86.5% accuracy, compared to 73.4% for the strongest baseline (MemGPT).
  • Superior Recall: As shown in Figure 1, TrackMem retrieves more "gold" (correct) evidence turns while maintaining a significantly smaller context window than a full-history approach.
  • High Precision Abstention: When a question is unanswerable, TrackMem is better at "declining" to answer (91.5% precision) rather than hallucinating a wrong answer.

Limitations and Future Work

  • LLM Dependency: The system relies on an LLM for the initial extraction and quantity resolution. If the model misidentifies an entity or merges two distinct figures during ingestion, that error persists in the memory.
  • Domain and Language: The current evaluation is limited to English-language financial dialogues between two speakers. Its performance in multi-party dialogues or other domains (like legal or medical) remains untested.
  • Ingestion Cost: Building the graph requires multiple model calls (extraction, resolution, synthesis), which is computationally more expensive than simple vector embedding used in standard RAG.

Summary updated

Abstract

Memory systems for LLM agents are evaluated mostly on social dialogue, where facts accumulate and rarely change, so remembering reduces largely to finding the turn in which something was said. Financial dialogue breaks this assumption: figures are revised, stances shift without being restated, rules stated once govern later questions, and a single company is discussed under several analytical lenses. Memory systems that retrieve the passages most similar to a question, or the most recent ones, therefore return superseded values, miss rules that share no wording with the question, and cannot recover a stance spread across sessions. We present TrackMem, a per-speaker graph memory that tracks the history of figures, separates perspectives by entity and lens, and preserves each speaker's actions in chronological order. To evaluate it, we introduce FinLoCoMo, a benchmark of 15 long-horizon investment conversations with 765 questions in two splits: one following the LoCoMo categories and one targeting four failure modes. With two readers, TrackMem outperforms existing memory systems on both splits. On the failure-mode split it reaches 73.9% macro accuracy against 69.4% for the strongest baseline, with the largest gains on questions about revised figures (86.5% against 73.4%).
BibTeX
@misc{cheung2026trackmem,
  author = {Austin M. Y. Cheung and Harsh Vardhan Gupta and Go Nam Lui and Lok Kan Chan and Ginny Y. Wong and Simon See},
  title = {{TrackMem: Tracking Evolving Beliefs in Long-Horizon Financial Dialogue}},
  year = {2026},
  eprint = {2611.02001},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02001},
}
Figure