TL;DR
CogniLife introduces a framework for continuous cognitive distillation, a process that captures a creator's unique behavioral patterns and domain expertise to power specialized AI agents in a marketplace. By distinguishing between a creator's core "Soul" and revisable "Expert Journals," it aims to solve the cold-expert problem—where new agents lack sufficient data to act authentically—by transferring creator traits across different agents they own.
What Problem Does This Paper Address?
As multi-expert agent marketplaces (like Poe or Delphi) emerge, a critical challenge arises: how can a creator ensure their published AI agents faithfully represent their specific expertise and behavioral style?
The authors identify the cold-expert problem: a newly published agent might receive many user requests but has very little interaction data from its creator to learn from. Meanwhile, that same creator might be highly active elsewhere on the platform, interacting with other agents. Current personalization research typically focuses on the user (the caller), but CogniLife shifts the focus to the creator. It seeks to bridge the gap between a creator's "tacit expertise"—knowledge that is hard to write down—and the active behavior of their deployed AI agents.
Methods
CogniLife frames the agent lifecycle as a continuous loop of Seeding (initial authoring), Immersion (accumulating observations), and Crystallization (abstracting observations into rules).
The framework relies on three key architectural components:
- The Soul: A creator-controlled repository of core behavioral commitments. Crucially, automated background processes cannot edit the Soul; only the creator can authorize changes, preserving their agency.
- Creator Profiles (Proposed): These hold transferable patterns that apply across all agents owned by a creator (e.g., "always ask for clarification if a prompt is ambiguous").
- Expert Journals (Proposed): These store knowledge and procedures specific to a single agent listing.
The system implements three "Control Points" to manage this evolution:
- Capability-owned proposals: Filters potential skill updates based on estimated quality and efficiency gains.
- Scoped memory maintenance: Extracts facts and communication patterns from dialogue while keeping user-specific data separate from creator-specific data.
- Creator-controlled deployment: Ensures that only authorized configurations and skills are released to the public.
Datasets
The authors introduce PROFYBENCH-V1, a new offline benchmark designed to evaluate professional expert-service outputs. It consists of 85 tasks across five domains:
- Evidence Advising (ADV): Grounding, uncertainty, and safety.
- Education (EDU): Pedagogical structure and completeness.
- Marketing (MKT): Platform fit and audience-specific rewriting.
- Multimodal Workflow (MMW): Workflow decomposition and text-image consistency.
- Personalization (PER): Style fidelity and long-turn coherence.
The benchmark includes variants for "Complete" requests, "Missing-information" (to test robustness to vague prompts), and "Adversarial" inputs. Additionally, the paper reports on a supplementary customs benchmark involving 52 tasks for structured data extraction from professional documents.
Key Insights and Ideas
- Creator vs. Caller Attribution: A major "aha moment" is the formal distinction between evidence belonging to the creator (how the agent should act) and preferences belonging to the caller (how the agent should adapt to a specific user). Mixing these leads to "identity drift" in agents.
- The "Soul" Boundary: By making the "Soul" immutable to automated updates, the authors prioritize creator intent over pure algorithmic optimization. This ensures the agent doesn't "evolve" away from the creator's fundamental values without permission.
- Distillation as Compression: The framework treats interaction history not just as a log, but as a source for "distilling" ephemeral chat turns into durable, revisable textual guidance (rules and procedures).
Key Results
In evaluations using an LLM-as-a-judge (o3-mini), the "Profy Full" configuration—which utilizes the CogniLife framework—significantly outperformed baselines.
- Overall Score: Profy Full achieved 4.65/5, compared to 4.01 for a raw LLM and 4.28 for a standard "Expert Prompt" baseline.
- Dimensional Gains: The largest improvements were seen in completeness (rising from 3.61 to 4.62) and specificity (3.38 to 4.26).
- Robustness: Profy Full was particularly effective at handling "missing-information" tasks, effectively using distilled context to fill in gaps that left baseline models struggling.
Limitations and Future Work
The authors are transparent about several key limitations:
- Proposed vs. Implemented: The "cross-expert attribution" protocol—the core solution for the cold-expert problem—is currently a proposed design and has not yet been implemented or evaluated.
- Judge Bias: The evaluation relies heavily on LLM-as-a-judge, which may be susceptible to known biases. Human validation of correctness and safety remains a necessary next step.
- Longitudinal Fidelity: While the system shows promise in offline tasks, it remains to be seen if "continuous distillation" truly preserves a creator's fidelity over long periods of real-world deployment.
- Privacy: Distilling creator traits from "activity elsewhere" requires careful governance to ensure that sensitive or private information isn't accidentally leaked into a public-facing agent.
Summary updated