TL;DR
Project Progress Desk is a centralized dashboard designed to manage the cognitive load of running dozens of AI coding agents (like Claude Code or Codex) simultaneously. By using a "receipt" system to link agent outcomes to verifiable evidence and weighted checklists, it provides developers with a clear, at-a-glance view of confirmed progress, blockers, and pending decisions across multiple projects.
What Problem Does This Paper Address?
As AI coding agents become more autonomous, developers are shifting from writing code to overseeing fleets of agents working in parallel. However, this creates a massive fragmentation problem. Project state is scattered across terminal sessions, chat histories, and different AI providers.
When a developer returns to their desk after a few hours (or days), they face a high cognitive cost to reconstruct what each agent did, which projects are blocked by errors (like expired API tokens), and where their manual intervention is required. Existing tools like Jira or GitHub Projects are often too heavy or disconnected from the raw agent sessions to provide the real-time, granular oversight needed for agentic workflows.
Methods
The authors developed a local Windows desktop application that aggregates data from various agent sessions into a unified "One Screen" overview. Key technical components include:
- Weighted Acceptance Checklists: Each project has a goal and a checklist where items are assigned weights (e.g., "Baselines" might be 30% of the total goal). Progress is calculated as $P = \sum w_i f_i$, where $w_i$ is the weight and $f_i$ is the completed fraction.
- Agent Receipts: This is the core innovation. Agents report their work using a JSON-based "receipt" (see Figure 3, Codex receipt example). These receipts link outcomes to specific checklist items, provide paths to evidence files (with SHA256 hashes for verification), and suggest the "next step."
- Progress Visualization: The dashboard distinguishes between "Confirmed" progress (dark bars supported by receipts) and "Estimated" progress (light bars based on agent guesses).
- Approach Branching: To help with decision-making, the "Details View" visualizes different implementation attempts as branches (see Figure 2, Approach branching visualization). This allows users to compare the success rates of different strategies—like "soft contact" vs. "rigid contact" in a robotics simulation—before choosing which path to continue.
Datasets
The paper does not use a traditional machine learning training dataset. Instead, it evaluates the system through:
- Daily Use Case: Observations from ~30 lab members using the tool to manage 36 active research projects.
- Simulated Demo Data: A set of 30 simulated projects used for the system demonstration to showcase real-time updates and blocker identification.
Key Insights and Ideas
- Evidence-Based Progress: The shift from "trusting the agent's chat summary" to "verifying the agent's receipt" is crucial. By requiring links to evidence files, the system reduces the risk of being misled by agent hallucinations regarding project status.
- Visualizing Uncertainty: By explicitly separating confirmed work from dated estimates in the progress bar (Figure 1, Overview dashboard), the tool provides a "honesty" layer that traditional progress bars lack.
- Surfacing the "Human-in-the-Loop" Moments: Rather than trying to automate everything, the Desk focuses on identifying where the human is needed. Status colors and attention counters (A5 in Figure 1) highlight projects that are "Blocked" or "Need You," allowing developers to prioritize their time effectively.
Key Results
- Scalability: The dashboard successfully fits the status of 36 projects on a single laptop screen, enabling a "scan" rather than a "search" workflow.
- Real-Time Feedback: The system updates the global overview within one minute of an agent publishing a new receipt.
- Improved Decision Making: In one hardware project, the approach history allowed the team to quickly identify that one implementation met timing requirements while another failed, leading to a faster integration choice.
Limitations and Future Work
The authors acknowledge that the current system was developed through "autobiographical design" (building for themselves), which may introduce bias. Future work includes:
- Controlled Comparisons: Testing the Desk against conventional project tables to quantify how much faster users can identify blocked tasks.
- Broader Deployment: Moving beyond the initial lab environment to see how the tool handles diverse engineering workflows.
- Integration: While it currently supports Claude Code and Codex, expanding to a wider variety of agent frameworks remains a goal.
Summary updated