Compiling Urban Sensing Goals into Checked Task Graphs

City agencies increasingly ask for things like "find out whether the potholes on Corridor B are getting worse after the rains," but crowd-sensing systems only understand a much narrower request: "send someone to point X at time t." Someone still has to translate the goal into concrete sensing jobs. Some data can be gathered passively, for example by vibration traces from phones or particulate sensors mounted on buses. Some needs a person to act, such as taking a photo of a crack. And some jobs should only start once another has found something, like dispatching a photo request only after a bus sensor reports a spike. Today engineers write this translation by hand, one objective at a time. We propose a system in which several language-model agents with separate roles compile a plain-language monitoring goal into a small, typed, executable dependency graph of sensing subtasks. One agent drafts the graph. A critic does not argue with it: the critic runs the draft in a simulator against the sensor catalog, the worker pool, and the budget, and sends back measured failures. Once the graph is accepted, it runs with no language model in the loop, using classical planners and statistical and vision tools.

The central bet is that ideas from multi-agent LLM work, namely separating planning from execution and giving the verifier its own independent check, pay off in physical sensing only if two conditions hold. The agents must work at compile time and never on the live, latency-critical path. The critic's judgment must rest on simulation rather than on conversation. If this holds, new monitoring objectives could be served without hand-authoring each pipeline, while keeping the timing and safety properties that sensing systems require.

Core Claim

A grounded generate-then-check loop can turn unseen natural-language sensing objectives into spatiotemporal task graphs that come close to the value of expert-written graphs. Its critic feeds back simulator output: coverage gaps, budget overruns, and dependency deadlines that cannot be met. We expect this loop to beat both a single LLM writing the graph in one pass and a loop whose critic is just another LLM reading the plan. A "task graph" here means nodes typed by modality, with dependency edges carrying spatial and temporal windows. The node types are opportunistic sweep, participatory inspection, cross-modal confirmation, and calibration check. A typical edge reads "inspect within 2 hours and 500 m of any confirmed spike."

This differs from the closest existing work in a specific way. AgentSense pairs a classical dynamic-programming allocator with LLM agents (a disturbance parser, a solver, an evaluator and a memory) that refine worker routes at runtime. It treats each sensing task as an atomic, pre-specified visit and trusts the incoming data. We leave allocation to the classical planner and put the agents one level up, where they decide what the subtasks are, how they depend on each other, and which modality must independently confirm which claim.

PBCT decomposes text crowdsourcing requests by detecting event triggers. Its authors state that it cannot represent concurrent events or temporal dependencies between subtasks, and it was tested only on text corpora. Our graph representation makes exactly those dependencies first-class. The classical allocators HyTasker and TaskMe handle the remaining part of the problem, assigning workers to spatiotemporal points, and we reuse them as the execution layer.

Why the design takes this shape

The multi-agent literature motivates role separation. LEGO splits causal explanation into analysts, an explainer and a critic, reaching F1 0.666 on WIKIWHY against 0.631 for ReAct. AgentVerse reports 89.0% pass@1 on HumanEval for a group against 83.5% for a single agent using chain-of-thought. Planner/executor splits in MACM and DRC-Coder reduce hallucinated steps. Tester roles in Self-collaboration and ReChisel work because the tester executes the artifact.

Other evidence constrains the design. AgentVerse also found GPT-3.5 groups swayed by wrong peer suggestions, to the point of doing worse than solo agents. The multi-agent survey documents hallucinations cascading through a group and growing token cost. On the sensing side, MyShake needs detection within seconds even though phone clocks drift by up to 2.5 s. These two findings together rule out agents in the runtime loop and rule out a conversational critic. The successful testers in Self-collaboration and ReChisel point the same way, because what made them useful was execution.

The sensing literature supplies the tools that run at execution time. One is the opportunistic calibration from OpenSense Zurich, which recalibrates a mobile sensor whenever its tram passes a reference station and so reduces ozone error from ±20 to ±2 ppb. Another is the field interpolation from AirDelhi. Road damage is handled by the detector trained on RDD2020 together with the metric measurement pipeline of Iqbal et al., which turns detected damage into physical lengths and areas, and by RoadCLIP.

Research Program

We will build three things. The first is a typed graph language. Each node names a modality, a tool, a region and a time window, and has a budget. Each edge carries a trigger condition and a spatiotemporal window. A compiler lowers the graph into the inputs a classical allocator expects.

The second is the compile loop. A Decomposer agent drafts a graph from the objective and a catalog of available sensors and tools. A Critic then executes the draft in a simulator and returns structured diagnostics: which events were missed, which dependency deadlines were violated, how far the budget was exceeded, and which claims rest on a single modality. The Decomposer revises the draft. The loop runs for at most K rounds.

The third is the simulator, which we call SenseSim. It is assembled from the datasets in the file. Worker mobility comes from T-Drive and Grab-Posisi trajectories, the same datasets AgentSense used. Particulate fields and the bias of fixed bus routes come from AirDelhi. Sensor drift comes from OpenSense. RDD2020 images are geolocated onto road segments to stand in for ground-truth damage, so that a simulated "photo task" returns a real image that a real detector then scores. Injected events, such as pollution episodes and damage growth after a storm, provide ground truth.

The objectives come from a held-out set of roughly 200 monitoring goals written by people other than the system designers, split by objective family. That split lets us test transfer to kinds of goals the system has not seen. For a 40-goal subset, domain practitioners will write expert graphs.

Baselines

The naive reference point is uniform coverage. The classical allocator spreads the whole budget evenly across the region and modalities, with no dependency structure. It shows how much value structure adds at all.

The strongest structural competitor is the expert-authored graph. For each objective in the 40-goal subset, a practitioner writes a graph in the same language, which is then executed by the same allocator and tools. It is effectively an upper bound for known objectives. We count success as reaching at least 90% of its detection utility on held-out families.

We also run a template-retrieval baseline. Expert graphs from the training families are stored, and the closest one is retrieved and has its parameters filled in for each new objective. This tests whether generation adds anything beyond lookup.

To measure the gain from moving the agents up a level, we reimplement an AgentSense-style baseline. A dynamic-programming plan over a fixed, atomic task list for each objective is refined by runtime LLM agents. We give it the same backbone model and the same token budget per objective, using only details stated in the paper. The file reports no numbers for AgentSense on T-Drive or Grab-Posisi, so "no reported result found on Lacuna for AgentSense Improvement Rate/Cost Savings" applies, and we will compare only within our own runs.

A single-agent compiler writes the graph in one pass with ReAct-style tool access. It is matched on total tokens.

The ablation that isolates our key ingredient is an LLM critic that reads the draft graph and critiques it in words but never runs the simulator. It is matched on rounds and tokens.

For the field-estimation component, the reference result is Random Forest interpolation on AirDelhi, with RMSE 22.24 as reported in the AirDelhi paper. Inside SenseSim, the opportunistic nodes must not do worse than this. For the vision component, the reference is RoadCLIP's 78.6% zero-shot accuracy on RoadBench. We use that detector as a fixed tool, so detector quality is held constant across all methods.

All methods share the same worker pool, events, tools, model backbone, and a tuning budget of 20 development objectives.

Why not just…?

Why not just use a classical optimizer with hand-written trigger rules? For objectives someone has already engineered, this is a real threat, and that is why the expert-graph and template-retrieval baselines exist. The claim we are making concerns new objectives. HyTasker and TaskMe treat tasks as fixed coordinate tuples and contain no mechanism for producing new dependency structure. If template retrieval comes within a few points of the compile loop on held-out families, we would conclude that a library of templates is enough, and we will report that outcome.

Why not just let one LLM write the plan? LEGO and AgentVerse report single-agent deficits only on text and code. Whether those deficits carry over to spatiotemporal plans is unknown. The single-agent compiler baseline, matched on tokens, settles the question directly. If it matches the loop, the multi-agent structure adds nothing here.

Why not just put LLM agents in the runtime loop, as AgentSense does? MyShake's timing requirements leave no room for multi-second calls to a language model. The conformity failures reported in AgentVerse also make runtime agent groups risky. We will measure the latency of decisions and the number of invalid route edits in the AgentSense-style baseline under the same disturbances.

Why not just have a critic agent verify the raw sensor readings? For numerical drift, statistical calibration such as OpenSense's ±2 ppb is very hard to beat, so we do not ask an LLM to do it. Our critic checks the structure of the plan, for example whether a claim gets confirmed by an independent modality. Checking individual data values stays with the statistical tools.

Proposed Experiments

The main experiment measures, for each method on held-out objective families, event detection recall and precision at a fixed budget. It also records median time from an event to an independent cross-modal confirmation, the rate of photo requests sent where no event exists, and the fraction of graphs that are valid and run.

The decisive check is the simulator-backed critic against the conversational critic. If simulation-grounded feedback does not beat talk-only feedback by a clear margin, the core claim fails. Our threshold is at least 5 points of recall at equal false-dispatch rate, with the difference holding across random seeds.

Next, we stress-test SenseSim with disturbances: worker dropout, roadblocks, a weather shift, and sensor drift. We then measure how much each compiled graph degrades without being recompiled.

Finally, we run the compiled graphs against a version of SenseSim with different city parameters. This guards against the possibility that the loop has simply overfit the simulator it was checked against.