The Human as Teacher: Offline Confidence Cascades for Field Data Entry

Survey workers in rural clinics, disaster zones and remote villages often capture data on phones with no reliable internet. In one Metro Manila health survey, connectivity was missing a quarter of the time and surveyors typed at 11.5 words per minute. Language models could draft these records for the worker to confirm, since reviewing a draft is much faster than writing from scratch. The problem is that the existing ways of making models cheap assume a large cloud model is always available as a fallback. We propose to replace that cloud fallback with the person already holding the phone.

The system is a three-level cascade that runs entirely on the device. A tiny nearest-neighbour classifier is tried first. A small compressed language model, prompted with few examples, handles the cases the classifier is unsure about. The field worker handles whatever the model is unsure about. Every correction the worker makes is stored locally, where it becomes a training example for the tiny classifier and an in-context example for the model. The claim is that this cascade, tuned for human seconds plus device energy per correct record, beats both manual entry and "upload later and process in the cloud." We also expect the best compression level to be set by a failure mode that benchmark accuracy cannot see: once model drafts get bad enough, reviewers either slow down or start approving errors.

Core Claim

The closest existing system is OCaTS. It routes queries between a cheap local student (a k-nearest-neighbour or small neural classifier over sentence embeddings) and GPT-4, using distance and entropy filters. On Banking77 it cuts GPT-4 calls by 34% at 82.68% accuracy, against 83.05% for the teacher alone. OCaTS assumes the teacher answers whenever the student hesitates, and it is weakest at cold start, which is exactly when a new field project begins. We keep its routing idea and change two things.

The first change is the teacher. A human reviewer is slower but more reliable than GPT-4. The human is also the only source of ground truth at the point of capture, because a participant cannot be re-asked after the surveyor leaves.

The second change is that a quantized on-device model sits between student and human. Its job is to cover the cold-start period before the student has seen enough corrections. We treat the model's bit-width and prompt length as parameters to tune, not fixed choices.

The objective is the expected cost per correct record:

$$C = \frac{\mathbb{E}[T_h] + \lambda\,\mathbb{E}[E_d]}{1 - \epsilon_{acc}}$$

The symbols are defined as follows. $T_h$ is reviewer time per record. $E_d$ is device energy per record. $\lambda$ converts joules to seconds and should reflect how scarce battery is in the field. $\epsilon_{acc}$ is the rate of wrong records that end up in the final dataset, including drafts the reviewer wrongly approved.

The $\epsilon_{acc}$ term carries the paper's main argument. A heavily compressed model can make $T_h$ look small, because reviewers accept quickly, while quietly inflating accepted errors. BrokenMath reports a related effect: models produce convincing justifications for flawed premises, and people reviewing plausible text are exposed to the same pull.

Research Program

We would build an offline-first data-entry client. It follows the local-write, sync-later pattern validated by the Manila deployment (Use of Mobile Forms in Low-Resource Areas) and by the offline Atekho collector in the LiFE Suite. The LiFE Suite still sends annotation and model work to the cloud; our client does both on the device.

The worker enters a short free-text or dictated note. The cascade then proposes structured outputs, either a category label (intent-style) or tagged spans (entity-style).

The first level is an embedding k-nearest-neighbour store, which starts empty and fills with confirmed records. The second level is a 1–3B instruction model compressed to 4, 3 or 2 bits. For the lowest bit-widths we use TACQ, which keeps task-critical weights at 16 bits and compresses the rest. For the KV cache (the stored attention keys and values that grow with prompt length) we use activation-aware low-rank compression from ASVD. This matters because the prompt fills with retrieved corrections and the KV cache grows with it. The third level is the human. Low-confidence items are shown to the worker as a pre-filled draft; high-confidence items are auto-filled but can still be tapped to edit.

Three analyses matter most.

The first is a calibration study, and everything downstream depends on it. We check whether the model's output entropy still separates right drafts from wrong ones after 2- and 3-bit compression. EHRSQL shows that entropy thresholding can flag unanswerable clinical queries at over 90% answerability F1, but only for uncompressed models. If compression destroys this signal, routing fails. In that case we fall back to per-bit-width threshold recalibration using the growing pool of human-confirmed records.

The second is a reviewer-behaviour model. It measures accept time, edit time, from-scratch time and false-accept rate as functions of draft quality. We fit it on a small user study and then use it in large-scale simulation. Bae et al. found that filtering LLM dialogues was 13.3 times faster than writing them from scratch. That figure is for good drafts from a very large cloud model, and we expect the ratio to shrink, perhaps below one, as drafts degrade.

The third is prompt-cost accounting for span tasks. ToPro shows that asking about each token separately ("the tag of [token] is a [MASK]") helps cross-lingual labelling. Its zero-shot in-context F1 with mt0-xxl was 18.09, where earlier methods scored 0.0, but it is about 4 times slower than fine-tuning. On a phone that cost may not be affordable. We compare it with a single-pass tagging prompt under a fixed per-record latency budget.

Baselines

The naive reference point is manual entry: the worker types every field with no model assistance. It sets the $T_h$ that everything else must beat. The Manila typing speed of 11.5 words per minute suggests it is expensive.

The second reference is store-and-forward to the cloud. Notes are saved locally and processed by a large cloud model after reconnection. A remote annotator reviews the model's output against the note alone, without access to the participant. This isolates the value of confirming records at the point of capture.

The strongest existing method is OCaTS, run exactly as published but with an outage schedule: the teacher is unreachable a set fraction of the time, 25% following the Manila measurement, and 100% for fully disconnected sites. When the teacher is unreachable, low-confidence items are queued. The paper's reported figure is 82.68% accuracy on Banking77 with 34% fewer GPT-4 calls, against a teacher at 83.05%. We will reproduce this with the connection always available before adding outages.

To beat OCaTS, the cascade must reach equal or better final-record accuracy on the same Banking77 stream, with the same initial examples per class and the same embedding model. It must also have lower $C$ at every outage level above zero, and no worse $C$ with full connectivity. All systems get the same cold-start examples, the same hyperparameter-search budget over thresholds, and the same simulated reviewer.

Three ablations isolate the key ingredients. The first removes the middle model, leaving student plus human only. This shows whether the compressed model earns its battery cost or whether a cold-start student plus a patient human already suffices. The second removes correction reuse: corrections are no longer fed back as examples. The third routes on raw entropy instead of recalibrated entropy, to measure how much compression-induced miscalibration costs.

Why not just…?

Why not just store everything and run a frontier model in the cloud later? For records that can be fixed later, this may well win. We say so openly and measure it with the store-and-forward baseline. The argument for local processing rests on two things. Omissions and ambiguities cannot be resolved after leaving the participant, as the point-of-care triage system of Michalowski et al. was designed around. And raw sensitive text may not leave the device, the constraint that led AWARE-TEXT to process text locally. The experiment reports the gap in final error between on-site and off-site review. If that gap is negligible for a task, local processing is not justified for that task.

Why not just train a small classifier from the corrections and skip the language model? This is the most serious threat, and it is included as the "student plus human only" ablation. OCaTS shows that a k-nearest-neighbour student can carry much of the load once it has data. We expect the compressed model to matter mainly over the first few hundred records and for unfamiliar inputs such as new languages. That expectation follows from ToPro's evidence that prompting transfers where fine-tuned heads fail across languages. If the crossover point turns out to come very early, the honest conclusion is to ship the model only for bootstrapping, and we would report that.

Why not just use standard 4-bit quantization? It may be enough. A 4-bit model of 1–3B parameters often fits on a phone, and TACQ's gains are concentrated at 2 bits, where it beats SliM-LLM by an average of 14.74%. We sweep bit-width rather than assume ultra-low bits are needed. TACQ and ASVD are included only in case 2–3 bits turn out to be required by battery or memory limits on low-end phones.

Why not drop the human and let the model judge itself? In the dialogue pipeline of Bae et al., raw in-context generation violated persona and policy rules 35.8% of the time until human filtering and training reduced this to 2.0%. For field records, a model with no human check also has no path to ground truth. We include a variant that uses the model's own confidence alone, with no human level, to quantify the final error it would admit.

Proposed Experiments

The simulation experiments use two data streams. The first is Banking77, standing in for categorical form fields. The second is PAN-X from XTREME, standing in for span extraction in languages the model saw little of in training. Each stream is ordered as a single field deployment and given connectivity schedules of 0%, 25% and 100% outage. Reviewer cost comes from the fitted behaviour model. For each system we report $C$, final-record accuracy, false-accept rate, device energy measured on two phones (low-end and mid-range Android), and $C$ as a function of records seen.

The user study recruits surveyors to process a few hundred records under drafts from models at 16, 4, 3 and 2 bits, plus a no-draft condition. The records include a planted set of subtly wrong drafts, so that we can measure false acceptance directly.

The decisive checks are four. First, is there a bit-width at which false-accept rate rises while reviewer time falls? That result would show benchmark accuracy is the wrong criterion for choosing compression. Second, does compressed-model entropy keep enough ranking power to route traffic, measured as AUROC for predicting draft error at each bit-width? Third, does the full cascade beat OCaTS under outage? Fourth, does the cascade beat student plus human over the first 500 records? If the third check fails, the proposal reduces to "queue and wait." If the fourth fails, the language model is unnecessary for these tasks.