Nursing homes are acoustically hostile to people with dementia in two opposite ways. Clattering carts and multi-talker noise overwhelm them, and long silences leave them disoriented and frightened. One promising response is "soundscape augmentation": small speakers play familiar, curated sounds such as rain, birdsong or distant cutlery, to give residents orienting cues and to mask disturbing noise. The best-tested system is a bedside player that learns each resident's preferences from caregivers pressing colored feedback buttons. It works only when staff press often. It also works poorly for people with severe dementia, and it starts every new resident from an essentially arbitrary sound library chosen by a handful of researchers. The obvious fix is to bring in structured expert consensus through a Delphi study, in which panelists rate items anonymously over several rounds and see the group's aggregate answers between rounds. Such a study would standardize which sounds suit which residents and moments. No such study exists in the material we reviewed.
Running that Delphi and publishing a taxonomy would be the wrong project, because the field already holds direct evidence that expert intuition about sounds fails. Six researchers rated 280 sounds and favored music, and the music agitated residents. Steady rain, which they overlooked, calmed them. This proposal therefore treats a Delphi panel's output as a prediction to be scored against what residents actually did. The consensus ratings become the starting preference estimate (the "prior") of the adaptive recommender. We then test, on held-out residents and held-out sounds, whether the consensus carries information beyond what the field feedback logs already contain. Inside the Delphi we also run a randomized experiment on its feedback step. Some panelists see only the group's statistics between rounds. Others also see real caregiver-feedback outcomes for a subset of sounds. We measure whether either kind of feedback makes the panel more accurate, or merely more uniform, on sounds it was never shown evidence for.
Core Claim
Our hypothesis is that a multidisciplinary Delphi panel weighted toward frontline caregivers, and shown field evidence between rounds, produces sound-by-context-by-resident-type suitability ratings that predict caregiver-rated resident responses better than the original researcher ratings. We expect the gain to be largest where field data are thinnest: severe dementia, night-time periods, and sounds the recommender has never played. Used as a prior, these ratings should reduce the number of caregiver button presses needed before the recommender settles on sounds that work. The claim is falsifiable on both counts. If a model trained only on the logged feedback and acoustic features matches the Delphi-informed prior, the consensus adds nothing for personalization. In that case its remaining value would be limited to safety bounds and settings with no data at all.
What the Two Directions Supply
The soundscape literature motivates the problem and supplies the ground truth. Soundscape augmentation for people with dementia built the 280-sound library and had six researchers rate each sound for 17 daily care activities. It tagged the library with a pretrained audio classifier covering the 527 AudioSet categories and deployed it in six homes. It also reported the failures that motivate us. Unfamiliar music did poorly, and sharp high-frequency sounds stood out against the low-frequency noise floor of care homes. Personalising augmented soundscapes for supporting persons with dementia supplies the method we build on. Its "Soundbox" player updates a per-sound score with learning rate 0.25 from a five-level caregiver button panel. Over 3,624 feedback events and 19 residents, it reports improvement in 68% of residents but only 40% in severe dementia, a correlation of 0.76 between feedback frequency and selection success, and 52% of feedback falling during sleep periods. Those last two numbers together say the system learns only where staff have time, and the hardest periods are exactly when staff are scarce.
The ethnographic work in The Influence of Everyday Acoustic Environments on the Challenging Behavior in Dementia and Designing Supportive Soundscapes for Nursing Home Residents with Dementia gives us four resident "personae." These are defined by whether a person can still interpret sounds and whether they can act on them, and we use them as the resident-type axis of the Delphi items. That work also constrains us. The same papers note that high-pitched sounds which cut through noise can startle people with heightened sound sensitivity, so a ranking for "stand-out" sounds needs consensus safety bounds alongside it. The pilot trial Effect of Soundscape Augmentation on Behavioral Symptoms in People With Dementia found no group difference on the global Neuropsychiatric Inventory, where the interaction explained 1% of variance. It did find a reduction in the "resisting care" subscale of the Pittsburgh Agitation Scale, which is why our clinical outcome is episode-level rather than global.
The Delphi literature supplies the instrument and its known weaknesses. The VICCCS consensus study used a 67% agreement threshold to fix vascular dementia terminology. VISION-Cog used Lawshe's content validity ratio, a per-item score of how many panelists call an item essential, with a five-expert panel to strip visual confounds from a cognitive battery. Both show that Delphi methods work in dementia, but both produce static artifacts whose correctness is never checked against outcomes. Byrne et al.'s behavioral-trials Delphi ranked "tailoring interventions to populations and contexts" and "specifying active ingredients" as top priorities, which is the agenda we address. It also documents two weaknesses we must design around: attrition from 77 to 57 panelists, and over-representation of academics. The NoGAPS project shows that a Delphi involving end-users can turn laboratory evidence into a usable field program. Our panel composition borrows from it. Finally, work on deliberation such as AI Agents and the Future of Deliberation raises the concern that Delphi convergence can reflect conformity rather than agreement. Scoring each round against field outcomes is how we tell the two apart.
Research Program
The project has three parts.
The first part assembles a scored evaluation set from existing deployments. This depends on a collaboration with the Soundbox and soundscape-augmentation groups to obtain the logged feedback events with sound, resident, time and context identifiers. If those logs cannot be shared, a new deployment of the same hardware in a comparable number of homes would rebuild them. For each sound and context pair, and for each resident's cognitive stage and persona, we compute the observed mean caregiver rating and its uncertainty. We also attach acoustic descriptors to every sound in the library: the audio classifier's tags, spectral centroid, and loudness, sharpness, roughness and fluctuation strength. This gives a purely data-driven competitor something to learn from.
The second part is the Delphi itself. We recruit roughly fifty panelists and require at least half to be frontline care staff or family caregivers. The rest are audiologists, environmental acousticians, geriatric psychiatrists and music therapists, recruited across several countries to counter the bias Byrne et al. document. Rating all 280 sounds across 17 activities and four personae is impossible. Instead we pick about sixty sounds that span the library's acoustic clusters, collapse the activities into five context groups (waking, meals, care tasks such as bathing, afternoon rest, night), and ask for 1–9 suitability ratings per sound, context and persona. Panelists also rate statements about psychoacoustic safety bounds, for example maximum sharpness, or playback level relative to ambient noise.
The sounds are split in advance into an "evidence" half and a "held-out" half. After round one, panelists are randomized into two feedback arms. One arm sees standard controlled feedback: medians, interquartile ranges and anonymized rationales. The other arm sees the same plus summaries of real caregiver outcomes for the evidence half only. A third round repeats this. Consensus is reported with content validity ratios and with round-to-round stability. We do not impose a single arbitrary cutoff.
The third part turns the ratings into a recommender prior and tests it. A rating converts to an initial per-sound score by a fixed linear map, chosen so that the Soundbox update rule runs unchanged from that starting point. We also fit a hierarchical model that predicts caregiver ratings from acoustic features, persona and context, with and without the Delphi score as an extra input. This tells us whether the consensus adds information beyond features the data could learn from.
Baselines
The naive reference point is the uniform prior. Every library sound starts with the same score, the recommender learns only from button presses, and the offline predictor guesses each resident's mean rating. Every other method must clear it. A Delphi prior that cannot is worthless.
The strongest existing method is the combination actually deployed. Its starting point is the six-researcher activity ratings of Soundscape augmentation for people with dementia, adapted online by the Soundbox score update of Personalising augmented soundscapes for supporting persons with dementia, which reports 68% of residents improved, 40% in severe dementia, and r = 0.76 between feedback volume and success. We found no reported result on Lacuna for how well the researcher ratings predict logged caregiver responses, so we compute that number ourselves under the same protocol as every other prior. The comparison is fair by construction. Every prior is mapped to initial scores by the same function, replayed through the identical update rule with learning rate 0.25, and evaluated leave-one-resident-out and leave-one-sound-cluster-out on the same logs. The data-driven competitors get the same tuning budget, with hyperparameters set by nested cross-validation inside the training folds.
The third baseline is the most dangerous. A data-only population model learns ratings from acoustic features, persona and context using the logs alone, with no expert input. Beating the deployed system means the Delphi prior gives lower held-out prediction error on caregiver ratings, and needs fewer simulated feedback events to reach the deployed system's final selection success. Beating the data-only model means that adding the Delphi score significantly improves held-out error. We look especially for that improvement on severe-dementia residents, night periods and unseen sound clusters.
Two ablations isolate the key ingredients. A single-round panel average with no iteration tests whether the Delphi's rounds matter at all. The randomized feedback arms test whether field evidence, rather than social feedback alone, is what improves accuracy on held-out sounds. A panel-composition ablation re-scores using frontline-only and academic-only subsets of the same panel.
Why not just…?
Why not just survey caregivers once and average their answers? Averaging many independent judgments often captures most of what iteration adds, and it avoids the attrition that took Byrne et al.'s panel from 77 to 57. This is a real threat, and we test it directly. Round-one ratings are the single-survey baseline, scored against field outcomes exactly as later rounds are. If later rounds narrow disagreement without lowering prediction error, the honest conclusion is that a one-shot frontline survey is the cheaper and equally good instrument. In that case only the safety-bound statements would justify multiple rounds.
Why not just let the data decide, by pooling feedback across residents? A population model fit to 3,624 events may already know that rain beats unfamiliar music. That model is our third baseline, and we expect it to win on sounds and residents well represented in the logs. The data, though, are thinnest exactly where the system fails. Severe dementia had 40% success, and success tracks feedback volume at r = 0.76. Sounds never played have no outcomes at all. The proposal stands or falls on whether the Delphi score improves held-out error in those strata. If it does not, we will report that expert consensus is redundant once modest field data exist.
Why not just hold a one-day workshop with experts and staff? It is cheaper, and the conversation is richer. But the curation failure documented in Soundscape augmentation for people with dementia came from a small group of researchers deliberating together. Face-to-face settings also mute the junior care staff whose observations, such as rain masking noise, turned out to be correct. We cannot afford to run a full workshop arm. Our panel-composition ablation addresses the underlying question: whose judgment predicts outcomes? If academic-only ratings predict as well as frontline ratings, the case for anonymity weakens.
Proposed Experiments
The decisive offline check is held-out prediction error on caregiver ratings. We compare the uniform prior, the researcher ratings, Delphi rounds one through three in both feedback arms, and the data-only model with and without Delphi scores. Results are stratified by dementia severity, time of day, and whether a sound fell in the evidence or held-out half. A clear success would look like this. Final-round ratings from the evidence-feedback arm beat the researcher ratings overall, beat the statistics-only arm on held-out sounds, and add significant predictive value to the data-only model in severe dementia and night periods. Calibration across rounds is the conformity test. If spread shrinks while error stays flat, convergence was social, not informative.
The second check is a simulated cold start. We replay each held-out resident's logged sequence through the Soundbox update from each prior and count the feedback events needed to match the deployed system's eventual selection success. A meaningful reduction would translate directly into fewer button presses for overstretched night staff.
Only if the offline checks pass do we run a small crossover pilot of Delphi-prior versus standard-prior Soundboxes in new residents. Its outcomes would be caregiver feedback volume to convergence, the Pittsburgh Agitation Scale resisting-care subscale, and nocturnal disruptions. This pilot is explicitly feasibility-scale, given that existing trials ran with 13 residents per group. Throughout, the safety-bound consensus statements set hard limits on playback and are reported separately as a design guideline. Their value does not depend on the predictive results.
References
- Soundscape augmentation for people with dementia
- Personalising augmented soundscapes for supporting persons with dementia
- The Influence of Everyday Acoustic Environments on the Challenging Behavior in Dementia: A Participatory Observation Study in Nursing Homes
- Designing Supportive Soundscapes for Nursing Home Residents with Dementia
- Effect of Soundscape Augmentation on Behavioral Symptoms in People With Dementia: A Pilot Randomized Controlled Trial
- The Vascular Impairment of Cognition Classification Consensus Study
- Feasibility, comprehensibility and acceptability of the VISION-Cog, a novel tool to assess cognitive impairment in visually impaired older adults: a cross-sectional pilot study in Singapore
- An international, Delphi consensus study to identify priorities for methodological research in behavioral trials in health research
- Bridging the gap between research evidence and real-world implementation: the nogaps project
- AI Agents and the Future of Deliberation: Designing Human-AI Collaboration for Democratic Dialogue