Counting Concurrent Sounds to Nowcast Neighborhood Economic Activity

Information about how a neighborhood is doing economically arrives slowly. Censuses come every five to ten years, and street-level imagery, the best visual proxy for local wealth, is refreshed only every year or few years. Urban microphones, by contrast, listen around the clock and cost little: the SONYC network in New York used $80 sensors that report which kinds of sounds they hear. A construction boom, a commercial corridor coming back to life, or a delivery economy expanding all change what a street sounds like, week by week. So far, though, urban sound recognition has asked only whether a sound type is present in a 10-second clip. That is a yes/no answer, and it saturates. One power saw and six power saws working at once produce the same label, even though they mean very different amounts of economic activity.

This proposal claims that the useful quantity for economic nowcasting (estimating the present state of a slowly reported indicator) is how many same-type sources are active at once, and for how long. Measuring this calls for polyphonic sound event detection: models that find when each sound starts and stops, even when sounds overlap. It specifically calls for the track-based variant, which can represent two sources of the same class at the same time. We will build a single-microphone, track-based counter of concurrent urban sources. We will aggregate its output into weekly activity-intensity indices for each sensor. Then we will test whether these indices track and anticipate administrative records of local economic activity better than clip-level tags, sound-level meters, or imagery. Clip-level tagging is the honest null hypothesis. If counting adds nothing over tag frequencies, that result is worth knowing too.

Core Claim and How It Differs From Prior Work

Two literatures sit on either side of this gap. The socioeconomic-sensing work estimates indicators from what a place looks like. Fine-Grained Car Detection for Visual Census Estimation classified 50 million street images by car make and year and correlated the results with tract income. CityLens benchmarked vision-language models on 11 indicators and found that fine-tuned models reach R² above 0.6 for GDP, but that street imagery is the strongest signal. MuseCL fuses imagery, mobility and point-of-interest text through contrastive learning. CityPulse detects change from street-view time series, but it can only run as often as the imagery is recaptured. These papers motivate the proposal, because they establish both that ground-level sensing carries socioeconomic signal and that its revisit rate limits it. None of them uses sound.

The urban-audio work runs the arrow the other way. SONYC-UST and SONYC-UST-V2 define multilabel tagging over 10-second clips, with an 8-coarse/23-fine taxonomy based on the noise code. Bai et al. use location and time as inputs to improve tagging. SONYC ties detections to enforcement and found acoustic evidence for 94% of after-hours construction complaints, against a 2% violation rate from inspections. None of these papers treats audio as a predictor of economic conditions, and none of them separates overlapping sources of the same class.

The method comes from polyphonic detection. Event-Independent Network predicts a fixed number of "tracks." Each track can carry any class, so two simultaneous saws occupy two tracks. Because track order is arbitrary, it trains with permutation-invariant training: the loss is computed under every assignment of tracks to ground-truth sources, and only the best assignment is used for the update. ACCDOA shows that detection and localization can share one regression target, reaching 7.9° localization error on TAU 2020. ACCDOA has one output vector per class, so on its own it cannot count same-class sources, and it needs a microphone array that SONYC's single MEMS (micro-electromechanical) microphone does not provide. We therefore reuse the track-and-permutation idea from Cao et al. without the direction-of-arrival head. ACCDOA informs us mainly as a constraint: the class-wise designs that dominate localization benchmarks are the wrong shape for counting.

Research Program

The first component is a counter. We adapt the event-independent architecture to single-channel log-mel input. It has K tracks, perhaps four, and each track emits a class from the SONYC fine taxonomy plus an activity flag at every frame. Strong labels with exact onsets and offsets are scarce, and clips with several sources of the same class are scarcer still. We will therefore train mostly on synthetic mixtures. These mix isolated urban source events, mined from confidently single-labeled SONYC-UST-V2 clips, at controlled counts, levels and overlaps over real background recordings from the same sensors. Synthetic training adds a gap between the training data and the field. To measure that gap, we will hand-annotate a few hundred real clips with per-class concurrent counts.

The second component turns detections into activity measures. For each sensor-week and each class, we compute three nested summaries of increasing richness. The first is clip-level presence frequency, which is what a tagger gives. The second is active minutes from frame-level detection. The third is concurrency-weighted source-minutes, meaning active minutes multiplied by the number of tracks simultaneously active for that class. We also compute these summaries separately for working and after-hours periods.

The third component is the latency test. With roughly 50 sensors, a cross-sectional regression of tract income on sound is underpowered and could be confounded by where sensors were placed. The main analysis is therefore within-sensor and over time. We ask whether a sensor's weekly index moves with, and ahead of, slowly reported local records of economic activity near that sensor. These records include construction and building activity records, business openings, and 311 complaint streams. Our method for picking which administrative series to use is to take those released with a known publication lag. Imagery will be used where its capture dates allow. A secondary, explicitly underpowered analysis relates long-run sensor averages to tract indicators, mainly to check that the signs make sense. The project depends on access to the continuous SONYC sensor archive rather than only the released clips. If that archive is unavailable, the fallback is the timestamped SONYC-UST-V2 clips, which are much sparser per node and would support only coarse monthly analyses.

Baselines

The naive reference point is persistence. The current-week estimate of each administrative indicator is set to its last published value, or to its value in the same week of the previous year. Any acoustic index that cannot beat persistence on out-of-sample nowcasting error has no practical value, however well it correlates in hindsight.

The second baseline is a sound-level index. It uses A-weighted equivalent level and the fraction of time spent 10 dB above background, the threshold SONYC used for its evidence. These are computed from the same audio and aggregated in the same weekly windows. This is what a city could deploy today without any machine learning. Sensing Urban Soundscapes and the Antwerp park study relied on indicators of this kind.

The strongest existing competitor is clip-level tag frequency from the best reported SONYC tagger. That tagger is the CNN9-Res system of Bai et al., which combines harmonic-percussive features, Mixup and a spatiotemporal context encoder, and reports macro-AUPRC 0.775 on SONYC-UST-V2, against 0.632 for the DCASE 2020 baseline. It is the strongest competitor because it already recognizes the right classes about as well as anything published. If counting helps, it must help beyond recognition. We will retrain it on the same SONYC-UST-V2 training split with the same tuning budget in GPU-hours as our counter. Both models will be fed identical sensor-weeks, and the downstream regressor will be identical: regularized linear regression with the same lag structure and the same cross-validation by time block.

The key ablation runs our own track model with its outputs collapsed to class presence, then to active minutes. This separates the value of concurrency counting from any improvement in the detector itself.

The proposal beats the tagger if three things hold. Concurrency-weighted indices must reduce out-of-sample nowcasting error relative to tag frequency on held-out time blocks. The reduction must be consistent across sensors, assessed with a paired test over sensors. And the gain must be concentrated in high-activity sites, where saturation predicts it should appear. On the counting task itself, no reported result was found on Lacuna for urban same-class source counting, so the baseline there is the tagger's implied count, which is at most one per class.

Why not just…?

Why not just use calibrated decibel meters? Level meters cannot tell a passing bus from a jackhammer. The economic signal is in which sources are active, as the SONYC finding that 51% of complaints concerned after-hours construction suggests. Still, overall loudness may correlate with overall activity well enough for some indicators. The level-index baseline tests this directly. If it matches the learned indices, we will report that cheap meters suffice.

Why not just count clip-level tags from an existing tagger? This is the real threat. A tag fired every 10 seconds, summed over a week, is already a crude measure of duration. Counting will only matter where several same-class sources overlap often. We test the saturation claim in three ways. First, on annotated real clips we measure how often the true count exceeds one. Second, we check whether the gap in nowcasting error between the tagger and the counter grows with site activity. Third, the collapse ablation removes concurrency while holding the detector fixed. If same-class overlap turns out to be rare in SONYC audio, we will drop the track machinery and present frame-level duration as the contribution.

Why not just use street-view or satellite imagery? Imagery is strong for levels: CityLens reports R² above 0.6 for GDP after fine-tuning. Its weakness is revisit rate, not accuracy. Our question concerns change within months, which imagery captured years apart cannot resolve. Where dated imagery brackets a change, we compare when each modality first registers it.

Why not just use 311 complaints? 311 data is continuous and free, and we include it as a predictor. But it reflects who complains as much as what happens. SONYC documents this reporting bias and a 5-day inspection lag. We will test whether acoustic indices add predictive value beyond 311 counts, and whether the residuals of 311 relative to sound vary with neighborhood income.

Decisive Checks

The first check is mechanistic. On synthetic and hand-annotated mixtures, the track model must estimate the number of concurrent same-class sources better than the tagger's implied count. Otherwise the rest of the program is moot.

The second check is the main result. Under time-blocked cross-validation, concurrency-weighted weekly indices should beat persistence, the level index and tag frequency on out-of-sample error for administrative activity series. The gain should grow at busier sensors.

The third check is about latency. Cross-correlation between each index and each lagged administrative series should peak at a lead. Acoustic activity should be visible before the corresponding records are published, and we will report the lead in weeks, with confidence intervals bootstrapped over sensors.

A null result on the second check, with a success on the first, would show that counting is technically possible but economically redundant with tagging. That conclusion would also change how urban acoustic networks should be built.