Correcting Confounded Price Logs Before Training Recommerce Pricing Agents

Recommerce firms are retailers that sell new goods, resell used goods, and buy used goods back from their own customers. They increasingly hand pricing to reinforcement-learning agents, which learn by trial and error. Those agents usually learn inside a simulator of customer behavior, and the simulator is fit to the firm's past sales and buyback records. Those records were produced by an earlier pricing rule that reacted to the market. Buyback offers rose when used stock ran low, and used-goods discounts fell when competitors were sold out. A simple regression on such records therefore mixes up "customers respond to price" with "prices respond to the situation." This proposal asks whether removing that mix-up during simulator fitting is what limits the agents. The correction uses propensity-score methods: statistical tools that compare like with like by modelling why each price was chosen.

The opportunity is concrete. Self-learning Agents for Recommerce Markets trains agents to set three prices: new, used, and buyback. Agents pre-trained in a simulator fit by ordinary least squares (OLS) on observable market data reached 94% of the performance of agents trained with perfect knowledge of the market. Nobody knows whether the remaining 6% comes from confounding in the price logs, from the wrong functional form, or from noise. These causes call for different fixes. A circular business depends on its buyback price drawing enough returns without starving new-product sales, so getting this right matters beyond the simulator.

Core Claim

The claim is testable and could fail. When historical prices were set by a pricing rule that reacted to observed market state, fitting the simulator's demand and return-supply curves with generalized propensity score adjustment should recover more accurate price sensitivities than OLS. That adjustment means weighting or subclassifying records by the estimated probability of the price level actually charged, given the state. Agents trained in the corrected simulator should then close a measurable part of the gap to the omniscient agent.

The adjustment cannot fix confounding the firm never recorded, so the proposal also measures how fast the benefit decays as unrecorded demand shocks grow. The novelty is not "apply PSM to circular economy." Existing bridges between the two literatures, such as the low-carbon pilot city PSM-DID study, estimate one policy effect on patents. The novelty here is treating causal adjustment as the calibration step of a sequential decision system, and judging it by downstream policy value rather than by a single effect estimate.

Research Program

The work has three parts.

The first part builds a confounded-log generator on top of the recommerce simulator from Self-learning Agents for Recommerce Markets. Ground-truth customer response is known inside this simulator. We run "historical" behavior policies, such as rule-based repricers and earlier trained agents with added noise, that set prices as functions of inventory, competitor prices, and a latent demand shock. Two dials control how much of the shock the logs record and how strongly the behavior policy reacts to it. This lets us sweep from fully observed confounding, where adjustment should help, to mostly unobserved confounding, where it should not.

The second part is the calibration method. We discretize each price into a small number of levels. The joint price setting becomes a multi-level treatment. Following Propensity Score Matching and Subclassification in Observational Studies with Multi-Level Treatments, we estimate the generalized propensity score $p(w \mid X)$. Here $w$ is the price level charged and $X$ is the recorded market state. We fit response curves within score subclasses after trimming records with no comparable counterparts.

The multi-level choice is deliberate. That paper shows pairwise binary matching can produce contradictory rankings, where $A > B$, $B > C$, and yet $C > A$, because each pair uses a different comparison population. A simulator built from contradictory pairwise elasticities would be internally inconsistent. The cross-effects matter most: a used-price cut steals new-product demand. To capture them, each subclass fits all three outcomes jointly: new sales, used sales, and returns.

The third part carries uncertainty through to training. Matching on the Estimated Propensity Score shows that treating the estimated score as known gives wrong variances, and that the standard bootstrap fails for nearest-neighbor matching. We therefore use its analytical variance correction for the matching variant. For the subclassification variant, we use a subclass-level resampling scheme that we validate by simulated coverage. The resulting intervals on each sensitivity define how widely we randomize simulator parameters during agent pre-training. The aim is to randomize only as much as the data actually leave uncertain.

Baselines

The naive reference point is the original pipeline: OLS calibration on the logs, followed by PPO pre-training. PPO, or proximal policy optimization, is the algorithm the source paper found most stable. Its reported result is 94% of omniscient performance Self-learning Agents for Recommerce Markets. We rerun it inside our generator so that every method sees identical logs.

The strongest competitor is a flexible outcome regression that includes the full recorded state as controls, implemented as gradient-boosted trees per outcome. If the state is recorded, this already handles confounding without any propensity model, so it is the real threat. We also run a doubly robust combination of the outcome model and the score weights. It is correct if either component is right and is the natural upper bound for this family. No reported result was found on Lacuna for either method in recommerce calibration.

The key ablation keeps everything fixed except the score. It runs subclassification on random rather than score-based strata, and pairwise binary matching instead of the multi-level score. This isolates whether gains come from the propensity model itself.

All methods get the same logs, the same simulator family, the same PPO budget and seeds, and the same hyperparameter search budget over a held-out log split. Beating a baseline means a higher fraction of omniscient profit in the true simulator at matched compute, with non-overlapping intervals across at least ten seeds. It also requires lower error in recovered price sensitivities, especially the new-versus-used cross-sensitivity.

Why not just…?

Why not just add the inventory and competitor state as controls in OLS? This is the most serious alternative. If the response is close to linear and the state is fully recorded, controlled OLS should match our method. The proposal expects an advantage only where behavior policies create poor overlap, for example when a repricer almost never offers high buybacks at high inventory. There, regression extrapolates and subclassification with trimming does not. The overlap dial in the generator settles this directly. If controlled boosting matches us across the sweep, we will report that the propensity step is unnecessary.

Why not just randomize a small share of prices and fit on that? Randomized price experiments remove confounding outright, and firms can run some. They are costly in lost margin and in disrupted buyback supply. We include a baseline that randomizes a fraction of logged decisions, ranging from 1% to 20%. We then report how much randomization the adjusted method is worth. A result like "matches a 5% experiment at zero experimental cost" is the practical claim. If even 1% randomization dominates, that is the finding.

Why not just randomize the simulator widely and let the agent be robust? Broad parameter randomization avoids committing to any calibration. It typically buys robustness at the cost of conservative pricing. The robustness premium observed for reverse-logistics design in the fuzzy robust optimization model, with cost rising from 1,769,715 to 1,882,260 yuan, is an analogue from a different problem. We compare wide uninformed randomization against randomization sized by our corrected intervals.

Proposed Experiments

The main experiment crosses three factors: the confounding-strength dial, the observed-fraction dial, and three market structures from the source simulator (monopoly, duopoly, and five-firm oligopoly). For each cell, we fit every calibration method and pre-train PPO. We then evaluate in the true simulator, reporting the fraction of omniscient profit, return volume, and new-product cannibalization.

The first decisive check targets the 6% gap. Under the source paper's conditions, give OLS the correct functional form. If its agents then reach near 100%, the gap is functional form, not confounding, and the core claim narrows to confounded regimes only. If OLS with the right form still falls short and our method closes the gap, confounding is the cause.

The second check is the unobserved-confounding decay curve. The advantage of our method should shrink toward zero as the recorded fraction of the demand shock falls. Gains that persisted at zero recorded confounding would signal a bug or an artifact.

The third check is interval coverage. Corrected intervals on price sensitivities should reach nominal coverage across replications, as Matching on the Estimated Propensity Score predicts for its adjusted estimator, while naive intervals should not.

Validation on real recommerce logs is outside what this testbed can show. It is the obvious next step once the simulator results say which fix is worth deploying.