TL;DR
This paper introduces a transformer-based time series model designed to forecast the "next step" in a criminal court case, moving beyond traditional models that only predict final outcomes like recidivism. By analyzing sequences of proceedings—combining structured data with free-text clerk notes—the authors provide a tool for researchers and oversight bodies to audit the cumulative impact of judicial decisions in real-time.
What Problem Does This Paper Address?
The criminal legal system is a "chain of decisions," from initial bond settings to final sentencing. However, most machine learning research treats these cases as single-point events, focusing on terminal predictions like whether a defendant will reoffend. This "collapses" the complexity of a case and hides how individual decisions—like a series of continuances or a specific bond amount—accumulate to affect the final resolution.
Furthermore, court data is notoriously messy. What actually happens in a courtroom is often buried in unstructured, free-text "clerk minutes" rather than clean, machine-readable fields. There is a significant technical barrier to identifying disparities or auditing decision-making at the level of individual proceedings. The authors aim to bridge this gap by modeling court cases as event sequences.
Methods
The authors propose a multi-modal transformer architecture to handle the irregular and heterogeneous nature of court records.
- Proceeding Representation: Each step in a case is represented by a fused feature vector. This includes frozen BERT embeddings of clerk notes, defendant demographics, current charges, prior criminal history, and bond information.
- Handling Irregular Time: Unlike many time-series models that assume fixed intervals, court proceedings happen sporadically. The model captures this by concatenating a one-hot month-year vector to the features, alongside standard learned positional embeddings that track the ordinal step in the case.
- Multi-label Prediction: Since multiple things can happen in one hearing (e.g., bond is set and a public defender is appointed), the model predicts a multi-label vector across 20 possible outcome categories.
- Sequence Modeling: The transformer uses a causal mask to ensure that at any step $t$, it only "sees" information from proceedings up to that point to predict the outcome at $t+1$.
Datasets
The study uses a newly constructed dataset from the public court records of a mid-sized U.S. city.
- Scope: 2,533 cases and 19,459 individual proceedings.
- Complexity: Case lengths vary significantly, with a median of 7 proceedings and a maximum of 61.
- Labels: 20 outcome labels (e.g., "Charges Refused," "Pled Guilty," "Bond Set") were extracted from clerk notes using a validated regular expression pipeline.
Due to privacy and legal restrictions, the raw records cannot be redistributed. However, the authors have released their full modeling pipeline and a schema-identical synthetic dataset to allow other researchers to inspect and rerun their methods.
Key Insights and Ideas
- Text is the Primary Signal: Ablation studies revealed that the free-text clerk notes are the most critical input. Removing text caused a much larger drop in performance than removing demographics or prior labels.
- Legal History Over Demographics: Counterfactual sensitivity analysis showed the model is most responsive to "legally relevant" features like prior case counts. While it showed some sensitivity to race and age (consistent with known systemic disparities), the magnitude was lower than that of criminal history.
- Discovering Procedural Trajectories: By clustering the learned case representations, the authors identified distinct "paths" cases take, such as early refusals at preliminary hearings versus cases that proceed through a full adversarial trial process.
- Flagging Anomalies: The model can act as an "outlier detector." For instance, if the model predicts a Domestic Stay-Away Order with 99% confidence but none is issued, it can help court observers identify procedural gaps—like missing victim information—that might otherwise go unnoticed.
Key Results
- Predictive Performance: The transformer achieved a micro-F1 of .5715 on the 20-class multi-label task. This was a 215% improvement over a frequency-based random baseline (.1812) and outperformed a static MLP baseline in predicting terminal outcomes.
- Sensitivity Analysis: Prior case count had the highest impact on predictions ($\Delta P = .0032$), followed by age (.0017) and race (.0009). For example, the model correctly learned that defendants with high prior case counts are less likely to be released on their own recognizance.
Limitations and Future Work
- Noisy Ground Truth: The model relies on labels extracted via regex from clerk notes. If a clerk makes a typo or uses non-standard phrasing, the model inherits that noise.
- Jurisdiction Specificity: The patterns identified reflect the specific practices of one U.S. city. Applying this framework elsewhere would require local adaptation of the label extractors and retraining on local data.
- Semantic Hierarchy: Currently, the model treats different charges (e.g., "Possession" vs. "Possession of 2 Grams") as unrelated features. Future work could use hierarchical embeddings to better capture the relationships between different types of crimes.
- Fairness Audits: While the authors performed sensitivity analysis, they did not conduct a full disaggregated evaluation of error rates across different demographic groups, which remains a vital area for future research.
Summary updated