Label-Free Triage of Non-Research Records with Factored Outlier Pursuit

Scholarly indexes such as OpenAlex and Semantic Scholar are full of records that are not research. These include journal mastheads, conference front matter, memorial tributes, scraped repository metadata, and abstracts that turn out to be a stray introduction paragraph. A recent audit found that 12% of a sample of cited English journal articles in OpenAlex had abstract integrity problems, and the authors call this a lower bound (One in Eight OpenAlex Abstracts Has Integrity Issues). The same study shows that a carefully prompted large language model agrees with human experts 96.0% of the time. Running such a model over the 81.1M papers in S2ORC or all of OpenAlex is expensive, though. The practical question is how to spend a limited budget of expensive model calls so that it catches as many bad records as possible.

We propose a cheap, label-free first stage built on one structural observation. Genuine abstracts across a corpus are well described by a modest number of shared topical directions, which makes them approximately low-rank. Many kinds of junk are individually idiosyncratic: each bad record breaks that structure as a whole, rather than in a few entries. That is exactly the setting of Outlier Pursuit, which splits a matrix into a low-rank part plus a part where only a few whole columns are nonzero (Robust PCA via Outlier Pursuit). Outlier Pursuit has only ever been solved with full singular value decompositions on matrices of a few thousand columns. We make it run at corpus scale by writing the low-rank part as a product of two thin matrices, an approach called Burer–Monteiro factorization. We then test whether its per-record outlier score improves which records get sent to the expensive model. Our hypothesis is that it does, mainly when the corpus differs from whatever a supervised filter was trained on.

Core Claim

A factored, column-sparse decomposition of a document-feature matrix gives a per-record anomaly score with three properties. It needs no labels. It uses memory linear in the number of documents. It is complementary to a cheap supervised classifier distilled from LLM labels. We specifically claim that, at a fixed budget of LLM calls, routing records by a combination of the outlier score and the classifier recovers more contaminated records than routing by the classifier alone. We expect the gain to concentrate on records whose era, language, or citation status differs from the classifier's training data.

We also commit in advance to a prediction about where the method should fail. Templated junk such as the near-identical IEEE "publication information" pages (IEEE Transactions on Cybernetics publication information, IEEE Transactions on Fuzzy Systems publication information) and conference front matter is repetitive. It therefore forms its own low-rank cluster rather than sparse outlier columns. That violates the conditions of Outlier Pursuit, so the decomposition will absorb such records into the low-rank part. We do not try to fix this with algebra, because title patterns and near-duplicate hashing already catch template junk trivially. The low-rank method is aimed at the idiosyncratic categories in the audit's taxonomy: insufficient content (29.3% of flagged abstracts), bibliographic metadata (15.4%), scrape artifacts, and truncation. Wrong-section text (13.9%) is fluent scientific prose and may well look like an inlier. Whether it does is one of the things we measure.

Research Program

Let $M \in \mathbb{R}^{d \times n}$ hold one column per record. Each column contains $d$ features of the abstract: either sparse word-frequency weights or a dense sentence embedding, extended with a few structural counts such as length, reference count, and in-citation count. We solve

$$\min_{U \in \mathbb{R}^{d\times r},\, V \in \mathbb{R}^{n\times r},\, C} \; \tfrac12 \| W^{1/2}(M - UV^\top - C)\|_F^2 + \tfrac{\mu}{2}\big(\|U\|_F^2 + \|V\|_F^2\big) + \lambda \sum_{j} \|C_{:,j}\|_2 .$$

Here $r$ is a chosen rank. The Frobenius penalty on the factors is the standard factored stand-in for the nuclear norm used in convex Outlier Pursuit. The last term is the column-sparse penalty from Robust PCA via Outlier Pursuit. $W$ is a diagonal matrix of per-record noise weights. The score of record $j$ is $\|C_{:,j}\|_2$.

The algorithm alternates between two steps. Given the factors, $C$ has a closed-form update that shrinks each column toward zero. Given $C$, $U$ and $V$ are updated with minibatch gradient steps. Records stream through in batches, so we never hold an $n \times n$ object or perform a full decomposition. This reuses the factored-gradient idea of Dropping Convexity for Faster Semi-definite Optimization. and the variance-reduced stochastic version of Finding Global Optima in Nonconvex Stochastic Semidefinite Optimization with Variance Reduction. On ordinal embedding, the latter reached target error in 40 epochs, against 1600 for projected gradient descent.

Neither paper handles the non-smooth column penalty. The landscape guarantees of The non-convex Burer-Monteiro approach works on smooth semidefinite programs require smooth constraint sets, so we make no claim of global optimality. Instead we check empirically, on matrices small enough for the convex solver, how often the factored solution matches the convex Outlier Pursuit solution. Initialization dependence is a known weak point of factored methods. We use a spectral initialization on a random subsample and report sensitivity to random restarts.

The weights $W$ address a real mismatch. Abstracts from different disciplines and eras have very different variance, as shown for low-rank estimation in ALPCAH: Subspace Learning for Sample-Wise Heteroscedastic Data. Without reweighting, a whole noisy-but-legitimate field could be flagged. We estimate per-venue noise levels from residuals and refit, and we ablate this step.

Theory constrains the design in two places. The tight recovery result of Exact Recoverability of Robust PCA via Outlier Pursuit with Tight Recovery Bounds allows the outlier count to grow linearly with $n$. It also suggests the setting $\lambda = 1/\sqrt{\log n}$, which we use as a starting point; it was derived for the exact-equality convex problem, so we expect to rescale it. That result also assumes the clean subspace is incoherent, meaning its energy is spread across many records. Highly cited hub papers and burst vocabulary may violate this. We measure incoherence on real corpora rather than assume it.

The final product is a cascade. The free outlier score and a cheap classifier are combined into a single routing rank. The top fraction goes to the LLM, and everything else is accepted or rejected cheaply.

Baselines

The do-nothing reference treats every indexed record as research. Under the audit's sample this lets roughly 12% bad records through, and about 21.1% for pre-2000 records (One in Eight OpenAlex Abstracts Has Integrity Issues). A second naive reference is a rule filter that flags titles matching front-matter, masthead, committee, correction, and tribute patterns, plus abstracts under a length threshold. It costs nothing and should handle most template junk.

The strongest existing method is the calibrated LLM auditor from the same study, which reports 96.0% overall agreement with human consensus. We treat it as the ceiling and as the labeler: it fixes the accuracy target, and the question is how many of its calls are needed to approach it.

The strongest cheap competitor is a supervised classifier distilled from LLM labels: logistic regression or a linear SVM on word-frequency features, and a small fine-tuned encoder. Evidence from scholarly classification suggests such models are hard to beat. On REF 2014 theme classification, an SVM came in only marginally below a BERT ensemble whose micro F1 was 0.577 (Benchmark for Research Theme Classification of Scholarly Documents). We found no reported result on Lacuna for a lightweight classifier on the OpenAlex integrity sample, so we will establish one. All cheap methods see the same features and the same records, and get the same hyperparameter search budget. Any method that uses labels gets the same number of LLM-labeled training records.

Two unsupervised baselines isolate our key ingredient. Plain truncated PCA residual norm, at the same rank, tests whether the column-sparse term matters at all. Convex Outlier Pursuit on subsamples of up to a few thousand records, the regime of Robust PCA via Outlier Pursuit, tests whether factoring loses accuracy relative to the exact problem. Ablating the reweighting $W$ isolates the heteroscedasticity fix.

Beating the classifier is the bar that matters. Specifically, the combined router must recover more contaminated records than the classifier alone at a fixed LLM budget (for example 10% of records), by a margin outside bootstrap confidence intervals. It must do so at least on the shifted test sets described below.

Why not just…?

Why not just distill the LLM into a cheap classifier and skip the matrix machinery? This is the real threat. On in-distribution records we expect the classifier to match or beat our score, and the proposal is built to show this honestly rather than avoid it. Our bet rests on shift. Contamination differs sharply across eras: 21.1% before 2000 against about 6% for 2020–2025. A classifier trained mostly on modern records may miss old scrape artifacts. Our outlier score is fit on the target corpus itself and needs no labels from it. If the combined router shows no gain over the classifier even on the pre-2000 and uncited splits, we conclude that the low-rank stage is not worth deploying, and we report that.

Why not just use PCA reconstruction error? It is the obvious unsupervised score and costs the same. Its weakness is that a large share of outliers drags the principal subspace toward them, so contaminated records partly explain themselves. The column-sparse term exists to prevent this, and the tight bounds of Exact Recoverability of Robust PCA via Outlier Pursuit with Tight Recovery Bounds show it tolerates outlier fractions growing linearly with $n$. At a 6–21% contamination rate, the difference should be visible. If the PCA residual does as well, the column penalty is unnecessary and we will say so.

Why not just use title rules and near-duplicate hashing? For mastheads, front matter, committee pages, and errata, we think rules are sufficient. That is why we exclude template junk from the claim. Rules cannot see a record whose title is a normal paper title but whose abstract is a copyright line, a truncated fragment, or a repository record. Those are the categories the audit found most common, and the per-category results will show whether rules already cover them.

Proposed Experiments

The first experiment checks the solver on synthetic column-corrupted matrices of 500 to 3,000 columns, the range studied in Exact Recoverability of Robust PCA via Outlier Pursuit with Tight Recovery Bounds. We compare the factored solver with convex Outlier Pursuit on outlier identification and running time across rank and outlier fraction.

The main evaluation uses the audit's data from One in Eight OpenAlex Abstracts Has Integrity Issues. The 1,000-record human golden sample serves as the final test, and the 10,000 LLM-annotated records serve as training labels for the supervised baselines. If those samples cannot be obtained, we will draw a fresh sample under the same filters from the 11-11-2025 OpenAlex snapshot and label it with the same protocol. We fit the decomposition on a much larger unlabeled pool from the same snapshot. Metrics are area under the precision-recall curve per failure category, and recall of contaminated records at fixed LLM budgets of 5%, 10%, and 20%.

The decisive check is the shift experiment. We train the classifier only on 2020–2025 records and evaluate the classifier alone, the outlier score alone, and the combined router on pre-2000 records. We repeat this with uncited records and non-English records, which the audit's filters excluded. The proposal succeeds if the combined router wins on the shifted splits. It fails cleanly if it does not.

The second decisive check is the per-category breakdown. Our prediction is high outlier scores for metadata, scrape, and truncation records, and low scores for template junk and wrong-section text. A wrong prediction would tell us the inlier/outlier picture of junk is incorrect.

Finally, we run the solver over the full S2ORC metadata (S2ORC: The Semantic Scholar Open Research Corpus) to report wall-clock time, peak memory, and stability of the top-ranked records across restarts. We hand-inspect a random sample of top-scored records for previously unlisted junk genres, since discovering such genres is the one thing a label-free method can do that a distilled classifier cannot.