A Dataset for Efforts Towards Achieving the Sustainable Development Goal of Safe Working Environments

Eirik Lund Flogard, Ole Jakob MengshoelNeurIPS 2022

Every year in the EU alone, poor working conditions lead to over 4,000 deaths and millions of serious accidents. To combat this, governmental agencies conduct labor inspections. However, these agencies have limited resources and must decide which of the thousands of businesses to visit and which specific regulations to check.

While machine learning (ML) could theoretically optimize this process—helping inspectors find "needles in the haystack" of non-compliant businesses—research has been stifled by a lack of high-quality, publicly available data. Most existing datasets are either proprietary or only include "positive" cases (where violations were found), making it impossible to train robust binary classifiers.

To address this gap, researchers have released the Labour Inspection Checklists Dataset (LICD), a first-of-its-kind public dataset containing over 63,000 Norwegian labor inspections. By providing 575 features and two target variables, the dataset enables machine learning research aimed at achieving UN Sustainable Development Goal 8: promoting safe and secure working environments for all.

The LICD Dataset

The core contribution of this work is the LICD, which spans 63,634 inspection instances from Norway between 2012 and 2019. Unlike previous datasets (such as US OSHA or Danish "Smiley" data), LICD includes both compliant and non-compliant cases, making it suitable for predictive modeling.

Dataset schema illustrating the relationship between inspections, checklists, and organizational features.

The dataset features 575 organizational and financial attributes (translated to English) and two primary targets: the Checklist ID (which set of rules was used) and a binary non-compliance label.

Methods and Baseline Tasks

The authors establish baselines for two primary tasks using a standard ML pipeline implemented in Scikit-learn:

  1. The Checklist Selection Problem (CLSP): A multi-class classification task where the model must predict which of the 369 available checklists is most relevant for a specific organization.
  2. The Non-compliance Classification Problem (NCP): A binary classification task to predict whether a violation will be found given a specific organization and a chosen checklist.

Because the dataset contains hundreds of features ranging from industry codes to detailed financial data, the authors placed a heavy emphasis on feature selection. They evaluated methods like Anova F, Mutual Information, and Recursive Feature Elimination to identify which organizational traits actually correlate with safety violations. They then tested a suite of classifiers, including Decision Trees (DT), Gradient Boosting, and Multi-layered Perceptrons (MLP).

Key Insights and Results

  • The Long-Tail Challenge: The usage of checklists follows a "long-tail" distribution. A few checklists are used constantly (e.g., for building construction), while hundreds are used rarely. This makes the Checklist Selection Problem (CLSP) particularly difficult for standard models.
Long-tail distribution of checklist usage across industries.
  • Counter-intuitive Features: Interestingly, for some feature selection methods, industry codes were not the top predictors. Instead, financial health indicators (like business age and payroll costs) often proved more indicative of potential safety violations.
  • Checklist Selection (CLSP) Performance: Decision Trees (DT) performed best but achieved a balanced accuracy of only 0.06. While this is better than a naive baseline (~0.04), it highlights the difficulty of choosing between 369 classes in a long-tailed distribution.
  • Non-compliance Prediction (NCP) Performance: This task showed more promise. AdaBoost and GradientBoost achieved an AUC of 0.68. Using the model for NCP resulted in a 66% increase in the odds of finding a violation compared to the dataset's base rate. While not perfect, this represents a significant potential efficiency gain for labor authorities.
  • Precision over Recall: In this domain, the authors argue that precision is the most vital metric. If an ML model predicts a violation, an inspector spends hours or days following up. High precision ensures that limited human resources aren't wasted on compliant firms.

Limitations and Future Work

The authors acknowledge several hurdles in their current work. Financial features often contain missing values for certain types of organizations, requiring careful imputation. Furthermore, the data is currently limited to the Norwegian context, and it remains to be seen how well these models generalize to different legal and economic systems.

Future research could treat this as a joint multi-objective optimization problem: selecting a checklist that is both relevant to the industry and likely to uncover a violation. Additionally, rather than a strict classifier, the authors suggest treating checklist selection as a recommendation problem, providing inspectors with a "top-K" list of suggestions.

By releasing LICD, the authors have provided the community with a vital benchmark for "AI for Social Good," specifically targeting the often-overlooked area of labor rights and workplace safety.

Summary updated

BibTeX
@inproceedings{flogard2022dataset,
  author = {Eirik Lund Flogard and Ole Jakob Mengshoel},
  title = {{A Dataset for Efforts Towards Achieving the Sustainable Development Goal of Safe Working Environments}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
}
Figure