Valuing Actions and Ranking Hockey Players With Machine Learning

Oliver Schulte

Introduction

Traditional hockey analytics have long relied on aggregate "box score" statistics like goals, assists, and plus/minus. While useful, these metrics often overlook the context and the sequence of events that lead to a goal. A defensive takeaway in a tied game or a cross-ice pass that opens up a shooting lane is frequently more valuable than the final shot itself. This paper introduces a machine learning framework to value every individual player action based on its "Impact"—the change it causes in the team's probability of scoring the next goal.

Methodology: Defining Impact

The foundation of this approach is the Success Probability $P(S_i | X_t)$, which estimates the likelihood that team $i$ will score given the current match context $X_t$. The context includes variables such as rink coordinates, game time, and manpower (e.g., power plays). The Impact of any action is then defined as the change in this probability: $$\text{impact}_i(t+1) = P(S_i | X_{t+1}) - P(S_i | X_t)$$ For example, if a successful pass increases a team's scoring probability from 10% to 15%, that action is assigned an impact value of +0.05.

The researchers explored two main architectures for estimating these probabilities:

  1. Classifier Approach: Treating events as discrete data points for binary classification (e.g., "did this lead to a goal?"). This often utilizes sliding windows or Recurrent Neural Networks (RNNs) to incorporate match history.
  2. Reinforcement Learning (RL): Modeling the game as a Markov Decision Process where success probability is treated as a "Value Function." RL is generally superior as it inherently accounts for the temporal flow and correlations between sequential actions, treating the game as a continuous stream rather than isolated incidents.

Valuing On-Ice Actions

By applying this model to NHL play-by-play data, we can quantify the average value of different types of actions. Not all shots are created equal, and not all turnovers are equally damaging. The model allows for a nuanced "valuation map" of every event on the ice.

Distribution of impact values across different game actions.

As shown in the distribution above, "Takeaways" and "Shots" typically provide the highest positive median impact. Conversely, "Giveaways" and "Penalties" represent the most significant negative contributions to a team's scoring probability. Interestingly, the impact of the same action can vary wildly based on context; a shot taken during a power play in a tied game carries a different statistical weight than a desperate shot at the end of a blowout.

Results and Player Rankings

Aggregating these individual action values over a season results in a Goal Impact Metric (GIM). This metric offers a more stable and predictive measure of player performance than traditional points. One of the key advantages of GIM is its early convergence; it can accurately project a player's end-of-season performance as early as the halfway mark, making it a powerful tool for mid-season scouting.

Top 20 NHL players by Impact Score for the 2015-2016 season.

For the 2015-2016 season, the model identified Taylor Hall, Joe Pavelski, and Johnny Gaudreau as the top performers by Impact Score. Notably, the metric highlighted "market inefficiencies" by identifying elite performers like Gaudreau and Mark Scheifele while they were still on low-value contracts, effectively predicting their subsequent multi-million dollar breakout contracts.

Conclusion and Future Frontiers

While the GIM provides a significant leap over traditional stats, it faces limitations. Discretizing the rink into zones can create artificial "jumps" in probability at zone boundaries. Furthermore, while play-by-play data is rich, it lacks the high-fidelity tracking of continuous player coordinates. The next frontier in hockey analytics lies in applying deep reinforcement learning to continuous tracking data, balancing high-level predictive accuracy with the interpretability required for coaching and scouting.

Summary updated

Abstract

A fundamental goal of sports analytics is to rank player performance. A common approach is to assign a value to each player action and rank a player by their aggregate action value. A recent AI-based approach is to measure the value of a player’s action by how much it increases their team’s chance of success, that is, their team’s chance of scoring the next goal. This requires a model that outputs a success probability estimate, given a match context and an action. This talk describes machine learning techniques for building success probability models from data. The techniques range from easy-to-implement probabilistic classifiers to advanced reinforcement learning methods. The results of success probability models are illustrated with action values and player rankings for the National Hockey League.
BibTeX
@inproceedings{schulte2022valuing,
  author = {Oliver Schulte},
  title = {{Valuing Actions and Ranking Hockey Players With Machine Learning}},
  booktitle = {LINHAC},
  year = {2022},
  doi = {10.3384/ecp191001},
}
Figure