In order to fully realise the value of open data researchers must first address the quality of the datasets

Marta Indulska, Shazia SadiqLondon 2017

Abstract There has been a phenomenal increase in the availability of data over the last decade. Open data is provided as a means of empowering users with information and in the hope of sparking innovation and increased efficiency in governments and businesses. However, in spite of the many success stories based on the open data paradigm, concerns remain over the quality of such datasets. Marta Indulska and Shazia Sadiq argue that in order to facilitate more effective and efficient realisation of value from open data, research must reach a shared consensus on the definition of data quality dimensions, provide methods and guidelines for assessing the potential usefulness of open datasets using exploratory tools and techniques, and develop rigorous theoretical underpinnings on effective use of open data.

Introduction: The Availability-Usability Gap

The "Open Data" movement has successfully pressured governments and organizations to release massive amounts of information. Portals in the US, UK, and Australia now host tens of thousands of datasets. However, "availability" does not equal "usability." While the volume of open data has exploded globally, its actual utility is severely bottlenecked by poor data quality and a lack of standardized assessment frameworks. To move from "quantity" to "value," researchers must establish a shared consensus on data quality dimensions and develop tools that allow users to assess a dataset's fitness for purpose before they invest resources into using it.

The Global Scale of Open Data

The growth of national open data portals serves as a benchmark for the movement's momentum:

  • USA (Data.gov): Over 192,000 datasets.
  • Canada: Over 118,000 datasets.
  • Australia: Jumped from 573 datasets in 2013 to over 49,000 in 2017.
  • United Kingdom: Over 42,000 datasets.
The open data movement provides the conceptual building blocks for innovation, but quality issues act as a roadblock.

Despite these numbers, many datasets suffer from the "garbage in, garbage out" trap. Issues such as missing metadata, inconsistent schemas, and poor data integrity create significant barriers. Because metadata is often incomplete or difficult to access, assessing whether a dataset is actually useful for a specific project becomes a resource-intensive and expensive exercise.

Barriers to Innovation: Metadata and Semantics

Metadata—the "data about the data"—is frequently the weakest link. Without it, understanding the generative process or the schema of a dataset is nearly impossible for outside researchers. Furthermore, there is no "one world" view of data quality. For example, "accuracy" might mean "closeness to reality" to one researcher but "precision of measurement" to another. This lack of consensus makes benchmarking impossible and risks leading to flawed decision-making if poor-quality data is used without scrutiny.

The Economic Case for Effective Use

Value isn't inherent in the data itself; it is realized through "effective use." This requires not just the data, but an understanding of its context and limitations. The economic stakes are high:

  • Landsat: Satellite imagery utilization saves the US government and private sector over $350 million annually.
  • EU Projections: Effective use of open data could reduce EU public administration costs by €1.7 billion and cut power consumption by 16%.
  • Citymapper: Demonstrates success by leveraging London’s open transport data to provide real-time value to millions.

A Research Agenda for Data Quality

While data exploration has been researched for a decade, there is still a critical lack of practical methods and guidelines for consumers to quickly assess a dataset's usefulness. To address this, a three-pronged research agenda is proposed:

  1. Standardization: Developing a unified vocabulary for data quality dimensions to allow for better communication between data curators and consumers.
  2. Exploratory Tools: Creating "bottom-up" data exploration techniques that allow users to identify quality issues before they commit to a project.
  3. Theoretical Development: Building rigorous theoretical models to understand the factors that lead to successful open data initiatives, informing better government policy on data release.

This article is a position piece based on research published in the International Journal of Information Management by Marta Indulska and Shazia Sadiq.

Summary updated

Abstract

There has been a phenomenal increase in the availability of data over the last decade. Open data is provided as a means of empowering users with information and in the hope of sparking innovation and increased efficiency in governments and businesses. However, in spite of the many success stories based on the open data paradigm, concerns remain over the quality of such datasets. Marta Indulska and Shazia Sadiq argue that in order to facilitate more effective and efficient realisation of value from open data, research must reach a shared consensus on the definition of data quality dimensions, provide methods and guidelines for assessing the potential usefulness of open datasets using exploratory tools and techniques, and develop rigorous theoretical underpinnings on effective use of open data.
BibTeX
@inproceedings{indulska2017order,
  author = {Marta Indulska and Shazia Sadiq},
  title = {{In order to fully realise the value of open data researchers must first address the quality of the datasets}},
  booktitle = {London},
  year = {2017},
}
Figure