A STUDY ON THE RELIABILITY OF AN ARTIFICIAL INTELLIGENCE MODEL FOR WATER ENVIRONMENT EVALUATION

Shengping ZhangInternational Journal of Geomate 2023 · DOI

TL;DR

This study investigates the optimal way to feed environmental data into deep learning models for water quality evaluation across Japan’s 109 major watersheds. The researchers discovered that training models on raw numerical data outperforms human-preprocessed categorical data, as preprocessing often introduces bias and leads to significant information loss.

What Problem Does This Paper Address?

Evaluating and planning for river health has traditionally relied on complex mathematical models that simulate physical and chemical processes. However, these models often struggle to incorporate "big data" or the nuanced expertise of planners across different regions.

While Artificial Intelligence (AI) is a powerful alternative, a critical question remains: What is the best way to present water environment data to an AI? For example, a metric like Biochemical Oxygen Demand (BOD) can be recorded as a raw number (mg/l), a class (Class AA, A, etc.), or a simple "yes/no" regarding whether a management goal was met. This paper tests which of these formats leads to the most reliable and accurate AI predictions.

Methods

The researchers developed a layered neural network model using Deep Learning technologies. The core of their experiment involved comparing three distinct data presentation methods for the "teacher data" (training set):

  1. Raw Numerical Data: Continuous values (e.g., exact pH or BOD levels) without any preprocessing.
  2. Environmental Quality Standard (EQS) Scores: Data converted into scores (0.1 to 0.9) based on established water quality classes.
  3. Management Goal Achievement: Categorical "yes/no" data indicating whether administrative goals were met.

The model utilized the Error Back Propagation Algorithm and the Method of Gradient Descent to minimize error, employing a Sigmoid function for unit output.

Datasets

The study utilized a massive dataset from the Ministry of Land, Infrastructure, Transport and Tourism of Japan, covering:

  • Scope: 104 of the 109 "Class-A" watersheds in Japan (5 were excluded due to missing data).
  • Timeframe: 21 years of records from 1998 to 2018 for training, with 2019 data used for evaluation.
  • Features: 58 items per river, categorized into Time of Sampling, River/Flow Conditions, Watershed Conditions, and various Water Quality Indexes (Living Environment, Human Health, etc.).

Key Insights and Ideas

  • Preprocessing is a Double-Edged Sword: While humans use categories (like "Class A") to make data easier to understand, this study suggests that such "filtering" damages the reliability of AI models. It strips away the subtle gradients that the AI needs to understand natural evolution processes.
  • The Bias of Human Logic: Preprocessing methods like EQS scoring are often subjective or arbitrary, designed for human convenience rather than mathematical precision. Feeding this "biased" information to an AI limits its ability to find the true underlying patterns.
  • AI's Native Language is Raw Data: AI models are inherently capable of processing various data types simultaneously. The study confirms that they are most "reliable" when allowed to interpret the rawest form of information available.

Key Results

The models were evaluated using Accuracy and "Threat Scores" (measuring how well the model hits the target for achieved vs. unachieved goals).

Presentation Way Accuracy Threat Score (Achieved) Threat Score (Unachieved)
Raw Data 96.88% 96.22% 84.71%
EQS Class Scores 92.55% 90.96% 70.19%
Management Goals 97.12% 96.57% 84.62%

While the "Management Goal" approach showed slightly higher accuracy, the authors argue this is a "circular" advantage because the output format perfectly matches the evaluation metric. When looking at the ability to capture the "un-achievement" rate (Threat Score U), the Raw Data model proved exceptionally robust, confirming it as the superior method for general water environment evaluation.

Limitations and Future Work

The authors acknowledge that while raw data is best for model reliability, preprocessing might still be necessary if the only goal is a simple binary answer (e.g., "Did we pass the test?").

Future research directions include: * Identifying which specific factors (out of the 58 items) are the most influential on water quality. * Determining the most effective specific improvement measures for each of the 104 watersheds. * Refining the methodology to help establish more reliable river environment planning and management standards globally.

Summary updated

Abstract

The objective of this study is to discover the most effective water environment improvement measures for the 109 most important watersheds of Japan, which are well-known Class-A watersheds under the jurisdiction of the Japanese government.An artificial intelligence (AI) model has been created by applying Deep Learning technologies with the expectation that an AI model is able to take adevantage of the multiple categorical water environmental data, and watershed information from 109 watersheds has been collected as teacher data to train the AI model.This study aims to find the best way to present the water environment data to an artificial intelligence model.To provide the most reliable water quality estimations, three different ways of water environment data presentation have been examined.It has been identified that presenting the raw environment data to the AI model as teacher data is the best way for building an AI model.This study concludes by pointing out that preprocessing data will cause information loss of teacher data and will also add biased information to teacher data.Therefore, using raw data without preprocessing as teacher data will help build a more reliable AI model.It is hoped that this study will contribute to establishing a more reliable river environment planning and management methodology.
BibTeX
@article{zhang2023study,
  author = {Shengping Zhang},
  title = {{A STUDY ON THE RELIABILITY OF AN ARTIFICIAL INTELLIGENCE MODEL FOR WATER ENVIRONMENT EVALUATION}},
  journal = {International Journal of Geomate},
  year = {2023},
  doi = {10.21660/2023.107.s8615},
}
Figure