DRLBTS: deep reinforcement learning-aware blockchain-based healthcare system

Abdullah Lakhan, Mazin Abed Mohammed, Jan Nedoma, Radek Martínek, Prayag Tiwari, Neeraj KumarScientific 2023 · DOI

Abstract

Industrial Internet of Things (IIoT) is the new paradigm to perform different healthcare applications with different services in daily life. Healthcare applications based on IIoT paradigm are widely used to track patients health status using remote healthcare technologies. Complex biomedical sensors exploit wireless technologies, and remote services in terms of industrial workflow applications to perform different healthcare tasks, such as heartbeat, blood pressure and others. However, existing industrial healthcare technologies still have to deal with many problems, such as security, task scheduling, and the cost of processing tasks in IIoT based healthcare paradigms. This paper proposes a new solution to the above-mentioned issues and presents the deep reinforcement learning-aware blockchain-based task scheduling (DRLBTS) algorithm framework with different goals. DRLBTS provides security and makespan efficient scheduling for the healthcare applications. Then, it shares secure and valid data between connected network nodes after the initial assignment and data validation. Statistical results show that DRLBTS is adaptive and meets the security, privacy, and makespan requirements of healthcare applications in the distributed network.

Introduction

As healthcare shifts toward the Industrial Internet of Things (IIoT)—enabling remote heartbeat monitoring and automated diagnostic services—underlying networks face a "triple threat" of challenges. First, medical services are not isolated tasks; they are sequential workflows that must be partitioned across heterogeneous devices. Second, resource constraints create a bottleneck: mobile devices lack power, while cloud nodes introduce high latency. Static schedulers often fail when network conditions fluctuate. Finally, security remains paramount; moving sensitive patient data between independent nodes (mobile to fog to cloud) creates vulnerabilities for data tampering.

Existing solutions often focus on financial IoT or simple task offloading, largely ignoring the specific inter-dependencies of healthcare workflows and the high overhead of traditional blockchain implementations. DRLBTS addresses these gaps by integrating adaptive scheduling with a lightweight security layer.

System Architecture and Methods

The DRLBTS framework utilizes a three-tier architecture comprising Mobile, Fog, and Cloud agents. This hierarchy allows for intelligent workflow partitioning: lightweight tasks remain on the mobile device, delay-sensitive tasks move to fog nodes, and data-heavy, delay-tolerant tasks are offloaded to the cloud.

Architecture of the DRLBTS framework across mobile, fog, and cloud tiers, illustrating the workflow submission and scheduling process.

Core Components

  • Multi-Agent Deep Reinforcement Learning (DRL): Instead of static logic, the system employs Q-learning. Agents learn through trial and error to predict the optimal node for a specific task based on real-time network states, deadlines, and resource availability.
  • Lightweight Blockchain: To secure data without the massive computational cost of traditional mining, the authors implemented a lightweight Proof of Work (PoW) and SHA-256 hashing. This ensures that every piece of data shared between nodes is validated and immutable.
  • Mathematical Optimization: The framework aims to minimize the makespan—the total time spent on both communication and computation—subject to strict deadline and resource constraints.

Adaptive Scheduling and Data Integrity

The synergy between DRL and blockchain is the framework's primary innovation. The DRL component allows the system to be "self-healing." If a fog node becomes congested, the Q-learning policy adapts in real-time to reschedule tasks. Simultaneously, the blockchain layer ensures that as tasks move between the mobile, fog, and cloud environments, their integrity is maintained through a cooperative validation scheme.

The DRL agent loop integrated with the multi-agent cooperative blockchain scheme for adaptive scheduling and secure data sharing.

Evaluation and Key Results

The study evaluated the framework using heartbeat datasets sourced from Kaggle and custom "self-workflows" designed to simulate real-world clinical scenarios. DRLBTS was compared against several baselines, including standard DQN, DDPG, and decentralized Ethereum-based scheduling.

Key findings include:

  • Makespan Efficiency: In tests with 1,000 workflow tasks, DRLBTS achieved a significantly lower Relative Percentage Deviation (RPD%)—roughly 20% compared to the 50-70% seen in standard DQN and DDPG models.
  • Scalability: The performance gap widened as complexity increased. When scaling to 2,000 tasks, DRLBTS maintained its efficiency while baselines struggled with increased deadline-missing rates.
  • Resource Management: The multi-agent approach proved more adept at managing "space leakage" (memory/resource waste) compared to existing Ethereum or Corda-based blockchain models.

Conclusion and Future Work

DRLBTS demonstrates that security does not have to come at the cost of performance. By using DRL to optimize when and where blockchain validation occurs, the authors achieved a system that is both more secure and faster than non-blockchain baselines.

Future research will focus on "mobility-enabled power efficiency" to reduce the computational intensity on mobile devices and extending the framework to support "hard" real-time healthcare applications where millisecond-level latency is critical for life-saving interventions.

Summary updated

Abstract

Industrial Internet of Things (IIoT) is the new paradigm to perform different healthcare applications with different services in daily life. Healthcare applications based on IIoT paradigm are widely used to track patients health status using remote healthcare technologies. Complex biomedical sensors exploit wireless technologies, and remote services in terms of industrial workflow applications to perform different healthcare tasks, such as like heartbeat, blood pressure and others. However, existing industrial healthcare technoloiges still has to deal with many problems, such as security, task scheduling, and the cost of processing tasks in IIoT based healthcare paradigms. This paper proposes a new solution to the above-mentioned issues and presents the deep reinforcement learning-aware blockchain-based task scheduling (DRLBTS) algorithm framework with different goals. DRLBTS provides security and makespan efficient scheduling for the healthcare applications. Then, it shares secure and valid data between connected network nodes after the initial assignment and data validation. Statistical results show that DRLBTS is adaptive and meets the security, privacy, and makespan requirements of healthcare applications in the distributed network.
BibTeX
@inproceedings{lakhan2023drlbts,
  author = {Abdullah Lakhan and Mazin Abed Mohammed and Jan Nedoma and Radek Martínek and Prayag Tiwari and Neeraj Kumar},
  title = {{DRLBTS: deep reinforcement learning-aware blockchain-based healthcare system}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-29170-2},
}
Figure