FEDSTR: Money-In AI-Out | A Decentralized Marketplace for Federated Learning and LLM Training on the NOSTR Protocol

Konstantinos E. Nikolakakis, George Chantzialexiou, Dionysis KalogeriasarXiv 2024 · DOI · arXiv

Abstract

As the demand for Large Language Model (LLM) training scales, the reliance on centralized cloud infrastructure introduces significant bottlenecks, including high costs, privacy risks, and censorship vulnerabilities. FEDSTR (fεdstər) introduces a decentralized marketplace for AI training by repurposing the NOSTR (Notes and Other Stuff Transmitted by Relays) protocol. By integrating Federated Learning (FL) with the Bitcoin Lightning Network, FEDSTR enables a "Money-In AI-Out" ecosystem. This system allows customers to trustlessly procure model training from a global network of service providers, utilizing cryptographic verification and micro-incentives to ensure integrity and efficiency without the overhead of traditional blockchain consensus.

Introduction

The current landscape of AI development is dominated by centralized entities that control the necessary compute and data silos. This centralization poses a threat to the democratization of AI, as access is often gated by high financial barriers and the risk of data exposure. While various "Web3" solutions have attempted to decentralize compute, they frequently suffer from high transaction latency, complex tokenomics, and scalability issues inherent to global consensus blockchains.

FEDSTR addresses these challenges by leveraging NOSTR, a lightweight, websocket-based communication protocol. Unlike traditional blockchains, NOSTR does not require global consensus for every message, making it an ideal backbone for the high-frequency coordination required in distributed machine learning. FEDSTR transforms NOSTR from a social media protocol into a robust marketplace for federated learning, where compute is commoditized and payments are settled instantly via the Bitcoin Lightning Network.

The FEDSTR Architecture

The core innovation of FEDSTR lies in its extension of the NOSTR standard to support machine learning workflows. This is achieved through the introduction of specific "Event Kinds" that facilitate the lifecycle of a training job:

  1. Job Requests (Kind 8000): Customers broadcast their training requirements, including model architecture, dataset pointers, and payment terms.
  2. Job Results (Kind 6000): Service providers (Data Vending Machines or DVMs) submit the trained model parameters or gradients.
  3. Job Feedback (Kind 7000): Participants exchange status updates and validation metrics.

Distributed Training Algorithms

FEDSTR is designed to support various federated learning strategies, most notably:

  • FedAvg (Federated Averaging): The standard approach for aggregating local model updates from multiple participants.
  • DiLoCo (Distributed Low-Communication): A specialized algorithm for training LLMs across devices with limited bandwidth, optimizing the communication-to-computation ratio.

Data and Model Handling

To maintain efficiency, model data is not stored directly on NOSTR relays. Instead, the protocol utilizes relays to pass cryptographically signed pointers (URLs and SHA-256 hashes). These pointers link to data stored on decentralized storage solutions like IPFS, standard HTTP servers, or specialized "Blossom" servers, ensuring that the protocol remains storage-agnostic and scalable.

Economic Incentives and Security

A critical component of any decentralized marketplace is the mitigation of counterparty risk. FEDSTR employs a "trust-minimized" approach through two primary mechanisms:

Lightning Network Integration

By utilizing the Bitcoin Lightning Network for "zaps" (instant micropayments), FEDSTR allows for granular payment structures. Training is broken down into multiple rounds, with payments released at each milestone. This minimizes the financial risk for both the customer (who only pays for verified progress) and the provider (who receives frequent compensation).

Validation Logic

To prevent "lazy" or malicious actors from submitting fake training results, FEDSTR implements a validation routine. The customer evaluates the "loss decay" of the returned model parameters against a local validation dataset. If the model performance does not improve as expected, the final payment is withheld. This creates a self-correcting marketplace where reputation—linked to a participant's public key (npub)—becomes a valuable asset.

Proof-of-Concept and Results

The authors demonstrated the viability of FEDSTR through a Proof-of-Concept (PoC) implementation using the MNIST dataset. The PoC successfully validated several key hypotheses:

  • Infrastructure Compatibility: The system operates seamlessly on existing, unmodified NOSTR infrastructure (e.g., the Damus relay).
  • Integrity: Schnorr signatures and SHA-256 hashing effectively prevented tampering during the transmission of model parameters.
  • Orchestration: The "JobRequest → JobFeedback → JobResult" flow successfully coordinated training across multiple simulated nodes.

Discussion and Future Work

While the PoC establishes a foundation for decentralized AI training, several areas for development remain. The current implementation uses simulated Lightning payments; full NIP-57 integration for production-ready Bitcoin transfers is a priority. Additionally, while the protocol supports reputation, an automated, cross-relay scoring system is needed to help customers identify high-quality service providers.

Future iterations of FEDSTR aim to incorporate NIP-94 for better model discovery and explore advanced verification methods like BitVM. By moving toward a zero-trust environment, FEDSTR seeks to provide a truly censorship-resistant and economically efficient alternative to centralized AI training giants.

Summary updated

Abstract

The NOSTR is a communication protocol for the social web, based on the w3c websockets standard. Although it is still in its infancy, it is well known as a social media protocol, with thousands of trusted users and multiple user interfaces, offering a unique experience and enormous capabilities. To name a few, the NOSTR applications include but are not limited to direct messaging, file sharing, audio/video streaming, collaborative writing, blogging and data processing through distributed AI directories. In this work, we propose an approach that builds upon the existing protocol structure with end goal a decentralized marketplace for federated learning and LLM training. In this proposed design there are two parties: on one side there are customers who provide a dataset that they want to use for training an AI model. On the other side, there are service providers, who receive (parts of) the dataset, train the AI model, and for a payment as an exchange, they return the optimized AI model. To demonstrate viability, we present a proof-of-concept implementation over public NOSTR relays. The decentralized and censorship resistant features of the NOSTR enable the possibility of designing a fair and open marketplace for training AI models and LLMs.
BibTeX
@misc{nikolakakis2024fedstr,
  author = {Konstantinos E. Nikolakakis and George Chantzialexiou and Dionysis Kalogerias},
  title = {{FEDSTR: Money-In AI-Out | A Decentralized Marketplace for Federated Learning and LLM Training on the NOSTR Protocol}},
  year = {2024},
  eprint = {2404.15834},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.15834},
}
Figure