No article text was recorded for this version.
human-in-the-loop scalability (human-in-the-loop scalability bottleneck)
Version 1 of 1253 papers257 conceptsfrozen 2026-10-04
Papers in this version
- Blog: VideoCoT: A Video Chain-of-Thought Dataset with Active Annotation Tool
- Blog: Human-in-the-loop Active Learning for Goal-oriented Molecule Generation
- Blog: DoTAT: A Domain-oriented Text Annotation Tool
- Blog: A Multi-module Robust Method for Transient Stability Assessment against False Label Injection Cyberattacks
- Blog: Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation
- Blog: Development and Validation of ML-DQA - a Machine Learning Data Quality Assurance Framework for Healthcare
- Blog: UNC-Utah NA-MIC framework for DTI fiber tract analysis
- Blog: NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples
- Blog: Fast building segmentation from satellite imagery and few local labels
- Blog: GFIE: A Dataset and Baseline for Gaze-Following from 2D to 3D in Indoor Environments
- Blog: Hybrid Reward Normalization for Process-supervised Non-verifiable Agentic Tasks
- Blog: Fine-tuning Language Models for Factuality
- Blog: Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images
- Blog: iTools: A Framework for Classification, Categorization and Integration of Computational Biology Resources
- Blog: Spying Browser Extensions: Analysis and Detection
- Blog: An Open Dataset and Model for Language Identification
- Blog: Causal Inference on Outcomes Learned from Text
- Blog: IntegralBench: Benchmarking LLMs with Definite Integral Problems
- Blog: Scaling Data Difficulty: Improving Coding Models via Reinforcement Learning on Fresh and Challenging Problems
- Blog: Targeted Data Acquisition for Evolving Negotiation Agents
- Blog: Building Interoperable Electronic Health Records as Purpose-Driven Knowledge Graphs
- Blog: COVID-19 non-pharmaceutical interventions: data annotation for rapidly changing local policy information
- Blog: VerificAgent: Integrating Expert Knowledge and Fact-Checked Memory for Robust Domain-Specific Task Planning
- Blog: SWE-bench-java: A GitHub Issue Resolving Benchmark for Java
- Blog: COCONut-PanCap: Joint Panoptic Segmentation and Grounded Captions for Fine-Grained Understanding and Generation
- Blog: Robustar: Interactive Toolbox Supporting Precise Data Annotation for Robust Vision Learning
- Blog: K3S: Knowledge-Driven Solution Support System
- Blog: AstroMMBench: A Benchmark for Evaluating Multimodal Large Language Models Capabilities in Astronomy
- Blog: Explore the Structure of Social Tags by Subsumption Relations
- Blog: Uncovering Misattributed Suicide Causes through Annotation Inconsistency Detection in Death Investigation Notes
- Blog: KU-DMIS at BioASQ 9: Data-centric and model-centric approaches for biomedical question answering
- Blog: Using Descriptive Video Services to Create a Large Data Source for Video Annotation Research
- Blog: Using Descriptions for Explaining Entity Matches (Discussion Paper)
- Blog: ScaleMAI: Accelerating the Development of Trusted Datasets and AI Models
- Blog: AgroTIC: Bridging the gap between farmers, agronomists, and merchants through smartphones and machine learning
- Blog: CYBORG: Blending Human Saliency Into the Loss Improves Deep Learning-Based Synthetic Face Detection
- Blog: Adapting the re-ID challenge for static sensors
- Blog: Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions
- Blog: Constructing a Scene-Based Knowledge System for E-Commerce Industries: Business Analysis and Challenges
- Blog: Seeing Culture: A Benchmark for Visual Reasoning and Grounding
- Blog: The Importance of Image Interpretation: Patterns of Semantic Misclassification in Real-World Adversarial Images
- Blog: Salutary Labeling with Zero Human Annotation
- Blog: Visualizing and Curating Knowledge Graphs over Time and Space
- Blog: Landsat30-AU: A Vision-Language Dataset for Australian Landsat Imagery
- Blog: MMED: A Multi-domain and Multi-modality Event Dataset
- Blog: Development and Evaluation of a Natural Language Processing Annotation Tool to Facilitate Phenotyping of Cognitive Status in Electronic Health Records: Diagnostic Study
- Blog: Actively Learning Ontology Matching via User Interaction
- Blog: Scaling data-driven robotics with reward sketching and batch reinforcement learning
- Blog: The PhotoSlap Game: Play to Annotate
- Blog: Interpretable Multimodal Zero-Shot ECG Diagnosis via Structured Clinical Knowledge Alignment
- Blog: RAD: On-line Anomaly Detection for Highly Unreliable Data
- Blog: Building a Foundational Guardrail for General Agentic Systems via Synthetic Data
- Blog: Real-Time Visual Feedback to Guide Benchmark Creation: A Human-and-Metric-in-the-Loop Workflow
- Blog: XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updates
- Blog: Auto-TA: Towards Scalable Automated Thematic Analysis (TA) via Multi-Agent Large Language Models with Reinforcement Learning
- Blog: Turning Web Text and Search Queries into Factual Knowledge: Hierarchical Class Attribute Extraction
- Blog: Data Cleaning Tools for Token Classification Tasks
- Blog: Improving Data Quality: Consistency and Accuracy
- Blog: Knowledge Graphs Evolution and Preservation - A Technical Report from ISWS 2019
- Blog: Rethinking Benchmarks for Cross-modal Image-text Retrieval
Showing 60 of 253 papers.
History & citation
Current · Permanent link
- The Human Verification Bottleneck in Machine Learning Benchmarks merged into this direction (51% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_human_in_the_loop_scalability_human_in_t_4e911afc,
title = {human-in-the-loop scalability (human-in-the-loop scalability bottleneck)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/human-in-the-loop-scalability-human-in-the-loop-scalability-bottleneck/rdr_4e911afc6f084247b3e598acae960389},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}This exact version (version 1, frozen text):
@misc{lacuna_human_in_the_loop_scalability_human_in_t_4e911afc_v1,
title = {human-in-the-loop scalability (human-in-the-loop scalability bottleneck)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/human-in-the-loop-scalability-human-in-the-loop-scalability-bottleneck/rdr_4e911afc6f084247b3e598acae960389/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}