No article text was recorded for this version.
LLM-based semantic evaluation
Version 1 of 1340 papers391 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Evaluating LLMs’ Capability to Identify Lexical Semantic Equivalence: Probing with the Word-in-Context Task
- Blog: Let your LLM generate a few tokens and you will reduce the need for retrieval
- Blog: Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction
- Blog: Bridging the Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
- Blog: Neptune: The Long Orbit to Benchmarking Long Video Understanding
- Blog: PM3-KIE: A Probabilistic Multi-Task Meta-Model for Document Key Information Extraction
- Blog: Learning Answer Generation using Supervision from Automatic Question Answering Evaluators
- Blog: Variable Extraction for Model Recovery in Scientific Literature
- Blog: UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning
- Blog: SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection
- Blog: Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs
- Blog: BERGEN: A Benchmarking Library for Retrieval-Augmented Generation
- Blog: Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge
- Blog: Fine-grained Gender Control in Machine Translation with Large Language Models
- Blog: OpenEQA: Embodied Question Answering in the Era of Foundation Models
- Blog: Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model
- Blog: Beyond Exact Match: Semantically Reassessing Event Extraction by Large Language Models
- Blog: WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-Grained Spatial-Temporal Understanding
- Blog: TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models
- Blog: Rethinking the Roles of Large Language Models in Chinese Grammatical Error Correction
- Blog: Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking
- Blog: Leveraging LLM For Synchronizing Information Across Multilingual Tables
- Blog: PropTest: Automatic Property Testing for Improved Visual Programming
- Blog: MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
- Blog: Definition Generation for Automatically Induced Semantic Frame
- Blog: Camoscio: An Italian Instruction-tuned LLaMA
- Blog: WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding
- Blog: Fine-grained Gender Control in Machine Translation with Large Language Models
- Blog: Are Large Language Models Effective in Clinical Trial Design? A Study on Baseline Feature Generation
- Blog: SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection
- Blog: Forecasting Live Chat Intent from Browsing History
- Blog: AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark
- Blog: Synthetic Document Question Answering in Hungarian
- Blog: On a Spurious Interaction between Uncertainty Scores and Answer Evaluation Metrics in Generative QA Tasks
- Blog: Evaluating Generative Language Models in Information Extraction as Subjective Question Correction
- Blog: Agentic AI for Improving Precision in Identifying Contributions to Sustainable Development Goals
- Blog: Analyzing Modular Approaches for Visual Question Decomposition
- Blog: Dial-In LLM: Human-Aligned Dialogue Intent Clustering with LLM-in-the-loop
- Blog: mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model
- Blog: Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
- Blog: Improving Automatic VQA Evaluation Using Large Language Models
- Blog: Evaluating Speech Recognition Performance Towards Large Language Model Based Voice Assistants
- Blog: Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries
- Blog: REGen: A Reliable Evaluation Framework for Generative Event Argument Extraction
- Blog: BERGEN: A Benchmarking Library for Retrieval-Augmented Generation
- Blog: HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning
- Blog: Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues
- Blog: SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection
- Blog: SURE-VQA: Systematic Understanding of Robustness Evaluation in Medical VQA Tasks
- Blog: Can Large Language Models be Effective Online Opinion Miners?
- Blog: JN666 at SemEval-2024 Task 7: NumEval: Numeral-Aware Language Understanding and Generation
- Blog: ChemActor: Enhancing Automated Extraction of Chemical Synthesis Actions with LLM-Generated Data
- Blog: DiscoveryBench: Towards Data-Driven Discovery with Large Language Models
- Blog: Evaluation of Question Generation Needs More References
- Blog: Enhancing Automated Essay Scoring Performance via Cohesion Measurement and Combination of Regression and Ranking
- Blog: Chinese UMR annotation: Can LLMs help?
- Blog: Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledge
- Blog: TD-EVAL: Revisiting Task-Oriented Dialogue Evaluation by Combining Turn-Level Precision with Dialogue-Level Comparisons
- Blog: ChemActor: Enhancing Automated Extraction of Chemical Synthesis Actions with LLM-Generated Data
- Blog: The Evolution of LLM Adoption in Industry Data Curation Practices
Showing 60 of 340 papers.
History & citation
Current · Permanent link
- Semantic Answer Equivalence for Generative Model Evaluation merged into this direction (85% overlap)
- Scaling Human Judgment with LLM Evaluators merged into this direction (50% overlap)
Cite this direction
Latest version (a living page):
@misc{lacuna_llm_based_semantic_evaluation_8e5239ce,
title = {LLM-based semantic evaluation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/llm-based-semantic-evaluation/rdr_8e5239ce58c94b5d855ec8cb76bf72d8},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}This exact version (version 1, frozen text):
@misc{lacuna_llm_based_semantic_evaluation_8e5239ce_v1,
title = {LLM-based semantic evaluation},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/llm-based-semantic-evaluation/rdr_8e5239ce58c94b5d855ec8cb76bf72d8/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}