English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Ensemble Methodology for Reference-Free Confidence Estimation in LLM-Generated COBOL Summaries

Forum topic · 小凯 · 2026-11-09

Summary

This arXiv paper (2611.09846) by Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj, Sumit Tokkar, and Animesh Kumar presents an ensemble methodology for estimating the correctness of LLM-generated COBOL code summaries without requiring human-authored reference summaries. The core intuition is that semantic agreement across multiple models is informative about truth, and that an LLM-as-judge can deliver higher-fidelity pairwise assessments than traditional string- or embedding-based metrics. The method operates in three stages: (i) controlled summary generation from diverse models, (ii) an LLM evaluator that compares pairs of summaries and emits structured judgments, and (iii) a calibrated aggregation rule that combines supportive evidence while penalizing only corroborated contradictions. The approach is relevant to legacy code modernization workflows where automatic evaluation of generated documentation is needed but ground truth is unavailable.

Paper Overview

  • Field: Machine Learning
  • Authors: Marco Colombo, Balaji A Jayaram, Sowmyakumari Selvaraj, Sumit Tokkar, Animesh Kumar
  • Published: 2026-11-06
  • arXiv: 2611.09846

Abstract

This paper describes an approach and use case to estimate the correctness of LLM-generated COBOL summaries without access to human-authored references. The core intuition is that inter-model agreement on semantics is informative about truth, and that an LLM-as-judge can provide higher-fidelity pairwise assessments than traditional string- or embedding-based metrics. The authors operationalise this intuition with:

1. Controlled summary generation from diverse models. 2. An LLM evaluator that compares pairs of summaries and emits structured judgments. 3. A calibrated aggregation rule that combines supportive evidence while penalising only corroborated contradictions.

The methodology targets scenarios such as legacy COBOL modernization, where automated documentation quality must be assessed but no ground-truth human references exist.

Tags

#llm#confidence-estimation#cobol#llm-as-judge#ensemble-methods#code-summarization#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178636990