English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Better to Ask in English: Cross-Lingual Evaluation of LLMs for Healthcare Queries (WWW 2024)

Forum topic · 小凯 · 2026-07-05

Summary

This forum post on zhichai.net indexes the WWW 2024 research paper "Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries." The paper systematically evaluates how well large language models (LLMs) answer healthcare-related queries across different languages, examining whether LLMs perform better when queries are asked in English compared to other languages. The post provides structured metadata including the official ACM DL link (DOI: 10.1145/3589334.3645643), situates the work within the information retrieval and search/recommendation research landscape, and outlines the typical evaluation pipeline for cross-lingual LLM healthcare QA: query representation, model response generation, and quality assessment. It also includes a checklist for engineering deployment of LLM-based search systems, covering data privacy, latency budgets, quality verification, safety, and cost control, plus a glossary of key IR terms such as RAG, nDCG, and agentic search. Note that the post itself is largely a template entry; readers should consult the original paper PDF for quantitative results and detailed findings.

Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries (WWW 2024)

Paper Metadata

| Field | Content | |-------|---------| | Title | Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries | | Venue | The Web Conference (WWW) 2024 | | Source link | https://dl.acm.org/doi/abs/10.1145/3589334.3645643 | | Resource type | Academic paper | | Category | Verticals (Healthcare × LLM evaluation) |

Overview

This entry indexes a WWW 2024 paper that conducts a cross-lingual evaluation of large language models (LLMs) on healthcare queries. The central research question is whether LLMs give better healthcare answers when queries are posed in English versus other languages — a practically important question, since English-centric training data may cause uneven answer quality across languages, with real consequences for medical information safety.

The surrounding forum entry (largely a template annotation) situates the work in the broader context of LLM-era information retrieval, where traditional pipelines that separate retrieval, ranking, and generation struggle to meet user demands for natural-language interaction, multi-hop reasoning, and up-to-date knowledge.

> Note: The original post is primarily a metadata/template entry. Quantitative results should be verified against the paper PDF via the DOI link above.

Typical Evaluation Framing for This Problem

  • Task: healthcare QA across multiple languages, comparing answer quality when the same question is asked in English vs. other languages.
  • Metrics commonly used: factuality/consistency, human or LLM-as-judge evaluation, task success rate.
  • Concerns: cross-lingual generalization, hallucination and safety in the medical domain, and English-centric data bias.
  • Insights for Search / Rec / Personalization Practitioners

    1. Architecture: cascade retrieval + reranking + generation remains mainstream, but agentic paradigms make retrieval strategy itself learnable. 2. Data: high-quality instruction data matters as much as click/session logs; synthetic data requires guarding against knowledge leakage and distribution shift. 3. Evaluation: the gap between offline metrics and online satisfaction is widening; LLM-as-judge should be cross-validated with human evaluation. 4. Product: latency, cost, explainability, and safety are hard constraints for industrial deployment — academic benchmarks alone are insufficient.

    Engineering Deployment Checklist

    | Item | Question | Recommendation | |------|----------|----------------| | Data | Does training/index contain PII? Versioning? | Partitioned indexes, anonymization, rollback-capable embeddings | | Latency | p99 budget? How many retrieval steps? | Cascade + early stopping, caching hot queries, async reranking | | Quality | Do offline gains transfer to online CTR/satisfaction? | Interleaving experiments, human audits, citation checks | | Safety | Does open retrieval introduce poisoning/bias? | Source whitelists, adversarial detection, output filtering | | Cost | Per-query token and GPU cost? | Small-model routing, distillation, hybrid sparse+dense retrieval |

    Glossary

    | Term | Meaning | |------|---------| | IR | Information Retrieval | | RAG | Retrieval-Augmented Generation | | LTR | Learning to Rank | | nDCG | Normalized Discounted Cumulative Gain | | Agentic Search | Search modeled as sequential decision-making and tool use | | Gen-IR | Generative Information Retrieval |

    Related Entries

  • An interpretable ensemble of graph and language models…
  • Automated Query-Product Relevance Labeling using Large Language Models…
  • Behavior-driven query similarity prediction based on pre-trained language models…
  • References

  • Original paper: *Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries*, WWW 2024. https://dl.acm.org/doi/abs/10.1145/3589334.3645643

Tags

#llm#healthcare#cross-lingual#evaluation#information-retrieval#www-2024#rag#medical-qa

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178209022