Paper Overview
Research area: ML Authors: Brian T. Weston, Alondra Torres Contreras, Aldair E. Gongora Release date: 2026-11-16 arXiv: 2611.21541
Abstract (translated)
We introduce SciResearch-Eval, an interactive benchmark of scientific judgment built on a standard laboratory problem: measuring a material's thermal properties by heating it and fitting a heat-conduction simulator. An agent designs its own experiments, uses a supplied simulator that may be under-resolved or physically wrong, and reports each property with a 95% interval. To our knowledge it is the first benchmark to combine agent-designed experiments, a deliberately flawed simulator, and proper scoring of calibrated intervals against exact ground truth. On a held-out split (93 tasks, 9 models, 2,511 episodes), models handle procedural traps. When the simulator is missing physics, most report wrong values while stating they are 95% sure: their ranges contain the truth in 20–62% of cases.
*Note: the source post's excerpt of the original abstract is truncated at this point.*
Related summary in the post
The post also carries a Chinese-language summary of a geolocation-aware multimodal framework, Wildfire-SSR, for building-level wildfire vulnerability assessment:
- Combines macro-scale satellite context with micro-scale street-view evidence; a vision-language model extracts wildfire-relevant building and site attributes from pre-fire street-view panoramas as structured semantic text.
- Features are spatially aligned with satellite imagery via a decoupled cross-attention fusion decoder, with a frozen satellite-pretrained DINOv2/DINOv3-style encoder capturing overhead visual context.
- Evaluated on ~37,000 affected buildings from the 2025 Los Angeles Eaton and Palisades fires; rotated positional embeddings gave the best spatial grounding (building localization F1 = 0.6735; five-class pixel-level risk harmonic-mean F1 = 0.2476), outperforming unimodal baselines and ungrounded multimodal configurations.
- The model shows a conservative bias toward higher risk classes, reducing severe risk underestimation, and completes an end-to-end study-area assessment in 37 minutes on a single GPU.