Search

AI can find the necessary data in a scientific article, perform the calculations, and still not understand exactly how the authors arrived at the result

Translated title. Original: ИИ может найти нужные данные в научной статье, выполнить расчёты и всё равно не понять, как именно авторы пришли к результату

AI-based replication studies reveal that matching published results does not prove that AI fully retraces authors' reasoning; multiple analysis paths and missing causal connections can lead to the same number. The authors advocate evaluating scientific AI systems across the entire reproduction process, linking data, settings, and conclusions.

[2611.08452] ReproBridge: Building Reproducibility into ML Research Artifacts

The work presents ReproBridge, a workflow to build refined, reproducible ML research artifacts (code, environment, launchers, guide) to improve reproduction across tasks. In experiments, refined artifacts boost reproduction success and reduce API costs, with ablations showing the importance of each component.

GitHub - hypneum-lab/rescience-nh01: ReScience C submission: pre-registered replication of the dreaming Hopfield kernel (Fachechi, Agliari, Barra 2019) at N = 1 000 — code, data, pre-registrations · GitHub

Independent NumPy replication of the dreaming Hopfield kernel with pre-registered design and provenance tracking; version 2 adds self-coupling ablation and diagonal experiments, with exact reproducibility across 1,000-node runs and published data/code artifacts.

winston-lin-research/papers/data-valuation at main · WINSTON672/winston-lin-research · GitHub

The note argues that dataset value depends on more than predictive usefulness and emphasizes reproducible valuation with full provenance and environment details for local, reproducible experiments; it includes limitations and disclosure about AI assistance.

Human written · Stylome receipt

The document emphasizes transparency by releasing traces and minimal reproduction materials for open-weight AI models, highlighting reproducibility, community discussion, and the role of open licenses in advancing AI research.

mloda-registry/docs/guides/data-operation-patterns/known-divergences.md at main · mloda-ai/mloda-registry · GitHub

A centralized audit detailing cases where non-PyArrow frameworks diverge from PyArrow semantics, with mitigations and tests to prevent silent regressions across data-operations. It guides when to update implementations or tests and documents three divergence kinds with concrete examples.

ferret: a Framework for Benchmarking Explainers on Transformers - ACL Anthology

The text explains how to submit corrections to an ACL Anthology record through a GitHub issue. Corrections must match the PDF and can cover the title, authors, or abstract.

[2611.01867] Fairness-Aware Design for Contextual Experiments: Guaranteeing Reliability and Equity in Heterogeneous Subgroups

The document proposes a Fairness-Aware Contextual Track-and-Stop Design (F-CTSD) for contextual experiments, delivering subgroup fairness guarantees while reducing sample size. It provides theoretical results on sample complexity and estimate reliability, plus numerical evidence showing improved efficiency and a 4.95% reduction in fairness violations.

ReproBridge: Building Reproducibility into ML Research Artifacts | DAIR.AI Academy | DAIR.AI Academy

ReproBridge defines a standardized workflow and artifacts to improve reproducibility in ML research; refining code, environment, and a reproduction guide boosts success rates and lowers API costs across models. An Auditor validates execution, and ablations show each component matters.

Publications: Research, Blog, Press | Artificial Societies

The document points to a piece about evaluating an adaptive sampling algorithm using the Gittins Index in experiments with exponential rewards, linked as a read article likely spanning research, blog, and press contexts.