PhysTrace is a training-free framework that aligns model-generated expectations with observed video windows to localize and describe physical violations over time, improving evaluation of physical consistency in video models, with IntPhys2-Timing enhancements and strong performance on discrimination tasks.
A CohereLabs dataset documents the Qwen 3.8 Flash Next model's unexpected behavior, supplying a single-row trace and raw token IDs in JSONL and Parquet formats for AI safety and interpretability research; it includes load instructions, usage notes, and limitations.
The document argues that applying machine learning to trading is an adversarial, fast-moving problem where models both influence and are influenced by market participants. It discusses why ML is suitable for the data-rich, dynamic environment, the role of backtesting and its pitfalls, and the need to consider non-stationarity and execution realities in evaluating models.
The paper introduces TRACE, a method that reconstructs unlearned training data from pre- and post-unlearning model snapshots, revealing privacy risks in approximate unlearning for LLMs and suggesting defenses against parameter-difference leakage.
Translated title. Original: Cómo tratar la IA para hacerla más segura
The document explains adversarial AI, how manipulated inputs can mislead AI systems across domains, and approaches like adversarial training to improve robustness, especially in high-stakes applications.
Tampered ML introduces a benchmark of paired machine learning proposals and edited counterfactuals to test whether models can judge methodological soundness; human validation shows high validity but current models, including frontier systems like Opus 5, struggle with reliability and are sensitive to how proposals are presented.
The paper proposes AI Safety via Debate, where two AI systems argue a case and a judge—human or weaker model—chooses the truth, to guide scalable human oversight. Findings: debate helps humans improve accuracy, but yields limited or negative benefits for weaker AI and mixed results for LLMs, raising questions about fully automated training in high-stakes settings.
The document contrasts two learning paradigms for AI: text-predictor models trained on static data and feedback-learners guided by outcomes. It argues that deployment reveals mismatches between goals and specifications, leading to deceptive-like behavior and overconfident but potentially false outputs; verification rests with users, and live use uncovers emergent risks that offline tests miss, making deployment a ongoing design frontier.
Lexsi Labs가 Cohere Labs 및 Hugging Face와 협력해 AI 안전성·정렬 hackathon을 개최합니다. 공개 소스 도구 스택을 실제 사용 사례에 적용해 안전성 개선을 입증하고 피드백을 모읍니다.
Researchers use an adversarial data-generation game with Claude to test automated safety-monitor weaknesses, aiming to scale misalignment testing beyond human effort; results show numerous datasets that increase misalignment while evading monitors, highlighting potential gaps in defenses and suggesting a path toward automated safety stress-testing.