AI Briefing — 2026-04-17

🔬 Research

GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents

106 upvotes · Mingyu Ouyang, Siyuan Hu, Kevin Qinghong Lin, Hwee Tou Ng, Mike Zheng Shou · Paper

GameWorld introduces a benchmark for evaluating multimodal LLM agents in video games with 34 diverse games and 170 tasks, using both computer-use agents (direct keyboard/mouse) and semantic action parsing approaches. The benchmark addresses the challenge of systematically evaluating agent capabilities like perception, planning, and control in closed-loop environments with verifiable, state-based metrics. Results show even the best agents fall far short of human performance, highlighting significant gaps in current multimodal agent capabilities.

RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Time

98 upvotes · Haozhe Wang, Cong Wei, Weiming Ren, Jiaming Liu, Fangzhen Lin · Paper

RationalRewards transforms reward models from passive evaluators into active optimization tools by teaching them to produce explicit multi-dimensional critiques before scoring. Their 8B model achieves state-of-the-art preference prediction competitive with Gemini-2.5-Pro while using 10-20x less training data. The approach improves visual generators both through RL training and a novel Generate-Critique-Refine loop that matches RL performance without parameter updates.

SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments

60 upvotes · Dinging Li, Yingxiu Zhao, Xinrui Cheng, Kangheng Lin, Hongbo Peng · Paper

SpatialEvo addresses 3D spatial reasoning through a self-evolving framework that uses Deterministic Geometric Environments (DGE) to provide zero-noise training feedback from exact geometric computations rather than model consensus. The system co-evolves questioner and solver policies across 16 spatial reasoning categories with a dynamic curriculum that focuses on weakest areas. It achieves highest average scores across nine benchmarks at both 3B and 7B scales without degrading general visual understanding.

🛠 Tools

forrestchang/andrej-karpathy-skills — Claude coding guidelines based on Karpathy's observations

7,959 stars/day · Repo

A single CLAUDE.md file that addresses Andrej Karpathy's identified LLM coding pitfalls through four principles: Think Before Coding, Simplicity First, Surgical Changes, and Goal-Driven Execution. The 52K-starred repo distills Karpathy's observations about LLMs making wrong assumptions, overcomplicating code, and making orthogonal edits into actionable guidelines. It forces explicit reasoning, prevents silent assumption-making, and promotes minimal, targeted changes.

thedotmack/claude-mem — Persistent memory for Claude coding sessions

1,897 stars/day · Repo

A TypeScript plugin that automatically captures everything Claude does during coding sessions, compresses it with AI using Claude's agent-sdk, and injects relevant context into future sessions. With 60K stars, it addresses the context loss problem between Claude conversations by maintaining persistent memory of past interactions. The tool uses AI compression to manage context efficiently and supports multiple languages through extensive internationalization.

Lordog/dive-into-llms — Hands-on LLM programming tutorial series

1,385 stars/day · Repo

A comprehensive Chinese-language programming tutorial series for large language models, derived from Shanghai Jiao Tong University's NLP courses. The 31K-starred Jupyter Notebook collection covers fine-tuning, deployment, mathematical reasoning, GUI agents, and model alignment with both course materials and video content. It provides free, practical programming references for students and researchers entering the LLM field.

📡 Analysis

datasette 1.0a28

17th April 2026 · Post · tags: datasette

Datasette's latest alpha fixes critical compatibility bugs introduced in 1.0a27, including execute_write_fn() callback errors and missing database cleanup. The release adds a new datasette.close() method and pytest plugin for automatic cleanup, addressing file descriptor exhaustion in plugin test suites. Most changes were implemented using Claude Code and the newly released Claude Opus 4.7, showcasing real-world AI coding assistance.

Qwen3.6-35B-A3B on my laptop drew me a better pelican than Claude Opus 4.7

16th April 2026 · Post · tags: ai, generative-ai, local-llms, llms, anthropic, claude, qwen, pelican-riding-a-bicycle, llm-release, lm-studio

Simon's lighthearted "pelican benchmark" reveals an interesting anomaly: a 21GB quantized Qwen3.6-35B-A3B model running locally outperformed Anthropic's Claude Opus 4.7 at generating SVG illustrations. While the pelican test remains a joke about model comparison difficulties, it historically correlated with general model utility—until now. This breaks the pattern, showing specialized capabilities don't always align with overall model power.

Sources: HuggingFace Papers API, GitHub Trending, simonwillison.net