AI Briefing — 2026-07-03
🔬 Research
Program-as-Weights: A Programming Paradigm for Fuzzy Functions
36 upvotes · Wentao Zhang et al. · Paper
PAW (Program-as-Weights) proposes replacing LLM API calls for "fuzzy" programming tasks (log filtering, JSON repair, intent-based ranking) with small, locally-executable neural artifacts. A 4B compiler model, trained on the 10M-example FuzzyBench dataset, generates parameter-efficient adapters for a frozen 0.6B Qwen3 interpreter — matching Qwen3-32B performance at ~1/50th the memory footprint, running at 30 tokens/s on a MacBook M3. The key insight is that the foundation model is invoked once to compile a reusable adapter, not once per query — making subsequent calls cheap and fully offline. A significant practical alternative to cloud LLM APIs for deterministic-ish tasks.
PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception
35 upvotes · Yana Wei et al. · Paper
PerceptionRubrics is a new benchmark targeting the gap between inflated benchmark scores and real-world multimodal brittleness. It pairs 1,038 dense images with 12,000+ atomic rubrics divided into "Must-Right" and "Easy-Wrong" criteria, scored via a gated mechanism that issues sharp binary penalties on failures of essential visual facts. Key finding: open-source models still lag proprietary ones by a persistent 8% perception deficit, and models that pass individual checks often fail strict conjunctive constraints. A more honest yardstick than current holistic semantic-matching benchmarks.
EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments
32 upvotes · Zhilin Wang et al. · Paper
EvoPolicyGym benchmarks how well LLM agents iteratively improve RL policies over multiple interaction rounds under a fixed budget, rather than measuring a single end-result score. Built on compact interactive RL environments, it provides trajectory-level diagnostics showing how agents allocate feedback, tune parameters, and discover task-appropriate mechanisms. On the 16-environment suite, GPT-5.5 takes the top aggregate rank. A useful diagnostic for anyone building or evaluating autonomous coding/optimization agents beyond simple pass/fail metrics.
🛠 Tools
msitarzewski/agency-agents — Curated AI agent personalities for popular coding tools
3,032 stars/day · Repo
A collection of 126K-star specialized AI agent system prompts (frontend wizards, Reddit community managers, reality checkers, etc.) installable into Claude Code, Cursor, Codex, and Gemini. A companion native app (macOS/Linux/Windows) handles one-click installation and auto-updates without requiring a repo clone. The agents are deliverable-focused rather than generic prompt templates, with defined workflows and success metrics. Its viral growth (born from a Reddit thread) reflects strong appetite for opinionated, off-the-shelf agent personas.
usestrix/strix — Open-source AI penetration testing agent
2,137 stars/day · Repo
Strix deploys autonomous AI "hackers" that scan your application for vulnerabilities and propose fixes, installable via pip install strix-agent (Apache 2.0). It integrates directly with GitHub Actions and CI/CD pipelines to block insecure code on every pull request, with no setup required via the hosted app.strix.ai. With 33K stars and active development (last push today), it's gaining fast traction as a practical alternative to manual pen-testing in dev workflows.
HKUDS/Vibe-Trading — LLM-powered personal trading agent
939 stars/day · Repo
Vibe-Trading (MIT, 17K stars) is a Python/FastAPI + React 19 agent that gives an LLM comprehensive trading capabilities via a single command. From HKUDS (Hong Kong University), it's installable as pip install vibe-trading-ai and supports multi-language docs (EN/ZH/JA/KO/AR). The agentic trading space is growing fast; this one stands out for its full-stack architecture and academic backing, though real-money use would warrant careful evaluation of risk controls.
📡 Analysis
llm-coding-agent 0.1a0
2026-07-02 · Post · tags: projects, llm, coding-agents, claude-code
Simon Willison bootstrapped a minimal coding agent (uvx --prerelease=allow --with llm-coding-agent llm code) on top of his LLM library, using Claude Fable 5 to write the spec and implement it via TDD in a single session. The resulting agent exposes tools for reading/editing files and running shell commands, plus an unanticipated CodingAgent Python class API. A clean illustration of how quickly a functional coding agent can be bootstrapped when the underlying LLM library provides an agent framework.
Using DSPy to evaluate and improve Datasette Agent's SQL system prompts
2026-07-02 · Post · tags: dspy, evals, datasette-agent, llms
Willison ran an async Claude Code task to apply DSPy — prompted by an AIE keynote — to systematically evaluate and optimize Datasette Agent's SQL system prompt against a gold-standard auto-generated dataset. DSPy identified a concrete flaw: sparse schema listings caused the model to guess column names and enter error-retry loops. A practical, reproducible example of using DSPy for prompt engineering beyond toy demos, directly applicable to any RAG or SQL agent pipeline.
Understand to participate
2026-07-02 · Post · tags: coding-agents, cognitive-debt, geoffrey-litt
Geoffrey Litt's AIE talk introduces the concept of cognitive debt — the drift between a developer's mental model and the actual code after an agent has made large changes. His "understand to participate" framing argues that you need genuine conceptual fluency, not just vague oversight, to remain an effective collaborator with coding agents. A timely philosophical check as agentic coding becomes the default mode: passive acceptance of agent output silently erodes the ability to steer future work.
Sources: HuggingFace Papers API, GitHub Trending, simonwillison.net