AI agent runs 40 ML experiments—linter silently broke results

An article describes how an AI agent ran 40 machine-learning experiments on a rented GPU overnight, based on Andrej Karpathy’s “autoresearch” pattern (edit one file, optimize one metric, use Git for checkpointing). The AI agent improved validation loss by 5.9% and reduced peak GPU memory from 44GB to 17GB, keeping 9 experiments and discarding 28 (3 crashed). The main failure came from environment instability: a linter on the remote machine silently modified a hyperparameter in train.py after each save. The agent set SCALAR_LR from 0.5 to 0.3, but the runtime used the linter-altered value, so experiments 30–38 plateaued with no alert or crash. The author lost about four hours of compute until the logs were reviewed. Before training, the same AI agent automation logic was applied to fixing 15 Claude Code skills. 13 were improved, but 3 had subtle regressions (e.g., removing an undocumented “AskUserQuestion” gate and narrowing triggers so real misspelled queries no longer matched). The piece also cites Gartner’s prediction that over 40% of agentic AI projects will be canceled by end-2027 due to escalating costs and insufficient risk controls. The author concludes that autonomous AI workflows need integrity checks (e.g., file integrity/compare-before-run), especially when scaling beyond a toy single-GPU setup. For traders, the takeaway is that “AI agent” demos can fail quietly when tooling or the runtime environment changes, which can affect sentiment around agentic AI investment cycles and cost narratives.