This is the detailed follow-up to the Astra video. We repeated 2 of the 8 experiments in MIT’s Sparks of Weight-Space Programming. The other 6 were not run, so nothing here confirms or disputes them.
Each model replied once per game, so Opus’s lead over Astra is unproven. Opus also wrote our code while being one of the models compared, which is why we ran MIT’s own networks and answers through it.
The game footage and network activations come from the saved network, replayed for this video. Published scores are the mean of 1,000 games.
Run the checks yourself: https://github.com/bolnet/context-clock/blob/master/WEIGHTSPACE_HOW_WE_DID_IT.md
Sources:
MIT CSAIL, Sparks of Weight-Space Programming (Takehi, Teoh, Zhang, Khattab): https://rikiyat.github.io/blog/weight-space-programming/
MIT’s logs: https://github.com/RikiyaT/lm_understand-logs
Our runs and checker: https://github.com/bolnet/context-clock
