In my opinion, the most important takeaway from this result is that our @OpenAI International Math Olympiad (IMO) gold model is also our best competitive coding model. 🧵

After the IMO, we ran full evals on the IMO gold model and found that aside from just competitive math, it was also our best model in many other areas, including coding. So folks decided to take the same exact IMO gold model, without any changes, and use it in the system for IOI.

The IOI scaffold involved sampling from a few different models and then using another model and a heuristic to select solutions for submission. This system achieved a gold medal, placing 6th among humans. The IMO gold model indeed did best out of all the models we sampled from.

To be clear, this system used scaffolding, though a lighter scaffold than last year. It only decided which samples from general-purpose models to submit. I’m optimistic that next year we’ll feel confident the model itself can do better than any scaffold we could come up with.

I was not involved in this work. Big congrats to @sherylhsu02, @alexwei_, @bminaiev and oleg murk, as well as @_lorenzkuhn, @MostafaRohani, @clavera_i, @andresnds, @ahelkky, and many many others on this result!