Releases
GPT 6.1 Terra
In a bit of a shock, OpenAI has brought back Terra, a model that was omitted from the original GPT 6 releases. For those that don’t remember, Terra is a mid-sized model, meant to fit in between Luna and Sol in the OpenAI lineup.
In terms of intelligence and pricing, the model is much closer to Sol 6.1 than it is to Luna, being half the price of Sol, and only a bit worse in terms of intelligence (whereas Luna is far behind intelligence wise).
| Model | $ per million (input) | $ per million (output) | Tokens per second |
|---|---|---|---|
| GPT 6.1 Terra | $1 | $5 | 62 |
| GPT 6.1 Sol | $2 | $10 | 31 |
| GPT 6 Luna | $0.10 | $0.50 | 35 |
| Claude Haiku 5.5 | $0.10 | $0.50 | 152 |
| Claude Opus 5.5 | $4 | $20 | 68 |
| Claude Sonnet 5.5 | $2 | $10 | 84 |
Compared to Sol and Astra, an OpenAI engineer summed it up best: “Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (while being faster / cheaper), and a bit worse on most other things.”
He goes on to say that this is not a bombshell release, but rather a nice mid range option to round out the OpenAI lineup. It is nice to see OpenAI working on the model’s communication capabilities, because that is one of the areas it feels quite behind Opus 5.5 in my view.
You should not that based on benchmark scores you should probably only be using Terra at extra high or max reasoning, as unlike other OpenAI models that don’t see to much of a jump between reasoning levels, Terra is very sensitive, with low reasoning basically being equivalent to Luna and max is just behind Sol, so quite a wide range.
Research
Frontier Code 2.0
Good benchmarks are hard to find. For coding benchmarks I had previously relied on DeepSWE since it aligned with my perception of how the models ranked in the real world.
Unfortunately it was a public benchmark, meaning that the model makers could see all of the questions and how they were being judged, and were able to overfit their models on the benchmark so that they would look good on it.
This meant we only got ~1 month of useful scores from the benchmark before it became obsolete, and I have needed a good coding benchmark to point at ever since.
Thankfully the team at Cognition (the team behind Devin and Windsurf) is here to fill the gap that has been left with their Frontier Code 2.0 benchmark.
They don’t just measure correctness of the implementation, but also the mergeability of the code as well. They worked with open source codebase maintainers to build a set of questions and acceptance criteria, along with scorers that not only made sure the new functionality was added or the bug was fixed, but also that the fixes took into account the context that could be found in the rest of the codebase and that it was done in a minimal and efficient way.
They spent over 40 hours per question, and also made prompts where the task was underspecified (as it often is in the real world) and the model must use its judgment to decide what actually needs to be done.
Unlike DeepSWE which was public, Frontier Code is fully private, so it will be much harder for AI labs to overfit, so hopefully we will be able to get more than a month of useful signal from it.
The models that I expect to see at the top of the benchmark are all there. This benchmark is slightly biased towards Claude models, as they tend to be more agentic than the GPT models which tend to be very strict instruction followers, which can be good or bad depending on what you are using the model for.
The benchmark has already been incorporated in the Artificial Analysis V5 Intelligence Index and I will be using it as my guiding star for coding capabilities for the foreseeable future.
Quick Hits
LFM 3
The LFM models have been the best models to run on low resource devices like phones or in the users browser using tools like Transformers.js, and they now have an updated version to further extend their lead in this space.
LFM 3 is a 1.7 billion parameter model whose architecture has been specifically optimized for mobile hardware. Don’t expect it to knock your socks off, but is capable enough to handle basic agentic tasks in a timely manner, fully locally.
Kimi K3.1 Pro
Moonshot AI released the new version of their Kimi model, but unfortunately it looks like there is no reason to use it.
Its main issue is its price given its performance. It is a decent model, but at $10 per million output tokens, and fairly high token usage, it is noticeably dumber than GPT 6.1 Sol while being 3x more expensive.
For coding tasks DeepSeek V4.1 Flash 1017 outdoes it while being 6x cheaper.
It is a fairly decent model, definitely top 3 from China (along with GLM 5.3 and DeepSeek V4.1 Flash) but its price is 2x and 6x higher respectively, which makes it hard to recommend.
Finish
I hope you enjoyed the news this week. If you want to get the news every week, be sure to join our mailing list below.
