PICK YOUR SUPPORT STYLE
MONTHLY SUPPORT
Reader
$5/mo
Contributor
$15/mo
Architect
$50/mo
Recurring subscriptions auto-bill monthly via Stripe Checkout. Cancel anytime from the receipt email.
What was talked about: Qwen 3.8 is a dense 27-billion-parameter model that can run on home hardware with about 24 GB of memory, or less with stronger quantization. Its benchmark results appear inflated, but early real-world use suggests capable coding and agentic performance near leading cloud models.
Takeaway: Qwen 3.8 is a practical local model for non-trivial work, but real-world tests are more useful than its benchmark scores.
Links shown: Qwen 3.8 open-weights announcement, Qwen 3.8 collection, Qwen 3.8-27B analysis
What was talked about: Qwen 3.8 supports low, medium, and extra-high reasoning levels. The lower settings can perform worse than earlier Qwen models, while extra-high reasoning provides the strongest results but consumes many more tokens and takes much longer to respond.
Takeaway: Use extra-high reasoning for difficult work, and include the added latency and token use in project planning.
Links shown: Qwen 3.8 reasoning benchmark, Qwen 3.8-27B token use
What was talked about: Compute Community lets people share models running on their own local hardware. Community members can access shared models through a chat interface with web search or call them through an OpenAI-compatible API from coding agents and other applications.
Takeaway: Idle local hardware can provide shared model access without relying on a commercial inference provider.
Links shown: Compute Community
What was talked about: llama.cpp can split a model between GPU memory and system memory, which makes Qwen 3.8 usable on GPUs that cannot hold the full model. It supports GPUs, CPUs, Macs, and hybrid configurations, with reported speeds near 30 tokens per second on an offloaded setup and 60 tokens per second on an RTX 3090.
Takeaway: Start with llama.cpp for local inference, then tune CPU offloading and memory settings for the available hardware.
Links shown: Compute Community, Georgi Gerganov’s Qwen llama.cpp commands, llama.cpp
What was talked about: Qwen 3.8 received rapid adoption, with millions of downloads for local quantized versions. Its popularity supports the view that more AI inference will move onto personal hardware and that shared local compute could become useful infrastructure.
Takeaway: Demand for capable local models is growing quickly, which makes local inference and compute-sharing tools increasingly relevant.
Links shown: Unsloth Qwen 3.8-27B GGUF
What was talked about: Liquid AI’s LFM 2.5 family was identified as a suitable starting point for local inference on iPhone and Android devices. These small edge models can support chat and basic search tasks, but they remain far below larger local models for coding and autonomous agent work.
Takeaway: Use small LFM models for lightweight mobile tasks, not demanding coding or agentic workflows.
Links shown: Unsloth Qwen 3.8-27B GGUF, Liquid AI models
What was talked about: Long-running agents make response speed an important operational limit. An OpenAI and Cerebras partnership reportedly runs GPT-5.6 Sol at about 750 tokens per second, roughly 14 times the standard speed, though the service is expected to have high prices and limited parallel capacity.
Takeaway: Very fast inference can shorten long agent tasks, but it will likely be a premium option for work where time matters most.
Links shown: Liquid AI models, Cerebras GPT-5.6 inference preview
What was talked about: OpenRouter was offering GPT-5.6 Sol output tokens at half the normal price, with further reductions for batch and flex tiers. The model’s token efficiency and the temporary discount could make it an economical production option.
Takeaway: Check current OpenRouter pricing before running large GPT-5.6 API workloads because the available discounts can materially reduce costs.
Links shown: OpenRouter GPT-5.6 Sol discount
What was talked about: OpenRouter provides one API key and interface for models from many labs, which makes model testing and switching easier. It also supports provider restrictions, free models, detailed cost metrics, and implementation-quality benchmarks. Stripe’s reported acquisition was connected to the potential value of model usage data and automatic request routing.
Takeaway: OpenRouter is useful for experimentation and multi-provider access, especially when projects frequently change models or require provider controls.
Links shown: OpenRouter GPT-5.6 Sol discount, OpenRouter, Kimi K3 providers, OpenRouter Free Models Router, Kimi K3 benchmarks
What was talked about: Live routing could inspect an agent session and move work when a model struggles, but task difficulty is often impossible to predict from the initial request. Switching models also invalidates the existing KV cache, which can require paying again for a large conversation history and make escalation more expensive than starting with the stronger model.
Takeaway: Automatic routing must account for task-specific difficulty and lost cache value, not only the price of each model.
Links shown: Kimi K3 performance
What was talked about: A mixture-of-models workflow can delegate focused tasks to fresh sub-agents instead of moving one long conversation between models. A short context summary lets models with different strengths review a problem without carrying millions of prior tokens, and a fresh context can expose missed issues.
Takeaway: Use concise handoffs between sub-agents when multiple models or fresh perspectives are useful.
Links shown: Kimi K3 performance
What was talked about: GLM 5.3 became available through the API after an earlier release limited to a coding plan. Early benchmarks and limited use suggested an improvement over GLM 5.2, with expected performance near the Kimi K3 tier.
Takeaway: API availability now makes independent benchmarks and broader practical testing of GLM 5.3 possible.
Links shown: GLM 5.3 API announcement, GLM 5.3 coding benchmark, GLM 5.3 on OpenRouter, Vector Lab: DeepSeek V4 Pro, Dillon Mulroy’s anti-slop post, anti-slop repository
What was talked about: Tools such as Herder.dev, Orca, Superset, Conductor, and T3 Code organize many coding-agent sessions across separate terminals and workspaces. They add status indicators, input alerts, clickable interfaces, and cross-terminal commands so an orchestrator can coordinate work and prepare a summary after unattended runs.
Takeaway: An agent multiplexer can make many parallel coding sessions easier to monitor and coordinate than plain terminal tabs or tmux alone.
Links shown: Herdr, Herdr repository, Superset vs. Mux, Conductor, T3 Code
What was talked about: A reported Google Cloud strategy emphasized fast specialized models, search knowledge, smaller local models, and expanded TPU and cloud infrastructure. The account suggested that Google can benefit when outside AI labs run workloads on Google Cloud, although parts of the strategy were treated as corporate messaging rather than confirmed direction.
Takeaway: Google’s AI position may depend as much on search, TPUs, and cloud hosting as on building the largest general-purpose model.
Links shown: T3 Code, Vector Lab: Google loses their key players, Susan Zhang on X, Compute Community
What was talked about: Anthropic extended increased subscription limits, while practical use still reached Claude limits sooner than Codex limits on comparable premium plans. Fable 5 appeared to improve after early weak results, and first-party coding harness updates seemed to improve model behavior. GPT-5.6 Sol worked best inside Codex or for focused one-shot tasks, while Luna Max provided strong general results and could select higher reasoning levels when needed.
Takeaway: Model quality depends strongly on the harness, task shape, and recent service tuning, so test models in their first-party environments before judging them.
Links shown: Claude Code limit extension, Claude status, Prime Intellect Lab, Thinking Machines Tinker, Tinker documentation, Tinker models and pricing
0:19 All right, I think we’re all set here, so let’s get started.
0:24 Welcome back to AI Tools Club this week.
0:27 If you have anything you want to discuss, this is a community meeting or ask.
0:31 Feel free to toss it in the chat here, whether it’s on YouTube or here in Discord.
0:37 I don’t have a ton to talk about today, so this will definitely be talking about what the community is interested in as well, not just the various things that I have to say.
0:48 So yeah, let’s get into things. I think we’ll start off this week with the latest model from the Alibaba Quinn team.
0:58 This is a dense 27 billion parameter model.
1:01 So if you don’t know what that means, it’s basically a small model that is runnable at home.
1:07 So this will take about 24 gigabytes of memory to run.
1:10 You could probably get it down into 16 gigabytes if you’re willing to quantitize it down and lose a little bit of quality from it.
1:18 Being a dense model instead of a mixture of experts model, it will be much better suited for GPUs rather than Macs, just because it requires a lot more computational capabilities versus
1:32 just memory bandwidth like Mixture of Experts models do.
1:37 But in terms of capabilities, this model seems to be very strong.
1:41 Benchmark-wise, on artificial analysis, it actually is around GLM 5.2 and GPT 5.6 Luna, but that is also a bit benchmarked, I would say.
1:53 The Quent team is pretty notoriously known.
1:56 Jordan, can you mute, please give that go?
1:59 Sorry, sorry. Yeah, the Quent team is historically known for benchmaxing their models.
2:05 They train on benchmarks or train on data that looks eerily similar to what these benchmarks are trying to measure.
2:12 So they tend to be a bit overzealous in terms of benchmark scores.
2:17 But in terms of real-world capabilities that I’ve seen from people, it seems to be around like Opus 4.5 levels or deep CP4 flash, somewhere around there, where it is a very capable real-world model.
2:30 That is, I’d say before you could do like very light coding with these local models like the Quen 3.6 series.
2:37 But now it seems like this is an actual sort of useful sub-agent potentially to be able to go and do these different tasks that are sort of like non-trivial agentic tasks.
2:49 I will say it does. So I guess actually in a departure from the previous Quenn models, which either just had reasoning on or off, this has a much more sort of modern approach, where it has different reasoning levels for it,
3:03 of low, medium, and extra high. And what I’ve seen from the community so far is that these low and medium settings aren’t that great, especially for more difficult problems.
3:13 You can see here actually the new 3.8 model actually does worse than the previous 3.6 mixture of X-rays model on low and medium reasoning levels.
3:22 And you only start to see those increased capabilities at extra high reasoning.
3:28 So most inference engines, this is what they will be defaulting to, is extra high.
3:32 But do know that it does go and use a whole bunch of tokens.
3:36 I believe somewhere on the artificial analysis page, of the models that I have up here that we’re comparing to, yeah, it uses the most tokens of pretty much any model.
3:46 It uses more than Opus or GLM. Compared to Luna, which remember, it is roughly the same benchmark-wise as it uses two and a half times more tokens, roughly.
3:57 So about two and a half times longer to reach a response.
4:02 But yeah, that being said, still a very interesting and useful model.
4:05 Like I said, it seems to pass the vibe check for sort of like real world use.
4:10 I haven’t seen anybody really complain.
4:12 It’s been a ton of people have been going and using it.
4:16 And if you want to use it as well, we’re actually hosting it on Compute Community.
4:22 Let me actually get the invite link.
4:24 But for those that don’t know, this is a platform I’ve made that I’ve sort of haven’t done too much with recently.
4:29 But I’ve recently, like this week, started revamping a bit.
4:32 But this is a way to go and share models that you are running locally with other people.
4:36 So you can go and create communities.
4:38 You can see here we have the Sunday Club community here.
4:41 And Nate and I were both running different Quen models.
4:45 So you can go in here, and if you’re part of this community, which I just sent the invite link to in the chat here, you can go and chat with it here, talk with it, say, hey.
4:56 Nice thing is by default on the Compute Community site, it has web search, so you can ask stuff like, what is Sunday Club?
5:05 And it won’t be able to answer. Oh, because I put in weird RAID limits.
5:09 But yeah, you can go chat with it. You can also use it via the API, so you can go and plug this in to stuff like clawed code.
5:16 If you go to MyNote specifically here, I give the code here for how to go and call this programmatically.
5:21 So you can go and use this like any of the other open AI models or anything like that.
5:28 So yeah, if you want to go give the model a try, you can go in here and try it out.
5:34 But yeah, has anybody been able to go and use this model, have any thoughts or comments about it at this point?
5:48 No specific questions about the model.
5:50 This is my first introduction to compute community.
5:54 Also, hi, this is Charlie. I was curious, who’s footing the bill for free LLMs?
6:01 So these are the ideas that I know there’s a decent number of us that Sunday that, or just in general, but we have like decent hardware, and a lot of times we’re running L1s at home locally.
6:11 But a lot of the time they’re just sitting there idle.
6:14 And so this is a way basically to go and share that compute.
6:16 So for instance, like this node, this is literally just running on my local PC right now, my 3090 that I’m sitting next to.
6:26 Yeah. So this isn’t like, we’re not, this isn’t a proxy like open router for another API provider.
6:32 This is for people who are actually running and like have the hardware to run these locally to go and share that.
6:37 Because yeah, like I said, we have all this, but it’s just sitting around.
6:41 And it’s gonna not to distract from the model question, though.
6:44 Yeah. Natural Stupid says they only have a 4080.
6:47 Seems the model’s still too large. So yeah, actually, interestingly, you can run it with tools like Llama C
6:55 They have… So Llama C
7:04 And so they have the ability to go and split the model between your system memory and your GPU memory.
7:10 So yeah, you can split the model so that, you know, two-thirds of it will live on your 4080, and then the other one-third will just sort of be moved back and forth between system memory and the GPU memory.
7:20 And they’ve optimized it to the point where it is fairly good, I would say.
7:25 Where it’s like actually usable. I think for this model, you’ll expect to see probably around like 30 tokens per second, which is definitely a usable number to go and use.
7:35 So let me get that for you. I forget, there’s basically just a set of flags that you can go and configure for CPU offloading in Lama C
7:45 You can go and ask your favorite local agent or cloud agent, and they will be able to tell you what to do for that.
7:55 But yeah, I’d also say, yeah, if anyone’s looking to run Quinn 3.8 locally, if you have a GPU or if you have a decent spec Mac, I would start out with Lama C here.
8:05 This works across pretty much all environments, like I said, like GPU, CPU, Mac, and then like hybrid, running it partly on the CPU and the GPU at the same time.
8:14 It’s got a huge community. They’re pretty much always up to date, all that sort of thing.
8:19 That’s how actually I’m running it right now on my 3090, just because it’s the easiest to set up.
8:23 If you don’t have to think or even ask an AI how to run it on Compute Community, I have the command that I ran to be able to go and run this model.
8:32 So you can just go and copy this, run it on your own machine.
8:34 This is meant specifically for running on a 3090.
8:37 You might need to tweak some things if you’re using other GPUs or on a different system.
8:42 But yeah, you can see what works for me and what I got it working.
8:47 But yeah, this runs at about 60 tokens per second for me.
8:52 So pretty usable speed. Anything else about this model that we want to talk about?
9:07 Having a local dev model, it’s amazingly fast how soon this has become available.
9:15 Even if it is benchmark max, like you said, it’s some indication that for normal people who don’t have 128 gigabytes, this is awesome.
9:29 And I definitely think this is sort of the future where I think the vast majority of LLMs will be moving locally.
9:36 And that’s sort of the bet of building compute community in the long-term version is that we are going to see more of this local inference and being able to harvest it in a meaningful way and share it.
9:46 That’s the goal. I do know this model specifically has gotten a huge or like a very strong reception.
9:53 I think it’s already like top five most liked models all time on Hugging Face.
9:58 It’s 600,000 downloads here, but I think for the actual quants that you’d run locally, I think, yeah, 3.5 million downloads already in like a week.
10:09 So yeah, definitely a lot of people want this and are interested in this.
10:13 And this is, yeah, just based on these downloads, I think it’s a useful model and people are finding reason to run it.
10:19 Mattro Stupid says, do you have any good LM recommendations that run locally on iPhone or Android phone?
10:26 Most likely the LFM models are probably the best ones.
10:30 I would go and look at, yeah, LFM like 2.5.
10:33 So this is from Liquid AI. They’re actually here in Boston.
10:37 But they have a variety of models here.
10:38 These are sort of meant to be edge inference models that are meant to run on stuff like your phone and whatnot.
10:45 So I would start here. They’re definitely not at the same tier as the Quen models are.
10:51 They are far behind. They are also much smaller.
10:54 But they are, you know, you can have a chat with them.
10:57 But I would not expect them to be doing any, you know, agentic coding or anything like that.
11:02 They could probably do some basic web search for you.
11:05 But anything beyond that, I think, would be a bit much for them.
11:09 Aiju says, token consumption is not only about spending more, but also about taking longer.
11:14 If you have any experiments at scale, need notice in the project planning.
11:20 That’s yeah. Definitely. It’s yeah, we’re getting to the point now where these long sort of horizon agents can spend, you know, double digit hours running on a single task.
11:35 So yeah, time is definitely becoming a factor for running faster is more important.
11:42 Which actually we can briefly touch on speed because we’re getting more speed from OpenAI allegedly in the future.
11:51 So for those that don’t know, we’ve talked about this previously, but OpenAI and Cerebris have partnered.
11:59 And Cerebris, for those that don’t know, they make really good inference chips.
12:03 They’re basically like physically massive inference chips that are meant for running LM super quick.
12:10 And so yeah, they have a partnership and they apparently have been able to get GPT 5.6 sole up on their own custom inference engine on their custom chips.
12:19 And now it runs 14 times faster than what you get in the regular sort of codecs default plan.
12:25 So 750 tokens per second. So you can imagine for these super long horizon tasks how important and useful this would be.
12:35 So yeah, this is currently, I believe, it’s sort of like a closed beta, but hopefully this will be coming out in the future.
12:44 This, though, I would guess is going to be very expensive, probably about 10x more expensive than the traditional GPT 5.6 prices, because the Cerebrus chips are not really meant for running large models.
12:59 They are meant for smaller models and running those very fast.
13:05 So yeah, this is probably going to be expensive.
13:07 And they’re also not very good at high batch sizes.
13:09 So they can’t serve a ton of users in parallel.
13:11 So yeah, but it will be if you need speed and speed is what really matters.
13:16 We will be getting this shortly with this.
13:21 And then I guess why not, since we’re also still talking about OpenAI and prices, for those that don’t know, if you’re using the OpenAI API for anything, if you’re not on, you know, like a Codex monthly subscription plan,
13:34 maybe you have an agent in production that you’re calling via API.
13:38 On OpenRouter right now, GPT 5.6 sole is half off.
13:43 So it’s $15 per million output tokens.
13:45 And the GPT 5.6 series models were already sort of the best price to performance models out there just because of how token efficient they are.
13:54 So this, yeah, having the price can get very, very cost efficient for you compared to any other model right now to go and use in production.
14:05 And yeah, they have a bunch of different tiers, so this applies to everything.
14:08 So I think like the batch API and the flex API, those are both half price.
14:12 And those that are already half price are now getting half.
14:14 So it’s a quarter of the usual $30 per million output tokens.
14:16 So that’s where we get the $7.50 per million output tokens here.
14:22 So yeah, if you’re using that for anything, know that that’s the case.
14:25 My guess this is probably also going to be you sort of like filtered down into the main API as well in the future.
14:33 But yeah. Any questions or comments about that?
14:43 I actually questioned, how many people are using OpenRouter right now?
14:46 So I know if you guys didn’t hear, OpenRouter got sold to Stripe for $9 billion.
14:53 So yeah, they’re doing well. So I wanted to know, do people actually use it here or is this some weird purchase by Stripe?
15:04 I just started to use it just today to get to some open source models because the open code Zen provider was very unreliable.
15:16 And it just happened today, so I have no information.
15:22 But it will be. And I should say, for those that don’t know, OpenRouter is a way to go and access basically every model through one unified interface.
15:31 So instead of needing to have an Anthropic API key, an OpenAI API key, a Quen API key, you just have one API key that allows you to access all of these models.
15:42 I’ve been using OpenRouter a lot for the open models.
15:47 I’ve got ZVR enabled. I’ve also done my own personal research to figure out what providers are domestic in the US and also legally incorporated in the US such that my
16:01 account will only hit those providers.
16:04 It does result in, like on this screen, on my account, like half of these providers are crossed out as a result of that.
16:14 But there’s still plenty that are available.
16:20 Nice, yeah. Siddharth and Zach, I saw you both unmuted.
16:25 No, yeah, really briefly, I just use it to swap between models for my own personal agent.
16:30 Just I’m constantly testing different models and see what output I get.
16:35 I just find it easy to switch out models for my Hermes agent.
16:41 Nice. Yeah, it’s definitely super useful for, yeah, just like when you’re tinkering or if you have stuff where you’re always updating the model, yeah, this is super convenient.
16:48 I’ve used this on a few projects before with different clients.
16:51 It just makes everything much easier for experimentation.
17:04 Siddhartha, if you’re trying to say something, I think you’re very laggy.
17:07 We cannot understand. And Nikolai points out that, yeah, OpenRouter, they also have a bunch of free models on here as well.
17:17 So yeah, he says, yeah, if you’re trying to use it for some hacks or just tinkering with some stuff, you can use the different models here.
17:24 I think actually high three here by 10 cent.
17:27 We haven’t discussed this before at all, but this is a fairly solid model, I believe, to go and use.
17:33 And so yeah, you can go and utilize this for free, it appears, on OpenRouter right now.
17:44 Yeah, we’ll see. Because I think the long-term play, the reason that Stripe is acquiring OpenRouter is because of this idea that, oh, because everyone uses them and all the requests are routed through them, they can build
17:57 a router to go and determine, oh, which model should I send this request to to sort of like cost optimize it.
18:04 This is something Sakana AI has also done before, and it seems like people actually used it.
18:11 So like routing between different models to get the best price to performance.
18:14 So I think that’s the bet with OpenRouter here and why Strike acquired them for the long term.
18:20 Yeah, they’re going to know more than anyone, more than any single lab about use.
18:28 I mean, even if it’s anonymized, they still will know a ton of how things are really used and what people need.
18:37 I know that they’ve, like over the past couple years now, I guess, they’ve been slowly adding in more and more of these metrics.
18:43 They’re measuring sort of like actual pricing, like factoring in like cached input prices and all that sort of thing.
18:49 So what is your actual output cost performance of the different providers as well?
18:54 And then they’ve also started benchmarking the providers to see how accurate their implementation is compared to the reference implementation from the lab itself.
19:05 So yeah, they have a ton of data like that to be able to go and effectively route these models.
19:10 And then also, yeah, they’re incorporating benchmarks in as well to know how good a given model is.
19:17 But yeah. I think it’s in, I’ve seen arguments on both sides.
19:22 I think I’m definitely more in the camp of routers aren’t going to be very useful.
19:28 I think that a lot of the questions you don’t know ahead of time if they’re going to be difficult or not.
19:33 But I also know there’s a lot of very smart people saying that routers are the future, and this is going to matter a lot.
19:38 So Stripe is obviously betting on that.
19:40 So we’ll see if that’s a correct bet.
19:43 I heard somebody from Open Router say today that they take a glance at the beginning and the middle and late in a session to see if the routing is correct.
19:55 And I haven’t really internalized that to figure out what that would look like.
20:00 But that’s like a live view. Is this model stumbling?
20:07 Does the context need to be set somewhere else to a smarter model?
20:14 I didn’t want to depend on it, I guess.
20:16 But maybe it’s good if you characterize two or three models that are good for you and you let the router move your work around.
20:26 There’s two issues that I see with model routing like that.
20:29 The first is that even if you learn and update your sort of routing model to know what a hard query might look like, it’s very task specific.
20:39 So you can take, for instance, like, oh, let’s ask my model, like, oh, can you get the number of users that my app has right now?
20:45 That might be super simple. You might just have, you know, a function or like a SQL query pre-baked to go and call the number of users that you have and find out.
20:54 Or it might be very difficult. You might not actually have production database credentials.
21:00 Or you might not actually store users in a normal way.
21:02 Maybe it’s all in local storage or something, and there’s no real good way for you to get it.
21:07 See, it’s hard to tell ahead of time whether or not a query is going to be difficult or not.
21:12 You could probably have a good sense, like if you structure your code base like a sane and normal person, then the routing probably could work.
21:19 But then you run into the second issue, which is KV cached reusage.
21:23 And so we find that the vast majority of tokens are part of the cached input tokens, you know, like the previous history of the model.
21:32 And whenever you go and switch models, that KP cache gets invalidated, and you have to go and pay for all those input tokens afresh all of a sudden.
21:41 And so a lot of the times, you will actually end up being more expensive using sort of like a cheap model and then switching to an expensive model, you know, and escalating it as the task that’s more difficult or it’s found
21:52 that it’s more difficult because you have to go and repay for all those input tokens for that big model, which the big model, if you’d just given it from the start, would have figured it out quicker.
22:02 And yeah, not used as many tokens. So it would have actually been cheaper.
22:10 Mike says, a strike up. Sorry, you got Will.
22:14 I found in that regard that using models that are equal in intelligence, but different in characterizations of how they handle things is good.
22:28 So I’ve been using like Opus 4.8 and then GLM 5.2 and then a little bit of 5.6 Sol and just kind of going between them.
22:39 And I’ve found that if you ask one to summarize it for a sub-agent, it doesn’t really care at this point who it sends it to.
22:50 It tends to give it enough context to be able to say, hey, I’m using this strategy for this concept with this information.
23:02 How would you go about looking at this?
23:05 And I’ve had a lot of success sending millions of tokens all at once and it just sending like maybe 20,000 tokens as a request and then getting like 7,000
23:20 back. Yes, yeah, sort of a mixture of models approach.
23:27 Instead of having one long thread with multiple models in it, you just delegate to sub-agents and have them communicate back and forth.
23:33 As I know some people have built like whole message boards and stuff for their agents to talk to each other about different tasks.
23:40 But yeah, that’s definitely another way.
23:42 It’s because yeah, these models usually, or even like the same model, but just a fresh perspective, like reset the context window.
23:48 A lot of times we’ll have a different view or different understanding or be able to point out some obvious issues a lot of the times.
23:56 But yeah, Mike says, is Stripe not public?
23:58 Was trying to look them up. Yeah, the er, actually yeah, Stripe is not public.
24:03 Open router was not either. Naturally Stupid says why Opening Anthropic has no model router.
24:09 I think this will be very good. I think once again, it’s like a very, it’s a hard, it’s a relatively intractable problem of just knowing what is difficult.
24:17 You can try and make a router, but there’s a reason nobody has been successful for this.
24:21 I mean, model routers have been an idea since, you know, like pre-Chat GPT.
24:27 People have talked about routing between different models and stuff for different types of queries.
24:31 And it’s a very difficult task. And it’s not reliable enough in production.
24:35 And so you’ll just get a lot of people complaining about how it doesn’t work.
24:39 That would be my guess. Mike says router encourages variety ETA providers.
24:44 As I assume Naturalist Dude was talking about internal, like only routing between open AI models or only routing between Anthropic models.
24:52 I think the other reason is that they have no real need to.
24:55 People will just pay for the tokens anyway.
24:58 And they’re sort of leaving it up to their customers to figure out when to switch up and down to different models.
25:03 Because I think by default, everyone just uses the smartest model and just pays the price for that instead of using the lesser models.
25:16 Or in the case of Fable, you know, unpromptably downgrading people.
25:22 The safety filters route you as needed.
25:24 That’s the only model router we have.
25:29 And yeah, any closing thoughts here about open router or routing or anything like that?
25:40 If not, we mentioned briefly there, where is it?
25:46 GLM 5.2. And so as of, I believe, was this 45 minutes ago now, GLM 5.3 is now officially live in the API.
25:56 So some of you probably saw last week that GLM 5.3 was kind of released.
26:03 It was only released in their coding plan on their platform.
26:08 There was no API release with it, which is why I did not cover it, because all the third-party benchmarks and stuff had not been able to be run, and a lot of people hadn’t really used it.
26:19 So yeah, performance-wise, at least from the benchmarks and the little that I’ve seen so far, it seems to be another step up from GLM 5.2.
26:28 My guess is this will put it around Kimi K3 level, maybe a little bit better.
26:34 Oh, lost audio. Did I just lose audio?
26:38 Oh, somebody here. Yeah, can you repeat yourself?
26:42 We lost, like, we can hear you now.
26:44 We lost 10 seconds of your audio. You’re good.
26:47 I think people are hammering on the compute community.
32:08 Jordan, you go. Go ahead. Sure. Okay.
32:12 I’ve been using agent multiplexers, like Herder.dev, but there’s also a few others.
32:17 This is the one I use. There’s others like Orca and Superset.
32:22 So it’s like if you run a lot of different agent jobs, I think it’s more relevant for mine because I have a bunch of issues that I just want to have agents one shot, or try to.
32:32 And then they’re not so tightly closely related.
32:36 But yeah, it’s just a way where you can see multiple agents and they can be using different harnesses.
32:41 And you can kind of have them work and they have their own Tmux pain.
32:47 But it also lets them have different workspaces and have different putting agents under it.
32:52 I wonder if people use stuff like this.
32:54 And I feel like it made me more productive in using more than eight agents.
33:00 Like before, I would have a bunch of terminals or shells and they’d all run their own thing.
33:05 But I feel like this is not bad too.
33:07 There’s another one like Orca and Superset that I want to try too, but it’s kind of like a lot of work to try these things.
33:14 These are both like YC-backed companies that are doing the same type of thing.
33:18 And those conductor.build. Yeah, I wonder if people have experience with this.
33:24 Yeah. I don’t understand what they do.
33:28 Can you help me out? Yeah, so do you know what Tmux is?
33:35 It’s just a way to run multiple terminals or coding agents at once.
33:42 And then they also tell you the status when stuff goes on.
33:44 So like Tmux is just like multiple different shells that you can have.
33:48 And then they just add a little bit more where it’s like agent-friendly.
33:52 They can see if your agent needs your input or whatever.
33:56 I think one other thing they do is they allow for orchestration outside of just Claude itself.
34:03 So a Claude terminal can message other Claude terminals.
34:08 But what these do is they have this little CLI.
34:10 So there’s this herder CLI. So anything that’s running in a herder terminal, you can tell it, it can pass commands to each other.
34:18 So you can have an orchestrator orchestrate all your other terminals.
34:22 So when I sleep, I have 20 terminals, and I’m like, all right, orchestrator, make these things keep working, and then give me a summary at the end of stuff I need to respond to in the morning.
34:32 And then it does it. And you can do this across different coding agents.
34:38 Yeah. Thank you. But yeah, I’m really interested in what’s the best one of these.
34:45 If people have experience, it’d be cool.
34:47 I’m using harder right now, but like, okay, now let’s try something else.
34:50 It’s kind of annoying to change that stuff.
34:53 Yeah, it is a meta harness. Is it really a harness, though?
34:56 It’s just like something that organizes other harnesses.
34:59 It’s not really a harness itself. Yeah, yeah, yeah.
35:08 And the last one was conductor.build.
35:12 I was using this back when I was in Boston also.
35:14 But it’s like Mac only. I have like a remote machine and a local.
35:19 Yeah, I just want things to work similarly across.
36:02 I was using Tmux and then I changed the herd just because like the little UX stuff, like the little bing when stuff is done, actually kind of made it better.
36:11 Yeah. Okay. Yeah,
36:30 I guess it’s similar. No one’s maybe
36:44 Nikolai just does this with all CLI or something.
36:53 Sorry, I dropped. Whoa, Tmux, what?
37:13 Oh, absolutely. Yeah, Tmux did. I installed it.
37:17 I don’t know how to use Tmux all that well, but Claude does.
37:20 So it’s been occasionally creating its own, creating and managing its own Tmux sections.
37:27 If you want a Tmux with a clickable UI, I recommend Herder.
37:32 Because I used using Tmux before, and I had to have a note thing on my thing to tell me the hotkeys so I would remember.
37:39 And then I was like, there has to be something better.
37:41 And then I ended up using Herder. Yeah, I saw somebody at DEF CON who had Ghosty running, Herder running in Ghosty, and fully customized.
37:52 And I was like, okay, this is nice.
37:55 Wait, who was it? He might have introduced me because I met some guy in SF that told me to use this.
38:01 I don’t know. Okay. Yeah. Yeah. If you’re in the same modes as Who Knows About Her, then you almost certainly
38:15 know about Ghostie as well. Okay. Well, that’s my contribution.
38:26 I’m definitely going to try Herder because I’ve just been using Ghosty tabs in Windows.
38:32 This seems better. Yeah, and if you don’t like Herder, T3 code is also pretty good.
38:42 It’s kind of a similar concept where it’s using your harness orchestration as opposed to agentic orchestration.
39:20 I mean, usually it doesn’t. Like, you can have stuff in different workspaces, but I have like this main review thing where it’s able to tell different review bots to keep going, and it works like how you’d expect it would.
39:35 It doesn’t really spam each other that much, but they have the ability to mess with each other, usually only when I ask for it explicitly.
39:41 I haven’t really noticed it like going off the rails and like randomly messaging each other.
40:05 No, yeah, some news I think it will be valuable to share with the community.
40:10 I know we spoke, Google’s kind of dead, but I spoke to a Google Cloud Enterprise lead there, and I was picking his brain on their AI strategy.
40:19 And like the community said, obviously they’re giving up on big models, and their whole priority now is focused on speed and general search knowledge.
40:29 They’re focusing on specialized models.
40:32 They’re kind of giving up on being the big dog because it’s just from a CapEx perspective on running the models.
40:39 It doesn’t make sense locally. This guy said the change up in DeepMind was overall good for the organization.
40:46 They’re going to get some new people in to help organize DeepMind, the focus on more smaller, specialized and fast models optimized for search.
40:58 And in terms of the AI arrays, they don’t really care who wins as long as they run on Google Cloud.
41:04 They’re going heavy in their TPU business and they’re supporting heavy supporters of local models and smaller models.
41:12 I just thought that would be some information that would be cool to share about the forward looking of DeepMind, Google’s AI strategy.
41:37 Who are the insiders? Where did they work in Google?
41:49 Yeah. Interesting. Interesting. My god, everything that you were told is essentially the corporate, is towing the corporate line.
42:03 And there’s absolutely no way that half of it’s like that half of it’s accurate.
42:10 But this guy’s leading the Google Cloud Enterprise team.
42:14 So do you think it doesn’t matter when he says that there when he says that there are people who left Gemini that don’t matter and you look at the people who left Gemini, then
42:28 that person doesn’t know what they’re talking about.
42:30 No, he didn’t say it doesn’t matter.
42:31 He said it’s his exact words was it’s a welcoming change and new leadership to focus on their new version of DeepMind.
42:42 But no, no, no, it’s interesting to get different perspectives.
43:01 It’s the only move they have. It’s sorry, but it’s the only move they have.
43:08 If their search business is dying and their cloud product, don’t tell anybody, but OpenAI is their biggest ad spend spender for ad spend and search.
43:22 No, that’s their core of their business.
43:24 And Anthropic, they have a huge business of Anthropic, and they’re running like, I think Anthropic is running like 40% of their models on Google Cloud.
43:31 So they kind of, their whole strategy is add, search, and OpenAI or Anthropic, they’re winning, no matter who comes out on top.
43:41 And I found that interesting. But that’s more business and less AI related.
46:58 I don’t know if everyone saw it, but Anthropic extended their increase in subscription rates till the end of the month.
47:33 Yeah, and I’m still getting a lot more out of Codex.
47:37 You know, I have both Max, both $200 plans, and I always starve out on Claude first.
47:42 And I’m using them both. I use Opus 5 and Fable.
47:53 I have found that Fable 5 got a lot better.
47:56 It was really weird earlier. Now it is at least as good as Opus 4.8 was.
48:04 But, you know, I think there was something, something has been tuned.
48:09 Of course, something has been tuned.
48:11 But, you know, my first experience was really bad.
48:15 And now it’s solid. I’ve been using 4.8 a lot ever since Opus 5 was just bad.
48:26 And even I’ve just been staying mostly on the cloud code harness.
48:32 And I think that they’ve made some updates to the harness that have significantly improved the way that the models, especially their first-party models, interact with the harness.
48:45 That might be it, yeah. I’ve also told all my models to explain things to me like I’m 15, which is helping with some of the super
49:00 dense sentences that were making me crazy.
49:05 Yep, ELI5, TLDR, ADHD, all those. I will say, are you using, you said you’re using codecs.
49:21 I haven’t found that 5.6 Sol works for anything good other than one-shotting prompts and then getting feedback or like staying inside of the codec
49:36 harness, but like on very specific tasks that I don’t really care what it’s doing on.
49:43 Because like if I try to use it with cloud code or any other harness, it just kind of the harness doesn’t really know what to do with it.
49:52 It doesn’t output it correctly. And then it’s just a mess.
49:58 So my experience is maybe the same as yours, but I don’t look at the code.
50:05 And so lately I’ve been running codex at Luna Max and then just saying, I have a scale fix issue.
50:14 Fix issue and pick the right model for it.
50:17 And that has the self-selection of the model level and it has gone up to, I think, Sol High, the highest that I’ve seen.
50:26 That’s worked out. LunaMax all by itself is great.
51:17 Oh, so Will, you’re not using the Codex desktop app or anything or the CLI?
51:22 I use all four of them. I use Claude, I use Claude Code, I use Codex, and I use Codex CLI.
51:29 And I think it’s a mix of what you said and what Andrew said, which was I am, in some respects, I don’t care sometimes when I’m looking at like specific code pieces.
51:43 Like if I just want something done and I just trust the model to go do it, then Codex is great because it doesn’t show I
51:57 think you dropped out well. I don’t know if you can see it.
53:17 And then this you can like they have servers or they have compute you can use to train these like models?
55:09 Thank you.
Subscribe to get the latest AI news in your inbox every week!
