PICK YOUR SUPPORT STYLE
MONTHLY SUPPORT
Reader
$5/mo
Contributor
$15/mo
Architect
$50/mo
Recurring subscriptions auto-bill monthly via Stripe Checkout. Cancel anytime from the receipt email.
What was talked about: DeepSeek V4 Flash adds stronger coding and agent training to a small, efficient model. It approaches the quality of much larger models such as GLM 5.2 while costing about $0.18 per million output tokens. Heavy parallel-agent use consumed little of an OpenCode Go subscription limit, which makes the model practical for continuous implementation work.
Takeaway: DeepSeek V4 Flash offers strong coding performance at a cost low enough for large-scale and long-running agent workloads.
Links shown: Vector Lab: Opus 5 Update, DeepSeek V4 usage-limit comparison, Vals AI DeepSeek V4 Flash results, Cline cost-per-task comparison
What was talked about: DeepSeek V4 Flash produces about 50 tokens per second, but it can use roughly twice as many tokens as comparable US models for the same task. Its effective task speed is therefore lower than raw generation speed suggests, though its cost remains competitive. The expected V4 Pro model may improve reasoning quality, but the earlier Flash model was already close to Pro performance.
Takeaway: Evaluate total tokens and task completion time, not only tokens per second; Flash may remain the best DeepSeek option for price-to-performance value.
Links shown: Cline cost-per-task comparison, DeepSeek V4 Flash on OpenRouter, Artificial Analysis, DeepSeek V4 Flash announcement
What was talked about: Pangram V4 identifies AI-generated text in unusual cases, including conversations made only from animal sounds, mixed human and AI passages, and generated number sequences. It can classify individual parts of a document and has resisted repeated attempts to optimize model output against its detector. Its training approach appears to use large sets of human and AI samples whose embedding patterns remain measurably distinct.
Takeaway: Pangram is a credible AI text detector, but results should still be treated as evidence rather than absolute proof.
Links shown: Pangram 4 announcement, Pangram cat-and-dog detection test, Pangram 4 technical report
What was talked about: An internal OpenAI model called Astra, believed to be part of the GPT-5 family, produced solutions for ten substantial open mathematics problems. The reported inference cost was about $2,000, and the results may represent work that would normally support major academic recognition. Released traces could help determine whether the proofs rely on novel insights or combinations of known methods.
Takeaway: Advanced models are becoming useful research systems for open mathematics, and previously unsolved problems may become important model benchmarks.
Links shown: Noam Brown on Astra’s mathematics results, OpenAI: Ten advances in mathematics
What was talked about: Apple alleges that OpenAI obtained confidential hardware information after hiring Apple engineers for its custom AI inference chip effort. OpenAI contests the account and claims Apple omitted evidence, contacted the wrong people, and continued seeking information from former employees. Public filings expose an unusually detailed conflict over employee movement, intellectual property, and custom silicon development.
Takeaway: AI hardware competition now includes serious legal risk around recruiting, confidential documents, and knowledge transferred by former employees.
Links shown: OpenAI: Apple is getting this wrong
What was talked about: A coding benchmark improvement loop ran for about 18 hours and consumed an estimated $800 of model usage without fixing the failing tests. The agents focused on Docker isolation, watchdog services, Perl scripts, C code, and other infrastructure that was outside the benchmark’s main need. Repeated self-critique created scope growth because each new concern was accepted without enough judgment about its value.
Takeaway: Open-ended self-improvement loops need strict scope, success criteria, and human checkpoints or they can spend heavily on irrelevant work.
Links shown: OpenAI: Apple is getting this wrong
What was talked about: GitHub issues, pull requests, CI results, and independent Claude and Codex checks can give agent loops feedback from outside their own conversations. Goals, hooks, cron jobs, and completion triggers can also form recurring automation patterns. These mechanisms make progress visible and provide concrete signals instead of allowing agents to validate their own work indefinitely.
Takeaway: Connect agent loops to deterministic tests, CI, and review events so that external evidence controls the next iteration.
What was talked about: Claude Code dynamic workflows can use JavaScript to define structured, repeated calls across a set of issues or tasks. Separate agents can read an issue, trace it through a codebase, create a report, and update the issue through a predefined sequence. This resembles recursive language model workflows and supports loops, branches, conditionals, and guaranteed coverage.
Takeaway: Code-defined workflows are more dependable than free-form recursion when every item must pass through a known sequence of checks.
Links shown: Claude Code dynamic workflows, λ-RLM GitHub repository, λ-RLM paper
What was talked about: Custom long-running harnesses behave differently across models, which makes model-agnostic loops difficult. GPT-5.6 variants sometimes added unrequested safety restrictions, refused authorized logins or Git pushes, skipped verification steps, or created incorrect Git identities. Luna also has limits as a Codex sub-agent because it does not reliably support the required inter-agent communication protocol.
Takeaway: Test each model against the exact tools and authorization steps in a workflow, and verify that long-running agents do not silently change policies or identities.
Links shown: λ-RLM GitHub repository, λ-RLM paper
What was talked about: OpenCode Go provides subscription access to several open and Chinese models, with controls for whether China-hosted endpoints are allowed. US hosting and zero-data-retention terms can be important for company purchasing, legal enforcement, and sensitive code. Direct DeepSeek endpoints may train on submitted data, while other providers offer logging-only or zero-retention policies. Hardware-backed verification remains uncommon despite Apple Private Cloud Compute and confidential-computing features on newer accelerators.
Takeaway: Choose model access based on hosting location, contract terms, retention policy, and training use rather than price alone.
Links shown: λ-RLM GitHub repository, λ-RLM paper, OpenCode Go, Cursor available models, DeepSeek V4 Flash on OpenRouter, Shared ChatGPT PCC comparison, Apple: Expanding Private Cloud Compute, Apple Private Cloud Compute Security Guide
What was talked about: Paper Cuts is a CLI that lets agents record obstacles, questions, and workflow problems while a task is running. This preserves issues that might disappear from the final response after an agent finds an unwanted workaround. Periodic retrospective agents can collect similar feedback, but automatic fixes may reverse one another when they lack broader context.
Takeaway: Capture agent friction during execution, then review reports with higher-level judgment before turning them into automatic changes.
Links shown: Steve Ruiz’s Paper Cuts post
What was talked about: MiniMax H3 is an open-weights text-to-video and image-to-video model with competitive benchmark performance and generated audio. It can run locally through ComfyUI on an RTX 3090, with low-resolution generations taking about five minutes. The video model has about nine billion parameters, while a larger Qwen text encoder requires offloading. Quantized and step-distilled variants can reduce memory use and generation time, and Blender layouts can guide camera motion and scene composition.
Takeaway: MiniMax H3 makes high-quality video experimentation practical on consumer hardware, especially with ComfyUI offloading, quantization, and distilled variants.
Links shown: Krea MiniMax H3 announcement, Krea video generator, Artificial Analysis text-to-video leaderboard, Comfy
0:05 All right. Let’s get started. Thank you, everyone, for Come Back to AI Tools Club.
0:12 Yeah, thank you for joining us on YouTube or here in the Discord for the discussion.
0:18 Today, I think I’ll kick things off with DeepSeek v4.
0:23 So, as usual, I always forget to pull up the actual news article for it.
0:28 But yeah, new model from DeepSeek. So, the previous DeepSeek V4 family of models, both Flash and Pro, they definitely, it seemed like very strong and efficient in terms of architecture models,
0:44 but they lacked a lot of polish on the post-training side of things.
0:49 So they weren’t very good for coding applications or in agent harnesses because they just had not seen very much agentic training.
0:58 They had a lot of intelligence in there, but they did not have that unlocked from them.
1:04 So DeepSeek has gone and trained their V4 flash model on a whole bunch more agent stuff, a whole bunch more coding, and now it is an absolute monster for its size.
1:15 It is about one-third the size of the likes of GLM 5.2, and compared to Kimbi K3, I think it’s 15 times smaller than that.
1:27 But it is still able to compete at least around the GLM 5.2 level in terms of quality.
1:34 And this is exceptionally impressive due to how little it costs.
1:39 So this model, similar to DeepSeek and their previous models, is extremely cheap.
1:47 So it is only 18 cents per million output tokens, which is almost 100 times cheaper than Kimi K3.
1:54 And I believe, what is that? About 20 times cheaper than GLM for similar performance.
2:03 So you’re able to get a ton of usage out of this model.
2:06 I’ve actually been using it with Open CodeGo.
2:09 This is a $10 a month subscription that you get with them.
2:13 You get to use all the different Chinese models.
2:15 And I used it basically non-stop with like 10 agents running in parallel doing benchmarking stuff for a couple hours.
2:24 And I used 2% of my five-hour time limit.
2:27 So yeah, you get a ton of usage out of this model.
2:30 Super low cost. I believe like 100 million tokens is around like a dollar or something like that.
2:38 I also went and did the math. And if you ran this model 24-7, it would only cost you $1.50 per day, roughly, per agent.
2:47 So about $0.07 per hour to go and run this model.
2:50 So yeah, extremely cheap, very capable model.
2:55 It’s primarily focused on coding is where it seems to be the best.
2:59 It also works very well in tandem with a larger model to go and do sort of like the reasoning and the high-level planning.
3:05 It struggles a bit more with those high-level sort of like conceptual design tasks.
3:10 But once you have sort of a clear idea of exactly what you want, this is an incredible model to go and give and just have it go and implement things.
3:19 So you can use Himik3 as the smart reasoning model and then DeepCP4 flash as the implementation model.
3:26 And with that, you can have a fully open source model stack that is also probably much cheaper than Anthropic or any of OpenAI’s offerings as well.
3:37 So yeah, I’ve been using this. I’ve been very impressed with it.
3:40 Has anybody else been able to start using it at all and have any thoughts on it?
3:53 Ask me next week, Andrew. I’m definitely going to try it.
3:57 Nice. Yeah, I would definitely… Yeah, Justin, you said it.
4:02 Same here. Yeah, same here. I haven’t tested it yet.
4:08 And this is a model I definitely would go and check out.
4:11 It is a bit sad, I would say, in some ways for me, because I really like the idea of running LLMs at home.
4:18 But this model is a bit too big to feasibly run at home, at least for a normal person.
4:23 You could run this on a gaming PC. But at the cost that it is, it sort of makes it irrelevant to go and build or buy anything that can run this.
4:33 Andrew, I haven’t read it, but can you hear me, by the way?
4:36 Yeah, we can. Okay, cool. I hope I’m not too staticky.
4:40 Or is it staticky? Okay, cool. Yeah, so I was wondering about the latency and for long-running processes, like or tasks rather, like what was your latency that you got, or did you measure that?
4:54 Yeah, looking at open router statistics, it didn’t seem slow by any means.
4:58 I know previous deep-seek models did seem fairly slow.
5:04 But yeah, I believe this, because it’s also open source, so everyone’s supporting it.
5:08 So yeah, it’s around 50 tokens per second.
5:09 So that’s actually the same as like GPT 5.6 sold is around 50 tokens per second, I believe.
5:15 And then, yeah, you can see that. Yeah, but then also then how many tokens does it output, right?
5:19 Like I think that’s another thing that I sometimes, like, I feel like a lot of the US models correct, they’re a little faster than the Chinese models because there’s just a lot of tokens that get outputted from the Chinese
5:30 model sometimes. Yeah, I believe compared to, like, I’d say for the US models, the main comparison I’d compare against is GPT 5.6 Luna, because that also got an 80% price decrease last week.
5:44 Yeah, but it’s about two times the number of tokens that get used in comparison, I believe, on artificial analysis here for their different benchmarks that they run.
5:55 So yeah, it is about two times slower when you factor in the token usage, but not egregiously slower, because tokens per second-wise, it should be roughly the same.
6:07 And yeah, for the price, I believe it’s like three to four times cheaper than GPT 5.60.
6:19 But yeah, it’s expected, I believe, for the DeepCv4 Pro model, the big model, to come out in the next couple of weeks, most likely.
6:27 I don’t have nearly as high hopes for that model, though, because even for the initial DeepCP4 release of these models, the Flash model was surprisingly close to the Pro model.
6:38 And so I don’t know if that’s because they’re putting less compute into the Pro model or if they’re just struggling to train it more or if, yeah, they’re able to train the flash model much more.
6:49 But yeah, hopefully V4 Pro is around Kimmy K3 or better levels of competence, but we’ll see.
6:57 I think it’s going to be very hard to beat v4 flashier in terms of price to performance usability, but we’ll see when that comes out.
7:10 Anything else about DeepSeek? If not, we can move along to Pangram.
7:22 I actually want to talk about this because it seems for a while now, AI text detection has always been very flaky and not very trustworthy, very high false positive rate.
7:35 Pangram in the last probably about like six-ish months now has come out as sort of like one of the better options in terms of AI text detection and one of the more reliable instances of it.
7:48 And now with their Pangram v4 model, they’ve sort of like taken it up a notch where it’s getting to like borderline ridiculous levels of AI text detection.
7:58 So this is just a classifier model that goes and gets run.
8:01 And people have been running it through just like weird text.
8:04 So they had this instance was they had AI go and generate a conversation between a cat and a dog using only meow and bark as the words.
8:15 And they were able to go and give this to Pangram and it was able to figure out that yes, this is an AI generated text.
8:23 And then they went and wrote the same version themselves or like their own variant of their own meows and barks.
8:29 And it was able to identify that as humans.
8:31 So even in like a non-sexual sort of like back and forth conversation in a fake language, the model is still able to figure out what is AI and what is not.
8:40 I’ve also seen it doesn’t just do text identification at the entire sort of like sequence level.
8:47 It does it word by word. So people have been going and they’ve taken previous articles that they have gone and written and they’ve replaced a single sentence with an AI generated sentence within there and then passed that
9:00 in. And Pangram has also been able to pluck that out as well as AI generated.
9:06 It’s also been one of the sort of like critiques of this sort of classifier is that you would be able to go and train against it, essentially.
9:18 And you use this classifier as basically the reward signal for your model, where you say, like, all right, reward it if it’s able to get past the Pangram filters.
9:26 So people have actually tried to run loops of either Claude or Codex trying to bypass the Pangram filters with its text that it’s generating.
9:38 And I’ve not really seen anyone have great success with it.
9:43 So yeah, it seems that, yeah, extremely robust, very strong, very hard to break.
9:49 So I would say at this point, I’d say this is somewhat reliable AI text detection now, where if I see that Pangram is saying something’s AI text, I would tend to believe it, which is the first system I would say.
10:05 Yeah. It’s one of the first. Sorry, Vlad posted something.
10:12 It’s very good. He said that his resume is 100% AI written, but it’s being detected as 74% AI written.
10:22 Very interesting. Oh, but that’s with text though, right?
10:25 So that’s, I assume a lot of code or markup in there.
10:30 Would be my guess. That’s being detected as not AI.
10:46 Yeah, so it looks like that. Oh, yeah, here’s another example of a random list of 200 numbers made by AI versus programmatically.
10:55 And it’s able to go and figure out whether or not which one is AI and which one is programmatic.
11:02 Okay, that challenge is… I don’t know.
11:06 That’s hard to believe. I don’t know.
11:12 I know for number generation, LLMs are not reliable to generate random numbers.
11:19 They are biased when you go and generate it.
11:23 So it’s, I know for previous versions, I don’t know what their current model is like.
11:27 I know they said they scaled things up, but previously, what they’ve gone and done is they’ve generated sort of like a diverse set of a million generations of both AI and non-AI writing and used that for training.
11:41 And it sounds like they scaled it up probably a couple order of magnitude.
11:44 It was very interesting because they embedded it all and sort of like measured the similarities or dissimilarities between them.
11:50 And they found that AI writing, when you go and embed it, is noticeably more different than human writing when you’re just looking at the sort of embedding level without actually going and training a model on it.
12:02 And it seems that it is pretty easily dividable.
12:07 Very interesting. Justin posts an archive link here.
12:15 That was the technical report to it.
12:16 Nice. Yeah, so my guess is that there’s probably in here they talk about how they go and build things out.
12:24 Have you been able to read this at all, Justin?
12:26 Or did you just find this as we were discussing it?
12:30 I just looked it up while we were discussing it.
12:32 Okay. I thought that was cool. So yeah, if you’re ever wondering, you know, what AI detection tool to go and use, I now recommend Pangram.
12:46 So yeah, if you have any successes or failures with it, please let us know.
12:51 Because we’d love to hear it. Okay.
13:01 Moving right along, we’ll talk about OpenAI and their big announcement this week.
13:09 So they used a new internal model called Astra, which is probably GPT 5.7.
13:17 I believe one of their employees said that this is not GPT-6.
13:21 This is just a continuation of the GPT-5 family that was able to go and solve 10 rather large open questions in the field of mathematics.
13:33 So this is building on top of the Jacobian conjecture being solved, I believe, two or three weeks ago now by Anthropic, which was a big breakthrough where they found a counterexample to a long-standing problem in math.
13:48 And yeah, now OpenAI is just going and basically throwing this at a whole variety of problems and having them go solve it.
13:56 Like I know the non-Sophic groups problem was a big open question and is pretty important.
14:03 I saw stuff about quantum parallel repetition being complex and very interesting as well.
14:10 But yeah, I think a few people, a few mathematicians said that any one of these would probably be worthy of a Fields Medal, or at least would be able to get you in as faculty at any of the top math departments around the
14:24 world. And OpenAI said this only cost about $2,000 worth of API tokens for it.
14:32 So yeah, rather large breakthrough.
14:35 It seems that we’re getting to the point now where AI models will probably be able to solve most of the open problems that are at least solvable by most of the math that we have now.
14:49 It’s interestingly, I think I saw something like an hour ago saying that OpenAI released all of the traces for this because people were wondering what was the intuition that the model had?
14:59 Like was this sort of like low-hanging fruit or just combining a bunch of different ideas together?
15:05 Or was this like some novel insights that it was coming up with?
15:09 Because based on what they were seeing, you couldn’t really tell.
15:14 And it seemed to just sort of like one sentence sort of gloss over the like main sort of like, ah, where did you get that insight from part of it in the proof.
15:23 So I think that they released more information for people to go over board.
15:28 Quinton says, prime factorization. Maybe I’ll buy gold.
15:34 Edzy, I don’t think that none of these, they’re open questions, but I don’t think any of them are like super important in terms of practical capabilities.
15:44 I don’t think, like, I don’t think this unlocks anything new all of a sudden.
15:49 Yeah, I don’t think we always knew that prime factorization was important for finance either.
15:57 Like public keys and everything. Anyway, it’s very interesting.
16:04 So yeah, my guess is that this is sort of the new benchmark, I feel like, for a lot of AI models, because these get a lot of hype whenever this sort of thing happens.
16:13 So I think we’ll see most of the known questions will become benchmarks in mathematics.
16:28 Sticking with OpenAI news, this time in the other direction, we’ll talk about how they have…
16:37 So we actually should back up. A few weeks ago, we discussed how Apple is currently trying to sue OpenAI for allegedly stealing a lot of their information about their hardware systems.
16:51 So specifically, the lawsuit was actually targeted at two individuals, and then OpenAI was listed as the third plaintiff on the motion filed by Apple.
17:04 Basically saying that OpenAI poached the Apple engineers and during the interview process and afterwards, basically used them to go and access a bunch of internal Apple documents that they should not have been allowed to
17:17 have. They were IP of Apple. So yeah.
17:21 But with this app or OpenAI, they are claiming that Apple didn’t really, they’re sort of leaving a bunch of things out and relying a bunch of things.
17:32 So they talk about how Apple was wrongly emailing the wrong members of the OpenAI team and how that they’ve met with OpenAI as well previously, but that was a lie
17:46 as well. And also how that they had continued to talk with their employees after they’d been hired at OpenAI about the confidential information and basically were getting those employees to give them the information that
18:00 they needed for this lawsuit. So yeah, they give a lot of depth and a lot of evidence in here, backing all of these claims up.
18:11 Yeah, exactly. We have just massive evidence chains here, which is very interesting to see this all happen out in the public.
18:20 Because I know, for instance, with the open, or sorry, the New York Times lawsuit that they’ve been dealing with for past three years now, none of that really gets talked about publicly all that much.
18:29 But this, they seem to be going at Apple head on and just directly calling them out publicly on it.
18:36 I do know OpenAI, they do have some of the scariest lawyers out there.
18:39 They have some very aggressive lawyers who will sort of like show up to your house and threaten you and that sort of thing.
18:46 So my guess is this is a part of their sort of guerrilla tactics around that.
18:51 But as yet, I would hate to be Chang Lu, who I believe he’s one of the people at the center of all of this.
18:56 Like I said, he’s one of the first two defendants named in the lawsuit.
19:00 As yeah, he’s probably not having a good time now.
19:06 And I should mention, this is all over OpenAI.
19:09 They’re trying to build their own AI inference chips.
19:12 And Apple with their M series of chips are all, those are all very strong sort of like custom-built silicon that Apple has made.
19:19 They’re also very good general sort of like AI processors as well.
19:24 So yeah, OpenAI has been trying to steal a bunch of that information from them.
19:35 Anyway, enough news talk. I wanted to talk a little bit more about the nitty-gritty, specifically loops.
19:44 Because I was trying out loops today.
19:47 So for those that don’t know, struggling right now.
19:51 For those that don’t know, loops are basically like the, it’s not even relatively new.
19:56 Like if you’re here for Gastown or beads and all of those token slop, like token burning different harnesses, loops are just the sort of more generalized idea of that where you have an agent going and trying to
20:10 verify what it’s doing or getting feedback from the environment or another agent and then continually iterating on that non-stop continuously.
20:20 So I had been using a loop this week because I was working on building out a coding benchmark for myself based on one of my coding templates.
20:30 I was just seeing like how well AI could interact with those templates and build things on top of them.
20:35 And so I had a bunch of deterministic tests that I built to go and verify correctness for these questions that I was asking it.
20:44 But I noticed that a lot of them had issues.
20:46 So I put GPT 5.6 Sol in a loop where it was spawning in a sub-agent to go and look at the tests and how the agents interacted with it previously, give feedback
21:00 for how to improve those tests, and then fixing those issues and then rerunning the sort of loop of like have the sub-agent go look and then have the main agent go and repair.
21:12 Ran that for basically two full usage limits for my OpenAI subscription because it got reset in the middle.
21:20 So I believe it was about $800 worth.
21:23 By the end, I did not have working benchmarks at all.
21:29 So the agent had gone and worked for about 18 hours across two days, just looping and trying to improve it.
21:36 And what I found is that the model got stuck on a lot of sort of like meaningless trivial things or like infrastructure things.
21:44 Like the majority of the code that it actually went and made was around the Docker environment and trying to make it as secure as possible, make sure stuff didn’t leak out.
21:53 We ended up with like three or four different services to go and help these Docker containers.
21:58 We had like a watchdog service to go and stop the containers and stuff like that.
22:02 We had Perl scripts sitting around and like C code, even though it was a JavaScript benchmark, trying to build out all around this.
22:09 And I found that, yeah, the actual tests that I cared about, that I wanted to have them get improved and be made less brittle or correct them, none of those actually got fixed.
22:20 They once again got over-engineered and never actually improved.
22:23 The same number of tests that failed before still failed after 18 hours.
22:27 And that’s also, I stepped in and tried to guide it a few times in the middle as well, or like check in on it.
22:33 But yeah, I think this is due to, like, for those of you that have used SpecKit before, I found this a lot when using SpecKit, where I’ll ask the model to sort of generate questions or issues that it sees with my SpecKit
22:45 plan. And then I’ll go and answer those, and we’ll go and improve the plan.
22:50 And I’ll iterate through that usually like anywhere between two to five times where I have it asked questions and I answer them and like we fill in the details that are missing.
22:58 But after a certain point, it starts asking questions or finding quote unquote issues that aren’t really issues, where it’s not meaningful or it sort of like misses the scope of the problem and it’s just not necessary to
23:11 add. But when you have it in this sort of generic loop where it’s the AI just blindly accepting whatever critique it goes and gets from the main agent, then I think that’s where we run into these issues where you have like
23:23 the scope creep or it gets lost going down sort of a side channel.
23:26 In my case, there’s like isolation for these Docker containers that just isn’t actually needed for the real thing where you need a human to step in instead.
23:37 So yeah, that’s my experience with loops, but I also wanted to hear what other people in the community have been doing.
23:44 Because I know a lot of people have probably more advanced agentic setups and use these sorts of things more than I do.
23:51 So yeah, have you guys had any success with that at all?
24:00 Yeah, I’ve been doing okay with I’m also using Sol for doing evaluations.
24:11 And it’s been running reasonably well.
24:14 I’ve had to make sure that it’s writing.
24:18 Basically, I’m using GitHub issues and PRs as my check.
24:24 So I have CI triggered, so it’s responding to that, which is an outside trigger, rather than it talking to itself all the time.
24:35 Okay, so that’s how you keep yourself in the loop?
24:37 Yeah, that’s how, well, I can see what’s going on because I get notifications.
24:42 And the CI there also has a clawed and a codex check.
24:49 They will respond and put in notes about it.
24:51 But that often seems to be the way that I’ve been able to get better results than it sounds like you did.
25:00 Edzie, I will say, I know there are more advanced harnesses.
25:04 I was just sort of like going through, like, just prompting it myself to basically build the loop.
25:08 But I know there are more advanced tools and like, you know, much similar to SpecKit, I think, where it sort of at the start defines sort of the scope of improvement and everything for it.
25:22 And those are supposedly better ways of going and using these.
25:28 Who says goals, hooks, cron are all loops?
25:33 Yeah, I think goal, I would say, like slash goal, I would consider a loop.
25:38 But what do you mean by hooks and cron?
25:41 Because I don’t, those I think are just more like modifiers and like additions.
25:50 Well, so those things are basically like things you tie into routines.
25:54 And routines, you can have continuously going, you can have them trigger on completion, on start, they trigger on different things.
26:04 And whilst they could just be one-off, you can set them up as loops.
26:09 And they often are set up as affordable loops.
26:14 So I think those are super important.
26:15 It’s very important to have them as part of your automation workflow.
26:23 Yeah, I could definitely see the idea of automatically triggering different checks and stuff is good.
26:29 Yeah, yeah, yeah. In cron’s and automatic jobs.
26:32 It’s because in my mind, loops are sort of like recursively doing something, whereas hooks are sort of like one-off.
26:39 You know, like agent completes, you know, a chat and then it automatically, a hook fires to go and run, I don’t know, like the CI CD and like the test suite automatically and give that feedback back to the model continue.
26:54 But yeah, I guess I can see how those are loops involved.
26:57 Yeah, I mean, if you have a cron job going, I mean, yeah, so I guess it’s like one shot versus multi-shot.
27:05 I mean, even a workflow, if you’re running a workflow every day, is that a loop?
27:10 You know, if it’s looking at the same thing?
27:15 We were talking about CI processes a second ago.
27:17 Like, if you have something triggered on the CI process, is that also a loop?
27:22 I don’t know. Yeah. Maybe I’m misusing the term.
27:25 Yeah. Yeah, it’s a semantic thing. So yeah, I think I could see the argument for why it would be.
27:30 I don’t think I would consider it a loop because I think loop is much more continuous where like that’s like one-off.
27:35 But yeah, like I said, that’s semantics is not an issue.
27:39 My stuff is definitely continuous. It’s been going on for about a week.
27:44 It’s yeah, that’s yeah. When I see like cron job, I think, yeah, it like runs every day, but like for like five, ten minutes versus, yeah, loops are the super long running processes instead.
27:57 Oh, okay, I was misunderstanding the loop.
27:59 I thought it was talking about you’re talking about like the slash loop command and clawed code.
28:04 But I’ve used the other kinds of loops quite a lot for I think my particular route was using the clawed code dynamic workflows.
28:16 And I’ve used, I had, I remember I had one set up a few weeks ago that ran for three days straight.
28:24 And I, for the time that 20, I had to wire clawed code to a local model.
28:31 But I managed to go through my entire issue board and update all of the stale issues.
28:38 Did you find it effective at doing that?
28:41 I found it far more effective than if I had just sicked the model on it without any guidance.
28:50 The nice thing about the loops, I think, is, at least for that particular use case, is that they guarantee coverage.
28:56 They guarantee that every issue is going to be looked at and assessed.
29:01 And for these dynamic workflows, I guess I haven’t worked with them too much.
29:05 Is this essentially just prompting on top of Claude?
29:08 Like, you have Claude as the main agent going and using a bunch of sub-agents to go and clear up all these issues?
29:14 Or is it more structured than that?
29:16 Like, is there sort of like a code, like, quote-unquote, harness that you’re building, like a for loop where it says, like, for each issue, pass it into claude?
29:24 Yeah, it’s exactly the latter. It’s literally a JavaScript, like an internal JavaScript with some kind of pain in the ass API contract.
29:36 But it lets Claude basically set up any kind of programmable pattern to call itself.
29:43 So you can have it, it can summon the data, like you can have, for a workflow, you can have it read an issue, have one agent read the issue, pass that to the next agent, which will trace it back through the code base to look
29:56 for anything stale, then trace that back to a reporter and then agent that writes out a report, and then trace that back to a final agent that goes back up and updates the issue.
30:08 Okay. Oh, so this is very similar then to RLMs.
30:12 Yes. Exactly. Okay. Sounds a bit more structured where you guys designing stuff upfront a bit.
30:21 I think that’s actually Lambda RLMs.
30:28 I believe. Yeah. Which, yeah, this sounds more where they sort of like define the different tasks that can be done beforehand.
31:01 Yeah, the dynamic workflows in terms of looping, they’re great.
31:05 And they definitely go beyond looping, too.
31:10 They’re also useful for doing pretty much any programmable pattern, like conditional.
31:19 The structure and patterns are written in code, and that’s done by the agents.
31:28 Yeah, that’s very cool. It’s one of those things where it’s, yeah, like, as someone who I don’t use anthropic models, I do miss that.
31:37 I feel like. It’s because you definitely are some much more quality of life features for building out these more bespoke agent workflows directly in the harness.
31:55 I just want to make sure the stream went.
31:56 It’s something, I had some memory leak somewhere.
32:00 I should have noticed I literally had the screen open buzz basically maxing out my memory.
32:04 So thankfully that got killed automatically, but we’re back now.
32:09 Cool. Yeah, has anybody else used loops here at all?
32:14 It’s like how popular are these? Because I know like on Twitter, a lot of people were talking about them.
32:19 But are actual sort of like people in the real world using it?
32:22 Like we have like three people here so far.
32:27 I haven’t been using loops out of the box.
32:30 I’ve been doing my own harness for like a while and it’s like a quite a journey taking different models into it of like trying to like have them like repeatedly recursively work
32:44 on something and like literally every model does something differently it’s a bit of a pain in the ass to do it on your own uh for example I just found like I had a workflow for
32:58 like that an agent had to like do something on my account and like if it logs out it has to log in and I put it to Luna and Luna is really really stubborn to not log in like I tell it like go ahead and log in you
33:12 have my password you have my authorization it’s like no you have to click the button like or something I need your authorization just so annoying and like I don’t know it might be like a Luna specific thing I don’t Sol was
33:23 also a bit annoying with like things like that like I noticed like drift still is like a quite an issue and it’s hard to prevent like when I had things running for for example I was migrating my harness from my like
33:38 Mac to a server and there was like a bunch of things that needed to be moved and like for example how like a browser used skill so they can use the browser locally there and Sol was doing the the migration and somehow for
33:52 some reason it from the like I for the browser use I use just like Firefox debugging protocol so like not like an extension but like a proper debugging protocol so you can like spoo the network and stuff and for some reason
34:05 like Sol out of its own initiative wrote in the skill like put hard blocks to not do network network fetching disabled then code and also wrote like warning do not reverse engineer private APIs
34:19 like out of its own will like without reporting anywhere it just like decided to be this like super safetyist model that you know doesn’t you know its task was to migrate it it never even like bothered to
34:33 follow the instructions to verify you know all the list that needs to be migrated just out of the sound initiative something like that it’s kind of my unrelated rent a little bit to loops but my point is overall that it’s
34:46 hard to uh yeah it’s hard to have a model agnostic loop from my experience generally they were like I have like a you could say a loop of like an agent’s like I’m looking for an apartment now that like wakes up every morning
34:58 and like tells me which apartments I should get like open them on Google Maps and like see if there’s like a big road nearby or something like that that works well but when it’s something that especially dealing with authorization
35:12 on GPT models like that’s just a recent thing that I found really really annoying.
35:19 I’ve noticed with the GPT 5.6 models is they seem to have a lot more sort of like safety training built in essentially where they will go and sort of like proactively try and do the safe thing.
35:30 I actually ran into this as well myself where I was trying to modify, I was like, oh, go clone this repo and like make these modifications and push it.
35:39 And it was like, oh, do you have permission to be able to go and do that?
35:42 And I was like, yes, it’s my own repository.
35:45 Also, it’s open source. Like it’s literally public.
35:47 But it’s like, oh, it doesn’t have a license file in there.
35:49 I don’t know if I can do it or not.
35:52 Yeah, okay, bud. Like, it’s fine. You can go and do it.
35:56 Yeah, this single-handedly makes me want to, like, pay for a Chinese model, because this is, like, obnoxious at this point.
36:02 I don’t know. It’s like, also, like, just like, I tell it, you know, I use Obsidian for Git syncing, like, this across things.
36:09 And, like, I don’t, like, it’s not a proper version control.
36:12 It’s more like for just syncing it across things.
36:14 And sometimes sync breaks. And I tell the agent, like, you know, just commit and push it.
36:19 And then it would, like, not listen to me tell it like five times that it has to push it manually.
36:24 But like, every time ask for my permission to push it to get to like a private repository.
36:29 You know, it’s just, yeah, it’s like they’re literally like losing, borderline, losing me as a customer because this is just obnoxious.
36:37 Like, and yeah, it’s not just me, seems like.
36:40 I’ve also run into the issue. Yeah, I’m like trying to get the models to push.
36:43 Is it basically found a way to, it didn’t, it couldn’t figure out how to use my actual Git credentials, which like literally just called the command line.
36:50 So instead it created a fake user that like pretended to be me.
36:54 It pushed all the changes to my GitHub.
36:57 And I was like, why did I not get credit for any of these commits?
37:00 It’s like, oh, it’s because it’s a custom user that I made instead.
37:04 So, yeah. Yeah, because they can’t use your credit, your authorization, because you didn’t authorize me to use your account or something.
37:10 Yeah, there’s compulsion like that.
37:12 And I’ll also say, you mentioned Luna in there not being as willing to do stuff.
37:15 I know OpenAI actually, in the codex harness, they don’t allow Luna to be a sub-agent because basically the communication protocol that they use with it, Luna isn’t advanced enough to be able to basically, like I
37:30 think in Codex, the models are able to chat to each other back and forth.
37:33 So the main model can go and check in on the sub-agent.
37:37 But Luna apparently isn’t good enough to be able to go and do that reliably.
37:40 So they don’t allow you to use Luna, at least out of the box.
37:42 You can go and modify it, but yeah, they don’t allow Luna to be used as a sub-agent for that reason.
37:47 Interesting. I actually had it like, I had it run a proper long task.
37:51 Like it did fix some like fire, like also some browser bug Firefox that was like crashing.
37:59 I had no idea what it was and I didn’t have to engage it.
38:02 Manage to like communicate across a bunch of 10 agents or something.
38:05 So that’s kind of strange. But yeah, I’ve tried Lona, so I’m moving off of that for now.
38:12 By the way, actually, a question to Andrew or to the group.
38:16 If you guys know if there is a way to, like if there is a subscription for some like open source Chinese models that is not given by Chinese, so it’s like more like for the purposes of like, so it can be
38:31 put on like US company account that it’s also not per token.
38:37 So it’s also as cheap as a subscription, but it’s also like on the US servers for like simply basically good reasons.
38:43 Is that a thing? Like is it open code?
38:45 Yes, open code. Yeah. So this is what I’ve been using.
38:48 They literally have a flag here for enabling models hosted in China or not.
38:52 So you can turn this off and I believe you can’t use DeepSeek v4 Flash right now.
38:57 I think they’re only going through DeepSeek directly for it.
39:01 But all the other models, I believe, they have hosted on US servers or something akin to that with zero data retention policies enabled.
39:11 Nice. Is it same for like Grog or is Grog, oh no, I mean Cursor or is Cursor now only Grog or how does it work?
39:18 Do they also have others? They do have others.
39:23 I just saw a list when I signed up, but I didn’t notice Rock.
39:29 I didn’t look for it either. But I saw GPT 5.6 and then a bunch of open source ones.
39:39 So Andrew, can I use the API key from OpenCodeGo for API access to Deep C?
39:46 Do you know? Or does that have to go through their framework, sort of like Cloud Code?
39:53 You can’t use that. Yeah. I believe you should be able to call it anywhere.
39:58 Yeah, very much like you can use it wherever.
40:00 They are the opposite of Anthropic, I think, in that regard.
40:03 Okay, I’ll give that a try. Thank you.
40:04 That’s simpler. The first month is five bucks as well.
40:08 So half off. My first task for DeepSeek V4 Flash was to configure DeepSeek V4 Flash to do code review for my refo.
40:20 We’ll see how that works out. Also, by the way, Anthropic is like they didn’t actually end up cracking down on like API kind of usage for their OAuth, at least for like claw-b and kind of other stuff
40:34 like that. They were announcing it for 15th of June and then on the 15th of June they sent an email like, hey, we changed our mind until further notice we’re pausing this.
40:46 So I was gonna migrate my harness off of it, but I, you know, kind of prolonging it even more.
40:52 So they, I think, only like not allow it explicitly in like open code or something like that, but for like you can still use like cloud-b and like just use their OAuth token for like just calling cloud API
41:06 and stuff like that. So not as bad.
41:09 I was going to say the same thing. Yeah, it looked like they were planning it out and then just decided not to.
41:15 And I’ve been using open code with my cloud subscription for a week or two now.
41:23 Oh. Yeah, they’re not in a dominant position to afford to lose a bit more mortgage share at this point.
41:31 Thank you, OpenAI. Thank you. Well, I was just curious, what’s the difference between hosted in China and not?
41:41 I guess that was actually a good question, I guess, for you, Daniel, because you were the one saying you didn’t want China.
41:46 More like for it to be on a company account, so that like a like an enterprise thing.
41:54 I think it’s just like generally if there’s a I just know like a general rule of thumb that they don’t want to be purchasing any software from outside of the US because it’s harder to like take legal actions
42:08 against companies that are out of the US.
42:11 That’s kind of the only reason, as far as I can tell.
42:16 One thing I will say with the DeepSeek endpoint is DeepSeek is known for training on whatever gets sent back to them.
42:22 You can consider this a good or bad thing.
42:25 I know with, I believe is the GPT, or sorry, the Deep C 3.2 release that a lot of people are like, oh, it’s way better on my use cases.
42:34 Like I can tell it has been trained on my data that I was talking to it previously.
42:37 So now it’s improved for my use cases.
42:41 So yeah, they will train on your data for better or worse, though.
42:45 Whereas yeah, pretty much every other provider, I think all the orange shields, this is on open router for those wondering, but all the orange shields mean they just log the information but don’t use it for training, and
42:54 then green means it’s full of zero data retention.
42:59 What happened to that thing? I remember like two years ago, there was a two years ago or so, like Apple was doing their like private cloud compute thing where they would like have trusted like execution environments, something
43:11 like that, that like you basically have a hardware guarantee that the firmware is signed and you get the signature in your API response or something that you’re guaranteed by the firmware cryptographically that no
43:26 one can access this data that you’re sending to them.
43:30 Has that died off entirely? Like is that not a thing?
43:33 You just have to take their word for it?
43:36 It’s yeah, I feel like that wasn’t part of the initial sort of like Siri updates, but I feel like those have been delayed and changed a bunch.
43:44 I’m not too sure where it is at this point.
43:48 But for like general, you know, normal LLM inference, they just, everyone just is okay with, you know, trusting, like, trust me, bro.
43:56 I’m not training on your data. Like, there is no, they’re okay without having any cryptographic signature for that.
44:03 I think a lot of these are contractual.
44:06 And because most of the companies that you see here are US-based companies, you can go and sue them if you find them breaking that contract.
44:14 The other nice thing is that none of these providers are actually model trainers.
44:18 So I guess they could be selling it behind the scenes to other labs, but they themselves will not be the ones to actually go and use this for training.
44:30 That makes sense. I do know, because I looked into this for compute community, actually, to see if you can verify what is being run on a given piece of hardware.
44:39 And it is very difficult to do so. I think the latest, like, last I knew H100s had the ability to have some form of verification of what is actually running there.
44:50 But any other GPUs, to my knowledge, you were not able to guarantee that the code you were seeing is the code that’s actually running on there when you’re accessing it via API.
45:12 Also, real quick, I just looked at Open Code.
45:14 Do they only have a $20 subscription?
45:16 They don’t have like a $200 subscription?
45:19 They only have a $10 a month, I think, Open Code Go.
45:23 Oh, yeah, $10 a month is. Okay, so it’s like very light.
45:26 So it’s not really an equivalent to $100 a month codex.
45:29 I mean, if you’re using like GPT, or sorry, Deep CP4 Flash, like I can get whatever they said, I can get 12,000 requests every five hours.
45:39 So you get a ton of usage out of it.
45:41 And I also believe that their usage rates are much more transparent, where I believe for $10 a month, you get $60 worth of usage for it.
45:50 You get 6x what you pay for. I know for using Kimmy and stuff, though, I ran out of usage fairly quickly with Kimmy because it’s obviously much more expensive model.
46:06 So it looks like Apple is using the private computer cloud, private cloud compute architecture.
46:13 And it’s expanded. It’s no longer purely Apple hardware because, well, they have to use other people’s GPUs.
46:24 I just shared a link to ChatGPT search.
46:34 I really like the architecture that they described a couple of years ago, but I have not been tracking that.
46:50 It still seems sort of up in air. They just like stopped doing it, like, out of what?
46:57 Like, it’s kind of strange. Well, I mean, this is June 8th, 2026.
47:01 They said they’re going to keep… They’re doing some more with it.
47:03 My guess is it’s Apple. They move relatively slow.
47:05 They’re like the last mover in most cases.
47:09 I mean, the NVIDIA one, you said only H100s have them.
47:12 Yeah. I assume if the H100s have it, probably V200s have it as well.
47:19 So it’s just underutilized because no one really cares that much, I guess.
47:23 Exactly. Anybody have to say that? If
47:44 not, we can continue actually with something that might help with loops and integrate well with your workflows.
47:52 And that’s this cool new tool I saw called Paper Cuts, where it’s basically a CLI tool for your agents to use that allow them to report any issues that they are encountering while they are running.
48:04 Because I feel like a lot of the times, like when I go and sort of like watch my agents work, I’ll see them run into issues or sort of like have questions or things that should be brought up to me or things that I could step
48:14 in and help with a bunch. But I never end up seeing those in the final model output once it completes the task.
48:20 And it’ll find a roundabout way of going and do things.
48:23 I think one of the most obvious cases of this is, let’s say you’re running your app on localhost 8000 already, and then the model tries to go and run it on port 8000 as well.
48:35 And it’ll go like, ah, crap, I’ve got to go run it on a different port.
48:37 And you could just be like, no, it’s already running on 8000.
48:39 I have it running somewhere else or something.
48:42 You don’t need to do that. So this is sort of a good way for giving the model the ability to give you that feedback while it’s running instead of only finding out about it at the end after it’s either found a workaround solution
48:54 that you wouldn’t have wanted or if it fails because it wasn’t able to find something.
48:58 So yeah. And then yeah, you can configure like I think yeah this guy mentions how he has it set up in his agents.md for sort of like what type of stuff to report and how to go and call it and that sort of thing.
49:14 I thought that this would be useful to me.
49:19 On that note I have a loop like that that like every hour or so or like every like 15 minutes forks an agent that was inactive for 15 minutes because Claude has like 60
49:33 minutes cash. So forks them and like gives them a prompt like write down all the things that went wrong etc and then I can accumulate them and like have agents try to like improve based on those which works about half the
49:46 time that they you know do actually useful things versus override each other’s work because sometimes they like complain about one thing they would fix that one thing and then next week they’ll complain about the opposite
49:57 thing and they would break it and so it’s that’s the downside of loops you have to have them somehow they have to have get a higher level like picture because they still get really really stuck in the details and they just
50:11 walk in circles work and walk in loops so I guess that’s okay design back to like the judgment capabilities of these models to assess what you can have I
50:35 just wanted to call out a new open source video model.
50:40 I know probably most of us here are focused on coding or LLMs, but I feel like it’s good every once in a while to talk about these more creative models instead.
50:48 This is Minimax H3. It’s actually the second best video generation model out there right now for both text and image to video generation.
51:00 And it also happens to be open weights.
51:02 So I’ve actually been running this model locally last night.
51:07 I was actually running a few tests.
51:08 And it’s been a while since we’ve had an actual model that is, I guess, new and open source that you can run at home.
51:17 But yeah, generations take like about five minutes on my 3090 at the lowest resolution.
51:22 So not very high resolution, but really good for experimenting and much cheaper than it would be normally.
51:28 Because looking at the prices here, you can see most of these are double digits or at least over five bucks per minute for video generation.
51:38 So experimenting and messing with video generation models is usually prohibitively expensive unless you have a very direct thing that you want to go and make with Impact.
51:47 But if you just want to play with them, it’s much harder.
51:49 But because this is open source, you can go and run it all locally.
51:53 So yeah. It also does audio as well.
51:57 So it’ll generate the audio for your clip.
52:00 I found it is okay. I wouldn’t say it’s very great audio generation, but it’s useful.
52:09 What’s your avenue for running it? I use it in Comfy UI.
52:15 For all my image and video generation stuff, I use Compi UI.
52:22 They also have their CompUI cloud as well, which last I remember checking when it first came out, you get a lot of very good deals on A100s for renting those out to go and use for generating images and video.
52:36 So you can go check that as well. Yeah, CompUI allows you to compose and it gives you a ton of fine-grained control.
52:44 I know I’ve been seeing people demoing Minimax H3 with Blender 2 video.
52:51 So they do a rough sort of sketch in Blender with just sort of camera tracking and sort of like big rectangles essentially.
52:58 And then they use Minimax H3 as the actual model to fill in all those details and make the real video.
53:06 So yeah. But you can do all that super easy without the open source model for since WAN 2.2,
53:20 right? I was recently trying to run an open model and checked it and the last one was Lily WAN from a year ago.
53:29 So that’s… I’m guessing it does not fit on a 3090, right?
53:33 So you have to swap it all the time or something.
53:36 But that’s still cool. Yeah. As well, yeah, the WAN model, yeah, that was from ages ago from Alibaba.
53:42 And yeah, they even stopped supporting the WAN series open source.
53:45 They closed source that with like 2.5 and onwards.
53:49 It’s Maymax H3. It does, like Comf UI handles all of the offloading and dynamically shifting stuff around for you.
53:57 I believe the actual model itself is relatively small.
54:00 It’s only about 9 billion parameters.
54:02 So it’s able to fit fairly comfortably on my 3090.
54:06 The main thing that it has to unload or sort of offload dynamically is the text encoder model.
54:13 So it uses a 32 billion parameter Quen model as the text encoder to pass into the image generation model.
54:21 And so that, because it’s 32 billion parameters, that gets shuffled around.
54:26 But then the actual video generation is fairly quick.
54:32 Cool. So I guess we’re going to be waiting for some quantizations and some.
54:36 Yeah. There’s already quantitizations out there to 4-bit for everything.
54:40 I know, I think it was Phal, I believe, went and distilled it as well, like time step distilled it.
54:46 So that normally for diffusion models, you have to iteratively run them a bunch of times to denoise the image.
54:54 Somewhere usually between 20 and 50 steps.
54:57 But FAL went and trained the model to go and only use, I believe, eight steps was the smallest that they got.
55:04 So you also get a pretty nice time speed up there as well.
55:07 As I haven’t had time to go and test out all those different variants yet, I was just throwing it together to go and mess around with last night.
55:12 But as yeah, I think I could probably like two or three X the speed of the generations, probably get it down to like a minute and a half for a video generation that’s eight seconds long.
55:28 Cheeseburger says CompuI recently has more support for quants such as int date, which can help with file size.
55:33 Yeah. I think the default for Minimax H3 that gets used is 8-bit.
55:38 And then yeah, you can go down to 4-bit as well.
55:40 They have full support for everything, because they’re sort of meant for running on local hardware that isn’t the most powerful.
55:53 Alright, and with that, we have run out of time for today.
55:59 Thank you all for coming, and can’t wait to see you all again next week.
56:09 See you all around.
Subscribe to get the latest AI news in your inbox every week!
