Scaling Autonomy Across the Organization

The upfront cost of autonomy

(51:08) >> Okay, cool. Uh, one question that had uh a couple of uh came up a couple times was just around like token usage. >> So the the question is like is is what you're describing a realistic thing for people who are on you know uh a normal set of token usage. They don't have you know basically unlimited tokens uh to work with.

(51:29) I think that's a really good point. I mean, like obviously, you know, I work at a AI lab where we have unlimited tokens. So, uh I definitely cannot say that, you know, this is something everyone should do in the exact same way that I did it. I think it's possible to get to this point without, you know, breaking the bank.

(51:50) But you know if you're like an engineering leader or you know you're you you have a startup that you lead um I think to me it's a question of ROI um and it's like uh yes you spend a lot of money on tokens in the upfront stage you know like refactoring a code base is going to take a lot of tokens uh adding all these things uh is going to take a bunch of tokens but if we're heading to a world where agents are writing all the code and you know You want to be very lean, right?

You don't want to have to hire, you don't want to be, you don't (52:24) want to become like meta, right? Like I mean like in terms of you don't want to become a 10,000 person engineering or because I mean that's a cool problem to have, but also you you have so much overhead. There's like planning, you know, like you it's it's a personally I I wouldn't uh it it's not super fun.

Stay nimble and evaluate the return

(52:44) But um I think you want to stay very nimble, right? And you want to you want to be like agents are all about allowing you to do things that you couldn't do before. That's really to me like the value of agents, you know, it's not just storing tokens on every single little thing, but um to me like the thing I couldn't do before is like enforce this level of constraints in a codebase by myself, right?

Like I'm just a single person, you know? Uh it would have taken me years to build this framework uh and do all the refactoring and (53:18) test everything myself and verify you know like run imagine if there it was just me right know in in pre- agent era just like running you know by it would take me so long right and my salary is pretty high right like so you know the the question I think an engineering leader might have is just then you know like what is there's a trade-off of do Do you hire someone to do this or do you spend the tokens to set up a codebase so that even the the most naive, right, the dumbest agents can do a good job?

And when you actually get to this point, (53:53) like even agents that are not, you know, fable size do an excellent job of writing code. And this pays a lot of dividends as well for me personally where I've empowered not just myself but again like PMs, designers, engineers who are not familiar with Grockbot to just contribute in a way that is sustainable.

Model size and cost tradeoffs

(54:17) So I think yeah it's definitely like a trade-off for sure. You know like nothing is like free for sure. Uh and tokens are pretty expensive. Uh but oh actually uh I I I don't know how many of you have seen this but we actually announced Grock 4.6 today so very exciting finally out um so yeah graph 4.

(54:37) 6 would be like a great it was very very smart uh it's really good on the on the benchmarks uh and it's the same the tokens uh well uh I hopefully I'm not saying this incorrectly but uh I believe the cost per token is the same as 4.5 so you're actually getting more intelligence for the same cost uh I think this is an area that cursor tries to cursor and spaceex AI try to really optimize for like that heredto frontier here of you know cost versus intelligence.

(55:09) Uh you know we don't necessarily want to build the biggest model ever because that is extremely expensive to run. It's really about like how do you find that sweet spot right? You don't you don't need a giant model but it's just super smart right and it's not very expensive for inference. Uh but um yeah, I think to kind of round it up, um I think it's like a it's it's there's a if you do your own analysis, I feel like it's pretty positive.

(55:36) It it'll be pretty positive that the ROI you get from investing in stuff like this, uh just empowers not just yourself, but your whole team to be so much more productive, right? Like imagine if you have an army of engineers like me who are shipping so much improvements and and bug fixes uh you know every day right like that is pretty exciting.

Let the wider team contribute

(56:00) >> Cool. Uh one last question before we wrap up. This one is for the people in product on the on the call. >> So let's say we do have an army of engineers who are shipping like Lauren. I'm just curious like how is the product team or other functions of your company keeping up given that like if you're shipping so quickly have are they using AI more to do their jobs?

Like as much as you can speak to that obviously you don't have like you're not in that role but just curious about how that works. (56:26) >> Um I think this is where Grothbot has been actually exceedingly powerful. uh where so before graphbot like you know uh obviously cursor only had cursor like we only had agents window we had a CLI we had an IDE and these are really like power user tools right like de they're designed for developers so it's very very developerentric you can do knowledge work in them but it like the UI is not really optimized for that so we actually didn't really have uh well I think like a lot of people like you know GTM product like they might have used (57:02) cursor uh to do their work but it definitely wasn't like a delightful experience for them.

Um I think now with Grogbot uh it's become Grogbot is basically like the kusher moment for people who are not in tech in my opinion like it's like it's like a very very accessible way to use agents in a very comfortable very familiar interface. (57:26) It looks like iMessage. Um, and it's very fun to, you know, you can give your agent a fun name.

Uh, you can have you can kind of do orchestration with in a very like natural way where you can sort of, you know, each agent's like a person and now you got a team of agents like working on you have one one agent per account that you manage as an example. (57:45) Or if you're a PM, you have, you know, you can have an agent that summarizes all the work that Lauren did last night and then now you know what I did, right?

So I think our PMs are leveraging that a lot and they're shipping code too. Uh so you know like often times they will just say oh here's a bug I fixed it can you look at it and then I'll go review it and actually it's just perfect. (58:05) I'm like okay stamp. Uh so uh that I think that shows that you know the the Dune architecture is holding up right the all the the really strict constraints allow people who are not experts in engineering to contribute at a high level.

Uh so I'm I feel like I'm already seeing that pay off a lot where uh you know designers and PMs are just able to to to ship features directly.

Invest in architecture and skills
                │
                ▼
Make verification reusable
                │
                ▼
Support engineering and product
                │
                ▼
Let more people deliver features

Closing thoughts

(58:31) Um and that just makes the Grockbot team super fast, right? Where we can ship so quickly. Um and we have a lot planned. So I'm very excited uh to you know uh to to ship more ship more stuff. >> Yeah, that's awesome. Uh well, we are at time. So, uh I guess Lauren, if if folks want to support you, maybe go try out Crockbot, try out uh 46 and uh you know, get provide some feedback.

(58:59) But yeah, this was awesome. Really appreciate you taking the time. Uh thanks everyone for all the messages in the chat. Lots of good questions. I know we didn't get through everything, but as I kind of said at the top, way more questions than than we could get through, but uh yeah, really really thanks thanks for for joining.

(59:13) Thanks everyone for joining and uh hopefully you enjoyed the session. >> Yep. >> All right. >> Yeah, I see it. Thank thanks for having me. And uh if you have any more questions yet, just DM me on Twitter. I'll I'll open them up. I guess I'll let the let the >> You're going to get a lot of DMs.

>> Yeah, I'll open the updates. So yeah, DM me. (59:30) Maybe I'll do like a Twitter space at some point. That's all for more questions. But >> really appreciate everyone for showing up uh you know, taking an hour out of your day. >> Yeah. All right. Thanks all. I'll see you the next one. >> Okay. Thanks everyone.

And bye.