Greenfield, Brownfield, and Guardrails

Encode engineering judgment

(31:33) Definitely encourage that. Uh but for me it's really all about it just keeps coming back to trust. You know every one of us here in this chat have a different standard for engineering. Uh and there are different things that are important for us in our codebase. And uh when you are able to encode all of that into skills and you can verify that your agent is actually doing them that allows you to really kind of ascend this curve and um uh you know start automating things.

The case for refactoring

(32:03) Uh there's another piece I wanted to talk about. Um if there's more >> Yeah, go for it. I'll I'll pick up more questions as I go. But >> yeah, I think there's a third part to this which I haven't talked about yet, which is kind of an interesting one which is like refactoring and rewriting like one of the uh I guess most controversial one of the most controversial topics in the industry I think is like should you rewrite your app or not?

Um because I think engineers are very prone to this where especially when you join a company you come in and (32:34) you see like the code base and you're like oh man this is like who wrote this code you know it's terrible I want to rewrite the whole thing there is a very common inclination and I think a lot of you know before agents um and I guess arguably even now people will definitely discourage you from re rewriting stuff but I'm actually here to make a case for why you might want to consider Um because I think it really depends.

Brownfield: existing guardrails

(33:03) Uh you know uh brownfield applications I think are actually in a pretty good spot especially if they're set up well already. Uh and like recently I've been talking to some people but uh you know I I was just observing I I just noticed this parallel which is that a lot of big tech company problems are now everybody's problems.

(33:26) Um, and the big tech company problem, you know, like when I was working at Meta, like we had this giant monora repo, we had like, I don't know, tens of thousands of engineers just, you know, like banging on their keyboards and and shipping code and a lot of really great engineers at Meta. Uh, but, uh, I'll say like, you know, you'll be surprised that the code quality is actually not that good.

(33:49) [laughter] Um, and so I often joke that like, you know, before AI sloth, we had human sloth. Um and so uh you know I think a lot of big tech infra like uh like what Meta has or Google you know you know really big tech companies are actually designed for that where you you're sort of like you're catering to the the you know like uh this sounds so bad to say but like the the least capable engineer on your team right you build you build frameworks you build conventions you build guard rails you know you restrict credentials so that (34:21) you know your intern doesn't wipe your production database Um there's uh you know if you have that level of infra already I think your agents can actually already do a very solid job right because they have the the guard rails are already in place for agents to not cause havoc or not cause too much havoc uh in your codebase um and you can always add more you know guard rails.

Greenfield: risk and opportunity

(34:49) Uh but I think like green field applications especially are you know like the brand new applications are like the biggest risk in my opinion. Uh and also the greatest opportunity because you know if you vibe code a project uh a prototype um like we did for Grockbot you know Grockbot was spun up very very very quickly.

Um and if you if you haven't heard of of Grockbot it's like a a new application we just launched yesterday. (35:15) Uh it's it's really cool. uh lets you orchestrate your create like individual agents that have their own identity and you can kind of orchestrate them. It's super cool. Definitely check it out.

Um but yeah, that was it's like a very it was a very green field application like most prototypes are so like vibe coded very quickly. (35:34) Humans were not reading the code at all. And uh I had this tweet recently uh where I said something about organic architecture. Um maybe I'll find it. Uh but the idea is that uh when you have a completely vibe coded application, you essentially have no guard rails whatsoever.

So uh your agents when you give them a task, they will just solve it in whatever method is the most convenient. (36:06) And over time you get into this uh situation where you have a codebase that is spiraling out of control because you don't understand it. Uh your agents understand it I guess in a way but like they've built something that is you know optimized for short for shortcuts.

Uh and uh you know it will you will suffer you'll have a lot of of issues with that application.

Build a prototype quickly
                  │
                  ▼
Leave architecture unconstrained
                  │
                  ▼
Agents choose convenient shortcuts
                  │
                  ▼
The codebase becomes harder to trust

Invest in constraints early

(36:30) Uh so I think starting your codebase with uh like very strong constraints is very much needed uh because like when you have a codebase that you can trust, right? when you have guardrails that actually help you uh uh help your agents write good code, you can get into the you know like into this part of the curve where I I where like I I said you know I woke up today and I had like 20 PRs merged u by my agents and that's because I invested a lot a lot of time uh over 600 PRs I I I calculated yesterday uh when I refactored all of (37:10) Grockbot to this new architecture that I've been Um, and yeah, I've gotten to a point where I I don't really look I really don't look at the code anymore.

And um, I say that not just, you know, to sell you tokens, but because I, you know, it it it took a lot of work to get to that point. I spent a lot of tokens to get the codebase to this point where I no longer have to look at it. (37:37) Uh but I'm very excited because you know of the potential where you know it's not just this doesn't just benefit me it benefits everyone contributing to Grockbot and it also empowers you know designers and product managers and you know pe uh even GTM people to add features to Grockbot and I don't have to worry you know I don't have to to wake up at night in in the middle of the night and worry like oh someone's just merged a perf regression right I have a ton of constraints and CI is like it's actually very annoying to write code in in graph (38:10) web but like agents absorb all of that annoyance.

Um but yeah I'm happy to talk about what exactly that is. Um >> yeah I think one question um >> before we get into the this part here is just around that element of like what your your your CI looks like or maybe some of the constraints and then also like the average PR size.

Keep pull requests focused

(38:31) I saw a question about that earlier just to give people you know kind of a a glance. So it doesn't have to be like mathematically average, but just uh you know like what generally the size of the a PR is. Um if it's only a couple lines of code or you know um yeah >> um I think it depends uh I'm trying to do this in a way where I'm not going to like >> you.

(38:55) Yeah, you don't have to share the actual number like an actual average. I think this this is fine >> benchmark >> but like we have so okay this is not that interesting but uh well fun fact is that virtualization in grockbot and in uh cursor is actually powered by uh pretext uh which is a sort of new library that someone's built um that's really interesting you should you should check it out but that's not really that important uh I think the average PR size I actually don't know.

(39:28) I pro I don't know if I want to click on these. Uh I probably can, but I would say like they can range anywhere from a few hundred lines or 50 lines to like a thousand depending on what the thing is doing. Uh so like here I'm actually like deleting a bunch of files. So I expect that it's just like mostly deletion.

(39:48) Uh but yeah, it kind of varies. >> There's no like Yeah, >> there's no like hard cap or hard limit. Are there all like 50 line P? >> There's no hard cap. Yeah, there's definitely no hard cap, but I I do encourage my agents to split up their work into multiple PRs. Uh I do that mostly because uh I like I like the idea of the I guess maybe this is much harder to do now as in the world of agents and you have like so many commits, but I like the idea that you know the git history is a very rich source of context.

Uh, and I like the I like each (40:19) PR to sort of atomically describe what that small piece of thing is doing, which also makes it easier for me to revert changes and like figure out, you know, oh, I shipped a bug and it's just it's here, right? It's not in this 40,000 line PR where who knows what landed in there.

Uh, but yeah, I don't have a hard cap on PR size.