Feature Maps and Reproducing User Reports

Give the agent control of the app

(11:38) uh where if if you're building an Electron app or a web app or even iOS uh applications uh you can teach your agent how to use like the ChromeDev tools protocol or through uh Apple has some utilities as well for running the simulator and taking traces and controlling programmatic control as well.

Uh so that's really useful. (12:04) Uh but one thing I want to talk about is uh the this thing uh where is the read me? Uh so this skill comes with this very unique feature called or not feature uh unique file called a feature map. And so the story then is like I built this skill and so now the agent was able to uh actually run the agent window and take traces and whatnot.

(12:34) Uh but it had no idea what what the agents window was. So um you know like someone would say like oh the the left sidebar is like laggy or something like that or you know the right side the the PR tab is not working and the agent would just be like kind of flailing around. It would spend a lot of time trying to like look up the code and you know where is this feature?

How do I actually get to it on the UI which made it basically completely useless.

Map features to their location

(13:01) uh you know like we would I would run this skill locally and you know it would spawn a dev build uh but then it' just be churning like I just try to click here it wouldn't know how to get to things um and it was just an awful experience uh so is putting arrows on my screen um so uh yeah this this feature map has been really useful uh because it teaches the agent how to get to all of the features that you have.

(13:33) Um, and in PAC, the plugin that I I've made, uh, if you search for PAC cursor on Google, you you'll find it. Uh, but there is a create verification skill in that plugin where it actually helps you set up something like this for yourself. Um, including the feature map. So, it will actually explore the code and build up this initial feature map that tells your agent how to get to all of the different features that you have.

Read the user report
               │
               ▼
Locate the feature in the map
               │
               ▼
Navigate to the relevant UI
               │
               ▼
Reproduce and inspect the issue

Reproduce and diagnose reports

(14:00) Uh and this is extremely powerful because now like you have these user reports that come in uh you you can actually map even like a vague report or even a screenshot. So we have this uh internally at cursor where uh we have a slack channel with you know lots of people giving us feedback on the agents window and rockbot and whatnot.

(14:22) Uh, and often times the report is very bad like very low quality like someone will just put very often we get like a screenshot like and then someone just says question mark question mark question mark like what is this and you know like without this your agent is like I have no clue right but with a feature map like this it has a lot more context and understanding of how to actually navigate how to get to all the different features.

(14:50) Uh so like you know example like I guess like the sidebar like what is the sidebar uh you know like all the different sub features that are present in it um like from the user point of view here's where how do I get to it all the different keyboard shortcuts uh even like the the what do you call it the DOM elements or yeah like the attributes that you use for selecting things through the CDP uh are all there.

(15:17) So uh again yeah this is like really really powerful uh for for agents >> uh and pstack ships uh that create verification skill but also a maintain verification skill uh so you can keep this up to date. >> Cool. Yeah, I was just going to ask how you created that. So do you mind sharing a little bit more about um that that process in the context of Pstack and maybe just what Pstack is for the folks who aren't familiar?

Yeah.