How to Build Quality Gates into Agentic Coding Workflows — Nnenna Ndukwe, Qodo AI

Um, my name is Nana, and Um, my name is Nana, and I'm the Head of I'm the Head of Developer Relations at Coda. We used to be just a We used to be just a platform for platform for platform for testing testing testing AI code, AI code, AI code, but now we're but now we're but now we're expanding beyond that expanding beyond that expanding beyond that . Everything is focused . Everything is focused . Everything is focused on code quality and on code quality and on code quality and management throughout the management throughout the software development lifecycle. And I software development lifecycle. And I spent a lot of spent a lot of spent a lot of time thinking time thinking time thinking and working on what a and working on what a structured structured AI coding process looks like? AI coding process looks like? AI coding process looks like? There are many different There are many different There are many different ways we ways we ways we all do this. I all do this. I all do this. I would love to hear from would love to hear from would love to hear from you all, especially towards the you all, especially towards the you all, especially towards the end. But I end. But I end. But I just want to explain just want to explain just want to explain what it might look like. what it might look like. what it might look like. I have a repository I have a repository I have a repository that anyone can that anyone can that anyone can create, and if you create, and if you create, and if you don't do anything else don't do anything else don't do anything else with it, with it, with it, have your AI have your AI have your AI analyze and analyze and analyze and take some tips from the take some tips from the take some tips from the structure of this structure of this structure of this repository and repository and repository and try to see try to see try to see what you can what you can what you can apply to your apply to your apply to your own own own workflow. OK? workflow. OK? Okay, so we'll take a little Okay, so we'll take a little look at the beginning of look at the beginning of look at the beginning of who I am, Nano. Definitely who I am, Nano. Definitely excited about AI for excited about AI for excited about AI for developers. This is my developers. This is my developers. This is my thing as a DevRel and as a thing as a DevRel and as a thing as a DevRel and as a former former full- full- time software engineer. And I live time software engineer. And I live time software engineer. And I live in Boston, and this is my in Boston, and this is my in Boston, and this is my cat Pippin. He was cat Pippin. He was cat Pippin. He was named after the named after the named after the Hobbit from The Lord of Hobbit from The Lord of Hobbit from The Lord of the Rings, not the the Rings, not the the Rings, not the basketball player basketball player basketball player I'm often I'm often I'm often asked about. And you know, that's what asked about. And you know, that's what asked about. And you know, that's what I'm here for, I'm here for, I'm here for, to implement a to implement a to implement a structured structured AI coding workflow AI coding workflow . We will . We will . We will use Coda.

use Coda. use Coda. You will be able to log in to the You will be able to log in to the You will be able to log in to the platform. If you don't platform. If you don't platform. If you don't want to want to want to use it use it use it for some of the early for some of the early for some of the early stages of the SDLC that we'll stages of the SDLC that we'll stages of the SDLC that we'll cover, you can cover, you can cover, you can just just just use it use it use it for PR code reviews. And for PR code reviews. And for PR code reviews. And first of all, if you first of all, if you first of all, if you want, you're going want, you're going want, you're going to need access to need access to need access to the slides, and I think the to the slides, and I think the to the slides, and I think the best way best way best way to do that, or the to do that, or the to do that, or the easiest way to easiest way to easiest way to do that, is to do that, is to do that, is to go to github.com, go to github.com, go to github.com, write my name, and write my name, and write my name, and then "repository then "repository then "repository should be at the top of the should be at the top of the should be at the top of the repository," repository," or these are slides that or these are slides that or these are slides that will link to will link to will link to my GitHub or my Google my GitHub or my Google my GitHub or my Google Slides. So hopefully Slides. So hopefully Slides. So hopefully everyone can, I'll give everyone can, I'll give everyone can, I'll give everyone a moment to everyone a moment to everyone a moment to do that. So, yes, the do that. So, yes, the do that. So, yes, the slides are here. slides are here. slides are here. The repository should be The repository should be The repository should be at the top. And at the top. And at the top. And someone, please someone, please someone, please raise your hand if raise your hand if raise your hand if you don't see this or you don't see this or you don't see this or don't have access. don't have access.

Thumbs up Thumbs up if I can move if I can move if I can move on, or I'll wait. on, or I'll wait. on, or I'll wait. Good. Um, so Good. Um, so Good. Um, so in general, where and how can in general, where and how can code quality deteriorate, who here, you know, code quality deteriorate, who here, you know, uses AI for uses AI for uses AI for coding? I think coding? I think coding? I think everyone. Do we have any everyone. Do we have any everyone. Do we have any Cloud Code fans here? And Codex? Cloud Code fans here? And Codex? Wow, wait, less Wow, wait, less hands for Codex. Okay, I hands for Codex. Okay, I hands for Codex. Okay, I thought they kind of thought they kind of thought they kind of exploded in the last exploded in the last exploded in the last couple of months. I couple of months. I couple of months. I use Codex for use Codex for use Codex for many things. But, many things. But, many things. But, um, where can um, where can code quality deteriorate in the code quality deteriorate in the agent agent agent development lifecycle? Everywhere. development lifecycle? Everywhere. development lifecycle? Everywhere. Planning and Planning and Planning and design, design, design, development, development, development, code review, testing, code review, testing, code review, testing, deployment. There are deployment. There are deployment. There are so many so many so many potential potential potential failure modes for failure modes for failure modes for using AI for using AI for using AI for coding just to coding just to coding just to deliver deliver deliver high-quality high-quality software, and that's where software, and that's where we get into we get into we get into this process where we this process where we this process where we have to have to have to over- over- over- manage manage manage AI coding agents to AI coding agents to AI coding agents to get what we get what we get what we need, just so need, just so need, just so we can support we can support we can support or stand behind the quality of or stand behind the quality of or stand behind the quality of the code and say, " the code and say, " Okay, you know what?" I Okay, you know what?" I Okay, you know what?" I can can can put my name on this.

put my name on this. put my name on this. That's good. We can That's good. We can That's good. We can push this further down the push this further down the push this further down the pipeline, or I pipeline, or I pipeline, or I feel safe feel safe feel safe having another developer having another developer having another developer review my code at review my code at review my code at this point. “And so I this point. “And so I this point. “And so I really believe that we have to really believe that we have to really believe that we have to create create create systems that are focused systems that are focused systems that are focused on quality. I think there on quality. I think there should be a should be a verification aspect to agentic engineering, and that verification aspect to agentic engineering, and that verification aspect to agentic engineering, and that starts with point starts with point starts with point 1, defining what 1, defining what 1, defining what code quality means code quality means code quality means to you and your to you and your to you and your team if you team if you team if you work on a work on a work on a development team. It development team. It development team. It needs to be defined, and needs to be defined, and needs to be defined, and then it needs to then it needs to then it needs to live somewhere. This needs to live somewhere. This needs to live somewhere. This needs to be codified so that our be codified so that our be codified so that our AI, no matter AI, no matter AI, no matter what interface what interface what interface you use, you use, you use, can extract this can extract this can extract this information and do its information and do its information and do its best to best to best to adhere to the adhere to the adhere to the code standards code standards code standards you have. And then once you have. And then once you have. And then once you decide you decide you decide where those standards where those standards where those standards are, whether it's are, whether it's are, whether it's locally in your locally in your locally in your specific individual specific individual specific individual repositories or repositories or repositories or at the context management level at the context management level at the context management level , which , which , which we'll look at we'll look at we'll look at later, then you later, then you later, then you can decide and can decide and can decide and port that to different port that to different port that to different interfaces. If it's interfaces. If it's interfaces. If it's your CLI, if it's your IDE, your CLI, if it's your IDE, your CLI, if it's your IDE, your CI workflows your CI workflows , it should be , it should be , it should be distributable. And distributable. And distributable. And we don't have to we don't have to we don't have to go through all of that go through all of that go through all of that . I already explained that.

. I already explained that. . I already explained that. We can just... We can just... Okay, here's what we're Okay, here's what we're Okay, here's what we're going to do. going to do. going to do. We are going to We are going to We are going to implement our..." implement our..." implement our..." context. I have a context. I have a context. I have a repository with repository with repository with rules, skills, rules, skills, rules, skills, a bunch of different things, and a bunch of different things, and a bunch of different things, and we want to we want to we want to use that use that use that for planning, for planning, for planning, generating, and generating, and generating, and testing code. And the whole testing code. And the whole testing code. And the whole point is that point is that point is that we want we want we want to check quality to check quality to check quality as early and as as early and as as early and as often as possible. And ideally, these are the often as possible. And ideally, these are the often as possible. And ideally, these are the same standards same standards same standards that we ask that we ask that we ask agents to check agents to check agents to check at every stage. That's the at every stage. That's the at every stage. That's the whole point. Oh, those are whole point. Oh, those are whole point. Oh, those are my skills. I have my skills. I have my skills. I have so many of them. Hmm. And so, so many of them. Hmm. And so, so many of them. Hmm. And so, so I think that, uh, I'll so I think that, uh, I'll so I think that, uh, I'll just move on to what just move on to what just move on to what we need to we need to we need to do. Hmm. So, do. Hmm. So, do. Hmm. So, first, let's first, let's first, let's move on to Kodo Quality move on to Kodo Quality move on to Kodo Quality Workshop Signal Pay. You all Workshop Signal Pay. You all Workshop Signal Pay. You all see it. Oh, I can't see it. Oh, I can't see it. Oh, I can't zoom in, uh.

zoom in, uh. Let's see. Good. So, this is the Good. So, this is the repository we're repository we're repository we're going to make. going to make. going to make. This is Kodo Quality Workshop Signal Pay. This is Kodo Quality Workshop Signal Pay. This is Kodo Quality Workshop Signal Pay. This is a payment app. This is a payment app. This is a payment app. And I want you to And I want you to And I want you to fork this fork this fork this repository and repository and repository and let me know when let me know when let me know when you do. And then you do. And then you do. And then you clone it. you clone it. If you want to If you want to figure this out...In your figure this out...In your figure this out...In your free time, all free time, all free time, all the instructions are in the readme. And the instructions are in the readme. And the instructions are in the readme. And I'm going to I'm going to I'm going to rely heavily on my rely heavily on my rely heavily on my coder agents to coder agents to coder agents to help me help me help me implement this, implement this, implement this, you know, at a meta level. you know, at a meta level. So, the So, the Signal Pay code quality seminar. And in Signal Pay code quality seminar. And in Signal Pay code quality seminar. And in this repository, we'll this repository, we'll this repository, we'll just walk through the just walk through the just walk through the process of planning a process of planning a process of planning a code change. We are code change. We are code change. We are going to read the going to read the going to read the repository rules repository rules repository rules to determine which to determine which to determine which rules are rules are rules are relevant to relevant to relevant to this particular this particular this particular code change. We're code change. We're code change. We're going to going to going to use use use the skills and then the skills and then the skills and then make sure we make sure we make sure we write proper tests write proper tests write proper tests using some using some using some TDD and BDD skills, TDD and BDD skills, test-driven development, and test-driven development, and behavior. Then we'll behavior. Then we'll behavior. Then we'll ask our ask our ask our coder agent coder agent coder agent to implement, um, the to implement, um, the to implement, um, the code change, and then code change, and then code change, and then run the local run the local run the local gates. We have linters gates. We have linters . We have, um, . We have, um, . We have, um, static analysis static analysis static analysis that we do that we do that we do because we need because we need because we need some deterministic some deterministic some deterministic checks and we checks and we checks and we need need need AI and a AI and a AI and a good mix of both, and

good mix of both, and good mix of both, and then we'll open a then we'll open a then we'll open a pull request and Codo pull request and Codo pull request and Codo will review our PR will review our PR will review our PR directly on GitHub. directly on GitHub. And after that, we're And after that, we're actually going to review actually going to review what problems with the code what problems with the code what problems with the code Codo found, and we're Codo found, and we're Codo found, and we're going to solve going to solve going to solve it automatically using the it automatically using the agent's skills. And what do we agent's skills. And what do we create? This is a create? This is a create? This is a standalone fast standalone fast standalone fast payments API and there payments API and there payments API and there 's no 's no 's no frontend or anything frontend or anything frontend or anything like that, so we like that, so we like that, so we can just can just can just focus on it focus on it focus on it being an API that you being an API that you being an API that you can view in the can view in the can view in the UI UI UI . Okay, . Okay, . Okay, some things to some things to some things to note note note about the about the about the repository structure. We have a repository structure. We have a repository structure. We have a file of MD agents. We file of MD agents. We file of MD agents. We have a hidden have a hidden have a hidden blueprint catalog and blueprint catalog and skill rules documentation. We have our skill rules documentation. We have our own API. We have own API. We have own API. We have tests and we also tests and we also tests and we also use semgrep use semgrep use semgrep for static for static for static analysis and we also analysis and we also analysis and we also have our have our have our CI workflow such as the CI workflow such as the CI workflow such as the validation process. So, first validation process. So, first validation process. So, first of all, we want to of all, we want to of all, we want to run this run this run this program locally.

program locally. program locally. So I'm going to So I'm going to So I'm going to use the use the use the make setup instructions make setup instructions make setup instructions after you after you after you cloned this cloned this cloned this repository. And, as I repository. And, as I repository. And, as I said, you can said, you can said, you can walk through it walk through it walk through it yourself by simply yourself by simply yourself by simply copying and copying and copying and pasting the instructions pasting the instructions . Let's see . Let's see . Let's see where I am...Okay. After where I am...Okay. After where I am...Okay. After we do that we do that , I believe there should be a , I believe there should be a , I believe there should be a make run. Okay, now it make run. Okay, now it make run. Okay, now it works. And works. And works. And it must be it must be it must be available locally. And this is the available locally. And this is the available locally. And this is the payments API that we have payments API that we have . Nothing special, . Nothing special, . Nothing special, very...Simple. Hmm, not very...Simple. Hmm, not very...Simple. Hmm, not quite the point of this, quite the point of this, quite the point of this, but, um, this is what you but, um, this is what you but, um, this is what you should see should see should see when the program runs when the program runs when the program runs locally. Okay, there are locally. Okay, there are locally. Okay, there are also some other also some other also some other endpoints if endpoints if endpoints if you want to look at them you want to look at them you want to look at them . Now you . Now you . Now you can, um, choose can, um, choose can, um, choose your lane if your lane if your lane if you want to just you want to just you want to just do it hands-on, do it hands-on, do it hands-on, or just or just or just observe. Here are the observe. Here are the quality checkpoints we will quality checkpoints we will go through. And, um, there's a go through. And, um, there's a go through. And, um, there's a whole process whole process whole process for setting up a gateway, for setting up a gateway, for setting up a gateway, right? We have these right? We have these right? We have these different teams that we different teams that we different teams that we run, and we have a run, and we have a run, and we have a validation team validation team validation team that is, um, pretty that is, um, pretty that is, um, pretty important. So you have important. So you have important. So you have this, um, already working in this, um, already working in this, um, already working in your local your local your local repository, and if repository, and if repository, and if you need you need you need help, ask help, ask help, ask your agent to, um,

your agent to, um, your agent to, um, change something change something change something locally to locally to locally to make sure it make sure it make sure it can run really can run really can run really smoothly. The next smoothly. The next smoothly. The next step we take step we take step we take is that we are is that we are is that we are going to going to going to use Codo. And use Codo. And , um, what's great about Codo is that , um, what's great about Codo is that , um, what's great about Codo is that we're not we're not we're not just going to just going to just going to run this in a run this in a run this in a pull request. We want pull request. We want pull request. We want to set up our to set up our to set up our account in Codo account in Codo account in Codo so that we can actually so that we can actually so that we can actually use use use the skills that are already in the skills that are already in the skills that are already in this repository and this repository and this repository and use use use the rules that are in this the rules that are in this the rules that are in this repository. And it repository. And it repository. And it will be critical for us to be will be critical for us to be will be critical for us to be able to able to able to apply this as a apply this as a apply this as a context to the context to the context to the code review phase, not code review phase, not code review phase, not just with your just with your just with your coding agents coding agents coding agents or during planning or during planning . Hmm, and like I said, . Hmm, and like I said, . Hmm, and like I said, you don't have to you don't have to you don't have to do this, but we can do this, but we can ...um, we'll get to ...um, we'll get to ...um, we'll get to that part in a that part in a that part in a second. So, some second. So, some second. So, some important standards important standards are the rules themselves. are the rules themselves. are the rules themselves. This repository contains This repository contains This repository contains various rules various rules various rules focused on the focused on the focused on the Payments API. And the Payments API. And the Payments API. And the agent's job is to determine agent's job is to determine agent's job is to determine when I need to when I need to when I need to implement this implement this implement this code change that you're code change that you're code change that you're requesting for me. requesting for me. requesting for me. I need to I need to I need to determine which of these determine which of these determine which of these rules will be rules will be rules will be relevant to relevant to relevant to this so I can this so I can this so I can follow them. They're follow them. They're follow them. They're all here, in this all here, in this all here, in this repository. And also repository. And also repository. And also skills for skills for skills for implementation. And then implementation. And then implementation. And then we have testing-driven planning and we have testing-driven planning and we have testing-driven planning and development development development . This . This . This repository contains a repository contains a repository contains a high- high- high- level planning template. And then we have an level planning template. And then we have an level planning template. And then we have an execution plan. And these are execution plan. And these are execution plan. And these are very detailed very detailed very detailed templates, because from what templates, because from what templates, because from what I've found, the more I've found, the more I've found, the more detailed you are, detailed you are, detailed you are, relatively speaking, the more relatively speaking, the more

relatively speaking, the more detailed you are or the detailed you are or the detailed you are or the more consistent and more consistent and more consistent and structured your structured your structured your planning is. phase and planning is. phase and planning is. phase and document that document that document that your agent should your agent should your agent should go back to and go back to and go back to and read, then they read, then they read, then they will follow will follow will follow a little better and also a little better and also a little better and also correct themselves correct themselves correct themselves when they realize when they realize when they realize they are missing some they are missing some they are missing some critical points of critical points of critical points of your process. And you your process. And you your process. And you want to be able to want to be able to want to be able to identify what the identify what the identify what the expected behavior is, expected behavior is, expected behavior is, what skills I'm what skills I'm what skills I'm going to going to going to use to use to use to implement it. It's all implement it. It's all implement it. It's all here as part of the here as part of the here as part of the planning template. And planning template. And planning template. And then we have then we have then we have local local local verification gateways. I verification gateways. I verification gateways. I use rough, pirate, use rough, pirate, use rough, pirate, bandit, pytest, semgrep, and bandit, pytest, semgrep, and bandit, pytest, semgrep, and regular commits as regular commits as regular commits as deterministic proof deterministic proof deterministic proof that changes that changes that changes made locally made locally made locally preserved the local preserved the local preserved the local contract. I am a firm contract. I am a firm contract. I am a firm believer in combining believer in combining believer in combining deterministic deterministic deterministic tools with tools with tools with artificial intelligence. And artificial intelligence. And artificial intelligence. And then the last gateway then the last gateway is the review of is the review of is the review of takedown requests and the takedown requests and the takedown requests and the correction gateway. When we correction gateway. When we correction gateway. When we open a PR request, open a PR request, open a PR request, Kodo's findings Kodo's findings Kodo's findings are reviewed, are reviewed, are reviewed, fixes and fixes and fixes and deferrals deferrals deferrals are documented, and are documented, and are documented, and then we can then we can then we can re-run re-run re-run our our our review team after review team after review team after we fix those we fix those we fix those code changes. And it will be a code changes. And it will be a code changes. And it will be a process of completing the process of completing the process of completing the code review cycle on code review cycle on code review cycle on its own, rather than its own, rather than its own, rather than just stopping just stopping just stopping right here at this right here at this right here at this local stage, local stage, local stage, going through this going through this going through this local run. Here are local run. Here are local run. Here are the prerequisites. We the prerequisites. We the prerequisites. We need Git, GitHub, GitHub need Git, GitHub, GitHub need Git, GitHub, GitHub CLI, Python, UV, Node.js, NPM, and

CLI, Python, UV, Node.js, NPM, and CLI, Python, UV, Node.js, NPM, and whatever whatever whatever coding agent you coding agent you coding agent you use, use, use, preferably Codex or Claude Code, preferably Codex or Claude Code, preferably Codex or Claude Code, and then we'll get and then we'll get and then we'll get our our our Codo account. Okay, here are the Codo account. Okay, here are the setup instructions. I'll setup instructions. I'll let you all let you all let you all do this, and we're do this, and we're do this, and we're going to log into the going to log into the going to log into the Codo app. This is app.codo.ai. Codo app. This is app.codo.ai. Codo app. This is app.codo.ai. And that's it. If you And that's it. If you And that's it. If you can log in here, you can log in here, you can log in here, you will be able to configure the will be able to configure the GitHub app to connect to a branched GitHub app to connect to a branched repository. So, repository. So, repository. So, app.codo.ai. Using. app.codo.ai. Using. app.codo.ai. Using. Who here already has a Who here already has a Who here already has a Codo account? Codo account? Codo account? Does anyone have one? Okay, Does anyone have one? Okay, Does anyone have one? Okay, great. So, yes, great. So, yes, great. So, yes, go ahead and sign in go ahead and sign in go ahead and sign in using your using your GitHub email address. I think this is the GitHub email address. I think this is the easiest way. easiest way. And, uh, and you'll be able to, uh , you'll be taken to, , you'll be taken to, , you'll be taken to, let's see, let's see, let's see, here on the here on the here on the integration page or the integration page or the integration page or the main page, and main page, and main page, and you'll need to you'll need to you'll need to add a connection, add a connection, add a connection, which will be the which will be the which will be the GitHub cloud connection, uh, GitHub cloud connection, uh, GitHub cloud connection, uh, and it will walk you and it will walk you and it will walk you through the through the through the setup guide. I setup guide. I setup guide. I RECOMMEND not RECOMMEND not RECOMMEND not to connect like all to connect like all to connect like all your repositories but your repositories but your repositories but only the forked only the forked only the forked repository, it should repository, it should repository, it should be listed there, be listed there, be listed there, but let me know but let me know but let me know if not. So if I if not. So if I if not. So if I click on GitHub and the click on GitHub and the installation guide, installation guide, connect your connect your connect your organization, hmm, I'm already organization, hmm, I'm already organization, hmm, I'm already connected, but it connected, but it connected, but it needs to be here so you needs to be here so you needs to be here so you can add it. Hmm,

can add it. Hmm, can add it. Hmm, this is just an example, this is just an example, this is just an example, for example we for example we for example we are using Codo as an are using Codo as an are using Codo as an example whether example whether example whether you use Codo you use Codo you use Codo or not, you will or not, you will or not, you will need skills need skills need skills and rules and rules and rules stored somewhere. stored somewhere. stored somewhere. Ideally, if you're Ideally, if you're Ideally, if you're working on a team, working on a team, working on a team, they'll be outside of they'll be outside of they'll be outside of your local your local your local repository unless repository unless repository unless you have a good you have a good use case for use case for putting them in that putting them in that putting them in that repository and it's repository and it's repository and it's something like something like something like hidden. If not, there hidden. If not, there hidden. If not, there are many are many are many levels of levels of levels of context control. I think context control. I think context control. I think Unblocked is one of them, Unblocked is one of them, Unblocked is one of them, and it's just a separate and it's just a separate and it's just a separate place to manage place to manage place to manage that context at that context at any time that you any time that you any time that you should be able to should be able to should be able to pull from anywhere pull from anywhere pull from anywhere . Does this make . Does this make . Does this make sense? And we sense? And we sense? And we use Kodo— use Kodo— this is just an example. this is just an example. Okay, once you're Okay, once you're logged into the Kodo portal, logged into the Kodo portal, logged into the Kodo portal, you've connected GitHub to you've connected GitHub to you've connected GitHub to your fork, then you'll your fork, then you'll your fork, then you'll need to need to need to generate an API key from the generate an API key from the generate an API key from the Kodo portal, and you Kodo portal, and you Kodo portal, and you can do that here can do that here by clicking on by clicking on API Keys in your API Keys in your API Keys in your profile, creating an profile, creating an profile, creating an API key, and giving API key, and giving API key, and giving it a name. You will probably it a name. You will probably it a name. You will probably want to save want to save want to save this API key somewhere this API key somewhere this API key somewhere locally so you can locally so you can locally so you can add it to add it to add it to kodo.kodo.forward/config.json later. So kodo.kodo.forward/config.json later. So , first generate an , first generate an , first generate an API key, and then you'll API key, and then you'll API key, and then you'll want to save want to save want to save it through this it through this it through this JSON structure. And again

JSON structure. And again JSON structure. And again , if you have , if you have , if you have access to the access to the access to the repository, you repository, you repository, you can have can have can have your your your coding agent do coding agent do coding agent do all of this for you. We all of this for you. We all of this for you. We also have Kodo skills. You also have Kodo skills. You also have Kodo skills. You can can can use use use npx skills by adding npx skills by adding npx skills by adding kodo-ai/kodo-skills/skills. If kodo-ai/kodo-skills/skills. If kodo-ai/kodo-skills/skills. If you don't want you don't want you don't want to use them, to use them, to use them, the skills are already the skills are already the skills are already local in the local in the local in the repository. Good. repository. Good. repository. Good. This is the task This is the task This is the task we will be working on, the we will be working on, the payment workflow security process. The challenge payment workflow security process. The challenge is to is to is to add one add one add one small small small payment workflow. This payment workflow. This payment workflow. This will be either a will be either a will be either a refund workflow refund workflow refund workflow or a retake or a retake or a retake processing workflow processing workflow . You can . You can choose which one choose which one choose which one you want, but these are... you want, but these are... intentionally small intentionally small intentionally small changes, right? With changes, right? With changes, right? With some potential some potential some potential risks risks risks built into the code, built into the code, built into the code, or is this a sensitive or is this a sensitive or is this a sensitive aspect of this, uh, aspect of this, uh, aspect of this, uh, this payments API. And, uh this payments API. And, uh , payment mutations , payment mutations are where a lot of AI-generated code are where a lot of AI-generated code can look can look plausibly plausibly plausibly correct, but can correct, but can correct, but can cause some cause some cause some problems. So, I think problems. So, I think this is a perfect this is a perfect this is a perfect example for us. Uh, and example for us. Uh, and example for us. Uh, and yeah, we're just yeah, we're just yeah, we're just going to going to going to use all of use all of use all of this text. Our coding agent this text. Our coding agent will use will use all of this to perform all of this to perform all of this to perform one of these tasks: one of these tasks: one of these tasks: refunds refunds refunds or or or retry processing. And retry processing. And retry processing. And here's where you can here's where you can here's where you can copy and copy and copy and paste paste paste agent requests to get started agent requests to get started . First, it's something

. First, it's something . First, it's something like: "Help like: "Help like: "Help me set up this me set up this me set up this repository for the workshop" repository for the workshop" . I have a Coda API key from the . I have a Coda API key from the . I have a Coda API key from the portal. "If you're portal. "If you're portal. "If you're using this, using this, using this, here are the requirements for the here are the requirements for the here are the requirements for the local local local setup. Hmm, I'm setup. Hmm, I'm setup. Hmm, I'm going to... Since going to... Since going to... Since I already have that part, I already have that part, I already have that part, I'm just going to copy and I'm just going to copy and I'm just going to copy and paste... install paste... install paste... install or test or test or test Coda skills, read Coda skills, read Coda skills, read agents.md and the rules, and agents.md and the rules, and agents.md and the rules, and pick the appropriate pick the appropriate pick the appropriate local local repository rule IDs repository rule IDs for this for this for this hands-on task hands-on task . And I'm doing this in the... er... . And I'm doing this in the... er... Codex. I would love to. Codex. I would love to. Codex. I would love to. Let's see. Let's see. So I'm in my... er... I'm in a project, and I'm I'm in a project, and I'm I'm in a project, and I'm just copying and just copying and just copying and pasting, getting pasting, getting pasting, getting my agent ready to my agent ready to my agent ready to start working on start working on start working on this project, giving this project, giving this project, giving it the context it it the context it it the context it needs. And needs. And needs. And raise your hand if raise your hand if raise your hand if you have any you have any you have any questions about questions about questions about where we're at right now where we're at right now , if you , if you want me to want me to want me to slow down or slow down or slow down or go back.

go back. Right here? Install Right here? Install or test or test or test Coda skills, we Coda skills, we Coda skills, we read agents.md and read agents.md and read agents.md and the rules, and then the rules, and then the rules, and then ask it to ask it to ask it to look at the look at the look at the repository rules for the repository rules for the repository rules for the hands-on task hands-on task that we're going to that we're going to that we're going to implement, and implement, and implement, and explain why each explain why each explain why each of these rules will be of these rules will be of these rules will be relevant for a relevant for a relevant for a future task future task . And then I will . And then I will . And then I will optionally compare these optionally compare these optionally compare these rules with..." rules with..." rules with..." Codo acquisition rules if these Codo acquisition rules if these Codo acquisition rules if these skills are available skills are available skills are available if you want to if you want to if you want to download them. Okay download them. Okay , let's see. What does it , let's see. What does it , let's see. What does it say? I will first say? I will first say? I will first justify this with the justify this with the repository guidelines and repository guidelines and check the local check the local check the local Codo settings and Codo settings and Codo settings and dependency status dependency status dependency status before choosing before choosing before choosing the rules. Okay, it the rules. Okay, it the rules. Okay, it has revealed that it knows has revealed that it knows has revealed that it knows we are we are we are inside the inside the inside the workshop project. And it workshop project. And it workshop project. And it also determined that also determined that also determined that I had Codo skills. Oh, the I had Codo skills. Oh, the I had Codo skills. Oh, the local local repository workshop skills aren't repository workshop skills aren't installed yet, so I'll installed yet, so I'll installed yet, so I'll run the run the run the repository installer, which will repository installer, which will repository installer, which will just copy the just copy the just copy the committed committed committed local skills folders to the local skills folders to the agent skills directory. Okay, great.

agent skills directory. Okay, great. So, it will process the So, it will process the So, it will process the local skills that local skills that local skills that we have. And I'll leave it to we have. And I'll leave it to we have. And I'll leave it to work. I'll get back work. I'll get back work. I'll get back to where to where to where we are now. This is a we are now. This is a we are now. This is a customization request. customization request. Let's see if Let's see if progress has been made. progress has been made. progress has been made. Once we Once we Once we have clarified the situation with the have clarified the situation with the have clarified the situation with the agents and they agents and they agents and they are ready to are ready to are ready to work, we will move work, we will move work, we will move into planning mode. into planning mode. into planning mode. So for this So for this So for this planning mode, we say planning mode, we say planning mode, we say use the use the use the local repository local repository local repository workshop planning skill workshop planning skill workshop planning skill to to to transform the transform the transform the practical task. in practical task. in practical task. in reading me into the reading me into the high-level implementation plan and the build session execution plan. And build session execution plan. And then we wanted to then we wanted to then we wanted to read the agents, read the agents, read the agents, read the rules, and, uh read the rules, and, uh , use whatever , use whatever skills we had to skills we had to figure out the constraints. figure out the constraints. And that plan should And that plan should include the transfer of include the transfer of include the transfer of implementation skills implementation skills , and that calls the first, uh , and that calls the first, uh , which calls the first , which calls the first , which calls the first implementation skill implementation skill implementation skill and the test-driven development request that and the test-driven development request that it will be it will be tied to and tied to and tied to and run next.

run next. Don't write any code yet. Don't write any code yet. I'm a big I'm a big I'm a big fan of fan of fan of planning mode, but if planning mode, but if planning mode, but if you don't want to you don't want to you don't want to use use use planning mode, it planning mode, it planning mode, it still has this still has this still has this prompt to, uh, prompt to, uh, prompt to, uh, make sure you're make sure you're make sure you're not going to not going to not going to implement anything. Oh, that implement anything. Oh, that implement anything. Oh, that takes some time. Well, takes some time. Well, takes some time. Well, that's because I have that's because I have that's because I have all the things set up all the things set up all the things set up for verification. Good. for verification. Good. Let's see. Good. Let's see. Good. Okay, this takes Okay, this takes Okay, this takes some time. Well, I'll some time. Well, I'll some time. Well, I'll probably make probably make probably make another attempt at another attempt at another attempt at capturing. Uh, I have.

capturing. Uh, I have. capturing. Uh, I have. What about you? What about you? What about you? Well, I was thinking about a Well, I was thinking about a Well, I was thinking about a refund, refund, refund, but where do you but where do you but where do you let him know let him know let him know you're doing that? you're doing that? you're doing that? Uh, he should Uh, he should Uh, he should prompt you and prompt you and prompt you and ask you. ask you. ask you. Good. Good. Uh, and if not, you Uh, and if not, you can...Make it can...Make it can...Make it clearly defined as clearly defined as clearly defined as part of the request. part of the request. part of the request. What stage are you at now What stage are you at now What stage are you at now ? ? ? I need to I need to I need to put the plan into action. put the plan into action. put the plan into action. Good. Perfectly. Okay, Good. Perfectly. Okay, Good. Perfectly. Okay, it's finally done. it's finally done. Repository instructions checked. The skill installation instructions skill installation instructions "Uh, um" or the " "Uh, um" or the " uh" command are launched. All local uh" command are launched. All local uh" command are launched. All local skills are now skills are now skills are now extracted. Here are the extracted. Here are the extracted. Here are the selected local selected local selected local repository rules repository rules repository rules for this for this payment workflow task. So, these are all the payment workflow task. So, these are all the rules that will be rules that will be rules that will be considered as considered as considered as part of this part of this part of this "wow" process. You know, "wow" process. You know, "wow" process. You know, maybe I'm also maybe I'm also maybe I'm also doing the doing the "uh" refund workflow with "uh" refund workflow with you. I'll copy and you. I'll copy and you. I'll copy and paste the paste the paste the scheduling request and scheduling request and scheduling request and go into go into go into scheduling mode. Hmm, it scheduling mode. Hmm, it scheduling mode. Hmm, it should ask me, should ask me, should ask me, but I'll say I'll but I'll say I'll but I'll say I'll work on a work on a work on a refund.

refund. What was that? What is it What was that? What is it called? called? called? Refund. Uh Refund. Uh , let's go. , let's go. , let's go. Refund workflow Refund workflow Refund workflow . Plan. Okay, . Plan. Okay, . Plan. Okay, let's see. let's see. let's see. So, I'm going to So, I'm going to So, I'm going to use the use the use the local repository local repository local repository workshop plan for the workshop plan for the workshop plan for the task skill and the task skill and the payment element potency skill, payment element potency skill, and then ground the plan and then ground the plan and then ground the plan in the agent.md readme file and in the agent.md readme file and in the agent.md readme file and the rules. I will only the rules. I will only the rules. I will only read and read and read and check the files. In check the files. In check the files. In this particular this particular this particular mode, no mode, no mode, no code or blueprint files will be written code or blueprint files will be written code or blueprint files will be written . So, Codex is now . So, Codex is now . So, Codex is now evaluating the current evaluating the current evaluating the current behavior of the program.

behavior of the program. We obviously haven't We obviously haven't implemented it, but implemented it, but implemented it, but we are defining what we are defining what we are defining what needs to be done, needs to be done, needs to be done, what the form of what the form of what the form of code changes that need to code changes that need to code changes that need to happen will be. This, um, happen will be. This, um, happen will be. This, um, I should probably I should probably I should probably dig into the dig into the dig into the templates now, since Codo templates now, since Codo templates now, since Codo is working so you can is working so you can is working so you can see what the see what the planning templates look like. So, here are the planning templates look like. So, here are the planning templates. I planning templates. I planning templates. I honestly think this is a honestly think this is a honestly think this is a really important really important really important artifact to artifact to artifact to take with you if you're take with you if you're take with you if you're not already using a not already using a not already using a structured structured structured template. Um, I think the template. Um, I think the template. Um, I think the plan should plan should plan should answer the answer the answer the questions: what questions: what questions: what behavior are we behavior are we behavior are we changing, what rules changing, what rules changing, what rules are being applied, what are being applied, what are being applied, what skills will skills will skills will drive the agent, drive the agent, drive the agent, what test what test what test will validate will validate will validate the expected behavior, the expected behavior, the expected behavior, and what and what and what quality criteria must the quality criteria must the quality criteria must the agent pass before we agent pass before we agent pass before we test it. And it test it. And it test it. And it all starts with a all starts with a all starts with a template for a template for a template for a general plan, and general plan, and general plan, and then a plan for then a plan for then a plan for executing the executing the executing the build session. And for this build session. And for this build session. And for this workshop, we expect it to workshop, we expect it to workshop, we expect it to look something like look something like look something like this. And here are this. And here are this. And here are the templates. We have a the templates. We have a the templates. We have a brief, the brief, the problem we are problem we are solving, any solving, any solving, any specific constraints specific constraints specific constraints we need to we need to we need to worry about, and then worry about, and then worry about, and then we have we have we have skill routing. This can...

skill routing. This can... be useful if be useful if be useful if you need to you need to plan in advance all plan in advance all the skills you the skills you the skills you think your coder will need to think your coder will need to use during the use during the implementation, implementation, implementation, one by one; one by one; one by one; especially if you especially if you especially if you want to want to want to use use use certain loops, um, or certain loops, um, or certain loops, um, or similar purposes, if it's similar purposes, if it's similar purposes, if it's in Codex or Claude code, to in Codex or Claude code, to in Codex or Claude code, to work on your work on your work on your behalf without your behalf without your behalf without your intervention. And we have a intervention. And we have a intervention. And we have a scope, scope, scope, behavior scenarios and behavior scenarios and behavior scenarios and verification gateways, verification gateways, verification gateways, failure and recovery rules, failure and recovery rules, failure and recovery rules, and a and a and a connectivity plan. This connectivity plan. This connectivity plan. This allows for a very allows for a very allows for a very structured structured structured process for, um, your process for, um, your process for, um, your coders to work coders to work coders to work on very on very on very specific specific specific tasks, tasks, tasks, organize the code in a organize the code in a organize the code in a nice and nice and nice and presentable presentable presentable way, make those way, make those way, make those commits in an organized way commits in an organized way commits in an organized way using using using regular commits, and regular commits, and regular commits, and then get to the point then get to the point then get to the point where you where you where you can actually submit a can actually submit a can actually submit a pull request; and everything is pull request; and everything is pull request; and everything is organized just as organized just as organized just as it would be when it would be when it would be when working in a team, working in a team, working in a team, when we didn't have when we didn't have when we didn't have artificial intelligence. And artificial intelligence. And artificial intelligence. And then the then the then the build execution template. What build execution template. What build execution template. What I've noticed is that it's very I've noticed is that it's very I've noticed is that it's very helpful to have helpful to have helpful to have documentation of the documentation of the documentation of the initial state.

initial state. initial state. What branch are we in What branch are we in What branch are we in ? What is the ? What is the ? What is the current local current local current local verification process? Are verification process? Are there any commands there any commands that should be that should be that should be executed to executed to executed to determine if determine if determine if this code meets this code meets this code meets any any any expectations from the linter? And expectations from the linter? And expectations from the linter? And of course, this is of course, this is of course, this is the transfer of the transfer of the transfer of implementation skills and implementation skills and implementation skills and execution steps. I execution steps. I execution steps. I want the agent to know want the agent to know want the agent to know every step in this every step in this every step in this process of making process of making process of making code changes, and I want them to code changes, and I want them to code changes, and I want them to also also also determine what the determine what the determine what the testing plan is. And then testing plan is. And then testing plan is. And then this definition of " this definition of " done" or done" or done" or completion notes. You completion notes. You completion notes. You need to need to need to make sure you make sure you make sure you have executed all the commands have executed all the commands have executed all the commands for this branch. You for this branch. You for this branch. You need to need to need to make sure you make sure you make sure you identify what has been identify what has been identify what has been postponed in case you postponed in case you postponed in case you change your mind at some change your mind at some change your mind at some point. Everything point. Everything point. Everything is documented in is documented in is documented in this plan. And now this plan. And now this plan. And now we have one. Refund we have one. Refund we have one. Refund Workflow Workshop Plan Workflow Workshop Plan . We have our . We have our summary.

summary. summary. We have associated We have associated We have associated repository rules, a repository rules, a high-level implementation plan, and a build session execution plan. And there build session execution plan. And there are also some are also some are also some assumptions here. Who here assumptions here. Who here assumptions here. Who here has reached has reached has reached this point? Okay, this point? Okay, this point? Okay, great. So, I'm great. So, I'm great. So, I'm going to go ahead going to go ahead going to go ahead and implement this. And and implement this. And and implement this. And while that's happening, while that's happening, we can move on to we can move on to some slides. So what some slides. So what some slides. So what I've been talking about I've been talking about I've been talking about operationalizing operationalizing operationalizing context, context, context, planning planning planning code generation, and reviewing code generation, and reviewing code generation, and reviewing code is something I code is something I code is something I find incredibly find incredibly find incredibly useful for useful for useful for consistency in the SDLC. I consistency in the SDLC. I consistency in the SDLC. I use use use a combination of different a combination of different a combination of different agents. skills, and it agents. skills, and it agents. skills, and it depends on depends on depends on the repositories the repositories the repositories I'm working with. It depends I'm working with. It depends I'm working with. It depends on the on the on the programming languages ​​and programming languages ​​and programming languages ​​and frameworks I frameworks I frameworks I use. But at a use. But at a use. But at a high level, so high level, so high level, so you'll see some you'll see some you'll see some Python skills up here, Python skills up here, Python skills up here, like Python, like Python, like Python, Pydantic, Pyright, or Python, which Pydantic, Pyright, or Python, which Pydantic, Pyright, or Python, which introduces contracts.

introduces contracts. introduces contracts. Many of these skills Many of these skills Many of these skills are available, uh, are available, uh, are available, uh, all over the internet, or all over the internet, or all over the internet, or you can ask you can ask you can ask your your your coding agent to create coding agent to create coding agent to create them yourself. And there are them yourself. And there are them yourself. And there are some that I got some that I got some that I got from some from some from some popular popular popular developers. Uh, these are developers. Uh, these are developers. Uh, these are some of Aaron some of Aaron some of Aaron Francis' skills, these are Francis' skills, these are Francis' skills, these are auditing skills. There are audit boundaries, auditing skills. There are audit boundaries, auditing skills. There are audit boundaries, drift, audit names, drift, audit names, drift, audit names, audit abstractions, audit abstractions, audit abstractions, dead code, and dead code, and dead code, and audit errors. I audit errors. I audit errors. I have found them to be have found them to be have found them to be incredibly powerful incredibly powerful incredibly powerful for truly for truly for truly simplifying and simplifying and simplifying and improving improving improving code quality during the code quality during the code quality during the implementation phase. I don't implementation phase. I don't implementation phase. I don't know if he has them know if he has them publicly available, publicly available, but if you can, uh but if you can, uh , find them, or , find them, or , find them, or ask your ask your ask your coding agent coding agent coding agent to create them to create them to create them for you, it's for you, it's for you, it's definitely a step definitely a step definitely a step in the right in the right in the right direction for quality. Uh, direction for quality. Uh, direction for quality. Uh, like I said, like I said, like I said, quality assurance quality assurance quality assurance early and early and early and often. So one of often. So one of often. So one of the values the values the values of using the skills of using the skills of using the skills and rules Codo get here is and rules Codo get here is and rules Codo get here is that... that... about extracting those about extracting those about extracting those rules from your rules from your context management layer and then context management layer and then determining which, uh, determining which, uh, determining which, uh, which rules are going to be which rules are going to be which rules are going to be relevant to the relevant to the relevant to the task. And then I task. And then I task. And then I believe in running believe in running believe in running local code local code local code review workflows review workflows review workflows . I don't care . I don't care . I don't care what tool you what tool you what tool you use, whether it's use, whether it's use, whether it's CodeX, whether it's Cloud Code, their CodeX, whether it's Cloud Code, their CodeX, whether it's Cloud Code, their built-in built-in built-in code review, or Codo, whatever it is code review, or Codo, whatever it is code review, or Codo, whatever it is , always , always , always run a local run a local run a local code review code review code review because there are always these because there are always these because there are always these small changes that small changes that small changes that you could probably you could probably you could probably make earlier, make earlier, make earlier, before another developer before another developer before another developer sees your code. And it's

sees your code. And it's sees your code. And it's worth being worth being worth being able to resolve able to resolve able to resolve this as quickly as possible. We this as quickly as possible. We this as quickly as possible. We are trying to reduce are trying to reduce are trying to reduce the problems that the problems that the problems that arise. And then we arise. And then we arise. And then we run an run an run an automatic automatic automatic code review in CI or the code review in CI or the code review in CI or the Git provider, which is the Git provider, which is the Git provider, which is the default. And default. And default. And when I was doing that, I when I was doing that, I when I was doing that, I walked through this entire walked through this entire workflow for an open source tool open source tool I'm building, and the proof I'm building, and the proof I'm building, and the proof was the number of was the number of was the number of lines I changed in a lines I changed in a lines I changed in a big PR. It was big PR. It was big PR. It was intentionally large intentionally large intentionally large because I wanted to because I wanted to because I wanted to test the quality of test the quality of test the quality of this this this actual actual actual workflow itself, and only one workflow itself, and only one workflow itself, and only one bug was bug was bug was found in over 1900 found in over 1900 found in over 1900 lines of code and lines of code and lines of code and using...Codo for using...Codo for using...Codo for this, like this, like this, like automated automated automated code review. And it code review. And it code review. And it was a was a was a reliability error, which made reliability error, which made reliability error, which made sense. Hmm, but it was a sense. Hmm, but it was a sense. Hmm, but it was a really good exercise really good exercise really good exercise for me to for me to for me to see that there see that there see that there is value in this process. is value in this process.

Yeah, so I'd like to Yeah, so I'd like to have a chart that would have a chart that would have a chart that would show you how I have show you how I have high-level planning patterns that I'll high-level planning patterns that I'll use use use no matter no matter no matter what repository I'm in what repository I'm in what repository I'm in . Here's the . Here's the . Here's the outline structure outline structure outline structure I expect, and then the I expect, and then the I expect, and then the overall code quality, overall code quality, overall code quality, like at the highest level like at the highest level , like engineering , like engineering , like engineering standards that are standards that are standards that are specific or that specific or that specific or that can be applied to can be applied to can be applied to any any any programming language. And programming language. And programming language. And then like below, then like below, then like below, okay, all the okay, all the okay, all the repositories that repositories that repositories that use Python, use Python, use Python, here are all the skills for here are all the skills for here are all the skills for that. And you need to that. And you need to that. And you need to decide what will be decide what will be decide what will be most relevant most relevant most relevant depending on your depending on your depending on your use case use case , but the more general the , but the more general the , but the more general the skill, the more skill, the more skill, the more access you access you access you need to have to need to have to need to have to many other many other many other repositories to have it. Does this make repositories to have it. Does this make repositories to have it. Does this make sense? Take a look at this, sense? Take a look at this, sense? Take a look at this, for example, let for example, let for example, let me make sure me make sure me make sure I limit this to just I limit this to just I limit this to just my repositories my repositories . Oh, okay. Okay, . Oh, okay. Okay, . Oh, okay. Okay, take...Look take...Look take...Look at this. We are at this. We are at this. We are on the Coda portal, and this is the on the Coda portal, and this is the on the Coda portal, and this is the rules section. So, in rules section. So, in rules section. So, in several repositories several repositories several repositories there are rules that have been there are rules that have been there are rules that have been derived, retrieved, and derived, retrieved, and derived, retrieved, and stored here on the stored here on the stored here on the portal. And they are portal. And they are portal. And they are divided by the name of the divided by the name of the divided by the name of the rule in general, and rule in general, and rule in general, and all the details that are all the details that are all the details that are generated here based on the generated here based on the generated here based on the context from context from

context from your repository. And your repository. And your repository. And then they are divided then they are divided then they are divided into categories. There is into categories. There is into categories. There is accessibility, accessibility, accessibility, correctness, correctness, correctness, ease of ease of ease of maintenance, maintenance, maintenance, architecture, and security. architecture, and security. architecture, and security. Hmm, and they are also Hmm, and they are also Hmm, and they are also organized by organized by organized by level of seriousness. level of seriousness. Not every, um, not every Not every, um, not every rule should be rule should be rule should be treated with the treated with the treated with the same level of same level of same level of seriousness. There are some seriousness. There are some seriousness. There are some rules that are rules that are rules that are strong enough strong enough strong enough to block mergers. to block mergers. to block mergers. There are others that are simply There are others that are simply There are others that are simply good practice, but good practice, but good practice, but it's not something that would be it's not something that would be it's not something that would be harmful if they were to harmful if they were to harmful if they were to pass. Hmm, and yes, here are pass. Hmm, and yes, here are pass. Hmm, and yes, here are some of the ways some of the ways some of the ways that, um, these rules that, um, these rules that, um, these rules can be categorized, can be categorized, can be categorized, and I believe they and I believe they and I believe they can also be combined can also be combined can also be combined with specific with specific with specific repositories if repositories if repositories if you want to get a you want to get a you want to get a detailed overview. Or detailed overview. Or detailed overview. Or if you want Coda to if you want Coda to if you want Coda to output it itself, um, output it itself, um, output it itself, um, or whatever other or whatever other or whatever other tool you tool you tool you use, you use, you use, you can actually... can actually... design this design this design this process yourself process yourself process yourself to capture these to capture these to capture these rules in a way that rules in a way that rules in a way that they rules or they rules or they rules or skills skills skills only apply only apply only apply to very specific to very specific to very specific repositories.

repositories. repositories. Can you...Can Can you...Can Can you...Can you...What was that? I do you...What was that? I do you...What was that? I do n't think so. I think, well, n't think so. I think, well, n't think so. I think, well, the only thing you can the only thing you can the only thing you can do is do is do is filter it on this filter it on this filter it on this dashboard dashboard . Hmm, but that's a really . Hmm, but that's a really . Hmm, but that's a really good question. I'll good question. I'll good question. I'll probably send this back to the probably send this back to the product development team for product development team for review. Hmm, but what review. Hmm, but what review. Hmm, but what sparked this idea? sparked this idea? Repositories? Right. They are now Right. They are now divided by divided by divided by organizational organizational organizational level and by level and by level and by repository level, repository level, repository level, naturally or naturally or naturally or natively. Those were...Those natively. Those were...Those natively. Those were...Those were good questions. were good questions. were good questions. Yes. Yes. That may be the

Yes. Yes. That may be the Yes. Yes. That may be the case, and so it's a great case, and so it's a great case, and so it's a great question. How do rules question. How do rules question. How do rules become rules? become rules? become rules? For example, how does Coda For example, how does Coda For example, how does Coda determine what is determine what is determine what is a rule or what it is a rule or what it is a rule or what it is , because it , because it , because it seems very seems very seems very automated. automated. automated. So, you can...it So, you can...it So, you can...it depends on how depends on how depends on how you already have you already have you already have things organized. If you have things organized. If you have things organized. If you have coding rules coding rules coding rules that that that are literally in the are literally in the are literally in the rules catalog, uh, rules catalog, uh, rules catalog, uh, you know, it's going to be you know, it's going to be you know, it's going to be really easy for Codo to say, " really easy for Codo to say, " Okay, every Okay, every Okay, every rule that I see here rule that I see here , I'm going to..." just copy , I'm going to..." just copy , I'm going to..." just copy and paste, like and paste, like and paste, like most of this most of this most of this content, um, and, um, content, um, and, um, content, um, and, um, upload it here upload it here upload it here to the portal. “But there are to the portal. “But there are to the portal. “But there are some rules that some rules that some rules that will appear as will appear as will appear as suggestions. And that's just suggestions. And that's just suggestions. And that's just from reading or analyzing the from reading or analyzing the from reading or analyzing the actual actual actual codebase. So you may codebase. So you may codebase. So you may not have written these not have written these not have written these rules rules rules directly into your directly into your directly into your code, but these are code, but these are code, but these are conventions that Codo conventions that Codo conventions that Codo noticed noticed noticed after analyzing the entire after analyzing the entire after analyzing the entire codebase. It won't codebase. It won't automatically create a rule for it, automatically create a rule for it, but it will appear as a but it will appear as a but it will appear as a suggestion for you to suggestion for you to suggestion for you to review. And this is what review. And this is what review. And this is what will be generated. You will be generated. You will be generated. You can can can edit all of this. There are edit all of this. There are edit all of this. There are examples of good and examples of good and examples of good and bad code. Just so that bad code. Just so that bad code. Just so that AI agents AI agents AI agents understand what is understand what is understand what is good and what is bad. And good and what is bad. And good and what is bad. And then you can then you can then you can define the category define the category define the category and and and warning levels. And then warning levels. And then , once you're , once you're , once you're happy with how it happy with how it happy with how it looks, you can looks, you can looks, you can accept it, uh, and accept it, uh, and accept it, uh, and activate that activate that activate that rule, and it will be rule, and it will be rule, and it will be automatically automatically automatically applied. Yes, applied. Yes, applied. Yes, continue. Uh, for

continue. Uh, for continue. Uh, for Codo, it's Codo, it's MGM. Uh, I think that's just loading the just loading the local local local repository rules, repository rules, repository rules, most most most if not all of which are related if not all of which are related if not all of which are related to the different paths you to the different paths you to the different paths you need to take need to take need to take in this workshop. "Uh in this workshop. "Uh , but overall, like , but overall, like , but overall, like the skill of getting a the skill of getting a the skill of getting a rule in Coda, it rule in Coda, it rule in Coda, it speaks for itself speaks for itself ." to the ." to the ." to the backend of the Coda platform, backend of the Coda platform, backend of the Coda platform, and it's kind of an and it's kind of an and it's kind of an agent-based process of agent-based process of agent-based process of identifying, um, the ones identifying, um, the ones identifying, um, the ones that are actually that are actually that are actually relevant and relevant and relevant and filtering. filtering. So, it's like a process. So, it's like a process. Yes. Hmm. That's a really Yes. Hmm. That's a really Yes. Hmm. That's a really good question. Um, good question. Um, good question. Um, okay, the plan was okay, the plan was okay, the plan was implemented, and I don't implemented, and I don't implemented, and I don't know why the know why the know why the draft was created.

draft was created. draft was created. I like I like I like real PRs, but well, real PRs, but well, real PRs, but well, whatever. Um, whatever. Um, whatever. Um, copy the link. copy the link. Okay, add a refund workflow with refund workflow with quality grids. Did you quality grids. Did you quality grids. Did you get there? Um, get there? Um, get there? Um, okay. Perfectly. And here okay. Perfectly. And here okay. Perfectly. And here we have the summary. I'm we have the summary. I'm we have the summary. I'm just going to just going to just going to make this a real make this a real make this a real PR because I want PR because I want PR because I want Coda to review this without Coda to review this without Coda to review this without merging merging merging pull requests. If Coda pull requests. If Coda pull requests. If Coda is connected to your is connected to your is connected to your repository, this is what repository, this is what repository, this is what you should see you should see pop up automatically. This pop up automatically. This will start the will start the will start the analysis process. And of course, analysis process. And of course, analysis process. And of course, I don't look at all the I don't look at all the I don't look at all the code because we code because we code because we don't have a whole day for don't have a whole day for don't have a whole day for that. Um, but you that. Um, but you that. Um, but you understand the point of understand the point of understand the point of implementing these implementing these implementing these processes. So in a processes. So in a processes. So in a few minutes, um, Coda few minutes, um, Coda few minutes, um, Coda will finish analyzing will finish analyzing will finish analyzing this code and output this code and output ...any potential ...any potential ...any potential problems, and we're problems, and we're problems, and we're going to going to going to look at what they are.

look at what they are. look at what they are. Um, as a result of Um, as a result of any potential any potential any potential code issues that code issues that code issues that arise, we're arise, we're arise, we're going to run an going to run an going to run an agent skill to agent skill to agent skill to automatically automatically automatically resolve them. Are there any other resolve them. Are there any other resolve them. Are there any other questions? They were questions? They were questions? They were really good. Yes. really good. Yes. really good. Yes. I have a question, and I have a question, and I have a question, and then the question that then the question that then the question that comes out of it is what comes out of it is what comes out of it is what I've seen with I've seen with I've seen with skills is that they skills is that they skills is that they can be kind of can be kind of can be kind of over-general, over-general, over-general, like dry code, and that's what like dry code, and that's what like dry code, and that's what good code looks like. good code looks like. good code looks like. And I argue with And I argue with And I argue with engineers who engineers who engineers who say, “This is say, “This is say, “This is too general. This too general. This too general. This should already be in the should already be in the should already be in the agent's hands. And I think agent's hands. And I think agent's hands. And I think you should wait you should wait you should wait until something comes up." until something comes up." until something comes up." “But to resolve these “But to resolve these “But to resolve these controversies, it would be controversies, it would be controversies, it would be really helpful to have really helpful to have really helpful to have data on such skills data on such skills data on such skills in a metric that in a metric that in a metric that shows that they are shows that they are shows that they are actually useful. Does actually useful. Does actually useful. Does Coda offer anything Coda offer anything Coda offer anything similar? Or how do you similar? Or how do you similar? Or how do you know that these skills are know that these skills are know that these skills are actually valuable and not actually valuable and not actually valuable and not just a choice of something? just a choice of something? just a choice of something? Hmm, Coda just Hmm, Coda just Hmm, Coda just takes skills into account. takes skills into account. takes skills into account. She doesn't rate their She doesn't rate their She doesn't rate their quality, but there is a quality, but there is a quality, but there is a tool that tool that tool that does, Tessell. Have you heard does, Tessell. Have you heard does, Tessell. Have you heard of Tessell before? T E S S E of Tessell before? T E S S E of Tessell before? T E S S E L—is a L—is a skills assessment company, and you can skills assessment company, and you can determine. She will tell determine. She will tell determine. She will tell you if there are any you if there are any you if there are any security issues with security issues with security issues with this skill, or is this skill, or is this skill, or is it actually as it actually as it actually as effective as you effective as you effective as you expect? This is the expect? This is the expect? This is the tool I would tool I would tool I would recommend for recommend for recommend for integrating this to integrating this to integrating this to determine this.

determine this. Thank you. Thank you. Yes. Good. Hmm, the PR Yes. Good. Hmm, the PR summary was summary was summary was generated generated generated automatically. We have a automatically. We have a automatically. We have a description and a diagram of the changes. description and a diagram of the changes. And it shows the And it shows the API legend in memory API legend in memory API legend in memory and the function. We have an and the function. We have an and the function. We have an API client and a fast API client and a fast API client and a fast API app, and API app, and API app, and different, let's different, let's different, let's see, see, see, endpoint..." endpoint..." Payment refund endpoint. This indicates Payment refund endpoint. This indicates four different fields. four different fields. That makes sense. So, That makes sense. So, one of the things I would one of the things I would one of the things I would recommend, but I recommend, but I recommend, but I realize I just realize I just realize I just missed, is a missed, is a missed, is a verification request. So, we verification request. So, we verification request. So, we can, after can, after can, after running the running the running the build process, making build process, making build process, making changes to the code, and changes to the code, and changes to the code, and actually actually actually doing a doing a doing a local review, local review, local review, run a run a run a validation request to validation request to validation request to make sure everything make sure everything make sure everything passed; but Codox already passed; but Codox already passed; but Codox already did this did this did this automatically on its automatically on its automatically on its own, simply own, simply own, simply because of how the because of how the repository is actually set up. But this would be a repository is actually set up. But this would be a best best best practice for practice for practice for going through this going through this going through this process. And any process. And any process. And any day Codox would be great

day Codox would be great . Well, we can . Well, we can . Well, we can take a look at the PR I take a look at the PR I take a look at the PR I already created. And this was a already created. And this was a already created. And this was a PR where I didn't run PR where I didn't run PR where I didn't run any of the processes any of the processes any of the processes we just we just we just talked about. But I have talked about. But I have talked about. But I have context obtained context obtained context obtained through the Codox portal. And that's something that through the Codox portal. And that's something that through the Codox portal. And that's something that , uh, is really , uh, is really , uh, is really important for you to important for you to important for you to consider when it comes to consider when it comes to consider when it comes to that that that level of context management that level of context management that I was talking about. Here, I was talking about. Here, during the code review, Codox during the code review, Codox during the code review, Codox discovered that there are rules in Codox discovered that there are rules in Codox discovered that there are rules in Codox that... that... that... Codox portal that are Codox portal that are Codox portal that are relevant to relevant to relevant to this actual this actual this actual code review. We are code review. We are code review. We are no longer talking about no longer talking about no longer talking about planning or planning or planning or implementation. We implementation. We implementation. We are talking specifically about are talking specifically about are talking specifically about code review. And code review. And code review. And then it also then it also then it also involved the appropriate involved the appropriate involved the appropriate skills, um, for this, skills, um, for this, skills, um, for this, um, this code change. And um, this code change. And um, this code change. And here came some here came some here came some results that results that results that required action, um, that required action, um, that required action, um, that required results required results . There's a rule within . There's a rule within . There's a rule within this, um, my this, um, my this, um, my organization where all organization where all organization where all Python functions must Python functions must Python functions must have a documentation line attached to it have a documentation line attached to it have a documentation line attached to it . And . And . And so it was shown so it was shown so it was shown as a rule violation and a as a rule violation and a support issue. The only support issue. The only reason this reason this reason this requires action is requires action is requires action is because our because our because our engineering engineering engineering organization organization organization has determined that it has a has determined that it has a has determined that it has a high level of high level of high level of severity.

severity. severity. Since you can Since you can Since you can adjust the adjust the adjust the severity level of your severity level of your severity level of your rules, it doesn't rules, it doesn't rules, it doesn't necessarily have to be necessarily have to be necessarily have to be something that deserves something that deserves something that deserves blocking. It blocking. It blocking. It just has to be something just has to be something just has to be something that gets resolved. It that gets resolved. It that gets resolved. It depends on you. And depends on you. And depends on you. And here is the evidence related here is the evidence related here is the evidence related to this. Require to this. Require to this. Require documentation lines for documentation lines for documentation lines for all Python functions. Well, all Python functions. Well, all Python functions. Well, I would say methods. I would say methods. I would say methods. Yes. Yes. Yes. Ahem. So, Yes. Yes. Yes. Ahem. So, Yes. Yes. Yes. Ahem. So, it's determined it's determined it's determined right during the right during the right during the code review phase, okay code review phase, okay , what rules will be , what rules will be , what rules will be relevant. relevant. relevant. The part that can The part that can The part that can be determined. be determined. be determined. is that you is that you is that you can decide, can decide, can decide, based on the based on the based on the severity level of the rule, that severity level of the rule, that severity level of the rule, that this code, this branch, or this code, this branch, or this code, this branch, or PR will be blocked. And PR will be blocked. And PR will be blocked. And so this is something you so this is something you so this is something you can configure can configure can configure in the Codota portal: in in the Codota portal: in in the Codota portal: in case of any case of any case of any required actions or required actions or required actions or detection of a detection of a detection of a high severity issue, high severity issue, high severity issue, we block the merge we block the merge we block the merge until you fix until you fix until you fix it and Codota it and Codota it and Codota will run again and will run again and will run again and check if it check if it check if it happened. Uh, there's happened. Uh, there's happened. Uh, there's a chance a chance a chance it won't determine what the it won't determine what the it won't determine what the relevant rule relevant rule relevant rule actually is...Yes. Um, and

actually is...Yes. Um, and actually is...Yes. Um, and I'm really glad you I'm really glad you I'm really glad you brought this up brought this up brought this up because it's something because it's something because it's something we've been talking about with the we've been talking about with the product development team. Um, and the product development team. Um, and the more corporate more corporate more corporate clients we clients we clients we talk to, talk to, talk to, especially in especially in especially in highly highly highly regulated industries, the regulated industries, the regulated industries, the more they want more they want more they want determinism in determinism in determinism in this vein. So I this vein. So I this vein. So I hope we hope we hope we can push can push can push for this to become a for this to become a for this to become a reality, because I reality, because I reality, because I think it's think it's think it's extremely important extremely important . I agree with you. Yes. Do . I agree with you. Yes. Do you mean this? After I After I solved the problems, yes solved the problems, yes , I think this is also a , I think this is also a , I think this is also a best practice, just in best practice, just in best practice, just in case. You case. You case. You never know, never know, never know, so... Um, you... I think so... Um, you... I think so... Um, you... I think it's more up it's more up it's more up to you, your agent to you, your agent to you, your agent might be... the one who might be... the one who might be... the one who would determine, would determine, would determine, like, "oh, am I like, "oh, am I like, "oh, am I going to do the background going to do the background going to do the background check or not," and you check or not," and you check or not," and you can suggest to can suggest to can suggest to your agent to your agent to your agent to do it do it anyway, regardless anyway, regardless anyway, regardless . But, yes, it . But, yes, it . But, yes, it depends on how depends on how depends on how your agent reacts.

your agent reacts. Hmm. And finally, here are the Hmm. And finally, here are the results. Oh, okay, results. Oh, okay, results. Oh, okay, five skills were involved five skills were involved five skills were involved . Okay, so . Okay, so . Okay, so we have one error, we have one error, we have one error, three rule violations, three rule violations, three rule violations, and two and two and two skill findings. There is a problem with skill findings. There is a problem with skill findings. There is a problem with the idempotence of the the idempotence of the the idempotence of the refund refund refund stored in stored in stored in memory. Therefore, the refund payment memory. Therefore, the refund payment uses a uses a local local local process dictionary to process dictionary to process dictionary to store and store and store and reproduce the reproduce the reproduce the fund results, which fund results, which fund results, which is not a long-lasting is not a long-lasting is not a long-lasting permanent storage. And permanent storage. And permanent storage. And we have a piece of code we have a piece of code we have a piece of code related to this, and related to this, and related to this, and we also have evidence. we also have evidence. Rules and subsequent Rules and subsequent files. Hmm, the important files. Hmm, the important files. Hmm, the important part that you part that you part that you all should definitely all should definitely experiment with experiment with is copying and is copying and is copying and pasting this pasting this pasting this agent query, which is agent query, which is agent query, which is designed to designed to designed to fix fix fix the suggestions that the suggestions that the suggestions that appear. Hmm, you appear. Hmm, you appear. Hmm, you can do this can do this can do this manually for each manually for each manually for each issue that occurs issue that occurs , or you can , or you can , or you can run Codium PR resolver run Codium PR resolver run Codium PR resolver for automatic for automatic for automatic resolution. Have you already resolution. Have you already resolution. Have you already started this skill?

started this skill? started this skill? No, yes. No, yes. No, yes. Yes, right now. Yes, right now. Yes, right now. Hmm. Hmm. Hmm. No, but we No, but we No, but we implemented it into our implemented it into our implemented it into our workflow workflow workflow today. today. today. Hmm. Hmm. Hmm. So, I'm going to be very So, I'm going to be very So, I'm going to be very similar because I don't similar because I don't similar because I don't want to get into the want to get into the want to get into the quality side, and I quality side, and I quality side, and I want the bug fixing to want the bug fixing to happen again through happen again through our own process. our own process. our own process. All Codium PR can All Codium PR can All Codium PR can reject because we reject because we reject because we may have a better may have a better may have a better way of using way of using way of using something, you may something, you may something, you may break something else. break something else. break something else. Absolutely. Absolutely. It is important, if we have It is important, if we have any automation, to any automation, to any automation, to re-enable the re-enable the re-enable the set section for set section for set section for verification. verification. verification. Hmm. Yeah, I think that's a Hmm. Yeah, I think that's a Hmm. Yeah, I think that's a best practice and best practice and best practice and that's, um, what I would that's, um, what I would that's, um, what I would do, or I think do, or I think do, or I think before the GenAI era, um, you just before the GenAI era, um, you just before the GenAI era, um, you just have to be have to be have to be careful and that's what careful and that's what careful and that's what you want you want you want your agents to do as well. So, your agents to do as well. So, your agents to do as well. So, that was a really, um, that was a really, um, that was a really, um, good point. Um, good point. Um, good point. Um, seeing all seeing all seeing all the results in this, the results in this, the results in this, um, PR, then I ran the um, PR, then I ran the um, PR, then I ran the Codium PR resolver skill, and Codium PR resolver skill, and Codium PR resolver skill, and I offered it for I offered it for I offered it for pull request pull request pull request number four, and I number four, and I number four, and I saw that it saw that it saw that it pulled up all the pulled up all the pulled up all the issues that Codium had already issues that Codium had already issues that Codium had already identified, from identified, from identified, from critical to high, critical to high, critical to high, and where did they go?

and where did they go? Oh, okay. So, it Oh, okay. So, it pulled all pulled all pulled all the results and now I the results and now I the results and now I can decide if can decide if can decide if I want to review I want to review I want to review each issue. each issue. each issue. separately, or do I want to separately, or do I want to separately, or do I want to automatically automatically automatically fix everything? Um, fix everything? Um, fix everything? Um, for the sake of this, uh, for the sake of this, uh, for the sake of this, uh, seminar, I'll just seminar, I'll just seminar, I'll just automatically fix automatically fix automatically fix everything. But, uh, here's the thing: everything. But, uh, here's the thing: everything. But, uh, here's the thing: if you want to be if you want to be if you want to be a little more a little more a little more hands-on and hands-on and hands-on and make decisions with make decisions with make decisions with your agent to your agent to your agent to actually plan out actually plan out actually plan out how you're going to how you're going to how you're going to make the fix, make the fix, make the fix, you can do it you can do it you can do it yourself yourself yourself with the " with the " review each review each review each issue" option, uh, " issue" option, uh, " Automatically Automatically Automatically fix everything." Now fix everything." Now fix everything." Now let's see let's see let's see how it reacts. So, how it reacts. So, how it reacts. So, I'm going to I'm going to I'm going to automatically automatically automatically fix all six fix all six fix all six Codo issues, and what's Codo issues, and what's Codo issues, and what's going to happen is... going to happen is... Okay, and you can Okay, and you can Okay, and you can see here, it see here, it see here, it says that then says that then a check will be run and a check will be run and commits will be made. It commits will be made. It commits will be made. It won't be a commit that won't be a commit that won't be a commit that will be blocked will be blocked will be blocked if it's not if it's not if it's not structured structured structured correctly, uh, but it's the correctly, uh, but it's the correctly, uh, but it's the point where it will be point where it will be point where it will be done after.

done after. And that will take the And that will take the next step, next step, next step, actually actually actually publishing it publishing it publishing it back to PR and, uh, with a back to PR and, uh, with a back to PR and, uh, with a summary of all the summary of all the summary of all the fixed changes. And fixed changes. And fixed changes. And from there, that's how the PR from there, that's how the PR from there, that's how the PR will be updated, and all the will be updated, and all the will be updated, and all the resolved issues resolved issues resolved issues will be, uh, will be, uh, will be, uh, crossed out. outside. crossed out. outside. I mean, I I mean, I used used used working trees for working trees for working trees for this. I...You...Did you this. I...You...Did you this. I...You...Did you use it? Yes. use it? Yes. Hmm. Hmm, yeah, that's not what Hmm. Hmm, yeah, that's not what Codo really Codo really Codo really does. I does. I does. I found that the coding agents seemed to found that the coding agents seemed to handle this themselves. handle this themselves. That's a good point, a That's a good point, a That's a good point, a good question. Are there good question. Are there good question. Are there any more questions before any more questions before any more questions before I wrap this up? Yes.

I wrap this up? Yes. Yes, you can Yes, you can do a do a do a code review, and I didn't run code review, and I didn't run code review, and I didn't run it for that because time was it for that because time was it for that because time was flying, but, um, you flying, but, um, you flying, but, um, you mean for mean for mean for Codium or for your Codium or for your Codium or for your coding, your coding, your coding, your coding agent. coding agent. So, we are developing a So, we are developing a local local local Codium validation workflow, Codium validation workflow, Codium validation workflow, and it will and it will interact directly with the Codium backend. So, all the Codium backend. So, all the agency capabilities agency capabilities that you would have in a that you would have in a that you would have in a withdrawal request, you would withdrawal request, you would withdrawal request, you would also have locally also have locally also have locally and could resolve them. and could resolve them. Hmm. What was that? How does it Hmm. What was that? How does it check all the code? check all the code? check all the code? It clones It clones It clones the repository. Hmm. Yes, the repository. Hmm. Yes, the repository. Hmm. Yes, this is a process we this is a process we this is a process we perform on the perform on the perform on the server side. It's server side. It's server side. It's like temporary like temporary like temporary cloning so that it cloning so that it cloning so that it understands, yes, a wider understands, yes, a wider understands, yes, a wider scope. changes that scope. changes that scope. changes that go beyond go beyond go beyond code differences.

code differences. code differences. Really good question. Really good question. Okay, I think Okay, I think time is almost up. time is almost up. So I couldn't go into So I couldn't go into detail on detail on detail on everything, but like I everything, but like I everything, but like I said, if nothing said, if nothing said, if nothing else you take away else you take away else you take away from this, take this from this, take this from this, take this repository and repository and repository and ask your AI for ask your AI for ask your AI for advice on what advice on what advice on what validation steps you'd validation steps you'd validation steps you'd like to implement like to implement like to implement or practice or practice or practice implementing implementing implementing locally. Um, and I locally. Um, and I locally. Um, and I think that would be a think that would be a think that would be a really important really important really important step to just step to just step to just automatically add automatically add automatically add more structure to more structure to more structure to your your your workflows and workflows and workflows and look at any look at any look at any tools. There are some, tools. There are some, tools. There are some, um, um, um, open source tools that open source tools that open source tools that have similar have similar context management layers like Coda. I context management layers like Coda. I think, um, there's think, um, there's think, um, there's no such thing, but if you no such thing, but if you no such thing, but if you look for it, you should look for it, you should look for it, you should try to try to try to practice having practice having practice having rules, skills, and rules, skills, and rules, skills, and other, um, engineering other, um, engineering other, um, engineering standards, whatever there standards, whatever there standards, whatever there are, stored are, stored are, stored somewhere that you, um, somewhere that you, um, somewhere that you, um, have access to so that you have access to so that you have access to so that you can pull that can pull that can pull that information, um, as information, um, as information, um, as it relates to your it relates to your it relates to your workflow stages workflow stages workflow stages . Oh yeah, it's . Oh yeah, it's . Oh yeah, it's called Packmind, called Packmind, called Packmind, I think. packmind.com So I think. packmind.com So I think. packmind.com So definitely check definitely check definitely check this out if you this out if you this out if you want an want an want an open source solution for, open source solution for, open source solution for, uh, managing your uh, managing your uh, managing your AI code, uh, AI code, uh, AI code, uh, through different through different through different rules and so on, rules and so on, rules and so on, consistently across your consistently across your consistently across your teams and your

teams and your teams and your repositories. Okay, repositories. Okay, repositories. Okay, I wish the I wish the I wish the internet was faster, internet was faster, internet was faster, but it isn't. So, I-I-I but it isn't. So, I-I-I can't-I-I can't, uh. can't-I-I can't, uh. can't-I-I can't, uh. This never This never This never happens, but here we are. happens, but here we are. Uh, let me know Uh, let me know if you have any if you have any if you have any questions. Uh, I'm questions. Uh, I'm questions. Uh, I'm probably here for the next probably here for the next probably here for the next few minutes. Thank you. few minutes. Thank you. Let's...Yes. I Let's...Yes. I did so. I'm at a did so. I'm at a did so. I'm at a point now where I don't point now where I don't point now where I don't quite understand the quite understand the context like in Compact, and context like in Compact, and I haven't had the I haven't had the I haven't had the problems that I would have had problems that I would have had problems that I would have had maybe three or maybe three or four months ago, with I four months ago, with I four months ago, with I think the progress that think the progress that think the progress that they've made, I they've made, I they've made, I just don't feel just don't feel just don't feel much degradation.

much degradation. much degradation. Good. Compact Good. Compact Good. Compact will like it. will like it. will like it. Yes, I would Yes, I would Yes, I would like it. Uh, but like it. Uh, but like it. Uh, but if I really feel if I really feel if I really feel like it, I'll just like it, I'll just like it, I'll just create a new, uh, create a new, uh, create a new, uh, new session and new session and new session and maybe copy and maybe copy and maybe copy and paste some paste some paste some stuff from the previous stuff from the previous stuff from the previous session into it. Yes, we have a session into it. Yes, we have a session into it. Yes, we have a huge stand here. So huge stand here. So , yes, come visit , yes, come visit , yes, come visit us. us. I assume this all I assume this all works...In the cloud, right? works...In the cloud, right? Um, Coda? Yes. Yes. Do Um, Coda? Yes. Yes. Do you have any local you have any local you have any local options for options for options for security reasons? security reasons? Yes, we have something like on- Yes, we have something like on- premise premise premise deployment and deployment and deployment and pretty much all the pretty much all the pretty much all the deployment options deployment options that you would want, that you would want, that you would want, because we work with because we work with because we work with banks and, you know, banks and, you know, banks and, you know, things like that, and, um, things like that, and, um, things like that, and, um, companies like companies like companies like some of the teams at Nvidia, there's some of the teams at Nvidia, there's some of the teams at Nvidia, there's no choice no choice no choice but to do it. but to do it. but to do it. So, yes, we are doing this So, yes, we are doing this So, yes, we are doing this integration. Hmm. Thank you.

integration. Hmm. Thank you. Let's see where Let's see where she comes from...Oh, there she is. she comes from...Oh, there she is. she comes from...Oh, there she is. Here is a brief summary of Here is a brief summary of Here is a brief summary of the fixes. Yes, she the fixes. Yes, she the fixes. Yes, she was. I don't even know was. I don't even know what time it is now. what time it is now. And how many things were And how many things were resolved? Everyone. All resolved? Everyone. All resolved? Everyone. All problems were problems were problems were solved. Perfectly. solved. Perfectly. solved. Perfectly. Good. Thank you all. Good. Thank you all. Good. Thank you all. If you have any If you have any If you have any questions, I'll be there. questions, I'll be there. Yes, girl.