Anthropic Replaced Git. Here's Why.
Read full transcript 10 segments
-
A few days ago, Anthropic released a A few days ago, Anthropic released a very interesting article on their very interesting article on their very interesting article on their engineering blog, where they introduced engineering blog, where they introduced engineering blog, where they introduced a new in-house source control system a new in-house source control system a new in-house source control system that is supposed to replace Git. This that is supposed to replace Git. This that is supposed to replace Git. This allowed them to scale from 2,500 merges allowed them to scale from 2,500 merges allowed them to scale from 2,500 merges per day a couple of months ago to more per day a couple of months ago to more per day a couple of months ago to more than 30,000 merges per day in September. than 30,000 merges per day in September. than 30,000 merges per day in September. In this video, I want to do an overview In this video, I want to do an overview In this video, I want to do an overview on this new source control system that on this new source control system that on this new source control system that they call Sion. This is their they call Sion. This is their they call Sion. This is their announcement on X, where they introduced announcement on X, where they introduced announcement on X, where they introduced their new in-house source control their new in-house source control their new in-house source control system, and they say that in the month system, and they say that in the month system, and they say that in the month of May they merged around 2,500 source of May they merged around 2,500 source of May they merged around 2,500 source control changes per day into their repo. control changes per day into their repo. control changes per day into their repo. In September, which is almost a month In September, which is almost a month In September, which is almost a month ago at the time of recording this video, ago at the time of recording this video, ago at the time of recording this video, they were merging more than 30,000 they were merging more than 30,000 they were merging more than 30,000 changes per day to their repo. So, they changes per day to their repo. So, they changes per day to their repo. So, they made a technical write-up on their made a technical write-up on their made a technical write-up on their engineering blog about this new source engineering blog about this new source engineering blog about this new source control system, and we're going to control system, and we're going to control system, and we're going to review it in this video as it offers review it in this video as it offers review it in this video as it offers some interesting insights into their some interesting insights into their some interesting insights into their technical choices, and I especially like technical choices, and I especially like technical choices, and I especially like their commentary about how software their commentary about how software their commentary about how software engineering is changing. So, here's the engineering is changing. So, here's the engineering is changing. So, here's the article. I'm going to leave it in the article. I'm going to leave it in the article. I'm going to leave it in the description of the video if you want to description of the video if you want to description of the video if you want to check it out, and it's titled check it out, and it's titled check it out, and it's titled Redesigning Source Control at Anthropic.
-
Redesigning Source Control at Anthropic. Redesigning Source Control at Anthropic. Whatever your feelings may be for Whatever your feelings may be for Whatever your feelings may be for Anthropic or any of the other AI labs, Anthropic or any of the other AI labs, Anthropic or any of the other AI labs, it's really interesting to consider what it's really interesting to consider what it's really interesting to consider what we're able to achieve with the current we're able to achieve with the current we're able to achieve with the current AI models that we have at hand. And take AI models that we have at hand. And take AI models that we have at hand. And take into consideration that these AI labs into consideration that these AI labs into consideration that these AI labs probably have a couple of generations probably have a couple of generations probably have a couple of generations improved models that they are using improved models that they are using improved models that they are using internally, and these are not available internally, and these are not available internally, and these are not available to the public. And another thing to keep to the public. And another thing to keep to the public. And another thing to keep in mind while reading this article is in mind while reading this article is in mind while reading this article is that they can probably burn an infinite that they can probably burn an infinite that they can probably burn an infinite amount of tokens while building this amount of tokens while building this amount of tokens while building this source control system or any other of source control system or any other of source control system or any other of their internal services. I'm going to their internal services. I'm going to their internal services. I'm going to zoom in a little so you can see this zoom in a little so you can see this zoom in a little so you can see this better, and in this article they explain better, and in this article they explain better, and in this article they explain how they were bottlenecked by the nature how they were bottlenecked by the nature how they were bottlenecked by the nature of Git and how repositories work, and of Git and how repositories work, and of Git and how repositories work, and they had a hard time scaling from their they had a hard time scaling from their they had a hard time scaling from their throughput in May, which was 2,500 throughput in May, which was 2,500 throughput in May, which was 2,500 merges per day, and so they figured they merges per day, and so they figured they merges per day, and so they figured they needed to build something else. So, you needed to build something else. So, you needed to build something else. So, you can see the curve here of how the number can see the curve here of how the number can see the curve here of how the number of merge changes per day was changing, of merge changes per day was changing, of merge changes per day was changing, and you can can a significant spike and you can can a significant spike and you can can a significant spike start to show up from the month of June start to show up from the month of June start to show up from the month of June all the way into September and it all the way into September and it all the way into September and it doesn't look like it's going to be doesn't look like it's going to be doesn't look like it's going to be slowing down. Now, the reason the left slowing down. Now, the reason the left slowing down. Now, the reason the left is in blue and the right one is in red is in blue and the right one is in red is in blue and the right one is in red is because they switched from pull is because they switched from pull is because they switched from pull requests to antique change requests and requests to antique change requests and requests to antique change requests and antique is another internal system which antique is another internal system which antique is another internal system which they don't cover in this article but they don't cover in this article but they don't cover in this article but they're using it as a code review tool.
-
they're using it as a code review tool. they're using it as a code review tool. Now, this new source control system Now, this new source control system Now, this new source control system which they called Scion works in a which they called Scion works in a which they called Scion works in a completely different way from how a Git completely different way from how a Git completely different way from how a Git repository works. Now, before they repository works. Now, before they repository works. Now, before they decided to switch, they did whatever decided to switch, they did whatever decided to switch, they did whatever they could to scale their monorepo on they could to scale their monorepo on they could to scale their monorepo on Git and I think it's interesting to note Git and I think it's interesting to note Git and I think it's interesting to note that a lot of these AI labs are working that a lot of these AI labs are working that a lot of these AI labs are working in monorepos and it makes a lot of sense in monorepos and it makes a lot of sense in monorepos and it makes a lot of sense when you think about it because an AI when you think about it because an AI when you think about it because an AI agent thrives with a lot of context and agent thrives with a lot of context and agent thrives with a lot of context and what better context could you give it what better context could you give it what better context could you give it than if all of your services are nicely than if all of your services are nicely than if all of your services are nicely stored inside of a monorepo that you can stored inside of a monorepo that you can stored inside of a monorepo that you can continue building on and this allows you continue building on and this allows you continue building on and this allows you to merge all of the conventions that you to merge all of the conventions that you to merge all of the conventions that you would want to enforce across your would want to enforce across your would want to enforce across your codebase into one single repository. And codebase into one single repository. And codebase into one single repository. And I've had this problem myself building I've had this problem myself building I've had this problem myself building some internal tools recently where all some internal tools recently where all some internal tools recently where all of them are currently in separate Git of them are currently in separate Git of them are currently in separate Git repos and I find myself often, whenever repos and I find myself often, whenever repos and I find myself often, whenever I want to build a new tool, I have to I want to build a new tool, I have to I want to build a new tool, I have to spin up a new Git repo and now I spin up a new Git repo and now I spin up a new Git repo and now I remember that I've got some conventions remember that I've got some conventions remember that I've got some conventions from this other repository that I would from this other repository that I would from this other repository that I would like to have in my new repo. So, I tell like to have in my new repo. So, I tell like to have in my new repo. So, I tell the agent to go check out that repo and the agent to go check out that repo and the agent to go check out that repo and pull the conventions from there and then pull the conventions from there and then pull the conventions from there and then use those when implementing my new tool. use those when implementing my new tool. use those when implementing my new tool. And come to think of it, I should And come to think of it, I should And come to think of it, I should probably merge all of my services into a probably merge all of my services into a probably merge all of my services into a monorepo and make my life easier. But monorepo and make my life easier. But monorepo and make my life easier. But instead of ranting about this, let's instead of ranting about this, let's instead of ranting about this, let's actually try to understand how Git works actually try to understand how Git works actually try to understand how Git works as a system. So, it's decentralized in as a system. So, it's decentralized in as a system. So, it's decentralized in nature and what this means is yes, nature and what this means is yes, nature and what this means is yes, there's a central place where you store there's a central place where you store there's a central place where you store your repo, let's say on GitHub, but your repo, let's say on GitHub, but your repo, let's say on GitHub, but effectively your local copy or clone is effectively your local copy or clone is effectively your local copy or clone is a fully fledged Git repository and you a fully fledged Git repository and you a fully fledged Git repository and you can use it to compute diffs, read can use it to compute diffs, read can use it to compute diffs, read history including the full history of history including the full history of history including the full history of your repo, you can blame a specific line your repo, you can blame a specific line your repo, you can blame a specific line of code to figure out who made the
-
of code to figure out who made the of code to figure out who made the change and when, you can see how a file change and when, you can see how a file change and when, you can see how a file changed through multiple versions as all changed through multiple versions as all changed through multiple versions as all of these are part of your Git history, of these are part of your Git history, of these are part of your Git history, and you clone the whole thing on your and you clone the whole thing on your and you clone the whole thing on your file system and append changes on top. file system and append changes on top. file system and append changes on top. Every now and then, you're going to push Every now and then, you're going to push Every now and then, you're going to push your changes to whatever is the central your changes to whatever is the central your changes to whatever is the central Git repository, and this is where issues Git repository, and this is where issues Git repository, and this is where issues like merge conflicts come in because like merge conflicts come in because like merge conflicts come in because another developer could be making another developer could be making another developer could be making concurrent changes, and in Anthropic's concurrent changes, and in Anthropic's concurrent changes, and in Anthropic's case, their main branch in the case, their main branch in the case, their main branch in the repository was 100 GB when downloaded on repository was 100 GB when downloaded on repository was 100 GB when downloaded on a file system. And why is this a a file system. And why is this a a file system. And why is this a problem? Well, when developing any problem? Well, when developing any problem? Well, when developing any serious project, you're going to have a serious project, you're going to have a serious project, you're going to have a number of continuous integration number of continuous integration number of continuous integration workflows. You're going to be building workflows. You're going to be building workflows. You're going to be building your code, testing it, running your code, testing it, running your code, testing it, running verification, scanning it for security verification, scanning it for security verification, scanning it for security vulnerabilities, and a bunch of other vulnerabilities, and a bunch of other vulnerabilities, and a bunch of other things. And your CI workflow needs to things. And your CI workflow needs to things. And your CI workflow needs to download a copy of your repo so that it download a copy of your repo so that it download a copy of your repo so that it can build it and run all the other can build it and run all the other can build it and run all the other checks that it needs. And if Anthropic checks that it needs. And if Anthropic checks that it needs. And if Anthropic was doing this naively, it would result was doing this naively, it would result was doing this naively, it would result in 600 petabytes of Git traffic per day. in 600 petabytes of Git traffic per day. in 600 petabytes of Git traffic per day. Now, there are ways to improve this Now, there are ways to improve this Now, there are ways to improve this using Git mirrors and repository using Git mirrors and repository using Git mirrors and repository snapshots, which are like point-in-time snapshots, which are like point-in-time snapshots, which are like point-in-time copies that you can reference, and these copies that you can reference, and these copies that you can reference, and these are stored in some blob storage, but all are stored in some blob storage, but all are stored in some blob storage, but all of this hits a limit when you are of this hits a limit when you are of this hits a limit when you are generating code at Anthropic's scale.
-
generating code at Anthropic's scale. generating code at Anthropic's scale. So, what they decided to do is So, what they decided to do is So, what they decided to do is completely rethink their source control completely rethink their source control completely rethink their source control approach to be more agent-centric. Now, approach to be more agent-centric. Now, approach to be more agent-centric. Now, they did base this on existing work like they did base this on existing work like they did base this on existing work like Google's Piper and Meta's Sapling, which Google's Piper and Meta's Sapling, which Google's Piper and Meta's Sapling, which are both source control systems, and are both source control systems, and are both source control systems, and they also use Google's Spanner as their they also use Google's Spanner as their they also use Google's Spanner as their database. So, a lot of these supporting database. So, a lot of these supporting database. So, a lot of these supporting infrastructure and concepts were already infrastructure and concepts were already infrastructure and concepts were already built, and they didn't really have to built, and they didn't really have to built, and they didn't really have to invent those from scratch. But, here are invent those from scratch. But, here are invent those from scratch. But, here are the core principles behind their new the core principles behind their new the core principles behind their new source control system called Scion. So, source control system called Scion. So, source control system called Scion. So, clients are always online. With a Git clients are always online. With a Git clients are always online. With a Git repo, you can work completely offline, repo, you can work completely offline, repo, you can work completely offline, disconnected from other developers, disconnected from other developers, disconnected from other developers, which also have offline copies of the which also have offline copies of the which also have offline copies of the repo, and everyone can work repo, and everyone can work repo, and everyone can work independently. With their source control independently. With their source control independently. With their source control system, you have to connect to a central system, you have to connect to a central system, you have to connect to a central server that allows you to fetch that server that allows you to fetch that server that allows you to fetch that allows you to fetch data, push changes, allows you to fetch data, push changes, allows you to fetch data, push changes, and do all the other operations that you and do all the other operations that you and do all the other operations that you would normally be able to do with a Git would normally be able to do with a Git would normally be able to do with a Git repo. And everything had to be repo. And everything had to be repo. And everything had to be horizontally scalable. And in Scion, the horizontally scalable. And in Scion, the horizontally scalable. And in Scion, the data is distributed, and every component data is distributed, and every component data is distributed, and every component is independently scalable. This is independently scalable. This is independently scalable. This practically means that you can grab a practically means that you can grab a practically means that you can grab a fraction of the source code that you fraction of the source code that you fraction of the source code that you need to do your work without having to need to do your work without having to need to do your work without having to download the entire repo. And you can download the entire repo. And you can download the entire repo. And you can see how advantageous this could be, see how advantageous this could be, see how advantageous this could be, especially when working with agents, as especially when working with agents, as especially when working with agents, as you could have multiple agents running you could have multiple agents running you could have multiple agents running in parallel working on distinct parts of in parallel working on distinct parts of in parallel working on distinct parts of the mono repo, and every agent can pull the mono repo, and every agent can pull the mono repo, and every agent can pull the source code that's required for the the source code that's required for the the source code that's required for the work they need to do. The part I found work they need to do. The part I found work they need to do. The part I found especially interesting is how Scion especially interesting is how Scion especially interesting is how Scion actually works. There's this data actually works. There's this data actually works. There's this data structure section, and their unit to structure section, and their unit to structure section, and their unit to work would think of as a commit is a work would think of as a commit is a work would think of as a commit is a change set. And this contains the author change set. And this contains the author change set. And this contains the author information, the change set message, and information, the change set message, and information, the change set message, and the changes to the files that you want
-
the changes to the files that you want the changes to the files that you want to make in a specific change. And all of to make in a specific change. And all of to make in a specific change. And all of these are appended to a write-ahead log. these are appended to a write-ahead log. these are appended to a write-ahead log. Now, why is this interesting? Well, it's Now, why is this interesting? Well, it's Now, why is this interesting? Well, it's interesting because you can replay a interesting because you can replay a interesting because you can replay a write-ahead log, and this gives you an write-ahead log, and this gives you an write-ahead log, and this gives you an option to compute a lot of data that you option to compute a lot of data that you option to compute a lot of data that you need instead of having to generate need instead of having to generate need instead of having to generate everything at the point a change is everything at the point a change is everything at the point a change is written. This also means you can written. This also means you can written. This also means you can generate point-in-time copies of this generate point-in-time copies of this generate point-in-time copies of this data by replaying only specific pieces data by replaying only specific pieces data by replaying only specific pieces of the write-ahead log. And they say so of the write-ahead log. And they say so of the write-ahead log. And they say so here, a change set record only what's here, a change set record only what's here, a change set record only what's changed, so it can't do a blame and tell changed, so it can't do a blame and tell changed, so it can't do a blame and tell you who last edited a line, or give you you who last edited a line, or give you you who last edited a line, or give you the full history for a file. Instead, the full history for a file. Instead, the full history for a file. Instead, they build additional data structures by they build additional data structures by they build additional data structures by replaying the write-ahead log, and they replaying the write-ahead log, and they replaying the write-ahead log, and they have data structures for file naming, have data structures for file naming, have data structures for file naming, for path history, and skip lists for the for path history, and skip lists for the for path history, and skip lists for the commit graph. The obvious benefits is commit graph. The obvious benefits is commit graph. The obvious benefits is that this allows them to at any time add that this allows them to at any time add that this allows them to at any time add a new type of derived data when they a new type of derived data when they a new type of derived data when they figure this could give them an figure this could give them an figure this could give them an operational benefit in some segment. operational benefit in some segment. operational benefit in some segment. Here's a quick visual comparison to help Here's a quick visual comparison to help Here's a quick visual comparison to help you understand the difference between a you understand the difference between a you understand the difference between a Git repo and Scion, their new system. Git repo and Scion, their new system. Git repo and Scion, their new system. So, with a Git repo, we have a remote So, with a Git repo, we have a remote So, with a Git repo, we have a remote repo, the source repo on a Git server repo, the source repo on a Git server repo, the source repo on a Git server like GitHub. But, you also have your like GitHub. But, you also have your like GitHub. But, you also have your local copy of this repo, which is a local copy of this repo, which is a local copy of this repo, which is a complete copy containing all of the complete copy containing all of the complete copy containing all of the history, all of the objects at the point history, all of the objects at the point history, all of the objects at the point in time that you downloaded this copy.
-
in time that you downloaded this copy. in time that you downloaded this copy. At any point, you can do a pull from the At any point, you can do a pull from the At any point, you can do a pull from the remote repo and download changes that remote repo and download changes that remote repo and download changes that were made in the meantime by other were made in the meantime by other were made in the meantime by other developers or agents. And because you developers or agents. And because you developers or agents. And because you have a local copy of the repo, you can have a local copy of the repo, you can have a local copy of the repo, you can also work offline without having to also work offline without having to also work offline without having to connect to the remote repo. With Scion, connect to the remote repo. With Scion, connect to the remote repo. With Scion, things are a bit different. You don't things are a bit different. You don't things are a bit different. You don't download the full copy of the repo. download the full copy of the repo. download the full copy of the repo. Instead, you do an on-demand fetch of Instead, you do an on-demand fetch of Instead, you do an on-demand fetch of only the parts of the source code that only the parts of the source code that only the parts of the source code that you need. And then when you make you need. And then when you make you need. And then when you make changes, you create what's called a changes, you create what's called a changes, you create what's called a change set that describes the files you change set that describes the files you change set that describes the files you changed, and you commit that to the changed, and you commit that to the changed, and you commit that to the central Scion service, which appends the central Scion service, which appends the central Scion service, which appends the changes to the write-ahead log, and then changes to the write-ahead log, and then changes to the write-ahead log, and then based on those changes, it computes the based on those changes, it computes the based on those changes, it computes the required data structure. This also means required data structure. This also means required data structure. This also means that if you want to do a get blame or that if you want to do a get blame or that if you want to do a get blame or view the history of a file, you do an view the history of a file, you do an view the history of a file, you do an API call to the Scion server to answer API call to the Scion server to answer API call to the Scion server to answer those questions for you, rather than those questions for you, rather than those questions for you, rather than being able to get that information being able to get that information being able to get that information locally from your get repo. And then locally from your get repo. And then locally from your get repo. And then here's an example of how their new here's an example of how their new here's an example of how their new system actually works. So, you've got a system actually works. So, you've got a system actually works. So, you've got a client that's either a developer or in client that's either a developer or in client that's either a developer or in their case an agent that commits a their case an agent that commits a their case an agent that commits a change set to the write-ahead log. Now, change set to the write-ahead log. Now, change set to the write-ahead log. Now, these changes are stored in blob these changes are stored in blob these changes are stored in blob storage, and then they use Google storage, and then they use Google storage, and then they use Google Spanner to store the derived data. This Spanner to store the derived data. This Spanner to store the derived data. This includes the blame history, the changes includes the blame history, the changes includes the blame history, the changes for a single file, and the entire commit for a single file, and the entire commit for a single file, and the entire commit graph. And this derived data can be graph. And this derived data can be graph. And this derived data can be computed either eagerly, because you computed either eagerly, because you computed either eagerly, because you often need it, or it can be computed on often need it, or it can be computed on often need it, or it can be computed on demand when you actually request it.
-
demand when you actually request it. demand when you actually request it. Here's what the architecture of their Here's what the architecture of their Here's what the architecture of their system looks like. So, they have several system looks like. So, they have several system looks like. So, they have several clients. This can be their code review clients. This can be their code review clients. This can be their code review tool, developer machines, or continuous tool, developer machines, or continuous tool, developer machines, or continuous integration jobs. Now, all of these talk integration jobs. Now, all of these talk integration jobs. Now, all of these talk to a Scion demon process, which contains to a Scion demon process, which contains to a Scion demon process, which contains the objects that the clients need to the objects that the clients need to the objects that the clients need to download to do development work or build download to do development work or build download to do development work or build and test the code base. If those objects and test the code base. If those objects and test the code base. If those objects aren't present on the demon, they can aren't present on the demon, they can aren't present on the demon, they can talk to a Scion cache node distributed talk to a Scion cache node distributed talk to a Scion cache node distributed per region. And then if the data isn't per region. And then if the data isn't per region. And then if the data isn't available on the cache, the cache can available on the cache, the cache can available on the cache, the cache can talk to the central Scion server to talk to the central Scion server to talk to the central Scion server to fetch the data from Spanner or cloud fetch the data from Spanner or cloud fetch the data from Spanner or cloud storage. And this type of architecture storage. And this type of architecture storage. And this type of architecture allows them to independently scale each allows them to independently scale each allows them to independently scale each layer. Now, if we scroll down, you can layer. Now, if we scroll down, you can layer. Now, if we scroll down, you can see the number of changes to the Scion see the number of changes to the Scion see the number of changes to the Scion and Antique code base. And obviously, and Antique code base. And obviously, and Antique code base. And obviously, there's the same rapid spike after the there's the same rapid spike after the there's the same rapid spike after the month of June, and it doesn't seem to be month of June, and it doesn't seem to be month of June, and it doesn't seem to be slowing. Now, I found this part of the slowing. Now, I found this part of the slowing. Now, I found this part of the article particularly interesting. So, a article particularly interesting. So, a article particularly interesting. So, a team of seven people built almost the team of seven people built almost the team of seven people built almost the entire thing, obviously using agents as entire thing, obviously using agents as entire thing, obviously using agents as massive leverage, but this is still a massive leverage, but this is still a massive leverage, but this is still a fairly small team and they managed to fairly small team and they managed to fairly small team and they managed to build a source control system for a build a source control system for a build a source control system for a company the size of Anthropic and then company the size of Anthropic and then company the size of Anthropic and then complete the entire migration in about 5 complete the entire migration in about 5 complete the entire migration in about 5 months. I also like this remark where months. I also like this remark where months. I also like this remark where engineers are no less valuable, but engineers are no less valuable, but engineers are no less valuable, but their work is changing. All of the their work is changing. All of the their work is changing. All of the engineers who worked on the system would engineers who worked on the system would engineers who worked on the system would now rate Claude as better at writing now rate Claude as better at writing now rate Claude as better at writing code than them, but the engineers were code than them, but the engineers were code than them, but the engineers were responsible for supplying the invaluable responsible for supplying the invaluable responsible for supplying the invaluable understanding of the high-level understanding of the high-level understanding of the high-level intersystem components and the design intersystem components and the design intersystem components and the design level trade-offs. Making informed level trade-offs. Making informed level trade-offs. Making informed choices around the problems we wanted to choices around the problems we wanted to choices around the problems we wanted to solve and the scale we knew we wanted to solve and the scale we knew we wanted to solve and the scale we knew we wanted to achieve. So, my takeaway from this is achieve. So, my takeaway from this is achieve. So, my takeaway from this is that you still need to be very damn good that you still need to be very damn good that you still need to be very damn good at building distributed systems,
-
at building distributed systems, at building distributed systems, software architecture, system design, software architecture, system design, software architecture, system design, and all the other fundamentals that make and all the other fundamentals that make and all the other fundamentals that make up software engineering so that you can up software engineering so that you can up software engineering so that you can steer the agents in the right direction steer the agents in the right direction steer the agents in the right direction and let them produce the code that you and let them produce the code that you and let them produce the code that you could manually type yourself, just a lot could manually type yourself, just a lot could manually type yourself, just a lot slower. And in some case, you probably slower. And in some case, you probably slower. And in some case, you probably could not generate all of that code and could not generate all of that code and could not generate all of that code and this now opens up a new options that are this now opens up a new options that are this now opens up a new options that are available using agents. Here's a quick available using agents. Here's a quick available using agents. Here's a quick walk-through of the performance walk-through of the performance walk-through of the performance improvements they saw comparing Git and improvements they saw comparing Git and improvements they saw comparing Git and their new source control system and you their new source control system and you their new source control system and you can pretty much see massive improvements can pretty much see massive improvements can pretty much see massive improvements across the board. I particularly like across the board. I particularly like across the board. I particularly like the generation of a new work tree, down the generation of a new work tree, down the generation of a new work tree, down from 7 minutes to 5 seconds. And this is from 7 minutes to 5 seconds. And this is from 7 minutes to 5 seconds. And this is very useful if you work with a lot of very useful if you work with a lot of very useful if you work with a lot of agents in parallel on your system. agents in parallel on your system. agents in parallel on your system. Instead of letting the agents fight it Instead of letting the agents fight it Instead of letting the agents fight it out inside of your repo, you create a out inside of your repo, you create a out inside of your repo, you create a work tree, which is like a local copy of work tree, which is like a local copy of work tree, which is like a local copy of the repo, and let the agent work in its the repo, and let the agent work in its the repo, and let the agent work in its own copy. Now, this is computationally own copy. Now, this is computationally own copy. Now, this is computationally expensive. It also takes up additional expensive. It also takes up additional expensive. It also takes up additional storage space, but it allows you to storage space, but it allows you to storage space, but it allows you to parallelize the work, which would be parallelize the work, which would be parallelize the work, which would be hard to do normally as the agents would hard to do normally as the agents would hard to do normally as the agents would run into a lot of conflicts. So, if run into a lot of conflicts. So, if run into a lot of conflicts. So, if you're not already using work trees for you're not already using work trees for you're not already using work trees for your agents, you should definitely check your agents, you should definitely check your agents, you should definitely check them out. So, what's the takeaway from them out. So, what's the takeaway from them out. So, what's the takeaway from this entire video? Should you go ahead this entire video? Should you go ahead this entire video? Should you go ahead and rewrite your source control system and rewrite your source control system and rewrite your source control system to some bespoke version for your to some bespoke version for your to some bespoke version for your company? Well, absolutely not. You company? Well, absolutely not. You company? Well, absolutely not. You should probably still be using Git as should probably still be using Git as should probably still be using Git as you're not operating at Anthropic scale.
-
you're not operating at Anthropic scale. you're not operating at Anthropic scale. You could consider switching to a mono You could consider switching to a mono You could consider switching to a mono repo if you're not already using it as repo if you're not already using it as repo if you're not already using it as it does come with some benefits for it does come with some benefits for it does come with some benefits for agentic work. But my takeaway is agentic work. But my takeaway is agentic work. But my takeaway is actually this and it's quite optimistic. actually this and it's quite optimistic. actually this and it's quite optimistic. Using agents, we can now do all of this Using agents, we can now do all of this Using agents, we can now do all of this fun work that was previously either fun work that was previously either fun work that was previously either impossible because it would take too impossible because it would take too impossible because it would take too much time or simply too expensive in much time or simply too expensive in much time or simply too expensive in terms of developer time when compared to terms of developer time when compared to terms of developer time when compared to other priorities for the business that other priorities for the business that other priorities for the business that you would normally be working on as an you would normally be working on as an you would normally be working on as an engineer. I think this now creates many engineer. I think this now creates many engineer. I think this now creates many new possibilities that previously new possibilities that previously new possibilities that previously weren't there of what we can build in weren't there of what we can build in weren't there of what we can build in the future. And this in turn ends up the future. And this in turn ends up the future. And this in turn ends up increasing the need for software increasing the need for software increasing the need for software engineers. Either way, let me know in engineers. Either way, let me know in engineers. Either way, let me know in the comments when you are switching to a the comments when you are switching to a the comments when you are switching to a mono repo, hit the smash button if you mono repo, hit the smash button if you mono repo, hit the smash button if you enjoyed this video, and until next time enjoyed this video, and until next time enjoyed this video, and until next time stay awesome.
No summary available yet.
View original episode ↗