Why We Deleted Our MCP Server and Rebuilt It — Abhi Arya, Reducto
Read full transcript 13 segments
-
So, I'm Avi, and I work So, I'm Avi, and I work on a product here at on a product here at on a product here at Reducto. And before I Reducto. And before I Reducto. And before I dive into the details, dive into the details, dive into the details, I'd like to give you I'd like to give you I'd like to give you a little more a little more a little more context on context on context on what Reducto is. We are an what Reducto is. We are an agent-based agent-based agent-based document platform, which document platform, which document platform, which means we means we means we help teams help teams help teams transform transform transform messy, messy, messy, unstructured unstructured unstructured documents into data documents into data documents into data that your that your that your tools and AI tools and AI agents understand. I'll agents understand. I'll agents understand. I'll get to the get to the get to the agents themselves in a moment, but first I agents themselves in a moment, but first I agents themselves in a moment, but first I want to give a quick want to give a quick want to give a quick overview of where overview of where overview of where we are now. we are now. we are now. Over the past 3 years, we have Over the past 3 years, we have Over the past 3 years, we have processed over 3 processed over 3 processed over 3 billion documents billion documents billion documents for clients such for clients such for clients such as Harvey, Scale AI, as well as the as Harvey, Scale AI, as well as the top five global top five global technology technology technology companies and hedge companies and hedge funds. Working at funds. Working at funds. Working at this scale this scale this scale means we've seen means we've seen means we've seen almost every way a almost every way a almost every way a document can document can document can go wrong in go wrong in go wrong in production, and we've production, and we've production, and we've adapted our adapted our adapted our product and models to product and models to handle anything. Today, it is not so difficult to get AI Today, it is not so difficult to get AI to work with a clean to work with a clean to work with a clean document document document using an advanced using an advanced using an advanced model model model . You can just . You can just . You can just upload something to the upload something to the upload something to the cloud and get cloud and get cloud and get about 60% of the result, about 60% of the result, about 60% of the result, but the most difficult thing is but the most difficult thing is but the most difficult thing is real production.
-
real production. real production. Scanned documents, Scanned documents, Scanned documents, unfolded pages, unfolded pages, unfolded pages, etc. The model starts etc. The model starts etc. The model starts to hallucinate to hallucinate to hallucinate because most because most corporate data is corporate data is unstructured. So unstructured. So unstructured. So Reducto is the layer that Reducto is the layer that Reducto is the layer that stands between your stands between your stands between your model and this model and this model and this data. We take on data. We take on data. We take on complex layouts complex layouts complex layouts and post-processing, which is why and post-processing, which is why and post-processing, which is why we we we work with work with work with and and and create many agents ourselves, create many agents ourselves, create many agents ourselves, designing these designing these designing these pipelines. Now that pipelines. Now that pipelines. Now that you know a little more you know a little more you know a little more about what we about what we about what we do, I'll move on do, I'll move on do, I'll move on to our main to our main to our main topic through something seemingly topic through something seemingly unrelated. unrelated. I visited I visited I visited my girlfriend this weekend and she my girlfriend this weekend and she my girlfriend this weekend and she gave me a gave me a gave me a sticker from the movie sticker from the movie Mean Girls, which Mean Girls, which Mean Girls, which adapted the adapted the adapted the agent version of the phrase. agent version of the phrase. agent version of the phrase. It says: "Sit down, It says: "Sit down, It says: "Sit down, loser." We create loser." We create software where software where agents come first agents come first agents come first .” And, as a .” And, as a .” And, as a true true software engineer, I had to software engineer, I had to software engineer, I had to put a put a put a sticker like this on my sticker like this on my sticker like this on my laptop. She laptop. She laptop. She brought it from work brought it from work brought it from work and asked me, "What and asked me, "What and asked me, "What is software that puts is software that puts agents first agents first ?" Because she doesn't ?" Because she doesn't ?" Because she doesn't work in the field of AI. And work in the field of AI. And work in the field of AI. And as I thought about it, as I thought about it, as I thought about it, I realized that I had I realized that I had I realized that I had many different many different many different answers. answers. Creating a nice Creating a nice wrapper, a nice API, wrapper, a nice API, wrapper, a nice API, writing or writing or writing or working on prompts working on prompts . All of this doesn't really . All of this doesn't really . All of this doesn't really put the agent put the agent put the agent first. It's first. It's first. It's more like just more like just more like just optimizing the model.
-
optimizing the model. Agent-based software is Agent-based software is inherently an inherently an inherently an architectural architectural architectural problem. And I think problem. And I think problem. And I think this structure, this this structure, this this structure, this architecture, architecture, architecture, boils down to one boils down to one boils down to one question: who pays question: who pays question: who pays for an agent's mistake? And in for an agent's mistake? And in for an agent's mistake? And in my opinion, there are my opinion, there are my opinion, there are three possible three possible three possible answers to this. So, answers to this. So, answers to this. So, first you have first you have first you have auto mode, where you auto mode, where you auto mode, where you optimize optimize optimize autonomy: you write autonomy: you write autonomy: you write a request and let a request and let a request and let the agent do whatever the agent do whatever the agent do whatever it wants. And when it wants. And when it wants. And when he makes a mistake, and he makes a mistake, and he makes a mistake, and he will, you he will, you he will, you won't even won't even won't even guess guess guess because he is very confident because he is very confident because he is very confident in his actions, and in his actions, and in his actions, and then you pay the price then you pay the price then you pay the price to your customers or to your customers or to your customers or users. Next is the users. Next is the user priority. Here you user priority. Here you optimize optimize optimize the interface, give the interface, give the interface, give the user full the user full the user full control, but build control, but build control, but build it on the basis of an agent it on the basis of an agent it on the basis of an agent who knows little, who knows little, who knows little, so it so it so it turns into a turns into a turns into a regular chatbot. You regular chatbot. You regular chatbot. You limit both limit both the user's understanding of the product, because he the user's understanding of the product, because he cannot fully cannot fully cannot fully form requests, and form requests, and form requests, and the capabilities of the the capabilities of the the capabilities of the agent itself within the agent itself within the agent itself within the product. And finally, product. And finally, product. And finally, the category that I the category that I the category that I think wins is the think wins is the customer service agent experience.
-
Here, the Here, the agent's capabilities are structured agent's capabilities are structured agent's capabilities are structured to provide to provide to provide the model with rich and the model with rich and the model with rich and clear functionality, clear functionality, clear functionality, thanks to which it thanks to which it thanks to which it performs what performs what performs what the user needs the user needs , helping him, , helping him, , helping him, and a person can and a person can and a person can check the check the check the agent's work. This approach agent's work. This approach agent's work. This approach transforms the transforms the transforms the agent's experience from a question of agent's experience from a question of agent's experience from a question of model to a question of model to a question of model to a question of context, context, context, opportunity, and opportunity, and opportunity, and overall overall overall self-improvement. self-improvement. Here's how I separated these Here's how I separated these categories and arrived at categories and arrived at categories and arrived at this final this final this final point. At Reducto, we point. At Reducto, we point. At Reducto, we create create create tools that tools that tools that automate automate automate end-to-end document workflows end-to-end document workflows using pipelines. using pipelines. Imagine that we Imagine that we Imagine that we first analyze first analyze first analyze the document, then the document, then the document, then classify it, classify it, classify it, perhaps divide it perhaps divide it perhaps divide it into sections, and only then into sections, and only then into sections, and only then extract the data. This extract the data. This extract the data. This applies to applies to applies to tasks like tasks like tasks like invoicing invoicing , , , contract management, and contract management, and contract management, and anything that requires anything that requires anything that requires human intervention. One thing human intervention. One thing human intervention. One thing I noticed I noticed I noticed early on: our early on: our early on: our users, users, users, especially the especially the especially the sales team at Reducto, sales team at Reducto, sales team at Reducto, were spending all their time on were spending all their time on were spending all their time on setup, not setup, not setup, not results. This is a broader results. This is a broader results. This is a broader problem, as problem, as problem, as legacy interfaces legacy interfaces legacy interfaces were designed were designed were designed specifically to build specifically to build specifically to build such pipelines.
-
such pipelines. such pipelines. Meanwhile, in the world of Meanwhile, in the world of Meanwhile, in the world of agent work, we are agent work, we are agent work, we are increasingly increasingly increasingly focused on the focused on the focused on the end result end result end result and how to and how to and how to get get get the user to it as quickly as possible. the user to it as quickly as possible. So the obvious step So the obvious step for us was the MCP server for us was the MCP server . Let the agent . Let the agent . Let the agent do the hard do the hard do the hard work, leaving work, leaving work, leaving the user only the user only the user only to think through to think through to think through the task and the task and the task and write the request. To write the request. To write the request. To confirm the confirm the confirm the correct correct correct direction, I direction, I direction, I quickly created an quickly created an quickly created an MVP with cloud code, where MVP with cloud code, where MVP with cloud code, where each API endpoint of each API endpoint of each API endpoint of our application was our application was our application was represented by the represented by the represented by the MCP tool in MCP tool in MCP tool in one huge one huge one huge file. I gave him a file. I gave him a file. I gave him a very clear and concise very clear and concise very clear and concise request. Added this to Slack, request. Added this to Slack, request. Added this to Slack, and honestly, it and honestly, it and honestly, it turned out pretty turned out pretty turned out pretty good. To some good. To some good. To some extent, it should have extent, it should have extent, it should have worked in those worked in those worked in those conditions. I clearly conditions. I clearly conditions. I clearly understood what I was understood what I was understood what I was doing. I was creating a doing. I was creating a doing. I was creating a very carefully very carefully very carefully curated demo. And that's why curated demo. And that's why curated demo. And that's why this particular this particular this particular demo showed me absolutely demo showed me absolutely demo showed me absolutely nothing. The nothing. The nothing. The real real real test test test began when I began when I began when I gave access to MCP to gave access to MCP to gave access to MCP to everyone on the team and it everyone on the team and it everyone on the team and it broke immediately. Here's broke immediately. Here's broke immediately. Here's how it happened. Someone would how it happened. Someone would how it happened. Someone would give him an incomplete give him an incomplete give him an incomplete description from a conversation with description from a conversation with description from a conversation with a client or a page a client or a page a client or a page from Notion, and the agent would just from Notion, and the agent would just from Notion, and the agent would just get to work. He get to work. He get to work. He built the entire built the entire built the entire workflow from workflow from workflow from start to finish. But start to finish. But start to finish. But he very confidently he very confidently he very confidently assured the user assured the user that he had done everything that he had done everything that he had done everything correctly, although neither correctly, although neither correctly, although neither the user nor the the user nor the the user nor the agent himself could agent himself could agent himself could explain what exactly explain what exactly explain what exactly this process did. And this this process did. And this this process did. And this became a harsh lesson in became a harsh lesson in became a harsh lesson in using using using automatic mode automatic mode automatic mode in a productive in a productive in a productive environment. The agent environment. The agent environment. The agent never admitted never admitted never admitted that he was unsure of the that he was unsure of the that he was unsure of the outcome. He outcome. He outcome. He used all the used all the used all the tools available tools available , and the person had no , and the person had no , and the person had no way to way to way to verify or verify or verify or comprehend what was comprehend what was comprehend what was happening. We
-
happening. We happening. We created a system that created a system that created a system that could communicate with could communicate with could communicate with our product, but we our product, but we our product, but we couldn't get it to couldn't get it to couldn't get it to work effectively. You might work effectively. You might work effectively. You might think think that the solution is that the solution is that the solution is more detailed more detailed more detailed system prompting or system prompting or system prompting or better better better model management, but it's model management, but it's model management, but it's actually a matter of actually a matter of actually a matter of architecture. I deleted the entire architecture. I deleted the entire architecture. I deleted the entire file we file we file we used to used to used to interact with MCP in interact with MCP in interact with MCP in one one one commit. And replaced commit. And replaced commit. And replaced it with no more it with no more tools. There were tools. There were even fewer of them, even fewer of them, even fewer of them, but they carried the but they carried the but they carried the structure of real structure of real structure of real work. For example, work. For example, work. For example, instead of instead of instead of constantly calling the constantly calling the constantly calling the creation of a creation of a creation of a workflow step and workflow step and workflow step and hoping that the agent hoping that the agent hoping that the agent would build would build would build the pipeline correctly, I built the pipeline correctly, I built the pipeline correctly, I built tools that tools that tools that understood how understood how understood how our backend our backend our backend or our product worked or our product worked . For example, a common . For example, a common . For example, a common use case for use case for use case for Reducto is to route Reducto is to route Reducto is to route our our document classification endpoint to a data extraction node to a data extraction node to a data extraction node . And this essentially . And this essentially . And this essentially extracts information extracts information extracts information for further for further for further processing. Instead of processing. Instead of letting letting letting the model guess or the model guess or the model guess or figure it out through a figure it out through a figure it out through a prompt, I turned prompt, I turned prompt, I turned it into a single it into a single it into a single " " classification- classification- extraction" tool. After extraction" tool. After extraction" tool. After each step, the agent each step, the agent each step, the agent receives contextual receives contextual receives contextual guidance on what to do guidance on what to do guidance on what to do next, next, next, with some with some with some tools tools tools automatically automatically automatically linking linking linking together, like this one. Instead of the together, like this one. Instead of the together, like this one. Instead of the agent agent agent guessing or guessing or guessing or thinking about what is thinking about what is thinking about what is correct, they are correct, they are correct, they are simply told the simply told the simply told the correct option.
-
correct option. correct option. Now the agent cannot Now the agent cannot Now the agent cannot improvise where improvise where improvise where we already know the truth. we already know the truth. we already know the truth. And since agents are And since agents are And since agents are stateless, we stateless, we stateless, we even added a even added a snapshot tool. This is a single snapshot tool. This is a single call that returns the call that returns the call that returns the current state of our current state of our current state of our pipeline: from pipeline: from pipeline: from the context of the entire the context of the entire the context of the entire process to any errors process to any errors process to any errors that occurred, with very that occurred, with very that occurred, with very specific codes. specific codes. This way, they This way, they can refer to can refer to can refer to real data, instead of real data, instead of real data, instead of guessing guessing guessing variable names, variable names, variable names, hallucinating, etc. hallucinating, etc. hallucinating, etc. By moving the context By moving the context By moving the context from the prompt to the from the prompt to the from the prompt to the structure, the agent structure, the agent structure, the agent stopped guessing and stopped guessing and stopped guessing and started started started using using using facts. And because facts. And because facts. And because the tools the tools the tools allow pipelines allow pipelines allow pipelines to take only the to take only the to take only the shape that we, as shape that we, as shape that we, as product engineers, product engineers, product engineers, wanted, the person can wanted, the person can wanted, the person can clearly track what clearly track what clearly track what happened, and we can happened, and we can happened, and we can explain the results of the explain the results of the explain the results of the agent's work to agent's work to agent's work to subsequent subsequent subsequent users of this users of this users of this MCP tool. So, MCP tool. So, MCP tool. So, to avoid to avoid to avoid automatic mode automatic mode , you don't give the agent an API , you don't give the agent an API , you give it , you give it , you give it tools that tools that tools that contain a common contain a common contain a common experience, such as the experience, such as the experience, such as the right templates, right templates, right templates, state, and validators, state, and validators, state, and validators, so that the right path so that the right path so that the right path also becomes the also becomes the also becomes the easiest path easiest path easiest path that the MCP can choose.
-
that the MCP can choose. that the MCP can choose. So, now we can So, now we can So, now we can build quality build quality build quality conveyors. We're conveyors. We're conveyors. We're done, aren't we? done, aren't we? done, aren't we? Not quite. When mentioning Not quite. When mentioning Not quite. When mentioning automatic mode, the automatic mode, the automatic mode, the agent was definitely agent was definitely agent was definitely wrong, and no one wrong, and no one wrong, and no one could have noticed. could have noticed. could have noticed. What we just What we just What we just talked about fixed talked about fixed talked about fixed this for the development phase this for the development phase . The tools made . The tools made . The tools made the structure quite the structure quite the structure quite understandable, and a person understandable, and a person understandable, and a person could follow the could follow the could follow the agent's actions. But agent's actions. But agent's actions. But now the problem now the problem now the problem appeared during appeared during appeared during execution. When execution. When execution. When the result an the result an agent is trying to produce turns out agent is trying to produce turns out to be wrong, who can to be wrong, who can to be wrong, who can tell? tell? tell? For example, if we For example, if we For example, if we had a bad schema or had a bad schema or had a bad schema or the user didn't the user didn't the user didn't specify the specify the usage scenario exactly, how do we usage scenario exactly, how do we know that the agent know that the agent know that the agent did something wrong? And did something wrong? And did something wrong? And with agents, it with agents, it with agents, it happens quietly. happens quietly. happens quietly. So instead of a So instead of a So instead of a segmentation fault or segmentation fault or segmentation fault or some real some real some real error, the pipeline error, the pipeline error, the pipeline is processing 100 is processing 100 is processing 100 documents, and documents, and documents, and maybe it's skipping maybe it's skipping maybe it's skipping the scan on the scan on the scan on page 30, or page 30, or page 30, or the user hasn't specified the user hasn't specified the user hasn't specified what types of documents are what types of documents are what types of documents are coming into the pipeline. coming into the pipeline. And, in essence, both of these And, in essence, both of these things are the same things are the same things are the same failure. The failure. The failure. The agent himself is very confidently agent himself is very confidently agent himself is very confidently mistaken, and his mistaken, and his mistaken, and his uncertainty uncertainty uncertainty remains remains remains completely imperceptible completely imperceptible . So, all we . So, all we . So, all we did to did to did to fix this fix this fix this came down to one came down to one came down to one change: we made change: we made change: we made the uncertainty the uncertainty the uncertainty quite visible. In quite visible. In quite visible. In our system or with our system or with our system or with Reducto, failures carry Reducto, failures carry Reducto, failures carry information. So, with information. So, with information. So, with Extract, we output Extract, we output confidence scores, confidence scores, bounding boxes, bounding boxes, bounding boxes, all this information about all this information about all this information about what went wrong, and we what went wrong, and we what went wrong, and we were able to create were able to create were able to create tools that tools that tools that feed that feed that feed that information back into the model, information back into the model, information back into the model, which helped us which helped us which helped us optimize the
-
optimize the optimize the extraction scheme over time and extraction scheme over time and extraction scheme over time and get more accurate get more accurate document results. But this is document results. But this is just the uncertainty of the just the uncertainty of the just the uncertainty of the outcome. The agent outcome. The agent outcome. The agent must also must also must also reveal its own reveal its own reveal its own uncertainty. When uncertainty. When uncertainty. When he encounters he encounters he encounters something ambiguous something ambiguous , such as two , such as two , such as two ways of reading a ways of reading a ways of reading a field or a type of field or a type of field or a type of document he has document he has document he has n't seen before, he n't seen before, he n't seen before, he shouldn't guess shouldn't guess shouldn't guess and shouldn't just and shouldn't just and shouldn't just stop there. stop there. stop there. Instead, it should Instead, it should Instead, it should ask, "I can ask, "I can ask, "I can read this two read this two read this two ways," and tell ways," and tell ways," and tell the user, "What did you the user, "What did you the user, "What did you mean?" So, mean?" So, mean?" So, essentially, during our essentially, during our essentially, during our evaluations, we started to evaluations, we started to evaluations, we started to de-rate the de-rate the de-rate the agent for any agent for any agent for any incorrect assumptions, and incorrect assumptions, and incorrect assumptions, and we gave our we gave our we gave our toolkit toolkit toolkit cues cues cues to use the to use the user question tools user question tools offered by Cloud Code and offered by Cloud Code and offered by Cloud Code and other advanced other advanced other advanced solutions instead of solutions instead of guessing. So guessing. So guessing. So we rewarded we rewarded we rewarded questions, not questions, not questions, not guesses. And guesses. And guesses. And the judgment does not belong the judgment does not belong the judgment does not belong only to the agent. This entire only to the agent. This entire only to the agent. This entire pipeline is still pipeline is still pipeline is still controlled controlled controlled by the user, and by the user, and by the user, and confidence in the confidence in the confidence in the agent's reasoning agent's reasoning agent's reasoning was then output as was then output as was then output as prompts in the MCP. So prompts in the MCP. So prompts in the MCP. So now, every time now, every time now, every time you create you create you create something new, it something new, it something new, it explains to explains to explains to the user what it did and the user what it did and the user what it did and why it did it so why it did it so why it did it so the user can the user can the user can review it, review it, review it, working solely for the working solely for the working solely for the user's benefit.
-
user's benefit. The agent is essentially The agent is essentially satisfying what you satisfying what you satisfying what you wrote, not the wrote, not the wrote, not the outcome you outcome you outcome you actually wanted. So what actually wanted. So what actually wanted. So what if we made if we made if we made the uncertainty a little the uncertainty a little the uncertainty a little clearer and clearer and clearer and left the final decision left the final decision left the final decision where a human could be where a human could be where a human could be held held held accountable accountable accountable ? Then one ? Then one ? Then one clarifying clarifying clarifying question does question does question does the work of three at once: it the work of three at once: it the work of three at once: it provides the model with provides the model with provides the model with context, opportunities, context, opportunities, context, opportunities, and also involves and also involves and also involves a person to a person to a person to form a judgment form a judgment form a judgment and check the facts. and check the facts. Going back to all Going back to all these failure modes these failure modes these failure modes in general, first in general, first in general, first we had auto mode, we had auto mode, we had auto mode, which no one could which no one could which no one could control. We were control. We were control. We were getting getting getting incorrect incorrect incorrect answers that just answers that just answers that just appeared without any appeared without any appeared without any explanation. And then an explanation. And then an explanation. And then an agent who passed agent who passed agent who passed the test by hacking the the test by hacking the the test by hacking the reward system reward system reward system or concealing or concealing or concealing their actions. At the heart of all of their actions. At the heart of all of their actions. At the heart of all of this is this is this is one common trait, where the one common trait, where the one common trait, where the agent had more agent had more agent had more autonomy and access autonomy and access autonomy and access than a human could than a human could than a human could verify. And this verify. And this verify. And this intelligence lives in the intelligence lives in the intelligence lives in the design itself as a whole.
-
design itself as a whole. Building a pipeline, Building a pipeline, generating a schema, and generating a schema, and generating a schema, and engaging the engaging the engaging the right right right context—this is where context—this is where context—this is where the agent is most the agent is most the agent is most powerful, and where powerful, and where powerful, and where a mistake costs a mistake costs a mistake costs the least, because a human is the least, because a human is the least, because a human is there. By designing there. By designing there. By designing the interface this the interface this the interface this way, we were able to way, we were able to way, we were able to increase increase MCP user satisfaction MCP user satisfaction because they because they because they could actually see what was could actually see what was could actually see what was happening. And happening. And happening. And by using Reducto by using Reducto by using Reducto for the document layer, for the document layer, for the document layer, we can we can we can guarantee accurate guarantee accurate guarantee accurate parsing and parsing and parsing and data extraction for those parts of the data extraction for those parts of the data extraction for those parts of the pipeline where we can't pipeline where we can't pipeline where we can't trust trust trust agents. Where an agents. Where an agents. Where an agent operates, it gains agent operates, it gains agent operates, it gains trust by being trust by being trust by being verifiable, verifiable, verifiable, correct, and working correct, and working correct, and working for the benefit of the user. for the benefit of the user. And all our talk And all our talk about clarity in about clarity in about clarity in general is what general is what general is what brings results brings results . Agents are just . Agents are just . Agents are just intelligence in a “box,” intelligence in a “box,” intelligence in a “box,” and it’s the and it’s the and it’s the toolkit toolkit toolkit you surround them with that you surround them with that you surround them with that gives them gives them gives them the ability to achieve the the ability to achieve the results you want. So once results you want. So once we got to we got to we got to that point, we that point, we that point, we focused on the focused on the focused on the tooling where tooling where tooling where every MCP session and every MCP session and every MCP session and request request request sent from the sent from the sent from the model was model was model was logged, and we logged, and we logged, and we even asked our even asked our even asked our sales team and sales team and sales team and other people on the team other people on the team other people on the team to export their to export their to export their cloud code sessions cloud code sessions cloud code sessions to understand how everything was to understand how everything was to understand how everything was behaving. Where people behaving. Where people behaving. Where people interfered with the interfered with the interfered with the agent's work, where the agent's work, where the agent's work, where the model's confidence model's confidence model's confidence was low, and where was low, and where was low, and where she asked she asked she asked questions. And as questions. And as questions. And as we added all of this we added all of this we added all of this toolkit, it toolkit, it toolkit, it impacted the impacted the impacted the long-term long-term long-term management that MCP had management that MCP had management that MCP had overall. Thus, overall. Thus, overall. Thus, the system became better at the system became better at the system became better at building pipelines, building pipelines, building pipelines, creating more
-
creating more creating more pipelines without our pipelines without our pipelines without our manual intervention. This manual intervention. This manual intervention. This became a cycle of became a cycle of became a cycle of self-improvement self-improvement self-improvement for each for each for each organization or organization or organization or user who user who user who created these created these created these pipelines. The foundation pipelines. The foundation pipelines. The foundation generally remained the generally remained the generally remained the same solution, same solution, same solution, where we kept the model where we kept the model where we kept the model under control and under control and under control and available for available for available for inspection, and inspection, and inspection, and the information around the information around the information around it became everything else it became everything else it became everything else that actually drove that actually drove that actually drove the model. So one thing the model. So one thing the model. So one thing that told me that told me that told me we were doing it we were doing it we were doing it right with these right with these right with these improvements improvements improvements is that I is that I is that I never directly told never directly told never directly told sales sales sales to use an to use an to use an agent to build agent to build agent to build pipelines, but simply pipelines, but simply pipelines, but simply provided it in our provided it in our provided it in our Cloud Enterprise as Cloud Enterprise as Cloud Enterprise as Reducto tools. And the Reducto tools. And the Reducto tools. And the sales team and sales team and sales team and other people on the team other people on the team other people on the team started to actually started to actually started to actually integrate it into integrate it into integrate it into their workflows. their workflows. their workflows. So before calling So before calling So before calling the customer, they the customer, they the customer, they used used used CircleBack or Slack to CircleBack or Slack to CircleBack or Slack to learn about learn about learn about the customer, and then the customer, and then the customer, and then used MCP used MCP used MCP to ask to ask to ask questions about questions about questions about the customer and build a the customer and build a the customer and build a pipeline for them. pipeline for them. So, when the agent experience was built for the benefit of was built for the benefit of users, we didn't users, we didn't users, we didn't push people push people push people directly to it, but directly to it, but directly to it, but on the contrary—they themselves on the contrary—they themselves on the contrary—they themselves started started started using it using it using it as a as a as a corporate-wide corporate-wide corporate-wide resource. And that's when resource. And that's when resource. And that's when they started to they started to they started to trust trust trust the results enough to the results enough to the results enough to demonstrate it to demonstrate it to demonstrate it to valuable customers, valuable customers, valuable customers, because the demo wasn't just because the demo wasn't just because the demo wasn't just working, but the person was working, but the person was working, but the person was actually saying, "Hey actually saying, "Hey , I built this." " , I built this." " , I built this." " Come and see."
-
Come and see." Come and see." And it really And it really And it really paid off paid off paid off during those during those during those sales calls. And I sales calls. And I sales calls. And I hope the hope the hope the reactions in Slack reactions in Slack reactions in Slack will also help will also help will also help to appreciate it a little. In general, to appreciate it a little. In general, to appreciate it a little. In general, a few months ago I would have a few months ago I would have a few months ago I would have told you that " told you that " agent agent agent first" means first" means first" means you have the best model, the you have the best model, the you have the best model, the best best best inference speed or inference speed or inference speed or intelligence of the model. And intelligence of the model. And intelligence of the model. And as models as models as models get better from get better from get better from closed to closed to closed to open, from open, from open, from expensive to cheap, I expensive to cheap, I expensive to cheap, I think it now think it now think it now means almost the means almost the means almost the opposite of what it opposite of what it opposite of what it seems. To some seems. To some seems. To some extent, the agent does not go extent, the agent does not go extent, the agent does not go first. And building an first. And building an first. And building an agent's experience agent's experience agent's experience means taking into account means taking into account means taking into account that the agent is in that the agent is in that the agent is in a loop with the people who are a loop with the people who are a loop with the people who are responsible for responsible for responsible for further work. So it's further work. So it's further work. So it's not just about what the not just about what the agent connects to? agent connects to? Today, everyone's agents Today, everyone's agents connect to connect to everything, and Claude can everything, and Claude can everything, and Claude can instantly create a instantly create a instantly create a new connector for you. new connector for you. But beyond that, when But beyond that, when your agent your agent your agent comes across as very comes across as very comes across as very confident but confident but confident but wrong, who wrong, who wrong, who notices and how quickly? notices and how quickly? If you take this into account, you If you take this into account, you can create can create can create something that people something that people something that people will actually trust in will actually trust in will actually trust in real work. Well, real work. Well, real work. Well, actually, that's all. Thank actually, that's all. Thank actually, that's all. Thank you, friends. We are also at you, friends. We are also at you, friends. We are also at Reducto on stand P8 and we are Reducto on stand P8 and we are Reducto on stand P8 and we are open to hiring for open to hiring for open to hiring for many positions. So many positions. So many positions. So please visit please visit please visit reductoai.notcurious. Thank you.
No summary available yet.
View original episode ↗