Reliable, Smart Workflows for Finance, Wealth and Insurance — Karthic Rao, Perpetual

Hi, I'm Karthik, Hi, I'm Karthik, CTO at CTO at CTO at Perpetual. In this Perpetual. In this Perpetual. In this presentation, I presentation, I presentation, I will talk about will talk about will talk about creating reliable creating reliable creating reliable and intelligent and intelligent and intelligent workflows for workflows for workflows for finance, finance, finance, wealth management, and wealth management, and wealth management, and insurance. In insurance. In insurance. In regulated areas, regulated areas, regulated areas, workflows must workflows must workflows must be be be robust enough to robust enough to robust enough to meet meet meet regulatory regulatory regulatory standards, standards, standards, yet intelligent enough yet intelligent enough to operate in to operate in real-world environments, such as real-world environments, such as finance and finance and finance and asset management. asset management. asset management. This creates both a This creates both a This creates both a technical problem and a technical problem and a regulatory compliance issue. regulatory compliance issue. Reliability and Reliability and Reliability and intelligence are not intelligence are not intelligence are not opposites. opposites. opposites. Only you decide Only you decide Only you decide where to draw the line between where to draw the line between where to draw the line between them, and that's what this them, and that's what this them, and that's what this conversation is about— conversation is about— conversation is about— determining where determining where determining where that line lies. For that line lies. For that line lies. For us, this is not just a us, this is not just a us, this is not just a thought experiment, but a thought experiment, but a thought experiment, but a question with question with question with serious consequences. serious consequences. serious consequences. Perpetual is an Perpetual is an Perpetual is an asset management company asset management company asset management company growing growing growing through acquisitions. We through acquisitions. We through acquisitions. We buy existing buy existing buy existing traditional traditional traditional asset management firms asset management firms asset management firms and maximize their and maximize their and maximize their performance and performance and performance and profits with profits with profits with AI. AI. AI. Asset management Asset management is a regulated is a regulated is a regulated industry, and this is where we industry, and this is where we industry, and this is where we put AI to put AI to put AI to work. Everything in work. Everything in work. Everything in this conversation this conversation this conversation is based on is based on is based on designing with designing with designing with these these these constraints in mind. constraints in mind. constraints in mind. Workflows in finance and Workflows in finance and Workflows in finance and asset management asset management asset management have four have four have four characteristics, and each characteristics, and each characteristics, and each of them drives of them drives of them drives the architecture in one the architecture in one the architecture in one direction. First, direction. First, direction. First, they are complex and they are complex and they are complex and require require real-time execution strategies as the process unfolds. You can't prescribe every step in advance.

prescribe every step in advance. Second, the data is Second, the data is Second, the data is confidential, so confidential, so confidential, so you need a way to you need a way to you need a way to fully migrate fully migrate fully migrate these processes to these processes to these processes to privately hosted privately hosted privately hosted models. Third, in a models. Third, in a models. Third, in a highly regulated highly regulated highly regulated industry, a industry, a industry, a policy violation is not policy violation is not policy violation is not just a just a just a bug report. This has bug report. This has bug report. This has serious consequences. serious consequences. serious consequences. Therefore, fourthly, Therefore, fourthly, Therefore, fourthly, execution must be execution must be execution must be extremely reliable extremely reliable extremely reliable and deterministic. and deterministic. and deterministic. Add intelligence Add intelligence Add intelligence only where execution only where execution only where execution judgment is truly needed judgment is truly needed judgment is truly needed , and , and , and minimize minimize minimize reliance on LLM otherwise reliance on LLM otherwise reliance on LLM otherwise . AI . AI . AI is spreading in is spreading in is spreading in regulated industries regulated industries , but there is still no , but there is still no , but there is still no consensus on an consensus on an consensus on an architecture that architecture that architecture that provides high provides high provides high reliability with reliability with reliability with minimal minimal minimal use of use of use of LLM intelligence. LLM intelligence. LLM intelligence. Meanwhile, developers from Meanwhile, developers from Meanwhile, developers from large labs large labs large labs mostly mostly mostly only hear about how to only hear about how to only hear about how to burn more burn more burn more tokens. But in a tokens. But in a tokens. But in a regulated industry, regulated industry, regulated industry, token maximization token maximization is the wrong goal. These is the wrong goal. These is the wrong goal. These tasks are close to the tasks are close to the tasks are close to the main ones. You can main ones. You can plan them out in advance and plan them out in advance and build a build a build a massive machine of massive machine of massive machine of states out of them that will states out of them that will states out of them that will only require a only require a only require a pinch of intelligence to pinch of intelligence to pinch of intelligence to navigate. Most navigate. Most navigate. Most steps simply steps simply steps simply follow the follow the follow the next node. next node. next node. Only at a few decision points does Only at a few decision points does the model choose a the model choose a course of course of course of events. So a smart events. So a smart events. So a smart approach to development approach to development approach to development is to is to is to use use use intelligence minimally intelligence minimally intelligence minimally and purposefully.

and purposefully. Let's back Let's back this up with an example of a this up with an example of a this up with an example of a real real real workflow in workflow in workflow in asset management. A financial asset management. A financial asset management. A financial advisor advisor advisor turns to his turns to his turns to his assistant and says one assistant and says one assistant and says one sentence: "Fill out sentence: "Fill out sentence: "Fill out IRS Form W-9 for Mr. IRS Form W-9 for Mr. IRS Form W-9 for Mr. Harper." For a person, Harper." For a person, Harper." For a person, this is a simple instruction. this is a simple instruction. this is a simple instruction. See what it See what it See what it actually triggers actually triggers actually triggers when a person starts to when a person starts to when a person starts to perform it. This perform it. This perform it. This task has task has task has complexities that complexities that complexities that go far go far go far beyond what beyond what beyond what appears at first appears at first appears at first glance. First, glance. First, glance. First, the assistant finds a the assistant finds a the assistant finds a W-9 form in W-9 form in W-9 form in the firm's records or on the IRS website. the firm's records or on the IRS website. the firm's records or on the IRS website. Then they look for Mr. Then they look for Mr. Then they look for Mr. Harper in the Harper in the Harper in the accounting system and find accounting system and find accounting system and find three three three Harper households. So they Harper households. So they Harper households. So they call the advisor: call the advisor: call the advisor: "Which Harper are "Which Harper are "Which Harper are you referring to?" you referring to?" you referring to?" Only after resolving Only after resolving Only after resolving this ambiguity do this ambiguity do this ambiguity do they proceed to they proceed to they proceed to fill out the form. fill out the form. fill out the form. Search, Search, Search, ambiguity, and ambiguity, and ambiguity, and a person in the process—all a person in the process—all a person in the process—all in the first minute of a in the first minute of a in the first minute of a simple task. simple task. Filling out a form Filling out a form sounds like regular sounds like regular sounds like regular data entry. This is not data entry. This is not data entry. This is not true. Each form is a true. Each form is a true. Each form is a small program. small program. small program. Form W-9 first Form W-9 first Form W-9 first identifies who the identifies who the identifies who the taxpayer is: taxpayer is: taxpayer is: an individual or the owner an individual or the owner an individual or the owner if it is a disregarded if it is a disregarded if it is a disregarded entity.

entity. entity. The foreign person The foreign person The foreign person stops here stops here stops here because they need a because they need a because they need a W-8 form. Then W-8 form. Then W-8 form. Then the method the method the method of taxation is determined. Exactly of taxation is determined. Exactly of taxation is determined. Exactly one mark in line one mark in line one mark in line 3a. And LLC adds its own 3a. And LLC adds its own 3a. And LLC adds its own letter. This decides letter. This decides letter. This decides which name goes on the first which name goes on the first which name goes on the first line and which on the second line and which on the second . Then whose number is it? . Then whose number is it? . Then whose number is it? Social Social Social Security Number (SSN) or Security Number (SSN) or Employer Identification Number (EIN) ? Never both. ? Never both. ? Never both. It is taken from a It is taken from a It is taken from a verified verified verified document, never document, never document, never from memory. Tax exemption codes from memory. Tax exemption codes only apply only apply to certain to certain to certain organizations and never organizations and never organizations and never to S corporations. And to S corporations. And to S corporations. And finally, who signs a finally, who signs a finally, who signs a document under penalty of document under penalty of perjury? Each perjury? Each answer changes the answer changes the answer changes the next question, and next question, and next question, and the people involved see the people involved see the people involved see only three types of only three types of only three types of obstacles. A question, a obstacles. A question, a obstacles. A question, a document to document to document to collect, or a collect, or a collect, or a signature. Now they signature. Now they signature. Now they start start start filling out the form and filling out the form and filling out the form and realize that the realize that the realize that the accounting system is accounting system is accounting system is missing three missing three missing three required values. required values. required values. They They They are checking the are checking the are checking the documents in the archive and the documents in the archive and the documents in the archive and the accounting system again—the data is still accounting system again—the data is still accounting system again—the data is still missing. Therefore, missing. Therefore, missing. Therefore, they begin they begin they begin correspondence with correspondence with correspondence with the client by asking for the client by asking for the client by asking for these three values. these three values. these three values. Now the task Now the task Now the task depends on a person depends on a person depends on a person outside the firm and their outside the firm and their outside the firm and their schedule. And now the schedule. And now the schedule. And now the workflow workflow workflow stops. Request from stops. Request from stops. Request from the client, and all three the client, and all three the client, and all three values ​​are still values ​​are still values ​​are still missing. If the client missing. If the client missing. If the client does not respond does not respond does not respond within two days, within two days, within two days, the assistant makes a the assistant makes a the assistant makes a follow-up request and follow-up request and follow-up request and informs the informs the informs the advisor at the daily advisor at the daily advisor at the daily meeting. If the client meeting. If the client meeting. If the client responds by providing responds by providing responds by providing only two of the only two of the only two of the three values, they three values, they three values, they ask again for the ask again for the ask again for the missing one. When all missing one. When all missing one. When all three values three values three values are received, they are received, they are received, they fill out a form and fill out a form and fill out a form and enter the new enter the new enter the new information back into the

information back into the information back into the accounting system, accounting system, accounting system, including how they including how they including how they obtained this data. obtained this data. obtained this data. In this field, how you In this field, how you In this field, how you learned the value learned the value learned the value is as important is as important is as important as the number itself. as the number itself. as the number itself. Once the form Once the form Once the form is completed, it is completed, it is completed, it is sent for is sent for is sent for electronic signature. electronic signature. electronic signature. If the client does not If the client does not If the client does not sign within a sign within a sign within a day or two, day or two, day or two, the assistant reminds the assistant reminds the assistant reminds him again. Once him again. Once him again. Once signed, they signed, they signed, they file the document file the document file the document for the client, for the client, for the client, send send send confirmation, and confirmation, and confirmation, and update the information update the information update the information for the advisor at the for the advisor at the for the advisor at the next daily next daily next daily meeting. One phrase meeting. One phrase meeting. One phrase from a consultant from a consultant from a consultant turns into a turns into a turns into a process that lasts for process that lasts for process that lasts for days, spans multiple days, spans multiple days, spans multiple systems, waits for two systems, waits for two systems, waits for two people, and repeatedly people, and repeatedly people, and repeatedly returns to the returns to the returns to the beginning. Look beginning. Look beginning. Look what is hidden in this what is hidden in this what is hidden in this one phrase. one phrase. one phrase. An uncertainty that An uncertainty that An uncertainty that needs to be eliminated. needs to be eliminated. needs to be eliminated. Data collection. Agreements Data collection. Agreements Data collection. Agreements measured in days, measured in days, measured in days, revision cycles, revision cycles, revision cycles, and fixing the and fixing the and fixing the evidence base evidence base evidence base at the end.

at the end. Workflow automation tools Workflow automation tools exist, but they exist, but they exist, but they rarely handle the rarely handle the rarely handle the complexity and complexity and complexity and ambiguity of the ambiguity of the ambiguity of the real world. A real world. A real world. A fixed flowchart fixed flowchart fixed flowchart breaks down at the first breaks down at the first breaks down at the first obstacle, and obstacle, and obstacle, and the lack of a reliable the lack of a reliable the lack of a reliable accounting system with accounting system with accounting system with integration and an integration and an integration and an agent capable agent capable agent capable of improvisation is of improvisation is of improvisation is exactly what the exactly what the exactly what the compliance compliance team cannot approve of. One approach team cannot approve of. One approach team cannot approve of. One approach is shattered by is shattered by is shattered by uncertainty, the uncertainty, the uncertainty, the other cannot other cannot other cannot be approved. None of be approved. None of be approved. None of them solves what is them solves what is them solves what is hidden in this hidden in this hidden in this phrase. So, what do we phrase. So, what do we phrase. So, what do we do about this? The fashionable do about this? The fashionable do about this? The fashionable approach is more approach is more approach is more agency and better agency and better agency and better models. Assign models. Assign models. Assign all tasks to the agent and all tasks to the agent and all tasks to the agent and wait for the next wait for the next wait for the next model to make it model to make it model to make it reliable. The reliable. The reliable. The knee-jerk knee-jerk knee-jerk response, especially in response, especially in response, especially in regulated regulated regulated companies, is the companies, is the opposite. Less opposite. Less opposite. Less agency, more agency, more agency, more rules. rules. rules. Block everything and Block everything and Block everything and strictly prescribe strictly prescribe strictly prescribe every path. Both every path. Both every path. Both options are false. In options are false. In options are false. In regulated industries, regulated industries, regulated industries, reliability depends reliability depends reliability depends on where you on where you on where you draw the line between draw the line between draw the line between them. Between the parts them. Between the parts them. Between the parts that should be that should be that should be deterministic and deterministic and deterministic and those where those where those where judgment is truly required.

judgment is truly required. judgment is truly required. That's the That's the That's the principle. Determinism principle. Determinism principle. Determinism governs the flow of governs the flow of governs the flow of control, evidence, control, evidence, control, evidence, and side and side and side effects. What effects. What effects. What happens next, what happens next, what happens next, what we know, how we we know, how we we know, how we know it, and anything that know it, and anything that know it, and anything that changes the world, like a changes the world, like a changes the world, like a recording made, an recording made, an recording made, an email sent, email sent, email sent, or a form filled out. or a form filled out. Intelligence provides Intelligence provides interpretation and interpretation and interpretation and evaluation. Reading evaluation. Reading evaluation. Reading ambiguous ambiguous ambiguous instructions, choosing instructions, choosing instructions, choosing an assistant, understanding an assistant, understanding an assistant, understanding the client's response, the the client's response, the the client's response, the model's response to model's response to model's response to the question, and a the question, and a the question, and a deterministic deterministic deterministic system deciding what system deciding what system deciding what to do with that to do with that to do with that response—and this response—and this response—and this interaction is interaction is interaction is recursive. recursive. recursive. A judgment can A judgment can A judgment can open a new, open a new, open a new, smaller plan that is smaller plan that is smaller plan that is recompiled, recompiled, recompiled, constrained, and becomes constrained, and becomes constrained, and becomes deterministic. You deterministic. You deterministic. You will see this in a moment with will see this in a moment with will see this in a moment with the example of the example of the example of further actions. The boundary further actions. The boundary further actions. The boundary between them is between them is between them is design. Here's how it design. Here's how it design. Here's how it works in the workflow engine works in the workflow engine we created. The request goes to The request goes to the scheduler. This is the the scheduler. This is the the scheduler. This is the model's challenge, and its model's challenge, and its model's challenge, and its only task is only task is to broadcast. It to broadcast. It to broadcast. It produces a typed produces a typed produces a typed semantic graph of semantic graph of semantic graph of intents, entities, intents, entities, intents, entities, facts, artifacts, facts, artifacts, facts, artifacts, events, and constraints, events, and constraints, events, and constraints, each each each based on the based on the based on the query words with all query words with all ambiguities found.

ambiguities found. Each node Each node Each node links back to the links back to the links back to the words from which it is words from which it is words from which it is derived. This graph derived. This graph derived. This graph goes to goes to goes to the compiler, which the compiler, which the compiler, which never calls never calls never calls the model. In 11 the model. In 11 the model. In 11 deterministic deterministic deterministic stages, it stages, it stages, it converts the graph into an converts the graph into an converts the graph into an execution plan or execution plan or execution plan or blocks in case of an blocks in case of an blocks in case of an error. There are only three error. There are only three error. There are only three verdicts: valid, verdicts: valid, verdicts: valid, needs clarification, needs clarification, needs clarification, or rejected. The choice or rejected. The choice or rejected. The choice of assistant of assistant of assistant is returned as is returned as is returned as a question, not as an a question, not as an a question, not as an assumption. The plan is assumption. The plan is assumption. The plan is compiled, compiled, compiled, immutable, and immutable, and immutable, and secured by its secured by its secured by its hash. The model hash. The model hash. The model performs the translation. performs the translation. performs the translation. The plan is being compiled. The plan is being compiled. The plan is being compiled. The compiler checks The compiler checks The compiler checks the plan for the plan for the plan for compliance with compliance with compliance with the grammar and the grammar and the grammar and dictionary of dictionary of dictionary of actions allowed for the firm. 42 types of actions allowed for the firm. 42 types of actions allowed for the firm. 42 types of action contracts: action contracts: action contracts: forms, documents, forms, documents, forms, documents, email, e- email, e- signature, calendar and signature, calendar and signature, calendar and human stages. Every human stages. Every human stages. Every action requested by action requested by action requested by the scheduler must the scheduler must the scheduler must comply with comply with comply with the contract. Here, the contract. Here, the contract. Here, filling out the form filling out the form filling out the form finds its finds its finds its contract and is contract and is contract and is fixed. If fixed. If fixed. If the planner requests an the planner requests an the planner requests an action that is not in action that is not in action that is not in the grammar or the grammar or the grammar or configures it configures it configures it incorrectly, incorrectly, incorrectly, the compiler rejects the compiler rejects the compiler rejects it. There are about 40 it. There are about 40 it. There are about 40 ways to ways to ways to fail safely: from a cycle in a fail safely: from a cycle in a fail safely: from a cycle in a graph to an graph to an graph to an unsupported action. The unsupported action. The unsupported action. The same same same contract digest contract digest contract digest is embedded in the engine, is embedded in the engine, is embedded in the engine, backend, and backend, and user interface. Therefore, user interface. Therefore, all three components all three components all three components understand understand understand the meaning of the action equally. That's what a the meaning of the action equally. That's what a the meaning of the action equally. That's what a private domain gives you private domain gives you private domain gives you . The model can . The model can . The model can only ask for what only ask for what only ask for what the firm already knows how the firm already knows how the firm already knows how to do safely. Here is a

to do safely. Here is a to do safely. Here is a compiled plan compiled plan compiled plan for requesting a Form W-9. for requesting a Form W-9. for requesting a Form W-9. Send the form to the client Send the form to the client Send the form to the client , wait for it , wait for it , wait for it to return, to return, to return, check it, and check it, and check it, and save it. Before save it. Before save it. Before anything is launched, anything is launched, anything is launched, the person approves the the person approves the the person approves the plan itself. Find plan itself. Find plan itself. Find the form, prepare it the form, prepare it the form, prepare it for for for the client to fill out, the client to fill out, the client to fill out, generate a PDF, and generate a PDF, and generate a PDF, and draft the draft the draft the letter. Drafting letter. Drafting letter. Drafting is a is a is a model challenge, so it is model challenge, so it is model challenge, so it is highlighted in purple. highlighted in purple. Sending is the Sending is the action checkpoint. action checkpoint. action checkpoint. Nothing will leave the firm Nothing will leave the firm Nothing will leave the firm until the person until the person until the person approves this approves this approves this specific letter. specific letter. specific letter. The plan then awaits a The plan then awaits a The plan then awaits a response, and a response, and a response, and a judge steps in to judge steps in to judge steps in to assess it. The resulting assess it. The resulting assess it. The resulting form is checked. form is checked. form is checked. If it matches, If it matches, If it matches, it is archived it is archived it is archived through another gateway. through another gateway. through another gateway. If not, the plan initiates a If not, the plan initiates a If not, the plan initiates a re-apply. re-apply. re-apply. Now look at the Now look at the Now look at the dotted steps. dotted steps. PDF generation, PDF generation, file attachment, file attachment, file attachment, response evaluation. response evaluation. response evaluation. The planner never The planner never The planner never asked them. asked them. asked them. The grammar added them The grammar added them The grammar added them because contracts require it because contracts require it because contracts require it . In this . In this . In this story, waiting, story, waiting, story, waiting, approval, and approval, and approval, and re-requests re-requests re-requests took up most of the took up most of the took up most of the work. Here work. Here work. Here they are nodes of the plan, they are nodes of the plan, they are nodes of the plan, not prompts. The not prompts. The default response time default response time is 72 hours, and is 72 hours, and is 72 hours, and the time can be from the time can be from the time can be from one minute to 30 days.

one minute to 30 days. one minute to 30 days. When time runs out, When time runs out, the rule, not the model, makes the decision. the rule, not the model, makes the decision. One approval One approval One approval covers the plan, and each covers the plan, and each covers the plan, and each external action, like external action, like external action, like sending a letter or sending a letter or sending a letter or requesting a signature, has requesting a signature, has requesting a signature, has its own gateway. its own gateway. its own gateway. The approval The approval The approval is tied to a hash of is tied to a hash of is tied to a hash of specific content specific content specific content and is verified and is verified and is verified immediately before immediately before immediately before sending. When sending. When sending. When a client a client a client only answers part of only answers part of only answers part of a request, the a request, the a request, the response judge marks response judge marks response judge marks it as partial. This it as partial. This it as partial. This decision is preliminary. decision is preliminary. decision is preliminary. The step remains The step remains The step remains blocked until blocked until blocked until the person the person the person confirms it. confirms it. confirms it. The person confirms and the The person confirms and the The person confirms and the step is unlocked step is unlocked . The . The . The requery is then executed as a requery is then executed as a requery is then executed as a new child plan. The new child plan. The new child plan. The approved approved approved parent element parent element parent element never changes, never changes, never changes, and there are a maximum of five and there are a maximum of five and there are a maximum of five rounds. When the plan rounds. When the plan rounds. When the plan is approved, it is approved, it is approved, it is executed by the is executed by the is executed by the state machine. He chooses state machine. He chooses state machine. He chooses every step and records every step and records every step and records every result. When every result. When every result. When a step needs to be evaluated, a step needs to be evaluated, a step needs to be evaluated, it makes a single query it makes a single query it makes a single query against a fixed against a fixed against a fixed model schema. Compose a letter, model schema. Compose a letter, model schema. Compose a letter, evaluate the response, evaluate the response, evaluate the response, find missing find missing find missing values, extract values, extract values, extract fields, check fields, check fields, check the document. The response the document. The response the document. The response is returned is returned is returned typed, and the typed, and the typed, and the state machine state machine state machine decides what decides what decides what to do with it. Side to do with it. Side to do with it. Side effects have their own effects have their own effects have their own life cycle.

life cycle. life cycle. Prepared, Prepared, Prepared, sent, executed sent, executed sent, executed successfully, or, if successfully, or, if successfully, or, if the world does not respond the world does not respond the world does not respond clearly, the result is clearly, the result is clearly, the result is unknown and unknown and unknown and manual verification is required. There manual verification is required. There are no automatic retries. A are no automatic retries. A chat message can confirm a chat message can confirm a chat message can confirm a step or answer a step or answer a step or answer a question. It question. It question. It cannot change an already cannot change an already cannot change an already approved plan. approved plan. approved plan. The model guides, but The model guides, but The model guides, but never controls, the never controls, the never controls, the process. Now process. Now process. Now look at what the look at what the models have left to do. Outside models have left to do. Outside the planner, the planner, the planner, judgments are small and judgments are small and judgments are small and typified. Is this typified. Is this typified. Is this answer complete? Does answer complete? Does answer complete? Does this this this document comply? Which field is document comply? Which field is document comply? Which field is which? They don't need which? They don't need which? They don't need advanced intelligence. advanced intelligence. advanced intelligence. A fast small model, A fast small model, A fast small model, flash level, can flash level, can flash level, can handle them. In handle them. In handle them. In this demonstration this demonstration this demonstration cycle, out of 20 solutions, 19 cycle, out of 20 solutions, 19 cycle, out of 20 solutions, 19 are at the are at the are at the flash level and one is at the flash level and one is at the flash level and one is at the advanced level. That's why advanced level. That's why advanced level. That's why we calculated that we calculated that we calculated that about 95% of about 95% of about 95% of model solutions in the model solutions in the model solutions in the workflow do not workflow do not workflow do not require a front-end require a front-end require a front-end model. And the flash layer model. And the flash layer model. And the flash layer can run on can run on can run on hardware you hardware you hardware you control. Only control. Only control. Only truly complex truly complex truly complex cases are passed on to the cases are passed on to the cases are passed on to the advanced model. And advanced model. And advanced model. And this 5% is not a permanent this 5% is not a permanent this 5% is not a permanent tax, since tax, since tax, since each judgment each judgment each judgment is typed, recorded, is typed, recorded, is typed, recorded, and previous ones and previous ones and previous ones are confirmed or are confirmed or are confirmed or rejected by a person rejected by a person . Each advanced . Each advanced . Each advanced solution, by its solution, by its solution, by its structure, creates an structure, creates an structure, creates an evaluation distribution and evaluation distribution and evaluation distribution and real data for real data for real data for training. This data training. This data training. This data can further train the can further train the can further train the flash level. Logging flash level. Logging flash level. Logging and evaluations already exist and evaluations already exist and evaluations already exist today. The today. The today. The learning cycle is the next learning cycle is the next learning cycle is the next step. So, the share of step. So, the share of step. So, the share of advanced models advanced models advanced models decreases with each

decreases with each decreases with each release until even release until even release until even complex cases complex cases complex cases start to be handled start to be handled start to be handled locally. Remember locally. Remember locally. Remember the requirement we the requirement we the requirement we started with? Because the data started with? Because the data started with? Because the data is sensitive, you is sensitive, you is sensitive, you need a way to need a way to need a way to fully migrate your fully migrate your fully migrate your workflows to workflows to workflows to privately hosted LLMs privately hosted LLMs . So, can this . So, can this . So, can this design really design really design really achieve that? Let's take achieve that? Let's take achieve that? Let's take one typed one typed one typed query from the Harper query from the Harper query from the Harper Bourbon 9 plan. Each Bourbon 9 plan. Each Bourbon 9 plan. Each model call is small, model call is small, model call is small, typed, and typed, and schema-validated. Therefore, schema-validated. Therefore, setting up a setting up a setting up a privately hosted privately hosted privately hosted model is a model is a model is a configuration change, not a configuration change, not a configuration change, not a code rewrite. code rewrite. code rewrite. One line One line One line changes, and the changes, and the changes, and the same answer is returned same answer is returned same answer is returned . All other . All other . All other queries have the same queries have the same queries have the same small typed small typed small typed form. Therefore, the same form. Therefore, the same form. Therefore, the same change applies change applies change applies to almost all of them. This is how to almost all of them. This is how to almost all of them. This is how about 95% of about 95% of local output local output local output becomes viable becomes viable becomes viable as an architectural as an architectural as an architectural consequence, not as a consequence, not as a consequence, not as a political promise. political promise. political promise. This is the path that This is the path that sensitive data requires, and sensitive data requires, and compliance risk compliance risk compliance risk is reduced is reduced is reduced as the engineering plan is as the engineering plan is as the engineering plan is released by release.

released by release. released by release. So, if you So, if you So, if you remember one thing, remember one thing, remember one thing, don't ask if don't ask if don't ask if your system should your system should your system should have more or less have more or less have more or less autonomy. autonomy. autonomy. Ask where the Ask where the Ask where the line is. Let line is. Let line is. Let determinism govern determinism govern determinism govern the flow of control, the flow of control, the flow of control, evidence, and evidence, and evidence, and side effects. side effects. side effects. Let the intellect be Let the intellect be Let the intellect be responsible for responsible for responsible for interpretation and interpretation and interpretation and judgment. At the judgment. At the judgment. At the typed points, typed points, typed points, you can check you can check you can check and use and use and use each solution you each solution you each solution you record so that record so that you need fewer of them next time. you need fewer of them next time. Ask where the Ask where the Ask where the line is. Thank you. I'm Karthik, line is. Thank you. I'm Karthik, line is. Thank you. I'm Karthik, CTO at CTO at CTO at Perpetual. If you Perpetual. If you Perpetual. If you are implementing AI in a are implementing AI in a are implementing AI in a regulated field, I would be regulated field, I would be regulated field, I would be happy happy happy to share my experiences.