Version-Controlled Search: The Web as It Was — Ryan Ahern, Exa

Greetings to everyone. I'm looking forward to introducing you to one introducing you to one of our newest of our newest of our newest Exa features, Exa features, Exa features, versioned search versioned search versioned search . I'm Ryan, a . I'm Ryan, a . I'm Ryan, a development engineer at development engineer at development engineer at Exa. Today I Exa. Today I Exa. Today I will tell you about two will tell you about two will tell you about two things. We have very little things. We have very little things. We have very little time. First, I want time. First, I want time. First, I want to explain what Exa is, to explain what Exa is, to explain what Exa is, what is agent-based web what is agent-based web search? Simply because search? Simply because search? Simply because you need to believe in you need to believe in you need to believe in this dream to this dream to this dream to understand why you understand why you understand why you want to want to want to use it use it use it for this. Secondly, I for this. Secondly, I for this. Secondly, I will talk about how to will talk about how to will talk about how to test financial test financial test financial models based on models based on models based on this. First, to this. First, to this. First, to understand what understand what understand what stage I'm at, has stage I'm at, has stage I'm at, has anyone here heard of Exa anyone here heard of Exa anyone here heard of Exa before or before or before or used it? used it? used it? Okay, please raise your Okay, please raise your Okay, please raise your hands. Who here hands. Who here hands. Who here has ever used Google for has ever used Google for has ever used Google for work or personal work or personal work or personal use, use, use, for example, to for example, to for example, to type in a Google search type in a Google search ? I hope, like everyone here ? I hope, like everyone here ? I hope, like everyone here , most , most , most people had a good people had a good people had a good market share, maybe market share, maybe you've heard of it. you've heard of it. you've heard of it. Last question: Last question: Last question: how many of you how many of you how many of you don't actually don't actually don't actually use Google anymore? You use Google anymore? You use Google anymore? You use ChatGPT, use ChatGPT, use ChatGPT, Claude, or even Instinct Claude, or even Instinct Claude, or even Instinct or Muse if you're on the or Muse if you're on the or Muse if you're on the front lines to front lines to front lines to actually actually actually do your do your do your searches. Yes, searches. Yes, searches. Yes, most people, most people, most people, right? So we're right? So we're right? So we're going to tie going to tie going to tie all of these things together. That all of these things together. That all of these things together. That last point, where last point, where last point, where a lot of people a lot of people a lot of people are now saying, "Hey, I are now saying, "Hey, I are now saying, "Hey, I don't even don't even don't even use use use computer search anymore. I ask computer search anymore. I ask computer search anymore. I ask an agent to do that" is something an agent to do that" is something an agent to do that" is something we're seeing in the we're seeing in the we're seeing in the market more broadly, right market more broadly, right ? The idea is ? The idea is ? The idea is that agents tend that agents tend to understand how to search pretty well to understand how to search pretty well because they are allowed to because they are allowed to because they are allowed to try multiple try multiple try multiple times. They don't times. They don't times. They don't get tired. They are generally get tired. They are generally get tired. They are generally quite quite quite friendly and friendly and friendly and work very hard work very hard . So when you say this is what . So when you say this is what . So when you say this is what I want, I want, I want, it will go out and do it will go out and do it will go out and do a lot of different

a lot of different a lot of different searches to searches to searches to satisfy your satisfy your satisfy your goal. And then there's the fact that goal. And then there's the fact that goal. And then there's the fact that they tend to they tend to they tend to look for things differently than look for things differently than look for things differently than people. So, as a human, people. So, as a human, people. So, as a human, you might type in you might type in you might type in a few keywords a few keywords a few keywords for something and for something and for something and hope you hope you hope you get the best get the best get the best results, like in the results, like in the results, like in the first few first few first few cases, and otherwise cases, and otherwise cases, and otherwise , maybe you , maybe you , maybe you try a different try a different try a different query and then query and then query and then give up. Whereas give up. Whereas give up. Whereas agents will be very agents will be very agents will be very verbose. They verbose. They verbose. They tend to describe what they tend to describe what they tend to describe what they want to find in want to find in want to find in great detail and great detail and great detail and will perform many will perform many will perform many searches to searches to searches to understand the space more fully understand the space more fully understand the space more fully before responding. before responding. before responding. And that means they And that means they And that means they perform many perform many perform many more searches for more searches for more searches for each task. each task. each task. So, the general idea So, the general idea So, the general idea is that: " is that: " Hey, most people these days Hey, most people these days use use agents to agents to agents to mediate mediate mediate your search your search your search tasks." And this is where tasks." And this is where tasks." And this is where Exa comes into play. Exa comes into play. Exa comes into play. So when you work So when you work So when you work with many agent with many agent with many agent systems. So, systems. So, systems. So, every time Cursor wants every time Cursor wants every time Cursor wants to find the latest to find the latest to find the latest documentation to documentation to documentation to help its help its help its coding agent coding agent coding agent complete its complete its complete its trajectory, every time trajectory, every time trajectory, every time you're talking to you're talking to you're talking to Gamma and you think, "Hey, it would be Gamma and you think, "Hey, it would be Gamma and you think, "Hey, it would be great if great if great if I had some SEC data I had some SEC data like Apple's, say, like Apple's, say, like Apple's, say, 2018, to finish 2018, to finish 2018, to finish my slide, that would my slide, that would my slide, that would make it make it make it shine." It shine." It shine." It will use Exa will use Exa will use Exa to search the entire to search the entire to search the entire web, so we web, so we web, so we will have our own will have our own crawling infrastructure, our own crawling infrastructure, our own database, our own database, our own database, our own embedding model, embedding model, embedding model, our own our own our own search architecture to get search architecture to get search architecture to get exactly what we need exactly what we need exactly what we need to answer a to answer a to answer a query. It's kind of like a query. It's kind of like a query. It's kind of like a few few few logos of big logos of big logos of big companies, but companies, but companies, but since this is since this is since this is artificial artificial artificial intelligence engineering, here are some intelligence engineering, here are some intelligence engineering, here are some archetypes that we archetypes that we archetypes that we work with in the work with in the work with in the financial sector. I financial sector. I financial sector. I work with a lot of work with a lot of work with a lot of hedge funds, hedge funds, hedge funds, asset managers, asset managers, asset managers, sovereign sovereign sovereign wealth funds, wealth funds, wealth funds, venture venture venture capitalists, on capitalists, on capitalists, on a lot of different a lot of different a lot of different assignments because assignments because assignments because they also need the they also need the they also need the latest information latest information . So, if you're a hedge

. So, if you're a hedge funder, you'll be funder, you'll be funder, you'll be trading trading trading information. You're information. You're information. You're looking for alpha in the market looking for alpha in the market looking for alpha in the market , right? As , right? As , right? As well as information that well as information that well as information that allows you allows you allows you to understand how to better to understand how to better to understand how to better position your position your position your organization. If you're, organization. If you're, organization. If you're, um, an insurer, you um, an insurer, you um, an insurer, you probably understand probably understand probably understand what what what counterparties you're counterparties you're counterparties you're working with or who you're working with or who you're working with or who you're insuring to insuring to insuring to make sure make sure make sure you're ultimately making the right you're ultimately making the right you're ultimately making the right decision. So you decision. So you decision. So you want to make sure you're want to make sure you're want to make sure you're doing this doing this doing this based on the best based on the best based on the best information and your information and your information and your risk models are the risk models are the risk models are the most accurate. Or most accurate. Or most accurate. Or even if you're even if you're even if you're in venture in venture in venture capital, you think, capital, you think, "I need the "I need the "I need the hottest startup hottest startup ." I need to know the ." I need to know the ." I need to know the next moment before next moment before next moment before they they they go bankrupt." And they wo go bankrupt." And they wo go bankrupt." And they wo n't necessarily be n't necessarily be n't necessarily be in some in some in some structured structured structured data set. I data set. I data set. I need to find them need to find them need to find them wherever they are on the wherever they are on the wherever they are on the internet. Hmm, that's why internet. Hmm, that's why internet. Hmm, that's why they're they're they're turning to Exa to turning to Exa to turning to Exa to support them. Hmm, and support them. Hmm, and support them. Hmm, and then we'll wrap up then we'll wrap up then we'll wrap up the last part, the last part, the last part, like why Exa, and like why Exa, and like why Exa, and why I think you're why I think you're why I think you're like, "Okay, I'm like, "Okay, I'm like, "Okay, I'm building agents, I'm building agents, I'm building agents, I'm interested in agents, and interested in agents, and interested in agents, and I definitely need I definitely need I definitely need generative web generative web search." Why Exa? Hmm, that's search." Why Exa? Hmm, that's search." Why Exa? Hmm, that's because we're like a because we're like a because we're like a laboratory. Um, in laboratory. Um, in laboratory. Um, in the sense that we're the sense that we're the sense that we're very, very interested in the very, very interested in the very, very interested in the search market, um, and search market, um, and search market, um, and the ways that agents the ways that agents the ways that agents search, and we're hyper- search, and we're hyper- search, and we're hyper- focused on that focused on that focused on that problem. There are three problem. There are three problem. There are three main principles, and main principles, and main principles, and then we'll talk then we'll talk then we'll talk about how that impacts about how that impacts version control to move on version control to move on to the next part of the to the next part of the to the next part of the conversation. But first conversation. But first , and this is quite obvious , and this is quite obvious , it's quality, right?

, it's quality, right? , it's quality, right? So you want to So you want to So you want to make sure that when make sure that when make sure that when the agent does this, um, the agent does this, um, the agent does this, um, tool call tool call tool call and search, they get what and search, they get what and search, they get what they need. they need. they need. Because the fewer Because the fewer tool calls it tool calls it needs to make, the more needs to make, the more needs to make, the more cost- cost- cost- effective it is for effective it is for effective it is for your conclusion, your conclusion, your conclusion, and the more and the more and the more likely it is to succeed likely it is to succeed likely it is to succeed on its overall on its overall on its overall trajectory. You're not trajectory. You're not trajectory. You're not going to going to going to bring in anything that bring in anything that bring in anything that seems to throw him off seems to throw him off seems to throw him off balance. Hmm, uh, we are balance. Hmm, uh, we are balance. Hmm, uh, we are very, very interested in the very, very interested in the very, very interested in the area of ​​quality. area of ​​quality. area of ​​quality. We recently We recently We recently released a new released a new released a new benchmark that we benchmark that we benchmark that we believe more accurately believe more accurately believe more accurately measures measures search search engine performance separately engine performance separately engine performance separately from findings, from findings, from findings, because the two because the two because the two can be mixed, can be mixed, can be mixed, right? It's a matter of which right? It's a matter of which right? It's a matter of which agent or which model agent or which model agent or which model is invoking the search, and is invoking the search, and is invoking the search, and which search is associated which search is associated which search is associated with which model. So you with which model. So you with which model. So you can take a look at can take a look at can take a look at that to see our that to see our that to see our latest thoughts on latest thoughts on latest thoughts on that. Secondly, regarding that. Secondly, regarding that. Secondly, regarding the cost. So it's not the cost. So it's not the cost. So it's not just a tool call string just a tool call string just a tool call string , right , right , right ? For example, ? For example, ? For example, how much does one how much does one how much does one search cost? Because agents search cost? Because agents search cost? Because agents can perform can perform can perform many different many different many different searches. So the idea searches. So the idea searches. So the idea is, "Hey, what's is, "Hey, what's is, "Hey, what's the value of this in the value of this in the value of this in completing your completing your completing your trajectory?" But trajectory?" But trajectory?" But also "What impact does this have also "What impact does this have also "What impact does this have on the overall cost of on the overall cost of on the overall cost of completing the task completing the task completing the task by this agent?" Because that's by this agent?" Because that's by this agent?" Because that's really what you really what you really what you care about, isn't it? You are care about, isn't it? You are care about, isn't it? You are usually less usually less usually less categorical about categorical about categorical about one one one tool call. The thing is, tool call. The thing is, tool call. The thing is, "How "How "How much did this much did this much did this agent cost?" Uh, and that's why agent cost?" Uh, and that's why agent cost?" Uh, and that's why we've been working on we've been working on we've been working on things like, things like, things like, say, dynamic say, dynamic say, dynamic selection, which means selection, which means that for each that for each that for each individual result, individual result, individual result, we're not just we're not just we're not just returning full returning full returning full pages. Uh, we're pages. Uh, we're pages. Uh, we're not even just not even just not even just taking one page, taking one page, taking one page, like we did with our like we did with our like we did with our previous previous previous selection functions, selection functions, selection functions, and figuring out exactly what are the and figuring out exactly what are the and figuring out exactly what are the right pieces right pieces right pieces on that page that we on that page that we on that page that we need to send need to send need to send back to the back to the back to the context window context window context window to complete the to complete the to complete the trajectory. We now trajectory. We now trajectory. We now do this for all do this for all do this for all search results, search results, search results, which means you which means you which means you can get can get can get higher higher higher quality results with 95% fewer

quality results with 95% fewer quality results with 95% fewer tokens tokens tokens than with the full text of than with the full text of than with the full text of the page. We have a blog the page. We have a blog the page. We have a blog about this, it's very interesting about this, it's very interesting , but basically it , but basically it , but basically it means you'll means you'll means you'll save a lot of save a lot of save a lot of money on all the money on all the money on all the input tokens that input tokens that input tokens that you'd spend on you'd spend on you'd spend on output for the agent output for the agent output for the agent to complete your to complete your to complete your trajectory. The last one is trajectory. The last one is trajectory. The last one is about latency, so about latency, so about latency, so since we have since we have since we have our own our own our own infrastructure, we infrastructure, we infrastructure, we can do can do can do searches with very low searches with very low searches with very low latency. Um, I'll latency. Um, I'll latency. Um, I'll just touch on just touch on just touch on the other side of the spectrum, the other side of the spectrum, the other side of the spectrum, which is like a very which is like a very which is like a very long in-depth long in-depth long in-depth study. So study. So study. So we have ExaAgent Ultra, which we we have ExaAgent Ultra, which we we have ExaAgent Ultra, which we released, um, not too released, um, not too released, um, not too long ago, maybe long ago, maybe long ago, maybe a few weeks ago, and a few weeks ago, and a few weeks ago, and which is state-of-the-art which is state-of-the-art which is state-of-the-art based on a lot of based on a lot of based on a lot of benchmarks, like benchmarks, like benchmarks, like very, very long very, very long very, very long search. And that's because we search. And that's because we search. And that's because we can do a lot of can do a lot of can do a lot of things "under the hood." things "under the hood." things "under the hood." We obviously We obviously We obviously own the search own the search own the search equipment, but we equipment, but we equipment, but we can also have can also have can also have our say on our say on our say on how it interacts with the how it interacts with the how it interacts with the agent to agent to agent to use it use it use it to perform these to perform these to perform these very, very long very, very long very, very long research research research tasks. So tasks. So tasks. So I highly recommend you I highly recommend you I highly recommend you familiarize yourself with these familiarize yourself with these familiarize yourself with these things. Hmm, it's just a things. Hmm, it's just a things. Hmm, it's just a transition to transition to transition to saying, "Okay, Exa saying, "Okay, Exa saying, "Okay, Exa really cares about this really cares about this really cares about this area. I think area. I think area. I think their search will be their search will be their search will be good." To good." To good." To motivate the next motivate the next motivate the next part, which is really part, which is really part, which is really interesting and is the topic of this interesting and is the topic of this interesting and is the topic of this conversation, which is conversation, which is conversation, which is version control. version control. version control. We recently We recently We recently released ExaSnapshot. That's released ExaSnapshot. That's released ExaSnapshot. That's over 400 billion over 400 billion over 400 billion images of the internet, and images of the internet, and images of the internet, and you can search them you can search them you can search them by date. Hmm, and this is by date. Hmm, and this is by date. Hmm, and this is really, really interesting in the really, really interesting in the really, really interesting in the financial financial financial services industry. We'll look at services industry. We'll look at services industry. We'll look at this in more detail, but this in more detail, but this in more detail, but first I'll talk a little first I'll talk a little first I'll talk a little about what this about what this essentially means.

essentially means. essentially means. So, we are a search So, we are a search So, we are a search company, so we are company, so we are company, so we are constantly searching the constantly searching the constantly searching the Internet for the latest Internet for the latest Internet for the latest information that information that information that we can add to we can add to we can add to our index. our index. our index. Every time we Every time we Every time we do this, we do this, we do this, we check it for check it for check it for relevance. relevance. relevance. Ultimately, you want to Ultimately, you want to Ultimately, you want to have the latest have the latest have the latest relevant information relevant information relevant information at the time of your search. But at the time of your search. But at the time of your search. But because we because we because we do all these do all these do all these checks, it means checks, it means we can we can we can get the version of these get the version of these get the version of these URLs at the time of URLs at the time of URLs at the time of the check and have a the check and have a the check and have a version history. You version history. You version history. You can can can access it in two access it in two access it in two ways. One of ways. One of ways. One of them is: "I know the URL I them is: "I know the URL I them is: "I know the URL I want to find, want to find, want to find, as it was as it was as it was on a certain date or on a certain date or on a certain date or earlier." You can earlier." You can earlier." You can give us a URL give us a URL give us a URL and say, “This is the date and say, “This is the date and say, “This is the date I want to find it. I want to find it. I want to find it. I want it on I want it on I want it on or before this date, and to provide or before this date, and to provide or before this date, and to provide it to me as you would have it to me as you would have it to me as you would have done if I had done if I had done if I had requested it at that requested it at that requested it at that time." The second, even time." The second, even time." The second, even more exciting, more exciting, is the concept of is the concept of is the concept of version search. So when there's a version search. So when there's a version search. So when there's a search API, the idea is, "What search API, the idea is, "What search API, the idea is, "What would happen if I would happen if I would happen if I did this search, did this search, did this search, but earlier, right?" but earlier, right?" » So I only get » So I only get » So I only get results that would have been results that would have been results that would have been provided at that moment provided at that moment provided at that moment or earlier. This has or earlier. This has or earlier. This has very interesting consequences. very interesting consequences.

very interesting consequences. I'll just give one I'll just give one I'll just give one very small very small very small example. OpenAI example. OpenAI example. OpenAI recently, I'd say as of recently, I'd say as of recently, I'd say as of October 1st, October 1st, October 1st, so on the left you'll so on the left you'll so on the left you'll see a snapshot of see a snapshot of see a snapshot of the search as it was on the search as it was on the search as it was on October 1st, and on the right October 1st, and on the right October 1st, and on the right was the active search was the active search was the active search as of a few days as of a few days as of a few days ago when I was making this ago when I was making this ago when I was making this slide. But, uh, in slide. But, uh, in slide. But, uh, in early October, there was early October, there was early October, there was some discussion that some discussion that OpenAI had an annual OpenAI had an annual OpenAI had an annual growth rate of growth rate of growth rate of about 70 billion, about 70 billion, about 70 billion, right? And that right? And that right? And that led to certain led to certain led to certain ways in which the market ways in which the market ways in which the market predicted how and where predicted how and where predicted how and where they would be they would be they would be valued. And then, valued. And then, valued. And then, more recently, there were more recently, there were more recently, there were some reports, and some reports, and some reports, and it was originally it was originally it was originally done by Axios. And done by Axios. And done by Axios. And recently, there have been recently, there have been recently, there have been publications that publications that publications that reported something reported something reported something like: "Actually, like: "Actually, like: "Actually, it's closer to 50 it's closer to 50 it's closer to 50 billion for interesting billion for interesting billion for interesting accounting accounting accounting reasons, right?" reasons, right?" reasons, right?" It turns out they It turns out they It turns out they were taking gross were taking gross were taking gross revenue into account, when in fact revenue into account, when in fact revenue into account, when in fact some of their some of their some of their revenue came from revenue came from revenue came from cloud cloud cloud providers, and providers, and providers, and technically that should be technically that should be technically that should be taken into account so there taken into account so there taken into account so there would be a transition would be a transition would be a transition period from the period from the period from the provider, so provider, so provider, so net annual revenue is net annual revenue is net annual revenue is better. That's how better. That's how better. That's how some some some other players etc. have reported, other players etc. have reported, other players etc. have reported, right? But right? But right? But basically that basically that basically that meant that the meant that the meant that the projected net projected net projected net annual annual annual execution rate went down from execution rate went down from execution rate went down from 70 billion to 50 70 billion to 50 70 billion to 50 billion, right?

billion, right? billion, right? It's just a financial It's just a financial It's just a financial thing that some of you thing that some of you thing that some of you may find interesting may find interesting may find interesting and some may not. But what and some may not. But what and some may not. But what is version search? is version search? is version search? Let's say you're Let's say you're Let's say you're interested in interested in interested in creating a product, creating a product, creating a product, solution, or agent solution, or agent solution, or agent designed to designed to find these things, find these things, right? Now you right? Now you right? Now you can go can go can go back and ask yourself, back and ask yourself, “Okay, before “Okay, before “Okay, before everyone figured it out, everyone figured it out, everyone figured it out, right? Before right? Before right? Before the market and some the market and some the market and some financial analyst financial analyst financial analyst published an published an published an analytical article analytical article analytical article that would sharply that would sharply that would sharply criticize this criticize this criticize this particular thing and everyone particular thing and everyone particular thing and everyone knew about it, what knew about it, what knew about it, what information existed information existed information existed on the Internet, right on the Internet, right ? For example, was there an ? For example, was there an ? For example, was there an SEC report that SEC report that SEC report that disclosed this in a footnote disclosed this in a footnote disclosed this in a footnote to the to the to the accounting? For example, accounting? For example, accounting? For example, were there any other reports were there any other reports were there any other reports from some other from some other from some other provider that provider that provider that you might have considered you might have considered you might have considered less authoritative, less authoritative, less authoritative, but were actually able to but were actually able to but were actually able to find these things, and find these things, and find these things, and maybe you should maybe you should maybe you should look at them look at them differently.” Hmm, you could differently.” Hmm, you could differently.” Hmm, you could potentially potentially potentially track these things down track these things down track these things down before the rest of the market before the rest of the market before the rest of the market found them. But found them. But found them. But most importantly, you most importantly, you most importantly, you can not only can not only can not only test it, but you can also test it, but you can also test it, but you can also do do backtesting, right? backtesting, right? Then you can Then you can Then you can adjust the ways you adjust the ways you adjust the ways you configure your configure your configure your model, you can model, you can model, you can adjust the ways you adjust the ways you adjust the ways you configure your configure your configure your agent, so that you can agent, so that you can agent, so that you can ask yourself, ask yourself, ask yourself, "Could it have detected "Could it have detected "Could it have detected this discrepancy?"

this discrepancy?" this discrepancy?" Oh, and more importantly, could Oh, and more importantly, could Oh, and more importantly, could he have identified this he have identified this he have identified this discrepancy in a discrepancy in a discrepancy in a general way, general way, general way, right? Let's right? Let's right? Let's create a set of create a set of create a set of about 50 about 50 about 50 sample sets, sample sets, sample sets, see if see if see if it can handle that, it can handle that, it can handle that, test it with test it with test it with a delay, and say, a delay, and say, a delay, and say, "Okay, I think "Okay, I think "Okay, I think my model will now be my model will now be my model will now be pretty good at pretty good at pretty good at finding finding finding features like that." And this is something features like that." And this is something features like that." And this is something new in the web new in the web search space, and you can search space, and you can search space, and you can test it in test it in test it in retrospect. This is kind of a retrospect. This is kind of a little diagram little diagram to illustrate to illustrate to illustrate the idea, but during the idea, but during any live search any live search any live search that you do with that you do with that you do with any existing any existing any existing search product, search product, search product, the idea is the idea is the idea is that it's active, so that it's active, so that it's active, so it has all the it has all the it has all the information. So in information. So in information. So in this particular this particular this particular little example, little example, little example, we're trying we're trying we're trying to predict whether to predict whether to predict whether a company has been acquired or a company has been acquired or a company has been acquired or not. In this case, not. In this case, not. In this case, if you do a live if you do a live if you do a live search, you will see that search, you will see that search, you will see that the deal has been announced, the deal has been announced, the deal has been announced, right? Since it right? Since it right? Since it happened, it has been happened, it has been happened, it has been publicized everywhere. publicized everywhere. publicized everywhere. You can say, " You can say, " Hey, I'm trying Hey, I'm trying Hey, I'm trying to train a model that to train a model that to train a model that can predict can predict can predict whether this company was acquired whether this company was acquired whether this company was acquired or not." So or not." So , I'm going to go in , I'm going to go in , I'm going to go in somewhere like 1 day before the somewhere like 1 day before the somewhere like 1 day before the announcement, 7 days announcement, 7 days announcement, 7 days before the announcement, a before the announcement, a before the announcement, a month before the announcement month before the announcement month before the announcement and see if and see if and see if I can I can I can predict it or not, predict it or not, predict it or not, right? Uh, and you could right? Uh, and you could right? Uh, and you could create a set of these create a set of these create a set of these companies, um, and companies, um, and companies, um, and essentially train your essentially train your essentially train your model, um, or model, um, or any other system that's any other system that's any other system that's trying to do trying to do trying to do this prediction, and this prediction, and this prediction, and see if see if see if it could reliably it could reliably it could reliably predict this predict this predict this particular feature. Uh particular feature. Uh , I'm going to , I'm going to , I'm going to spend a little time spend a little time spend a little time here just here just here just talking about how talking about how talking about how different systems are currently different systems are currently different systems are currently being backtested being backtested using Snapshot. So, there using Snapshot. So, there are three types of are three types of are three types of archetype here. The first is that I archetype here. The first is that I archetype here. The first is that I work as a very work as a very work as a very traditional model.

traditional model. traditional model. Maybe it's machine Maybe it's machine Maybe it's machine learning, maybe it's learning, maybe it's learning, maybe it's even simpler, like a even simpler, like a even simpler, like a decision tree or decision tree or decision tree or something. Uh, and all I something. Uh, and all I something. Uh, and all I want to do is want to do is want to do is just collect the just collect the just collect the unstructured data unstructured data unstructured data from the web version into some kind of from the web version into some kind of from the web version into some kind of structured set of structured set of structured set of information. It could information. It could information. It could be something like be something like be something like software pricing software pricing software pricing , it could , it could , it could be some be some be some sentiment analysis, it could sentiment analysis, it could sentiment analysis, it could be the terms and be the terms and be the terms and conditions of a particular conditions of a particular conditions of a particular site. Whatever site. Whatever your model provides, not all your model provides, not all information is always information is always information is always available. For example, available. For example, available. For example, if you are trying if you are trying if you are trying to understand to understand software prices, the software price page is not software price page is not necessarily necessarily necessarily stored in the stored in the stored in the version for this website in a way that version for this website in a way that you can reliably you can reliably access it access it access it . Or maybe . Or maybe . Or maybe you have a you have a you have a sentiment analysis that you sentiment analysis that you sentiment analysis that you want to do, but want to do, but want to do, but it will depend on it will depend on it will depend on what point in what point in what point in time that sentiment was released time that sentiment was released time that sentiment was released , right? And , right? And , right? And you want to be you want to be you want to be able to get a able to get a able to get a reliable set of these reliable set of these reliable set of these signals. And then you signals. And then you signals. And then you can just can just can just turn it into an turn it into an turn it into an array of signals that array of signals that array of signals that goes into your goes into your goes into your model, and you can model, and you can model, and you can predict it. This is a kind of predict it. This is a kind of predict it. This is a kind of base base base space. Some people are space. Some people are space. Some people are simply enriching simply enriching simply enriching data that they wouldn't have data that they wouldn't have data that they wouldn't have access to for access to for access to for their existing their existing their existing models. So it's kind of like the models. So it's kind of like the models. So it's kind of like the status quo status quo status quo plus one. The interesting plus one. The interesting plus one. The interesting space lies in space lies in space lies in the following two the following two the following two areas. The first of these areas. The first of these is the idea of ​​adding an is the idea of ​​adding an is the idea of ​​adding an agent function. Um, agent function. Um, agent function. Um, and this is it: “Okay, and this is it: “Okay, and this is it: “Okay, I have this existing I have this existing I have this existing model. It works model. It works model. It works pretty well, but pretty well, but pretty well, but I have an idea that if I I have an idea that if I I have an idea that if I add one or add one or add one or two agent two agent two agent characteristics to it, it characteristics to it, it characteristics to it, it could work could work could work better." So, let's say better." So, let's say better." So, let's say something like

something like negative media scanning. negative media scanning. Of course, Of course, Of course, negative media negative media negative media goes online, goes online, goes online, searches for all searches for all searches for all the information that exists there, the information that exists there, the information that exists there, potentially doing potentially doing potentially doing it in an agentic way, it in an agentic way, it in an agentic way, right? When you right? When you right? When you do multiple do multiple do multiple searches, you searches, you searches, you exhaust several exhaust several exhaust several different trajectories, and different trajectories, and different trajectories, and then you take all of that then you take all of that then you take all of that and compress it into some kind of and compress it into some kind of and compress it into some kind of structure. Um, another structure. Um, another structure. Um, another example here would be, example here would be, example here would be, for example, for example, justice research, justice research, pretty much the same. pretty much the same. pretty much the same. Like, “Hey, I want to Like, “Hey, I want to Like, “Hey, I want to go out. I want to go out. I want to go out. I want to look at the entire look at the entire look at the entire world of companies and world of companies and world of companies and find potentially find potentially find potentially all of their, for example, all of their, for example, all of their, for example, approximate revenue, approximate revenue, approximate revenue, number of number of number of employees, or all of the employees, or all of the employees, or all of the important important important characteristics. The idea characteristics. The idea characteristics. The idea is that you is that you is that you would use would use would use something like ExaSearch, something like ExaSearch, something like ExaSearch, where you would where you would where you would run your run your run your prediction to prediction to prediction to gather all this gather all this gather all this information and then information and then information and then feed it into your, feed it into your, feed it into your, ahem, traditional model ahem, traditional model . And now you can . And now you can . And now you can use, use, use, for example, ExaSnapshot to for example, ExaSnapshot to for example, ExaSnapshot to test these test these test these specific specific specific characteristics. So you characteristics. So you characteristics. So you can go can go can go back and say, “Hey, what back and say, “Hey, what back and say, “Hey, what if I had done this, if I had done this, if I had done this, like, negative like, negative like, negative media scan media scan media scan a month ago?” “Or a month ago?” “Or a month ago?” “Or maybe I had already maybe I had already maybe I had already brought a bunch of people into brought a bunch of people into brought a bunch of people into my portfolio my portfolio my portfolio about 4 months about 4 months about 4 months ago. What would I say ago. What would I say ago. What would I say if they were going to if they were going to if they were going to join my join my join my portfolio? And how does that portfolio? And how does that portfolio? And how does that compare to compare to compare to today? Hmm today? Hmm , and that's motivating, , and that's motivating, , and that's motivating, especially with so many especially with so many especially with so many financial financial financial services customers like now, you services customers like now, you services customers like now, you don't have to just don't have to just don't have to just trust that search will be trust that search will be trust that search will be better. You can better. You can better. You can go back and go back and go back and test it test it test it against your against your against your existing processes and existing processes and existing processes and say, "Hey, this is say, "Hey, this is say, "Hey, this is better." “Um, so this is better." “Um, so this is better." “Um, so this is maybe the next maybe the next maybe the next evolution, something evolution, something evolution, something like one step like one step like one step plus plus. Um, plus plus. Um, plus plus. Um, on the right, it's kind of like the on the right, it's kind of like the on the right, it's kind of like the future.

future. So, there are several So, there are several startups startups startups working in the field of working in the field of working in the field of forecasting. The idea forecasting. The idea forecasting. The idea is that the is that the is that the complete modeling complete modeling complete modeling or prediction or prediction or prediction is driven by an agent. So is driven by an agent. So is driven by an agent. So in the middle in the middle in the middle is a is a is a machine learning model machine learning model machine learning model that maybe does that maybe does that maybe does prediction, prediction, prediction, some kind of algorithm, or a some kind of algorithm, or a some kind of algorithm, or a decision tree. decision tree. decision tree. On the right is an agent On the right is an agent On the right is an agent who just goes out and who just goes out and who just goes out and does a bunch of things. It does a bunch of things. It does a bunch of things. It could be a search for could be a search for could be a search for inner knowledge. This inner knowledge. This inner knowledge. This could be access to could be access to could be access to data channels. It data channels. It data channels. It could be something else. could be something else. could be something else. But one of the valuable But one of the valuable But one of the valuable things it can things it can things it can do is do is do is call up a tool call up a tool , uh, and , uh, and , uh, and access the Internet. access the Internet. access the Internet. The idea is that The idea is that The idea is that now you don't now you don't now you don't have to try to have to try to have to try to create some kind of create some kind of create some kind of internal shell internal shell internal shell that tries to that tries to that tries to represent what it's like to represent what it's like to access the access the access the full Internet. You full Internet. You full Internet. You can just can just can just say, "Hey, I want to say, "Hey, I want to say, "Hey, I want to make sure I can make sure I can make sure I can control this control this control this function so I know that function so I know that function so I know that future information future information future information isn't just going to isn't just going to isn't just going to leak leak leak directly into my directly into my directly into my model." “I’m trying model." “I’m trying model." “I’m trying to predict the to predict the to predict the acquisition of companies acquisition of companies acquisition of companies through some through some through some very crazy very crazy very crazy agency process agency process agency process that has access to that has access to that has access to everything. Now you everything. Now you everything. Now you can say something can say something can say something like, “Okay, like, “Okay, like, “Okay, let’s let’s let’s make sure it’s make sure it’s make sure it’s not just one not just one not just one web search that web search that web search that detects that something was detects that something was detects that something was purchased, purchased, purchased, comes back and comes back and comes back and predicts it with 100% predicts it with 100% accuracy. So this is kind of like the accuracy. So this is kind of like the accuracy. So this is kind of like the right right right side. I'll just give a side. I'll just give a side. I'll just give a little example of the little example of the little example of the middle ground to middle ground to middle ground to wrap up this wrap up this wrap up this discussion. Yesterday we discussion. Yesterday we discussion. Yesterday we did a sort of did a sort of did a sort of extended extended snapshot workshop , and we had about , and we had about , and we had about 100 people in the room 100 people in the room 100 people in the room looking at the looking at the looking at the same example same example same example we had earlier about we had earlier about we had earlier about trying to predict trying to predict trying to predict company acquisitions.

company acquisitions. Interestingly, this was one Interestingly, this was one of the examples from the of the examples from the of the examples from the seminar from one of the seminar from one of the seminar from one of the people who got a people who got a people who got a relatively high score relatively high score .” But the interesting thing here .” But the interesting thing here .” But the interesting thing here was just to was just to motivate. The first motivate. The first idea was idea was idea was that within the past that within the past that within the past year, an investor or year, an investor or year, an investor or significant shareholder significant shareholder significant shareholder had publicly called for had publicly called for had publicly called for the sale of the company as a the sale of the company as a the sale of the company as a whole. And the second, which is whole. And the second, which is whole. And the second, which is interesting, is interesting, is interesting, is that the share price of that the share price of that the share price of this company was this company was this company was at least 60% at least 60% lower than the highest lower than the highest lower than the highest share price in the last share price in the last share price in the last 5 years, meaning it 5 years, meaning it 5 years, meaning it seemed to have declined. seemed to have declined. seemed to have declined. The last interesting idea The last interesting idea The last interesting idea is that it's a is that it's a is that it's a company called company called company called Dominion. This is a bit of Dominion. This is a bit of Dominion. This is a bit of a re-training for a re-training for a re-training for people who are familiar people who are familiar people who are familiar with retrospective with retrospective with retrospective testing. The idea testing. The idea testing. The idea is that this is that this is that this might be something that an might be something that an might be something that an agent could do agent could do agent could do correctly if correctly if correctly if trained on a live trained on a live trained on a live set, it set, it set, it could potentially could potentially could potentially just retrain just retrain just retrain to the available to the available to the available training data. training data. training data. The idea with a snapshot The idea with a snapshot The idea with a snapshot is that you is that you is that you can not only can not only can not only go back and go back and go back and test these things test these things ...Um, you can ...Um, you can ...Um, you can go back and go back and go back and adjust your adjust your adjust your score and your score and your score and your identifier in a identifier in a identifier in a way that detects way that detects way that detects these things, but you these things, but you these things, but you can also try can also try can also try to see if it to see if it to see if it generalizes. And that would be generalizes. And that would be generalizes. And that would be very, very very, very very, very difficult to do if difficult to do if difficult to do if you're limited to, you're limited to, you're limited to, for example, for example, real-time testing, real-time testing, like in like in like in production. And you production. And you production. And you can only can only can only change the ways you change the ways you change the ways you describe these functions over describe these functions over describe these functions over time and test them time and test them time and test them in real time, in real time, in real time, right? You can right? You can right? You can wait months to wait months to wait months to try to see try to see try to see how and why your how and why your how and why your identical system, ahem identical system, ahem , is actually , is actually , is actually predictive or predictive or predictive or over- over- over- tuned in tuned in tuned in various ways. Hmm various ways. Hmm , so you can , so you can , so you can describe these fairly describe these fairly describe these fairly complex functions complex functions complex functions like, um, these, maybe you

like, um, these, maybe you like, um, these, maybe you can imagine can imagine can imagine trying trying trying to reproduce, um, uh, in to reproduce, um, uh, in to reproduce, um, uh, in some, uh, non- some, uh, non- some, uh, non- semantic semantic semantic search. But you can search. But you can search. But you can go wild as you go wild as you go wild as you wish within the wish within the wish within the existing, uh, world of existing, uh, world of existing, uh, world of incremental incremental incremental semantic search. semantic search. semantic search. So you can say So you can say So you can say something like, “Hey, something like, “Hey, something like, “Hey, I want to know if there have I want to know if there have I want to know if there have been any negative been any negative been any negative comments in the media about comments in the media about comments in the media about high-profile high-profile VPs who VPs who VPs who work, um, um, in work, um, um, in work, um, um, in venture capital funds, who venture capital funds, who , for example, were on the , for example, were on the , for example, were on the AI ​​Engineer podcast and AI ​​Engineer podcast and AI ​​Engineer podcast and they said something they said something they said something bad about it, bad about it, bad about it, right? You can right? You can right? You can put all your put all your put all your effort into these features effort into these features that you think that you think that you think will be predictive, will be predictive, will be predictive, because that's what because that's what because that's what basic incremental basic incremental basic incremental search allows, but then you search allows, but then you search allows, but then you can test can test can test it on hindsight, it on hindsight, it on hindsight, right? And right? And right? And see if it see if it see if it works or not. Then I'll works or not. Then I'll works or not. Then I'll just summarize. just summarize. just summarize. The first thing we The first thing we The first thing we talked about here is talked about here is talked about here is using Exa, using Exa, using Exa, as it is a high-quality as it is a high-quality as it is a high-quality and cost- and cost- and cost- effective effective effective identical web search identical web search identical web search with any latency with any latency . We have a lot of . We have a lot of . We have a lot of blogs on this topic, blogs on this topic, blogs on this topic, so I encourage you, so I encourage you, so I encourage you, if you haven't if you haven't if you haven't tried Exa yet, tried Exa yet, tried Exa yet, to give it a try, because the to give it a try, because the to give it a try, because the first thing you want first thing you want first thing you want to know is, "Hey, do to know is, "Hey, do to know is, "Hey, do I really think this is going to I really think this is going to I really think this is going to add any add any add any benefit to my model?" benefit to my model?" benefit to my model?" “The second question, as we’ve “The second question, as we’ve “The second question, as we’ve discussed, discussed, discussed, is, ‘Hey, is, ‘Hey, is, ‘Hey, using an using an using an Exa snapshot to test these Exa snapshot to test these Exa snapshot to test these financial models financial models financial models or any other or any other or any other risk model, risk model, risk model forecasting, models risk model forecasting, models using these using these using these web search features web search features web search features to see, to see, to see, like, ‘Hey, is like, ‘Hey, is like, ‘Hey, is this going to add value to this going to add value to this going to add value to you and your you and your you and your identity processes?

identity processes? ""Thank you very much".