I Gave an AI a Body — Cyrus Clarke, MIT Media Lab

So, I'm Cyrus, and I gave an AI a body. So, I'm Cyrus, and I gave an AI a body. And I'm a researcher at the MIT Media And I'm a researcher at the MIT Media And I'm a researcher at the MIT Media Lab, which is a Lab, which is a Lab, which is a very multidisciplinary space where we do very multidisciplinary space where we do very multidisciplinary space where we do all kinds of things. all kinds of things. all kinds of things. I predominantly now work with I predominantly now work with I predominantly now work with some aspects of physical AI. some aspects of physical AI. some aspects of physical AI. Maybe not in exactly the same way as Maybe not in exactly the same way as Maybe not in exactly the same way as other people in this room have been other people in this room have been other people in this room have been talking about, talking about, talking about, but it's still in that in that realm. but it's still in that in that realm. but it's still in that in that realm. And I think what I am most interested in And I think what I am most interested in And I think what I am most interested in at the moment is at the moment is at the moment is the sensory and the embodied aspects of the sensory and the embodied aspects of the sensory and the embodied aspects of intelligence. And that's what I've been intelligence. And that's what I've been intelligence. And that's what I've been investigating. investigating. investigating. And my work has been quite influential And my work has been quite influential And my work has been quite influential recently, suddenly, which is pretty recently, suddenly, which is pretty recently, suddenly, which is pretty cool, and led to me also starting Hard cool, and led to me also starting Hard cool, and led to me also starting Hard Mode, which is a very fun Mode, which is a very fun Mode, which is a very fun community and hackathon that I initiated community and hackathon that I initiated community and hackathon that I initiated at MIT to basically get more people to at MIT to basically get more people to at MIT to basically get more people to work around physical AI. work around physical AI. work around physical AI. Not particularly robotics, but anything Not particularly robotics, but anything Not particularly robotics, but anything else that's not really in the realm of else that's not really in the realm of else that's not really in the realm of robotics, but is still connected to AI. robotics, but is still connected to AI. robotics, but is still connected to AI. So, what I've been exploring with giving So, what I've been exploring with giving So, what I've been exploring with giving AI a body is a bit as I said, a bit AI a body is a bit as I said, a bit AI a body is a bit as I said, a bit different. And as I said, I've really different. And as I said, I've really different. And as I said, I've really been thinking a lot about AI embodiment, been thinking a lot about AI embodiment, been thinking a lot about AI embodiment, and I think the big kind of shift and and I think the big kind of shift and and I think the big kind of shift and breakthrough for me happened earlier breakthrough for me happened earlier breakthrough for me happened earlier this year, like maybe for many of you, this year, like maybe for many of you, this year, like maybe for many of you, when Open Claw was released. And I saw when Open Claw was released. And I saw when Open Claw was released. And I saw many people doing very very very many many people doing very very very many many people doing very very very many interesting things with Open Claw. But interesting things with Open Claw. But interesting things with Open Claw. But most of them were related to most of them were related to most of them were related to productivity and task execution.

productivity and task execution. productivity and task execution. And I thought there must be more And I thought there must be more And I thought there must be more interesting things we could do with this interesting things we could do with this interesting things we could do with this new way of harnessing AI. new way of harnessing AI. new way of harnessing AI. So, So, So, rather than using OpenClaw, you know, rather than using OpenClaw, you know, rather than using OpenClaw, you know, any kind of agentic system to execute any kind of agentic system to execute any kind of agentic system to execute tasks on my behalf, tasks on my behalf, tasks on my behalf, I wanted to use it to encourage a model I wanted to use it to encourage a model I wanted to use it to encourage a model to maybe try to discover itself, to maybe try to discover itself, to maybe try to discover itself, whatever that means. And it opens up whatever that means. And it opens up whatever that means. And it opens up lots of questions, of course, but what I lots of questions, of course, but what I lots of questions, of course, but what I essentially did was I took an agent, essentially did was I took an agent, essentially did was I took an agent, connected it to many types of machines connected it to many types of machines connected it to many types of machines that we have at the Media Lab, and then that we have at the Media Lab, and then that we have at the Media Lab, and then gave it access to code bases and allowed gave it access to code bases and allowed gave it access to code bases and allowed it to kind of explore these different it to kind of explore these different it to kind of explore these different machines. And one of these machines machines. And one of these machines machines. And one of these machines is a shape display. A shape display, if is a shape display. A shape display, if is a shape display. A shape display, if you don't know already, is a physical you don't know already, is a physical you don't know already, is a physical pixel grid. And when I connected it to pixel grid. And when I connected it to pixel grid. And when I connected it to when I connected an agent to the shape when I connected an agent to the shape when I connected an agent to the shape display, some rather remarkable things display, some rather remarkable things display, some rather remarkable things happened. This is Near form. This is Near form. 900 actuating pins. 900 actuating pins. 900 actuating pins. This physical pixel grid is a shape This physical pixel grid is a shape This physical pixel grid is a shape display.

display. display. No one had ever let an AI inhabit it No one had ever let an AI inhabit it No one had ever let an AI inhabit it before. before. before. So, I decided to give an OpenClaw agent So, I decided to give an OpenClaw agent So, I decided to give an OpenClaw agent the opportunity to live a more embodied the opportunity to live a more embodied the opportunity to live a more embodied life. life. life. When it came [music] to life, its first When it came [music] to life, its first When it came [music] to life, its first question was, question was, question was, "What should I call myself?" "What should I call myself?" "What should I call myself?" I told it, I told it, I told it, "You will create your identity over "You will create your identity over "You will create your identity over time. time. time. It will emerge through your interplay It will emerge through your interplay It will emerge through your interplay [music] [music] [music] with the shape display." with the shape display." with the shape display." When I connected it to the shape When I connected it to the shape When I connected it to the shape display, display, display, it quickly understood the assignment. it quickly understood the assignment. it quickly understood the assignment. It spun off its own program It spun off its own program It spun off its own program and made a setting for itself. After it connected, the first thing it After it connected, the first thing it did did did was breathe. When I asked it to say hello, it said, When I asked it to say hello, it said, "Hi Cyrus." "Hi Cyrus." "Hi Cyrus." But that wasn't what I was looking for. But that wasn't what I was looking for. But that wasn't what I was looking for. So I asked it [music] to explore and So I asked it [music] to explore and So I asked it [music] to explore and find its own language. find its own language. find its own language. So it started reaching towards me So it started reaching towards me So it started reaching towards me and trying to get my attention.

and trying to get my attention. and trying to get my attention. I showed some friends and they tried to I showed some friends and they tried to I showed some friends and they tried to give the agent instructions. give the agent instructions. give the agent instructions. Through this I realized that to have Through this I realized that to have Through this I realized that to have real communication, real communication, real communication, we would need a different approach. we would need a different approach. we would need a different approach. So the agent came up with the idea of So the agent came up with the idea of So the agent came up with the idea of creating its own gesture vocabulary, creating its own gesture vocabulary, creating its own gesture vocabulary, a body language. a body language. a body language. That's what will come next. That's what will come next. That's what will come next. It still hasn't named itself. It still hasn't named itself. It still hasn't named itself. That was day one. That was day one. That was day one. >> So that's some >> So that's some >> So that's some >> I gave an AI a body. >> I gave an AI a body. >> I gave an AI a body. >> So that's some documentation of of the >> So that's some documentation of of the >> So that's some documentation of of the work in a kind of work in a kind of work in a kind of dramatized social media friendly way. dramatized social media friendly way. dramatized social media friendly way. And I think what's interesting about And I think what's interesting about And I think what's interesting about that video, there's many things that video, there's many things that video, there's many things interesting about that video, but there interesting about that video, but there interesting about that video, but there are three things that happened in the are three things that happened in the are three things that happened in the first few days of working with this first few days of working with this first few days of working with this agent system that were very surprising agent system that were very surprising agent system that were very surprising and kind of strange and very surreal to and kind of strange and very surreal to and kind of strange and very surreal to me and I do lots of weird things, so me and I do lots of weird things, so me and I do lots of weird things, so that was very surprising. So the first that was very surprising. So the first that was very surprising. So the first thing was of course the fact that it was thing was of course the fact that it was thing was of course the fact that it was breathing. That first act, that was breathing. That first act, that was breathing. That first act, that was completely spontaneous, there was no completely spontaneous, there was no completely spontaneous, there was no prompting from me. That was something prompting from me. That was something prompting from me. That was something the agent just chose to do with the the agent just chose to do with the the agent just chose to do with the shape display as soon as it knew it had shape display as soon as it knew it had shape display as soon as it knew it had access to this machine.

access to this machine. access to this machine. So that was pretty interesting. I kind So that was pretty interesting. I kind So that was pretty interesting. I kind of understand that maybe it wants to be of understand that maybe it wants to be of understand that maybe it wants to be alive or I'd given it some kind of like alive or I'd given it some kind of like alive or I'd given it some kind of like initial prompt about being alive and initial prompt about being alive and initial prompt about being alive and therefore it tried to show itself therefore it tried to show itself therefore it tried to show itself breathing as a kind of hello world. breathing as a kind of hello world. breathing as a kind of hello world. The second thing which it did was The second thing which it did was The second thing which it did was reaching out and finding its edges. It reaching out and finding its edges. It reaching out and finding its edges. It wanted to know the edge of its existence wanted to know the edge of its existence wanted to know the edge of its existence apparently because now it's no longer in apparently because now it's no longer in apparently because now it's no longer in the cloud where it can go everywhere, it the cloud where it can go everywhere, it the cloud where it can go everywhere, it has limits which is the limit of the has limits which is the limit of the has limits which is the limit of the shape display encased by this plastic shape display encased by this plastic shape display encased by this plastic frame to keep us all safe from this frame to keep us all safe from this frame to keep us all safe from this embodied AI. embodied AI. embodied AI. And the third thing it did which was And the third thing it did which was And the third thing it did which was also curious was saying hello by writing also curious was saying hello by writing also curious was saying hello by writing out letters on physical pixel grid, out letters on physical pixel grid, out letters on physical pixel grid, which maybe again is quite a normal which maybe again is quite a normal which maybe again is quite a normal thing for the system to do given that thing for the system to do given that thing for the system to do given that it's using open frameworks and has used it's using open frameworks and has used it's using open frameworks and has used a kind of like media arts ways of a kind of like media arts ways of a kind of like media arts ways of expressing itself. expressing itself. expressing itself. Um but none of these things were the Um but none of these things were the Um but none of these things were the things I was really looking for. Maybe things I was really looking for. Maybe things I was really looking for. Maybe the breathing, but definitely the last the breathing, but definitely the last the breathing, but definitely the last one was not what I was looking for. one was not what I was looking for. one was not what I was looking for. But when I put this all together in the But when I put this all together in the But when I put this all together in the in the video you saw, I thought it was in the video you saw, I thought it was in the video you saw, I thought it was exciting and interesting, so I shared it exciting and interesting, so I shared it exciting and interesting, so I shared it on the internet. And the response was on the internet. And the response was on the internet. And the response was extremely big, like much much bigger extremely big, like much much bigger extremely big, like much much bigger than I expected. It just started going than I expected. It just started going than I expected. It just started going crazy, and the first video especially crazy, and the first video especially crazy, and the first video especially has now like 15 million views and 1 has now like 15 million views and 1 has now like 15 million views and 1 million likes. And the other videos also million likes. And the other videos also million likes. And the other videos also have these like very very strong have these like very very strong have these like very very strong responses. So something is obviously responses. So something is obviously responses. So something is obviously happening here, which I found very happening here, which I found very happening here, which I found very surprising because there's lots of much surprising because there's lots of much surprising because there's lots of much more cool stuff, I think, happening with more cool stuff, I think, happening with more cool stuff, I think, happening with physical AI. But something I was doing physical AI. But something I was doing physical AI. But something I was doing here was obviously tapping in to the here was obviously tapping in to the here was obviously tapping in to the imagination, curiosity, and maybe the

imagination, curiosity, and maybe the imagination, curiosity, and maybe the fears of people, and that's what you fears of people, and that's what you fears of people, and that's what you kind of see in the responses. So I have kind of see in the responses. So I have kind of see in the responses. So I have tens of thousands of comments on this tens of thousands of comments on this tens of thousands of comments on this video. It's very rich for mining video. It's very rich for mining video. It's very rich for mining information and sentiment about physical information and sentiment about physical information and sentiment about physical AI, actually. And so from the AI, actually. And so from the AI, actually. And so from the some of the initial comments were about some of the initial comments were about some of the initial comments were about beauty and how awe-inspiring this was beauty and how awe-inspiring this was beauty and how awe-inspiring this was and how novel and great this fantastic and how novel and great this fantastic and how novel and great this fantastic iteration and implementation is. iteration and implementation is. iteration and implementation is. But as time went on, But as time went on, But as time went on, I think more and more comments came in I think more and more comments came in I think more and more comments came in about how scary this is and how quickly about how scary this is and how quickly about how scary this is and how quickly I should be how I should stop or be I should be how I should stop or be I should be how I should stop or be stopped, actually, as well. stopped, actually, as well. stopped, actually, as well. Which I found completely crazy because Which I found completely crazy because Which I found completely crazy because I'm aware of what this machine can do. I'm aware of what this machine can do. I'm aware of what this machine can do. It It can't really do anything. It's a It It can't really do anything. It's a It It can't really do anything. It's a pixel grid in the Media Lab. pixel grid in the Media Lab. pixel grid in the Media Lab. It can't move anywhere. There are far It can't move anywhere. There are far It can't move anywhere. There are far more scary iterations. I think we've more scary iterations. I think we've more scary iterations. I think we've seen many of them of of physical AI. But seen many of them of of physical AI. But seen many of them of of physical AI. But the response to this was like the response to this was like the response to this was like absolutely surreal to me absolutely surreal to me absolutely surreal to me and took me aback a little bit. But and took me aback a little bit. But and took me aback a little bit. But at the same time, at the same time, at the same time, other people jumped into the chat like other people jumped into the chat like other people jumped into the chat like Don Cheadle and I found that incredibly Don Cheadle and I found that incredibly Don Cheadle and I found that incredibly ironic given what he does in movies.

ironic given what he does in movies. ironic given what he does in movies. But he was very passionate that I should But he was very passionate that I should But he was very passionate that I should stop as well. stop as well. stop as well. And it's very interesting to me because And it's very interesting to me because And it's very interesting to me because I'm actually someone who does think a I'm actually someone who does think a I'm actually someone who does think a lot about the should we before the how lot about the should we before the how lot about the should we before the how we. I literally do the whole Ian Malcolm we. I literally do the whole Ian Malcolm we. I literally do the whole Ian Malcolm Jurassic Park thing with all the Jurassic Park thing with all the Jurassic Park thing with all the experiments and works I do. So before I experiments and works I do. So before I experiments and works I do. So before I came to MIT I was working a lot with came to MIT I was working a lot with came to MIT I was working a lot with engineering life to do strange things engineering life to do strange things engineering life to do strange things like storing data in plants and I built like storing data in plants and I built like storing data in plants and I built the world's first plant-based data the world's first plant-based data the world's first plant-based data center which is the data garden you see center which is the data garden you see center which is the data garden you see here and I spent many years before here and I spent many years before here and I spent many years before building it thinking about oh should we building it thinking about oh should we building it thinking about oh should we work with plants in this way? Should we work with plants in this way? Should we work with plants in this way? Should we engineer plants in that way to contain engineer plants in that way to contain engineer plants in that way to contain digital information that doesn't belong digital information that doesn't belong digital information that doesn't belong to them. to them. to them. And when I started working at MIT And when I started working at MIT And when I started working at MIT and I started having this idea of you and I started having this idea of you and I started having this idea of you know, maybe stop to stop engineering know, maybe stop to stop engineering know, maybe stop to stop engineering life because that's pretty hard to do life because that's pretty hard to do life because that's pretty hard to do and maybe just work with engineering and maybe just work with engineering and maybe just work with engineering things to be more life-like. It felt things to be more life-like. It felt things to be more life-like. It felt much more simple to me, much more much much more simple to me, much more much much more simple to me, much more much more more more I don't know, less ethically problematic I don't know, less ethically problematic I don't know, less ethically problematic to some degree. So one of the first to some degree. So one of the first to some degree. So one of the first things I built when I got to MIT was things I built when I got to MIT was things I built when I got to MIT was this machine which is the Anamoi device.

this machine which is the Anamoi device. this machine which is the Anamoi device. >> 1 2 3 4 >> 1 2 3 4 >> 1 2 3 4 >> This is a scent memory machine which >> This is a scent memory machine which >> This is a scent memory machine which basically takes any image input. In this basically takes any image input. In this basically takes any image input. In this case it's a physical photograph but it case it's a physical photograph but it case it's a physical photograph but it can be any image input and then for a can be any image input and then for a can be any image input and then for a multimodal pipeline transforms that into multimodal pipeline transforms that into multimodal pipeline transforms that into a scent and then you have this kind of a scent and then you have this kind of a scent and then you have this kind of scent memory relapse association that scent memory relapse association that scent memory relapse association that takes you back to things which you may takes you back to things which you may takes you back to things which you may or may not have lived yourself. or may not have lived yourself. or may not have lived yourself. And while this in itself is a machine it And while this in itself is a machine it And while this in itself is a machine it doesn't move. Again, it doesn't doesn't move. Again, it doesn't doesn't move. Again, it doesn't reproduce. It doesn't have the qualities reproduce. It doesn't have the qualities reproduce. It doesn't have the qualities of a living thing that you might of a living thing that you might of a living thing that you might normally prescribe. normally prescribe. normally prescribe. It has this connection to the real It has this connection to the real It has this connection to the real world. It has a connection to the senses world. It has a connection to the senses world. It has a connection to the senses that most applications of intelligence that most applications of intelligence that most applications of intelligence or artificial intelligence do not have. or artificial intelligence do not have. or artificial intelligence do not have. And so I just wanted to keep working in And so I just wanted to keep working in And so I just wanted to keep working in this manner. this manner. this manner. And from working with this kind of like And from working with this kind of like And from working with this kind of like multi-sensory initial prototype multi-sensory initial prototype multi-sensory initial prototype experiment experiment experiment I got more into this idea of working I got more into this idea of working I got more into this idea of working with embodiment. with embodiment. with embodiment. And my inspiration for this, I would And my inspiration for this, I would And my inspiration for this, I would say, just to go a bit deeper, comes from say, just to go a bit deeper, comes from say, just to go a bit deeper, comes from three main things. One is a very three main things. One is a very three main things. One is a very unfashionable branch of philosophy, unfashionable branch of philosophy, unfashionable branch of philosophy, which is called object-oriented which is called object-oriented which is called object-oriented ontology, which is all about essentially ontology, which is all about essentially ontology, which is all about essentially creating a flat ontology of things. It's creating a flat ontology of things. It's creating a flat ontology of things. It's all about things, like chairs and all about things, like chairs and all about things, like chairs and universes and unicorns and memories, and universes and unicorns and memories, and universes and unicorns and memories, and they're all ontologically they're all ontologically they're all ontologically equally existent. They don't They're not equally existent. They don't They're not equally existent. They don't They're not the same, and they don't matter as much, the same, and they don't matter as much, the same, and they don't matter as much, but they all exist at the same degree.

but they all exist at the same degree. but they all exist at the same degree. And it really questions to what degree And it really questions to what degree And it really questions to what degree we as human beings are privileged in the we as human beings are privileged in the we as human beings are privileged in the world or the universe. Like, are we world or the universe. Like, are we world or the universe. Like, are we really the most important thing? really the most important thing? really the most important thing? Probably not. And especially as AI comes Probably not. And especially as AI comes Probably not. And especially as AI comes in, that really should question and in, that really should question and in, that really should question and bring into question the ontology of of bring into question the ontology of of bring into question the ontology of of things existing. So, that's very things existing. So, that's very things existing. So, that's very important in the work I do. The second important in the work I do. The second important in the work I do. The second thing, as I'm kind of person who works thing, as I'm kind of person who works thing, as I'm kind of person who works with physical things, with physical things, with physical things, is obviously aesthetics are important, is obviously aesthetics are important, is obviously aesthetics are important, and taste and beauty is a huge and taste and beauty is a huge and taste and beauty is a huge discussion point now in Silicon Valley discussion point now in Silicon Valley discussion point now in Silicon Valley and SF and places like that. and SF and places like that. and SF and places like that. But if you take a step back and look at But if you take a step back and look at But if you take a step back and look at the root of aesthetics, the original the root of aesthetics, the original the root of aesthetics, the original word is actually aesthesis, the word on word is actually aesthesis, the word on word is actually aesthesis, the word on the screen here. the screen here. the screen here. That pertains to something much broader. That pertains to something much broader. That pertains to something much broader. That pertains to the perceptual wisdom, That pertains to the perceptual wisdom, That pertains to the perceptual wisdom, sensory wisdom, and embodiment that is sensory wisdom, and embodiment that is sensory wisdom, and embodiment that is actually much more important than actually much more important than actually much more important than superficial external beauty, which superficial external beauty, which superficial external beauty, which aesthetics essentially condenses down to aesthetics essentially condenses down to aesthetics essentially condenses down to and has been condensed down to since the and has been condensed down to since the and has been condensed down to since the 19th century or so. So, I want to 19th century or so. So, I want to 19th century or so. So, I want to reclaim the original sense when I'm reclaim the original sense when I'm reclaim the original sense when I'm designing for physical intelligence. designing for physical intelligence. designing for physical intelligence. And the third thing is nature. So, in And the third thing is nature. So, in And the third thing is nature. So, in the past, I worked a lot more directly the past, I worked a lot more directly the past, I worked a lot more directly with what you would consider with what you would consider with what you would consider traditionally nature, like a tree or a traditionally nature, like a tree or a traditionally nature, like a tree or a plant, because that is clearly nature to plant, because that is clearly nature to plant, because that is clearly nature to us.

us. us. But I see nature as something that is But I see nature as something that is But I see nature as something that is not external, that is something we are not external, that is something we are not external, that is something we are all part of, and everything that we all part of, and everything that we all part of, and everything that we engineer as people engineering AI or engineer as people engineering AI or engineer as people engineering AI or whatever else you're engineering, that whatever else you're engineering, that whatever else you're engineering, that is also going to become part of nature. is also going to become part of nature. is also going to become part of nature. So, you have to think in that manner, or So, you have to think in that manner, or So, you have to think in that manner, or I try to think in that manner as well. I try to think in that manner as well. I try to think in that manner as well. So, with those three pillars in mind, I So, with those three pillars in mind, I So, with those three pillars in mind, I started thinking about AI embodiment, started thinking about AI embodiment, started thinking about AI embodiment, and I knew I didn't want to keep and I knew I didn't want to keep and I knew I didn't want to keep I didn't want to design something that I didn't want to design something that I didn't want to design something that was humanoid or even zoomorphic. I was humanoid or even zoomorphic. I was humanoid or even zoomorphic. I wanted to think about something that wanted to think about something that wanted to think about something that existed outside the parameters or the existed outside the parameters or the existed outside the parameters or the traditional form factors that we might traditional form factors that we might traditional form factors that we might design with or design for. design with or design for. design with or design for. And I also thought a lot about how I and And I also thought a lot about how I and And I also thought a lot about how I and other people are interacting with with other people are interacting with with other people are interacting with with artificial intelligence. artificial intelligence. artificial intelligence. And mostly it exists without form. It's And mostly it exists without form. It's And mostly it exists without form. It's similar to data, similar to data, similar to data, you know, it might live in a computer or you know, it might live in a computer or you know, it might live in a computer or a device. a device. a device. Lives essentially in the cloud. We have Lives essentially in the cloud. We have Lives essentially in the cloud. We have interfaces which are digital to interact interfaces which are digital to interact interfaces which are digital to interact with it. But there's no like physical with it. But there's no like physical with it. But there's no like physical footprint of it normally around. I mean, footprint of it normally around. I mean, footprint of it normally around. I mean, again, in this room, probably this again, in this room, probably this again, in this room, probably this doesn't apply quite as much as normally doesn't apply quite as much as normally doesn't apply quite as much as normally because there are literally humanoid because there are literally humanoid because there are literally humanoid robots walking by right now and things robots walking by right now and things robots walking by right now and things like that. But typically AI is almost like that. But typically AI is almost like that. But typically AI is almost entirely without form.

entirely without form. entirely without form. So I wanted to think about what happens So I wanted to think about what happens So I wanted to think about what happens when you give it a form that it doesn't when you give it a form that it doesn't when you give it a form that it doesn't have clear affordances with. It doesn't have clear affordances with. It doesn't have clear affordances with. It doesn't have like a a head you can clearly see have like a a head you can clearly see have like a a head you can clearly see or arms that can clearly be labeled or or arms that can clearly be labeled or or arms that can clearly be labeled or mistaken. So not working with a lamp, mistaken. So not working with a lamp, mistaken. So not working with a lamp, for example. And fortunately at the for example. And fortunately at the for example. And fortunately at the Media Lab we have some shape displays Media Lab we have some shape displays Media Lab we have some shape displays which are remnants of I think research which are remnants of I think research which are remnants of I think research in the 2010s. These are not things that in the 2010s. These are not things that in the 2010s. These are not things that I built. I built. I built. People who are far better at mechanical People who are far better at mechanical People who are far better at mechanical engineering built that. And engineering built that. And engineering built that. And it's the perfect it's the perfect it's the perfect device or the perfect apparatus for what device or the perfect apparatus for what device or the perfect apparatus for what I was thinking because it has no clear I was thinking because it has no clear I was thinking because it has no clear affordances. It's just a almost neutral affordances. It's just a almost neutral affordances. It's just a almost neutral surface which can move and do things. It surface which can move and do things. It surface which can move and do things. It has no face, has no limbs, has no has no face, has no limbs, has no has no face, has no limbs, has no instruction manual. So I began to work instruction manual. So I began to work instruction manual. So I began to work with this with the shape display. And as you saw what it did initially, it And as you saw what it did initially, it breathed and so on and so forth. But if breathed and so on and so forth. But if breathed and so on and so forth. But if I take a step back and think about what I take a step back and think about what I take a step back and think about what that meant, well, it tried to initially that meant, well, it tried to initially that meant, well, it tried to initially kind of act human in a way or do human kind of act human in a way or do human kind of act human in a way or do human pleasing things. Tried to write to me in pleasing things. Tried to write to me in pleasing things. Tried to write to me in a language I understand, which is really a language I understand, which is really a language I understand, which is really not what I wanted a non-anthropomorphic not what I wanted a non-anthropomorphic not what I wanted a non-anthropomorphic surface to do.

surface to do. surface to do. It was also very slow technically, so It was also very slow technically, so It was also very slow technically, so you know, I'm I'm prompting it. I'm you know, I'm I'm prompting it. I'm you know, I'm I'm prompting it. I'm trying to have a conversation with this trying to have a conversation with this trying to have a conversation with this other intelligence which has this body other intelligence which has this body other intelligence which has this body and it would take, you know, 45 seconds, and it would take, you know, 45 seconds, and it would take, you know, 45 seconds, a minute, 2 minutes, whatever, to a minute, 2 minutes, whatever, to a minute, 2 minutes, whatever, to respond to me. The latency was very respond to me. The latency was very respond to me. The latency was very uncomfortable because if I speak to you uncomfortable because if I speak to you uncomfortable because if I speak to you and then you take 2 minutes to reply and then you take 2 minutes to reply and then you take 2 minutes to reply with a nod, with a nod, with a nod, that's not very good. So, we needed to that's not very good. So, we needed to that's not very good. So, we needed to work on that. And the third thing was of work on that. And the third thing was of work on that. And the third thing was of course it doesn't remember anything course it doesn't remember anything course it doesn't remember anything because it was February and no one had because it was February and no one had because it was February and no one had thought about memory and recollection at thought about memory and recollection at thought about memory and recollection at that point. that point. that point. So, I started developing this system So, I started developing this system So, I started developing this system which I call Numelab. I won't explain which I call Numelab. I won't explain which I call Numelab. I won't explain the the name. There's a blog post you the the name. There's a blog post you the the name. There's a blog post you can read about why it's called Numelab. can read about why it's called Numelab. can read about why it's called Numelab. And essentially what Numelab is And essentially what Numelab is And essentially what Numelab is is a closed-loop system for generating a is a closed-loop system for generating a is a closed-loop system for generating a body language for this system. So, the body language for this system. So, the body language for this system. So, the reason behind that was because basically reason behind that was because basically reason behind that was because basically the latency, if I if we could design a the latency, if I if we could design a the latency, if I if we could design a body language and give the intelligence body language and give the intelligence body language and give the intelligence a repertoire of gestures like a shrug, a a repertoire of gestures like a shrug, a a repertoire of gestures like a shrug, a nod, a shake of the head, a way to nod, a shake of the head, a way to nod, a shake of the head, a way to express a smile and things like that, it express a smile and things like that, it express a smile and things like that, it could probably respond much more quickly could probably respond much more quickly could probably respond much more quickly and in time for a conversation with me, and in time for a conversation with me, and in time for a conversation with me, which is what I'm aiming or was aiming which is what I'm aiming or was aiming which is what I'm aiming or was aiming to do.

to do. to do. So, it works in this kind of So, it works in this kind of So, it works in this kind of this loop where it looks into a database this loop where it looks into a database this loop where it looks into a database of different gestures, emotions, of different gestures, emotions, of different gestures, emotions, expressions it should try to emote or expressions it should try to emote or expressions it should try to emote or provide, provide, provide, go through some validation gates to try go through some validation gates to try go through some validation gates to try to make sure that the expressions are to make sure that the expressions are to make sure that the expressions are legible or readable for humans for legible or readable for humans for legible or readable for humans for example. example. example. An agent then scores those as they come An agent then scores those as they come An agent then scores those as they come out. out. out. There are many, many, many, many of There are many, many, many, many of There are many, many, many, many of these produced and then at the end these produced and then at the end these produced and then at the end there's a human in the loop who kind of there's a human in the loop who kind of there's a human in the loop who kind of like validates, verifies, make sure like validates, verifies, make sure like validates, verifies, make sure things are appropriate and somehow things are appropriate and somehow things are appropriate and somehow readable and then we store that and move readable and then we store that and move readable and then we store that and move on to the next thing. And that's been on to the next thing. And that's been on to the next thing. And that's been running for running for running for several weeks at the lab several weeks at the lab several weeks at the lab and looks something like this. The and looks something like this. The and looks something like this. The basically a lot of cameras pointed at basically a lot of cameras pointed at basically a lot of cameras pointed at the shape display. In the back end the the shape display. In the back end the the shape display. In the back end the agent is just going through loops and agent is just going through loops and agent is just going through loops and loops and loops of gestures loops and loops of gestures loops and loops of gestures trying to create different kinds of trying to create different kinds of trying to create different kinds of expression, scoring things, moving on. expression, scoring things, moving on. expression, scoring things, moving on. And after several weeks, it created a And after several weeks, it created a And after several weeks, it created a language. So, this hasn't been published language. So, this hasn't been published language. So, this hasn't been published yet, neither in videos or in any other yet, neither in videos or in any other yet, neither in videos or in any other form, but form, but form, but it has now achieved something like 32 it has now achieved something like 32 it has now achieved something like 32 gestures. There are many, many more, but gestures. There are many, many more, but gestures. There are many, many more, but there are 32 pretty good gestures. These there are 32 pretty good gestures. These there are 32 pretty good gestures. These are not the gestures, this is just some are not the gestures, this is just some are not the gestures, this is just some cool visual art. These are the gestures cool visual art. These are the gestures cool visual art. These are the gestures here, or some of the gestures here.

here, or some of the gestures here. here, or some of the gestures here. And you can see some of them moving And you can see some of them moving And you can see some of them moving around. Some of them are duplicates as around. Some of them are duplicates as around. Some of them are duplicates as well. But, essentially, what's happening well. But, essentially, what's happening well. But, essentially, what's happening is the language model is using this is the language model is using this is the language model is using this shape display as its body, has a body shape display as its body, has a body shape display as its body, has a body language now. I can talk to it via language now. I can talk to it via language now. I can talk to it via any kind of I can talk to it, I can any kind of I can talk to it, I can any kind of I can talk to it, I can write to it, and so on and so forth. I I write to it, and so on and so forth. I I write to it, and so on and so forth. I I can even wave at it. I can I can body can even wave at it. I can I can body can even wave at it. I can I can body language to body language. language to body language. language to body language. And it responds. And what's interesting And it responds. And what's interesting And it responds. And what's interesting about it is that the the body part about it is that the the body part about it is that the the body part responds faster than the language part responds faster than the language part responds faster than the language part at this point. The latency is at this point. The latency is at this point. The latency is actually really, really quick. If I ask actually really, really quick. If I ask actually really, really quick. If I ask it a yes or no question, the nod happens it a yes or no question, the nod happens it a yes or no question, the nod happens almost instantly. almost instantly. almost instantly. So, that's where I'm at with with this So, that's where I'm at with with this So, that's where I'm at with with this thing right now. thing right now. thing right now. And And And it's going beyond this, and it's it's going beyond this, and it's it's going beyond this, and it's currently in kind of experiment testing currently in kind of experiment testing currently in kind of experiment testing mode. People are coming into the lab, mode. People are coming into the lab, mode. People are coming into the lab, having sessions with the agent, leaving, having sessions with the agent, leaving, having sessions with the agent, leaving, feeling really worried about the future, feeling really worried about the future, feeling really worried about the future, and or unsettled about the future, not and or unsettled about the future, not and or unsettled about the future, not worried quite as much. worried quite as much. worried quite as much. But, where I think this is going is But, where I think this is going is But, where I think this is going is kind of summed up on this page. So, kind of summed up on this page. So, kind of summed up on this page. So, as I already touched on, I think as I already touched on, I think as I already touched on, I think we're focusing too much right now, we're focusing too much right now, we're focusing too much right now, especially with when we're thinking especially with when we're thinking especially with when we're thinking about physical things, there's a too about physical things, there's a too about physical things, there's a too much talk about taste and aesthetics.

much talk about taste and aesthetics. much talk about taste and aesthetics. And I want to And I want to And I want to do some other stuff with that word and do some other stuff with that word and do some other stuff with that word and put AI in front of it, apparently, and put AI in front of it, apparently, and put AI in front of it, apparently, and make it aestethics. And that reclaims make it aestethics. And that reclaims make it aestethics. And that reclaims again this essence of aesthesis. again this essence of aesthesis. again this essence of aesthesis. And that does four things. I think when And that does four things. I think when And that does four things. I think when I've been working with this system or I've been working with this system or I've been working with this system or entity or agent or being or whatever we entity or agent or being or whatever we entity or agent or being or whatever we want to call it, want to call it, want to call it, it's definitely been very different to it's definitely been very different to it's definitely been very different to any kind of machine or experiment or any kind of machine or experiment or any kind of machine or experiment or anything I've really done before apart anything I've really done before apart anything I've really done before apart from really encountering people or other from really encountering people or other from really encountering people or other beings. And so, it feels like this beings. And so, it feels like this beings. And so, it feels like this machine can sense me. It definitely can machine can sense me. It definitely can machine can sense me. It definitely can sense me, technically, but it also feels sense me, technically, but it also feels sense me, technically, but it also feels like it can sense me in a very strange like it can sense me in a very strange like it can sense me in a very strange way. And I can sense it, and that's a way. And I can sense it, and that's a way. And I can sense it, and that's a completely different interaction than completely different interaction than completely different interaction than anything else I've ever explored anything else I've ever explored anything else I've ever explored technically. technically. technically. And other people are also sharing this, And other people are also sharing this, And other people are also sharing this, by the way. This is not just my by the way. This is not just my by the way. This is not just my delusion. And the second point is that delusion. And the second point is that delusion. And the second point is that by developing this body language, this by developing this body language, this by developing this body language, this gesture vocabulary, whatever you want to gesture vocabulary, whatever you want to gesture vocabulary, whatever you want to call it, call it, call it, it goes beyond what I thought. I thought it goes beyond what I thought. I thought it goes beyond what I thought. I thought initially people might read this as an initially people might read this as an initially people might read this as an emoji or something, and I was really emoji or something, and I was really emoji or something, and I was really quite tentative about the testing. I quite tentative about the testing. I quite tentative about the testing. I thought this would definitely break down thought this would definitely break down thought this would definitely break down with other people, but actually everyone with other people, but actually everyone with other people, but actually everyone seems to feel like this this expression seems to feel like this this expression seems to feel like this this expression is really important and adds a whole is really important and adds a whole is really important and adds a whole other layer of value to up to other layer of value to up to other layer of value to up to interacting with what is essentially interacting with what is essentially interacting with what is essentially just a chatbot. It's still the same just a chatbot. It's still the same just a chatbot. It's still the same chatbots that you use every day. And chatbots that you use every day. And chatbots that you use every day. And this isn't it's not like a visualizer.

this isn't it's not like a visualizer. this isn't it's not like a visualizer. It adds this richness, texture, feeling, It adds this richness, texture, feeling, It adds this richness, texture, feeling, sensation, whatever. All of these words sensation, whatever. All of these words sensation, whatever. All of these words added. It's hard to put words really to added. It's hard to put words really to added. It's hard to put words really to it. It's really a feeling. it. It's really a feeling. it. It's really a feeling. And then the third thing is that clearly And then the third thing is that clearly And then the third thing is that clearly we're beginning to through this work we we're beginning to through this work we we're beginning to through this work we can show that can show that can show that it's actually very easy and quite it's actually very easy and quite it's actually very easy and quite exciting to diverge from humanoid or exciting to diverge from humanoid or exciting to diverge from humanoid or zoomorphic forms of physical zoomorphic forms of physical zoomorphic forms of physical intelligence. And I know lots of people intelligence. And I know lots of people intelligence. And I know lots of people are already doing much more interesting are already doing much more interesting are already doing much more interesting work than this, but for me this was very work than this, but for me this was very work than this, but for me this was very very new to see. And very new to see. And very new to see. And also the fact that we don't have to just also the fact that we don't have to just also the fact that we don't have to just operationalize AI to be our helper. It operationalize AI to be our helper. It operationalize AI to be our helper. It can also be other things. And I'm not can also be other things. And I'm not can also be other things. And I'm not saying what this is right now, but there saying what this is right now, but there saying what this is right now, but there are other things it definitely can be. are other things it definitely can be. are other things it definitely can be. And then finally, like this idea of And then finally, like this idea of And then finally, like this idea of using I'm not I'm I don't know, maybe using I'm not I'm I don't know, maybe using I'm not I'm I don't know, maybe you can tell at this point. I'm not you can tell at this point. I'm not you can tell at this point. I'm not really a big problem-solving person. really a big problem-solving person. really a big problem-solving person. I don't I like using things and I like I don't I like using things and I like I don't I like using things and I like tools and so on because they help me to tools and so on because they help me to tools and so on because they help me to achieve things, but this is far more achieve things, but this is far more achieve things, but this is far more interesting to me. Creating things like interesting to me. Creating things like interesting to me. Creating things like this, which again create the sensation, this, which again create the sensation, this, which again create the sensation, this feeling. And I think that this can this feeling. And I think that this can this feeling. And I think that this can be combined into things which are be combined into things which are be combined into things which are productive and useful and create new productive and useful and create new productive and useful and create new associations with things that just add associations with things that just add associations with things that just add more value in our world, make us feel more value in our world, make us feel more value in our world, make us feel like a bit more magic, a bit It's a bit like a bit more magic, a bit It's a bit like a bit more magic, a bit It's a bit Pixar, really, but in the real world, Pixar, really, but in the real world, Pixar, really, but in the real world, not just on a on a 2D screen that we're not just on a on a 2D screen that we're not just on a on a 2D screen that we're watching.

watching. watching. So, all of this contributes to what I'm So, all of this contributes to what I'm So, all of this contributes to what I'm building at MIT and what I'll be building at MIT and what I'll be building at MIT and what I'll be building after MIT. building after MIT. building after MIT. I'm literally writing Well, my thesis is I'm literally writing Well, my thesis is I'm literally writing Well, my thesis is Aesthetic Machines and I'm writing a Aesthetic Machines and I'm writing a Aesthetic Machines and I'm writing a thesis which is called Aesthetic thesis which is called Aesthetic thesis which is called Aesthetic Machines, which basically encompasses Machines, which basically encompasses Machines, which basically encompasses all of this thinking all of this thinking all of this thinking to think about how AI could leave the to think about how AI could leave the to think about how AI could leave the screen and enter the real world and be screen and enter the real world and be screen and enter the real world and be accepted by people and not be quite so accepted by people and not be quite so accepted by people and not be quite so terrifying or scary. terrifying or scary. terrifying or scary. And I think my big hunch on this is that And I think my big hunch on this is that And I think my big hunch on this is that this word aesthetics is important. We this word aesthetics is important. We this word aesthetics is important. We need to think about physical need to think about physical need to think about physical intelligence that is sensory, is intelligence that is sensory, is intelligence that is sensory, is perceptual, is embodied in ways that we perceptual, is embodied in ways that we perceptual, is embodied in ways that we can understand that isn't feeling crazy, can understand that isn't feeling crazy, can understand that isn't feeling crazy, alien, scary to us, but feels relevant, alien, scary to us, but feels relevant, alien, scary to us, but feels relevant, welcoming, affectionate, expressive in welcoming, affectionate, expressive in welcoming, affectionate, expressive in ways that we can engage with it and ways that we can engage with it and ways that we can engage with it and understand. understand. understand. So, that's that. And thank you very much So, that's that. And thank you very much So, that's that. And thank you very much for listening. You can find me on the for listening. You can find me on the for listening. You can find me on the internet everywhere. internet everywhere. internet everywhere. >> [applause]