Podcasts›Automated›Tony Zhao on why home robots could unlock general intelligenceLike💬 Be the first to comment →AudioSeptember 18, 2026Tony Zhao on why home robots could unlock general intelligenceFrom Automated, hosted by Brian HeaterSunday Robotics is training Memo, a home robot, with human data collected through a skill-capture glove.Play episodeSaveOpen the audio file TranscriptTranscript from the show's feedSearch the transcript0:00If we find a way to scale data and compute, we're able to solve robotics. But this is actually not everything. There's one missing piece here, which is like, where's the data? You do not need a robot to collect data to train a robot. You can make a glove. And the data you collected from those cheap wearable devices works just as well. I think in robotics, there's a huge disconnect between a video and some system that actually works in the real world. That's the largest chasm that we as a whole industry needs to cross.0:31Much of the industry is saying we're gonna do industrial first and then we'll look at the home. And you ended up in the home. If you think about homes, the same reason that makes it hard is the same reason that it will facilitate the right amount of intelligence that can actually adapt. When people talk about the difficulty of the home and the timeline, much of the industry is putting it behind industrial tasks, you know, like manufacturing. Is it possible that people are overstating the problems relative to other large-scale robotics challenges?1:29We have a very fun episode for you today with Tony Zhao of Sunday Robotics. it's a really interesting company in the humanoid space. Tony has some unique views on tackling unstructured environments like the home. I'm also going to put him on blast right now during this intro and remind him that he agreed to send me one of those robot heads. Thanks to Tony and Sunday. Thanks to you for tuning in. Don't forget to like, subscribe, review the show, and check out our newsletter and news site over at automated.fm.2:02Automated's next happy hour and live podcast recording are going to be in the Bay Area. On Tuesday, September 22nd, we will be at Circuit Launch in Mountain View. Thanks to our friends at Silicon Valley Robotics for helping to set that up. More information and a link to sign up in the show notes. Now please enjoy this episode of Automated. Vision and AI are changing what's possible in industrial automation, from quality inspection and predictive maintenance to robotics, logistics, medical devices, aerospace, and manufacturing.2:34At the Advanced Vision and AI conference, you'll hear from industry leaders, see real-world AI applications in action, explore the latest in machine vision, computer vision, sensors, edge computing, and automation technology. You'll also be able to connect with the people who are shaping what comes next. Whether you're building, buying, or implementing these systems, this is a chance to stay ahead of the curve and see how vision and AI are moving from promise to practical impact. Learn more and make sure to buy your ticket early at the Advanced Vision and AI conference September 23rd and 24th in Santa Clara.3:11Mom and Dad, I'm thinking of dropping out of Stanford to launch a robotics startup. So funny.3:59So how how do you how do you sort of set the stage for something like that? I think because I I I was very close with my parents when I was in college and in PhD, and I've been like telling them like what I saw in robotics. And I think it's like for them, they also understand it's pretty clear that we're on an exponential path, that a lot of things that were previously thought to be impossible start to become possible. And they they can get a sense that like I'm feeling a little bit restless in academia, that like I want to do things that you know is not just not the best thing to do in academia when the objective is mostly about publication, but not like real products.4:44And I think they're also aware that like me as a kid, I'm always the you know, slightly reckless, slightly ambitious kind of kids. And so I think there are not super surprises coming out of me. Yeah, I I I I know I know next to nothing, or actually nothing, about your history prior to Stanford. So did you like move out to California specifically to go to school? Yes. I went to Berkeley for my undergrad and then go across the Bay to Sanford for PhD. Okay. So I I I ask partially because you know this the this is the thing with like startups in general, and and and I can ask this now, you know, now that you've had some like you know, prolonged success, you that there's been a lot of interest, you're launching a product in beta later this year, a ton of investor interest as well.5:32But I'm very I would say pragmatic to a fault when it comes to to risk taking. I'm risk adverse. That's probably a better way of putting it. So I always have I always have my fallback. And and and in a case like this, you know, was was going back to university, was that your fallback? Did you have a fallback? Is it do you find that it's a good idea to just not have a fallback? Yeah, I think one thing that I don't really talked about, and it's like when I left Stanford, I'm not on leave. I'm leaving. So there's no that door is closed.6:04Yes. because I think to me is that I actually don't enjoy having a fallback a lot of the times. When I started my PhD, I think one advice I got was that like, hey, you know, of course, PhD is about doing research, but you should have like a few other things that you really care about so that when your research is not going well, you're feeling okay. And I think I'm always the kind of people who are more inclined to and more excited about just being all-in on things. I was on research, I was on the company right now.6:36I I want to get back to this idea or maybe maybe even like this moment when it became clear to you as you said that it's on this exponential path. you know, there obviously in in technology we find that there are cycles. certainly right now, everybody I'm talking to that who has been in this for a very long time says that this moment is different. But I'm sure that, you know, prior to you, a lot of grad students and undergrad students felt like their moment was different as well, right?7:08They felt like, okay, we're close to this breakthrough. what was it that you were seeing at the time that gave you confidence that, like, okay, now is really, really the time to do this? Yeah, so I think maybe a little bit of history here. So me and my co-founder Chung, we're both PhD students at Stanford. And the kind of works that we're doing is to bring the methods that works really well for things like NLP, like large language models, into robotics.7:38And how can we make robotics feel more like the scaling loss, the you know, the LLMs? this was back in like 2023, when this was, I know this sounds like perfectly reasonable right now, back in 2023 is pretty contrarian. and I think when we think about the exponential path, it's less about like, hey, everything's dumb, but more about we see light at the end of the tunnel. There's a clear direction and there's a clear recipe of how to scale things up. So for me, I think the work I was working on is called Aloha.8:10This is actually one of the biggest projects I was working on when I was a PhD student myself. And I think what I was observing is that as we scale data, as we scale compute, it turns out you can solve really, really dexterous tasks. And one example we did was using this tabletop mounted gripper robots to tie a shoelace, which is like really, really dexterous and needs like thousands of hours of data. But just by scaling, we're able to do it. And that's like there are literally like remember professors from MIT who was like, I will quit my tenure if this task is solved.8:47I remember this. And and and you know, when obviously like we talk to a lot of folks who are in the industrial space, and for them it's like it's wired harnesses, right? I mean that that's kind of an equivalent on the industrial side. Like just taking such a flexible, skinny piece of material and weaving it through something is incredibly difficult to do. Exactly. Yeah, deformable objects are hard to assimilate. like traditional methods just really like struggles in those settings. and I think that was like one direction of like scaling data and compute makes dexterity, like the ceiling seems to be like really, really high.9:23And the other thing is generalization, which is like how much task can I do? Like in how many places can this happen? And the biggest data point for me is actually Tesla FSD or autopilot. And what we are seeing there is that they have the largest fleet, they have like almost infinite data, and they're able to train this end-to-end neural network to drive pretty well in many, many places. Of course, like the deployment threshold in like autonomous vehicles is incredibly high in robotics is a lot lower. But to me, that also shows that as you scale up data and compute, the model will generalize really, really well.9:59And these two combined is essentially telling me that if we find a way to scale data and compute, we're able to solve robotics. And but this is actually not everything. there's one missing piece here, which is like kind of the elephant in the room for robotics, is like where's the data? Right. Yeah. It's like unlike LLM, where like videos, there's this whole internet, and there are like all the books that people wrote over the past like thousands of years. and Teslas, Teslas have all like a fleet of people driving around.10:30Exactly. And it took them like 10, 20 years to get there, right? It's really, really difficult. So I think the the sort of light bulb moment is really when I look at my co-founder's work before it's released. That work is called UMI. And these say it's like UMI, universal manipulation interface. And that is, to me, the actual why now of like, I'll quit my PhD, do this company. It's not because my own research, it's because my co-founder's research. And what that research showed is that you do not need a robot to collect data to train a robot.11:06You can make this like handheld gripper thing, or in our case, make a glove. And the data you collected from those cheap wearable devices works just as well to train a robot than a whole like teleoperated, like full human, whatever it is, because those are really, really expensive. But the devices only cost like a few hundred dollars to make. And it's way more easier to distribute, it's way more easier to scale up. And to me, like more data is better. And then a scalable way to get data like combined is kind of forming the full picture for me that like early 2024, we should go do it.11:40Instead of like doing more papers, we should scale up the system, see the limitations, and do good research on top of that foundation, which is like impossible to do in academia just because the scale is so much smaller. So I'm gonna take a slightly off track, but before we before we get too far away from this, this is something that I really want to get some clarity on. I was speaking to Genesis, I think it might have been Genesis AI, and you know, they're doing they're doing five-fingered hands, and a lot of their thesis there is it makes sense to do five-fingered hands because you know you could do like a one-to-one port, right?12:18If you're using human hands to collect the data, that makes sense. When I look at your robot, it's I would say it's a a simpler manipulator from the standpoint of like digits at very least, right? You know, it's two fingers and a thumb. It's almost like a Lego hand in a sense. And then there's an analog to that on the glove itself. So the the development of that specific gripper, is that a product of developing the robot hand first or developing the data collection device?12:52Yeah. So one assumption that is that we didn't violate, that sometimes other kind of data collection violates, is that there is an exact match between the data collection device and the robot. And what it means is that anything that can be done with the data collection device can also be done with the robot. In our case, is that all the manipulation services, like where the glove, when we make contact with the world, are exactly the same, same material, same stiffness across the glove and the robot.13:25And this is something that is quite nuanced because this allows us, the exact match allows us to use the glove for both the pre-training and the post-training of the data set. And we do not need teleoperation in and of itself at all. Because the data itself is so good, it's as if it's coming from the robot. And having this unbiased data source is actually what allows us to push the success rate to like 99, 99.2, 99.5 eventually. And because the data source itself is unbiased.13:58And what people are seeing in LM these days is that like as you scale up pre-training, all the little biases that you have in the data set sort of manifests itself in the final behavior of the model. And for us, we want to think about that in the beginning to make sure our data is as good, as high quality, as high fidelity as it can get, so that asymptotically we didn't hit a wall later on, that we need to like remove some of the data set and recollect it. So I think this is kind of the kind of in the weeds explanation of like why we do that at all.14:30Is that like and I think the I think another way to think about it is like through the lens of evolution. And like even if you don't think about nature, if you think about evolution of a product, like iPhone, right? The first iPhone don't have a front-facing camera. The first iPhone does not have 3G, it doesn't have an app store. And over time it gets better and better and better. I think there is always a risk to be thinking about like what's going to be looking like in 20, 30 years too early.15:06And one example of that is I'm not sure if you heard about a company called General Magic. Oh, yeah, of course. Yeah, this was Tony Fidel's company, right? Exactly. They're trying to build an iPhone in the 1990s, right? And what it turns out is just technology is not there. And they're trying to start with a too ambitious, too, like kind of all encompassing of a goal that it turns out to be really hard to execute in a time frame that matters. So I think this is kind of the lemma I see in in robotics is that nobody will disagree that the end game of robotics should look like a humanoid, should have five fingers, should have all the things that humans have.15:48But I think the harder question to us is how to get there, as opposed to what is the end game. And to us, we always think about it as an iterative approach, that we can get something to be providing value, we can have other research, we can have all the recipe of how to train this financial model and see it all the way through before we start to like scope creep on like, you know, more exciting hardware, like five finger hands and all these things. That will eventually come, but I think the sequencing is what matters the most.16:18Yeah, and another point here, I mean, something that that jumped out at me when I was looking at some of the Aloha papers that you published. you know, one of the papers was headed learning bimanual mobile manipulation with low-cost whole body teleoperation. Low cost jumped out at me. you know, obviously research and mass massive manufacturing are really different. Obviously, teleop versus you know these like generalized robots, those are different as well. But low cost relative to other humanoid robots is going to be incredibly important if we're actually going to start introducing these into the home.16:56Yeah, absolutely. Like we're creating like a new category of general purpose robots. And I think there's really like when you think about low cost, there are two ways to think about it. One is the simplicity of the hardware and the fact that when you combine a hardware with a smart enough brain, the brain can actually compensate for some of the insufficiencies of hardware. And what I mean by that is that if you think about industrial robotics, you know exactly where the NF vector is, like sub-millimeter, right?17:30But if you look at human, do you know how far your current thump is from your eyes? Right? You actually don't know. But intelligence is so strong that you can close a loop so that you can still do things that are like super precise, super like like accurate. That's an interesting point, you know, and and and I think we're seeing this now in a lot of the training that's happening. There, there's a lot of surprise results as far as the decision making that the or or or you know, the paths that the robot themselves are taking, that they aren't ultimately doing the thing that the human would do.18:01Like they're taking a different approach to it because you know they they've got a different set of sensors, and even if their hardware is very similar to ours, like they have a more optimal idea of how to do something. Yeah. And I think is like they need to. I think the essentially like a smart enough brain will be able to go beyond some of the hardware limitations. this is something that never like if someone says like 10 years ago, people will be like, this is crazy. But I think this is something that Aloha observed, and we continue to observe that across a lot of low-cost hybrid platforms.18:36And I think the other thing that is maybe the biggest difference between like starting a company and doing research is that simplicity itself is not enough. We need to be like ruthless when it comes to supply chain, when it comes to cutting costs, when it comes to negotiating these offers. So I think that is also something that we as a team spend an incredible amount of time with like engineers, with supply chain specialists to be working on. That's so like at the end, think about it as like building an iPhone is not just about the like the design of it, but the manufacturing.19:09And design manufacturing is really one thing as opposed to two things that are optimized sequentially. Yeah, okay. So let's talk about that, right? I mean, you know, the the the reason why, one of the key reasons why we're able to well, let me back up. Two of the key reasons why we're all able to have these phones in our pockets is one, all of the years of RD that went into these devices to get things smaller and and more efficient. and two, like the the massive scale. And you we're hearing huge humanoid numbers, but you've even I would say the most optimistic projections aren't really getting them to smartphone scale.19:47So, you know, certainly like costs aren't going to come down that much. When you're looking at at a system like this, and when you're looking at all of the RD and everything else that goes into creating what is effectively like a full stack robot, how do you assure that it is within reach for consumers? Within reach in terms of the price point and the time market, right? Yes. So this is a great question. So the way we think about price point is really it only makes sense in the context of the capability.20:23Like if the robot can only do this one thing, like do your dishes, right? And now you're like, hey, how much am I playing my paying my housekeeper to do it? Of course, like the robot is actually a better service because it respects your privacy, it will be there 24-7, it's like all the good stuff. But I think there will be comparisons drawn with a human who's like only able to do that one task. But I think the the core thing that is really cool about general purpose robot as a new category of hardware is that its usefulness is more decided by the software and the AI than the hardware.21:00I think there are those videos of like 10, 20 years ago that there is this like simple like gripper robot, the PR2, cleaning up a whole like living room, right? This is the Willow Garage robot. Yes, the Willow Garage robot. and what it's showing is that intelligence is really the bottleneck, right? Because that robot is like pretty simple in in today's standard. Of course, in those days is like pretty expensive. Yeah. and I think what to us, I think maybe more a more day-to-day kind of comparison when it's the iPhone, is that its value is really decided by how many apps that you can run in a phone.21:37And if you think about Tesla, I think a lot of the value these days is like the autonomous vehicle, the FSD that is running on it. So I think we need to think about the pricing of the robot in the context of the capability that it will be running on. And the question then becomes like. Over time, how like how can more and more people become viable? Like the problem isn't viable to them.22:08Because in the beginning, let's say the robot can only do one task and we charge like X thousand dollars for it. There are like a ton of people who are like pretty well off, they're excited about the technology, they will buy it, right? Yes, they will buy it. But then imagine if the robot becomes the cost of a gaming PC or your phones, and it can do all your chores and also be your butler and also be maybe a teacher to your kids, like all the apps that could potentially run the robot. And that suddenly becomes a product that is for the maths as opposed to for only the people who are well off.22:45So I think how we glide, how fast we glide in this along this path is determined by how fast we can scale the capability of robot, the intelligence of the robot, which is something that we observed can scale with data, with compute. And also there's a ton of like tailwinds from just general foundation models that helps the robot reason better and think about like reasoning through like images or videos and all these advances. Yeah, yeah, which loops back to a point you made earlier, which was that the first iPhone had apps on it, but the app store came around with I think the second generation device.23:21So early adopters felt there was enough functionality to go out and buy that system, and then it was really enough to you know keep interest up to create a second device and then get developers on board. Yeah. And I think we like I think through all these iterations, like I think earning people's trust, early earning the trust of these early adopters, then the larger population is kind of the central thing that we want to optimize for, like like not just the technology, but as a company as a whole.23:58Yeah. So so so there's there are a number of things that I I you know I can see working like for and against the system. And and this isn't even getting into you know, because we've had this conversation a million times. I'm sure you've had this several orders of magnitude more than we have about the difficulty of unstructured environments in the home. So we don't have to dive super deep into that. But I'm thinking about a few things in terms of this parallel to whether it's a a Tesla or an iPhone or any kind of like first-generation device.24:35As I'm talking to a lot of industrial humanoid manufacturers, their sense is that, okay, we really need to get this thing out in the field doing one thing really, really well. And that's part of like really getting the data flywheel out there. And it's like, well, you know, how much ROI is that really gonna get to you? And like whatever the equivalent to ROI in the in the home is. The second thing is speed, you know, it does it have to be if we're talking about effectively the equivalent of a human worker in the home, does it have to be as fast or faster than than that?25:13Does it have to be as accurate or or more accurate? But certainly one thing that I see working in the system's favor is the ability to like more or less operate 24-7. Yeah, for sure. I think the I think it's really interesting to think about home in the context of industrial applications as well. Yeah. And I think for me, based on what I'm seeing in as how fast the field is moving, I think what will happen is just like everything will be solved. I think in a few years, yeah, the AI will be so strong.25:46Like, think about it, right? Like five years ago, you think about the coding capability, solving math Olympias of these like LLMs is like crazy. But because we found a recipe, because there is a scalable way to scale things, like it all eventually happens. And so I think is isn't I think between home and industrial is less about like, hey, I can only do one kind of thing, but more like I think the wave of this like like physical AGI, if you will, will be so large that everything will be solved.26:20And to us, what we are optimizing for is what is the biggest market? What is the most exciting thing to work on? And my answer to that is homes, because it's so relatable. I want it myself, my mom wanted, all my friends want it. And it's just like so relatable, and and this is a nice thing about working on consumer is that everyone around you will appreciate the work that you're doing. So I think this is kind of the high-level view when I think about like home.26:52And going into the Vs a little bit, you mentioned RLI and speed, right? And speed is actually something that we're explicitly not prioritizing at the moment. The thought experiment is that let's say the robot can do your laundry, do your dishes, and tidying up your whole house, right? And let's say just a living room. use that as small example. And turns out it's so slow. Like if you had a human to do it, like people can get it done in like 30 minutes.27:24But a robot will do it for like eight hours. And what it ends up looking like is the consumer experience is you go to work in the morning and you come back home, and all of these things are done. Does it really matter if the robot does it in eight hours or four hours or two hours or one hour? So this is kind of like the Roombo lesson to a certain extent. Yes. Is that I think the capability and like raising on this like exponential path of like more capabilities is a lot more important than optimizing for speed and throughput from the very beginning.28:00And this is something that is different, I think between industrial as well, is that like in order for a robot to have positive ROI in the production line, if it is a bottleneck, it slows down the whole line. And but for homes, they're just like this the thrower of time that the robot can just slowly get a lot of things done. and it's not blocking anyone, as long as like you know, it's finishing the job in a way that is high quality.28:31It's really interesting to take a step back and sort of look at where we are and the conversation that we're having right now, and you know, the conversation around projecting what's going to happen over the next few years as we're talking about this kind of, again, like exponential trajectory that we're on in terms of like technological breakthroughs. And and you know, it seems like and and it's hard for me, like a layman, to necessarily judge true breakthroughs versus you know, obviously we're getting a lot of interesting, you know, cool videos and thing and and and things like that.29:08But we are undoubtedly seeing a lot of generational technological breakthroughs. Tasks that that roboticists have been attempting to do for decades, like they're starting to be solved right now. But even looking at that, like you and all these other companies are still taking a bet that there are going to be additional generational breakthroughs between now and then that are going to get us there in a manageable amount of time.29:44The question here is what are we kind of betting on? And well, yeah, the the the yeah, the c the the qu yeah, it's it's a little bit what we're betting on, but it's also that it's it's like, you know, to take a step back, it is a little bit it is a little bit scary to to to think like, okay, well we we have to assume that this pace is probably not only gonna continue, but is really gonna kind of accelerate. Yeah. I think the I think if you take a step back here and think about I think the framing that you mentioned, what's really interesting, you mentioned like the videos and all the things online, right?30:25And I think in robotics, this is actually a a there's a huge disconnect between a video and some system that actually works in the real world, right? Yeah, and repeatable at scale, yeah. Exactly. And I think that's the largest chasm that we as a whole industry needs to cross. Because like if you look at the internet, like you saw like Boston Dynamics robots doing backflips like 10 years ago, and we're like, Robotics seems like itself. It's it looks superhuman to me.30:55Like I cannot do that. but the reality is I look around in your home in factories, and well, factories are robots, but like in hospitals, let's say, like there's really no robots deployed in the open world. And the big disconnect here is that having a robot working in one place is very, very different from having a robot working in unseen environments and unseen, like with unseen objects, right? So there's a big disconnect here. that I think this is actually a big point that we are trying to make in the Act Two announcement that we can talk about maybe later.31:28But I think we're definitely in a point that is a little bit scary, a little bit surprising all the time. And we're not, I'm not saying in the context of like, oh, Sunday is the only company doing this or something, is like how fast the whole field has been moving. The results that we are getting, like all the other companies are getting, are like unheard of. Like if you talk about it like a year ago or two years ago.32:00I think this kind of reminds me of like when the first time I saw so I was actually an NLP researcher back when I was in undergrad. I was, I think, among the first to get access to the GPT-3 playground. This is the first time they opened up the API. And I think it's the level of surprise when you talk to a chatbot that it felt like there's a there's a human on the other side of it.32:30And it's kind of scary. and I think these days when we look at our robot moves, it actually moves in a way that is really similar to human because I learned it from human. I think there is a similar kind of feeling that is invoked when I look at it. Yeah. yeah, that that that we're kind of right in the middle of that uncanny valley right now, where it's it's hard to determine like what is in isn't human. stepping back again into the Aloha paper, I noticed that one of your co-authors was Sergei Levine at Stanford.33:01And again, as as we're looking at the landscape, like like you said, like there are right now at least a handful of really incredible physical AI companies just like really objectively doing some really incredible stuff, taking very different approaches, you know, video training versus this kind of real-world data gathering that you're doing versus I think a physical intelligence is approach is more like what's really get robots out in the world doing something.33:33Is there a sense in which you know that the the the truth to all of this maybe lies in some combination of all of these solutions, or you really get the sense that, okay, we're we're we're doing it right and people are gonna kind of come around to our way of thinking? I think it's it's really a mixture of both. I would say one way to think about all these technical decisions, whether it be like video models, role models, like ego videos, like all these things, is that what are the things that are incompatible?34:07What are the things that are one-way doors? And it turns out like all of them are neither one-way doors nor incompatible with whatever other framework it is. They can all be combined in a way that is very natural. Is that if you look at LLM, it's actually a combination of all kinds of approaches. There is this auto-aggressive pre-training, there is, there was SFT for a while against RLHF, there is now reasoning, there's like it's a is a it's a whole stack of approaches that build on top of each other and combine into a system that's actually performant.34:38So I think the I think a lot of these ideas are not mutually exclusive nor a one-way door at the end of the day. And we as a company is like, we're so pragmatic, we just want things to work, and we are very open-minded when it comes to how things should work. But I I think coming back to the question of like like what should we work on? How should we focus?35:09I think one thing that we are really, really believes in is vertical integration and being a full-stack company. And what I mean by that is if you think about our whole pipeline from data collection, we design the gloves ourselves. We run our own data operations with like a thousand memory developers, and we train our own models, we have our own software that is running on the robot, and we'll do our own deployment and product and marketing. And all the videos that we've shown are like edited by ourselves.35:42And there's a belief I have is that it's very tempting to be working on a slice of the problem and feel like, oh, it's like, you know, it's a bite-able, it's a decent chunk, but it feels manageable. But at the same time, I think you'll be fooling yourself if you think you can do only 50% of the hard work, but captures 100% of the value. And in robotics, everything is so hard.36:12The model is hard, the hardware is hard, the deployment is hard. And I think we set out to be like, hey, we should not be lazy, we should not avoid these things, but we should own as much of this that is critical inside a company. And this is a belief that I think drives a lot of the decisions that we're making. Yeah, yeah. there's a there's a great there's a great video that I think it's like a four or five-minute video, and it and it and it sort of explains the company.36:44You know, you, your co-founders, and and a lot of folks in the company are featured in it. And it tells a story about the early days when you were working out of a house in like maybe Mountain View, right around Stanford area, Google area, telling the neighbors that you're running an NFC company because it was easier than telling them you're building a robot. Yeah. At the top of the conversation, we talked about you know the difficulty of saying, like, I'm you know, dropping out of you know, one of the best schools in the world to make this big bet.37:18Also I'm making a bet on the technology, I'm making a bet on the people, I'm making a bet on myself, right? You know, I'm I'm becoming a technical founder and a CEO of the company. I know that we're starting, we have to start small. We're starting in in a in a house. I mean, what was there ever a point when you felt like you had bit off way more than you could chew? Definitely. Yeah. I think we we enjoyed it.37:49It's actually really, really hard to train your like there's so the dependency tree is so long, right? Is that if your glove has some problem in it, that pollutes the data, and no matter what you do to the model, you're not going to save it, like because your data is wrong, right? And or there is a a small bug in the robot controller, and turns out the tracking of the tracking accuracy is like high. in those cases, like no matter how good your model is, it's also not going to work as reliable as it should be.38:22So I think there was, I think the first year is really, really painful because there are so many things that we're working on in parallel with such a small team. Like at the end of the first year, we have like 12, 13 people. And I think it's just like pretty crazy to look back that, like, like Tony, why are you doing that? but I think it is kind of the the foundation that we are building on, both in terms of the culture of the team, that we still value the full-stack aspect.38:53We want everyone to be like a polymath and be a generalist, and also like the intensity, like the amount how much we care. yeah. So I think it was it was a very, very interesting journey for sure. Is it true that you and Chang met on Twitter slash X? Is that is that a true fact? It is absolutely true. Which is funny because you were both you know publishing papers in in in Stanford at the time. You know, you're working on Aloha, you you know, you were at Berkeley, you're at Stanford at the time, you're working with objectively brilliant people.39:28You know, I I Sara Gay just jumped out at me because he's been on the show, and again, they're like doing really incredible work over there. we talked about the technology, but what what did you just speaking with Chang, what was it that made you say, hey, okay, this is this is the guy that I'm going all in with? Yeah. I think system thinking. System thinking. Yes. that's a really rare thing to see in researchers because lots of researchers are ideas person, right?40:00They have this brilliant idea and they pursue it, and it turns out it works. And then like the the industry take it from there to make it into a real system. I think what Chung had was a rare combination that there's both the ingenuity of coming up with these like crazy ideas and the system level thinking that is able to scale it up. And the way it kind of manifested is like even you look at his diffusion policy codebase, it's an incredibly written code base. And still like I think thousands of people are building on it these days, if not like hundreds, like tens of thousands of people.40:35And I think that was like kind of the first sign of that. And over time, I think the system thinking evolves with what a company needs. In the beginning, it was like structuring the code base, and it's like how these different components of the technology work together. hardware, software, like this kind of system that is beyond just a code base. And now it becomes more like how do we design the right organization structure so that as we scale, the whole team can work in a way that is efficient and like you know, collaborative and with the highest amount of throughput that we can get, the high like at the moment most amount of like potential out of people who are in the team.41:20So I think the system level thinking is the thing that is this one theme that like shines throughout his whole career from a researcher to now the CTO and co-founder of the company. Yeah, so I I would say that that he has the crazy ideas and then the ability to uncrazify them. Exactly. To make that to make them to make them achievable. In that first year, you know, you said again, by the end you've got about a dozen people, and obviously you're working in all these projects in in tandem, but like you know, coming off of having published these papers and all the work you were doing, looking down the road at having to solve all of these incredibly difficult problems at the same time, like literally, where do you start on day one?42:10Like, how do we even get started to tackle this big you rent the house, you know, you buy the 3D printers, you sit down, and then what? So I think everything started with a glove, actually. Which is how can we design a glove such that it's both ergonomic to wear, it's dexterous, it can do like it's kind of the 80-20 thing, can get like 80% of the capability with 20% of the cost. And also you're able to build a hand that is going to be manufacturable and cheap.42:43So everything starting with that, we're actually starting with like off-the-shelf hardware because they're the easiest to get. And then we just have our hand attached to those like off-the-shelf arms. And over time, I think we realize like I think it's only by using them you deeply understand the limitation of it. And then we start to swap out the arm with the arm that we built, and then we swap out the stand with an actual mobile base. And it is also like kind of an evolution that the robot grows from the hand all the way to the rest of the body.43:15That's interesting. And and and is there a sense in which that offers kind of a broader roadmap for the company going forward as you continue to sort of like iterate the hardware and software? Yes. I think this is the kind of the spirit of the company is that again, we're like we're pragmatic, we want to get things to work, we want to get things done. And and that can mean different things. And maybe that means a more sophisticated design of the robot, and we'll do that.43:49Maybe it means like a different way to architect the neural network. I think what is unchanged is being pragmatic. What's unchanged is the operation. Powers we have to gather the data that we need and the talent that we're having across the board and all the rest of the things, like we're always like challenging ourselves over time. Of course, that we're actually very consistent over the past time because I think we happen to be right in the beginning, but I think we're not like the type of company who are so attached to this one idea that it will never change.44:24Yeah, yeah. And and I was a I was a creative writing major, and in in creative writing they call it killing your darlings, which is the idea of like not being too attached or in love with any single idea that when it you know that you can't get rid of it. And that's a difficult thing, especially, you know, for a technology company that is very that is built around like a few core technologies. but also the thing that we've seen over and over again with robotics companies, and I'm sure that you had this conversation with every investor you talk to, is the idea of kind of almost like backwards engineering a solution.45:04We have this great technology, like what let's see what it can actually fix. And you and you ended up in the home. And I assume that part of the process, and certainly part of that first year, was really determining like not only the the need and the want, but like what actually is plausible. Yes. I think the way we think about that is that like as you said, right? Like at the end of 2023, it's so obvious that this is amazing technology, UMI, there's transformers, there's diffusion policies, all these things.45:37Like, where do we apply these? These are so powerful and universal, you can pretty much, if you want to apply to like any robots, right? and I think the way we think about it, like I think these days we're able to distill that into a word we call research market fit. And what it means is that the research that we're doing, which is oriented towards general intelligence, to be able to solve like any task on any robot and everything, is aligned with the product and the market that we are trying to address.46:11And if you think about homes, is that it's such a like the same reason that makes it hard, is the same reason that it will facilitate the right amount of intelligence that can actually adapt. If you think about like deploying a robot into a factory to do this one thing, yes, there will be data flywheels that come in out of those tasks, but the task is like narrow in the sense that all the flywheels come back looking the same, right? Because you mean that the diversity of the data set that you're trading on allows it to be more capable across a wider variety of different tasks?46:47Yes. As in solving homes is oriented in a way that is aligned with solving general intelligence as opposed to a very specialized kind of intelligence. Because if the goal is to get this one high throughput task to be really, really good, there's already a solution, which is called industrial automation. There are machines that can do it really, really well, right? And I think Holmes is is really the unstructured environment that will kind of push us to arrive at intelligence that is so adaptable that can interact with humans and can do many, many tasks in a way that's reliable.47:24Yeah, you spent a little bit of time at the Moonshot Factory. Right. yes. You were you were over there. Is that part of where this way of thinking comes from? Of like, let's take on an extremely difficult problem because there are ways in which like like that, like like, you know, I I'm sure everybody listening or watching this knows this to a certain extent, but like that's the basic thesis of the moonshot, right? Is that like you go for the hardest thing and there's gonna be value in that, you know, regardless of of of what the outcome is.47:54Was it your time sort of like being at a place like that, or you know, being being in and around Tesla or DeepMind that really convinced you that you wanted to go after such a hard problem? Yeah, I think my time at Tesla and DeepMind definitely shaped how I think about these. And also just as a previous NLP researcher, seeing how language model as a field is taking off from something that looks like pretty ordinary in the early days, huge inspiration.48:27And I think just at the end of the day, I think everyone has a decision to make, which is you can be an optimist or you can be a pessimist. And I'm very firmly on the side of being an optimist, that I believe the technology will work, that I believe the technology will benefit humanity. And this is why we're starting the company. Yeah, yeah. And and I I can't remember if it was you or or or Cheng who in in the video that I mentioned before, mentioned this that like there is this broad consensus that, okay, we're gonna have home robots, but it's you know, it's like how long it takes to get there.49:02And we're coming up on time, so we can close on this. But let me ask you one last big question. When people talk about the difficulty of the home and the timeline, much of the industry is putting it behind industrial tasks, you know, like manufacturing. Is it possible that people are overstating the problems relative to other large-scale robotics challenges?49:34As in the difficulty of it? Is in the difficulty of deploying a robot in the home at scale effectively. Yes. I think the way I will think about it is that if the goal is to deploy a robot to one home versus one factory, I think deploying to one factory is a lot easier than deploying to one home. Because within the home, there is already so much reliability. You can have a cat jumping around, right? Like the environment is dynamic and it's unpredictable, unstructured.50:05But I think the goals that all the companies, including us, is want to have is have a robot to be able to deploy to any factory or any homes without additional adaptation. And if we're thinking in that context, actually the difficulty of these two problems shrinks a lot because now you have to handle the like the variability across factories. All the factories are built so differently, right? Their whole IT systems, like how the conveyor belt talks to your system and how your system talks to the downstream system, is so different.50:38So I think the objective shouldn't be about having one pilot run. It should be having a generalized machine and intelligence that's able to go into any home or any factories. And I would argue if that is the goal, these two things are actually very, very similar to each other. That's interesting. So I I lied when I said that was gonna be the last one, and this is probably one you aren't ready to answer anyway, but much of the industry is saying we're gonna do industrial first and then we'll look at the home.51:13You know, it and i if your sense is that you know you can really chain train an incredibly robust, generalized robot model with all of the diverse data sets on the home, do you get the sense that like once this thing is out there and and in the home, then that sort of makes it easier then to start deploying into industrial settings?51:44Yeah, for sure. if you think about the two directions, right? Either they're like imagine an industrial robot coming out of a factory and now they need to work alongside a human, versus a robot that can already work alongside human in a diverse environment going to factories. Like which path does a human take, right? Like we clearly take the home path, and then we practice and we get better, like get faster, and then kind of growl our way out of homes, maybe first into getting a grocery runs, or like doing the dry cleaning, and then eventually into these really sophisticated factories.52:20But I think the I think the to me like the maybe the closest, like the analogy thing is like, does there exist an iPhone or iPad for business? It doesn't exist. No. They tried. Samsung tried, yeah. Yeah. And the thing is, like, once you solve the hard problems and build a network that is like a model that is so general and intelligent, and a form factor that is both universal and like reliable and cheap, I think the world kind of opens up in front of you.53:03And this is how like iPhones started as a like consumer product and iPad. And these days, if you go to check in, they will just give you an iPad, right? They get they don't give you like an iPad for for hotels as an example. So I think this is kind of a to us, this is a kind of reward of working on hard problems and solving things in a way that is like truly general and universally applicable across different fields. And so by starting with the hard part.53:34Well, Tony, it's been an absolute pleasure. Thank you so much for taking the time. Yeah, thank you so much, Brian. Absolutely enjoyed it. Thanks so much to Tony and Sunday Robotics. Thanks to you for tuning in. Please like, subscribe, share, review, and check out the rest of our work over at automated.fm. Automated's next happy hour and live podcast recording are going to be in the Bay Area very soon on Tuesday, September 22nd. We are going to be at Circuit Launch in Mountain View. Thank you to our friends at Silicon Valley Robotics for helping to set that up.54:08More information and a link to sign up can all be found in the show notes. Stick around, and we will be back here very soon for another episode of Automated.
