Creativity, Art, and AI: Dive into Digital Tooling with Adobe's Daniel Robbins

Daniel Robbins, Principal Designer at Adobe, delves into his work in 3D and generative AI, discussing how Adobe democratizes 3D creation. He highlights the combination of 3D technology and generative AI to generate 3D images from simple object guides, making these tools accessible to a wider audience. Robbins addresses challenges such as job displacement, equity, and ethics, emphasizing the importance of control and consistency in AI outputs. He explores the future of 3D and 2D video convergence. Throughout the episode, Robbins underscores the significance of understanding human needs, creating tangible artifacts, and storytelling, advising those in the field to focus on meaningful work and networking for career growth.

Notable Quotes
🎨 "The ability for anybody, a school kid, to describe a scene and have the system produce something that they can move around in." Daniel
🤔 "Be very thoughtful about which technologies we bring to bear and what human problems or tasks we're trying to address." Daniel

Resources
Podcast Feedback, Guests
Khan Academy
Hugging Face
Comfy UI 
Allie K Miller, Karen X Cheng

Daniel Bio
Daniel C. Robbins is a Principal Designer in Seattle, working on both strategy and interface design across all of Adobe's 3D/AR/VR projects with a focus on generative AI. He's also helped craft integrity aspects of Horizon Worlds at Meta, advanced AR/VR projects at HTC, VR for architects at Visual Vocal, and capped a long tenure at Microsoft by helping bring the Microsoft Envisioning Center to life. His art projects have ranged from an art car for Burning Man to very ambitious Halloween costumes for his kids. Dan regularly mentors and sponsors people from underrepresented groups who are entering the design field and he brings a passion for joy, equity, anti-racism, and justice through the lenses of curiosity, compassion, and contradiction.

Connect w/ Daniel on LinkedIn
Daniel’s Website

Chapters

• 0:00 Podcast Survey, Intro & Context
• 2:52 Daniel Robbins: A Creative Journey at Adobe
• 3:24 Democratizing 3D: Bringing 3D Art to the Masses
• 4:25 Generative AI and 3D: Simplifying Complex Creations
• 5:33 The Role of Hallucination in AI Art
• 9:39 The Rise of the Creator Economy: Democratization of Tools
• 10:18 Navigating Risk and Ethics in Generative AI
• 13:03 Deepfakes and the Brave New World
• 17:10 Fast-Moving Industry: Team Collaboration
• 18:01 The Hype Cycle: A Hunger for Innovation
• 18:45 Data Curation & Ethical Considerations
• 20:02 Adobe's Approach to Training Data and Bias
• 21:26 Representation in AI: Idealism vs. Realism
• 22:16 Challenges of Democratizing AI Tools
• 23:27 Future of AI: Quality, Control, Consistency
• 26:50 3D Light Fields and Generative Video
• 30:16 Intentional Tech: Solving Human Problems
• 31:26 The ROI of Generative AI
• 32:53 Hidden Cost of Compute: Environmental and Financial Impacts
• 33:18 Daniel's Career Path: From Academia to Industry
• 35:07 Democratization of Tech: Accessibility & Opportunities
• 38:47 Beyond Being in the Room: The Importance of Being Heard
• 43:28 The Balance of R&D and Marketing in Tech Companies
• 44:59 Portfolios Over Credentials
• 46:09 Upskilling in Generative AI Era
• 48:22 Finding Joy and Purpose in Your Career Path
• 48:57 Leveraging Generative AI for Opportunities
• 49:05 Daniel Robbins: Mentoring & Networking
• 49:32 Closing Thoughts & Career Guidance

Transcript

0:00

Hey everyone, as we near the end of season one, we are gearing up for season two, already recording some amazing stuff. I literally have chills hearing the stories of some of our guests. Here's the thing, we wanna make it even better. We want constructive feedback. You have phenomenal ideas that I know can take this podcast to the next level. If you wanna share some of that constructive feedback, just takes a few seconds, I swear. Go to yourairoadmap .com/podcastfeedback podcast feedback, share your thoughts. Maybe you'll get a shout out. Seriously, but if you've listened to this, you know that I love data. I crave knowing more, being curious, getting better together. I can't wait to hear from you. My team will read every single one, so we're really excited. Okay, thank you so much. Woohoo! Hi, my name is Joan Palmiter Bajorek. I'm on a mission to decrease fluffy hype and talk about the people actually building in AI. Anyone can build in AI, including you. Whether you're terrified or excited, there's been no better time than today to dive in. Now is the time to be curious and future -proof your career, and ultimately, your income. This podcast isn't about white dudes patting themselves on the back. This is about you and me. and all the paths into cool projects around the world. So what's next on Your AI Roadmap? Let's figure it out together. You ready? This is Your AI Roadmap, the podcast. Hey folks, this is Joan sharing a little bit of context about this episode. So this is an episode with Dan Robbins from Adobe. He and I have met here in Seattle, gosh, how many years ago when I visited the Adobe campus. And I really have enjoyed how I've learned from his career, how he thinks about risk management being an intersectional. feminist, I love meeting people, especially men, who are really thinking about their positionality in the world the way they build product and teams and respect and learn from everybody on their team. You'll hear about quality and controls and the really cool things Adobe's building out as much as he can share, right? And I'm just so grateful. for the ability to really expand my mind and hopefully your mind in what Adobe is building and how that is differentiated from other products like Canva and other really cool suites. It's just really changed Adobe has as a company since I was a power user of their product circa 2012 with Photoshop that I would fight with on a daily basis. Anyway, that's another story, but. I hope you have a fun time listening to this episode. Let's dive Hello, hello. You too, I'm so jazzed. Well, can you introduce yourself please? Sure. I am Daniel Robbins. I am a father, a husband, a human. And during the day, I also work at Adobe as a principal designer focused on 3D and what we are now calling generative AI. And I am based in Seattle, also known as the land of the Duwamish. Land of the Duwamish. That is absolutely correct. Well, you're doing a lot of cool stuff. I'd love to hear what concretely could you talk to people about that you do in your day job? Sure. So Adobe, as people might know, makes a variety of tools that are usually used by creative people to make visual and audio and other kinds of artifacts. I specifically work on a set of tools that are geared toward people who want to make artifacts that involve three dimensional images. So think of I have something like a car and I want to spin that around or I have a character and I want that to walk across the screen. That has been the province of very skilled people with a lot of training. Typically, we are trying to democratize that ability to make 3D things and bring that to a wider audience. As people may know, one of the new technologies that has lots of hype and money behind it, as you can imagine, is generative AI and machine learning. And we are trying to figure out how can we bring the best of 3D and the best of generative AI together so that we can bring these capabilities to a... wider audience. Cool. So can you give, and I haven't tried out the new suite and it probably changes daily. So when, could you give it like a concrete example? Are these things where you like make a box and it fills things in or So I'll start at the high level abstractly, and then I'll come down the very specifics. So as you mentioned, when we talk about 3D, we're talking about shapes. We're talking about things that live in space and that can move around and at least represent volume. So yes, if you want to do something like make a city scene using traditional techniques, you'd have to say make a bunch of boxes and line them up and say here are the windows. And then you'd have to get artists who draw the textures and put them on there. And then the cars. and such. What we're working on, if I come be a little bit more specific, instead of having to give all that detail, which takes a lot of skill and time, is tedious and is required, makes a very linear workflow, is we let you place a lot of very simple objects in your scene. So just a cube, say you tell the system, this cube is really a high rise building. And maybe you have something like a elongated sphere and you say, this is a sports car. And then you have a bunch of spheres sitting on top of cylinders. And you say, These are trees. And you tell the system this, and you pass this information off to the system. And it thinks for a few seconds, and it comes back with something that looks essentially like a photograph, where those have been, I don't want to say they've been replaced, but they have been used as guides to direct the hallucination of this new image. Hmm to direct guide the hallucination. Well, oh interesting that what it would interesting framing because I think right now a lot of people Are thinking of hallucinations as the bad the evil Are you are you joking? Are you is it place? I mean, we want to be honest with these terms. I mean, that's not a word you'll see in product necessarily or press releases, but it is a sort of colloquial term of art that we try to, at least I try to remind people of, that with traditional digital creative tools are very deterministic. You draw a line somewhere, that line is exactly in that place. You type a piece of text, that text is there. You put a three-d representation of cube in an exact place, and that's where it is. And so that's deterministic. Now we are embracing probabilistic tools, things that are using statistical inference, which means using lots and lots of data about how things have been done in the past, whether that be in the image domain or the text domain or the audio domain to guide judgments about how one thing would follow after another thing. And so because of that, because it's probabilistic, because it's based on statistics, it's kind of a roll of the dice every time you engage one of these systems. And you know, as if in this discussion, we may get into more detail about how there's real sort of a tension and trade-offs often between quality and control. And so, you know, with deterministic systems, you have ultimate control. But the quality may not be where you want it to be because not everybody has the skills or the tools necessarily or the training or the time or the resources to get to the quality level they want. So we see some of these generative AI techniques as a way of taking a shortcut to get to quality much more quickly in terms of production, but also in terms of ideation and serendipity at the front end of a project. Oh, that's so cool. Oh, I got so many follow-up questions to this. do you currently focus on one product or are you looking at different suite of products? So I'm very fortunate that I get to operate across a whole bunch of products. They're all in 3D domain. Adobe has a bunch of in-market products that address 3D creatives. And then we also have things that we're not at liberty to talk about yet that hopefully will bring some of that 3D to a wider audience. So again, I'm focused on all those 3D products. But to do that, we're a large company. So I need to... coordinate with people who work on the traditional flagship products like Photoshop, Illustrator, After Effects, things like that. And then we also have another team that's working on an in-market web-based product called Firefly that's had a lot of press recently. And they are sort of on the cutting edge of what are the user interface patterns. And when I say user interface, I mean, what are the buttons and the sliders and the different affordances that someone might use to control these systems? So we definitely. I won't say we're in lockstep, but we definitely are learning from each other as we move in these domains. Oh, that's so cool. Well, and when you, and excuse me, this is my own like, who are Adobe's buyers and the buyer persona, you think about like, I'm designing for Margo who's working specifically on this type of creation. When you think about who's using these tools on the daily hours, pouring their time into using these tools, how do you think about, or is that a part of your process? Oh, it is. I mean, whether someone is designing a car or a container for a new sports drink or shoe, one has to think about their audience. If you don't, you are just making something for yourself, essentially. So, as I said, we have some in-market 3D products. Historically, those have been targeted at a very high-end audience. People are making visual effects and video games and things like that. We have a... a very heartfelt interest in bringing that to a wider audience. The observation being that, A, there are some people who actually do want to do 3D work or their jobs might require you to deal with 3D work, but then also being able to work with 3D objects and 3D scenes gives you a degree of control and mutability that you don't get if you're in the 2D domain. So to come back to your question, like, who is this stuff for? So it's evolving. It started with very high-end stuff. but now we are very intentionally bringing in the generative AI stuff so that we can bring it to a broader audience. Our first steps in that area will be definitely toward the flagship product, so people who are typically using things like Photoshop and Illustrator. You can still think of those as everything from hobbyists to professional artists, but even kids in schools, and maybe they're working on their yearbook, or maybe they want to put something on social media that has that extra level of quality and polish. That's awesome. Yeah, I'll admit, or I'll admit, people may not remember, I had a BFA in photography, photo media, and spent a whole lot of hours and months of my life getting better at Photoshop. And I'd try other tools, and I was like, this is nice. If I want the really technical nitty gritty details, I have to go to a power tool like that. So I'm really excited for, as other tools join the marketplace, the canvas of the world. I don't know if you've seen Pica recently, but these different other, you know, everyone's, because I think these are creator economies taking off, so many more people also are trying to learn it upscale and as you mentioned, democratization of the ability to create phenomenal things. Yeah, for good and also risk comes along with that as well. Absolutely. Yeah, how do you think about risk? I think it across a bunch of different facets. When some of the generative AI stuff started coming up maybe say a year, year and a half ago, there was quite a lot of talk, at least in the professional artist market around job displacement. And that's definitely something to pay attention to. You'll hear me talk about the pyramid often of different industries. And I would like, this is my opinion, not necessarily Adobe's, but my opinion is those who are at the top, highly, highly skilled, will always have an audience for what they do. Those who are maybe not necessarily at the top should think about expanding their skill sets to include some of these new tools. So job displacement's one area. I'm much more interested and sort of focused as a human being on issues around equity and ethics. And that is everything from who gets to use these tools, who gets to learn how to use them. who gets economic access to these tools. So that's one side. Who gets input into how these tools are created. And it's not just the tools like the levers and buttons, but the data sets and the training that are used to make these models. And then there's the other side, which is the output. What are people making with these tools? And as we know that any tool, independent of whether it's using generative AI or not, People can make things that are harmful or people, whether it be intentional or unintentional. And that's something I'm very involved in trying to both track and try to figure out other ways that we can nudge people toward doing things that are better for the world, ideally. And this is a particular concern, I think, with the generative AI stuff, again, on those two fronts, one about quality. So if I can make something that looks photorealistic, that is. say video, and we've seen some very interesting stuff come out in the last few months. If I can do that, that is both exciting, but also potentially worrisome. So that's an area we have to track, so that's the quality. And then around control, because some of these generative AI techniques, especially as I can start driving them with synthetic 3D scenes, I get a lot more control over what shows up in these things. so I can have certain characters do things that they might not be doing in the real world. And that is a, I wouldn't say that it's a showstopper, but that is certainly an area that we need to watch closely and learn very quickly from what we see happening in the world. Be very sensitive around that. We're very fortunate that Adobe has devoted real resources of time and money to having a dedicated product equity team. These are people I work very closely with. I'm not in that team, but they are great partners in what we do. And so they are focused very much on the training of these systems and also who do we engage from the community to help craft these systems and also give us reality checks of, you know, are we doing good? So that was a very long-winded answer, but thank you for asking the question. Well, that's a fantastic answer. I love your bird's eye view to the concrete view and thinking about equity in our workflows and who we work with and how we build. I'm reminded of, I don't know, there's so many different reference points we could talk to, but like the pope in that white big puffer coat that looked so photorealistic, whereas the pope was probably not, I think, wearing that big white puffer. But I mean, I think the, where there's so many cool creative aspects of what we can build. skyscrapers, et cetera. And then there's like politicians, you know, the implications to the world of deep fakes and things that people might take as real and then certainly aren't. So I think there's the brave new world concept. And so one of the things that we're seeing, or at least I'm seeing from within, is it is forcing a lot of people to come up with a point of view about these things. I would say many companies, including Adobe, traditionally have had the orientation that we're making tools and tools are neutral and people can make good things or bad things with them. But it is a step change. It's not just a change in degree. It's a change in magnitude. when anybody now, in the very soon near future, will be able to make things that are indistinguishable from things captured in the real world. And that is not, again, that is not the same as saying, well, anybody can get in Photoshop and do manipulation, or in the old days of certain government regimes, people could modify film and negatives. This is now an entirely different level. So we do have to proceed. very deliberately on this and have our ears and our eyes wide open to what's happening in the world. Absolutely. Well, this directly matches a conversation I had with a different podcast guest who works on social listening tools and tracking virality for mostly big corporations like Hulu and Warner Brothers. But really thinking about the implications to our society, right? What we consume, how we build things, how people take those into real world context. It's a whole flywheel. Okay, so very cool stuff you're doing. I talked about metrics. how long have you been in this part of the industry would you say? So it depends how you divide that up. I've been in Adobe for almost two years. Interestingly, when I was hired into Adobe, they brought me in to focus on an area that I'd been working on for quite a while, which is virtual reality or augmented reality or spatial computing, however you want to call that. But pretty quickly, it became apparent and urgent that we have more people focused on how to mesh together 3D and generative AI. So I'm actually not working on any of the spatial. I'm 100% focused on generative AI. So that's fairly new for me. But in the areas of 3D interaction, I've been working on that since around 1987, just to age myself there. Yeah, okay, cool. Well, my question is about kind of longitudinal then. What surprises and challenges have you seen along the way? Because you're saying that this is a step change. You know, what little nodes have you been watching and like, whoa, this surprised me, or like, this is a challenge I thought was gonna be easy, or like any of those kind of moments. Sure. So some of my background is I come from academia. I spent about eight years working in academic research lab. And then I worked in industrial research lab and Microsoft research. And there are always new developments in technology and new bodies of work, whether it be immersive computing or mobile computing or cloud computing. And now we have this new area of LLMs and generative AI. And an observation I have that is surprising, because that was your question, is what's actually surprising me, is kind of an inversion in how the two parts of industry are working, and one being sort of the research and academic end, and the other being productization. In the past, maybe to be a little reductionist about it, research tend to operate it on its own, kind of in isolation. And there were huge challenges to doing tech transfer, of getting things out to market. Now there's an inversion where the product teams are begging the research teams to go faster and to give them more. And that's very interesting. A week doesn't go by that I see new developments from our own internal research teams, from external stuff. And as soon as something comes up, there is a product-led desire. to weave that stuff into something that people in the real world can actually use. And so again, that is a surprising change. And that's one of the things that to me seems like a signal that this is different than other technological changes. Do you feel like you're more in collaboration then with some of these other teams and colleagues than you were before? Yeah, I mean, it's a very fast moving vehicle that we are both peddling and being pushed and pulled all at the same time, whichever metaphor you want to use. Yes. that's awesome. That's very cool. Yeah. I certainly have seen as an ex-academic myself, like in the laboratory, working on something, wearing on something. Oh, what did we sell it? Oh no. Like this commercialization and then working at enterprise companies where product and R and D like never talked. Like throw things over and be like, hi, goodbye. And I mean, like, and you know, months later, um, you know, is it relevant anymore at all to be, but literally, uh, I can't remember maybe one meeting or one synchronous meeting in my whole time of these teams ever actually talking, just wild. And now, you know, everyone's benchmarking and deploying and running around in circles and trying to get paid. Okay. so again, I would say, what makes this different than any other hype cycle? And I've lived through a bunch of them. Again, is that I've never seen such a hunger from the product teams to put stuff from research into flight quickly. You think it's too quick? I know some people are worried about that pace part. I would say the quick, yeah, I mean, because I mentioned all the risks, definitely. I would also say that there, you know, with any new technology, there's kind of a mania around, you know, first mover advantage. And I think with some of these things, time to market is not actually the first mover advantage. I think the advantages, the moats, as some people may call it, will be much more around the training sets and who has the best training set by whatever metrics you might qualify best. Yeah. Yep. Well, I think that the data piece, I follow an influencer in the data space who was like, your model may only be as good as your data. Like, reminder, like data important. How do you all think about data curation to the extent you can share, obviously? could you just speak a little bit about the data piece? Sure, and I'll be putting on my Adobe hat for this part of it. So one of Adobe's value propositions is that for the data that we're using to train these, and I'm going to talk about the visual domain at least. We're talking about imagery and video. All comes from Adobe Stock and other licensed content, which means that it's commercially safe. So that's one aspect of it. Much more interesting to me, as I mentioned before, deep involvement of the product equity team. So that comes in and everything from when someone types in a prompt, how is that prompt interpreted? What kinds of terms do we flag? And then also when we get certain kinds of output, we actually have processes that look at the output and say, well, some of the images or video, whatever it is that we're generating don't meet our standards for sort of ethical or equitable output. And then we'll intercept those and we'll... will request more results from the engine, essentially. So there's sort of two parts of that. It's not just the training of the data. It's also how do you intercept the prompts, and then also how do you interpret the artifacts before they go out the door, before they're even presented to the user. And then, but then, there is a step that comes even before that, which you alluded to, which is our curation of the stuff that's even going into the training set. I guess, as I said right now, our training is very much based on what's in Adobe Stock. We down select from that as to what goes into the training set. Not everything that's in there is appropriate to be trained on. Some of it may be redundant. Some of it may not be up to the standards that we need. I think there's a lot of interesting work to be done, and I'm standing on the shoulders of people much smarter around this. There's a lot of work to be done. around starting from scratch with new training sets. Ovetta Sampson, who is one of the great, wonderful voices in this area, talks a lot about, there's only so far you can go with curation and prompt engineering. At some point, we need to just start over and have new people, people from more diverse backgrounds, who are actually making these data sets. Because as you know, any corpus is a way of reinforcing biases that come along, whether they're intentional or unintentional, that come along from the people who made those things. Absolutely. Yeah, it's, I think one of the interesting things about as we do benchmarking of different outputs and you know, like he is a doctor, she is a nurse and people are like, oh, it's affronted. Like, why are we putting these gender things? And then you look at the data set that has been built since the 1920s, right? And then we all shocker that these are the mapping outputs that may no longer represent the world we see or want for today. but sometimes even there, what is the right answer? Some of this is a corporate governance question. Some of this is just sort of a moral or ethical question. So say someone types in a prompt like doctor. Should the results that we give you reflect what you generally see in society? Should it reinforce the biases that one would actually see if you go to doctor's offices that are... reflecting of institutional bias and racism and inequities? Or should we give you results that are more idealized to try to sort of nudge your notion of representation toward a more ideal situation? That's one area. Another is around localization. You know, if I type in dancer and I'm in the US, I might get a set of dancers that looks a certain way. If I'm in India and I type in dancer, should I see localized things or not? I'm not going to, you know, I think these are great questions for listeners to grapple with. on their choices, right? I think, as you mentioned, some people think these tools are so neutral, that they've all the vestiges of the choices that are made by humans. Again, by humans in this AI thing, and I think also, as you mentioned, kind of these seasoned professionals and some entrants coming, we have such opportunity to make those choices and the democratization of creation. I keep talking to people about hugging faces, like... You can build these data sets, you can build these models. It's more accessible, I believe, than it has ever been, which is a really exciting opportunity. And it has challenges as well. If you go into any of these corpuses like Civite or whatever that host these models and these lores and stuff, there'll be monsters there. And I would definitely caution anybody before they start wading into the world of models. And when I say model, I mean a generative system or an LLM that's been trained on a certain subset, a certain corpus, to get toward a certain desired outcome. For sure, for sure. Well, I think I mean just the ability for anyone to do it. I guess is kind of what my, but I agree with you, question what's in it, who made it, and what it's gonna do. When you think about, you know, we've seen such transformations happen in such a short span of time, so maybe not a fair question, but when you look to the horizon of your part of the field, what do you think is coming down the pike? Sure, I can talk generally in some sort of hand wavy stuff without revealing too much of the secret sauce. So as I mentioned several times, there are these two pillars, two things that we want to optimize for, one being quality and one being control. I tend to be much more interested in the control aspect, especially as we think about how do you give control to people who don't have traditional art or digital tool backgrounds. But to really get, the degree of control people want and the degree of consistency. And let me unpack that word consistency. So if anyone's used any of these systems, whether they be Dolly, Runway, Firefly, you know, any number of these fairly new generative AI systems, you'll notice every time you change anything in the inputs, whether the inputs be text or some slider or some visual input, you're re-rolling the dice and you get... results that while some aspects may be very similar to previous results, there are a lot of things that keep changing. They keep wiggling around. And really, there is a strong point of view, at least with the researchers that I work with, that the only way to get to real strong consistency is to have a much richer 3D representation of these scenes. And when I say 3D, I don't mean traditional 3D graphics. I don't mean that I have a box here and a sphere here and a bunch of cones and things like that with textures and materials. I mean that essentially we need light fields and light fields is a term of art. And what it really talks about is at every point in space, and I'm talking about real space, you could think at every point in space, as you move around it, whether there'd be an object, something like this as reflection and diffraction and refraction in it, at every point in space, The light changes how it hits that object and then comes to my eye. So as I move around it, I have a lot of view-dependent effects. And so the way to represent that in its very heavy weight, it's very expensive, is what we call a light field. So there's been new techniques fairly recently, generally two of them, one called NERFs, that's an acronym, another called Gaussian Splats. But what they are essentially doing, both of them, is finding... new ways of representing that light field in much more compact ways and in ways that use some of these generative eye techniques in using inference to fill in between sparse sets of these data points. That's a lot of big terminology, but the long and the short of it is saying instead of having traditional graphics where we have these shapes lying around, we really have a space that is full of information about how light is moving through the world. And when you do that, you get a really very rich and very repeatable way of looking at the world. And so one of our goals is to be able to take these kinds of inputs, whether that be a text prompt, like say I want a sports car on the street, or whether it be I have an image of a car I already like or a sketch I made, to take all those inputs and to generate these light fields. And again, the... the terms you may see floating around are splats and nerfs, but to generate these light fields so that I can do things like move objects to the scene, move the camera, move my viewpoint, and I have consistency, and I have realism. So again, trying to bring up the quality and trying to keep that control so that I have that consistency. So that's one area that I think is gonna be very fruitful for bringing much more realistic and controllable and consistent. things to bear. And then the other one is around video, sort of generated video. And you've seen some very exciting things. Sora came out a month or so ago. And what those are typically doing is they're still generating 2D images, but you keep generating an image based on what happened in the previous image, and then you're giving inputs about how things change over time. What will happen at some point is there's going to be a convergence or that video generation in these 3D. And you've actually seen some pretty interesting experiments where people have taken these synthetic videos that have been generated by AI and then actually backported them to light fields, have created 3D worlds that you can move around in a very limited way that came from the video. And so I think there's going to be a lot of back and forth between the 3D to 2D to 3D to 2D. So that was, again, a long answer to your short question. And then if we move into the more esoteric, it raises some very interesting sort of questions, you know, cognitive, neurological and cognitive questions about how do we perceive the world? What are the models for the world inside our own heads? Are we fundamentally actually processing the world in 2D and assembling these things and then making inferences? Or are we doing it intrinsically in 3D? I don't have the answer to it. It's not my own field of work, but it's It has a lot to bear on which of these sort of systems we do more of the input and the processing and the output in, whether it be 2D or 3D. And then tied into that, and this is maybe more detailed than you want, there are some schools of thought that says it's sort of immaterial, how we think as humans, whether it be 3D or 2D internally. because fundamentally we have a ton more 2D data, whether it be videos on YouTube, whether it be pictures people have taken, whether it be commercially viable corpuses like Adobe Stock, it's all 2D to a first approximation. I mean, yes, we have some synthetic 3D data, there is way more 2D. So that's what the training is built on. And so there's a lot of effort on taking that two-dimensional data and deriving three-dimensional representations from it. Wow. But now, let me get the TLDR. What are you going to get? Words, words, words. What are you going to actually get with all that? Is the ability for anybody, a school kid, to say, whether it be typing or drawing or speaking, to describe a scene, whether it be realistic or magical, and have the system produce something that they can move around in, whether it be immersively putting it on their head or whether it be on their phone or their desktop. It'll fundamentally be a 3D scene that they and other people can experience. Very cool, wow. My brain goes so many places. I think one of my, it's so interesting when you talk about kind of light fields. This is certainly not something I think about much in my work today. But I have seen this, my friend sent me this video of marbles in the dark. Have you seen this video? And Nvidia put out this, I was like, marbles? Like, sorry, I don't care, or you know, marbles, sure. But I opened it up and it had, as you are talking about kind of these light sources and they appear to be marbles, this is all apparently synthetic, but these glass orbs, you know, rolling down different things, different textures, and really thinking about, in my mind, my brain was like, well, this is a real scene. And then I was like, wait a minute, this was generated and kind of just showing off different abilities of the things that, you know, the Nvidia team, et cetera, are working on. But, you know, I work in software. Or at least we just, or maybe somewhere, I mean like just data API calls, just the way this other part of the field is working and thinking about light and video and you know as you mentioned kind of 2D, 3D perceptual abilities of our brains. I'm thinking about memories like I'm like where's the breeze? Like I'm when I think about some of the most vivid memories of my life like you know there's audio, there's heat, cool, etc. So I think it's really delightful, for me at least, and hopefully for the listeners, to hear about different aspects of the field taking off and kind of the big picture futures that become integrated as these things kind of eventually do and don't converge and get integrated better. Now, I would like to say to our audience, part of the onus on us as technologists is to be very thoughtful and intentional about which technologies we bring to bear and what actual human problems or tasks we're trying to address. There have been certain... products that have been released very recently, where there have been some fairly negative reviews of these products that combine hardware, the real world and AI and audio input and audio output. And I think there's some really interesting lessons from these around, are you solving an actual need and what are the thresholds for usability? Are they actually more convenient than patterns that people are already using? So I'd like anybody who's dipping their toes into this area or has the good fortune to be able to actually driving these ships to really ask themselves not so much how cool is this stuff, but what human task or problem or desire are you actually addressing? Does someone really want this thing that we're making? A, and then also the thing that we actually can make with the technology that we can make today at a reasonable cost and hopefully not destroying the environment. Does it actually meet the thresholds that would be required so that someone would actually want to use this? Absolutely. Yeah, well, and I think the use case is actually helpful, but also just a wild amount of money being dumped into this. I don't know if you saw Allie Miller's post, but like all these companies are spending money on generative AI. Are they documenting ROI? Like, is this going to be fiscally responsible eventually? You know, as anyway, corporations make choices. No, but this is something we definitely grapple with all the time. You know, let's take something like that. So every time you hit a button on a computer and it's running something off on the cloud or even on a graphics card you might have locally, there is a cost. It's using electricity, it's using compute and someone had to make those chips. And so, you know, we as a designer, when I say designer, user interface designer, someone who cares about how people interact with these systems, we're trying to be very thoughtful about... what signals do you give to someone as to what the total cost of an action might be? You know, we might be abstracting some of that way. We may decide that we're going to subsidize certain things, or you may have a subscription or whatever. But still, we'd like to give you some clue. When you do this thing, this is what the impact of that might be. And I don't mean it's going to say when you click a button, it's going to say three trees were just burned up because you did this. But we should be able to say, OK, when you want to do this, we're going to give you a very, you know, a... First, we'll give you a low quality version. And we want you to triage sort of these lower quality things before you do the next step, the thing that's gonna require more compute and more energy. Not only because that costs money and takes time, but also having those pauses sometimes, while it might seem a little heretical, sometimes having the pause has a better outcome because you're thinking about things and you're more intentional. Absolutely. Yeah, I love that. I don't think enough people think about GPUs, frankly. I agree with you. Well, I wanna make sure that we have time to pivot to you and your path. This podcast is called Your AI Roadmap. And certainly there's the us building it and actually talking about what we're doing. And there's the people who wish they were in the room. People want to be building it, whether they're pivoters, whether they're rising from college, et cetera. Would you mind telling us kind of how you got here on your career journey. Sure, and first I want to qualify that by saying I have many advantages and privileges that most people in the world don't have. Everything from my gender, the color of my skin, my accidents of birth, where I was born, and certain family advantages and stuff like that. So I just want to state that up front. That said, there are certain things that certain... things around technology that are much more available to a much broader set of people in our world than ever were. When I was getting to these areas many decades ago, it required going to very fancy schools and having degrees and having very extremely expensive equipment at your disposal. And when I say expensive equipment, I mean, the first virtual reality gear that I used at university, this device. which is very primitive by today's standards, cost about a quarter of a million dollars. Today I have a VR headset literally next to me that costs around probably $300. So definitely much more accessible in terms of that area. And then if you think about the learning itself, anybody can go on YouTube or Khan Academy or Hugging Face, as you said, and learn these things. in a way that the only limiting factor for a lot of us is time and energy. That said, some of us have more privilege of time and luxury and leisure than others. But in an absolute, these things are much more available now to anybody. So I'm stating all that up front as a disclaimer, saying my path from many decades ago is going to be vastly different than someone coming into it right now. My path was very traditional, academic, going into school, and credentials, and all that other stuff. With a caveat that when I was coming up in the world and doing interface design, that wasn't even a term. There were no textbooks, there were no classes. I'm pretty sure there weren't degree programs in interface design when I was starting out doing this. The people who I hung out with were the inventors of the field, for the most part grandfathers, not... grandmothers because there was a lot of bias and inequity in the field even then. Now many of these areas around interface design are much more codified and we have different disciplines within interface design so it is a different path right now than my own path. So my own path was university, worked in a research group at university, invented some of the foundational techniques and then went to work for a industrial research group at Microsoft for many, many years, then transitioned into making products from Microsoft. Then I went to a series of other companies, some tiny, some much, much larger, but always at the cutting edge of immersive technologies and now into this brave new world of generative stuff. So that's my path. If someone, as you said, wants to knock on the door, wants to get a piece of the pie, is everybody... Should you know what are those paths about? So the hope out there the bright light is that because so much of this is available and I'm not gonna say free Because everything has a cost whether it be direct or indirect But things are tremendously more accessible now than they than they were when I was coming up in this stuff So if anybody wants to come up in the world now the first things I would say are Learn all the things on your own to the degree that is necessary for you to make stories that you can share with people. So what do I actually mean by that? It's not necessary that you have a degree next to your name, you know, to put it very bluntly, when we're hiring people, at least designers, I care much less about what people's degrees are and much more about what did you make. And so what does it mean to make something? You know, right now we've got a landscape, everything from influencers who may be making or maybe just be sharing things that other people made. or may just be talking about things other people made. But there is a large group of people who are, in some sense, amateur makers or untrained makers or self-trained makers. And that's tremendously exciting that they can do that. Think of someone like Karen X. Cheng, who she used to be just one person. Now she has sort of a little mini studio that makes these things, as do various other people out there. So the first thing I would say, again, coming down, boiling down the teal of the art is, if you want to get into this world, if you want to get hired, if you want to get noticed, is make something. An artifact, whether it be something you write, a video, a website, an app, or a mock-up of an app is way more impressive and interesting than just talking about something or telling me that you did three classes in something or of such and such degree. Make something, learn how to tell a story about that. Try to really bring to the fore what the impact of this thing is. And I can unpack what I mean by impact a little bit. Impact doesn't necessarily mean you made a company $5 million. Impact is really about how did your efforts change the world? If you hadn't done this thing, what would not be different? And so there's a lot of different vectors for impact. Impact could be, I caused these two people to talk to each other who weren't already talking to each other. I caused someone to be surprised. I caused someone to learn something they didn't know before. I got someone to change their mind. Those are all impact. I got someone to stop doing something they were doing before. All of those are impact. It's not necessarily about money. It's not necessarily about shipping something. It's about change. It's about moving the needle. So when I'm looking at resumes, or I'm looking at portfolios, or I'm talking to people, the things I look for are, did you make something that is beautiful, that gets me emotionally engaged, and does it demonstrate impact? And do you know how to tell a good story around that? That's a lot of different pieces. Some of those are easier to learn than others. I would say one of the hardest. areas for anybody to learn around this domain of getting a job is actually networking and getting connected to people. There aren't really meaningful playbooks for that, and there's tremendous inequities in that process. But we can talk about that more at part two. Yeah, well, I couldn't agree with you more. And frankly, a huge chunk of my book is about networking and storytelling. So we are certainly on the same page about how to be in those rooms with these technical opportunities that you're building and iterating and then communicating them. I think it really has to have both of those parts for me. So I do want to say, so you mentioned the phrase be in the room and you'll often hear people say, you know, how do you get in the room? How do you get a seat at the table? I think those are great first steps. I think it's also really important to say, so you got in the room, you got to the table. Did anybody listen to you? Did they listen to you? Did they make a decision based on what you're saying? Maybe you did you get to create the next table in the next room for people? And these aren't necessarily my sentiments. You know, I'm trying to echo some of the sentiments of other people in the product equity world who really have some wonderful turns or phrases and metaphor to really capture that it's not just about sort of pithy statements, it's about actually power in some sense. You know, who has power in these, who is able to enact change. And I think we are at some very interesting moments in how available these technologies are to a much broader swath of people than previously. Absolutely, I couldn't agree more. Yeah, well, one of the things I'd love to talk to you about, especially as you have this long, illustrious, very cool career, is some of those decision points that you might've made along the way that maybe maps on to one of our listeners' experiences. Like even going back to your college days, being interested in this field, trying early VR sets, how did, amongst all the majors and things you could've pursued, what drew you to this path? This may sound a little counterintuitive, but I think being lazy led to a lot of the decisions. And so what I actually mean by that, that sort of flip statement is when I'm making something, whether it be in the digital domain or whether it be physical, because I have a background in sculpture and actually making physical things, there's a limit to my skills and my patience in making things. And that's why I have... always embraced these new tools that help bring the, that do the precise part that I can't necessarily do and do the things that scale that I can't necessarily do. And then even, again, going from the abstract to the specific, if I come up to sort of that nugget of what's driving all that, you know, it may be a little flip to say lazy, but really underneath it is very selfishly, I feel like I have amazing things in my head and so does everybody. So, you know, someone wakes up, in the morning and they had this amazing dream, these cool visuals and they saw this stuff. But it's so friggin' hard to make that, to bring that out into the outside world, outside of your head in a form that other people can experience. We have some people out here who are geniuses, who may be poets, who may be spoken word people, who are rappers, who are painters. I'm not those people. I'm not a genius in taking what's in my head and bringing it. to the world that other people can experience in the way I want them to. So that desire to be able to take something out of my head and enact it has driven all of my decisions. Almost all of them. Yeah. Oh, that's incredible. Well, and I love your version of even the concept of lazy. I recently heard an interview of someone who has ADHD and talks about it very publicly. And he says it's one of his superpowers because his ability to clear out all the other stuff and just focus and be wildly unavailable to these other things has been one of his strongest points in driving some of the companies he's built. And so I think it's just, you know, we all have different ways our brains and... and mine's work. I'd love to also speak a little bit about maybe your time at Microsoft and how you mentioned kind of you pivoted to product or there was a transition during that point. Would you speak to that? Sure. So there are multiple reasons I did that. One is I felt like I had hit a wall in terms of what I could make in the research context that I was in at that point. The kinds of problems we were trying to tackle, the richness of interaction that we wanted to bring really required a much larger effort. Most research groups are actually run very, very lean and rely on kind of a PhD model where people tend to be self-sufficient where not only can they come up with the idea, but they can make their own prototype, whether that be in code or by other technical means. I'm not that person. I'm not a coder. I'm not a person. I'm a draw and sculpt and saw kind of person. So I hit a wall in terms of how many of my ideas I could realistically make. And I didn't have the... storytelling skill or charismatic skill, whatever it is, to marshal the resources around me within the context of research to get the resources to make these things. So that's one of the reasons I wanted to move to product. Also, as I talked about much earlier in this session, at that time there was this huge divide between the research world and the product world and very little of the stuff we were doing in research made it out the door to the broader audience. So I was very coming back to that word, impact, and one of the vectors of impact is actually getting people to see your things. So then I did move on to working on creative tools in a product team. There were still very interesting problems around interaction, around scale, around representation, still around 3D, things like that. Trade-offs between parametric design and sort of handcrafted design. But the things we made did ship. That was very rewarding. Yeah, that's so cool. Well, I think it's interesting as you talk about who drives what's being built and the thoughtfulness and which teams are speaking to each other. It reminds me of a story my friend told me recently, which seems so bizarre, but his company recently, he was to lay off a lot of R&D because they couldn't sell it. And marketing got so frustrated that all these cool things were being created and no one wanted to buy them. But they were like, something is wrong about this sell and buy or build and sell. adventure and we really need to like streamline and transition to what's actually like the marketing team was doing more listening to users and then choosing what to ask R&D to really double down on. And I thought it was such an interesting way of thinking about resources and sprints and I was like cutting R&D why would you ever do that as someone who works more in that space. There's a fascinating kind of push and pull I think for that company and at that stage. Yeah, and I feel very fortunate where I work now. So I am in a product team, but we actually have a small group about maybe four or five people within our product team who their sole purpose is to do tech transfer from research. And that is a luxury and it's an amazing luxury. And I really value that and don't take it for granted because I haven't had that anywhere else. Yeah, I was going to say, for people interested in working in Adobe, you're just making it sound better and better all the moment. When you had great recommendations for folks thinking about jobs and storytelling and building projects and so forth, when people ask you about kind of certificates or programs or kind of that educational piece in programmatic form. Are there places you might recommend to people or kind of news outlets or what kind of resources might you So I'm going to be, again, this may be heresy, but I'm going to say, when it comes down to it, when the rubber hits the road and we're looking at portfolios and resumes, and that is the main sort of thing that we look at when we're screening people, I don't care where you went to school, I don't care what certificates or bootcamp you went to, I am looking at the thing you made. And I don't care if you did it for school, I don't care if you did it for a company, I don't care if you just did it on your own, I'm looking at the thing you made. So, you know, take that and do with it as you will. But if you have a fixed amount of energy, and most humans do have a fixed amount of energy, and you're choosing to say, you know, do I do this boot camp? Do I get this certificate? Or do I actually sit down and watch a bunch of videos on how to make something? Make the thing. That would be my leaning. That said, that's very particular to areas where you can make an artifact. So, you know, the visual domain. things like programming. It's not so true if you're say you want to be a user researcher, which is someone who's looking at the phenomena of how people move through the world, whether it be qualitative or quantitative. For that you absolutely have to be degree and credentialed and very rigorous training and established techniques. sense? I like all of it. when you think about, are there certain social media things that you read a lot, different medium people, or because a lot of people are always asking me like how do I upskill? How do I keep my knowledge fresh? What recommendations might you make to the people who ask those questions? So again, making a contrast between when I was coming up in now, not only is much more of the technology accessible, but people are sharing how they make things. So if we go very specific, say generative AI, there's generally sort of two sort of free systems out there where people are crafting their own pathways, their own workflows, one being hugging face that you mentioned, another being comfy UI. And both of these consume various open source or freely available models. So what I suggest is you start getting on the communities, whether it be Reddit, Discord, wherever they are, LinkedIn, and start first gravitate toward the artifact. When you see a thing, it's like, look at that cool synthetic video. How did they make that? And then try to get past the influencer and try to get connected with the actual person who made the thing. And... Some of those people are actually very generous in sharing, not necessarily one-to-one, but with a broader broadcast, how they actually made the thing. And there's a reason that they're comfortable doing that is because the technology is moving so fast. So if they tell you how they made it last week, they're probably gonna make it in a different way next week. But in any case, there's an amazing amount of just public sharing. And again, I'll say Reddit, Discord, LinkedIn, those are the three main. things that, and then there's that other social network with a single letter that I don't browse myself, but other people send me things from. And there's a tremendous amount of generosity out there. Again, don't expect people to write you back and then give you information. They're not more broadly sharing, but people do say, you know, I here's, here's the thing I made on comfy UI. And if you crack open this PNG, it actually has the network in it. And here's the things, and you can make this thing yourself and do your own tuning. Again, that's a lot of word soup right there. But it is something that is learnable. That said, if you start going down one of those roads and you find this isn't fun, it makes me feel stupid, and I'm not getting the results I want, well, then maybe that's not the right path for you. And you may be barking up the wrong tree. think it's good to turn into yourself instead of just saying like, this is prestigious or I think this will make me money. If you keep getting negative feelings bubbling up and it isn't joyful or playful in any respect, then to really listen to oneself. But, you know, and we're almost at the end, but I'll come back to the money thing because everybody has to pay the rent. I mean, you know, in the country I live in, people have to pay for healthcare, unfortunately. When we are hiring right now for designers, and that's the field I'm in, user interface design, really that's what we're looking for. We're looking for people who have experience or a point of view or skills in the areas of generative AI to a first approximation. So if you want to get a job, I would say, you know, get on that bandwagon, but find your own particular, you know, the way to differentiate yourself is to have a strong point of view, be good at storytelling, demonstrate impact. that. I love that. Well, that sounds like a really great place to leave it. If people want to learn more about your stuff, I know you have a fantastic website. Where can I find you? The first is a start on LinkedIn. I'm Daniel Robbins. I think I'm pretty easy to find on LinkedIn. I do a lot of informal mentoring. People reach out to me all the time. I will be honest. I am pretty protective of my time, and I tend to give my time more toward those from underrepresented groups. But please reach out to me. I love giving career guidance and coaching on an informal ad hoc basis. And especially because it's informed by being on the other side of the table where we're hiring people and I know what we look for Well, thank you so much for sharing your time here with us. I'm sure that listeners are going to utterly love it. Thank you and have a wonderful rest of your day. Thank you so much and have a restorative weekend. too. Oh gosh, was that fun. Did you enjoy that episode as much as I did? Well, now be sure to check out our show notes for this episode that has tons of links and resources and our guest bio, etc. Go check it out. If you're ready to dive in to personalize your AI journey, download the free Your AI Roadmap workbook at yourairoadmap .com / workbook. Well, maybe you work at a company and you're like, hey, we want to grow in data and AI and I'd love to work with you. Please schedule an intro and sync with me at Clarity AI at hireclarity .ai. We'd love to talk to you about it. My team builds custom AI solutions, digital twins, optimizations, data, fun stuff for small and medium sized businesses. Our price points start at five, six, seven, eight figures, depends on your needs, depending on your time scales, et cetera. If you liked the podcast, please support us. Can you please rate, review, subscribe, send it to your friend, DM your boss, follow wherever you get your podcasts. I certainly learned something new and I hope you did too. Next episode drops soon. Can't wait to hear another amazing expert building in AI. Talk to you soon!