Behind the Scenes of MailChimp and TurboTax: AI Literacy with Nicolle Merrill of Intuit

In this conversation, guest Nicolle Merrill (she/her), Principal Conversation Designer at Intuit, discusses her role in designing conversational AI experiences and prompt engineering for generative AI products. She emphasizes the importance of understanding the problem being solved and setting clear boundaries for the AI solution. Nicolle also highlights the challenges of evaluating and measuring success in generative AI and the need for experimentation with metrics. She shares her experiences with prompt engineering and the creative aspect of writing prompts, as well as the surprises and complexities that come with it. Finally, Nicolle and Joan discuss the evolving nature of AI design and the democratization of AI technology. Nicolle discusses the evolving field of conversational design and the challenges and opportunities it presents. She shares her career journey and offers advice for aspiring conversation designers. Nicolle also talks about her company, Boring AI, which focuses on AI literacy and training teams to apply AI in the workplace.


Nicolle Merrill Quotes
🎙️ "How do you design with AI? I mean, this is such an exciting field."
💡 "You have to anticipate what the user is going to do and design for it."
🤝 "The ability to dispel assumptions and co-create with stakeholders is crucial in AI design."
💬 "Anyone can build a chatbot. And that is nerve-wracking."
🔧 "What are all the pieces that come together as I'm building out this particular solution?"
📊 "You're on point with the data part. I can't not think about data all the time."


About Nicolle Merrill
Nicolle Merrill is a Principal Conversation Designer for fintech products at Intuit. For the past 6 years, she's worked at the intersection of AI and design. In 2023, Nicolle shifted into prompt engineering and generative AI product design. Nicolle is also the founder of Boring AI, an AI literacy training company and author of the book, Punch Doubt in the Face: How Upskill, Change Careers, and Beat the Robots. She spends her work days asking a lot of questions and investigating how humans interact with AI.

Connect with Nicolle on LinkedIn: https://www.linkedin.com/in/nicollemerrill/
Connect with Boring AI:

Chapters

• 0:00 Introductions & Overview
• 1:52 Intuit Assist and Generative AI
• 3:25 Conversational Design and Prompt Engineering
• 5:04 Designing for Happy Paths and Unhappy Paths
• 6:40 Setting Boundaries for Conversational AI Experiences
• 9:49 Collaboration between Designers, Engineers, and PMs
• 12:42 Metrics and Evaluation in Generative AI
• 15:27 Challenges in Evaluating Generative AI
• 18:00 Novel Use Cases and Metrics in Generative AI
• 20:15 The Future of Prompt Engineering and AI Design
• 23:08 Architecture and Dataset Quality
• 24:40 Representation in Datasets
• 26:22 Transitioning from Career Coaching to Conversational AI
• 28:51 Creating a Portfolio and Showcasing Skills
• 31:36 Learning Conversation Design
• 32:49 Qualities of a Conversation Designer
• 34:21 Advice for Job Seekers
• 35:42 Boring AI
• 36:45 AI Literacy and Diversity
• 37:55 Balancing AI and Human Interaction
• 40:16 Connect with Nicolle
• 40:32 Outro

Transcript

0:00

Hi, my name is Joan Palmiter Bajorek. I'm on a mission to decrease fluffy hype and talk about the people actually building in AI. Anyone can build in AI, including you. Whether you're terrified or excited, there's been no better time than today to dive in. Now is the time to be curious and future-proof your career, and ultimately, your income. This podcast isn't about white dudes patting themselves on the back. This is about you and me. and all the paths into cool projects around the world. So what's next on your AI roadmap? Let's figure it out together. You ready? This is Your AI Roadmap, the podcast. Hello, hello! Good to see you. Could you please introduce yourself? Yes, my name is Nicolle Merrill and I am a Principal Conversation Designer at Intuit and also the Founder and Principal Trainer at Boring AI, which is an AI literacy training company. Boring AI, I dig that. Okay. Well, I'd love to dig into some of your stuff at Intuit, it's the company behind TurboTax, QuickBooks, Mailchimp. Basically, everyone listening to this has probably used one of those products. Oh, yeah! No, can you share a little bit about what you do? Yeah, so I'm about three years into that role, and I'm a conversation designer, and so I've been primarily focused on the enterprise chatbot for QuickBooks. So I work a lot with solving problems for small business owners, whether it's like accounting or finance, helping them understand their finances and so on. And in the last year, it's gotten even more exciting considering Generative AI came out, and I've evolved more into moved from kind of conversation design to prompt engineering and designing for Generative AI products. Totally. Well, and for people who don't know that, oh gosh, I'm like, do we talk about the product? You're like, no, yes to all the above. I think, I just looked at the keynote and seeing the opportunities for little query bars or kind of chat bots integrated into product suites. Can you talk, people who haven't seen it, through kind of features that are being publicly displayed? So in the last year, Intuit released Intuit Assist and it's a generative AI financial assistant. So it helps people understand their finances better. So like financial insights, get answers to questions about their business and taxes and so on. And so it's accessible in different areas of the product. It depends on which part of the product you're using. You may not see it at times, you know, like all products that are in the generative AI space. It's certainly an evolution, right? And so it is assistant based in kind of that chatbot way, but it's also integrated into the product as well as kind of a feature. Yeah, that makes sense. And certainly, aren't we all asking ourselves questions when we look at, you know, like, and look at my finances, hey, wait a minute, this invoice, did it go through? Like, we are looking there for insights in theory. So that's, well, and as people, I think people are more comfortable with prompt engineering these days, but as far as like conversational design, could you talk about some of your background and kind of your... how your vestiges of your hands are like touching a product. I love to hear. Yeah, so this is my sixth year in conversational AI. So working as a designer, I describe it as making computers talk. I've been primarily in the, originally I was in HR, so working on HR chatbots that help companies hire candidates, so like talent screening. And now I'm working for Intuit, I've been in the FinTech banking payments space, pretty regulated spaces that I've been in, which is kind of funny, I'm a very creative person, so never would have thought I'd be working in these very, regulated spaces, but it's been very interesting to kind of learn about how I can, you know, design experiences, conversational experiences to help users, right? Get things done. That's really what I'm doing with this. I started out kind of as a conversational analyst and then moved into more of like working with client facing and helping design conversations. And I was laid off at COVID then got hired at Intuit. And it was like, okay, I never thought I'd be But it was such an interesting learning experience to move into enterprise, right? Here I'd been in startup land, kind of doing everything. I was the only designer on an AI team. And then I moved into this enterprise space, with like a team of conversation designers and a manager who was incredibly knowledgeable in the space. So I've had this really cool opportunity to grow my skillset and then just get exposure to product. Right? I started on just a chat bot. So then I moved on to like... a chatbot within a product. So what's the relationship to the product? And then the last year I've moved more into product itself. So an enterprise product and looking at how features like generative AI features are integrated into the product. And so it's kind of this evolution of AI. How do you design with AI? That's kind of where I'm at right now. My specialty at the high level is being able to... design interactive experiences with AI and with language models, it's the only model you're going to talk to. You're not talking to a recommendation model. So it's really interesting to be able to use that skill set over time and in the right place at the right time for generative AI to come out. Totally. And how do you design with AI? I mean, this is such an exciting field. And frankly, I feel like people keep forgetting. Or do you go to happy hours too, where people like, someone built that? Or somebody who's a human? But I'm happy. Yeah. And really thinking about like, I'm always thinking about how are people going to interact with this? I'm a customer first mentality where it's like, okay, you're a brilliant data scientist and a brilliant engineer, and you've built this solution. But like, customers aren't necessarily going to do that. Right? Everyone makes these assumptions about what customers are going to do. We know from conversational AI, we have what's called unhappy paths, right? Those are all the paths that a user might take that you did not plan for. Yeah, and it's the same with, you know, when you're designing with AI and product, right? Customers may not do the thing that you think they're going to do. So you have to anticipate that and design for it. Is there a chance you could give us an example of the happy path and unhappy paths or I think from a conversational point of view, right? You're like, okay, I'm going to design a pass where you're going to create an invoice. It's like, great. Ideally, a user will ask, can you create an invoice for Joan for $100. And you're like, sure, here's your invoice. Well, what about the user who's like, what if the user asks for an invoice for Nicolle? But there's no Nicolle in the customer database. Then what? What do you do? Right? And if you don't have a solution for that, You might show something like, I don't have a customer like that. What can I help you with? Or you can respond with nothing, right? And sometimes people, if you're not used to designing for conversations, you don't design for those paths. You have to have a path for, to help when someone hits an error, right? So you have to think through all these different scenarios in a conversational experience and really anticipate what is the user going to do? What are all the ways this can go wrong? I'm good at thinking about things that go wrong, right? Like it's like kind of my specialty. I don't think that's. It's not helpful sometimes when they're trying to build, you know, and I'm like, wait a minute. But it's something you got to anticipate in, you know, language models now, like with LLMs, are better about, you know, being able to adapt to that unhappy path, but you still have to have, you have to set boundaries, right? So that way things don't go off track. And that's another, that's also something that I do in my role, defining the boundaries of your experience. Yeah, I make jokes with people about like bots that I've seen broken with emojis Like someone wants someone to put the smile emoji errors error error. We don't know Or hi just hi. Oh, no error error. We sent you to an agent Yeah, I did one the other day I was trying to break and I started translating things in French and then I got it to speak French and then it later told me I'm only, you know, I can only answer in English and I was like, well, that's not true. So like, it's like satisfying every time I get like a, you know, like a win. I'm obsessed with it. there are so many fun ways to break boundaries. Yeah, when you think about boundaries? I think people right now are thinking so expansively. Hopefully, they're listening to this. You know, thinking about like large language models. What are they? Where are they? Boundaries, hallucinations. I feel like anxiety just begins bubbling real fast. When you think about kind of boundaries reading experiences, how do you think about boundaries, guardrails, that kind of stuff? Yeah, I think about it in terms of conversations with stakeholders. Right? So a lot of times what will happen both internally and just externally, when I talk with folks on projects is you've got someone that's like, we got to make it generative, right? And it's like, okay, now we're leading with technology and not an actual business solution. Right? And my specialty is having worked in this space for, you know, six years is mapping an AI solution to a business problem, right? Or not, or saying, actually, you don't need to solve it with AI. And so for me, when we talk about boundaries, it's really getting clear on what's the problem we're solving here. Are we solving a pain point for your customer? Is your customer internal or external, And then is generative AI actually the best solution here? Because there are trade-offs, right? And so really getting clear on, I do this exercise called should, should not do, where we really define like, what should it do? Let's get specific. And then let's get specific about what it shouldn't do. And that starts to create these edges to the experience. And when you have multiple stakeholders in a room and having that conversation, so I'm used to working hyper cross-functional, right? I got data scientists, engineers, PMs, leadership often, right, because they're there too to understand ideally. You get to have a better conversation about the solution, right, you get to have a better solution coming out of the gate rather than waiting to after you've kind of implemented and you're looking at it and you're like, well, what is going on with this product, right? What's going on with this solution? So for me, Boundaries is about what does it look like in the beginning of the project, to be able to kind of shape what we're building, right? There's a tendency to kind of just let the model do it, but that doesn't work, You have, otherwise you're gonna get some weird stuff coming out of the model, you know? And I think that just depends on kind of your perspective and what you know, and because I sit cross-functionally, I understand the different functions and approaches, and I can bring that together to kind of facilitate those conversations. Absolutely. Yeah, I know that makes a lot of my practice right now. We're doing like we start with a stakeholder workshop because I find biggest derailments has happened in projects where a few quarters in. Someone says, but that's not what I wanted. You should have known. And I was like in the meeting or like one in the meeting versus like handing the mic. Like has everyone spoken and been given space to say what their goals are? Because if people articulate what their actual goals are, as you mean, with this should and should not, it's like should not build X. and you should not. You should not build these. I thought that's what we were thinking. Yeah. I find that, no, you're absolutely on point with that. I find that ability to actually dispel the assumptions. There are so many assumptions right now that people bring to building these experiences. We've heard about it, or I've got pressure from so and so. I got to implement, or the PM's like, we got to ship it. There's all these perspectives that are coming in. So it's really my role to kind of like... you know, as a designer, right, is to bring those perspectives in and say, okay, are we all on the same page here, you know, and what does this realistically look like? And then turn to my partners and say, can we build this? Right, is this feasible? And many times we think of the PM as that, but the PM is really just kind of, you're partnering this to map that out, right? I'm the person who says, okay, let's get everyone together and make sure we've got everyone on the same page and co-create together. I find that when you're building with generative AI, and also other AI solutions, it's a co-creation process. Right? It's not, okay, PD hands off, then engineering, design says something and then hands to engineering and then we implement and ship. Right? It's far more iterative and testing and coming back and saying, especially with generative AI, the outputs are just so wild sometimes that you're like, you gotta get everyone together and be like, is this what we agreed on? Right? Like, and that helps with those assumptions and that brings people along on the journey. that's really what my role is. Absolutely, yeah, I was just speaking with Stephanie from Microsoft and she was saying like the role of design and PMs she feels like has been shifting and I feel like it has been so previously in projects I worked on so hierarchical and I feel like as you're talking you're like there's a respect or like equity that is changing in dynamics not to say people aren't treating each other well but that just that shared vocabulary or responsibility Yeah. looks different today perhaps. Yeah, I would agree. Especially when you get a good PM partner that it's like, you know, where they're on board, they're learning too. I find this space, you know, the people who are really open to learning right now are doing pretty well, right? That ability to just take information in and learn from those perspectives. You know, I'm not an engineer, but I need to co-create with engineers. And so I'm finding the engineers that teach, right? And I'm sitting with them and they're telling me things and I'm saying, okay, but that can't work because of this, so tell me what could work. Right? And so like being able to, you know, have that partnership is so important. The same with data science, too. I have a data science certification, but I'm not a data scientist. Right. And so I know enough to be able to like talk with them. But in the end, they're the experts in that. And so really, it's that ability to kind of know where your knowledge area ends. Right. And ask the questions and kind of bring those things together. And I think that's that to me is what it's like to really design with AI. Right. And as we move into the generative space, and I think just in general as a skill set, as we look at the future of work, as IT is not separate, right? Tech is embedded in what we do. So that ability to kind of not be so siloed and work with these different perspectives and across functions is really critical. Right, well, and this bidirectional curiosity, like to learn from each other. It's like, I respect you, you do your own thing. as an ex-academic, like just the interdisciplinary approach, which is still relatively modern. Literally in different ivory towers or brick buildings that are separate things. Okay, so are there potentially different. metrics of success you could share about products. on this one, I think with generative AI, we're still figuring it out. What does it look like? Because it's hard to evaluate. There are traditional metrics that you can use, but it still does require some degree of human review. I know there are debates out there. We should automate all the evaluation, but I actually quite disagree with that because it's language, right? And language has nuance, and your customers aren't going to interpret it like a machine. So for me, while I don't necessarily can't really expand on the metrics right now, I think it's more of an approach to metrics and that kind of experimentation right now on metrics. I think in a conversational space, we have some metrics about conversational turns and completion rates. I think that just gets harder to do in a generative context, right? If you're having a generative conversation, who's to say that like 10 turns in a conversation is too much now if the bot understands context really well and it can switch contexts, right? It's harder to know when things go off track, if it's a generative conversational experience, right? There are tools out there that are evolving, but I think it's just such a new space that evaluation is just such an open field right now in generative AI that metrics get kind of fuzzy. And I think that's where I really am in conversation with engineering partners on what does that look like? yeah, I can imagine even as an end user, like you're using a novel product. How can you, you know, a traditional chat bot versus, or like I saw the demo related to MailChimp and different like email marketing and different products, adding different things. My mind just processing that I can do this within the interface about a newsletter I'm sending. Honestly, I need a minute. I'm like an end user to, and of course in two years, this will be just potentially a norm. But right now my brain is trying to catch up. Yeah, well and then you have, you know, you've got your generative. This is another one too, because I work primarily in conversational AI and from that background, you know, now the conversations I'm trying to teach conversational designers like think outside the conversational box, right? Because generative is also just these, generative content experiences. We've got generative experiences that complete a task in product, right? We've got, you know, generative experiences that... read data and spit out a summary of it, right? These are very different experiences than a conversational experience. And yet we're using a language model for both. And so, you know, when I talk about, you know, generative AI and metrics or like how it's used, I try to, I need to always remember to separate. Like it's not necessarily always conversational. Could very well be something like what you're talking about in a newsletter, right? Like I click the button and it generates something for me. in a chatbot, I find it really interesting because not all chatbots do the same thing, right? I have a chatbot that I'm obsessed with. It's called Pi, and it's by Inflection AI, and they, I love that bot. I use it so much, more than I would have ever thought, and primarily for brainstorming, sometimes for refining things, but mostly for brainstorming, because it's better than ChatGPT. And I think about that in the context of traditional metrics. If I'm having a conversation that goes on for five minutes, from a quantitative point of view, that's a lot of conversational turns. I might find that's like, wow, you're outside of my expected five turns, if that's my metric of a good conversation, right? So how do you apply these metrics to new types of conversations? Before generative AI, we weren't necessarily having a brainstorm conversation with chatbots, right? You might mostly no. Right? So even the type of tasks that these assistants are doing have changed, which means our metrics have to change. Absolutely, right. And I was like, oh, is Nicolle making an outlier? And then I'm like, no, it's just a novel use case. Like we can't, it's blue and green. It's just very, absolutely. Are there surprises with these new challenges that you're finding? Any personal Easter eggs? I think prompt engineering is a surprise. It's, I think I wrote this before, but I love it and I hate it and I love it all over again. I find it to be one of the most interesting, interesting skills to learn because one, I love the creative aspect. I genuinely believe like creative writing, if you're good at creative writing and words, you're gonna make a great prompt engineer. And I know right now we really see it in the domain of engineers because engineers can build the backend and they can. you know, chain your prompts together. You can do a lot of good stuff, right? Helps to have an engineering partner when you're prompt engineering. But as personally as the person who writes prompts, like it's super fun. And I love to do it. And then sometimes I hit a wall and I'm just like, I hate this. Like, why can't I just get the output? You know, like I can't tell you how many times I couldn't get it to stop saying a phrase over and over and over. And I was like, I don't want you to say it. And I'm just trying and I can't get any more. You know, and so then I just get really frustrated and it's like, Oh, what is it with these things? Because there's no explanation. It is a black box. You don't necessarily know why it did something or did not. And that's hard to work with. You're used to trouble. If you've even been in the technical space, you're used to troubleshooting, you're used to kind of explanations or something you can backtrack. And it's not that way with product engineering. And so I was surprised by it because I thought it was a lot more straightforward. And the more I work with it, the more it's like, everything depends on your use case. So when they're like, do this, do this, this one thing will work. And it's like, well, that might work for your use case. But if we're combining all sorts of things with the prompt, if we're using other techniques, if we're, you know, it's, if it's part of a chain, so many things go into the making of the prompt. And I was really surprised by that. It's not necessarily as cut and paste as what I thought it was going to be. Absolutely, I would agree. And I think that I teach some ChatGPT courses in so far. It's prompt engineering could be used for I think that like people still are shocked that like you can make tables, like you can write code or like the it's it doesn't just have to be reformatting emails or popping out poems. I frequently have to refresh it though, because sometimes it does just spew out who knows what. Um, but when it when it is really helpful and I need to just reformat something easily, instead of looking at the code, I can ask in relatively plain language what I want. I bet people are widely underusing it, utilizing it. I teach the same courses. I was just doing one the other day on using it for high-level data analysis, like contextual data. And I was like, okay, I went over to one of the websites, downloaded a bunch of data, a data set on employee reviews. I was teaching a group of HR folks, and I was like, okay, let's see what the employees had to say. And it was like sentiment analysis, but actually defining what categories you'd like to analyze through. So getting real specific. high level takeaways and then can you create a, you know, recommendations for improvements, like things like that where it's like, going beyond just kind of a, make me an email, right? And I think that's just because we're all still getting used to it and at the same time I've noticed that the needs for prompt engineering have changed over the last like six months I've been teaching it and people are a little bit more interested in like, okay, but what can I really do? And it's that kind of like next level, like what you're talking about, where it's like they want to know more, right? And can you contextualize this for me? Because I think, you know, I saw some stat with like 40 something percent of companies like banned generative tools at work, but like 40 something percent of people are using them anyways at work. And it's like, yeah, I mean, like, how could you not? You know? And I think there's that innate curiosity there and people see the value in them in getting work done, right? And I always talk in my courses, but this is the first time you can get hands on with AI. Most people weren't building recommendation models or pricing models before they had GPT, right? So you can get hands on and play. And this is just such a different relationship to AI now. So I don't know. I'm all for it, but it's definitely, it's still, we're still evolving inside of companies, right? Companies are still trying to figure it out. Oh, for sure. Well, and then just the democratization. I mean, just to have it like you can have ChatGPT on your phone, like power that you were just touching. Like it's kind of mind boggling. I would love to. I'm sure people are hearing this and being like, how do I get into this field? How do I get Nicolle's job? Before we get into career stuff, like when you think about prompt engineering and we're talking like wild things that have happened just in the last six months. when you kind of extrapolate out in our fields, where do you see this part of the field heading? Well, you know, it's interesting. I was thinking about it from a conversational design point of view because, you know, it's interesting to see now anyone can build a chatbot. Right? And before it was like, oh, you know, I'm building the chatbot, right? Like I have these special skills. And now anyone can do it. And that is nerve wracking, right? At the same time, I know that I have like a lot of skills around it's not just creating a chatbot, right? Like we said before, it's like design and design facilitation and things like that. And so for me, I really think about, you know, as I go, what am I learning through prompt engineering, right? So prompt engineering is just the task I'm doing, right? But because, so I do prompt engineering in my day job, right, as a prompt engineer, but I also do it on the side. I'm building all kinds of bots that I just wanna build, right, because like all these custom GPTs for fun, because it's like, can I get the bot to do this? And it's like this creative writing trick, right? And so I'm doing it, and I just do it a lot for my workflows now, right, to run a business. So I'm doing so much of it. And for me, I'm really paying attention to, you know, from what I'm doing in my day job, I'm paying attention to, OK, what else is connected to this? I learned so much more about engineering because the engineering shapes, you know, how the architecture on the back end is built shapes how I'm going to write my prompt, right? If I'm using data. I need to understand the data structures. I need to understand the types of data, right? And so it's really been this learning process of kind of looking behind the curtains and saying, okay, yes, I do prompt engineering, but what are all the factors that go into making this happen? And I think, you know, I did some of that before, but it's far more intentional now because I feel a little bit threatened in my job of like conversational AI, like not knowing. It feels very ambiguous right now what's gonna happen in our space, to designers in particular. And I know that's a conversation we're having in our industry. And it's a conversation a lot of people are having in their industries, thanks to generative AI. But I think in particular for conversation designers, it can feel like, are we, what are we doing? You know, like in the future. And so for me, I'm very much curious about how does this all fit together? I'm a systems person. Um, so I do systems design as well. And so what are all the pieces that come together as I'm building out this particular solution, right? And I think in prompt engineering, you do have that kind of bird's eye view. of because you are the product, you. Your outputs are the product, right, experience. So yes, you have the scaffolding of the product and the visual designers, I don't want to discredit any of that, right? And the engineering that goes into build that is incredible. But you are the one creating the experience, the interactive experience. And so you really have to know how all these parts fit together. And I've found that to be one of the more interesting pieces in the last year, it really surprised me because I'm used to working in chatbots. right? Like I say I'm making this square over here because that's usually where I work in this tiny square of a chatbot. And so you know when I look to the future I'm looking at like what does it look like to put all these pieces together? Absolutely. Yeah. Well and, I think I've been thinking far more in architectures and like the quality of different calls. Like I've been thinking far more in like modularity of APIs is kind of where my brain is really thinking about, but also like, uh, datasets and leverage and how clean they are and who they don't represent. I wonder, or just, I've been doing research on rag models and like, How big of language or data sets do we need if they're linguistic sets? Like size of data, quality of data, like is this gonna be one of the next kind of product things that a lot of companies need to be, or just this is as I, if you're worried about my forum, and daydreaming about where is the field headed, what differentiation is gonna happen at different companies or for different use cases. you're on point with the data part. Like, I can't not think about data all the time. I think about it, especially if you're using a rag solution. I'm constantly, I've even experimented with my own. I've got some, I got one invoice floor right now that is, I have a rag solution in there, but I'm customizing the data sets and the original data set that I thought would be useful once I put it in the bot was not right. And so I went back, so I've actually been iterating on my data set. It's a small data set because the solution is... Just a small solution, but I'm still using it, you know, to experiment with it. And it's been really interesting because usually I'm the person that asks the questions about the data set to get to know the data sets because it informs the prompt. And also like if outputs are coming out that are wildly off, I have to ask, what data are we using here? Right? What did you train this on? And ideally you're brought in earlier, but most of the time you're not. Right? So you're like, I recognize that the data scientists aren't necessarily going to come to you and say, I'm thinking about this data in an ideal world. They would. But so that whole point about who's represented, what's represented, do the data scientists know the subject matter? This is something I'm actually really passionate about. Having been a conversational designer, I've asked that I would consult with subject matter experts all the time. I do my independent research and then I consult with them. And I always found that the two merged together so well. And so for my push on data scientists and engineering is like, who's your subject matter expert here? What assumptions are you making? As a designer, it's natural for me to do that. but it might not be as natural for them. So coming back to that data set, how do we know it's informed by subject matter experts, that the right voices have been consulted and so on? I think that is one of the most interesting areas to go into and I think certainly in the prompt engineering space and conversational AI space, you're set up to move into some of those areas and be in those conversations. You kind of have to beat the doors down to get in those conversations, but okay. I hear that. And I think, or at least having worked in data science before, I'm like, oh, we try to future proof. We didn't know that variable was important. We're all trying our best. I do believe, but I heard about just, there are so many, I think as I am joined this field, realizing how many choices are being made on a daily basis by all these different parties. co-constructing this thing and how, you know, like this AI is actually just for humans in a room or right now, distributed around the world on computers. But like functionally, it's their choices that's deployed to millions. And so that's how I, anyway, my PhD research and Women in Voice, but that's a fundamental thing that when I realized it, it really blew my mind. Okay, Nicolle, let's talk about you. and how you got into this space. You alluded to some of your conversational, just current trajectory stuff. Back in the day, what's the career path? How did you get? Ha! well this is my fourth career. So before this I was working as a global international career coach. So I was doing career coaching and executive training and I was talking to a lot of employers in tech who were like, hey, you know, like, do they know how to work with data, these MBAs? And I was like, I don't know. What do you mean work with data? At the same time, this was around the time of like, if you can believe it, in 2016 the headlines were, AI is going to take your job. And I was like, wait, what? So those headlines were there in 2016 that we're seeing now, same headlines. And I was like, something's up, I gotta learn more. So I was moving, I was moving, so I had to leave my job and I quit and started doing a lot of research, ended up writing a book on the future of work in AI and how to change careers for it all. And in the midst of that, in doing my research, I was going down the vertical, the HR vertical, and started noticing chat bots that interviewed people. I was really skeptical and I was like, no way, they can't do that. And so I was just breaking about them, breaking them and writing about the poor experiences and how they could be better for candidates. And then I ended up seeing a job and they hired me at a startup, right? I was like, listen, I can do this. I can do this. And it got me in and it was really cool. And I think that's what I love about startups is that flexibility. Cause I can't imagine doing that necessarily in corporate. But in startups, like they've just got that like kind of openness to different people. And like I said, I was. I was the only designer on an AI team. That's kind of weird, you know? I didn't know nothing. I just didn't know a lot about AI. But I knew a lot about language. I knew a lot about user experiences, right? And I knew a lot about HR. So I leveraged that skillset to get in, to really find what is my niche where I can really compete. And then I did my research on conversation design. I had like a, hey, I know what NLU is, you know? And I had the basics down. I love that contrarian, like I could make this, I am such an optimizer also, like I could make this better, like this, this could be. Wait, but so the career, so do you think those is distinct? It seems like they flow together, but like this career coach and then into the HR space, into the chatbot space, and then into this AI space, I had careers before this. I was a luxury travel writer before that. And then I was a study abroad advisor before that. And so I sold adventure travel in different countries a long time ago. So like, I've just had this kind of like, I don't know, what's interesting? So I just got to go in the interesting and to get into AI, it was very much like, I was a career coach. I understood HR because I used to be a staffing recruiter a very long time ago. So I was like, I just knew HR enough that when I saw these bots in the HR space, although I didn't understand bots, I was like, I know HR and I know you're not creating a good talent experience for your job candidates. And that's where I think when we look at, I used to coach people all the time how to get a job and I was like, well, do something that gets their attention. And I'm a writer, so I was like, I will write about these things. I don't know that I was trying to get a job, I was just trying to get my thoughts out because I was writing a book. And it just so happens that... Two things happen at once. I wrote a book and I got this job. So I've been living a hybrid lifestyle, you know, split personality since then, right? I think it's that standing out point that a lot of candidates I think miss potentially or I'd love to see. I think it's that standing out point that a lot of candidates I think miss potentially having a portfolio, right? And if you're brand new, yes, it can feel like, oh, I don't have projects, but now anyone can create a chatbot, right? And then start tearing down other people's chatbots. There's so many of them now, right? And talk about what you do better. And I think, you know, we do, in my job, we do experience reviews. Like that is a normal thing that you do as a designer is an experience review. So going through and saying like, this worked well, this didn't work well. testing them out on different users and getting their feedback and presenting it. There's these very creative ways that you can show you understand conversation design that don't involve actually working on a project and building the bot. Because building the bot is one thing, but also being able to understand just kind of these design tactics that you can use to improve experiences. Because we don't just create new experiences, we also improve on the existing ones. iteration is all the time I talk to customers who are like, oh, and then we deploy and like, just deploy. Like that's the beginning of the journey. We're going to learn. We're going to get much better. Right? Or what are your metrics? Like how would you take this bot that you found online and say like how would you measure success? Or compare different bots, right? In the same, there's just so many ways to kind of go about that. And man, write that up, put that in a deck, I don't know, and then send it to the employer that you're applying to. Like they'll love it. That's amazing. Well, and that's exactly, you know, if people wanted to get into this field, they're like, this is awesome. I want that job. I want these projects. You know, what guidance would you give them? Are there boot camps? Are there certificates? Are there degree? Like what tangibly could you offer people as kind of actionable? You mentioned a portfolio and creating. I think having your own, I think if you're gonna be in design at this point, you have to have your own website, you have to have your own just simple, I'm not talking big, just like, who are you, what are you thinking about? Right, I think that's pretty much standard, and I think that's even more so as we start to see kind of the merging of product design and conversation design, because more designers and content designers are trying to move into the conversational AI space. So it's a very interesting time right now, so that does mean more competition, right? And so really thinking about like, how do I showcase myself? And like, I use Squarespace. I used to do custom WordPress. No, I just use Squarespace now to get my websites up, right? And I think being able to just put it out there, put yourself out there and say, this is like what I'm thinking. As a designer, you have to get comfortable with putting yourself out there, your ideas out there, what you've built out there. And then you have to get comfortable with people rejecting it. Right, it's just the nature of it. And so I'd say like portfolio, but yes, go build a custom bot with ChatGPT, go build a custom bot with Voiceflow, go build a custom bot. Like Hugging Face now has some open source, you know, you want to take it step further and go get more technical. Great. Then, then, you know, learn some Python. I think there's a big push to learn to code, but it's like, sometimes when you learn to code, it's like, but what am I coding? Right. But maybe you learn a little bit of Python just to get you to get your first bot built, right? Um, I think that's like really. That hands-on approach combined with what I talked about earlier where you're doing experience reviews, I think are great You know the Conversation Design Institute they have courses on conversation design So if you can afford that or if your employer, you know sponsors You you might be in a job right now where you're a UX writer and you're like, I want to be a conversation designer. Great. Does your employer offer any free assistance? Right, so I would check in with their with your employer or some of these conferences now like, you know I know there's a bunch of upcoming conferences They've got workshops, like hands-on workshops, and you can go, and I've met new aspiring designers there. So I think it's that initiative where you're like, you just can't wait in this space. You have to actually just go do. It's a very much hands-on space. Yeah, absolutely. Couldn't agree more. Well, and I don't know, you mentioned like the size of your team. When you, I don't know how often you bring on new team members, but when you're potentially on an interview loop and looking at other candidates, you know, what do you look for and colleagues? Yeah, well, when I'm looking for people that have a perspective, right? And that can be hard if you're a junior designer, right? But like have a perspective on something that you built. Tell me the why. I'm always looking at like, okay, you built this. What was your thinking behind it? Why did you make those choices? And it can be really nerve wracking, I know, you know, to be, to have to like explain, especially as a junior designer, we've all been there. But that is still what I'm looking for. I'm looking to hear your why, not just your what. So having that story, because a lot of what we do is storytelling, right? I designed this because of this, the business requested this, so my solution fits this need because of this. And here's the data actually that's supporting that, right? We use a lot of data. So if you've already been in a job where you're using data, then you're gonna be in a good place because conversational design requires you to look at the transcripts or look at the quantitative data about how many intents were hit and make a decision based on that. Create a story that you can tell your stakeholders, right? So I'd say that I'm looking for the knowledge, sure, but like the why and like some of these soft skills. Can you communicate well? Can you present? Can you write? Because documentation is something that we have to do. Not all of us are great at it, you included. You get busy, but things like that. So there's like, I think it can be intimidating because it... I've talked to candidates who are like, oh, you know, I don't necessarily have the foundation like in the technical side. That's okay, you can learn that, right? But if you're showing that you're curious and you've taught yourself and you're going after things and you're building things like that to me, she was like, okay, yeah, come on in. Let's get you in here. We're gonna teach you some things, right? So I'm looking for all of that. Yeah, absolutely. I think especially in this job market, and as you talk about, there's different competition, as different people are switching fields or pivoting, you know, what advice might you give people, especially during this job market? I mean, there are like layoffs and thousands of layoffs, like, just what advice might you give someone who is literally going through this right now? Yeah, I think that creativity part, what can you do? I think I know what it's like to be laid off. I've job searched a ton of times, you know, it gets really down, right? You can get really down when you're not getting any responses. I think being able to fill yourself up with people. So whether that's the people in your life that bring you joy during the hard times, or the people that you know in the field, that you're just like, you know, we've all had our former coworkers or our former besties at work that we can turn to, right? So like, you know. checking in with them and then talking to the people. I mean, we call it networking, but in reality, it's checking in with people and saying, hey, like I'm really curious about what you're doing. You know, I just talked to a student actually the other day. It was really cool because I used to be a career coach and from this program, an alum reached out to me who I had coached. He was like, hey, there's a current student interested in conversational AI. It turned out she'd been working on a project, a conversational AI product in her MBA program. And I was like, yes, I would love to talk to you. Like, what are you doing? And like, I'm like, here's this resource and here's this resource. So like people wanna help. And I know it's tough to put yourself out there, but keep going, right? There's no getting around it at this point. Yeah. Well, and you mentioned at the top of the episode, but I'd love to hear more about Boring AI, which is such a great title. Trust me, sir. I picked that title because before General of AI came out, which is kind of like before and after, I would tell people like I teach people about, you know, I teach people AI literacy skills. So if you're from non-technical school teams or you're not a programmer, like I teach you how to understand AI and its impact and how to apply it. And people are like, oh, that's so boring. And I was like, yeah, but I'm not, you know, like I'm going to make it interesting. I guarantee you. And so when it was time to like actually formalize it, it was like, yeah, Boring AI. Yeah. And if you look at my website, like it's So Boring AI. It's filled with donuts and bright colors, right? So it's very different from what you would expect from like AI, because like I do think AI is really interesting and the way that it's been taught in the past is it's so much more focused on developers and data scientists and learning to code and really focused on like, is it supervised learning or un-supervised learning? Like who cares? Quite honestly, if you're not building with it, it doesn't matter. And so really helping teams understand, especially with generative AI, it's much easier now because it's hands-on, really understanding how does it work, how is it applied in the workplace, and then who does it impact? What is the impact of these tools? Because I'm really focused on that impact piece because we are impacted by it, you know, both as a society and as an individual and in the workplace. And so that's my goal with this company is to really train teams because I want more diverse voices in the conversation. You know, I love the data scientists and the engineers. They're building great things, but we need more people from outside of that space to come in and apply their perspectives, you know? And these type of training gives them the confidence, right? You don't have to know everything, but I give you the foundation to really build on and give you tools to move further so you can be in those conversations. Oh, so cool. Well, even the phrase like AI literacy, I feel like this is not yet common vocabulary, or I do. I, in my university is like, there's so many computer science majors around me and it's really mostly one demographic. And so the opportunity today that it's so democratized, like you don't have to go to a class where there's just, you are othered in whatever respect. Like you can just do it at home in your pajamas. Like, I think it's such an amazing opportunity and you're providing such a phenomenal resource. So. I mean, it's a lot of fun. I love because then you get to, I love, you know, it's ironic. I build chat bots and work on AI because I love people. You put me on a stage, you put me in front of people. I am like my happiness right there. So it's wonderful to connect with people. And then they fill me up with their questions and curiosity, right? I am fueled by that curiosity. I'm trying to get everyone to be curious about these tools, right? So it's really reciprocal. So funny, well, and to hear someone who's like working on building AI and bots, and you say how much you love humans, or some kind of... I don't know, is that like dystopian? Or like, I don't, I don't know, do you hear that well? Yeah. No, it's a very I've thought about it, right? It's like it's interesting and I can find myself sometimes working on these products and being like, you know I've said I've literally said I gotta go touch grass Right because the conversations can be so abstract sometimes that it's like I just let's go have a real experience in life Offline, you know, and so really trying to balance that out because it does get very intense. I'd say there are just times where were really caught up in some very specific details about, you know, data or an algorithm. And it's like, yeah, that's not very concrete at all. Right. And you spend days doing that and you're not really interacting with people. I mean, you're interacting with your teammates. When you think about your impact and what your work, what the impact of your work is and your impact as an algorithm, right. For me, it's not as that it's not what I'm craving. You know what I mean? So I have to seek out these tangible experiences elsewhere, you know what I mean? With people and good conversation, right? So yeah, it's really kind of like an opposite end sometimes. I'm so gonna use that like, let's go touch grass. Like that's a, my dog is excellent for that. Of like, she screens, she's like, let's go out. Like immediacy and attention, like the way she thinks about the world and kind of literally pulling me out the door is extremely helpful as a counterpoint to back to back Zoom calls, building architectures. That I hope my most utopian world is that you know, these tools like automate some perfunctory things that don't bring joy to my life and allow time to be human, to go on the hikes and runs that I cherish. Still far from that, you know, as we get faster and faster and faster. But I think that's where I hope the next years are going. Yeah. Well, I'm sure she said on the rap. I'm not sure she said it exactly like that, no. time right now. We're all trying to figure it out. Well, thank you so much for your time with us. If people want to learn more from you, hear about you, where should they go? You just follow me on LinkedIn, connect with me on LinkedIn. I'm pretty much there asking people about things all the time. And then you can go to my website, SoBoringAI.com to find out more about my work. Cool, well thank you so much, it's been a pleasure. you! It's been so great! Bye! Oh gosh, was that fun. Did you enjoy that episode as much as I did? Well, now be sure to check out our show notes for this episode that has tons of links and resources and our guest bio, etc. Go check it out. If you're ready to dive in to personalize your AI journey, download the free Your AI Roadmap workbook at yourairoadmap .com / workbook. Well, maybe you work at a company and you're like, hey, we want to grow in data and AI and I'd love to work with you. Please schedule an intro and sync with me at Clarity AI at hireclarity .ai. We'd love to talk to you about it. My team builds custom AI solutions, digital twins, optimizations, data, fun stuff for small and medium sized businesses. Our price points start at five, six, seven, eight figures, depends on your needs, depending on your time scales, et cetera. If you liked the podcast, please support us. Can you please rate, review, subscribe, send it to your friend, DM your boss, follow wherever you get your podcasts. I certainly learned something new and I hope you did too. Next episode drops soon. Can't wait to hear another amazing expert building in AI. Talk to you soon!