AI in Law: Revolutionizing the Legal Field with Peter Geovanes

Peter Geovanes, Chief Innovation and AI Officer at McGuireWoods LLP, discusses AI's transformative impact on the legal sector. He emphasizes AI tools like ChatGPT-4 for enhancing efficiency, competitive advantage, and prompt engineering. Highlighting data privacy and ethical use, he advocates a "trust but verify" approach. Geovanes also stresses the importance of education and a multidisciplinary strategy to harness AI's full potential.

Peter Geovanes Notable Quotes
🚀 "We were one of the first firms in the world to sign up and have access to ChatGPT-4 specifically fine-tuned for legal."
💡 "It's really important that we become masters of prompt engineering and use AI to drive efficiencies and competitive advantages."
🛡️ "For any law firm, the biggest concern is data privacy and always protecting attorney-client privilege."
⚖️ "In the legal community, it's crucial to trust but verify when using AI-assisted legal research."
💼 "AI tools can do the drudgery work, allowing lawyers to focus on more substantive and value-added tasks."

Resources Mentioned
CaseText CoCounsel
RCode
RTutor
Coursera
Udemy

About Peter:
As the Chief Innovation and AI Officer at McGuireWoods LLP, Peter directs the development and implementation of innovative business and legal solutions, leveraging AI-enhanced tools to better serve clients and boost firm performance. His expertise spans data science, advanced analytics, AI/ML, business strategy, innovation, R&D, project management, and management consulting. He has successfully implemented cutting-edge analytics and AI solutions for leading law firms and Fortune 1000 corporations. Peter has been honored with the 2024 OnCon Icon Awards "Top 100 Data & Analytics Professional," CDO Magazine’s "Champions of Data in the Midwest 2024," and Corinium Intelligence's 2022 Data Value Award, and has been a finalist for the "Legal Innovator of the Year" Award. He holds both a J.D. and an MBA and proudly served as an Officer in the U.S. Navy for eight years.

Connect w/ Peter on LinkedIn

Chapters

• 0:00 Introduction and Catching Up
• 3:08 The Role of AI in the Legal Field
• 7:01 Safeguards and Privacy Concerns
• 9:26 AI-Assisted Legal Research
• 13:09 AI in Cross-Examinations
• 14:57 The Impact of AI on Legal Professionals
• 20:54 The Future Role of Lawyers
• 23:37 The Union of Large Language Models and Workflow Tools
• 26:18 The Transformational Power of AI
• 28:03 Ethical Boundaries and Responsibilities
• 29:49 Peter's Career Journey
• 35:18 The Importance of Legislation and Ethics

Transcript

0:00

Hi, my name is Joan Palmiter Bajorek. I'm on a mission to decrease fluffy hype and talk about the people actually building in AI. Anyone can build in AI, including you. Whether you're terrified or excited, there's been no better time than today to dive in. Now is the time to be curious and future-proof your career, and ultimately, your income. This podcast isn't about white dudes patting themselves on the back. This is about you and me. and all the paths into cool projects around the world. So what's next on your AI roadmap? Let's figure it out together. You ready? This is Your AI Roadmap, the podcast. Hey folks, this is Joan dropping in to say hello. I'm so excited to share with you this episode with me and Peter. Peter and I met at a conference just a few years ago where I saw his work on the intersection of AI and law. And whoa, I have been working out and thinking about this episode, it has blown my mind. Not that the other episodes haven't, but like I - because this is a part of the field I don't think about very often, it really expands my understanding of AI in different domains. And I think it really could expand your mind also. Maybe you're a lawyer and you're like, hey, I already know this. Okay, well, good for you. The rest of us are learning also. And that's what this podcast is about. I will say as a disclaimer, neither me nor Peter are giving you direct legal advice. That seems obvious, but we are really thinking about how you think about... changes in fields, changes in the legal field, medical field, et cetera, et cetera. So get ready to laugh hopefully with us. Here's some amazing examples of what we're talking about and the changes in the field and the way you could interact with the law may radically change in the next few years. Ready? Let's dive Fabulous. How you doing? I know, it's been a minute, almost exactly a year. of last year in San Diego. Okay, yeah, well it's good to see you. How's it going? Yeah? a new job So, um, you know, I, it was one of these things, be careful what you ask for. I got everything I wanted plus more. Whoa, that's exactly what we like. title I wanted and I got the position and the salary and all the perks. So it really worked out well. Thank you. having seen your work in San Diego in person, I was like, ooh, Peter's cooking. So that's amazing. Well, would you mind introducing yourself, your name, your title, your current company? Sure. Well, thank you for having me on the podcast today. My name is Peter Geovanes I am the Chief Innovation and AI Officer at the law firm of McGuire Woods. Cool beans. Well, and a lot of our listeners, I think, will be, you know, aware of AI and kind of software stuff, but the legal side of it is still nascent. And people's a lot of understanding of like, oh, lawyers, like, wait a minute, like legislation, you know, there's these concepts. I'd love if you wouldn't mind sharing kind of what projects you're working on, the scope of your role. Like, what are you doing on a day to day? Yeah, I think it's pretty interesting. My first day at my new firm was in mid-August and the managing partner took me out for dinner and he said, Peter, it's time we kind of view AI, specifically legal AI, at being an inflection point. And it's really important that we become masters of prompt engineering and being able to use artificial intelligence to help us not only for efficiencies, which also helps drives down costs for our clients, but also there are some really true competitive advantages. And I'll share some of those use cases with you. But we are in the midst of testing a product, which is by a company called Case Text. and in full disclosure they were acquired by Thomson Reuters later this summer, but they were one of the first partners of OpenAI and just to kind of tell the story back in October of 22, they had access to ChatGPT 4 and were building out this product and I was one of the first people to see it and I was awestruck with the possibility. I kind of took a career move and I said I'm going to show this to the partners at the firm, specifically the ones that are on the technology committee, because I also agree this is a game changer. So long story short is I showed it to the partners. They had a similar kind of reaction. And we were one of the first firms in the world to sign up and have access to this. So as of January 1st, 2023, we were using Chat GPT-4 that had been specifically fine tuned for legal. It had used a RAG as an opportunity to have it focused just on like cases and statutes and regulations that were updated on a daily basis. And if it didn't know the answer, um, it would not hallucinate. It will tell you, I don't know the answer. And as you can imagine for any law firm, the biggest concern is data privacy and always protecting attorney-client privilege. And this tool met all those safeguards as well. Thanks. So long story short, I was the first one to start it at one firm. Uh, I got hired by my new firm to do the same thing. And it's been a fantastic, uh, journey. So I've got about a hundred attorneys that are testing the tool, both transactional attorneys and litigators. Um, and really what I'm getting excited about is the opportunity to kind of strategize with the partners of the firm on their specific cases and how we could use this. So. I'll pause here to take a breath so I don't die on you, but I'm happy to share some of these specific use cases. I think that's where it's really exciting. And I think too, for your listeners, more often than not, they have a legal department or they have an in-house general counsel. A lot of these applications and these tricks and tools I think are applicable to the in-house counsel as well. So hopefully there's some knowledge sharing here. Oh, definitely. When I think these legal, I mean, to see it so early. And I think honestly, Peter, just your mindset around instead of like, ah, I'm scared, like, you know, run away, put my head in the sand. You dove head first, you know, being an early adopter, like seeing how it's going to transform your field, sharing it with your colleagues. I hope you realize or maybe you're in an echo chamber of people who think that way. But I know a lot of people. Or I don't know if you've heard the stat from Pew Research that only 14% of Americans have tried ChatGPT. Yeah. So that was as of last- like the last 400 days of my life, that's all I've been focused on, right? It's like everywhere. Everything I read is about it. So we may be in a unique little niche. I do believe so. I think, or, and that was, I think, roughly 60 to 70% of people had heard about it. And then the smaller group that's tried it, and then people like you and me who are building on it, looking at APIs, understanding how it's going to take our fields and in new places. But I think especially, I've only heard of a very few legal startups, use cases, firms. So it's just, I'm delighted to know you and be able to share your knowledge with other folks. and so forth. So can you tell me especially I think the kind of privacy of so many people are worried about their data. Samsung had that huge data breach that scared everybody. When people say like, hey Peter I'm terrified. Like what kind of safeguards would you recommend or that you use in your practices? Could you talk to that? Yeah, so I think the first kind of differentiation point is there's the public ChatGPTs or whether it's Gemini or Lama, whatever you may be using. And then there are the private versions. So one of the examples that I'm going to share with you, we are using a private instance of ChatGPT-4. And as I kind of alluded to, it does have some additional safeguards. So if you remember months ago, the issue was ChatGPT had not been trained since September of 2021. So it wasn't getting updated information. That could be a big problem and legal when you're looking for precedence and something may have changed. So having the ability of a Thompson Reuters to have their corpus and their basically infrastructure of having up to date. legal precedent available and updating it on a daily basis was one of the first things that we had to ensure was there. The second part is, you know, the famous story was Samsung, but the other one that really rocked the legal community was the Matta case. And the background was this gentleman, Roberto Matta, was on an international airliner. He alleged that he became injured, came back to the United States, and he sought legal representation. They filed a lawsuit in the federal courts, Southern District of New York. His legal team put together a beautifully written motion. The only problem was they used ChatGPT for the legal research and the 18 cases that were cited were all completely fabricated. They never existed. So the opposing counsel was reading this brief and they went to the judge and said, Judge, we can't find any of these cases. I mean they just – we've done everything. They just don't exist. So the judge called in the two attorneys and said, Okay, guys, tell me what happened. It's like, did you use Westlaw or did you use Lexus for legal research? No judge, we used ChatGPT for thinking it was like a super search engine. So the judge ended up reprimanding them. It became huge headlines. This is like May of last year, really rocked the entire legal community about can we trust what's coming out of AI with these hallucinations. And they were both fined $5,000 and they were sanctioned by the court, but they became the poster childs of what not to do. of the legal community seeing the promise of these tools and then all of a sudden somebody throwing cold water on the whole experiment. So we all kind of took a step back and we all put safeguards in place. And I would say to your earlier question, as we're going through the testing, I think one of my edicts is trust but verify. So while we're still using AI and the AI-assisted legal research is one of the most used tools and capabilities that we're finding. value for our attorneys and our clients, but we're having our attorneys check everything right now in this Trust But Verified. Kind of a funny story I'll share with you. I was at the firm maybe a month and we started getting questionnaires from clients and they all kind of had this similar discussion. So the first question was, hey, we hear AI is a big buzz. Are you using it in any way? Please let us know how. Second question. We hear AI may give efficiency gains. Could you tell us what you're finding? And you know what question three was. When will you be passing on the cost savings to us as a result of these efficiency gains, right? So I had to, again, kind of throw cold water on them saying very early days we are testing, but. While there may be potential efficiency gains because we're in this trust but verify mode, in many cases, I'm still going through the same timeframe and maybe even extended as we're trying to learn the capabilities, the risks, the limitations of these tools. Yeah, wow, that's fascinating. Well, and I hadn't heard about that example, but what a salient, well, and part of me is like, or five grand is a lot of money for some folks to be publicly reprimanded and your reputation, oh, that's another ball game. But I understand people wanting to use these tools. I think that's one of my reasons. I don't know if you've tried Perplexity by chance. It's a really interesting tool, you know, similar to ChatGPT, but it gives citations with links. So if you did get spit out 18 references or so forth, it would literally have the links and you can go, I mean, these attorneys should have checked those cases. But in this case, I have never had it hallucinate a mistaken link. So it's really interesting to have citations because of course it's like any of these are leveraging different data sets, like large language model. It's leveraging data of some sort. And so I think that's why RAG is getting more and more popular. And as you mentioned, like up to date cases, quality of our datasets is crucial to them being valuable in the slightest. Otherwise, it is Goobledee-gawk. right, understanding how fundamentally a large language model works. And this was one issue that came up talking with my intellectual property attorneys. You know, they were of the mindset that... as part of the training data set that the large language model was basically keeping a snapshot or keeping a copy of all that data. And I said, no, that's not the case at all. It basically has read everything. It's made connections, inferences from it, but it should not be able to regurgitate for you verse chapter section of a document or something. And that was a complete different mindset for them. They were unaware that that's what was happening on a fundamental basis. So that part of education. I think also an interesting point was, so in addition to just getting the tool, it's been an amazing change management project for the firm. So in addition to giving them kind of the basics of navigating and using the tool. We also had to do responsibility and ethics training around AI for every attorney. We had to do two classes on prompt engineering, kind of the basics and then more advanced prompt engineering. And then we work with them, they have a use case or we have a workshop with them. So I'll give you one example and I just came off the phone. We have a client, we are preparing to go to court and do cross examinations. The plaintiff's firm. will be presenting 10 different experts in their field. All these experts are well known and they have published a number of documents and in some case textbooks on the subject. So we're on the defendant side and as we're preparing for cross examination, we're thinking, can we use AI and specifically our tool here at Case Text Co-Counsel to upload maybe everything they've written. and give some prompts like help us identify inconsistencies. So on a very simple basis, right, on Monday, the expert said something was red, and three days later they said it was blue. We want to be aware of that. And from a legal standpoint, right, for cross-examination, you can use that to impeach the expert or maybe even go against their credibility, right? You know, you say, this is never done, but a week later you've given me 25 different exceptions when it has been done. So whatever it may be. The feedback I'm getting from the attorneys is there is so much information. We would not have the physical time or the budget from our client to be able to do this thorough of due diligence and preparation. Now we've got this new tool. You could argue this is an efficiency gain, but I would also argue this is really competitive advantage. You would think in the very future. Going out on a limb here, I would say it would almost be professional malpractice if you didn't use these tools to the benefit of your client. Lawyers have an ethical responsibility for what we call zealous representation. I think it's foreseeable that we're going to get there in the not too distant future where somebody will lose a case and they will bring up the point, did you use AI to support me? Did you use every tool and every technique that was available in a reasonable, I should say not every, reasonable techniques to assist me in zealous representation? So it may be, you know, five years out, 10 years out, but to me, it seems foreseeable that the tools are so powerful and they can give you such a competitive advantage that that's a foregone conclusion in my mind. Absolutely. When it seems like these tools, I don't know how you think about them. I think about them as kind of an augmentation and we're certainly seeing the word copilot come out, but it's almost like you have like 10 highly trained assistants that suddenly can go through these documents and highlight those inconsistencies or otherwise. But I totally hear you. I had a legal issue and my lawyer wasn't using every tool in the book, you know. trying to get me out of some serious problems, hot water, I would be annoyed or more than annoyed. I'd be quite angry actually. So I think that's an interesting point of competitive. Yeah, all right. you seeing AI as kind of the equalizer or I think I've used the example of, it's almost like performance enhancing drugs, right? It's you get a young associate and you give them AI. Are they able to do 133% of a, of a normal person? I don't know. Is that what you're sensing as well? Gosh, I think it's early days, I would say. I think that for people who are leveraging it well, it is a game changer and extremely efficient, but sometimes you always need a human in the loop. I think that's one of the things that I just like, do not deploy if you care about the product, which anything anyone builds, or at least again with reputation, the quality of my work, I can't have it spit all googly gock and put my name on there. that for any professional, I would think that's the case. But again, not, but as you mentioned, like not using it. or not considering here's what that tool is today to help me, where's that tool gonna be in six months? How am I future-proofing my team? How are we preparing ourselves? Again, I'm consulting right now, Peter with different startups who are thinking or talking to their investor and their investor says, what is your AI roadmap? How are we future-proofed? How is our investment future-proofed? You might be in agriculture, you might be in robotics. Seriously, if you don't have a 10-year plan, at least in a vision concept of what it could be, are not necessarily interested in investing or we're very concerned about your future proving yourself. So I think a lot of companies, I'm grateful to say it's fun to partner up with people in completely different fields from mine where we have a shared goal, a common mission. But I think it's really interesting what you talk about with kind of a legal representation, talking to lawyers. I mean, could I say that almost all companies of a certain size need this type of legal training for employees, kind of guidelines and practices? I mean, to me, it seems like the liability there could be quite extreme. Yeah, and I think right now, if you look across the legal environment, they're kind of grouped like I'm one of the, they call it the AMLO 100 or 200 being the 200 largest law firms in the US. And then there's kind of, and I would say typically the definition is 500 or more attorneys that are firm. And then you have mid-sized firms, so typically 100 to 200 attorneys. And then you can go all the way down to your solo practitioners. And what I think about the AI... which maybe at this early stage, the larger firms were early adopters, but for those mid-size and smaller firms, it's also an opportunity to proverbially kind of punch above your weight class. and go after more work and get more done with your resources. I think the other aspect that we're seeing that I didn't really expect right off the bat was the quality of life improvements. A lot of a young associate's work is absolute drudgery. I'll give you an example. Let's say it's a merger and acquisition, right? Company A wants to acquire company B, they set up a deal room, they put maybe 10,000 executive contracts, and the investment bankers and the lawyers wanna know of these contracts, in the next 60, 90, 180 days, which ones are auto renewing, which ones have a favored nation clause, and let's sort those by dollar amount. So in the past, it'd be an army of first and second year associates looking through these contracts, 50, 60, 80 pages each, pulling out the data points and maybe collaborating on Excel spreadsheet until they were cross-eyed. Right now with AI and a large language model, we could basically write a prompt, upload those documents and say, extract these data points for us and put it into a CSV. And then have the attorney basically review and spot check and make sure the data is correct. But that drudgery work, right, is over and we're doing more value add work, which is a win for the clients and our bill rates and also a win for the law firm being able to do more substantive work. being able to use legal judgment instead of just kind of rote, awful, relaboring work. I think I made my point. Drudgery. Yeah, oh for sure. Yeah, I was recently watching Lincoln Lawyer on Netflix. I don't know if you've seen that show and just the amount of paperwork and as you mentioned, several people working through the night trying to find those tiny details that you know, make or break the case. Is there a lot of I wonder though as I'm thinking through that example, is there a huge amount of like paper that needs to be digitized like do you find that most of your customers already have that? well obviously, we'll digitize or OCR things, but for the most part, everything is now electronic, everything that we need. Just another example, I love telling this story. A friend of mine literally wrote the book on immigration law, and he uploaded it into and was asking questions to the book. And he said, this is the best day of my life. I said, why? He said, for the first time, I can have a conversation with myself and I can get that, right? It's like, this is great. I wrote the book, I'm asking myself questions. I'm getting answers from the book, but having that interaction is also really powerful. So again, maybe we'll use the example of the expert witness. Maybe they published the book. You could upload that book in a PDF form or electronic version. you could ask specific questions. How did the expert opine on X, Y, or Z and be able to prepare yourself? I think the other thing that we're really finding powerful is the large language model's ability to summarize. So from a legal context, let's say the SEC comes out with a new regulation tomorrow. Maybe it's 60, maybe it's 90, maybe it's 300 pages long. The ability to have that large language model summarize it in a very accurate fashion and allow the lawyer basically what's changed or what the implication is for my client. Not only does it allow them to synthesize that information much more quickly before they go out and read it, but also... I think just being more agile, right? I can read it. I understand from a summary what the impact is. Then maybe I put another couple of paragraphs below that summary and I write the impact. This is what I think it's going to mean for the agriculture community or for investment bankers. This is what this means today. We should probably have a further conversation on this. But the ability for the attorney to be almost in real time. acting as that fiduciary for their clients and being a step ahead and alerting them to this change and this is what the implication is for you. That's something that just wasn't possible before. So I really embrace AI as being that change agent for us. Absolutely. Things I implicitly hear of these examples you're giving is kind of this plain language, like reducing these hundreds of pages to something that's potentially readable and consumable in a shorter amount of time. But also, I hear the democratization of these legal texts, massive amounts of jargon, to something that's consumable and actionable. Do you think that potentially, I mean, as you mentioned, kind of these. early stage folks who are working on their careers, who are going through the drudgery, do you feel like that may shift the role of lawyers in the future or kind of how people prepare their legal practices? Yeah, I think that's a really fascinating question and we kind of are struggling with it now in our firm is. Do you need to walk the proverbial miles in the shoes of those that have gone before you and kind of learn the traditional ways of legal research and doing the drudgery that we described of going through a thousand contracts? Because I would argue there is some learning there, right? You are going through your learning. But at what point, right, after I've done 50 of these contracts or a hundred of these contracts is the learning kind of ending and now it's just becoming just the drudgery? And is that maybe the point too where we need to, within six months or maybe we, you know, the other thing going on too is that law schools, they're starting to embrace these AI tools and technology. So we're seeing it too as new associates are coming into the firm during the recruiting process. They're asking the tough questions. Has your firm made these technology investments? because otherwise you're kind of viewed as that old fashioned, maybe even stodgy law firm. And if I'm kind of this new wave attorney to be, and I've used these tools, it's like, I don't want to take a step backwards. I don't want to go to my father's law firm or my mother's law firm. I want to be a 21st century attorney and I expect to be able to use these tools and do less of the grunt work, so to speak, and really use my... business acumen and my legal judgment for the benefit of my clients, what I thought I was going to law school for. Totally, it almost seems like one could be more human or it's very interesting to see, excuse me, there's an age gap between us, but I think somewhat potentially ageism or like someone who's ambitious, who wants to have a long career, who says, wait a minute, if I don't have people who are mission aligned, who are not mindset aligned, this could be a huge career roadblock, frankly. I totally get that. and it's interesting that you brought that up too is I had some people at the firm, some of our paralegals came up to me and said, listen, we're hearing that these tools can summarize. That was tasks that we used to do or still do. Are we worried? Are we going to lose a job? And I guess my thought to them was I would embrace this opportunity. I have license here. I want to train you on these tools. I need kind of that grassroots champion as we roll this out to the entire firm. I don't want you to be scared of it. I think it's an opportunity for you to learn the capabilities of these tools, but more importantly become an expert in prompt engineering and make yourself invaluable to these case or matter teams going forward. But if you don't do that, I mean, I think the writing's on the wall. I think that fossilizing and kind of expecting it to stay the same is the axe. I would agree and I've certainly worked on projects where we're creating bots, automating things and suddenly they're like, wait a minute, is Lisa, you know, Mark, are these people losing their jobs? And I'm like, do they like doing that drudgery of answering phones or whatever? Can we please upskill Lisa to be doing something more in her zone of genius? I think there's a huge opportunity, especially if we know she's a hard worker. There's no reason to lose those folks. But there is a transition that needs to happen. Yeah, and I'm sensing too what we're gonna see in the next year or two years is kind of a union of large language models and the AI components that we've been speaking of, but also with kind of the workflow tools, whether it's low-code, no-code tools or other things. So complete processes, we may be able to replace or augment and get these efficiency gains. And again, put the humans in positions to do more value add work than just the... the grunt work. Totally, and that's what my, it's so interesting talking to people at different career stages. My friend, acquaintance Victor, he is trying to figure out his next career steps and upskilling. And he's realized recently that like becoming a developer may not be advantageous as he projects out what companies will need of him in the next five years. And so it's interesting as he thinks about, he's been talking to me about like, is it data science? Is it how we build use cases or studies of ROI? Like where is the value add for enterprise in the future? tailor his skills toward, I mean, again, we're all making projections, right? We can't know the future, but I think as talent is considering, how do I upscale? What is valuable? Honestly, I feel like there's a bifurcation of kind of the intern level and the senior executives and less of that middle ground of people able to, you know, pivot and that middle level. It's harder, I think, for at least that's what I'm seeing on my side, harder for people. to your point about coding, I think it's an absolute amazing development to be able to go to copilot or GitHub and basically describe what you're looking to do and have it give you Python code or even Excel macro for what you wanted to do. Um, Wow. What a, what a performance enhancement. What a, um, a scary time for people that that's what they were kind of betting their careers on because I think it's going to be completely disrupted. Yeah, I talked to my CTO about this often and she says like, just because photography was created doesn't mean that painters aren't still valuable. And frankly, someone, my CTO says, seniority, I always need a human in the loop. We cannot deploy stuff willy nilly. And I think I will say, I code in R. And when I saw some of these tools that spit out beautiful code in R, you know, like solid A, A plus, just draft. how many hours I spent in my PhD, just like hours of my life, just gone. But now someone using one of these tools, it's called, I think, R code, R tutor, something like this. Wow, just, as you mentioned, just. So fast. Okay, well, so this podcast is called Your AI Roadmap to hear about projects, to hear about things in flight, but also to hear about trajectories, to hear about how you got here, kind of where you see the future, and you've already alluded to this, but before we jump into career stuff, I'd love to hear, you know, as you project and you're on the forefront of this stuff, clearly, when you think about kind of the next five, ten years, what do you see kind of the realities going forward, like What are you seeing potentially for your field? I think that's a really interesting question because just the last 12 months have been so transformational. And as I was talking about that tool, Case Text Co-Console, it didn't get released until March of 2023. and it's been completely disruptive. The amount of change, I think the other part of this, which is really fascinating is the large language models are getting more and more powerful. Google rebranded Bard Gemini. of tokens that this has. I think it's like 1.5 million tokens and the amount that means the large language model can basically keep in its internal memory while it's comparing documents and comparing documents one-to-one has just gone up exponentially. And again, this is one year into this kind of cycle. So when we start thinking about five to 10 years out, you start thinking of AI being able to not only do these tasks, but also perhaps start to exercise professional judgment, whether it's legal judgment or medical judgment, um, kind of opine on subjects. Um, I think a lot of the information that we do today for preparing for trial will be able to be automated, but to your point, it's always going to be the skill of a trial lawyer communicating and advocating. If you remember when we were in San Diego at a chief data officer conference, I presented one use case, which was kind of scary where AI was basically kind of in the judge's role. where it was doing some sentencing, which is a really scary thought, but this was a Wisconsin case from 2016. And basically the judgment was AI or some type of analytics can be used for sentencing a suspect or someone found guilty. However, that can't be the only basis. And there has to be a human in the loop. So I think that's gonna be part of it too. I think it's gonna be more. detailed information on ethical boundaries and ethical responsibilities. I think our legislature is a little bit behind in times. We're starting to see some states like California pass an AI Bill of Rights. And I think we're gonna have more information going forward about kind of the rights and the wrongs of this. And of course, everyone is concerned about bias and when AI is starting to make the decisions that is done in a very ethical responsible way. But to your point, I don't know if my mind can handle what's going to happen in five years just based on what's happened in the last 12 months. Absolutely, it's a big question. I just I love asking what people are seeing from their perspective and their part of the field and I totally hear you about, you know, sentencing and kind of biases that are propagated and really at her folks Um in ways that are potentially not deliberate. I think one of our other guests talked about kind of actually, um maybe neutralizing bias actually and being used in the medical field for, unfortunately, there's many, many biases, especially in Black women's pregnancies and so forth and kind of pain mitigation, these systemic issues. But if there is a tool that is far more equitable and is an intervention tool to say, hey, this patient has mentioned X number of times that they are in pain, you know, this tracks with some kind of analytics of... This is escalating. We need to take intervention that potentially urges I think there are so many opportunities to use these in a different safeguard way to help advocate for Patients for example in this context and in a way that unfortunately in our medical system here in the US Still really needs that triaging because humans do have bias We all have different biases and really working to make the world a more equitable, safe place for everybody is I think, you know, if we create those tools correctly, can live into our best values. Yeah. Okay. Well, Peter, how did you get into this job? If people are hearing about this and they're like, Whoa, this is like, yes, sign me up. I want this career path. How did you get here? so this is gonna sound like I had a plan. There was no plan. So let me tell you the background and it was a little bit serendipity, but I started my career as a naval officer. I did eight years active duty. Toward the end of that I went back to school and got my MBA. So I kind of think of that as chapter one. Chapter two then was a 20-year consulting career working at firms like Accenture and PWC. But where I really got lucky was I'm gonna age myself here. from 2000 to 2005, I accepted a role for an analytic company called SPSS. You may know of SPSS and SAS as kind of the two industry leaders back in the day that kind of created the whole field that we would consider today data mining or data science. So I really got in at our early level. And it was funny, back in 2001, 2002, we'd be elbowing each other saying, I think this is the year this analytics AI thing is going to take off, right? Who knew? We were literally 20 years ahead of the curve. It was just like I said, it was a PhD level education from some wonderful thought leaders in the field. It really got me on this kind of analytical data driven path. And I would say it was probably the most career defining job that I had. All my jobs subsequently after that in consulting were all still around data and analytics and helping Fortune 1000 companies kind of the ideation and implementation of business analytical solutions. So jump ahead toward the end of that 20 year period. Um, I'll admit it. I had the midlife crisis. So most people go out and they buy a red sports car or convertible. I decided instead I'm going to go to law school full time while I continue to work and raise my family. So while I was going to law school, um, you know, people were asking why are you doing it? And you know, the original, I guess, impetus was in consulting. I would end up basically, I'll use the word selling. partnering with my clients and selling a statement of work to them about what we were going to implement. But more often than not, I would find myself in front of the general counsel's office trying to defend the statement of work that I wrote with no legal training. And I just thought, wow, I'd be so much more efficient if in arm for a conversation, if I knew what their objections were and if I understood more, maybe I should take a class in contracts. So when I was looking at law school, first year subject matter at my law school was contracts, torts, and criminal law. I thought, well, I'll do the first year. When I got into it, and I think this is our analytical background, I loved it. I loved the learning. In law school, you get these crazy hypotheticals and it's rules-based and you're thinking through this and it's an analytical write-up. It was like puzzles to me and I really, really enjoyed it in a very hedonistic way. Long story short, I graduated law school and right around 2016, IBM was running the Watson commercials on TV. The law firm that recruited me ultimately, the leadership had seen this. And they had their own kind of epiphany thinking, we can see in the very near future that data and analytics and perhaps even AI will augment an attorney's legal judgment and experience. Let's go out and find somebody. And I was lucky that the job req made it to me. I would say I read it, it was all wrong. I helped them rewrite it. Thankfully I got the job. My title was head of data strategy, AI and analytics. I was at the firm for seven years. while I was there and did a lot of really cool things and won a number of industry awards, including Legal Innovator of the Year. And then as we started this podcast, I was sharing with you this summer, I accepted a new role, which is at my current firm. And I accepted the role as Chief Innovation and AI Officer. And one other side note, I was at a big legal conference at the end of August. I had just started at the firm. I had my new title walking around and people were stopping me going, my God, that is the sexiest title I've ever heard. It's like, oh, thank you. You know, so AI, maybe the first in law to have AI in their title. Seriously, you were the first, or it's just a unique role, at least, but I don't know that, I mean, right now it might, or it sounds new to me, I don't think it'll be unique longer. It seems like this might be fundamental at some of the big firms. Would you agree with that? love to get your perspective on it. I was just at a data conference and they said, you know, I don't think this AI office, chief AI officer is going to persist. And their logic was AI is so important to every C level executive that it'll be part of their roles as well. So I don't know. I thought that was a different way of thinking about it. Yeah, that's so fascinating. I... believe that the jargon that we're using today will evolve significantly. Um, I think about multimodal, I think about integrated, I think like, you know, is software going to eat AI? Is AI going to eat software? Like that part of it, I honestly don't care what the jargon is. Um, but I do have seen different roles, you know, chief data officer, chief AI officer, and as you mentioned, like some of these job roles, someone's thought through, like I know they have their stuff together, other ones I'm like, there's nothing about PII. There's nothing about ethics. there is nothing about legal protections, for example. And I'm like... it's clear that whoever wrote this and deployed it has no idea what they're up against. And frankly, that scares me, but also I think as, again, there's a need to be a human in the room saying here's our agenda, here's our strategy, here's how we're rolling this out. And we need to be at the forefront to look at legislation that's coming down the pike for our field and being on top of it and having team members who are actively working on it. So at least in the next 5, 10, I'll make a, you can quote me on this now, I think that type of role on the C-suite invaluable and frankly, it's not a CEO role. There is vision related to it, but there's like grunt work, unfortunately, at least at the moment, to operationalize those data analytics protections. To me, that's a role and that's a team. Yeah, you used the word multimodal in what popped in my mind and the role that I'm in now. It's very much multidiscipline in the sense that if you look at my career, like I said, it wasn't planned out. I'm happy where I am right now, but it's really a culmination of all my experiences. So I look at my experience as a business consultant and building my business acumen, my technical chops from analytics and AI and NLP. just having empathy for people and being able to change management and communication. And it's really an amalgamation of all those skills, which is allowing me to kind of lead the charge in this brand new wild frontier that we're in. Well, it's so interdisciplinary. I think, Peter, especially as you talk about, like, the... Having all these legal documents, you know, done faster potentially, but, like, be able to be more human or have that emotional intelligence with a client. It seems like to me that, as you mentioned, like, why do lawyers even get into this? Like, it's... I don't know. Maybe it is the paperwork side. But potentially also to work with people and advocate for different causes you believe in. It seems like with these changes, one could be more human, leverage that interdisciplinary... and execute different projects better, at least what I'm hearing from my years. and I think the other part that you kind of alluded to too was the need for transparency. So not only within our firm of how we're using these tools and part of that is kind of champion these new best practices that we're learning and that are constantly evolving, but also with our clients, keeping them in the loop, letting them know that we're uploading data. In some cases, it could be considered sensitive data, but we're taking these precautions. I gave. We're reading through tens of thousands of documents and we're able to find the proverbial Needle in the haystack which is going to give you the best representation going forward But having that dialogue and I guess that was one of the surprises in my role. I thought it would be much more Technical my role, but I find myself kind of being that evangelist that educator internally and externally to the firm as well Well, it's also, I think, one of your superpowers. Your ability to translate huge technical things to a broader audience. That's amazing. Well, if people are interested in this field, I mean, I just heard the number of degrees you have. Do you think people will be needing law degree, MBA? Like, what type of background do you think folks might need to pursue? So again, going back to my time, and again, I'm going to date myself 20 years, but when I was really at the very forefront of predictive analytics, we kind of walked away with this thought that You could become a data scientist and add value, but if you could also build up your business acumen or combine that with, in my case, legal acumen, that's when the real magic happens. And that's what my recommendation would be is absolutely I would encourage people to, if their aspiration is to be a data scientist, but you're really going to learn more, too, working at an organization and talking to the business people and understanding the business challenges that they have. and be part of a cross-functional team and contribute that way, that's where I think the career growth is and that's where I think the real opportunities are in the future. Absolutely, that makes sense. Well, you've mentioned this software several times. I don't know if it's an investment tip or otherwise, if they've already IPO'd. Have they? Okay. when you think about the shape of your team and who, you know, people-wise is needed, what is your current shape of your team today? You mentioned 100 folks maybe around you, or what is the shape of your team today? So the law firm itself is about a thousand attorneys and I have a hundred of those attorneys actually actively working with me as kind of early beta testers. Some are litigators. Some are transactional attorneys. But as far as my team today. Again, it's a cross-functional team. So in a law firm setting, I have IT project managers and IT folks that are working with me. I have people from our learning and development team. I have people that are trained as legal librarians. And I also have some of the paralegals. And part of that is I want that entire team to walk away from this beta test experts in not only the software, but understanding the capabilities and really learning the nuances of prompt engineering. because if the prompts are wrong, you know, garbage in, garbage out, it's really that important. And I think we're all learning together, right? A year ago, I don't even think that word prompt engineering was in anyone's vernacular. And now, you know, it's talked about commonplace, but we're still learning. And you know, one of the... tricks or tips that I've been giving people is if you grew up, if you ever used, uh, it was kind of like a grade school game. They call it mad libs and it would be a sentence in like, I need an adjective here or I need a proper noun. But if you kind of use your prompt engineering like that, it works very well. Or, uh, my daughter is a theater major and I use the analogy of a director talking to an actor and you're kind of setting the stage. Like the actor wants to know what's the stage, what's my motivation. Tell the large language model, right? I'm an attorney. I've uploaded a number of documents. I'm going to be cross-examining a witness in this case. I'm interested in finding inconsistent statements that this person may have said over the course of time. But basically set the stage in your prompt and it's amazing what you get out. And then I think the other thing that we kind of stumbled upon is what we're calling meta-prompting, where you're using the large language model to help you craft the prompt. So you basically say, I helped me, you know, assist me in crafting a prompt. I'd like to do kind of the following and kind of spell it out in natural language. And it's amazing what you get back. And I think that's like, you're starting to see the complete power of the 360 of the AI was actually running the prompt for you to run the prompt. Right? Because that blank slate is one of the hardest things, just like a blank piece of paper. It's like, where do I even start? Like, how do I get the best outputs from you? That's amazing. Well, as you think about, you know, if people are listening to this and they're like, whoa, I don't know if I'm getting an MBA and a law degree and like, Peter's got a cool gig. What do you look at for people you're hiring? You know, like what kind of skill sets, what kind of mindset? Like, what are you looking for? Yeah, and I think legal is a very, this is a bad pun. It's a hard bar to get into because typically you have to have some type of legal background, whether it's a paralegal or experienced at a prior firm. But for my team and going forward, I think I want people that would be attorney facing. that can understand some of the challenges that they have. And then just an entrepreneurial spirit, right? We're kind of making the rules up as we go along here. We're trying new use cases. We're being very innovative. We're not dismissing anything. We're testing a lot of crazy hypotheses. So just that kind of can do. And if you need basically... a checklist of how you're going to do your job, you're probably not going to do well in this role because we're making it up as we go along. What kind of a roll up your sleeves kind of attitude, let's try it out. Yeah, yeah, open my. not be afraid of failure. I mean, that's part of it too, is if we're going to fail, fail fast and learn from it, and then we'll try another iteration. But that's part of the mindset right now that we're using. Absolutely. Well, I feel like my legal problems would be well safe in your hands with that kind of mindset. I do wonder, have you consulted with government? I feel like as you talk about the way you're shaping this practice in your company, are there opportunities to have this value in other places? Yes, that's a really interesting question too, is I was honored to be asked to speak at the American Bar Association on a panel on AI. Our firm also has, we call it a consulting division, but it's basically working in DC with different representatives for different states and advising them too on some of these AI policies that are going forward. So a lot of the little work that we're doing and the things that we're doing with clients and the feedback that we're getting, we're able to basically feed in for the good of the of the So that's very good Yeah, thank you for your work. I'm extremely grateful because this stuff, I feel like on the legislative level, at least seems so slow compared to on the ground. That's honestly one of the reasons I have this podcast is there's so much hype. There's so much discussion. What are people actually building? Are they running those experiments? Are we learning things? Are we failing? Are we talking to customers? I think that's the real hands-on is where the deep learning happens. And then the legislation seems to be totally divorced, at least in my field right now. It's just kind of the wild, wild west. So any advice you might give for folks who want to learn more, want to upskill? Yeah, I think, you know, two things that come to mind is, um, I just read everything I can. It seems like every day, whether it's the Wall Street Journal or bearings or whatever your newspaper of choice is, um, there are really interesting AI stories, the business of law. We talked about Samsung, we talked about the mat the case. That's where they were all published first, the copyright infringements with New York times and open AI. I think those are really fascinating things to kind of watch and see how they develop. The other is I'll put a plug in is I love like the Courseras and the Udemy's of the world. Yeah. the ability for relatively small amount of money, especially catching one that are on sale, but to get really good quality information about natural language processing. And years ago it was birthed and now it learns language models and understanding how they work. I just see that as just an invaluable resource in this world that things are changing so quickly, just to keep relevant. Absolutely. I couldn't agree more. Well, if people hear about this and they're like, Whoa, I need to learn more. I need to follow these resources. Our show notes will have a lot of these. But where can people find you? Oh I am on LinkedIn, feel free, please connect with me. That's probably the easiest way. Absolutely. Well, thank you so much for sharing your time and expertise. This has been fascinating. I feel like my mind is just going to swirl on this all day long. Any last things you might want to mention? Hello. just kind of a brief encounter at a conference and you were so kind to follow up. And I appreciate you for reaching out. It was my pleasure. Well thank you so much for your time. It was great to catch up. Have a good rest of your day. Absolutely, you too. Cheers. Bye. Oh gosh, was that fun. Did you enjoy that episode as much as I did? Well, now be sure to check out our show notes for this episode that has tons of links and resources and our guest bio, etc. Go check it out. If you're ready to dive in to personalize your AI journey, download the free Your AI Roadmap workbook at yourairoadmap .com / workbook. Well, maybe you work at a company and you're like, hey, we want to grow in data and AI and I'd love to work with you. Please schedule an intro and sync with me at Clarity AI at hireclarity .ai. We'd love to talk to you about it. My team builds custom AI solutions, digital twins, optimizations, data, fun stuff for small and medium sized businesses. Our price points start at five, six, seven, eight figures, depends on your needs, depending on your time scales, et cetera. If you liked the podcast, please support us. Can you please rate, review, subscribe, send it to your friend, DM your boss, follow wherever you get your podcasts. I certainly learned something new and I hope you did too. Next episode drops soon. Can't wait to hear another amazing expert building in AI. Talk to you soon!