Alter Everything Podcast

AlteryxMatt · ‎10-08-2024

In this episode, we sit down with Luke Cornetta, Senior Director at Alvarez & Marsal, to explore the role of generative AI in the tax industry. Luke shares his experience in implementing AI-powered solutions to streamline projects, using tools like Alteryx to scale these initiatives. We dive deep into the business impacts of generative AI, including key trends, the intricacies of prompt building and the growing importance of private LLM solutions in ensuring data privacy and security.

Panelists

Luke Cornetta, Senior Director @ Alvarez & Marsal - @Luke_C, LinkedIn
Megan Bowers, Sr. Content Manager @ Alteryx - @MeganBowers, LinkedIn

Topics

Ep 169 (YT thumb).png

Transcript

Episode Transcription

Ep 169 Implementing Generative AI and Alteryx Solutions

[00:00:00] Megan Bowers: Welcome to Alter Everything, a podcast about data science and analytics culture. I'm Megan Bowers, and today I am talking with Luke Cornetta, senior Director at Alvarez and Marsal. In this episode, we chat about how he's implemented generative AI in projects in the tax space. How he uses Alteryx to scale these projects and the business impacts and themes he's been seeing from generative AI so far.

Let's get started.

Well, it's awesome to have you on our podcast today. Luke, could you give a quick introduction to yourself for our listeners?

[00:00:37] Luke Cornetta: Sure. Megan. My name is Luke Cornetta. I'm a senior director at var. I work in their practice based Alteryx user. Just over seven years now, I'd say.

[00:00:50] Megan Bowers: And you mentioned that you're expert certified, is that right?

[00:00:54] Luke Cornetta: That's right. So I have always been using Alteryx Expert was a goal of mine and about three years ago, I wanna say I, I was able to pass that certification, which was quite the challenge. But it's been very helpful to hold that certification and continue to push Alteryx to, to its limits.

[00:01:10] Megan Bowers: Definitely. Yeah.

That's a huge accomplishment. So congrats on that for sure. I'm really excited to talk to you today to hear about some of the cool stuff you've been doing with Alteryx in combination with generative ai. So if you could tell us a little bit about how your team has used generative AI to help with an ERP system implementation and how that went.

[00:01:33] Luke Cornetta: Sure. So part of what we do in the tax practice is, is dealing with a lot of data. My team focuses on mostly data, despite sitting in the tax practice, we deal with all sorts of data from ERP systems, from PDF files, from unstructured Excel files, you name it. But we see it in the tax world. And it's really less so a tax challenge and more so a data challenge.

And the Alteryx background that I have lends nicely to helping on those types of projects. And Alteryx is a really great tool from for sucking data out of one legacy system and transforming it into whatever the new system needs. So when the past, I've done these types of projects before generative AI was really mainstream.

And always struggled with the, some of the one-to-one mapping this field in the old system is this field. In the new system, Alteryx is great at, maybe there's some business logic where you combine two fields or split two fields out into whatever the new system needs. But one thing that was always a struggle, I'm thinking back to one project two or three years ago, what was comments fields?

There's just a spot in the old system where people can dump whatever they want into it, whether it's an email or. Just their thoughts or some notes about a client or a customer or a location. And there's really not a great way to deal with that. And in the past there's just been teams of people reading those comments or maybe we do some sort of keyword check or some sort of logic and Alteryx to try to make an attempt at looking at them.

And things like RegX and text parsing can get you so far, but it turns into a brute force exercise. So that's been a struggle on these types of projects in. When we got on the opportunity to work on this latest project earlier in the year, generative AI was much more mainstream at that point. We had stood up an internal secure place for us to use generative ai, I think an internal chat CPT that was secure on on our organization's network.

And we started playing around with a very similar need. You know, the project we were helping on had comments fields and notes fields that were containing a lot of really critical business information. That our client was using to run their business. Things like product pricing and hours of operation and in all sorts of formats, you know, just think dozens and dozens of people over the years typing in information in no standardized manner.

So things like RegX and text parsing weren't gonna work. So we started playing around with pasting some of these comments into just, uh, internal chat. GT is probably the best way to think of it and asking it to pull out some of this key information. We found that it was actually not too bad at reading it because its large language model is meant to understand language.

So, uh, it was doing a very good job at being able to read that and pull out some of the information that we wanted. So the next step was to leverage Alteryx. So that's the background on the journey of how do we see a need for generative AI in this type of, what's traditionally been an ETL data prep exercise.

[00:04:36] Megan Bowers: You're using an internal LLM model to extract the relevant data from these comment fields. I thought it was funny when you mentioned anything from business hours to an email or something.

[00:04:48] Luke Cornetta: Oh, yes. Um,

[00:04:49] Megan Bowers: maybe someone put a shopping list in there. So you're using that model to get that information out, and then how are you scaling that in Alteryx?

[00:04:57] Luke Cornetta: Yeah, so Alteryx was very good at helping us scale that. So the biggest challenge, like you said, some of these comments could have been 10 characters, they could be 5,000 characters. Their whole email chains faced it in. So it was very interesting. It was an interesting challenge to try to extract the, the necessary data, and luckily no one was expecting a hundred percent accuracy on that.

I don't think it's reasonable to expect a hundred percent accuracy on this, but we were targeting roughly about 80, 90% accuracy, and we got comfortable with that number. Alteryx was really helpful in scaling this because what we were able to do was with some of the APIs that Azure has available for its LLMs, we were able to leverage just the traditional download tool in Alteryx to make those API calls and essentially pass each comment field through that API applying more or less the same prompt to it.

So basically just doing the same thing over and over again by passing multiple records through with each record an a p. Go and extract the data that we needed, and then we were able to use Alteryx to then parse those results out into a more structured way to load into the target system. So it was relatively streamlined.

There was definitely a lot of trial and error in this process, but we were able to get it to a pretty good spot that traditional methods just weren't gonna be able to get to in the same, in the same timeframe.

[00:06:19] Megan Bowers: Very cool. Yeah. Obviously there's so much chatter around generative AI and to hear. Kind of implemented use case like that is really interesting and hopefully our listeners can think about ways they can implement generative AI in their Alteryx flows as well.

But you mentioned trial and error. Did you face any challenges when you were going through like prompt engineering really getting what you wanted out of the LLM or what did that look like?

[00:06:47] Luke Cornetta: Certainly. So some of the challenges, it's really all about the prompting at the most basic level. And what we were finding was, um, sometimes the comments contain multiple types of data that we needed, whether it's, like I said, pricing or hours or a few other different things.

And if we tried to ask for it all at once, it would get confused or not give us as good of a result. If we, if we broke it up into a few separate different prompts, asking pricing in one prompt or hours in another prompt, or whatever it might be, that tended to give us better results depending on how we worded it.

We had to be very particular in how we wanted our outputs, 'cause we wanted our outputs in a structured format that we could then further parse down the line. Sometimes it. Try to be too helpful. It would try to be very polite. It's like, here's your answer and then this. And I was like, we don't really need it to say here's your answer.

We just need the answer. So it, it would try to be too helpful at times, which we appreciated, but it, we had to be very precise in the way that we worded things. So the prompts for these ended up, if you were to look at the prompt, it looks like a giant paragraph. And that's really, uh, good prompt. It shouldn't be a sentence isn't gonna get us answer.

We prescriptive give some.

Pretty good for most of the records.

[00:08:04] Megan Bowers: That's awesome and super interesting. I have used chat, GBT, various things for like random one-off projects, but building out a prompt like that went to the point where it's a very large paragraph, I'm sure. Yeah. Best practice for really dialing in on what kind of results you wanna see.

[00:08:24] Luke Cornetta: Yeah, I, I think that's exactly right. And I'd echo, you know, it was super exciting to see this generative AI technology be actually applicable in a real world scenario. Because one of the things that drives me crazy is when everything's just kind of buzzword driven and it's all marketing and no substance behind it.

So I had used generative AI previously to help draft paragraphs do what everyone does with it, draft an email, draft, whatever, and see it actually. Significantly impact in the, uh, an actual engagement that, that our firm was working on was very interesting to see, and very positive to see and really has opened up a lot of doors for how we do work here.

[00:09:02] Megan Bowers: Totally. Were there any security or governance concerns when you were going about this project?

[00:09:09] Luke Cornetta: Sure. So when we, as a firm, when we started going down generative ai, going down that rabbit hole about maybe a year and a half ago at this point, one of the first and foremost things for us was security and governance.

Being a firm that deals with confidential information with me and the tax practice, we deal with all sorts of PII and other types of data that we have to be very careful with. And as we work through our projects, security was tantamount to what we were doing. So we obviously are not allowed to use the public facing chat, GPT, nor even the some of the APIs we have to be careful about are they saving our data for training purposes.

We spent a lot of time building out the backend and making sure that when we set it up on our environment, that it was truly on our environment and not going out anywhere else. That was the biggest piece, and once we got comfortable with the security side of it and the architecture side of it, we, we've been able to scale it out a bit more to give it, you know, we have a.

Teams can go to and ask questions and put their PDFs in and ask questions about whatever they really want, if they put their own documents in or in these instances set up an A PIA connection to using an Alteryx workflow or other types of processes. But all of that goes back to the security and architecture of it, and that's where my team has spent a lot of time on and, and has done a very good job at setting up.

Somewhere that we're confident that our data is staying within our walls,

[00:10:34] Megan Bowers: and that's an impressive end-to-end solution that you guys have. I think it's cool that your company has a private LLM that you're continuing to build out and improve and use internally. Again, there's a lot of talk, like you said, a lot of buzzwords and.

To see it. Actually, the value realized, like you're talking about, it provided a lot of value saved time on the project was implemented in a solution I just think is really interesting. And we had chatted earlier about another use case that you had, that your team used AI to solve some challenges when classifying support tickets for a client.

Could you share about that project and the impact that solution had?

[00:11:17] Luke Cornetta: Sure. Yeah, so definitely a common theme of comment fields being really the most troublesome part of any sort of data professional's life is just unstructured free text where users or humans can put really whatever they want into it.

So it was a very similar exercise in the sense that we had a large data set of support tickets. You might interact with your internal IT to log, maybe you need a password change or you're locked out of something, or you need access to something. And the data had a a category column or some equivalent of a category column, but it really wasn't specific enough to drill into what is the team spending its time on?

What are all of these tickets and where can we have opportunities to streamline things potentially, or really just review what the support teams are spending their time on. So a category. Might have been very broad, might have just been like computer issue or something, and that's, that doesn't really tell you what the issue is.

A computer issue could be a password reset or a, a software request or my charger stopped working. There's a lot of different things, and I'm just making up examples as we go here, but this, there's a lot of different things that were embedded in those support comments. So we were able to come up with a standard framework of what are all of these types of tickets, what's all of our subcategories that we want?

Then we did a very similar exercise of passing those comments in and having it try to classify against those categories. And there's multiple ways we could have done this. We could have looked at a classified more traditional classification model, but just given the size of these comments and everything, it just was the most expedient answer to hook it up and run it through Alteryx and get our results in.

Probably a matter of hours. I think the one thing I'd highlight here is the processing time is limited, obviously by API rate calls and limiting of how many calls you can make in a certain period of time. So we had a throttle, I think we were averaging about one to 2000 records every hour or so. So there's a timing component of it, but definitely was able to get us the results we need to get a preliminary analysis back to our back, to our client.

[00:13:20] Megan Bowers: What was the business impact for the client like having those. Tickets classified correctly.

[00:13:27] Luke Cornetta: I'm not too up to speed on the business impact, but my understanding is they were looking for areas where is the most impactful place they could leverage potentially new automations or new processes or outsourcing, or just really trying to understand where the team's time was going, and then based on what those answers were, coming up with ways to reduce the time or improve.

Throughput the tickets, or I guess the big thing in support in the support land is how long is a ticket open for and how responsive is the team? So they were looking to understand what is actually happening on that.

[00:13:59] Megan Bowers: Awesome. We talked about two projects already. What are some common themes that you've been seeing across these successful implementations of AI and data projects?

[00:14:11] Luke Cornetta: So I, I think the biggest theme is really just the presence of unstructured data, and there's a lot of data sets where that exists. So the common theme is you have unstructured texts that actually contains pretty important information or could be abstracted to drive some sort of insight based on what it is.

And it's, it becomes, comes down to the repeatability of that. So if there's unstructured text that you can look at and see the information. Successfully get that information out, then that's where Alteryx can come in and really run through all your data and give you an an output that's helpful. We certainly came into instances where even as a human reading it, it wasn't really clear to me what the comment was trying to say, and the AI is not gonna work any sort of magic there if it's not even clear to a human what it's trying to say.

But in general, if there's unstructured text that can be extracted, that's really the biggest theme. I think we're just scratching the surface of it. But even in the tax world, depending on certain wording, could change to the taxable nature of a, of an invoice or a product or all that stuff that is traditionally variable.

One company could phrase something one way. Another company could phrase something another way. The language interpretation is really a big piece of it and is able to get us. Definitely a spot where it's a first level review, quality of output. We definitely do a lot of time reviewing the data that we get out of it to get comfortable with it.

We're not going through and just blindly trusting the outputs because obviously there's risks with AI around hallucinations and just misinterpretations of a prompt. Or in some cases we would actually find there is more data than what we might have been expecting. Two key things out of it, and there's actually in.

And made us, made 'em think and see, do we have to change how we're structuring our outputs? So I'd say the biggest theme is one, having that type of data. And two, the other theme that makes this to make it successful is making sure one, everyone is on the same page, all the stakeholders and everyone on the project is on the same page about how this works.

Making sure expectations are realistic. Like I said, AI's not gonna magically get a hundred percent accuracy, probably not for a long time, if not ever. But getting on the same page that this is gonna help jumpstart and get us 80, 80, 90% of the way there. And then there's still gonna be some that probably need to be looked at after that, after we get comfortable with it.

So that's the other hallmark for, uh, successful projects of these, of this nature, is just making sure expectations are all aligned and everyone's comfortable with how it's working.

[00:16:50] Megan Bowers: And I think that was important that you pointed out. One of the things that stood out to me was. You have to have some quality of data coming in.

If it's not readable by a human, if it's just absolute nonsense, then you know you're gonna get nonsense out. I think that's a common thing across all AI projects is that you have to have meaningful data going into the model, and then what you said about reviewing the outputs. I think that's important to highlight that this isn't just a quick solution where you can just trust a hundred percent and move on.

While it does save a lot of time, it sounds like there is still time being put into reviewing and taking a look at that 10% or whatever, and also. Probably spot checking some of the other outputs to make sure it's working as intended and not hallucinating like you mentioned.

[00:17:42] Luke Cornetta: For sure. Yeah, you're a hundred percent right and we've all seen, I've started to see people get emails or messages on LinkedIn and you can tell it's just AI written.

So without that review, you run some risks there that are just not for the type of work that we do at my firm's not accept, we can't take blindly trust in the accept that. So we do a lot of validation and when you do get it out a structured way, it's much easier to build validations on structured data expecting.

The values we were extracting to be within a certain range or a certain set of values. So we were able to pluck out the outliers pretty easily and then get comfortable with the rest based on, like you said, doing a sampling of the output to get comfortable with it.

[00:18:22] Megan Bowers: Yeah. And I'm, I'm just curious, since this LOM model is built by your company, as you're going through these projects, are you able to like, provide feedback to them so that.

They're continuing to tweak and adjust the model, or is it more like a closed system that you're just tweaking your inputs to get what you're looking for? So

[00:18:42] Luke Cornetta: maybe one point of clarification, I mean, we're using Microsoft Azure's infrastructure, so it's not necessarily a homegrown LLM. Within the projects, we're not allowed to use one client's data to train a model that could then later benefit another client's data without some pretty specific contracting language.

So it's all done in in a vacuum, more or less. But within the context of a project, we have some flexibility to supplement with some background information, which is always helpful. But I wouldn't say we've come across a project where we've needed to do a lot of yet. But lot of this, my experience. One project at a time, so very point solution oriented, I'd call it.

So we're using it at a point in time to extract data. At this point, I haven't been involved with like implementing an ongoing system that's gonna run business operations every day into the foreseeable future. The projects are, here's a set of data process into a new set of data for review and into a new system.

Take I there you could probably. Create something with Alteryx Gallery to really impact the ongoing business operations using this technology. I, I just haven't come across that use case just yet, but I'm sure as things continue to evolve, that's certainly a very real possibility with a tool like Alteryx.

[00:20:00] Megan Bowers: Definitely. So you've talked about some really cool projects and. I'm just wondering what advice you have for other professionals that are looking to learn more about incorporating generative AI into their data work.

[00:20:14] Luke Cornetta: It's a really good question. I think the biggest advice I'd have is the same advice I have when people ask, how do I learn Alteryx, or how do I learn any other tool?

And my answer is just go and try it and go and find a. Use case and see how it can work and not give up when it doesn't necessarily give you the right output on the first try. Like I mentioned earlier, we spent hours and hours honing in on exactly how to prompt it correctly to give us what we wanted. I think AI is probably a little trickier than other types of tools because shouldn't be going and putting their company's data into chat gt, right?

It's just not a secure thing to do. So. Some of it is gonna be limited to what specific organizations have available. I've been hearing more and more from various clients in various industries. Their companies stood up some variation of a center of excellence or a task force or something within their IT function to start investigating or building out fully some sort of internally secure interface.

And I'd say if a professional has access to, that's a great starting point to go and just see how it can shave 10, 15 minutes out of. Tasks, you know, drafting emails, drafting slide decks. It's great at even giving you Alteryx formulas. I ask it all the time to help draft an Alteryx formula at this point, not because I don't know how to do it, it's just I know that it'll take me more time than it, it would take the AI to write it.

So that's probably the biggest thing. And I find I use it every day, honestly, to, I have it up and just ask it questions or I can use it to, to use Microsoft's term like a copilot to ask questions and just bounce. If someone's, if I'm on the phone with someone and they're using acronyms I don't know, or they're using industry language, I can quickly ask it, Hey, what is this?

And get an answer. So at least I have some context. Or even this morning I was using it just to help pull together outlines that I can use in the PowerPoint. I still don't trust it blindly enough to draft everything for me, but I use it to get outlines for starting points, and then I fill in content with my own, my own.

I know plenty of people that are afraid of it or intimidated by using new technology this, and I'd say just give it a shot. Give it an earnest real chance, and I think it might surprise you.

[00:22:24] Megan Bowers: I love that. And as you start to use it more and more, learn more about how to prompt it better and it's integrating it into even the simple things like writing emails could be a good start to learning the ins and outs of, okay, here's.

How much detail I need to add to really get what I want out of these responses and what you were saying reminded me of this person, this AI expert on LinkedIn in this course she was giving where she talked about having an AI first mindset and she just has a sticky note on her computer that says AI question book.

And she really is thinking about for everything, for any kind of manual test, is there a way I could use ai? And starting off with that. Curiosity, I think can take you into cool places and learning things on the fly that you didn't even think about before, just because you're trying and experimenting. So I love that advice.

[00:23:17] Luke Cornetta: I think that's exactly right. Yeah.

[00:23:19] Megan Bowers: Awesome. Well, thanks so much for joining us today, Luke. It's been really a pleasure to hear about your projects and what you're doing at your company. For listeners that are curious to learn more, they can always comment on our community page. For this episode, we'd love to hear about how you all are using AI as well.

And thanks for joining us, Luke.

[00:23:40] Luke Cornetta: Yeah, anytime. Thanks for having me back in.

[00:23:43] Megan Bowers: Thanks for listening to download Alteryx's ebook on three generative AI use cases for finance departments, head over to our show notes on alteryx.com/podcast. See you next time.

Ep 169 Implementing Generative AI and Alteryx Solutions [00:00:00] Megan Bowers: Welcome to Alter Everything, a podcast about data science and analytics culture. I'm Megan Bowers, and today I am talking with Luke Cornetta, senior Director at Alvarez and Marsal. In this episode, we chat about how he's implemented generative AI in projects in the tax space. How he uses Alteryx to scale these projects and the business impacts and themes he's been seeing from generative AI so far. Let's get started. Well, it's awesome to have you on our podcast today. Luke, could you give a quick introduction to yourself for our listeners? [00:00:37] Luke Cornetta: Sure. Megan. My name is Luke Cornetta. I'm a senior director at var. I work in their practice based Alteryx user. Just over seven years now, I'd say. [00:00:50] Megan Bowers: And you mentioned that you're expert certified, is that right? [00:00:54] Luke Cornetta: That's right. So I have always been using Alteryx Expert was a goal of mine and about three years ago, I wanna say I, I was able to pass that certification, which was quite the challenge. But it's been very helpful to hold that certification and continue to push Alteryx to, to its limits. [00:01:10] Megan Bowers: Definitely. Yeah. That's a huge accomplishment. So congrats on that for sure. I'm really excited to talk to you today to hear about some of the cool stuff you've been doing with Alteryx in combination with generative ai. So if you could tell us a little bit about how your team has used generative AI to help with an ERP system implementation and how that went. [00:01:33] Luke Cornetta: Sure. So part of what we do in the tax practice is, is dealing with a lot of data. My team focuses on mostly data, despite sitting in the tax practice, we deal with all sorts of data from ERP systems, from PDF files, from unstructured Excel files, you name it. But we see it in the tax world. And it's really less so a tax challenge and more so a data challenge. And the Alteryx background that I have lends nicely to helping on those types of projects. And Alteryx is a really great tool from for sucking data out of one legacy system and transforming it into whatever the new system needs. So when the past, I've done these types of projects before generative AI was really mainstream. And always struggled with the, some of the one-to-one mapping this field in the old system is this field. In the new system, Alteryx is great at, maybe there's some business logic where you combine two fields or split two fields out into whatever the new system needs. But one thing that was always a struggle, I'm thinking back to one project two or three years ago, what was comments fields? There's just a spot in the old system where people can dump whatever they want into it, whether it's an email or. Just their thoughts or some notes about a client or a customer or a location. And there's really not a great way to deal with that. And in the past there's just been teams of people reading those comments or maybe we do some sort of keyword check or some sort of logic and Alteryx to try to make an attempt at looking at them. And things like RegX and text parsing can get you so far, but it turns into a brute force exercise. So that's been a struggle on these types of projects in. When we got on the opportunity to work on this latest project earlier in the year, generative AI was much more mainstream at that point. We had stood up an internal secure place for us to use generative ai, I think an internal chat CPT that was secure on on our organization's network. And we started playing around with a very similar need. You know, the project we were helping on had comments fields and notes fields that were containing a lot of really critical business information. That our client was using to run their business. Things like product pricing and hours of operation and in all sorts of formats, you know, just think dozens and dozens of people over the years typing in information in no standardized manner. So things like RegX and text parsing weren't gonna work. So we started playing around with pasting some of these comments into just, uh, internal chat. GT is probably the best way to think of it and asking it to pull out some of this key information. We found that it was actually not too bad at reading it because its large language model is meant to understand language. So, uh, it was doing a very good job at being able to read that and pull out some of the information that we wanted. So the next step was to leverage Alteryx. So that's the background on the journey of how do we see a need for generative AI in this type of, what's traditionally been an ETL data prep exercise. [00:04:36] Megan Bowers: You're using an internal LLM model to extract the relevant data from these comment fields. I thought it was funny when you mentioned anything from business hours to an email or something. [00:04:48] Luke Cornetta: Oh, yes. Um, [00:04:49] Megan Bowers: maybe someone put a shopping list in there. So you're using that model to get that information out, and then how are you scaling that in Alteryx? [00:04:57] Luke Cornetta: Yeah, so Alteryx was very good at helping us scale that. So the biggest challenge, like you said, some of these comments could have been 10 characters, they could be 5,000 characters. Their whole email chains faced it in. So it was very interesting. It was an interesting challenge to try to extract the, the necessary data, and luckily no one was expecting a hundred percent accuracy on that. I don't think it's reasonable to expect a hundred percent accuracy on this, but we were targeting roughly about 80, 90% accuracy, and we got comfortable with that number. Alteryx was really helpful in scaling this because what we were able to do was with some of the APIs that Azure has available for its LLMs, we were able to leverage just the traditional download tool in Alteryx to make those API calls and essentially pass each comment field through that API applying more or less the same prompt to it. So basically just doing the same thing over and over again by passing multiple records through with each record an a p. Go and extract the data that we needed, and then we were able to use Alteryx to then parse those results out into a more structured way to load into the target system. So it was relatively streamlined. There was definitely a lot of trial and error in this process, but we were able to get it to a pretty good spot that traditional methods just weren't gonna be able to get to in the same, in the same timeframe. [00:06:19] Megan Bowers: Very cool. Yeah. Obviously there's so much chatter around generative AI and to hear. Kind of implemented use case like that is really interesting and hopefully our listeners can think about ways they can implement generative AI in their Alteryx flows as well. But you mentioned trial and error. Did you face any challenges when you were going through like prompt engineering really getting what you wanted out of the LLM or what did that look like? [00:06:47] Luke Cornetta: Certainly. So some of the challenges, it's really all about the prompting at the most basic level. And what we were finding was, um, sometimes the comments contain multiple types of data that we needed, whether it's, like I said, pricing or hours or a few other different things. And if we tried to ask for it all at once, it would get confused or not give us as good of a result. If we, if we broke it up into a few separate different prompts, asking pricing in one prompt or hours in another prompt, or whatever it might be, that tended to give us better results depending on how we worded it. We had to be very particular in how we wanted our outputs, 'cause we wanted our outputs in a structured format that we could then further parse down the line. Sometimes it. Try to be too helpful. It would try to be very polite. It's like, here's your answer and then this. And I was like, we don't really need it to say here's your answer. We just need the answer. So it, it would try to be too helpful at times, which we appreciated, but it, we had to be very precise in the way that we worded things. So the prompts for these ended up, if you were to look at the prompt, it looks like a giant paragraph. And that's really, uh, good prompt. It shouldn't be a sentence isn't gonna get us answer. We prescriptive give some. Pretty good for most of the records. [00:08:04] Megan Bowers: That's awesome and super interesting. I have used chat, GBT, various things for like random one-off projects, but building out a prompt like that went to the point where it's a very large paragraph, I'm sure. Yeah. Best practice for really dialing in on what kind of results you wanna see. [00:08:24] Luke Cornetta: Yeah, I, I think that's exactly right. And I'd echo, you know, it was super exciting to see this generative AI technology be actually applicable in a real world scenario. Because one of the things that drives me crazy is when everything's just kind of buzzword driven and it's all marketing and no substance behind it. So I had used generative AI previously to help draft paragraphs do what everyone does with it, draft an email, draft, whatever, and see it actually. Significantly impact in the, uh, an actual engagement that, that our firm was working on was very interesting to see, and very positive to see and really has opened up a lot of doors for how we do work here. [00:09:02] Megan Bowers: Totally. Were there any security or governance concerns when you were going about this project? [00:09:09] Luke Cornetta: Sure. So when we, as a firm, when we started going down generative ai, going down that rabbit hole about maybe a year and a half ago at this point, one of the first and foremost things for us was security and governance. Being a firm that deals with confidential information with me and the tax practice, we deal with all sorts of PII and other types of data that we have to be very careful with. And as we work through our projects, security was tantamount to what we were doing. So we obviously are not allowed to use the public facing chat, GPT, nor even the some of the APIs we have to be careful about are they saving our data for training purposes. We spent a lot of time building out the backend and making sure that when we set it up on our environment, that it was truly on our environment and not going out anywhere else. That was the biggest piece, and once we got comfortable with the security side of it and the architecture side of it, we, we've been able to scale it out a bit more to give it, you know, we have a. Teams can go to and ask questions and put their PDFs in and ask questions about whatever they really want, if they put their own documents in or in these instances set up an A PIA connection to using an Alteryx workflow or other types of processes. But all of that goes back to the security and architecture of it, and that's where my team has spent a lot of time on and, and has done a very good job at setting up. Somewhere that we're confident that our data is staying within our walls, [00:10:34] Megan Bowers: and that's an impressive end-to-end solution that you guys have. I think it's cool that your company has a private LLM that you're continuing to build out and improve and use internally. Again, there's a lot of talk, like you said, a lot of buzzwords and. To see it. Actually, the value realized, like you're talking about, it provided a lot of value saved time on the project was implemented in a solution I just think is really interesting. And we had chatted earlier about another use case that you had, that your team used AI to solve some challenges when classifying support tickets for a client. Could you share about that project and the impact that solution had? [00:11:17] Luke Cornetta: Sure. Yeah, so definitely a common theme of comment fields being really the most troublesome part of any sort of data professional's life is just unstructured free text where users or humans can put really whatever they want into it. So it was a very similar exercise in the sense that we had a large data set of support tickets. You might interact with your internal IT to log, maybe you need a password change or you're locked out of something, or you need access to something. And the data had a a category column or some equivalent of a category column, but it really wasn't specific enough to drill into what is the team spending its time on? What are all of these tickets and where can we have opportunities to streamline things potentially, or really just review what the support teams are spending their time on. So a category. Might have been very broad, might have just been like computer issue or something, and that's, that doesn't really tell you what the issue is. A computer issue could be a password reset or a, a software request or my charger stopped working. There's a lot of different things, and I'm just making up examples as we go here, but this, there's a lot of different things that were embedded in those support comments. So we were able to come up with a standard framework of what are all of these types of tickets, what's all of our subcategories that we want? Then we did a very similar exercise of passing those comments in and having it try to classify against those categories. And there's multiple ways we could have done this. We could have looked at a classified more traditional classification model, but just given the size of these comments and everything, it just was the most expedient answer to hook it up and run it through Alteryx and get our results in. Probably a matter of hours. I think the one thing I'd highlight here is the processing time is limited, obviously by API rate calls and limiting of how many calls you can make in a certain period of time. So we had a throttle, I think we were averaging about one to 2000 records every hour or so. So there's a timing component of it, but definitely was able to get us the results we need to get a preliminary analysis back to our back, to our client. [00:13:20] Megan Bowers: What was the business impact for the client like having those. Tickets classified correctly. [00:13:27] Luke Cornetta: I'm not too up to speed on the business impact, but my understanding is they were looking for areas where is the most impactful place they could leverage potentially new automations or new processes or outsourcing, or just really trying to understand where the team's time was going, and then based on what those answers were, coming up with ways to reduce the time or improve. Throughput the tickets, or I guess the big thing in support in the support land is how long is a ticket open for and how responsive is the team? So they were looking to understand what is actually happening on that. [00:13:59] Megan Bowers: Awesome. We talked about two projects already. What are some common themes that you've been seeing across these successful implementations of AI and data projects? [00:14:11] Luke Cornetta: So I, I think the biggest theme is really just the presence of unstructured data, and there's a lot of data sets where that exists. So the common theme is you have unstructured texts that actually contains pretty important information or could be abstracted to drive some sort of insight based on what it is. And it's, it becomes, comes down to the repeatability of that. So if there's unstructured text that you can look at and see the information. Successfully get that information out, then that's where Alteryx can come in and really run through all your data and give you an an output that's helpful. We certainly came into instances where even as a human reading it, it wasn't really clear to me what the comment was trying to say, and the AI is not gonna work any sort of magic there if it's not even clear to a human what it's trying to say. But in general, if there's unstructured text that can be extracted, that's really the biggest theme. I think we're just scratching the surface of it. But even in the tax world, depending on certain wording, could change to the taxable nature of a, of an invoice or a product or all that stuff that is traditionally variable. One company could phrase something one way. Another company could phrase something another way. The language interpretation is really a big piece of it and is able to get us. Definitely a spot where it's a first level review, quality of output. We definitely do a lot of time reviewing the data that we get out of it to get comfortable with it. We're not going through and just blindly trusting the outputs because obviously there's risks with AI around hallucinations and just misinterpretations of a prompt. Or in some cases we would actually find there is more data than what we might have been expecting. Two key things out of it, and there's actually in. And made us, made 'em think and see, do we have to change how we're structuring our outputs? So I'd say the biggest theme is one, having that type of data. And two, the other theme that makes this to make it successful is making sure one, everyone is on the same page, all the stakeholders and everyone on the project is on the same page about how this works. Making sure expectations are realistic. Like I said, AI's not gonna magically get a hundred percent accuracy, probably not for a long time, if not ever. But getting on the same page that this is gonna help jumpstart and get us 80, 80, 90% of the way there. And then there's still gonna be some that probably need to be looked at after that, after we get comfortable with it. So that's the other hallmark for, uh, successful projects of these, of this nature, is just making sure expectations are all aligned and everyone's comfortable with how it's working. [00:16:50] Megan Bowers: And I think that was important that you pointed out. One of the things that stood out to me was. You have to have some quality of data coming in. If it's not readable by a human, if it's just absolute nonsense, then you know you're gonna get nonsense out. I think that's a common thing across all AI projects is that you have to have meaningful data going into the model, and then what you said about reviewing the outputs. I think that's important to highlight that this isn't just a quick solution where you can just trust a hundred percent and move on. While it does save a lot of time, it sounds like there is still time being put into reviewing and taking a look at that 10% or whatever, and also. Probably spot checking some of the other outputs to make sure it's working as intended and not hallucinating like you mentioned. [00:17:42] Luke Cornetta: For sure. Yeah, you're a hundred percent right and we've all seen, I've started to see people get emails or messages on LinkedIn and you can tell it's just AI written. So without that review, you run some risks there that are just not for the type of work that we do at my firm's not accept, we can't take blindly trust in the accept that. So we do a lot of validation and when you do get it out a structured way, it's much easier to build validations on structured data expecting. The values we were extracting to be within a certain range or a certain set of values. So we were able to pluck out the outliers pretty easily and then get comfortable with the rest based on, like you said, doing a sampling of the output to get comfortable with it. [00:18:22] Megan Bowers: Yeah. And I'm, I'm just curious, since this LOM model is built by your company, as you're going through these projects, are you able to like, provide feedback to them so that. They're continuing to tweak and adjust the model, or is it more like a closed system that you're just tweaking your inputs to get what you're looking for? So [00:18:42] Luke Cornetta: maybe one point of clarification, I mean, we're using Microsoft Azure's infrastructure, so it's not necessarily a homegrown LLM. Within the projects, we're not allowed to use one client's data to train a model that could then later benefit another client's data without some pretty specific contracting language. So it's all done in in a vacuum, more or less. But within the context of a project, we have some flexibility to supplement with some background information, which is always helpful. But I wouldn't say we've come across a project where we've needed to do a lot of yet. But lot of this, my experience. One project at a time, so very point solution oriented, I'd call it. So we're using it at a point in time to extract data. At this point, I haven't been involved with like implementing an ongoing system that's gonna run business operations every day into the foreseeable future. The projects are, here's a set of data process into a new set of data for review and into a new system. Take I there you could probably. Create something with Alteryx Gallery to really impact the ongoing business operations using this technology. I, I just haven't come across that use case just yet, but I'm sure as things continue to evolve, that's certainly a very real possibility with a tool like Alteryx. [00:20:00] Megan Bowers: Definitely. So you've talked about some really cool projects and. I'm just wondering what advice you have for other professionals that are looking to learn more about incorporating generative AI into their data work. [00:20:14] Luke Cornetta: It's a really good question. I think the biggest advice I'd have is the same advice I have when people ask, how do I learn Alteryx, or how do I learn any other tool? And my answer is just go and try it and go and find a. Use case and see how it can work and not give up when it doesn't necessarily give you the right output on the first try. Like I mentioned earlier, we spent hours and hours honing in on exactly how to prompt it correctly to give us what we wanted. I think AI is probably a little trickier than other types of tools because shouldn't be going and putting their company's data into chat gt, right? It's just not a secure thing to do. So. Some of it is gonna be limited to what specific organizations have available. I've been hearing more and more from various clients in various industries. Their companies stood up some variation of a center of excellence or a task force or something within their IT function to start investigating or building out fully some sort of internally secure interface. And I'd say if a professional has access to, that's a great starting point to go and just see how it can shave 10, 15 minutes out of. Tasks, you know, drafting emails, drafting slide decks. It's great at even giving you Alteryx formulas. I ask it all the time to help draft an Alteryx formula at this point, not because I don't know how to do it, it's just I know that it'll take me more time than it, it would take the AI to write it. So that's probably the biggest thing. And I find I use it every day, honestly, to, I have it up and just ask it questions or I can use it to, to use Microsoft's term like a copilot to ask questions and just bounce. If someone's, if I'm on the phone with someone and they're using acronyms I don't know, or they're using industry language, I can quickly ask it, Hey, what is this? And get an answer. So at least I have some context. Or even this morning I was using it just to help pull together outlines that I can use in the PowerPoint. I still don't trust it blindly enough to draft everything for me, but I use it to get outlines for starting points, and then I fill in content with my own, my own. I know plenty of people that are afraid of it or intimidated by using new technology this, and I'd say just give it a shot. Give it an earnest real chance, and I think it might surprise you. [00:22:24] Megan Bowers: I love that. And as you start to use it more and more, learn more about how to prompt it better and it's integrating it into even the simple things like writing emails could be a good start to learning the ins and outs of, okay, here's. How much detail I need to add to really get what I want out of these responses and what you were saying reminded me of this person, this AI expert on LinkedIn in this course she was giving where she talked about having an AI first mindset and she just has a sticky note on her computer that says AI question book. And she really is thinking about for everything, for any kind of manual test, is there a way I could use ai? And starting off with that. Curiosity, I think can take you into cool places and learning things on the fly that you didn't even think about before, just because you're trying and experimenting. So I love that advice. [00:23:17] Luke Cornetta: I think that's exactly right. Yeah. [00:23:19] Megan Bowers: Awesome. Well, thanks so much for joining us today, Luke. It's been really a pleasure to hear about your projects and what you're doing at your company. For listeners that are curious to learn more, they can always comment on our community page. For this episode, we'd love to hear about how you all are using AI as well. And thanks for joining us, Luke. [00:23:40] Luke Cornetta: Yeah, anytime. Thanks for having me back in. [00:23:43] Megan Bowers: Thanks for listening to download Alteryx's ebook on three generative AI use cases for finance departments, head over to our show notes on alteryx.com/podcast. See you next time.

This episode was produced by Megan Bowers (@MeganBowers), Mike Cusic (@mikecusic), and Matt Rotundo (@AlteryxMatt). Special thanks to @andyuttley for the theme music track, and @mikecusic for our album artwork.

Alter Everything Podcast

Podcast Guide

169: Exploring Generative AI Solutions Using Alteryx

Panelists

Topics

Transcript

This episode was produced by Megan Bowers (@MeganBowers), Mike Cusic (@mikecusic), and Matt Rotundo (@AlteryxMatt). Special thanks to @andyuttley for the theme music track, and @mikecusic for our album artwork.