Meeting Title: Brainforge x EY AI Adoption Discussion Date: 2025-06-30 Meeting participants: Uttam Kumaran, Vincent Dipalma
WEBVTT
1 00:01:02.370 ⇒ 00:01:03.710 Uttam Kumaran: Hey!
2 00:01:03.710 ⇒ 00:01:05.590 Vincent DiPalma: Hey? What’s going on? How are you?
3 00:01:05.896 ⇒ 00:01:07.119 Uttam Kumaran: Great to meet you.
4 00:01:07.330 ⇒ 00:01:12.799 Vincent DiPalma: Yeah. Great to meet you as well. Yeah. Thanks for for hopping on. Thanks for being so accommodating on the time, too.
5 00:01:12.800 ⇒ 00:01:18.540 Uttam Kumaran: Oh, man, no, thank you for being commenting. I mean I that’s it’s an incredible achievement, you know.
6 00:01:18.540 ⇒ 00:01:18.930 Vincent DiPalma: So.
7 00:01:18.930 ⇒ 00:01:24.567 Uttam Kumaran: I’m glad everybody’s doing well. And yeah, hopefully, non eventful.
8 00:01:25.080 ⇒ 00:01:33.449 Vincent DiPalma: Good. Yeah, not good. Yeah, no. So far, so good. It’s just there’s like, they they do a whole bunch of tests on babies. And do you have any kids yet.
9 00:01:33.450 ⇒ 00:01:41.600 Uttam Kumaran: No, I don’t have any kids. But a close friend of mine just had a kid, and she had a lot of blood sugar problems. So she was in the IC.
10 00:01:41.600 ⇒ 00:01:41.930 Vincent DiPalma: Yeah.
11 00:01:41.930 ⇒ 00:01:51.680 Uttam Kumaran: And like they’re transferring. But it’s like, yeah. But you know, what was kind of crazy was the whole time he was using Chat Gpt to help him learn.
12 00:01:52.300 ⇒ 00:01:52.540 Vincent DiPalma: Okay.
13 00:01:52.540 ⇒ 00:01:58.849 Uttam Kumaran: They were doing, and, in fact, it helped get her transferred to the right facility when they were trying to like.
14 00:01:58.850 ⇒ 00:01:59.590 Vincent DiPalma: Oh, wow!
15 00:01:59.590 ⇒ 00:02:10.710 Uttam Kumaran: Prevent it from happening, because, like long story short, they were in they’re in Maryland, and they were at like Baltimore Hospital need to get transferred somewhere else. And she had these blood sugar issues. And he was like.
16 00:02:10.910 ⇒ 00:02:22.509 Uttam Kumaran: you know, typically, I think people just call like Facebook groups and stuff, but he was like asking a lot of questions to it, and then was able to come prepared to those conversations with the doctors. So you’re not just like.
17 00:02:22.920 ⇒ 00:02:23.490 Vincent DiPalma: Very cool.
18 00:02:23.490 ⇒ 00:02:28.740 Uttam Kumaran: Day goes, and it was actually I told him you should write. You should write an article or something about it, because.
19 00:02:28.740 ⇒ 00:02:29.350 Vincent DiPalma: Yeah.
20 00:02:29.350 ⇒ 00:02:39.840 Uttam Kumaran: He really, I think, was able to actually add to the discussion, and got them to get her to the where they would have ended up in terms of care. Cycle a lot faster.
21 00:02:40.230 ⇒ 00:02:40.760 Uttam Kumaran: Thank you.
22 00:02:40.760 ⇒ 00:02:43.298 Vincent DiPalma: That’s cool. Yeah, that’s funny. You say that my
23 00:02:43.710 ⇒ 00:02:59.579 Vincent DiPalma: My wife had gestational diabetes, and then the baby came out and had low blood sugar, too. And so she was almost going to the Nicu. But they they prevented that. But the chat Gpt thing is funny, because I was listening to this podcast the other day. You know Brian Johnson.
24 00:02:59.580 ⇒ 00:03:00.410 Uttam Kumaran: Yes.
25 00:03:00.410 ⇒ 00:03:11.342 Vincent DiPalma: Yeah, they’re like, I want to stay alive forever. He was on this podcast and they were talking about how nurses are now starting to like, have chat, gpt on the computer to to help them diagnose issues.
26 00:03:11.630 ⇒ 00:03:17.849 Uttam Kumaran: It’s actually like you would you would want your. I kind of would want my doctor to be using it. You know, I actually know no problem with that.
27 00:03:17.850 ⇒ 00:03:34.212 Vincent DiPalma: Yeah, I don’t either, as long as they don’t take it, for you know. Yeah, as long as just give me an idea of like what this is. I mean, it’s just like it’s like Googling, but better, I mean much better for that world, you know. I’m sure they Google things all the time.
28 00:03:34.510 ⇒ 00:03:50.700 Uttam Kumaran: And it’s also, you know, like we really expect them to keep them. Such a broad memory from like medical. It’s like, Keep you want them to learn the principles and then layer on AI to help you like diagnose further.
29 00:03:50.700 ⇒ 00:04:06.319 Vincent DiPalma: Yeah, I have. When I was born I had a disease called hirschpron’s, which is like very rare, and it’s like a hole in your intestines, basically. And when I tell doctors that I don’t expect them all to know it at all. But some of them like I feel like lying to me, and they’re like, Oh, I know what that is and like, do you really? And I’m like.
30 00:04:06.320 ⇒ 00:04:07.219 Uttam Kumaran: Yeah, yeah.
31 00:04:07.220 ⇒ 00:04:09.719 Vincent DiPalma: Use chat gpt, I mean, they get a lot of information about it.
32 00:04:09.720 ⇒ 00:04:12.509 Uttam Kumaran: They may recall like a book, or whatever to go or like.
33 00:04:12.510 ⇒ 00:04:20.959 Vincent DiPalma: Yeah, right? Right? Just tell me, like, where to find. I use chat. Gp, recently, because it’s silly. I want to make a Linkedin post, and I read a book.
34 00:04:21.209 ⇒ 00:04:28.449 Vincent DiPalma: and I forgot like which page this thing I was thinking about was on, and I asked Chatgpt to like find the page, and it did it really well.
35 00:04:28.450 ⇒ 00:04:29.380 Uttam Kumaran: Oh, nice!
36 00:04:29.380 ⇒ 00:04:35.830 Vincent DiPalma: Yeah. So same thing with doctors. Right? It’s like, Hey, go find, find this one disease, and how you know everything about it.
37 00:04:35.830 ⇒ 00:04:46.580 Uttam Kumaran: Yeah, you know, yesterday I was in I’m here in New Hampshire. I was in New York just for the past few days, and I was in Boston and South Station waiting to get picked up and
38 00:04:46.908 ⇒ 00:05:11.839 Uttam Kumaran: we’ve I’ve been thinking a lot more about like time management with my time and like was talking to chat to bt about like we’re trying. I’m I think we need to measure like how much of my time via my calendar, and like we should do some stuff in slack. I was like, is there a science behind this? And if there is like, what are some books that you can recommend, like the canonical books around, like CEO, time management, or like executive time management, and it gave me some like.
39 00:05:11.960 ⇒ 00:05:16.510 Vincent DiPalma: 2 great books that are like literally, just on that topic, perfect.
40 00:05:16.510 ⇒ 00:05:18.481 Uttam Kumaran: And I’m like, this is great.
41 00:05:18.810 ⇒ 00:05:43.799 Vincent DiPalma: Yeah. So when I’m talking to clients, one of the things that I get a lot like, what’s the difference between Google and Chatgpt? And I always say that with Google like to try to break it down again, I’m intact. So like they’re they’re very like different minded. I try to break it down like when you Google, something I feel like you have to almost have like a Phd, and how to like research, because you have to like drill down with chat. Gpt! It’s like it gives you the answer.
42 00:05:43.800 ⇒ 00:05:59.200 Vincent DiPalma: I feel way quicker. I know prompting is very important, but like you can kind of get around it. Just ask it like, like you just said it gave you back the books. And I bet you it took maybe one, maybe 2 like prompts with Google. You have to go down this like rabbit hole, and it’s just like not as efficient, in my opinion.
43 00:05:59.200 ⇒ 00:06:21.220 Uttam Kumaran: Yeah, like, I had this sense of like, Hey, I’m I’m running at a time, having our time prioritizing. We’re in this sort of digital world. We’re fully remote company. So I can’t like not be on slack. Everything goes through email. But those tools have no prioritization. And I’m like, but then I also have this, like, we have a little bit of management structure. So I have a few managers. And I’m like.
44 00:06:21.220 ⇒ 00:06:29.999 Uttam Kumaran: Okay, how can I? What what system should I put in place where they could come to me with like risks? And we can batch that versus me, having to be always on, and I.
45 00:06:30.000 ⇒ 00:06:30.510 Vincent DiPalma: Yeah.
46 00:06:30.510 ⇒ 00:06:41.000 Uttam Kumaran: Take some time away from that, and I was like trying to get it to. It was giving me some good tips, but then I was also like, Look, tell me like I like to go read. If there’s a subject I’m learning about like, just give me the one book that.
47 00:06:41.000 ⇒ 00:06:41.670 Vincent DiPalma: Yeah.
48 00:06:41.670 ⇒ 00:06:43.700 Uttam Kumaran: Would recommend reading, or like.
49 00:06:43.700 ⇒ 00:06:44.130 Vincent DiPalma: And then.
50 00:06:44.130 ⇒ 00:06:53.339 Uttam Kumaran: Was like written on, based on that, and and like it, it gave me like 3 or 4. Then it was like. But honestly, you should read this one. This is probably gonna be perfect.
51 00:06:53.340 ⇒ 00:07:00.580 Vincent DiPalma: Very cool. And did you have I know we’ll probably get into it. But like any background in corporate world before you started your company.
52 00:07:00.580 ⇒ 00:07:20.189 Uttam Kumaran: Yeah. So I I worked in New York my background is in engineering. So I worked as a data engineer. So I went to Bucknell. I worked as a data engineer at Wework and then I worked at a company called Flow Code. After that, and then I was sort of started to lead data teams. And then I led product at a data startup was my last role before
53 00:07:20.190 ⇒ 00:07:41.786 Uttam Kumaran: Brain Forge. But nothing like I had done some sort of contract work, and some internships at like companies like service now, and things like that. So I and then I grew up in the Bay Area. So a lot of family friends and friends that work in like big corporate tech but nothing. But like most of my career, is built in like increasingly smaller startups.
54 00:07:42.100 ⇒ 00:07:51.779 Vincent DiPalma: Got it? Yeah? I asked, just because, like, I don’t know, I’ve been in corporate world for, like almost all of my career. And I. Every other year I try to figure out a new way, to be like better in productivity.
55 00:07:51.780 ⇒ 00:08:01.649 Uttam Kumaran: Yeah, this is like on this is really hard one, like, I feel. Usually I’m pretty good and growing up. I’ve always be like, had my life in trello, or maps things out, but.
56 00:08:02.200 ⇒ 00:08:08.470 Uttam Kumaran: This one is pretty hard because I’m simultaneous. The person who cares the most, and
57 00:08:08.810 ⇒ 00:08:18.499 Uttam Kumaran: also the person with the least time, but also I certainly probably have every answer. So it’s so hard to like what to do. I mean.
58 00:08:18.500 ⇒ 00:08:18.890 Vincent DiPalma: It’s.
59 00:08:18.890 ⇒ 00:08:27.940 Uttam Kumaran: We’re layering on a ton of AI. So I’m happy to share a lot of stuff we’re doing internally, because for me, as a weird data and AI consultancy.
60 00:08:27.960 ⇒ 00:08:51.450 Uttam Kumaran: If your AI consultancy is not using AI like to the gills, I feel like that’s a red flag. So we started using it. A lot of the way I got into being an AI consultancy is like we’ve started doing stuff. And then I was like, Oh, we should totally offer these as because we’re learning how. Okay? I I’ve been using fireflies and otter. But there’s like, I want those in slack. And I want like tailored summaries.
61 00:08:51.450 ⇒ 00:09:05.081 Uttam Kumaran: Okay, we should build something for that, like, okay, how do we do that? We. We have a internally. We started building a little bit of a platform for ourselves where across we have, like maybe 10 or so clients, there’s always meetings happening for all of those.
62 00:09:05.360 ⇒ 00:09:21.520 Uttam Kumaran: And for our really like, as you know, this is like a headcount business, like I worked in b 2 b Saas and Sas for a while, but this business is tough, because every road points to hiring more, and it’s really like not something. I want to be our 1st answer, and.
63 00:09:21.520 ⇒ 00:09:22.210 Vincent DiPalma: Right.
64 00:09:22.210 ⇒ 00:09:22.789 Uttam Kumaran: Part of this.
65 00:09:22.790 ⇒ 00:09:25.000 Uttam Kumaran: He he wants, like, yeah, yeah.
66 00:09:26.030 ⇒ 00:09:27.269 Vincent DiPalma: Okay, where’s mommy?
67 00:09:29.590 ⇒ 00:09:33.559 Vincent DiPalma: Standard close. Okay. Can you go ask it for helping with TV? I’m on a call right now.
68 00:09:34.280 ⇒ 00:09:37.279 Vincent DiPalma: She’s not here. Calm down. Go ahead, boss. Thank you.
69 00:09:37.880 ⇒ 00:09:43.889 Vincent DiPalma: Sorry. No daycare day we do. Daycare. Tuesday, Wednesday, Thursday. So Fridays.
70 00:09:43.890 ⇒ 00:09:44.610 Uttam Kumaran: Sorry.
71 00:09:44.610 ⇒ 00:09:44.930 Vincent DiPalma: Stuff.
72 00:09:44.930 ⇒ 00:09:51.599 Uttam Kumaran: So yeah, so basically, we we wanted to create a system where anyone can go back and look at all the meetings that are happening per client.
73 00:09:51.850 ⇒ 00:09:52.250 Vincent DiPalma: And also.
74 00:09:52.250 ⇒ 00:10:01.939 Uttam Kumaran: Chat with those meeting transcripts right? And then the next layer is like cool. But what are the common asks are like, Hey, generate me these tickets from those.
75 00:10:01.940 ⇒ 00:10:12.950 Vincent DiPalma: Yeah, that’s the important part, and not to cut you off, because, like what you just said sounds very familiar to like copilot, Microsoft co-pilot right? So. But like, if you can do those downstream things where like, it actually takes action.
76 00:10:12.950 ⇒ 00:10:14.380 Uttam Kumaran: Level 2, yeah.
77 00:10:14.380 ⇒ 00:10:17.160 Vincent DiPalma: And that’s what I think is gonna be required to like.
78 00:10:17.160 ⇒ 00:10:26.069 Uttam Kumaran: So like for all of our meetings, that our clients, one of the most common tasks, is taking it and turning it into tickets for a project management system, which is a huge
79 00:10:26.460 ⇒ 00:10:38.239 Uttam Kumaran: like, I worked as a product manager for a while. That’s like it’s a huge job. It takes so long. And AI get 80% of the way there, you as a Pm can go just like tweak it, make sure it’s right. Go right. And so.
80 00:10:38.240 ⇒ 00:10:39.990 Vincent DiPalma: Percent like I have an agent that
81 00:10:40.540 ⇒ 00:10:43.879 Vincent DiPalma: I call it the oh, I have a Mr. Ticket guy, or something.
82 00:10:43.880 ⇒ 00:10:44.220 Uttam Kumaran: Yeah.
83 00:10:44.220 ⇒ 00:11:12.899 Vincent DiPalma: And we have a Prd like a product requirement document, Guy and I. We do that now where we it’s kind of bootleg. We’ll take like a conversation with next actions and then upload it to those agents. And then say, create me a user story, right? Create me a Prd document, and then we send that over to the we don’t have it like actually connected to ado or anything else. It’s just kind of like. Then the persons to re, you know, recraft it. But same kind of idea, just automatic automation is where like it is key. I think.
84 00:11:12.900 ⇒ 00:11:23.969 Uttam Kumaran: Yeah, and exactly, it’s like in our business. That’s like, I want our Pm to be going and calling, get another 30 min on with client like do something that’s like actually gonna move the needle versus
85 00:11:23.990 ⇒ 00:11:43.379 Uttam Kumaran: like. And and this that’s the work that I know as a Pm. People don’t want to do so. They they half ask their tickets, and then if they do that, then the engineers eat the sort of badly groomed tickets, you know, and stuff like that. And so those are the things that we try to automate a lot internally, mainly forced by constraint because I just didn’t have money to
86 00:11:43.680 ⇒ 00:11:47.019 Uttam Kumaran: be like, okay, I can go hire super senior folks and.
87 00:11:47.290 ⇒ 00:11:47.810 Vincent DiPalma: Yeah, right.
88 00:11:47.810 ⇒ 00:11:48.579 Uttam Kumaran: You know.
89 00:11:48.910 ⇒ 00:12:09.819 Vincent DiPalma: Yeah, that’s I mean, I think you’re on the right path with that. So is that kind of what? So if you do that internally. So I wanna let me start here, I went on your website and saw that, like you are, it’s interesting because it looks like everything is really AI, now like. So I guess, let me ask, did you guys start as a data company 1st and foremost and then move into AI right? Is that what it was.
90 00:12:09.820 ⇒ 00:12:14.679 Uttam Kumaran: Yeah, that’s correct. So my background is really heavily in internal reporting, internal analytics. And then later.
91 00:12:15.210 ⇒ 00:12:38.120 Uttam Kumaran: more like building customer facing analytics systems. But everything, you know that’s related to Kpi measurement across all business functions within a company. But then that’s also like the infrastructure setup. So everything from setting up data warehouses, moving data in modeling it, making it available to reporting and then sending it to other systems. So basically, when you bring us on. You’re kind of bringing on a full stack data team.
92 00:12:38.540 ⇒ 00:13:01.579 Uttam Kumaran: And that’s sort of that’s what we I done in my career. And so that’s where we started. The AI piece really came on as like well one I was used like. I don’t think I could have done this company without it. Like as lean I am. I don’t know. It would have been way harder. So we, I’ve been using AI. I started the company about 2 years ago. So right when Chatgpt 3.5 came out.
93 00:13:01.580 ⇒ 00:13:12.350 Uttam Kumaran: So I’ve been using it for as much as possible since then, and then, sort of in about September last year I brought on our 1st dedicated AI engineer just to work with me to automate the business.
94 00:13:12.350 ⇒ 00:13:38.170 Uttam Kumaran: And then at that point I was like, okay, well, there’s so many people need this work that let’s just offer AI as a service. But we kind of didn’t figure out like what the how to join those 2 together. And I think the existing brand sort of reflects those sort of separation not really clear, like why one company should be doing both. So where we’ve kind of landed now, and I think the website will reflect that in a few weeks is basically
95 00:13:38.170 ⇒ 00:14:02.490 Uttam Kumaran: you in order to get the most out of AI. You actually need your context to be amazing. Right? Claude versus Gemini versus Openai is not really where the challenges the challenge is, can you get your most up to date? Crm information there? Can you get a great prompt in there? And can you route like your latest meeting transcript in there.
96 00:14:02.490 ⇒ 00:14:19.750 Uttam Kumaran: all on the fly, all for anybody who uses that agent. Right? So there, it’s actually more of a data engineering problem that I found versus a AI problem. Like. I think you will find that everybody will get trained on writing great prompts, and a lot of these prompts will get
97 00:14:19.750 ⇒ 00:14:33.929 Uttam Kumaran: sort of given to you. So that problem will get eliminated. But getting like the right data in is still really difficult. And second, it’s a routing it out, doesn’t need to go to your teams. Doesn’t need to go to like an email. Draft doesn’t need to go to another agent.
98 00:14:33.930 ⇒ 00:14:59.469 Uttam Kumaran: So those are all rhyme with what we do on the data side. They’re very similar. Where it’s different is the people on the data side don’t have any clue about the AI stuff and the people who have any clue about the AI agentry, especially if they in the last few years. They’re not like classical data. They’re not classical software engineers. Typically so they don’t have any clue. What this is. So they they’re used to like, sort of like.
99 00:14:59.570 ⇒ 00:15:20.400 Uttam Kumaran: yeah, like, bootstrapping stuff into context where the data people are used to writing flows. And you know. So for us, I was like, Oh, wow! It’s perfect. Because when we come into a company, we structure organize all their data. And then I really, where our output typically is like a dashboard or a meeting or report right?
100 00:15:20.400 ⇒ 00:15:44.119 Uttam Kumaran: But actually, that leads to a decision getting made. And so ultimately, the AI agent is probably a more better representation of what needs to happen with that data which an action needs to get taken or a recommendation needs to get made. And so for us, we really come in. We do all the data cleanup, we model it, and then we make it available. We make it available to Llms to take action on right.
101 00:15:44.290 ⇒ 00:15:45.639 Vincent DiPalma: That’s so funny, so that.
102 00:15:45.640 ⇒ 00:15:49.720 Uttam Kumaran: Yeah, that’s sort of like what I’ve landed on is like.
103 00:15:49.720 ⇒ 00:16:04.770 Vincent DiPalma: That’s super smart. 1. 1 thing I’m gonna say, before I get in, the next thought is, I got that from your website. I was a little confused to be honest, like where you guys wanted to go at the current moment. It felt like you. It felt like where we are right now, which is and and like I,
104 00:16:05.200 ⇒ 00:16:06.989 Vincent DiPalma: I’ve been thinking about this
105 00:16:07.630 ⇒ 00:16:10.969 Vincent DiPalma: again. I’m on maternity leave. They give me 4 months. I’m I don’t go back to October and.
106 00:16:10.970 ⇒ 00:16:11.859 Uttam Kumaran: Oh, okay. Okay.
107 00:16:11.860 ⇒ 00:16:18.800 Vincent DiPalma: I keep thinking, though, me because I am like a workaholic. So but like, where I’m thinking, is like
108 00:16:19.000 ⇒ 00:16:44.989 Vincent DiPalma: where your website is is exactly where we are in the market right now, and that is, we just are coming with AI because it’s hot. And then we know they need data, though. But no one wants to actually hear that. So we’re like, Okay, we’ll we’ll talk to you about AI. But we really want to sell you data. And then they sit there and for 6 months and make the decision because they don’t want to pay the ticket price for this data transformation. And not only the ticket price.
109 00:16:45.130 ⇒ 00:17:11.910 Vincent DiPalma: But also we’re in this fine line of do we build it on Prem, or do we host it as a Sas model? Right? And your website gave me that vibe where it was like, I feel like, you guys are coming in like, Hey, like we have these things. What do you want. So like, I agree with whatever you’re thinking about changing definitely would maybe resonate more. But we are like ey and and I’m sure many other consultancies are like in the same boat we’re like, we’re just hoping they they grab something. But we’re not actually like leading them to the water. You know what I mean.
110 00:17:11.910 ⇒ 00:17:22.499 Uttam Kumaran: Exactly so for me. The AI is the grab like right? So to get me in a conversation with folks, the AI allows us to have that meeting. But I also tell them, I said.
111 00:17:22.520 ⇒ 00:17:48.540 Uttam Kumaran: you guys may go and build a system in like a few weeks that like, seems like, it’s anecdotally working but you want to build production systems with security, and you want to get great adoption in your org. It’s not like that. It’s not like that. It’s not that easy. And so I. So I actually like it because it forces the data conversation and the data conversation again, it takes a long time, but also
112 00:17:48.540 ⇒ 00:17:59.689 Uttam Kumaran: we need to do that in order to make give you the best AI outputs and so there are people that still come to us like, can you build me this quick thing? And like, I mean, depending on how much they pay? We will consider it.
113 00:17:59.690 ⇒ 00:18:00.170 Vincent DiPalma: He did.
114 00:18:00.170 ⇒ 00:18:22.490 Uttam Kumaran: Like, I am more interested in actually getting our existing data clientele to adopt AI which we are like. Not that. Far from because we’ve model other data. And then for the AI folks, they, I think, even if we go and do the data work one, it increases our scope. And second, they’re gonna be there. You want that like, you want to be measuring their business anyways. And so I do think that’s gonna be our mix.
115 00:18:22.757 ⇒ 00:18:41.489 Uttam Kumaran: People are. Gonna realize that if you want to do things like chat with your Crm or get an get an update like in English of like which one of your clients, based on their meetings that have been having, or the activities related to them, are like most likely to churn. And who should you pay attention to. Those are things that I’m building for us right now. Right?
116 00:18:41.490 ⇒ 00:18:42.310 Uttam Kumaran: Yeah, right? Right?
117 00:18:42.310 ⇒ 00:18:53.249 Uttam Kumaran: Taking all of our Zoom Meetings and figure out like, who’s about to like who’s like probably having a bad time. Right? So then I go as a leader, go, and like, focus on that.
118 00:18:53.680 ⇒ 00:19:05.409 Uttam Kumaran: I needed to go. Have all my Zoom Meetings in one place transcribed clean transcriptions made available to then process via Llm. Like those. That’s a data, that’s all. Until the last part
119 00:19:05.540 ⇒ 00:19:10.330 Uttam Kumaran: there. It’s purely a data problem. There’s like no AI in the 1st part at all.
120 00:19:10.330 ⇒ 00:19:12.680 Vincent DiPalma: As much as people wanted to believe it is. There’s no AI.
121 00:19:12.680 ⇒ 00:19:41.890 Uttam Kumaran: There’s no AI in the 1st part at all. In fact, the AI is just like right when you need like a sort of like. For example, if you want to set up like a meeting scoring system like, take this qualified on these certain things, and and produce a score, and like a summary of why, that’s an AI thing. But for that to happen, not just like Oh, I have my transcript conveniently, and I have an agent, and I can paste it in like one thing I’m realizing to in our company’s adoption is really hard, like. I think there’s folks like us who are like.
122 00:19:42.090 ⇒ 00:19:53.600 Uttam Kumaran: take every I I would record every single thing in my life, and pass to it at this point. But even in my company, people aren’t using it, not because I think I think there was some like, Oh, I’m worried. It’s gonna like.
123 00:19:53.600 ⇒ 00:20:14.679 Uttam Kumaran: take my job and stuff. And that was easy to push, because I’m like nobody’s getting fired, in fact, want you to push off all the stuff that’s that’s like not working, or that you can push off or procrastinate on to the AI. Really, where I came in, it was like people, I just think, need AI where they work. So that’s for us in slack. They really. So we do a lot of AI agency that talks directly to slack.
124 00:20:15.482 ⇒ 00:20:21.200 Uttam Kumaran: Realize that having them open open, AI figure out the prompts do that it’s just
125 00:20:21.360 ⇒ 00:20:27.172 Uttam Kumaran: unfortunately one step too far for the average person, even at a company like ours.
126 00:20:27.650 ⇒ 00:20:31.020 Vincent DiPalma: And then try to make it more available in slack.
127 00:20:31.030 ⇒ 00:20:50.710 Uttam Kumaran: And and then also, it’s like, I think I want to move away from this like, Hey, you have an agent. It could do a hundred things instead, it should do just 3 the 3 things you need really, often, and we should start to eat those like transcripts. The tickets is a perfect one, or we have a meeting with a lead, and then we need to write a follow up perfect one, right? So
128 00:20:50.710 ⇒ 00:21:03.209 Uttam Kumaran: eat the really like easy ones, and those should happen more automatically versus like, Hey, you guys have the agent like, why aren’t you using the agent? And I don’t. I just I don’t know. I’m trying been trying to push this in our company for a long.
129 00:21:03.500 ⇒ 00:21:17.630 Uttam Kumaran: I just realized, like there, maybe in 5 years it will be much more common, everybody. But now I think still, people in their personal lives aren’t using AI, and so they don’t really know how to prompt, or they’re like get overwhelmed. And so
130 00:21:17.780 ⇒ 00:21:33.939 Uttam Kumaran: I maybe in 5 years is there. But if I need it today, and so we are gonna start to then we now have the layer of agents. We’re not gonna have agents doing very specific tasks more frequently. And make that like somewhat like happen automatically, you know, in.
131 00:21:33.940 ⇒ 00:22:01.060 Vincent DiPalma: That’s right. Yeah. So like my, so I guess a 2 second background, and then I’ll get into the point I was going to make. But I graduated a finance degree. I thought I was going to be a broker. To be quite honest, I went, and I was a credit analyst for a little while, and then I went into sales for 4 years. Really, truthfully, I was 23, and I want to make more money at the time, so I went to sales. It was good. Money was great. I was at a barbecue, and someone’s like, Hey, from arson, young like we need sales. Minded people. I’ll teach you how to do tax something. Okay.
132 00:22:01.060 ⇒ 00:22:01.580 Vincent DiPalma: wow.
133 00:22:01.580 ⇒ 00:22:16.670 Vincent DiPalma: So I did tack. I jumped over. I took a leap of faith. I took a pay cut a pretty big one, but I went over and I was helping with tax. So I was in tax for 8 years, and then the last 2 years I moved into technology 2 and a half now.
134 00:22:17.780 ⇒ 00:22:18.740 Vincent DiPalma: And
135 00:22:18.790 ⇒ 00:22:46.439 Vincent DiPalma: as soon as I came over I was I became like a product manager. Because what I needed to do what I found in tax is that the technology team would build us things. And it wasn’t anywhere near what I wanted. So I stopped using technology team. I thought they were like the worst, you know, and the only reason I went over is because during the pandemic. I saw a product being built. And I was like, this is cool like, I’m good at talking to the business team. I’m talking to the clients. I’m talking to the engineers. I’m talking designers, and I’m helping them build the product like this is what I want to do. I didn’t know what it was.
136 00:22:46.440 ⇒ 00:22:52.060 Vincent DiPalma: and I figured out. And that’s why I moved over. But I thought the technology team was like incompetent at that point.
137 00:22:52.060 ⇒ 00:22:58.049 Uttam Kumaran: What happened die because of that, because there just isn’t that person. That’s the bridge.
138 00:22:58.050 ⇒ 00:22:58.900 Vincent DiPalma: Correct. Yeah.
139 00:22:58.900 ⇒ 00:23:06.370 Uttam Kumaran: They have priority. Someone’s yelling at them to get something out. These guys to use it. No, never asked them any questions.
140 00:23:06.780 ⇒ 00:23:11.750 Uttam Kumaran: The engineer building the core stuff will be like, I don’t. He doesn’t really get what this is, even.
141 00:23:11.750 ⇒ 00:23:13.549 Vincent DiPalma: Yeah. And they don’t even care, though, like.
142 00:23:13.550 ⇒ 00:23:14.659 Uttam Kumaran: They don’t care at all. Yeah.
143 00:23:14.660 ⇒ 00:23:15.020 Vincent DiPalma: Pardon.
144 00:23:15.020 ⇒ 00:23:16.339 Uttam Kumaran: Taking tickets down and.
145 00:23:16.340 ⇒ 00:23:16.660 Vincent DiPalma: Right.
146 00:23:16.660 ⇒ 00:23:43.860 Uttam Kumaran: No problem. So in my company I am. I am. I run our AI team as a Pm. Right? So right before this, we had our planning for this week, because I’m the only person that like can see both sides. Unfortunately, like, I wish I I trying to hire someone who can see that. But I’m the only one. Really. It’s like, I get all the AI stuff. I’m really motivated to get it to work on technology. That is not really like figured out yet. So there’s gonna pick ups.
147 00:23:43.860 ⇒ 00:23:54.967 Uttam Kumaran: And then the data side I know super well. And of course, what we need as a company. It’s like that’s in my brain. So I’m the. I tried to put another Pm. On it totally. Didn’t work.
148 00:23:55.260 ⇒ 00:23:59.320 Vincent DiPalma: I bet. Yeah, especially like it will never work to what your your standards are at.
149 00:23:59.320 ⇒ 00:23:59.820 Uttam Kumaran: Yeah.
150 00:23:59.820 ⇒ 00:24:01.250 Vincent DiPalma: Like, yeah, it’s hard.
151 00:24:01.250 ⇒ 00:24:03.670 Uttam Kumaran: So, yeah, totally feel your point. Yeah.
152 00:24:03.670 ⇒ 00:24:17.719 Vincent DiPalma: And then like. So so before I left on leave, so I got a platform. I don’t know if Clarence told you, but we got a geni platform off the ground. We got 30 million dollars of funding and we launched it, and it was great. But what I’m going back to
153 00:24:17.980 ⇒ 00:24:23.979 Vincent DiPalma: is adoption. So this is why this is hitting home. Because what I’m finding from all the data is that
154 00:24:24.330 ⇒ 00:24:38.859 Vincent DiPalma: I’m giving them these tools like the firm gave everyone copilot. Okay, we’re giving them this massive Gen. AI 30 million dollar platform and no one’s using it. And I. And you know what like, he kind of put it on me a little bit. But at the end of the day, like ey
155 00:24:39.450 ⇒ 00:24:54.179 Vincent DiPalma: like, or any company like that, that’s not really a Tech Focus Company doesn’t really say, like, Okay, figure out why the people aren’t using it and go make it work. But now I told them before I left, I said, look, I don’t feel like I’ll be doing any justice if I don’t come back as an adoption lead.
156 00:24:54.180 ⇒ 00:25:11.299 Vincent DiPalma: So I’m going to be adoption, lead and external client facing. I just got promoted. So like I think that I’ll have a little more steam to do that. But what you’re saying is resonating very well, because you’re going to give people these things, and they’re not going to use them. So how do you do it? In my opinion, what you’re saying is right. You just make it part of their day to day. Life.
157 00:25:11.300 ⇒ 00:25:11.670 Uttam Kumaran: Yeah.
158 00:25:11.670 ⇒ 00:25:13.629 Vincent DiPalma: Make it like, I think, a good idea.
159 00:25:13.630 ⇒ 00:25:14.570 Uttam Kumaran: Unavoidable.
160 00:25:14.570 ⇒ 00:25:20.860 Vincent DiPalma: Unavoidable like. I actually say it all the time. I say, like, be undeniable. So like and to do that is like
161 00:25:21.240 ⇒ 00:25:31.499 Vincent DiPalma: you have to just let make it easy for them, like I just did a so real quick. I just did a an app this morning because I’m building an AI app on my my end behind the scenes. But
162 00:25:31.780 ⇒ 00:25:41.689 Vincent DiPalma: I was looking at some competitors, and one competitor had me fill out 25 questionnaire, and I was like this sucks, you know, like cause I’m I already want to stop at Number 15, you know.
163 00:25:41.690 ⇒ 00:25:42.110 Uttam Kumaran: Yeah.
164 00:25:42.110 ⇒ 00:25:59.050 Vincent DiPalma: But back to my point. So one thing is, you like you said something about like, you have all the data and like. What if you had something that could like build trends off your conversations like we’re having a conversation. I know it’s being recorded. It’s gonna go into your piggy bank. It’s gonna like, Ha Get hit up against AI right like
165 00:25:59.050 ⇒ 00:26:15.730 Vincent DiPalma: if I was a client, though, and you have 10 clients like 10 data clients like, where are those data clients on their journey to incorporating AI. And if you get a win like one client wins like what made them win. AI would be great for that like, what did AI know that you didn’t maybe realize on the online.
166 00:26:15.730 ⇒ 00:26:16.230 Uttam Kumaran: A 100 people.
167 00:26:16.230 ⇒ 00:26:19.699 Vincent DiPalma: Data analysis, right? Like, what? What was their spark? You know.
168 00:26:19.700 ⇒ 00:26:25.600 Uttam Kumaran: Exactly. And so for us, it’s like I want to. I first, st I want to get all of our teams to work on the
169 00:26:26.370 ⇒ 00:26:48.730 Uttam Kumaran: move our business forward, so that not going from transcript to tickets, I want to eat that first.st The second thing is, I want to have the ability to ask question across all of the meetings we’ve had about a client that’s like never been. I’ve never been a part of a company that has been able to do something like that like, tell me what like, for example, we’re coming up on renewal. Look through everything we’ve done, and propose a couple of things.
170 00:26:48.840 ⇒ 00:26:52.869 Uttam Kumaran: Yeah, yes, if I was really good, I could go get every transcript
171 00:26:53.180 ⇒ 00:27:05.470 Uttam Kumaran: put into chat gpt, then get it once right? And yeah, that may be a 4 h task to get one. But I need this to happen across all clients. And ultimately it needs to be someone with less AI equipped than me way less.
172 00:27:05.771 ⇒ 00:27:14.829 Vincent DiPalma: Fluidity is gonna be important for you. Cause like, you’re gonna wanna ask a question. But you’re gonna wanna rebuttal, too. You know what I mean, yeah, like, you know.
173 00:27:14.830 ⇒ 00:27:37.980 Uttam Kumaran: The other. The other thing is, for all of our clients actually built an agent for each, and the agent has access to all of the lip tickets created for it has access to all the slack messages like in the in our internal channels that we sent with it, and all the Zoom Meetings that we’ve had associated with that client. We’ll layer on code and emails. So we basically build like a client agent.
174 00:27:38.070 ⇒ 00:27:45.060 Uttam Kumaran: What that helps with, though, is not that like you? Go to the you now have a client agent, you can ask it questions. It’s like anything associated with any sort of work.
175 00:27:45.460 ⇒ 00:27:45.860 Uttam Kumaran: No.
176 00:27:45.860 ⇒ 00:28:10.100 Uttam Kumaran: for that client runs through its own client agent. So when we say, Go from transcript to ticket, you’re not just like it’s not like a blank copy paste into Chatgpt fresh. It’s going into this client agent which has all this nuance right about like. So then it helps just improve it a little bit more. But to your point it’s like it needs to be undeniable. It can’t make mistakes. And so constantly
177 00:28:10.200 ⇒ 00:28:20.370 Uttam Kumaran: I push back on our team where it’s like, Hey, the output was bad, or it messed up the formatting, and these may seem like small things to the engineers. But I’m like this is the stuff that kills this right.
178 00:28:20.370 ⇒ 00:28:21.359 Vincent DiPalma: Oh, yeah. Oh, yeah.
179 00:28:21.360 ⇒ 00:28:28.330 Uttam Kumaran: It doesn’t get it right. Second is, I don’t think people are gonna wake up in my company and be like I’m so excited to go use the AI thing.
180 00:28:28.330 ⇒ 00:28:28.980 Vincent DiPalma: Yeah, right.
181 00:28:29.282 ⇒ 00:28:51.949 Uttam Kumaran: Instead, it should be almost like one of the things I’m planning on working on is we now have the AI, those AI agents in slack where you can say, Hey, tell me what happened last meeting, or hey? The client just asked this like, How would you solve this? That’s that’s like, okay, that’s relying on the engineer or the Pm. To be sort of proactive. The second step I wanted to do is
182 00:28:52.020 ⇒ 00:29:00.429 Uttam Kumaran: when in the slack channel. If there is a question that gets asked, the AI should actually process every question and say, Can I answer this.
183 00:29:00.430 ⇒ 00:29:01.200 Vincent DiPalma: Yeah. Oh, yeah.
184 00:29:01.200 ⇒ 00:29:08.499 Uttam Kumaran: Escalate to me, or a group of people saying, Hey, I think I can answer this, and here’s what I’m thinking about sending. Should I send it.
185 00:29:08.980 ⇒ 00:29:09.770 Vincent DiPalma: 100%.
186 00:29:09.770 ⇒ 00:29:26.679 Uttam Kumaran: So it’s a all. It’s it’s an always on meaning. And at some point I can just hit, send, and it’ll respond. The last part is, after a while we will score those and say anything above 90. Rip it just like. So then it just is almost like it is a person on your team that just answers
187 00:29:27.030 ⇒ 00:29:30.189 Uttam Kumaran: 100%. A proactive person on your team. Yeah.
188 00:29:30.190 ⇒ 00:29:46.629 Uttam Kumaran: right right now, it’s being described. This is almost being stripped like an ambience. AI. I don’t know. The terms are all changing, but something that is sort of scanning it and understanding. Can I answer this? I’ll answer, because then it’s unavoidable. If it’s already in thread saying like, Hey, I think I can answer this.
189 00:29:46.820 ⇒ 00:29:52.003 Uttam Kumaran: Then that person starts conversating with it, and then it goes right. That’s what you expect out of each of us.
190 00:29:52.250 ⇒ 00:29:55.170 Vincent DiPalma: It’s so funny you’re saying this. Oh, dude! It said, resonating so.
191 00:29:55.170 ⇒ 00:29:56.120 Uttam Kumaran: Because like, if you.
192 00:29:56.120 ⇒ 00:30:03.090 Vincent DiPalma: And like, alright. So what we’re also trying to do is like exactly what you’re saying, which is like, Hey, let’s in tax again, like I have a tax return.
193 00:30:03.380 ⇒ 00:30:25.710 Vincent DiPalma: and I want something to come in and say, Hey, like I’ve already analyzed the tax return like line one is showing a charitable donation. You should probably start talking about like safe harvesting or something, and then, like I can say no like, there will be a point where it’ll get annoying. So what you need to do is, in my opinion, is it has to be something where it’s like a customized digital agent assistant for that person where it learns the way the person wants them.
194 00:30:25.710 ⇒ 00:30:27.950 Uttam Kumaran: Yes, they have also profile, right?
195 00:30:27.950 ⇒ 00:30:28.390 Vincent DiPalma: Correct.
196 00:30:28.390 ⇒ 00:30:32.449 Uttam Kumaran: So one of the things I was thinking about this, the other, I think this is where I have some ideas. And I’m like.
197 00:30:32.910 ⇒ 00:30:51.970 Uttam Kumaran: that’s gonna be 6 months before I can build this. But you know all of our employees. We were thinking of trying to get them to take like Myers Briggs, or something like that, right? Talking to Clarence about this. And I was like, Why don’t we create a profile for every person, so that when that person interacts with the agent. It’s tailored towards the way they understand stuff. Right?
198 00:30:52.080 ⇒ 00:30:55.109 Uttam Kumaran: You sort of build a context for that person. So that
199 00:30:55.500 ⇒ 00:31:08.789 Uttam Kumaran: more effective. But all this is towards is actually the fact that we are the limiting factor, like the human beings, are the blocker between this working, not the I think the technology is there to do all these things?
200 00:31:08.790 ⇒ 00:31:09.390 Vincent DiPalma: I agree with you.
201 00:31:09.390 ⇒ 00:31:28.959 Uttam Kumaran: A company like mine, with no money is able to even like try to get to there. I think it’s the human beings, and that’s not just yelling at people saying, Go, use the agents, it’s it is this like, it just like the door handle is at the height of your arm like this is, we have to design the systems.
202 00:31:29.130 ⇒ 00:31:41.799 Uttam Kumaran: And unfortunately, we are limited by we. We all use teams. We’re on meetings like that’s just the way we work now. And so our systems have to sort of mold around those versus like, go use a tool over here, learn like.
203 00:31:42.190 ⇒ 00:31:51.660 Uttam Kumaran: learn how to talk to AI and like I don’t know. I just don’t think that after trying this for for a while. Now I don’t think that everybody is going to be as
204 00:31:52.030 ⇒ 00:32:02.009 Uttam Kumaran: don’t go for the people that are, you know, give them everything because their leverage is gonna go crazy. But for the average folks. I think it has to just intrude into their.
205 00:32:02.010 ⇒ 00:32:02.450 Vincent DiPalma: Because.
206 00:32:02.450 ⇒ 00:32:03.300 Uttam Kumaran: To the day, to day.
207 00:32:03.300 ⇒ 00:32:17.879 Vincent DiPalma: I think because I think we’re coming to the same conclusion here. And I, AI, we keep saying, AI is cool, and I think it’s interesting. I think it’s fun, but the people that it does it doesn’t. I don’t really give a crap, because if I was a CEO of a company, or if I’m now a senior manager at Ey, and I want my people to use this.
208 00:32:18.460 ⇒ 00:32:21.919 Vincent DiPalma: I know that it will help their efficiency whether they like it or not.
209 00:32:21.940 ⇒ 00:32:50.569 Vincent DiPalma: so I don’t really care, so I don’t. I’m no longer trying to sell. And I’m doing this even with clients, too. How cool and fiery and hot it is. I’m now trying to sell. You need this stuff. So as a CEO or a director, or someone high up in a company. You just implement it into your workforce. And you’re going to see the efficiencies gained. Yeah. And that’s it. Like, that’s it. That’s the bottom line, like, it’s kind of stuffing it down their throat. But if you just keep saying and say isn’t so cool like, don’t you want to use it like people are gonna be like and like not.
210 00:32:50.570 ⇒ 00:33:01.020 Uttam Kumaran: No, I say, like, basically it’s here. And like you’re sitting. I so where I describe it is, I describe it as like, look, I think if you implement this to the extent you can, you’ll find 20 to 40% productivity.
211 00:33:01.020 ⇒ 00:33:02.160 Vincent DiPalma: Yeah, yeah, easy.
212 00:33:02.160 ⇒ 00:33:07.880 Uttam Kumaran: That is so tremendous. What initiative are you working on? That is also gonna give you that. And.
213 00:33:07.880 ⇒ 00:33:08.329 Vincent DiPalma: A 100%.
214 00:33:08.330 ⇒ 00:33:09.909 Uttam Kumaran: Put this up against that right and like
215 00:33:10.490 ⇒ 00:33:35.190 Uttam Kumaran: that’s what it is. And and so for me, that’s that’s sort of what I try to talk to our folks about. The other thing is on the adoption piece, because we are a data company. First, st anytime we do agent work, we measure it. So I’m measuring how many interactions we get a list of the internal employees. And we look at who’s using it? Who’s not using it so that we can say, Hey, let’s set up an office hours, or let’s go call those people the Pm’s can interview, saying, Hey, why aren’t using it
216 00:33:35.370 ⇒ 00:33:56.330 Uttam Kumaran: like, are you trained. Do you feel comfortable with it? So? And then we also do Eval. So we score. We score the AI responses. So when we get on a client. We have them build what’s called like a golden data set. So question, answer pairs like common questions that we want, and the common answers, so that I can score. And then, ideally, we want to see those scores going up over time like our.
217 00:33:56.330 ⇒ 00:33:57.780 Vincent DiPalma: Yeah. But are you seeing that.
218 00:33:57.960 ⇒ 00:34:19.400 Uttam Kumaran: Yeah, we are. We are starting to, I think what is hard about it is clients never know the scope of questions that they want the AI to answer upfront it. It just keeps expanding. So we go in there like, Hey, we want to answer these specific like customer, request type things. And then, of course, their employees ask like a hundred more things. They’re like, Yeah, we’d like to support that.
219 00:34:19.400 ⇒ 00:34:19.890 Vincent DiPalma: Yeah.
220 00:34:20.380 ⇒ 00:34:40.030 Uttam Kumaran: So we have to sort of build. But the, it’s actually not the fact that we get it right the 1st time it is, how fast can we improve like? Can we see the questions being asked isolate the bottom? 20% have a meeting to get like from our stakeholder? What the right answer should be, get that back into the training data set. And like.
221 00:34:40.230 ⇒ 00:34:42.029 Uttam Kumaran: how fast can we do that? You know.
222 00:34:42.030 ⇒ 00:34:47.636 Vincent DiPalma: But what I’m finding is that which I feel it’s actually really hurting me, and I don’t know how to fix it yet.
223 00:34:48.090 ⇒ 00:35:05.849 Vincent DiPalma: They. So we ask them for feedback. They give it feedback, and they usually I have a star rating and thumbs up. But usually I tell them these star ratings. Let’s say they give it a 2 star, and I interview them. And I say, like, why is it a 2 star? What they’re telling me is like it wasn’t a hundred percent correct. And I’m like, Wait a minute. So this saved you
224 00:35:06.050 ⇒ 00:35:18.060 Vincent DiPalma: 30 min out of an hour. But it’s not 100% correct. But it still saves you time to me. That’s a 5 star. Because it saved you. It might not be a hundred percent correct or a hundred percent finished. But it’s 5 star. And that’s what I think.
225 00:35:18.060 ⇒ 00:35:27.140 Uttam Kumaran: Thing we did is we did thumbs up and thumbs down, but we force people to give a short one line description right? So that sorry.
226 00:35:27.140 ⇒ 00:35:28.440 Vincent DiPalma: Did you find value in that.
227 00:35:28.440 ⇒ 00:35:50.969 Uttam Kumaran: Yeah, it’s helping because some people will be like, actually, this little cause, it’ll be like, we have to pull something from a spreadsheet. It’ll be like this was wrong. Okay, that’s fine. But then, if it’s like it was slow. Okay, that’s a different category of problem. The other thing is, we don’t only go off that we also build our own evaluation. So we build this. The final score takes an account. If there is user feedback
228 00:35:50.970 ⇒ 00:36:12.049 Uttam Kumaran: to use that as 50%. And then if we also evaluate based on like, okay, how grounded it was in certain things I don’t know. If, like that, 50% is the way it should be. But I also don’t like I don’t want us to be like, Oh, it is right, based on our scoring. And then it’s like, completely wrong. But then I also don’t want that person. It’s so subjective sometimes. Yeah.
229 00:36:12.050 ⇒ 00:36:15.069 Vincent DiPalma: And it’s so subjective, are you? Are you running automated, testing yet.
230 00:36:15.270 ⇒ 00:36:41.350 Uttam Kumaran: Yeah. So we’re using this company called Brain Trust. We’re running Evals based from them. We’ve tried a bunch of the ones we’re not so everything automatically goes every question. Answer, pair after the gets sent to the user, through whatever it goes through an evaluation to save a score. So on. Everything we’re we’re running a score on not doing is we’re the the feedback loop, though, I think, is really the bottleneck here in that.
231 00:36:41.390 ⇒ 00:36:58.960 Uttam Kumaran: the the faster. I can get the worst scoring stuff to our Pm. To go isolate and fix and then get put back into the training data like, okay, for example, some people are like, Hey, this is, this is totally wrong. We look at the documents that’s supposed to pull from the company has no documents on that topic.
232 00:36:58.960 ⇒ 00:36:59.650 Vincent DiPalma: Yep, that’s right.
233 00:36:59.650 ⇒ 00:37:01.000 Uttam Kumaran: What kind of cooked like? What can I do?
234 00:37:01.000 ⇒ 00:37:03.029 Uttam Kumaran: No, I’m gonna guess, because it wants to please you.
235 00:37:03.030 ⇒ 00:37:10.700 Uttam Kumaran: What can I? What can I do there right? But then so then it’s like that’s it. But then I want to be able to show the client that that is their problem. That’s not like a Us problem.
236 00:37:10.940 ⇒ 00:37:11.420 Uttam Kumaran: There are
237 00:37:11.420 ⇒ 00:37:20.969 Uttam Kumaran: times where it asks a question. And then, yeah, there’s maybe conflicting information. Or it gets the formatting. Okay, that’s a Us. Problem to go fix. But see, these are the things where I don’t think they’re.
238 00:37:21.110 ⇒ 00:37:26.340 Uttam Kumaran: It’s not. It’s it’s sort of like bug fixing for typical software
239 00:37:26.630 ⇒ 00:37:33.489 Uttam Kumaran: kind like, that’s sort of the model I base it on. But again, you need like this unit testing philosophy. So for us.
240 00:37:33.490 ⇒ 00:37:33.950 Vincent DiPalma: There’s no.
241 00:37:33.950 ⇒ 00:37:45.950 Uttam Kumaran: Scoring is the way I isolate, because otherwise I can’t look through hundreds of question answers. So I need some ability to say, every week your jobs take the bottom 20. And really just triage.
242 00:37:46.080 ⇒ 00:37:54.150 Uttam Kumaran: Was it our fault? Was it their fault? Was it a fault at all, and then get it fixed and over time. It’s just gonna get way better, you know.
243 00:37:54.150 ⇒ 00:38:09.819 Vincent DiPalma: Yeah, I’m trying to implement. Clarence has been preaching it for before he left Ui so much. But he also Clarence also helped us with our feedback components, by the way. But he was the design, the menu X designer on my platform. But like the fact that, like the reinforced learning model.
244 00:38:09.820 ⇒ 00:38:10.310 Uttam Kumaran: Yeah.
245 00:38:10.310 ⇒ 00:38:17.629 Vincent DiPalma: I need that on the fly, and I need like the weighted like you. You know, you asked a question to get. Do you get. You can get back a answer or B answer.
246 00:38:17.630 ⇒ 00:38:18.060 Uttam Kumaran: Yeah, it’s.
247 00:38:18.060 ⇒ 00:38:22.790 Vincent DiPalma: You can get either one. It just guesses, really. And then like, if I can start getting it to like, wait.
248 00:38:22.790 ⇒ 00:38:23.280 Uttam Kumaran: Yeah.
249 00:38:23.280 ⇒ 00:38:37.390 Uttam Kumaran: to give me the a answer like, I want that on the fly, because what I want is, I want someone to be like, no, that’s wrong, because XY. And Z. And I don’t want to evaluate anything. I just want them to go back into the model and know to tell the Llm. To now start shifting the weight like that’s what I want. Automatically.
250 00:38:37.400 ⇒ 00:38:42.519 Uttam Kumaran: There’s this whole thing I haven’t. I’ve bookmarked couple of things on like what’s called self healing agents. Right?
251 00:38:42.520 ⇒ 00:38:44.240 Vincent DiPalma: Cool, nice.
252 00:38:44.240 ⇒ 00:39:04.759 Uttam Kumaran: This concept. Maybe I’ll try to dig up. I it’s I like my, it’s in some reading list I have. Yes, it should learn, and and I don’t know whether it goes and improve the prompt, or it goes and improve something in the training set. But ideally, yes, both of those should be like in my example, where I said, the Pm. Is in the loop. There, that sounds like another agent should be figuring out the triage.
253 00:39:04.760 ⇒ 00:39:07.040 Vincent DiPalma: Oh, yeah, so it’s called Llm. As a judge I don’t know.
254 00:39:07.040 ⇒ 00:39:08.330 Uttam Kumaran: Yes. LM. As a judge.
255 00:39:08.330 ⇒ 00:39:08.680 Uttam Kumaran: Exactly.
256 00:39:08.680 ⇒ 00:39:09.130 Uttam Kumaran: Yeah.
257 00:39:09.130 ⇒ 00:39:14.050 Vincent DiPalma: That’s what we’re trying to implement as a judge with, we’re using data bricks, or we’re gonna be using databricks.
258 00:39:14.050 ⇒ 00:39:14.530 Uttam Kumaran: Cool.
259 00:39:14.870 ⇒ 00:39:15.890 Vincent DiPalma: So, but they.
260 00:39:15.890 ⇒ 00:39:28.129 Uttam Kumaran: It was too hard. It was too hard for me to see the whole thing in my head without like doing the manual loops a couple of times. I don’t wanna like I don’t wanna just build like the whole system. And then, like there’s gonna be too many moving pieces.
261 00:39:28.130 ⇒ 00:39:28.720 Vincent DiPalma: Yeah.
262 00:39:28.720 ⇒ 00:39:35.330 Uttam Kumaran: So first, st it’s like, maybe we should include Lm. As a judge should just catch like the easy one, and then we sort of work it up, you know.
263 00:39:35.330 ⇒ 00:39:45.380 Vincent DiPalma: So they Llm. Could be wrong. Llm. As a judge could be wrong as well. But what I’m when I come back to work and again adoption. But what I’m really trying to get my clients had when I do, the client facing stuff
264 00:39:45.440 ⇒ 00:40:08.120 Vincent DiPalma: is, it’s going to be wrong. It’s going to be wrong a lot. And like I need them to be all in with us, because it’s like it needs to be a team effort. We’re doing this from the ground up and like, because if they come into this thinking like any other technology, a power bi report is wrong, we get killed. But like that’s not this. That’s that’s not this at all. This is totally different, and it’s not pointing fingers and blames. I would want to get rid of whose fault.
265 00:40:08.120 ⇒ 00:40:09.880 Uttam Kumaran: You need them to work with you. Yeah.
266 00:40:09.880 ⇒ 00:40:23.600 Vincent DiPalma: It has to be, and I know that we, you know, at my company we have big dogs like a lot of big fish that like really important, if something’s wrong, could really screw you. But, like right now, let’s not have that decision like the Llm. Is never making a decision for you you always have. That’s a hundred.
267 00:40:23.600 ⇒ 00:40:47.659 Uttam Kumaran: So I always describe it as like, this is a human in the loop thing, like someone gets a a jurisdiction, someone gets a judgment, and it takes that as information. So my job is, instead of looking through 10 documents finding, and instead of you missed 5 of them. And then now you have to go waste someone else’s time like all those downstream. That’s what we’re replacing here. Really, AI is a 1st point of escalation, like even in my.
268 00:40:47.660 ⇒ 00:40:48.030 Vincent DiPalma: Exactly.
269 00:40:48.030 ⇒ 00:40:56.449 Uttam Kumaran: I’m like, I just want to replace me as a 1st level of escalation. So AI, better human than manager than me.
270 00:40:56.450 ⇒ 00:40:57.180 Vincent DiPalma: A 100%.
271 00:40:57.180 ⇒ 00:40:59.890 Uttam Kumaran: One more barrier, and those are the things I think
272 00:41:00.080 ⇒ 00:41:15.260 Uttam Kumaran: I I never try to describe it, because this is where, like there is what’s being written in like the journal. And then there’s like, what’s on the ground. I’m like, this is not here to like. We’re not replacing 100% of people. It’s not gonna do everything, but also partnership, like we have to work with you to write that question. Answer, document.
273 00:41:15.260 ⇒ 00:41:15.660 Vincent DiPalma: Yeah.
274 00:41:15.660 ⇒ 00:41:35.112 Uttam Kumaran: We need weekly or bi weekly like these triage, or where you’re looking through the responses because you’re gonna be surprised on what your people are trying to ask. 3rd is like, the adoption is going to be very hard, like, even in my company. We are AI company. It’s absolutely brutal like, and I disappointed
275 00:41:35.460 ⇒ 00:41:36.170 Vincent DiPalma: It is disappointing.
276 00:41:36.170 ⇒ 00:41:43.329 Uttam Kumaran: That is the chat. Oh, you know when, when? Yeah, you know how, as powerful the tech is, and I’m like, Damn, we are just.
277 00:41:43.610 ⇒ 00:41:44.000 Vincent DiPalma: Set.
278 00:41:46.180 ⇒ 00:41:47.009 Uttam Kumaran: Care about this.
279 00:41:47.010 ⇒ 00:41:56.529 Vincent DiPalma: Yeah, we just got the other day I got a my counterpart still working, obviously because I’m the one that had the baby. But he he pings he text me the other day and was like.
280 00:41:56.530 ⇒ 00:42:17.750 Vincent DiPalma: I just got the you spend 30 million dollars on a platform, and no one’s using the AI. Why question? And it’s like, and they’re using it. But like one little thing throws them off right like it didn’t give me the right answer. So I’m just gonna go chat, Gpt, instead of your platform. It’s like that also is probably not giving you the right answer. But maybe it’s giving a little bit better of an answer like, it’s so frustrating, man. Or did you guys get funding yet? Or did you go through.
281 00:42:17.750 ⇒ 00:42:20.380 Uttam Kumaran: No dude. We are completely bootstrapped.
282 00:42:20.380 ⇒ 00:42:21.560 Vincent DiPalma: Oh, no!
283 00:42:21.560 ⇒ 00:42:22.949 Vincent DiPalma: So so it’s.
284 00:42:23.750 ⇒ 00:42:39.279 Uttam Kumaran: A long journey. I don’t. So again, my background is not in sort of consulting, but I went and read all the big books about how to run some of these, and and even our team, you know most of the folks. There’s only one person who interned at Ui. Everybody else is worked internally, like as Ng.
285 00:42:39.525 ⇒ 00:42:42.219 Uttam Kumaran: well, you’re very well spoken, so you can totally do it.
286 00:42:42.220 ⇒ 00:42:44.111 Uttam Kumaran: I appreciate it, appreciate it.
287 00:42:44.490 ⇒ 00:42:44.870 Vincent DiPalma: Yeah.
288 00:42:44.870 ⇒ 00:42:49.230 Uttam Kumaran: I learned I worked with executives my whole career, like as being a data engineer, you know.
289 00:42:49.230 ⇒ 00:42:49.560 Vincent DiPalma: Yeah.
290 00:42:49.720 ⇒ 00:43:04.880 Uttam Kumaran: And so I have this sort of business sense. I also studied a little bit of finance in school, and so I don’t know but the business is not hard. I don’t think the technology stuff we’re doing is the hardest part for me, one just running a business. This is like it’s like chewing glass every day, like some.
291 00:43:04.880 ⇒ 00:43:05.929 Vincent DiPalma: Oh, yeah, I bet.
292 00:43:05.930 ⇒ 00:43:19.129 Uttam Kumaran: Something is going wrong every day. It’s brutal. Second is running a cash flow. Heavy business like this, where it’s casual problems, where like it’s money comes into spikes. But like it’s just that’s brutal.
293 00:43:19.130 ⇒ 00:43:19.490 Vincent DiPalma: Oh, yeah.
294 00:43:20.309 ⇒ 00:43:26.049 Uttam Kumaran: But like the every day that we’re in business longer, it gets easier like.
295 00:43:26.600 ⇒ 00:43:39.060 Uttam Kumaran: Come back to want to work with us, we still find ourselves in doing great client work. I think the AI stuff we’re doing. I’m not reading many people doing in in like our context, in in consulting or professional services.
296 00:43:40.370 ⇒ 00:43:42.470 Vincent DiPalma: And if they are, I think they’re lying. To be quite honest.
297 00:43:42.470 ⇒ 00:43:46.929 Uttam Kumaran: I agree. I also agree because I read a lot of stuff. And I’m like, I don’t know, like.
298 00:43:46.930 ⇒ 00:43:50.619 Vincent DiPalma: No, it just looks good on paper, but it’s not really get under the hood, and it’s not real.
299 00:43:50.620 ⇒ 00:43:57.749 Uttam Kumaran: The AI shops. They’re like trying to be like, yeah, we can help you find, like leads faster like these are like, not the core problems.
300 00:43:57.750 ⇒ 00:44:02.900 Vincent DiPalma: No, not at all at all, and I think they’re all just trying to wait to get bought out. To be honest, I got.
301 00:44:02.900 ⇒ 00:44:03.290 Uttam Kumaran: Yeah, and.
302 00:44:03.290 ⇒ 00:44:07.900 Vincent DiPalma: There’s like a certain level of value that they really think they can give. And they just they’re trying to get seen to get bought.
303 00:44:07.900 ⇒ 00:44:28.770 Uttam Kumaran: And all the really core. Smart people, I think, are in products like building products. But a lot of those 2 I’m like these are these are, you know, I was having this whole thought process where I’m like these tool, the systems we’re building are so bespoke that you can’t put out like a generalized platform. The platforms already exist like Openai already is all their things. Api. Still, it’s not working right.
304 00:44:28.770 ⇒ 00:44:29.909 Vincent DiPalma: Is it 100%.
305 00:44:29.910 ⇒ 00:44:35.209 Uttam Kumaran: I don’t think these are gonna end up being like you could buy sas off the shelf to do this.
306 00:44:35.210 ⇒ 00:44:35.779 Vincent DiPalma: I agree.
307 00:44:35.780 ⇒ 00:44:40.289 Uttam Kumaran: Couple of different like data movements as and like a prompting one. But like.
308 00:44:40.580 ⇒ 00:45:04.580 Uttam Kumaran: I just don’t see this working the same way as like. And also there’s a lot of wrong with, like, okay, Hubspot doesn’t work for every single company, right? But then, for the 40% doesn’t work, they should be able to build their own thing right, like they should be able to vibe code their own thing. Why not or if they like? If their feature set fits 20% with Hubspot. Why would they even buy that like nobody’s built that like, I think it’s gonna go more towards
309 00:45:04.840 ⇒ 00:45:09.360 Uttam Kumaran: just like really narrow, more custom software that people will build for themselves
310 00:45:10.000 ⇒ 00:45:22.090 Uttam Kumaran: the most generic thing like salesforce. Yes, it’s a platform. Yes, you could do everything. But like, do you want to do that? And why not? Just buy the thing that works directly for you at the stage? You’re in.
311 00:45:22.268 ⇒ 00:45:26.199 Vincent DiPalma: Yeah. So what I’m trying to do, I mean, I don’t know if I should. I know we’re recorded. But whatever like.