Meeting Title: Lead Agent Feedback Date: 2025-01-08 Meeting participants: Nicolas Sucari, Uttam Kumaran, Miguel De Veyra, Casie Aviles, Connor Fenn
WEBVTT
1 00:00:25.250 ⇒ 00:00:30.149 Miguel de Veyra: Need a phone is no, Scott’s not gonna be here right? Is he.
2 00:00:31.390 ⇒ 00:00:35.429 Uttam Kumaran: No, no, not for this. This is for this is for Casey’s agent.
3 00:00:35.430 ⇒ 00:00:36.970 Miguel de Veyra: Oh, okay. Okay. Yeah. Yeah. Okay.
4 00:00:37.410 ⇒ 00:00:38.129 Miguel de Veyra: Oh, yeah.
5 00:00:42.858 ⇒ 00:00:51.760 Uttam Kumaran: Cool. I think, Casey, if you want to just take it away, and then I can. I’m recording this meeting so I can share it broadly, but I’ll also just ping
6 00:00:56.200 ⇒ 00:00:57.570 Casie Aviles: Okay. Yeah.
7 00:00:58.180 ⇒ 00:00:58.770 Uttam Kumaran: Go ahead!
8 00:00:58.770 ⇒ 00:01:04.050 Casie Aviles: So, okay, so just to yeah, I guess to start
9 00:01:05.062 ⇒ 00:01:11.000 Casie Aviles: so for the lead agent, I think, yeah, everyone’s been using it already for some time now. And
10 00:01:12.818 ⇒ 00:01:18.869 Casie Aviles: yeah, and over over time, we’ve had some iterations like, we’ve we’ve added some features. And
11 00:01:19.550 ⇒ 00:01:23.059 Casie Aviles: yeah, so currently, I’m demoing the latest version that we have.
12 00:01:24.020 ⇒ 00:01:32.669 Casie Aviles: So yeah, so just to give you an intro basically, we know that it takes an input so we give it what to research for
13 00:01:33.392 ⇒ 00:01:38.230 Casie Aviles: the the AI processes that calls its tools, and then it produces the output.
14 00:01:39.030 ⇒ 00:01:49.299 Casie Aviles: So yeah, we could try it on slack. Wait, let me just yeah. And
15 00:01:49.910 ⇒ 00:01:52.770 Casie Aviles: I think, yeah, we we use this already. But
16 00:02:00.090 ⇒ 00:02:05.385 Casie Aviles: so before, like I, I think I already mentioned, was using Zapier before, and now we have.
17 00:02:07.060 ⇒ 00:02:15.839 Casie Aviles: it’s all on any 10, and we have this it’s much cleaner now, and we have this app over here. So we we invited to the channels.
18 00:02:16.440 ⇒ 00:02:21.590 Casie Aviles: and the bot should leave a reaction this like this.
19 00:02:21.730 ⇒ 00:02:28.059 Casie Aviles: So to talk about talk more about the tools that it’s equipped with. So currently we have.
20 00:02:29.180 ⇒ 00:02:35.249 Casie Aviles: So we have the output. So this is how it looks like. And just to talk more about the tools that it’s equipped.
21 00:02:35.580 ⇒ 00:02:37.869 Casie Aviles: We have mainly 2
22 00:02:38.801 ⇒ 00:02:44.370 Casie Aviles: for the research for actually researching. So for the 1st one is a web scrape.
23 00:02:44.660 ⇒ 00:02:48.529 Casie Aviles: So this for for the web script we have. We’re using Exa.
24 00:02:49.950 ⇒ 00:02:56.020 Casie Aviles: We could inspect much it more in detail, like these are like the parameters that we are sending.
25 00:02:56.380 ⇒ 00:03:02.210 Casie Aviles: So we have, like the Ids, which should contain a list of the Urls
26 00:03:02.630 ⇒ 00:03:07.339 Casie Aviles: that we want to scrape and then we have sub page targets so
27 00:03:07.750 ⇒ 00:03:16.819 Casie Aviles: kind of what? Which pages or sub pages do we want to target when doing our scrape, and also the limit of
28 00:03:17.340 ⇒ 00:03:19.370 Casie Aviles: the sub pages to crawl?
29 00:03:20.020 ⇒ 00:03:24.480 Casie Aviles: And just text is true is to just get the text, yeah, sorry. Go ahead.
30 00:03:25.170 ⇒ 00:03:28.650 Uttam Kumaran: Is there a reason? For for 5.
31 00:03:30.856 ⇒ 00:03:37.430 Casie Aviles: Yeah, because when I when I raised the numbers of sub pages, sometimes it would get the
32 00:03:38.070 ⇒ 00:03:41.070 Casie Aviles: context, error like limit exceeded. Yeah.
33 00:03:41.670 ⇒ 00:03:46.280 Casie Aviles: So for now I I limited it to 5,
34 00:03:50.220 ⇒ 00:03:54.960 Casie Aviles: right so the next one is for the Apollo tool.
35 00:03:55.740 ⇒ 00:04:02.539 Casie Aviles: So for the Apolitol, it takes this a profile, URL, and it performs enrichment.
36 00:04:03.010 ⇒ 00:04:08.015 Casie Aviles: So given the profile, URL, it should output like,
37 00:04:08.740 ⇒ 00:04:11.680 Casie Aviles: yeah, we could check the executions over here real quick.
38 00:04:13.090 ⇒ 00:04:16.829 Casie Aviles: And so we could inspect much more in detail what the.
39 00:04:17.029 ⇒ 00:04:25.789 Casie Aviles: what kind of data it produces. But yeah, it’s responsible for, like getting the email and the contact details among other things.
40 00:04:28.170 ⇒ 00:04:30.150 Casie Aviles: So yeah, over here.
41 00:04:31.610 ⇒ 00:04:35.670 Casie Aviles: So yeah, we could see here the name of the person. Their link.
42 00:04:35.670 ⇒ 00:04:39.019 Uttam Kumaran: And this only happens if you give a Linkedin.
43 00:04:40.390 ⇒ 00:04:43.250 Casie Aviles: Yes, yes, if you give the linking profile.
44 00:04:44.150 ⇒ 00:04:44.750 Uttam Kumaran: Okay.
45 00:04:46.560 ⇒ 00:04:54.020 Casie Aviles: Right? So, yeah, those are the 2 main tools that we use that the A the agent uses for doing their research.
46 00:04:54.860 ⇒ 00:04:58.769 Casie Aviles: Yeah. And so the output would look something like this.
47 00:04:59.380 ⇒ 00:05:06.439 Casie Aviles: So we’ve adjusted the format. So now it should return the company overview
48 00:05:07.718 ⇒ 00:05:14.180 Casie Aviles: the key contact. So this is from the enrichment tool, opportunity, assessment
49 00:05:15.150 ⇒ 00:05:19.280 Casie Aviles: qualification, and next steps and recommended next steps.
50 00:05:20.230 ⇒ 00:05:25.219 Casie Aviles: And also, yeah. So there’s also the relevant case studies. So from
51 00:05:26.389 ⇒ 00:05:31.330 Casie Aviles: this is from our services. Or like, I mean, case studies that we got from.
52 00:05:32.950 ⇒ 00:05:34.529 Casie Aviles: yeah from the website.
53 00:05:35.420 ⇒ 00:05:38.769 Casie Aviles: And also we have reflections over here. So
54 00:05:39.170 ⇒ 00:05:45.619 Casie Aviles: the the agent would kind of provide its reasoning why it chose to highlight certain areas.
55 00:05:46.690 ⇒ 00:05:48.590 Casie Aviles: So and yeah, there’s also.
56 00:05:49.250 ⇒ 00:05:52.339 Nicolas Sucari: Can I ask you the case? Studies are coming from the.
57 00:05:52.640 ⇒ 00:05:59.160 Nicolas Sucari: from the workflow, like what we have there already in the, in the website, or we are using beehive.
58 00:05:59.430 ⇒ 00:06:00.510 Nicolas Sucari: Do you know what I’m.
59 00:06:01.280 ⇒ 00:06:10.059 Uttam Kumaran: So we’re gonna actually be moving this to all pull from notion, because notion is gonna have like a way more content. So I I think that’s probably one
60 00:06:10.730 ⇒ 00:06:15.019 Uttam Kumaran: one thing to note here is that all the case studies and services are all going to be pulled
61 00:06:15.180 ⇒ 00:06:20.030 Uttam Kumaran: basically live from notion which we just finished this week, so that should be done soon.
62 00:06:20.030 ⇒ 00:06:24.590 Nicolas Sucari: Okay. So maybe maybe we’ll have their information that is not actually published in anywhere. Right?
63 00:06:24.590 ⇒ 00:06:25.180 Uttam Kumaran: Correct.
64 00:06:25.180 ⇒ 00:06:27.090 Nicolas Sucari: It’s just internally. Okay, okay.
65 00:06:27.090 ⇒ 00:06:27.770 Uttam Kumaran: Correct.
66 00:06:29.790 ⇒ 00:06:33.630 Casie Aviles: Yes. So yeah, what else?
67 00:06:35.130 ⇒ 00:06:41.939 Casie Aviles: Right? So the other thing that we’ve equipped the agent with is the this notion agent.
68 00:06:42.230 ⇒ 00:06:45.213 Casie Aviles: So yeah, Miguel’s set up this
69 00:06:45.990 ⇒ 00:06:49.699 Casie Aviles: way to extract content from notion. And
70 00:06:49.930 ⇒ 00:06:58.779 Casie Aviles: we he already created this agent for that and what basically, it acts like a tool for for this lead research agent.
71 00:06:59.360 ⇒ 00:07:06.020 Casie Aviles: So we could ask it some questions like, what are
72 00:07:12.010 ⇒ 00:07:14.339 Casie Aviles: okay? Yeah. So I.
73 00:07:15.100 ⇒ 00:07:15.990 Uttam Kumaran: So it does the threading.
74 00:07:15.990 ⇒ 00:07:17.040 Casie Aviles: Yeah, and it
75 00:07:18.733 ⇒ 00:07:27.070 Casie Aviles: yeah, over here. But it doesn’t like reply to. If if we don’t target it, it won’t reply. But we have to target here. Still.
76 00:07:28.410 ⇒ 00:07:30.950 Casie Aviles: So, but yeah, it has like one thread.
77 00:07:32.420 ⇒ 00:07:34.220 Connor Fenn: Can you guys hear me? Yeah.
78 00:07:34.220 ⇒ 00:07:34.880 Uttam Kumaran: Yeah.
79 00:07:35.070 ⇒ 00:07:41.516 Connor Fenn: So I just care. As we build out notion, the responses for these will just get better and better. Right is that.
80 00:07:43.927 ⇒ 00:07:51.069 Miguel de Veyra: Casey. There, I think the reason the answer is kind of messed up is because I reprompted the entire agent.
81 00:07:52.050 ⇒ 00:07:53.190 Casie Aviles: Oh, okay.
82 00:07:54.120 ⇒ 00:07:58.239 Miguel de Veyra: So maybe it might be better to just connect the rag, since it’s just normal rag.
83 00:08:00.240 ⇒ 00:08:05.850 Casie Aviles: Okay, then, okay, but yeah, ideally, this is like, kind of the the results that we would pull from
84 00:08:06.520 ⇒ 00:08:10.379 Casie Aviles: notion. Right? So we have this.
85 00:08:10.920 ⇒ 00:08:18.129 Casie Aviles: So the the good thing here is that instead of that, just hard coding, because previously I just hard coded it into the prompt.
86 00:08:19.279 ⇒ 00:08:22.690 Casie Aviles: Now we could like, you know, dynamically, pull from
87 00:08:22.990 ⇒ 00:08:26.209 Casie Aviles: the database, which is super base. And
88 00:08:26.690 ⇒ 00:08:31.360 Casie Aviles: yeah, and then it also, it’s also from notion. So yeah,
89 00:08:33.970 ⇒ 00:08:37.770 Casie Aviles: right. So those are like the tools that we have right now and then.
90 00:08:38.140 ⇒ 00:08:42.689 Casie Aviles: I I guess the other thing, the other tool that we have is this feedback alert.
91 00:08:45.070 ⇒ 00:08:48.790 Casie Aviles: So for the feedback, alert, we could
92 00:08:49.000 ⇒ 00:08:53.409 Casie Aviles: do something like if you’re not like satisfied with the
93 00:08:53.720 ⇒ 00:08:57.599 Casie Aviles: or you, you would 1st give it like this. Numerical rating from one to 5.
94 00:08:58.280 ⇒ 00:09:03.030 Casie Aviles: So it’s indicated here. On with each reply.
95 00:09:03.410 ⇒ 00:09:08.330 Casie Aviles: So we have 4 out of 5, and then we give it the reasoning. So let’s say, incomplete data.
96 00:09:10.190 ⇒ 00:09:11.255 Casie Aviles: So it should
97 00:09:12.060 ⇒ 00:09:18.688 Casie Aviles: receive this. And the idea behind is, we want to have like, we want to be notified of the
98 00:09:19.790 ⇒ 00:09:26.750 Casie Aviles: of your feedback. So it, it sends one to the AI team, and then we we have
99 00:09:27.140 ⇒ 00:09:32.489 Casie Aviles: it over here. So we got the rating, and then and then we also have, like a suggested option.
100 00:09:33.010 ⇒ 00:09:38.160 Casie Aviles: So that’s what one of the new features that we’ve implemented for the bot.
101 00:09:40.620 ⇒ 00:09:44.889 Nicolas Sucari: You just you just directly. You just directly type like the rate
102 00:09:45.638 ⇒ 00:09:48.079 Nicolas Sucari: you don’t type like feedback or anything.
103 00:09:49.770 ⇒ 00:09:51.770 Casie Aviles: You could you could.
104 00:09:52.320 ⇒ 00:09:53.730 Uttam Kumaran: It. It figures it out.
105 00:09:53.730 ⇒ 00:09:54.480 Casie Aviles: Over here.
106 00:09:56.480 ⇒ 00:10:00.280 Nicolas Sucari: Oh, yeah. But go to the thread. Go to the thread with the message, please.
107 00:10:01.845 ⇒ 00:10:02.170 Nicolas Sucari: Sure.
108 00:10:02.170 ⇒ 00:10:02.870 Nicolas Sucari: Let me check.
109 00:10:04.440 ⇒ 00:10:07.940 Nicolas Sucari: Okay, you just ugly researcher again. And okay.
110 00:10:08.090 ⇒ 00:10:14.880 Nicolas Sucari: And the message you don’t like start with a feedback word or something like that, so that
111 00:10:15.150 ⇒ 00:10:16.999 Nicolas Sucari: it understands its feedback.
112 00:10:19.150 ⇒ 00:10:21.198 Uttam Kumaran: I think it knows.
113 00:10:21.910 ⇒ 00:10:29.260 Nicolas Sucari: I mean, because what I’m saying is as this is, gonna be a thread like you can ask like different questions. So how it’s gonna I didn’t.
114 00:10:29.260 ⇒ 00:10:29.609 Uttam Kumaran: But it.
115 00:10:29.610 ⇒ 00:10:30.210 Nicolas Sucari: Twenties.
116 00:10:30.210 ⇒ 00:10:33.210 Uttam Kumaran: It knows. Yeah, yeah, no, it understands. Yeah.
117 00:10:33.890 ⇒ 00:10:38.790 Miguel de Veyra: Can we move this feedback thing? Not into AI team, maybe like a feedback.
118 00:10:38.790 ⇒ 00:10:40.369 Miguel de Veyra: I agree we should put that.
119 00:10:41.220 ⇒ 00:10:44.540 Uttam Kumaran: We should put this in the in the Feedback channel.
120 00:10:44.740 ⇒ 00:10:45.870 Miguel de Veyra: Yeah, yeah.
121 00:10:46.320 ⇒ 00:10:48.051 Uttam Kumaran: I guess I wanted to leave.
122 00:10:48.440 ⇒ 00:10:58.160 Uttam Kumaran: I think this primarily is built for the sales team. So I want to leave Connor. I know we’re up against time. But I want to leave. I want to ask him for feedback on how this is going and
123 00:10:58.470 ⇒ 00:11:01.133 Uttam Kumaran: and like how it’s been working for him and given
124 00:11:02.300 ⇒ 00:11:05.560 Uttam Kumaran: what he saw today, like what else we could try to do here.
125 00:11:06.340 ⇒ 00:11:25.579 Connor Fenn: I think it’s really cool. The interaction piece was the one like the big thing that I always, when I was using it in the past that I wish that I had had, but it seems like I can do that now, which is pretty cool. My one question would be when the agent is pulling data from notion.
126 00:11:25.860 ⇒ 00:11:32.920 Connor Fenn: Is it only like the specific things that it’s like told to or like? If I write notes
127 00:11:33.220 ⇒ 00:11:42.040 Connor Fenn: in my, you know in the client page, is it also gonna have access to those so like when I’m asking it questions, it’ll have that information, too.
128 00:11:45.240 ⇒ 00:11:46.280 Casie Aviles: At the moment.
129 00:11:46.280 ⇒ 00:11:48.660 Uttam Kumaran: Good for yeah, for Casey and Miguel.
130 00:11:48.990 ⇒ 00:11:52.080 Miguel de Veyra: Sorry. Can you repeat it? I was. I was checking something in motion.
131 00:11:52.080 ⇒ 00:11:53.656 Connor Fenn: Yeah, yeah, no worries.
132 00:11:54.340 ⇒ 00:12:05.900 Connor Fenn: So the data that the agent pulls from notion is that only specific data that it’s told to look at or like, if I write
133 00:12:06.020 ⇒ 00:12:15.110 Connor Fenn: notes about a specific client that like we’ve been working with over a couple of weeks, is it also gonna have access to that data. So when I’m.
134 00:12:15.382 ⇒ 00:12:29.569 Miguel de Veyra: No, no, not yet like. So the the setup right now for the notion sync, because it’s a bit not really hard, but tedious is that we have to, basically, you know, direct it to haste. Get this data from this page and the sub pages and get this from this page.
135 00:12:29.570 ⇒ 00:12:33.770 Uttam Kumaran: Yeah, but dude long story short. Yes, in the future. Right?
136 00:12:33.770 ⇒ 00:12:34.460 Miguel de Veyra: Yeah, yeah.
137 00:12:34.460 ⇒ 00:12:37.039 Connor Fenn: If it’s pulling, though from like
138 00:12:37.180 ⇒ 00:12:41.790 Connor Fenn: client pages already, though then it then it essentially is.
139 00:12:41.790 ⇒ 00:12:44.510 Uttam Kumaran: I just said, Yes, yeah. No. Long story short. Yes.
140 00:12:44.510 ⇒ 00:12:47.050 Connor Fenn: Oh, okay, got it.
141 00:12:47.900 ⇒ 00:12:51.509 Uttam Kumaran: It’s not. It’s just not happening right now. But I think, Miguel, my own.
142 00:12:51.910 ⇒ 00:12:54.726 Uttam Kumaran: My only question would be
143 00:12:55.650 ⇒ 00:13:14.490 Uttam Kumaran: like, how does it know, like, are we gonna go check to see whether a client exists basically to pull in additional information. I just put my, that was kind of related to my second question, I put in Zoom. Sometimes we already speak to the leads, or we have some context. Usually a lot of that context goes into notion in the lead page.
144 00:13:15.390 ⇒ 00:13:23.059 Uttam Kumaran: It would be great for the bot to understand that, like a lead already exists, and to use that when it basically produces its report.
145 00:13:26.110 ⇒ 00:13:30.679 Uttam Kumaran: But yeah, I feel like this is something that we should add to the backlog Casey.
146 00:13:32.840 ⇒ 00:13:37.479 Miguel de Veyra: Yeah, I mean the the leads table, the clients. They’re all tables right?
147 00:13:37.650 ⇒ 00:13:38.340 Uttam Kumaran: Yeah, yeah.
148 00:13:38.340 ⇒ 00:13:41.869 Miguel de Veyra: Databases. Basically. So it should be, yeah, yeah, okay, yeah.
149 00:13:42.410 ⇒ 00:13:46.709 Miguel de Veyra: do you want me to prioritize that over the the target industries? I think it’s more important.
150 00:13:47.570 ⇒ 00:13:51.839 Uttam Kumaran: I think there’s as as you get through everything. It’s on the list.
151 00:13:52.140 ⇒ 00:13:52.750 Connor Fenn: Utah man.
152 00:13:53.640 ⇒ 00:13:54.819 Connor Fenn: Question for you
153 00:13:55.560 ⇒ 00:14:08.420 Connor Fenn: the bigger picture I this is probably down the road. I I know we’re just starting with this, but the bigger picture to eventually get like this kind of out of slack, too. And like, maybe we have, like an actual work, dashboard.
154 00:14:10.310 ⇒ 00:14:15.400 Uttam Kumaran: I guess. Tell me what what you mean by work dashboard! Tell me what that like would look like for you.
155 00:14:15.590 ⇒ 00:14:17.490 Connor Fenn: Oh, I’m just like
156 00:14:18.150 ⇒ 00:14:26.019 Connor Fenn: have, like an internal like, sign up something kind of like the notion, or but more of like a web page kind of view
157 00:14:26.160 ⇒ 00:14:29.840 Connor Fenn: where it’s not so clunky, where I can like, you know.
158 00:14:30.050 ⇒ 00:14:35.230 Connor Fenn: not have to go into slack and scroll through chats and things like that just have, like a.
159 00:14:35.350 ⇒ 00:14:43.950 Connor Fenn: you know, clean, organized way of viewing everything and then clicking on the different folders of like, maybe clients or chats. Be cool
160 00:14:44.720 ⇒ 00:14:45.330 Connor Fenn: to add.
161 00:14:45.330 ⇒ 00:14:47.950 Miguel de Veyra: I can. I can cool something like that.
162 00:14:48.340 ⇒ 00:14:57.120 Uttam Kumaran: Yeah. But here, I guess, Miguel, I guess, for Pm. 1. 0. 1, you never build what the client asks. You gotta understand what they’re asking about first, st
163 00:14:57.280 ⇒ 00:15:02.630 Uttam Kumaran: right, we’re gonna if we go build something and nobody uses it. It’s kind of useless. So the question here is like
164 00:15:02.950 ⇒ 00:15:11.110 Uttam Kumaran: what I wanna know what slack isn’t doing today that you feel is like, approved, basically.
165 00:15:11.481 ⇒ 00:15:23.370 Connor Fenn: This is essentially like it would be cool to eventually have, like just a chat Gdp, kind of dashboard of this, but that you can like organize the different, you know, folders.
166 00:15:23.370 ⇒ 00:15:24.190 Uttam Kumaran: I see.
167 00:15:24.190 ⇒ 00:15:30.968 Connor Fenn: Like that. I was just curious. If that was, you know, a bigger picture thought, I’m not expecting that anytime soon. I was just thinking.
168 00:15:31.440 ⇒ 00:15:35.380 Uttam Kumaran: Yeah, our, I have like 2 sort of ideas here, one
169 00:15:35.480 ⇒ 00:15:41.009 Uttam Kumaran: for us in our company. Our work happens in notion work happens in slack work happens in zoom
170 00:15:41.300 ⇒ 00:15:51.220 Uttam Kumaran: so as much as the AI can be integrated where work happens is like my number one priority, right? So anywhere where you’re in a channel in slack.
171 00:15:51.400 ⇒ 00:16:11.290 Uttam Kumaran: you should be able to query notion. You should also be able to ask one or many AI AI agents to help you. The second thing we are working on, you know, and you kind of saw a little bit of it is sort of this sort of home for all of the AI agents, because some AI agents slack may not be the best place to interact.
172 00:16:11.400 ⇒ 00:16:17.129 Uttam Kumaran: And so we’re sort of working on this. But the use case, for this initially was really for Demos.
173 00:16:17.290 ⇒ 00:16:19.590 Uttam Kumaran: like, I don’t think we really designed this.
174 00:16:20.820 ⇒ 00:16:30.319 Connor Fenn: Well, yes, this is actually kind of what I was talking about. But I knew this was a demo. But like this, if we have like an Yeah, this is perfect, like an internal one is cool. That’s kind of.
175 00:16:31.380 ⇒ 00:16:36.320 Uttam Kumaran: I think I would just continue to like for me. And building this is
176 00:16:37.280 ⇒ 00:17:00.540 Uttam Kumaran: like this versus slack. Yeah, there definitely are some improvements. But for me the core thing is making sure that what you get out of the agent is right. Be before working on the form factor. So that’s that’s the thing that for me. I want everybody here to stress test the actual. What’s getting returned by the agent. Because, as you guys know, I mean, you know, we could whip together whatever Ui, and we can build the Chat
177 00:17:00.560 ⇒ 00:17:23.510 Uttam Kumaran: Gpt clone basically for us in terms that’s fine. But I want to press on like, are the agents actually giving us information that’s valuable, that that it doesn’t matter, the form factor necessarily right? And then the nice thing is the reason why I don’t want to hide like the problem with Chat Gpt now is, it’s individualized right? And the problem with
178 00:17:23.700 ⇒ 00:17:31.310 Uttam Kumaran: when we’re working in our company is that if other people can benefit from sort of the queries you’re writing, it’s really really hard.
179 00:17:32.590 ⇒ 00:17:50.979 Uttam Kumaran: right? So I’m almost like of the sense of like, I may even bring chat, gpt into slack, and basically just have a channel where, like, you can interact with chat, gpt directly there, because there’s a lot of stuff I do with Chat Gpt, that I’m sure would be really helpful for everybody to see sort of how I
180 00:17:51.120 ⇒ 00:17:54.589 Uttam Kumaran: prompt and do things that unfortunately, it’s just on my.
181 00:17:54.950 ⇒ 00:17:57.519 Uttam Kumaran: it’s just on my thing. So my prior like
182 00:17:57.650 ⇒ 00:18:02.299 Uttam Kumaran: to give you a sense for me. My point is one everybody accelerate learning
183 00:18:02.540 ⇒ 00:18:10.729 Uttam Kumaran: right? So the second thing is just making it easy to access. So that’s sort of how, I’m thinking. But I think your points are are also really valid.
184 00:18:12.440 ⇒ 00:18:21.500 Connor Fenn: One thing that I also was thinking about can do you mind going back to the output page where you can see like a response
185 00:18:28.419 ⇒ 00:18:30.099 Connor Fenn: for the research.
186 00:18:31.510 ⇒ 00:18:36.380 Connor Fenn: Is this like, if you put in a like a client, Linkedin, or something like that, or web page.
187 00:18:36.670 ⇒ 00:18:39.020 Connor Fenn: isn’t, it’s in slack, normally right?
188 00:18:40.980 ⇒ 00:18:41.620 Casie Aviles: This one!
189 00:18:41.830 ⇒ 00:18:52.270 Connor Fenn: Yeah, so we have. If you it go down a little, it does the like opportunity assessment
190 00:18:52.600 ⇒ 00:18:57.740 Connor Fenn: with relevant services. Would we be able to like. Add, in
191 00:18:57.990 ⇒ 00:19:07.990 Connor Fenn: a section of that where it maybe like goes and searches for recent news that maybe that company did. That might be.
192 00:19:08.140 ⇒ 00:19:11.499 Connor Fenn: you know, a talking point that we could use.
193 00:19:11.500 ⇒ 00:19:13.000 Uttam Kumaran: Hmm, yeah.
194 00:19:14.428 ⇒ 00:19:18.549 Casie Aviles: Brings in like a combo starter, too.
195 00:19:20.170 ⇒ 00:19:26.600 Uttam Kumaran: Yeah, we should look at just bringing in like recent headlines or recent news, and then also link them. Casey.
196 00:19:28.440 ⇒ 00:19:29.469 Casie Aviles: Yeah, sure, sure.
197 00:19:30.170 ⇒ 00:19:35.760 Casie Aviles: Actually, the the 1st version of this on relevance already. Did that. But yeah,
198 00:19:36.830 ⇒ 00:19:37.180 Uttam Kumaran: Cool.
199 00:19:37.430 ⇒ 00:19:41.459 Casie Aviles: Yeah, we could bring that over here with another tool.
200 00:19:44.160 ⇒ 00:20:10.010 Uttam Kumaran: So I think the biggest thing honestly, overall is, Connor. As you’re using this, if something in there is not helpful, or you are like, I wish you could do more. The feedback loop process is like out of everything we did here. I think the number one feature that’s actually like, gonna help us accelerate is the feedback like it’s not gonna be enough. Cause the AI team and me
201 00:20:10.220 ⇒ 00:20:14.540 Uttam Kumaran: like we’re not direct users of this every day. So.
202 00:20:15.020 ⇒ 00:20:25.950 Uttam Kumaran: and I want you to think big and think about like, actually your entire process. And like, we want this agent to do a lot of stuff. So this is the 1st
203 00:20:26.090 ⇒ 00:20:49.599 Uttam Kumaran: sort of like part of like sort of this sort of sales automation that we’re working on. But like, I want us to push this to the limit. And so think big and think about like when you’re using it like, Hey, I wish it could do this, or this would have been helpful. Given this client. Those are the stuff that’s gonna really help us improve. And again, the nice thing is, you can just either use the feedback to send that to us, or just tag
204 00:20:50.200 ⇒ 00:20:54.829 Uttam Kumaran: tag me or tag someone in slack, and we’ll take a look and prioritize
205 00:20:55.160 ⇒ 00:21:02.320 Connor Fenn: Just to clarify on that feedback loop. Is that just going to you guys? Or is also the AI like, take that into consideration.
206 00:21:02.650 ⇒ 00:21:03.540 Miguel de Veyra: Business.
207 00:21:04.120 ⇒ 00:21:08.080 Uttam Kumaran: I mean, the AI doesn’t. The AI is not building itself like at the moment. So it’s just
208 00:21:09.330 ⇒ 00:21:10.210 Uttam Kumaran: us. Yeah.
209 00:21:10.210 ⇒ 00:21:10.650 Connor Fenn: I don’t know.
210 00:21:10.650 ⇒ 00:21:15.389 Uttam Kumaran: But we are, we are like saving all that feedback. Or we’re gonna start saving it. And then.
211 00:21:15.730 ⇒ 00:21:18.670 Uttam Kumaran: yeah, we have plans for basically like how
212 00:21:19.020 ⇒ 00:21:22.749 Uttam Kumaran: we can use AI to basically learn from feedback over time.
213 00:21:23.740 ⇒ 00:21:27.390 Uttam Kumaran: But that will be that’s gonna take a little bit longer.
214 00:21:29.280 ⇒ 00:21:32.679 Connor Fenn: That’s what you define as like a layer. 3 AI, right.
215 00:21:33.710 ⇒ 00:21:39.290 Uttam Kumaran: Yeah, it’s just like it’s sort of like, not, I mean, it’s just not the most efficient like
216 00:21:39.620 ⇒ 00:21:51.550 Uttam Kumaran: it could. The one thing we want to do is basically the biggest things that we could do is one are the prompts, the best prompts. So over time, as we collect more feedback, we will analyze the feedback, and then basically use that to say, like.
217 00:21:51.840 ⇒ 00:21:56.039 Uttam Kumaran: what are, what do we know anecdotally versus? What are we seeing from the feedback?
218 00:21:56.570 ⇒ 00:22:19.400 Uttam Kumaran: Right? And we’ll and eventually we’ll have agents deployed within engineering within operations, within sales within marketing. And so this process of collecting feedback isn’t just for this agent. It’s gonna be anytime. We have an a checkpoint anytime. We have an interaction with the AI. I want to know whether that was positive or negative. I think even this one to 5 sort of thing is too heavy.
219 00:22:19.610 ⇒ 00:22:26.880 Uttam Kumaran: I may even do like thumbs up or thumbs. We may think about even easier ways to do it, but the feedback loop has to be
220 00:22:27.000 ⇒ 00:22:32.380 Uttam Kumaran: for us to build fast. The feedback loop has to be extremely fast.
221 00:22:32.835 ⇒ 00:22:41.679 Uttam Kumaran: So everything we do on the AI side. You know, I’ve been pushing the team to see how we can build in feedback and build in evaluations and things like that
222 00:22:45.520 ⇒ 00:22:59.805 Uttam Kumaran: cool. Okay? So I think, Casey, we have a couple more. I think we have a couple of things we can work on. I think. Maybe let’s just like centralize sort of notes in the AI Channel, and then we could sort of add that to the backlog. I know we have a couple of other things we’re doing. So we’ll
223 00:23:00.410 ⇒ 00:23:02.610 Uttam Kumaran: well, that’s the backlog, and we can get to it.
224 00:23:03.700 ⇒ 00:23:04.120 Casie Aviles: Okay.
225 00:23:04.120 ⇒ 00:23:08.010 Uttam Kumaran: Do you? Wanna do you want to start a thread and AI team and just sort of like, dump
226 00:23:08.220 ⇒ 00:23:12.109 Uttam Kumaran: any notes there and then we can add those tickets.
227 00:23:13.750 ⇒ 00:23:14.569 Casie Aviles: Okay. Yeah.
228 00:23:15.050 ⇒ 00:23:15.690 Uttam Kumaran: Okay.
229 00:23:16.540 ⇒ 00:23:18.260 Uttam Kumaran: Okay, awesome.
230 00:23:19.220 ⇒ 00:23:20.830 Uttam Kumaran: Thanks. S. Thanks. Casey. Appreciate it.
231 00:23:20.830 ⇒ 00:23:21.380 Miguel de Veyra: Thanks. Everyone.
232 00:23:21.380 ⇒ 00:23:21.820 Uttam Kumaran: Right.
233 00:23:22.440 ⇒ 00:23:23.060 Casie Aviles: I.
234 00:23:23.060 ⇒ 00:23:24.260 Nicolas Sucari: Thanks, guys. Bye-bye.
235 00:23:24.260 ⇒ 00:23:24.880 Uttam Kumaran: Bye.