Meeting Title: AI Service Office Hours Block Date: 2026-03-18 Meeting participants: Casie Aviles, Pranav Narahari
WEBVTT
1 00:02:33.770 ⇒ 00:02:34.790 Pranav Narahari: Hey, Casey.
2 00:02:36.980 ⇒ 00:02:38.000 Casie Aviles: Hey, Pranav.
3 00:02:39.670 ⇒ 00:02:51.860 Pranav Narahari: Yeah, so I just saw your, your message. Yeah, that sounds great. Where… so, okay, I see that there’s a spreadsheet link here. Okay, that’s awesome.
4 00:02:53.080 ⇒ 00:02:54.530 Pranav Narahari: Yeah, perfect.
5 00:02:55.510 ⇒ 00:02:57.849 Pranav Narahari: And so this is basically just, like, a…
6 00:02:58.110 ⇒ 00:03:04.330 Pranav Narahari: I just want to get a sense of, like, how long it’s taking to create these spreadsheets. It’s not a long time, right? It’s just, like, a download from Snowflake?
7 00:03:05.430 ⇒ 00:03:10.809 Casie Aviles: Yeah, I mean, last week I already worked on, like, creating the script.
8 00:03:11.450 ⇒ 00:03:16.099 Casie Aviles: That lets me basically do, like, a run and a comparison test.
9 00:03:17.300 ⇒ 00:03:23.300 Casie Aviles: So, what I’ve been doing is I would just ask Cursor to modify those whenever I need to.
10 00:03:23.770 ⇒ 00:03:26.700 Casie Aviles: do another set of tests, you know, so…
11 00:03:27.920 ⇒ 00:03:33.349 Pranav Narahari: Gotcha. What I’m wondering is if we can, like, leverage Rill here in any way?
12 00:03:34.970 ⇒ 00:03:41.500 Pranav Narahari: I don’t want it to be, like, a big lift, right? So if it ends up being a big lift, and it’s not something that we can…
13 00:03:42.270 ⇒ 00:03:52.860 Pranav Narahari: that we can leverage going forward for any of these, like, one-off, like, okay, we need to assess the difference in execution times, or accuracy, or whatever it may be.
14 00:03:54.130 ⇒ 00:04:03.059 Pranav Narahari: is there something, like, an architecture that comes to mind where, like, we could easily build that in real? Or is Google… or do you think… and you can be…
15 00:04:03.230 ⇒ 00:04:06.869 Pranav Narahari: straight up with me here, too, if, like, you think Google Sheet is probably the best way for this.
16 00:04:10.790 ⇒ 00:04:19.490 Casie Aviles: I’m… I guess… I’m thinking, like, it might be more of a lift to have… to have it set up with real… Yeah.
17 00:04:20.529 ⇒ 00:04:21.180 Pranav Narahari: However.
18 00:04:21.180 ⇒ 00:04:21.570 Casie Aviles: I’m thinking.
19 00:04:21.570 ⇒ 00:04:34.839 Pranav Narahari: about real is, like, it’s probably better for weekly type of things, like, when you’re doing, like, differences, because then that way, like, yeah, you want to be able to see, like, the latest data, or maybe you want to see weekly data, at the very least.
20 00:04:34.840 ⇒ 00:04:35.160 Casie Aviles: Yes.
21 00:04:35.960 ⇒ 00:04:55.920 Pranav Narahari: Yeah, okay. Yeah, that sounds good. Just one question that Utam was asking, like, why are we having, like, spreadsheets? But I think for this specific scenario, where we’re trying to just show them, like, one-off differences in how, the new Andy is working, it makes sense.
22 00:04:56.270 ⇒ 00:05:02.900 Casie Aviles: Yeah, exactly, like, it’s not going to be recurring, you know? We won’t be, like, constantly showing them
23 00:05:03.100 ⇒ 00:05:05.269 Casie Aviles: Or, like, yeah, I mean…
24 00:05:06.780 ⇒ 00:05:14.370 Casie Aviles: It’s just going to be, like, for, like, for example, for this particular week where we’re testing out some optimizations, and…
25 00:05:15.280 ⇒ 00:05:19.899 Casie Aviles: It’s not going to be the same case for, like, the following weeks anymore, so…
26 00:05:20.580 ⇒ 00:05:26.139 Casie Aviles: I think what we’re showing them is just, you know, like, the current Andy, you know, the current setup.
27 00:05:26.310 ⇒ 00:05:27.370 Casie Aviles: Which…
28 00:05:27.700 ⇒ 00:05:31.140 Pranav Narahari: Yeah, so one question that I just have here, and maybe…
29 00:05:31.140 ⇒ 00:05:31.490 Casie Aviles: actually.
30 00:05:31.490 ⇒ 00:05:38.209 Pranav Narahari: Do you want to just, share the spreadsheet? Or maybe… actually, let me just do that, I can do that, since I have it here.
31 00:05:38.540 ⇒ 00:05:43.039 Casie Aviles: Okay, cool, yeah, I’ll… I can answer, like, if you have any questions about…
32 00:05:43.590 ⇒ 00:05:46.050 Pranav Narahari: Yeah, cause I haven’t been able to…
33 00:05:46.180 ⇒ 00:05:48.619 Pranav Narahari: dive into it yet, but I think this is a good time.
34 00:05:58.530 ⇒ 00:06:04.130 Pranav Narahari: Yeah, let me… Do you have it up on your end, too, or should I share my screen?
35 00:06:04.560 ⇒ 00:06:05.870 Casie Aviles: Yeah, I also have it up.
36 00:06:06.270 ⇒ 00:06:10.690 Pranav Narahari: Okay, perfect. Alright, so… let me just look around here…
37 00:06:12.640 ⇒ 00:06:19.550 Pranav Narahari: Okay, so, yeah, this, 4 to 6 seconds is the end-to-end execution time, right?
38 00:06:21.040 ⇒ 00:06:22.670 Casie Aviles: Or just…
39 00:06:22.790 ⇒ 00:06:26.460 Pranav Narahari: Is this just for… .
40 00:06:28.480 ⇒ 00:06:30.810 Casie Aviles: Are you looking at… .
41 00:06:30.810 ⇒ 00:06:33.070 Pranav Narahari: I’m looking at the one you’re highlighting right now, yeah.
42 00:06:34.020 ⇒ 00:06:38.349 Casie Aviles: Yeah, that’s end-to-end, because that’s, that’s the call.
43 00:06:38.950 ⇒ 00:06:41.369 Casie Aviles: That… so it’s the… it’s the…
44 00:06:42.010 ⇒ 00:06:46.159 Casie Aviles: time that it took to get to… so from sending the…
45 00:06:46.440 ⇒ 00:06:53.160 Casie Aviles: message to the endpoint, and then getting that, right? So that’s… that’s… that should be end-to-end.
46 00:06:54.160 ⇒ 00:06:58.640 Pranav Narahari: Gotcha. And so, why didn’t we test on the full 811 records?
47 00:06:59.300 ⇒ 00:07:06.130 Casie Aviles: Oh, this is live right now, so… It only logs, like.
48 00:07:06.460 ⇒ 00:07:10.530 Casie Aviles: what is currently being asked by the CSRs.
49 00:07:12.800 ⇒ 00:07:13.170 Pranav Narahari: Oh.
50 00:07:13.170 ⇒ 00:07:14.170 Casie Aviles: Yeah.
51 00:07:14.400 ⇒ 00:07:21.389 Casie Aviles: Yeah, so… I… And I also get, like, what you’re trying to say, so…
52 00:07:21.630 ⇒ 00:07:27.479 Casie Aviles: What I did there, though, was not, like, the full 800 records, I did…
53 00:07:27.640 ⇒ 00:07:33.689 Casie Aviles: Since I was just focusing on optimizing the… the database querying step.
54 00:07:34.160 ⇒ 00:07:39.190 Casie Aviles: I took the… so if you recall the report from last week…
55 00:07:39.600 ⇒ 00:07:46.419 Casie Aviles: Let me look for that sheet. I think it’s… And hold on.
56 00:07:46.530 ⇒ 00:07:47.570 Casie Aviles: resort…
57 00:07:53.040 ⇒ 00:07:53.750 Casie Aviles: No.
58 00:07:54.860 ⇒ 00:07:56.459 Casie Aviles: Sure, it is known.
59 00:07:57.110 ⇒ 00:07:59.760 Casie Aviles: Oh, there you go, the resolved one.
60 00:08:01.110 ⇒ 00:08:07.429 Casie Aviles: I’ll send, yeah, I’ll send the sheet link as well, here.
61 00:08:08.240 ⇒ 00:08:09.289 Casie Aviles: on Zoom.
62 00:08:13.150 ⇒ 00:08:21.640 Casie Aviles: this, basically I ran this script that will… Go through, like… All the zip…
63 00:08:22.040 ⇒ 00:08:22.940 Pranav Narahari: Which, page?
64 00:08:22.940 ⇒ 00:08:23.480 Casie Aviles: wood.
65 00:08:23.750 ⇒ 00:08:27.060 Casie Aviles: It’s called Result Dumps Down Feedback.
66 00:08:27.860 ⇒ 00:08:30.780 Pranav Narahari: Resolved, thumbs down, feedback.
67 00:08:37.890 ⇒ 00:08:39.140 Pranav Narahari: Yep, okay.
68 00:08:39.270 ⇒ 00:08:42.029 Pranav Narahari: Resolve, thumbs down, feedback, and then at the bottom.
69 00:08:43.700 ⇒ 00:08:52.830 Casie Aviles: Yes, so where I’m highlighting it, if you can see that one, I added a new… yeah, this was from last week.
70 00:08:53.210 ⇒ 00:08:58.590 Casie Aviles: Oh, so basically, after I made optimizations to the DB querying step.
71 00:08:59.040 ⇒ 00:09:03.859 Casie Aviles: Yeah. That should… that should be under the… this column, K, and… yeah.
72 00:09:04.270 ⇒ 00:09:08.779 Casie Aviles: This should be… it’s the column K, master accept time, new.
73 00:09:09.200 ⇒ 00:09:15.799 Casie Aviles: So that’s the optimized one for the zip code-related questions. So all of these questions
74 00:09:15.960 ⇒ 00:09:24.350 Casie Aviles: Only had to do with… The database querying. So now it should be…
75 00:09:24.900 ⇒ 00:09:30.720 Casie Aviles: there should be some noticeable improvements as well. So last time, when I didn’t optimize for the DB,
76 00:09:31.170 ⇒ 00:09:33.470 Casie Aviles: It took around 8.9.
77 00:09:33.570 ⇒ 00:09:37.210 Casie Aviles: Here in the column I, so this is the average.
78 00:09:37.580 ⇒ 00:09:39.100 Casie Aviles: Here at the bottom.
79 00:09:39.690 ⇒ 00:09:43.919 Casie Aviles: So now, at least for this test, it shows up as 3.6.
80 00:09:44.520 ⇒ 00:09:45.570 Casie Aviles: Okay.
81 00:09:46.370 ⇒ 00:09:48.460 Pranav Narahari: That’s, I mean, that’s a noticeable improvement, like…
82 00:09:48.690 ⇒ 00:09:52.019 Pranav Narahari: So, that’s… that’s great. And so…
83 00:09:52.240 ⇒ 00:10:06.800 Pranav Narahari: the master exec time… this sounds like it’s end-to-end, right? Like, we’re not just assessing, like, the DB routing part of things. DB routing is maybe one thing that we’re changing, but we’re still assessing end-to-end, right?
84 00:10:07.170 ⇒ 00:10:09.520 Casie Aviles: Yes, yes, this is from… this… yeah.
85 00:10:10.190 ⇒ 00:10:13.909 Pranav Narahari: That’s great. How are we just, like, choosing? Because I know there’s probably been, like.
86 00:10:14.020 ⇒ 00:10:18.519 Pranav Narahari: thousands and thousands of, like, prompts being asked, like, how did we choose these 143?
87 00:10:20.410 ⇒ 00:10:26.859 Casie Aviles: So basically, I just use, like, a simple filtering, or, like, a regex filtering.
88 00:10:27.380 ⇒ 00:10:32.869 Casie Aviles: Basically, it has to look for…
89 00:10:33.380 ⇒ 00:10:37.780 Casie Aviles: Questions where there are technicians involved or inspectors.
90 00:10:38.060 ⇒ 00:10:45.219 Casie Aviles: And then… If there are, like, zip codes that are included in the query, basically, so that’s…
91 00:10:45.590 ⇒ 00:10:48.009 Casie Aviles: That’s how I filtered these out.
92 00:10:48.440 ⇒ 00:10:53.230 Casie Aviles: Yeah, that’s how it works right now.
93 00:10:54.580 ⇒ 00:10:55.380 Pranav Narahari: Gotcha.
94 00:10:58.800 ⇒ 00:11:02.779 Casie Aviles: And also, I… since these are, like, the… the…
95 00:11:03.190 ⇒ 00:11:09.980 Casie Aviles: the ones that received, like, a thumbs-down feedback, so I also used that to filter from, filter out, like.
96 00:11:10.380 ⇒ 00:11:12.130 Casie Aviles: other records.
97 00:11:12.760 ⇒ 00:11:18.440 Casie Aviles: And also, this is from December to March.
98 00:11:19.180 ⇒ 00:11:23.550 Pranav Narahari: December. Okay, so December to March, there’s only 143.
99 00:11:24.810 ⇒ 00:11:31.600 Casie Aviles: Well, it’s, I think it’s just… it’s because of, you know, it’s… it’s the thumbs-down feedback, and…
100 00:11:31.900 ⇒ 00:11:33.979 Casie Aviles: Oh, right. Is this zip code related.
101 00:11:34.670 ⇒ 00:11:37.620 Pranav Narahari: Totally. That makes… oh, so is it thumbs down?
102 00:11:37.940 ⇒ 00:11:40.470 Pranav Narahari: Or zip code, or thumbs down and zip code?
103 00:11:40.680 ⇒ 00:11:42.499 Casie Aviles: Thumbs down and zip code, yeah.
104 00:11:42.630 ⇒ 00:11:44.569 Pranav Narahari: I see. Okay.
105 00:11:44.850 ⇒ 00:11:53.530 Pranav Narahari: Alright, well, yeah, I would also just like to see more generalized data, which I think is that, the live NADEN.
106 00:11:53.960 ⇒ 00:11:56.510 Pranav Narahari: output comparison, right? So yeah, if we go back there.
107 00:11:57.210 ⇒ 00:12:04.139 Pranav Narahari: Could we rerun the script pretty easily tomorrow, just to see? Because there’ll be more than 44 records, hopefully…
108 00:12:04.710 ⇒ 00:12:12.490 Pranav Narahari: Yeah, so, I mean, what I’d actually really like to do is,
109 00:12:15.510 ⇒ 00:12:27.009 Pranav Narahari: I would like to have a report where we can have the data be more convincing, where it’s not just 44 records, it’s maybe, like, close to 1,000 records.
110 00:12:27.560 ⇒ 00:12:40.469 Pranav Narahari: And so, yeah, running basically a one-time script. We have all of the historical data, so that shouldn’t be an issue for just getting to see how it functioned with N8N.
111 00:12:40.470 ⇒ 00:12:41.210 Casie Aviles: Yes.
112 00:12:41.770 ⇒ 00:12:51.220 Pranav Narahari: But what I would like to see is just, like, okay, with the new Andy, how is it functioning, just from an execution time perspective?
113 00:12:52.080 ⇒ 00:12:57.709 Pranav Narahari: On… yeah, just, like, in terms of That we’re seeing. Yeah, so…
114 00:12:57.710 ⇒ 00:12:58.270 Casie Aviles: Okay.
115 00:12:59.550 ⇒ 00:13:04.260 Pranav Narahari: what is the lift to do that? If, say, if, like, we want that on Monday of next week.
116 00:13:05.260 ⇒ 00:13:08.030 Casie Aviles: Oh, that report? I think I can do that.
117 00:13:08.400 ⇒ 00:13:14.000 Casie Aviles: Yeah, maybe tomorrow we could have it already, by tomorrow.
118 00:13:14.360 ⇒ 00:13:22.470 Pranav Narahari: Yeah, that would be great. If we could get it tomorrow, that’s even better. So yeah, if you can get, like, at least a thousand records,
119 00:13:23.180 ⇒ 00:13:25.510 Pranav Narahari: That would, that would be… that would be great.
120 00:13:25.510 ⇒ 00:13:31.120 Casie Aviles: And we’re not… we’re not gonna filter on anything, basically, right? And we just wanna get, like, around 1,000.
121 00:13:32.390 ⇒ 00:13:35.039 Pranav Narahari: Yes, yeah, I don’t think we should filter on anything.
122 00:13:35.300 ⇒ 00:13:42.310 Pranav Narahari: And then… but also, we should have a direct comparison to the specific questions in N8N.
123 00:13:42.970 ⇒ 00:13:46.090 Pranav Narahari: So, just sort of comparing apples to apples, like…
124 00:13:46.240 ⇒ 00:13:49.519 Pranav Narahari: The execution time difference should be seen for…
125 00:13:49.790 ⇒ 00:13:52.949 Pranav Narahari: the same question on NNN and Andy.
126 00:13:54.350 ⇒ 00:13:55.040 Casie Aviles: Okay.
127 00:13:55.350 ⇒ 00:13:58.190 Casie Aviles: Okay, yeah, I think that’s doable, so I’ll…
128 00:13:58.490 ⇒ 00:14:02.460 Casie Aviles: Run another script in, right after our call.
129 00:14:03.130 ⇒ 00:14:04.530 Pranav Narahari: Cool.
130 00:14:04.730 ⇒ 00:14:09.520 Pranav Narahari: Let’s just take a look at Linear 2, because I think this was the main thing that you were working on, right?
131 00:14:09.910 ⇒ 00:14:10.850 Casie Aviles: Yeah, yeah.
132 00:14:11.260 ⇒ 00:14:13.739 Pranav Narahari: Okay, so I added,
133 00:14:14.160 ⇒ 00:14:20.070 Pranav Narahari: And then you did mention a couple more things, so yeah, thank you for already reaching out to the client.
134 00:14:22.620 ⇒ 00:14:27.089 Pranav Narahari: And when you do that, do you just do that via the Google Chat or email?
135 00:14:27.390 ⇒ 00:14:36.470 Casie Aviles: Yeah, I just… I typically just, you know, reach out to them via Google Chat. Mostly it’s, yeah, the CSRs and Jenny’s.
136 00:14:37.820 ⇒ 00:14:38.400 Pranav Narahari: Cool.
137 00:14:38.840 ⇒ 00:14:44.339 Pranav Narahari: what I kind of want to do is just, like, let’s just create a group chat in Google Chat.
138 00:14:44.550 ⇒ 00:14:45.180 Casie Aviles: Sure.
139 00:14:45.180 ⇒ 00:14:51.389 Pranav Narahari: And then going forward, let’s just, like, pipe the questions through there, okay? Just so I can just, like, have a little bit more visibility.
140 00:14:52.490 ⇒ 00:14:54.780 Casie Aviles: Oh, okay, let’s see how…
141 00:14:54.780 ⇒ 00:15:02.050 Pranav Narahari: Yeah, so, like, also, maybe one thing that I’m doing differently now is I’m not using that, that anteater
142 00:15:02.220 ⇒ 00:15:09.280 Pranav Narahari: email for the group chat. I’m just using our Brainforge one, and then having a chat with their external, like, anteater…
143 00:15:09.460 ⇒ 00:15:11.810 Pranav Narahari: emails. Does that make sense?
144 00:15:12.500 ⇒ 00:15:18.190 Casie Aviles: I see, yeah. Since what I do is when I ask them a message, like, I just
145 00:15:18.640 ⇒ 00:15:21.079 Casie Aviles: Add my name, so they know who’s…
146 00:15:21.390 ⇒ 00:15:24.820 Casie Aviles: Sending them a message, so… but yeah, I get it, it makes sense to.
147 00:15:24.820 ⇒ 00:15:25.510 Pranav Narahari: Yeah.
148 00:15:25.920 ⇒ 00:15:27.180 Casie Aviles: use.
149 00:15:27.670 ⇒ 00:15:29.809 Pranav Narahari: Better if we do it that way, that way, like…
150 00:15:30.250 ⇒ 00:15:37.809 Pranav Narahari: we don’t need a sign… like, it’s just gonna be more difficult for all of us to just be monitoring this. It’s mostly on me to monitor this, right? .
151 00:15:38.720 ⇒ 00:15:41.690 Pranav Narahari: And so… Yeah, like…
152 00:15:42.570 ⇒ 00:15:50.469 Pranav Narahari: I’ll… I’ll set up that… that, like, honestly, let me just do that right now. I’ll just be like, hey, let’s just continue all comms in this conversation.
153 00:15:52.950 ⇒ 00:16:00.790 Casie Aviles: or, and they also have, like, department group chats, so I think.
154 00:16:00.790 ⇒ 00:16:06.550 Pranav Narahari: Yeah, those ones we’ll leave there, those aren’t gonna be, like, talked about as much, I think.
155 00:16:06.550 ⇒ 00:16:10.020 Casie Aviles: But it’s mainly just Yvette and Janice then, and then us.
156 00:16:10.420 ⇒ 00:16:16.679 Pranav Narahari: Yeah, so actually, if we go in here, and then, you see that one with Yvette and Janiece? .
157 00:16:17.210 ⇒ 00:16:17.850 Casie Aviles: Yes.
158 00:16:18.400 ⇒ 00:16:20.079 Pranav Narahari: Can you just add people here?
159 00:16:20.720 ⇒ 00:16:21.860 Casie Aviles: Yeah, I think…
160 00:16:21.860 ⇒ 00:16:27.130 Pranav Narahari: Yeah, do you want to add just me, you, Mustafa, Sam?
161 00:16:28.020 ⇒ 00:16:28.720 Pranav Narahari: Yeah, I think.
162 00:16:28.720 ⇒ 00:16:29.930 Casie Aviles: I think this is what I was seeing.
163 00:16:29.930 ⇒ 00:16:33.990 Pranav Narahari: Oh, yeah. So I think we need to create it from Brainforge.
164 00:16:35.190 ⇒ 00:16:39.050 Casie Aviles: Oh… So… Okay.
165 00:16:41.600 ⇒ 00:16:44.379 Casie Aviles: So it’s going to be within our workspace, then.
166 00:16:45.490 ⇒ 00:16:46.809 Pranav Narahari: Yeah, which is fine.
167 00:16:50.090 ⇒ 00:16:50.840 Casie Aviles: Okay.
168 00:16:55.190 ⇒ 00:16:57.839 Pranav Narahari: I don’t see any downside to that.
169 00:16:59.880 ⇒ 00:17:04.829 Pranav Narahari: Okay, so I’ll… I have to figure out all their emails and stuff, so I’ll do it right after this. I have to jump…
170 00:17:05.170 ⇒ 00:17:05.510 Casie Aviles: Okay, yeah.
171 00:17:06.990 ⇒ 00:17:12.870 Pranav Narahari: But, yeah, it seems like you already created a ticket for, like, what we need to do next. What is the timeline on that?
172 00:17:14.309 ⇒ 00:17:21.769 Casie Aviles: I’ll try to have it by end of week, but it shouldn’t take too long, since it’s just…
173 00:17:22.179 ⇒ 00:17:27.309 Casie Aviles: moving, you know, the existing admin UI to their Google Cloud Run.
174 00:17:28.560 ⇒ 00:17:43.599 Pranav Narahari: Okay, cool. Yeah, that sounds good. I’m just gonna make another ticket for you to create a new script or update your current script to get those, like, thousand-plus record, comparison for execution time, and then… yeah, that should be good enough for the end of this week.
175 00:17:44.210 ⇒ 00:17:45.909 Casie Aviles: Okay. Okay, sounds good.
176 00:17:46.700 ⇒ 00:17:48.069 Pranav Narahari: Thanks, Casey. We’ll talk soon.
177 00:17:48.070 ⇒ 00:17:50.450 Casie Aviles: Alright, thank you. Bye-bye.