Meeting Title: ABC Project Weekly Sync Date: 2026-04-01 Meeting participants: Samuel Roberts, Pranav, Casie Aviles
WEBVTT
1 00:00:48.050 ⇒ 00:00:48.970 Pranav: Hey, Sam.
2 00:00:53.490 ⇒ 00:00:54.360 Samuel Roberts: Hey.
3 00:00:57.610 ⇒ 00:00:58.680 Samuel Roberts: How’s it going?
4 00:01:00.940 ⇒ 00:01:02.790 Pranav: Things are good, things are good.
5 00:01:04.230 ⇒ 00:01:07.700 Pranav: I was just working on, like, a bunch of Eden stuff this morning.
6 00:01:07.700 ⇒ 00:01:10.610 Samuel Roberts: Yeah, I saw a couple of the tickets get resolved and stuff, so…
7 00:01:10.790 ⇒ 00:01:11.500 Pranav: Yeah.
8 00:01:12.030 ⇒ 00:01:16.320 Samuel Roberts: Cool. Yeah, I figured it wasn’t worth… there was a few tickets that I’m like, I think this is…
9 00:01:16.480 ⇒ 00:01:23.650 Samuel Roberts: pretty simple and basic, but I wanted to make sure it was in there to block other things and stuff, so… I’m glad you were able to knock this out.
10 00:01:23.950 ⇒ 00:01:29.590 Pranav: Yeah, yeah. I think a lot of them already have been worked on, I just haven’t… I’ve just been doing it kind of in my head.
11 00:01:29.970 ⇒ 00:01:31.059 Samuel Roberts: No, totally, totally.
12 00:01:36.890 ⇒ 00:01:37.510 Pranav: Hey, JZ.
13 00:01:37.510 ⇒ 00:01:38.740 Samuel Roberts: One sec, yeah.
14 00:01:39.870 ⇒ 00:01:40.750 Casie Aviles: Hey, guys.
15 00:01:44.750 ⇒ 00:01:57.340 Pranav: Yeah, so I just wanted to go over a few things. So with linear right now, I just want to close out everything migration-related, so that might require us to, like, maybe spin down a few things,
16 00:01:58.710 ⇒ 00:02:02.239 Pranav: I… Yeah, I think since it’s just…
17 00:02:03.210 ⇒ 00:02:15.040 Pranav: nothing’s going to break at this point, right? Like, if there’s any small patches we need to make, I think we’re still feeling pretty good about leaving it on the mash wrap and not having to go back to the N8ana one.
18 00:02:15.100 ⇒ 00:02:24.429 Pranav: So, I think it’s… we’re in a pretty good spot to just close out any of those final tickets that were, like, end-to-end deprecation, Azure deprecation.
19 00:02:25.710 ⇒ 00:02:30.479 Pranav: And so, yeah, how about we just start there, and then we can… we can hop into other things.
20 00:02:30.960 ⇒ 00:02:33.139 Pranav: So I’ll, I’ll share my screen real quick.
21 00:02:34.080 ⇒ 00:02:34.660 Samuel Roberts: Nice.
22 00:02:47.410 ⇒ 00:02:49.209 Samuel Roberts: -Oh. Oh, no.
23 00:02:49.480 ⇒ 00:02:50.689 Samuel Roberts: You guys hear me still?
24 00:02:51.160 ⇒ 00:02:52.290 Pranav: Yeah, we can hear you.
25 00:02:52.440 ⇒ 00:02:54.110 Samuel Roberts: Alright, my whole screen just froze.
26 00:02:54.600 ⇒ 00:02:56.520 Samuel Roberts: There we go. Okay, got nervous.
27 00:02:56.720 ⇒ 00:02:57.590 Pranav: Perfect.
28 00:02:58.830 ⇒ 00:03:00.200 Pranav: Alright…
29 00:03:00.960 ⇒ 00:03:05.930 Samuel Roberts: Oh, yeah, I forgot, I… so these were all… all these CDC and the, that was, like, how…
30 00:03:06.130 ⇒ 00:03:08.830 Samuel Roberts: Cursor had added initial names to tickets.
31 00:03:09.290 ⇒ 00:03:09.650 Pranav: I just…
32 00:03:09.650 ⇒ 00:03:15.860 Samuel Roberts: left it because they kind of reference each other sometimes, but I’m gonna next time make sure it doesn’t create those, but I didn’t want.
33 00:03:15.860 ⇒ 00:03:17.209 Pranav: Yeah, all good.
34 00:03:17.210 ⇒ 00:03:21.910 Samuel Roberts: everything. I mean, the dependency should still be there, but, yeah.
35 00:03:23.590 ⇒ 00:03:24.330 Samuel Roberts: So…
36 00:03:24.680 ⇒ 00:03:28.460 Pranav: I also don’t think we need this, fallback plan anymore.
37 00:03:28.460 ⇒ 00:03:33.900 Samuel Roberts: Well, I mean, the only thing was just to switch one link. That’s why I figured it was pretty simple, so I just added that.
38 00:03:33.900 ⇒ 00:03:34.500 Pranav: Okay, perfect.
39 00:03:34.500 ⇒ 00:03:40.450 Samuel Roberts: comment to the ticket, so, like, I didn’t mark it as anything, I just figured we had it there, but it’s, yeah, that’s fine. It was literally…
40 00:03:40.780 ⇒ 00:03:44.460 Samuel Roberts: flip back to the previous one. That being said, once we…
41 00:03:44.660 ⇒ 00:03:47.020 Samuel Roberts: Spin things down, that is no longer applicable.
42 00:03:47.900 ⇒ 00:03:48.410 Pranav: Yeah.
43 00:03:48.410 ⇒ 00:03:48.990 Samuel Roberts: So…
44 00:03:49.450 ⇒ 00:03:58.589 Pranav: Okay, that’s totally fine. Anything we can just do in this call right now to just, fully deprecate these things? Superbase, NIDN…
45 00:04:01.640 ⇒ 00:04:06.589 Samuel Roberts: Yeah, probably, I mean… go ahead, Casey. Sorry.
46 00:04:06.590 ⇒ 00:04:12.600 Casie Aviles: Yeah, we can probably turn off the N8N… Live already since.
47 00:04:12.960 ⇒ 00:04:16.109 Casie Aviles: Nothing’s going… no, there are no executions there anymore.
48 00:04:17.269 ⇒ 00:04:22.059 Samuel Roberts: I would definitely confirm That, make sure there’s not something…
49 00:04:22.610 ⇒ 00:04:24.019 Casie Aviles: I don’t think there could be, but…
50 00:04:24.300 ⇒ 00:04:26.899 Samuel Roberts: Just double-check the executions and make sure they’re days ago.
51 00:04:28.210 ⇒ 00:04:28.910 Pranav: Yeah.
52 00:04:29.330 ⇒ 00:04:37.819 Pranav: And we can do that now, or if you guys want to do that later, I think if we can just, like, close out these two things end of day today, that’d be great.
53 00:04:38.090 ⇒ 00:04:45.970 Samuel Roberts: Yeah, the Supabase is the same thing, you could probably just… probably even just turn it off. Casey, is there anything in there that…
54 00:04:46.170 ⇒ 00:04:49.030 Samuel Roberts: We might need in the future that hasn’t been moved.
55 00:04:49.230 ⇒ 00:04:53.309 Samuel Roberts: I feel like the ZipsDB and the embeddings were the only things I was thinking about.
56 00:04:54.450 ⇒ 00:04:57.700 Casie Aviles: Yeah, at least for ABC-related.
57 00:04:57.700 ⇒ 00:04:58.250 Samuel Roberts: Yeah.
58 00:04:58.890 ⇒ 00:05:03.600 Samuel Roberts: Yeah, yeah, this is a specific… that one is its own project on Sipabase, so…
59 00:05:04.560 ⇒ 00:05:10.320 Casie Aviles: Yeah, I think those are, like, the major ones that we moved over, so I don’t think we have any…
60 00:05:11.580 ⇒ 00:05:12.450 Casie Aviles: Okay.
61 00:05:12.590 ⇒ 00:05:13.400 Casie Aviles: Yeah.
62 00:05:13.690 ⇒ 00:05:15.880 Casie Aviles: I don’t think anything will break if we…
63 00:05:16.210 ⇒ 00:05:17.629 Samuel Roberts: Okay, yeah, I think if…
64 00:05:17.860 ⇒ 00:05:22.919 Samuel Roberts: if everything’s working as is, and we just… I can probably just disable it before I delete it, even, so…
65 00:05:27.910 ⇒ 00:05:31.210 Pranav: Feels like everything in review is… Pretty good.
66 00:05:31.210 ⇒ 00:05:36.880 Samuel Roberts: Yeah, Casey was telling me this morning, I figured we’d just talk it through and make sure that things are pretty good before we close them out, you and I, so…
67 00:05:36.880 ⇒ 00:05:38.650 Pranav: Yeah, sounds good. And this one…
68 00:05:38.750 ⇒ 00:05:41.570 Pranav: Enable Master Studio and GCP deployment.
69 00:05:42.240 ⇒ 00:05:42.970 Pranav: Is this something.
70 00:05:42.970 ⇒ 00:05:43.870 Samuel Roberts: That’s all set, too.
71 00:05:44.290 ⇒ 00:05:44.980 Pranav: proposals.
72 00:05:45.330 ⇒ 00:05:50.440 Samuel Roberts: Yeah, he did that already. I think that’s why it was in review. Yeah, that’s the link. I was gonna put together a little…
73 00:05:50.540 ⇒ 00:05:52.730 Samuel Roberts: Pin it to the channel, because it’s been useful.
74 00:05:53.290 ⇒ 00:05:54.550 Pranav: Perfect. That’s awesome.
75 00:05:54.550 ⇒ 00:05:57.340 Samuel Roberts: And it’s not a pretty URL, and I want to make sure we…
76 00:05:57.990 ⇒ 00:06:06.920 Pranav: Yeah. Okay, so, Casey, I think you gave us an update on this on Monday? The benchmark.
77 00:06:06.920 ⇒ 00:06:07.370 Casie Aviles: I’m Gemini.
78 00:06:07.470 ⇒ 00:06:08.500 Pranav: Models.
79 00:06:09.840 ⇒ 00:06:14.319 Casie Aviles: Hmm, yeah, these were… I worked on… I worked with Sam.
80 00:06:14.540 ⇒ 00:06:15.930 Casie Aviles: Probably diesel.
81 00:06:17.250 ⇒ 00:06:24.590 Casie Aviles: what I found… basically is that the Gemini 2.5 flash light was…
82 00:06:25.680 ⇒ 00:06:30.479 Casie Aviles: Doing… what’s the best, like, among the… at least the ones that we tested.
83 00:06:30.620 ⇒ 00:06:36.800 Casie Aviles: So if we were to… choose, like, a fallback. Like, in terms of latency for now, so…
84 00:06:37.250 ⇒ 00:06:40.920 Casie Aviles: I think that’s… that’s a good… Fallback option.
85 00:06:41.200 ⇒ 00:06:44.260 Casie Aviles: I tried mo- Go ahead.
86 00:06:44.260 ⇒ 00:06:48.730 Pranav: It’s fallback work, though, because it’s not failing, right?
87 00:06:48.940 ⇒ 00:06:56.449 Samuel Roberts: Sometimes it was failing, that’s why I was worried about it, because there were 503 errors, and there were just slowdowns.
88 00:06:57.040 ⇒ 00:07:01.059 Pranav: Okay, so this doesn’t really affect slowdowns, I guess, this fallback, right?
89 00:07:01.990 ⇒ 00:07:06.180 Samuel Roberts: Unless we want to… Do some kind of timeout check.
90 00:07:06.440 ⇒ 00:07:10.069 Samuel Roberts: But then that’s gonna make it take longer anyway, I feel like, in the long run?
91 00:07:10.070 ⇒ 00:07:10.470 Pranav: Yeah.
92 00:07:10.850 ⇒ 00:07:17.440 Samuel Roberts: So I don’t think that’s really a benefit there. It was more just like, if there is a 503, can we redirect it so they don’t get no output? Yeah.
93 00:07:17.580 ⇒ 00:07:18.480 Pranav: I like that.
94 00:07:18.480 ⇒ 00:07:25.550 Samuel Roberts: Okay. It’s… it’s… we didn’t necessarily confirm yet, Casey, correct me if I’m wrong, how the outputs are from that older model, though?
95 00:07:27.340 ⇒ 00:07:28.200 Samuel Roberts: You just confirmed this.
96 00:07:28.200 ⇒ 00:07:28.650 Casie Aviles: I’m sorry.
97 00:07:28.650 ⇒ 00:07:32.660 Samuel Roberts: Right. We just confirmed the speed of the 2.5.
98 00:07:32.920 ⇒ 00:07:34.100 Casie Aviles: Yes, yes, just to.
99 00:07:34.100 ⇒ 00:07:44.749 Samuel Roberts: Not that… yeah. So there would be some more testing to actually set it up as a fallback, but now that things… the other question there is if these 503s are just Googled infra being…
100 00:07:45.810 ⇒ 00:07:46.710 Samuel Roberts: Bad.
101 00:07:47.140 ⇒ 00:07:49.619 Samuel Roberts: Will that affect the 2.5 as well?
102 00:07:51.860 ⇒ 00:07:52.710 Samuel Roberts: So…
103 00:07:53.410 ⇒ 00:07:56.779 Samuel Roberts: If that’s the only one that’s good, I’m not sure we really need to worry about it right now.
104 00:07:57.240 ⇒ 00:08:02.489 Samuel Roberts: I was kind of hoping that, like, another one would be, like, a… non-Gemini one would be good.
105 00:08:03.940 ⇒ 00:08:08.149 Samuel Roberts: But since it’s not, I’m not super worried about it right now, especially with the spikes, kind of.
106 00:08:09.210 ⇒ 00:08:13.190 Samuel Roberts: Leveling out a little bit. And the downtime, too, so…
107 00:08:14.970 ⇒ 00:08:17.909 Samuel Roberts: I would say, like, let’s keep that in our back pocket more than anything.
108 00:08:19.500 ⇒ 00:08:20.140 Pranav: Okay.
109 00:08:22.760 ⇒ 00:08:28.449 Casie Aviles: Yeah, that heat map that we have now should help us, like, see…
110 00:08:30.190 ⇒ 00:08:32.539 Casie Aviles: We’re getting… you’re still getting spikes.
111 00:08:33.610 ⇒ 00:08:38.839 Pranav: Okay, perfect, yeah, I actually did want to take a look at all those, dashboards that you made. Yeah.
112 00:08:38.840 ⇒ 00:08:39.550 Samuel Roberts: They’re pretty cool.
113 00:08:39.559 ⇒ 00:08:42.579 Pranav: So these, these 3 here I can just put into done?
114 00:08:45.910 ⇒ 00:08:52.580 Samuel Roberts: Yeah, I think if we’re satisfied with all that, there’s no real follow-up for…
115 00:08:53.870 ⇒ 00:08:57.290 Samuel Roberts: Which one? Hold on, sorry. Validate peak time, spikes…
116 00:08:58.400 ⇒ 00:09:03.900 Samuel Roberts: That was just the… why… that was the real stuff, basically, right?
117 00:09:04.980 ⇒ 00:09:10.140 Casie Aviles: Hmm, so basically we wanted to see if… the test script.
118 00:09:10.460 ⇒ 00:09:14.640 Casie Aviles: Matches, like, the execution times we got with the live one, and…
119 00:09:14.810 ⇒ 00:09:15.410 Samuel Roberts: Bye.
120 00:09:16.310 ⇒ 00:09:19.049 Casie Aviles: Yeah, so I ran another script for that.
121 00:09:19.480 ⇒ 00:09:22.240 Samuel Roberts: Yeah, I think we’re good to mark all those as done then.
122 00:09:23.040 ⇒ 00:09:23.770 Samuel Roberts: Huh.
123 00:09:28.930 ⇒ 00:09:32.909 Pranav: And this template embeddings are not retrievable in AMD…
124 00:09:32.910 ⇒ 00:09:37.430 Samuel Roberts: That was… Yeah, I thought he dealt with that pretty quickly, I don’t know.
125 00:09:37.620 ⇒ 00:09:39.030 Pranav: Yeah, I think this is done.
126 00:09:39.320 ⇒ 00:09:39.900 Samuel Roberts: Yeah.
127 00:09:42.320 ⇒ 00:09:48.679 Pranav: Okay… Alright, yeah, so, Casey, do you wanna maybe just share your screen?
128 00:09:49.830 ⇒ 00:09:52.819 Casie Aviles: Sure, I’m in A10 right now.
129 00:09:57.390 ⇒ 00:09:58.210 Casie Aviles: Hold.
130 00:10:06.860 ⇒ 00:10:08.280 Casie Aviles: Do you guys see it now?
131 00:10:08.690 ⇒ 00:10:09.300 Pranav: Yep.
132 00:10:11.240 ⇒ 00:10:17.009 Casie Aviles: Okay, so the last… Executions that we were getting was 28.
133 00:10:19.180 ⇒ 00:10:23.460 Casie Aviles: So that was… Yeah, that’s… It’s not a third day for me.
134 00:10:23.780 ⇒ 00:10:24.770 Pranav: So…
135 00:10:25.350 ⇒ 00:10:28.739 Casie Aviles: I think we can… I’m just gonna deactivate this now, since…
136 00:10:29.400 ⇒ 00:10:31.230 Samuel Roberts: Yeah. Saturday, though.
137 00:10:31.490 ⇒ 00:10:32.629 Samuel Roberts: Seems weird.
138 00:10:32.960 ⇒ 00:10:33.590 Samuel Roberts: What’s that?
139 00:10:33.590 ⇒ 00:10:39.079 Casie Aviles: I think they’re still using it in the weekend site, I believe. I saw some usage.
140 00:10:39.580 ⇒ 00:10:42.140 Samuel Roberts: But I’m wondering why the NAN was getting used in the weekend.
141 00:10:43.780 ⇒ 00:10:47.689 Casie Aviles: Oh, maybe… oh, I think it’s the… might be the time, though.
142 00:10:47.940 ⇒ 00:10:49.160 Casie Aviles: My time zone.
143 00:10:49.880 ⇒ 00:10:56.659 Samuel Roberts: Oh, of course, of course. I hate that it does that, okay. Yeah, that’s not… yeah. I wish it just gave us, like, a UTC time, but…
144 00:10:58.310 ⇒ 00:10:59.150 Samuel Roberts: Good call.
145 00:11:00.630 ⇒ 00:11:04.780 Casie Aviles: That’s… The fact that there’s no more after that.
146 00:11:05.400 ⇒ 00:11:05.910 Samuel Roberts: Okay.
147 00:11:07.800 ⇒ 00:11:09.170 Casie Aviles: Yeah, all clear.
148 00:11:11.940 ⇒ 00:11:15.239 Pranav: Cool. Okay, so I’ll just, move any data in deprecation.
149 00:11:15.780 ⇒ 00:11:16.830 Pranav: To done?
150 00:11:18.540 ⇒ 00:11:20.319 Casie Aviles: Let’s see here… I don’t think we need to delete it, necessarily.
151 00:11:20.320 ⇒ 00:11:21.520 Samuel Roberts: necessarily, so I would.
152 00:11:21.520 ⇒ 00:11:22.540 Casie Aviles: NBC.
153 00:11:23.910 ⇒ 00:11:25.749 Samuel Roberts: It’s probably the embedding pipeline too, right?
154 00:11:27.720 ⇒ 00:11:29.169 Pranav: Oh, there’s a few different things, yeah.
155 00:11:30.820 ⇒ 00:11:32.770 Casie Aviles: Yeah, we had a lot of workflows.
156 00:11:33.190 ⇒ 00:11:33.660 Pranav: Yep.
157 00:11:33.660 ⇒ 00:11:34.230 Samuel Roberts: Yeah.
158 00:11:44.870 ⇒ 00:11:48.609 Casie Aviles: These are… Word pencil again.
159 00:11:51.390 ⇒ 00:11:52.000 Pranav: Okay.
160 00:11:52.870 ⇒ 00:11:55.460 Pranav: Yeah. Yeah, that’s my gap.
161 00:11:58.720 ⇒ 00:12:02.110 Casie Aviles: Yeah, so I’ve also deactivated the others, I believe.
162 00:12:04.220 ⇒ 00:12:09.839 Casie Aviles: Last week, so… I think that should be all that’s running on N8N.
163 00:12:12.090 ⇒ 00:12:13.080 Pranav: Okay, cool.
164 00:12:14.630 ⇒ 00:12:16.389 Pranav: We’ll move this to done…
165 00:12:16.390 ⇒ 00:12:19.909 Samuel Roberts: Yeah, I’m looking at Superbase now. It does not look like you can pause if you’re on a…
166 00:12:20.360 ⇒ 00:12:21.710 Samuel Roberts: Paid plan?
167 00:12:24.460 ⇒ 00:12:27.710 Samuel Roberts: Which I don’t… like.
168 00:12:27.840 ⇒ 00:12:33.969 Samuel Roberts: But we can maybe just move it to a, like, a separate org that is not on a paid plan, maybe.
169 00:12:36.180 ⇒ 00:12:38.439 Samuel Roberts: It’s the only thing that their docs say.
170 00:12:48.790 ⇒ 00:12:52.560 Samuel Roberts: So… How do I know?
171 00:13:03.620 ⇒ 00:13:13.080 Pranav: Yeah. So we will be flying…
172 00:13:13.300 ⇒ 00:13:21.170 Samuel Roberts: I think I’m gonna make a separate… yeah, you can’t do that. That’s what I was just like, man. But we can make, like, a Brainforge-free organization, it says, and like…
173 00:13:21.510 ⇒ 00:13:22.790 Samuel Roberts: Transfer it.
174 00:13:22.960 ⇒ 00:13:25.210 Samuel Roberts: And you’re not allowed to pause ones on a free one.
175 00:13:26.010 ⇒ 00:13:31.850 Samuel Roberts: I just feel like deleting data makes me nervous right now until we’re, like, a little farther down.
176 00:13:31.850 ⇒ 00:13:32.860 Pranav: Yeah, I like that.
177 00:13:32.860 ⇒ 00:13:37.160 Samuel Roberts: So, I think just keep it for now, probably throw a ticket in, like, a month, maybe, to get rid of it.
178 00:13:37.650 ⇒ 00:13:42.500 Samuel Roberts: Okay. Everything’s been good. I just don’t want to get to a point when we’re, like, Oh, shit.
179 00:13:43.980 ⇒ 00:13:45.360 Pranav: Yeah, that’s fair.
180 00:13:45.620 ⇒ 00:13:51.810 Pranav: Okay, cool, that’s… that’s that. Casey, do you want to just show a real now?
181 00:13:54.150 ⇒ 00:13:55.110 Casie Aviles: Sure, sure.
182 00:13:55.110 ⇒ 00:13:55.944 Samuel Roberts: 3…
183 00:13:56.780 ⇒ 00:13:59.140 Casie Aviles: Did we just go through, like, the dashboard now?
184 00:13:59.530 ⇒ 00:14:00.030 Pranav: Yeah.
185 00:14:01.860 ⇒ 00:14:03.849 Casie Aviles: Deprecating something else.
186 00:14:05.010 ⇒ 00:14:07.159 Pranav: No, just the dashboard.
187 00:14:08.010 ⇒ 00:14:08.630 Casie Aviles: Okay.
188 00:14:08.820 ⇒ 00:14:09.220 Pranav: Yep.
189 00:14:09.890 ⇒ 00:14:14.640 Casie Aviles: So this is just the local one. I submitted the PR for this,
190 00:14:15.710 ⇒ 00:14:18.030 Casie Aviles: Basically, I just added this,
191 00:14:19.860 ⇒ 00:14:23.139 Casie Aviles: This, dimension here, input intent category.
192 00:14:23.440 ⇒ 00:14:30.660 Casie Aviles: Nice. So it’s the same… It’s the same categories that we have in the… in the CSV.
193 00:14:31.180 ⇒ 00:14:35.419 Pranav: Oh, okay. So there’ll be consistent categories per week, okay.
194 00:14:35.810 ⇒ 00:14:36.760 Pranav: But… Yep.
195 00:14:36.970 ⇒ 00:14:43.939 Casie Aviles: I think what’s missing for now is… the actual… Automated process of…
196 00:14:45.110 ⇒ 00:14:52.700 Casie Aviles: assigning these categories, right? So this, this is only for… this is the one-time Okay. Run that we did.
197 00:14:53.800 ⇒ 00:14:57.480 Casie Aviles: So for… from 16th to 26. So…
198 00:14:58.480 ⇒ 00:15:04.430 Casie Aviles: what… basically, this… this exposes, you know, what we have in BigQuery now.
199 00:15:06.050 ⇒ 00:15:11.779 Casie Aviles: I still need to work on, like, Populating that automatically.
200 00:15:15.280 ⇒ 00:15:19.650 Pranav: Okay, yeah, that’s… That’s honestly okay, I actually do…
201 00:15:20.180 ⇒ 00:15:23.280 Pranav: I mean, yeah, I guess it depends.
202 00:15:23.460 ⇒ 00:15:35.220 Pranav: I see pros and cons to having dynamically created categories by the AI. The pro is, like, by having fixed categories, is that you can track over time how things are changing.
203 00:15:37.090 ⇒ 00:15:42.749 Pranav: But then, the definite, like, how we’ve defined these categories is just using AI. We didn’t… I think the…
204 00:15:43.270 ⇒ 00:15:47.790 Pranav: If we’re gonna have fixed categories, we probably want to, like, put some thought into, like.
205 00:15:48.030 ⇒ 00:15:52.350 Pranav: what are the right categories to… to look at? .
206 00:15:53.380 ⇒ 00:16:12.660 Samuel Roberts: My thought there is to probably, like, yeah, do a quick pass on these, see if there’s any categories we can either combine, remove, you know, add, and then settle on, like, a fix, and then have an other, and that way, if the AI really doesn’t think anything belongs, it can dump it in other, and then periodically we do a…
207 00:16:13.180 ⇒ 00:16:14.730 Samuel Roberts: a scan of other…
208 00:16:15.230 ⇒ 00:16:16.980 Pranav: Yeah, I like that a lot, actually.
209 00:16:16.980 ⇒ 00:16:25.900 Samuel Roberts: That way, it’s not generating things on the fly and changing things, but, like, we can periodically go back and say, like, okay, this really should have been a new category, or no, no, the AI messed this one up, or, you know, whatever.
210 00:16:26.190 ⇒ 00:16:27.679 Pranav: Yeah, I like that.
211 00:16:27.950 ⇒ 00:16:28.650 Samuel Roberts: Okay.
212 00:16:30.950 ⇒ 00:16:38.409 Casie Aviles: Okay. Yeah. So that’s… that’s all for at least this one. I just also cleaned up
213 00:16:38.760 ⇒ 00:16:42.930 Casie Aviles: some of the… Dimensions that weren’t as useful.
214 00:16:43.630 ⇒ 00:16:44.830 Casie Aviles: So…
215 00:16:49.760 ⇒ 00:16:57.340 Casie Aviles: Yeah, like, for example, there were some quality scores that are just novels right now, combined scores, that…
216 00:16:57.760 ⇒ 00:17:05.550 Casie Aviles: Are, by default, set within so I hid these for now, until we…
217 00:17:06.190 ⇒ 00:17:09.099 Casie Aviles: Clean, fix that, like, the data.
218 00:17:11.099 ⇒ 00:17:16.100 Casie Aviles: what else? And then expected output, like, we don’t really have… because in the past, like, we wanted to…
219 00:17:18.210 ⇒ 00:17:27.460 Casie Aviles: Score the output, the actual output, against unexpected output, so that’s why we had this dimension here.
220 00:17:27.859 ⇒ 00:17:35.370 Casie Aviles: But we’re not… we’re not doing that, necessarily, so… and that’s why we have a lot of nulls, so I… I hid these for now.
221 00:17:36.390 ⇒ 00:17:37.120 Pranav: Gotcha.
222 00:17:37.470 ⇒ 00:17:41.790 Pranav: If you go up a little bit for that isTestUser,
223 00:17:42.390 ⇒ 00:17:44.849 Pranav: Is that the same thing as, like, testers?
224 00:17:46.060 ⇒ 00:17:48.289 Casie Aviles: S users, right?
225 00:17:49.430 ⇒ 00:17:52.320 Pranav: Right, like, to the left. Yeah, right there.
226 00:17:54.380 ⇒ 00:17:59.600 Casie Aviles: Oh, okay, yeah, this is just the definition of, like.
227 00:17:59.750 ⇒ 00:18:03.220 Casie Aviles: This is a different definition for now, so I need to include, like.
228 00:18:03.980 ⇒ 00:18:13.140 Casie Aviles: What’s kind of tricky right now is because the testing was, like, from a certain period only, like, right? Like, that was…
229 00:18:13.930 ⇒ 00:18:17.979 Casie Aviles: 25, I think, last week, so starting… Yeah.
230 00:18:18.530 ⇒ 00:18:23.390 Casie Aviles: 25 to sometime in the early… in early Friday, so…
231 00:18:23.390 ⇒ 00:18:24.170 Pranav: Yep.
232 00:18:24.880 ⇒ 00:18:27.689 Casie Aviles: We didn’t really tag them as…
233 00:18:27.880 ⇒ 00:18:32.679 Casie Aviles: When we were logging them, they weren’t tagged as test users right now, so I have to…
234 00:18:33.210 ⇒ 00:18:36.990 Casie Aviles: I guess, define… A test user filter that…
235 00:18:38.350 ⇒ 00:18:42.720 Casie Aviles: Looks at that, those, those, those, that period only, and…
236 00:18:42.850 ⇒ 00:18:45.590 Casie Aviles: Include, like, the trainers and Jenny’s.
237 00:18:46.500 ⇒ 00:18:55.700 Pranav: So, I think it needs to be that period only, because I think usage of ANDI is mostly from CSRs, right? So.
238 00:18:56.090 ⇒ 00:19:00.649 Pranav: If the trainers are ever using it, if Janice is ever using it, she is just testing it.
239 00:19:01.260 ⇒ 00:19:02.700 Casie Aviles: Oh.
240 00:19:02.700 ⇒ 00:19:11.910 Pranav: So, it doesn’t matter the time period, actually. So, this is just a good toggle for them to have, like, okay, were we just testing it out, or were we, actually using it?
241 00:19:13.680 ⇒ 00:19:24.370 Pranav: I think at the end of the day, like, when we’re getting billed, like, when we’re billing them, it’s overall usage, so even if Janice or the trainers were using it, there’s no language in there talking about, like.
242 00:19:24.370 ⇒ 00:19:26.520 Samuel Roberts: Oh shit, really? Who’s using it?
243 00:19:26.570 ⇒ 00:19:29.980 Pranav: Interesting. I mean, it has to be ABC, right?
244 00:19:30.370 ⇒ 00:19:34.299 Samuel Roberts: No, yeah, our usage definitely should not count, but you’re right.
245 00:19:34.410 ⇒ 00:19:34.890 Samuel Roberts: That’s interesting.
246 00:19:34.890 ⇒ 00:19:42.910 Pranav: But, I mean, it is all just, like, what was the intention, right? But if they are doing, like, 10, 20 tests per week.
247 00:19:43.200 ⇒ 00:19:46.130 Pranav: like… just the trainers and Janiece, like…
248 00:19:46.540 ⇒ 00:19:46.940 Samuel Roberts: Gotcha.
249 00:19:46.940 ⇒ 00:19:51.200 Pranav: It’s not a big deal if that gets included, but that being said, too, like…
250 00:19:52.210 ⇒ 00:19:56.390 Pranav: they’re probably never using ANDI in the way that a CSR is using ANDI.
251 00:19:56.570 ⇒ 00:19:57.630 Pranav: So…
252 00:19:58.710 ⇒ 00:20:06.470 Pranav: if we can just put them into a list where we can just filter them out from the actual usage, that’s… that’s probably the number we want to be reporting to them. That’s what’s actually…
253 00:20:06.470 ⇒ 00:20:09.659 Samuel Roberts: True, like, they definitely don’t, like, act as a CSR ever.
254 00:20:10.310 ⇒ 00:20:11.390 Samuel Roberts: Are you sure about that?
255 00:20:12.040 ⇒ 00:20:15.350 Pranav: That’s my assumption.
256 00:20:15.590 ⇒ 00:20:19.340 Samuel Roberts: Okay, I’m just not sure, I don’t… 100% know that.
257 00:20:20.130 ⇒ 00:20:23.330 Pranav: Yeah, so let’s… let’s work under that assumption for now, and I’ll…
258 00:20:23.330 ⇒ 00:20:23.890 Samuel Roberts: Fine.
259 00:20:24.100 ⇒ 00:20:25.220 Pranav: Yeah.
260 00:20:25.220 ⇒ 00:20:25.550 Samuel Roberts: It doesn’t.
261 00:20:25.970 ⇒ 00:20:29.370 Pranav: But, the trainers, I’m not entirely.
262 00:20:29.370 ⇒ 00:20:30.080 Samuel Roberts: Yeah.
263 00:20:30.250 ⇒ 00:20:36.740 Samuel Roberts: and I… all it’ll really impact is, like, a few… you know.
264 00:20:37.230 ⇒ 00:20:42.389 Samuel Roberts: few metrics, a small amount, I’m not too worried about that, especially if it’s not affecting the billing, I don’t think it’s gonna, like…
265 00:20:42.920 ⇒ 00:20:44.260 Samuel Roberts: be an issue.
266 00:20:46.630 ⇒ 00:20:53.329 Samuel Roberts: Because, yeah, unless they’re testing it massively, we shouldn’t have to worry about that too much. So it’s… and yeah, I think that’s probably fine.
267 00:20:53.970 ⇒ 00:20:55.440 Samuel Roberts: Yeah. Export that assumption.
268 00:20:56.000 ⇒ 00:20:58.229 Samuel Roberts: If it becomes an issue, we can clarify.
269 00:20:58.540 ⇒ 00:20:59.150 Pranav: Yep.
270 00:21:00.390 ⇒ 00:21:06.539 Casie Aviles: Yeah, right now, the test users just, you know, us, so it’s excluding us.
271 00:21:06.850 ⇒ 00:21:10.759 Casie Aviles: everything that uses the Brainforge service account, and then…
272 00:21:10.900 ⇒ 00:21:19.250 Casie Aviles: If it matches our names, so that’s us, Misa, and other past users, so… And also Pranov, so…
273 00:21:19.600 ⇒ 00:21:21.180 Casie Aviles: Yeah, those are filtered out.
274 00:21:21.590 ⇒ 00:21:23.329 Casie Aviles: Yeah, like, for example here.
275 00:21:23.880 ⇒ 00:21:24.810 Pranav: Nice, nice.
276 00:21:26.390 ⇒ 00:21:27.230 Casie Aviles: So I’ll just…
277 00:21:27.650 ⇒ 00:21:28.380 Pranav: give a…
278 00:21:28.380 ⇒ 00:21:29.250 Casie Aviles: I’ll just…
279 00:21:29.500 ⇒ 00:21:33.209 Pranav: To, to see what’s the next tier we have to get to.
280 00:21:33.330 ⇒ 00:21:38.660 Pranav: So, right now, we’re at 2,000 sessions per month. If we get to 5,000,
281 00:21:38.920 ⇒ 00:21:45.009 Pranav: That’s, like, the next revenue tier. So, not sure if you guys already knew that, but…
282 00:21:45.690 ⇒ 00:21:49.249 Samuel Roberts: No, I didn’t really know what… I knew Amber was talking about tears, I didn’t actually see the…
283 00:21:49.470 ⇒ 00:21:53.300 Pranav: Yeah, so let me… this is probably just interesting for you guys, you know, it’s like a little.
284 00:21:53.300 ⇒ 00:21:54.630 Samuel Roberts: Yeah, definitely good to know.
285 00:21:54.630 ⇒ 00:21:59.170 Pranav: Right, like… I’ll… I’ll throw this into our… to our chat.
286 00:22:01.900 ⇒ 00:22:07.059 Samuel Roberts: 5,000, and so we’re at… We’re at the 2,000 tier, is what you’re saying, right?
287 00:22:07.060 ⇒ 00:22:09.500 Pranav: Yeah, it’s a little bit confusing.
288 00:22:10.320 ⇒ 00:22:12.210 Pranav: But that’s why I was like.
289 00:22:12.370 ⇒ 00:22:17.320 Pranav: took me a while, I was just like, let me see the actual contract, let me see, like, what we’re actually invoicing to see, like.
290 00:22:17.320 ⇒ 00:22:17.780 Samuel Roberts: Right.
291 00:22:17.780 ⇒ 00:22:20.830 Pranav: They’re understanding this.
292 00:22:21.830 ⇒ 00:22:30.139 Pranav: But, yeah, so basically if you go above 2,000, you’re in Tier 2, which is the 5,000… up to 5,000 sessions per month.
293 00:22:30.300 ⇒ 00:22:35.760 Pranav: Which then brings us to 12K per month.
294 00:22:37.160 ⇒ 00:22:40.459 Samuel Roberts: And this is, like, just every month.
295 00:22:40.580 ⇒ 00:22:44.440 Samuel Roberts: which tier are you in? It’s not like you pass one and stay there, kind of thing.
296 00:22:44.440 ⇒ 00:22:46.290 Pranav: Yeah, yeah, it’s just…
297 00:22:47.530 ⇒ 00:22:48.250 Samuel Roberts: Okay.
298 00:22:48.250 ⇒ 00:22:51.040 Pranav: Hasn’t happened yet that we’ve gone down in usage by that much.
299 00:22:51.050 ⇒ 00:22:55.790 Samuel Roberts: Right, but I’m just saying, like, you know, getting up to 5, and then if something’s 4 the next month, it’s like…
300 00:22:55.790 ⇒ 00:22:56.790 Pranav: Yeah.
301 00:22:57.000 ⇒ 00:22:57.510 Pranav: Yeah.
302 00:22:57.510 ⇒ 00:22:58.160 Samuel Roberts: Good.
303 00:22:58.470 ⇒ 00:22:59.440 Pranav: Right, right.
304 00:23:00.610 ⇒ 00:23:03.010 Pranav: I would assume that’s how it works, to be honest.
305 00:23:03.010 ⇒ 00:23:08.200 Samuel Roberts: Yeah, I think I vaguely remember hearing that, but I don’t know if that was… You know.
306 00:23:08.630 ⇒ 00:23:09.210 Pranav: Yeah.
307 00:23:09.210 ⇒ 00:23:10.279 Samuel Roberts: My imagination or not.
308 00:23:10.470 ⇒ 00:23:12.349 Pranav: So February, okay.
309 00:23:12.620 ⇒ 00:23:15.799 Pranav: Do you want to take a look at… March?
310 00:23:17.520 ⇒ 00:23:20.960 Pranav: And then we have to remove a bunch of… well, let’s just see if we have to.
311 00:23:21.780 ⇒ 00:23:23.419 Pranav: Do we even get to 5K?
312 00:23:23.930 ⇒ 00:23:30.920 Pranav: Yeah, okay, so it doesn’t really matter. Okay. Because, I mean, this is including their QA testing, which was about, like, 500…
313 00:23:31.210 ⇒ 00:23:33.220 Samuel Roberts: I was gonna say, yeah, so it’s still not… I mean, that’s…
314 00:23:33.390 ⇒ 00:23:34.960 Pranav: It’s actually still pretty good, though.
315 00:23:34.960 ⇒ 00:23:39.360 Samuel Roberts: Good, yeah, it’s closer than I was thinking it would be, based on, like, just knowing 2000 was the bottom.
316 00:23:39.710 ⇒ 00:23:40.320 Pranav: Yeah.
317 00:23:41.290 ⇒ 00:23:47.570 Pranav: Oh, so that’s including our testing, but then we didn’t include their QA testing, so that’s within this 3,800.
318 00:23:47.670 ⇒ 00:23:48.210 Pranav: Right.
319 00:23:48.960 ⇒ 00:23:52.130 Pranav: Okay, so then, yeah, it’s probably around 3K, which is…
320 00:23:52.130 ⇒ 00:23:52.590 Samuel Roberts: Okay.
321 00:23:52.590 ⇒ 00:23:56.229 Pranav: What was February? February’s 2-something? 2-4, right?
322 00:23:57.420 ⇒ 00:23:58.240 Pranav: Yeah.
323 00:23:58.240 ⇒ 00:23:59.140 Casie Aviles: Yeah.
324 00:24:01.950 ⇒ 00:24:03.950 Casie Aviles: Oh, 2,200.
325 00:24:03.950 ⇒ 00:24:06.689 Pranav: Yeah, so I mean, that’s actually a lot.
326 00:24:06.850 ⇒ 00:24:08.430 Pranav: Of improvement in that month.
327 00:24:09.510 ⇒ 00:24:10.780 Pranav: That’s it, guys.
328 00:24:12.390 ⇒ 00:24:15.180 Samuel Roberts: I like seeing stuff like, where’s my mouse here?
329 00:24:15.730 ⇒ 00:24:21.450 Samuel Roberts: This is nice to see, because everyone’s, like, just seeing all the thumbs downs come through, you don’t really have a sense of the volume.
330 00:24:21.760 ⇒ 00:24:22.370 Pranav: Per se.
331 00:24:22.370 ⇒ 00:24:26.339 Samuel Roberts: Yes. So that is nice to see a… can we go to,
332 00:24:26.870 ⇒ 00:24:30.229 Samuel Roberts: last month now, too, just to see that real quick, now that I’m noticing that.
333 00:24:30.680 ⇒ 00:24:31.430 Pranav: Yeah.
334 00:24:37.150 ⇒ 00:24:38.050 Pranav: Okay.
335 00:24:38.770 ⇒ 00:24:41.330 Pranav: Still looks like the proportions went up a little bit.
336 00:24:41.540 ⇒ 00:24:42.280 Samuel Roberts: Yeah.
337 00:24:42.620 ⇒ 00:24:45.409 Samuel Roberts: But I mean, thumbs up and thumbs down, like, you know what I mean? Both of those, like.
338 00:24:45.410 ⇒ 00:24:45.980 Pranav: Oh, that’s true.
339 00:24:45.980 ⇒ 00:24:46.799 Samuel Roberts: good, so…
340 00:24:46.800 ⇒ 00:24:51.279 Pranav: Yeah, yeah. I think probably what happened there was just, like, during the QA testing, they just…
341 00:24:51.820 ⇒ 00:24:53.050 Pranav: Gave more feedback.
342 00:24:53.220 ⇒ 00:24:54.820 Samuel Roberts: That’s totally true, that’s totally true.
343 00:24:55.450 ⇒ 00:24:58.130 Pranav: Because, like, yeah, it doesn’t look like the ratio changed at all.
344 00:24:58.130 ⇒ 00:25:08.339 Samuel Roberts: I was gonna say, I don’t know real super well, Casey, so tell me if there’s a good way to do this, but is there a way to then make this, like, to show this percentage in here somewhere?
345 00:25:09.640 ⇒ 00:25:11.950 Samuel Roberts: So that we can actually see, like, you know, we…
346 00:25:12.200 ⇒ 00:25:18.370 Samuel Roberts: We know the totals, but we can compare month to month on, like, the percentage of thumbs up, thumbs down, no.
347 00:25:19.520 ⇒ 00:25:23.040 Casie Aviles: Hmm. I haven’t tried that one. Okay.
348 00:25:23.580 ⇒ 00:25:24.810 Casie Aviles: Yeah.
349 00:25:25.440 ⇒ 00:25:26.070 Samuel Roberts: Okay.
350 00:25:26.450 ⇒ 00:25:30.459 Samuel Roberts: Yeah, I just… I’m not super familiar with it yet, I haven’t done a lot of real work yet, so…
351 00:25:30.700 ⇒ 00:25:32.979 Pranav: Yeah, and then also this thumbs down…
352 00:25:33.520 ⇒ 00:25:38.449 Pranav: What Janiece told me was that some of the trainers were putting thumbs downs for things that are actually not thumbs down.
353 00:25:39.140 ⇒ 00:25:41.679 Samuel Roberts: I believe that, and based on some things I see, it’s like…
354 00:25:42.090 ⇒ 00:25:46.239 Samuel Roberts: just in the… in the channel, I’m like, okay, that’s a… that’s a good one, that’s not a good one, you know?
355 00:25:46.840 ⇒ 00:25:47.980 Pranav: Yeah, and so…
356 00:25:48.250 ⇒ 00:25:55.460 Pranav: What we might want to do at some point is then have a real dashboard that pulls from linear.
357 00:25:55.580 ⇒ 00:26:00.350 Pranav: To then look at all the triage tickets to see, okay, which ones were actually canceled.
358 00:26:00.350 ⇒ 00:26:01.120 Samuel Roberts: Mmm.
359 00:26:01.490 ⇒ 00:26:04.920 Pranav: They were falsely classified as, like, thumbs down.
360 00:26:05.330 ⇒ 00:26:05.890 Samuel Roberts: Yeah.
361 00:26:07.160 ⇒ 00:26:10.870 Pranav: But… That’s… that’s for the future, it’s not super important. Definitely, yeah.
362 00:26:10.870 ⇒ 00:26:12.740 Samuel Roberts: Yeah, no, I mean, the data’s there, so…
363 00:26:12.920 ⇒ 00:26:16.710 Pranav: Cool. No, Casey, this is… this is amazing. Thanks for doing all this.
364 00:26:18.330 ⇒ 00:26:19.480 Casie Aviles: Yeah.
365 00:26:19.750 ⇒ 00:26:24.959 Casie Aviles: Do we… should we also look at, like, this other… this execution?
366 00:26:24.960 ⇒ 00:26:26.609 Samuel Roberts: I was gonna ask the heat map one?
367 00:26:26.610 ⇒ 00:26:27.860 Pranav: Oh, yeah, yeah.
368 00:26:29.930 ⇒ 00:26:32.430 Casie Aviles: So yeah, I was… I was doing this…
369 00:26:32.740 ⇒ 00:26:37.419 Casie Aviles: Manually on BQ, so I figured, like, it would have to be also B.
370 00:26:37.940 ⇒ 00:26:40.200 Casie Aviles: good to have it on real, so…
371 00:26:40.620 ⇒ 00:26:44.469 Casie Aviles: This lets us see, like, for example, this period.
372 00:26:45.130 ⇒ 00:26:52.780 Casie Aviles: per day, like, we can compare as well, like, the execution times for, like, March 9 and 23, if you can see here.
373 00:26:54.410 ⇒ 00:26:58.459 Casie Aviles: So it’s… we had some spikes here.
374 00:26:58.570 ⇒ 00:27:04.040 Casie Aviles: From the testing… And then we’re seeing, like, We’re getting lower again.
375 00:27:06.020 ⇒ 00:27:10.120 Casie Aviles: Then we’ll… same thing for, like, the median, P95…
376 00:27:10.730 ⇒ 00:27:18.360 Casie Aviles: And then here, this is, like… The percentage of, like, how many…
377 00:27:18.590 ⇒ 00:27:23.549 Casie Aviles: Or, like, the percent of records that are, like, less than or equal to 5 seconds.
378 00:27:24.150 ⇒ 00:27:26.920 Casie Aviles: So… Okay.
379 00:27:26.920 ⇒ 00:27:27.980 Samuel Roberts: Nice looking.
380 00:27:28.340 ⇒ 00:27:29.650 Samuel Roberts: Into the graph there.
381 00:27:30.320 ⇒ 00:27:30.910 Pranav: Nice.
382 00:27:30.910 ⇒ 00:27:37.339 Casie Aviles: For yesterday, that was… we are at 67%, so that’s 67%.
383 00:27:37.710 ⇒ 00:27:45.190 Casie Aviles: Less than or equal to 5 seconds, so we have, like, around… 33, that’s…
384 00:27:45.980 ⇒ 00:27:49.969 Casie Aviles: Still, like, above 5 seconds. So those are what we can.
385 00:27:49.970 ⇒ 00:27:52.820 Samuel Roberts: Totally. Overall improvement, though, except for the…
386 00:27:53.010 ⇒ 00:27:56.270 Samuel Roberts: This period… oh, sorry, yeah, except for that period.
387 00:27:56.530 ⇒ 00:27:59.100 Samuel Roberts: Looks like overall improvement, which is…
388 00:28:00.070 ⇒ 00:28:04.479 Pranav: Yeah, and so this week, and then next week will probably be, like, really good to just…
389 00:28:04.820 ⇒ 00:28:09.740 Pranav: see that there’s no more, like, kind of fluctuation like that… like that.
390 00:28:10.210 ⇒ 00:28:18.639 Pranav: Yeah, very easy way to describe it is just, like, follow a yellow line here, follow the other line, make sure that’s below the previous.
391 00:28:19.360 ⇒ 00:28:25.790 Pranav: But I guess that’s not always gonna be the case, that’s only gonna be the case right now, right? Because just comparing against 2 weeks ago.
392 00:28:26.320 ⇒ 00:28:26.930 Samuel Roberts: Yeah.
393 00:28:27.930 ⇒ 00:28:31.750 Pranav: Okay, so I guess this dashboard is really going to be useful for just…
394 00:28:33.170 ⇒ 00:28:38.179 Pranav: like, right now, I guess. And then maybe, like, going forward, if we’re seeing…
395 00:28:38.320 ⇒ 00:28:42.900 Pranav: Differences in execution time, then it’s good for us to clock, but…
396 00:28:42.900 ⇒ 00:28:49.980 Samuel Roberts: I think, yeah, it’ll be good to be able to see where there are spikes and stuff, too, because, like, if there’s no changes and we see stuff like this, like…
397 00:28:50.270 ⇒ 00:28:53.660 Samuel Roberts: It’s pretty clear it’s a Gemini thing, probably, again.
398 00:28:53.960 ⇒ 00:29:00.459 Samuel Roberts: Gotcha. Yeah, we’re not changing, you know what I mean? If we haven’t made any changes, like, substantial changes at least,
399 00:29:00.680 ⇒ 00:29:04.520 Samuel Roberts: I would imagine, like, if they’re starting to report, oh, a bunch of this stuff, a bunch of this stuff…
400 00:29:06.050 ⇒ 00:29:10.340 Samuel Roberts: we can probably come in here and say, like, oh yeah, there’s a huge spike going on there.
401 00:29:10.580 ⇒ 00:29:11.300 Pranav: Yeah.
402 00:29:11.460 ⇒ 00:29:17.730 Pranav: Cool. Yeah, it builds our case for why, like, hey, we’re doing everything here, Gemini’s the one that’s…
403 00:29:18.120 ⇒ 00:29:18.770 Samuel Roberts: Yeah.
404 00:29:18.770 ⇒ 00:29:22.780 Pranav: modeling and affecting execution time. Okay, that’s great.
405 00:29:23.070 ⇒ 00:29:30.909 Pranav: And then this will be a great dashboard for me to just show on Friday, just to be like, hey, look how much better we’re doing compared to the N8N app.
406 00:29:32.320 ⇒ 00:29:32.830 Casie Aviles: Yeah.
407 00:29:33.350 ⇒ 00:29:34.469 Samuel Roberts: Yeah, definitely.
408 00:29:36.480 ⇒ 00:29:37.020 Pranav: Cool.
409 00:29:37.330 ⇒ 00:29:44.309 Casie Aviles: That’s great. I think that’s all for this dashboard. This is just, like, a more… a simple table here that also shows
410 00:29:44.410 ⇒ 00:29:45.660 Casie Aviles: per day.
411 00:29:47.090 ⇒ 00:29:48.030 Casie Aviles: But, yeah.
412 00:29:48.800 ⇒ 00:29:49.690 Casie Aviles: That’s cool.
413 00:29:51.670 ⇒ 00:30:01.000 Pranav: Okay, awesome, yeah, and the last part I wanted to do today for this working session was, let’s talk about the Central Doc Copilot.
414 00:30:02.830 ⇒ 00:30:06.300 Pranav: For this week, all that needs to get done is just that,
415 00:30:06.770 ⇒ 00:30:11.270 Pranav: the new triage workflow. Basically.
416 00:30:12.950 ⇒ 00:30:21.479 Pranav: what I wanted to set up is, like, within Linear, having certain gates for the tickets to go through, based on what
417 00:30:21.630 ⇒ 00:30:26.619 Pranav: based on what decision Janice makes,
418 00:30:27.730 ⇒ 00:30:29.960 Pranav: And so, yeah, that’s kind of just on me.
419 00:30:30.460 ⇒ 00:30:36.049 Pranav: I’ll figure that out today, and then I’ll post that for you guys, in the channel.
420 00:30:36.240 ⇒ 00:30:37.270 Pranav: Sam, I…
421 00:30:37.270 ⇒ 00:30:38.820 Samuel Roberts: the… sorry, what?
422 00:30:39.060 ⇒ 00:30:43.210 Pranav: I just know you, worked on a bunch of, like, the tickets.
423 00:30:43.770 ⇒ 00:30:50.680 Samuel Roberts: Yeah, I was just gonna say, there’s one there, it was such a yes. I mean, we can tweak the dates, because once we finally got to it all, but there’s the…
424 00:30:51.020 ⇒ 00:30:58.520 Samuel Roberts: First one here is define, triage, state machine label submission format, lock and lock approval triggers. So you’re talking… you’re saying you’ll… do that?
425 00:30:58.720 ⇒ 00:31:00.600 Pranav: Yeah, yeah, yeah, yeah, I’ll do that.
426 00:31:01.320 ⇒ 00:31:03.820 Samuel Roberts: Alright, and then that blocks…
427 00:31:04.780 ⇒ 00:31:05.120 Pranav: Yeah.
428 00:31:05.120 ⇒ 00:31:10.700 Samuel Roberts: that is actually blocked by something else, which is evaluate input mechanism option, but I don’t think that really…
429 00:31:11.160 ⇒ 00:31:15.540 Samuel Roberts: So we actually, if you want to talk through this a little bit more, how…
430 00:31:15.780 ⇒ 00:31:22.420 Samuel Roberts: Or what you were thinking, maybe? Because, like, there were some other thoughts about how to handle that blah.
431 00:31:22.880 ⇒ 00:31:29.220 Samuel Roberts: the… linear submission versus the Google Doc comments.
432 00:31:30.020 ⇒ 00:31:33.310 Pranav: Oh, yeah, so for the google.comment.
433 00:31:33.840 ⇒ 00:31:37.550 Pranav: I think we shouldn’t go down that route. Okay.
434 00:31:38.210 ⇒ 00:31:39.070 Samuel Roberts: Because…
435 00:31:40.290 ⇒ 00:31:41.260 Pranav: So…
436 00:31:41.410 ⇒ 00:31:49.169 Pranav: there’s two… so I think in the linear ticket, it might have been confusing what I meant by linear.comment… I mean, sorry, google.comment.
437 00:31:49.170 ⇒ 00:31:50.120 Samuel Roberts: Oh, okay.
438 00:31:50.120 ⇒ 00:32:03.209 Pranav: Because I don’t want the trainers themselves to be making comments in there. The comment would happen from… from our script, so then we would know exactly where to paste the central doc update.
439 00:32:03.580 ⇒ 00:32:09.390 Samuel Roberts: Oh, my… okay, okay, sorry, so I think when we had talked about it, my thought was that they would make a comment that says…
440 00:32:09.550 ⇒ 00:32:16.790 Samuel Roberts: or, like, even an edit suggestion or something, that would be a… in the Google Doc, we could use the…
441 00:32:17.250 ⇒ 00:32:21.100 Samuel Roberts: GWS CLI We can pull that.
442 00:32:21.630 ⇒ 00:32:24.070 Samuel Roberts: Confirm it through all the stuff we’re gonna build.
443 00:32:24.070 ⇒ 00:32:36.590 Pranav: Yeah, the problem with that, though, is I don’t want them to be thinking about where it should go in the dock. That should be us. Yeah. They can give, like, some suggested heading. I think I wrote out…
444 00:32:36.590 ⇒ 00:32:37.200 Samuel Roberts: Okay.
445 00:32:37.200 ⇒ 00:32:38.020 Pranav: Somewhere, like…
446 00:32:38.020 ⇒ 00:32:40.489 Samuel Roberts: We can resolve that ticket then.
447 00:32:40.490 ⇒ 00:32:41.330 Pranav: Yeah.
448 00:32:41.800 ⇒ 00:32:43.920 Pranav: But, yeah, I don’t think it…
449 00:32:44.630 ⇒ 00:32:46.629 Samuel Roberts: You have, you say you wrote something out?
450 00:32:47.350 ⇒ 00:32:49.100 Pranav: I wrote it out a while ago, it might just be.
451 00:32:49.100 ⇒ 00:32:49.770 Samuel Roberts: Okay.
452 00:32:49.770 ⇒ 00:32:57.479 Pranav: I can find that. Yeah, if you… I mean, you can leave that ticket open, and I’ll give it a full read, and then we can…
453 00:32:57.940 ⇒ 00:33:01.050 Pranav: I’ll think about it a little bit more.
454 00:33:01.420 ⇒ 00:33:02.300 Samuel Roberts: Yeah.
455 00:33:02.300 ⇒ 00:33:02.900 Pranav: Yo.
456 00:33:04.320 ⇒ 00:33:06.500 Samuel Roberts: Yeah, cause I see exactly what you’re saying.
457 00:33:09.340 ⇒ 00:33:10.230 Samuel Roberts: I just…
458 00:33:12.350 ⇒ 00:33:18.029 Samuel Roberts: Yeah, they need to know where it’s gonna… I mean, there is some amount of, like, human readability we probably still want on the central dock.
459 00:33:18.450 ⇒ 00:33:21.940 Samuel Roberts: So, like, but then, will they know it, you know what I mean? I don’t know.
460 00:33:21.940 ⇒ 00:33:28.940 Pranav: Yeah, yeah, like, at the end of the day, it will be human-readable, right? Wherever we end up putting it, it’s going to make sense if the AI decides it.
461 00:33:28.940 ⇒ 00:33:33.519 Samuel Roberts: Yeah, that I get. I’m just debating if they would even need to have any understanding of it, which…
462 00:33:33.530 ⇒ 00:33:42.259 Pranav: Yeah… to me, I don’t think they need to, like, at the end of the day, we should be maximizing this for Andy, not for the trainers.
463 00:33:43.120 ⇒ 00:33:46.500 Pranav: And so… Yeah, like.
464 00:33:46.500 ⇒ 00:33:46.820 Samuel Roberts: Yeah.
465 00:33:47.050 ⇒ 00:33:53.120 Pranav: Yeah, so we need to kind of make it so they’re looking at the central dock less and less.
466 00:33:54.400 ⇒ 00:34:00.680 Samuel Roberts: Yeah, my only… Pushback there would be, this central dock was, like, a…
467 00:34:01.580 ⇒ 00:34:15.970 Samuel Roberts: amalgamation of all their other random places, or all their other sources, and so part of me wants to say, like, well, they should… we should enforce to them, like, only ever look here, you know what I mean? Like, if you need to update something, like.
468 00:34:16.730 ⇒ 00:34:26.679 Samuel Roberts: because there was a while where they were still changing other docs and things, and it was kind of a push to, like, care about the central dock more. But if we can skip that, I just…
469 00:34:26.940 ⇒ 00:34:31.899 Samuel Roberts: I don’t see really a huge problem there, I’m just, you know, more plain devil’s advocate here than anything.
470 00:34:32.030 ⇒ 00:34:37.619 Samuel Roberts: So I think if there’s a good… a good, like, this is the information we need
471 00:34:37.819 ⇒ 00:34:43.299 Samuel Roberts: And maybe that’s sort of what you had put together, like, the format for a submission.
472 00:34:43.619 ⇒ 00:34:46.389 Samuel Roberts: To update the information in the central doc.
473 00:34:46.779 ⇒ 00:34:53.459 Samuel Roberts: And as long as that is clear to them, that it’s just…
474 00:34:53.749 ⇒ 00:34:58.119 Samuel Roberts: Andy needs to know this, not this, and right now it’s returning this, kind of thing.
475 00:34:58.549 ⇒ 00:35:01.489 Samuel Roberts: we can probably handle the rest of that, you’re right.
476 00:35:02.380 ⇒ 00:35:03.170 Pranav: Okay.
477 00:35:04.460 ⇒ 00:35:06.999 Pranav: I’ll give it a… I’ll give it… I’ll think about it.
478 00:35:07.000 ⇒ 00:35:12.720 Samuel Roberts: Yeah, I want to put you on this one too, you can do whatever you need with that ticket, resolve it whichever way you need there.
479 00:35:13.050 ⇒ 00:35:15.110 Samuel Roberts: So that’s blocking that, and then…
480 00:35:15.950 ⇒ 00:35:20.080 Samuel Roberts: Proposal detection listener, that depends on that first one, I think.
481 00:35:20.780 ⇒ 00:35:24.910 Samuel Roberts: And then train them on how to use it as just another ticket there that I don’t think has a…
482 00:35:25.690 ⇒ 00:35:31.140 Samuel Roberts: Ton following it, and then… Yeah, okay. I think,
483 00:35:31.600 ⇒ 00:35:34.449 Samuel Roberts: Yeah, give that some more thought. I think you’re probably…
484 00:35:34.840 ⇒ 00:35:38.690 Samuel Roberts: Not really wrong either way, it just kind of depends how we want to…
485 00:35:38.970 ⇒ 00:35:39.569 Pranav: There’s supposed to be a…
486 00:35:39.570 ⇒ 00:35:40.680 Samuel Roberts: Interface with it, yeah.
487 00:35:40.900 ⇒ 00:35:44.249 Pranav: No, like, cause this should be just, like…
488 00:35:45.030 ⇒ 00:35:49.100 Pranav: very e… like, the easiest solution, you know? .
489 00:35:49.100 ⇒ 00:35:49.760 Samuel Roberts: Yeah.
490 00:35:49.760 ⇒ 00:36:00.479 Pranav: Google Docs seems easier, but then also I… it’s like, we need ease, but then we also need accuracy. So we need to get as much context from them as possible, so I’m even thinking maybe…
491 00:36:00.680 ⇒ 00:36:10.719 Pranav: they could go through the Google Doc, but then still have to fill out a templated comment, and if they don’t fill out the full, like, templated comment, like, you need to hit these, like, four different.
492 00:36:10.800 ⇒ 00:36:14.439 Samuel Roberts: Yeah, that’s a… that’s an option, where, like, we could give them…
493 00:36:15.190 ⇒ 00:36:23.029 Samuel Roberts: suggestion access or whatever, and they could be like, I think this should get deleted, I think we should add this, and then they leave a comment, and we pull that section.
494 00:36:23.690 ⇒ 00:36:27.320 Samuel Roberts: And resolve that.
495 00:36:27.620 ⇒ 00:36:31.650 Samuel Roberts: With the detection, the duplicate detection and the conflicting detection and stuff.
496 00:36:33.570 ⇒ 00:36:34.200 Pranav: Yeah.
497 00:36:34.200 ⇒ 00:36:40.120 Samuel Roberts: Either way, yeah, I mean, I think we can make it work. The question, I think you’re right, is does it matter if they’re in there or not?
498 00:36:40.120 ⇒ 00:36:40.730 Pranav: Yep.
499 00:36:41.050 ⇒ 00:36:41.860 Pranav: Yup.
500 00:36:42.020 ⇒ 00:36:46.850 Pranav: Okay, I’ll give this some thought. Okay, and then…
501 00:36:46.850 ⇒ 00:36:50.389 Samuel Roberts: Oh, sorry, no, you’re good, you’re good. No, I was just… there was more I was gonna say about the, like.
502 00:36:50.770 ⇒ 00:36:56.019 Samuel Roberts: part of this is, like, doing their data integrity sort of thing, and so I just wasn’t sure if, like.
503 00:36:56.630 ⇒ 00:36:59.859 Samuel Roberts: Where they need to be in that, if it’s important.
504 00:36:59.860 ⇒ 00:37:01.149 Pranav: Yeah, so, like, when something.
505 00:37:01.150 ⇒ 00:37:03.769 Samuel Roberts: Everything does change in the company.
506 00:37:04.310 ⇒ 00:37:07.929 Samuel Roberts: What is that… what is their process for getting it into here, and does it matter if they’re…
507 00:37:08.530 ⇒ 00:37:09.090 Pranav: Yeah.
508 00:37:09.090 ⇒ 00:37:10.439 Samuel Roberts: For the dock or not, yeah.
509 00:37:10.440 ⇒ 00:37:23.140 Pranav: That is important for them. Their thing is, like, they just want to be the final say, and so they’ll see the full activity of, like, what we’re proposing to change, and then they’ll then give the final sign-off for us.
510 00:37:23.140 ⇒ 00:37:26.929 Samuel Roberts: Yeah, I think that’s fine, as long as they know that, that’s totally cool. Yeah, I think you’re right. I think you’re right.
511 00:37:27.220 ⇒ 00:37:32.030 Pranav: Cool. And then, transcripts is one other thing. Yeah, Sam, it completely, like.
512 00:37:32.430 ⇒ 00:37:39.010 Pranav: I forgot about just, like, looking at that, conversation you had with, 8x8 last week, so I’m gonna.
513 00:37:39.010 ⇒ 00:37:39.520 Samuel Roberts: Yeah.
514 00:37:39.600 ⇒ 00:37:40.130 Pranav: learn.
515 00:37:40.130 ⇒ 00:37:45.739 Samuel Roberts: So… The context there is just, they knew we were ingesting transcripts.
516 00:37:46.330 ⇒ 00:37:51.540 Samuel Roberts: In some form. And that has gone through, like, me doing a quick spike.
517 00:37:51.650 ⇒ 00:37:55.799 Samuel Roberts: And then, Utam’s saying, like, let’s wait and get NSOW for that kind of work?
518 00:37:56.100 ⇒ 00:37:56.930 Samuel Roberts: Yep.
519 00:37:57.200 ⇒ 00:38:04.930 Samuel Roberts: And then that came back recently, because that was in, like, November, I think I first touched that. Gotcha. And then recently…
520 00:38:05.190 ⇒ 00:38:08.940 Samuel Roberts: Got some stuff loaded in, Amber did, like, a pass on, like, what was there.
521 00:38:09.060 ⇒ 00:38:11.209 Samuel Roberts: Analytically, because there was…
522 00:38:11.350 ⇒ 00:38:17.199 Samuel Roberts: I’m not 100% sure what the engagement is like for, like, the transcript analysis stuff, because there’s…
523 00:38:17.390 ⇒ 00:38:23.019 Samuel Roberts: the transcript and Andy kind of piece, and then there’s just, like, data mining the transcripts for…
524 00:38:23.470 ⇒ 00:38:25.550 Samuel Roberts: what are the CSRs doing, kind of thing.
525 00:38:25.760 ⇒ 00:38:26.230 Pranav: Yeah.
526 00:38:26.230 ⇒ 00:38:35.239 Samuel Roberts: makes sense. So I don’t know, for the Evolve thing, it sounded like they knew we were ingesting transcripts, so if we were doing that, they wanted an endpoint that could update the Evolve
527 00:38:35.920 ⇒ 00:38:47.350 Samuel Roberts: ticket or whatever it is, I don’t know that much about Evolve, but Evolve was basically like, okay, we can build an endpoint, you can just send some JSON there, and as long as you know, like, the customer and the agent, you can update a comment or something.
528 00:38:47.460 ⇒ 00:38:48.480 Samuel Roberts: So… Yep.
529 00:38:49.130 ⇒ 00:39:00.169 Samuel Roberts: that’s probably fine, as long as once we build that ingestion pipeline, you know, however frequently we’re doing it, however often we’re doing that, we can hit that. I just wanted to make sure, from, like, the business side of it, that we’re not
530 00:39:01.000 ⇒ 00:39:04.270 Samuel Roberts: Beyond our scope there, without… Yeah.
531 00:39:05.220 ⇒ 00:39:11.500 Pranav: Yeah, I mean, They’re still just not… I think what they’re doing is just trying to…
532 00:39:12.500 ⇒ 00:39:18.850 Pranav: have… couple everything more tightly with Evolve, right? Yeah.
533 00:39:19.020 ⇒ 00:39:21.519 Pranav: It’s just a business tactic, I think.
534 00:39:21.520 ⇒ 00:39:33.210 Samuel Roberts: I mean, whatever. I mean, this was a call with Evolve and with, Tim and Yvette. So, like, they had already clearly talked about stuff, and there were a few other things that, like, wasn’t related to us that they talked about. So…
535 00:39:33.210 ⇒ 00:39:42.959 Pranav: Because I also know, look, I don’t think this is super important, because Janiece is talking to other, CRM providers. So it’s just like, they’re thinking about leaving, like.
536 00:39:42.960 ⇒ 00:39:44.200 Samuel Roberts: That’s hilarious, okay.
537 00:39:44.200 ⇒ 00:39:48.700 Pranav: I’m just like, I don’t even want to build on top of that, like, they’re not even sold on that, right?
538 00:39:48.700 ⇒ 00:40:02.640 Samuel Roberts: Well, the thing about our impact would just be, if there’s an endpoint for Evolve, we can send the transcript to that. If there’s an endpoint for some other CRM, we can do that. I just wanted to make sure that I wasn’t, like, promising something to them that was beyond the scope of the work we signed for.
539 00:40:02.830 ⇒ 00:40:05.660 Samuel Roberts: So, that’s… that’s it there.
540 00:40:05.850 ⇒ 00:40:11.440 Pranav: On the transcripts part, though, like, you did do, like, a spike on this before to, like, ingest transcripts.
541 00:40:11.590 ⇒ 00:40:12.309 Pranav: Where’s a.
542 00:40:12.310 ⇒ 00:40:19.030 Samuel Roberts: Some of those tickets that are there are, which one? The department-based insights? That’s the project, right? So, like…
543 00:40:19.030 ⇒ 00:40:19.600 Pranav: Yup.
544 00:40:19.800 ⇒ 00:40:25.290 Samuel Roberts: some of the initial tickets here are, like, not… I don’t want to say done by any means, but,
545 00:40:26.430 ⇒ 00:40:28.790 Samuel Roberts: Where is it? That’s a weird order, okay.
546 00:40:28.790 ⇒ 00:40:39.410 Pranav: Because I’ll be working on some of this stuff today. I’ll just probably be pinging you today and tomorrow to just see, like, okay, where are we at currently on some of this stuff? Yeah.
547 00:40:39.410 ⇒ 00:40:45.919 Samuel Roberts: Let me… double-check the… there’s a… a branch, I think, that’s got some 8x8 work on it.
548 00:40:46.280 ⇒ 00:40:47.140 Pranav: Okay.
549 00:40:47.140 ⇒ 00:40:53.539 Samuel Roberts: it’s a little… it’s in, like, a very rough state, because it was just, like, I was throwing some scripts together to…
550 00:40:53.980 ⇒ 00:41:01.549 Samuel Roberts: it’s a little annoying, because 8x8’s annoying, but I can explain all that in more depth, or put together a README or something for you. Okay. But,
551 00:41:02.040 ⇒ 00:41:04.050 Samuel Roberts: It’s, it’s not…
552 00:41:04.380 ⇒ 00:41:13.420 Samuel Roberts: ready-to-run kind of thing, but, like, I know how to do it, and the scripts all… how I’ll have the information of, like, the process that I followed to get the transcripts into BQ.
553 00:41:13.710 ⇒ 00:41:15.320 Samuel Roberts: Gotcha. So.
554 00:41:15.320 ⇒ 00:41:21.000 Pranav: Maybe what actually makes sense here is if, casey, I don’t know what your capacity is for the rest of this week.
555 00:41:21.220 ⇒ 00:41:27.409 Pranav: Would you be able to, like, sync with Sam on this to kind of get a better understanding of 8x8?
556 00:41:27.540 ⇒ 00:41:28.970 Pranav: And the scripts he built?
557 00:41:30.250 ⇒ 00:41:31.570 Casie Aviles: Sure, yeah.
558 00:41:31.860 ⇒ 00:41:35.539 Pranav: Okay, I just don’t know if, like, you’re already kind of, like, at time with…
559 00:41:35.760 ⇒ 00:41:38.160 Pranav: There’s not gonna be this week.
560 00:41:40.450 ⇒ 00:41:43.069 Casie Aviles: No, I should have some capacity, yeah.
561 00:41:43.350 ⇒ 00:41:48.319 Pranav: Okay, okay, that sounds good then. Yeah, I’ll let you guys maybe…
562 00:41:48.500 ⇒ 00:42:08.490 Pranav: So then for the… I’ll handle the… what’s… what’s relevant for the co-pilot this week, and then Casey can probably figure out transcript this week. If transcript goes into next week, it’s not a huge deal, because we’re only focusing on co-pilot right now. Transcript was intentionally supposed to be, like, a spike that we just started in the beginning, but…
563 00:42:08.490 ⇒ 00:42:09.240 Samuel Roberts: Yeah, it was like…
564 00:42:09.240 ⇒ 00:42:10.530 Pranav: I thought it backload to cut it off.
565 00:42:11.160 ⇒ 00:42:11.670 Pranav: Yeah.
566 00:42:12.180 ⇒ 00:42:17.980 Samuel Roberts: Yeah, and some of that spike, like I said, is partially done, but I wanted to make sure the ticket was still there to, like, document the full…
567 00:42:18.570 ⇒ 00:42:21.059 Samuel Roberts: Like… Here’s how it works, kind of.
568 00:42:21.060 ⇒ 00:42:21.750 Pranav: Yeah.
569 00:42:21.970 ⇒ 00:42:23.379 Pranav: Yup, yep, makes sense.
570 00:42:23.380 ⇒ 00:42:23.920 Samuel Roberts: Okay.
571 00:42:24.120 ⇒ 00:42:27.389 Samuel Roberts: So I’m gonna… so there’s one ticket there…
572 00:42:28.320 ⇒ 00:42:35.679 Samuel Roberts: That’s document, transcript storage, keying, retention, PII constraints. I’m gonna assign that to me, because I need to put together that kind of README, I think.
573 00:42:35.680 ⇒ 00:42:36.310 Pranav: Okay.
574 00:42:36.570 ⇒ 00:42:38.690 Samuel Roberts: And then the next one is…
575 00:42:38.890 ⇒ 00:42:42.190 Samuel Roberts: Pull and validate one week of transcript, pilot group, client readout.
576 00:42:42.530 ⇒ 00:42:43.409 Pranav: Yeah, send that.
577 00:42:43.410 ⇒ 00:42:45.050 Samuel Roberts: That I’ll probably throw to Casey.
578 00:42:45.190 ⇒ 00:42:45.860 Pranav: Yeah.
579 00:42:46.900 ⇒ 00:42:55.339 Samuel Roberts: And then, the other thing about these tickets… so, casey, just for your,
580 00:42:55.510 ⇒ 00:43:02.619 Samuel Roberts: full view. Like, Panav had put together the project plan. I don’t know if you saw some of those documents in Notion or anything, Casey, but, like, Utam…
581 00:43:02.620 ⇒ 00:43:17.129 Samuel Roberts: kind of laid out a new project plan way to do it for the CSOs and SLs. And so, with that, there was, like, a technical plan, that technical plan then became these tickets, these tickets got loaded in, but one thing I did was try to tag them for, like.
582 00:43:17.360 ⇒ 00:43:19.889 Samuel Roberts: Is a human required to do this?
583 00:43:20.510 ⇒ 00:43:39.860 Samuel Roberts: So you’ll notice in some of the new tickets for ABC, there’s, like, an agent, maybe, and a human-required tag. So, like, certain things are like, okay, we need a person to go and, like… like, I forget what the other one was for Eden, but it was, like, service accounts, and, like, get the right things, you know, like, things we can’t just, like, kick off to a cloud agent.
584 00:43:40.080 ⇒ 00:43:47.800 Samuel Roberts: And then agent maybe is probably either using cursor in, like, an in-loop thing, or kicking off to a cloud agent.
585 00:43:49.890 ⇒ 00:43:56.230 Samuel Roberts: I think this is still kind of new, especially on ABC, where I don’t think we have cloud agent environments set up yet.
586 00:43:56.620 ⇒ 00:44:03.190 Samuel Roberts: But I just wanted to bring that to your attention, that, like, hopefully we can get to a point when, like, some of these things can be running in parallel in the cloud.
587 00:44:03.600 ⇒ 00:44:10.509 Samuel Roberts: But some of the things will be definitely, like, human involvement. A lot of decisions and, like.
588 00:44:10.630 ⇒ 00:44:20.649 Samuel Roberts: setup kind of stuff that way. But just keep that in mind when those labels are there, and we can try to… I want to push a little bit more, because, like, we have a lot of that stuff built for, like, the platform and everything.
589 00:44:20.890 ⇒ 00:44:23.159 Samuel Roberts: But… We don’t have a lot of, like.
590 00:44:23.550 ⇒ 00:44:31.630 Samuel Roberts: environment set up for Cursor to go pull the ABC repo and do all that. So I think I’d like to get to that point,
591 00:44:32.890 ⇒ 00:44:39.309 Samuel Roberts: But also know that, like, if it says Asian maybe, you might just be able to, like, feed the ticket to Cursor, and just answer a few questions.
592 00:44:39.550 ⇒ 00:44:41.259 Samuel Roberts: So, don’t, don’t, like…
593 00:44:42.700 ⇒ 00:44:48.109 Samuel Roberts: Don’t just, like, dive into a ticket, necessarily, but, like, look at those labels, and if there’s any questions or anything like that, definitely ping us, so…
594 00:44:49.050 ⇒ 00:44:54.610 Samuel Roberts: I tried to get as much context in the ticket this time as possible, too, so hopefully it’s, like, all there.
595 00:44:54.610 ⇒ 00:44:56.269 Pranav: Tickets are very, like, verbose, like, it’s…
596 00:44:56.270 ⇒ 00:45:06.640 Samuel Roberts: Yeah, and I had Cursor just, like, even when I was doing the sub-tickets this past time, it was, like, it made the sub-tickets with, like, no context, and just, like, referenced the parents, and I was like, no, no, no, dump everything into the ticket.
597 00:45:06.640 ⇒ 00:45:07.260 Pranav: Yeah.
598 00:45:07.260 ⇒ 00:45:13.769 Samuel Roberts: everything we need. So I wanna… I don’t think it’ll be, like, perfect, this first, kind of, project or two.
599 00:45:14.050 ⇒ 00:45:33.589 Samuel Roberts: Because we’ve still gotta figure out, like, getting that set up for ABC, because this is the one thing, like, I feel like if we were a product company, it’s easy to set this all up for, like, one repo, but since we’re, like, ABC’s got their own, our platform has their own, like, you know, Eden’s gonna have it, you know, it’s like, we have to do things a few more times, which is not bad, but we gotta figure out the good process for, like…
600 00:45:33.770 ⇒ 00:45:38.509 Samuel Roberts: spinning these things up quickly. Yeah. So this’ll be a good… this’ll be a good trial for, like.
601 00:45:38.640 ⇒ 00:45:53.100 Samuel Roberts: Are these tickets clear enough? What are the pre-tickets we have to put in there to, like, get a cloud agent set up for a new repo and stuff like that, and linearance? So I… I just want to bring that to your attention, Casey, so if you’re… as you’re grabbing tickets or designing tickets or whatever, like, keep that in mind.
602 00:45:53.350 ⇒ 00:46:02.619 Samuel Roberts: And if anything seems off with it, too, where it’s like, no, no, no, this is… the agent did a really bad job here, or I think a decision wasn’t outlined in the ticket, highlight that so we can…
603 00:46:03.500 ⇒ 00:46:04.709 Samuel Roberts: Fix that next time.
604 00:46:06.560 ⇒ 00:46:07.150 Casie Aviles: Okay.
605 00:46:08.860 ⇒ 00:46:11.780 Samuel Roberts: Yeah, I think besides that,
606 00:46:12.150 ⇒ 00:46:16.670 Samuel Roberts: What else is in here for that? Yeah, those are the two main ones there.
607 00:46:17.630 ⇒ 00:46:19.399 Samuel Roberts: Anything else to do? No, it’s all…
608 00:46:20.430 ⇒ 00:46:23.620 Samuel Roberts: past. I’m gonna have to change some of the dates on these, because I think they’re…
609 00:46:24.060 ⇒ 00:46:28.620 Samuel Roberts: They were from when I first put the… Docked together, not necessarily, like…
610 00:46:28.990 ⇒ 00:46:31.080 Samuel Roberts: But I wouldn’t stress about this too much.
611 00:46:31.950 ⇒ 00:46:32.720 Samuel Roberts: Okay.
612 00:46:33.550 ⇒ 00:46:36.720 Samuel Roberts: Same thing with the, estimates?
613 00:46:37.460 ⇒ 00:46:46.340 Samuel Roberts: it’s… I had a hard time being like, if an agent’s gonna do it, what do I assign this? So, like, we’ll have to play with that. I tried to make sure nothing was bigger than 5. If it was bigger than 5, I broke it down.
614 00:46:47.440 ⇒ 00:46:53.919 Samuel Roberts: But feel free to break it down another level if it’s too much, and we can talk that through for, you know, ticket-wise.
615 00:46:53.920 ⇒ 00:46:57.160 Pranav: And do you think of, like, the points as ours?
616 00:46:57.900 ⇒ 00:47:01.279 Samuel Roberts: That’s how… yeah, so we… we ha- I don’t know when that change happened.
617 00:47:01.420 ⇒ 00:47:04.320 Samuel Roberts: we had, like, a…
618 00:47:04.320 ⇒ 00:47:04.880 Casie Aviles: prompted.
619 00:47:05.110 ⇒ 00:47:08.549 Casie Aviles: One point equals 1R, I believe, yeah.
620 00:47:08.890 ⇒ 00:47:10.330 Samuel Roberts: It had been…
621 00:47:10.710 ⇒ 00:47:20.300 Samuel Roberts: not that… I don’t know when the timing of all that was relative to, like, when you started, but I remember we had to go through and, like, run a whole… Casey, I think, ran a script that was, like, redoing all the…
622 00:47:20.440 ⇒ 00:47:30.999 Samuel Roberts: estimates for everything to go to this… one ticket is one hour. And we had to switch it from, like, exponential Fibonacci to just linear, things like that, but…
623 00:47:31.000 ⇒ 00:47:31.630 Pranav: Yep.
624 00:47:31.630 ⇒ 00:47:41.450 Samuel Roberts: the way I want to think about it is, if anything’s bigger than 5, it’s too big a ticket. 5 might even be too big a ticket, but I just… I drew a line there, and if we need to make that smaller, definitely do.
625 00:47:42.990 ⇒ 00:47:49.060 Samuel Roberts: But this, again, might change as we throw some more stuff to agents and see what it’s capable of with the context it has.
626 00:47:49.450 ⇒ 00:47:50.100 Pranav: Yeah.
627 00:47:50.370 ⇒ 00:47:55.239 Samuel Roberts: So… Yeah, one ticket, one hour was the…
628 00:47:55.390 ⇒ 00:47:58.210 Samuel Roberts: Easiest way for everyone to just get on board with that.
629 00:47:58.410 ⇒ 00:48:03.149 Samuel Roberts: Instead of story points, and who can do how many story points, and stuff like that. Like, I think…
630 00:48:03.380 ⇒ 00:48:08.640 Samuel Roberts: I don’t mind that system, but I also think with AI now, it’s a whole different kind of…
631 00:48:08.970 ⇒ 00:48:09.450 Pranav: Yeah.
632 00:48:09.450 ⇒ 00:48:10.599 Samuel Roberts: Game to play.
633 00:48:11.300 ⇒ 00:48:13.239 Samuel Roberts: Yeah, and for me, it will, like…
634 00:48:13.400 ⇒ 00:48:20.460 Pranav: just see, like, what’s taking more time, what’s taking less time, like, for me to just be like, okay, points equals hours, it’s easy.
635 00:48:21.030 ⇒ 00:48:31.970 Samuel Roberts: Yeah, exactly, exactly. And if it’s off, we can correct, and it’s much easier to be like, oh, I thought it was this many story points, but it turned out to be this many story points, and it’s like, whatever. Not a game I want to play either.
636 00:48:31.970 ⇒ 00:48:32.580 Pranav: Yep.
637 00:48:32.840 ⇒ 00:48:34.340 Pranav: Okay, awesome. Cool.
638 00:48:34.490 ⇒ 00:48:35.810 Pranav: I feel pretty good, though.
639 00:48:36.180 ⇒ 00:48:36.770 Samuel Roberts: Okay.
640 00:48:40.030 ⇒ 00:48:56.920 Pranav: Yeah, so, once we merge that PR, I’ll just send over, an update to Janiece, or… actually, Casey, if you want to do that into the external channel, and just tag Janice, I know she’ll… and tag the Yvette and Janice, because they’ll be super excited, specifically for the… the weekly categorization.
641 00:48:57.800 ⇒ 00:48:58.440 Casie Aviles: Okay.
642 00:48:59.540 ⇒ 00:49:00.840 Casie Aviles: Are we…
643 00:49:00.840 ⇒ 00:49:01.519 Samuel Roberts: Sorry, go ahead.
644 00:49:02.050 ⇒ 00:49:02.679 Samuel Roberts: benefit yourself.
645 00:49:02.680 ⇒ 00:49:07.460 Casie Aviles: that… there’s… I don’t… there’s no, like, categories yet for, like.
646 00:49:07.840 ⇒ 00:49:11.930 Casie Aviles: Past 20… yeah, past last week, though.
647 00:49:12.710 ⇒ 00:49:19.429 Pranav: That’s fine. Yeah, I think they’ll just be excited to see that, you know, this dashboard’s there, and then,
648 00:49:19.560 ⇒ 00:49:23.740 Pranav: They’ll look at last week maybe a little bit longer, because all the…
649 00:49:23.950 ⇒ 00:49:31.700 Pranav: all they were able to see is just what I was able to show them during our meeting. But then also, I’ll let them know, like, on Friday, this will regenerate.
650 00:49:34.100 ⇒ 00:49:34.700 Casie Aviles: Got it.
651 00:49:35.380 ⇒ 00:49:39.439 Pranav: Yeah, and that’s how it works, right? Like, every Friday morning or something, like.
652 00:49:39.440 ⇒ 00:49:44.010 Samuel Roberts: That’s what I was about to ask. I don’t… I don’t think it’s… we have to run it ourselves right now, right, Casey?
653 00:49:44.810 ⇒ 00:49:47.490 Casie Aviles: Yeah, I just had Cursor do it last time.
654 00:49:48.120 ⇒ 00:49:53.130 Samuel Roberts: So we… I was gonna say, if… separate from these other two projects, we might want to ticket out a, like…
655 00:49:54.170 ⇒ 00:50:01.669 Samuel Roberts: process to have an agent do that on a weekly basis, because right now… or we just have a ticket every Friday to manually do it in Cursor, I don’t know.
656 00:50:01.830 ⇒ 00:50:03.309 Samuel Roberts: Whatever the… whatever the…
657 00:50:03.860 ⇒ 00:50:09.199 Samuel Roberts: you know, I don’t know how in-depth we want to go with building something that’s going to do that processing right now, but…
658 00:50:09.320 ⇒ 00:50:12.029 Samuel Roberts: if Cursor can do it, and you can just have a ticket every…
659 00:50:12.400 ⇒ 00:50:14.330 Samuel Roberts: Friday to run it, I think that’s fine.
660 00:50:14.330 ⇒ 00:50:19.090 Pranav: Yeah, we just have automated scripts to just do it and just click a few buttons, like, that’s totally fine for right now.
661 00:50:20.160 ⇒ 00:50:23.470 Samuel Roberts: Yeah, I would say then the idea would be put a skill together.
662 00:50:23.700 ⇒ 00:50:24.620 Samuel Roberts: Maybe.
663 00:50:26.130 ⇒ 00:50:26.800 Pranav: Yeah.
664 00:50:27.280 ⇒ 00:50:27.770 Pranav: Okay.
665 00:50:27.770 ⇒ 00:50:29.370 Samuel Roberts: And,
666 00:50:29.590 ⇒ 00:50:42.229 Samuel Roberts: we’ll just make a recurring ticket to run that skill every Friday. Yeah. Or maybe even eventually get Linear to run the skill? I don’t know. This is where the cloud stuff, I’m not sure where ABC’s gonna fall when it’s, like, all the access and everything, so…
667 00:50:42.230 ⇒ 00:50:48.459 Pranav: Yeah, I mean, how I think about it is just, like, right before my meeting, I’ll just run the skill, it’ll populate…
668 00:50:48.460 ⇒ 00:50:52.040 Samuel Roberts: Oh, that works, yeah, that’s right. Okay, so, Casey, if you can put the skill together…
669 00:50:52.160 ⇒ 00:50:57.150 Samuel Roberts: Get that into the… ABC repo, I guess?
670 00:50:59.080 ⇒ 00:50:59.730 Casie Aviles: Okay.
671 00:51:00.740 ⇒ 00:51:07.519 Samuel Roberts: Alright, we might as well keep the skill there, because that’s where it’s gonna be. I guess it doesn’t matter. It doesn’t really matter, we’re running it on data, it’s not like the code matters.
672 00:51:08.300 ⇒ 00:51:14.350 Samuel Roberts: you could put a scale in the ABC, in the… Client folder in the…
673 00:51:15.240 ⇒ 00:51:17.319 Samuel Roberts: Monorepo, or whatever we’re calling it now.
674 00:51:18.060 ⇒ 00:51:22.549 Samuel Roberts: Right, because it’s hitting BQ, I don’t think it needs to be in the ABC repo.
675 00:51:22.770 ⇒ 00:51:24.130 Samuel Roberts: This is our…
676 00:51:24.870 ⇒ 00:51:30.480 Samuel Roberts: you know, process. Once we build a thing that might do it, I would say it would go in the repo.
677 00:51:31.190 ⇒ 00:51:32.180 Samuel Roberts: Does that make sense?
678 00:51:35.390 ⇒ 00:51:44.329 Casie Aviles: Yeah, so it’s… currently, I’m working off of, like, the ABC report, not the one in the platform, or, like, in the… Okay.
679 00:51:44.920 ⇒ 00:51:56.010 Samuel Roberts: Okay, then I would say, yeah, that’s fine. Wherever it’s easiest to make that skill, just make sure that that’s, clearly documented, and that when Pranav opens up that repo, I guess, it’s easy to run.
680 00:51:56.010 ⇒ 00:51:56.430 Casie Aviles: That’s good.
681 00:51:56.430 ⇒ 00:52:02.989 Samuel Roberts: And then, once that’s done, I’d say make a recurring ticket for, like, Friday morning or something.
682 00:52:04.060 ⇒ 00:52:04.900 Casie Aviles: Oh, wow, yeah, that’s…
683 00:52:04.900 ⇒ 00:52:07.059 Samuel Roberts: That way we don’t, you know, lose track of that.
684 00:52:09.820 ⇒ 00:52:12.800 Samuel Roberts: And then if that starts to be, you know.
685 00:52:13.040 ⇒ 00:52:15.860 Samuel Roberts: Really useful, and we want to dig in and do…
686 00:52:16.310 ⇒ 00:52:21.010 Samuel Roberts: The kind of thing we talked about earlier, where we define the categories and really nail it.
687 00:52:22.580 ⇒ 00:52:27.270 Samuel Roberts: Down and do the other, like, category thing, we can figure that out, too.
688 00:52:28.900 ⇒ 00:52:34.330 Samuel Roberts: But this seems like enough right now, in my mind, but… Sound good?
689 00:52:35.990 ⇒ 00:52:36.710 Pranav: Perfect.
690 00:52:37.260 ⇒ 00:52:37.900 Samuel Roberts: Cool.
691 00:52:38.320 ⇒ 00:52:39.540 Samuel Roberts: Anything else?
692 00:52:39.710 ⇒ 00:52:41.380 Samuel Roberts: ABC-related, then?
693 00:52:46.000 ⇒ 00:52:47.049 Samuel Roberts: Feeling pretty good.
694 00:52:48.210 ⇒ 00:52:49.279 Pranav: Cool. Yeah.
695 00:52:49.280 ⇒ 00:52:50.360 Casie Aviles: That’s all I had.
696 00:52:50.730 ⇒ 00:52:51.480 Samuel Roberts: Alright.
697 00:52:54.020 ⇒ 00:52:54.550 Pranav: Alright.
698 00:52:54.550 ⇒ 00:52:54.980 Samuel Roberts: Thanks, Al.
699 00:52:54.980 ⇒ 00:52:56.379 Pranav: Talk to you guys later. See ya.
700 00:52:56.380 ⇒ 00:53:08.549 Samuel Roberts: Yeah. Do you want to talk Eden stuff at all, or are you good to plow through those for a little bit until… I just wasn’t sure, like, how resourcing-wise, like, I… I don’t know. It’s still in, like, a weird flux where, like, SL stuff, so I’m still trying to… I have a meeting tomorrow about…
701 00:53:09.410 ⇒ 00:53:15.489 Samuel Roberts: SL things, and I don’t know if that’s gonna include, like, resource allocation and stuff, so I just wanted to make sure that, like, you’re good for that for now until…
702 00:53:15.490 ⇒ 00:53:20.839 Pranav: Yeah, I mean, with Eden, I was just like, am I just gonna be the… the IC?
703 00:53:20.840 ⇒ 00:53:28.470 Samuel Roberts: So this is where I’m not really… I feel like if you can do it for now, like, go ahead, all the tickets are there, we can chat if you need to, but,
704 00:53:29.250 ⇒ 00:53:37.109 Samuel Roberts: I think if you’re… if you have the time and the capacity right now to do that, I think you have the most context, so it’s probably not worth handing off to someone else right now, so…
705 00:53:37.980 ⇒ 00:53:42.459 Pranav: Yeah, I think, at least for this week, it makes sense for me to do that. Yeah.
706 00:53:42.690 ⇒ 00:53:43.310 Samuel Roberts: Okay.
707 00:53:43.310 ⇒ 00:53:50.320 Pranav: Yeah, and then, yeah, I guess for your meeting tomorrow, you can just, like, mention how that’s all I’m thinking about. Actually, are we in that meeting together tomorrow?
708 00:53:50.320 ⇒ 00:53:56.499 Samuel Roberts: No, I think it’s just me and, actually, I don’t know, maybe you’re in there, I don’t know. I think it was Kayla and Utah.
709 00:53:56.500 ⇒ 00:53:57.139 Pranav: Oh, yeah, I’m gonna.
710 00:53:57.140 ⇒ 00:54:06.559 Samuel Roberts: They’re having, like, a meeting with each of the SLs, so I think they have one with Awash and Demi to just talk through, like… which I think is gonna be more, like, SL playbooks and stuff, too, but as part of that was also, like.
711 00:54:06.760 ⇒ 00:54:09.960 Samuel Roberts: Resourcing and allocation stuff, because, like…
712 00:54:10.470 ⇒ 00:54:15.219 Samuel Roberts: I haven’t had much insight into that yet, so I don’t know too much yet, so I’m hoping that that’ll clarify, because…
713 00:54:15.380 ⇒ 00:54:19.380 Samuel Roberts: as we’re structuring this a bit more, I want to make sure I’m… keeping us…
714 00:54:19.540 ⇒ 00:54:24.699 Samuel Roberts: Yeah. Yeah, I don’t want to get to a point where you’re like, oh shit, there’s too much Eden and no one else has any context kind of thing, you know?
715 00:54:24.820 ⇒ 00:54:25.680 Pranav: Yep, yep.
716 00:54:26.230 ⇒ 00:54:28.670 Samuel Roberts: Yeah, if you’re good, just keep running with that.
717 00:54:28.840 ⇒ 00:54:34.830 Samuel Roberts: let me know if there’s anything I can do ticket-wise, or… or architecturally, or whatever, but… Yeah.
718 00:54:34.830 ⇒ 00:54:36.160 Pranav: Thank you.
719 00:54:36.360 ⇒ 00:54:37.340 Samuel Roberts: Cool, alright.
720 00:54:38.750 ⇒ 00:54:39.770 Samuel Roberts: Thanks to y’all later.
721 00:54:39.960 ⇒ 00:54:41.610 Pranav: Cool. Talk soon, guys. See ya.
722 00:54:42.090 ⇒ 00:54:42.830 Casie Aviles: Thank you.