Meeting Title: Internal AI Team Thanksgiving Planning Date: 2025-11-26 Meeting participants: Gabriel Lam, Mustafa Raja, Samuel Roberts, Casie Aviles
WEBVTT
1 00:00:28.590 ⇒ 00:00:29.640 Mustafa Raja: Ayy.
2 00:00:34.670 ⇒ 00:00:35.600 Gabriel Lam: Hello?
3 00:00:36.970 ⇒ 00:00:37.659 Mustafa Raja: How are you?
4 00:00:38.100 ⇒ 00:00:40.409 Gabriel Lam: I’m doing well, how are you?
5 00:00:40.980 ⇒ 00:00:42.270 Mustafa Raja: Yeah, doing good, doing good.
6 00:00:43.540 ⇒ 00:00:54.399 Mustafa Raja: So, for the error, the way… the reason it’s, it’s coming up is, for this particular meeting, it cannot find…
7 00:00:54.600 ⇒ 00:00:55.639 Gabriel Lam: You can’t find a…
8 00:00:55.640 ⇒ 00:00:56.230 Mustafa Raja: tea.
9 00:00:56.960 ⇒ 00:00:57.909 Gabriel Lam: A proper what?
10 00:00:58.500 ⇒ 00:00:59.769 Mustafa Raja: a proper team.
11 00:01:00.460 ⇒ 00:01:01.560 Gabriel Lam: I see.
12 00:01:02.480 ⇒ 00:01:06.870 Mustafa Raja: So, what it does is it does not return the team with the ticket, and that…
13 00:01:07.250 ⇒ 00:01:10.070 Mustafa Raja: Gets caught by the structure that.
14 00:01:10.070 ⇒ 00:01:10.430 Gabriel Lam: I see.
15 00:01:10.430 ⇒ 00:01:18.359 Mustafa Raja: it shouldn’t be the way that it is. So, maybe what I’m trying to do is maybe adjust some things to ask it to…
16 00:01:18.510 ⇒ 00:01:24.669 Mustafa Raja: You know, if it doesn’t find proper match during exact match, maybe the next possible one.
17 00:01:24.790 ⇒ 00:01:32.770 Mustafa Raja: But you should always return team, because otherwise, I guess I’ll have to change a lot of structure on how things are going.
18 00:01:33.180 ⇒ 00:01:36.300 Mustafa Raja: Maybe that… that’s an easier… easier fix.
19 00:01:36.970 ⇒ 00:01:37.810 Gabriel Lam: Hmm.
20 00:01:40.300 ⇒ 00:01:45.390 Mustafa Raja: Yeah, with that, I’m also going to push, what we discussed yesterday?
21 00:01:45.650 ⇒ 00:01:49.179 Mustafa Raja: Okay. And that comes down to transcript request also.
22 00:01:49.530 ⇒ 00:01:50.260 Gabriel Lam: Awesome.
23 00:01:52.350 ⇒ 00:01:52.880 Mustafa Raja: It’s.
24 00:01:52.880 ⇒ 00:01:54.459 Samuel Roberts: That bug from before?
25 00:01:55.930 ⇒ 00:01:56.550 Mustafa Raja: Yeah?
26 00:02:00.190 ⇒ 00:02:02.790 Gabriel Lam: I have a question, is there a reason why…
27 00:02:02.980 ⇒ 00:02:11.319 Gabriel Lam: You know, let’s say when you first click into a meeting, the summary is not the latest prompt, it’s the old prompt, and you have to press trigger.
28 00:02:11.320 ⇒ 00:02:15.980 Mustafa Raja: Oh, it might be because, yeah, yeah, it might be because the N10 one is still triggering.
29 00:02:15.980 ⇒ 00:02:17.139 Gabriel Lam: Got it. Not, not…
30 00:02:17.140 ⇒ 00:02:17.779 Samuel Roberts: Oh, okay.
31 00:02:17.780 ⇒ 00:02:18.210 Gabriel Lam: Makes sense.
32 00:02:18.210 ⇒ 00:02:21.340 Mustafa Raja: Let me turn that off.
33 00:02:22.290 ⇒ 00:02:22.890 Samuel Roberts: Green.
34 00:02:24.840 ⇒ 00:02:29.090 Mustafa Raja: Yeah, didn’t… I forgot… I forgot to turn it off.
35 00:02:33.080 ⇒ 00:02:34.300 Mustafa Raja: Okay…
36 00:02:40.830 ⇒ 00:02:45.619 Mustafa Raja: Yeah, if I turn it off, I’m wondering if I also need to, you know, adjust the windmill code.
37 00:02:48.540 ⇒ 00:02:50.629 Samuel Roberts: Is the window code fire that webhook?
38 00:02:51.230 ⇒ 00:02:53.869 Mustafa Raja: Yeah, windmill calls this one.
39 00:02:54.370 ⇒ 00:02:56.699 Samuel Roberts: Yeah, I mean, at some point we should.
40 00:02:57.220 ⇒ 00:02:58.220 Samuel Roberts: Probably worth it.
41 00:02:59.290 ⇒ 00:03:00.629 Mustafa Raja: If you’re going to want to give…
42 00:03:00.790 ⇒ 00:03:04.520 Mustafa Raja: If Superbase calls it, or if Windmill calls it.
43 00:03:06.970 ⇒ 00:03:10.970 Mustafa Raja: Windmill has proper tri-catches, so it shouldn’t, you know…
44 00:03:11.440 ⇒ 00:03:12.300 Samuel Roberts: Oh, okay.
45 00:03:12.600 ⇒ 00:03:13.340 Mustafa Raja: Stuff.
46 00:03:13.680 ⇒ 00:03:14.510 Mustafa Raja: Whoa.
47 00:03:15.020 ⇒ 00:03:15.900 Mustafa Raja: Yeah.
48 00:03:20.670 ⇒ 00:03:23.110 Mustafa Raja: I’m just quickly double-checking which one called it.
49 00:03:32.120 ⇒ 00:03:33.200 Samuel Roberts: Yeah.
50 00:03:34.440 ⇒ 00:03:41.570 Mustafa Raja: It’s pretty… okay. Yeah, I’m just deactivating it, and hopefully even mill does not break.
51 00:03:43.040 ⇒ 00:03:45.460 Samuel Roberts: Okay, yeah, definitely check that later then, yeah.
52 00:03:45.460 ⇒ 00:03:47.870 Mustafa Raja: Yeah, I’ll definitely take a look at that.
53 00:03:48.220 ⇒ 00:03:48.820 Samuel Roberts: Okay.
54 00:03:51.330 ⇒ 00:03:52.480 Mustafa Raja: Okay.
55 00:03:55.310 ⇒ 00:04:05.180 Gabriel Lam: Yeah, I know, Sam, you have to head out soon, so I wanted to just take some time to go over what we can do the next… at least two days, if not until the next week.
56 00:04:05.620 ⇒ 00:04:06.580 Gabriel Lam: Yeah.
57 00:04:06.580 ⇒ 00:04:07.060 Samuel Roberts: That’s true.
58 00:04:07.060 ⇒ 00:04:09.590 Gabriel Lam: specifically migration stuff, because I know…
59 00:04:10.760 ⇒ 00:04:14.029 Gabriel Lam: You… you know much more than I do there, so…
60 00:04:14.220 ⇒ 00:04:15.490 Samuel Roberts: Sure, okay.
61 00:04:15.640 ⇒ 00:04:24.410 Samuel Roberts: I’m trying to think where to begin with that. So I guess… Well, I don’t know.
62 00:04:24.890 ⇒ 00:04:29.239 Samuel Roberts: Yeah, it’s gonna be hard next week, isn’t it? I wasn’t thinking I should have prepped a little more.
63 00:04:29.670 ⇒ 00:04:30.430 Gabriel Lam: You’re good.
64 00:04:32.260 ⇒ 00:04:32.740 Samuel Roberts: this hour.
65 00:04:32.740 ⇒ 00:04:33.660 Gabriel Lam: lots of prep.
66 00:04:33.960 ⇒ 00:04:47.629 Samuel Roberts: Exactly, okay. So let’s, the… We talked about… I’m trying to remember what… the new client hub stuff.
67 00:04:48.390 ⇒ 00:04:51.929 Samuel Roberts: we kind of talked about at the end.
68 00:04:56.880 ⇒ 00:05:00.599 Samuel Roberts: I’ve got this up now, let me maybe share my screen.
69 00:05:05.240 ⇒ 00:05:13.599 Samuel Roberts: Sorry, I just had to reboot my computer, like, 30 minutes ago, so I’m still trying to, like, get things back open, and in their state, I’m used to working with them on, so one second.
70 00:05:25.250 ⇒ 00:05:28.540 Samuel Roberts: So this is the notion that we had talked through a little bit, right?
71 00:05:30.040 ⇒ 00:05:36.140 Samuel Roberts: So I think the plan of attack… is…
72 00:05:36.720 ⇒ 00:05:41.950 Samuel Roberts: And guys, step in, correct me if I’m… Wrong here, but…
73 00:05:46.600 ⇒ 00:05:51.089 Samuel Roberts: Was this stuff we wrote yesterday, or was this from the… was this stuff I had in here before?
74 00:05:51.640 ⇒ 00:05:53.620 Samuel Roberts: Or 2 days ago, when we did it.
75 00:05:54.010 ⇒ 00:05:55.810 Gabriel Lam: I think this was 2 days ago.
76 00:05:56.780 ⇒ 00:05:57.370 Samuel Roberts: Okay.
77 00:05:58.640 ⇒ 00:06:02.230 Samuel Roberts: So we want to do the ingestion pipeline. So basically, we need to create a second
78 00:06:02.950 ⇒ 00:06:04.789 Samuel Roberts: Like, a duplicate copy of…
79 00:06:08.790 ⇒ 00:06:13.439 Samuel Roberts: Pipeline, and we’re gonna go to that… a new generic table, right?
80 00:06:14.690 ⇒ 00:06:15.290 Casie Aviles: Yeah.
81 00:06:15.650 ⇒ 00:06:17.539 Casie Aviles: the master table, I believe.
82 00:06:17.540 ⇒ 00:06:21.720 Samuel Roberts: Yeah, so then it’s probably, like, a master raw and embedding for each one, is that what it was?
83 00:06:23.150 ⇒ 00:06:23.830 Samuel Roberts: Okay.
84 00:06:31.410 ⇒ 00:06:33.029 Casie Aviles: And then we need to update.
85 00:06:33.100 ⇒ 00:06:34.090 Samuel Roberts: Sorry, go ahead.
86 00:06:35.440 ⇒ 00:06:40.070 Casie Aviles: Yeah, just, also, like, being able to dynamically…
87 00:06:40.940 ⇒ 00:06:45.599 Casie Aviles: Get the clients, instead of just having, like, a config.yaml file.
88 00:06:47.560 ⇒ 00:06:52.619 Samuel Roberts: Right, right, okay. So, let me just… Stop down here, maybe, then.
89 00:06:53.620 ⇒ 00:06:55.439 Samuel Roberts: Okay, so what is today?
90 00:06:57.300 ⇒ 00:07:16.570 Samuel Roberts: Okay, so we need to, New pipeline for… Master… handle of… Alliance… Slack and Zoom.
91 00:07:23.200 ⇒ 00:07:29.660 Samuel Roberts: logic to fetch clients from Supabase.
92 00:07:30.350 ⇒ 00:07:33.049 Samuel Roberts: Right? That’s in Dagster, for Daxter.
93 00:07:34.990 ⇒ 00:07:37.080 Samuel Roberts: So we’re not hard-coding it, right?
94 00:07:38.150 ⇒ 00:07:38.880 Casie Aviles: Yes.
95 00:07:38.880 ⇒ 00:07:56.630 Samuel Roberts: And then, we need to update… new… Supabase Vector Query… function… to take client… ID, I guess, as… oops.
96 00:07:56.910 ⇒ 00:07:57.870 Gabriel Lam: Oops, sorry.
97 00:07:58.230 ⇒ 00:08:00.359 Samuel Roberts: You’re fine. As an input.
98 00:08:00.660 ⇒ 00:08:04.840 Samuel Roberts: So those were the, kind of, the first things to get this, like, new data set up right.
99 00:08:07.000 ⇒ 00:08:08.680 Samuel Roberts: Am I missing anything there?
100 00:08:09.060 ⇒ 00:08:19.429 Samuel Roberts: Oh, I’m missing one big thing, which is to migrate Or, yeah, copy… or I’ll say backfill.
101 00:08:20.720 ⇒ 00:08:36.769 Samuel Roberts: data from… individual tables… into… master client tables, right? Eventually. That’s probably not the first priority yet, right?
102 00:08:38.400 ⇒ 00:08:42.859 Samuel Roberts: Because we want to make sure that everything’s working, and then we can fill the data. Who shouldn’t necessarily waste time doing that?
103 00:08:45.570 ⇒ 00:08:49.840 Samuel Roberts: Yeah, that’s the next… that’s the next step.
104 00:08:50.360 ⇒ 00:08:54.050 Samuel Roberts: I don’t know who typed that, because I don’t see a name there, but…
105 00:08:55.500 ⇒ 00:08:58.460 Samuel Roberts: So then, yeah, then the thing is to…
106 00:08:59.250 ⇒ 00:09:02.569 Samuel Roberts: create a generic client hub agent.
107 00:09:04.450 ⇒ 00:09:08.220 Samuel Roberts: That takes a client as an input, fetches from the right tables.
108 00:09:09.970 ⇒ 00:09:13.140 Samuel Roberts: But it has the same… Sorry?
109 00:09:13.850 ⇒ 00:09:15.650 Casie Aviles: Yeah, I just typed it, feel free.
110 00:09:15.650 ⇒ 00:09:25.679 Samuel Roberts: Yeah, yeah, okay, so I think that’s… I think that’s really, yeah, that’s all… that’s the next, that’s the next big step. I was trying to think of it in, like, stages, so this was, like, to get the data set up, ready to go for this new agent.
111 00:09:28.040 ⇒ 00:09:32.460 Samuel Roberts: the agent probably needs a little more…
112 00:09:32.980 ⇒ 00:09:36.159 Samuel Roberts: This is… this is kind of the rough… That’s not getting anymore.
113 00:09:36.910 ⇒ 00:09:40.370 Samuel Roberts: This is kind of… kind of Ohio Highlander? What? Okay, hold on.
114 00:09:41.110 ⇒ 00:09:42.519 Samuel Roberts: This is kind of the rough…
115 00:09:42.860 ⇒ 00:09:45.550 Samuel Roberts: way I was thinking about it,
116 00:09:47.120 ⇒ 00:09:50.240 Samuel Roberts: From way back when. This was what I’d thrown together.
117 00:09:51.090 ⇒ 00:09:54.710 Samuel Roberts: This is basically my,
118 00:09:55.120 ⇒ 00:10:00.029 Samuel Roberts: understanding of how the current N8 N flow worked, and trying to just move that to be more generic.
119 00:10:01.030 ⇒ 00:10:05.410 Samuel Roberts: So, I don’t know… do we have the…
120 00:10:07.650 ⇒ 00:10:12.629 Samuel Roberts: Did we talk through the… besides the pipeline the other day? Refresh my memory here.
121 00:10:12.630 ⇒ 00:10:18.060 Gabriel Lam: I think it was just the pipeline and making sure that we Had a new pipeline?
122 00:10:18.130 ⇒ 00:10:20.500 Samuel Roberts: Yes, that went to a…
123 00:10:20.740 ⇒ 00:10:22.300 Gabriel Lam: like, master table?
124 00:10:22.810 ⇒ 00:10:23.410 Samuel Roberts: Yep.
125 00:10:23.410 ⇒ 00:10:29.849 Gabriel Lam: But also… Not to touch the existing one, so that it could run in parallel.
126 00:10:30.260 ⇒ 00:10:32.330 Samuel Roberts: Okay, so that’s, that’s this, let’s this…
127 00:10:41.100 ⇒ 00:10:43.209 Casie Aviles: Is it just me, or… I think it’s different.
128 00:10:43.210 ⇒ 00:10:43.800 Mustafa Raja: Yeah…
129 00:10:43.800 ⇒ 00:10:44.740 Casie Aviles: It’s choppy.
130 00:10:44.990 ⇒ 00:10:46.609 Gabriel Lam: I think it’s Sam.
131 00:10:46.610 ⇒ 00:10:50.720 Samuel Roberts: New pipeline… My internet connection is unstable, there it goes.
132 00:10:50.720 ⇒ 00:10:52.210 Gabriel Lam: There we go.
133 00:10:52.960 ⇒ 00:10:54.119 Gabriel Lam: We heard you, though.
134 00:10:55.360 ⇒ 00:10:57.509 Samuel Roberts: Okay, hold on one sec.
135 00:11:05.250 ⇒ 00:11:06.130 Samuel Roberts: Can y’all hear me?
136 00:11:06.130 ⇒ 00:11:08.270 Mustafa Raja: Casey? Yeah, yeah, yeah.
137 00:11:09.590 ⇒ 00:11:22.840 Mustafa Raja: Also, Casey, correct me if I’m wrong, but I’m… but I think the way the pipeline is currently set up is we… we remove all of the data from the table, and then we repopulate it with all of the data that… that is right, right?
138 00:11:24.360 ⇒ 00:11:26.260 Casie Aviles: Yeah, that’s pretty much how it works.
139 00:11:27.440 ⇒ 00:11:29.690 Samuel Roberts: Wait, we removed the whole table for what?
140 00:11:29.690 ⇒ 00:11:30.550 Mustafa Raja: Yeah.
141 00:11:31.890 ⇒ 00:11:35.570 Mustafa Raja: For Zoom, let me see for Slack…
142 00:11:36.600 ⇒ 00:11:37.140 Casie Aviles: That’s weird.
143 00:11:39.330 ⇒ 00:11:41.410 Samuel Roberts: we remove… all the…
144 00:11:43.010 ⇒ 00:11:44.629 Mustafa Raja: Yeah, all the current status.
145 00:11:46.180 ⇒ 00:11:54.539 Casie Aviles: It’s… it’s… it’s not really deleting it, like, it’s just the… the copy that we filtered out of, like, the main recordings table.
146 00:11:55.620 ⇒ 00:11:56.410 Samuel Roberts: Oh, okay.
147 00:11:56.410 ⇒ 00:11:58.889 Casie Aviles: And, yeah, and then we…
148 00:11:59.470 ⇒ 00:12:05.650 Casie Aviles: I believe we’re redoing, like, the embedding, so it’s a bit expensive that way, though, for now.
149 00:12:06.700 ⇒ 00:12:07.130 Mustafa Raja: Hmm.
150 00:12:07.130 ⇒ 00:12:07.980 Casie Aviles: I guess.
151 00:12:08.540 ⇒ 00:12:11.769 Mustafa Raja: So each client table, gets refilled, right?
152 00:12:16.850 ⇒ 00:12:20.100 Samuel Roberts: Wait, when is that? I’m confused. So when a new meeting comes in, that’s what happens?
153 00:12:20.310 ⇒ 00:12:36.469 Mustafa Raja: No, no, no. It happens… Daxter does it when, Daxter does it on schedule, I believe, right? Yes. On schedule, what happens is, we have these tables, client… client… whatever the client name, and then the meetings, right?
154 00:12:36.470 ⇒ 00:12:36.930 Samuel Roberts: Yeah.
155 00:12:36.930 ⇒ 00:12:42.300 Mustafa Raja: Those tables are deleted, and then those are created again with whatever the new data we have.
156 00:12:47.890 ⇒ 00:12:50.800 Mustafa Raja: Does that make sense? I can share my screen, we can take a.
157 00:12:50.800 ⇒ 00:12:53.489 Samuel Roberts: Yeah, go ahead and share, because I’m not sure I’m following, yeah.
158 00:12:56.800 ⇒ 00:13:04.909 Mustafa Raja: Okay, so… yeah, I think it’s happening over here, create or re-override sub-table. So we drop the table.
159 00:13:05.120 ⇒ 00:13:14.059 Mustafa Raja: And then we create a new one, and then we have this query. There’s a query also…
160 00:13:16.660 ⇒ 00:13:20.129 Mustafa Raja: copy acceptable, yeah. What we are doing is, we are…
161 00:13:20.490 ⇒ 00:13:31.909 Mustafa Raja: So it’s a folder that really is the name of the meeting, and then, if the… if the folder has the client name in it, we fetch all of the meetings, and then push.
162 00:13:32.450 ⇒ 00:13:33.600 Mustafa Raja: To the sub-table.
163 00:13:35.670 ⇒ 00:13:37.730 Samuel Roberts: Populous subtle, that’s what populates.
164 00:13:37.830 ⇒ 00:13:38.820 Samuel Roberts: Okay.
165 00:13:39.580 ⇒ 00:13:44.260 Mustafa Raja: Original table would be, I believe, the… where all of the meetings are.
166 00:13:47.170 ⇒ 00:13:51.949 Samuel Roberts: Oh, and wait, when is this function run, populate subtle with SQL?
167 00:13:53.840 ⇒ 00:14:00.470 Mustafa Raja: Let’s go to that step cloud… Kishi, do you know?
168 00:14:00.600 ⇒ 00:14:02.699 Mustafa Raja: What the scheduling is like for this?
169 00:14:03.810 ⇒ 00:14:05.299 Casie Aviles: It’s every 5 hours.
170 00:14:05.920 ⇒ 00:14:08.730 Samuel Roberts: So we’re re-embedding every meeting every 5 hours?
171 00:14:10.220 ⇒ 00:14:10.720 Casie Aviles: Meh.
172 00:14:10.720 ⇒ 00:14:12.429 Mustafa Raja: I think, I’m not 100% sure.
173 00:14:12.870 ⇒ 00:14:16.159 Mustafa Raja: But the code looks like to be doing that. Okay.
174 00:14:17.370 ⇒ 00:14:20.900 Samuel Roberts: I thought it was just adding new meetings and doing it.
175 00:14:26.030 ⇒ 00:14:26.610 Mustafa Raja: Okay.
176 00:14:26.610 ⇒ 00:14:31.480 Samuel Roberts: I guess I’m wondering why that’s the… why is it set up that way? Like, was there,
177 00:14:32.190 ⇒ 00:14:34.870 Samuel Roberts: A reason we had to drop and redo the tables?
178 00:14:40.730 ⇒ 00:14:42.550 Mustafa Raja: I don’t really have an idea.
179 00:14:43.130 ⇒ 00:14:44.690 Casie Aviles: Yeah. Okay.
180 00:14:44.690 ⇒ 00:14:46.270 Mustafa Raja: Yeah, it’s this one, I believe.
181 00:14:46.270 ⇒ 00:14:50.929 Samuel Roberts: Yeah, okay, yeah, 1AM, 5 AM… 10 a.m. Okay.
182 00:14:53.080 ⇒ 00:14:53.560 Samuel Roberts: Anyone.
183 00:14:53.560 ⇒ 00:14:54.610 Mustafa Raja: triple check, too.
184 00:15:00.460 ⇒ 00:15:02.030 Samuel Roberts: Okay, so…
185 00:15:09.120 ⇒ 00:15:09.910 Samuel Roberts: Oh.
186 00:15:15.860 ⇒ 00:15:17.240 Samuel Roberts: Okay.
187 00:15:21.950 ⇒ 00:15:24.819 Mustafa Raja: Yeah, it’s this one, yeah. Embed Zoom transcription.
188 00:15:26.000 ⇒ 00:15:26.820 Samuel Roberts: Okay.
189 00:15:30.510 ⇒ 00:15:32.699 Samuel Roberts: Interesting, I didn’t realize this.
190 00:15:33.130 ⇒ 00:15:39.250 Samuel Roberts: 30 minutes to… I mean, we could just… Go ahead.
191 00:15:39.910 ⇒ 00:15:45.610 Mustafa Raja: I guess for now, we could keep it that way, or maybe change it, I don’t know.
192 00:15:49.880 ⇒ 00:15:52.250 Samuel Roberts: Yeah, it takes 30 minutes.
193 00:15:52.650 ⇒ 00:15:56.489 Samuel Roberts: Yeah, and it’s just re-embedding everything, is that it?
194 00:15:56.600 ⇒ 00:16:00.110 Samuel Roberts: Or, like, including new meetings, but it’s also doing old meetings.
195 00:16:01.280 ⇒ 00:16:10.530 Mustafa Raja: Yeah, yeah, the code… just the code snippet, it looks like it drops the previous table. Let’s actually…
196 00:16:10.740 ⇒ 00:16:14.319 Mustafa Raja: Take a look at how the tables look like, also.
197 00:16:14.320 ⇒ 00:16:17.789 Samuel Roberts: Yeah, yeah, okay, I’m sorry, I knew what Dave was doing all that.
198 00:16:19.620 ⇒ 00:16:25.970 Mustafa Raja: Yeah, I just, while we were talking about this stuff, I… I thought maybe this…
199 00:16:25.970 ⇒ 00:16:26.600 Samuel Roberts: Yeah.
200 00:16:26.600 ⇒ 00:16:28.700 Mustafa Raja: It’d be something to bring up.
201 00:16:29.230 ⇒ 00:16:29.880 Samuel Roberts: Thank you, yeah.
202 00:16:29.880 ⇒ 00:16:30.490 Mustafa Raja: Nope.
203 00:16:33.410 ⇒ 00:16:34.310 Mustafa Raja: I did.
204 00:16:34.980 ⇒ 00:16:38.770 Mustafa Raja: Okay, let’s see, actually, from the table name…
205 00:16:39.200 ⇒ 00:16:42.840 Mustafa Raja: which sort of tables this… these are. Sorry.
206 00:16:46.170 ⇒ 00:16:52.099 Mustafa Raja: So, table name… Table name.
207 00:16:52.420 ⇒ 00:16:55.209 Mustafa Raja: Okay, Zoom recording files…
208 00:16:55.730 ⇒ 00:16:57.800 Samuel Roberts: So that’s the… that’s the table…
209 00:16:57.800 ⇒ 00:16:58.879 Mustafa Raja: Does the main thing.
210 00:16:59.150 ⇒ 00:16:59.820 Samuel Roberts: Okay.
211 00:17:00.350 ⇒ 00:17:06.089 Mustafa Raja: And then… Subtable name… Get subtabl name…
212 00:17:06.099 ⇒ 00:17:08.859 Samuel Roberts: If it’s printing all those things, can we see those outputs in Dagster?
213 00:17:08.859 ⇒ 00:17:12.179 Mustafa Raja: Yeah, so it does this for the raw.
214 00:17:12.930 ⇒ 00:17:13.349 Samuel Roberts: Okay.
215 00:17:13.359 ⇒ 00:17:24.649 Mustafa Raja: And for these… these tables, yeah, so it puts… puts the meetings in the raw tables, and then maybe the embeddings tables are generated from them, these ones. Yep.
216 00:17:25.839 ⇒ 00:17:26.799 Mustafa Raja: I think.
217 00:17:27.109 ⇒ 00:17:28.679 Mustafa Raja: Let’s actually…
218 00:17:28.680 ⇒ 00:17:30.280 Samuel Roberts: Every 5 hours for every meeting?
219 00:17:31.970 ⇒ 00:17:34.870 Mustafa Raja: Let’s actually see if we can…
220 00:17:35.050 ⇒ 00:17:38.589 Samuel Roberts: Yeah, can we see the Dagster, like, output log? Is there…
221 00:17:38.590 ⇒ 00:17:39.270 Mustafa Raja: Okay.
222 00:17:45.360 ⇒ 00:17:48.579 Casie Aviles: You can… SDD out, I think.
223 00:17:49.910 ⇒ 00:17:51.630 Mustafa Raja: Oh, this one, sorry.
224 00:17:53.720 ⇒ 00:17:54.769 Mustafa Raja: Select a step.
225 00:17:54.770 ⇒ 00:17:57.569 Casie Aviles: I’ll be able to select a step.
226 00:17:57.570 ⇒ 00:17:58.170 Mustafa Raja: on this one?
227 00:17:58.170 ⇒ 00:17:58.870 Casie Aviles: this…
228 00:17:59.130 ⇒ 00:18:00.810 Samuel Roberts: Yeah, there you go, okay, cool.
229 00:18:01.630 ⇒ 00:18:03.409 Samuel Roberts: So.
230 00:18:03.410 ⇒ 00:18:05.240 Mustafa Raja: Let’s actually go to the top.
231 00:18:05.240 ⇒ 00:18:06.470 Samuel Roberts: Yeah, yeah.
232 00:18:07.380 ⇒ 00:18:08.160 Mustafa Raja: Okay.
233 00:18:09.310 ⇒ 00:18:10.030 Mustafa Raja: So…
234 00:18:10.030 ⇒ 00:18:11.170 Samuel Roberts: which contains…
235 00:18:12.520 ⇒ 00:18:14.009 Mustafa Raja: Virtual records.
236 00:18:15.900 ⇒ 00:18:19.740 Mustafa Raja: Okay, so it started with, hype access, and it…
237 00:18:24.450 ⇒ 00:18:32.010 Samuel Roberts: Yeah, dropping table raw, creating table… What? Hmm… Okay, is this…
238 00:18:32.250 ⇒ 00:18:35.769 Samuel Roberts: So is it doing anything with, like… so is it embedding with all the…
239 00:18:36.800 ⇒ 00:18:38.859 Samuel Roberts: I’m just trying to understand why we’re doing that.
240 00:18:40.910 ⇒ 00:18:41.320 Casie Aviles: Yeah.
241 00:18:41.320 ⇒ 00:18:41.870 Samuel Roberts: because…
242 00:18:41.870 ⇒ 00:18:45.050 Casie Aviles: We definitely have to update that.
243 00:18:45.050 ⇒ 00:18:45.539 Samuel Roberts: Yeah, that was true.
244 00:18:45.540 ⇒ 00:18:46.820 Casie Aviles: done out of.
245 00:18:47.200 ⇒ 00:18:51.130 Casie Aviles: Like, to get something out quickly, to be honest. Okay.
246 00:18:51.500 ⇒ 00:18:53.510 Samuel Roberts: Okay, okay, if that’s the case, I’m just making sure.
247 00:18:53.510 ⇒ 00:18:54.180 Casie Aviles: design.
248 00:18:54.390 ⇒ 00:19:00.770 Samuel Roberts: No, no, yeah, it’s fine. I’m just making sure it’s not, like, having more of the context changes the embedding or something, I don’t know.
249 00:19:01.900 ⇒ 00:19:06.150 Mustafa Raja: Yeah, I just want to make sure if, if this table
250 00:19:06.330 ⇒ 00:19:12.069 Mustafa Raja: If this table is also, you know, being deleted and then regenerated.
251 00:19:12.950 ⇒ 00:19:18.360 Mustafa Raja: It just says creating, and then generating. I don’t see it drop one like I do.
252 00:19:18.750 ⇒ 00:19:21.260 Samuel Roberts: Yeah, but that could just be a missing message, you know.
253 00:19:21.260 ⇒ 00:19:24.169 Casie Aviles: Yeah, that’s probably just a missing output flow.
254 00:19:24.170 ⇒ 00:19:25.070 Samuel Roberts: Yeah…
255 00:19:25.220 ⇒ 00:19:26.000 Mustafa Raja: Okay.
256 00:19:27.100 ⇒ 00:19:27.470 Samuel Roberts: Okay.
257 00:19:27.470 ⇒ 00:19:30.350 Mustafa Raja: Let me actually just… Pick this.
258 00:19:31.240 ⇒ 00:19:39.479 Mustafa Raja: Go here, and let’s see… create… Okay, we created, and then…
259 00:19:42.690 ⇒ 00:19:43.260 Samuel Roberts: So wait, when…
260 00:19:43.260 ⇒ 00:19:45.240 Mustafa Raja: Creative not exists.
261 00:19:46.620 ⇒ 00:19:49.429 Mustafa Raja: So we create this only if it doesn’t exist, right?
262 00:19:49.430 ⇒ 00:19:53.370 Samuel Roberts: Okay, we’re not… so we’re not recreating the table for the embeddings.
263 00:19:54.080 ⇒ 00:19:58.790 Mustafa Raja: Yes, create table if not exists. Okay. And then let’s see what happens…
264 00:20:00.240 ⇒ 00:20:03.760 Mustafa Raja: Okay, we generate this, and then generate embeddings.
265 00:20:04.700 ⇒ 00:20:09.180 Mustafa Raja: Processing a record… Okay.
266 00:20:09.180 ⇒ 00:20:10.250 Samuel Roberts: Hmm…
267 00:20:16.120 ⇒ 00:20:18.429 Mustafa Raja: Records, okay, we need to see what the records are.
268 00:20:18.430 ⇒ 00:20:19.790 Samuel Roberts: Yeah.
269 00:20:20.930 ⇒ 00:20:23.199 Mustafa Raja: We need to see what the records are. Okay.
270 00:20:23.530 ⇒ 00:20:26.010 Mustafa Raja: God, gotcha.
271 00:20:26.010 ⇒ 00:20:26.920 Samuel Roberts: from Brazil.
272 00:20:26.920 ⇒ 00:20:27.550 Mustafa Raja: Okay.
273 00:20:29.230 ⇒ 00:20:32.130 Mustafa Raja: filtered data. Well, this is,
274 00:20:32.790 ⇒ 00:20:38.699 Mustafa Raja: Yeah, let’s see what the filtered data is. Sorry, I…
275 00:20:38.700 ⇒ 00:20:42.689 Samuel Roberts: No, you’re okay, you’re okay, this is exactly what I would be doing if I were in the code, so…
276 00:20:42.690 ⇒ 00:20:47.170 Mustafa Raja: Okay, so it’s, morning, Dora.
277 00:20:48.380 ⇒ 00:20:54.189 Mustafa Raja: Yeah, yeah, it doesn’t seem to be filtering on… based on names, or… based on names or whatever.
278 00:20:54.550 ⇒ 00:20:55.190 Samuel Roberts: Okay.
279 00:20:55.430 ⇒ 00:20:56.200 Samuel Roberts: Okay.
280 00:20:57.200 ⇒ 00:20:57.560 Samuel Roberts: Mmm.
281 00:20:57.560 ⇒ 00:20:59.249 Mustafa Raja: Indeed, I meant.
282 00:21:00.180 ⇒ 00:21:02.950 Mustafa Raja: So, I don’t know if it knows…
283 00:21:04.600 ⇒ 00:21:07.739 Mustafa Raja: That we have data already in the embedded.
284 00:21:08.240 ⇒ 00:21:14.399 Samuel Roberts: So, is there something about doing it this way, where… it’s still doing them individually, though, right?
285 00:21:15.350 ⇒ 00:21:22.689 Mustafa Raja: Yes, for each client, yeah. For each client? Is it… so is it doing, like, a…
286 00:21:23.370 ⇒ 00:21:29.489 Samuel Roberts: It goes client by client and reruns an embedding across all those transcripts, or one by one?
287 00:21:30.580 ⇒ 00:21:33.459 Mustafa Raja: By the looks of it, one by one.
288 00:21:33.770 ⇒ 00:21:39.680 Mustafa Raja: For, for each meeting, then… for one meeting, the next, the next, the next, right? Like that?
289 00:21:39.680 ⇒ 00:21:43.859 Samuel Roberts: Yeah, okay, so I was just wondering, I didn’t know if it was, like, because we’re doing it
290 00:21:44.040 ⇒ 00:21:50.419 Samuel Roberts: Together, we’re getting more context in the embedding about the client.
291 00:21:54.420 ⇒ 00:21:58.340 Samuel Roberts: Okay, I… Maybe I need to rethink a little bit how we’re gonna…
292 00:21:59.940 ⇒ 00:22:09.090 Samuel Roberts: Because, like I said, I don’t want to mess with this yet. I don’t really want to change how it’s doing things. I wanted to just make a new system that was doing it in parallel that we could flip over to. But maybe that’s…
293 00:22:11.220 ⇒ 00:22:14.369 Mustafa Raja: Oh yeah, the filter might really just be…
294 00:22:14.770 ⇒ 00:22:17.620 Mustafa Raja: We are getting the meetings for the client.
295 00:22:19.270 ⇒ 00:22:19.950 Samuel Roberts: Okay.
296 00:22:22.830 ⇒ 00:22:24.440 Mustafa Raja: So, when a meeting…
297 00:22:24.440 ⇒ 00:22:29.870 Samuel Roberts: When a meeting gets… When a meeting gets… Completed.
298 00:22:30.090 ⇒ 00:22:32.020 Samuel Roberts: Zoom triggers windmill.
299 00:22:33.520 ⇒ 00:22:34.810 Samuel Roberts: Windmills? Yes.
300 00:22:34.980 ⇒ 00:22:41.640 Samuel Roberts: saves it to Superbase, with the… In that file,
301 00:22:41.750 ⇒ 00:22:44.880 Samuel Roberts: I forget the name of the table, but… In the…
302 00:22:44.880 ⇒ 00:22:45.920 Casie Aviles: Meeting recording.
303 00:22:45.920 ⇒ 00:22:46.250 Mustafa Raja: this one.
304 00:22:46.250 ⇒ 00:22:46.920 Samuel Roberts: Yeah.
305 00:22:47.040 ⇒ 00:22:51.390 Samuel Roberts: So then, then, I guess, what happens from there, is maybe my new question.
306 00:22:51.790 ⇒ 00:22:58.429 Mustafa Raja: What happens is, this job runs every 5 hours, and…
307 00:22:58.680 ⇒ 00:23:01.199 Mustafa Raja: Yeah, it doesn’t have to run immediately.
308 00:23:01.830 ⇒ 00:23:08.879 Samuel Roberts: Oh, I see. Okay, that’s where I meant misunderstanding. Okay, I thought it was happening. So the embeddings don’t get run except on this schedule.
309 00:23:08.880 ⇒ 00:23:10.319 Casie Aviles: And it just is.
310 00:23:10.320 ⇒ 00:23:14.540 Samuel Roberts: Backfilling all the… all the meetings at once, so it handles all the new ones.
311 00:23:15.430 ⇒ 00:23:15.880 Casie Aviles: Yeah.
312 00:23:16.290 ⇒ 00:23:18.370 Samuel Roberts: And clear… and the old ones. Okay, okay.
313 00:23:22.240 ⇒ 00:23:23.729 Samuel Roberts: Okay, so I think…
314 00:23:27.000 ⇒ 00:23:30.959 Samuel Roberts: Is there… A good reason to not…
315 00:23:31.420 ⇒ 00:23:34.150 Samuel Roberts: Do it on demand for the meetings?
316 00:23:37.520 ⇒ 00:23:38.340 Samuel Roberts: I don’t know.
317 00:23:38.340 ⇒ 00:23:40.940 Mustafa Raja: On demand, as in, as the meeting comes.
318 00:23:41.340 ⇒ 00:23:42.070 Samuel Roberts: Yeah.
319 00:23:43.750 ⇒ 00:23:45.339 Samuel Roberts: Are we… am I missing, like…
320 00:23:45.340 ⇒ 00:23:46.110 Mustafa Raja: to put it?
321 00:23:46.110 ⇒ 00:23:54.369 Samuel Roberts: Yeah, I’m just wondering, is it… is it missing any context by not doing them all together? But it doesn’t seem like it’s doing them in, like, a big chunk, it’s just doing per meeting, per client.
322 00:23:55.940 ⇒ 00:23:56.650 Samuel Roberts: Okay.
323 00:24:02.080 ⇒ 00:24:06.319 Samuel Roberts: Alright, so okay, maybe, maybe I don’t want to mess with this right now, because I don’t want to break…
324 00:24:06.320 ⇒ 00:24:08.410 Mustafa Raja: Yeah, that makes sense, I just wanted to bring it up.
325 00:24:08.770 ⇒ 00:24:09.250 Samuel Roberts: Yeah, no.
326 00:24:09.250 ⇒ 00:24:10.870 Mustafa Raja: I appreciate that.
327 00:24:11.340 ⇒ 00:24:14.429 Samuel Roberts: I do appreciate that, because I did not realize it was doing it this way.
328 00:24:14.920 ⇒ 00:24:17.460 Samuel Roberts: okay.
329 00:24:17.620 ⇒ 00:24:23.019 Samuel Roberts: I might need to think a little more, but I don’t have too much time left right now.
330 00:24:23.590 ⇒ 00:24:27.229 Samuel Roberts: So, do you guys think it’s a good idea
331 00:24:27.950 ⇒ 00:24:30.549 Samuel Roberts: To not mess with the pipelines for now, then?
332 00:24:30.940 ⇒ 00:24:33.989 Mustafa Raja: The… the embedding pipelines.
333 00:24:34.390 ⇒ 00:24:38.169 Samuel Roberts: Well, so, okay, because my, kind of, overall plan here…
334 00:24:39.200 ⇒ 00:24:44.100 Samuel Roberts: Was to get all that stuff into one table, which would make it easier to query
335 00:24:45.240 ⇒ 00:24:48.840 Samuel Roberts: from a generic Mastra… agent.
336 00:24:49.000 ⇒ 00:24:49.850 Samuel Roberts: Right?
337 00:24:51.010 ⇒ 00:24:54.689 Samuel Roberts: But we don’t have to do it that way. We could just move the monster, all the N8N stuff.
338 00:24:55.150 ⇒ 00:24:58.159 Samuel Roberts: Into code, and still use our current data setup for now.
339 00:25:01.060 ⇒ 00:25:06.280 Samuel Roberts: Like, either we do both, or we just do one, and if we want to get off N8N, we could do that.
340 00:25:06.780 ⇒ 00:25:08.759 Samuel Roberts: We just have to be a little more…
341 00:25:08.960 ⇒ 00:25:16.789 Samuel Roberts: aware of the fact that we have a whole bunch of different tables, and we will still have to create those tables per client, which is what I was kind of trying to avoid.
342 00:25:17.770 ⇒ 00:25:18.330 Mustafa Raja: Yes.
343 00:25:18.620 ⇒ 00:25:19.290 Casie Aviles: Hmm.
344 00:25:21.670 ⇒ 00:25:31.259 Mustafa Raja: Yeah. Okay. One single table… if we try and restructure this, maybe, you know, handling one single table…
345 00:25:31.420 ⇒ 00:25:36.280 Mustafa Raja: Might be… might be easier, I believe.
346 00:25:37.140 ⇒ 00:25:38.650 Samuel Roberts: Yeah, okay, okay.
347 00:25:38.650 ⇒ 00:25:43.549 Mustafa Raja: We would only have to rethink the logic for that particular table and all.
348 00:25:43.550 ⇒ 00:25:48.690 Samuel Roberts: Yes, okay, so let’s, let’s… okay, maybe we still want to do it this way. So, basically, we don’t have to change anything…
349 00:25:49.120 ⇒ 00:25:57.970 Samuel Roberts: for Zoom, Windmill will send that to that master record table.
350 00:26:00.880 ⇒ 00:26:10.380 Samuel Roberts: I guess… We could then say… Well, so right now, When it gets there.
351 00:26:10.860 ⇒ 00:26:12.809 Samuel Roberts: Let me, let me get super close up, hold on.
352 00:26:13.120 ⇒ 00:26:17.590 Samuel Roberts: What knows… how do they know which client these meetings are at that point?
353 00:26:17.870 ⇒ 00:26:18.820 Samuel Roberts: Folder, is that what it.
354 00:26:18.820 ⇒ 00:26:26.960 Mustafa Raja: Yes, the name of the meeting, that’s all. The name of the meeting must have the client’s name.
355 00:26:26.960 ⇒ 00:26:30.089 Samuel Roberts: For it to be embedded with the client.
356 00:26:30.670 ⇒ 00:26:36.469 Mustafa Raja: And that is where I was thinking maybe, maybe Turbo Puffer could help us solve that.
357 00:26:38.610 ⇒ 00:26:39.280 Samuel Roberts: Okay.
358 00:26:40.900 ⇒ 00:26:50.359 Samuel Roberts: Yes, that is something that could be helped. Okay, so let me… let me take a step back here. We don’t have to mess with it dropping into that recordings files…
359 00:26:50.640 ⇒ 00:26:51.590 Samuel Roberts: table.
360 00:26:51.780 ⇒ 00:26:57.500 Samuel Roberts: What we could do is set up a separate trigger That takes that.
361 00:27:03.830 ⇒ 00:27:09.920 Samuel Roberts: And makes an embeddings table, From that storing, so a master embeddings table.
362 00:27:10.040 ⇒ 00:27:14.870 Samuel Roberts: that just pulls the client out for now. Maybe eventually we use TurboPuffer to identify it.
363 00:27:15.810 ⇒ 00:27:16.230 Mustafa Raja: Oh, boy.
364 00:27:16.230 ⇒ 00:27:17.500 Samuel Roberts: Okay, okay.
365 00:27:17.770 ⇒ 00:27:19.100 Mustafa Raja: Yeah, this makes sense.
366 00:27:19.820 ⇒ 00:27:23.150 Samuel Roberts: Yeah, so I think we don’t have to mess with the current system, like I said.
367 00:27:23.850 ⇒ 00:27:28.230 Samuel Roberts: But we can just add another… another branch here, basically.
368 00:27:28.230 ⇒ 00:27:29.999 Mustafa Raja: Yes, yes, yeah.
369 00:27:30.220 ⇒ 00:27:33.940 Mustafa Raja: Yeah, we might… that way we might solve both problems.
370 00:27:34.650 ⇒ 00:27:35.879 Samuel Roberts: Yeah. We don’t dispute.
371 00:27:35.880 ⇒ 00:27:39.590 Mustafa Raja: by our structure, and we… we also addressed the problem.
372 00:27:40.090 ⇒ 00:27:43.680 Samuel Roberts: Okay, so… What we need to do, then, is…
373 00:27:43.800 ⇒ 00:27:46.229 Samuel Roberts: On… have an on insert, maybe?
374 00:27:47.160 ⇒ 00:27:49.800 Mustafa Raja: Yes, that would be uninsert.
375 00:27:50.050 ⇒ 00:27:56.830 Samuel Roberts: Okay, so… I’m gonna update the… Okay, so… on…
376 00:27:57.480 ⇒ 00:28:01.420 Samuel Roberts: Insert 2, and it’s called… let me make sure I got the name right here…
377 00:28:02.010 ⇒ 00:28:05.300 Samuel Roberts: The Zoom meetings recording file… okay.
378 00:28:07.270 ⇒ 00:28:11.900 Samuel Roberts: Can’t easily copy that name, how do I do that? Copy… Coming in there, okay.
379 00:28:17.730 ⇒ 00:28:23.540 Casie Aviles: I think I was just trying to remember, now that you guys mentioned the on-demand thing.
380 00:28:24.650 ⇒ 00:28:26.110 Samuel Roberts: Yeah. I believe, like…
381 00:28:26.110 ⇒ 00:28:33.489 Casie Aviles: the first thing we tried was actually to have the embedding step right as part of, like, the NADN workflow.
382 00:28:33.890 ⇒ 00:28:35.020 Casie Aviles: So…
383 00:28:35.150 ⇒ 00:28:44.199 Casie Aviles: in a way, it was actually on demand before, but I think that the issue that we were having was we had to backfill every time. That’s why we did, like.
384 00:28:44.830 ⇒ 00:28:46.710 Casie Aviles: The scheduled thing, but…
385 00:28:46.710 ⇒ 00:28:48.330 Samuel Roberts: Oh…
386 00:28:48.330 ⇒ 00:28:49.300 Casie Aviles: Interesting.
387 00:28:50.290 ⇒ 00:28:54.020 Casie Aviles: Because, essentially, it’s just going to be for new meetings.
388 00:28:54.670 ⇒ 00:29:02.630 Casie Aviles: That will be in that… Table in that… in the embeddings table, so… .
389 00:29:02.630 ⇒ 00:29:02.980 Mustafa Raja: So…
390 00:29:02.980 ⇒ 00:29:04.049 Casie Aviles: Yeah, no.
391 00:29:04.080 ⇒ 00:29:07.759 Mustafa Raja: So what I’m understanding is that…
392 00:29:08.120 ⇒ 00:29:12.970 Mustafa Raja: When the trigger, initiated the embedding, it would…
393 00:29:13.140 ⇒ 00:29:17.489 Mustafa Raja: Not have the previous meeting, and just only that meeting would get embedded?
394 00:29:18.920 ⇒ 00:29:19.700 Mustafa Raja: Is that correct?
395 00:29:21.060 ⇒ 00:29:24.860 Casie Aviles: Yeah, just the upcoming meetings, so we didn’t have, like.
396 00:29:25.420 ⇒ 00:29:30.550 Casie Aviles: Because we had, like, the, the… Like, all the meetings.
397 00:29:30.550 ⇒ 00:29:34.179 Samuel Roberts: And yet old… so initially, there were old meetings that weren’t yet embedded.
398 00:29:34.180 ⇒ 00:29:35.740 Casie Aviles: Yes, yes, exactly.
399 00:29:35.740 ⇒ 00:29:37.539 Samuel Roberts: And then as this was built out.
400 00:29:38.250 ⇒ 00:29:40.810 Samuel Roberts: That was a good way to get them all backfilled.
401 00:29:40.810 ⇒ 00:29:44.080 Casie Aviles: Yes, yes, that’s an hour now.
402 00:29:44.230 ⇒ 00:29:46.380 Samuel Roberts: We’re re-backfilling everything every time.
403 00:29:46.900 ⇒ 00:29:49.350 Casie Aviles: Yeah. Okay. I forgot the RMD.
404 00:29:49.660 ⇒ 00:29:50.240 Casie Aviles: that was…
405 00:29:50.240 ⇒ 00:29:51.299 Mustafa Raja: That makes sense.
406 00:29:51.700 ⇒ 00:29:55.749 Samuel Roberts: Okay, well… again, I don’t want to break anything, so I don’t want to, like…
407 00:29:56.080 ⇒ 00:30:03.080 Samuel Roberts: make that change yet, but I think eventually we can get rid of that, probably, because I think what we should maybe do…
408 00:30:03.900 ⇒ 00:30:09.029 Samuel Roberts: And this might be something good to look into a little bit, because I did a few things with, like, on inserts.
409 00:30:09.130 ⇒ 00:30:18.869 Samuel Roberts: for the… users, where, like, when a new… User gets created.
410 00:30:19.200 ⇒ 00:30:23.569 Samuel Roberts: it fetches stuff from Google to sync their data properly.
411 00:30:24.320 ⇒ 00:30:24.700 Mustafa Raja: Hmm.
412 00:30:25.180 ⇒ 00:30:25.640 Casie Aviles: Hmm.
413 00:30:25.640 ⇒ 00:30:30.629 Samuel Roberts: So there’s a way to do it, it’s a… it’s… it’s… we can do it in Supabase, I’m thinking?
414 00:30:31.310 ⇒ 00:30:32.060 Mustafa Raja: Yes.
415 00:30:32.340 ⇒ 00:30:33.770 Samuel Roberts: I don’t know if.
416 00:30:33.770 ⇒ 00:30:35.180 Mustafa Raja: It wouldn’t live…
417 00:30:35.410 ⇒ 00:30:42.899 Mustafa Raja: Yeah, I think it wouldn’t live on Daxter then, because Daxter is more, more for, more for a scheduler, right?
418 00:30:43.160 ⇒ 00:30:43.730 Samuel Roberts: Yeah.
419 00:30:43.730 ⇒ 00:30:44.160 Casie Aviles: Yeah.
420 00:30:44.160 ⇒ 00:30:46.530 Mustafa Raja: on-demand solution, so platform might be…
421 00:30:46.640 ⇒ 00:30:58.860 Samuel Roberts: Yeah, so on insert to the Zoom meetings, we would say, make a new record in the embedding table, and run the embedding.
422 00:30:59.590 ⇒ 00:31:02.640 Samuel Roberts: On that meeting’s transcript.
423 00:31:04.040 ⇒ 00:31:08.700 Samuel Roberts: On that meeting’s, we could probably even do the condensed transcript, right?
424 00:31:09.520 ⇒ 00:31:10.179 Mustafa Raja: Oh, yeah.
425 00:31:11.640 ⇒ 00:31:14.600 Mustafa Raja: We already have that logic in there, so we could…
426 00:31:14.600 ⇒ 00:31:15.330 Samuel Roberts: Yeah. Okay.
427 00:31:15.330 ⇒ 00:31:16.440 Mustafa Raja: As we go.
428 00:31:16.440 ⇒ 00:31:24.429 Samuel Roberts: We’ll do that. We’ll do that at least for Zoom. What’s happening with Slack, then? When are those embeddings happening? Is this something else that I’m not familiar with now?
429 00:31:26.080 ⇒ 00:31:27.869 Mustafa Raja: Black, I think? Yeah, yeah.
430 00:31:27.870 ⇒ 00:31:29.010 Casie Aviles: Those are scheduled.
431 00:31:29.770 ⇒ 00:31:32.659 Samuel Roberts: Okay, and is it doing a backfill every time, too, you think?
432 00:31:32.970 ⇒ 00:31:38.919 Mustafa Raja: Yes, there is also an additional step there for pre-retrieval, if I remember correctly.
433 00:31:42.360 ⇒ 00:31:52.759 Mustafa Raja: I think it’s… I think what it really is, is it just restructures the Slack messages, in a better presentability. That makes sense.
434 00:31:52.760 ⇒ 00:31:54.699 Samuel Roberts: Yeah, that makes some sense. Okay.
435 00:31:55.650 ⇒ 00:32:02.310 Mustafa Raja: So, and that is… that is one table. So one table might be for raw data, one table might.
436 00:32:02.310 ⇒ 00:32:02.830 Samuel Roberts: Oh.
437 00:32:02.830 ⇒ 00:32:08.289 Mustafa Raja: For retrieval, meaning some cleaned data, and then embeddings.
438 00:32:09.520 ⇒ 00:32:13.250 Samuel Roberts: Sure. Okay, so let me go to the Slack project real quick.
439 00:32:13.730 ⇒ 00:32:18.859 Samuel Roberts: So where is Slack Messages? Is that the main one? Nope, that’s empty, okay.
440 00:32:21.320 ⇒ 00:32:23.899 Mustafa Raja: Let me… let me see, actually, I haven’t taken…
441 00:32:23.900 ⇒ 00:32:30.870 Samuel Roberts: Client messages and embeddings… I see client, client, client… Internal AI… okay.
442 00:32:38.340 ⇒ 00:32:39.100 Samuel Roberts: Alright.
443 00:32:39.100 ⇒ 00:32:49.569 Mustafa Raja: Yeah, I think… I think that transformed tables are the ones that have… You know?
444 00:32:51.070 ⇒ 00:32:55.660 Casie Aviles: These are basically client-filtered, or client-specific tables.
445 00:32:56.470 ⇒ 00:33:03.330 Samuel Roberts: then what are… so, like, what is ABC Home Transformed versus Client ABC Home Embeddings and client ABC Home Messages?
446 00:33:06.840 ⇒ 00:33:09.510 Samuel Roberts: Transformed is the one that’s, like, prettier?
447 00:33:11.270 ⇒ 00:33:12.800 Casie Aviles: Let me recall.
448 00:33:14.670 ⇒ 00:33:16.220 Casie Aviles: Okay.
449 00:33:16.530 ⇒ 00:33:19.109 Mustafa Raja: I think content blocks is the one.
450 00:33:20.540 ⇒ 00:33:23.420 Samuel Roberts: Content blocks, oh, right, okay.
451 00:33:25.520 ⇒ 00:33:35.450 Mustafa Raja: Yeah, Hydra has it, Default has it, Hypex has it, yeah, yeah, every client has that, so… so it should be the content blocks one, so if we could, you know, search up blocks, that should be…
452 00:33:35.840 ⇒ 00:33:37.010 Mustafa Raja: That should come up.
453 00:33:37.580 ⇒ 00:33:38.200 Samuel Roberts: Okay.
454 00:33:39.030 ⇒ 00:33:43.519 Samuel Roberts: I think we may need to think a little more about this then.
455 00:33:44.660 ⇒ 00:33:52.600 Samuel Roberts: So what we would need then, if we’re gonna do this basically by combining them, we would need a content blocks with a client field.
456 00:33:55.400 ⇒ 00:33:56.570 Samuel Roberts: Right?
457 00:33:58.820 ⇒ 00:33:59.620 Mustafa Raja: Yeah.
458 00:34:00.650 ⇒ 00:34:01.840 Samuel Roberts: So, I’m just trying to think, if we’re trying.
459 00:34:01.840 ⇒ 00:34:03.020 Mustafa Raja: You can identify…
460 00:34:03.730 ⇒ 00:34:04.420 Samuel Roberts: Yeah, go ahead.
461 00:34:04.420 ⇒ 00:34:10.690 Mustafa Raja: I’m wondering if we could identify a client with this current structure, if we have the Slack channel, because they…
462 00:34:10.690 ⇒ 00:34:13.750 Samuel Roberts: The channel… the channel definitely makes a difference, too, you’re right.
463 00:34:13.750 ⇒ 00:34:14.699 Mustafa Raja: Yeah, yeah, yeah.
464 00:34:15.179 ⇒ 00:34:19.860 Mustafa Raja: But I’m seeing for Eden, there are a lot of channels.
465 00:34:19.860 ⇒ 00:34:23.799 Samuel Roberts: Yeah, so… I think there’s gonna be multiple channels per client, potentially, yeah.
466 00:34:25.300 ⇒ 00:34:30.039 Mustafa Raja: Let me actually see how we are structuring the teams, or the clients.
467 00:34:30.440 ⇒ 00:34:41.950 Samuel Roberts: Okay, maybe I need to spend a little more time thinking about this a little bit before I… they were… get to run with this then. Because I feel a little bit like I didn’t realize how that was going for Zoom, for example.
468 00:34:43.030 ⇒ 00:34:43.690 Mustafa Raja: Ow.
469 00:34:44.060 ⇒ 00:34:54.929 Samuel Roberts: So I’m just a little nervous about making… Making big decisions about… Ow… So…
470 00:34:57.400 ⇒ 00:35:03.729 Samuel Roberts: So we have a content… so let’s just talk through Slack real quick so I can get a feel for it. We have content blocks per client.
471 00:35:04.670 ⇒ 00:35:05.569 Samuel Roberts: We also then have.
472 00:35:05.570 ⇒ 00:35:08.289 Mustafa Raja: Let me actually open up the code also, so…
473 00:35:08.290 ⇒ 00:35:09.530 Samuel Roberts: Oh, sure, yeah, yeah, that’s a good idea.
474 00:35:09.530 ⇒ 00:35:10.610 Mustafa Raja: clarity.
475 00:35:10.610 ⇒ 00:35:15.749 Samuel Roberts: We also have… transformed, but that’s not for all clients, I don’t see here.
476 00:35:16.040 ⇒ 00:35:17.439 Samuel Roberts: Looks like it’s old.
477 00:35:18.500 ⇒ 00:35:22.159 Samuel Roberts: I’m assuming that’s not used as much, then we have embeddings and messages.
478 00:35:23.330 ⇒ 00:35:24.550 Samuel Roberts: Okay.
479 00:35:30.790 ⇒ 00:35:31.600 Mustafa Raja: Let me share.
480 00:35:33.990 ⇒ 00:35:40.910 Mustafa Raja: So, Casey, correct me if I’m wrong, but I think these three… these three files should be concerned with the… with the…
481 00:35:41.390 ⇒ 00:35:51.579 Mustafa Raja: embeddings for Slack, so that’s the pre-retrieval, and then we have, Slack data transformation, and I think it starts from here.
482 00:35:52.870 ⇒ 00:35:54.619 Samuel Roberts: I was just typing, okay.
483 00:35:55.690 ⇒ 00:36:00.899 Casie Aviles: Yeah, it starts with Slack data transformation, then Transform Job, then Embedding Job.
484 00:36:02.210 ⇒ 00:36:03.980 Mustafa Raja: Okay, transform job.
485 00:36:04.280 ⇒ 00:36:07.040 Mustafa Raja: Slide your… okay, okay, okay.
486 00:36:07.520 ⇒ 00:36:08.030 Casie Aviles: Yep.
487 00:36:08.750 ⇒ 00:36:10.919 Mustafa Raja: We shot from here, then this, and then this.
488 00:36:17.310 ⇒ 00:36:18.660 Samuel Roberts: Alright, maybe…
489 00:36:26.120 ⇒ 00:36:26.890 Samuel Roberts: Alright.
490 00:36:28.100 ⇒ 00:36:42.380 Samuel Roberts: So, re-architecting the data seems a little more than maybe… like, we could make moves to get off of N8N and still use this data for now. The problem is creating the new client tables every time, which we can make as a separate project to migrate that over.
491 00:36:43.350 ⇒ 00:36:46.230 Samuel Roberts: I think the priority here should be getting off Neden.
492 00:36:47.780 ⇒ 00:36:53.520 Casie Aviles: Yeah, I think that’s the most painful part, because we have to copy-paste a new workflow every time.
493 00:36:53.520 ⇒ 00:37:01.030 Samuel Roberts: We can still make new tables, we could even make a script that makes tables when we add new clients or something, that’s not crazy.
494 00:37:02.110 ⇒ 00:37:02.660 Mustafa Raja: Yeah.
495 00:37:02.800 ⇒ 00:37:10.689 Samuel Roberts: But for now, let’s focus on using the tables we have with a Moscra agent, but we want to
496 00:37:10.960 ⇒ 00:37:14.950 Samuel Roberts: Plan ahead by making the client
497 00:37:15.270 ⇒ 00:37:18.430 Samuel Roberts: ID part of the parameters for it?
498 00:37:21.770 ⇒ 00:37:22.730 Samuel Roberts: so that…
499 00:37:24.800 ⇒ 00:37:37.509 Samuel Roberts: So that way, for now, it’ll just look up the right tables, but eventually it’ll use, hopefully, eventually, one master table with the client IDs as part of the lookup.
500 00:37:38.960 ⇒ 00:37:39.690 Casie Aviles: Okay.
501 00:37:40.090 ⇒ 00:37:54.370 Samuel Roberts: Does that make sense as a plan here? For, like… at least for the next, like, you know, week and a half, maybe, to make some moves on it? I think moving to… getting the Maestra connected, not changing the data, just setting up, like, a separate, new
502 00:37:54.560 ⇒ 00:37:58.470 Samuel Roberts: generic Mastra Client Hub agent.
503 00:38:00.580 ⇒ 00:38:02.149 Casie Aviles: Yes, I think that… that makes sense.
504 00:38:02.150 ⇒ 00:38:15.440 Samuel Roberts: And we’ll just have to… we’ll have to architect it in such a way that we can replace the, like, multi-table part of it with a call to one table with a client. So maybe we want to just, like, make that a special function that gets the right data.
505 00:38:19.190 ⇒ 00:38:29.790 Samuel Roberts: Okay, so that was… okay, that’s good that we sorted that out, at least. Now we have a better path forward. So let’s forget about the pipeline, changing plans. Let’s not re-architect that.
506 00:38:30.920 ⇒ 00:38:33.980 Samuel Roberts: Let’s think of, okay.
507 00:38:34.680 ⇒ 00:38:39.440 Samuel Roberts: New pipeline… Let’s go with…
508 00:38:41.670 ⇒ 00:38:44.380 Mustafa Raja: Can you share the screen, if you’re writing Notion doc?
509 00:38:44.380 ⇒ 00:38:46.240 Samuel Roberts: Oh, yeah, thank you, thank you, thank you.
510 00:38:48.670 ⇒ 00:38:49.510 Samuel Roberts: Okay.
511 00:38:50.420 ⇒ 00:38:58.590 Samuel Roberts: Gen… ner… Monster… Agent.
512 00:39:00.720 ⇒ 00:39:07.719 Samuel Roberts: So this is what we need to do, which we haven’t really talked about much, because we were talking about the pipeline, but now we’re… so that’s really this piece here.
513 00:39:10.240 ⇒ 00:39:17.629 Samuel Roberts: So, let’s… Quickly go through the main… parts of the client hub.
514 00:39:18.120 ⇒ 00:39:20.970 Samuel Roberts: So I think… is this… is this mostly… does this look…
515 00:39:27.740 ⇒ 00:39:32.110 Samuel Roberts: Does this look like a rough… Way that it currently works.
516 00:39:32.900 ⇒ 00:39:33.970 Samuel Roberts: The N8N.
517 00:39:35.890 ⇒ 00:39:36.890 Casie Aviles: Let me see…
518 00:39:51.000 ⇒ 00:39:52.450 Samuel Roberts: Hmm, okay.
519 00:39:58.730 ⇒ 00:40:01.209 Mustafa Raja: I don’t think that it has, so…
520 00:40:01.590 ⇒ 00:40:06.220 Mustafa Raja: It has context on how… how… how much passed.
521 00:40:06.690 ⇒ 00:40:09.579 Mustafa Raja: It’s looking in, you know?
522 00:40:09.580 ⇒ 00:40:12.160 Samuel Roberts: Oh, okay, okay, that’s fair. Can we actually.
523 00:40:12.160 ⇒ 00:40:21.839 Mustafa Raja: But this might be good if the agent has context on, on the dates, that might be a… might be good.
524 00:40:22.370 ⇒ 00:40:29.470 Samuel Roberts: Yeah, maybe. Okay, let’s… can… can someone jump to an NN client hub, for example, and we can…
525 00:40:31.230 ⇒ 00:40:32.519 Mustafa Raja: Just talk through.
526 00:40:33.860 ⇒ 00:40:35.089 Mustafa Raja: Because I think…
527 00:40:35.350 ⇒ 00:40:37.270 Samuel Roberts: I think the best thing to do.
528 00:40:39.510 ⇒ 00:40:44.310 Samuel Roberts: Let’s figure out a few of the agents, a few parts that we can really,
529 00:40:55.880 ⇒ 00:40:56.840 Samuel Roberts: Okay, yeah.
530 00:40:57.000 ⇒ 00:41:01.689 Samuel Roberts: So… This is one, this is what actually is, like.
531 00:41:02.740 ⇒ 00:41:05.920 Samuel Roberts: messages from Slack or webhook or something, right?
532 00:41:06.330 ⇒ 00:41:12.389 Mustafa Raja: Yeah, this one, yeah. Slack event. Either Slack, or Webhook, or what this is, is.
533 00:41:12.390 ⇒ 00:41:13.020 Samuel Roberts: the other…
534 00:41:13.020 ⇒ 00:41:13.670 Mustafa Raja: Linear together.
535 00:41:13.670 ⇒ 00:41:14.490 Samuel Roberts: DMs.
536 00:41:15.250 ⇒ 00:41:16.460 Samuel Roberts: Right, okay.
537 00:41:16.750 ⇒ 00:41:17.640 Samuel Roberts: Okay.
538 00:41:20.130 ⇒ 00:41:24.800 Samuel Roberts: So we have… let’s… so this is the actual client hub agent.
539 00:41:25.570 ⇒ 00:41:33.640 Samuel Roberts: That is, what is the kind of knowledge base, the RAG agent, effectively, for Slack and Zoom?
540 00:41:33.930 ⇒ 00:41:34.790 Samuel Roberts: Is that ES?
541 00:41:34.790 ⇒ 00:41:35.310 Casie Aviles: Which is…
542 00:41:35.310 ⇒ 00:41:36.190 Samuel Roberts: statement.
543 00:41:36.640 ⇒ 00:41:38.450 Samuel Roberts: Okay, so then we also have…
544 00:41:39.550 ⇒ 00:41:44.749 Samuel Roberts: So then this gets called by the other parts, like that Zoom meeting summarizer? Was that…
545 00:41:45.230 ⇒ 00:41:50.639 Samuel Roberts: Does that make use of this? The linear ticket generator makes use of this? The…
546 00:41:50.640 ⇒ 00:41:53.539 Mustafa Raja: Linear ticket does, I don’t know about any other.
547 00:41:54.070 ⇒ 00:41:54.800 Samuel Roberts: Okay.
548 00:41:55.420 ⇒ 00:41:57.730 Samuel Roberts: Well… Okay.
549 00:41:59.240 ⇒ 00:42:04.459 Mustafa Raja: Yeah, now, since we have, you know, migrated to Mastra.
550 00:42:04.590 ⇒ 00:42:10.240 Mustafa Raja: A linear ticket agent would be using this webhook to communicate with this.
551 00:42:10.470 ⇒ 00:42:17.460 Samuel Roberts: So effectively here, let’s just look real quick. Message from Slack, gets the time, thread, what’s this Python code.
552 00:42:19.960 ⇒ 00:42:20.900 Samuel Roberts: Here, because…
553 00:42:20.900 ⇒ 00:42:23.620 Casie Aviles: It has something to do with, some…
554 00:42:24.010 ⇒ 00:42:26.700 Casie Aviles: Like, filtering and handling thread IDs.
555 00:42:26.700 ⇒ 00:42:27.440 Mustafa Raja: Wow.
556 00:42:27.770 ⇒ 00:42:31.479 Casie Aviles: Yeah, thread timestamps, thread IDs, for threading, basically.
557 00:42:31.480 ⇒ 00:42:33.599 Samuel Roberts: Python is not supported, what does that mean?
558 00:42:34.410 ⇒ 00:42:34.979 Casie Aviles: Oh, yeah.
559 00:42:35.970 ⇒ 00:42:36.730 Casie Aviles: broken now.
560 00:42:36.730 ⇒ 00:42:38.630 Mustafa Raja: I… .
561 00:42:38.630 ⇒ 00:42:39.370 Samuel Roberts: That’s so…
562 00:42:39.370 ⇒ 00:42:42.719 Mustafa Raja: This trigger might not even be working now.
563 00:42:43.090 ⇒ 00:42:45.629 Casie Aviles: I mean… Look at that.
564 00:42:45.630 ⇒ 00:42:47.319 Samuel Roberts: We’ll get that in. Yeah, it’s 5.
565 00:42:47.320 ⇒ 00:42:50.810 Casie Aviles: What are less… Yeah. Or less of a priority, probably.
566 00:42:50.810 ⇒ 00:42:52.839 Samuel Roberts: Really, what we want to do is get…
567 00:42:54.970 ⇒ 00:42:57.380 Samuel Roberts: We want to duplicate this in code.
568 00:42:57.510 ⇒ 00:42:59.279 Samuel Roberts: So that we can hit a route.
569 00:43:01.050 ⇒ 00:43:05.279 Samuel Roberts: that maybe we pass in the client ID as well, right?
570 00:43:05.280 ⇒ 00:43:05.960 Casie Aviles: Yes.
571 00:43:06.180 ⇒ 00:43:09.430 Samuel Roberts: And then it… Goes through whatever it has to do here.
572 00:43:09.850 ⇒ 00:43:11.850 Samuel Roberts: This is an edit fields…
573 00:43:12.140 ⇒ 00:43:18.820 Samuel Roberts: you know, whatever this logic is up to this point, this then becomes an AI agent, and we can use a PG vector
574 00:43:20.010 ⇒ 00:43:21.899 Samuel Roberts: To do this, right?
575 00:43:22.620 ⇒ 00:43:23.650 Mustafa Raja: Yeah, yep.
576 00:43:23.650 ⇒ 00:43:29.910 Samuel Roberts: And then we re… Turn… the webhook response somewhere?
577 00:43:30.780 ⇒ 00:43:40.829 Mustafa Raja: So this doesn’t take, what’s it called? Much time, right after this is done, so we needed… we didn’t need a specific response to webhook.
578 00:43:40.830 ⇒ 00:43:41.520 Samuel Roberts: Got it, okay.
579 00:43:41.520 ⇒ 00:43:42.060 Mustafa Raja: Thank you.
580 00:43:42.060 ⇒ 00:43:47.760 Samuel Roberts: So, okay, so, okay, let’s… let’s make that the first priority, then. Like, this piece, we want to make…
581 00:43:48.170 ⇒ 00:43:53.849 Samuel Roberts: this, especially the webhook side, because Slack could be a webhook thing too, I imagine.
582 00:43:54.200 ⇒ 00:43:55.469 Samuel Roberts: Once we redo the slide.
583 00:43:55.470 ⇒ 00:43:56.250 Mustafa Raja: Oh my gosh.
584 00:43:56.460 ⇒ 00:44:01.610 Samuel Roberts: So it’s kind of all the same, except we have to know the format coming in, right? Because it’s going to be coming into our…
585 00:44:02.500 ⇒ 00:44:03.729 Samuel Roberts: Next app.
586 00:44:06.330 ⇒ 00:44:08.700 Samuel Roberts: So the pieces here are… Yeah, go ahead.
587 00:44:08.700 ⇒ 00:44:16.030 Mustafa Raja: I’m thinking maybe we could leave this trigger here, have a webhook that calls our platform.
588 00:44:18.290 ⇒ 00:44:25.879 Samuel Roberts: Yeah, well, I think… I think that’s basically what’s gonna happen, and even that, we can probably change it to be… to not even use N8N, because…
589 00:44:26.580 ⇒ 00:44:31.359 Samuel Roberts: Slack, instead of this being the thing that does it, Slack could just probably hit a web… a web…
590 00:44:31.550 ⇒ 00:44:33.420 Samuel Roberts: A webhook. That would be the plot.
591 00:44:33.420 ⇒ 00:44:35.809 Mustafa Raja: Yeah, yeah, yeah, yeah, we could do that.
592 00:44:35.810 ⇒ 00:44:37.949 Samuel Roberts: Basically, all we need to do is… so forget about…
593 00:44:38.370 ⇒ 00:44:40.169 Samuel Roberts: This whole side of things for now.
594 00:44:40.710 ⇒ 00:44:44.880 Samuel Roberts: We want to create a route that’s, like, you know.
595 00:44:45.090 ⇒ 00:44:52.810 Samuel Roberts: platform.brainforge.ai slash API slash client hub slash client ID,
596 00:44:53.760 ⇒ 00:44:59.769 Samuel Roberts: slash query, you know, something like that. And so we basically put a file in Next that’s gonna do that. It’s gonna run…
597 00:45:00.030 ⇒ 00:45:05.040 Samuel Roberts: based on that client ID, the right… piece here…
598 00:45:06.260 ⇒ 00:45:10.640 Samuel Roberts: using PGVector, and using the right lookups for the tables.
599 00:45:12.970 ⇒ 00:45:13.660 Casie Aviles: Yes.
600 00:45:14.270 ⇒ 00:45:16.319 Samuel Roberts: I think that’s really it for this piece.
601 00:45:16.570 ⇒ 00:45:20.550 Samuel Roberts: Eventually, we’ll integrate Slack back into it, but does that seem…
602 00:45:23.110 ⇒ 00:45:24.920 Samuel Roberts: Does that seem like a plan?
603 00:45:25.860 ⇒ 00:45:27.460 Casie Aviles: Yeah, I think that’s clear.
604 00:45:28.010 ⇒ 00:45:45.339 Mustafa Raja: I’m just wondering, rather than having, individual folders for each client, maybe, we could just have a client hub, and then, we would be passing the ID, to the… Yes, but we don’t have to have separate folders, we have the client ID. Yeah, we have the master folder, and that’s all.
605 00:45:45.490 ⇒ 00:45:49.370 Samuel Roberts: We have the client ID as a param in the URL.
606 00:45:49.930 ⇒ 00:45:50.500 Mustafa Raja: Yeah.
607 00:45:52.330 ⇒ 00:45:52.990 Samuel Roberts: Okay.
608 00:45:54.820 ⇒ 00:45:55.970 Samuel Roberts: So I think that’s…
609 00:45:56.830 ⇒ 00:46:04.309 Samuel Roberts: I don’t want to say straightforward, because obviously it’s not necessarily straightforward, but I think that’s a better plan than… it just makes… makes this piece a little more complicated.
610 00:46:04.450 ⇒ 00:46:08.430 Samuel Roberts: Because we have to make sure we look up the right tables, so as long as we have the names right.
611 00:46:09.450 ⇒ 00:46:11.330 Samuel Roberts: Per client, that’s fine.
612 00:46:12.090 ⇒ 00:46:14.300 Samuel Roberts: But we want to keep it as generic as possible.
613 00:46:14.610 ⇒ 00:46:15.230 Mustafa Raja: Hmm.
614 00:46:16.260 ⇒ 00:46:17.510 Samuel Roberts: Does that seem like a…
615 00:46:17.510 ⇒ 00:46:18.000 Mustafa Raja: So.
616 00:46:18.000 ⇒ 00:46:19.160 Samuel Roberts: like a… go ahead.
617 00:46:19.930 ⇒ 00:46:24.579 Mustafa Raja: Yeah, so master table is excluded from the plan for now?
618 00:46:25.610 ⇒ 00:46:28.049 Samuel Roberts: Yes, let’s not worry about that for now.
619 00:46:28.390 ⇒ 00:46:37.020 Samuel Roberts: We’re just gonna use the data that’s already there, we’re not gonna change that, because I don’t want to break anything on the platform, especially, like, if I’m not around to, like…
620 00:46:37.020 ⇒ 00:46:37.790 Mustafa Raja: Okay.
621 00:46:37.790 ⇒ 00:46:56.279 Samuel Roberts: I don’t want to try and, like, you know, mess up too much stuff here, because I don’t want you guys flying without me if that’s a problem, but… I’m sure you guys can handle it, because you guys know this better than I do at this point, but I don’t… I don’t feel comfortable saying, go do that until I’m back, but… I think this piece, we can set up a route, then we can test.
622 00:46:57.070 ⇒ 00:47:04.680 Samuel Roberts: and we can see how good the PGVector is for both Zoom and Slack.
623 00:47:06.630 ⇒ 00:47:07.400 Mustafa Raja: Yes.
624 00:47:07.770 ⇒ 00:47:11.160 Samuel Roberts: So we basically need… two monster tools.
625 00:47:13.640 ⇒ 00:47:15.560 Samuel Roberts: That we pass in the right table.
626 00:47:16.130 ⇒ 00:47:18.100 Samuel Roberts: And one agent.
627 00:47:18.100 ⇒ 00:47:18.700 Casie Aviles: Oops.
628 00:47:18.970 ⇒ 00:47:21.359 Samuel Roberts: That is gonna call those tools as needed.
629 00:47:23.360 ⇒ 00:47:26.310 Samuel Roberts: And then just the right routing and everything.
630 00:47:28.500 ⇒ 00:47:35.889 Samuel Roberts: Okay, that seems pretty… Like, a good piece to bite off, that is… Gonna get us…
631 00:47:36.720 ⇒ 00:47:39.509 Samuel Roberts: Towards that… okay, so we have the Slack messages table.
632 00:47:40.660 ⇒ 00:47:42.409 Mustafa Raja: I think… This is on defense?
633 00:47:42.940 ⇒ 00:47:48.629 Mustafa Raja: Yes, maybe we could, you know, the embedding stables could live here, no?
634 00:47:48.630 ⇒ 00:47:50.180 Samuel Roberts: Yeah, no, I was thinking…
635 00:47:50.180 ⇒ 00:47:50.850 Mustafa Raja: Don’t worry about that.
636 00:47:51.000 ⇒ 00:47:53.150 Samuel Roberts: But you’re absolutely right, we could just look them right up.
637 00:47:53.690 ⇒ 00:47:54.270 Samuel Roberts: So, okay.
638 00:47:54.270 ⇒ 00:47:54.650 Mustafa Raja: Yeah.
639 00:47:54.650 ⇒ 00:47:58.670 Samuel Roberts: So then two plans, then, would be…
640 00:48:00.050 ⇒ 00:48:02.730 Samuel Roberts: Update the client’s table with the right…
641 00:48:03.360 ⇒ 00:48:06.869 Samuel Roberts: like, Zoom embeddings and Slack embeddings look up.
642 00:48:08.960 ⇒ 00:48:14.610 Samuel Roberts: build… or code up the two tools to use the PG vector.
643 00:48:14.980 ⇒ 00:48:18.200 Samuel Roberts: And then code up the master agent that will use those tools.
644 00:48:20.950 ⇒ 00:48:22.239 Mustafa Raja: Yeah, pretty simple.
645 00:48:22.710 ⇒ 00:48:30.599 Samuel Roberts: Yeah, I think it’s not… and that gets us a good chunk of the way there to being able to move stuff off of N8N without having to redo the pipeline yet.
646 00:48:30.990 ⇒ 00:48:36.640 Samuel Roberts: And we can make a plan for that later. The other side of this is you can test with Maestra.
647 00:48:36.840 ⇒ 00:48:43.150 Samuel Roberts: and then put it into the web route on Next. So you can use a little monster dev environment to, like.
648 00:48:43.450 ⇒ 00:48:48.799 Samuel Roberts: test the vector embeddings and everything without having to build a whole UI for doing…
649 00:48:48.930 ⇒ 00:48:51.990 Samuel Roberts: The right webhook for… for everything.
650 00:48:52.730 ⇒ 00:48:53.140 Mustafa Raja: Yeah.
651 00:48:53.300 ⇒ 00:48:58.010 Samuel Roberts: Does that… Sound good?
652 00:48:59.400 ⇒ 00:49:00.459 Mustafa Raja: Sounds good to me.
653 00:49:00.730 ⇒ 00:49:01.200 Samuel Roberts: Okay.
654 00:49:01.200 ⇒ 00:49:01.820 Casie Aviles: Hmm.
655 00:49:02.080 ⇒ 00:49:10.819 Samuel Roberts: So, I guess the big tasks are updating the client table with the right embeddings table’s names, right?
656 00:49:11.980 ⇒ 00:49:13.670 Mustafa Raja: Yeah, the union, right?
657 00:49:14.570 ⇒ 00:49:15.240 Samuel Roberts: Sorry?
658 00:49:15.820 ⇒ 00:49:17.249 Mustafa Raja: the query engines.
659 00:49:19.590 ⇒ 00:49:23.669 Samuel Roberts: Oh, right, we have the actual query engine. Well, the PGVector might be able to do it.
660 00:49:24.090 ⇒ 00:49:25.319 Samuel Roberts: We might not even need that now.
661 00:49:25.330 ⇒ 00:49:27.690 Mustafa Raja: Mmm… Yeah, yeah.
662 00:49:30.080 ⇒ 00:49:32.590 Samuel Roberts: I’m actually not sure… Can we jump back to.
663 00:49:32.590 ⇒ 00:49:37.650 Mustafa Raja: I mean, if I could do… yeah, yeah, let’s jump back.
664 00:49:38.150 ⇒ 00:49:41.930 Samuel Roberts: So, that function that we create, is that being used by this?
665 00:49:42.800 ⇒ 00:49:43.479 Samuel Roberts: Hold on.
666 00:49:43.480 ⇒ 00:49:47.649 Mustafa Raja: That… the function that we would create would replace this one, right? These two.
667 00:49:47.650 ⇒ 00:49:51.680 Samuel Roberts: Well, we have a function that’s stored on Superbase for doing that. Where is that getting used?
668 00:49:52.070 ⇒ 00:49:53.200 Casie Aviles: Yeah, you can click that.
669 00:49:53.200 ⇒ 00:49:55.489 Mustafa Raja: No, that is used by Spurbase itself, I think.
670 00:49:55.490 ⇒ 00:49:56.949 Samuel Roberts: Right, so, like, is this…
671 00:49:56.950 ⇒ 00:49:57.389 Casie Aviles: It’s a match.
672 00:49:57.390 ⇒ 00:50:04.599 Samuel Roberts: If you open this, it’s to match the query name there. Okay, yeah. So, either we do that, or we do it in PGVector. I don’t know.
673 00:50:06.590 ⇒ 00:50:08.780 Samuel Roberts: That would be something we have to figure out.
674 00:50:09.350 ⇒ 00:50:14.629 Casie Aviles: Yes, if we could do it in PGVector, then that’s much better, if we don’t have to, like, create.
675 00:50:15.120 ⇒ 00:50:15.750 Samuel Roberts: Yeah.
676 00:50:15.750 ⇒ 00:50:16.290 Casie Aviles: each time.
677 00:50:16.290 ⇒ 00:50:21.330 Samuel Roberts: That’s definitely good. Just, do Monster PGVector, maybe, or using tools, yeah, they probably should get to it.
678 00:50:25.070 ⇒ 00:50:26.410 Samuel Roberts: Is there anything here?
679 00:50:26.590 ⇒ 00:50:27.559 Casie Aviles: Or.
680 00:50:28.610 ⇒ 00:50:30.760 Samuel Roberts: Yeah, what’s on the side? Is there anything…
681 00:50:32.900 ⇒ 00:50:38.509 Samuel Roberts: Memory agent… just search PD vector, yeah. Or RAG, yeah, that’s probably good, too.
682 00:50:39.870 ⇒ 00:50:47.759 Samuel Roberts: Master text SQL, right there, look at that. Yeah, yeah, vector query, okay. So yeah, I would say this is a good bet that we might be able to get it out of the…
683 00:50:49.420 ⇒ 00:50:51.839 Samuel Roberts: Out of the Supabase and into ours.
684 00:50:51.950 ⇒ 00:50:54.150 Samuel Roberts: And then that vector query tool…
685 00:50:55.930 ⇒ 00:50:58.320 Mustafa Raja: Oh, this actually uses PGVector.
686 00:50:58.480 ⇒ 00:51:04.539 Samuel Roberts: Yeah, exactly. So I’m thinking, basically, it’s not going to be much different than what they have laid out here, hopefully.
687 00:51:04.560 ⇒ 00:51:11.090 Mustafa Raja: But any logic from that other one for matching might be important, so… Yeah, yeah.
688 00:51:11.520 ⇒ 00:51:11.880 Samuel Roberts: Okay.
689 00:51:12.450 ⇒ 00:51:13.320 Mustafa Raja: That’s nice.
690 00:51:13.960 ⇒ 00:51:15.349 Samuel Roberts: Yeah, okay, cool.
691 00:51:16.480 ⇒ 00:51:21.039 Samuel Roberts: So yeah, if we can stop using those functions, we can just use it right here, that’s probably good, and then…
692 00:51:21.990 ⇒ 00:51:27.680 Samuel Roberts: That’s for Slack and Zoom, and then the agent is just the prompt that has the right tools
693 00:51:28.940 ⇒ 00:51:30.480 Samuel Roberts: passed into it, right?
694 00:51:33.510 ⇒ 00:51:34.150 Samuel Roberts: Okay.
695 00:51:34.820 ⇒ 00:51:36.869 Samuel Roberts: I think that’s pretty much the plan, then.
696 00:51:37.820 ⇒ 00:51:46.870 Samuel Roberts: I guess, yeah, the tickets are effectively, like, update the client table, With the embeddings, table names.
697 00:51:47.270 ⇒ 00:51:53.690 Samuel Roberts: update the… or create the Slack embedding tool.
698 00:51:54.890 ⇒ 00:52:00.010 Samuel Roberts: create the Zoom embedding tool, which are gonna be very similar, probably.
699 00:52:00.400 ⇒ 00:52:06.449 Samuel Roberts: and then create the agent that calls those tools.
700 00:52:07.000 ⇒ 00:52:08.250 Samuel Roberts: And then…
701 00:52:08.390 ⇒ 00:52:14.260 Samuel Roberts: get that working, put it in a route that includes a client ID, and then we’re good, I think.
702 00:52:15.360 ⇒ 00:52:16.909 Samuel Roberts: For this first phase.
703 00:52:17.450 ⇒ 00:52:18.220 Mustafa Raja: Yes.
704 00:52:18.790 ⇒ 00:52:19.480 Samuel Roberts: Okay.
705 00:52:23.000 ⇒ 00:52:30.100 Samuel Roberts: Yeah, I think that’s it. I think you guys can kind of run with that. I don’t know how long that’ll take. It probably isn’t gonna be crazy, so…
706 00:52:30.270 ⇒ 00:52:36.210 Samuel Roberts: Like, getting that working and testing it against stuff is gonna… like, comparing the results and everything will be really…
707 00:52:36.450 ⇒ 00:52:38.899 Samuel Roberts: Good, to see how it does.
708 00:52:39.550 ⇒ 00:52:41.849 Samuel Roberts: Against the current, like, status quo.
709 00:52:43.070 ⇒ 00:52:47.179 Samuel Roberts: And then from there, we can, if it’s working.
710 00:52:48.370 ⇒ 00:52:56.090 Samuel Roberts: we can go two routes. We could either change the underlying data, like we talked about, the pipelines and stuff, if we think it would streamline things at all.
711 00:52:56.340 ⇒ 00:52:57.569 Samuel Roberts: We could certainly probably.
712 00:52:57.570 ⇒ 00:52:57.990 Mustafa Raja: Maybe…
713 00:52:58.280 ⇒ 00:53:00.889 Samuel Roberts: backfill every time, but we could also…
714 00:53:00.890 ⇒ 00:53:01.470 Mustafa Raja: Yeah.
715 00:53:01.470 ⇒ 00:53:05.289 Samuel Roberts: Just switch this over so that things start using this instead and see how it does.
716 00:53:05.980 ⇒ 00:53:06.630 Mustafa Raja: Yep.
717 00:53:06.740 ⇒ 00:53:09.800 Samuel Roberts: Yeah, maybe we could… maybe Casey and I could…
718 00:53:10.100 ⇒ 00:53:15.739 Mustafa Raja: Get together and, document how… This stuff is working, right?
719 00:53:17.700 ⇒ 00:53:18.689 Samuel Roberts: Yeah, that would…
720 00:53:18.690 ⇒ 00:53:20.010 Mustafa Raja: Have a high-level look at it.
721 00:53:20.010 ⇒ 00:53:26.329 Samuel Roberts: That would definitely be great for the… certainly for the pipeline, yeah, definitely, as it is. Like, the current Dagster…
722 00:53:26.860 ⇒ 00:53:32.820 Samuel Roberts: functions and where things end up, because I definitely didn’t understand that all clearly, so…
723 00:53:33.260 ⇒ 00:53:35.650 Mustafa Raja: I also don’t, for now.
724 00:53:35.650 ⇒ 00:53:47.859 Samuel Roberts: Yeah, no, no, you’re good, you’re good. That’s definitely why we gotta get together and do that then. So yeah, if you guys could do… if you guys could document the pipelines then, and then throw together that generic client hub and try that out, and see how it does against…
725 00:53:48.460 ⇒ 00:53:50.200 Samuel Roberts: The current status, yeah.
726 00:53:51.980 ⇒ 00:53:53.080 Mustafa Raja: Yes.
727 00:53:53.890 ⇒ 00:54:01.569 Samuel Roberts: Alrighty, yeah, so I… this is where I disappear, but I will,
728 00:54:01.760 ⇒ 00:54:05.620 Samuel Roberts: Probably not have my notifications on the whole time. In the next 2 days.
729 00:54:05.840 ⇒ 00:54:12.240 Samuel Roberts: will definitely be worse, but next week, if you need me, you can probably push notifications through, even if I haven’t paused.
730 00:54:12.490 ⇒ 00:54:13.899 Samuel Roberts: If anything comes up.
731 00:54:15.230 ⇒ 00:54:20.470 Samuel Roberts: I don’t know when I’ll be able to get back to them and stuff, but I will have my laptop with me, I just won’t be on it necessarily, so…
732 00:54:21.260 ⇒ 00:54:21.920 Mustafa Raja: Yep.
733 00:54:22.180 ⇒ 00:54:26.370 Mustafa Raja: Have a good weekend. Have a great weekend. I’ll see you the next week also.
734 00:54:26.370 ⇒ 00:54:35.699 Samuel Roberts: Yeah, yeah, yes, thank you, thank you. It should be nice to… I’m going to my wife’s family for Thanksgiving, and then my family next week, so… should be nice to see everyone. Yeah.
735 00:54:36.260 ⇒ 00:54:51.720 Samuel Roberts: Alright, well, like I said, if you have any questions or anything, or any thoughts, or, you know, need… need just another brain to bounce things off of if things aren’t going well, like, don’t hesitate. I just might not be immediately responsive, but I will be eventually, so…
736 00:54:51.720 ⇒ 00:54:52.380 Mustafa Raja: Fuck it.
737 00:54:52.990 ⇒ 00:54:53.690 Samuel Roberts: Cool.
738 00:54:54.150 ⇒ 00:54:58.819 Samuel Roberts: Alright, I’m actually really excited about this, this seems like a good path forward, and now that I understand the pipeline stuff.
739 00:55:01.080 ⇒ 00:55:03.820 Samuel Roberts: Yeah, maybe Casey and I could get together and…
740 00:55:04.990 ⇒ 00:55:07.659 Mustafa Raja: You know, outline what’s happening in the league.
741 00:55:08.240 ⇒ 00:55:12.669 Mustafa Raja: Yeah, that would be very helpful, too, for the next few steps after this, so I think that’s a great plan.
742 00:55:14.680 ⇒ 00:55:16.510 Mustafa Raja: Alrighty. Yeah, this was… thank you.
743 00:55:17.510 ⇒ 00:55:18.789 Samuel Roberts: Alright, sounds good, guys.
744 00:55:19.700 ⇒ 00:55:23.400 Samuel Roberts: I will, yeah, maybe I’ll talk to you. If not, I’ll see you in a couple weeks.
745 00:55:23.560 ⇒ 00:55:24.270 Gabriel Lam: Alright.
746 00:55:24.270 ⇒ 00:55:24.850 Mustafa Raja: Yep.
747 00:55:25.180 ⇒ 00:55:25.950 Samuel Roberts: Alright.
748 00:55:27.420 ⇒ 00:55:29.699 Gabriel Lam: Okay, I’ll end the meeting. Thanks, guys.
749 00:55:29.700 ⇒ 00:55:30.390 Samuel Roberts: Alrighty.
750 00:55:30.390 ⇒ 00:55:32.000 Mustafa Raja: Thank you. Bye.
751 00:55:32.010 ⇒ 00:55:32.580 Samuel Roberts: Bye.
752 00:55:32.580 ⇒ 00:55:33.010 Mustafa Raja: Hi, thanks.