Meeting Title: AI-Default-Interlude-Insomnia-ABC Standup Date: 2025-10-23 Meeting participants: Casie Aviles, Mustafa Raja, Samuel Roberts, Uttam Kumaran, Rico Rejoso, Demilade
WEBVTT
1 00:00:07.140 ⇒ 00:00:08.489 Mustafa Raja: Oh, you’re busy.
2 00:00:10.710 ⇒ 00:00:11.450 Casie Aviles: And…
3 00:00:19.470 ⇒ 00:00:20.300 Casie Aviles: Sorry?
4 00:00:21.120 ⇒ 00:00:22.879 Casie Aviles: Did you say something?
5 00:00:23.510 ⇒ 00:00:24.879 Mustafa Raja: Oh, yeah. Hey, how are you?
6 00:00:25.800 ⇒ 00:00:30.020 Casie Aviles: Yeah, yeah, doing good, yeah, alright.
7 00:00:30.510 ⇒ 00:00:31.330 Casie Aviles: How about you?
8 00:00:31.900 ⇒ 00:00:33.180 Mustafa Raja: Yeah, doing good.
9 00:00:37.720 ⇒ 00:00:38.520 Samuel Roberts: No.
10 00:00:39.430 ⇒ 00:00:40.210 Mustafa Raja: Great.
11 00:00:42.210 ⇒ 00:00:43.549 Samuel Roberts: How are you guys doing today?
12 00:00:44.600 ⇒ 00:00:46.040 Mustafa Raja: Yeah, I’m doing good, how well?
13 00:00:46.790 ⇒ 00:00:48.090 Samuel Roberts: Doing alright.
14 00:01:01.360 ⇒ 00:01:02.690 Samuel Roberts: Excuse me.
15 00:01:12.490 ⇒ 00:01:13.390 Uttam Kumaran: Hey, guys.
16 00:01:14.360 ⇒ 00:01:14.680 Mustafa Raja: Oh.
17 00:01:16.600 ⇒ 00:01:17.150 Samuel Roberts: Ayy.
18 00:01:17.150 ⇒ 00:01:17.930 Uttam Kumaran: Hello.
19 00:01:21.540 ⇒ 00:01:22.770 Uttam Kumaran: How’s everything?
20 00:01:26.430 ⇒ 00:01:27.320 Casie Aviles: Good.
21 00:01:27.930 ⇒ 00:01:28.530 Casie Aviles: Good.
22 00:01:29.730 ⇒ 00:01:32.329 Samuel Roberts: It’s real rainy here, but besides that…
23 00:01:32.430 ⇒ 00:01:34.710 Samuel Roberts: You know, nice day to stay inside.
24 00:01:35.560 ⇒ 00:01:38.910 Uttam Kumaran: Nice, yeah, it’s finally getting cooler here, so…
25 00:01:39.100 ⇒ 00:01:40.649 Samuel Roberts: Oh, good, yeah.
26 00:01:49.950 ⇒ 00:01:58.339 Samuel Roberts: It was raining to and from the class yesterday, too, and it was pouring, and I was like, of course, I think it did that last week, too, just seems every Wednesday, it just seems every Wednesday now.
27 00:02:00.200 ⇒ 00:02:01.120 Uttam Kumaran: Nice.
28 00:02:03.400 ⇒ 00:02:05.720 Uttam Kumaran: Okay, cool, let’s talk about…
29 00:02:05.970 ⇒ 00:02:11.540 Uttam Kumaran: stuff this week. So yeah, one thing is, if you guys can come to the next meeting for…
30 00:02:11.790 ⇒ 00:02:13.770 Uttam Kumaran: Today and tomorrow as well.
31 00:02:13.920 ⇒ 00:02:19.639 Uttam Kumaran: And then I’m gonna kinda change a little format for Monday.
32 00:02:19.790 ⇒ 00:02:23.589 Uttam Kumaran: That would be great, because we’re going to talk about insomnia and stuff in the next meeting.
33 00:02:25.140 ⇒ 00:02:26.050 Samuel Roberts: Cool, yeah.
34 00:02:28.950 ⇒ 00:02:33.680 Uttam Kumaran: So, let’s talk about, tickets this week. So, the one thing,
35 00:02:34.320 ⇒ 00:02:40.749 Uttam Kumaran: Casey, I just said, like, this is way too many stuff in blocks right now.
36 00:02:41.010 ⇒ 00:02:56.129 Uttam Kumaran: And in our conversation yesterday, we talked a little bit about ABC and grooming, and basically, I just, like, I’m trying to intervene a bit and make sure that you guys aren’t spending, like.
37 00:02:56.720 ⇒ 00:03:04.360 Uttam Kumaran: like, I just don’t think you should be doing triage work for ABC every single day. Like, I want to find a way for this team to batch it, like.
38 00:03:04.600 ⇒ 00:03:05.780 Uttam Kumaran: twice a week.
39 00:03:05.900 ⇒ 00:03:15.429 Uttam Kumaran: But also, I think my larger point is that the fact that there are this many issues indicates that there is some fundamental architecture issue.
40 00:03:16.430 ⇒ 00:03:16.900 Casie Aviles: Yes.
41 00:03:16.900 ⇒ 00:03:23.330 Uttam Kumaran: Like, there is either… And it’s like, we need to find, like, basically what the culprit is.
42 00:03:23.660 ⇒ 00:03:26.349 Uttam Kumaran: To, like, at a high level, it’s either, like.
43 00:03:26.780 ⇒ 00:03:37.820 Uttam Kumaran: we’re… like, our system is, like, not appropriate for the expectation, which that’s… I don’t think that’s the majority of the problem, but for example, tracking errors.
44 00:03:39.030 ⇒ 00:03:50.879 Uttam Kumaran: some, like, UX for, like, selecting departments, there is some stuff we can do. The other thing is, I just don’t think they have all this information in one place, and that is not something that I want us to, like, spend time doing, especially you guys, because
45 00:03:51.040 ⇒ 00:03:56.470 Uttam Kumaran: your engineers, like, I don’t want you, like, wrangling like… Google Docs.
46 00:03:57.220 ⇒ 00:03:57.600 Casie Aviles: And so…
47 00:03:57.600 ⇒ 00:04:04.320 Uttam Kumaran: So, I think after… basically, by the end of this week, I’ve asked Amber to sort of give me
48 00:04:04.550 ⇒ 00:04:09.190 Uttam Kumaran: A little bit of an understanding of, like…
49 00:04:09.380 ⇒ 00:04:26.840 Uttam Kumaran: like, Amber and Mustafa worked yesterday in a bit of a spike, but on Friday, I think we’ll spend some time talking through, like, the architecture here. Basically, or maybe even sometime next week. But basically, I think I need to… I want to go back to the…
50 00:04:28.440 ⇒ 00:04:30.429 Uttam Kumaran: ABC team and say, hey, like.
51 00:04:30.550 ⇒ 00:04:34.060 Uttam Kumaran: 90% of the problems are the fact that you guys don’t have this information.
52 00:04:34.320 ⇒ 00:04:37.569 Uttam Kumaran: I can’t have my team work on these types of problems anymore.
53 00:04:37.680 ⇒ 00:04:42.460 Uttam Kumaran: So, like, either we… you need to pay us for more, or, like…
54 00:04:42.600 ⇒ 00:04:48.400 Uttam Kumaran: someone on your team has to be assigned to this full-time. So that’s something that I’m gonna push for them. So…
55 00:04:48.890 ⇒ 00:04:56.700 Uttam Kumaran: I guess my… my point is that, like, we can chat about it in Slack, but, like, I don’t want us working on, like.
56 00:04:57.430 ⇒ 00:05:00.759 Uttam Kumaran: There’s just, like, other stuff that’s way more important than this work right now.
57 00:05:00.940 ⇒ 00:05:15.609 Uttam Kumaran: And a lot of this work seems, like, so… like, it’s just, like, kind of, like, useless. Like, I just want to, like, figure out, like, how we can get your guys to not spend time on this stuff. It’s just, like, taking a long time, and we’re not even making that much money from this client, so…
58 00:05:16.840 ⇒ 00:05:26.229 Casie Aviles: Yeah, I understand. Yeah, a lot of the triage tickets that come in are just, you know, like, formatting stuff, and then some are, like, missing data.
59 00:05:26.340 ⇒ 00:05:30.729 Casie Aviles: And, yeah, it’s true, like, they have a lot of data.
60 00:05:31.340 ⇒ 00:05:36.260 Casie Aviles: And we have to kind of consolidate all of that, and then clean it.
61 00:05:36.650 ⇒ 00:05:42.429 Casie Aviles: And then, once we do that, we find out there’s, like, another missing piece of data, so that’s usually where…
62 00:05:43.220 ⇒ 00:05:49.430 Casie Aviles: the triage tickets come in. But partly, there’s also some things we could maybe improve with
63 00:05:49.930 ⇒ 00:05:53.239 Casie Aviles: the way we retrieve the context for RAG,
64 00:05:53.770 ⇒ 00:06:01.370 Casie Aviles: So, specifically with… I think with mechanical… the mechanical department, that’s where, like, the… issues were…
65 00:06:02.000 ⇒ 00:06:05.139 Casie Aviles: I think that’s maybe on us, but yeah.
66 00:06:05.920 ⇒ 00:06:09.520 Samuel Roberts: Is Mechanical still coming from the sheets, or is that stuff in the database now?
67 00:06:13.170 ⇒ 00:06:21.169 Casie Aviles: I believe there’s, like, there’s… we set up, like, a central dock where every time it’s being updated.
68 00:06:21.360 ⇒ 00:06:23.669 Casie Aviles: It gets, vectorized.
69 00:06:24.610 ⇒ 00:06:27.090 Samuel Roberts: Okay. Similar to what we did to PES.
70 00:06:27.590 ⇒ 00:06:28.180 Samuel Roberts: Okay.
71 00:06:28.820 ⇒ 00:06:32.619 Samuel Roberts: But it’s not in the Supabase database that we set up for the other stuff.
72 00:06:35.640 ⇒ 00:06:39.240 Casie Aviles: I believe it should be there, but I need to… I need to check again.
73 00:06:39.240 ⇒ 00:06:46.429 Samuel Roberts: Oh, okay, yeah. That’s something I don’t have a great sense about, is, like, what is, like, in Superbase versus what’s still coming from that central dock.
74 00:06:47.130 ⇒ 00:06:57.040 Samuel Roberts: Which I think is probably some of the issue here, is that some things are different places, and I don’t personally love the central dock as, like, a solution, because it’s… it’s messy, but…
75 00:06:57.650 ⇒ 00:07:00.720 Samuel Roberts: Yeah, okay.
76 00:07:03.050 ⇒ 00:07:04.229 Casie Aviles: Yeah, that’s all.
77 00:07:08.660 ⇒ 00:07:14.520 Uttam Kumaran: Yeah, so I think, I think the biggest thing…
78 00:07:16.270 ⇒ 00:07:27.970 Uttam Kumaran: I want to do next week is do, like, an architecture review of this whole thing, and then, like, find a way out of this, because, yeah, this is just, like, ridiculous, so… Like, what’s on your guys’ plate for ABC, like, today and tomorrow?
79 00:07:28.110 ⇒ 00:07:34.239 Uttam Kumaran: Casey Mustafa, and then if you let me know that, I will just make sure that that’s, like, reflected accurately here.
80 00:07:34.240 ⇒ 00:07:35.239 Casie Aviles: Or we can make some…
81 00:07:35.240 ⇒ 00:07:36.330 Uttam Kumaran: decisions now, yeah.
82 00:07:36.330 ⇒ 00:07:43.650 Casie Aviles: Sure. Yeah, for me, I think I have this urgent ticket that I started just a few minutes ago. This one, the spike.
83 00:07:44.090 ⇒ 00:07:49.060 Casie Aviles: So I’ll focus on this. AVC for… Oh, this one. Yes. Okay.
84 00:07:49.740 ⇒ 00:07:56.790 Mustafa Raja: For me… yeah, for me, I, I’m just writing on those bank documents for the banks that I did.
85 00:07:59.280 ⇒ 00:08:01.030 Mustafa Raja: There’s nothing else.
86 00:08:01.430 ⇒ 00:08:03.719 Mustafa Raja: Abc Foods.
87 00:08:05.160 ⇒ 00:08:05.810 Uttam Kumaran: Okay.
88 00:08:09.100 ⇒ 00:08:12.179 Uttam Kumaran: So, for all of these, I may move these out.
89 00:08:18.660 ⇒ 00:08:23.580 Uttam Kumaran: And then… Anything here I can move out, similarly?
90 00:08:30.960 ⇒ 00:08:33.400 Uttam Kumaran: Like, this one we may talk about next week, right?
91 00:08:34.640 ⇒ 00:08:35.390 Casie Aviles: Okay.
92 00:08:36.620 ⇒ 00:08:38.700 Uttam Kumaran: Yeah, like, I just… yeah, okay.
93 00:08:40.789 ⇒ 00:08:41.860 Uttam Kumaran: These ones?
94 00:08:46.910 ⇒ 00:08:51.170 Casie Aviles: And I haven’t taken a look at a lot of these. Okay.
95 00:08:52.570 ⇒ 00:08:55.400 Uttam Kumaran: I’m just gonna move them out, because I just wanna figure it out.
96 00:08:56.360 ⇒ 00:09:00.690 Uttam Kumaran: Hold on one sec, there’s just someone at my door, let me go grab that one up.
97 00:09:32.960 ⇒ 00:09:33.690 Samuel Roberts: Excuse me.
98 00:09:42.740 ⇒ 00:09:45.689 Samuel Roberts: Well, I guess while he’s gone, I’m curious. So, like…
99 00:09:46.520 ⇒ 00:09:52.800 Samuel Roberts: The stuff that’s still in the central dock, that is the… like, are there still… Assignments and stuff there.
100 00:09:53.010 ⇒ 00:09:55.450 Samuel Roberts: Or is that all in the database, and is it certain departments?
101 00:09:55.450 ⇒ 00:09:55.830 Mustafa Raja: dismissed.
102 00:09:55.830 ⇒ 00:09:57.470 Samuel Roberts: Something I’m still a little hazy on.
103 00:09:57.470 ⇒ 00:10:04.010 Mustafa Raja: Yeah, so the central doc, is a Google Doc, right? So whenever it’s updated, we have a.
104 00:10:04.030 ⇒ 00:10:08.739 Samuel Roberts: vector store in Superbase that also gets updated with it.
105 00:10:09.360 ⇒ 00:10:11.890 Samuel Roberts: And that’s just the, like, the policies and stuff?
106 00:10:13.510 ⇒ 00:10:14.830 Mustafa Raja: Yeah, I believe so.
107 00:10:15.100 ⇒ 00:10:19.589 Samuel Roberts: Okay, then… and that master spreadsheet of all the things, that’s all in the database now?
108 00:10:20.950 ⇒ 00:10:22.340 Casie Aviles: Yeah, the… I love that.
109 00:10:22.480 ⇒ 00:10:25.080 Casie Aviles: Yeah, the inspector thing, with all the… Yes, that’s.
110 00:10:25.080 ⇒ 00:10:25.540 Samuel Roberts: Okay.
111 00:10:25.540 ⇒ 00:10:29.330 Casie Aviles: With the zip codes and the assignments, that should be in the database, yeah.
112 00:10:29.700 ⇒ 00:10:31.350 Samuel Roberts: For… for all the departments?
113 00:10:33.490 ⇒ 00:10:47.709 Casie Aviles: No, just… so we started with pest, and then for inspectors, or for technicians, I mean, we had pest, and then for inspectors, we have… yeah, yeah, that should be, like, not all of the departments.
114 00:10:48.060 ⇒ 00:10:48.840 Samuel Roberts: Okay.
115 00:10:49.220 ⇒ 00:10:56.710 Samuel Roberts: So I’m wondering, I don’t know the details of all the mechanical stuff that was in the triage and things, but I don’t know if that was related to not being in the database, maybe.
116 00:10:56.940 ⇒ 00:10:57.610 Mustafa Raja: Boom.
117 00:10:58.880 ⇒ 00:11:01.899 Casie Aviles: Yeah, if it’s a… if it’s a zip code thing, then…
118 00:11:02.460 ⇒ 00:11:03.170 Samuel Roberts: Okay.
119 00:11:03.560 ⇒ 00:11:10.460 Casie Aviles: If it’s mechanical inspectors, then… It should be in the database, but if they, if they, like.
120 00:11:11.320 ⇒ 00:11:14.479 Casie Aviles: The problem is if they create, like, updates to…
121 00:11:15.110 ⇒ 00:11:21.140 Casie Aviles: If they have, like, new, new people, or they want to change, reassign people… Then…
122 00:11:22.020 ⇒ 00:11:26.410 Casie Aviles: I’m not sure, like, if the CSRs expect that to happen, like, immediately.
123 00:11:27.370 ⇒ 00:11:29.430 Samuel Roberts: That’s true. Okay, good call.
124 00:11:30.760 ⇒ 00:11:33.360 Casie Aviles: So sometimes they probably ask, and then…
125 00:11:34.620 ⇒ 00:11:42.559 Casie Aviles: They would say, oh, this is not right. But we haven’t really updated it, and we weren’t really informed that we should update it, stuff like that.
126 00:11:42.910 ⇒ 00:11:43.830 Samuel Roberts: Right.
127 00:11:44.750 ⇒ 00:11:50.350 Samuel Roberts: Okay, so we probably need a better, like… Process for that anyway, maybe.
128 00:11:52.370 ⇒ 00:11:53.250 Samuel Roberts: Okay.
129 00:12:00.300 ⇒ 00:12:01.240 Samuel Roberts: Excuse me.
130 00:12:01.980 ⇒ 00:12:02.680 Samuel Roberts: Alright.
131 00:12:03.540 ⇒ 00:12:04.630 Uttam Kumaran: Hey guys, sorry.
132 00:12:05.070 ⇒ 00:12:05.870 Samuel Roberts: Yeah, you’re good.
133 00:12:06.660 ⇒ 00:12:12.499 Uttam Kumaran: Okay. So, yeah, I think we’re just gonna do some type of, like.
134 00:12:12.870 ⇒ 00:12:15.359 Uttam Kumaran: refresh on this client, I think, next week.
135 00:12:16.970 ⇒ 00:12:26.219 Uttam Kumaran: I think we’ve done the… I think we have the foundation. I think there’s probably a couple of, like, architecture tickets that, Sam, I can have you take.
136 00:12:26.320 ⇒ 00:12:38.239 Uttam Kumaran: And then we can just pause on, like, some triage for a sec, or just, like, limit the amount of triage. Additionally, like, if some of this triage can be done by non-engineering folks, like, I’d rather loop other people in.
137 00:12:38.540 ⇒ 00:12:50.269 Uttam Kumaran: Like, if Rico can help, or Amber can do some of this, or other non-technical people, part of the architecture here is for us to think about how triage can be done by other people.
138 00:12:50.500 ⇒ 00:12:54.770 Uttam Kumaran: And so I want us to sort of spend time thinking about that, so…
139 00:12:57.720 ⇒ 00:12:58.789 Samuel Roberts: Yeah, that makes sense.
140 00:13:00.860 ⇒ 00:13:01.440 Uttam Kumaran: Cool.
141 00:13:01.900 ⇒ 00:13:14.719 Uttam Kumaran: Okay, great. For default, yeah, I think, I’m gonna meet with Caitlin later, to talk through requirements for the Customer Health Dashboard, and then share with her the…
142 00:13:14.970 ⇒ 00:13:16.640 Uttam Kumaran: PDL stuff.
143 00:13:16.840 ⇒ 00:13:20.510 Mustafa Raja: Yes, so there are two tickets in review.
144 00:13:21.450 ⇒ 00:13:25.310 Mustafa Raja: And I think Caitlin said, it looks good to her.
145 00:13:26.150 ⇒ 00:13:26.790 Uttam Kumaran: Yeah.
146 00:13:27.040 ⇒ 00:13:31.389 Uttam Kumaran: So, yeah, for these, I’ll, I’m gonna move to Don.
147 00:13:35.370 ⇒ 00:13:35.980 Mustafa Raja: Yeah.
148 00:13:37.360 ⇒ 00:13:38.350 Uttam Kumaran: Okay, great.
149 00:13:43.500 ⇒ 00:13:44.300 Uttam Kumaran: Okay.
150 00:13:44.450 ⇒ 00:13:47.940 Uttam Kumaran: Cool. And then for, PDL, we have the,
151 00:13:48.410 ⇒ 00:13:51.789 Uttam Kumaran: the people stuff? We have anything to share on the people stuff yet?
152 00:13:52.130 ⇒ 00:13:52.820 Mustafa Raja: No.
153 00:13:53.630 ⇒ 00:13:54.100 Uttam Kumaran: Okay.
154 00:13:54.100 ⇒ 00:13:57.550 Mustafa Raja: I’m still building up the, sheet.
155 00:13:59.300 ⇒ 00:14:04.550 Uttam Kumaran: Okay, the meeting’s at 10.30 my time. You think you’ll have anything by then?
156 00:14:08.020 ⇒ 00:14:13.499 Mustafa Raja: Pdl doesn’t really give us data for… oh, it does give us data for people.
157 00:14:14.160 ⇒ 00:14:18.589 Mustafa Raja: Okay, so we want, we want, a report for that also.
158 00:14:19.560 ⇒ 00:14:20.380 Uttam Kumaran: Yes.
159 00:14:21.030 ⇒ 00:14:28.219 Mustafa Raja: I don’t think, because, the building of the sheet is a lot of manual work.
160 00:14:28.980 ⇒ 00:14:31.239 Uttam Kumaran: Okay. For all of the companies that we have.
161 00:14:33.230 ⇒ 00:14:33.830 Uttam Kumaran: Okay.
162 00:14:42.450 ⇒ 00:14:49.650 Uttam Kumaran: So, I guess, but the plan we talked about last night was that you were gonna just find a couple people for each company, right, and then run it through each of them?
163 00:14:50.310 ⇒ 00:14:56.139 Mustafa Raja: Yeah. So, I’m about, I have about…
164 00:14:59.200 ⇒ 00:15:02.830 Mustafa Raja: About, 76 people right now.
165 00:15:03.340 ⇒ 00:15:08.870 Mustafa Raja: And… I have a lot of companies in the backlog.
166 00:15:09.820 ⇒ 00:15:10.630 Mustafa Raja: to find people.
167 00:15:10.630 ⇒ 00:15:14.259 Uttam Kumaran: So can… can you share… can you just throw that into the Google Sheet, and then…
168 00:15:14.260 ⇒ 00:15:15.560 Mustafa Raja: is in the Google Sheet.
169 00:15:15.740 ⇒ 00:15:17.160 Uttam Kumaran: Oh, okay, okay, great.
170 00:15:19.640 ⇒ 00:15:25.570 Uttam Kumaran: Let me just check that I can see that.
171 00:15:29.470 ⇒ 00:15:31.020 Uttam Kumaran: Oh, okay, great, okay.
172 00:15:31.260 ⇒ 00:15:33.730 Uttam Kumaran: Cool. Alright, let me… let me ask her about this, then.
173 00:15:34.550 ⇒ 00:15:35.220 Mustafa Raja: Oh, yeah.
174 00:15:36.020 ⇒ 00:15:36.740 Uttam Kumaran: Okay, cool.
175 00:15:38.350 ⇒ 00:15:48.690 Uttam Kumaran: Okay, and then Insomnia, we’ll talk in the next meeting. I think the biggest update that you guys will hear is, like, Amber, like, is going to start to,
176 00:15:48.980 ⇒ 00:15:52.069 Uttam Kumaran: query, we have some of the Braze exports, so,
177 00:15:52.110 ⇒ 00:16:09.979 Uttam Kumaran: Casey, when you were out, we were able to get into S3, we have it connected to MotherDuck. We need to do some data transformation in order to make it more available for analysts. Like, they… basically, one of the Braze exports is a large text file. So, Mustafa, you’re… did you end up kicking that off last time.
178 00:16:09.980 ⇒ 00:16:15.839 Mustafa Raja: Yes, yes, I did end up kicking it off, and, before going to bed, I did look.
179 00:16:15.990 ⇒ 00:16:22.720 Mustafa Raja: In AWS on how it looks, so we have 2,718 files.
180 00:16:23.190 ⇒ 00:16:31.800 Mustafa Raja: Okay. And each file has about 5,000 entries, so we have 13.4… 4 million.
181 00:16:31.960 ⇒ 00:16:32.820 Mustafa Raja: People.
182 00:16:33.250 ⇒ 00:16:41.740 Uttam Kumaran: Cool. And so, what’s the plan for that? Like, are you gonna run it locally, like your transformation script, and then re-upload it to S3?
183 00:16:41.740 ⇒ 00:16:44.819 Mustafa Raja: Yes, this is what it looks like to me.
184 00:16:45.110 ⇒ 00:16:48.419 Mustafa Raja: So I’ll have to download all 2700 files.
185 00:16:48.700 ⇒ 00:16:53.880 Mustafa Raja: And then transform it, possibly into 3 CSVs.
186 00:16:54.100 ⇒ 00:16:57.109 Mustafa Raja: And then upload it to AWS.
187 00:16:58.040 ⇒ 00:17:01.069 Uttam Kumaran: So, one thing I’m gonna… yeah, go ahead.
188 00:17:01.480 ⇒ 00:17:05.730 Samuel Roberts: I’m wondering if there’s a way to do… so, like, right now, Mother Duck is just querying the S3, right?
189 00:17:06.040 ⇒ 00:17:07.190 Samuel Roberts: And you want to keep…
190 00:17:07.190 ⇒ 00:17:08.270 Mustafa Raja: HD files.
191 00:17:08.270 ⇒ 00:17:18.399 Samuel Roberts: Right, right. Yeah, yeah, I saw… yeah, they were zip files, I made them… I unzipped them all, and that took a lot on my machine. So I’m wondering if there’s a better way to do it than doing it locally for you.
192 00:17:19.609 ⇒ 00:17:23.360 Samuel Roberts: But is the goal to get it all still in S3, or are we… like, we just want Mother.
193 00:17:23.369 ⇒ 00:17:23.959 Uttam Kumaran: I mean…
194 00:17:24.180 ⇒ 00:17:24.980 Samuel Roberts: Alright, Larry?
195 00:17:24.980 ⇒ 00:17:29.160 Uttam Kumaran: Well, I want there to be a source of truth in S3.
196 00:17:30.190 ⇒ 00:17:30.820 Samuel Roberts: Okay.
197 00:17:31.380 ⇒ 00:17:33.199 Uttam Kumaran: And then, Mother Duck.
198 00:17:33.580 ⇒ 00:17:42.670 Uttam Kumaran: you could query it, we can re-transform it after that, but I want there to be a source of truth for the raw, rawest data as upstream as possible.
199 00:17:42.820 ⇒ 00:17:48.180 Uttam Kumaran: Got it. Because we may not be in Mother Duck, like, I’m not… that’s, like, a shaky ground.
200 00:17:48.310 ⇒ 00:17:51.029 Uttam Kumaran: So, S3 is gonna be, like, yeah, exactly.
201 00:17:51.620 ⇒ 00:17:59.439 Samuel Roberts: Okay, that’s what I was gonna say, because I didn’t know if there was a way to just use that, S3 query and put it into Mother Duck, but if you want to keep it, that makes sense.
202 00:18:00.240 ⇒ 00:18:06.129 Uttam Kumaran: So, I guess my suggestion… my suggestion to Mustafa was, like, you either download it, run a script locally.
203 00:18:06.430 ⇒ 00:18:08.900 Uttam Kumaran: To transform this into…
204 00:18:09.120 ⇒ 00:18:17.779 Uttam Kumaran: I mean, you can… I don’t know, can… can Motherdeck do Parquet? I mean, you could decide. Like, you could do CSVs, you could do Parquet, whatever it takes.
205 00:18:17.780 ⇒ 00:18:20.300 Samuel Roberts: I’m pretty sure it can do parquet. Yeah.
206 00:18:22.850 ⇒ 00:18:25.090 Uttam Kumaran: And then re-upload it, basically.
207 00:18:25.470 ⇒ 00:18:28.420 Uttam Kumaran: The other alternative is, if you write a script.
208 00:18:28.580 ⇒ 00:18:33.280 Uttam Kumaran: I can run it on my machine, it’s just gonna depend on, like, how big those files are.
209 00:18:36.260 ⇒ 00:18:37.470 Uttam Kumaran: Yeah. The other alternative…
210 00:18:37.470 ⇒ 00:18:37.910 Mustafa Raja: Mr. Runner.
211 00:18:37.910 ⇒ 00:18:39.640 Uttam Kumaran: that in Dagster or something.
212 00:18:39.640 ⇒ 00:18:44.829 Mustafa Raja: Yeah, each file is, like, a couple of MBs, like, 20… around 20 MBs each file.
213 00:18:46.600 ⇒ 00:18:49.790 Uttam Kumaran: So it’s gonna be, like, Tens of gigs.
214 00:18:49.790 ⇒ 00:18:51.820 Samuel Roberts: Yeah, that’s what I was finding.
215 00:18:51.820 ⇒ 00:18:53.260 Uttam Kumaran: Oh, okay, then that’s fine.
216 00:18:53.470 ⇒ 00:18:59.030 Samuel Roberts: Yeah, I think it’s doable, I just wasn’t sure what the… if it was… if it needed to be, but yeah, if you wanted an S3, definitely, I think that’s…
217 00:18:59.030 ⇒ 00:19:14.710 Mustafa Raja: I just… I was just wondering if there’s a better way to download all of these files, like a batch download, because yesterday I tried to multi-select the files to download, and I wasn’t able to find any option for that.
218 00:19:15.520 ⇒ 00:19:20.970 Samuel Roberts: So when I wrote the script, I used… Bunn has a built-in S3 client, that’s pretty.
219 00:19:20.970 ⇒ 00:19:21.300 Mustafa Raja: Okay.
220 00:19:21.300 ⇒ 00:19:24.060 Samuel Roberts: That’s pretty fast. So I had…
221 00:19:24.240 ⇒ 00:19:29.290 Samuel Roberts: cursor using that to probably download, unzip, re-upload, was all I did, basically.
222 00:19:29.290 ⇒ 00:19:30.619 Mustafa Raja: Okay, yeah, I’ll do that then.
223 00:19:30.620 ⇒ 00:19:32.890 Samuel Roberts: you can probably… yeah, you can probably have it do that. I know…
224 00:19:33.380 ⇒ 00:19:37.300 Samuel Roberts: From what I’ve seen online, that seems like the fastest S3 client, so…
225 00:19:37.300 ⇒ 00:19:38.619 Mustafa Raja: Okay, yeah, thank you.
226 00:19:38.620 ⇒ 00:19:40.670 Uttam Kumaran: Can you put… can you put that script in here, Sam?
227 00:19:40.670 ⇒ 00:19:42.440 Mustafa Raja: Yeah, that’ll… that’ll really help.
228 00:19:42.440 ⇒ 00:19:46.980 Samuel Roberts: Yeah, I’ll drop, I’ll, yeah. Let me see…
229 00:19:46.980 ⇒ 00:19:49.089 Uttam Kumaran: Or you can throw it… you can throw it into a gist.
230 00:19:49.360 ⇒ 00:19:52.410 Samuel Roberts: That’s exactly what I was gonna do, I’ll probably… it’s a little big, I think, let me…
231 00:19:52.610 ⇒ 00:19:53.299 Samuel Roberts: Figure out where this is.
232 00:19:53.300 ⇒ 00:19:59.269 Uttam Kumaran: Yeah, just toss into, like, toss into, like, a gist, and then… yeah.
233 00:19:59.270 ⇒ 00:20:01.090 Samuel Roberts: Yeah. Yeah, I’ll do that.
234 00:20:03.770 ⇒ 00:20:04.400 Uttam Kumaran: Okay.
235 00:20:04.750 ⇒ 00:20:09.330 Uttam Kumaran: So yeah, this one is really important today, that we get that done.
236 00:20:15.110 ⇒ 00:20:19.150 Uttam Kumaran: Okay, so that’s that,
237 00:20:19.620 ⇒ 00:20:22.279 Uttam Kumaran: So, I’m gonna mark this one as done.
238 00:20:22.530 ⇒ 00:20:23.650 Uttam Kumaran: Then, Sam?
239 00:20:24.610 ⇒ 00:20:26.180 Samuel Roberts: Sorry, hold on, I was…
240 00:20:26.680 ⇒ 00:20:33.660 Samuel Roberts: Which one? Yeah, yeah, that one’s done, if it was just get that… Yeah. Yeah, it was just explore, and it ended up being just unzipping them, but that’s done, yes.
241 00:20:37.600 ⇒ 00:20:44.070 Uttam Kumaran: Okay, this is also… This is also done.
242 00:20:53.630 ⇒ 00:21:01.119 Uttam Kumaran: And then, one thing, Casey, that, Sam and I started was a little bit of, like, an architecture
243 00:21:01.420 ⇒ 00:21:08.739 Uttam Kumaran: of the daily impact scorecard. This is probably another, like, hour-long meeting we should do.
244 00:21:09.070 ⇒ 00:21:14.570 Uttam Kumaran: Where we… we just go through the architecture,
245 00:21:15.310 ⇒ 00:21:24.379 Uttam Kumaran: And then we identify sources, transform outputs, and then we sort of propose a new solution of, like, how we’re gonna migrate this to, like, a data warehouse.
246 00:21:24.580 ⇒ 00:21:26.970 Uttam Kumaran: And, like, a true, like, data environment.
247 00:21:29.630 ⇒ 00:21:36.159 Uttam Kumaran: Do you want to, like, poke at that today and see if you can add a couple things there? Because I… me and Sam started, but it was a little bit difficult, because you…
248 00:21:36.160 ⇒ 00:21:37.040 Casie Aviles: Yeah, sure, sure.
249 00:21:38.150 ⇒ 00:21:40.020 Uttam Kumaran: Okay, I will.
250 00:21:40.990 ⇒ 00:21:43.220 Casie Aviles: Where can I find the… yeah.
251 00:21:43.590 ⇒ 00:21:46.200 Uttam Kumaran: Yeah, let me, grab that right now.
252 00:21:53.900 ⇒ 00:21:55.350 Uttam Kumaran: Okay.
253 00:21:55.660 ⇒ 00:22:01.220 Uttam Kumaran: So, I’m gonna just put a ticket here for, like, well…
254 00:22:01.850 ⇒ 00:22:04.539 Uttam Kumaran: It’s part of, like, this spike.
255 00:22:04.900 ⇒ 00:22:11.520 Uttam Kumaran: But, I guess I will… Tag you here, Casey.
256 00:22:13.480 ⇒ 00:22:19.419 Uttam Kumaran: If you can just take a look at that today. Well, I’ll make… Just so it’s… it’s clear.
257 00:22:19.570 ⇒ 00:22:21.249 Uttam Kumaran: I don’t want to do self-issues.
258 00:22:21.620 ⇒ 00:22:28.990 Uttam Kumaran: Expand on… Daily reporting… Architecture.
259 00:22:38.380 ⇒ 00:22:45.019 Uttam Kumaran: Need… Sources… Key Transformation Points.
260 00:22:45.650 ⇒ 00:22:48.190 Uttam Kumaran: And where they happened.
261 00:22:48.720 ⇒ 00:22:55.470 Uttam Kumaran: Need… like, key… output.
262 00:22:56.930 ⇒ 00:22:59.050 Uttam Kumaran: Reports, and links.
263 00:22:59.250 ⇒ 00:23:02.910 Uttam Kumaran: And then, yeah, we can propose basically a new solution for that. That’s…
264 00:23:03.540 ⇒ 00:23:06.150 Uttam Kumaran: This would also be pretty important to do.
265 00:23:06.760 ⇒ 00:23:08.370 Uttam Kumaran: Book.
266 00:23:08.480 ⇒ 00:23:12.740 Uttam Kumaran: And then… Great.
267 00:23:13.150 ⇒ 00:23:19.620 Uttam Kumaran: I think that’s all… I had… so, oh, this is the one…
268 00:23:25.130 ⇒ 00:23:26.839 Uttam Kumaran: Oh, so this is done now.
269 00:23:27.800 ⇒ 00:23:29.689 Uttam Kumaran: We have the extract working.
270 00:23:29.810 ⇒ 00:23:31.910 Uttam Kumaran: Okay, so that’s fine.
271 00:23:33.150 ⇒ 00:23:43.530 Uttam Kumaran: One thing I’m gonna create in the next cycle here is document phrase, export to S3.
272 00:23:44.180 ⇒ 00:23:49.380 Uttam Kumaran: to transform… Process…
273 00:23:55.600 ⇒ 00:23:59.879 Uttam Kumaran: I’m trying to use, like… see, it says A, what do I… how do I, like, get to this shortcut here?
274 00:24:00.530 ⇒ 00:24:03.039 Samuel Roberts: Hold on, I’m somewhere else, one sec, let me see what we got.
275 00:24:03.040 ⇒ 00:24:06.979 Uttam Kumaran: Assigned to A, right? Like, should I… if I just press A, it’s not gonna work.
276 00:24:06.980 ⇒ 00:24:10.109 Samuel Roberts: You gotta hit escape, maybe? Yeah. And get out of bed.
277 00:24:10.600 ⇒ 00:24:12.240 Samuel Roberts: Yeah, and then, yeah.
278 00:24:13.300 ⇒ 00:24:13.760 Uttam Kumaran: Okay.
279 00:24:13.760 ⇒ 00:24:18.590 Samuel Roberts: Yeah, because you were… the cursor was blinking in the, description, so it’s in text.
280 00:24:18.730 ⇒ 00:24:19.580 Samuel Roberts: But, yeah.
281 00:24:20.930 ⇒ 00:24:22.000 Uttam Kumaran: Okay, great.
282 00:24:33.760 ⇒ 00:24:36.369 Uttam Kumaran: I’m not gonna get to this stuff right now.
283 00:24:37.080 ⇒ 00:24:42.750 Uttam Kumaran: Well, basically, this is gonna… I can’t do this until we have the new architecture, so I’m just gonna move it to next.
284 00:24:43.350 ⇒ 00:24:51.450 Uttam Kumaran: Okay, alright. The rest of the stuff I need to… we’ll talk to the whole team about. On the internal side,
285 00:24:52.590 ⇒ 00:25:04.199 Uttam Kumaran: Yeah, so Sam, I… I… we created some tickets in the backlog for the Master Migration Plan. I just still don’t have, like, a ton of context on,
286 00:25:04.590 ⇒ 00:25:12.689 Uttam Kumaran: like… how to… Basically, like, how to ticket them out.
287 00:25:12.960 ⇒ 00:25:14.269 Uttam Kumaran: Yeah. For these ones.
288 00:25:14.390 ⇒ 00:25:19.320 Uttam Kumaran: So, one thing I did is, like, there’s this simple template, if you can even just put that.
289 00:25:19.770 ⇒ 00:25:23.009 Uttam Kumaran: That would be really nice. Yeah.
290 00:25:23.010 ⇒ 00:25:31.550 Samuel Roberts: I can do that. Yeah, I was trying to just get out some thoughts, so, like, I listed a bunch of the requirements and what’s gonna have to happen, then I just kind of made a few quick tickets, but I can fill this in with that stuff now.
291 00:25:32.130 ⇒ 00:25:34.300 Uttam Kumaran: Okay, cool. Yeah, that would be…
292 00:25:34.510 ⇒ 00:25:36.300 Samuel Roberts: And those are tagged.
293 00:25:36.740 ⇒ 00:25:43.649 Uttam Kumaran: Yeah, those are tagged as needs grooming, and then… Yeah, I’m gonna put…
294 00:25:45.390 ⇒ 00:25:49.159 Uttam Kumaran: Yeah, all of these are sort of… basically, we need to…
295 00:25:50.640 ⇒ 00:25:58.900 Uttam Kumaran: we need to… I need to work on the grooming for this, so… yeah, these are… I would say that these are probably the most urgent ones.
296 00:25:59.050 ⇒ 00:25:59.570 Samuel Roberts: Yeah.
297 00:26:00.180 ⇒ 00:26:03.029 Samuel Roberts: Algorimnos, tonight, then.
298 00:26:03.720 ⇒ 00:26:09.829 Uttam Kumaran: Okay, and then for next cycle, basically, we’ll… these are a couple of them that we can take on.
299 00:26:10.000 ⇒ 00:26:11.759 Uttam Kumaran: And then…
300 00:26:11.980 ⇒ 00:26:21.449 Uttam Kumaran: for this cycle, do you think we can, like, present something on Friday to maybe even, like, Awash and Demolade on, like, Texas Equal?
301 00:26:21.910 ⇒ 00:26:31.349 Samuel Roberts: Excuse me, yeah, probably. I didn’t get a ton of time yesterday to keep going with that, but, I think I have a pretty good path forward with the dbt stuff, and getting Carser to, like.
302 00:26:31.550 ⇒ 00:26:38.630 Samuel Roberts: Generate, kind of a context file from those, so it knows a little bit better about what it’s doing.
303 00:26:38.880 ⇒ 00:26:41.860 Samuel Roberts: And get some feedback, at least, would be good. So yeah, that’s fine.
304 00:26:42.610 ⇒ 00:26:43.030 Uttam Kumaran: Okay.
305 00:26:45.000 ⇒ 00:26:52.230 Demilade: Sorry to take us back, but I… For the… Cold. Insomnia Cookies.
306 00:26:52.760 ⇒ 00:26:55.850 Demilade: I… the… we’ve done the segment stuff.
307 00:26:56.360 ⇒ 00:26:56.940 Demilade: like, the…
308 00:26:56.940 ⇒ 00:26:57.390 Uttam Kumaran: Yeah.
309 00:26:57.390 ⇒ 00:26:58.300 Demilade: Segmentation.
310 00:26:58.800 ⇒ 00:27:04.280 Uttam Kumaran: Yeah, so, sorry, I probably didn’t make that clear. We’re gonna do the insomnia as part of the next call, most of it.
311 00:27:04.280 ⇒ 00:27:06.130 Demilade: Okay, okay, gotcha, gotcha.
312 00:27:06.130 ⇒ 00:27:13.690 Uttam Kumaran: Because, yeah, like, Amber’s involved now, and like, so… sorry, I should have sent a message earlier, but yeah, that’s why we just glossed over it.
313 00:27:14.360 ⇒ 00:27:15.280 Demilade: Okay, gotcha.
314 00:27:15.880 ⇒ 00:27:16.440 Uttam Kumaran: Cool.
315 00:27:16.750 ⇒ 00:27:23.679 Uttam Kumaran: Okay, and then… I guess that’s… Probably it. I think,
316 00:27:24.230 ⇒ 00:27:31.269 Uttam Kumaran: probably Casey, I think, let me know, like, where you get to today. I… I… I’m feeling, like.
317 00:27:31.660 ⇒ 00:27:42.480 Uttam Kumaran: closer to, like, where I want to be on, like, PMing everything, so I think, like, starting this weekend or next week, I should have more time. I think the one thing I want to figure out is…
318 00:27:42.530 ⇒ 00:27:56.849 Uttam Kumaran: like, starting next week, I think, again, we’re no longer on Interlude, and then hopefully ABC kind of stays at that level. We’ll kind of see what the load is on Insomnia and,
319 00:27:57.120 ⇒ 00:28:06.889 Uttam Kumaran: Insomnia, ABC, and, like, any other clients, and then we can see how much bandwidth we have for internal stuff, so… that’s kind of my thought for next week.
320 00:28:07.250 ⇒ 00:28:14.720 Uttam Kumaran: Okay, guys, I’ll give you, a minute or two back, and then we can talk.
321 00:28:14.720 ⇒ 00:28:15.100 Samuel Roberts: Okay.
322 00:28:15.100 ⇒ 00:28:15.740 Uttam Kumaran: speeding.
323 00:28:16.330 ⇒ 00:28:19.679 Samuel Roberts: Yeah, I don’t know if I’m on the event. Can you just share the link?
324 00:28:19.680 ⇒ 00:28:22.119 Uttam Kumaran: Oh, yeah, let me… I’ll just… I’ll add you guys.
325 00:28:22.410 ⇒ 00:28:23.340 Samuel Roberts: Perfect, thank you.
326 00:28:23.720 ⇒ 00:28:24.690 Mustafa Raja: Thank you.
327 00:28:24.690 ⇒ 00:28:25.240 Samuel Roberts: See you there.
328 00:28:25.240 ⇒ 00:28:25.970 Uttam Kumaran: Thank you.
329 00:28:26.180 ⇒ 00:28:26.810 Mustafa Raja: Bye.