Meeting Title: Data platform planning sprint Date: 2025-05-19 Meeting participants: Annie Yu, Luke Daque, Demilade Agboola, Awaish Kumar, Caio Velasco
WEBVTT
1 00:01:58.240 ⇒ 00:01:59.230 Awaish Kumar: Hello!
2 00:02:02.560 ⇒ 00:02:03.020 Annie Yu: Hello!
3 00:02:03.860 ⇒ 00:02:04.570 Caio Velasco: No.
4 00:02:05.430 ⇒ 00:02:07.049 Awaish Kumar: You’re welcome. Bye!
5 00:02:07.180 ⇒ 00:02:08.790 Awaish Kumar: Can you hear me?
6 00:02:09.150 ⇒ 00:02:10.120 Caio Velasco: Yes.
7 00:02:11.980 ⇒ 00:02:13.660 Awaish Kumar: Okay. Thank you.
8 00:02:14.060 ⇒ 00:02:16.970 Awaish Kumar: How has been the weekend.
9 00:02:20.000 ⇒ 00:02:25.430 Caio Velasco: On my end. All good, lots of some much better weather.
10 00:02:25.650 ⇒ 00:02:28.090 Caio Velasco: Lots of people outside was nice.
11 00:02:30.000 ⇒ 00:02:30.860 Awaish Kumar: Yeah.
12 00:02:31.590 ⇒ 00:02:33.621 Demilade Agboola: Mine was pretty good as well.
13 00:02:34.630 ⇒ 00:02:37.209 Demilade Agboola: The weather wasn’t great, but like it pretty good.
14 00:02:37.580 ⇒ 00:02:38.859 Demilade Agboola: We can all know.
15 00:02:40.370 ⇒ 00:02:48.229 Annie Yu: Nice. I went to a a soccer game, but that was like a 1 to one so. But it was still fun.
16 00:02:49.710 ⇒ 00:02:50.410 Caio Velasco: Did locally.
17 00:02:50.410 ⇒ 00:02:52.930 Awaish Kumar: Yeah, soccer, game.
18 00:02:56.350 ⇒ 00:02:57.610 Caio Velasco: Local soccer, event.
19 00:02:58.150 ⇒ 00:03:04.209 Annie Yu: Yeah. Portland timbers to Seattle sounders. They’re like rivals.
20 00:03:04.210 ⇒ 00:03:05.310 Caio Velasco: Nice. Okay.
21 00:03:09.480 ⇒ 00:03:16.020 Awaish Kumar: Yeah like, and it’s very hot here. So I’ve been just
22 00:03:16.200 ⇒ 00:03:19.940 Awaish Kumar: relaxing at home. It was so hot outside, but.
23 00:03:21.420 ⇒ 00:03:23.169 Annie Yu: How hot is it? Is it.
24 00:03:23.620 ⇒ 00:03:26.939 Awaish Kumar: It’s like almost 50 centigrades.
25 00:03:28.920 ⇒ 00:03:31.130 Annie Yu: Oh, wait! 50 Celsius.
26 00:03:31.540 ⇒ 00:03:32.410 Awaish Kumar: Yes.
27 00:03:32.410 ⇒ 00:03:35.109 Annie Yu: Wow, wow, okay, that’s a lot.
28 00:03:35.280 ⇒ 00:03:38.980 Demilade Agboola: That’s very, very hot. That’s almost desert. Sort of temperatures.
29 00:03:40.690 ⇒ 00:03:46.839 Awaish Kumar: Yes, yeah, it’s it’s quite. It’s getting even more harder day by day.
30 00:03:48.400 ⇒ 00:03:50.319 Caio Velasco: You can fry an egg outside.
31 00:03:53.980 ⇒ 00:04:02.670 Awaish Kumar: Yeah, like, it’s. It’s even better here at some places here they’re very hot, like, even it goes beyond like 54, 55.
32 00:04:04.160 ⇒ 00:04:04.840 Awaish Kumar: It’s correct.
33 00:04:05.310 ⇒ 00:04:10.520 Awaish Kumar: It’s almost like Uae. But yeah, Uae has much better systems, like
34 00:04:11.460 ⇒ 00:04:14.679 Awaish Kumar: for people to operate. Yeah, here, it’s it’s kind of tough.
35 00:04:17.805 ⇒ 00:04:19.479 Awaish Kumar: Yeah, let’s get into it.
36 00:04:21.660 ⇒ 00:04:25.040 Awaish Kumar: Okay, now, like today, I won’t do the talking like
37 00:04:25.300 ⇒ 00:04:28.009 Awaish Kumar: I was just trying to do it for few
38 00:04:28.190 ⇒ 00:04:34.240 Awaish Kumar: days to get things up to speed. But yeah, here, like, like as a regular
39 00:04:34.420 ⇒ 00:04:37.020 Awaish Kumar: stand up, everybody can give their own
40 00:04:37.170 ⇒ 00:04:41.610 Awaish Kumar: updates and what he might want to work on next.
41 00:04:42.400 ⇒ 00:04:43.270 Awaish Kumar: Prove me.
42 00:04:43.500 ⇒ 00:04:46.650 Awaish Kumar: Anybody can can start voluntarily.
43 00:04:50.030 ⇒ 00:04:54.870 Caio Velasco: No, I can go so, for I think what I have on
44 00:04:55.710 ⇒ 00:05:06.680 Caio Velasco: on my plate right now is like the documentation part and the spreadsheets part. The documentation is basically done. I’m finalizing. I’m finishing the
45 00:05:07.540 ⇒ 00:05:13.330 Caio Velasco: pull parts one which I don’t even have much to put, because I couldn’t find any
46 00:05:13.630 ⇒ 00:05:19.660 Caio Velasco: business related information on notion. And the only client page we have, it’s empty.
47 00:05:20.000 ⇒ 00:05:24.660 Caio Velasco: So I mean, I’m just gonna create this structure. And basically.
48 00:05:24.660 ⇒ 00:05:26.979 Awaish Kumar: You can ask Luke. Right, Luke.
49 00:05:27.110 ⇒ 00:05:30.010 Awaish Kumar: You have context about poor parts.
50 00:05:33.220 ⇒ 00:05:41.119 Caio Velasco: Yeah, okay, I was gonna post, and maybe I can tag Luke on the on the slack message. Then he can help me out.
51 00:05:42.560 ⇒ 00:05:44.809 Awaish Kumar: Are you? Yeah, go ahead.
52 00:05:44.810 ⇒ 00:05:48.129 Luke Daque: And yeah, yeah, sure sounds good.
53 00:05:49.470 ⇒ 00:05:57.690 Caio Velasco: And for the spreadsheets. I, as I remember, I think we we have put a deadline for next week.
54 00:05:58.704 ⇒ 00:06:02.736 Caio Velasco: I can already start working tomorrow and Wednesday
55 00:06:03.360 ⇒ 00:06:05.689 Caio Velasco: but then only next week, when I’m back.
56 00:06:07.770 ⇒ 00:06:10.259 Awaish Kumar: Like you. You are off after Wednesday or.
57 00:06:10.890 ⇒ 00:06:12.640 Caio Velasco: I’m off Thursday and Friday.
58 00:06:15.230 ⇒ 00:06:21.389 Awaish Kumar: Okay. So you have, like, 3 days, if like, you can finish it like, I have just put it so
59 00:06:21.890 ⇒ 00:06:26.010 Awaish Kumar: you don’t feel pressure. But yeah, if you can do it, you can. You are.
60 00:06:26.920 ⇒ 00:06:29.460 Awaish Kumar: You’re free to finish it in this week.
61 00:06:30.050 ⇒ 00:06:33.450 Caio Velasco: Okay, I can do it Tuesday and Wednesday. Then.
62 00:06:34.720 ⇒ 00:06:39.380 Awaish Kumar: Okay? And we have maybe one more here for metabol. Right?
63 00:06:42.680 ⇒ 00:06:46.600 Caio Velasco: Yeah, I think from all of them, right? Yeah, for both parts. Yeah.
64 00:06:47.710 ⇒ 00:06:49.330 Awaish Kumar: I can assign to you
65 00:06:55.800 ⇒ 00:07:02.240 Awaish Kumar: like you can ask in the Channel if you need help from anyone.
66 00:07:05.490 ⇒ 00:07:10.460 Caio Velasco: But I think other than that. That’s basically it for me.
67 00:07:11.560 ⇒ 00:07:20.479 Awaish Kumar: So, yeah, so like, after this documentation part I’m working on moving these documents to the
68 00:07:21.370 ⇒ 00:07:28.670 Awaish Kumar: there’s 3 and if if you want to collaborate with me on that? That’s kind of more
69 00:07:28.850 ⇒ 00:07:29.970 Awaish Kumar: tinkle stuff.
70 00:07:31.770 ⇒ 00:07:33.200 Caio Velasco: What part? Sorry.
71 00:07:34.332 ⇒ 00:07:43.850 Awaish Kumar: Like, we have these notion docs and the Google Docs, right? Google sheets you are building. After that, these all needs to be moved to S. 3
72 00:07:44.483 ⇒ 00:07:44.930 Awaish Kumar: like this.
73 00:07:44.930 ⇒ 00:07:46.490 Awaish Kumar: Yes, don’t quite
74 00:07:47.275 ⇒ 00:08:04.499 Awaish Kumar: so I am working on that to move it, move it to S. 3. But as you mentioned. You want to like, get familiarize yourself with more technical stuff. If you want to collaborate on that with me. You have like, you can just let me know, and you can just.
75 00:08:04.857 ⇒ 00:08:08.432 Caio Velasco: I won’t just let me know when for next week.
76 00:08:10.220 ⇒ 00:08:13.110 Awaish Kumar: Okay, calming.
77 00:08:13.940 ⇒ 00:08:19.739 Awaish Kumar: Okay. Next, maybe any you can give updates. So you are already done with this.
78 00:08:20.710 ⇒ 00:08:21.400 Awaish Kumar: That would be.
79 00:08:22.600 ⇒ 00:08:24.810 Awaish Kumar: Do you share the Faqs with no.
80 00:08:26.280 ⇒ 00:08:31.900 Annie Yu: I think I this one, yeah, yeah, the yet. And.
81 00:08:31.900 ⇒ 00:08:32.689 Awaish Kumar: This one is.
82 00:08:32.690 ⇒ 00:08:41.989 Annie Yu: Extended, Kyle, and I am still working on the spreadsheets, but I am on track, so I should be able to get that done either today or tomorrow.
83 00:08:43.110 ⇒ 00:08:43.720 Awaish Kumar: Okay.
84 00:08:44.330 ⇒ 00:08:48.610 Annie Yu: And I’ll send. Send that to Kyle as well. Right. Is that correct?
85 00:08:49.390 ⇒ 00:08:51.680 Awaish Kumar: Yeah, you can just create that sheet and
86 00:08:52.700 ⇒ 00:08:56.899 Awaish Kumar: and and share it in the tip data platform channel.
87 00:08:57.050 ⇒ 00:08:57.800 Annie Yu: Okay.
88 00:08:57.800 ⇒ 00:08:58.370 Annie Yu: Okay.
89 00:08:58.770 ⇒ 00:09:01.119 Awaish Kumar: Yeah. Like, think who?
90 00:09:01.260 ⇒ 00:09:08.739 Awaish Kumar: That’s all for you here, and and anything else you are interested in, like we talked about.
91 00:09:09.430 ⇒ 00:09:15.040 Awaish Kumar: and more technical stuff like moving these sheets to S, 3.
92 00:09:17.030 ⇒ 00:09:20.890 Awaish Kumar: We talked about Meta plan previously. So
93 00:09:21.070 ⇒ 00:09:25.220 Awaish Kumar: are you interested in doing anything apart from documentation.
94 00:09:26.250 ⇒ 00:09:27.650 Annie Yu: Not at the moment.
95 00:09:28.880 ⇒ 00:09:41.059 Awaish Kumar: Okay, like we like, I have a ask, like I with the exact team I have
96 00:09:41.330 ⇒ 00:09:47.939 Awaish Kumar: asked to like, give it like, because you are interested in internal client work. I have asked them to give you like maybe
97 00:09:48.220 ⇒ 00:09:54.109 Awaish Kumar: 5, 10 h between that and even to work on data platform work.
98 00:09:54.380 ⇒ 00:09:58.589 Awaish Kumar: So if you want to work on that, let me know. Otherwise maybe I can
99 00:09:58.850 ⇒ 00:10:04.349 Awaish Kumar: inform them so like so they can adjust your availability in client work.
100 00:10:04.910 ⇒ 00:10:08.770 Annie Yu: And is this for Metaplank? Specifically.
101 00:10:09.800 ⇒ 00:10:19.860 Awaish Kumar: It’s date like this is for data platform work overall, not just a tool. So like, if you have any ideas on a data platform side. You can bring that in.
102 00:10:21.350 ⇒ 00:10:35.710 Awaish Kumar: I have like, said the team, that you I might need you for data platforms, work for 5 to 10 h in a week, maybe. But like, if if you want to move further into client work, then it’s okay, right? We can just adjust that.
103 00:10:36.680 ⇒ 00:10:39.600 Annie Yu: Okay, I’ll I’ll let you know.
104 00:10:41.270 ⇒ 00:10:41.730 Awaish Kumar: Okay.
105 00:10:41.880 ⇒ 00:10:48.640 Annie Yu: Cause. You probably have a a planning session today. Right? So I’ll I’ll let you decide.
106 00:10:48.640 ⇒ 00:10:49.150 Awaish Kumar: Thank you.
107 00:10:50.770 ⇒ 00:10:53.739 Awaish Kumar: Like, this is the session we are doing right now, like
108 00:10:53.950 ⇒ 00:11:01.100 Awaish Kumar: I like right now in my mind, I only have these 2 tasks which I give one want to like, get
109 00:11:01.370 ⇒ 00:11:10.779 Awaish Kumar: done like number one, all the documentation and number 2 this Meta plane for Eden. And but what we like. Why, I’m mentioning
110 00:11:10.820 ⇒ 00:11:31.739 Awaish Kumar: Meta plane because Uttam and I myself we want to roll out this Meta plane for all our other clients. So Luke is busy with fixing it for Eden. So like we can ask you to work on, maybe for for other. Some of the client like ribbon stems or anyone else.
111 00:11:33.390 ⇒ 00:11:43.429 Awaish Kumar: if we want to roll it out for all our existing clients. So that’s the basically plan. So if you are interested. I can assign it to you. Otherwise maybe I, Luke or Demo, we can decide
112 00:11:43.700 ⇒ 00:11:44.910 Awaish Kumar: in between us.
113 00:11:51.320 ⇒ 00:11:55.390 Annie Yu: Yeah, I I wanna say, I’m I’m okay, either way. So.
114 00:11:59.610 ⇒ 00:12:04.020 Awaish Kumar: But like you are not, you don’t have to like, do it alone. Right?
115 00:12:06.340 ⇒ 00:12:10.769 Awaish Kumar: Programming. You can always say, like, Okay, let’s let’s have a
116 00:12:11.030 ⇒ 00:12:20.790 Awaish Kumar: walk. Walk me through it like how to do it like things like that. And I can pair. Luke Luke is has been working on it for for some time. He can pair also.
117 00:12:21.430 ⇒ 00:12:30.580 Annie Yu: Yeah, that would be great. Yeah. So maybe the next relate the ticket. You can include me.
118 00:12:30.710 ⇒ 00:12:34.969 Annie Yu: I can be like shadowing or like pairing with someone else.
119 00:12:38.050 ⇒ 00:12:43.150 Awaish Kumar: Yeah, like, I can assign it to you. And then, you can
120 00:12:43.760 ⇒ 00:12:46.219 Awaish Kumar: collaborate with anyone in the team.
121 00:12:49.960 ⇒ 00:12:56.460 Awaish Kumar: basically. And so maybe I will set up by end of this week.
122 00:12:56.930 ⇒ 00:13:00.360 Awaish Kumar: We have something for my open steams.
123 00:13:02.870 ⇒ 00:13:08.870 Awaish Kumar: I want to extend this bigquery or snowflake downloading.
124 00:13:11.210 ⇒ 00:13:12.860 Demilade Agboola: Robin stems is redshift.
125 00:13:14.220 ⇒ 00:13:22.319 Awaish Kumar: That’s okay. So which one was basically snowflake, it was heated.
126 00:13:25.220 ⇒ 00:13:27.769 Awaish Kumar: Okay, I don’t. I don’t remember what. So like.
127 00:13:28.410 ⇒ 00:13:34.090 Awaish Kumar: Well, let’s I want like something to. Very simple
128 00:13:34.230 ⇒ 00:13:49.029 Awaish Kumar: for for any like for redshift, you might have to create new connections and things like that. Let’s see how it goes, and if you need like, I can be there, because I know, like we have to set up a new source, and it may require us to run some
129 00:13:49.450 ⇒ 00:13:51.309 Awaish Kumar: queries on a client side.
130 00:13:52.340 ⇒ 00:13:59.179 Awaish Kumar: Create some users and passwords and things like that. So yeah, you can just log in and see this stuff. And then we can discuss.
131 00:14:00.260 ⇒ 00:14:08.449 Annie Yu: Okay. Okay? So if I do have any cause, I probably will have many questions. Is it more like a demote question? Or Luke.
132 00:14:10.060 ⇒ 00:14:25.700 Awaish Kumar: Like like for Meta plan specific. You can ask me or Luke like we have worked on. It has not worked on it yet. He might have some knowledge already, but for our urban systems, for specific to client, demote has much more knowledge than us.
133 00:14:26.490 ⇒ 00:14:28.189 Annie Yu: Okay, that’s fair. Okay?
134 00:14:29.730 ⇒ 00:14:32.869 Awaish Kumar: Okay, let’s move on.
135 00:14:33.140 ⇒ 00:14:53.169 Caio Velasco: I wish a quick thing just before the goes, so that I don’t forget. I don’t have access to the other repos. I only have, as in Github, only have access to Javi and pull parts if I’m not wrong. So when I get to the part of Google sheets, I need to have access to the repo, to check.
136 00:14:55.740 ⇒ 00:14:57.587 Awaish Kumar: Yeah, okay, you can.
137 00:15:00.100 ⇒ 00:15:03.339 Awaish Kumar: you can. Ping, maybe I
138 00:15:03.670 ⇒ 00:15:08.429 Awaish Kumar: I don’t have either, after all that. So yeah, you can just bring in data, flip Platform Channel.
139 00:15:09.070 ⇒ 00:15:10.959 Awaish Kumar: Tom and me and.
140 00:15:11.220 ⇒ 00:15:13.840 Caio Velasco: Okay, okay, perfect.
141 00:15:17.080 ⇒ 00:15:17.650 Awaish Kumar: Crooked.
142 00:15:20.750 ⇒ 00:15:22.449 Awaish Kumar: Yep, I’m not in.
143 00:15:24.480 ⇒ 00:15:27.220 Demilade Agboola: So a couple of things for me.
144 00:15:27.905 ⇒ 00:15:35.609 Demilade Agboola: One is the pr summaries demo. So the pr summaries is fine.
145 00:15:36.930 ⇒ 00:15:45.680 Demilade Agboola: I have been able to build it out. Extract the what’s it called the reptile information, because we’ve used
146 00:15:46.772 ⇒ 00:15:50.220 Demilade Agboola: reptile instead of code rabbit.
147 00:15:50.400 ⇒ 00:15:58.139 Demilade Agboola: And then the idea now, or the only thing left really is I I need to be able to create an integration with notion.
148 00:15:58.821 ⇒ 00:16:02.609 Demilade Agboola: But for that I require like ownership.
149 00:16:03.430 ⇒ 00:16:06.789 Demilade Agboola: They I require some access. So I’ve requested that from Utam.
150 00:16:07.920 ⇒ 00:16:09.270 Awaish Kumar: I can send you a message.
151 00:16:09.980 ⇒ 00:16:15.259 Demilade Agboola: Yeah. So once that is done, I will just integrate it so that when pr.
152 00:16:15.370 ⇒ 00:16:23.620 Demilade Agboola: some, when the Pr is merged, it would just send that summary instead of creating a markdown file within the repository, it will send it
153 00:16:24.090 ⇒ 00:16:29.160 Demilade Agboola: to the notion documents instead. So that’s the that’s the final step left.
154 00:16:29.903 ⇒ 00:16:33.710 Demilade Agboola: The other things are in progress, but not. There’s
155 00:16:34.040 ⇒ 00:16:39.510 Demilade Agboola: I plan to finish out the sheet for urban stems by tomorrow and
156 00:16:42.000 ⇒ 00:16:43.150 Demilade Agboola: Let me see.
157 00:16:43.780 ⇒ 00:16:50.119 Demilade Agboola: and then the core metrics for urban stems that’s that’ll be like across this week, cause that the the due date is
158 00:16:50.430 ⇒ 00:16:53.030 Demilade Agboola: at the end of this week, or like middle of this week. So.
159 00:16:53.230 ⇒ 00:16:55.650 Awaish Kumar: I just split it out for you.
160 00:16:55.750 ⇒ 00:16:58.010 Awaish Kumar: It’s the same sheet. But I okay.
161 00:16:58.010 ⇒ 00:17:03.989 Awaish Kumar: print out like the simpler ones. This one is like all other sheets except this long
162 00:17:05.312 ⇒ 00:17:10.800 Awaish Kumar: column with core metrics and lineage like this sheet is takes a bit longer.
163 00:17:10.930 ⇒ 00:17:19.789 Awaish Kumar: So this one is all other simpler things, and then this one. So you can like finish it in this weekly sport, like maybe tomorrow, and then
164 00:17:20.040 ⇒ 00:17:21.300 Awaish Kumar: end of this week.
165 00:17:21.440 ⇒ 00:17:24.469 Awaish Kumar: And for this one you said. It’s just simple.
166 00:17:25.380 ⇒ 00:17:29.493 Awaish Kumar: You just waiting on some permissions
167 00:17:30.480 ⇒ 00:17:47.479 Awaish Kumar: bio thumb, so you can put paste it on notion, Doc, right and it will be like, not just for so once it is done for Eden. So I want like this, you’re okay. You are doing it as a kit of action. Right?
168 00:17:48.100 ⇒ 00:17:53.490 Demilade Agboola: Yes. So once once I get the record the access from Utam.
169 00:17:53.600 ⇒ 00:18:07.220 Demilade Agboola: I would use the Api. So the out integrated the Api into the Github action. So once it is merged, it would just do the entire process and then send the summary to the Api. The ignition Api
170 00:18:07.350 ⇒ 00:18:08.129 Demilade Agboola: would be integrator.
171 00:18:08.130 ⇒ 00:18:15.190 Awaish Kumar: So we need replicas of this ticket as well. Right? Because this is will be for Eden. But
172 00:18:15.360 ⇒ 00:18:21.009 Awaish Kumar: even even we just have to copy paste, but we have to do it for all other clients as well. Right.
173 00:18:21.520 ⇒ 00:18:25.040 Demilade Agboola: Definitely. So I’ll I will create a template version.
174 00:18:26.126 ⇒ 00:18:28.749 Demilade Agboola: So that one will.
175 00:18:29.610 ⇒ 00:18:34.950 Demilade Agboola: So I think the only thing I will need to change. To be honest really is the Api call to the
176 00:18:35.877 ⇒ 00:18:38.799 Demilade Agboola: like one like the notion Api.
177 00:18:39.390 ⇒ 00:18:44.180 Demilade Agboola: exactly, because at the end of the processes are the same, it’s just basically going through the every Pr.
178 00:18:44.180 ⇒ 00:18:51.310 Awaish Kumar: Yeah, yeah. Only the notion. Page Api is going to be diff different. And we just have to copy paste. The code. Right? I.
179 00:18:51.790 ⇒ 00:18:52.110 Demilade Agboola: Yes.
180 00:18:52.679 ⇒ 00:18:58.949 Awaish Kumar: To the to the repository. That’s it. Maybe we can.
181 00:19:00.490 ⇒ 00:19:11.156 Awaish Kumar: Oh, okay, maybe I can just create another ticket, which basically for?
182 00:19:21.460 ⇒ 00:19:24.220 Awaish Kumar: Okay. Should I sign it to you?
183 00:19:24.950 ⇒ 00:19:26.509 Demilade Agboola: Yeah, sure, it’s all good.
184 00:19:27.370 ⇒ 00:19:32.609 Awaish Kumar: Oh, sorry you we can have.
185 00:19:45.050 ⇒ 00:19:46.390 Awaish Kumar: Okay, thank you.
186 00:19:47.150 ⇒ 00:19:49.590 Awaish Kumar: So we are almost set for this week.
187 00:19:50.254 ⇒ 00:19:56.680 Awaish Kumar: Look, we can talk about this Meta and stuff now so like, where are we at
188 00:19:57.140 ⇒ 00:19:58.680 Awaish Kumar: adding test for Eden?
189 00:20:03.420 ⇒ 00:20:08.500 Luke Daque: To be honest, I haven’t really gotten any updates since the last time we
190 00:20:08.680 ⇒ 00:20:14.809 Luke Daque: talked. But yeah, I’m still unable to figure out the Cicd
191 00:20:15.000 ⇒ 00:20:23.679 Luke Daque: issue. I didn’t reach out to the support team from Meta plane, and it looks like they did reply.
192 00:20:24.360 ⇒ 00:20:34.129 Luke Daque: trying giving some advice, or maybe like how to set it up, so I might try to to do that today and see where we can go from there.
193 00:20:34.290 ⇒ 00:20:35.179 Luke Daque: But other than.
194 00:20:35.180 ⇒ 00:20:41.519 Awaish Kumar: What about these 3 like these? These words, like we, we have talked about these 3 things. This should be simple enough.
195 00:20:41.690 ⇒ 00:20:42.579 Awaish Kumar: So is it done.
196 00:20:42.580 ⇒ 00:20:47.230 Luke Daque: That’s good we should have the monitors.
197 00:20:48.140 ⇒ 00:20:52.989 Awaish Kumar: Not all the like I I like. We talked about this, that all the.
198 00:20:52.990 ⇒ 00:20:53.600 Luke Daque: Yeah, yeah.
199 00:20:53.600 ⇒ 00:21:02.829 Awaish Kumar: Custom test in DVD project, and it didn’t, which it’s like just 4 and 5 tests. We just just want to move them over to Meta plan.
200 00:21:04.280 ⇒ 00:21:07.720 Luke Daque: Yeah, I haven’t done that yet. I’ll I’ll work on.
201 00:21:08.220 ⇒ 00:21:09.090 Luke Daque: Good day.
202 00:21:09.410 ⇒ 00:21:10.060 Awaish Kumar: Okay.
203 00:21:10.060 ⇒ 00:21:10.650 Luke Daque: And food.
204 00:21:10.650 ⇒ 00:21:11.070 Awaish Kumar: Only.
205 00:21:11.680 ⇒ 00:21:12.360 Luke Daque: For the source.
206 00:21:12.360 ⇒ 00:21:14.249 Awaish Kumar: This you mentioned. It’s not possible, right?
207 00:21:14.810 ⇒ 00:21:22.209 Luke Daque: Yes, the number 5 1, but the number 4. I’ve already created that as well as number 3. So it’s just that
208 00:21:22.330 ⇒ 00:21:25.440 Luke Daque: the Pr comments, yeah, number 2.
209 00:21:25.440 ⇒ 00:21:26.000 Awaish Kumar: It’s.
210 00:21:26.480 ⇒ 00:21:26.870 Luke Daque: That’s
211 00:21:27.260 ⇒ 00:21:34.030 Awaish Kumar: Okay, so focus on this one. And I think this is possible. But I want to
212 00:21:34.270 ⇒ 00:21:40.620 Awaish Kumar: test it with maybe snowflake some snowflake client. So we can verify.
213 00:21:40.620 ⇒ 00:21:41.480 Awaish Kumar: Okay, cool.
214 00:21:43.130 ⇒ 00:21:49.470 Awaish Kumar: Because I like, I see this. When we add a connection to Github, it’s it does say
215 00:21:50.370 ⇒ 00:21:55.380 Awaish Kumar: asked us to like. Select 2 different databases. Maybe that gives something.
216 00:21:55.790 ⇒ 00:21:57.409 Luke Daque: Let’s try that.
217 00:21:58.510 ⇒ 00:21:59.430 Awaish Kumar: No.
218 00:22:00.390 ⇒ 00:22:03.969 Luke Daque: Read us in the client that’s using Snowflake right.
219 00:22:03.970 ⇒ 00:22:04.740 Awaish Kumar: Yeah.
220 00:22:06.290 ⇒ 00:22:11.629 Awaish Kumar: And then, if not, then I like not here. But we are.
221 00:22:12.180 ⇒ 00:22:17.789 Awaish Kumar: I want to try. Maybe I can try it myself. I have. We have a tool called data diff.
222 00:22:18.180 ⇒ 00:22:22.709 Awaish Kumar: Maybe we can see if we can bring that in. It’s an open source tool.
223 00:22:23.880 ⇒ 00:22:24.570 Luke Daque: Okay.
224 00:22:25.800 ⇒ 00:22:29.659 Awaish Kumar: Okay, so like, when do you think this would be done?
225 00:22:31.469 ⇒ 00:22:36.849 Luke Daque: Let me check, I think, yeah, I can. I can work on that today. So I think we should be
226 00:22:37.600 ⇒ 00:22:39.739 Luke Daque: should have some updates by by then
227 00:22:41.260 ⇒ 00:22:45.870 Luke Daque: for today. So maybe by tomorrow we should have some updates from my end.
228 00:22:45.870 ⇒ 00:22:46.883 Luke Daque: Okay, let’s
229 00:22:48.250 ⇒ 00:22:56.489 Awaish Kumar: Okay, when it’s done, let me know, like we, we want to roll it out for other clients. So after no, before that, let’s
230 00:22:57.190 ⇒ 00:23:08.150 Awaish Kumar: also finalize this thing like so everyone’s input are welcome. We want to finalize the process for
231 00:23:09.030 ⇒ 00:23:19.329 Awaish Kumar: alerting and resolving from the incidents.
232 00:23:19.330 ⇒ 00:23:19.840 Luke Daque: Like the.
233 00:23:19.840 ⇒ 00:23:20.280 Awaish Kumar: So.
234 00:23:20.280 ⇒ 00:23:20.640 Luke Daque: Make more.
235 00:23:22.310 ⇒ 00:23:39.419 Awaish Kumar: Yes. But number one thing is like we have to decide on what data sets to monitor like we don’t want to monitor like, get a rough from like year 4 tables or things like that. So number one, like we want to monitor large data only
236 00:23:40.000 ⇒ 00:23:42.799 Awaish Kumar: for at the at least for for now
237 00:23:42.900 ⇒ 00:23:46.770 Awaish Kumar: number 2, then we want to row
238 00:23:47.620 ⇒ 00:23:53.759 Awaish Kumar: march, Beta, and then we want to decide on thresholds as well like
239 00:23:53.940 ⇒ 00:23:56.950 Awaish Kumar: like. I see this freshness row counters.
240 00:23:57.060 ⇒ 00:24:02.640 Awaish Kumar: and there’s only difference of one row or 10 rows. It it sends in it alert.
241 00:24:03.100 ⇒ 00:24:07.753 Awaish Kumar: so like. And it’s going to spam us. And it’s it’s kind of
242 00:24:08.370 ⇒ 00:24:16.089 Awaish Kumar: known behavior as well because we know order. Summary table is going to have some more rows today than yesterday.
243 00:24:16.250 ⇒ 00:24:17.819 Awaish Kumar: And that’s okay. Right.
244 00:24:17.970 ⇒ 00:24:20.799 Awaish Kumar: So we have to design what?
245 00:24:21.010 ⇒ 00:24:25.330 Awaish Kumar: When it should send us the alerts, not every
246 00:24:25.640 ⇒ 00:24:34.149 Awaish Kumar: changes and all needs to be alerted. And then then we have a process for like who is going to take?
247 00:24:34.280 ⇒ 00:24:43.880 Awaish Kumar: Look at the if there is any alert after these changes, then who is going to look at at them like one person assigned
248 00:24:45.000 ⇒ 00:24:48.469 Awaish Kumar: for a full week, and then we rotate them, or
249 00:24:48.710 ⇒ 00:24:50.840 Awaish Kumar: should it be by client so.
250 00:24:53.350 ⇒ 00:24:56.970 Luke Daque: Yeah, that’s a good point. Because, like, even now.
251 00:24:57.952 ⇒ 00:25:02.889 Luke Daque: we are receiving a lot of alerts in the Meta plane alerts channel. And
252 00:25:03.150 ⇒ 00:25:08.020 Luke Daque: I don’t know if, like all of those need to be worked on right?
253 00:25:09.720 ⇒ 00:25:15.620 Luke Daque: so yeah, we might, we might need to adjust like some of the settings on on the thresholds and stuff.
254 00:25:21.120 ⇒ 00:25:22.230 Awaish Kumar: You know so.
255 00:25:25.600 ⇒ 00:25:31.600 Luke Daque: Because we we like alerts, but, like too many alerts, might also not be beneficial.
256 00:25:34.460 ⇒ 00:25:41.563 Awaish Kumar: So like at large, I will assign assign it to around. But
257 00:25:43.070 ⇒ 00:25:49.099 Awaish Kumar: basically we can. I can like, we all can put our own inputs for this
258 00:25:49.633 ⇒ 00:26:02.650 Awaish Kumar: and then we can share it with Luke, and he maybe prepares a final document out of it. Then we can meet again in in some meeting and finalize that maybe then also shared with with Autum for suggestions.
259 00:26:05.380 ⇒ 00:26:06.379 Luke Daque: Should we also.
260 00:26:06.380 ⇒ 00:26:06.990 Awaish Kumar: So do you.
261 00:26:08.170 ⇒ 00:26:15.719 Luke Daque: The cost of meta plane, like the the more alerts we have is it like related to the cost.
262 00:26:16.700 ⇒ 00:26:21.679 Awaish Kumar: So how the Meta plan like, have you seen the pricing? How they cost us by alert? Or
263 00:26:22.170 ⇒ 00:26:23.279 Awaish Kumar: let’s fix.
264 00:26:23.280 ⇒ 00:26:23.630 Luke Daque: They did.
265 00:26:23.630 ⇒ 00:26:24.350 Awaish Kumar: Question.
266 00:26:25.260 ⇒ 00:26:26.250 Luke Daque: They’re not.
267 00:26:26.560 ⇒ 00:26:32.630 Luke Daque: It doesn’t really necessarily show how much they’re pricing is just like Call us, or something.
268 00:26:37.220 ⇒ 00:26:44.560 Awaish Kumar: Okay, like, that’s a like a separate. Maybe we can just look at it after. Let’s let’s just
269 00:26:45.290 ⇒ 00:26:49.909 Awaish Kumar: focus on like, it’s not really. You have to decide on thresholds here. But like
270 00:26:50.670 ⇒ 00:26:59.300 Awaish Kumar: this, this will be just a kind of a talk motion talk where we decide on what
271 00:26:59.590 ⇒ 00:27:03.330 Awaish Kumar: like, which data we want to monitor at first, st like march data.
272 00:27:03.680 ⇒ 00:27:07.859 Awaish Kumar: or all of the data, or what like, but I prefer is to be the
273 00:27:08.340 ⇒ 00:27:11.220 Awaish Kumar: only the march table. But then.
274 00:27:11.570 ⇒ 00:27:22.770 Awaish Kumar: alerts like we have to decide on thresholds for each client, or maybe for each data set, we define some thresholds, and when to alert or things like that. So?
275 00:27:23.010 ⇒ 00:27:45.700 Awaish Kumar: And how the process for resolving issues like, if there’s a Meta plane alert, then we might like one person, is assigned kind of for a full week, and he’s going to look at it for every client. But if there is a some urgent issue raised by the client that. Okay, we are seeing data issues. Then immediately that’s get assigned to client A,
276 00:27:45.990 ⇒ 00:27:53.370 Awaish Kumar: so you can resolve it faster because there is, might spend some more time understanding what is the issue?
277 00:27:54.110 ⇒ 00:27:59.070 Awaish Kumar: So things like that like, based on urgency, who is going to get?
278 00:27:59.450 ⇒ 00:28:03.602 Awaish Kumar: So we just create create a motion dock for that. And
279 00:28:05.280 ⇒ 00:28:08.789 Awaish Kumar: everybody’s input is welcome. They can comment.
280 00:28:10.600 ⇒ 00:28:15.469 Awaish Kumar: Then we yeah, then we can see, like, what is the.
281 00:28:18.810 ⇒ 00:28:19.710 Luke Daque: Sounds good.
282 00:28:23.420 ⇒ 00:28:29.659 Awaish Kumar: Outcome of that like. After that we would be in a good place to to resolve, actually resolve issues. Right
283 00:28:34.450 ⇒ 00:28:37.789 Awaish Kumar: so far we are getting alerts. We are seeing
284 00:28:38.411 ⇒ 00:28:42.639 Awaish Kumar: some charts and data plan. But yeah, we are not able to take any action.
285 00:28:42.820 ⇒ 00:28:50.729 Awaish Kumar: So this is going to moved us towards taking some action after an an incident is reported.
286 00:28:53.530 ⇒ 00:28:59.350 Awaish Kumar: That’s it for today, and we are on time as well. So thank you. Everyone
287 00:29:00.110 ⇒ 00:29:08.579 Awaish Kumar: for this last ticket like, I encourage everyone to give your thoughts inputs to the team and maybe share it with Luke or or in the Channel.
288 00:29:10.927 ⇒ 00:29:11.970 Caio Velasco: Thank you. Rich.
289 00:29:12.620 ⇒ 00:29:13.469 Demilade Agboola: Thank you. Guys.
290 00:29:13.470 ⇒ 00:29:14.010 Awaish Kumar: Right.
291 00:29:14.010 ⇒ 00:29:15.307 Demilade Agboola: Thank you. Bye.