Meeting Title: Metaplane Data Issues Date: 2025-11-11 Meeting participants: Awaish Kumar, Uttam Kumaran
WEBVTT
1 00:30:28.210 ⇒ 00:30:28.970 Awaish Kumar: Hmm.
2 00:30:29.730 ⇒ 00:30:30.570 Awaish Kumar: Nope.
3 00:30:33.200 ⇒ 00:30:34.400 Awaish Kumar: Do you agree with it?
4 00:30:37.850 ⇒ 00:30:38.860 Awaish Kumar: Lia?
5 00:31:16.300 ⇒ 00:31:21.350 Awaish Kumar: And there are… What?
6 00:31:35.140 ⇒ 00:31:37.480 Awaish Kumar: No more for you.
7 00:33:42.690 ⇒ 00:33:43.480 Awaish Kumar: Thanks.
8 00:34:13.219 ⇒ 00:34:14.620 Awaish Kumar: It is dominant.
9 00:34:14.860 ⇒ 00:34:16.990 Awaish Kumar: Reading Academy, and it’s very interesting.
10 00:34:17.270 ⇒ 00:34:18.000 Awaish Kumar: Nope.
11 00:35:45.740 ⇒ 00:35:47.870 Awaish Kumar: Not after I just moved in.
12 00:35:48.140 ⇒ 00:35:48.840 Awaish Kumar: Huh?
13 00:36:22.720 ⇒ 00:36:26.890 Awaish Kumar: It can be logical.
14 00:37:15.880 ⇒ 00:37:16.279 Awaish Kumar: Adam
15 00:37:19.660 ⇒ 00:37:20.330 Awaish Kumar: Have a look.
16 00:39:08.620 ⇒ 00:39:12.350 Awaish Kumar: Good job once!
17 00:39:12.850 ⇒ 00:39:15.069 Awaish Kumar: Let’s go to the world.
18 00:39:30.330 ⇒ 00:39:32.510 Awaish Kumar: I have updated.
19 00:39:43.340 ⇒ 00:39:44.090 Awaish Kumar: Excellent.
20 00:40:54.690 ⇒ 00:40:55.510 Awaish Kumar: Happy birthday!
21 00:41:04.990 ⇒ 00:41:05.700 Awaish Kumar: I agree.
22 00:42:35.270 ⇒ 00:42:38.630 Awaish Kumar: You gotta do it?
23 00:53:54.810 ⇒ 00:53:55.490 Awaish Kumar: Okay.
24 00:56:40.460 ⇒ 00:56:41.519 Awaish Kumar: You know?
25 01:01:09.040 ⇒ 01:01:12.719 Awaish Kumar: But, thank you.
26 01:01:23.120 ⇒ 01:01:24.270 Awaish Kumar: Ain’t done it at all.
27 01:07:47.650 ⇒ 01:07:48.700 Uttam Kumaran: Hello, sir.
28 01:07:53.130 ⇒ 01:07:54.020 Awaish Kumar: Hello.
29 01:07:57.720 ⇒ 01:07:58.780 Uttam Kumaran: How’s everything?
30 01:07:59.330 ⇒ 01:07:59.910 Awaish Kumar: Oops.
31 01:08:03.180 ⇒ 01:08:04.260 Awaish Kumar: How about you?
32 01:08:05.600 ⇒ 01:08:06.490 Uttam Kumaran: Good.
33 01:08:09.970 ⇒ 01:08:13.680 Uttam Kumaran: Busy. I just had a good interview with another…
34 01:08:13.950 ⇒ 01:08:17.649 Uttam Kumaran: Kind of, like, data engineer, analytics engineer?
35 01:08:17.790 ⇒ 01:08:19.310 Uttam Kumaran: Kind of, like, our level.
36 01:08:19.609 ⇒ 01:08:26.489 Uttam Kumaran: So I may have you talk to him Tomorrow, maybe you and Demolade.
37 01:08:26.710 ⇒ 01:08:31.149 Uttam Kumaran: But… Yeah, things are good.
38 01:08:33.279 ⇒ 01:08:34.079 Awaish Kumar: Okay.
39 01:08:44.969 ⇒ 01:08:47.179 Awaish Kumar: to Paulo.
40 01:09:24.460 ⇒ 01:09:27.969 Uttam Kumaran: Okay, cool. So, yeah, walk me through what you’re thinking.
41 01:09:29.170 ⇒ 01:09:41.130 Awaish Kumar: So, basically, regarding… like, I will move… we will come to Metaplane with no staging, like, I…
42 01:09:41.279 ⇒ 01:09:49.489 Awaish Kumar: I think runs are… runtime is… Getting, like, like, shorter and shorter.
43 01:09:50.950 ⇒ 01:09:53.260 Awaish Kumar: For the trading runs, to…
44 01:09:53.689 ⇒ 01:09:59.470 Awaish Kumar: And some of it is because of, like, the… some of the old models are not incremental.
45 01:09:59.820 ⇒ 01:10:00.670 Awaish Kumar: So, I will…
46 01:10:00.670 ⇒ 01:10:01.330 Uttam Kumaran: Okay.
47 01:10:01.330 ⇒ 01:10:05.930 Awaish Kumar: I have, like… Made some of them incremental, and
48 01:10:07.840 ⇒ 01:10:11.679 Awaish Kumar: also, like, have one schema now, and I also copied all…
49 01:10:12.140 ⇒ 01:10:15.749 Awaish Kumar: Tables for production to staging, and also dev analytics, so…
50 01:10:15.750 ⇒ 01:10:16.330 Uttam Kumaran: Okay.
51 01:10:16.720 ⇒ 01:10:18.330 Awaish Kumar: The runs will be smoother.
52 01:10:19.260 ⇒ 01:10:20.100 Uttam Kumaran: Okay.
53 01:10:20.100 ⇒ 01:10:31.830 Awaish Kumar: But I… I don’t know, like, if I should spend time on this. I had a question, which is, like, so I can see in the… in the documentation that I can… we can select, like,
54 01:10:32.050 ⇒ 01:10:35.320 Awaish Kumar: Different databases for staging and production.
55 01:10:35.880 ⇒ 01:10:38.899 Awaish Kumar: For the Redshift, LDT.
56 01:10:39.570 ⇒ 01:10:40.270 Uttam Kumaran: Okay.
57 01:10:40.920 ⇒ 01:10:44.460 Awaish Kumar: But I don’t know if I can separate out the raw.
58 01:10:44.800 ⇒ 01:10:50.669 Awaish Kumar: I… I’m thinking about this because, like, we now… right now, we only have one database.
59 01:10:50.770 ⇒ 01:10:56.689 Awaish Kumar: Which is being accessed by quite a lot of people, and then a lot of applications.
60 01:10:56.800 ⇒ 01:10:57.800 Awaish Kumar: And that…
61 01:10:57.800 ⇒ 01:10:58.220 Uttam Kumaran: Okay.
62 01:10:58.220 ⇒ 01:10:59.679 Awaish Kumar: It’s low as well.
63 01:11:01.210 ⇒ 01:11:02.510 Awaish Kumar: So, multiple people…
64 01:11:02.510 ⇒ 01:11:03.219 Uttam Kumaran: Oh, okay.
65 01:11:03.220 ⇒ 01:11:06.109 Awaish Kumar: Same input system, same server.
66 01:11:07.040 ⇒ 01:11:09.920 Awaish Kumar: Even in the same database, like,
67 01:11:10.070 ⇒ 01:11:16.469 Awaish Kumar: And then, like, all our PRs, our staging, dev, everything is going into the same database.
68 01:11:17.280 ⇒ 01:11:17.950 Uttam Kumaran: Okay.
69 01:11:18.760 ⇒ 01:11:31.670 Awaish Kumar: So, one of the way, like, one of the things I’ve been thinking of is if I can somehow, like, I know, like, they have staging and production, their databases are possible, that’s…
70 01:11:32.250 ⇒ 01:11:32.859 Awaish Kumar: Not a problem.
71 01:11:32.860 ⇒ 01:11:33.440 Uttam Kumaran: Yeah.
72 01:11:33.440 ⇒ 01:11:39.610 Awaish Kumar: I don’t know if I can also create a raw, so that raw data lives at this one place.
73 01:11:39.930 ⇒ 01:11:40.910 Awaish Kumar: You know…
74 01:11:40.910 ⇒ 01:11:44.410 Uttam Kumaran: Oh, don’t we already have… Oh, everything’s going to analytics, you mean?
75 01:11:44.570 ⇒ 01:11:46.300 Awaish Kumar: Everything is going to analytics.
76 01:11:46.300 ⇒ 01:11:49.899 Uttam Kumaran: But I guess, like, how does that… how does that change any, like.
77 01:11:51.050 ⇒ 01:11:55.230 Awaish Kumar: I guess, tell me what that affects, because will that really affect the job?
78 01:11:55.260 ⇒ 01:11:56.240 Uttam Kumaran: Speed?
79 01:11:56.240 ⇒ 01:12:01.870 Awaish Kumar: affects this, like, the… obviously the database’s capability, like, the system has…
80 01:12:02.180 ⇒ 01:12:12.819 Awaish Kumar: maybe 4 CPUs, 8 cores, I think, like, 16 cores, maybe? So, like, it has its own limit… hardware limitations, right?
81 01:12:13.190 ⇒ 01:12:14.910 Awaish Kumar: And then…
82 01:12:14.910 ⇒ 01:12:15.700 Uttam Kumaran: Okay.
83 01:12:16.250 ⇒ 01:12:22.960 Awaish Kumar: So multiple requests are coming, multiple connections are being created, maybe some of the connections left over.
84 01:12:23.080 ⇒ 01:12:27.820 Awaish Kumar: So they’ll create, kind of a…
85 01:12:28.850 ⇒ 01:12:31.729 Awaish Kumar: Load on the database, and it can make it slow.
86 01:12:32.440 ⇒ 01:12:33.170 Uttam Kumaran: Okay.
87 01:12:33.470 ⇒ 01:12:36.190 Awaish Kumar: while executing queries. So yeah, that’s the…
88 01:12:37.540 ⇒ 01:12:40.120 Awaish Kumar: That’s why I haven’t been thinking about it. I’m…
89 01:12:40.350 ⇒ 01:12:44.010 Awaish Kumar: Not sure how significant an impact it will make, but…
90 01:12:44.740 ⇒ 01:12:48.970 Awaish Kumar: It can… it will improve some of the… Hmm.
91 01:12:50.260 ⇒ 01:12:52.040 Awaish Kumar: Like, execution times.
92 01:12:53.710 ⇒ 01:13:00.379 Uttam Kumaran: Okay, so that’s one thing. I would say probably more of the priority right now is getting the…
93 01:13:01.760 ⇒ 01:13:05.880 Uttam Kumaran: Basically, getting alerts on the core… .
94 01:13:06.380 ⇒ 01:13:07.440 Awaish Kumar: Yeah, so…
95 01:13:08.190 ⇒ 01:13:09.090 Uttam Kumaran: Yeah.
96 01:13:09.090 ⇒ 01:13:17.160 Awaish Kumar: Yeah, for that, I also have, like… I’m seeing, since being on the upper stem, like, I see only… we are getting
97 01:13:17.410 ⇒ 01:13:26.199 Awaish Kumar: like, pings only for revenue and inventory dashboards, which are, like, kind of active departments.
98 01:13:26.710 ⇒ 01:13:30.689 Awaish Kumar: Or… these are, like, making you…
99 01:13:30.980 ⇒ 01:13:40.409 Awaish Kumar: trying… like, I’m more, like, concerned about these… these two things. I’ve added a lot of, like, monitors already on the revenue.
100 01:13:40.650 ⇒ 01:13:45.529 Uttam Kumaran: We have the inventory. Like, we did do have someone in inventory as well.
101 01:13:45.530 ⇒ 01:13:51.700 Awaish Kumar: But I can recheck, like, if we are not missing any of the inventory tables. And the news is that…
102 01:13:51.980 ⇒ 01:13:57.049 Awaish Kumar: the… the issues they shared, like, MetaBlit did capture it.
103 01:13:57.330 ⇒ 01:14:00.820 Awaish Kumar: So, like, if I can click on the… go on these links.
104 01:14:01.140 ⇒ 01:14:09.090 Awaish Kumar: So, like, it shows, like, fact orders, basically, is inflated by… since yesterday. But the only problem was that
105 01:14:09.230 ⇒ 01:14:17.289 Awaish Kumar: Because I try to, reduce the sensitivity, sensitivity, so that it doesn’t spam us.
106 01:14:17.630 ⇒ 01:14:19.259 Awaish Kumar: It didn’t notify us in Slack.
107 01:14:19.950 ⇒ 01:14:21.840 Uttam Kumaran: Okay. Okay, okay.
108 01:14:23.720 ⇒ 01:14:24.650 Uttam Kumaran: Alright.
109 01:14:24.880 ⇒ 01:14:27.749 Uttam Kumaran: So, I guess my point is, like.
110 01:14:27.920 ⇒ 01:14:32.020 Uttam Kumaran: I want to move into an environment where like…
111 01:14:32.670 ⇒ 01:14:38.660 Uttam Kumaran: We are getting alerted each time, like, and each alert
112 01:14:39.250 ⇒ 01:14:47.060 Uttam Kumaran: Basically, someone says, I’m gonna investigate, and then someone says, this is an issue, I need to alert the client.
113 01:14:48.430 ⇒ 01:14:55.050 Uttam Kumaran: So, part of this is process, and part of this is establishing with our whole team that
114 01:14:55.170 ⇒ 01:15:04.580 Uttam Kumaran: These alerts channels exist, and if someone is on call, Right? So… the on-call, I guess.
115 01:15:07.620 ⇒ 01:15:15.160 Uttam Kumaran: you can tell me what you think. Like, maybe for… at least for Eden, Urban Stems, we need someone on call each week.
116 01:15:15.610 ⇒ 01:15:18.880 Uttam Kumaran: That kind of, like, rotates. Their job is to go
117 01:15:19.980 ⇒ 01:15:23.409 Uttam Kumaran: Anytime that an alert comes, their job’s a triage.
118 01:15:23.540 ⇒ 01:15:26.360 Uttam Kumaran: Their job is not to fix it, necessarily.
119 01:15:26.530 ⇒ 01:15:27.690 Uttam Kumaran: triage?
120 01:15:28.730 ⇒ 01:15:37.149 Uttam Kumaran: clearly explain what the issue is, what the recommended fix is, and then what’s… if… do we need to communicate to the client? Then.
121 01:15:37.470 ⇒ 01:15:39.620 Uttam Kumaran: Anybody on the team can execute the work.
122 01:15:39.740 ⇒ 01:15:47.979 Uttam Kumaran: Right? So, I guess the first part of that is just… is making… me being able to say, anytime there is an alert.
123 01:15:48.560 ⇒ 01:15:50.950 Uttam Kumaran: A ticket is gonna get created for triage.
124 01:15:51.770 ⇒ 01:15:54.850 Uttam Kumaran: And then, who is it assigned to? Who’s on call?
125 01:15:55.670 ⇒ 01:16:02.669 Uttam Kumaran: So do we… I guess, do we feel like all the alerts going into Urban STEM’s channel and for Eden are valid at this point?
126 01:16:05.400 ⇒ 01:16:07.560 Awaish Kumar: Yes.
127 01:16:07.850 ⇒ 01:16:11.859 Uttam Kumaran: The alternative is we… As a client group.
128 01:16:12.120 ⇒ 01:16:15.870 Uttam Kumaran: We take a look at it during stand-up once a week, you know?
129 01:16:16.260 ⇒ 01:16:31.630 Awaish Kumar: No, like, we… we have to, like, compromise on… on… on a thing, like… like, we will start spending time on it, and I can say, like, 5% of our time can go on false positives. That’s, like…
130 01:16:31.630 ⇒ 01:16:32.150 Uttam Kumaran: Okay.
131 01:16:32.790 ⇒ 01:16:39.990 Awaish Kumar: We have to… make it, because, like, for example, I have a, like, row count monitor.
132 01:16:40.090 ⇒ 01:16:49.020 Awaish Kumar: Sometimes it is a real outlier. Sometimes the number of rows just in inventory increased
133 01:16:50.100 ⇒ 01:16:57.400 Awaish Kumar: which was not in the Python, that model can realize, but… it basically…
134 01:16:58.820 ⇒ 01:17:04.540 Awaish Kumar: is a real inventory, so, like, things like that. So it’s a real notification, but it’s still,
135 01:17:05.350 ⇒ 01:17:08.010 Awaish Kumar: No, no, like, action on that.
136 01:17:08.470 ⇒ 01:17:09.150 Uttam Kumaran: Okay.
137 01:17:10.080 ⇒ 01:17:10.880 Uttam Kumaran: Okay.
138 01:17:12.460 ⇒ 01:17:18.669 Uttam Kumaran: So, I guess, like, I guess, can you pull up, like, can we pull up both of those channels, and…
139 01:17:18.940 ⇒ 01:17:19.760 Uttam Kumaran: like…
140 01:17:20.500 ⇒ 01:17:20.850 Awaish Kumar: Does that…
141 01:17:20.850 ⇒ 01:17:25.560 Uttam Kumaran: I would just like to… like, let’s pull up the external… let’s pull up Urban Stems Alerts.
142 01:17:27.880 ⇒ 01:17:31.820 Uttam Kumaran: So today, for example, there’s this Metaplane daily update.
143 01:17:32.750 ⇒ 01:17:34.289 Uttam Kumaran: Can we get rid of that?
144 01:17:34.880 ⇒ 01:17:39.290 Uttam Kumaran: And similarly, look, there’s a job issue, and there’s a duration issue.
145 01:17:39.490 ⇒ 01:17:47.809 Uttam Kumaran: these… I guess today, looks like there’s 4 messages in here, or 3 messages in here, so I guess, like, that would be my question, is, like, what…
146 01:17:48.240 ⇒ 01:17:56.050 Awaish Kumar: So, like, I don’t know who, but somebody created these… dbt job run monitors.
147 01:17:57.670 ⇒ 01:18:03.670 Awaish Kumar: It’s monitoring the execution time of our… production jobs on DBT.
148 01:18:04.410 ⇒ 01:18:05.070 Uttam Kumaran: Okay.
149 01:18:06.040 ⇒ 01:18:09.419 Awaish Kumar: So, the duration monitors are those, and the…
150 01:18:10.810 ⇒ 01:18:13.839 Uttam Kumaran: So the duration monitor, but see, like, let’s look at this one.
151 01:18:14.640 ⇒ 01:18:18.519 Uttam Kumaran: This one is… Expected is 2 minutes.
152 01:18:18.790 ⇒ 01:18:20.839 Uttam Kumaran: And the actual 7 minutes.
153 01:18:21.620 ⇒ 01:18:22.080 Awaish Kumar: Yeah.
154 01:18:22.080 ⇒ 01:18:24.470 Uttam Kumaran: I don’t care about anything that’s running
155 01:18:25.100 ⇒ 01:18:27.590 Uttam Kumaran: Less than 30 minutes, at this point.
156 01:18:28.460 ⇒ 01:18:33.960 Awaish Kumar: Okay, yeah, that, that adjustment I can make, right? We… We can have…
157 01:18:37.410 ⇒ 01:18:47.669 Awaish Kumar: like, for each of these jobs, I can basically add… manually add, like, what are the valid values, the ranges. Okay.
158 01:18:48.030 ⇒ 01:18:50.809 Awaish Kumar: So that is… that is an easy fix.
159 01:18:51.190 ⇒ 01:18:55.370 Awaish Kumar: for Redshift, like… We have these monitors.
160 01:18:59.430 ⇒ 01:19:00.390 Awaish Kumar: So…
161 01:19:00.560 ⇒ 01:19:09.990 Awaish Kumar: So, like, basically, we got today notification for two things, and we both cap… both of these are captured, like, one on the fact order, revenue seems…
162 01:19:12.590 ⇒ 01:19:14.560 Awaish Kumar: Like, the inflator?
163 01:19:14.690 ⇒ 01:19:22.289 Awaish Kumar: Secondly, on inventory, mod table, row count is also inflated here.
164 01:19:23.410 ⇒ 01:19:24.090 Uttam Kumaran: Okay.
165 01:19:24.350 ⇒ 01:19:31.470 Awaish Kumar: Have… But, like, I don’t know why I’m not getting notifications, although it is connected with Slack.
166 01:19:35.450 ⇒ 01:19:37.909 Uttam Kumaran: Well, this… because this incident happened on 8-11.
167 01:19:38.960 ⇒ 01:19:41.780 Uttam Kumaran: So… oh, it’s today. Okay.
168 01:19:43.710 ⇒ 01:19:44.340 Awaish Kumar: Hell yeah.
169 01:19:44.340 ⇒ 01:19:45.040 Uttam Kumaran: Well…
170 01:19:45.740 ⇒ 01:19:51.810 Uttam Kumaran: Oh, I see. Okay, so would you say this is a va… but see, would you say this is valid? Because…
171 01:19:53.710 ⇒ 01:19:55.229 Uttam Kumaran: I don’t know, it seems like…
172 01:19:56.970 ⇒ 01:19:59.190 Uttam Kumaran: Well, it seems like something here happened, right?
173 01:20:02.450 ⇒ 01:20:03.160 Awaish Kumar: Yeah.
174 01:20:18.340 ⇒ 01:20:22.690 Awaish Kumar: Maybe… maybe it’s not in the… Who they are talking about.
175 01:20:28.460 ⇒ 01:20:31.200 Awaish Kumar: But, like, this seems valid here.
176 01:20:36.400 ⇒ 01:20:37.910 Awaish Kumar: If we look, look at this.
177 01:20:38.010 ⇒ 01:20:39.600 Awaish Kumar: some monitor?
178 01:20:40.150 ⇒ 01:20:41.000 Awaish Kumar: for revenue.
179 01:20:41.000 ⇒ 01:20:41.590 Uttam Kumaran: Yeah.
180 01:20:45.090 ⇒ 01:20:47.279 Uttam Kumaran: Okay, yeah, that’s a clear issue, right?
181 01:20:47.750 ⇒ 01:20:48.380 Awaish Kumar: Yep.
182 01:21:14.820 ⇒ 01:21:21.200 Uttam Kumaran: Okay, so I guess today… so can we note down all of the currently active issues, and we can investigate?
183 01:21:22.630 ⇒ 01:21:23.610 Awaish Kumar: Okay, yeah.
184 01:21:25.030 ⇒ 01:21:30.780 Uttam Kumaran: Right? So, I’m gonna say active investigations.
185 01:21:33.090 ⇒ 01:21:35.790 Uttam Kumaran: So yeah, or let’s just put it in Notion.
186 01:21:36.150 ⇒ 01:21:39.879 Uttam Kumaran: So I just want to note down what all the active investigations are.
187 01:21:40.170 ⇒ 01:21:44.699 Uttam Kumaran: and sort of, like, what our takeaways are. And so the other thing is, like.
188 01:21:45.210 ⇒ 01:21:50.259 Uttam Kumaran: Can we… oh, so we want to keep this daily… okay, whatever, we can keep the daily update, that’s fine.
189 01:21:51.160 ⇒ 01:21:57.950 Uttam Kumaran: So, let’s say an active investigation, so the first one,
190 01:21:58.610 ⇒ 01:22:03.080 Uttam Kumaran: The first one we have is… yeah, if you just want to list them out, I can take a note.
191 01:22:04.510 ⇒ 01:22:06.400 Awaish Kumar: Like, let me…
192 01:22:07.410 ⇒ 01:22:08.919 Uttam Kumaran: the fact orders…
193 01:22:10.440 ⇒ 01:22:11.120 Awaish Kumar: Thank you.
194 01:22:11.240 ⇒ 01:22:14.150 Uttam Kumaran: So, I guess my other thing, if you go to… if you go to Overview.
195 01:22:16.040 ⇒ 01:22:17.320 Uttam Kumaran: If you go to Overview?
196 01:22:22.510 ⇒ 01:22:26.750 Uttam Kumaran: like… Click on Analytics.
197 01:22:33.410 ⇒ 01:22:37.190 Uttam Kumaran: I guess my question is, if you scroll down… Here.
198 01:22:37.840 ⇒ 01:22:41.359 Uttam Kumaran: Are… do all of these tables need monitors on them?
199 01:22:41.780 ⇒ 01:22:46.510 Awaish Kumar: So… I only have them on the…
200 01:22:46.650 ⇒ 01:22:50.939 Awaish Kumar: ones which has downstream importance. Not all of them.
201 01:22:51.210 ⇒ 01:22:51.830 Uttam Kumaran: Okay.
202 01:22:52.250 ⇒ 01:22:53.290 Uttam Kumaran: Okay, okay.
203 01:22:56.090 ⇒ 01:23:02.539 Uttam Kumaran: Okay, so for rev… so for fact orders, there is an accrued… Revenue metric increase.
204 01:23:03.680 ⇒ 01:23:05.370 Uttam Kumaran: What other ones do we have?
205 01:23:07.190 ⇒ 01:23:15.870 Awaish Kumar: Like, on effect orders, we have, adjustment space table, Suborders with lots.
206 01:23:19.830 ⇒ 01:23:24.529 Awaish Kumar: And we’re in marked inventory adjustment and marked inventory lot hourly changes.
207 01:23:24.830 ⇒ 01:23:31.699 Awaish Kumar: So, basically… Both on the inventory lot and the revenue tables.
208 01:23:35.160 ⇒ 01:23:37.899 Uttam Kumaran: Okay, one second, let me write that down.
209 01:23:38.910 ⇒ 01:23:42.480 Uttam Kumaran: So… Inventory lot.
210 01:23:44.750 ⇒ 01:23:46.230 Uttam Kumaran: Or what’s the table name?
211 01:23:46.410 ⇒ 01:23:49.390 Uttam Kumaran: Oh, mart inventory law changes, Mart?
212 01:23:52.010 ⇒ 01:23:53.530 Uttam Kumaran: Oh, all of those.
213 01:23:53.700 ⇒ 01:23:56.120 Uttam Kumaran: Okay, yeah, if you want to just paste those in.
214 01:24:00.400 ⇒ 01:24:00.940 Awaish Kumar: Wait.
215 01:24:02.240 ⇒ 01:24:02.840 Awaish Kumar: So.
216 01:24:22.040 ⇒ 01:24:26.230 Awaish Kumar: Let me change it to the… Numbered list.
217 01:25:24.440 ⇒ 01:25:26.780 Uttam Kumaran: Okay, so I’m gonna put,
218 01:25:47.860 ⇒ 01:25:59.520 Uttam Kumaran: I’m gonna put manual for the dbt jobs, and I’m gonna put… At or above… 45 minutes.
219 01:26:00.380 ⇒ 01:26:01.200 Uttam Kumaran: For now.
220 01:26:02.110 ⇒ 01:26:03.400 Uttam Kumaran: Let’s start there.
221 01:26:04.650 ⇒ 01:26:07.440 Awaish Kumar: Are you writing somewhere or anything?
222 01:26:08.160 ⇒ 01:26:10.759 Uttam Kumaran: Well, I just changed it in Metaplane.
223 01:26:11.130 ⇒ 01:26:11.800 Awaish Kumar: Okay.
224 01:26:12.590 ⇒ 01:26:13.949 Awaish Kumar: The duration.
225 01:26:14.480 ⇒ 01:26:15.090 Uttam Kumaran: Yeah.
226 01:26:15.370 ⇒ 01:26:15.950 Awaish Kumar: Okay.
227 01:26:16.710 ⇒ 01:26:30.620 Uttam Kumaran: So, I’m gonna say… Updates here. One adjusted… Job duration, Monitor… To… greater than…
228 01:26:31.300 ⇒ 01:26:33.130 Awaish Kumar: Greater than 45 minutes.
229 01:26:34.000 ⇒ 01:26:34.830 Uttam Kumaran: Okay.
230 01:26:35.720 ⇒ 01:26:41.890 Awaish Kumar: So far, which one, like… this is the one that maybe needs… to be under…
231 01:26:42.850 ⇒ 01:26:44.330 Awaish Kumar: Yeah, the items in my…
232 01:26:44.330 ⇒ 01:26:47.139 Uttam Kumaran: But see, I already… but this should be already covered.
233 01:26:49.320 ⇒ 01:26:49.870 Awaish Kumar: Weird.
234 01:26:49.870 ⇒ 01:26:51.169 Uttam Kumaran: I just changed it.
235 01:26:55.140 ⇒ 01:27:00.479 Uttam Kumaran: So, that’s why for this one, I’m gonna… I’m gonna mark this incident
236 01:27:16.260 ⇒ 01:27:17.190 Uttam Kumaran: like…
237 01:27:18.360 ⇒ 01:27:21.499 Awaish Kumar: Like, you change for all of them somewhere, or…
238 01:27:22.210 ⇒ 01:27:24.850 Uttam Kumaran: Yeah, I just… I went to the, oh, wait.
239 01:27:25.110 ⇒ 01:27:27.700 Awaish Kumar: I can’t see the ginger.
240 01:27:27.700 ⇒ 01:27:28.670 Uttam Kumaran: go to Rules.
241 01:27:34.520 ⇒ 01:27:35.190 Awaish Kumar: Okay.
242 01:27:35.830 ⇒ 01:27:37.090 Uttam Kumaran: Yeah, click on that.
243 01:27:39.230 ⇒ 01:27:40.780 Uttam Kumaran: And then go to monitors.
244 01:27:42.930 ⇒ 01:27:44.860 Uttam Kumaran: Or go to Overview and go to Edit.
245 01:27:44.980 ⇒ 01:27:45.960 Uttam Kumaran: On the bottom.
246 01:27:53.420 ⇒ 01:27:54.780 Uttam Kumaran: And then click on this.
247 01:27:59.360 ⇒ 01:28:00.020 Uttam Kumaran: Yeah.
248 01:28:02.800 ⇒ 01:28:06.340 Awaish Kumar: Okay, so you’ve added a rule for all the dbt jobs.
249 01:28:06.970 ⇒ 01:28:07.690 Uttam Kumaran: Yeah.
250 01:28:10.720 ⇒ 01:28:11.420 Awaish Kumar: Okay.
251 01:28:13.420 ⇒ 01:28:17.730 Awaish Kumar: But then, is this valid for all, like…
252 01:28:18.880 ⇒ 01:28:22.140 Awaish Kumar: Some of the monitors are, like, running for…
253 01:28:23.970 ⇒ 01:28:26.939 Awaish Kumar: More than 30 minutes, and it says a valid time.
254 01:28:28.090 ⇒ 01:28:34.070 Uttam Kumaran: So, but that’s, I guess, my point, is that we should be dec… all of them should get pushed down.
255 01:28:34.290 ⇒ 01:28:37.240 Uttam Kumaran: To… at least 45, you know?
256 01:28:54.560 ⇒ 01:28:57.869 Uttam Kumaran: So then, yeah, looks like there is a job failure this morning.
257 01:28:58.480 ⇒ 01:29:03.039 Uttam Kumaran: So this one is worth investigating, right? Or this is like, oh, it went to 11 minutes.
258 01:29:03.140 ⇒ 01:29:08.240 Uttam Kumaran: But see, this is not valid, because this… the whole monitor is, like, not accurate.
259 01:29:08.620 ⇒ 01:29:15.090 Uttam Kumaran: Like, I don’t care whether this runs for 5 minutes or 15 minutes, doesn’t matter to me. So, I’m gonna mark as normal.
260 01:29:15.670 ⇒ 01:29:17.820 Awaish Kumar: Yeah, but, like…
261 01:29:18.680 ⇒ 01:29:24.039 Awaish Kumar: I’m… I’m thinking they’re just these values per job, like, we don’t have a lot of them.
262 01:29:25.770 ⇒ 01:29:30.679 Uttam Kumaran: Yeah, but I guess this is where, like, I don’t want… I just don’t care at the moment. It’s not a priority.
263 01:29:30.990 ⇒ 01:29:31.810 Uttam Kumaran: Right?
264 01:29:32.710 ⇒ 01:29:37.200 Uttam Kumaran: As long as things aren’t taking more than 45 minutes, we don’t have much to worry about.
265 01:29:37.200 ⇒ 01:29:43.199 Awaish Kumar: It’s okay, but for some of them which are taking more than 45 minutes,
266 01:29:43.630 ⇒ 01:29:45.839 Awaish Kumar: Should we get notified? Like, is that a priority?
267 01:29:45.840 ⇒ 01:29:49.120 Uttam Kumaran: Yeah, we should, I mean, we should, we should attack it, right?
268 01:29:49.460 ⇒ 01:29:56.709 Uttam Kumaran: Okay. That should be our next task, yeah. We can decrease those. So I want to start by saying 45, next one we’ll do 30.
269 01:29:57.030 ⇒ 01:29:57.770 Awaish Kumar: Okay.
270 01:29:58.860 ⇒ 01:29:59.749 Uttam Kumaran: See what I mean?
271 01:30:01.150 ⇒ 01:30:01.790 Awaish Kumar: Okay.
272 01:30:03.030 ⇒ 01:30:06.669 Awaish Kumar: And I also looked at the Looker monitors, and…
273 01:30:06.780 ⇒ 01:30:10.030 Awaish Kumar: There’s no way we can add monitors, like, on…
274 01:30:10.140 ⇒ 01:30:14.070 Awaish Kumar: If it is updated or not, or fresh, like, freshness test.
275 01:30:14.960 ⇒ 01:30:19.980 Awaish Kumar: It’s just like… we can only track the views of the dashboard on a view.
276 01:30:21.480 ⇒ 01:30:23.040 Awaish Kumar: For the local monitors.
277 01:30:23.610 ⇒ 01:30:24.280 Uttam Kumaran: Okay.
278 01:31:11.630 ⇒ 01:31:16.120 Uttam Kumaran: Okay, so those are all things I think we can talk about today.
279 01:31:17.600 ⇒ 01:31:22.639 Uttam Kumaran: And so, I guess I’m okay with that, but how do you feel about Eden? Like.
280 01:31:22.890 ⇒ 01:31:25.479 Uttam Kumaran: Wondering if we can get a similar setup there.
281 01:31:28.850 ⇒ 01:31:33.440 Awaish Kumar: like, for Edin, oh, actually, I…
282 01:31:33.600 ⇒ 01:31:37.369 Awaish Kumar: did some work on Metaplane, and that is, like.
283 01:31:38.670 ⇒ 01:31:41.719 Awaish Kumar: That is working, but the only thing is that…
284 01:31:41.910 ⇒ 01:31:44.760 Awaish Kumar: After we made quite a lot of
285 01:31:45.080 ⇒ 01:31:48.319 Awaish Kumar: We added a couple of new models and things, and…
286 01:31:48.480 ⇒ 01:31:52.140 Awaish Kumar: And we didn’t maintain Metaplane up to that level.
287 01:31:52.290 ⇒ 01:31:54.759 Awaish Kumar: So I can add more.
288 01:31:55.280 ⇒ 01:31:55.910 Uttam Kumaran: Okay.
289 01:31:56.380 ⇒ 01:32:03.740 Awaish Kumar: Like, monitors, but, yeah, we just need a process of driving, like, we sometimes get notifications, and…
290 01:32:04.870 ⇒ 01:32:07.990 Awaish Kumar: And getting involved. We need process for that.
291 01:32:09.250 ⇒ 01:32:10.020 Uttam Kumaran: Okay.
292 01:32:10.460 ⇒ 01:32:15.150 Uttam Kumaran: Alright, so for both of those, we need… process.
293 01:32:15.810 ⇒ 01:32:21.020 Uttam Kumaran: Are we… do we feel like… I mean, is this something you think you can own? Like, what do you think?
294 01:32:23.400 ⇒ 01:32:27.420 Uttam Kumaran: Because I guess I don’t need you to own solving the problems.
295 01:32:27.870 ⇒ 01:32:34.130 Uttam Kumaran: But I do need you to own… like, error… Reliability.
296 01:32:37.900 ⇒ 01:32:39.240 Uttam Kumaran: Look, what do you think?
297 01:32:42.510 ⇒ 01:32:45.859 Awaish Kumar: Like, adjusting the monitors and things like that.
298 01:32:46.250 ⇒ 01:32:49.290 Uttam Kumaran: more teaching people how to do the monitors, right? But…
299 01:32:49.640 ⇒ 01:32:53.950 Uttam Kumaran: I want to start with these two clients, then I want to make this available for more clients.
300 01:32:54.120 ⇒ 01:32:57.109 Awaish Kumar: Yeah, I can own that, and I’ll, like…
301 01:32:57.240 ⇒ 01:33:02.129 Awaish Kumar: like, I’m more… like, what about, like, on-call thing?
302 01:33:03.390 ⇒ 01:33:08.580 Uttam Kumaran: On call, we’ll decide tomorrow morning, so I’ll… I’ll nominate someone on each client team to be on call.
303 01:33:08.720 ⇒ 01:33:09.520 Uttam Kumaran: you know.
304 01:33:10.420 ⇒ 01:33:11.260 Awaish Kumar: Okay.
305 01:33:12.890 ⇒ 01:33:16.409 Uttam Kumaran: So for urban stems, it’ll rotate between me, you, and Debilade.
306 01:33:17.390 ⇒ 01:33:21.190 Uttam Kumaran: on Eden, it’ll rotate as well.
307 01:33:24.120 ⇒ 01:33:25.989 Awaish Kumar: But, okay.
308 01:33:32.810 ⇒ 01:33:33.630 Uttam Kumaran: What do you think?
309 01:33:34.320 ⇒ 01:33:38.149 Awaish Kumar: Yeah, like, I’m okay with the idea, I’m just concerned, like.
310 01:33:38.260 ⇒ 01:33:44.940 Awaish Kumar: sometimes, like, majority of time, I get… messages when, like, I’m already…
311 01:33:45.210 ⇒ 01:33:49.240 Awaish Kumar: allowed off, like, around maybe 2, 3 AM, 11.
312 01:33:49.240 ⇒ 01:33:49.760 Uttam Kumaran: Yeah.
313 01:33:49.760 ⇒ 01:33:55.810 Awaish Kumar: Computer, and then… Next morning, I open the computer, and there’s a lot of… Messages?
314 01:33:56.890 ⇒ 01:33:58.630 Awaish Kumar: Fire, like…
315 01:33:59.670 ⇒ 01:34:03.940 Uttam Kumaran: But I guess my point would be, you’re not gonna be on call, Every week.
316 01:34:04.630 ⇒ 01:34:05.480 Uttam Kumaran: Right.
317 01:34:05.740 ⇒ 01:34:06.819 Uttam Kumaran: So I guess…
318 01:34:08.430 ⇒ 01:34:12.709 Awaish Kumar: Yeah, but yeah, but that’s what I’m saying, like, even, like…
319 01:34:12.830 ⇒ 01:34:15.849 Awaish Kumar: Or, like, all the week, if I’m on the…
320 01:34:16.540 ⇒ 01:34:21.460 Awaish Kumar: On, on call, like, after 2 or 3… Yeah, I’m, like, I’m.
321 01:34:21.460 ⇒ 01:34:23.070 Uttam Kumaran: Or you think we should do two people?
322 01:34:25.240 ⇒ 01:34:28.250 Awaish Kumar: I’m saying 2AM here, it’s, like, almost 4…
323 01:34:28.850 ⇒ 01:34:29.440 Uttam Kumaran: Yeah.
324 01:34:29.440 ⇒ 01:34:36.770 Awaish Kumar: Eastern time, so it almost the day end, but after that, normally I’m not… Like, focused on…
325 01:34:37.550 ⇒ 01:34:38.400 Uttam Kumaran: Yeah, yeah.
326 01:34:46.700 ⇒ 01:34:47.540 Uttam Kumaran: Okay.
327 01:34:48.020 ⇒ 01:34:48.800 Awaish Kumar: Like, we could…
328 01:34:48.800 ⇒ 01:34:54.519 Uttam Kumaran: Let’s… Let’s talk about it tomorrow with everybody. I don’t know what to do. Yeah, it’s tough.
329 01:34:55.760 ⇒ 01:34:59.110 Awaish Kumar: No, but, like, Raquel said, like, maybe…
330 01:34:59.350 ⇒ 01:35:02.379 Awaish Kumar: Every day, like, the person who’s on call.
331 01:35:02.610 ⇒ 01:35:05.119 Awaish Kumar: Like, the first half, like, until…
332 01:35:05.410 ⇒ 01:35:08.859 Awaish Kumar: Maybe 2, 3 p.m. Eastern.
333 01:35:09.010 ⇒ 01:35:12.500 Awaish Kumar: He is responsible for traging the…
334 01:35:13.550 ⇒ 01:35:16.980 Awaish Kumar: Issues, and we can communicate with the client that we are
335 01:35:17.150 ⇒ 01:35:21.709 Awaish Kumar: And the first half of each day, we are… Like, keep it in mind.
336 01:35:21.710 ⇒ 01:35:24.890 Uttam Kumaran: Yeah, so we have a plan for business hours and off hours.
337 01:35:25.850 ⇒ 01:35:26.670 Uttam Kumaran: Okay.
338 01:35:27.510 ⇒ 01:35:29.430 Uttam Kumaran: Okay, let’s discuss it tomorrow.
339 01:35:29.610 ⇒ 01:35:32.949 Uttam Kumaran: With everybody, and I can send a little bit of a summary of this meeting.
340 01:35:33.710 ⇒ 01:35:38.040 Uttam Kumaran: Do you want to take this transcript and maybe put a… you can put a little summary
341 01:35:38.380 ⇒ 01:35:41.610 Uttam Kumaran: In, like, internal documentation?
342 01:35:42.190 ⇒ 01:35:49.539 Uttam Kumaran: I think it would be great if we have a document just on data reliability, and we can summarize this meeting, kind of like what our outcomes are.
343 01:35:50.750 ⇒ 01:35:51.500 Awaish Kumar: Okay.
344 01:35:53.790 ⇒ 01:35:56.320 Uttam Kumaran: Okay, cool. Alright, let’s jump to the other meeting.
345 01:35:56.560 ⇒ 01:35:57.500 Awaish Kumar: Okay, sure.
346 01:35:57.500 ⇒ 01:35:58.500 Uttam Kumaran: Okay, thanks.