Meeting Title: Zoom Meeting Date: 2025-04-07 Meeting participants: Demilade Agboola, Awaish Kumar, Uttam Kumaran
WEBVTT
1 00:00:21.990 ⇒ 00:00:23.130 Uttam Kumaran: Hey! Weish.
2 00:00:24.010 ⇒ 00:00:24.770 Awaish Kumar: Hello!
3 00:00:26.200 ⇒ 00:00:27.590 Awaish Kumar: How are you doing.
4 00:00:28.020 ⇒ 00:00:29.000 Uttam Kumaran: How’s it going.
5 00:00:31.093 ⇒ 00:00:33.640 Awaish Kumar: It’s good.
6 00:00:35.410 ⇒ 00:00:38.069 Uttam Kumaran: I say that one more time. It’s sort of cutting out a little bit.
7 00:00:44.920 ⇒ 00:00:45.730 Awaish Kumar: Hello! Can you hear me?
8 00:00:54.110 ⇒ 00:00:55.669 Uttam Kumaran: It’s cutting out a little bit.
9 00:01:11.565 ⇒ 00:01:13.195 Awaish Kumar: Okay. Now.
10 00:01:14.010 ⇒ 00:01:15.310 Uttam Kumaran: Yes, it’s better now.
11 00:01:17.070 ⇒ 00:01:21.550 Awaish Kumar: Okay, yeah, I said, it’s all going good. How about you.
12 00:01:21.550 ⇒ 00:01:23.126 Uttam Kumaran: Yeah, it’s going well,
13 00:01:23.720 ⇒ 00:01:30.799 Uttam Kumaran: excited to start this month off we just you know, sent some messages, some potential new clients. So
14 00:01:31.030 ⇒ 00:01:33.260 Uttam Kumaran: yeah, it’s it’s going. Well, I’m excited.
15 00:01:37.676 ⇒ 00:01:41.820 Uttam Kumaran: Yeah, I just wanted to have this meeting to sort of chat through data quality
16 00:01:41.930 ⇒ 00:01:46.080 Uttam Kumaran: sort of issues. And then anything we can do today on the even side sort of like
17 00:01:46.390 ⇒ 00:01:49.540 Uttam Kumaran: assist with like setting up tests, or
18 00:01:50.090 ⇒ 00:01:52.680 Uttam Kumaran: I don’t know. I was talking to dev a Lotte and I just wanted to
19 00:01:53.400 ⇒ 00:02:01.510 Uttam Kumaran: try to find a way for us to all like, basically make some decisions on the data quality side. And then we can start to have that across all clients, basically.
20 00:02:06.390 ⇒ 00:02:07.370 Awaish Kumar: Okay.
21 00:02:12.770 ⇒ 00:02:14.620 Uttam Kumaran: What do you think is the
22 00:02:14.860 ⇒ 00:02:18.669 Uttam Kumaran: like? I mean? I I suggested some items, but
23 00:02:18.840 ⇒ 00:02:22.899 Uttam Kumaran: I mean, what do you think is like a best path forward right now. Just short term for Eden.
24 00:02:31.560 ⇒ 00:02:33.760 Awaish Kumar: Okay, did you wrote it somewhere.
25 00:02:34.940 ⇒ 00:02:35.953 Awaish Kumar: Yeah, right.
26 00:02:36.460 ⇒ 00:02:40.980 Uttam Kumaran: Yeah. But I guess just overall wanted to get your your thoughts.
27 00:02:42.240 ⇒ 00:02:44.500 Awaish Kumar: Yeah, like, I, I think the
28 00:02:44.850 ⇒ 00:02:48.689 Awaish Kumar: the best way is like number one thing which we could write
29 00:02:53.057 ⇒ 00:02:58.319 Awaish Kumar: some tests like some Dbg test, but on our metric level,
30 00:03:00.220 ⇒ 00:03:03.940 Awaish Kumar: to Co. And compare it to like. For example, we can write some test
31 00:03:04.160 ⇒ 00:03:11.464 Awaish Kumar: which calculates the metric on staging and production, and compares it them together. And
32 00:03:12.660 ⇒ 00:03:18.319 Awaish Kumar: if there is the significant difference, it can like fail the check.
33 00:03:18.520 ⇒ 00:03:22.010 Awaish Kumar: So we we know before it, merging it.
34 00:03:23.670 ⇒ 00:03:25.300 Uttam Kumaran: Hmm, okay. Okay.
35 00:03:25.890 ⇒ 00:03:32.289 Awaish Kumar: So, for example, like and Cac is, was dependent on 2 things like they already spend.
36 00:03:32.540 ⇒ 00:03:34.289 Awaish Kumar: and the new customer count.
37 00:03:34.470 ⇒ 00:03:42.579 Awaish Kumar: So either we can directly calculate Ncac, or we can just take the spend in new customer count and see if there is significant difference
38 00:03:42.980 ⇒ 00:03:44.800 Awaish Kumar: in any of them like.
39 00:03:45.140 ⇒ 00:03:55.659 Awaish Kumar: then we can just in in these values between staging and prod. We can just fail those checks and and like block the Pr from from machines.
40 00:03:56.690 ⇒ 00:03:57.125 Uttam Kumaran: Okay.
41 00:03:58.253 ⇒ 00:03:59.640 Awaish Kumar: Further investigation.
42 00:03:59.640 ⇒ 00:04:05.369 Uttam Kumaran: So then that’s all. Just like Dbt tests. I guess my other point was like, what about like dashboard tests?
43 00:04:05.500 ⇒ 00:04:08.818 Uttam Kumaran: Right? So I was. I was telling them a lot of that we should have
44 00:04:09.490 ⇒ 00:04:13.680 Uttam Kumaran: we should basically create a staging dashboard on top of the staging data as well.
45 00:04:15.498 ⇒ 00:04:22.559 Awaish Kumar: Yeah, that like that is also like th, that is useful
46 00:04:22.790 ⇒ 00:04:25.899 Awaish Kumar: to for investigation after that, because
47 00:04:26.590 ⇒ 00:04:33.270 Awaish Kumar: whenever we make a Pr like, it’s possible like we when I have lots of
48 00:04:34.162 ⇒ 00:04:42.619 Awaish Kumar: con things to context switch. Then maybe I could. I don’t review it thoroughly, or whatever. For some reason, miss something
49 00:04:43.118 ⇒ 00:04:52.029 Awaish Kumar: in in a staging dashboard like I made a change. I want to review. I have to review the staging dashboard before I merge the Pr. I may
50 00:04:52.290 ⇒ 00:04:57.530 Awaish Kumar: over oversighted, or like maybe miss something, but having these
51 00:04:58.422 ⇒ 00:05:05.189 Awaish Kumar: checks like we, we don’t enforces like you cannot just merge it, but we can like it could be a useful
52 00:05:05.340 ⇒ 00:05:09.216 Awaish Kumar: in knowing that. Okay, the check failed. There means there is significant
53 00:05:09.680 ⇒ 00:05:14.870 Awaish Kumar: change. We should investigate it and have proper answer before merging.
54 00:05:15.120 ⇒ 00:05:21.070 Awaish Kumar: and when in when we are in this scenario, we, these staging dashboard, can be useful in.
55 00:05:21.370 ⇒ 00:05:23.500 Awaish Kumar: in, in investigating or
56 00:05:24.305 ⇒ 00:05:33.740 Awaish Kumar: in in giving a second eye. Okay, what is the difference? And is, is it if it looks good and also sharing them with analysts? So to review is will be much easier
57 00:05:34.449 ⇒ 00:05:42.229 Awaish Kumar: when we have our staging dashboards, but I think these both should be like together. Not a singular
58 00:05:42.630 ⇒ 00:05:44.360 Awaish Kumar: like, not a single item.
59 00:05:44.800 ⇒ 00:05:45.450 Uttam Kumaran: Okay?
60 00:05:47.680 ⇒ 00:05:51.210 Uttam Kumaran: So I guess in the in the short term like, what do you think are
61 00:05:51.660 ⇒ 00:05:56.180 Uttam Kumaran: our options are? I mean, should we just prioritize like creating
62 00:05:56.590 ⇒ 00:06:06.240 Uttam Kumaran: tests for Eden. And then cause this is the thing we also have to create the run book. Basically, right? So my question is like, yes, we can set up tests.
63 00:06:06.560 ⇒ 00:06:11.870 Uttam Kumaran: We can have the test come to slack. But are we gonna should we should we basically
64 00:06:12.150 ⇒ 00:06:15.860 Uttam Kumaran: have like an on call process, or what? How do you think we should handle.
65 00:06:17.770 ⇒ 00:06:23.730 Awaish Kumar: Okay, like this Pr stuff like the task which went last last digit.
66 00:06:23.910 ⇒ 00:06:27.169 Uttam Kumaran: Yeah, but not just that. Like, let’s just say like, so that Pr.
67 00:06:27.170 ⇒ 00:06:28.009 Awaish Kumar: The Pr process.
68 00:06:28.010 ⇒ 00:06:34.129 Uttam Kumaran: Would have got caught by a test right? The test would have alerted. Then who would have? Who would have been like, Okay, it’s it’s like.
69 00:06:34.560 ⇒ 00:06:39.469 Uttam Kumaran: should we do one person among us that’s on call or like? How do you think we should do it?
70 00:06:43.280 ⇒ 00:06:50.789 Awaish Kumar: Yeah, so what? I’m, what? I’m like, the what i’m right now talking about is not
71 00:06:51.482 ⇒ 00:06:53.170 Awaish Kumar: data quality on
72 00:06:53.290 ⇒ 00:07:02.570 Awaish Kumar: like number one thing is that we have data quality checks on existing things right? There is like 2 points here, number one.
73 00:07:02.710 ⇒ 00:07:20.340 Awaish Kumar: that the ex we have existing flow. And there can be some some data quality issues in existing flow. Right? That is something different. Right? But the other thing is that a breaking change like we bring some updates that will affect something
74 00:07:20.880 ⇒ 00:07:23.929 Awaish Kumar: so like there should be different
75 00:07:24.080 ⇒ 00:07:31.200 Awaish Kumar: process to handle both like 2 2 different process to handle those 2 things when we have
76 00:07:31.350 ⇒ 00:07:40.740 Awaish Kumar: this Pr thing like when we make a change, that and and we we we don’t know how. Maybe it can affect other things or
77 00:07:41.980 ⇒ 00:07:49.199 Awaish Kumar: and maybe we neglect some. It can break something, and for that we can seal.
78 00:07:49.656 ⇒ 00:08:01.310 Awaish Kumar: We have a staging dashboard in a like. The 1st step could be to have have a staging dashboard which which could be easiest, because we have to copy the dashboards and change the data source right.
79 00:08:01.420 ⇒ 00:08:11.469 Awaish Kumar: so it could be the if the like 1st step could be this one that we we move on with the having staging dashboards, and we review
80 00:08:11.988 ⇒ 00:08:18.799 Awaish Kumar: like like the Pr reviews. We also get this staging like the overall dashboard reviews after the change.
81 00:08:19.100 ⇒ 00:08:25.340 Awaish Kumar: So, and then we merge the Pr right? So this is this is one way to handle it.
82 00:08:25.610 ⇒ 00:08:27.880 Awaish Kumar: Then we can add the checks
83 00:08:28.250 ⇒ 00:08:35.010 Awaish Kumar: for that in the Pr process as well, which can automatically suggest something which can be in a long term
84 00:08:35.230 ⇒ 00:08:52.399 Awaish Kumar: solution and for the overall data quality, that’s all have always been challenging that we want to have, how we want to have it, because if we just send the notifications as you know, it can just spam us
85 00:08:52.610 ⇒ 00:08:55.250 Awaish Kumar: and the number one way. I I
86 00:08:55.420 ⇒ 00:09:01.690 Awaish Kumar: I want. I tried to handle it in my one of the previous companies was Create, create the data quality dashboard.
87 00:09:02.170 ⇒ 00:09:03.680 Awaish Kumar: right? So.
88 00:09:04.145 ⇒ 00:09:04.610 Uttam Kumaran: Yes!
89 00:09:04.610 ⇒ 00:09:10.320 Awaish Kumar: Instead of instead of alerting, it will show us, like, Okay, total count
90 00:09:11.014 ⇒ 00:09:29.580 Awaish Kumar: total, like revenue, or the customer count, or whatever order count is going this like this in a dashboard. I can see. Okay, it’s like increasing by a constant number. And then it’s significantly increased by 100% or like 500% like, okay, there’s some outlier here.
91 00:09:29.690 ⇒ 00:09:31.919 Awaish Kumar: So we can review that and find out. Okay.
92 00:09:32.040 ⇒ 00:09:41.740 Awaish Kumar: whenever we find some outliers we? We can see that in data quality dashboard. And we then take create a ticket for that and investigate
93 00:09:43.950 ⇒ 00:09:50.410 Awaish Kumar: like. That’s that’s 1 way to solve it without spamming. And
94 00:09:50.550 ⇒ 00:09:54.690 Awaish Kumar: when we get a lot of alerts and spams, then it’s difficult.
95 00:09:54.850 ⇒ 00:10:00.960 Awaish Kumar: as you know, that, like we after some time, we just ignore those messages. Because okay for me.
96 00:10:02.370 ⇒ 00:10:02.940 Uttam Kumaran: I mean, but.
97 00:10:02.940 ⇒ 00:10:03.680 Awaish Kumar: That’s the thing.
98 00:10:03.680 ⇒ 00:10:06.390 Awaish Kumar: So we we only want to have tests that are like.
99 00:10:06.520 ⇒ 00:10:11.570 Uttam Kumaran: Okay, if this test goes, everything has to stop. Basically right?
100 00:10:11.570 ⇒ 00:10:12.150 Awaish Kumar: Okay.
101 00:10:12.340 ⇒ 00:10:17.239 Uttam Kumaran: We only want to have high high high priority tests.
102 00:10:20.190 ⇒ 00:10:23.753 Uttam Kumaran: Let me. Yeah. So one of the things that
103 00:10:24.747 ⇒ 00:10:27.220 Uttam Kumaran: hey, demo idea. Sorry I didn’t notice you were a
104 00:10:28.181 ⇒ 00:10:36.140 Uttam Kumaran: you are here. Yeah. So in the past we have, we actually, we use elementary a bit. It’s kind of quite expensive.
105 00:10:36.820 ⇒ 00:10:39.720 Uttam Kumaran: We’re testing foundational right now.
106 00:10:40.435 ⇒ 00:10:43.360 Uttam Kumaran: Which? And they just sent me like, that’s something.
107 00:10:46.180 ⇒ 00:10:50.539 Uttam Kumaran: That something on their end is working like, I’m gonna I’m gonna check. But
108 00:10:51.098 ⇒ 00:10:58.460 Uttam Kumaran: basically, I think the 1st piece is just the nail. Can we add tests and then do those test alerts
109 00:10:58.640 ⇒ 00:10:59.820 Uttam Kumaran: come somewhere.
110 00:10:59.960 ⇒ 00:11:03.310 Uttam Kumaran: Right? So let me just share
111 00:11:03.630 ⇒ 00:11:05.959 Uttam Kumaran: this. And this is something we worked on
112 00:11:06.400 ⇒ 00:11:11.619 Uttam Kumaran: last year, and we haven’t really taken a lot of action on it. But basically, we said.
113 00:11:11.730 ⇒ 00:11:16.189 Uttam Kumaran: we said that no tests with a severe, with a severity being less than high
114 00:11:17.970 ⇒ 00:11:22.840 Uttam Kumaran: meaning as soon as it hits the slack channel. It needs to get triaged.
115 00:11:24.740 ⇒ 00:11:31.670 Uttam Kumaran: you know. And so and the second piece is like the alert should actually be able to send what like? What is the failure?
116 00:11:32.717 ⇒ 00:11:34.119 Uttam Kumaran: Like, what table?
117 00:11:34.290 ⇒ 00:11:39.860 Uttam Kumaran: And then, basically, we just really really want to confirm that
118 00:11:40.936 ⇒ 00:11:45.630 Uttam Kumaran: all the tests that get sent, or what’s called a game breaking alerts, basically, which is like
119 00:11:46.474 ⇒ 00:11:50.519 Uttam Kumaran: and one of the things we talked about was like, do we need to have a goalie system?
120 00:11:51.417 ⇒ 00:11:58.190 Uttam Kumaran: For every client, basically. Or do we have like someone on call with it on every client that has to just triage.
121 00:11:58.410 ⇒ 00:12:01.299 Uttam Kumaran: which is just like, Hey, you have to go through.
122 00:12:01.600 ⇒ 00:12:08.049 Uttam Kumaran: assign this ticket and then escalate if if needed, I think, for now it can be like maybe the 3 of us
123 00:12:08.876 ⇒ 00:12:14.799 Uttam Kumaran: and then we can. The 3 of us can handle it for maybe one or 2 weeks, and then we can see how we wanna
124 00:12:15.460 ⇒ 00:12:21.749 Uttam Kumaran: assign everything to to other to other analytics engineers. Right? So an analyst, basically.
125 00:12:24.900 ⇒ 00:12:29.089 Uttam Kumaran: So that seems like a good point. I mean, if I was just to take notes on our
126 00:12:30.675 ⇒ 00:12:38.670 Uttam Kumaran: conversation today. Like.
127 00:12:38.870 ⇒ 00:12:40.380 Uttam Kumaran: One thing I would say is.
128 00:12:40.560 ⇒ 00:12:43.200 Uttam Kumaran: if we were to have like the dues.
129 00:12:43.700 ⇒ 00:12:57.409 Uttam Kumaran: One is like tickets for adding game breaking tests for Eden, sending those alerts
130 00:13:00.330 ⇒ 00:13:02.409 Uttam Kumaran: to the new alerts channel.
131 00:13:05.360 ⇒ 00:13:09.790 Uttam Kumaran: The last piece is thinking.
132 00:13:10.730 ⇒ 00:13:14.069 Uttam Kumaran: The the 3rd piece is a data quality dashboard.
133 00:13:15.980 ⇒ 00:13:27.160 Uttam Kumaran: This, this has common things like comparing raw with Martz, comparing legacy models with new models.
134 00:13:28.538 ⇒ 00:13:34.169 Uttam Kumaran: Overall counts and sums data freshness.
135 00:13:36.120 ⇒ 00:13:39.109 Uttam Kumaran: So these seem like pretty good short term
136 00:13:41.530 ⇒ 00:13:43.740 Uttam Kumaran: fixes like we could probably do this this week.
137 00:13:44.300 ⇒ 00:13:45.580 Awaish Kumar: Right.
138 00:13:47.697 ⇒ 00:13:48.739 Awaish Kumar: What do you.
139 00:13:48.740 ⇒ 00:13:49.430 Uttam Kumaran: Guys think.
140 00:13:51.616 ⇒ 00:13:56.949 Awaish Kumar: Yeah, but like for one and point number one and 2, like, maybe we
141 00:13:57.310 ⇒ 00:14:00.539 Awaish Kumar: like the the thing I mentioned that we want to have it in a
142 00:14:01.880 ⇒ 00:14:09.270 Awaish Kumar: on the time when we want to ship any changes like you know what like
143 00:14:09.700 ⇒ 00:14:12.279 Awaish Kumar: at the time of Pr reviews, right.
144 00:14:12.687 ⇒ 00:14:22.460 Uttam Kumaran: Yeah. So then test test run during pr review the Icd right?
145 00:14:22.840 ⇒ 00:14:24.150 Awaish Kumar: 1st is brought right.
146 00:14:26.000 ⇒ 00:14:26.660 Uttam Kumaran: Hmm.
147 00:14:27.630 ⇒ 00:14:32.669 Awaish Kumar: Like kind of make tests which compares staging versus production and highlight.
148 00:14:32.670 ⇒ 00:14:33.120 Uttam Kumaran: Yeah.
149 00:14:33.120 ⇒ 00:14:35.280 Awaish Kumar: There is a significant change yet.
150 00:14:36.300 ⇒ 00:14:40.050 Uttam Kumaran: Yeah. So then I guess this is where it’s like, what are the what are the tests?
151 00:14:42.400 ⇒ 00:14:45.040 Uttam Kumaran: The tests are gonna be like comparing
152 00:14:45.280 ⇒ 00:14:49.309 Uttam Kumaran: tables me and dum a lot, I said. Mainly we should be comparing.
153 00:14:49.770 ⇒ 00:14:55.750 Uttam Kumaran: I guess. Yeah, we could be comparing staging versus production.
154 00:14:57.358 ⇒ 00:15:01.959 Uttam Kumaran: I think it’s probably more healthy to compare like raw versus
155 00:15:02.790 ⇒ 00:15:08.559 Uttam Kumaran: production or something. Right? I I don’t know. This is where it’s like, what if there is a Pr that has to remove.
156 00:15:09.370 ⇒ 00:15:11.019 Uttam Kumaran: Yeah, like, when we have a.
157 00:15:11.630 ⇒ 00:15:23.040 Awaish Kumar: Like the production is already running right for the Pr test. We want to see that the changes we have made are they how they are, how much they are different from production.
158 00:15:27.990 ⇒ 00:15:28.909 Uttam Kumaran: Yeah, I agree.
159 00:15:28.910 ⇒ 00:15:38.959 Awaish Kumar: If I add a filter, if I add a filter like how like it’s going to impact the total number of revenue or totally total number of orders, and the total revenue
160 00:15:40.370 ⇒ 00:15:43.030 Awaish Kumar: alright compared to production.
161 00:15:46.500 ⇒ 00:15:51.500 Demilade Agboola: I mean, yeah, we could compare it to production. I I think it’s just a thing of
162 00:15:52.460 ⇒ 00:15:54.639 Demilade Agboola: if we’re trying to.
163 00:15:55.860 ⇒ 00:15:57.840 Demilade Agboola: I think we’re trying to like
164 00:15:58.320 ⇒ 00:16:03.900 Demilade Agboola: if you have the raw tables, and we know what the raw tables should constantly give us. So, for instance, we know
165 00:16:05.170 ⇒ 00:16:10.120 Demilade Agboola: the number of orders consistently in our table is this amount?
166 00:16:10.735 ⇒ 00:16:21.459 Demilade Agboola: We can just reference it to the actual raw table, so we can say the sum of all other accounting, like product summary by transaction, should be equals to the count.
167 00:16:22.035 ⇒ 00:16:26.550 Demilade Agboola: Of all orders from this table, and we can create those aggregation in such a way that
168 00:16:28.640 ⇒ 00:16:31.969 Demilade Agboola: like, even within the same system, we can
169 00:16:32.210 ⇒ 00:16:37.820 Demilade Agboola: easily flag when things are like things, the numbers are very different.
170 00:16:40.880 ⇒ 00:16:43.440 Demilade Agboola: But there’s also no problem, also comparing to production.
171 00:16:47.890 ⇒ 00:16:49.140 Uttam Kumaran: Hmm.
172 00:16:52.410 ⇒ 00:17:01.959 Demilade Agboola: I’m also just thinking part of the reasons why, you know ideally, we might want to compare within the same system is so that if anything does happen in production for some weird reason.
173 00:17:06.230 ⇒ 00:17:07.819 Demilade Agboola: we could also catch it.
174 00:17:10.230 ⇒ 00:17:11.490 Uttam Kumaran: Yeah, I agree.
175 00:17:15.390 ⇒ 00:17:18.769 Uttam Kumaran: This one just seems a little bit more complicated than
176 00:17:21.560 ⇒ 00:17:23.020 Uttam Kumaran: yeah. I see what you mean.
177 00:17:26.209 ⇒ 00:17:32.849 Awaish Kumar: Like what I’m saying, that we have just simple tests like edit.
178 00:17:33.449 ⇒ 00:17:38.319 Awaish Kumar: At the time when we are having a a Pr, we just
179 00:17:38.889 ⇒ 00:17:42.219 Awaish Kumar: like, see the top level, maybe metrics.
180 00:17:42.369 ⇒ 00:17:45.499 Awaish Kumar: or like or top level fields, which are
181 00:17:45.899 ⇒ 00:17:56.099 Awaish Kumar: which are being used in metrics like for 18, like revenue. And Ltv. These are important things, and maybe there are 8, Max. 7, 8 fields
182 00:17:56.309 ⇒ 00:18:01.979 Awaish Kumar: which are responsible to calculate these all. So we can just add, for those 7 8 fields.
183 00:18:02.119 ⇒ 00:18:08.619 Awaish Kumar: add some test, compare it staging versus broad after every Pr and see if if that impacts.
184 00:18:08.919 ⇒ 00:18:15.946 Awaish Kumar: And if we see that okay, there is greater than 10% difference. Okay, let’s review it.
185 00:18:17.198 ⇒ 00:18:27.349 Awaish Kumar: let’s edit in a review before merging, and then do a review find some answers for this big change, and if it is justified, then we merge the Pr. Otherwise we we block it.
186 00:18:30.019 ⇒ 00:18:30.739 Awaish Kumar: Hold on.
187 00:18:32.000 ⇒ 00:18:36.180 Uttam Kumaran: I see, but that’s but that’s the thing, I guess. So this is like a non blocking.
188 00:18:37.380 ⇒ 00:18:39.060 Uttam Kumaran: That’s a non-blocking test.
189 00:18:40.598 ⇒ 00:18:49.300 Awaish Kumar: Yeah, like testing itself doesn’t block. But we know that if check failed, then it’s there’s a need to further review before merging.
190 00:18:54.250 ⇒ 00:18:56.690 Uttam Kumaran: Okay? So we can also come. We can also try that.
191 00:18:57.260 ⇒ 00:19:01.310 Uttam Kumaran: But there is also, like, non breaking.
192 00:19:02.300 ⇒ 00:19:09.700 Uttam Kumaran: I see the track X total pro account aging.
193 00:19:29.600 ⇒ 00:19:33.879 Uttam Kumaran: Okay? And then long term, we definitely wanna want to have like.
194 00:19:35.570 ⇒ 00:19:37.210 Uttam Kumaran: we need to have like run books.
195 00:19:38.630 ⇒ 00:19:42.800 Uttam Kumaran: We need to think through like on call strategy.
196 00:19:47.010 ⇒ 00:19:56.780 Uttam Kumaran: Basically like, consider a platform or testing foundational
197 00:19:58.354 ⇒ 00:20:02.489 Uttam Kumaran: data fold, etc, like some of the ones that we’re testing.
198 00:20:20.750 ⇒ 00:20:21.370 Uttam Kumaran: Hmm.
199 00:20:21.370 ⇒ 00:20:28.720 Demilade Agboola: Out of curiosity when you say game breaking test like the number one you mean like. If the test fails, everything should the entire run should fail.
200 00:20:29.360 ⇒ 00:20:29.950 Uttam Kumaran: Yeah.
201 00:20:30.090 ⇒ 00:20:33.419 Uttam Kumaran: Well, no, this just means that like.
202 00:20:35.020 ⇒ 00:20:41.210 Uttam Kumaran: there is a a game. Breaking basically means like, we’re not gonna have any alerts go to slack that aren’t like a p. 1
203 00:20:42.680 ⇒ 00:20:46.409 Uttam Kumaran: brings it, for example, like, if there’s a test that says, Hey, they eaten.
204 00:20:46.650 ⇒ 00:20:48.929 Uttam Kumaran: The Eden Ncac is negative.
205 00:20:49.400 ⇒ 00:20:56.340 Uttam Kumaran: There’s no world in which that should be the case. So it’s we’re gonna get a slack, alert right?
206 00:21:00.200 ⇒ 00:21:02.619 Uttam Kumaran: That’s what I’m sort of considering.
207 00:21:05.770 ⇒ 00:21:06.205 Demilade Agboola: Okay.
208 00:21:09.165 ⇒ 00:21:14.770 Uttam Kumaran: That way. It it we we don’t get hit with like a hundred alerts every day to start with.
209 00:21:15.650 ⇒ 00:21:18.289 Uttam Kumaran: and we can begin to work through what
210 00:21:19.270 ⇒ 00:21:23.620 Uttam Kumaran: we can begin to work through, like how the 3 of us sort of take on tests.
211 00:21:24.300 ⇒ 00:21:26.529 Uttam Kumaran: And then we can start to think about around books.
212 00:21:29.300 ⇒ 00:21:34.130 Demilade Agboola: There’s another. Here’s something we should do in our like one of my previous companies.
213 00:21:34.500 ⇒ 00:21:37.239 Demilade Agboola: But every March model has to have tests.
214 00:21:38.160 ⇒ 00:21:45.789 Demilade Agboola: whether because that’s literally what people are using to build their models. So we need to be able to ensure and
215 00:21:45.950 ⇒ 00:22:03.609 Demilade Agboola: ensure that there is the highest quality possible. It doesn’t always have to be the most complex. Sometimes it’s just a simple comparison. Hey? Is the count of this table equals to the count of the initial row table. In some cases it’s just unique tests, unique and all null test on the primary keys.
216 00:22:04.050 ⇒ 00:22:08.520 Demilade Agboola: In some cases it’s more convoluted to be honest.
217 00:22:08.640 ⇒ 00:22:10.240 Demilade Agboola: But the idea was.
218 00:22:10.670 ⇒ 00:22:15.589 Demilade Agboola: for the most part we were. We were sure of the data that was being used
219 00:22:15.910 ⇒ 00:22:22.390 Demilade Agboola: by the analysts, because we kind of made sure that they use intermediate models or rob or rob tables. So
220 00:22:22.889 ⇒ 00:22:35.170 Demilade Agboola: for for the raw tables, the huge focus there was largely on like source freshness. So if the data was scale, we would know as well. So like having that sort of understanding of what tests are important, for what stage
221 00:22:35.430 ⇒ 00:22:40.239 Demilade Agboola: is very important because you don’t again like you said you don’t want to do too many.
222 00:22:40.420 ⇒ 00:22:47.189 Demilade Agboola: but you also want to be able to be sure, for for every table that you’re giving the analyst to use. What are the assumptions that
223 00:22:47.390 ⇒ 00:22:49.480 Demilade Agboola: must never fail.
224 00:22:50.020 ⇒ 00:22:50.700 Uttam Kumaran: Yes.
225 00:22:51.140 ⇒ 00:22:52.430 Demilade Agboola: And puts it out there for that.
226 00:22:53.360 ⇒ 00:22:58.439 Uttam Kumaran: Okay? I mean, I think that’s totally fair. Like, every March model needs to have a test
227 00:22:59.840 ⇒ 00:23:02.580 Uttam Kumaran: part of Pr review process.
228 00:23:03.110 ⇒ 00:23:03.920 Uttam Kumaran: Okay.
229 00:23:23.060 ⇒ 00:23:27.430 Awaish Kumar: Yeah, like, this is the thing which I
230 00:23:27.600 ⇒ 00:23:32.919 Awaish Kumar: like. We we discussed many times that when we have this lot of these tests and notifications
231 00:23:33.930 ⇒ 00:23:39.319 Awaish Kumar: for that. Like I’m I’m more inclined towards having a dashboard like for these kind of
232 00:23:41.300 ⇒ 00:23:48.810 Awaish Kumar: data quality dashboard for for all the March model. And then we can maybe just go and review, maybe
233 00:23:49.140 ⇒ 00:23:58.980 Awaish Kumar: give 30 min every day to to review their data quality and and wrote it between the ease.
234 00:24:01.980 ⇒ 00:24:03.819 Uttam Kumaran: Yeah. So some sort of like.
235 00:24:12.360 ⇒ 00:24:16.990 Uttam Kumaran: So for this week, I mean, maybe the 3 of us should just try to split up
236 00:24:17.540 ⇒ 00:24:19.869 Uttam Kumaran: getting Prs out for tests.
237 00:24:20.290 ⇒ 00:24:22.559 Uttam Kumaran: I mean, Eden is the number one.
238 00:24:23.320 ⇒ 00:24:31.509 Uttam Kumaran: So let me know. Like, how do you think we should start to divvy it up? And then I can tell Akash to just put tickets up on the board.
239 00:24:34.260 ⇒ 00:24:37.789 Uttam Kumaran: So that we can make sure that all Mart’s models have tests.
240 00:24:45.290 ⇒ 00:24:46.430 Uttam Kumaran: Is there like a.
241 00:24:46.430 ⇒ 00:24:46.810 Demilade Agboola: With it.
242 00:24:46.810 ⇒ 00:24:49.140 Uttam Kumaran: Is there a good like, yeah, yeah, go ahead.
243 00:24:50.590 ⇒ 00:24:57.658 Demilade Agboola: I was gonna say, like, cause we have a lot of models in Aden. So, like all the legacy, we don’t necessarily have to care about that?
244 00:24:58.990 ⇒ 00:25:07.290 Demilade Agboola: Well, especially things that are being used actively. I think the highest priority should be things that have been used actively. So we have a list of models that are being used actively.
245 00:25:07.500 ⇒ 00:25:09.169 Demilade Agboola: those number one
246 00:25:09.797 ⇒ 00:25:22.479 Demilade Agboola: and then I think the second level of priority will be the other ones that are not legacy, but still exist. So again, they’re not actively being used. But we just want to be sure that we future proof it.
247 00:25:28.400 ⇒ 00:25:33.480 Demilade Agboola: So I’ve actually kind of started working on the active like the ones that actively being used like creating
248 00:25:34.061 ⇒ 00:25:42.489 Demilade Agboola: so like team products, back transactions, product sale, summary by and by transaction things like that. I’m just trying to get
249 00:25:42.700 ⇒ 00:25:44.610 Demilade Agboola: those tests on them.
250 00:25:46.690 ⇒ 00:25:47.580 Awaish Kumar: Don’t know.
251 00:25:48.640 ⇒ 00:25:50.970 Uttam Kumaran: So let’s just like, how about we just split it up.
252 00:25:51.440 ⇒ 00:25:53.814 Awaish Kumar: We? Let’s just split up the
253 00:25:56.470 ⇒ 00:25:59.740 Uttam Kumaran: The marts like, let me just pull this up on my side.
254 00:26:08.220 ⇒ 00:26:12.210 Uttam Kumaran: like. Why don’t we just split up the non legacy ones between us.
255 00:26:13.860 ⇒ 00:26:19.220 Uttam Kumaran: I guess. Like, if someone I mean the biggest ones are gonna be marketing sales.
256 00:26:19.870 ⇒ 00:26:26.330 Uttam Kumaran: I mean, the most, the most important ones are gonna be marketing sales customers right
257 00:26:32.210 ⇒ 00:26:34.059 Uttam Kumaran: sales probably has the most.
258 00:26:38.800 ⇒ 00:26:52.950 Uttam Kumaran: So why don’t we? Why don’t each of us just take one of these folders and just try to rip them? Because I mean for me like I can do customers. You guys are probably closer to the marketing and sales in terms of like what the number. If you guys have to have fixed number tests.
259 00:26:53.948 ⇒ 00:26:57.839 Uttam Kumaran: But maybe we just split it up that way. That way. We can get test out like.
260 00:26:58.320 ⇒ 00:27:01.369 Uttam Kumaran: get tests open for review by tomorrow, basically
261 00:27:01.700 ⇒ 00:27:04.129 Uttam Kumaran: or or like, sometime tomorrow. Wednesday.
262 00:27:10.860 ⇒ 00:27:12.090 Uttam Kumaran: What do you guys think.
263 00:27:14.800 ⇒ 00:27:16.810 Awaish Kumar: Oh, yeah, we can take it, yeah.
264 00:27:17.270 ⇒ 00:27:19.990 Awaish Kumar: And like, but it, we like.
265 00:27:20.720 ⇒ 00:27:25.739 Awaish Kumar: how like, how we want, how deep we want to go in the test like it should be just the
266 00:27:25.870 ⇒ 00:27:30.400 Awaish Kumar: top level, like unique nominal count or or.
267 00:27:31.300 ⇒ 00:27:36.850 Uttam Kumaran: No, I think we should do like some value tests like, for example, like
268 00:27:37.190 ⇒ 00:27:43.710 Uttam Kumaran: about us, like we know we know certain order counts for for Eden and for Joby.
269 00:27:43.830 ⇒ 00:27:47.080 Uttam Kumaran: like we should say this, this number shouldn’t go past.
270 00:27:47.750 ⇒ 00:27:49.429 Awaish Kumar: This value basically.
271 00:27:55.520 ⇒ 00:27:58.589 Demilade Agboola: Yeah, I mean, we could do value test. I also think we could do
272 00:28:00.400 ⇒ 00:28:03.460 Demilade Agboola: if possible, some certain comparison tests.
273 00:28:03.930 ⇒ 00:28:09.220 Demilade Agboola: But for the most part. Yeah, like, I think we could just simply do on body tests.
274 00:28:09.370 ⇒ 00:28:10.510 Demilade Agboola: lot of time.
275 00:28:14.750 ⇒ 00:28:17.610 Demilade Agboola: So if we know, or if we know what the
276 00:28:19.990 ⇒ 00:28:29.759 Demilade Agboola: if we know what the like, a low number looks like. If we know things that should not. You know things that should appear in a certain formats, we, we can always ensure that that happens.
277 00:28:37.860 ⇒ 00:28:38.750 Uttam Kumaran: Okay.
278 00:28:40.390 ⇒ 00:28:44.480 Uttam Kumaran: So then, how about I? I mean, we’re gonna figure like, I think we’re the 3 of us will figure
279 00:28:44.630 ⇒ 00:28:51.790 Uttam Kumaran: we’ll figure out what’s possible while we work on this, maybe what I’ll do is I’ll ask Akash
280 00:28:51.910 ⇒ 00:28:54.379 Uttam Kumaran: to create 3 tickets for each of us.
281 00:28:54.550 ⇒ 00:28:57.160 Uttam Kumaran: maybe. Demolati, do you want to take sales.
282 00:28:57.450 ⇒ 00:28:59.329 Uttam Kumaran: since like is that like the
283 00:28:59.600 ⇒ 00:29:02.270 Uttam Kumaran: I mean, what is the one that that really like needs.
284 00:29:03.720 ⇒ 00:29:04.540 Awaish Kumar: Yeah, I guess.
285 00:29:04.540 ⇒ 00:29:05.069 Demilade Agboola: Yeah, it’s up to.
286 00:29:05.200 ⇒ 00:29:05.990 Uttam Kumaran: Yeah.
287 00:29:05.990 ⇒ 00:29:07.420 Demilade Agboola: Yeah, I’m out of hand of sales.
288 00:29:08.300 ⇒ 00:29:17.869 Uttam Kumaran: Okay. If you want to take sales and away she want to take marketing. I’ll take customers. If I get done with customers I’ll move on to. I’ll move on to customer service.
289 00:29:18.421 ⇒ 00:29:26.369 Uttam Kumaran: I think I’m gonna I’ll ask Akash to create these tickets. Let’s maybe let’s regroup the 3 of us again on Wednesday.
290 00:29:27.051 ⇒ 00:29:33.090 Uttam Kumaran: and then we can talk about like, okay, what do we learn in the Pr and like creating these tests and things like that
291 00:29:33.780 ⇒ 00:29:41.560 Uttam Kumaran: because our goal is to allow every single analyst right? Everybody on the team should now be creating tests. Right? So
292 00:29:41.680 ⇒ 00:29:46.889 Uttam Kumaran: we need to make sure that. Okay, it’s a process that’s really clear for us. We’ll end up with some more documentation.
293 00:29:47.439 ⇒ 00:29:50.429 Uttam Kumaran: But then let’s plan on chatting again on like Wednesday.
294 00:29:50.800 ⇒ 00:29:53.330 Uttam Kumaran: I’ll shoot over Prs as I finish them.
295 00:29:53.974 ⇒ 00:30:04.670 Uttam Kumaran: I know Robert wanted to get some of these tests done on the 9, th so I’m gonna tell Akash that, hey? The like. All of us need to have bandwidth to basically do that.
296 00:30:05.413 ⇒ 00:30:09.489 Uttam Kumaran: And then let’s just send each other the Prs, and we’ll send the alerts to the channel and sort of
297 00:30:09.870 ⇒ 00:30:14.030 Uttam Kumaran: test out how it works. And I’m gonna start to add it to other clients as well.
298 00:30:14.290 ⇒ 00:30:15.130 Uttam Kumaran: So
299 00:30:15.620 ⇒ 00:30:19.860 Uttam Kumaran: let’s just see how it goes, and we can make come, make some long term decisions, maybe end of the week.
300 00:30:22.120 ⇒ 00:30:24.700 Awaish Kumar: So, okay.
301 00:30:24.700 ⇒ 00:30:26.450 Demilade Agboola: I think that’s good.
302 00:30:26.450 ⇒ 00:30:31.599 Uttam Kumaran: Okay. I’m also. So I’m also testing like foundational, I’m gonna try data fold. So
303 00:30:32.020 ⇒ 00:30:41.959 Uttam Kumaran: we’re gonna I’ll I’ll try to test some other like out of the box. Tools as we like start to go along. But let’s just try to rip it up today this week
304 00:30:42.310 ⇒ 00:30:48.809 Uttam Kumaran: in terms of tests, and then at least, we’ll have something going, and then we can start to use that, because ultimately
305 00:30:49.030 ⇒ 00:30:55.239 Uttam Kumaran: we can do tests, or we can buy something off the shelf, and so we’ll I want us to have the test so that we can start to compare
306 00:30:55.480 ⇒ 00:30:58.340 Uttam Kumaran: what’s what’s good, or if we want to pay for something.
307 00:30:58.850 ⇒ 00:31:01.249 Uttam Kumaran: But I think this is fine, for now.
308 00:31:04.270 ⇒ 00:31:05.870 Awaish Kumar: Cool. Okay.
309 00:31:08.760 ⇒ 00:31:11.159 Uttam Kumaran: Anything else, for today.
310 00:31:17.760 ⇒ 00:31:19.110 Demilade Agboola: Yeah, I think that’s it.
311 00:31:20.780 ⇒ 00:31:32.979 Uttam Kumaran: Okay, cool, alright. Slack me if slack me if you need anything. Yeah, Demo, I’m I’m following the stuff in Eden Channel. But in case you need any help. There, just let me know. And then, yeah, I’m sort of preparing now for the urban stems chat later. So.
312 00:31:33.830 ⇒ 00:31:35.070 Demilade Agboola: Okay. Sounds good.
313 00:31:35.070 ⇒ 00:31:36.979 Uttam Kumaran: Okay. Alright. Thanks. Everyone.
314 00:31:37.845 ⇒ 00:31:38.710 Awaish Kumar: Okay.