Meeting Title: Metaplane Data Quality Sync Date: 2025-06-19 Meeting participants: Demilade Agboola, Uttam Kumaran, Robert Tseng, Luke Daque


WEBVTT

1 00:00:06.390 00:00:07.730 Uttam Kumaran: Hello!

2 00:00:11.540 00:00:12.369 Demilade Agboola: Hello, Tom!

3 00:00:13.000 00:00:14.269 Uttam Kumaran: Hey! How are you?

4 00:00:14.880 00:00:16.219 Demilade Agboola: I’m pretty good. How are you?

5 00:00:16.730 00:00:19.260 Uttam Kumaran: Good, good.

6 00:01:34.170 00:01:35.050 Demilade Agboola: That’s all.

7 00:01:38.440 00:01:39.420 Uttam Kumaran: I just.

8 00:01:39.830 00:01:41.570 Demilade Agboola: Okay, sure. Show you something.

9 00:01:42.030 00:01:42.789 Uttam Kumaran: Yeah, yeah.

10 00:01:44.376 00:01:52.769 Demilade Agboola: So far, even, I was able to just kind of put together potential potential like

11 00:01:54.110 00:02:00.410 Demilade Agboola: failure points in terms of like, what could go wrong in in giving them the answers to

12 00:02:00.920 00:02:07.379 Demilade Agboola: like get like whatever data questions they have. So from from the bigquery perspective, like the raw data

13 00:02:07.620 00:02:10.949 Demilade Agboola: we might have ingestion issues from like the web hook and all that.

14 00:02:11.200 00:02:18.530 Demilade Agboola: Or, like, you know, just late data, basically from the Dbt perspective.

15 00:02:19.440 00:02:22.550 Demilade Agboola: we can have still, without being used in transformations.

16 00:02:22.680 00:02:30.259 Demilade Agboola: we can have github actions failing, we can have by joins or filters.

17 00:02:30.860 00:02:37.510 Demilade Agboola: Then basically like, yeah, kind of like tied to Github action. Here we can just have, like field model built without letting.

18 00:02:37.700 00:02:43.837 Demilade Agboola: and then obviously kind of also under Github actions. If their schema changes

19 00:02:45.240 00:02:47.820 Demilade Agboola: that are not accounted for, and things just start breaking.

20 00:02:49.146 00:02:56.480 Demilade Agboola: and a couple of things that came to mind are like the extraction are running or failing silently on

21 00:02:56.650 00:03:05.540 Demilade Agboola: or like the email subscriptions fire before the extracts are done. So we send them reports like Pdf. Reports that are.

22 00:03:05.540 00:03:06.190 Uttam Kumaran: Yeah.

23 00:03:06.360 00:03:09.110 Demilade Agboola: Yeah, sure, they have all data data, basically.

24 00:03:10.157 00:03:15.160 Demilade Agboola: and then obviously, things can break you to like Renames or type mismatches or.

25 00:03:15.160 00:03:18.020 Uttam Kumaran: Yeah. One item I would add to tableau is also just like

26 00:03:18.767 00:03:33.000 Uttam Kumaran: filters being wrong or like undiscussed. For example, we had a we did like previously. We had. There was some issue right with like showing completed days or completed weeks. And they’re like, why is the why is the bar, low.

27 00:03:33.270 00:03:40.300 Uttam Kumaran: classic, the classic thing where you’re like. Oh, why are we going down? You’re like dude. It’s the 8th of the month, like

28 00:03:40.660 00:03:41.740 Uttam Kumaran: so.

29 00:03:41.910 00:03:47.760 Uttam Kumaran: But that’s not like a that is just like, because we didn’t get like A, we didn’t get it

30 00:03:47.940 00:03:50.749 Uttam Kumaran: agreed upon like, whether we wanted to show that or not.

31 00:03:51.140 00:03:57.690 Uttam Kumaran: So yeah, filters can be on incomplete or like on or cause unexpected changes.

32 00:04:12.040 00:04:17.720 Demilade Agboola: So I think the idea for this document. Well, 2 things I was thinking of. One is just like.

33 00:04:17.890 00:04:18.800 Demilade Agboola: if

34 00:04:19.440 00:04:27.219 Demilade Agboola: this can be like troubleshooting guidelines like, if anything goes wrong, you’re like, hey? Which of these criteria did?

35 00:04:27.790 00:04:31.576 Demilade Agboola: Did we fail on like what failed? Basically

36 00:04:32.340 00:04:59.210 Demilade Agboola: and 2, it’s also obviously, we don’t want to always like work from a reactive perspective. But just also like, Hey, how do we mitigate this? Or how do we catch anything that happens like under 12 h of it happening. For instance, whatever number we want to come up with, but like, how do we ensure that these potential issues are addressed as quickly as possible is basically another thing to just think about with this.

37 00:04:59.340 00:05:16.730 Demilade Agboola: So as much as possible. When I’m thinking, Meta plane, I’m thinking, okay, how do we use Meta plane to target as many of these use cases as possible. So I like, we have a list of like the importance quote unquote, important models that kind of power like our dashboard. So how do we translate

38 00:05:16.930 00:05:24.530 Demilade Agboola: those like these like these potential failure points into

39 00:05:25.470 00:05:33.150 Demilade Agboola: capture like actionable things for those dashboards. So we always know, like, Hey, things have not run.

40 00:05:33.270 00:05:36.030 Demilade Agboola: or we know that like there’s a bad.

41 00:05:36.030 00:05:36.460 Uttam Kumaran: Perfect.

42 00:05:36.460 00:05:37.579 Demilade Agboola: Yeah, things like that.

43 00:05:38.470 00:05:43.199 Uttam Kumaran: So maybe. Do you want to share this with me? And maybe let’s just start. Let’s start here, and then we can.

44 00:05:43.590 00:05:45.150 Uttam Kumaran: We can talk about this.

45 00:05:47.150 00:05:49.300 Uttam Kumaran: Gotcha. Thank you.

46 00:06:05.990 00:06:06.740 Uttam Kumaran: Cool.

47 00:06:08.450 00:06:15.400 Uttam Kumaran: Okay, so yeah, in terms of the. So I’m kind of at the checklist piece. So

48 00:06:17.652 00:06:20.570 Uttam Kumaran: let me just clean this up a little bit.

49 00:06:39.140 00:06:47.710 Uttam Kumaran: Okay? Cool hold on.

50 00:07:04.320 00:07:05.920 Uttam Kumaran: Okay. So

51 00:07:18.180 00:07:21.699 Uttam Kumaran: I’m just gonna kind of comment on like what I think the solution is.

52 00:07:55.130 00:07:57.680 Uttam Kumaran: Okay. So stale data being used.

53 00:07:59.560 00:08:00.900 Uttam Kumaran: Yeah, so.

54 00:09:32.220 00:09:34.360 Demilade Agboola: Okay.

55 00:12:32.030 00:12:33.129 Demilade Agboola: what’s up.

56 00:12:33.850 00:12:34.740 Uttam Kumaran: Yeah.

57 00:12:34.740 00:12:36.459 Demilade Agboola: What do you think the threshold

58 00:12:37.060 00:12:47.159 Demilade Agboola: in terms of like? So if we have certain, I mean, we have a lot of models, obviously, but for in terms of like core models, so fact transactions, product sales, summary.

59 00:12:47.490 00:12:48.260 Uttam Kumaran: Yeah.

60 00:12:48.260 00:12:55.890 Demilade Agboola: And as well as I’m trying to think of the other number as well as like theme products.

61 00:12:56.070 00:12:56.830 Uttam Kumaran: Yeah.

62 00:12:56.830 00:12:58.520 Demilade Agboola: What do you think?

63 00:12:58.670 00:13:00.599 Demilade Agboola: An appropriate like

64 00:13:01.510 00:13:11.350 Demilade Agboola: window, in which, if we don’t have data after like, say, 5 HI think 6 h should be good window. It should definitely be an error we need to look into.

65 00:13:12.740 00:13:23.260 Demilade Agboola: because I feel like, if we do it 1 h, 1 h might be too short 2 h, maybe also be too short. But I think after 6 h giving our our model runs every hour.

66 00:13:23.820 00:13:29.119 Demilade Agboola: If we don’t have data for the past like 6 h. Then we need.

67 00:13:29.120 00:13:33.419 Uttam Kumaran: I guess I guess it would. It would more be so. You’re talking about the freshness.

68 00:13:33.800 00:13:34.450 Demilade Agboola: Yeah.

69 00:13:35.300 00:13:39.230 Uttam Kumaran: Yeah, I would put 6 h minimum.

70 00:13:39.840 00:13:45.729 Demilade Agboola: Alright, so we won’t do for every single table, because that will be too too much of a hassle. But I think

71 00:13:46.260 00:13:49.190 Demilade Agboola: what we what we can do is we can do a current timestamp

72 00:13:50.035 00:13:53.840 Demilade Agboola: as a column on like the Dbt runs.

73 00:13:55.770 00:13:56.580 Uttam Kumaran: Okay.

74 00:13:56.980 00:13:58.310 Uttam Kumaran: Yes. Yes.

75 00:13:58.310 00:14:00.090 Demilade Agboola: And then we can just do it

76 00:14:01.920 00:14:04.710 Demilade Agboola: such that, like for the final tables.

77 00:14:06.880 00:14:08.939 Demilade Agboola: we get to know if that happens.

78 00:14:10.540 00:14:11.230 Uttam Kumaran: Oh!

79 00:15:03.990 00:15:05.200 Demilade Agboola: I’m Robert.

80 00:15:06.210 00:15:11.619 Demilade Agboola: I I was going to ask you. I was thinking of like having P. 0 dashboards.

81 00:15:12.349 00:15:19.049 Demilade Agboola: I mean, we have a bunch of dashboards. But I was thinking like, Okay, let’s have some P. 0 dashboards, dashboards that like, if

82 00:15:19.890 00:15:38.349 Demilade Agboola: we put like warning like, we set up monitoring for the the data that supplies that dashboard. If we have an error on those dashboards like all hell good breaks loose, basically like, because I mean, we have a bunch of dashboards, and not everything, obviously, is the same weight.

83 00:15:40.430 00:15:41.999 Uttam Kumaran: No, we should have P. 0.

84 00:15:42.690 00:15:48.889 Robert Tseng: I mean, I feel like, isn’t that just like a staging dashboard where we just dump everything we have on there? But it seems like it.

85 00:15:48.890 00:15:49.690 Uttam Kumaran: Yes.

86 00:15:49.810 00:15:57.870 Robert Tseng: Really noisy and like, who’s gonna notice? I guess it’ll just give us a heads up if things are breaking before we push it to production. But.

87 00:15:58.500 00:16:08.659 Uttam Kumaran: No, but see what what we’re gonna like. I think for the if you have one or 2 dashboards that matter, you should create a version that points to the staging database

88 00:16:09.740 00:16:12.270 Uttam Kumaran: that way. When you when you create a Pr

89 00:16:13.290 00:16:16.659 Uttam Kumaran: you can actually see the impact in a dashboard.

90 00:16:17.220 00:16:20.059 Uttam Kumaran: And you have, like once more step of confidence.

91 00:16:20.280 00:16:27.350 Uttam Kumaran: It this, what this will require, though, is, you have to then maintain 2 dashboards worth it.

92 00:16:28.150 00:16:39.789 Demilade Agboola: Yeah, I mean, I get that. But what I what I’m trying to say is like, so, for instance, before Josh Dashboard is very important. But we can’t compare that to like a type form response, explorer, dashboard, like if that.

93 00:16:39.790 00:16:40.560 Uttam Kumaran: Yes.

94 00:16:41.020 00:16:42.059 Demilade Agboola: Yeah, it goes bad. Sure.

95 00:16:42.060 00:16:49.849 Robert Tseng: Yeah, no, I’ve only only like the revenue and orders reports who really what? I consider we would need to do something like this for.

96 00:16:50.312 00:16:53.640 Demilade Agboola: The stuff that people are actually looking at every single morning.

97 00:16:54.300 00:17:01.290 Demilade Agboola: Alright cool! That’s that. That helps me filter out like unnecessary, like things to monitor.

98 00:17:02.930 00:17:03.560 Robert Tseng: Yeah.

99 00:17:03.560 00:17:06.569 Uttam Kumaran: Okay. So let let me. Let’s, I think we’re good.

100 00:17:07.030 00:17:10.519 Uttam Kumaran: like, I feel good on this. I think I can.

101 00:17:10.849 00:17:14.320 Uttam Kumaran: I mean, I can have. I can ask away to help me with this doing this.

102 00:17:17.140 00:17:26.229 Robert Tseng: Yeah. So I need to take this, Doc. And I need to like share something with Eden’s team. Probably today, just kind of sharing how our Qa process is gonna be better.

103 00:17:26.684 00:17:29.360 Robert Tseng: I mean, I’m happy to just write it.

104 00:17:29.680 00:17:31.220 Robert Tseng: And I can.

105 00:17:31.390 00:17:51.350 Robert Tseng: I? I think we’ll be using the same material, anyway. So if anything, I just want to be able to show them what we’re doing before, what we investigated this week and then, like a checklist that we’re implementing in the future. I feel like that’s easy for everyone to understand. So I’m just that’s how I’m gonna frame frame it to them.

106 00:17:51.830 00:17:53.159 Uttam Kumaran: Okay. Okay.

107 00:17:53.160 00:17:57.400 Robert Tseng: Yeah. So obviously like, there’s all this other stuff that goes on in the background that

108 00:17:57.880 00:18:04.630 Robert Tseng: I don’t think that people necessarily need to know. But I’m just telling you. I think that’s that’s how I’m gonna frame the message.

109 00:18:05.000 00:18:05.640 Uttam Kumaran: Okay.

110 00:18:06.200 00:18:06.890 Robert Tseng: Yes.

111 00:22:52.110 00:22:55.520 Uttam Kumaran: I’m just reading a little bit what the options are.

112 00:23:00.700 00:23:07.530 Demilade Agboola: That’s fine. I’m creating a list of the dashboards and trying to rank them. P. 0 to p. 3

113 00:23:08.375 00:23:10.990 Demilade Agboola: right now. So if anyone else.

114 00:23:11.170 00:23:13.700 Demilade Agboola: It’s a long we can.

115 00:31:38.062 00:31:40.949 Demilade Agboola: Do you need the list of like

116 00:31:41.280 00:31:48.409 Demilade Agboola: important models? Or do you just are you just kind of like figuring out how to make the monitors. In the 1st place.

117 00:31:50.252 00:32:00.210 Uttam Kumaran: Yes, I I know we. I think a waste gave me like 5 of them in the other, Doc. But yes, I would love to have everything you would consider a P. 0 model.

118 00:32:00.620 00:32:09.499 Uttam Kumaran: This can include, this can include the core, dim fact tables, and like any of the summary tables.

119 00:32:10.830 00:32:11.390 Demilade Agboola: Okay.

120 00:32:11.390 00:32:12.970 Uttam Kumaran: That would be. That’s ideal.

121 00:32:14.600 00:32:19.090 Uttam Kumaran: I don’t want to do on intermediate tables unless, like it’s you you really think like we have to.

122 00:32:21.550 00:32:23.540 Demilade Agboola: No, the intermediate levels are fine.

123 00:32:24.160 00:32:24.710 Uttam Kumaran: Okay.

124 00:32:51.370 00:32:54.850 Demilade Agboola: Robert, I robot! I made some changes to the

125 00:32:55.450 00:33:00.469 Demilade Agboola: the dashboard I missed on dashboard priorities. Let me tag you in it. Let’s see.

126 00:33:01.490 00:33:03.339 Robert Tseng: On the dock. Yeah, I can go back into it.

127 00:33:03.500 00:33:04.270 Demilade Agboola: Alright!

128 00:35:34.409 00:35:41.259 Robert Tseng: Okay? So, Peter, executive dash, yeah. Anything. Roas related.

129 00:35:41.639 00:35:45.489 Robert Tseng: Revenue, snapshot retention, dashboard profitability.

130 00:35:46.589 00:35:54.319 Robert Tseng: Yeah. I mean, I think the underlying models are and product sales summary.

131 00:35:55.089 00:35:56.679 Uttam Kumaran: Order, Summary.

132 00:35:58.699 00:36:00.039 Robert Tseng: Stuff like that.

133 00:36:01.599 00:36:04.909 Robert Tseng: q, 1 marketing.

134 00:36:13.359 00:36:16.350 Robert Tseng: Yeah. I mean, I think I think this seems right to me.

135 00:36:18.699 00:36:26.809 Robert Tseng: I think I had asked also for this to be in our data platform documentation, I mean, Annie should be kind of filling it in every time. She

136 00:36:29.229 00:36:42.509 Robert Tseng: Well, I don’t know who owns the documentation. I think it’s assigned to you. Demo. So yeah, there’s probably some, you know. Probably you’ll probably just. I mean, this is this is great. But I wish we had this as well. So

137 00:36:48.649 00:36:59.729 Robert Tseng: okay, so I mean, that’s fine. I think I will pretty much in my message. Talk about the P. 0 dashboards and kind of emphasize like the checklist we’re using for anything that pushes changes to these dashboards.

138 00:36:59.899 00:37:02.559 Robert Tseng: We’re just gonna be a bit more careful about it.

139 00:37:05.500 00:37:12.750 Uttam Kumaran: I mean I would. I would only say, Robert, that like part of this also, is that they cannot expect

140 00:37:12.930 00:37:17.930 Uttam Kumaran: stuff to change this fast meaning, like, like

141 00:37:18.110 00:37:25.010 Uttam Kumaran: the 24 h turnaround on anything is really hard to sustain. If we need to make sure things are right.

142 00:37:25.340 00:37:26.100 Uttam Kumaran: Yeah.

143 00:37:26.100 00:37:27.790 Robert Tseng: No, I yeah, I think

144 00:37:28.530 00:37:44.469 Robert Tseng: no. I mean the the straw that broke the camel’s back was this like same day change that like, I guess. Didn’t want to push on Friday. It wasn’t. That wasn’t why everything broke. I think there are other things I feel like we just got lucky getting up to this point, where.

145 00:37:44.840 00:37:45.330 Uttam Kumaran: Yes.

146 00:37:45.330 00:38:06.189 Robert Tseng: Big model changes that were being pushed. But they launched a bunch of new products. They’re asking for a new level of grain in terms of what we drill down into. And yeah, even the way that we think about product categories updated. So it was like 3 pretty big changes to our schema that, like all happened pretty much in the past week or 2 weeks. And then.

147 00:38:06.330 00:38:07.240 Robert Tseng: yeah, we just.

148 00:38:07.240 00:38:09.570 Uttam Kumaran: It would have happened at some point like.

149 00:38:09.570 00:38:10.240 Robert Tseng: Yeah.

150 00:38:10.240 00:38:15.459 Uttam Kumaran: In fact, it’s happened before, which is why we we got this close with Meta plane, and we didn’t

151 00:38:15.760 00:38:16.610 Uttam Kumaran: sort of finish the.

152 00:38:16.610 00:38:23.320 Robert Tseng: Yeah, last time it happened, I feel like like 2 months ago. So like, you know, everyone thought we were in a good place. And then, yeah, it just all.

153 00:38:23.980 00:38:25.449 Robert Tseng: It is what it is. So.

154 00:38:26.860 00:38:27.450 Uttam Kumaran: Yeah.

155 00:38:31.210 00:38:59.440 Robert Tseng: Yeah. So I mean, definitely, it’s on us to kind of push back and enforce some of our heuristics more. Which is why I think, yeah, with, there’s a checklist on our side that we’re gonna hold ourselves accountable to. But I’m also going to. I also gonna let them know that like, if you’re gonna request a change to any of these things like we’re not gonna do it like same day, because there’s just a lot of dependencies that we need to go and test before and make sure we don’t have a blow up like this again. So I think people will will be fine with that.

156 00:39:17.890 00:39:22.320 Robert Tseng: Okay, cool. So I’m gonna go and do that now.

157 00:44:47.640 00:44:56.499 Robert Tseng: Okay, so in the doc in the deliverable section, I mean, I think I’m just. I haven’t turned these to tickets, I think. Just want to make sure that these are our action items.

158 00:44:57.502 00:45:05.690 Robert Tseng: I think a lot of this will come to you. So just want to make sure that we’re aligned there, and then I’ll I’ll paste in the message.

159 00:45:06.820 00:45:09.700 Robert Tseng: and small for our view.

160 00:54:29.006 00:54:34.225 Uttam Kumaran: One other thing, demolates. I think there are some

161 00:54:35.020 00:54:44.259 Uttam Kumaran: data sources that are unused, that we should just deprecate or move to or yeah, like these 2.

162 00:54:48.790 00:54:54.250 Demilade Agboola: Sure, I mean, if like, what’s the check for their like? Their status.

163 00:54:56.074 00:55:02.430 Uttam Kumaran: They just haven’t been extracted in a in a long time. 5 days.

164 00:55:02.930 00:55:05.429 Demilade Agboola: Yeah, sure, we could always deprecate those.

165 00:55:07.289 00:55:11.180 Uttam Kumaran: Cause. I’m gonna assume that any data source in tableau

166 00:55:11.580 00:55:14.620 Uttam Kumaran: is live like, is that a fair assumption.

167 00:55:15.990 00:55:18.190 Demilade Agboola: Yeah should be fair.

168 00:55:19.320 00:55:20.439 Uttam Kumaran: Okay, then let me

169 00:55:22.970 00:55:24.429 Uttam Kumaran: Let’s send this and.

170 00:58:19.110 00:58:21.393 Robert Tseng: Okay, I updated the doc.

171 00:58:27.870 00:58:32.530 Uttam Kumaran: Okay, I gotta. I gotta. I gotta jump to ABC thing. But I’m I’m gonna

172 00:58:32.720 00:58:34.599 Uttam Kumaran: basically work on this. The rest of the day.

173 00:58:35.450 00:58:37.209 Robert Tseng: Okay, yeah, they want.

174 00:58:37.210 00:58:47.290 Robert Tseng: We can just review this and then let me know if this is something that you’re good with. And after this. I think this is the expectation moving forward. So I’m gonna send the message.

175 00:58:47.710 00:58:54.980 Uttam Kumaran: Yeah. And if you need any nice looking screenshots, I invited you to Meta plane. You can go get some or tell me if you want any.

176 00:58:55.290 00:58:55.780 Robert Tseng: Okay.

177 00:58:55.780 00:58:56.340 Demilade Agboola: Yeah.

178 00:58:56.680 00:59:04.620 Demilade Agboola: Also, I’ve I’ve created the list of the P. 0 models. I’ll cross reference them against Dbt and send them to you so you can

179 00:59:05.515 00:59:06.609 Demilade Agboola: set up the.

180 00:59:07.100 00:59:18.939 Uttam Kumaran: Oh, nice, perfect. Yeah. The P. 0 models. The P. 0 dashboards. And then ideally, if you have an extra 5 min, the P. 0 sources.

181 00:59:19.360 00:59:22.549 Uttam Kumaran: They don’t have. They don’t really, necessarily have to be the

182 00:59:23.110 00:59:26.989 Uttam Kumaran: I mean ideally, this, the tables like a bask, like

183 00:59:27.280 00:59:30.749 Uttam Kumaran: things like that if I can just get that list as well, that’d be great.

184 00:59:31.570 00:59:33.070 Demilade Agboola: Okay. Sounds good.

185 00:59:33.470 00:59:34.210 Uttam Kumaran: Okay.

186 00:59:35.130 00:59:36.130 Robert Tseng: Thanks guys.

187 00:59:36.130 00:59:36.900 Uttam Kumaran: Thanks, guys, bye.