Meeting Title: DBT Model Deprecation Sync Date: 2025-07-02 Meeting participants: Demilade Agboola, Caio Velasco
WEBVTT
1 00:00:29.350 ⇒ 00:00:30.909 Caio Velasco: Hey? There, Mandy! How’s it going.
2 00:00:34.394 ⇒ 00:00:41.860 Demilade Agboola: Hey, it’s going. Okay, just a lot of meetings. I think I’ve had like, almost like 3 h worth of meetings. Yeah.
3 00:00:42.060 ⇒ 00:00:42.670 Caio Velasco: Oh, my God!
4 00:00:43.010 ⇒ 00:00:43.740 Demilade Agboola: Yeah.
5 00:00:44.180 ⇒ 00:00:45.499 Caio Velasco: Hate those things.
6 00:00:46.282 ⇒ 00:00:51.447 Demilade Agboola: Yeah, at this point my days are like that. Most of my days are like that.
7 00:00:52.070 ⇒ 00:00:55.508 Caio Velasco: Okay, let’s be. Let’s be quick.
8 00:00:55.980 ⇒ 00:00:56.660 Demilade Agboola: Yeah, sure.
9 00:00:58.200 ⇒ 00:01:03.850 Demilade Agboola: So for this, what I’m what I’m trying to say is, if
10 00:01:07.660 ⇒ 00:01:14.120 Demilade Agboola: if you’re able to communicate like what appears to be wrong, or appears to be like
11 00:01:17.050 ⇒ 00:01:18.699 Demilade Agboola: what appears to be
12 00:01:19.120 ⇒ 00:01:29.760 Demilade Agboola: the disparity, like, what’s the cost of the disparity? It makes it easier for me to say, oh, just like use manifesto Json, or just use like the Dvt audit
13 00:01:30.653 ⇒ 00:01:36.290 Demilade Agboola: but like as it is right now, I’m not exactly sure what like what is causing that difference.
14 00:01:37.570 ⇒ 00:01:38.740 Caio Velasco: No problem.
15 00:01:39.030 ⇒ 00:01:46.917 Demilade Agboola: Yeah. So it makes it hard for me to, unless I’m going to have to dive in and start going through it like bit by bit, and then say everything.
16 00:01:51.610 ⇒ 00:01:54.190 Caio Velasco: Let me share the screen. Then.
17 00:01:55.050 ⇒ 00:02:02.160 Caio Velasco: okay, I can guide you with briefly, and then we can see. And you, with your experience, can see like what would be the best thing to do.
18 00:02:03.350 ⇒ 00:02:04.840 Caio Velasco: Do. Do you see my screen.
19 00:02:05.190 ⇒ 00:02:06.019 Demilade Agboola: Yeah, proceed.
20 00:02:06.500 ⇒ 00:02:08.940 Caio Velasco: Yeah, let me put it larger.
21 00:02:11.270 ⇒ 00:02:13.380 Caio Velasco: Okay, so
22 00:02:13.650 ⇒ 00:02:22.950 Caio Velasco: yeah, so we have this main sheet now, which has been updated last week or so because Utm asked me to
23 00:02:23.070 ⇒ 00:02:27.569 Caio Velasco: do the usage, which is basically all this
24 00:02:28.272 ⇒ 00:02:36.679 Caio Velasco: based on 180 days that things were queried or scanned because Redshift used the the scan
25 00:02:36.820 ⇒ 00:02:37.940 Caio Velasco: language.
26 00:02:39.170 ⇒ 00:02:48.300 Caio Velasco: Okay, so yeah, so this is the 1st thing we have. Some are true, a lot of false. This would be the 1st thing we deprecate which would be my my next task.
27 00:02:48.540 ⇒ 00:02:51.409 Caio Velasco: Then I would be basically deprecating.
28 00:02:51.790 ⇒ 00:02:52.990 Caio Velasco: Well.
29 00:02:53.660 ⇒ 00:03:00.370 Caio Velasco: all these things, and then I don’t think we care about this part. So this is my 1st question, does it make sense? Because we do have
30 00:03:00.510 ⇒ 00:03:02.569 Caio Velasco: things from analytics here?
31 00:03:02.810 ⇒ 00:03:04.859 Caio Velasco: But they haven’t been queried.
32 00:03:05.200 ⇒ 00:03:09.860 Caio Velasco: So I am assuming that we can archive them.
33 00:03:12.250 ⇒ 00:03:19.770 Demilade Agboola: And I think we should make exceptional things that analytics, because things in analytics are being created by Dbt
34 00:03:20.550 ⇒ 00:03:27.710 Demilade Agboola: even though they’ve not been used in a while not been query in a while. We need to, I think, for those ones. We might need to ask questions as to
35 00:03:28.100 ⇒ 00:03:30.140 Demilade Agboola: why that is.
36 00:03:31.900 ⇒ 00:03:37.920 Caio Velasco: Okay. As, for example, if I were to just put these ones, we have
37 00:03:38.650 ⇒ 00:03:43.410 Caio Velasco: 230 like, it’s a lot. And like all views and tables.
38 00:03:43.700 ⇒ 00:03:48.060 Demilade Agboola: Yeah, that’s a lot actually, is a lot.
39 00:03:51.160 ⇒ 00:03:59.650 Caio Velasco: Because this came basically. And this there’s no space, I believe to be mistaken, because this came directly
40 00:03:59.840 ⇒ 00:04:02.270 Caio Velasco: from redshift, from querying.
41 00:04:02.780 ⇒ 00:04:12.549 Caio Velasco: The let’s say the the the analogous of information. Schema in redshift.
42 00:04:15.360 ⇒ 00:04:16.140 Caio Velasco: So.
43 00:04:16.870 ⇒ 00:04:21.410 Demilade Agboola: Give me one second. Let’s see if I can see one of these tables in like the Dvc model.
44 00:04:22.840 ⇒ 00:04:25.850 Demilade Agboola: If it still exists as a model.
45 00:04:26.120 ⇒ 00:04:27.679 Caio Velasco: Oh, that’s fine.
46 00:04:27.680 ⇒ 00:04:30.280 Caio Velasco: I do that. Okay. Otherwise I can open here.
47 00:04:31.670 ⇒ 00:04:39.390 Demilade Agboola: I’m actually just looking at the the easy post underscore base.
48 00:04:40.750 ⇒ 00:04:43.069 Demilade Agboola: All right. So my guess is okay.
49 00:04:44.010 ⇒ 00:04:46.590 Demilade Agboola: These are models that have been renamed.
50 00:04:47.520 ⇒ 00:04:53.349 Demilade Agboola: So the old models that exist. But like they’re not running based on the score convey.
51 00:04:55.200 ⇒ 00:04:59.130 Demilade Agboola: Yeah, okay, yeah, we can delete them. We can delete them.
52 00:04:59.790 ⇒ 00:05:02.099 Caio Velasco: Okay? And are you sure for all of this.
53 00:05:03.740 ⇒ 00:05:07.069 Demilade Agboola: Let me see, because we have even like.
54 00:05:07.070 ⇒ 00:05:10.290 Caio Velasco: When message stuff orders.
55 00:05:11.760 ⇒ 00:05:14.330 Demilade Agboola: Let me see. Oms, orders or Ms.
56 00:05:14.630 ⇒ 00:05:15.840 Demilade Agboola: Oh, there it is.
57 00:05:18.180 ⇒ 00:05:21.749 Demilade Agboola: Yeah, there’s no, there isn’t an oms orders anymore.
58 00:05:23.780 ⇒ 00:05:28.070 Demilade Agboola: Let me see, where am I? Stop order.
59 00:05:30.140 ⇒ 00:05:36.190 Demilade Agboola: I can look as there isn’t almost a lot of items guessing. It’s just not running
60 00:05:47.220 ⇒ 00:05:48.350 Demilade Agboola: interesting.
61 00:05:50.860 ⇒ 00:05:58.580 Demilade Agboola: But we have a list of Dbt models. Is it possible to just like search against this and remove them.
62 00:06:02.170 ⇒ 00:06:03.449 Caio Velasco: Sorry I didn’t get it.
63 00:06:03.850 ⇒ 00:06:04.929 Demilade Agboola: Only such
64 00:06:06.270 ⇒ 00:06:12.199 Demilade Agboola: for the one stop here like it’s like the manifest Jason, which is part of what I was speaking to.
65 00:06:12.390 ⇒ 00:06:17.119 Demilade Agboola: but like they appear in the Dbt. Models.
66 00:06:21.310 ⇒ 00:06:27.450 Caio Velasco: Well, if we just look at the manifest, then well, with the S.
67 00:06:27.760 ⇒ 00:06:33.900 Caio Velasco: We still have a lot, I mean, not a lot, but a few like 62 that
68 00:06:34.390 ⇒ 00:06:36.749 Caio Velasco: are not being queried in redshift.
69 00:06:37.770 ⇒ 00:06:40.230 Caio Velasco: but it is present in the manifest.
70 00:06:41.880 ⇒ 00:06:46.320 Demilade Agboola: Also those rest of your queries also count like
71 00:06:46.900 ⇒ 00:06:53.149 Demilade Agboola: if they’re used by looker, or if they’re being used by any other like, if they even use within Dbt. Upstream.
72 00:06:54.870 ⇒ 00:06:56.320 Caio Velasco: Let me check. Here.
73 00:06:59.910 ⇒ 00:07:03.240 Demilade Agboola: But like, for instance, if something’s a downstream model in Dvt.
74 00:07:03.570 ⇒ 00:07:09.920 Demilade Agboola: so an option model in Dvt and downstream models use it good.
75 00:07:09.920 ⇒ 00:07:11.789 Caio Velasco: From the queried pipe.
76 00:07:12.120 ⇒ 00:07:18.630 Caio Velasco: Probably I’m not sure, because I don’t know much about this, but we’ll have a lot of users here. Look a user.
77 00:07:18.750 ⇒ 00:07:20.910 Caio Velasco: a polyatomic axis.
78 00:07:25.110 ⇒ 00:07:26.540 Demilade Agboola: Can I go? Can you go?
79 00:07:27.940 ⇒ 00:07:31.740 Demilade Agboola: So yeah. Dbt, part as well.
80 00:07:33.400 ⇒ 00:07:35.190 Caio Velasco: Yeah, we brought the stage.
81 00:07:40.780 ⇒ 00:07:42.610 Caio Velasco: Okay, mode analytics.
82 00:07:44.110 ⇒ 00:07:45.580 Caio Velasco: This is probably good.
83 00:07:45.960 ⇒ 00:07:48.579 Caio Velasco: Read replica. I don’t know. What is this.
84 00:07:54.950 ⇒ 00:07:58.360 Demilade Agboola: Okay, alright. So what we’re gonna do is I think we should
85 00:08:00.740 ⇒ 00:08:11.860 Demilade Agboola: the ones that are not being used by anyone’s. So it’s not like it’s not true, for is in manifest is not true, for is used by Dvt. Let’s get rid of those ones. Those ones we can be
86 00:08:12.240 ⇒ 00:08:19.930 Demilade Agboola: like those ones that were like, oh, it’s not being used, and it’s let me see.
87 00:08:20.780 ⇒ 00:08:24.779 Caio Velasco: Yeah. And that’s where we started. If we would put this as precedence here.
88 00:08:24.920 ⇒ 00:08:30.039 Caio Velasco: because theoretically it just is scripted and and then join here.
89 00:08:34.110 ⇒ 00:08:42.340 Caio Velasco: Then it would be putting all this out. But then you from yours. There is some yes, which I also don’t know why. There was differences.
90 00:08:43.620 ⇒ 00:08:47.359 Demilade Agboola: Yeah. So that’s why I was saying like it would be. Also, if you like.
91 00:08:47.520 ⇒ 00:08:53.990 Demilade Agboola: Look into them like, is anything of like those models have their renamed now, because
92 00:08:54.970 ⇒ 00:08:59.319 Demilade Agboola: that’s also protect. That’s also a possibility. So like, if you can look through like
93 00:08:59.710 ⇒ 00:09:04.939 Demilade Agboola: Dbt’s the repo right now, and see if you can find the names of these things.
94 00:09:05.360 ⇒ 00:09:15.170 Demilade Agboola: if you can’t find them, then maybe that’s what’s happened. Maybe like, Oh, these number, these models don’t exist anymore. But they’ve been renamed because Emily does love things. She’s.
95 00:09:22.050 ⇒ 00:09:24.310 Caio Velasco: Yeah, apparently this.
96 00:09:24.650 ⇒ 00:09:26.000 Caio Velasco: It’s not there.
97 00:09:28.400 ⇒ 00:09:31.129 Demilade Agboola: That shouldn’t be there, because it’s it’s even N.
98 00:09:31.640 ⇒ 00:09:33.390 Caio Velasco: Yeah, this is also my meeting. Right?
99 00:09:34.780 ⇒ 00:09:35.370 Caio Velasco: Alright.
100 00:09:35.370 ⇒ 00:09:36.249 Demilade Agboola: And as well.
101 00:09:36.250 ⇒ 00:09:38.100 Caio Velasco: Yeah, you you’re looking for this one. Right?
102 00:09:38.550 ⇒ 00:09:39.095 Demilade Agboola: Here
103 00:09:40.000 ⇒ 00:09:42.500 Caio Velasco: A good one here, Fedex.
104 00:09:51.290 ⇒ 00:09:58.039 Caio Velasco: not in the main name, just as a city name.
105 00:09:58.610 ⇒ 00:09:59.170 Caio Velasco: Hmm!
106 00:09:59.170 ⇒ 00:10:04.880 Demilade Agboola: Yeah. So that that’s why I’m guessing. Some of these things are like, maybe things where the names of these things have changed.
107 00:10:05.000 ⇒ 00:10:08.119 Demilade Agboola: That’s why, I said, like getting an idea of what’s going on.
108 00:10:08.370 ⇒ 00:10:14.210 Demilade Agboola: So the main thing of these things have changed. And that’s why they don’t appear anymore.
109 00:10:15.200 ⇒ 00:10:16.070 Caio Velasco: Okay.
110 00:10:16.570 ⇒ 00:10:17.280 Demilade Agboola: Yeah.
111 00:10:17.790 ⇒ 00:10:22.389 Demilade Agboola: So obviously, if the names have changed, then, yeah, it doesn’t matter anymore. Because,
112 00:10:25.560 ⇒ 00:10:31.190 Demilade Agboola: if the names have changed doesn’t matter anymore. Because, like Dbt will not produce those models anymore. So that’s fine.
113 00:10:32.660 ⇒ 00:10:38.580 Caio Velasco: True, true. So theoretically, what we are saying is that we are disregarding this one.
114 00:10:39.270 ⇒ 00:10:40.040 Demilade Agboola: Yes.
115 00:10:41.290 ⇒ 00:10:44.029 Caio Velasco: Okay, okay, okay, you wanna put this red.
116 00:10:44.500 ⇒ 00:10:52.409 Caio Velasco: Okay? So we keep this this. So basically, our 1st layer of deprecation will be
117 00:10:52.530 ⇒ 00:10:59.969 Caio Velasco: not only the false here, but the ones that are not presently manifest. So basically, this 2 filters.
118 00:11:01.120 ⇒ 00:11:01.810 Demilade Agboola: Yeah.
119 00:11:02.070 ⇒ 00:11:11.649 Caio Velasco: Okay, and then for well, we start with this one. And then we see about the accurate models.
120 00:11:14.220 ⇒ 00:11:21.819 Caio Velasco: okay, I still have to update this. So this is, don’t worry about this one now. But then we can move to then second layer and include the accuracy idea.
121 00:11:23.150 ⇒ 00:11:25.860 Caio Velasco: Yeah, okay, I think with this, I’m good.
122 00:11:26.020 ⇒ 00:11:28.210 Caio Velasco: Do we need anything else? I don’t think so. Right.
123 00:11:28.622 ⇒ 00:11:30.270 Demilade Agboola: No, no, that’s fine.
124 00:11:30.270 ⇒ 00:11:34.501 Caio Velasco: Okay, cool. So, man, have a good day, and I hope you don’t have any meetings again.
125 00:11:35.080 ⇒ 00:11:38.050 Demilade Agboola: Have to get some models out, so we’ll see.
126 00:11:38.050 ⇒ 00:11:39.450 Caio Velasco: Okay. Okay. Cool.
127 00:11:39.450 ⇒ 00:11:39.890 Demilade Agboola: Alright. Thank you.
128 00:11:40.050 ⇒ 00:11:40.569 Caio Velasco: See you.
129 00:11:41.420 ⇒ 00:11:41.980 Caio Velasco: Good day.