Meeting Title: AE-Planning-Session Date: 2024-07-22 Meeting participants: Nicolas Sucari, Uttam Kumaran, Brian Pei, Ryan
WEBVTT
1 00:00:53.920 ⇒ 00:00:54.830 Nicolas Sucari: Hi! Ryan.
2 00:00:57.980 ⇒ 00:00:59.764 Ryan: I don’t know.
3 00:01:01.940 ⇒ 00:01:03.999 Nicolas Sucari: Oh, I can’t hear you very well.
4 00:01:06.115 ⇒ 00:01:07.939 Ryan: Can you hear me now?
5 00:01:09.000 ⇒ 00:01:10.919 Nicolas Sucari: Okay. Now, yes. How are you?
6 00:01:12.020 ⇒ 00:01:13.990 Ryan: Yeah, I’m doing. Well, how are you guys.
7 00:01:15.580 ⇒ 00:01:16.480 Nicolas Sucari: All good.
8 00:01:19.000 ⇒ 00:01:23.410 Ryan: Yeah, I’m in a library at the moment, like somewhere in
9 00:01:23.820 ⇒ 00:01:27.440 Ryan: Japan. So I I will be like talking much, I guess.
10 00:01:31.900 ⇒ 00:01:35.699 Nicolas Sucari: You’re in Japan. Yeah, I don’t know. Sometimes your audio is.
11 00:01:36.310 ⇒ 00:01:36.970 Nicolas Sucari: yeah.
12 00:01:36.970 ⇒ 00:01:39.410 Ryan: Yeah. Can you hear me? Can you hear me?
13 00:01:39.710 ⇒ 00:01:47.750 Nicolas Sucari: Yeah, I can hear you. But sometimes, yeah, I think you’re you’re in a coffee shop or something like that, because something serious, like background noise. But that’s fine. Don’t worry.
14 00:01:47.750 ⇒ 00:01:52.360 Ryan: Yeah, yeah, I won’t talk on that. Because, yeah, I’m in a public space.
15 00:01:55.010 ⇒ 00:01:55.820 Nicolas Sucari: Nice
16 00:01:57.590 ⇒ 00:02:02.020 Nicolas Sucari: you went to Japan. I really, I really wanted to know Japan.
17 00:02:02.620 ⇒ 00:02:08.019 Nicolas Sucari: It’s 1 of my trips that I’m trying to plan for the next years. I don’t know when.
18 00:02:09.410 ⇒ 00:02:10.314 Ryan: Yeah, it’s pretty cool.
19 00:02:15.030 ⇒ 00:02:15.890 Nicolas Sucari: Hi Brian.
20 00:02:16.310 ⇒ 00:02:17.010 Brian Pei: Morning.
21 00:02:19.490 ⇒ 00:02:20.430 Nicolas Sucari: Morning.
22 00:02:21.710 ⇒ 00:02:23.400 Nicolas Sucari: Okay, guys, let’s
23 00:02:23.650 ⇒ 00:02:30.500 Nicolas Sucari: get going. So if we have some spare time after this meeting, probably we can review
24 00:02:30.620 ⇒ 00:02:31.720 Nicolas Sucari: the
25 00:02:31.990 ⇒ 00:02:35.419 Nicolas Sucari: the test stuff that we have there on slack.
26 00:02:35.490 ⇒ 00:02:39.420 Nicolas Sucari: But for now wanna share a screen and show the
27 00:02:41.100 ⇒ 00:02:43.009 Nicolas Sucari: project, adap project
28 00:02:43.460 ⇒ 00:02:46.970 Nicolas Sucari: as we can go and see what we can achieve this week. Okay.
29 00:02:51.590 ⇒ 00:02:52.390 Nicolas Sucari: dare
30 00:02:52.920 ⇒ 00:02:54.310 Nicolas Sucari: are you seeing
31 00:02:54.750 ⇒ 00:02:56.010 Nicolas Sucari: my?
32 00:02:56.940 ⇒ 00:02:58.750 Nicolas Sucari: Are you seeing the screen? Yes.
33 00:03:09.280 ⇒ 00:03:11.493 Nicolas Sucari: okay. Are you seeing? Yes or no?
34 00:03:12.850 ⇒ 00:03:13.640 Ryan: It’s perfect.
35 00:03:14.290 ⇒ 00:03:15.860 Nicolas Sucari: Yeah. Okay. So
36 00:03:15.870 ⇒ 00:03:27.939 Nicolas Sucari: I think, yeah. Last week we completed a lot of a lot of different tasks. We move everything from port. I think we we finished that, Ryan. I don’t know if we are still missing on some of the reports
37 00:03:28.334 ⇒ 00:03:32.309 Nicolas Sucari: on the migration to real, but I think all, all of them are ready. Right.
38 00:03:33.960 ⇒ 00:03:36.360 Ryan: Yeah, it should be. Everything should be good.
39 00:03:40.286 ⇒ 00:03:44.760 Nicolas Sucari: Okay, and we only have in progress these tasks now.
40 00:03:45.395 ⇒ 00:03:48.869 Nicolas Sucari: I don’t know if you’re still working on this Ryan.
41 00:03:49.340 ⇒ 00:03:52.320 Ryan: Yeah, I still haven’t found where
42 00:03:52.850 ⇒ 00:03:57.130 Ryan: that data is coming from. So yeah, it’s in progress. I’m still looking for that.
43 00:03:57.580 ⇒ 00:03:58.220 Ryan: He’s out.
44 00:03:58.220 ⇒ 00:03:58.640 Nicolas Sucari: Perfect.
45 00:03:59.060 ⇒ 00:03:59.900 Ryan: Warehouse? Yeah.
46 00:04:01.450 ⇒ 00:04:02.940 Nicolas Sucari: Okay, great.
47 00:04:03.150 ⇒ 00:04:22.570 Nicolas Sucari: And then I think for this week we don’t have something like it’s really urgent. But we can start tackling down tasks that we have here in ready this week and backlog. So probably what we should do is try to achieve, to look at what we have here, and see if we can.
48 00:04:22.880 ⇒ 00:04:27.489 Nicolas Sucari: We can start like tackling down this task and see what we can achieve this week.
49 00:04:29.040 ⇒ 00:04:42.589 Nicolas Sucari: apart from these, the only thing that we need to review is all of the alerts that we are receiving on slack. So I don’t know guys how you wanna like split the tasks. Probably, Brian, you were going to work on that one. I think we do some
50 00:04:43.297 ⇒ 00:04:45.800 Nicolas Sucari: so I don’t know if. Yeah, okay.
51 00:04:45.800 ⇒ 00:04:46.870 Brian Pei: Yeah, it.
52 00:04:46.870 ⇒ 00:04:55.460 Uttam Kumaran: Yeah, I think if we have time today, we can probably tackle the shipments warehouse and that one on this call the warehouse. One is probably coming from Primus
53 00:04:55.590 ⇒ 00:04:59.310 Uttam Kumaran: or something where they just like they’re just changing something.
54 00:04:59.618 ⇒ 00:05:03.500 Uttam Kumaran: But we can take a look at all that we should be able to close that.
55 00:05:06.140 ⇒ 00:05:07.540 Nicolas Sucari: Yeah, okay, perfect.
56 00:05:10.702 ⇒ 00:05:15.910 Nicolas Sucari: And yeah. Then it’s just going back to backlog and see what we can
57 00:05:15.940 ⇒ 00:05:19.499 Nicolas Sucari: adding into the week. I don’t know if the week will be like
58 00:05:19.670 ⇒ 00:05:24.490 Nicolas Sucari: maxed out with these tasks, or we need to add something else. But I think probably we can.
59 00:05:24.500 ⇒ 00:05:33.850 Nicolas Sucari: Yeah, start working on on those 2 things. And if we finish that by Wednesday, or I don’t know when. We can add more stuff. Okay.
60 00:05:36.520 ⇒ 00:05:44.539 Ryan: Sounds good. I may not have a lot of time this week, though. But yeah, I can work on those issues.
61 00:05:46.030 ⇒ 00:05:51.250 Uttam Kumaran: Yeah, Ryan, what’s your availability this week? Are you off like the back half of the week, or
62 00:05:52.170 ⇒ 00:05:58.279 Ryan: Yeah, like the whole week I’m I’m away. But I I’ll log in and like from time to time.
63 00:05:58.660 ⇒ 00:06:00.509 Uttam Kumaran: Okay. Okay, okay, that’s just good to know.
64 00:06:01.890 ⇒ 00:06:02.570 Ryan: Thanks.
65 00:06:04.290 ⇒ 00:06:05.380 Nicolas Sucari: Okay.
66 00:06:07.470 ⇒ 00:06:13.880 Nicolas Sucari: yeah, I don’t know. I don’t know if we have something else for the week that we need to discuss about tasks or anything that it’s like
67 00:06:14.550 ⇒ 00:06:19.050 Nicolas Sucari: that we need to work on so if not, we can jump into the
68 00:06:19.330 ⇒ 00:06:21.130 Nicolas Sucari: alert stuff
69 00:06:21.820 ⇒ 00:06:22.610 Nicolas Sucari: right?
70 00:06:23.010 ⇒ 00:06:27.090 Uttam Kumaran: Yeah, I don’t know, Brian. If you’re at your computer, you wanna drive
71 00:06:29.132 ⇒ 00:06:33.929 Uttam Kumaran: I mean, I just know where everything is, but maybe helpful to walk through with all 3 of us.
72 00:06:37.410 ⇒ 00:06:38.840 Brian Pei: You say Brian or Ryan.
73 00:06:38.840 ⇒ 00:06:40.090 Uttam Kumaran: Brian. Sorry.
74 00:06:40.580 ⇒ 00:06:41.280 Brian Pei: Okay.
75 00:06:48.110 ⇒ 00:06:50.729 Ryan: Maybe you guys should start calling me Luke
76 00:06:52.360 ⇒ 00:06:53.000 Ryan: don’t get.
77 00:06:53.000 ⇒ 00:06:54.810 Uttam Kumaran: Yeah, we could go middle names.
78 00:06:55.530 ⇒ 00:06:57.789 Ryan: That’s my second name, Luke.
79 00:06:58.620 ⇒ 00:07:00.660 Uttam Kumaran: And I can just say, Bpay.
80 00:07:05.670 ⇒ 00:07:13.216 Uttam Kumaran: yeah, I don’t know, Brian, whether it’s like helpful to go through the github alerts first, st or if you start taking a look at stuff?
81 00:07:14.043 ⇒ 00:07:14.490 Brian Pei: You actually.
82 00:07:14.490 ⇒ 00:07:17.269 Uttam Kumaran: Actually mentioned, we’re gonna go through. Yeah. Actually, we could do a year.
83 00:07:17.635 ⇒ 00:07:18.000 Brian Pei: Idea.
84 00:07:18.770 ⇒ 00:07:23.230 Uttam Kumaran: Oh, oh, well, yeah, or we could do the I want to talk about alerts for sure
85 00:07:23.740 ⇒ 00:07:26.289 Uttam Kumaran: while we have. And the shipments. One. I think
86 00:07:26.860 ⇒ 00:07:28.499 Uttam Kumaran: I know where it’s coming from.
87 00:07:37.498 ⇒ 00:07:38.930 Brian Pei: Can you drive? Then?
88 00:07:40.350 ⇒ 00:07:41.075 Uttam Kumaran: Yes,
89 00:07:43.240 ⇒ 00:07:45.419 Uttam Kumaran: Yeah. Give me one sec.
90 00:07:49.200 ⇒ 00:07:50.580 Uttam Kumaran: Just pop in there now.
91 00:08:36.210 ⇒ 00:08:41.160 Uttam Kumaran: Okay, I think I’m I’ll go through it. But for the I kinda want
92 00:08:41.539 ⇒ 00:08:43.590 Uttam Kumaran: like, of course, like, I’m not gonna be
93 00:08:43.870 ⇒ 00:08:46.799 Uttam Kumaran: owning the process. So
94 00:08:47.370 ⇒ 00:08:50.200 Uttam Kumaran: I kind of would love
95 00:08:51.035 ⇒ 00:08:54.859 Uttam Kumaran: for you guys to kind of give feedback on like what we should actually do.
96 00:08:55.456 ⇒ 00:08:57.120 Uttam Kumaran: But let me share
97 00:09:03.590 ⇒ 00:09:05.510 Uttam Kumaran: cool. So
98 00:09:07.360 ⇒ 00:09:11.060 Uttam Kumaran: let’s just start. Let’s actually just start straight from like
99 00:09:11.430 ⇒ 00:09:12.125 Uttam Kumaran: the
100 00:09:14.540 ⇒ 00:09:17.760 Uttam Kumaran: we’ll start straight from the alert, and we’ll just kind of like walk through.
101 00:09:25.210 ⇒ 00:09:27.959 Uttam Kumaran: so this is the most recent alert
102 00:09:28.994 ⇒ 00:09:30.930 Uttam Kumaran: go to view failure.
103 00:09:37.190 ⇒ 00:09:41.570 Uttam Kumaran: Great. So it looks like there are 7 errors and 2 warnings.
104 00:09:44.580 ⇒ 00:09:45.770 Uttam Kumaran: there’s like.
105 00:09:46.160 ⇒ 00:09:49.679 Uttam Kumaran: I’m just gonna take this and copy this into something.
106 00:09:52.820 ⇒ 00:09:54.395 Uttam Kumaran: Hey? What I’ll do is
107 00:09:57.400 ⇒ 00:10:05.899 Uttam Kumaran: and the spirit of the meeting on Friday. I’m just gonna have Claude side by side, because basically, we’ll this is kind of like how I would do this.
108 00:10:08.700 ⇒ 00:10:10.290 Uttam Kumaran: go to projects.
109 00:10:11.010 ⇒ 00:10:14.809 Uttam Kumaran: I’m gonna have pool parts engineer. And I’m basically gonna say, like.
110 00:10:15.400 ⇒ 00:10:17.600 Uttam Kumaran: we’re working on
111 00:10:17.650 ⇒ 00:10:19.180 Uttam Kumaran: cleaning up
112 00:10:21.470 ⇒ 00:10:23.030 Uttam Kumaran: 30 plus
113 00:10:23.560 ⇒ 00:10:24.310 Uttam Kumaran: her
114 00:10:33.021 ⇒ 00:10:35.429 Uttam Kumaran: and then I wanna put this in notion
115 00:10:36.730 ⇒ 00:10:40.089 Uttam Kumaran: first, st and then we can kind of like go through
116 00:10:42.700 ⇒ 00:10:46.470 Uttam Kumaran: Let’s see what it says. And I’m gonna paste this into notion.
117 00:10:57.640 ⇒ 00:11:00.259 Uttam Kumaran: okay, so go to
118 00:11:02.390 ⇒ 00:11:03.060 Uttam Kumaran: alert.
119 00:11:04.720 ⇒ 00:11:05.400 Uttam Kumaran: Where?
120 00:11:12.839 ⇒ 00:11:14.130 Uttam Kumaran: I’m just gonna
121 00:11:18.350 ⇒ 00:11:19.774 Uttam Kumaran: Where did we do,
122 00:11:21.130 ⇒ 00:11:23.900 Uttam Kumaran: Brian? We never moved your dock into here, did we?
123 00:11:25.740 ⇒ 00:11:27.090 Brian Pei: My!
124 00:11:27.290 ⇒ 00:11:27.900 Brian Pei: Oh.
125 00:11:27.900 ⇒ 00:11:28.760 Uttam Kumaran: Yeah, and not. You’re like.
126 00:11:28.760 ⇒ 00:11:31.169 Brian Pei: I didn’t. I didn’t move it myself.
127 00:11:31.610 ⇒ 00:11:34.040 Uttam Kumaran: Okay, then I’m just gonna create.
128 00:11:34.910 ⇒ 00:11:36.769 Uttam Kumaran: I’m just gonna create one here
129 00:11:37.090 ⇒ 00:11:38.260 Uttam Kumaran: and
130 00:11:41.620 ⇒ 00:11:45.390 Uttam Kumaran: for now. We’ll move that in there.
131 00:11:46.890 ⇒ 00:11:47.840 Uttam Kumaran: But
132 00:11:56.460 ⇒ 00:11:58.419 Uttam Kumaran: And then I’m just gonna create it.
133 00:12:08.430 ⇒ 00:12:09.310 Uttam Kumaran: Okay.
134 00:12:11.850 ⇒ 00:12:13.560 Uttam Kumaran: great. So
135 00:12:14.700 ⇒ 00:12:16.890 Uttam Kumaran: let’s see what it said.
136 00:12:22.650 ⇒ 00:12:27.049 Uttam Kumaran: okay, so we’re gonna let’s just go through these guys today.
137 00:12:29.990 ⇒ 00:12:35.019 Uttam Kumaran: And the nice thing about this is it basically told us, probably where stuff is.
138 00:12:36.425 ⇒ 00:12:37.290 Uttam Kumaran: So
139 00:12:38.440 ⇒ 00:12:41.389 Uttam Kumaran: let’s just try and see if we can walk through this.
140 00:12:41.830 ⇒ 00:12:46.569 Uttam Kumaran: So the 1st issue is null value constraints.
141 00:12:46.660 ⇒ 00:12:49.129 Uttam Kumaran: all order items. Line. Item, id.
142 00:12:50.623 ⇒ 00:12:55.180 Uttam Kumaran: this is unique value constraint. So I just want to go see this test first.st
143 00:12:55.930 ⇒ 00:12:57.559 Uttam Kumaran: So if we go to code
144 00:12:57.720 ⇒ 00:12:59.530 Uttam Kumaran: and we go to
145 00:13:00.080 ⇒ 00:13:02.390 Uttam Kumaran: probably all
146 00:13:02.400 ⇒ 00:13:04.400 Uttam Kumaran: order items.
147 00:13:08.230 ⇒ 00:13:10.510 Uttam Kumaran: actually did they move everything to test.
148 00:13:14.500 ⇒ 00:13:17.779 Ryan: No, I don’t think so. It’s still in the models. The apple file.
149 00:13:18.410 ⇒ 00:13:19.310 Uttam Kumaran: Oh.
150 00:13:20.420 ⇒ 00:13:22.719 Uttam Kumaran: so I wonder what happened to
151 00:13:25.850 ⇒ 00:13:28.540 Uttam Kumaran: what happened to author’s Pr. About that.
152 00:13:36.030 ⇒ 00:13:44.899 Ryan: What I noticed, though I guess it’s not pr that they they consolidated all the yaml files to just a single yaml file for each.
153 00:13:44.900 ⇒ 00:13:45.365 Uttam Kumaran: Yeah.
154 00:13:45.830 ⇒ 00:13:46.970 Ryan: Folder. Yeah.
155 00:13:49.450 ⇒ 00:13:50.319 Ryan: it’s not difficult.
156 00:13:50.320 ⇒ 00:13:50.680 Uttam Kumaran: ‘kay.
157 00:13:50.680 ⇒ 00:13:51.670 Ryan: We want more.
158 00:13:53.937 ⇒ 00:13:57.522 Uttam Kumaran: Ideally. Yeah, I just wonder.
159 00:14:02.380 ⇒ 00:14:06.959 Uttam Kumaran: well, let’s just see what. Let’s just see where this what it looks like. Then staging
160 00:14:47.000 ⇒ 00:14:50.569 Uttam Kumaran: oh, it’s gonna be one per folder.
161 00:14:52.480 ⇒ 00:14:53.120 Ryan: Yeah.
162 00:15:06.640 ⇒ 00:15:08.189 Uttam Kumaran: Okay, I mean, this is fine.
163 00:15:08.310 ⇒ 00:15:10.160 Uttam Kumaran: Don’t really care either way.
164 00:15:12.640 ⇒ 00:15:13.520 Uttam Kumaran: but
165 00:15:13.620 ⇒ 00:15:16.399 Uttam Kumaran: this should have this needs to happen for prod
166 00:15:17.090 ⇒ 00:15:18.300 Uttam Kumaran: as well.
167 00:15:22.000 ⇒ 00:15:26.819 Uttam Kumaran: Okay, but let’s just let’s just keep going while we’re while we’re on this subject.
168 00:15:37.640 ⇒ 00:15:40.765 Uttam Kumaran: Okay, so this is this, but I don’t know where the
169 00:15:41.870 ⇒ 00:15:44.410 Ryan: I guess it’s the reporting that. Yeah.
170 00:15:45.010 ⇒ 00:15:45.800 Ryan: look.
171 00:15:45.800 ⇒ 00:15:47.810 Uttam Kumaran: Oh, yeah, okay, great, cool, cool.
172 00:15:49.590 ⇒ 00:15:55.479 Uttam Kumaran: Good point. Okay. So we had this line item, order, id test.
173 00:15:57.103 ⇒ 00:15:59.150 Uttam Kumaran: which is
174 00:16:00.100 ⇒ 00:16:02.500 Uttam Kumaran: somewhere here.
175 00:16:15.300 ⇒ 00:16:18.882 Uttam Kumaran: Okay, so it’s not unique at the moment.
176 00:16:19.800 ⇒ 00:16:26.548 Uttam Kumaran: we can’t see exactly, can I? Is there an rise there anywhere in elementary? I can see the sequel, or like, should I just
177 00:16:27.610 ⇒ 00:16:29.050 Uttam Kumaran: try to run this, and
178 00:16:29.350 ⇒ 00:16:29.970 Uttam Kumaran: no thanks.
179 00:16:29.970 ⇒ 00:16:31.350 Ryan: Think if we go through the
180 00:16:31.720 ⇒ 00:16:33.030 Ryan: the Ui.
181 00:16:33.180 ⇒ 00:16:39.119 Ryan: I think we should be able to see there. But we can also do like Brian’s approach, where we query.
182 00:16:39.870 ⇒ 00:16:40.570 Ryan: Yeah.
183 00:16:40.570 ⇒ 00:16:44.319 Uttam Kumaran: Let’s do what? Let’s do that. Whatever we agreed on.
184 00:16:45.040 ⇒ 00:16:48.820 Uttam Kumaran: so let me let me just go straight to Snowflake then.
185 00:16:55.710 ⇒ 00:16:56.560 Uttam Kumaran: and I’ll
186 00:16:56.710 ⇒ 00:16:57.380 Uttam Kumaran: ‘kay
187 00:17:13.170 ⇒ 00:17:16.090 Uttam Kumaran: gonna create a new worksheet called
188 00:17:25.190 ⇒ 00:17:26.460 Uttam Kumaran: or so
189 00:17:27.630 ⇒ 00:17:30.819 Uttam Kumaran: and investigation.
190 00:17:35.960 ⇒ 00:17:36.965 Uttam Kumaran: okay,
191 00:17:38.070 ⇒ 00:17:40.920 Uttam Kumaran: and where? Where was that again? Somewhere here.
192 00:17:42.887 ⇒ 00:17:44.439 Ryan: Should be dbt.
193 00:17:44.850 ⇒ 00:17:48.590 Ryan: then dbt, and then Terry add reporting
194 00:17:48.880 ⇒ 00:17:50.113 Ryan: or part of a different.
195 00:17:50.360 ⇒ 00:17:51.319 Uttam Kumaran: It’s in heat.
196 00:17:51.550 ⇒ 00:17:53.430 Uttam Kumaran: it’s in. It’s in each of them, or.
197 00:17:56.520 ⇒ 00:18:01.149 Ryan: actually, I’m not sure. Was it in in here, Brian? The elementary.
198 00:18:01.450 ⇒ 00:18:03.199 Ryan: most technical study.
199 00:18:06.450 ⇒ 00:18:08.300 Uttam Kumaran: Brian, you know which one it’s in.
200 00:18:10.520 ⇒ 00:18:11.710 Brian Pei: You mean the
201 00:18:11.870 ⇒ 00:18:13.310 Brian Pei: the big table.
202 00:18:13.650 ⇒ 00:18:14.500 Uttam Kumaran: Yeah.
203 00:18:14.790 ⇒ 00:18:16.629 Brian Pei: Yeah, one second, it’s in my
204 00:18:18.190 ⇒ 00:18:19.440 Brian Pei: to my document.
205 00:18:20.870 ⇒ 00:18:23.290 Uttam Kumaran: Maybe it’s just dbt.
206 00:18:23.420 ⇒ 00:18:25.220 Uttam Kumaran: rotten results.
207 00:18:25.880 ⇒ 00:18:26.840 Ryan: Yeah, that’s fine. Yeah.
208 00:18:26.840 ⇒ 00:18:27.869 Brian Pei: Think it’s that one
209 00:18:28.170 ⇒ 00:18:29.550 Brian Pei: dbt run results.
210 00:18:30.250 ⇒ 00:18:31.290 Uttam Kumaran: Okay, cool.
211 00:18:42.380 ⇒ 00:18:46.010 Uttam Kumaran: Alright, but these are old ones.
212 00:18:46.010 ⇒ 00:18:47.790 Brian Pei: It shows successes as well.
213 00:18:52.390 ⇒ 00:18:54.860 Uttam Kumaran: See these. This are 7, 16,
214 00:18:58.850 ⇒ 00:19:01.010 Uttam Kumaran: like this one is from this morning.
215 00:19:02.190 ⇒ 00:19:03.160 Uttam Kumaran: But what?
216 00:19:03.310 ⇒ 00:19:04.640 Uttam Kumaran: And
217 00:19:19.200 ⇒ 00:19:20.740 Uttam Kumaran: Oh, it’s probably this.
218 00:19:25.550 ⇒ 00:19:27.869 Uttam Kumaran: So this we need to do something about
219 00:19:30.290 ⇒ 00:19:31.750 Uttam Kumaran: yeah, just need to.
220 00:19:31.750 ⇒ 00:19:32.690 Ryan: Thing, about.
221 00:19:32.960 ⇒ 00:19:38.640 Uttam Kumaran: I tried to ask last week what the steps were, and I don’t. I think, Ryan, you created an issue. But
222 00:19:39.540 ⇒ 00:19:41.130 Uttam Kumaran: yeah, this is horrible.
223 00:19:46.480 ⇒ 00:19:50.149 Uttam Kumaran: This is Dbt. Elementary. Mart. Oh, but this isn’t.
224 00:19:51.970 ⇒ 00:19:53.700 Uttam Kumaran: Oh, this isn’t even a
225 00:19:54.510 ⇒ 00:19:56.030 Uttam Kumaran: elementary thing.
226 00:19:58.570 ⇒ 00:19:59.965 Ryan: Yeah, it’s confusing.
227 00:20:04.070 ⇒ 00:20:06.449 Uttam Kumaran: Okay, so it’s probably just elementary. Then.
228 00:20:10.300 ⇒ 00:20:12.140 Uttam Kumaran: yeah, okay, so
229 00:20:13.330 ⇒ 00:20:15.080 Uttam Kumaran: I don’t know what this is.
230 00:20:22.300 ⇒ 00:20:27.769 Ryan: I wonder if it’s it’s related to what Bryce did? Where.
231 00:20:27.770 ⇒ 00:20:28.509 Uttam Kumaran: Oh, okay.
232 00:20:28.810 ⇒ 00:20:29.110 Ryan: Kiri.
233 00:20:29.110 ⇒ 00:20:29.840 Uttam Kumaran: He changed it.
234 00:20:29.840 ⇒ 00:20:33.070 Ryan: All the DVD prefixes. So.
235 00:20:34.450 ⇒ 00:20:35.710 Uttam Kumaran: That affect our
236 00:20:36.870 ⇒ 00:20:39.150 Uttam Kumaran: Did that affect historical data here.
237 00:20:39.930 ⇒ 00:20:40.770 Ryan: I.
238 00:20:41.850 ⇒ 00:20:49.182 Ryan: I don’t think it affects historical data, but like after that point after he pushed the Pr. It’s
239 00:20:50.420 ⇒ 00:20:52.820 Ryan: no, no new daytime, too.
240 00:20:54.110 ⇒ 00:20:55.939 Ryan: to those standards. Yes.
241 00:20:58.250 ⇒ 00:21:03.150 Uttam Kumaran: I mean like, can we still see before, like can we see test before the 16.th
242 00:21:04.770 ⇒ 00:21:06.160 Ryan: Oh, that’s a good
243 00:21:06.580 ⇒ 00:21:07.899 Ryan: question. I don’t know.
244 00:21:11.290 ⇒ 00:21:11.699 Ryan: Checked.
245 00:21:11.700 ⇒ 00:21:12.790 Brian Pei: 20 slack.
246 00:21:12.790 ⇒ 00:21:15.830 Uttam Kumaran: I mean the the Ui doesn’t even have the Ui doesn’t have.
247 00:21:18.390 ⇒ 00:21:19.510 Ryan: Yeah, I.
248 00:21:19.510 ⇒ 00:21:23.099 Brian Pei: This table. The table I shared in chat has 20. Second
249 00:21:23.170 ⇒ 00:21:24.660 Brian Pei: July 20, second.
250 00:21:25.460 ⇒ 00:21:26.340 Uttam Kumaran: Oh, really.
251 00:21:26.660 ⇒ 00:21:31.029 Brian Pei: It’s instead of Dbt underscore elementary. It’s just elementary.
252 00:21:31.380 ⇒ 00:21:36.599 Uttam Kumaran: Yeah, so that’s exactly. But this, this ui isn’t pointing to that.
253 00:21:41.840 ⇒ 00:21:43.419 Uttam Kumaran: okay, I mean.
254 00:21:43.970 ⇒ 00:21:50.170 Ryan: We’ll have to fix that. I think that’s because the workflow file has the Dbt prefix in it.
255 00:21:50.738 ⇒ 00:21:52.969 Ryan: That’s why it’s like pointing to that
256 00:21:53.520 ⇒ 00:21:55.770 Ryan: table with a Dbt. Prefix.
257 00:21:57.140 ⇒ 00:21:59.370 Uttam Kumaran: Yeah, that’s okay. It’s just like
258 00:21:59.780 ⇒ 00:22:05.729 Uttam Kumaran: it’s just like, every time I’m looking in here, there’s an issue, and it’s just a little bit frustrating.
259 00:22:07.157 ⇒ 00:22:17.000 Uttam Kumaran: But let’s just keep going. We need to note all these things down. And if they’re gonna make changes to the schemas, they need to basically clean this whole thing up.
260 00:22:20.040 ⇒ 00:22:25.760 Uttam Kumaran: yeah, or it’s like, basically unusable, like, how do you? We can’t. There’s not like you can’t. You don’t know, or anything is in here.
261 00:22:29.540 ⇒ 00:22:30.420 Uttam Kumaran: like.
262 00:22:30.940 ⇒ 00:22:38.060 Uttam Kumaran: yeah, I mean Ryan or Brian. If you guys see this, and they’re like making these changes. And it’s like, look clearly like blew this whole thing up
263 00:22:38.330 ⇒ 00:22:40.369 Uttam Kumaran: like we need to basically
264 00:22:40.510 ⇒ 00:22:42.679 Uttam Kumaran: like, not have this. So
265 00:22:45.360 ⇒ 00:22:46.790 Uttam Kumaran: okay, let’s keep going.
266 00:22:47.240 ⇒ 00:22:48.420 Uttam Kumaran: though.
267 00:22:48.520 ⇒ 00:22:50.790 Uttam Kumaran: we want to look at failures.
268 00:22:51.740 ⇒ 00:22:52.570 Uttam Kumaran: And
269 00:22:55.200 ⇒ 00:22:57.099 Uttam Kumaran: I don’t know what the status
270 00:22:58.730 ⇒ 00:23:01.310 Uttam Kumaran: column is for these guys. But
271 00:23:02.270 ⇒ 00:23:04.410 Uttam Kumaran: status is just success.
272 00:23:06.740 ⇒ 00:23:08.420 Uttam Kumaran: Let’s just do like.
273 00:23:22.000 ⇒ 00:23:22.890 Uttam Kumaran: okay.
274 00:23:23.450 ⇒ 00:23:25.979 Uttam Kumaran: looks like these are the ones we’re looking at today.
275 00:23:30.680 ⇒ 00:23:34.470 Uttam Kumaran: and we’re going to start with the all orders line items got 3 and a 3 results
276 00:23:35.950 ⇒ 00:23:38.308 Uttam Kumaran: cool. So let’s take the
277 00:23:40.480 ⇒ 00:23:41.640 Uttam Kumaran: query.
278 00:24:04.250 ⇒ 00:24:07.649 Uttam Kumaran: okay. So look, there’s a bunch that are duplicated.
279 00:24:08.650 ⇒ 00:24:10.330 Uttam Kumaran: This one is weird.
280 00:24:12.160 ⇒ 00:24:14.170 Uttam Kumaran: and some of these are like 6.
281 00:24:15.400 ⇒ 00:24:17.939 Uttam Kumaran: So I guess we’ll just go one by one.
282 00:24:18.020 ⇒ 00:24:19.899 Uttam Kumaran: Take a look what’s going on.
283 00:24:20.550 ⇒ 00:24:24.299 Uttam Kumaran: And actually, better yet, I’m just gonna do this in real.
284 00:24:25.770 ⇒ 00:24:27.620 Uttam Kumaran: It’ll be way easier.
285 00:24:59.050 ⇒ 00:25:03.350 Uttam Kumaran: Looks like it’s probably something to do with has refund?
286 00:25:11.770 ⇒ 00:25:14.020 Uttam Kumaran: So let’s take a look at that.
287 00:25:25.300 ⇒ 00:25:27.129 Uttam Kumaran: So where is the latest?
288 00:25:30.750 ⇒ 00:25:33.170 Uttam Kumaran: like all orders like.
289 00:25:34.100 ⇒ 00:25:34.620 Uttam Kumaran: is it.
290 00:25:34.620 ⇒ 00:25:36.439 Ryan: July 19.th Yeah, yeah.
291 00:25:37.040 ⇒ 00:25:37.780 Uttam Kumaran: Okay.
292 00:26:20.570 ⇒ 00:26:25.359 Uttam Kumaran: okay, so looks like the top one and the bottom one are the same.
293 00:26:27.340 ⇒ 00:26:28.730 Uttam Kumaran: basically.
294 00:26:31.660 ⇒ 00:26:34.340 Uttam Kumaran: But then there’s something to do with this refund
295 00:26:39.230 ⇒ 00:26:43.129 Uttam Kumaran: and then I don’t know why these are both separated, though.
296 00:26:52.950 ⇒ 00:26:55.180 Uttam Kumaran: okay. But like, let’s go look at the order.
297 00:27:04.270 ⇒ 00:27:06.109 Uttam Kumaran: Okay? So it was a refund
298 00:27:18.300 ⇒ 00:27:19.440 Uttam Kumaran: from, I guess.
299 00:27:19.440 ⇒ 00:27:22.059 Ryan: Something. Yeah, we’ll have to look at the
300 00:27:22.780 ⇒ 00:27:23.820 Ryan: the sequel
301 00:27:23.920 ⇒ 00:27:27.238 Ryan: like how the Reclams are being joined and stuff like that.
302 00:27:29.390 ⇒ 00:27:35.509 Uttam Kumaran: Okay. So let’s just do that. I’m just gonna kind of paste these in notion. So we can just keep continuing.
303 00:27:39.280 ⇒ 00:27:41.590 Nicolas Sucari: Even though it still has like
304 00:27:41.690 ⇒ 00:27:44.640 Nicolas Sucari: duplicate the same order. Item, right?
305 00:27:45.483 ⇒ 00:27:53.090 Nicolas Sucari: I mean, we identify the refund. The refund is another line. Yeah, we need to look into that. But we still have, like 2 lines with the same information. There.
306 00:27:53.670 ⇒ 00:27:54.590 Uttam Kumaran: Yeah.
307 00:27:57.320 ⇒ 00:27:58.240 Uttam Kumaran: okay.
308 00:28:15.630 ⇒ 00:28:16.630 Uttam Kumaran: okay.
309 00:28:16.960 ⇒ 00:28:21.839 Uttam Kumaran: let’s look at. Let’s look at. So there’s probably something to do with this that’ll clear up a bunch of them.
310 00:28:22.010 ⇒ 00:28:24.020 Uttam Kumaran: I’ll leave that to y’all.
311 00:28:25.165 ⇒ 00:28:25.790 Uttam Kumaran: Let’s
312 00:28:26.410 ⇒ 00:28:27.700 Uttam Kumaran: look at a note.
313 00:28:35.954 ⇒ 00:28:42.879 Nicolas Sucari: What you’re saying, Brian, through the chat is the 2 lines of orders, or they have 300 stuff.
314 00:28:44.680 ⇒ 00:28:47.586 Brian Pei: That’s my best guess of where the those
315 00:28:48.060 ⇒ 00:28:50.310 Brian Pei: refund duplicates would be coming from. Yeah.
316 00:28:50.310 ⇒ 00:28:50.990 Uttam Kumaran: Yeah.
317 00:28:51.800 ⇒ 00:28:52.990 Uttam Kumaran: Okay, cool.
318 00:28:54.220 ⇒ 00:28:56.239 Uttam Kumaran: Gonna just paste that into here.
319 00:29:03.100 ⇒ 00:29:10.239 Uttam Kumaran: Okay, let’s look at okay, let’s not. I don’t care that much about inventory. Let’s look at the shipments.
320 00:29:13.330 ⇒ 00:29:14.320 Uttam Kumaran: Okay.
321 00:29:26.060 ⇒ 00:29:29.540 Uttam Kumaran: alright. So we got a bunch that are duplicate
322 00:29:29.700 ⇒ 00:29:31.200 Uttam Kumaran: ids.
323 00:29:31.210 ⇒ 00:29:33.690 Uttam Kumaran: Let’s go do the same thing.
324 00:29:40.260 ⇒ 00:29:41.790 Uttam Kumaran: Our shipments
325 00:29:42.670 ⇒ 00:29:44.710 Uttam Kumaran: go to Id.
326 00:29:45.890 ⇒ 00:29:49.189 Uttam Kumaran: or what? What is what? Exactly what id is this?
327 00:29:53.660 ⇒ 00:29:56.070 Uttam Kumaran: Oh, wait. This. Did I just put the same one in here?
328 00:29:57.990 ⇒ 00:29:59.210 Uttam Kumaran: I copied it.
329 00:30:03.152 ⇒ 00:30:06.530 Uttam Kumaran: Okay, so it’s unique shipments. Order. Id.
330 00:30:08.650 ⇒ 00:30:11.519 Uttam Kumaran: it looks like only 3. So not bad.
331 00:30:29.870 ⇒ 00:30:33.030 Uttam Kumaran: Okay, so let’s take a look at one of these
332 00:30:41.750 ⇒ 00:30:44.250 Uttam Kumaran: and what the order item, Id.
333 00:30:57.620 ⇒ 00:31:01.250 Uttam Kumaran: yeah, 4 shipments seems odd.
334 00:31:02.280 ⇒ 00:31:04.540 Uttam Kumaran: So let’s go look this guy up
335 00:31:09.500 ⇒ 00:31:10.449 Uttam Kumaran: that we go.
336 00:31:10.680 ⇒ 00:31:11.360 Uttam Kumaran: B.
337 00:31:12.650 ⇒ 00:31:15.910 Uttam Kumaran: If I just held shift and you click, it’ll copy.
338 00:31:16.910 ⇒ 00:31:18.759 Uttam Kumaran: Let’s go. Look at this.
339 00:31:23.140 ⇒ 00:31:24.879 Uttam Kumaran: It’s like 2 items.
340 00:31:29.050 ⇒ 00:31:30.580 Uttam Kumaran: June 15, th
341 00:31:35.660 ⇒ 00:31:37.000 Uttam Kumaran: 2 shipments.
342 00:31:40.780 ⇒ 00:31:42.099 Uttam Kumaran: but it is 2,
343 00:31:42.380 ⇒ 00:31:43.500 Uttam Kumaran: though.
344 00:31:44.000 ⇒ 00:31:46.619 Uttam Kumaran: Oh, I see what’s happening.
345 00:31:50.210 ⇒ 00:31:51.840 Uttam Kumaran: So in the case when
346 00:31:53.240 ⇒ 00:31:58.529 Uttam Kumaran: it’s cause it’s it’s not. So actually, let’s go look at okay, let’s go. Do some.
347 00:32:24.930 ⇒ 00:32:33.980 Uttam Kumaran: Okay? I mean in. So in, in all in all order items, this is gonna come in as one item. But 2 quantities
348 00:32:34.750 ⇒ 00:32:37.649 Uttam Kumaran: or, yeah, quantity is 2
349 00:32:43.340 ⇒ 00:32:50.100 Uttam Kumaran: which means that when we do the join between all order items and shipments. We’re gonna get
350 00:32:50.270 ⇒ 00:32:52.979 Uttam Kumaran: multiple shipments for one order. Item.
351 00:32:57.110 ⇒ 00:33:03.070 Uttam Kumaran: yeah, I guess I’m not sure like thinking out loud. The goal was originally just to be like.
352 00:33:03.280 ⇒ 00:33:04.400 Uttam Kumaran: every
353 00:33:04.620 ⇒ 00:33:07.069 Uttam Kumaran: order item has a shipment attached.
354 00:33:08.120 ⇒ 00:33:11.030 Uttam Kumaran: So we probably can get rid of
355 00:33:12.400 ⇒ 00:33:14.889 Uttam Kumaran: that. And instead, Justin.
356 00:33:15.480 ⇒ 00:33:17.770 Uttam Kumaran: yeah, I guess I don’t know. What do you guys think.
357 00:33:19.750 ⇒ 00:33:20.400 Brian Pei: Aggregate
358 00:33:21.080 ⇒ 00:33:22.620 Brian Pei: the shipments by
359 00:33:23.040 ⇒ 00:33:24.090 Brian Pei: motored
360 00:33:24.870 ⇒ 00:33:25.600 Brian Pei: order.
361 00:33:25.600 ⇒ 00:33:26.020 Uttam Kumaran: Well.
362 00:33:26.020 ⇒ 00:33:26.470 Brian Pei: And.
363 00:33:26.470 ⇒ 00:33:32.459 Uttam Kumaran: An order could have multiple shipments. And also the I didn’t think about this case where, like
364 00:33:32.900 ⇒ 00:33:35.260 Uttam Kumaran: double quantity, same order. Item.
365 00:33:38.030 ⇒ 00:33:40.410 Uttam Kumaran: So in this case it’s actually
366 00:33:40.530 ⇒ 00:33:43.667 Uttam Kumaran: like it breaches the rule we had. But
367 00:33:45.580 ⇒ 00:33:46.900 Uttam Kumaran: I wonder
368 00:33:47.720 ⇒ 00:33:52.040 Uttam Kumaran: like what a test, because basically, every order item needs to have a shipment
369 00:33:53.520 ⇒ 00:33:56.660 Uttam Kumaran: like how so? How do I? How do we put that test in place.
370 00:34:02.580 ⇒ 00:34:03.870 Brian Pei: Second.
371 00:34:12.079 ⇒ 00:34:14.899 Ryan: What shipment data do we need, like the shipment?
372 00:34:15.309 ⇒ 00:34:19.319 Ryan: Wait and shipment? About right? Just those 2
373 00:34:19.969 ⇒ 00:34:21.959 Ryan: and all the all organizations.
374 00:34:24.440 ⇒ 00:34:26.899 Uttam Kumaran: we don’t really need any shipment data there.
375 00:34:27.120 ⇒ 00:34:27.830 Uttam Kumaran: Cause? Yeah.
376 00:34:27.830 ⇒ 00:34:30.729 Ryan: Yeah, cause we already have shipments, right? Model.
377 00:34:30.739 ⇒ 00:34:32.919 Uttam Kumaran: Yeah, we don’t need any shipment data there.
378 00:34:33.529 ⇒ 00:34:37.649 Uttam Kumaran: But the the the thing isn’t like, it’s not this. This question isn’t like
379 00:34:37.929 ⇒ 00:34:43.369 Uttam Kumaran: putting shipment data into all order items. This is like, how do we enforce
380 00:34:43.479 ⇒ 00:34:44.889 Uttam Kumaran: that? Every
381 00:34:45.939 ⇒ 00:34:49.239 Uttam Kumaran: every line item has an associated shipment.
382 00:34:53.360 ⇒ 00:34:54.060 Brian Pei: I th
383 00:34:54.320 ⇒ 00:34:57.209 Brian Pei: think it’s this one
384 00:34:58.780 ⇒ 00:35:01.160 Brian Pei: relationship test
385 00:35:03.510 ⇒ 00:35:09.640 Brian Pei: test validates that all the records in a trial table have a corresponding record in a parent table. This property
386 00:35:09.720 ⇒ 00:35:12.409 Brian Pei: is referred to as referential integrity.
387 00:35:14.930 ⇒ 00:35:15.430 Brian Pei: Yeah.
388 00:35:15.430 ⇒ 00:35:16.120 Ryan: Yeah, I think.
389 00:35:16.120 ⇒ 00:35:16.849 Uttam Kumaran: All that.
390 00:35:16.850 ⇒ 00:35:18.419 Ryan: That’s about right? Yeah.
391 00:35:19.570 ⇒ 00:35:23.590 Uttam Kumaran: And also this may take a few days like
392 00:35:23.660 ⇒ 00:35:27.060 Uttam Kumaran: not every order. Item, because shipments happen a few days later.
393 00:35:32.170 ⇒ 00:35:36.360 Ryan: So so we just has to have a filter, maybe like a week or something
394 00:35:38.560 ⇒ 00:35:39.629 Ryan: like it. Doesn’t.
395 00:35:40.110 ⇒ 00:35:41.500 Ryan: This shouldn’t
396 00:35:43.710 ⇒ 00:35:46.780 Ryan: work a week or something.
397 00:35:46.780 ⇒ 00:35:48.370 Uttam Kumaran: Last after
398 00:35:50.180 ⇒ 00:35:52.119 Uttam Kumaran: and order item.
399 00:35:52.300 ⇒ 00:35:53.000 Uttam Kumaran: breathe
400 00:35:55.540 ⇒ 00:35:58.020 Uttam Kumaran: needs to have a filter.
401 00:35:58.560 ⇒ 00:35:59.400 Uttam Kumaran: Okay?
402 00:35:59.590 ⇒ 00:36:05.299 Uttam Kumaran: Nice thing is, there’s only a few of these. But we are seeing duplicates, so we should fix that
403 00:36:06.890 ⇒ 00:36:09.299 Uttam Kumaran: all discounts. I guess we can
404 00:36:09.630 ⇒ 00:36:12.053 Uttam Kumaran: take a look at that. Or should we look at some of these?
405 00:36:14.330 ⇒ 00:36:18.480 Uttam Kumaran: Some of these I feel like are going to be okay. Well, let’s just look at some of these.
406 00:36:18.560 ⇒ 00:36:22.259 Uttam Kumaran: No value issues in all in line. Item, order. Id.
407 00:36:33.970 ⇒ 00:36:35.939 Uttam Kumaran: did I just click on my own
408 00:36:49.180 ⇒ 00:36:54.520 Uttam Kumaran: yeah. So line item or order, Id should not be null in this table. So that’s what we’re gonna look at.
409 00:36:58.840 ⇒ 00:37:01.139 Uttam Kumaran: I don’t honestly don’t have any guesses.
410 00:37:04.740 ⇒ 00:37:08.039 Uttam Kumaran: Okay. So what I’m gonna do is just look at where it’s all
411 00:37:23.830 ⇒ 00:37:25.080 Uttam Kumaran: bear
412 00:37:32.400 ⇒ 00:37:33.690 Uttam Kumaran: nice.
413 00:37:37.530 ⇒ 00:37:38.880 Ryan: I think it’s like it’s
414 00:37:39.000 ⇒ 00:37:41.119 Ryan: is it all from Walmart cause? I don’t think.
415 00:37:41.120 ⇒ 00:37:41.700 Uttam Kumaran: Yeah.
416 00:37:41.740 ⇒ 00:37:43.720 Ryan: Line items in Walmart, right?
417 00:37:44.590 ⇒ 00:37:46.549 Uttam Kumaran: I don’t think so.
418 00:37:47.270 ⇒ 00:37:47.995 Uttam Kumaran: but
419 00:37:49.790 ⇒ 00:37:50.310 Ryan: Yeah.
420 00:37:50.310 ⇒ 00:37:51.530 Uttam Kumaran: All from Walmart, which is.
421 00:37:51.960 ⇒ 00:37:53.439 Uttam Kumaran: which is a good sign.
422 00:37:54.080 ⇒ 00:37:56.550 Uttam Kumaran: because this this isn’t causing duplicates.
423 00:38:00.850 ⇒ 00:38:02.830 Uttam Kumaran: yeah. And I don’t believe
424 00:38:05.180 ⇒ 00:38:05.970 Uttam Kumaran: the
425 00:38:11.990 ⇒ 00:38:14.280 Uttam Kumaran: yeah. We just have orders.
426 00:38:15.270 ⇒ 00:38:15.940 Ryan: Yeah.
427 00:38:16.920 ⇒ 00:38:18.630 Uttam Kumaran: I guess we have. Item.
428 00:38:19.370 ⇒ 00:38:21.809 Uttam Kumaran: item, line number. Oh, there might be here.
429 00:38:28.810 ⇒ 00:38:32.000 Ryan: It’s just the item, the last number, like the
430 00:38:33.020 ⇒ 00:38:34.919 Ryan: one with 2 or something. Oh.
431 00:38:35.140 ⇒ 00:38:35.830 Ryan: yeah.
432 00:38:37.570 ⇒ 00:38:42.739 Uttam Kumaran: Okay. But let’s look at this one. So this one looks like it’s a there’s 2 items for this order.
433 00:38:43.400 ⇒ 00:38:46.640 Uttam Kumaran: 1, 0, 8, 9, 1, 3. Let’s look at this one
434 00:38:47.460 ⇒ 00:38:50.479 Uttam Kumaran: cause. It looks like we probably do have to solve this problem
435 00:38:51.120 ⇒ 00:38:52.970 Uttam Kumaran: order. Id.
436 00:39:01.050 ⇒ 00:39:02.070 Uttam Kumaran: What.
437 00:39:08.300 ⇒ 00:39:10.090 Nicolas Sucari: Isn’t order. Item id.
438 00:39:14.980 ⇒ 00:39:18.129 Uttam Kumaran: This isn’t. Oh, sorry. I’m not even looking. Well.
439 00:39:22.050 ⇒ 00:39:25.729 Uttam Kumaran: okay, well, let’s go. Look at the Walmart code. I don’t know what’s going on.
440 00:39:37.820 ⇒ 00:39:39.650 Uttam Kumaran: Oh, okay.
441 00:39:40.990 ⇒ 00:39:42.130 Uttam Kumaran: okay.
442 00:39:49.470 ⇒ 00:39:51.439 Uttam Kumaran: and get rid of this stuff, too.
443 00:39:54.310 ⇒ 00:39:55.020 Uttam Kumaran: Yeah,
444 00:39:59.920 ⇒ 00:40:02.010 Uttam Kumaran: let’s just look at. Let’s just look at.
445 00:40:02.260 ⇒ 00:40:02.970 Uttam Kumaran: There’s
446 00:40:12.630 ⇒ 00:40:15.910 Uttam Kumaran: I don’t think Dbt staging is the right thing.
447 00:40:17.880 ⇒ 00:40:18.890 Uttam Kumaran: or maybe
448 00:40:31.600 ⇒ 00:40:32.590 Uttam Kumaran: I see.
449 00:40:48.920 ⇒ 00:40:52.030 Ryan: Looks like it’s mics dot mics.
450 00:40:54.120 ⇒ 00:40:55.070 Uttam Kumaran: Oh, okay.
451 00:40:59.250 ⇒ 00:41:02.130 Uttam Kumaran: did I just hallucinate that like entire thing?
452 00:41:02.360 ⇒ 00:41:03.970 Uttam Kumaran: Was that not the right one?
453 00:41:25.130 ⇒ 00:41:30.230 Uttam Kumaran: Okay, well, what did I even do? Start with. So let’s go to brainboard.
454 00:41:30.550 ⇒ 00:41:32.410 Uttam Kumaran: Etl Walmart
455 00:41:32.510 ⇒ 00:41:33.700 Uttam Kumaran: orders.
456 00:41:36.180 ⇒ 00:41:36.980 Uttam Kumaran: that
457 00:41:38.900 ⇒ 00:41:39.960 Uttam Kumaran: orders.
458 00:41:43.800 ⇒ 00:41:46.640 Uttam Kumaran: Pretty sure. I was looking at this lady.
459 00:41:51.870 ⇒ 00:41:54.059 Uttam Kumaran: Right? Okay, let’s just look at like, what?
460 00:41:59.720 ⇒ 00:42:01.330 Uttam Kumaran: When did she order this?
461 00:42:03.400 ⇒ 00:42:05.749 Uttam Kumaran: I can’t even tell what the order date is.
462 00:42:10.240 ⇒ 00:42:14.299 Uttam Kumaran: Okay, Margaret. Something she ordered from grapevine.
463 00:42:15.190 ⇒ 00:42:15.920 Uttam Kumaran: Okay.
464 00:42:24.230 ⇒ 00:42:26.419 Uttam Kumaran: okay, well, I can’t really even tell.
465 00:42:48.900 ⇒ 00:42:50.340 Uttam Kumaran: Okay, here we go.
466 00:42:51.540 ⇒ 00:42:53.010 Uttam Kumaran: these 3.
467 00:43:33.430 ⇒ 00:43:34.020 Uttam Kumaran: What am I.
468 00:43:34.020 ⇒ 00:43:35.599 Ryan: And wasn’t that order? Id.
469 00:43:36.530 ⇒ 00:43:37.650 Uttam Kumaran: Yeah. Thanks.
470 00:43:38.590 ⇒ 00:43:39.440 Uttam Kumaran: Okay, cool.
471 00:43:42.970 ⇒ 00:43:46.560 Uttam Kumaran: Okay. Some other weirdness. Then what this cost?
472 00:44:02.830 ⇒ 00:44:04.510 Uttam Kumaran: Let’s look at this top one.
473 00:44:12.630 ⇒ 00:44:13.640 Ryan: So the last
474 00:44:14.780 ⇒ 00:44:15.800 Ryan: to our.
475 00:44:15.880 ⇒ 00:44:20.100 Ryan: it’s basically because it’s a different part number product skew.
476 00:44:20.440 ⇒ 00:44:21.230 Ryan: But the first.st
477 00:44:21.665 ⇒ 00:44:22.100 Uttam Kumaran: Yeah.
478 00:44:22.100 ⇒ 00:44:23.670 Ryan: 2 duplicates is.
479 00:44:24.330 ⇒ 00:44:26.229 Uttam Kumaran: Okay. But this looks like a.
480 00:44:26.780 ⇒ 00:44:29.830 Uttam Kumaran: this looks so these 2 actually looks like some weird duplicates.
481 00:44:30.120 ⇒ 00:44:33.265 Uttam Kumaran: This one looks like it’s a order. It’s like, actually
482 00:44:34.560 ⇒ 00:44:36.370 Uttam Kumaran: multiple order items.
483 00:44:36.960 ⇒ 00:44:37.480 Ryan: Right.
484 00:44:40.000 ⇒ 00:44:44.229 Uttam Kumaran: So I’m just gonna take this and paste that in somewhere. But that’s actually
485 00:44:44.290 ⇒ 00:44:45.940 Uttam Kumaran: doesn’t look like it’s kind of that.
486 00:44:46.790 ⇒ 00:44:49.569 Uttam Kumaran: It’s up. Seems like, there’s a mix of problems on this one.
487 00:45:06.360 ⇒ 00:45:11.410 Ryan: Seems like it’s all mark for now. For the knowledge in all orders.
488 00:45:11.410 ⇒ 00:45:11.970 Uttam Kumaran: Yeah.
489 00:45:14.350 ⇒ 00:45:15.045 Ryan: Tattoo.
490 00:45:20.980 ⇒ 00:45:22.892 Uttam Kumaran: So it looks like
491 00:45:24.610 ⇒ 00:45:27.200 Uttam Kumaran: resolve. No value issues. So
492 00:45:27.430 ⇒ 00:45:28.630 Uttam Kumaran: one is
493 00:46:17.000 ⇒ 00:46:19.280 Uttam Kumaran: I also think that maybe this
494 00:46:22.860 ⇒ 00:46:24.950 Uttam Kumaran: they may have just ordered 2 of these.
495 00:46:29.540 ⇒ 00:46:34.159 Ryan: Possible as well like. It’s also a different line. But it’s the same item.
496 00:46:35.190 ⇒ 00:46:38.773 Uttam Kumaran: Yeah. So that’s I’m gonna I’m just gonna put that in notes that like,
497 00:47:01.980 ⇒ 00:47:02.700 Uttam Kumaran: okay.
498 00:47:08.780 ⇒ 00:47:11.110 Uttam Kumaran: shipments on the order. Item, id.
499 00:47:28.020 ⇒ 00:47:30.099 Uttam Kumaran: it looks like there’s a bunch here.
500 00:47:50.870 ⇒ 00:47:52.760 Uttam Kumaran: It’s all the primus ones.
501 00:48:00.710 ⇒ 00:48:01.590 Uttam Kumaran: but
502 00:48:05.360 ⇒ 00:48:10.349 Uttam Kumaran: these should be associated with orders right like with shopify orders.
503 00:48:13.040 ⇒ 00:48:14.770 Brian Pei: These are where
504 00:48:16.100 ⇒ 00:48:18.880 Brian Pei: the order id from primus is this.
505 00:48:18.930 ⇒ 00:48:20.060 Brian Pei: coalesce
506 00:48:21.430 ⇒ 00:48:22.340 Brian Pei: from.
507 00:48:22.340 ⇒ 00:48:23.160 Uttam Kumaran: Yeah.
508 00:48:23.530 ⇒ 00:48:25.059 Brian Pei: Primus source, and
509 00:48:25.310 ⇒ 00:48:29.610 Brian Pei: remember a lot of them just come in either as null or as a
510 00:48:31.480 ⇒ 00:48:33.570 Brian Pei: not decipherable string.
511 00:48:35.550 ⇒ 00:48:36.937 Uttam Kumaran: So I wonder
512 00:48:40.040 ⇒ 00:48:44.910 Ryan: Shouldn’t we filter these out in just they don’t have like order ids associated to them.
513 00:48:46.450 ⇒ 00:48:51.050 Uttam Kumaran: No cause we’re gonna miss out on this cost. So I I actually like, I’m
514 00:48:51.824 ⇒ 00:48:55.460 Uttam Kumaran: in all situations I’d rather the cost. Behold
515 00:48:56.116 ⇒ 00:49:00.913 Uttam Kumaran: and then the secondary thing is like that. There are duplicates. And of course there’s some issues. But
516 00:49:01.740 ⇒ 00:49:04.920 Uttam Kumaran: it looks like, probably in this case we need to just
517 00:49:06.520 ⇒ 00:49:09.799 Uttam Kumaran: figure out a way to join these with the proper
518 00:49:10.260 ⇒ 00:49:12.580 Uttam Kumaran: order. And it looks like also
519 00:49:12.630 ⇒ 00:49:16.229 Uttam Kumaran: you look at this warehouse, Brian. This is where that issue is coming from.
520 00:49:17.110 ⇒ 00:49:17.670 Ryan: Yeah.
521 00:49:19.189 ⇒ 00:49:20.659 Uttam Kumaran: I don’t actually.
522 00:49:20.660 ⇒ 00:49:21.860 Brian Pei: Out of Primus, too.
523 00:49:23.140 ⇒ 00:49:23.813 Uttam Kumaran: Yeah.
524 00:49:30.050 ⇒ 00:49:31.770 Ryan: I guess we need some sort of mapping.
525 00:49:31.770 ⇒ 00:49:32.330 Uttam Kumaran: Easy.
526 00:49:32.330 ⇒ 00:49:38.260 Ryan: Like. So we know which previous warehouse maps to the actual warehouse, or something.
527 00:49:38.780 ⇒ 00:49:46.340 Uttam Kumaran: No, I don’t know what this. I don’t know what. I don’t know what this is like. All the premise stuff is gonna come from ya pank
528 00:49:49.223 ⇒ 00:49:52.126 Brian Pei: That warehouse name from Shopify Primus
529 00:49:52.710 ⇒ 00:49:54.780 Brian Pei: comes from the column shipper name.
530 00:49:54.900 ⇒ 00:49:56.790 Brian Pei: We don’t join to anything.
531 00:49:57.430 ⇒ 00:50:02.779 Uttam Kumaran: But that’s the thing like, I don’t know what shipper name is. It’s actually looks like this is some guy’s name.
532 00:50:02.930 ⇒ 00:50:05.530 Uttam Kumaran: So the one thing I would just.
533 00:50:07.370 ⇒ 00:50:10.250 Uttam Kumaran: I would just, can you guys just ask chuck
534 00:50:12.320 ⇒ 00:50:17.150 Uttam Kumaran: like, does every primus thing come from New York cause? Then we can just fix this
535 00:50:17.829 ⇒ 00:50:20.700 Uttam Kumaran: cause. I don’t think this is. I don’t think this is the
536 00:50:21.360 ⇒ 00:50:22.380 Uttam Kumaran: right field.
537 00:50:23.110 ⇒ 00:50:25.240 Brian Pei: Primus booking data is all
538 00:50:26.820 ⇒ 00:50:27.600 Brian Pei: missed.
539 00:50:27.600 ⇒ 00:50:29.819 Uttam Kumaran: Pretty sure all. I’m pretty sure all these come from New York.
540 00:50:31.340 ⇒ 00:50:35.339 Uttam Kumaran: so I just need to ask him, and then we can. We can hard code this as yah bank
541 00:50:37.920 ⇒ 00:50:39.530 Uttam Kumaran: and put that somewhere.
542 00:51:03.800 ⇒ 00:51:04.480 Uttam Kumaran: Hey?
543 00:51:05.940 ⇒ 00:51:08.700 Uttam Kumaran: But otherwise, yeah, we these need
544 00:51:09.420 ⇒ 00:51:18.440 Uttam Kumaran: order ids, because all of these associated orders are not gonna show that they have been shipped. So this join is gonna fail. Basically.
545 00:51:18.950 ⇒ 00:51:21.160 Uttam Kumaran: that relationship join is going to fail.
546 00:51:21.490 ⇒ 00:51:24.680 Uttam Kumaran: So I know you mentioned that you
547 00:51:24.780 ⇒ 00:51:29.430 Uttam Kumaran: we did this coalesce. We basically probably need to add a 3rd option, which is just
548 00:51:30.090 ⇒ 00:51:31.085 Uttam Kumaran: join
549 00:51:33.730 ⇒ 00:51:37.179 Uttam Kumaran: join to shopify orders on a number of these fields.
550 00:51:39.260 ⇒ 00:51:40.410 Uttam Kumaran: You see what I mean.
551 00:51:41.810 ⇒ 00:51:43.110 Brian Pei: On all the other fields.
552 00:51:44.100 ⇒ 00:51:45.260 Uttam Kumaran: Kind of yeah.
553 00:51:45.590 ⇒ 00:51:48.360 Uttam Kumaran: to basically identify what the order is like.
554 00:51:48.970 ⇒ 00:51:49.380 Uttam Kumaran: The
555 00:51:49.790 ⇒ 00:51:51.920 Brian Pei: Many as possible.
556 00:51:51.920 ⇒ 00:51:54.290 Uttam Kumaran: As many as we can. Yeah.
557 00:51:54.956 ⇒ 00:51:56.954 Ryan: Like we have.
558 00:52:03.920 ⇒ 00:52:04.940 Uttam Kumaran: Yeah,
559 00:52:07.260 ⇒ 00:52:13.355 Uttam Kumaran: let’s see how far we get on that. And then I can make we can. We can call Primus and ask like what the deal is.
560 00:52:15.573 ⇒ 00:52:19.290 Uttam Kumaran: But I do think that with the Zip code
561 00:52:19.920 ⇒ 00:52:21.309 Uttam Kumaran: the state.
562 00:52:24.117 ⇒ 00:52:27.399 Uttam Kumaran: You can probably figure it out.
563 00:52:28.010 ⇒ 00:52:28.980 Uttam Kumaran: but
564 00:52:31.250 ⇒ 00:52:35.610 Uttam Kumaran: I don’t know what else we’re getting from Primus like we’re getting probably getting an address right.
565 00:52:39.279 ⇒ 00:52:40.759 Brian Pei: There’s an address. Yeah.
566 00:52:40.760 ⇒ 00:52:44.499 Uttam Kumaran: Yeah. So then I’m sure you can just find the who the order is from.
567 00:52:45.750 ⇒ 00:52:52.640 Uttam Kumaran: And the nice thing is the these will always come after. So you’ll never have a case where there this isn’t associated.
568 00:52:56.350 ⇒ 00:53:00.279 Uttam Kumaran: and yeah, I mean, this is just it’s just stupid that the data sucks.
569 00:54:00.290 ⇒ 00:54:03.540 Uttam Kumaran: Okay, these are freshness anomalies.
570 00:54:08.470 ⇒ 00:54:11.659 Uttam Kumaran: let’s just take a look at those. I don’t know what’s going on.
571 00:54:17.270 ⇒ 00:54:19.080 Uttam Kumaran: Will we see those here, Ryan?
572 00:54:19.190 ⇒ 00:54:21.049 Uttam Kumaran: The freshness stuff or no.
573 00:54:25.510 ⇒ 00:54:26.270 Ryan: Us
574 00:54:27.630 ⇒ 00:54:29.120 Ryan: not sure as well.
575 00:54:29.960 ⇒ 00:54:30.640 Uttam Kumaran: Okay.
576 00:54:31.170 ⇒ 00:54:34.000 Ryan: Maybe there’s a freshness table in elementary.
577 00:54:34.670 ⇒ 00:54:36.230 Uttam Kumaran: Yeah. Dbt.
578 00:54:37.010 ⇒ 00:54:40.410 Ryan: Resource, freshness results. Maybe that table.
579 00:54:46.400 ⇒ 00:54:47.090 Uttam Kumaran: In.
580 00:55:09.360 ⇒ 00:55:12.539 Uttam Kumaran: But oh, these are! What have we decided? It’s not the right
581 00:55:12.590 ⇒ 00:55:14.159 Uttam Kumaran: place to go.
582 00:55:15.140 ⇒ 00:55:17.470 Ryan: Yeah, I think that’s correct. We don’t. I don’t
583 00:55:17.580 ⇒ 00:55:20.699 Ryan: think we have any freshness. And normally, since
584 00:55:21.070 ⇒ 00:55:22.130 Ryan: last week.
585 00:55:22.890 ⇒ 00:55:23.780 Ryan: so.
586 00:55:25.130 ⇒ 00:55:25.970 Uttam Kumaran: Oh.
587 00:55:27.800 ⇒ 00:55:30.629 Uttam Kumaran: but what is this one that we looked at then.
588 00:55:38.360 ⇒ 00:55:41.600 Ryan: Oh, yeah, there’s probably it’s like there is one.
589 00:55:41.960 ⇒ 00:55:45.749 Uttam Kumaran: Yes, so elementary source freshness.
590 00:55:47.260 ⇒ 00:55:49.929 Uttam Kumaran: That’s why I don’t know where I can go to see that.
591 00:55:54.360 ⇒ 00:55:56.770 Ryan: I guess it should be that tables but
592 00:55:56.940 ⇒ 00:55:58.259 Ryan: doesn’t look like it’s
593 00:55:59.730 ⇒ 00:56:01.639 Ryan: it has something from today, though.
594 00:56:11.550 ⇒ 00:56:14.689 Uttam Kumaran: Okay? So you can see these anomaly detection ones.
595 00:56:15.080 ⇒ 00:56:17.127 Uttam Kumaran: I’m gonna go ahead and just do
596 00:56:17.880 ⇒ 00:56:20.859 Uttam Kumaran: order by detected at.
597 00:56:27.190 ⇒ 00:56:32.189 Ryan: I guess, cause they’re just warnings, not failures. And we filtered earlier. Just
598 00:56:32.760 ⇒ 00:56:33.490 Ryan: light
599 00:56:33.590 ⇒ 00:56:34.410 Ryan: fail.
600 00:56:36.340 ⇒ 00:56:39.189 Uttam Kumaran: Okay. But why is it coming up? Then?
601 00:56:44.860 ⇒ 00:56:46.509 Uttam Kumaran: These are all warrants.
602 00:56:47.710 ⇒ 00:56:48.870 Uttam Kumaran: Are errors.
603 00:56:50.260 ⇒ 00:56:52.000 Uttam Kumaran: When were these done?
604 00:56:53.230 ⇒ 00:56:55.559 Uttam Kumaran: 7, 22. See that errors, though.
605 00:56:56.610 ⇒ 00:56:57.010 Ryan: Yeah.
606 00:56:57.840 ⇒ 00:56:59.090 Uttam Kumaran: Okay? So
607 00:56:59.560 ⇒ 00:57:02.215 Uttam Kumaran: oh, but these are unique. Okay, so these aren’t
608 00:57:03.400 ⇒ 00:57:04.190 Uttam Kumaran: anomaly.
609 00:57:04.450 ⇒ 00:57:04.710 Ryan: Normal.
610 00:57:04.710 ⇒ 00:57:06.169 Uttam Kumaran: Just unique stuff. Okay?
611 00:57:09.780 ⇒ 00:57:12.160 Uttam Kumaran: And the one that we were looking at is
612 00:57:15.370 ⇒ 00:57:16.910 Uttam Kumaran: Amazon orders.
613 00:57:20.230 ⇒ 00:57:21.170 Uttam Kumaran: This one
614 00:57:27.750 ⇒ 00:57:29.410 Uttam Kumaran: email change.
615 00:57:32.300 ⇒ 00:57:36.140 Uttam Kumaran: Okay? I mean, I guess I don’t know exactly what’s going on. Then.
616 00:57:42.820 ⇒ 00:57:46.800 Uttam Kumaran: Okay, so it’s a warning. So this it’s not gonna fail. If this happens.
617 00:57:48.500 ⇒ 00:57:50.319 Ryan: Yeah, it’s just warning.
618 00:57:51.010 ⇒ 00:57:51.960 Ryan: So.
619 00:57:53.500 ⇒ 00:57:54.140 Uttam Kumaran: Okay.
620 00:58:06.910 ⇒ 00:58:08.500 Uttam Kumaran: okay, great. Okay, cool.
621 00:58:09.830 ⇒ 00:58:14.216 Uttam Kumaran: Okay. I know we’re at time. We got through a couple of them. But
622 00:58:15.430 ⇒ 00:58:20.319 Uttam Kumaran: yeah, I think, basically, guys, we know the process now. So I kind of just want to clear these up.
623 00:58:21.101 ⇒ 00:58:23.590 Uttam Kumaran: I think that, Brian, basically, the way you
624 00:58:23.860 ⇒ 00:58:27.899 Uttam Kumaran: wrote it out is exactly the way we’re gonna do. It seems pretty straightforward.
625 00:58:28.523 ⇒ 00:58:33.510 Uttam Kumaran: The refunds. One is the biggest issue that’s definitely causing some duplicates. So
626 00:58:35.160 ⇒ 00:58:38.009 Uttam Kumaran: but I think there’s looks like there’s
627 00:58:39.182 ⇒ 00:58:43.519 Uttam Kumaran: still a couple remaining but each of these.
628 00:58:44.150 ⇒ 00:58:46.430 Uttam Kumaran: like probably I will just
629 00:58:46.720 ⇒ 00:58:48.810 Uttam Kumaran: we’ll create a ticket for
630 00:58:50.570 ⇒ 00:58:54.950 Uttam Kumaran: this whole thing. And if you guys just want to crank through that this week, that would be
631 00:58:55.150 ⇒ 00:58:56.030 Uttam Kumaran: dope
632 00:58:56.710 ⇒ 00:58:57.570 Uttam Kumaran: charm.
633 00:58:57.940 ⇒ 00:59:01.049 Brian Pei: Can you link this notion, Link? And oh, wait here on the app.
634 00:59:01.560 ⇒ 00:59:06.490 Uttam Kumaran: I’m going to send it to you. I’m gonna put it in here.
635 00:59:06.760 ⇒ 00:59:07.790 Brian Pei: Okay. Cool.
636 00:59:15.740 ⇒ 00:59:19.712 Uttam Kumaran: Cool. Okay. Alright. Great meeting guys.
637 00:59:21.010 ⇒ 00:59:24.370 Uttam Kumaran: Guess I’ll chat with you guys on slack.
638 00:59:25.560 ⇒ 00:59:26.330 Brian Pei: Thank you.
639 00:59:27.090 ⇒ 00:59:28.130 Nicolas Sucari: Thanks guys.
640 00:59:28.410 ⇒ 00:59:29.083 Uttam Kumaran: Appreciate it.
641 00:59:29.420 ⇒ 00:59:30.580 Ryan: Nice, you. Bye-bye.
642 00:59:31.050 ⇒ 00:59:31.820 Uttam Kumaran: Talk, soon
643 00:59:32.010 ⇒ 00:59:33.010 Uttam Kumaran: enjoy Ryan.
644 00:59:34.560 ⇒ 00:59:35.340 Ryan: Thanks.