Meeting Title: US x BF | Standup Date: 2025-11-13 Meeting participants: Uttam Kumaran, Awaish Kumar, Demilade Agboola, Amber Lin, Robert Tseng, Emily Giant
WEBVTT
1 00:00:23.380 ⇒ 00:00:28.569 Uttam Kumaran: How’s the, how’s the copy… oh, I should merge or copy PR, right?
2 00:00:29.330 ⇒ 00:00:30.679 Awaish Kumar: Yeah, I already did.
3 00:00:30.850 ⇒ 00:00:32.060 Awaish Kumar: Crimilarity.
4 00:00:32.310 ⇒ 00:00:33.520 Uttam Kumaran: Nice. Approved it.
5 00:00:39.580 ⇒ 00:00:41.719 Uttam Kumaran: Can I actually also… Yeah, yeah, go ahead, go ahead.
6 00:00:42.860 ⇒ 00:00:48.969 Awaish Kumar: Yeah, initially, why… why I was asking for that integration, it was because
7 00:00:49.050 ⇒ 00:01:02.869 Awaish Kumar: I didn’t have the IAM role, like, in this copy command requires IAM role parameter in the command itself. It does not use, like, dbt credentials to execute this copy command.
8 00:01:03.240 ⇒ 00:01:10.390 Awaish Kumar: So, for that IAM role, I… I just went into your… I logged in using your username and figured out that.
9 00:01:10.390 ⇒ 00:01:10.840 Uttam Kumaran: Okay.
10 00:01:10.840 ⇒ 00:01:17.730 Awaish Kumar: They had a… a role called Redshift S3 Access. That worked, so I just moved it into a…
11 00:01:20.300 ⇒ 00:01:20.880 Uttam Kumaran: Okay.
12 00:01:31.400 ⇒ 00:01:35.400 Awaish Kumar: Right now, yeah, for Nosbee, I… I think…
13 00:01:35.890 ⇒ 00:01:42.119 Awaish Kumar: It runs every day, but it, like, gets the data until yesterday, so I think that’s okay.
14 00:02:32.770 ⇒ 00:02:34.869 Uttam Kumaran: And then Awish, I might get your help.
15 00:02:35.170 ⇒ 00:02:40.380 Uttam Kumaran: after this call, having some DPT issues with another client.
16 00:02:41.260 ⇒ 00:02:43.480 Uttam Kumaran: And I don’t know what I’m doing wrong.
17 00:02:45.910 ⇒ 00:02:49.340 Uttam Kumaran: But I should have done a lot of, like, the first
18 00:02:49.450 ⇒ 00:02:53.640 Uttam Kumaran: kind of core. I set up all the raw marts for Hydra and stuff like that, so…
19 00:02:55.200 ⇒ 00:02:56.049 Demilade Agboola: Okay, sounds great.
20 00:02:56.050 ⇒ 00:02:57.150 Uttam Kumaran: progress.
21 00:04:06.180 ⇒ 00:04:09.320 Robert Tseng: Hello. Is this an internal or external?
22 00:04:09.600 ⇒ 00:04:12.370 Uttam Kumaran: This isn’t… this is, external.
23 00:04:12.680 ⇒ 00:04:13.930 Robert Tseng: Oh, okay.
24 00:04:14.810 ⇒ 00:04:17.290 Amber Lin: I checked Emily to see if she’s coming.
25 00:04:17.290 ⇒ 00:04:18.659 Robert Tseng: No, no, it’s all good, I mean, yeah.
26 00:04:19.350 ⇒ 00:04:21.889 Uttam Kumaran: No, let me ping… I was about to send her a message.
27 00:04:40.350 ⇒ 00:04:46.680 Uttam Kumaran: Yeah, Amber, you could probably drop, I feel like we… Four of us are probably… Save you some time.
28 00:04:46.780 ⇒ 00:04:48.189 Amber Lin: Yeah, talk to you later.
29 00:04:48.470 ⇒ 00:04:49.090 Uttam Kumaran: Thanks.
30 00:04:59.100 ⇒ 00:05:00.290 Uttam Kumaran: Hello.
31 00:05:01.320 ⇒ 00:05:02.260 Emily Giant: Hi!
32 00:05:03.310 ⇒ 00:05:05.330 Uttam Kumaran: We had some forecast questions.
33 00:05:05.330 ⇒ 00:05:06.440 Emily Giant: Yeah, yeah!
34 00:05:06.440 ⇒ 00:05:08.950 Uttam Kumaran: Yeah!
35 00:05:09.250 ⇒ 00:05:21.469 Uttam Kumaran: So, Robert’s been poking at forecast stuff. His first point was like, yeah, I don’t think some of these dashboards are working, and I’m like… Oh, yeah.
36 00:05:21.470 ⇒ 00:05:22.750 Emily Giant: That sounds accurate.
37 00:05:24.130 ⇒ 00:05:24.650 Robert Tseng: Hey, bro.
38 00:05:25.050 ⇒ 00:05:27.999 Robert Tseng: It’s been a while. I think we talked, like, I don’t know, a couple months ago.
39 00:05:28.000 ⇒ 00:05:28.800 Emily Giant: Yeah.
40 00:05:28.800 ⇒ 00:05:29.359 Robert Tseng: Good to see you
41 00:05:29.990 ⇒ 00:05:35.059 Robert Tseng: Yeah, good to see you again. Yeah, yeah, I think I… I don’t know, I was able to…
42 00:05:35.280 ⇒ 00:05:40.380 Robert Tseng: kind of look through the… everything in the forecast folders, and also looking around at Tus.
43 00:05:40.410 ⇒ 00:05:53.460 Robert Tseng: I mean, I think at this point, I just, I have some hypotheses around, kind of, the forecasting. I think, consistently, it looks like we’re off… we’re off base by, like, 25 to 30%, and so…
44 00:05:53.460 ⇒ 00:06:09.860 Robert Tseng: I think there’s probably low-hanging fruit to be able to get it down, because, I’ve… I’ve had to… I’ve had to bring forecasts down to within 10% of accuracy before, so, yeah, I guess for me, it’s just, like, kind of poking around in… in SQL to figure out, like, what the…
45 00:06:09.960 ⇒ 00:06:12.869 Robert Tseng: what the models are actually built on, like, I…
46 00:06:13.490 ⇒ 00:06:18.949 Robert Tseng: And then, like, I can suggest, like, how we can make that better.
47 00:06:19.470 ⇒ 00:06:26.180 Robert Tseng: yeah, I guess, you know, you… I mean, we were just going to… I was just gonna go into Redshift, and
48 00:06:26.290 ⇒ 00:06:31.419 Robert Tseng: and poke around there. I don’t know if you have any other guidance on what else I should be looking at.
49 00:06:33.530 ⇒ 00:06:47.819 Emily Giant: Not really, because all the forecasting documents are ingested into Redshift, so they’re there. Okay. Can I… I’m wondering which, like, more specifically, like, what are you looking at? Like, I can point you in directions if I know
50 00:06:47.990 ⇒ 00:06:51.270 Emily Giant: The question that’s being asked of
51 00:06:51.470 ⇒ 00:06:56.690 Emily Giant: Forecasting, like, there’s so many different… Departments that do it.
52 00:06:58.220 ⇒ 00:07:12.190 Robert Tseng: Yeah, I mean, I guess the question to me, I’m still trying to find, like, what the right question is. I think I was starting from the, this is what we do have currently, and, I’m gonna kind of back into the questions from there. Yeah.
53 00:07:12.190 ⇒ 00:07:15.460 Emily Giant: definitely point you from there. Okay, so, there are…
54 00:07:16.090 ⇒ 00:07:24.480 Emily Giant: There are a couple key forecasting documents that I would point you to. One would be the,
55 00:07:25.170 ⇒ 00:07:30.269 Emily Giant: And this is just in terms of, like, where the forecast lives currently that we’re using, or, like…
56 00:07:31.080 ⇒ 00:07:48.739 Robert Tseng: Yeah, I guess, like, I started off with the annual revenue plan. I mean, that’s one model, right? It’s basically, like, from a budget perspective, like, what they forecasted out, so maybe there are other models that I haven’t seen, but this is, like, their component level category performance. That’s kind of where I started.
57 00:07:48.740 ⇒ 00:07:52.150 Robert Tseng: yeah, I guess if there are other ones that I should look at, I think that’d be helpful.
58 00:07:52.570 ⇒ 00:07:54.760 Emily Giant: The main one is called, like.
59 00:07:55.400 ⇒ 00:07:59.339 Emily Giant: Let me find it so that I’m not guessing what this is called.
60 00:07:59.560 ⇒ 00:08:00.270 Robert Tseng: Okay.
61 00:08:00.270 ⇒ 00:08:03.369 Emily Giant: I want to say it’s called the Consolidated Mart.
62 00:08:04.640 ⇒ 00:08:06.280 Emily Giant: Something.
63 00:08:06.280 ⇒ 00:08:07.070 Robert Tseng: Okay.
64 00:08:07.070 ⇒ 00:08:15.640 Emily Giant: Well, let me see, it’s in… the… Staging… hold on here.
65 00:08:20.010 ⇒ 00:08:21.710 Emily Giant: Forecast uploads…
66 00:08:26.030 ⇒ 00:08:27.940 Emily Giant: Forecast and planning feeds.
67 00:08:32.120 ⇒ 00:08:36.450 Emily Giant: So, Mart… Forecast and planning combined.
68 00:08:36.600 ⇒ 00:08:43.930 Emily Giant: is… what is powering all of the Looker information in terms of forecasts at the moment?
69 00:08:44.360 ⇒ 00:08:45.070 Robert Tseng: Okay.
70 00:08:45.830 ⇒ 00:08:47.730 Emily Giant: Let me copy-paste that.
71 00:08:48.430 ⇒ 00:08:50.529 Emily Giant: Phrase into the chat.
72 00:08:50.920 ⇒ 00:08:51.530 Robert Tseng: Yeah.
73 00:08:51.730 ⇒ 00:08:53.179 Robert Tseng: That’d probably be best.
74 00:08:55.140 ⇒ 00:08:58.800 Emily Giant: I do not like what dbt has done with these updates.
75 00:08:58.920 ⇒ 00:09:04.539 Emily Giant: it’s like they went to, like, a DOS-based system last night, or something when we updated this. It’s…
76 00:09:06.140 ⇒ 00:09:06.790 Emily Giant: Yeah.
77 00:09:06.790 ⇒ 00:09:07.809 Uttam Kumaran: Yeah, it doesn’t look good.
78 00:09:07.810 ⇒ 00:09:08.440 Emily Giant: No.
79 00:09:08.440 ⇒ 00:09:11.320 Uttam Kumaran: Continue to not be… spend too much time in there.
80 00:09:11.720 ⇒ 00:09:16.739 Emily Giant: You oughta Okay, here, Mart, forecast and planning, oop!
81 00:09:27.990 ⇒ 00:09:33.270 Emily Giant: So this combines the marketing and the revenue team’s forecast?
82 00:09:33.820 ⇒ 00:09:34.430 Robert Tseng: Okay.
83 00:09:35.410 ⇒ 00:09:39.490 Emily Giant: And then there’s the FY26 marketing forecast.
84 00:09:40.060 ⇒ 00:09:42.659 Emily Giant: Let me find that…
85 00:10:01.320 ⇒ 00:10:03.330 Awaish Kumar: Alright.
86 00:10:09.450 ⇒ 00:10:20.049 Emily Giant: I don’t know why this is not in the folder that I expected it to be in. Maybe marketing? Forecast uploads, okay. And then… staging, marketing, purchase forecast…
87 00:10:23.250 ⇒ 00:10:24.400 Awaish Kumar: Oh, yeah.
88 00:10:24.400 ⇒ 00:10:27.380 Emily Giant: Okay, and then there’s Marketing Purchase Forecast.
89 00:10:28.430 ⇒ 00:10:28.980 Robert Tseng: Okay.
90 00:10:31.870 ⇒ 00:10:33.500 Emily Giant: And that should be, like.
91 00:10:33.650 ⇒ 00:10:45.670 Emily Giant: joined to all of the new, like, North Beam data, et cetera, GA4, like, they do not have a meaningful tie in Looker right now to this document, because I don’t think they know
92 00:10:45.730 ⇒ 00:11:00.069 Emily Giant: what they want from it, but I think that, Robert, you will know what they want from it. Okay. This is just the document that they update with all of the, like, various attribution platforms for marketing, and how much money they’re forecasting to spend, and how much they think are
93 00:11:00.170 ⇒ 00:11:05.950 Emily Giant: is coming from each of those channels. So those are the two main forecasting documents that,
94 00:11:06.680 ⇒ 00:11:18.380 Emily Giant: That I would say are used the most. Now, the component level one, that is a good question. I might have to ask Dean Capel about that, or look through Perry’s notes. Like, our…
95 00:11:18.380 ⇒ 00:11:23.259 Robert Tseng: Yeah, I saw Perry’s name on there, yeah. Maybe he built… Perry, he’s gone.
96 00:11:23.260 ⇒ 00:11:23.819 Emily Giant: So that’s…
97 00:11:23.820 ⇒ 00:11:25.270 Awaish Kumar: Right, okay.
98 00:11:25.470 ⇒ 00:11:31.560 Emily Giant: But, okay, staging GA4 Ecom Marketing… Products, no.
99 00:11:33.100 ⇒ 00:11:35.099 Uttam Kumaran: Is she still consulting, Emily?
100 00:11:35.100 ⇒ 00:11:37.549 Emily Giant: She is, yeah, I can still ask her.
101 00:11:37.550 ⇒ 00:11:38.070 Uttam Kumaran: Okay.
102 00:11:38.070 ⇒ 00:11:48.849 Emily Giant: So there’s another thing with forecasting, which I don’t think we really will have time for in this engagement, but each holiday, there’s new forecasts, and they do, like, a whole new upload every time, and it’s really…
103 00:11:49.540 ⇒ 00:11:53.410 Emily Giant: Not the way that I would…
104 00:11:54.610 ⇒ 00:11:57.729 Emily Giant: Say to go about it, because it creates a lot of clutter.
105 00:11:57.970 ⇒ 00:12:10.460 Emily Giant: But, yeah, I think starting with those two documents in terms of forecasting is the most useful. And anything else, we can kind of model after what you… what you work on with those two.
106 00:12:10.730 ⇒ 00:12:14.609 Emily Giant: Because the rest are a little bit more ad hoc for, like, holiday plans.
107 00:12:15.230 ⇒ 00:12:16.530 Robert Tseng: Sure. Okay.
108 00:12:17.860 ⇒ 00:12:22.730 Uttam Kumaran: And then, are there any Google Sheets that, like, that drive those, or, like… Yeah. Okay.
109 00:12:22.770 ⇒ 00:12:24.300 Emily Giant: Yeah, let me…
110 00:12:25.130 ⇒ 00:12:28.040 Uttam Kumaran: I feel like I’ve looked at a couple of them, but… Yeah.
111 00:12:30.180 ⇒ 00:12:37.890 Emily Giant: they’re in Stitch, so you’ll see, like, what the table is. I’m looking in the YAML file at the moment, but…
112 00:12:38.250 ⇒ 00:12:43.889 Emily Giant: it might be easier to just go, okay, so it’s called FY25 Budget Purchase Feed.
113 00:12:45.420 ⇒ 00:12:46.550 Uttam Kumaran: Yeah, let me…
114 00:12:46.550 ⇒ 00:12:47.839 Emily Giant: Let me…
115 00:12:52.040 ⇒ 00:12:58.859 Emily Giant: So this is the… in Stitch, the schema… and table…
116 00:13:02.020 ⇒ 00:13:04.590 Emily Giant: For the marketing forecast.
117 00:13:06.850 ⇒ 00:13:10.709 Uttam Kumaran: So that’s FY25 budget purchase feed.
118 00:13:11.150 ⇒ 00:13:11.720 Emily Giant: So the one…
119 00:13:11.720 ⇒ 00:13:12.219 Uttam Kumaran: I mean, there’s…
120 00:13:12.220 ⇒ 00:13:17.289 Emily Giant: Yeah, the marketing purchase forecast, FY26.fy26 purchased…
121 00:13:17.770 ⇒ 00:13:28.480 Emily Giant: formatted for upload. And then for the, the combined… forecast…
122 00:13:29.470 ⇒ 00:13:31.270 Robert Tseng: Route budget purchase…
123 00:13:31.470 ⇒ 00:13:34.979 Emily Giant: raw planning. Those are two different documents that I…
124 00:13:35.370 ⇒ 00:13:38.310 Emily Giant: Created into, like, one final model, because it was…
125 00:13:38.620 ⇒ 00:13:41.189 Emily Giant: creating so much duplication in Looker.
126 00:13:41.960 ⇒ 00:13:44.489 Emily Giant: Let’s see where this comes from…
127 00:13:52.800 ⇒ 00:13:56.869 Emily Giant: Schema is Forecast and Planning FeedV2.feed.
128 00:14:01.210 ⇒ 00:14:05.770 Emily Giant: And that is the… Sorry.
129 00:14:10.800 ⇒ 00:14:12.940 Emily Giant: Budget Purchase Forecast.
130 00:14:24.870 ⇒ 00:14:28.260 Uttam Kumaran: Yeah, I don’t think I… I… I don’t think I have access to…
131 00:14:28.380 ⇒ 00:14:31.809 Uttam Kumaran: the actual sheets, but all the data is in Redshift.
132 00:14:32.040 ⇒ 00:14:32.900 Emily Giant: Yeah.
133 00:14:34.220 ⇒ 00:14:37.350 Uttam Kumaran: I don’t know, Robert, if you need, like, the actual sheets or anything.
134 00:14:38.020 ⇒ 00:14:43.290 Robert Tseng: Yeah, I mean, that would be ideal. I mean, I kind of get the sense of, like, I…
135 00:14:43.810 ⇒ 00:14:48.780 Robert Tseng: So… There are…
136 00:14:50.110 ⇒ 00:14:59.770 Robert Tseng: Yeah, I mean, to me, there are multiple models in here, which is better. I think that’s better than what I first saw. So, I mean, I guess, like, there’s…
137 00:15:00.420 ⇒ 00:15:12.839 Robert Tseng: You know, and this is pretty typical, like, you do have forecasts from different, from different… from different models, and then you’re basically triangulating to come to one, like, master, like, demand model, and then one, like.
138 00:15:12.850 ⇒ 00:15:23.430 Robert Tseng: like, I guess one demand forecast, and then one, like, I guess, operational supply… supply forecast that has, like, the materials planned, and all the raw material, whatever. I mean, I guess this is not…
139 00:15:23.510 ⇒ 00:15:34.339 Robert Tseng: exactly this typical manufacturing situation, so maybe that’s probably less… less important, but I guess it… I think it’s always good for me to see both sides of the equation.
140 00:15:34.510 ⇒ 00:15:43.450 Emily Giant: Okay, I can reach out to, supply chain and see if they have their own forecast. I don’t think that’s ever been ingested into Looker, but that’s definitely a good point that it probably exists.
141 00:15:44.290 ⇒ 00:15:47.069 Emily Giant: Let me reach out to Yvonne right now.
142 00:15:47.300 ⇒ 00:15:50.549 Uttam Kumaran: And I just, also, I just requested access to…
143 00:15:51.180 ⇒ 00:15:56.160 Uttam Kumaran: couple of those sheets with the Urban Stems and Brain Forge email, so…
144 00:15:56.160 ⇒ 00:15:59.820 Emily Giant: I’ll tell Perry, because I think she’s the owner of those.
145 00:15:59.820 ⇒ 00:16:02.790 Uttam Kumaran: She should… she should have gotten an email, yeah.
146 00:16:02.790 ⇒ 00:16:03.330 Emily Giant: Damn.
147 00:16:04.530 ⇒ 00:16:08.389 Uttam Kumaran: So that was, like, the marketing purchase forecast, the…
148 00:16:08.660 ⇒ 00:16:12.960 Uttam Kumaran: FY25 budget, and then I also requested the
149 00:16:13.200 ⇒ 00:16:18.929 Uttam Kumaran: the V-Day 2025, like, so some of the holiday-specific forecasts.
150 00:16:22.170 ⇒ 00:16:36.060 Robert Tseng: Yeah. I guess, like, at a high level, like, the goal of this is to identify, like, all the different methods that are there, figure out which ones are most trustworthy, and then we’re gonna build, like, a base case. It may just be, like, a combination of what you already have.
151 00:16:36.490 ⇒ 00:16:36.870 Emily Giant: Right.
152 00:16:36.870 ⇒ 00:16:48.370 Robert Tseng: And then you can start to set, like, the upper and lower bounds in case, like, things start to kind of go off track, so you can proactively triangulate what you need to to, I guess.
153 00:16:48.530 ⇒ 00:16:58.770 Robert Tseng: you know, I don’t know what the levers are, like, I don’t know how your purchasing orders and everything go, but that’s where I’ve seen this be helpful.
154 00:16:58.770 ⇒ 00:16:59.570 Emily Giant: Yeah, totally.
155 00:16:59.570 ⇒ 00:17:00.130 Robert Tseng: Yeah.
156 00:17:00.490 ⇒ 00:17:10.009 Robert Tseng: So, like, it would be something that, like, a COO or VP of Ops would be, like, they would probably be the main consumer of this, is typically what I’ve seen.
157 00:17:10.460 ⇒ 00:17:11.300 Emily Giant: Okay, yeah.
158 00:17:11.300 ⇒ 00:17:20.670 Robert Tseng: And then… and then obviously the marketing side, like, whoever, I guess, CMO or VP of Marketing would also care about, like, on the… on the spend side, like, if they…
159 00:17:21.119 ⇒ 00:17:30.999 Robert Tseng: Does what they’re forecasting to spend and, like, the ac… and does that actually track with, like, what we’re able to, you know, support?
160 00:17:31.160 ⇒ 00:17:43.620 Emily Giant: Yeah, I think that that’s a really important one for our team, because we do try to, like, rein in the marketing spend quite a bit, and, there has been, like, a lot of data silos around the marketing spend up until now.
161 00:17:44.010 ⇒ 00:17:54.280 Robert Tseng: I saw that it was around, like, $100K or slightly less per week, with some variation. Is that… is that… is that true across all marketing spend?
162 00:17:55.930 ⇒ 00:18:01.530 Emily Giant: I don’t know off the top of my head, but that doesn’t sound that incorrect. That sounds… Okay.
163 00:18:02.010 ⇒ 00:18:02.810 Emily Giant: Yeah.
164 00:18:04.300 ⇒ 00:18:08.440 Emily Giant: Given all our affiliate channels and stuff, I wouldn’t… I wouldn’t be surprised.
165 00:18:08.710 ⇒ 00:18:13.040 Emily Giant: Okay. But I can… that would be a good question for PK.
166 00:18:13.350 ⇒ 00:18:30.559 Robert Tseng: Okay, because at your size company, I would expect it to actually be, like, double. Like, typically, it’s like, I don’t know, like, 20 to 30% in our… in marketing spend, but I just feel like that’s 5… like, kind of 100K seems kind of low, from what I… just first glance.
167 00:18:30.900 ⇒ 00:18:38.909 Emily Giant: Yeah, I agree with you. I know that’s been, like, a huge initiative of ours, though, is to, like, rein in the marketing spend. Okay.
168 00:18:39.210 ⇒ 00:18:47.579 Emily Giant: I’m not… not an expert on marketing. That is, like, the department that’s, like, a little bit outside of my purview.
169 00:18:47.890 ⇒ 00:18:48.420 Robert Tseng: Okay.
170 00:18:48.780 ⇒ 00:18:59.539 Emily Giant: Yeah, I can… I can also ask PK, like, what is in the sheet and what we’re ingesting should be correct, so if that’s what you’re seeing, then it probably is true. Okay.
171 00:19:00.080 ⇒ 00:19:03.569 Emily Giant: Awish, do you know if that’s, like, part of Northbeam? Do they…
172 00:19:04.920 ⇒ 00:19:08.180 Emily Giant: Did they do any kind of, like, spend?
173 00:19:08.370 ⇒ 00:19:09.380 Emily Giant: Data?
174 00:19:13.200 ⇒ 00:19:13.940 Awaish Kumar: like…
175 00:19:15.130 ⇒ 00:19:17.410 Uttam Kumaran: It should be in the Northview data, right?
176 00:19:17.410 ⇒ 00:19:18.840 Emily Giant: That’s what I was thinking.
177 00:19:19.380 ⇒ 00:19:22.890 Awaish Kumar: In North Beam, we do have, like, the spend data.
178 00:19:24.370 ⇒ 00:19:28.550 Awaish Kumar: But, like, right now, I think we are just bringing in the touchpoints.
179 00:19:29.180 ⇒ 00:19:32.589 Emily Giant: Okay. I don’t know if we have worked on getting the span data from…
180 00:19:34.800 ⇒ 00:19:39.270 Emily Giant: Would that also be something that would be in GA4, or just only North Beam?
181 00:19:40.380 ⇒ 00:19:41.440 Awaish Kumar: Only Northam.
182 00:19:41.440 ⇒ 00:19:45.080 Emily Giant: Only North… okay. We probably do want to work on…
183 00:19:46.150 ⇒ 00:19:54.829 Emily Giant: Bringing that in so that we can do actuals versus forecasts, and that the team, like, right now they’re hard-coding it, and that opens up a lot of…
184 00:19:55.880 ⇒ 00:19:58.609 Emily Giant: Exclamation points in my brain.
185 00:20:02.150 ⇒ 00:20:02.740 Robert Tseng: Yeah.
186 00:20:04.350 ⇒ 00:20:08.540 Emily Giant: Is that something that we could do, Awash? Like, I think that that would be really helpful to have.
187 00:20:08.540 ⇒ 00:20:09.720 Awaish Kumar: No, we do not.
188 00:20:09.720 ⇒ 00:20:11.199 Emily Giant: Yeah, that’d be awesome.
189 00:20:16.650 ⇒ 00:20:26.209 Robert Tseng: Okay, cool. Well, I guess as much as I can get access to, I’ll poke around and definitely have something to present by Tuesday, I guess, when the next time we’re meeting on this, so…
190 00:20:26.400 ⇒ 00:20:27.140 Robert Tseng: Yeah.
191 00:20:28.080 ⇒ 00:20:33.659 Emily Giant: That sounds great. That’s good stuff. Alright, and then, Tom, if you don’t hear from Perry.
192 00:20:33.830 ⇒ 00:20:37.490 Emily Giant: Let me know, so I can give her, like, a little nudge.
193 00:20:37.940 ⇒ 00:20:38.610 Uttam Kumaran: Okay.
194 00:20:39.490 ⇒ 00:20:44.539 Uttam Kumaran: Perfect. And then, today, we’re just kind of cruising through more Looker stuff, so I…
195 00:20:44.650 ⇒ 00:20:49.960 Uttam Kumaran: couldn’t do much yesterday, but I think we’ll… More stuff later today, so…
196 00:20:50.160 ⇒ 00:20:53.799 Emily Giant: Okay, that sounds good, and then I’m just working on,
197 00:20:54.330 ⇒ 00:21:11.180 Emily Giant: revenue QA, I was asked by our, CFO to do an analysis of, like, product revenue, and I pulled the numbers from our historical mart, and they were just really low for the, area of time, so hopefully gonna, like, use.
198 00:21:11.180 ⇒ 00:21:15.459 Uttam Kumaran: Demolati’s new deployments from this morning to… Right.
199 00:21:15.640 ⇒ 00:21:32.000 Emily Giant: look instead of the historical, the isolated historical model, but that model does feed the end table, so might have to do some debugging there, just seeing what’s going on with that. Like, I would expect it to be, like, triple what.
200 00:21:32.000 ⇒ 00:21:32.720 Uttam Kumaran: Okay.
201 00:21:32.720 ⇒ 00:21:34.029 Emily Giant: Pulled up in the report.
202 00:21:34.630 ⇒ 00:21:35.240 Uttam Kumaran: Okay.
203 00:21:36.450 ⇒ 00:21:39.080 Uttam Kumaran: Yeah, just Slack me, I’m happy to also look into it.
204 00:21:39.240 ⇒ 00:21:40.240 Emily Giant: Okay, cool.
205 00:21:41.280 ⇒ 00:21:41.890 Uttam Kumaran: Okay.
206 00:21:42.620 ⇒ 00:21:50.079 Uttam Kumaran: Yeah, so I think we should… we also have the North Beam sort of copy command in, so I guess, Awash, maybe if you want to send an update
207 00:21:50.260 ⇒ 00:21:56.469 Uttam Kumaran: Or… update Kristen, whoever, I guess I wasn’t following on, like, what’s next steps after that.
208 00:21:59.110 ⇒ 00:22:02.260 Awaish Kumar: Like, for the North Beam, we have now data…
209 00:22:02.560 ⇒ 00:22:08.649 Awaish Kumar: coming in on a schedule, and it is modeled out. We have fields in the fat orders.
210 00:22:08.890 ⇒ 00:22:11.300 Awaish Kumar: So I can share all of that.
211 00:22:12.770 ⇒ 00:22:13.350 Awaish Kumar: Hello?
212 00:22:13.350 ⇒ 00:22:13.700 Uttam Kumaran: Perfect.
213 00:22:13.700 ⇒ 00:22:14.450 Awaish Kumar: Selection.
214 00:22:15.830 ⇒ 00:22:19.730 Uttam Kumaran: Yeah, maybe we can… yeah, I would… yeah, if you want to share that in Slack, and then…
215 00:22:20.480 ⇒ 00:22:38.260 Uttam Kumaran: if it’s easiest, I would also just, like, commit a README file to, or commit a… commit some type of docs file to the repo. I was like, maybe we should put a Notion or Confluence, but it’s gonna get lost, so maybe if you just create a short, like, document in the repo.
216 00:22:38.450 ⇒ 00:22:41.140 Uttam Kumaran: Next to where the North Beach source is, or whatever.
217 00:22:41.330 ⇒ 00:22:43.110 Uttam Kumaran: That way, it’s there forever.
218 00:22:44.960 ⇒ 00:22:46.029 Awaish Kumar: Okay, sure.
219 00:22:46.450 ⇒ 00:22:50.250 Uttam Kumaran: And then, yeah, I would just let PK or whoever know in the channel, or Kristen, yeah, okay.
220 00:22:51.070 ⇒ 00:22:53.940 Emily Giant: Awash, my, staging run.
221 00:22:54.170 ⇒ 00:23:00.130 Emily Giant: It timed out again, but it did get through, OMS orders, so I…
222 00:23:00.130 ⇒ 00:23:00.810 Awaish Kumar: Yes.
223 00:23:01.100 ⇒ 00:23:05.540 Emily Giant: I think I only need to run… Suborders.
224 00:23:05.540 ⇒ 00:23:06.710 Awaish Kumar: Yeah, well…
225 00:23:07.120 ⇒ 00:23:13.379 Awaish Kumar: I tried that, but I don’t know, my job stuck for, like, 20 minutes in queue, 14 minutes.
226 00:23:15.520 ⇒ 00:23:17.110 Awaish Kumar: I didn’t execute it.
227 00:23:17.270 ⇒ 00:23:27.270 Emily Giant: Boo! Any… any ideas, Utam, on how we can… I… I feel like I’m being a numb-numb.
228 00:23:27.780 ⇒ 00:23:36.660 Emily Giant: With this, like, staging thing, because I don’t know why I thought, like, unlike production, staging will just, like, magically update to our instance without full refreshes.
229 00:23:37.120 ⇒ 00:23:39.339 Emily Giant: Well, like, I guess, like…
230 00:23:39.340 ⇒ 00:23:44.609 Uttam Kumaran: I guess, Awash, like, are we running full refreshes in state? Are we, like…
231 00:23:44.770 ⇒ 00:23:49.319 Uttam Kumaran: Copying over data from prod to staging every day, or…
232 00:23:49.610 ⇒ 00:23:51.170 Uttam Kumaran: Should we be doing that?
233 00:23:52.590 ⇒ 00:23:53.440 Awaish Kumar: -Oh.
234 00:23:53.870 ⇒ 00:23:59.209 Awaish Kumar: Like, the only thing that full refreshes are running only because we changed the schema.
235 00:23:59.550 ⇒ 00:24:01.810 Awaish Kumar: Like, we can’t do anything about it.
236 00:24:01.810 ⇒ 00:24:02.410 Uttam Kumaran: Yeah, yeah, yeah.
237 00:24:02.410 ⇒ 00:24:09.479 Awaish Kumar: If we try the schemas, like, there is… we have to run it in production as well, where, like.
238 00:24:09.680 ⇒ 00:24:12.629 Awaish Kumar: We don’t have any staging, we don’t have it in production.
239 00:24:12.880 ⇒ 00:24:15.929 Awaish Kumar: Okay. So, from where we are going to bring it from?
240 00:24:16.380 ⇒ 00:24:19.709 Uttam Kumaran: Yeah, but I guess, like, what’s the, like… yeah, go ahead.
241 00:24:20.020 ⇒ 00:24:24.519 Demilade Agboola: I’m gonna say, could we do a limit? So instead of having the full, like.
242 00:24:24.870 ⇒ 00:24:29.599 Demilade Agboola: Aging, can we limit it to, like, maybe 1,000 rows per whatever time period?
243 00:24:30.080 ⇒ 00:24:37.390 Uttam Kumaran: Yeah, I think the problem is here is, like, the schema is actually changing, like, there’s new columns and stuff, so it will trigger a full refresh.
244 00:24:38.030 ⇒ 00:24:42.670 Uttam Kumaran: I mean, in the past, like.
245 00:24:43.340 ⇒ 00:24:51.719 Uttam Kumaran: We’ve just done these, like, off hours, basically, where you just… you run it in prod, you let it run the whole way through, and then you copy that back to staging.
246 00:24:52.750 ⇒ 00:25:03.269 Emily Giant: Yeah, I think just at least for the next two weeks, we need to do that. It doesn’t need to be a forever thing, but the schemas are gonna change in the next few weeks. Like, the, the two…
247 00:25:03.310 ⇒ 00:25:19.960 Emily Giant: PRs that… well, one is now… I already ran it in production, and it was successful, so we should be good there, but, like, the staging orders, staging OMS sub-orders, like, that’s gonna be a big change, because we’re gonna, like, unplug everything from the deprecated model. Yeah. But after that, it’s not gonna happen as much. It’s just…
248 00:25:19.960 ⇒ 00:25:32.780 Uttam Kumaran: Yeah, so I wish I’d rather us do, I’d rather us have a script that copies over production, and then on… if there are schema changes, then there’s, like, nothing we can really do, like…
249 00:25:33.440 ⇒ 00:25:34.280 Uttam Kumaran: we’re not doing.
250 00:25:34.280 ⇒ 00:25:34.680 Awaish Kumar: Yeah, he’s.
251 00:25:34.680 ⇒ 00:25:36.879 Uttam Kumaran: of advanced promotion process.
252 00:25:37.000 ⇒ 00:25:44.119 Uttam Kumaran: Like, typically, in this situation, in the past, we’ve done, like, we’ve had a pre-prod, and you can run it there, and then do a swap, like.
253 00:25:44.290 ⇒ 00:25:49.760 Uttam Kumaran: it’s just too complicated, I feel like, to maintain, so… Yeah.
254 00:25:53.850 ⇒ 00:25:59.239 Awaish Kumar: Okay, are you saying that, like, if we have any staging, then we copy over to production?
255 00:25:59.800 ⇒ 00:26:07.149 Uttam Kumaran: No, I’m saying, like, Like, in this situation, like, we don’t have a pre-prod environment, right?
256 00:26:07.260 ⇒ 00:26:10.819 Uttam Kumaran: So, we can’t… we can’t, like, run this in a product… like.
257 00:26:11.030 ⇒ 00:26:22.989 Uttam Kumaran: the point of these environments is that you run this in, like, as close to production as possible. In this situation, in the past, like, when I dealt with this, we created a pre-prod environment, so what we would do is, like, you’d run it in staging.
258 00:26:23.140 ⇒ 00:26:33.969 Uttam Kumaran: it would run in pre-prod, and then we would… we would then do a swap. But in this situation, I don’t… I feel like that’s, like, too much to do, so…
259 00:26:34.190 ⇒ 00:26:37.370 Uttam Kumaran: I guess what I would say is, like, we should…
260 00:26:38.010 ⇒ 00:26:42.549 Uttam Kumaran: if the… we should… we have to merge the PR, run it in production.
261 00:26:42.690 ⇒ 00:26:54.379 Uttam Kumaran: like, run the full refresh, and then copy it back over to staging, and then that’s… on schema change, like, that’s still… that’s gonna be the process. I don’t see how there’s an… there’s not really a clear other process.
262 00:26:54.560 ⇒ 00:26:56.280 Uttam Kumaran: Unless you wanna, yeah.
263 00:26:56.970 ⇒ 00:27:02.619 Awaish Kumar: For production, that is also going to take the similar amount of time to get executed on production.
264 00:27:03.050 ⇒ 00:27:06.099 Awaish Kumar: To get all these tables, fully refreshed.
265 00:27:06.370 ⇒ 00:27:08.699 Awaish Kumar: Otherwise, it’s going to fail there as well.
266 00:27:15.550 ⇒ 00:27:17.820 Emily Giant: I… Dink!
267 00:27:18.470 ⇒ 00:27:21.910 Emily Giant: I… this is a real I think. But, the…
268 00:27:22.170 ⇒ 00:27:35.680 Emily Giant: PR with the staging orders and suborders, that’s gonna be the biggest change that there is, because it’s going from, like, hundreds of columns to, like, the change in those is, like, every time we’ve changed a system.
269 00:27:36.030 ⇒ 00:27:37.320 Emily Giant: Instead of, like.
270 00:27:37.550 ⇒ 00:27:56.919 Emily Giant: like, concatenate… not concaten… instead of coalescing the columns, it just creates a whole new set of columns. So those… the changes in that PR are just taking every iteration of the same column and coalescing them so that that table is, like, way smaller. That’s gonna be the biggest schema change that there is.
271 00:27:57.020 ⇒ 00:27:59.590 Emily Giant: In the next 2 weeks, I’m guessing.
272 00:27:59.590 ⇒ 00:28:07.610 Uttam Kumaran: I guess, like, my only ask is, like, if we push this to production and run it, like, are we gonna get a bunch of errors for the rest of the day that, like.
273 00:28:07.610 ⇒ 00:28:08.850 Emily Giant: We may not.
274 00:28:08.990 ⇒ 00:28:11.539 Awaish Kumar: So then I…
275 00:28:13.180 ⇒ 00:28:14.020 Uttam Kumaran: Why?
276 00:28:15.910 ⇒ 00:28:20.720 Awaish Kumar: Because these tables, like in production, they don’t have the updated schema.
277 00:28:20.870 ⇒ 00:28:23.040 Awaish Kumar: Either… So…
278 00:28:23.370 ⇒ 00:28:30.520 Awaish Kumar: There are… like, if we added 10 more columns, they were not in production, so now it needs to run
279 00:28:31.070 ⇒ 00:28:37.190 Awaish Kumar: Like, if it runs in an incremental mode, it is going to fail, because it will say these columns does not exist.
280 00:28:37.370 ⇒ 00:28:46.020 Emily Giant: Until there’s a full refresh. So what we would have to do, if we ran it during the day, is, like, turn off the other jobs for an hour, run this, then turn the jobs back on.
281 00:28:46.020 ⇒ 00:28:49.809 Uttam Kumaran: So I guess my only… that’s my only suggestion, is, like, can we just do this at the end of the day?
282 00:28:49.810 ⇒ 00:28:50.629 Emily Giant: Yeah, that’s fine.
283 00:28:50.630 ⇒ 00:28:51.040 Uttam Kumaran: Okay.
284 00:28:51.040 ⇒ 00:28:52.620 Emily Giant: It’s gonna be hard for me.
285 00:28:52.620 ⇒ 00:28:53.539 Awaish Kumar: So what I’m…
286 00:28:54.370 ⇒ 00:28:55.310 Awaish Kumar: And what I’m thinking.
287 00:28:55.310 ⇒ 00:28:55.950 Emily Giant: get around it, it’s.
288 00:28:55.950 ⇒ 00:29:04.079 Awaish Kumar: Yeah, I don’t know if it works, but what I’m thinking of is, now that we have already executed some of it in staging.
289 00:29:04.420 ⇒ 00:29:09.010 Awaish Kumar: I can go in and copy these tables to production.
290 00:29:09.010 ⇒ 00:29:12.830 Uttam Kumaran: Oh, and then so the production doesn’t have to go full refresh?
291 00:29:12.830 ⇒ 00:29:13.180 Awaish Kumar: Yeah.
292 00:29:13.180 ⇒ 00:29:14.860 Emily Giant: Oh, okay.
293 00:29:14.860 ⇒ 00:29:15.320 Demilade Agboola: And…
294 00:29:15.320 ⇒ 00:29:16.380 Emily Giant: That’s pretty cool.
295 00:29:17.130 ⇒ 00:29:22.010 Uttam Kumaran: Yeah, so that’s, like, that was basically identical to, like, yeah, the promotion. I mean, yeah.
296 00:29:22.240 ⇒ 00:29:31.629 Uttam Kumaran: I would say… the only thing is, like, I’m busy a little bit. I just don’t want things to fail, and then we have to hop on, so I’ll wish I can leave that call to you.
297 00:29:32.400 ⇒ 00:29:38.330 Uttam Kumaran: If you want to do that, keep a backup, and then revert. If things go haywire, then do that.
298 00:29:38.580 ⇒ 00:29:45.750 Emily Giant: Y’all, it’s fine. Let’s just do it at the end of the day. Like, if you want to spend time on it, like, because I can just create a new branch with those changes, like.
299 00:29:45.750 ⇒ 00:29:46.280 Uttam Kumaran: Okay.
300 00:29:46.280 ⇒ 00:30:02.200 Emily Giant: I can merge it with what I just deployed, because I knew that those wouldn’t fail the jobs, or they should not fail the jobs, because the schema didn’t change. It was just, like, adding staging tables so that there weren’t so many transformations in the intermediate, and
301 00:30:02.420 ⇒ 00:30:07.679 Emily Giant: The incremental models that did have a schema change were brand new, so that won’t matter.
302 00:30:09.060 ⇒ 00:30:13.179 Demilade Agboola: So as long as those jobs… I don’t need to do it till the end of the day, it’s really okay.
303 00:30:13.540 ⇒ 00:30:14.440 Uttam Kumaran: Okay, okay.
304 00:30:15.460 ⇒ 00:30:18.019 Emily Giant: I shouldn’t have done it last night. I…
305 00:30:18.020 ⇒ 00:30:25.789 Uttam Kumaran: No, this is just the… this is just, like, the pain of this. I just want to move these type of migration activities to off hours, because it’ll just cause a bunch of issues, so…
306 00:30:25.790 ⇒ 00:30:26.590 Emily Giant: Yeah.
307 00:30:26.590 ⇒ 00:30:33.070 Uttam Kumaran: And then… okay. So yeah, I would… let’s… maybe I’ll just put a… I can put a placeholder meeting for later.
308 00:30:33.470 ⇒ 00:30:45.430 Emily Giant: That sounds good. And then, Awash, like, if you could write up any amount of documentation on, like, how that works, like, the staging runs and production, I think I’m just personally still confused, because, like.
309 00:30:45.490 ⇒ 00:31:00.880 Emily Giant: jobs were my… my biggest blank spot coming into this whole engagement. Like, I know how to write SQL, but, like, the jobs were, like, well. So I would love to know how it is that, like, you can copy that run over to production, because that seems, like, very useful.
310 00:31:03.010 ⇒ 00:31:03.760 Awaish Kumar: Okay.
311 00:31:06.250 ⇒ 00:31:06.870 Uttam Kumaran: Perfect.
312 00:31:07.390 ⇒ 00:31:08.200 Uttam Kumaran: Okay.
313 00:31:08.590 ⇒ 00:31:09.460 Uttam Kumaran: Alright.
314 00:31:09.600 ⇒ 00:31:10.770 Uttam Kumaran: Thanks, everyone!
315 00:31:10.770 ⇒ 00:31:11.670 Emily Giant: Thanks, bye.
316 00:31:11.670 ⇒ 00:31:12.730 Uttam Kumaran: Talk to you soon. Bye.