Meeting Title: Magic Spoon Data Pipeline Sync Date: 2026-01-29 Meeting participants: Uttam Kumaran, Ashwini Sharma
WEBVTT
1 00:00:12.820 ⇒ 00:00:13.670 Ashwini Sharma: Hey, Utam.
2 00:00:13.870 ⇒ 00:00:15.470 Uttam Kumaran: Hey, how are you?
3 00:00:15.740 ⇒ 00:00:19.930 Ashwini Sharma: I’m good. Let me show what’s the current status, right?
4 00:00:20.040 ⇒ 00:00:26.370 Ashwini Sharma: So yesterday, they sent out an email, with the new filter, right?
5 00:00:26.370 ⇒ 00:00:26.710 Uttam Kumaran: Okay.
6 00:00:27.100 ⇒ 00:00:27.920 Ashwini Sharma: Pretty good.
7 00:00:29.170 ⇒ 00:00:30.250 Ashwini Sharma: Exertude.
8 00:00:30.600 ⇒ 00:00:31.190 Uttam Kumaran: Great.
9 00:00:32.150 ⇒ 00:00:37.200 Ashwini Sharma: Yeah. Yes. Right, they have added additional geographies, right?
10 00:00:37.200 ⇒ 00:00:37.760 Uttam Kumaran: Okay.
11 00:00:38.280 ⇒ 00:00:43.079 Ashwini Sharma: Okay, so which means that I’ll have to reload, rerun the pipeline with these geographies in place.
12 00:00:43.550 ⇒ 00:00:47.540 Ashwini Sharma: Right? This is 11428 geographies.
13 00:00:47.540 ⇒ 00:00:47.910 Uttam Kumaran: Okay.
14 00:00:47.910 ⇒ 00:00:51.660 Ashwini Sharma: Whereas I have, only 254 over here.
15 00:00:52.030 ⇒ 00:00:52.770 Ashwini Sharma: Right?
16 00:00:52.900 ⇒ 00:01:00.219 Ashwini Sharma: Now, second thing is, yesterday when I did a reload, right, I saw something
17 00:01:00.690 ⇒ 00:01:03.380 Ashwini Sharma: Weird. Wait, let me show that to you.
18 00:01:04.610 ⇒ 00:01:10.030 Ashwini Sharma: Yeah. So if you see this, this is for Oklahoma, Oklahoma City, okay, for.
19 00:01:10.030 ⇒ 00:01:10.550 Uttam Kumaran: Yeah.
20 00:01:11.490 ⇒ 00:01:13.810 Ashwini Sharma: You see here, right?
21 00:01:14.850 ⇒ 00:01:17.000 Ashwini Sharma: These three are blank, right?
22 00:01:18.090 ⇒ 00:01:22.679 Ashwini Sharma: So this is… this is… this is the same API call, okay? Brand.
23 00:01:23.460 ⇒ 00:01:27.359 Ashwini Sharma: it’s for brand, so it means, like, it’s for the same API call. This…
24 00:01:27.630 ⇒ 00:01:29.840 Ashwini Sharma: And these are two different API calls, okay?
25 00:01:30.260 ⇒ 00:01:33.310 Ashwini Sharma: as a brand level, right? At reporting level, so…
26 00:01:33.510 ⇒ 00:01:41.399 Ashwini Sharma: And if you see here, right, for other items, we are getting spinstaller, but for these 3 items, we didn’t get it.
27 00:01:41.890 ⇒ 00:01:42.470 Uttam Kumaran: Yeah.
28 00:01:42.700 ⇒ 00:01:49.730 Ashwini Sharma: Now, these are part of the same call, right? So I don’t see any issue with missing filters or anything like that over here.
29 00:01:49.960 ⇒ 00:01:50.520 Uttam Kumaran: Okay.
30 00:01:50.760 ⇒ 00:01:57.459 Ashwini Sharma: Somehow, the subcategory, this particular subcategory, it didn’t written. The category is there, for example, the same category.
31 00:01:57.460 ⇒ 00:01:58.320 Uttam Kumaran: Yeah, yeah.
32 00:01:58.770 ⇒ 00:02:10.109 Ashwini Sharma: For one, it is there, but for somehow this subcategory, there is no data in spins. Okay. Similarly, like, if I see here, there is a bunch of such geographies where we don’t have any data, right? For example.
33 00:02:10.220 ⇒ 00:02:12.889 Ashwini Sharma: Dallas, Fort Worth, Texas, something.
34 00:02:12.890 ⇒ 00:02:13.710 Uttam Kumaran: API.
35 00:02:14.000 ⇒ 00:02:20.039 Ashwini Sharma: No data, right? Yeah, no data, again. These have data. These have a mixture of data. Yeah.
36 00:02:20.220 ⇒ 00:02:27.330 Ashwini Sharma: So, just the one that I showed, example, for this one, Oklahoma. Same case for this one, but for others, there is a
37 00:02:27.670 ⇒ 00:02:32.249 Ashwini Sharma: missing data. So if I just look into these things, right,
38 00:02:36.170 ⇒ 00:02:37.400 Ashwini Sharma: Installer.
39 00:02:46.460 ⇒ 00:02:49.040 Ashwini Sharma: So, these many records don’t have.
40 00:02:49.400 ⇒ 00:02:50.600 Ashwini Sharma: Data in them.
41 00:02:50.850 ⇒ 00:02:54.609 Ashwini Sharma: Almost. Okay, it’s not showing the count, but…
42 00:02:57.700 ⇒ 00:02:58.530 Ashwini Sharma: Wow.
43 00:03:01.130 ⇒ 00:03:02.470 Ashwini Sharma: How many are there?
44 00:03:04.090 ⇒ 00:03:05.140 Ashwini Sharma: 44.
45 00:03:06.630 ⇒ 00:03:08.650 Ashwini Sharma: 44 don’t have any data in them.
46 00:03:09.040 ⇒ 00:03:15.220 Ashwini Sharma: And the total amount for these… What, 118,000.
47 00:03:19.570 ⇒ 00:03:26.609 Ashwini Sharma: So the thing is, like, what I’ll do now is I’m going to rerun the pipeline with the additional geographies that has been requested.
48 00:03:26.850 ⇒ 00:03:27.300 Uttam Kumaran: Okay.
49 00:03:27.300 ⇒ 00:03:33.550 Ashwini Sharma: Good, sorry, where it is, right? This one.
50 00:03:34.320 ⇒ 00:03:38.089 Ashwini Sharma: With all these geographies, and limited to
51 00:03:39.070 ⇒ 00:03:43.500 Ashwini Sharma: Limited to these… oh, yeah, these product levels are already there.
52 00:03:43.930 ⇒ 00:03:51.499 Ashwini Sharma: Geography level, I’m going to put everything, because this is the whole list of geographies, right? There is not any more geographies apart from these.
53 00:03:51.710 ⇒ 00:03:54.599 Ashwini Sharma: And then category will be limited to these.
54 00:03:54.600 ⇒ 00:03:57.950 Uttam Kumaran: Okay. Product universe only these, and brand will be these.
55 00:03:58.270 ⇒ 00:04:00.700 Ashwini Sharma: Magic Spoon. I’m going to rerun the pipeline.
56 00:04:00.990 ⇒ 00:04:06.440 Ashwini Sharma: But then again, their latest QA data is also not available, so I don’t know, like, what to compare.
57 00:04:06.440 ⇒ 00:04:09.209 Uttam Kumaran: Fine, yeah, let’s… let’s just go ahead and do our side.
58 00:04:09.640 ⇒ 00:04:10.340 Ashwini Sharma: Okay.
59 00:04:10.510 ⇒ 00:04:16.500 Uttam Kumaran: And then, so, I guess how… like, what’s… how long do you think it’s gonna take? And I’ll send an… I’ll send an update this morning.
60 00:04:17.589 ⇒ 00:04:23.989 Ashwini Sharma: So, to me, I mean, for me to run this pipeline, it’s going to take about 1 hour if I start it now.
61 00:04:23.990 ⇒ 00:04:24.650 Uttam Kumaran: Okay.
62 00:04:26.130 ⇒ 00:04:31.350 Uttam Kumaran: Okay. And then, how many more geographies is it than… Last time.
63 00:04:31.350 ⇒ 00:04:38.820 Ashwini Sharma: This is… About… a lot more, right? Last time, I had 254 geographies.
64 00:04:39.100 ⇒ 00:04:40.350 Ashwini Sharma: In the filter.
65 00:04:40.350 ⇒ 00:04:40.670 Uttam Kumaran: Okay.
66 00:04:40.670 ⇒ 00:04:47.329 Ashwini Sharma: Right now, they have, yeah, 1428 journal fees.
67 00:04:47.330 ⇒ 00:04:49.160 Uttam Kumaran: Okay, so I’ll say 1200 more.
68 00:04:49.370 ⇒ 00:04:50.530 Ashwini Sharma: 1200 more, yeah.
69 00:04:50.920 ⇒ 00:04:52.290 Uttam Kumaran: geographies…
70 00:04:58.210 ⇒ 00:05:01.530 Uttam Kumaran: And then that file Michael sent didn’t come into Redshift, right?
71 00:05:02.190 ⇒ 00:05:05.170 Ashwini Sharma: That, no, it hasn’t come to that shift.
72 00:05:06.190 ⇒ 00:05:06.730 Uttam Kumaran: Okay.
73 00:05:10.120 ⇒ 00:05:16.230 Ashwini Sharma: Probably it failed because it has been quite some time. Yesterday, it did it about 12 hours ago, right?
74 00:05:17.430 ⇒ 00:05:17.880 Uttam Kumaran: Yeah.
75 00:05:17.880 ⇒ 00:05:25.570 Ashwini Sharma: it has not come by this time. Last time, also, that QA did data, the way he seeded, it didn’t come, so I had to reseed it again from a different…
76 00:05:26.130 ⇒ 00:05:27.370 Ashwini Sharma: Okay. Oh, please. Okay.
77 00:05:27.690 ⇒ 00:05:35.740 Uttam Kumaran: Okay, so I think on our side… If we can just produce… the… Yeah, the report…
78 00:05:36.120 ⇒ 00:05:38.060 Uttam Kumaran: Kind of the aggregations.
79 00:05:38.550 ⇒ 00:05:39.430 Uttam Kumaran: That they want.
80 00:05:39.430 ⇒ 00:05:40.210 Ashwini Sharma: Yeah.
81 00:05:40.210 ⇒ 00:05:40.590 Uttam Kumaran: Yeah.
82 00:05:40.590 ⇒ 00:05:41.150 Ashwini Sharma: Okay.
83 00:05:41.150 ⇒ 00:05:48.979 Uttam Kumaran: That’s great. I guess my other question is, your… that you produced a new, like, the new data sheet yesterday.
84 00:05:49.280 ⇒ 00:05:51.980 Uttam Kumaran: With the category, geography pivots.
85 00:05:52.370 ⇒ 00:05:52.770 Ashwini Sharma: Hmm.
86 00:05:52.770 ⇒ 00:05:54.680 Uttam Kumaran: Like, so I guess, like.
87 00:05:54.820 ⇒ 00:05:59.290 Uttam Kumaran: Should we… should we ditch that? Or, like, what do you think we should use… should we use that at all?
88 00:06:00.620 ⇒ 00:06:06.250 Ashwini Sharma: They wanted it in this format, right? That’s what, Demilade and I discussed, and then…
89 00:06:06.930 ⇒ 00:06:09.470 Uttam Kumaran: But I guess my question is, like, did we,
90 00:06:10.700 ⇒ 00:06:15.480 Uttam Kumaran: Do we want to wait for the new spins data to then put together the category pivot?
91 00:06:16.450 ⇒ 00:06:22.399 Ashwini Sharma: Yeah, I’ll get the new Sprint data and create this one, because this is… this won’t take a lot of time doing.
92 00:06:22.400 ⇒ 00:06:25.560 Uttam Kumaran: So this, so this data right now, this is from yesterday.
93 00:06:26.200 ⇒ 00:06:27.100 Ashwini Sharma: This is from.
94 00:06:27.100 ⇒ 00:06:29.059 Uttam Kumaran: Is this a rerun, or is this the same…
95 00:06:30.080 ⇒ 00:06:32.340 Ashwini Sharma: No, this was a rerun, this was a rerun.
96 00:06:32.740 ⇒ 00:06:38.190 Uttam Kumaran: Okay, so what I’m gonna do is I’m just gonna put this in, and I’m gonna say that we went ahead and made the pivots.
97 00:06:38.320 ⇒ 00:06:46.250 Uttam Kumaran: with… the width… Like, the data we have right now, We’re gonna do this again?
98 00:06:46.460 ⇒ 00:06:48.670 Uttam Kumaran: For the new filters.
99 00:06:48.670 ⇒ 00:06:50.670 Ashwini Sharma: Geographies, new geographies, yeah.
100 00:06:51.050 ⇒ 00:06:52.720 Ashwini Sharma: Okay, okay.
101 00:06:52.720 ⇒ 00:06:55.020 Uttam Kumaran: Okay, alright. Let me do that. And then…
102 00:06:55.020 ⇒ 00:06:55.360 Ashwini Sharma: So.
103 00:06:55.360 ⇒ 00:06:57.210 Uttam Kumaran: Yeah, I would say… yeah, go ahead, go ahead.
104 00:06:57.210 ⇒ 00:07:05.920 Ashwini Sharma: Yeah. This is not formatted like we had done for the previous sheet. That’s right, where we added a dollar difference and all those things, right? So that, that wasn’t done.
105 00:07:06.940 ⇒ 00:07:08.679 Uttam Kumaran: That’s fine, I can take care of that, yeah.
106 00:07:09.470 ⇒ 00:07:11.780 Ashwini Sharma: Yeah, and then this… this is…
107 00:07:12.060 ⇒ 00:07:17.500 Ashwini Sharma: Do you want me to add a filter or something like this? No, no, no, no, I think, I think leave this, leave this, I can take this.
108 00:07:17.700 ⇒ 00:07:18.580 Ashwini Sharma: Okay, alright.
109 00:07:18.580 ⇒ 00:07:22.969 Uttam Kumaran: Yeah, and then I think, yeah, if we can… if you just… just ping me whenever, like.
110 00:07:23.540 ⇒ 00:07:31.040 Uttam Kumaran: yeah, like, the new data is there, and then, yeah, I think I want to just try to make some good progress on CTA for the rest of the week, I feel like.
111 00:07:31.610 ⇒ 00:07:32.230 Ashwini Sharma: Yeah, yeah.
112 00:07:32.230 ⇒ 00:07:38.270 Uttam Kumaran: Well, the waste will also help there, so in case you guys can get as much data loaded, then I can help start modeling.
113 00:07:38.790 ⇒ 00:07:44.509 Ashwini Sharma: Yeah, there is only one folder to be loaded. That’s, like, 4 tables. I think I should be able to do that.
114 00:07:46.010 ⇒ 00:07:48.339 Ashwini Sharma: Then maybe the next one today.
115 00:07:48.520 ⇒ 00:07:51.530 Uttam Kumaran: Okay, then maybe the next thing is to just… then we can…
116 00:07:51.800 ⇒ 00:07:56.069 Uttam Kumaran: Start modeling, like, that… those first, kind of, like, members tables.
117 00:07:56.490 ⇒ 00:07:57.180 Ashwini Sharma: Sure.
118 00:07:57.310 ⇒ 00:08:01.840 Ashwini Sharma: Okay. So, okay, reiterate, I’m not going to touch this file, right?
119 00:08:01.840 ⇒ 00:08:02.320 Uttam Kumaran: Okay.
120 00:08:02.320 ⇒ 00:08:04.050 Ashwini Sharma: I’m going to rerun the pipeline with…
121 00:08:04.360 ⇒ 00:08:16.540 Ashwini Sharma: new, new geographies. Okay, and once it is complete, I’m going to create this sheet again. But this sheet won’t have a comparison with the PF dollars sheet. We don’t have the PF dollars amount right now.
122 00:08:16.770 ⇒ 00:08:24.430 Uttam Kumaran: Yeah, what I’ll actually do is let’s try to… let’s create everything in… the… QA sheet?
123 00:08:25.150 ⇒ 00:08:29.150 Uttam Kumaran: I will create for you the sheets to put this in.
124 00:08:29.560 ⇒ 00:08:34.599 Uttam Kumaran: And you could just work directly in there, that way we don’t get disorganized on, like, multiple sheets.
125 00:08:34.950 ⇒ 00:08:36.270 Ashwini Sharma: Okay, alright.
126 00:08:36.270 ⇒ 00:08:38.489 Uttam Kumaran: Okay. Okay. Thank you, sir.
127 00:08:38.490 ⇒ 00:08:39.900 Ashwini Sharma: Okay, yeah, thanks.
128 00:08:39.909 ⇒ 00:08:40.529 Uttam Kumaran: Alright, talk to you soon.
129 00:08:40.590 ⇒ 00:08:41.340 Ashwini Sharma: Meh.