Meeting Title: Javy-Data-Engineering-Weekly Date: 2024-09-24 Meeting participants: Nicolas Sucari, Aman Nagpal, Brian Pei, Payas Parab
WEBVTT
1 00:00:42.450 ⇒ 00:00:43.090 Nicolas Sucari: My end.
2 00:00:45.230 ⇒ 00:00:49.409 Brian Pei: Hey, is this always the the meeting link like, should I save this.
3 00:00:50.000 ⇒ 00:00:54.390 Nicolas Sucari: I don’t know. Can’t you see on the calendar invite.
4 00:00:55.240 ⇒ 00:00:59.215 Brian Pei: I don’t know when I click on the calendar. It doesn’t have a
5 00:00:59.590 ⇒ 00:01:01.759 Brian Pei: it just says data engineering weekly.
6 00:01:03.138 ⇒ 00:01:06.170 Brian Pei: Oh, now it says it. Okay, that’s weird.
7 00:01:07.510 ⇒ 00:01:08.339 Brian Pei: I didn’t. But about.
8 00:01:08.340 ⇒ 00:01:09.850 Nicolas Sucari: Why, sometimes.
9 00:01:09.850 ⇒ 00:01:11.930 Brian Pei: A couple of minutes before the meeting. The
10 00:01:14.320 ⇒ 00:01:15.250 Nicolas Sucari: I don’t know.
11 00:01:15.580 ⇒ 00:01:16.250 Brian Pei: It’s fun.
12 00:01:16.560 ⇒ 00:01:17.600 Brian Pei: No big deal.
13 00:01:18.110 ⇒ 00:01:18.969 Nicolas Sucari: How you doing.
14 00:01:20.370 ⇒ 00:01:21.239 Brian Pei: I’m pretty good.
15 00:01:22.075 ⇒ 00:01:22.400 Brian Pei: Okay.
16 00:01:23.420 ⇒ 00:01:25.860 Brian Pei: My mom is hang on one second.
17 00:01:52.180 ⇒ 00:01:53.490 Nicolas Sucari: I was trying to
18 00:01:53.860 ⇒ 00:01:57.530 Nicolas Sucari: do something in Blender. Have you used it before?
19 00:01:58.804 ⇒ 00:02:00.569 Brian Pei: I have not used blender. No.
20 00:02:03.310 ⇒ 00:02:04.430 Brian Pei: isn’t that like
21 00:02:05.030 ⇒ 00:02:06.200 Brian Pei: animation?
22 00:02:06.630 ⇒ 00:02:07.510 Nicolas Sucari: Yeah.
23 00:02:07.540 ⇒ 00:02:10.150 Brian Pei: Right? Okay, yeah, no. I have no idea how to use it.
24 00:02:13.250 ⇒ 00:02:14.310 Nicolas Sucari: Paper. Yes.
25 00:02:14.800 ⇒ 00:02:16.869 Payas Parab: Hey, Brian? Hey, Nicholas, how you guys doing.
26 00:02:17.850 ⇒ 00:02:18.770 Brian Pei: Good, good.
27 00:02:19.360 ⇒ 00:02:22.651 Brian Pei: nice to meet you, I’ll say.
28 00:02:23.200 ⇒ 00:02:23.890 Payas Parab: As well.
29 00:02:24.580 ⇒ 00:02:28.879 Brian Pei: I’ll set up in a snowflake account for you later today, after this meeting.
30 00:02:28.880 ⇒ 00:02:31.312 Payas Parab: Yeah, no rush. I just wanna make sure it’s on your your radar.
31 00:02:31.850 ⇒ 00:02:32.510 Payas Parab: Oh.
32 00:02:34.690 ⇒ 00:02:35.760 Brian Pei: Sounds good.
33 00:02:38.820 ⇒ 00:02:45.080 Nicolas Sucari: Yeah. Aman should be joining in like some minutes. But yeah, nice to meet you, I guess.
34 00:02:45.619 ⇒ 00:02:48.940 Nicolas Sucari: Any question you have anything you’d like to ask.
35 00:02:48.940 ⇒ 00:02:58.029 Payas Parab: Yeah, I’d love to know, like what roles you guys each have kind of for kind of for this project. I know I’m like, Brian, you’re the like solutions, engineer. Is that like what? What the role is.
36 00:02:58.030 ⇒ 00:02:58.520 Brian Pei: Yeah.
37 00:02:58.520 ⇒ 00:03:01.259 Payas Parab: Just kind of like when this project kicked off. I was out of office.
38 00:03:02.220 ⇒ 00:03:04.282 Brian Pei: Sure data
39 00:03:05.150 ⇒ 00:03:08.870 Brian Pei: data, architect data infra engineer. I guess setting up systems.
40 00:03:10.350 ⇒ 00:03:11.200 Payas Parab: Got it? Yeah.
41 00:03:11.200 ⇒ 00:03:11.950 Nicolas Sucari: Name.
42 00:03:12.350 ⇒ 00:03:14.929 Nicolas Sucari: I’m project manager from Brainforge.
43 00:03:16.520 ⇒ 00:03:20.059 Nicolas Sucari: yeah, I’m kind of managing all communications with the man
44 00:03:20.100 ⇒ 00:03:24.190 Nicolas Sucari: trying to be like the intermediate people that’s between
45 00:03:24.920 ⇒ 00:03:27.090 Nicolas Sucari: Brian, Aman.
46 00:03:27.300 ⇒ 00:03:31.669 Nicolas Sucari: Robert. Maybe you trying to coordinate all digital things.
47 00:03:32.310 ⇒ 00:03:39.120 Payas Parab: Got it got it? Is everything kind of going kind going kind of smoothly so far, any like big roadblocks you guys have hit yet, or
48 00:03:39.360 ⇒ 00:03:40.640 Payas Parab: things are going okay.
49 00:03:41.000 ⇒ 00:03:46.429 Nicolas Sucari: Yeah, things are going. Okay. We started. I think it was last week. Maybe
50 00:03:46.460 ⇒ 00:03:52.169 Nicolas Sucari: yeah. 2 weeks ago, something like that. We just started integrating
51 00:03:52.510 ⇒ 00:03:56.060 Nicolas Sucari: different data sources into Snowflake. So that’s.
52 00:03:56.060 ⇒ 00:03:56.360 Payas Parab: Got it.
53 00:03:56.360 ⇒ 00:03:57.659 Nicolas Sucari: Most of the updates.
54 00:03:57.910 ⇒ 00:03:58.350 Payas Parab: Sweet.
55 00:03:58.350 ⇒ 00:04:03.400 Brian Pei: I’ll talk about that today. Yeah, I really only started last week. So this would be second weekend.
56 00:04:04.500 ⇒ 00:04:05.320 Nicolas Sucari: Hi aman!
57 00:04:05.860 ⇒ 00:04:07.080 Aman Nagpal: Hey, guys, how’s it going.
58 00:04:08.450 ⇒ 00:04:10.120 Nicolas Sucari: Doing good. How about you?
59 00:04:10.880 ⇒ 00:04:15.880 Aman Nagpal: Good. Good. I’m just looking over your updates you just sent a few minutes ago.
60 00:04:16.459 ⇒ 00:04:17.849 Nicolas Sucari: Excellent. Yeah.
61 00:04:17.879 ⇒ 00:04:20.889 Nicolas Sucari: he got that answer from 5 trump people
62 00:04:21.614 ⇒ 00:04:26.299 Nicolas Sucari: yesterday. So I was just checking all of the different connectors that they have.
63 00:04:26.539 ⇒ 00:04:29.539 Nicolas Sucari: And yeah, maybe that’s helpful. Moving forward.
64 00:04:31.160 ⇒ 00:04:37.297 Brian Pei: Yeah, and I just send an email because of that desync thing.
65 00:04:37.940 ⇒ 00:04:44.670 Brian Pei: I went in and I tested Snowflake. And it wasn’t the IP issue, which I will talk about in a little bit.
66 00:04:44.790 ⇒ 00:04:48.390 Brian Pei: So my, my email is just like, I think it is just that
67 00:04:48.440 ⇒ 00:04:51.050 Brian Pei: the free trial ended. Maybe unless
68 00:04:51.467 ⇒ 00:04:56.249 Brian Pei: or at least I get the pop up when I log into 5 tran that says 0 days in the trial.
69 00:04:56.580 ⇒ 00:04:58.010 Brian Pei: and they might just
70 00:04:58.330 ⇒ 00:05:01.229 Brian Pei: block them when the trial’s over. But I’m not really sure.
71 00:05:01.980 ⇒ 00:05:03.299 Nicolas Sucari: Yeah, maybe we can
72 00:05:03.350 ⇒ 00:05:07.370 Nicolas Sucari: try them and see if that’s the issue. And if we need to scale it up.
73 00:05:09.260 ⇒ 00:05:10.530 Brian Pei: Yeah, but
74 00:05:10.590 ⇒ 00:05:13.787 Brian Pei: other than that which was
75 00:05:15.080 ⇒ 00:05:18.319 Brian Pei: in the past couple of days it.
76 00:05:18.350 ⇒ 00:05:22.094 Brian Pei: The 5 trend has been running smoothly up until
77 00:05:22.690 ⇒ 00:05:30.460 Brian Pei: the the bug that you emailed us about this morning. So I’ll talk about all of that pre connection stopping but
78 00:05:31.990 ⇒ 00:05:33.872 Brian Pei: yeah, in snowflake
79 00:05:34.720 ⇒ 00:05:37.100 Brian Pei: in the past. I’ve done
80 00:05:37.240 ⇒ 00:05:41.170 Brian Pei: IP network policies for different tools to get access to Snowflake.
81 00:05:41.220 ⇒ 00:05:54.669 Brian Pei: And as you know, in the 1st couple of days of last week I accidentally set it so that only Ips from Dbt. Cloud and 5 tran work, and not our logins. So that was my bad. That was on me I said.
82 00:05:54.670 ⇒ 00:05:55.300 Aman Nagpal: All good.
83 00:05:55.300 ⇒ 00:06:04.160 Brian Pei: Full support tickets, and one of them ghosted me, so Tom called them. Anyway, it’s all figured out now. We deleted all that stuff and everything’s fine. So
84 00:06:04.230 ⇒ 00:06:20.398 Brian Pei: over the weekend, when the connectors were re-syncing they finished their historical sync. And then and I checked that data on Monday. And then today, which is weird, that it failed. But today I checked to make sure that it was incrementally inserting
85 00:06:20.880 ⇒ 00:06:23.200 Brian Pei: So I did a couple of tests on
86 00:06:23.962 ⇒ 00:06:28.479 Brian Pei: I noticed that yesterday. Your
87 00:06:28.510 ⇒ 00:06:38.430 Brian Pei: or at least I did it. I just did like a distinct customer. Id count. I got yesterday like 1.5 million in shopify.
88 00:06:38.450 ⇒ 00:06:43.169 Brian Pei: which is a lot. I don’t know if that sounds right or wrong to you, but that’s what I see in.
89 00:06:43.170 ⇒ 00:06:45.870 Aman Nagpal: 5 million new orders.
90 00:06:45.870 ⇒ 00:06:53.409 Brian Pei: No 1.5 million distinct customer ids from shopify whether or not they made an order, I guess like could have signed up.
91 00:06:53.860 ⇒ 00:06:55.490 Aman Nagpal: If you’re not sure.
92 00:06:55.590 ⇒ 00:06:57.669 Aman Nagpal: Oh, you like new ones.
93 00:06:58.388 ⇒ 00:07:00.859 Brian Pei: Total sorry. I looked at the whole table.
94 00:07:00.860 ⇒ 00:07:05.849 Aman Nagpal: Oh, total. Yeah, total. I mean, I I’ve no idea if if it was new that that sounds way too high.
95 00:07:05.860 ⇒ 00:07:08.169 Aman Nagpal: I’m not sure about the the total.
96 00:07:08.770 ⇒ 00:07:13.319 Brian Pei: Okay. I didn’t do any like checking, for I don’t know
97 00:07:13.780 ⇒ 00:07:18.269 Brian Pei: customers that aren’t real people or anything. I just did some quick counts to make sure that we had data.
98 00:07:18.828 ⇒ 00:07:20.579 Brian Pei: And then, oh, yeah.
99 00:07:20.580 ⇒ 00:07:23.910 Aman Nagpal: The customer from shopify does say 1.5 2 8, so.
100 00:07:24.090 ⇒ 00:07:25.590 Brian Pei: Oh, great.
101 00:07:27.140 ⇒ 00:07:33.209 Brian Pei: okay, that’s awesome. So then today I ran the query, and it had an extra like 10,000 or something. So
102 00:07:33.340 ⇒ 00:07:41.940 Brian Pei: at least for me that checked that yesterday or this morning. It is syncing I did the same for orders. In Snowflake yesterday
103 00:07:42.320 ⇒ 00:07:43.330 Brian Pei: I
104 00:07:43.340 ⇒ 00:07:45.599 Brian Pei: got what is it
105 00:07:45.950 ⇒ 00:07:49.779 Brian Pei: like? 400,000 distinct orders? And today I see 400,
106 00:07:49.970 ⇒ 00:07:58.030 Brian Pei: actually a lot more, 437,000 shopify orders. I guess all time in the orders table is what I was seeing.
107 00:07:58.130 ⇒ 00:08:06.809 Brian Pei: I also I didn’t filter for refunds or orders that weren’t fulfilled. It’s just like every distinct order. Id, just to make sure that there was data in there.
108 00:08:06.840 ⇒ 00:08:08.650 Brian Pei: did a check on that
109 00:08:09.115 ⇒ 00:08:10.815 Brian Pei: and then for Amazon.
110 00:08:11.390 ⇒ 00:08:14.639 Brian Pei: I saw 73,000
111 00:08:14.850 ⇒ 00:08:17.339 Brian Pei: distinct order ids from from Amazon.
112 00:08:18.870 ⇒ 00:08:44.320 Brian Pei: so just checking that that sounds I mean, even just with the data point for customers, if that’s what it says in shopify and hopefully fingers crossed that. It is accurate. I’ll do a little bit more data validation on my own end. And I did. Also, I just wrote a quick query based on last week’s conversation on Tiktok. I do have tiktok tags coming in through shopify
113 00:08:44.790 ⇒ 00:08:52.169 Brian Pei: I got a hundred 12,933 orders that have some sort of tick tock tag on it
114 00:08:52.320 ⇒ 00:08:56.149 Brian Pei: when I’ve ran my validation. Query. So
115 00:08:56.689 ⇒ 00:08:59.349 Brian Pei: that was mostly just for me to
116 00:09:01.110 ⇒ 00:09:16.039 Brian Pei: trust that there is some sort of tick tock and shopify integration happening in the back end that comes through in the data. So that’s good as well. Some of them say like Tiktok, and some of them say tick tock with like a string and a bunch of random letters.
117 00:09:16.623 ⇒ 00:09:17.590 Brian Pei: We can
118 00:09:17.640 ⇒ 00:09:20.849 Brian Pei: go through what those mean after I
119 00:09:21.040 ⇒ 00:09:33.129 Brian Pei: do any sort of modeling. So have not started any sort of modeling, because we got the tables in I need to do some validation, and we need to make sure that that 5 trend bug is not
120 00:09:33.210 ⇒ 00:09:36.010 Brian Pei: because of some other issue or
121 00:09:36.150 ⇒ 00:09:38.853 Brian Pei: free trial to enterprise, whatever it is.
122 00:09:39.270 ⇒ 00:09:44.899 Brian Pei: But yeah, at the end there were 70 shopify tables and 59 Amazon tables.
123 00:09:46.290 ⇒ 00:09:49.440 Brian Pei: as it comes to data modeling. We.
124 00:09:49.740 ⇒ 00:09:52.729 Brian Pei: it’s there. There’s really no.
125 00:09:53.130 ⇒ 00:10:02.219 Brian Pei: I guess there is a world but for the purposes of like enterprise reporting when we use either dbt, or whatever you know, when we start to write sequel.
126 00:10:02.310 ⇒ 00:10:13.324 Brian Pei: there’s no model that uses all 70 tables. In any of the projects I’ve ever been on. It’s like the most important ones, customers and orders and stuff like that. So through this process
127 00:10:13.750 ⇒ 00:10:18.160 Brian Pei: we’ll go through like the shopify tables
128 00:10:18.270 ⇒ 00:10:33.429 Brian Pei: for reporting that we’ll need, and then at some point in the process, if there are some tables out of the 70 that we just don’t need, for whatever reason we can turn those off to save a little bit on storage and and compute
129 00:10:34.569 ⇒ 00:10:36.129 Brian Pei: so yeah, lots
130 00:10:36.210 ⇒ 00:10:43.650 Brian Pei: of tables that I just threw in all the table names, just so that you and the team know what they are into that Google sheet in the slack channel
131 00:10:44.651 ⇒ 00:10:48.860 Brian Pei: with Amazon and shopify. So
132 00:10:49.020 ⇒ 00:10:53.790 Brian Pei: shopify. I’ve kind of been going through Amazon. I only kind of looked at orders.
133 00:10:53.980 ⇒ 00:10:55.690 Brian Pei: The Amazon tables look
134 00:10:55.760 ⇒ 00:10:57.789 Brian Pei: a lot more
135 00:10:58.670 ⇒ 00:11:06.250 Brian Pei: accounting. E so I need to wrap my head around all of these like financial tables in there.
136 00:11:07.770 ⇒ 00:11:11.619 Brian Pei: and then, before we get into modeling.
137 00:11:12.270 ⇒ 00:11:16.699 Brian Pei: the question is, gonna be around what you guys have
138 00:11:17.030 ⇒ 00:11:19.330 Brian Pei: previously set up an amplitude
139 00:11:19.670 ⇒ 00:11:24.280 Brian Pei: in the sense of if there I have
140 00:11:24.710 ⇒ 00:11:30.322 Brian Pei: pretty basic limited knowledge of amplitude, and I haven’t done an amplitude
141 00:11:31.910 ⇒ 00:11:39.640 Brian Pei: project in the past, but so I don’t know. I know it like acts as a database, and it has reports, but I don’t know if you write sequel in it.
142 00:11:39.750 ⇒ 00:11:42.679 Brian Pei: But basically any SQL.
143 00:11:42.930 ⇒ 00:11:51.269 Brian Pei: That you have, which I don’t know if you do or not, because you didn’t have Snowflake before. Usually we, we look at either the like.
144 00:11:51.430 ⇒ 00:11:59.539 Brian Pei: even like arithmetic metrics that you have in like excel, but mostly sequel that you may have written in the past
145 00:12:00.127 ⇒ 00:12:06.950 Brian Pei: to use as a reference, or if you don’t have like any like
146 00:12:07.130 ⇒ 00:12:08.630 Brian Pei: production level
147 00:12:08.780 ⇒ 00:12:09.920 Brian Pei: business.
148 00:12:10.010 ⇒ 00:12:12.319 Brian Pei: SQL. SQL. Business logic.
149 00:12:12.330 ⇒ 00:12:15.021 Brian Pei: Then I I go into
150 00:12:15.810 ⇒ 00:12:17.280 Brian Pei: kind of like a
151 00:12:18.723 ⇒ 00:12:23.019 Brian Pei: what’s the word? The the default shopify.
152 00:12:23.510 ⇒ 00:12:23.800 Nicolas Sucari: Yeah.
153 00:12:23.800 ⇒ 00:12:26.909 Brian Pei: Aggregated metric tables that we would do
154 00:12:27.130 ⇒ 00:12:41.249 Brian Pei: for for for anybody, for kind of orders, customers, refunds, net revenue, gross revenue, blah, blah blah, like all that stuff. I’ve done it enough times where I can just try to do it, and then we can validate the the data
155 00:12:41.600 ⇒ 00:13:00.029 Brian Pei: once I complete it. But it doesn’t hurt to ask because every company will have some custom sequel logic for everything so figured. I’d ask that before. I kind of like try to create any cleaned up version of orders or cleaned up version of customers and anything but.
156 00:13:01.140 ⇒ 00:13:11.719 Aman Nagpal: Yeah, maybe pies can speak on what he’s seen so far in our amplitude. But I mean, as far as I know, we’re not using any sequel we do have with an amplitude some basic formulas like
157 00:13:11.840 ⇒ 00:13:24.490 Aman Nagpal: prop, some of like order, total subtracting taxes, things like that. Nothing crazy or like Aov formulas but no sequel, as far as I know, so.
158 00:13:24.640 ⇒ 00:13:25.700 Brian Pei: That’s okay.
159 00:13:25.700 ⇒ 00:13:48.430 Payas Parab: Yeah, there’s there’s no sequel. But there are like certain certain things have like like you said like formulas with tabular transformation. So if there’s like key formulas we can like pull out for you. We can like pull that out if that’s helpful. But more or less, it’s like all pretty standard like your like. Aov is just like exactly what you would expect right like. It’s just like a like it’s nothing too crazy. So whatever standard team that you guys have
160 00:13:48.520 ⇒ 00:13:50.340 Payas Parab: probably would be okay
161 00:13:50.860 ⇒ 00:13:54.979 Payas Parab: or standard like method for trial, like shopify.
162 00:13:57.220 ⇒ 00:13:58.710 Brian Pei: Yeah. And I think Rob
163 00:13:59.710 ⇒ 00:14:02.510 Brian Pei: Has an amplitude account.
164 00:14:02.790 ⇒ 00:14:05.710 Brian Pei: do you? Is, do you think it’s worth
165 00:14:05.870 ⇒ 00:14:10.545 Brian Pei: me poking around in amplitude, or do you want me to just run with,
166 00:14:11.500 ⇒ 00:14:14.250 Brian Pei: just go straight to modeling, since I have all these tables.
167 00:14:14.270 ⇒ 00:14:28.170 Payas Parab: I. I think I think we should move into modeling. But if you want like, what you can do is you can basically like for. Okay, here’s the customer table. Here’s what that looks like. And I can just quickly double check an amplitude like, okay, this kind of makes sense. Like, if you want to like, use us as a reference check. I just like, if you’re not familiar, it’s like
168 00:14:28.180 ⇒ 00:14:33.349 Payas Parab: it’s just not like a good use of your time, I think, to like poke around in there. We can kind of validate some of the
169 00:14:33.380 ⇒ 00:14:41.479 Payas Parab: the methodology as needed. But I nothing. Nothing is going to be like, non, like, not standard. Let’s just put it that way.
170 00:14:41.690 ⇒ 00:14:48.579 Payas Parab: There might be certain attributes that need to like flow through. So we we, if you send them to me and Robert. We can like take a look and confirm.
171 00:14:48.940 ⇒ 00:14:49.740 Brian Pei: Sweet.
172 00:14:50.170 ⇒ 00:14:59.050 Brian Pei: Alright, that sounds good. I can get going with that even while 5 train is being hung hung up right now. Just start playing around with some
173 00:14:59.160 ⇒ 00:15:00.970 Brian Pei: sequel modeling.
174 00:15:01.561 ⇒ 00:15:06.478 Brian Pei: For my, I think my last question for Aman is
175 00:15:07.350 ⇒ 00:15:10.769 Brian Pei: You have. You have shopify orders, and you have Amazon orders.
176 00:15:11.570 ⇒ 00:15:17.490 Brian Pei: Some sort of Consolidated Orders table. Will
177 00:15:17.530 ⇒ 00:15:19.494 Brian Pei: union those together?
178 00:15:20.320 ⇒ 00:15:21.819 Brian Pei: do you are. Is there?
179 00:15:22.010 ⇒ 00:15:28.320 Brian Pei: Will will there ever be any overlap between an order in shopify and an order in Amazon.
180 00:15:28.799 ⇒ 00:15:37.179 Brian Pei: I don’t think they talk to each other, and they’re different platforms, obviously. But I just wanna make sure that if I do some sort of union that I’m not
181 00:15:37.240 ⇒ 00:15:44.660 Brian Pei: duplicating orders for any sort of systemic reason why the same order would be fed to both systems? Or will that never happen.
182 00:15:45.830 ⇒ 00:15:47.030 Aman Nagpal: So
183 00:15:47.420 ⇒ 00:16:07.171 Aman Nagpal: the same order, I don’t believe will ever be within both systems. It’s you know, Tiktok. I know we spoke about where tick tock orders are coming in. Shopify Amazon should be completely separate. Your your point about putting them all in one combined orders. Table I think that’s fine. As long as you know most of the stuff we look at
184 00:16:07.510 ⇒ 00:16:13.860 Aman Nagpal: visualizations, data, everything is typically going to be separate, you know. We’ll look at shopify separately. We’ll look at Amazon separately.
185 00:16:14.299 ⇒ 00:16:32.070 Aman Nagpal: But you know some things that I’m sure you know, like like we’re trying to do already, right? So just on the Amazon side. I know they don’t give customer identifying information. They think a fake email address assigned to each customer, and where we’ve been kind of looking at that when we
186 00:16:32.070 ⇒ 00:16:44.746 Aman Nagpal: backfilled Amazon orders into amplitude to say, Hey, look this person with this fake email their next order. If it uses that same fake email. We can kind of look at, repeat, purchase rates that way within it.
187 00:16:45.570 ⇒ 00:16:46.559 Brian Pei: Oh, yeah, that’s.
188 00:16:47.440 ⇒ 00:16:52.509 Aman Nagpal: Yeah, I mean, I’m assuming it’s the same fake email that they use hopefully. That’s the case. But
189 00:16:52.944 ⇒ 00:17:06.030 Aman Nagpal: ideally, if we’re ever able to kind of take any sort of customer information from Amazon and match that up with a shopify customer. That would be amazing. I don’t know if that’s possible.
190 00:17:06.414 ⇒ 00:17:17.119 Aman Nagpal: But yeah, I mean, I don’t know if that’s a huge task to combine them into one orders, table or not, or if there’s benefits of doing it or not, but you know I’ll leave that up to you.
191 00:17:17.900 ⇒ 00:17:18.520 Brian Pei: Okay.
192 00:17:19.320 ⇒ 00:17:24.738 Brian Pei: cool. I actually didn’t see a customer’s table in Amazon, but maybe they just called it something else.
193 00:17:25.520 ⇒ 00:17:27.919 Brian Pei: if they mask everything with a fake email.
194 00:17:28.359 ⇒ 00:17:33.330 Brian Pei: Then it’s hard like in in the past. If we wanted to consolidate customer
195 00:17:33.400 ⇒ 00:17:39.010 Brian Pei: information, you do a you do a join on the email address and maybe the address and the phone number
196 00:17:39.583 ⇒ 00:17:41.550 Brian Pei: so I, basically, I need
197 00:17:41.880 ⇒ 00:17:48.960 Brian Pei: one column of any sort of customer information to be able to try to get it, to talk, to shopify, but if I don’t have
198 00:17:49.010 ⇒ 00:17:52.610 Brian Pei: any, then it might be difficult, unless it’s.
199 00:17:52.610 ⇒ 00:18:00.100 Payas Parab: Oh, sorry. A quick question, Brian. Could you clarify what you mean by so fake emails that, like Amazon, assigns it like some type of like an anonymized
200 00:18:00.150 ⇒ 00:18:04.819 Payas Parab: like. So you’ll we’ll never really know what the full email address is, but
201 00:18:04.910 ⇒ 00:18:14.359 Payas Parab: it can be used as an identifier in other situations like it can be. I use an identifier for repeat orders, but not to join with other data sets is that the limitation.
202 00:18:15.190 ⇒ 00:18:20.500 Aman Nagpal: So. Yes, and let me clarify. What I’m talking about is the data we’re getting from the seller
203 00:18:20.590 ⇒ 00:18:22.450 Aman Nagpal: reports. Api.
204 00:18:22.868 ⇒ 00:18:26.060 Aman Nagpal: Not from Amazon dashboard right, so I don’t.
205 00:18:26.060 ⇒ 00:18:26.940 Payas Parab: That’s what I do.
206 00:18:27.220 ⇒ 00:18:33.972 Aman Nagpal: Method when 5, trying to grab the data. I don’t know if it’s giving you that information or not. But I can tell you right now if I pull up
207 00:18:34.440 ⇒ 00:18:45.970 Aman Nagpal: an event for Amazon. So we get email is a randomized email that they assigned I don’t think we’re getting name. I don’t think we’re getting even street address.
208 00:18:47.960 ⇒ 00:18:57.439 Aman Nagpal: I think at 1 point some orders we did maybe see the state and city potentially even that. I’m not sure about. But yeah, I don’t. I don’t think there’s
209 00:18:57.670 ⇒ 00:19:02.540 Aman Nagpal: at least from the Api. I don’t think there’s any identifying information, so there is state there is city.
210 00:19:04.260 ⇒ 00:19:10.630 Aman Nagpal: but yeah, I I don’t know if you’re getting the same data with through 5 Tran and all. I don’t know if there’s anything that we can kind of use to match up with shopify.
211 00:19:10.630 ⇒ 00:19:13.740 Nicolas Sucari: No, we are. We are getting the shipping address.
212 00:19:14.060 ⇒ 00:19:18.100 Nicolas Sucari: or yeah, some fields for shipping, but I don’t know if we can use that.
213 00:19:18.630 ⇒ 00:19:19.290 Aman Nagpal: The full address.
214 00:19:19.290 ⇒ 00:19:20.120 Nicolas Sucari: That’s true.
215 00:19:20.670 ⇒ 00:19:23.400 Nicolas Sucari: I haven’t checked. I’m checking the columns.
216 00:19:23.920 ⇒ 00:19:27.120 Brian Pei: I’m seeing apartment numbers and streets in here.
217 00:19:27.720 ⇒ 00:19:31.770 Aman Nagpal: That’s perfect. That’s the case. We can try to identify using address.
218 00:19:32.460 ⇒ 00:19:36.620 Brian Pei: So we can try it that way. I do see some phone numbers, but not all.
219 00:19:37.400 ⇒ 00:19:37.990 Payas Parab: From a company.
220 00:19:37.990 ⇒ 00:19:38.340 Brian Pei: This is.
221 00:19:38.340 ⇒ 00:19:49.409 Payas Parab: Any like overlap like, do you think it’s a lot of overlap? Or do you think that like that selling motion is separate, right like? Is it like a separate group of customers on Amazon, or something like, how how often do you think this would be even the case, that it’s like
222 00:19:49.420 ⇒ 00:19:51.190 Payas Parab: worth us wrangling with it.
223 00:19:52.070 ⇒ 00:19:56.499 Aman Nagpal: I think the overlap. It’s not that big. I think there probably is
224 00:19:56.870 ⇒ 00:20:07.769 Aman Nagpal: small, even tiny overlap. I know the way. Justin thinks he’s eventually, if it’s possible, he’s gonna want to report. Say, hey, look! These customers bought on both channels.
225 00:20:07.770 ⇒ 00:20:09.469 Payas Parab: Sure, sure, fair.
226 00:20:13.470 ⇒ 00:20:13.750 Brian Pei: All right.
227 00:20:13.750 ⇒ 00:20:17.270 Aman Nagpal: So I’ll give you another example. Just came up yesterday. I know we talked about Klaviyo
228 00:20:17.410 ⇒ 00:20:19.679 Aman Nagpal: now he’s asking me to
229 00:20:19.870 ⇒ 00:20:26.050 Aman Nagpal: have an email received event firing for every customer that receives an email.
230 00:20:26.200 ⇒ 00:20:39.309 Aman Nagpal: And I told them, you know, this is gonna you know, we gotta bump up our our number of events and amplitude again. Then this is gonna be a huge amount of emails, whether it’s coming from a campaign, a campaign or coming from a flow. I haven’t heard back yet, but you know, just.
231 00:20:39.770 ⇒ 00:20:47.170 Aman Nagpal: you know, thinking in that line, this is probably something that you know. With this data warehouse we might be doing as well.
232 00:20:50.800 ⇒ 00:20:52.390 Aman Nagpal: So just super granular.
233 00:20:52.710 ⇒ 00:20:53.430 Brian Pei: Okay?
234 00:20:54.295 ⇒ 00:20:59.654 Brian Pei: My, my final last last final question, when we’re doing
235 00:21:02.940 ⇒ 00:21:08.877 Brian Pei: when we’re doing business logic that isn’t included in like just a column in the order table. One of them is
236 00:21:09.230 ⇒ 00:21:15.060 Brian Pei: subscriptions. I did see that recurring subscription is a tag in shopify.
237 00:21:16.820 ⇒ 00:21:19.359 Brian Pei: Is that the best way for me to identify
238 00:21:19.780 ⇒ 00:21:25.029 Brian Pei: orders that are on a subscription. I know that’s a really important data point for all companies.
239 00:21:25.060 ⇒ 00:21:30.469 Brian Pei: Is there another system like, how do you manage subscript, or report on subscriptions right now, I guess, is what I should be asking.
240 00:21:31.130 ⇒ 00:21:33.320 Aman Nagpal: Absolutely. Yeah. So let me just
241 00:21:33.370 ⇒ 00:21:35.949 Aman Nagpal: give you the exact info for so for Tiktok.
242 00:21:36.020 ⇒ 00:21:47.309 Aman Nagpal: like you mentioned, we do have the tags. But there’s also the Channel Tiktok. So all Tiktok orders are coming up as Tiktok Channel. I don’t think it’ll make much of a difference using
243 00:21:47.330 ⇒ 00:21:49.879 Aman Nagpal: channel versus using Tiktok, but
244 00:21:50.120 ⇒ 00:21:52.180 Aman Nagpal: maybe because
245 00:21:53.090 ⇒ 00:21:55.650 Aman Nagpal: it it might be ideal just to stick with the
246 00:21:56.590 ⇒ 00:22:20.830 Aman Nagpal: I don’t know right now. We’re using the Tiktok native integrate native app within shopify. You know. What if we were to switch to a different app that handles, syncing tick tock orders to shopify? So in that case I don’t know that the channel will still show up as Tiktok so maybe, you know, rewinding back to everything. I’m just saying, maybe tags is the best way to go for Tiktok, where we have the Tiktok tag, and then we also have
247 00:22:20.830 ⇒ 00:22:33.719 Aman Nagpal: Tiktok free sample tags, and each Tiktok order is also tagged with a Tiktok order number, since that’s different from the shopify order number. So that’s on Tiktok
248 00:22:34.060 ⇒ 00:22:36.289 Aman Nagpal: in terms of subscriptions.
249 00:22:36.890 ⇒ 00:22:39.390 Aman Nagpal: The way it typically works
250 00:22:39.430 ⇒ 00:22:47.222 Aman Nagpal: is, I think the channel changes. Let me confirm here. So we have the renewal tag.
251 00:22:48.000 ⇒ 00:22:52.060 Aman Nagpal: that populates when it’s a renewal order. I’m just trying to remember
252 00:22:52.780 ⇒ 00:22:57.500 Aman Nagpal: how we have that tagging, and if there’s a better source of
253 00:22:57.600 ⇒ 00:22:59.650 Aman Nagpal: a better source where that comes from.
254 00:22:59.760 ⇒ 00:23:04.530 Brian Pei: You. You don’t have to answer right now. But subscriptions is is one of the
255 00:23:05.900 ⇒ 00:23:07.350 Brian Pei: parts that
256 00:23:07.740 ⇒ 00:23:10.739 Brian Pei: get messy in terms of context for
257 00:23:11.194 ⇒ 00:23:16.389 Brian Pei: I guess. Like, if I’m ordering on a subscription, it’s like every month I I get
258 00:23:16.440 ⇒ 00:23:18.682 Brian Pei: your product versus.
259 00:23:19.460 ⇒ 00:23:43.760 Brian Pei: if I just happen to order it myself every month versus I had a subscription, and it canceled, but then I order it myself, and then I get another subscription. It’s like all all those kinds of questions that I’ll kind of like leave to to you and your team to to give us some business logic on how you perceive what a subscription is like, which then sometimes is calculated into what is it like?
260 00:23:43.990 ⇒ 00:23:52.909 Brian Pei: Customer? Total value of a customer? Versus? How many subscriptions do I have if I just keep going on and off and on, and off and on and off, and then I just buy
261 00:23:53.080 ⇒ 00:23:56.849 Brian Pei: 5 products at one time like I like that that kind of stuff.
262 00:23:57.370 ⇒ 00:24:05.709 Brian Pei: I I won’t spend too much time trying to detangle it myself, but in the data and the tags. Whenever I see some any sort of like
263 00:24:06.030 ⇒ 00:24:14.040 Brian Pei: subscription renewal, Boolean or date range, I’ll just include that and deal with arithmetic. Later.
264 00:24:15.410 ⇒ 00:24:25.040 Aman Nagpal: Yeah, no, I I definitely want to get you these details. So I’ll work on that list and tell you where it’s coming from but we do have one time ordered.
265 00:24:25.090 ⇒ 00:24:27.220 Aman Nagpal: We have subscription orders
266 00:24:27.684 ⇒ 00:24:39.419 Aman Nagpal: which is both 1st order and renew renewing orders, and then we separate, you know, 1st time order versus renewal orders, and the last thing we have.
267 00:24:39.957 ⇒ 00:24:47.440 Aman Nagpal: If I’m not forgetting anything is reactivation orders, which, is, when they cancel the subscription.
268 00:24:47.500 ⇒ 00:24:55.500 Aman Nagpal: They came back to our reactivation page. We restarted their subscription. That 1st initial order we call a reactivation.
269 00:24:56.490 ⇒ 00:24:57.170 Brian Pei: Got it.
270 00:24:57.530 ⇒ 00:24:59.369 Aman Nagpal: But I’ll get you all the details on that stuff.
271 00:24:59.370 ⇒ 00:24:59.950 Brian Pei: Awesome.
272 00:25:00.853 ⇒ 00:25:03.266 Brian Pei: That’s that’s that’s great.
273 00:25:05.100 ⇒ 00:25:07.980 Brian Pei: cool. I think those are all the questions I had. I think now that
274 00:25:08.480 ⇒ 00:25:15.750 Brian Pei: I, our team, is seeing data and stuff like these meetings will just be like random fire rapid fire questions. But
275 00:25:15.990 ⇒ 00:25:18.708 Brian Pei: other than the IP thing, we’re in a good state
276 00:25:19.070 ⇒ 00:25:21.139 Brian Pei: The fact that I have all these questions
277 00:25:21.320 ⇒ 00:25:22.879 Brian Pei: should show that, you know, we’re
278 00:25:22.890 ⇒ 00:25:27.140 Brian Pei: looking at the data. We’re gonna start to model it. And we’ll get you something.
279 00:25:28.970 ⇒ 00:25:32.240 Brian Pei: we’ll get you something in your hands as soon as
280 00:25:33.760 ⇒ 00:25:44.769 Brian Pei: well, I guess. Yeah. Would you prefer iterative like, Hey, I have a table that you can look at now, but it might not be ready. Or do you want me to like? Finish, finish something and then deliver it.
281 00:25:47.930 ⇒ 00:25:56.799 Aman Nagpal: I don’t. I don’t mind looking at the work in progress. I mean, if it’s if it’s just kind of like, hey, I just did this. We’re still working on it, and you want to throw it in slack. That’s great. Or you know.
282 00:25:57.260 ⇒ 00:26:12.490 Nicolas Sucari: Yeah, we can send. We can send some updates once we are starting with the modeling so that you can take a look to the tables. But then what we will try to do is to start using real to share with you that information in real, to set up all of those tables
283 00:26:12.807 ⇒ 00:26:19.180 Nicolas Sucari: and then you can. You will be able to check a little bit more visually that information and see if the numbers are
284 00:26:19.536 ⇒ 00:26:36.019 Nicolas Sucari: yeah, are are okay, or if something or something, if something is really strange, and we can, we need to check the modeling. But once we do that, yeah, you will be able to check easily if the shopify Amazon orders are there. And then we can start
285 00:26:36.040 ⇒ 00:26:40.150 Nicolas Sucari: doing some like specifically dashboards regarding some stuff.
286 00:26:41.000 ⇒ 00:26:47.120 Aman Nagpal: Yeah, that works. And on my side I reached out to. I know Robert wanted to talk to the
287 00:26:47.350 ⇒ 00:26:54.040 Aman Nagpal: stakeholders, so I reached out to Justin and Jared to see if and when they can hop on a call with Robert.
288 00:26:54.190 ⇒ 00:26:55.420 Aman Nagpal: Okay.
289 00:26:55.570 ⇒ 00:27:05.120 Aman Nagpal: yeah. So that’s that’s the update there. And again. Just if if there’s anything you’re waiting on me ever just keep feel free to keep paying back I’ll try to get it to you asap.
290 00:27:05.810 ⇒ 00:27:26.970 Nicolas Sucari: Yeah, excellent. And then, if there is any dashboard in amplitude that you are using, that you would like us to know how to or to replicate or to try to check their data. Let us know, and we can. We can go through it. I have access to amplitude, but I don’t know if which of the dashboards you are looking at just let me know, and they can. Yes, start to poke around a little bit.
291 00:27:27.700 ⇒ 00:27:28.920 Aman Nagpal: Yeah, maybe a
292 00:27:29.000 ⇒ 00:27:38.009 Aman Nagpal: deliverable from me. To you guys. The requirement, I guess, is a list of all of our most used reports and dashboards.
293 00:27:38.287 ⇒ 00:27:48.089 Aman Nagpal: So I will try my best to get that to you. I know there’s a ton floating everywhere, but I will say, you know, I think we brought this up last time is just to make sure that
294 00:27:48.430 ⇒ 00:27:55.425 Aman Nagpal: how are we handling this right? So all of the reports that we currently have an amplitude which ones stay, as is versus.
295 00:27:55.920 ⇒ 00:28:01.339 Aman Nagpal: Once we kind of either redo or move over with the data from the web data warehouse.
296 00:28:01.340 ⇒ 00:28:10.410 Nicolas Sucari: Yeah, yeah, I’m I’m not saying we’re gonna redo everything right now, we’re just just to check. If what if our modeling reflects that reality, in in, up, to.
297 00:28:10.880 ⇒ 00:28:11.590 Aman Nagpal: Yeah, yeah.
298 00:28:11.590 ⇒ 00:28:19.210 Nicolas Sucari: For us is to to double, cross the the checks and see that our modeling with the new data in Snowflake is reflecting exactly the same that you have on amplitude.
299 00:28:19.670 ⇒ 00:28:21.120 Aman Nagpal: You got it. That’s perfect.
300 00:28:21.660 ⇒ 00:28:23.170 Nicolas Sucari: Okay, excellent.
301 00:28:24.400 ⇒ 00:28:25.240 Nicolas Sucari: Okay.
302 00:28:25.715 ⇒ 00:28:28.960 Nicolas Sucari: I don’t have anything else. I don’t know. Guys. If you
303 00:28:29.010 ⇒ 00:28:30.360 Nicolas Sucari: have any other question.
304 00:28:33.100 ⇒ 00:28:34.809 Nicolas Sucari: Okay, we’re done.
305 00:28:34.920 ⇒ 00:28:43.360 Nicolas Sucari: Thank you guys, we’ll still, we’ll keep posting questions and giving updates through slack. And yeah, let’s hope in a meeting next next Tuesday. Okay.
306 00:28:43.970 ⇒ 00:28:45.610 Aman Nagpal: Sounds good. Thanks. A lot. Guys.
307 00:28:45.610 ⇒ 00:28:46.140 Brian Pei: Thanks. Everyone.
308 00:28:46.140 ⇒ 00:28:47.470 Nicolas Sucari: Thanks, guys. Bye-bye.