Meeting Title: Zoom Meeting Date: 2025-04-09 Meeting participants: Annie Yu, Robert Tseng
WEBVTT
1 00:00:27.200 ⇒ 00:00:28.590 Annie Yu: Hello, Robert!
2 00:00:30.980 ⇒ 00:00:31.870 Robert Tseng: Hey! Annie!
3 00:00:31.870 ⇒ 00:00:32.790 Annie Yu: Hi!
4 00:00:34.160 ⇒ 00:00:34.920 Annie Yu: Oh.
5 00:00:38.210 ⇒ 00:00:43.640 Annie Yu: I feel like my morning is moving so fast.
6 00:00:43.640 ⇒ 00:00:44.240 Robert Tseng: Yeah.
7 00:00:45.086 ⇒ 00:00:56.930 Annie Yu: Okay, thank you so much for the time. I think I do have one small question. Maybe we can start with that for the cohort. I’m
8 00:00:57.090 ⇒ 00:01:01.700 Annie Yu: cause I wanna get back to a wish on this as soon as possible.
9 00:01:02.340 ⇒ 00:01:02.960 Robert Tseng: Okay.
10 00:01:03.760 ⇒ 00:01:24.569 Annie Yu: So I’m gonna pull up like something just really scrappy here. So just showing you what our table install fake is is looking like. So for monthly cohort, we’re trying to get the cumulative sales over each monthly cohort numbers. So that means
11 00:01:24.680 ⇒ 00:01:30.790 Annie Yu: whoever placed their 1st order in January. We we count them in here, and we want to fix that
12 00:01:31.756 ⇒ 00:01:32.930 Annie Yu: throughout the
13 00:01:33.360 ⇒ 00:01:47.519 Annie Yu: the future months. And then we get the cumulative sales from that 1st month over the next few months, divided by the same denominators. So the same customers within that cohort
14 00:01:49.100 ⇒ 00:01:53.929 Annie Yu: And then I know that you mentioned that the team would want to know.
15 00:01:54.210 ⇒ 00:01:59.980 Annie Yu: kind of be able to filter on App source. And this is where I am like a
16 00:02:00.340 ⇒ 00:02:04.739 Annie Yu: not sure what what to go about it. So where should I
17 00:02:05.200 ⇒ 00:02:25.649 Annie Yu: agree on like kind of this table? Because with this all we can get what we want with the same view as lifetimely because we aggregated all the new customers within this cohort across app source. And now the question is
18 00:02:26.260 ⇒ 00:02:31.159 Annie Yu: so, if we want to be able to see different app source.
19 00:02:31.740 ⇒ 00:02:38.010 Annie Yu: So for the aggregation here, let’s maybe just look at the distinct customers.
20 00:02:38.290 ⇒ 00:02:39.570 Annie Yu: So
21 00:02:40.990 ⇒ 00:02:53.440 Annie Yu: the problem with this one is, let’s say, someone. Okay, this is like a 1 place his very 1st order in January on shopify. So that’s he’s like
22 00:02:53.950 ⇒ 00:02:55.759 Annie Yu: actual 1st order.
23 00:02:56.020 ⇒ 00:03:03.890 Annie Yu: And then, if he placed another order in Amazon next month, we wouldn’t count him
24 00:03:04.680 ⇒ 00:03:12.680 Annie Yu: as a new customer in Amazon. Is that okay? So I’m I’m like trying to wrap my head around that I’m not sure
25 00:03:14.160 ⇒ 00:03:15.090 Annie Yu: which way.
26 00:03:23.220 ⇒ 00:03:33.219 Annie Yu: And yeah, so that’s where I’m but with all, we don’t have to worry about that, and we will have all either way. I’m just not sure for each app source
27 00:03:35.280 ⇒ 00:03:36.939 Annie Yu: how to go about it.
28 00:03:37.800 ⇒ 00:03:42.449 Robert Tseng: Yeah, I mean, I kind of see this as 2 separate models. Right? It’s kind of like we have one. That’s
29 00:03:43.680 ⇒ 00:03:53.490 Robert Tseng: it’s 1st order of all time, like kind of blended across all marketplaces. And then like, eventually, I mean, maybe there’d be one where we would need to.
30 00:03:54.120 ⇒ 00:03:59.049 Robert Tseng: Not just maybe. But oh, yeah, if we wanted to do like 1st time shopify orders.
31 00:04:00.550 ⇒ 00:04:05.750 Robert Tseng: Then you would need this view that you’re this sheet that you’re just that you have here.
32 00:04:06.960 ⇒ 00:04:08.306 Annie Yu: So if
33 00:04:09.090 ⇒ 00:04:29.349 Annie Yu: okay, I think wish was saying, if we do the 1st order. So this one a 1 place his 1st order in January 25. So we count him so think of this like a unique id. So we count as monthly cohort January
34 00:04:29.670 ⇒ 00:04:33.510 Annie Yu: and shopify. So that means, even if he plays
35 00:04:33.640 ⇒ 00:04:41.720 Annie Yu: other orders and other app stores, we would still track his cumulative sales. Within this this cohort.
36 00:04:41.880 ⇒ 00:04:42.480 Robert Tseng: Yeah.
37 00:04:42.830 ⇒ 00:04:44.699 Annie Yu: Okay. And that’s okay.
38 00:04:44.700 ⇒ 00:04:45.290 Robert Tseng: Yeah.
39 00:04:45.890 ⇒ 00:04:47.130 Annie Yu: Okay, then that
40 00:04:47.630 ⇒ 00:04:59.610 Annie Yu: then I think I think I’m good. I think we’re good. We can get it moving. And idea, I just wanted to run that by you. I’m I’m not sure what’s the best way to go about that.
41 00:05:00.370 ⇒ 00:05:08.059 Robert Tseng: Okay, yeah, I mean, so you have this customer id, and then you have shopify. So
42 00:05:08.610 ⇒ 00:05:10.919 Robert Tseng: if I wanted to.
43 00:05:12.250 ⇒ 00:05:35.360 Robert Tseng: Well, then, you’re tracking lifetime value kind of like since then, which it should be across any platform. There aren’t that many customers that like order across platform. So I don’t think it’s that big of a deal. Yeah? Then, if I want it. So yeah, I think that makes sense. If I wanted to just look at shopify lifetime value for that customer. I could just join on that customer and and sum up shopify sales from fact for fact, orders so
44 00:05:35.910 ⇒ 00:05:38.939 Robert Tseng: feel like I don’t really need that in this model, right?
45 00:05:39.800 ⇒ 00:05:44.959 Annie Yu: Yes, but this summary table would be aggregated without customer. Id.
46 00:05:46.350 ⇒ 00:05:46.840 Robert Tseng: Right?
47 00:05:51.327 ⇒ 00:05:54.370 Robert Tseng: So the cumulative sales would be
48 00:05:55.040 ⇒ 00:05:58.850 Robert Tseng: cumulative sales across all channels, even though that yeah.
49 00:05:58.850 ⇒ 00:06:02.949 Annie Yu: No, if if the I mean for these ones yes.
50 00:06:02.950 ⇒ 00:06:03.420 Robert Tseng: Yeah.
51 00:06:03.420 ⇒ 00:06:05.610 Annie Yu: For these ones. It would be
52 00:06:05.770 ⇒ 00:06:13.900 Annie Yu: someone who plays their 1st order very 1st order within this source, and then their future sales.
53 00:06:14.750 ⇒ 00:06:17.289 Robert Tseng: But but yeah, but their future sales could be from any source.
54 00:06:17.290 ⇒ 00:06:18.669 Annie Yu: Yes. Yes. Yeah.
55 00:06:18.990 ⇒ 00:06:20.539 Annie Yu: Okay. Cool. Cool.
56 00:06:20.540 ⇒ 00:06:21.080 Robert Tseng: Yep.
57 00:06:21.080 ⇒ 00:06:23.770 Annie Yu: Then that clears it.
58 00:06:23.870 ⇒ 00:06:28.960 Annie Yu: And then the other thing I know I would.
59 00:06:29.830 ⇒ 00:06:31.070 Annie Yu: Okay, let me.
60 00:06:34.480 ⇒ 00:06:41.120 Annie Yu: So this one and I I took a just a quick look.
61 00:06:41.570 ⇒ 00:06:43.550 Annie Yu: I I think I
62 00:06:45.440 ⇒ 00:06:58.200 Annie Yu: kind of understand the demand challenge now and then. My question is, then would be, and also this one. Is this more like a potential future solution that you’re proposing.
63 00:06:59.490 ⇒ 00:07:03.980 Robert Tseng: This is just me, like, basically
64 00:07:04.240 ⇒ 00:07:09.280 Robert Tseng: writing out how I think incremental does incrementality.
65 00:07:09.280 ⇒ 00:07:10.260 Annie Yu: Oh, okay.
66 00:07:10.260 ⇒ 00:07:17.770 Robert Tseng: Yeah, so, yeah, I mean, this is, I.
67 00:07:17.770 ⇒ 00:07:30.289 Annie Yu: So, okay, so these, okay, that makes sense. Because I was looking at this. And I think this is like a full on engine rather than like an analyst
68 00:07:31.110 ⇒ 00:07:40.290 Annie Yu: who will be capable of like doing all these so that that will make sense. So this is more like a how incremental does their.
69 00:07:40.610 ⇒ 00:07:44.389 Annie Yu: They’re like like the engine of this.
70 00:07:44.790 ⇒ 00:08:02.430 Robert Tseng: Yeah, but I guess this is kind of like I mean, you could describe this as future state, like I mean, I I think it’d be cool to be able to build this in house, and then we’d be able to apply it across other customers. But yeah, that’s not like the short term ask. I guess that’s just giving you an idea of like what I think.
71 00:08:03.570 ⇒ 00:08:04.780 Annie Yu: So that oh.
72 00:08:05.725 ⇒ 00:08:06.220 Robert Tseng: Yeah.
73 00:08:06.580 ⇒ 00:08:17.729 Annie Yu: So the goal for this ticket would be figure out if we can kind of simulate what they do. Like a lighter version. And then what that would look like, or.
74 00:08:18.960 ⇒ 00:08:41.310 Robert Tseng: Yeah, so I guess I’m I have my other screen pull up here. So look at this in more detail. Yes, I shared like a couple notion tests or like notion docs, for I mean, I could write this a bit clearer, but I think the main objective is like, like, can we measure in like? Can we measure incrementality? Can we have like a baseline measurement for incrementality?
75 00:08:41.626 ⇒ 00:08:50.799 Robert Tseng: As well? So there were like a couple of ways that we thought about approaching approaching it. One is like running the geolift test. So we can know, like.
76 00:08:52.470 ⇒ 00:08:59.740 Robert Tseng: if you basically like, what what does like
77 00:09:01.220 ⇒ 00:09:06.890 Robert Tseng: what like? What’s the impact of non of of like paid and non.
78 00:09:07.400 ⇒ 00:09:14.410 Robert Tseng: or of a non non product specific, just like general awareness campaigns.
79 00:09:17.120 ⇒ 00:09:18.160 Robert Tseng: Organic
80 00:09:18.980 ⇒ 00:09:28.022 Robert Tseng: it like we have like attribution. Okay, I don’t. I don’t wanna be throwing around all this terminology, I mean, how do I simplify this.
81 00:09:30.640 ⇒ 00:09:36.439 Robert Tseng: so we have a way to like measure
82 00:09:36.730 ⇒ 00:09:42.199 Robert Tseng: campaign like ad spend campaign performance at the product level. But not every
83 00:09:42.808 ⇒ 00:10:00.750 Robert Tseng: ad that they run is product specific, right? Some of it is just like brand campaigns that are more general awareness like this is the Eden brand, like, we’re not selling you a specific product. And then there’s also going to be like fluctuations in traffic from just organic, or
84 00:10:00.910 ⇒ 00:10:05.290 Robert Tseng: because maybe their brand has like gotten more
85 00:10:05.500 ⇒ 00:10:16.200 Robert Tseng: renown over over time. And like, I don’t think they have a way to measure like the impact of those types of campaigns. If it’s not specifically like
86 00:10:16.620 ⇒ 00:10:31.910 Robert Tseng: product X campaign, like like what they don’t. Yeah, those are. Those campaigns are more straightforward to track. But then, like kind of other marketing efforts are harder to track. So incrementality is just like in supposed to be a way to
87 00:10:33.043 ⇒ 00:10:35.909 Robert Tseng: figure out like, are your.
88 00:10:36.180 ⇒ 00:10:40.070 Robert Tseng: is your, are your marketing investments actually making
89 00:10:40.740 ⇒ 00:10:51.990 Robert Tseng: are adding additional customers or or like, are they just like kind of cannibalizing or replacing, like other marketing efforts that you would have already that you already have gotten
90 00:10:53.830 ⇒ 00:11:00.600 Robert Tseng: So I mean, I maybe I could write the objectives out a bit clearer. But I think to me, this is, I just wanted you to really
91 00:11:01.455 ⇒ 00:11:06.510 Robert Tseng: understand and like, how are we handling attribution for these
92 00:11:06.760 ⇒ 00:11:20.859 Robert Tseng: not very straightforward types of marketing efforts? And like, how do we evolve it to the next level. So if incremental is like kind of the step one through 4 like engine that you’re describing.
93 00:11:20.960 ⇒ 00:11:32.410 Robert Tseng: we’re at 0 right now, like there must be like an intermediary step that we can really push the like the the efforts towards so it’s it’s kind of more of a
94 00:11:32.890 ⇒ 00:11:36.849 Robert Tseng: like. That’s that’s kind of the stage that we’re at. Like, I, I yeah.
95 00:11:38.260 ⇒ 00:11:39.090 Annie Yu: Okay?
96 00:11:40.900 ⇒ 00:11:45.099 Annie Yu: One clarifying question again, I know you mentioned there are
97 00:11:45.400 ⇒ 00:12:04.749 Annie Yu: kind of, or did you? Did you mean like organic revenue and some campaigns that are generic, not specific for some products. And then there’s also campaigns for specific product. So which ones are the one that
98 00:12:05.460 ⇒ 00:12:08.109 Annie Yu: the team is not able to track.
99 00:12:09.380 ⇒ 00:12:15.950 Robert Tseng: So it’s really the ones that are like organic or non product specific.
100 00:12:16.600 ⇒ 00:12:17.580 Annie Yu: Okay.
101 00:12:17.580 ⇒ 00:12:18.200 Robert Tseng: Yeah.
102 00:12:20.160 ⇒ 00:12:22.950 Annie Yu: Okay.
103 00:12:27.920 ⇒ 00:12:35.627 Annie Yu: alright. Well, I think these are all new. So I will definitely,
104 00:12:37.640 ⇒ 00:12:40.509 Annie Yu: need to spend some more time on this.
105 00:12:45.750 ⇒ 00:12:49.690 Annie Yu: okay, so
106 00:12:55.090 ⇒ 00:12:58.729 Annie Yu: okay, no. I think that makes sense.
107 00:13:00.980 ⇒ 00:13:03.480 Annie Yu: Okay, I think I’m just gonna just
108 00:13:05.500 ⇒ 00:13:08.770 Annie Yu: spend some time on it and then let you know if I
109 00:13:09.240 ⇒ 00:13:13.569 Annie Yu: like any, any other help, but I think there’s already enough
110 00:13:13.890 ⇒ 00:13:20.694 Annie Yu: for me to learn. I mean on on this ticket. So it’s not like I I won’t have anything to do, so.
111 00:13:20.990 ⇒ 00:13:27.860 Robert Tseng: Yeah, yeah, I’m not asking you to like produce anything. Wherever by end of the week I just want you to kind of spend some time like learning, investigating.
112 00:13:27.860 ⇒ 00:13:28.180 Annie Yu: Yeah.
113 00:13:29.260 ⇒ 00:13:38.389 Robert Tseng: I’ll try to like, write, do do some more documentation on, like what we’re trying to accomplish with it. I think I tried to explain it on this call. But
114 00:13:39.160 ⇒ 00:13:40.290 Robert Tseng: yeah, I think
115 00:13:44.660 ⇒ 00:13:52.499 Robert Tseng: I mean, how much are you familiar? How familiar are you with like the marketing data and stuff like I mean, I feel like you can learn it. But I just
116 00:13:52.770 ⇒ 00:13:57.530 Robert Tseng: sometimes I don’t know what terms I should use and not use, because I don’t know how much knowledge you have. Yeah.
117 00:13:57.530 ⇒ 00:13:58.983 Annie Yu: So far.
118 00:14:00.180 ⇒ 00:14:21.300 Annie Yu: well, when you select incrementality definitely, that’s new to me, I think for me, I’ve tracked like Mql click through rates. So all the like, I think the basic ones. But and like Utm tagging and tracking, that’s pretty much it, for like marketing, I also like have tracked like
119 00:14:22.950 ⇒ 00:14:30.050 Annie Yu: like what? Like targeted marketing for different persona, and also like the time of day or time of week.
120 00:14:30.180 ⇒ 00:14:30.740 Annie Yu: But that’s.
121 00:14:30.740 ⇒ 00:14:31.330 Robert Tseng: Yeah.
122 00:14:31.810 ⇒ 00:14:33.930 Robert Tseng: So I think, like a descriptive.
123 00:14:34.180 ⇒ 00:14:35.389 Annie Yu: Type of thing.
124 00:14:35.970 ⇒ 00:14:41.419 Robert Tseng: Yeah, yeah. So I mean, what you’re describing is kind of that’s what baseline attribution is. I think it’s just like
125 00:14:41.670 ⇒ 00:14:49.359 Robert Tseng: making sure that every ad or campaign that you run is like labeled properly, so that you can tie an order back to a source right.
126 00:14:49.360 ⇒ 00:14:50.080 Annie Yu: No, no.
127 00:14:50.375 ⇒ 00:15:02.469 Robert Tseng: So yeah, you’re right. It’s like descriptive statistics, pretty much. I mean, I know this is more of a stretch project, but I mean, I know you have your. You have interest in data science work. So I wanted to.
128 00:15:02.470 ⇒ 00:15:02.820 Annie Yu: Good.
129 00:15:02.820 ⇒ 00:15:05.970 Robert Tseng: Kind of like, put something that was more like
130 00:15:06.810 ⇒ 00:15:09.840 Robert Tseng: like a little bit of a harder problem, like a harder problem
131 00:15:10.550 ⇒ 00:15:17.169 Robert Tseng: and kind of learn about it. Yeah. And so like, beyond, just like us
132 00:15:17.560 ⇒ 00:15:25.824 Robert Tseng: associating orders being able to tie orders back to like the source that they came in from
133 00:15:26.940 ⇒ 00:15:32.589 Robert Tseng: like that that. Tell that helps you to cast like that’ll that’ll help you explain, like.
134 00:15:32.790 ⇒ 00:15:50.579 Robert Tseng: where, how the Roi of different marketing dollars for very specific product campaigns that you can actually associate back to. But you know, like marketing is so broad, not every, not every like campaign that you launch is going to promote a specific product, right? So some of them are going to promote just like
135 00:15:51.340 ⇒ 00:16:11.759 Robert Tseng: it’s like lifestyle, like like ads that I’m sure. You see all the time on Youtube or whatever. It’s just like brand awareness brand promotion. That’s not necessarily selling a specific product. But I would say, like, you know, they spend a lot of dollars doing that like trying to get people to know like about the Eden brand, right? And so
136 00:16:12.000 ⇒ 00:16:18.150 Robert Tseng: that’s like, that’s kind of been a challenge for a lot of companies to like know how to measure that?
137 00:16:18.250 ⇒ 00:16:34.079 Robert Tseng: Because it’s not very clear, like, okay, if I just launch a general brand campaign. How does that actually drive sales like I, I don’t know. And like which products do people buy from those campaigns? It’s not like you can tie Utm back to it, because if they click.
138 00:16:34.535 ⇒ 00:16:37.270 Annie Yu: To like traffic is that it?
139 00:16:37.890 ⇒ 00:17:05.789 Robert Tseng: Yeah. Well, like any paid traffic on utms that are tied to your product like, that’s for a product specific campaign. I think brand awareness. Campaigns don’t necessarily have like a Cta. Or if they do, it’ll just. It’ll just go to the main like landing page or something. You know. It’s it’s not like, it won’t bring them to a specific product. So you just think about like you’re watching Youtube, there’s like a TV ad or like a Youtube ad, and if they’re just, you know.
140 00:17:06.550 ⇒ 00:17:24.750 Robert Tseng: do you want to like lose weight like Eden Brand like, Come, check us out. Then they click on the website. It just brings them to the homepage. It’s not like the that particular ad was selling a specific product. It was just like getting you to think about the Eden brand, you know.
141 00:17:26.040 ⇒ 00:17:29.910 Robert Tseng: So it’s, you know.
142 00:17:30.400 ⇒ 00:17:40.369 Robert Tseng: so that I think that’s like 1 1 like area of like marketing. Spend that like we’re trying to measure with this? But then, also.
143 00:17:41.100 ⇒ 00:17:59.299 Robert Tseng: yeah, like, you have organic traffic that’s coming in from the search terms, the SEO stuff that you’re doing. Maybe. Yeah, just being in the market. Longer, your brand is more well known. And so people just naturally come to your site more often. We don’t really know what that baseline is, either like.
144 00:18:00.470 ⇒ 00:18:04.039 Annie Yu: And how? How is it that we can’t track that?
145 00:18:04.620 ⇒ 00:18:06.819 Annie Yu: Not through like or.
146 00:18:06.820 ⇒ 00:18:16.380 Robert Tseng: You can use Google, search, console to kind of get a gauge for like organic baseline. Yeah, that’s like, that’s true. But I think, like running
147 00:18:16.910 ⇒ 00:18:30.925 Robert Tseng: like a geolift study. You should look at the Geolift one. I think that’s a it’s like a it’s pretty well documented. Facebook kind of released this this model here. So I think you can study the methodology kind of see what it tries to do.
148 00:18:31.810 ⇒ 00:18:44.339 Robert Tseng: yeah, I mean, we we already tried. We already designed the experiment, and, like I wrote out a bunch of stuff for Bo to kind of like try to tackle. But he didn’t end up getting around to it. So I’m hoping that I’ll be able to work with you on it.
149 00:18:45.920 ⇒ 00:18:46.780 Robert Tseng: yeah.
150 00:18:47.210 ⇒ 00:18:58.457 Robert Tseng: And then last thing I’ll say is like to break down the increment like what incremental does. And with these different steps. So yeah, you know, they only look at a few things they look at
151 00:18:58.930 ⇒ 00:19:00.869 Robert Tseng: ad spend by channel.
152 00:19:01.060 ⇒ 00:19:28.429 Robert Tseng: And they look at like your orders by date. Right? So it’s like, daily ad spend by channel, and also like daily orders. And yeah, they like, talk about how they bring in all these other factors, weather patterns, stock market trends like whatever like these are just all different, publicly available enrichment sources that how important they are. Like, I think that’s all kind of fluff to be honest, like, I think, yeah. And they’re they’re trying to infer, like.
153 00:19:29.080 ⇒ 00:19:32.050 Robert Tseng: okay, based on the
154 00:19:32.210 ⇒ 00:19:50.339 Robert Tseng: the trends that you’ve been based on the trends for each of the ad spend channels. How does that like correlate with like the trend of the orders that are being placed right? So obviously, like, if you turned on Facebook and you were increasing budget by 20% there.
155 00:19:50.340 ⇒ 00:20:04.189 Robert Tseng: And you’re seeing like a 20% increase in orders. Then the algorithm will wait. Facebook to probably be like, yeah, this channel is probably driving like the growth here or whatever it’s. That’s a very simplified way of doing it.
156 00:20:04.280 ⇒ 00:20:12.709 Robert Tseng: But yeah, it’s just like trying to use various like time series models to like, have some sort of causal inference for, like
157 00:20:12.820 ⇒ 00:20:21.769 Robert Tseng: ad spend in across channels to like orders. So I wouldn’t be like overwhelmed by what they’re trying to what what they claim they do.
158 00:20:22.190 ⇒ 00:20:26.580 Robert Tseng: I think they’ve overcomplicated. It is my kind of take.
159 00:20:26.720 ⇒ 00:20:54.189 Robert Tseng: I feel like we could. We could build something that’s just like running the Geolift test or implementing the profit model. That’ll probably do like the same thing that it does, I’m sure, like there’s some nuance to how they find to the model and stuff, and we can learn from that. But I think that’s where I find that there’s it’s valuable for us to run this exercise as well. So that we can kind of just
160 00:20:54.620 ⇒ 00:20:56.430 Robert Tseng: keep that. Yeah, like.
161 00:20:56.900 ⇒ 00:21:03.780 Robert Tseng: eventually, like, yeah, so that we know we know how to do this type of measurement for the client, and also future clients.
162 00:21:04.450 ⇒ 00:21:05.080 Annie Yu: Okay.
163 00:21:05.300 ⇒ 00:21:06.240 Robert Tseng: Yeah, yeah.
164 00:21:06.940 ⇒ 00:21:10.067 Robert Tseng: So I know it’s kind of niche. But yeah.
165 00:21:11.270 ⇒ 00:21:19.999 Annie Yu: The kind of the output for this one. If, I said, I think we can do this one, it doesn’t have to mean I am capable of doing this right.
166 00:21:20.000 ⇒ 00:21:25.279 Robert Tseng: Yeah, yeah, no, totally not. Yeah. I mean, I give you a screenshot for like what I think like it.
167 00:21:25.600 ⇒ 00:21:41.360 Robert Tseng: that’s the output, the incremental, does they? They pay like 20 grand 20 grand or something a month for this tool, just so that they can see this pivot table by channel. And it’s like, Okay, that’s the spend. And then that’s like how much money you get from each channel. Like.
168 00:21:41.740 ⇒ 00:21:45.109 Robert Tseng: I mean, who really knows? I think that’s just what. But that’s
169 00:21:45.920 ⇒ 00:21:52.239 Robert Tseng: that’s what they. That’s how they make their business. So I’m I’m just wanting to see, like.
170 00:21:52.800 ⇒ 00:21:59.379 Robert Tseng: yeah, we don’t have to build it out entirely. But, like, what can can we do? Something that’s like a step in that direction?
171 00:21:59.620 ⇒ 00:22:02.719 Annie Yu: You know, and so we’ll like kind of like.
172 00:22:03.630 ⇒ 00:22:08.079 Annie Yu: like a B testing. Be like an option for the team at all or not.
173 00:22:09.252 ⇒ 00:22:11.680 Robert Tseng: I guess. What do you mean by a B testing.
174 00:22:11.680 ⇒ 00:22:16.735 Annie Yu: Like if we do like, because we can always just
175 00:22:18.810 ⇒ 00:22:27.950 Annie Yu: I don’t know the right word, but to like, get the ads to a sample a group, and then don’t do that with the other group, and then compare.
176 00:22:28.610 ⇒ 00:22:39.849 Robert Tseng: Yeah. So yeah, I mean, I think that’s also one of the risks with the Geolift study, because it requires you to turn off certain like ads in order to like, see?
177 00:22:40.210 ⇒ 00:23:02.469 Robert Tseng: You basically turn off an ad for 2 weeks, and then you turn it back on and just see what happened like, did did things actually drop like, did sales actually drop or not? And like that kind of helps, the model kind of learn from like the impact of that. So, yeah, I mean, we we have to, you know, if we have like a well, if we design the experiment, and we can give instructions to
178 00:23:03.100 ⇒ 00:23:25.499 Robert Tseng: the the marketing team like we know what are their biggest campaigns and stuff. So maybe we pick a campaign that’s like, not as high budget and high impact. And we just run the test there. That could be like a lower risk way to like, you know, test like our approach. And then, you know, if it’s valuable, then we can roll it out to something bigger. So I can like help guide like once you’ve
179 00:23:25.790 ⇒ 00:23:40.379 Robert Tseng: once you have a clear kind of this like way to design the methodology, I can tell you, like which campaigns we should run it on 1st and like make the rollout a bit smoother. So I think that’s that’s kind of where I feel like my my role in this would be.
180 00:23:40.880 ⇒ 00:23:47.889 Annie Yu: Okay. Yeah, I’ll I’ll I’ll spend some time there and then start. We’ll go from there.
181 00:23:47.890 ⇒ 00:24:02.710 Robert Tseng: Yeah. Yeah. And if you if you do it, and you’re just like I have no idea or like, I don’t wanna do this like. That’s fine. You can just tell me like I. But I’m just like looking out for, like more interesting projects here and there to try to push towards you towards you to see which ones you would like.
182 00:24:04.220 ⇒ 00:24:10.190 Robert Tseng: yeah, I know your background is more in like voc and stuff. So I want to be heading that direction, but.
183 00:24:10.190 ⇒ 00:24:18.948 Annie Yu: I think this is this is, gonna be interesting. I I am definitely more interesting, like predictive modeling on all that. So I feel like this has part of that
184 00:24:19.370 ⇒ 00:24:24.787 Annie Yu: so it would be like I. I can learn something either way. I think.
185 00:24:25.610 ⇒ 00:24:35.540 Robert Tseng: Alright, yeah. So that’s that’s what it is. Yeah, let me know if you have any other questions. But yeah, I think this is more of a stretch kind of project, I suppose.
186 00:24:35.540 ⇒ 00:24:42.559 Annie Yu: I’ll I’ll have fun. I’ll have fun with it. And and hopefully we get to a better place with my.
187 00:24:42.804 ⇒ 00:24:43.049 Robert Tseng: Right.
188 00:24:43.050 ⇒ 00:24:45.380 Annie Yu: Okay, cool. Thank you. Robert.
189 00:24:45.560 ⇒ 00:24:46.580 Robert Tseng: Talk to you later.