Meeting Title: Data Analysis Office Hours Date: 2025-10-14 Meeting participants: Robert Tseng, Mustafa Raja, Henry Zhao, Casie Aviles
WEBVTT
1 00:01:25.200 ⇒ 00:01:26.080 Mustafa Raja: Hmm.
2 00:01:27.840 ⇒ 00:01:29.500 Robert Tseng: Hey, Mustafo.
3 00:01:30.670 ⇒ 00:01:31.420 Mustafa Raja: How are you?
4 00:01:32.630 ⇒ 00:01:34.200 Robert Tseng: Doing well, how are you?
5 00:01:34.990 ⇒ 00:01:35.810 Mustafa Raja: Yeah.
6 00:01:36.000 ⇒ 00:01:37.039 Mustafa Raja: I’m doing good.
7 00:01:51.490 ⇒ 00:01:55.399 Robert Tseng: I guess you’ve been doing some of the data work for a few weeks now.
8 00:01:55.790 ⇒ 00:02:01.249 Robert Tseng: How do you feel like it’s going? Are you… I guess… I know you started off boring, kind of a…
9 00:02:01.390 ⇒ 00:02:06.990 Robert Tseng: AI engineer role here, and then I guess you’ve taken on new scope, how does that benefit you?
10 00:02:07.900 ⇒ 00:02:15.490 Mustafa Raja: Yeah, so mostly what I’ve been doing is analysis and dashboarding, and yeah, that has been pretty good.
11 00:02:16.000 ⇒ 00:02:22.409 Mustafa Raja: I have a… I have a polyatomic sink to create in Omni, so this is a new thing for me.
12 00:02:22.850 ⇒ 00:02:24.330 Mustafa Raja: Oh, cool.
13 00:02:25.050 ⇒ 00:02:29.080 Mustafa Raja: Yeah, the data, I’ve been very interested in it, and…
14 00:02:29.570 ⇒ 00:02:34.269 Mustafa Raja: Maybe over the weekends, I’ll put some more time in learning new things about it.
15 00:02:34.790 ⇒ 00:02:35.410 Robert Tseng: Yeah.
16 00:02:35.840 ⇒ 00:02:39.820 Robert Tseng: Okay, cool. I mean, I think if you… I feel like you have a good,
17 00:02:41.040 ⇒ 00:02:46.500 Robert Tseng: Deads for, like, how to approach things, so, yeah, like,
18 00:02:47.570 ⇒ 00:02:54.920 Robert Tseng: for somebody who’s new to data, I feel like you’re doing, like, I feel like you’re… you do a pretty good job.
19 00:02:55.560 ⇒ 00:03:00.820 Robert Tseng: Yeah, I feel like your technical skills, obviously, there. You know, if anything, it’s just like…
20 00:03:01.420 ⇒ 00:03:11.780 Robert Tseng: knowing what, like, how much is enough, and like, how do you tell a story with the data, that I feel like, you know, you’ll, you’ll, you’ll get more experience on over…
21 00:03:11.780 ⇒ 00:03:12.230 Mustafa Raja: Yeah.
22 00:03:12.230 ⇒ 00:03:13.900 Robert Tseng: as you’re in anymore. Yeah.
23 00:03:13.900 ⇒ 00:03:31.469 Mustafa Raja: Yeah, yeah, the last, last call we had, for this really helped me do the analysis for, default. What they are doing is, comparing the data vendors they want to, partner up with for their internal product.
24 00:03:33.160 ⇒ 00:03:40.889 Mustafa Raja: And yeah, that last meeting then helped, in that, so I was able to then curate a story in MotionDoc.
25 00:03:41.140 ⇒ 00:03:45.210 Mustafa Raja: A few of the vendors that I tested, so that’s helpful.
26 00:03:45.920 ⇒ 00:03:48.100 Robert Tseng: Amazing. Yeah, love to hear that.
27 00:03:48.510 ⇒ 00:03:50.660 Robert Tseng: Yeah, I guess,
28 00:03:50.830 ⇒ 00:03:56.260 Robert Tseng: Hey, Casey and Henry, yeah, we were just kind of catching up a little bit before you guys joined.
29 00:03:56.860 ⇒ 00:04:04.380 Robert Tseng: I’m probably just gonna jump into it for today, because we’re gonna wrap this up a bit earlier, at, like.
30 00:04:04.570 ⇒ 00:04:06.180 Robert Tseng: 3.30 Eastern.
31 00:04:06.430 ⇒ 00:04:18.530 Robert Tseng: we’re supposed to have, some… some client calls, so… I’m not, like, confident that guy will actually join and confirm the invite, but I’m still gonna just jump off of my bed to see if he’ll,
32 00:04:18.579 ⇒ 00:04:29.749 Robert Tseng: If you’ll… if you’ll come back to it. So, yeah, I guess, for today’s session, I will do just kind of a short walkthrough of, like, the analysis that I kind of…
33 00:04:29.950 ⇒ 00:04:32.650 Robert Tseng: recently built for, Insomnia.
34 00:04:32.990 ⇒ 00:04:35.080 Robert Tseng: So I’ll kind of talk through…
35 00:04:35.630 ⇒ 00:04:40.369 Robert Tseng: The outline of, like, you know, how we even kind of got to…
36 00:04:41.510 ⇒ 00:04:45.389 Robert Tseng: the question that I’m trying to answer, and then kind of,
37 00:04:46.490 ⇒ 00:04:50.069 Robert Tseng: what I was thinking going into it, and…
38 00:04:50.510 ⇒ 00:05:02.339 Robert Tseng: I did have to pivot along the way, so I think that’s also part of the analysis journey, too, so I wanted to, like, speak to some of the nuances there. And then, obviously, I’ll, like, talk about the insights. So,
39 00:05:02.760 ⇒ 00:05:10.350 Robert Tseng: Yeah, I felt like this was pretty fresh in my mind, and I need to present this soon anyway, so I thought it’d be good to run through it with you guys.
40 00:05:12.220 ⇒ 00:05:18.360 Robert Tseng: Okay, so with that, I will, my screen…
41 00:05:21.320 ⇒ 00:05:26.200 Robert Tseng: Yeah, I mean, I guess, Casey and Mustafa have some context on…
42 00:05:26.540 ⇒ 00:05:30.650 Robert Tseng: insomnia already, and Henry, you do not, but, you know, it’s…
43 00:05:30.870 ⇒ 00:05:38.330 Robert Tseng: you understand CPG businesses and stuff, so I don’t really have to go into too much. The driving question that we’re, you know, that’s…
44 00:05:38.680 ⇒ 00:05:46.579 Robert Tseng: that I felt like the leadership has been asking consistently is, like, What is driving…
45 00:05:46.740 ⇒ 00:06:03.100 Robert Tseng: like, a softness in owned channel performance. Owned channel meaning email, SMS, and push notifications. So, these are channels that we’re… that our clients are using to
46 00:06:03.290 ⇒ 00:06:15.010 Robert Tseng: retarget existing customers, or at least people who have signed up for newsletters and given us access to some of their information. So, this is not like paid media spend, where we’re
47 00:06:15.390 ⇒ 00:06:24.879 Robert Tseng: you know, buying more ad space on, like, Meta or TikTok or whatever it is to go and acquire new customers.
48 00:06:25.000 ⇒ 00:06:29.829 Robert Tseng: And… Yeah, so I think there are a lot of different ways to approach this question. I think…
49 00:06:30.030 ⇒ 00:06:47.740 Robert Tseng: how I even got to this question. You know, we went through a few different exercises. One was kind of calling out that there was some softness, and so we kind of outlined some analysis here. Mustafa, you’ve helped out on some of these tickets, kind of, like, drilling into, okay, well, we can see, like.
50 00:06:47.930 ⇒ 00:06:57.640 Robert Tseng: Overall, when we’re looking at things like the marketing performance tracker, like, and if we just exported this spreadsheet and we looked at
51 00:06:59.840 ⇒ 00:07:07.470 Robert Tseng: I guess somebody’s in here filtering by date, but if you exported, like, daily, kind of performance
52 00:07:07.720 ⇒ 00:07:18.080 Robert Tseng: since, I guess, this data goes all the way back to January of 2024, you would be able to see it pretty clearly on a line chart that, like, revenue has dipped, or whatever.
53 00:07:18.860 ⇒ 00:07:24.509 Robert Tseng: But, like, that doesn’t really give us, much insight, and I think…
54 00:07:24.620 ⇒ 00:07:29.100 Robert Tseng: something that I want to emphasize here is, like, yeah, I think…
55 00:07:29.100 ⇒ 00:07:51.210 Robert Tseng: you know, for the first few weeks of us being on this client, we were… and we still are, kind of, like, at the… at the surface, where we’re just talking about, like, bigger trends, and we’re not able to break down the analysis much more. Something that I learned when I started, kind of, more in, like, a consulting role before, it was an internship I did, like, back in college, like.
56 00:07:51.480 ⇒ 00:07:56.609 Robert Tseng: I just remember, my manager specifically telling me, like.
57 00:07:57.000 ⇒ 00:08:11.189 Robert Tseng: averages are misleading, you know, or, like, they don’t… they don’t really tell you much. Like, be very skeptical of averages. So, I feel like that’s one kind of heuristic I always kind of have in the back of my mind. There’s always more of a story to be told than the averages.
58 00:08:11.420 ⇒ 00:08:12.490 Robert Tseng: Because…
59 00:08:12.490 ⇒ 00:08:36.619 Robert Tseng: Yeah, I think the average kind of tells you, like, general, like, trends, which I… which we’ll kind of… we’ll talk through, but it doesn’t tell you anything about what’s driving those averages. And so, you know, without looking to the data too much, like, we know what the different cuts that we could make are, especially for a product like… a company like Insomnia, right? So, you know, Mustafa, you’re working on something right now that’s relevant to this.
60 00:08:36.620 ⇒ 00:08:42.890 Robert Tseng: you’re helping me to create the levels of, like, granularity that we need to be able to drill into, right? So we’re looking at, like.
61 00:08:43.370 ⇒ 00:08:52.150 Robert Tseng: Like, sales by product. Sales by segment. RFM segment is just, like, one very, like, basic way to
62 00:08:52.360 ⇒ 00:08:58.260 Robert Tseng: Segment your existing customer base by recency, frequency, and monetary value.
63 00:08:58.400 ⇒ 00:09:03.789 Robert Tseng: So it’s, like, a very rudimentary way of, like, trying to kind of dis…
64 00:09:03.790 ⇒ 00:09:23.539 Robert Tseng: figure out who your highest value customer segments are. I think it’s a pretty outdated, approach, to be honest. Like, I… I think, you know, Facebook has, like, an out-of-the-box, like, model called Profit that’s pretty good at creating customer segments, and what it does better than the RFM model is that it actually bakes into
65 00:09:23.540 ⇒ 00:09:36.900 Robert Tseng: bakes, like, seasonality into it, and, like, other, like, external macro factors that are more than just this kind of three, three, variable model. So, it’s… it’s obvious to me that, you know, where the
66 00:09:37.070 ⇒ 00:09:56.309 Robert Tseng: I guess I wish I could zoom into this, but where Insomnia currently is at with how they set up their customer segments is very basic, and it doesn’t really tell them very much. I can’t really zoom in too closely, but you can tell that, like, even for their, like, highest value segment right now, what they’re calling champions.
67 00:09:56.310 ⇒ 00:10:04.609 Robert Tseng: 80% of their revenue, and it’s, like, 30% of their customer base. You know, if your highest value segment is, you know.
68 00:10:04.640 ⇒ 00:10:12.459 Robert Tseng: 30% of your customer base and, you know, 80% of your revenue? I don’t know, like, that to me is not that helpful, because it’s like…
69 00:10:13.310 ⇒ 00:10:15.599 Robert Tseng: Like, there’s…
70 00:10:15.690 ⇒ 00:10:31.689 Robert Tseng: yeah, you want more of your customers to become like that, but usually I would expect that, like, kind of cream of the crop segment to be smaller and be a smaller portion of revenue, and it’s more of, like, a normal distribution, kind of referencing the histogram concept we talked about last week, where
71 00:10:31.690 ⇒ 00:10:48.040 Robert Tseng: you know, in the spread of different segments, you would expect most of them to kind of fall in the middle… in the middle segments, and then you’re finding ways to push them towards, you know, towards the top, which I think the way that they’ve laid out their segmentation, like, that’s not kind of what’s happening right now.
72 00:10:48.040 ⇒ 00:10:52.630 Robert Tseng: But, not to diverge too much from, kind of, the topic at hand.
73 00:10:53.000 ⇒ 00:11:05.249 Robert Tseng: We did diagnose, kind of, like, what was going on. I had some, like, kind of, insights that I shared last time. And so, just, I kind of teased up this analysis, and I originally had this outline where
74 00:11:05.530 ⇒ 00:11:21.679 Robert Tseng: I kind of started off just, like, I knew I wanted to kind of tell the story, kind of some hypotheses I made. So, within these slides that didn’t end up kind of taking place, the hypotheses I had, one was that I felt like, if you took
75 00:11:21.800 ⇒ 00:11:24.120 Robert Tseng: Insomnia’s email conversion rate
76 00:11:24.430 ⇒ 00:11:35.420 Robert Tseng: Because I just wanted to isolate one specific channel. Email is 80, you know, more than 80% of their… of their… actually, it’s less. It’s… it’s like 70% of owned channel revenue.
77 00:11:35.770 ⇒ 00:11:44.420 Robert Tseng: So I’m sure it behaves differently than, like, the push channel and the SMS, so I just wanted to just focus on, like, what the biggest driver
78 00:11:44.830 ⇒ 00:12:02.329 Robert Tseng: So I just focused on email, and I was, like, kind of hypothesizing. I believe that there’s probably something to be said about their email performance is just, like, not as high as industry standards, is what I would expect. So, I wanted some slide to kind of just help me visualize that and communicate that to the client.
79 00:12:04.090 ⇒ 00:12:09.449 Henry Zhao: Do you do anything to get industry standards, or do you kind of just have an idea of that?
80 00:12:09.450 ⇒ 00:12:24.290 Robert Tseng: Yeah, no, so that’s a good question. I was gonna… yeah, so I… I had a few… I have a few ideas for, like, how I would do it. So, the easiest thing is to go and get, like, industry reports. I mean, depending on who you are, how reliable they are, kind of, you know, TBD, but…
81 00:12:24.290 ⇒ 00:12:31.840 Robert Tseng: At least, like, my… I’ve worked with vendors before, like Braze. They have, like, you know, 150,000, 200,000 customers.
82 00:12:31.840 ⇒ 00:12:39.960 Robert Tseng: They should know, like, all the food and beverage, like, customers within their portfolio, and they should be able to give me some, like.
83 00:12:40.220 ⇒ 00:12:46.360 Robert Tseng: benchmark from their internal data. If we had a good relationship with your account executive, we could ask for that.
84 00:12:46.700 ⇒ 00:13:02.289 Robert Tseng: I’ve done that for other tools before. I’ve done that specifically in the logistics sector, and getting returns data, in my shipping… in my shipping data, stuff like that. So, usually, your vendors are pretty okay with sharing that type of anonymized, data.
85 00:13:02.310 ⇒ 00:13:15.709 Robert Tseng: unfortunately, I don’t know who our rep is with Insomnia, and, like, whatever, I asked for them to get it, but it’s been over a week, and they haven’t gotten to me yet. So, I had to kind of look for other ways. So, the other way…
86 00:13:15.710 ⇒ 00:13:24.489 Henry Zhao: As we grow, it might be cool to have, like, a Notion or a database where we have, like, by industry, like, industry standards of things that we commonly analyze. Might be cool.
87 00:13:24.490 ⇒ 00:13:29.370 Robert Tseng: Yeah, I have that already for other sectors, so…
88 00:13:29.520 ⇒ 00:13:48.990 Robert Tseng: I guess we haven’t had any mobile app clients in a while, but, like, I kind of put together a doc that’s, like, all the benchmarks that I would expect to see on, like, product analytics for mobile apps. So yeah, like, I did put together something like that, but yes, it’s… you know, ideally, we would curate our own, like, benchmarks for… across all of this
89 00:13:48.990 ⇒ 00:13:51.070 Robert Tseng: All of these different industries.
90 00:13:51.070 ⇒ 00:13:56.229 Robert Tseng: But yeah, because I did not have it, I kinda had to, like, do something kind of…
91 00:13:56.350 ⇒ 00:14:15.149 Robert Tseng: I mean, it’s not great. Like, it’s not what I wanted to show. I just, like, Klaviyo released some data, they said that their conversion rates are 1.7% in 2025. I could not find 2024, so it’s just a straight curve. So, obviously, the client’s like, well, how do you really know that, like, things were actually underperforming?
92 00:14:15.210 ⇒ 00:14:16.580 Robert Tseng: And, you know.
93 00:14:17.190 ⇒ 00:14:31.899 Robert Tseng: I do know this, but, like, I cannot show it visually. Like, I think every kind of report out there is basically saying the same thing, where email conversion rates continue to decline, and there’s been a decline, especially from 2024 to 2025.
94 00:14:32.110 ⇒ 00:14:40.370 Robert Tseng: So, I mean, like, I’ll be able to, like, kind of couch your expectations around this, so I think my hypothesis is right. Like, I think you can see…
95 00:14:40.410 ⇒ 00:14:54.720 Robert Tseng: looking at the data from January 2024 through, kind of, to date, yeah, like, conversion rates have never kind of returned to what they were in early January… in early 2024. And what we are seeing is that, kind of, like, month to month.
96 00:14:54.720 ⇒ 00:15:03.210 Robert Tseng: there’s, like, a dip, or there’s, like, a spike and dip around, kind of, March and April 20, March and April time, that seems to be consistent year over year.
97 00:15:03.470 ⇒ 00:15:12.760 Robert Tseng: Now, obviously, I’m only looking at 2 years’ worth of data. This is… I mean, I’m sure the company has been around for longer, but they have not been with Braves that long, so they don’t really have that much more data.
98 00:15:12.940 ⇒ 00:15:28.320 Robert Tseng: So I would say this is a pretty, like, level one type of macro trend insight. If I were to make this better, I would get the monthly performance data from Brace, and I’d be able to map it month to month, and then I would look at correlation to see how closely our curve
99 00:15:28.470 ⇒ 00:15:36.489 Robert Tseng: actually matches the curve that Brace gives us. And that will give me a better sense of, like, whether or not it’s an appropriate benchmark.
100 00:15:36.800 ⇒ 00:15:46.580 Robert Tseng: But then there’s also limitations to this, because we don’t know, are they really the same size as Insomnia? Obviously, like, a smaller company is going to have different,
101 00:15:46.670 ⇒ 00:16:02.720 Robert Tseng: kind of performance to, like, a larger company. So there’s, you know, I would just take this kind of stuff with a grain of salt. We’re just trying to let… have some sort of explanation for, like, some external factors that are not within our control on how to set our expectations on this particular metric.
102 00:16:04.060 ⇒ 00:16:09.180 Robert Tseng: And then the other call-out I put here was that, hey, actually in Klaviyo’s database, like.
103 00:16:09.870 ⇒ 00:16:24.859 Robert Tseng: top 10% of their customers, they’re seeing conversions of upwards of 4.5%. And I kind of know this from, like, my other experience in DEC. I think the top performers on lifecycle, yeah, they’re hitting consistently over 4%. Like, that’s what I would expect.
104 00:16:24.860 ⇒ 00:16:34.679 Robert Tseng: So, the fact that, you know, insomnia is hovering at under 1% over the past, I don’t know, 6 months, they’re not doing well. And I think…
105 00:16:34.690 ⇒ 00:16:39.049 Robert Tseng: I can… I don’t have to do that much research to know that.
106 00:16:39.770 ⇒ 00:16:48.839 Robert Tseng: But anyway, so that’s… I’ll just pause there. This is just kind of, like, one approach to doing, like, a macro analysis, when you’re trying to, like, make sense of, like.
107 00:16:49.060 ⇒ 00:17:02.069 Robert Tseng: well, you’re seeing this internal performance data, well, how do you actually, communicate to a stakeholder, is this good or bad? Like, are we, you know, should they be worried, or should they, should, like, what should it mean to them?
108 00:17:04.020 ⇒ 00:17:05.460 Robert Tseng: Any questions on this one?
109 00:17:08.460 ⇒ 00:17:09.440 Henry Zhao: Not really.
110 00:17:09.819 ⇒ 00:17:11.199 Robert Tseng: Okay, cool.
111 00:17:11.679 ⇒ 00:17:16.659 Robert Tseng: Yeah, so then, I think kind of just going back, my next hypothesis,
112 00:17:17.259 ⇒ 00:17:23.129 Robert Tseng: And actually, I’ll just kind of go through the deck. Yeah, so then I ended up kind of just, you know, doing a little bit more, kind of fleshing out
113 00:17:23.349 ⇒ 00:17:34.039 Robert Tseng: This is pretty straightforward, just letting them know, okay, we’re generating this much weekly revenue, sending this many messages across different channels, you know, most of it is driven by email. So, pretty straightforward there.
114 00:17:34.219 ⇒ 00:17:50.769 Robert Tseng: Another point that I wanted to make was that I saw this trend where as email… as we were sending more emails, or, like, and volume continued to increase, or kind of stay the same, the revenue continues to dip. So, there’s something about this, like, efficiency of, like, I don’t know, like.
115 00:17:50.769 ⇒ 00:18:05.609 Robert Tseng: dollar, percent, or, you know, whatever you want to call it, but I just call it email revenue efficiency. That’s clearly dropping, like, over… over time. So, it’s like sending, you know, the same number of emails no longer kind of drives the same number of… same… same dollar amount.
116 00:18:05.609 ⇒ 00:18:14.199 Robert Tseng: Which, intuitively, that probably makes sense to, you know, some of us, and I think where I would like to improve this analysis is
117 00:18:14.249 ⇒ 00:18:36.019 Robert Tseng: this is kind of where I would want, you know, to have the better customer segments. I would want to segment messages sent by, you know, customers that… and start to do a cohort… a cohorted version of this, where we’re not just looking at, you know, all revenue generated across messages from all customers, but I’m able to kind of break it out by
118 00:18:36.609 ⇒ 00:18:47.879 Robert Tseng: you know, new customers in Q1 of 2024, Q2 2024, Q3 2024, etc. I would expect that, like, newer cohorts are more responsive and
119 00:18:47.879 ⇒ 00:18:57.359 Robert Tseng: have a higher efficiency than the later cohorts. But I would like to know, is it that… is it, like, customers that have been getting our emails for…
120 00:18:57.359 ⇒ 00:19:13.459 Robert Tseng: three months start to, like, fade, like, in their efficiency, or does it, like, all burn out within the first month, and these are mostly new customers? I think, from what I saw in the data, these are not mostly new customers. Like, they don’t get that many new customers every… every month.
121 00:19:13.459 ⇒ 00:19:17.999 Robert Tseng: At least, like, on their email list. So, a lot of it is… this is probably a mix of both.
122 00:19:18.929 ⇒ 00:19:31.999 Robert Tseng: And so, yeah, I think that’s kind of, like, something that I’m calling out to them as well. And what I want… what I’m going to share with them as a next step, well, where do you get the most leverage? Like, if we assume that
123 00:19:32.499 ⇒ 00:19:43.329 Robert Tseng: you’re gonna get the most revenue out of your newest customers. That’s why onboarding or new customer, like, life cycle flows are the most important, in terms of, you know, that’s…
124 00:19:43.329 ⇒ 00:20:06.289 Robert Tseng: that’s the window where you’re going to really be able to get the most value, out of your customer, and so that’s kind of where you would focus on. Otherwise, like, life cycle, like, I don’t know, who do you go after? The people that have been with you for 2 plus years, or the people that have been with you for less than a month? Like, it kind of is… it’s too wide of a range, and so my default, and I think this is a common principle across
125 00:20:06.319 ⇒ 00:20:20.699 Robert Tseng: any product company as well, even in product analytics, is to go after the new customers, the new subscribers, the new users, and try to, you know, they have the highest potential of them not fully realizing their LTV,
126 00:20:20.709 ⇒ 00:20:40.709 Robert Tseng: there’s opportunity to really, like, influence them the most early… early in the journey. So, I think that’s usually kind of, like, where I would tackle first. So, I think that’s kind of where I’d like to steer the Insomnia team to really focus on… on their early flows that they reach… that they reach their, their early customers with.
127 00:20:40.899 ⇒ 00:20:44.439 Robert Tseng: Yeah, any questions on this one?
128 00:20:50.049 ⇒ 00:21:02.899 Robert Tseng: Okay, yeah, and then I think this is pretty straightforward too, but, you know, this is just another view of the email efficiency, so I’m kind of just, like, putting on a scatter plot, trying to look at the R-square, just trying to better understand
129 00:21:03.059 ⇒ 00:21:11.459 Robert Tseng: Okay, well, like, how strong is, the relationship between, revenue and messages sent?
130 00:21:11.599 ⇒ 00:21:29.859 Robert Tseng: And the answer is it’s not very strong. It’s, like, it’s moderate. There is some relationship. Obviously, like, volume has to tie to revenue in some way, but if you look at all of these different bubbles that are kind of above the curve, most of them are from 2024. There’s very few that are from 2025. So it’s not like they’re not hitting anything that’s, like.
131 00:21:29.859 ⇒ 00:21:32.859 Robert Tseng: They’re not sending any… they are sending campaigns that are still, like.
132 00:21:32.889 ⇒ 00:21:46.059 Robert Tseng: hitting well, generating good revenue, but it’s not very consistent, and it seems like it’s… they’re just kind of, like, random shotting. It’s not… it’s not really… it’s clearly not that strategic or… or… or,
133 00:21:46.059 ⇒ 00:21:55.889 Robert Tseng: predictable for them. So, I think my goal is to be able to, you know, buy… if you kind of imagine changing the…
134 00:21:56.309 ⇒ 00:22:08.439 Robert Tseng: changing the, the legend here, and instead of by month, if I were able to really nail campaign types, and I can show that different campaign types are,
135 00:22:08.689 ⇒ 00:22:22.889 Robert Tseng: And if there’s a time component to this, too, so maybe the visualization actually has to change. But the idea is, like, I would like this team to get to a place where they can launch a type of campaign and kind of know, like, what…
136 00:22:23.399 ⇒ 00:22:28.989 Robert Tseng: I… Is it gonna generate all the revenue, like, in the first, in the first…
137 00:22:29.369 ⇒ 00:22:34.599 Robert Tseng: 80% of the revenue in the first day, or kind of, like, is it gonna be, like, later on in the week?
138 00:22:35.499 ⇒ 00:22:44.989 Robert Tseng: And with lifecycle campaigns, I think what’s… why this is an interesting problem to me is that the, yeah, like, the, campaign…
139 00:22:44.989 ⇒ 00:22:55.749 Robert Tseng: duration is so short. It’s only one week. Generally speaking, you’re looking at 7-day attribution. So, as opposed to other types of marketing campaigns, paid media is usually 30-day.
140 00:22:55.749 ⇒ 00:23:14.689 Robert Tseng: offline, where you’re, like, I don’t know, billboard or, like, mailers or whatever, that’s pretty hard to measure. People can do up to, like, 30 to 60 days or whatever. But generally speaking, you know, when you’re launching a lifecycle campaign, like a push notification, that attribution is, like, one day or less. Like, you’re getting a notification on your phone.
141 00:23:14.689 ⇒ 00:23:24.329 Robert Tseng: you’re expected to press into it and take an action, like, pretty much immediately. So, in some way, the feedback loop is much shorter, but then you’re also having to
142 00:23:24.659 ⇒ 00:23:44.079 Robert Tseng: the, yeah, the volume of campaigns that you’re producing is much higher. So, I think, you know, there… I think there is good opportunity, and we’ll be able to get a lot of data points. Like, there’s, I don’t know, something like 4,000… 3,000 to 4,000 campaigns that were launched in just the past 2 years, which is great. Like, I do think that we can really influence, like, what,
143 00:23:44.079 ⇒ 00:23:47.629 Robert Tseng: What this chart and the future versions of this chart will end up looking like.
144 00:23:47.729 ⇒ 00:23:52.869 Robert Tseng: From a… from a revenue efficiency perspective. Yeah.
145 00:23:53.039 ⇒ 00:23:58.469 Robert Tseng: Okay, any questions on, kind of, this, on the takeaway on this slide?
146 00:24:03.579 ⇒ 00:24:11.259 Robert Tseng: Okay, if not, then lastly, like, kind of this… I just do this appendix, this kind of, like, okay, well, if email sends and… or, like.
147 00:24:11.279 ⇒ 00:24:33.029 Robert Tseng: sent messages and revenue are not, like, the most highly correlated, then, like, what are some other options? Well, you know, there are, like, 15 other variables that are, you know, better than the blue dotted line is kind of, like, the same R-squared that we saw here. So yeah, there are, like, plenty of other explanatory variables that help us to predict revenue that we obviously need to look into. It’s never going to be a
148 00:24:33.029 ⇒ 00:24:38.569 Robert Tseng: one very, like, one thing causes, like, the other. Like, causation is very hard to prove in marketing.
149 00:24:38.679 ⇒ 00:24:42.289 Robert Tseng: We’re only looking really at relationships and correlation, so…
150 00:24:42.709 ⇒ 00:24:51.609 Robert Tseng: Yeah, I think with more time and more sophistication, we’ll be able to, like, group variables and be able to, like, tell, like, a more robust story of, like.
151 00:24:51.689 ⇒ 00:25:01.759 Robert Tseng: hey, actually, people who get… who get sent many messages, but they open the… they open the messages within the first… first X number of hours.
152 00:25:01.759 ⇒ 00:25:13.039 Robert Tseng: And then they also see, like, content, banners, or kind of, like, they, you know, they see a couple other touchpoints. This is kind of where you start to build out your own kind of, like.
153 00:25:13.039 ⇒ 00:25:15.619 Robert Tseng: View of, like, multi-touch attribution.
154 00:25:15.979 ⇒ 00:25:33.139 Robert Tseng: Where you’re trying to see, like, what is really the most significant relationship between customer behaviors, and reacting to the campaigns that you’re giving them, and, like, revenue, which is kind of usually what you’re optimizing for. So, I think this was really just, like, a…
155 00:25:33.139 ⇒ 00:25:36.939 Robert Tseng: teaser exercise to show them that, like, I think there are a lot of other things I want
156 00:25:37.009 ⇒ 00:25:46.879 Robert Tseng: permission to go and, like, dig into. And yeah, so that’s… that’s kind of… kind of the extent of what I’m sharing with them.
157 00:25:47.259 ⇒ 00:25:52.789 Robert Tseng: Probably the end of day today, I think there’s a few other things that they wanted to, like, dial in specifically on.
158 00:25:53.039 ⇒ 00:25:58.449 Robert Tseng: But yeah, I mean, I would say, like, kind of this deck on the right is kind of where
159 00:25:58.769 ⇒ 00:26:09.789 Robert Tseng: this is what they review on a… by… on a… every other week, and they send this to the board. Yeah, it’s like 20 slides, but a lot of it is… to me, this is not really insights. This… these are just, like.
160 00:26:09.899 ⇒ 00:26:15.979 Robert Tseng: performance, like, snapshots of, like, okay, like, what happened at X, like…
161 00:26:16.779 ⇒ 00:26:22.019 Robert Tseng: you know, we launched X campaigns, and they performed, like, whatever, you know, and
162 00:26:22.049 ⇒ 00:26:38.539 Robert Tseng: it’s, you know, it’s what I would expect from their team, and yes, Casey, like, a lot of your work is kind of helping their team to be able to, you know, as you’re probably seeing this, and like, yeah, like, all of the metrics that you’re helping us track, they’re using your resource to basically dump it into this deck.
163 00:26:38.689 ⇒ 00:26:55.609 Robert Tseng: But it doesn’t really, like, tell the story of, like, well, what should they do about it? So, I think, like, our job on the data side, we don’t have to be as, like… we’re not operations people, right? We don’t need to go and build this stuff out all the time, like…
164 00:26:55.609 ⇒ 00:27:03.709 Robert Tseng: we’re supposed to be able to step back and be able to influence the strategy of life. Okay, well, what should they do about it? Is, like, a 70%
165 00:27:03.919 ⇒ 00:27:19.179 Robert Tseng: you know, from a portfolio management perspective, 70% being overly indexed on email is not great, because if email tanks, which it does during October and November, then your revenue tanks. And so, like, how do we, like, what are the different levers that we can pull in order to extend
166 00:27:19.739 ⇒ 00:27:33.569 Robert Tseng: kind of revenue in other channels. And we can see, even just from looking at a couple examples, it seems like, you know, October to November, December, like, there’s… that’s kind of when, like, banner channels and, like, banner seems to be a higher percentage.
167 00:27:33.739 ⇒ 00:27:46.759 Robert Tseng: Maybe it’s because email dropped significantly, but it’s like, I don’t think seasonality affects every channel the same way. And so, what we’re really trying to get towards is to build, like, a kind of portfolio of different
168 00:27:46.759 ⇒ 00:27:57.889 Robert Tseng: Channels, their performance, and the levers that we can pull, so that we can stabilize growth more consistently across other channels, and be able to tell them when they should lean into one over the other.
169 00:27:57.989 ⇒ 00:28:10.869 Robert Tseng: like, from the data that I’ve seen last year, at least, like, around this time, email is not the best channel. Like, October to November, huge dip relative to other months. Maybe this is the time to really just focus on
170 00:28:11.159 ⇒ 00:28:26.489 Robert Tseng: push on SMS, and this doesn’t mean you stop email, but, like, there’s, like, it’s a kind of a waste of, you know, you’re not really gonna see, like, great returns on email at this time, at this time of year. So, I think, those are the types of, kind of.
171 00:28:26.849 ⇒ 00:28:32.809 Robert Tseng: opinions that we should be able to have, like, from examining the data. And as you guys kind of…
172 00:28:32.879 ⇒ 00:28:42.799 Robert Tseng: you guys, as in Mustafa and Casey, continue to work on this client and help me to be able to get deeper into these segments, I’ll be able to give even more specific recommendations on, like.
173 00:28:42.829 ⇒ 00:28:50.609 Robert Tseng: what products should they actually be focused on? What segment is the actual segment that they should be targeting, etc. So,
174 00:28:50.639 ⇒ 00:29:04.409 Robert Tseng: Yeah, I think, that’s… that’s kind of… sorry, I didn’t really leave that many… that much time for, questions. I mean, I can take a few, but, yeah, that’s kind of the overview of this analysis,
175 00:29:04.539 ⇒ 00:29:06.879 Robert Tseng: For, for, for Insomnia this week.
176 00:29:10.769 ⇒ 00:29:22.449 Robert Tseng: Any questions, kind of, from… I mean, whether you have about the analysis itself, kind of, like, how anything, like, methodology, how it was done, or, like, kind of things that you’re thinking about on… in the other clients that you’re working on.
177 00:29:22.749 ⇒ 00:29:31.559 Robert Tseng: That, like, maybe you could try to apply or kind of think about how this style of thinking and analysis may apply to your work?
178 00:29:36.680 ⇒ 00:29:50.899 Casie Aviles: More on just, you know, with the data that I’ve been seeing all the time, like, I guess the surface level thing that I would… I would think is that their email is, like, doing better than the other channels, but…
179 00:29:51.330 ⇒ 00:29:54.370 Casie Aviles: Actually, it’s not that. There’s more to it.
180 00:29:54.530 ⇒ 00:29:58.450 Casie Aviles: And yeah, this was definitely insightful and…
181 00:29:59.110 ⇒ 00:30:01.810 Casie Aviles: You know, and being able to understand that.
182 00:30:04.160 ⇒ 00:30:09.089 Robert Tseng: Okay, great. Yeah, I mean, I’m sure you’re seeing, yeah, email dollar amounts are definitely the highest, but, like.
183 00:30:09.090 ⇒ 00:30:09.590 Casie Aviles: Yeah.
184 00:30:09.590 ⇒ 00:30:15.499 Robert Tseng: just because it’s generating the most revenue doesn’t mean that that’s, like, the best use of, like, their efforts? Like.
185 00:30:15.620 ⇒ 00:30:31.890 Robert Tseng: it… I mean, it may not be, like, I haven’t built this view for the other channels. I mean, I would expect that it’s… the proficiency is actually better, like, I know that from some of the other exploratory work that I did. But yeah, like, it is… it is not the most efficient channel right now, so…
186 00:30:39.190 ⇒ 00:30:53.280 Robert Tseng: Cool. Yeah, so I’m… I will drop… jump in a minute or two, but yeah, I guess, for the next session, I mean, I’ll have another thing queued up, we can go more into, kind of.
187 00:30:53.490 ⇒ 00:31:09.190 Robert Tseng: we could do more… something more focused on, like, the Python kind of piece, if we want to, like, really know how we’re generating these charts and stuff, like, I’m happy to do more of a code walkthrough, and I was more focused on analysis, outlining, kind of, like, how I got to the driving question, and then adjustments I had to make along the way.
188 00:31:09.630 ⇒ 00:31:13.069 Robert Tseng: But yeah, I guess, you know, I’ll, I’ll, kind of
189 00:31:13.270 ⇒ 00:31:15.530 Robert Tseng: Draw some ideas for the next session.
190 00:31:19.640 ⇒ 00:31:21.300 Robert Tseng: Okay, cool.
191 00:31:22.070 ⇒ 00:31:27.050 Robert Tseng: Carl, thanks for… thanks for being here. Sorry to cut this one short a little bit, this time.
192 00:31:27.830 ⇒ 00:31:33.450 Robert Tseng: Yeah, I just couldn’t find another time on the client’s calendar, so we’re gonna try to see if I can get ahold of him now.
193 00:31:34.020 ⇒ 00:31:34.850 Mustafa Raja: Thank you.
194 00:31:34.850 ⇒ 00:31:36.600 Robert Tseng: Cool. Thanks, everyone. Thank you. Bye.