Meeting Title: ReadMe <> Brainforge Check-In Date: 2025-11-20 Meeting participants: Elizabeth Conference Room, Robert Tseng, Uttam Kumaran
WEBVTT
1 00:00:19.000 ⇒ 00:00:20.000 Robert Tseng: Hello!
2 00:00:24.320 ⇒ 00:00:25.560 Elizabeth Conference Room: Hello?
3 00:00:26.170 ⇒ 00:00:27.510 Elizabeth Conference Room: Hi!
4 00:00:29.700 ⇒ 00:00:30.380 Speaker 3 (Elizabeth Conference Room): here.
5 00:00:31.390 ⇒ 00:00:32.390 Robert Tseng: Yeah, I can hear you, I can hear you.
6 00:00:32.390 ⇒ 00:00:33.470 Elizabeth Conference Room: Cool.
7 00:00:34.580 ⇒ 00:00:35.700 Speaker 3 (Elizabeth Conference Room): How are you?
8 00:00:36.610 ⇒ 00:00:39.040 Robert Tseng: I’m good, I’m actually in Hong Kong, so…
9 00:00:39.040 ⇒ 00:00:40.660 Elizabeth Conference Room: Wow! Cool.
10 00:00:40.660 ⇒ 00:00:41.370 Robert Tseng: Yeah.
11 00:00:42.200 ⇒ 00:00:46.120 Robert Tseng: It’s a 16-hour flight, and got in a few hours ago, just kinda…
12 00:00:47.050 ⇒ 00:00:51.570 Robert Tseng: trying to manage things, and then I’ll probably go to bed shortly after this.
13 00:00:51.570 ⇒ 00:00:53.380 Elizabeth Conference Room: Yeah, yeah.
14 00:00:53.380 ⇒ 00:00:56.389 Speaker 2 (Elizabeth Conference Room): They’re for work or leisure?
15 00:00:56.820 ⇒ 00:01:05.650 Robert Tseng: Yeah, I’m visiting my in-laws. We usually, come here around Thanksgiving or Christmas. We alternate every year, so.
16 00:01:06.030 ⇒ 00:01:09.130 Elizabeth Conference Room: Yeah, I mean, I will be working, I just…
17 00:01:09.130 ⇒ 00:01:13.999 Robert Tseng: We usually won’t be taking that many meetings, like, in the afternoon Eastern time.
18 00:01:14.290 ⇒ 00:01:16.009 Elizabeth Conference Room: Cool. Paula, are you there?
19 00:01:17.320 ⇒ 00:01:20.849 Robert Tseng: A week? A little over a week, like, I’ll come back on Friday, next Friday.
20 00:01:21.220 ⇒ 00:01:23.080 Elizabeth Conference Room: Okay. Cool.
21 00:01:23.350 ⇒ 00:01:23.910 Robert Tseng: Yeah.
22 00:01:24.700 ⇒ 00:01:30.600 Speaker 1 (Elizabeth Conference Room): Awesome. And then it’s, are your teammates joining us, or is it just you?
23 00:01:30.600 ⇒ 00:01:31.310 Robert Tseng: Just me.
24 00:01:31.500 ⇒ 00:01:32.040 Speaker 1 (Elizabeth Conference Room): Okay.
25 00:01:32.040 ⇒ 00:01:32.690 Robert Tseng: Yeah.
26 00:01:32.970 ⇒ 00:01:34.710 Elizabeth Conference Room: Cool.
27 00:01:34.719 ⇒ 00:01:40.458 Speaker 1 (Elizabeth Conference Room): Well, thanks for sending over the notion, and also pulling together the amplitude dash.
28 00:01:40.619 ⇒ 00:01:51.539 Speaker 1 (Elizabeth Conference Room): I think we should just go pull up both deliverables and just start walking through them. Sure. Let’s start with the Notion doc, since the funnel is a top priority for us.
29 00:01:51.779 ⇒ 00:01:53.119 Speaker 1 (Elizabeth Conference Room): And that’s live.
30 00:01:53.120 ⇒ 00:01:54.730 Robert Tseng: God, we got through that.
31 00:01:54.970 ⇒ 00:01:55.700 Speaker 1 (Elizabeth Conference Room): Yeah, that’d be.
32 00:01:55.700 ⇒ 00:01:57.520 Robert Tseng: Sorry. Yeah, okay.
33 00:01:57.950 ⇒ 00:02:01.890 Robert Tseng: I will do…
34 00:02:03.340 ⇒ 00:02:04.110 Speaker 1 (Elizabeth Conference Room): Goodbye.
35 00:02:05.120 ⇒ 00:02:06.346 Robert Tseng: This… hmm…
36 00:02:06.960 ⇒ 00:02:08.840 Speaker 1 (Elizabeth Conference Room): from the Instagram.
37 00:02:09.009 ⇒ 00:02:11.039 Robert Tseng: Okay, let’s do it.
38 00:02:12.169 ⇒ 00:02:13.349 Robert Tseng: Here we… oof.
39 00:02:13.699 ⇒ 00:02:17.969 Robert Tseng: Great. Is that okay? Can you see that?
40 00:02:17.970 ⇒ 00:02:18.360 Speaker 1 (Elizabeth Conference Room): Yeah.
41 00:02:18.360 ⇒ 00:02:20.520 Robert Tseng: Let me see who minimize that…
42 00:02:21.340 ⇒ 00:02:34.729 Robert Tseng: Great. So, okay, so the, you know, we were always kind of referencing this, like, four-step funnel, and kind of, like, verifying each of these steps, and so we’ve finally been able to do that. Couple call-outs here, so…
43 00:02:34.790 ⇒ 00:02:43.680 Robert Tseng: I trust the number of projects that are being created, that… the projects created in Amplitude I trust. Subscription success, I trust in Amplitude.
44 00:02:43.870 ⇒ 00:02:47.230 Robert Tseng: So I think those… that’s the good news.
45 00:02:47.570 ⇒ 00:02:53.730 Robert Tseng: I think, like, around signups, I think it’s just, like, a matter of maybe we’re not able to, like.
46 00:02:53.980 ⇒ 00:03:12.929 Robert Tseng: pattern match the right filters from Mongo to Amplitude, and so we’re off by, you know, 60% or whatever. I think it’s actually lower. Once we add the email verification, I’m able to bring that down to 5K, so, you know, whatever that top of funnel metric of, like, user signups is, like, we’re not exactly matching Amplitude, but
47 00:03:12.950 ⇒ 00:03:17.519 Robert Tseng: I, you know, I think maybe we just need to figure out, like, what else
48 00:03:17.690 ⇒ 00:03:33.199 Robert Tseng: like, Mongo is capturing more than amplitude, which is surprising. I thought it would be the inverse. And so we kind of have to see, you know, if there is anything else that we need to consider on the Mongo side that, like, would, would help get this number closer.
49 00:03:33.800 ⇒ 00:03:34.990 Elizabeth Conference Room: And that covers.
50 00:03:34.999 ⇒ 00:03:45.309 Speaker 2 (Elizabeth Conference Room): Is there anything… is there anything we can do to help, figure that out? Because that’s, I mean, I hear you that there’s, like, you have some ideas, but it’s a huge delta.
51 00:03:47.070 ⇒ 00:03:54.490 Robert Tseng: Yeah, so, I mean, I would say I should really say this is more like 5,000, so what, this delta is more like…
52 00:03:55.250 ⇒ 00:03:59.950 Robert Tseng: Yes. Yeah, it’s still, like, you know, some, some, something.
53 00:03:59.950 ⇒ 00:04:00.490 Speaker 2 (Elizabeth Conference Room): Yeah.
54 00:04:00.490 ⇒ 00:04:06.690 Robert Tseng: 60% or whatever. Yeah. So, yeah. I guess…
55 00:04:06.910 ⇒ 00:04:18.570 Robert Tseng: I mean, we’ve just been going in and, like, clicking… like, I’ve learned a lot more about every object in Mongo. I mean, ideally, this is just something, like.
56 00:04:18.579 ⇒ 00:04:28.140 Robert Tseng: whoever set it up, like, that knows what the sign-up event is firing off of in Mongo might know, like, hey, it’s just, like, a couple…
57 00:04:28.540 ⇒ 00:04:31.769 Robert Tseng: additional filters that, like, you’re missing here.
58 00:04:31.770 ⇒ 00:04:32.160 Speaker 2 (Elizabeth Conference Room): Exactly.
59 00:04:32.160 ⇒ 00:04:34.630 Robert Tseng: We’ll just keep, like, guessing until we get there.
60 00:04:34.630 ⇒ 00:04:43.750 Speaker 2 (Elizabeth Conference Room): Right, yeah, so that’s… Alicia and I chatted earlier today. I think that… because Falco… well, actually, I don’t know if Falco set up the sign-up, event.
61 00:04:44.000 ⇒ 00:04:52.979 Speaker 2 (Elizabeth Conference Room): That might have already existed when we kicked this off, but we will have someone on our side verify the, sign-up event.
62 00:04:53.160 ⇒ 00:04:54.729 Speaker 2 (Elizabeth Conference Room): And then… Yeah.
63 00:04:55.350 ⇒ 00:04:57.659 Speaker 2 (Elizabeth Conference Room): We’ll go from there. Okay.
64 00:04:57.660 ⇒ 00:05:03.870 Speaker 1 (Elizabeth Conference Room): Because you would prefer to have everything and then filter down, Robert, within Amplitude, is that the goal?
65 00:05:05.160 ⇒ 00:05:07.040 Robert Tseng: Well, I… yeah, so I was…
66 00:05:07.230 ⇒ 00:05:21.329 Robert Tseng: I actually thought amplitude would be higher than in Mongo, in which case I could use amplitude filters, which I’m more familiar with, to, like, kind of cut it down, but it actually ended up being the other way around, so then in Mongo, I’m just, like, I’m just filtering, I’m just, like, randomly guessing.
67 00:05:21.330 ⇒ 00:05:21.600 Speaker 1 (Elizabeth Conference Room): So.
68 00:05:21.870 ⇒ 00:05:28.720 Robert Tseng: The email verification seemed right, I think it got us closer, but it didn’t get us all the way, so I’m like, what else is in this delta? I don’t know.
69 00:05:29.130 ⇒ 00:05:40.630 Speaker 2 (Elizabeth Conference Room): Yeah, I don’t think we want to exclude the email verification, because it… it’s… while we ask people to do it, it doesn’t prevent them from using the product.
70 00:05:41.010 ⇒ 00:05:43.700 Robert Tseng: That’s true. So, maybe I’ll just not.
71 00:05:43.700 ⇒ 00:05:44.960 Speaker 2 (Elizabeth Conference Room): Yeah, at least for now.
72 00:05:44.960 ⇒ 00:05:46.110 Robert Tseng: This is what I’m, yeah.
73 00:05:46.110 ⇒ 00:06:01.350 Speaker 2 (Elizabeth Conference Room): Yeah, let’s see if we can update. So, our potential options, for now at least, is take a look at the events used for science success in Amplitude, and see if there are any updates there that can better mirror how Mongo is recognizing.
74 00:06:02.070 ⇒ 00:06:02.640 Robert Tseng: Correct.
75 00:06:02.960 ⇒ 00:06:15.789 Speaker 2 (Elizabeth Conference Room): Okay, so we’ll start with that, and then project created, I agree, that is quite close. Attempted launch, I prefer that to be closer, but whatever, for now, it’s fine. Why’d you say you’re okay with subscription success?
76 00:06:16.440 ⇒ 00:06:21.019 Robert Tseng: So yeah, actually, like, this is… it’s hard to, like, illustrate this in a table, so…
77 00:06:21.710 ⇒ 00:06:29.900 Robert Tseng: In amplitude, we have attempted launch. We don’t actually have, like, launch. Like, so this is true launches, this is attempted launches.
78 00:06:29.900 ⇒ 00:06:30.560 Speaker 2 (Elizabeth Conference Room): I don’t know.
79 00:06:30.560 ⇒ 00:06:35.029 Robert Tseng: And so, like, I feel like, you know, maybe I should split this up a bit more.
80 00:06:35.330 ⇒ 00:06:36.619 Speaker 2 (Elizabeth Conference Room): If that makes sense.
81 00:06:37.570 ⇒ 00:06:42.509 Robert Tseng: Yeah, so this is more like… I don’t know, true launch.
82 00:06:42.650 ⇒ 00:06:43.459 Elizabeth Conference Room: Huh.
83 00:06:44.000 ⇒ 00:07:02.900 Robert Tseng: So I’m just gonna… Yeah, and so… for whatever reason, attempted launch, subscription success as a… as a… as, like, that follows attempted launch, it just doesn’t capture the right one. But once I remove that step, we match up perfectly. It’s 31. It’s the same as what we have.
84 00:07:02.900 ⇒ 00:07:03.300 Speaker 2 (Elizabeth Conference Room): Oh.
85 00:07:03.300 ⇒ 00:07:09.299 Robert Tseng: Here. So, so yeah, it’s like, whatever is happening with attempted launch, like.
86 00:07:10.410 ⇒ 00:07:16.199 Robert Tseng: And because it’s not a true launch event, like, I just think that that’s what’s messing with the number here.
87 00:07:16.200 ⇒ 00:07:18.770 Speaker 2 (Elizabeth Conference Room): Why can’t we get true launch and amplitude?
88 00:07:19.440 ⇒ 00:07:23.409 Robert Tseng: It’s just… it’s just not… we just don’t have that event in… in Amplex.
89 00:07:23.410 ⇒ 00:07:23.860 Speaker 2 (Elizabeth Conference Room): Okay.
90 00:07:23.860 ⇒ 00:07:28.539 Robert Tseng: So, kind of like what… yeah, we should just… if we can’t send that in, like, that would be ideal.
91 00:07:28.540 ⇒ 00:07:40.230 Speaker 2 (Elizabeth Conference Room): Yeah, well, someone’s… so we’re gonna have to have someone from Eng do something to figure out the user sign-up events, so we might as well have them create… launch… a launch event as well.
92 00:07:40.235 ⇒ 00:07:43.824 Speaker 1 (Elizabeth Conference Room): Sorry, can you just define a true launch? How are you defining that?
93 00:07:44.410 ⇒ 00:07:53.740 Robert Tseng: So this is just, like, like, a project actually got launched. I don’t know, there’s, like, a… it’s… it’s… there’s a log for that in…
94 00:07:54.000 ⇒ 00:07:54.730 Speaker 1 (Elizabeth Conference Room): So the delta.
95 00:07:54.730 ⇒ 00:07:55.450 Robert Tseng: launched.
96 00:07:55.450 ⇒ 00:07:56.520 Speaker 1 (Elizabeth Conference Room): Straight lines.
97 00:07:56.520 ⇒ 00:07:56.960 Robert Tseng: project.
98 00:07:56.960 ⇒ 00:08:05.039 Speaker 2 (Elizabeth Conference Room): I think that’s what we call them. So, Robert, for attempted launch, the event is triggering on when someone clicks the button, launch, right?
99 00:08:05.040 ⇒ 00:08:06.380 Robert Tseng: Yeah, it’s just a button, yeah.
100 00:08:06.380 ⇒ 00:08:18.100 Speaker 2 (Elizabeth Conference Room): And then… because… so they hit the button launch, and then it takes them to manage plan, and then they have to actually select a plan to launch, which would be subscription success.
101 00:08:18.365 ⇒ 00:08:22.194 Speaker 1 (Elizabeth Conference Room): The 196 would be… Delta would be free of launch.
102 00:08:23.630 ⇒ 00:08:35.640 Speaker 2 (Elizabeth Conference Room): I don’t know. I don’t… that’s probably the only thing it could be, right? Right. What’s the call-out? Free launching canceled, right? After reviewing the project, then it’s perfectly fine. I found only two, I need to be excluded.
103 00:08:36.950 ⇒ 00:08:37.770 Robert Tseng: Yeah.
104 00:08:39.720 ⇒ 00:08:40.480 Speaker 2 (Elizabeth Conference Room): Okay.
105 00:08:41.919 ⇒ 00:08:43.379 Robert Tseng: So yeah, you’re right. So this is, like.
106 00:08:43.380 ⇒ 00:08:43.950 Speaker 2 (Elizabeth Conference Room): Yeah.
107 00:08:43.950 ⇒ 00:08:46.120 Robert Tseng: Free launch, really.
108 00:08:46.520 ⇒ 00:08:57.150 Robert Tseng: And then this is, like… well, we were… I mean, I just… we call it paid projects, but, like, yeah, once there’s a payment, what we’re calling subscription success… I know this language needs to be normalized, but…
109 00:08:57.450 ⇒ 00:08:57.960 Speaker 2 (Elizabeth Conference Room): Anybody’s…
110 00:08:57.960 ⇒ 00:09:00.300 Robert Tseng: could launch. Yeah. But…
111 00:09:00.800 ⇒ 00:09:10.990 Robert Tseng: Once it’s actually a successful launch, there is, like, a gate field that comes through that you log in Mongo that’s called launched at. Yeah.
112 00:09:10.990 ⇒ 00:09:11.600 Speaker 2 (Elizabeth Conference Room): And they’re called…
113 00:09:11.600 ⇒ 00:09:12.460 Robert Tseng: That’s not being kept.
114 00:09:12.460 ⇒ 00:09:17.640 Speaker 2 (Elizabeth Conference Room): Yeah, wait, can you just call that quick, just paid launch?
115 00:09:17.880 ⇒ 00:09:27.609 Speaker 2 (Elizabeth Conference Room): Okay. So, time to launch… Yeah, you’re right. Mongo doesn’t have… That as a data point.
116 00:09:29.120 ⇒ 00:09:33.109 Speaker 2 (Elizabeth Conference Room): interacting with the attempted launch? Mongo doesn’t have…
117 00:09:33.110 ⇒ 00:09:37.080 Robert Tseng: Yeah, Margo doesn’t store, like, clicks and buttons or whatever, so that’s, like, kind of
118 00:09:37.470 ⇒ 00:09:40.160 Robert Tseng: discrepancy, like, that’s, like, the duality here.
119 00:09:40.160 ⇒ 00:09:46.239 Speaker 2 (Elizabeth Conference Room): Great. And then, Amplitude, can we get free launch tracked?
120 00:09:47.040 ⇒ 00:09:48.339 Speaker 2 (Elizabeth Conference Room): Do we have a chart?
121 00:09:49.280 ⇒ 00:09:52.559 Robert Tseng: No. So, we don’t.
122 00:09:52.970 ⇒ 00:09:57.490 Robert Tseng: Yeah, I mean, we have it, like, post-mortem, like, if there is actually, like.
123 00:09:58.050 ⇒ 00:10:05.469 Robert Tseng: Like, it comes up as, like, a prop… like, a property, whether or not it’s paid or paid or unpaid, and so you can, like.
124 00:10:05.770 ⇒ 00:10:13.299 Robert Tseng: You can either guess at when they launched, or you can look at, a user that you, yeah, you…
125 00:10:13.560 ⇒ 00:10:22.700 Robert Tseng: behaves like someone who’s launched, and then look at their plan type. So, it’s, like, kind of just… both are incomplete ways of looking at it. Like, we just… this would be ideal to have.
126 00:10:23.200 ⇒ 00:10:23.990 Speaker 2 (Elizabeth Conference Room): Yup.
127 00:10:24.110 ⇒ 00:10:26.559 Speaker 2 (Elizabeth Conference Room): I think, I think we should be able to get that, too.
128 00:10:26.565 ⇒ 00:10:27.325 Speaker 1 (Elizabeth Conference Room): I, I don’t know.
129 00:10:27.325 ⇒ 00:10:28.064 Speaker 2 (Elizabeth Conference Room): love about signing up.
130 00:10:28.070 ⇒ 00:10:28.810 Speaker 4 (Elizabeth Conference Room): Yeah.
131 00:10:28.810 ⇒ 00:10:29.940 Speaker 2 (Elizabeth Conference Room): But we’ll… I’ll talk to most respect.
132 00:10:30.325 ⇒ 00:10:35.614 Speaker 2 (Elizabeth Conference Room): because ultimately, Robert, what we need to be able to do is,
133 00:10:36.285 ⇒ 00:10:42.525 Speaker 2 (Elizabeth Conference Room): We’re reintroducing the 14-day trial, at some point in the next couple of weeks.
134 00:10:42.685 ⇒ 00:10:51.514 Speaker 2 (Elizabeth Conference Room): And so what that will do is, you know, constrain the amount of time that someone doesn’t launch, and at 14 days.
135 00:10:51.645 ⇒ 00:10:56.785 Speaker 2 (Elizabeth Conference Room): People are going to be required to pick a plan, or basically no doubt.
136 00:10:56.790 ⇒ 00:10:57.329 Robert Tseng: Yeah, yeah.
137 00:10:57.330 ⇒ 00:11:03.919 Speaker 2 (Elizabeth Conference Room): And so we’re gonna see an uptick in people picking free, free launch.
138 00:11:04.750 ⇒ 00:11:05.480 Robert Tseng: Yeah.
139 00:11:05.480 ⇒ 00:11:07.880 Speaker 2 (Elizabeth Conference Room): And I just want to make sure we can…
140 00:11:08.020 ⇒ 00:11:10.890 Speaker 2 (Elizabeth Conference Room): Speak really intelligently about those numbers.
141 00:11:13.820 ⇒ 00:11:15.249 Robert Tseng: Yep, yeah, that makes sense.
142 00:11:15.250 ⇒ 00:11:19.660 Speaker 2 (Elizabeth Conference Room): Okay, so I’ll see if we can get better events for sign-up success.
143 00:11:19.990 ⇒ 00:11:26.770 Speaker 2 (Elizabeth Conference Room): And better events for… Or, it’s really just, like, plan selection.
144 00:11:28.560 ⇒ 00:11:34.029 Speaker 2 (Elizabeth Conference Room): Because you don’t have… because in paid, we don’t have it broken out by startup or business either.
145 00:11:34.840 ⇒ 00:11:38.340 Speaker 2 (Elizabeth Conference Room): 400 risk, right? Should we just call it Dog Sponge? Like…
146 00:11:38.980 ⇒ 00:11:41.860 Speaker 2 (Elizabeth Conference Room): Well, no, I think we wanna… I think we want…
147 00:11:41.860 ⇒ 00:11:58.510 Robert Tseng: There are a few different, fields in Longo. It’s kind of messy, to be honest, so, like, you have, like, there’s a lot of redundancy here. So, there’s actually 3 different plan statuses, 3 different plans, and so, like, we had to, like, add all these, like, conditions in order to guess at it. But anyway, so…
148 00:11:59.450 ⇒ 00:12:05.919 Speaker 2 (Elizabeth Conference Room): So, if we wanted to track… if we wanted to track the amount of customers by plan type…
149 00:12:07.620 ⇒ 00:12:10.220 Speaker 2 (Elizabeth Conference Room): Is that possible at this current moment, or no?
150 00:12:10.460 ⇒ 00:12:12.570 Speaker 2 (Elizabeth Conference Room): Yeah, we can.
151 00:12:12.840 ⇒ 00:12:19.809 Speaker 2 (Elizabeth Conference Room): So, because we have, we have user sign-up, project creation, attempted launch, and then it’s, like.
152 00:12:21.190 ⇒ 00:12:28.570 Speaker 1 (Elizabeth Conference Room): It’s a breakout of paid launch, basically, right? Right. Yeah. We just want to know what plans the paid launch users selected. Yeah.
153 00:12:28.570 ⇒ 00:12:29.140 Robert Tseng: Yeah.
154 00:12:29.420 ⇒ 00:12:39.699 Robert Tseng: There’s still, like, a small share of users that, like, are on null or whatever, so, like, you know, we had tabled that, like, whether or not we wanted to clean it up, but I don’t know what.
155 00:12:39.700 ⇒ 00:12:40.629 Speaker 1 (Elizabeth Conference Room): What percentage of…
156 00:12:40.630 ⇒ 00:12:41.720 Robert Tseng: This is… yeah.
157 00:12:41.720 ⇒ 00:12:52.429 Speaker 1 (Elizabeth Conference Room): Okay, we really should clean that up, or at least understand why they are coming up as null, so yeah, maybe you guys can dig into that, and then just… we can make a decision from there, because that… that shouldn’t happen.
158 00:12:53.180 ⇒ 00:12:57.520 Speaker 2 (Elizabeth Conference Room): Yeah, not with me. I feel like it’s something… how it was set up, like…
159 00:12:57.525 ⇒ 00:12:59.855 Speaker 1 (Elizabeth Conference Room): Yeah, it just doesn’t make sense. Okay.
160 00:13:01.430 ⇒ 00:13:07.269 Robert Tseng: Yeah, so, I mean, in Mongo, I could end up breaking this next stage out and buy plan type.
161 00:13:07.270 ⇒ 00:13:23.880 Robert Tseng: I can’t go to the use… you can’t go to user level, that’s, like, the difference. But with amplitude, at amplitude, you’re always tracing, like, the same… well, you think you’re tracing the same person. But yeah, so we can get at what it should be versus, like, what amplitude has, but yeah.
162 00:13:25.400 ⇒ 00:13:29.380 Speaker 2 (Elizabeth Conference Room): Sorry, so if you were to break it out by plan type, it would be…
163 00:13:29.510 ⇒ 00:13:31.530 Speaker 2 (Elizabeth Conference Room): On a user basis, right?
164 00:13:32.210 ⇒ 00:13:35.409 Robert Tseng: In amplitude, it would be. In Mongo, it’s not.
165 00:13:35.410 ⇒ 00:13:36.030 Speaker 3 (Elizabeth Conference Room): See what I’m.
166 00:13:36.030 ⇒ 00:13:43.930 Robert Tseng: Mongo is more, like, yeah, it’s just… these are just, like, different types of aggregations, like, it’s just, like, pivots on pivots on pivots, so…
167 00:13:43.930 ⇒ 00:13:44.560 Speaker 3 (Elizabeth Conference Room): Yep.
168 00:13:44.560 ⇒ 00:13:45.090 Robert Tseng: Yeah.
169 00:13:45.400 ⇒ 00:13:45.714 Speaker 3 (Elizabeth Conference Room): Cop.
170 00:13:46.340 ⇒ 00:13:47.340 Speaker 3 (Elizabeth Conference Room): Okay.
171 00:13:47.950 ⇒ 00:13:49.250 Speaker 3 (Elizabeth Conference Room): Yeah. Awesome.
172 00:13:49.580 ⇒ 00:13:53.140 Speaker 3 (Elizabeth Conference Room): I think we’re getting closer, this is exciting.
173 00:13:53.145 ⇒ 00:14:02.764 Speaker 2 (Elizabeth Conference Room): That’s exactly right. It’s like, it’s user science success, it’s project creation, it’s attempted launch, and then it’s free or paid launch, and of the paid launch, is it startup, business, or enterprise?
174 00:14:03.405 ⇒ 00:14:05.545 Speaker 2 (Elizabeth Conference Room): It’s not gonna be Enterprise, but that’s fine.
175 00:14:07.680 ⇒ 00:14:09.600 Robert Tseng: Yeah, yeah, we’re excluding Enterprise from this, yeah.
176 00:14:09.600 ⇒ 00:14:10.910 Speaker 3 (Elizabeth Conference Room): Yeah, great.
177 00:14:12.620 ⇒ 00:14:14.990 Speaker 3 (Elizabeth Conference Room): Okay.
178 00:14:14.990 ⇒ 00:14:15.590 Robert Tseng: Okay.
179 00:14:15.870 ⇒ 00:14:19.149 Speaker 1 (Elizabeth Conference Room): Let’s move to… yes.
180 00:14:19.350 ⇒ 00:14:37.160 Robert Tseng: Yeah, so we… you asked quickly about the linting and docs audit option. I mean, it… this is, like, a different view from… because we had already kind of, shown the linting stuff before, but, you know, this is just isolating those two. So, seems like you launched a feature in mid-November, I guess a week ago.
181 00:14:37.230 ⇒ 00:14:42.959 Robert Tseng: That adoption has not been great compared to the other…
182 00:14:43.290 ⇒ 00:14:46.260 Robert Tseng: like, linter… like, linter ran versus Linter Fix.
183 00:14:46.380 ⇒ 00:14:54.989 Robert Tseng: Yeah, I mean, you can see all the different error messages that we have here. I don’t know, we can kind of click into anything that you want to double-click and do.
184 00:15:03.810 ⇒ 00:15:06.360 Speaker 1 (Elizabeth Conference Room): The usage from a pivot is really important.
185 00:15:06.900 ⇒ 00:15:08.829 Speaker 1 (Elizabeth Conference Room): That becomes, like, the trigger.
186 00:15:10.110 ⇒ 00:15:15.169 Speaker 1 (Elizabeth Conference Room): And, like, I’d be curious what we’re doing for those users, like, when they open.
187 00:15:15.630 ⇒ 00:15:15.990 Robert Tseng: Hmm.
188 00:15:15.990 ⇒ 00:15:20.040 Speaker 3 (Elizabeth Conference Room): What’s going on here? So… Interesting.
189 00:15:20.270 ⇒ 00:15:20.849 Elizabeth Conference Room: I’m sorry.
190 00:15:20.850 ⇒ 00:15:21.480 Speaker 3 (Elizabeth Conference Room): suck.
191 00:15:22.500 ⇒ 00:15:23.710 Robert Tseng: Let’s click into that.
192 00:15:23.710 ⇒ 00:15:24.600 Speaker 3 (Elizabeth Conference Room): Damn.
193 00:15:25.220 ⇒ 00:15:29.640 Robert Tseng: So, yeah, there’s, like, weekly, kind of, like, Cyclical behavior here.
194 00:15:30.290 ⇒ 00:15:36.700 Robert Tseng: Seems like… it’s like the latter half of the week is when people are trying it. For some reason, during the past week.
195 00:15:37.690 ⇒ 00:15:39.529 Robert Tseng: Quite a few people hit their limit.
196 00:15:40.290 ⇒ 00:15:41.010 Speaker 3 (Elizabeth Conference Room): Cool.
197 00:15:44.120 ⇒ 00:15:45.850 Speaker 3 (Elizabeth Conference Room): So that means these are…
198 00:15:46.610 ⇒ 00:15:55.290 Speaker 1 (Elizabeth Conference Room): oh, actually, we should probably filter this by people who have the AI booster or not.
199 00:15:56.950 ⇒ 00:15:57.640 Robert Tseng: Okay.
200 00:15:58.460 ⇒ 00:16:11.050 Speaker 1 (Elizabeth Conference Room): Are they able to do it without the booster? Yes. That’s how they hit the limit, but I think we just want to capture, like, true users versus people who have added it on, because I think we’ll want to track that behavior separately.
201 00:16:11.790 ⇒ 00:16:12.380 Speaker 1 (Elizabeth Conference Room): It’s like…
202 00:16:12.380 ⇒ 00:16:14.970 Robert Tseng: So this is only AI users now.
203 00:16:15.560 ⇒ 00:16:18.799 Speaker 1 (Elizabeth Conference Room): Okay, and what about the inverse, if, people who don’t have it?
204 00:16:24.320 ⇒ 00:16:24.900 Elizabeth Conference Room: Yeah.
205 00:16:31.770 ⇒ 00:16:33.250 Speaker 4 (Elizabeth Conference Room): What’s the ask on this?
206 00:16:33.950 ⇒ 00:16:35.939 Speaker 1 (Elizabeth Conference Room): From Pat.
207 00:16:36.260 ⇒ 00:16:41.999 Speaker 1 (Elizabeth Conference Room): adoption and revenue metrics. Okay. So I think we’ve hit some good start for adoption.
208 00:16:42.960 ⇒ 00:16:46.219 Speaker 1 (Elizabeth Conference Room): What about revenue? I saw on the bottom you guys had started
209 00:16:46.580 ⇒ 00:16:49.139 Speaker 1 (Elizabeth Conference Room): pulling something together, I can’t remember exactly.
210 00:16:49.140 ⇒ 00:16:55.300 Robert Tseng: Yeah, we’re, like, mapping it to billing events. It’s not exactly revenue yet, so this is kind of where it was, like,
211 00:16:56.430 ⇒ 00:17:01.939 Robert Tseng: what do we say here? Yeah, no, this is just, like, whether or not they actually, ended up
212 00:17:07.589 ⇒ 00:17:09.009 Speaker 1 (Elizabeth Conference Room): It’s gonna be hard to let this start.
213 00:17:10.540 ⇒ 00:17:15.179 Robert Tseng: Yeah, okay, well, so, I mean, I know this is… this is not… this is not ready for, revenue.
214 00:17:15.560 ⇒ 00:17:16.250 Robert Tseng: So…
215 00:17:16.250 ⇒ 00:17:27.160 Speaker 1 (Elizabeth Conference Room): No, I think the adoption metrics are really great. It’s, like, a level that we haven’t been looking at yet. Okay. I think through… I don’t know the…
216 00:17:27.160 ⇒ 00:17:32.390 Speaker 2 (Elizabeth Conference Room): the limits on the non-paid plan versus the paid plan, I don’t know what they are. It’s like…
217 00:17:32.390 ⇒ 00:17:35.780 Speaker 4 (Elizabeth Conference Room): Well, I don’t… yeah.
218 00:17:38.100 ⇒ 00:17:40.320 Speaker 1 (Elizabeth Conference Room): How do you know that they fit the limit?
219 00:17:42.090 ⇒ 00:17:44.510 Speaker 1 (Elizabeth Conference Room): On your end.
220 00:17:47.480 ⇒ 00:17:48.860 Speaker 1 (Elizabeth Conference Room): How is that.
221 00:17:50.930 ⇒ 00:17:54.380 Robert Tseng: I believe this is a custom event. Let me see…
222 00:17:54.380 ⇒ 00:17:56.920 Speaker 3 (Elizabeth Conference Room): happens at expense, huh? Yes.
223 00:17:57.030 ⇒ 00:17:57.870 Speaker 3 (Elizabeth Conference Room): So she probably just…
224 00:17:57.875 ⇒ 00:18:02.134 Speaker 2 (Elizabeth Conference Room): created a… an event writing limit in the database. Okay.
225 00:18:04.260 ⇒ 00:18:07.349 Speaker 1 (Elizabeth Conference Room): This is where my knowledge goes.
226 00:18:10.750 ⇒ 00:18:11.390 Speaker 4 (Elizabeth Conference Room): Okay.
227 00:18:11.390 ⇒ 00:18:16.679 Robert Tseng: Yeah… Yeah, it looks like someone just added this event when we used it.
228 00:18:16.680 ⇒ 00:18:24.199 Speaker 1 (Elizabeth Conference Room): Okay, cool. And is it possible to add filters for all these adoption metrics on plan type, specifically, like, startup and business?
229 00:18:24.780 ⇒ 00:18:25.450 Robert Tseng: Yeah.
230 00:18:25.780 ⇒ 00:18:27.970 Speaker 1 (Elizabeth Conference Room): Okay, I think that will be helpful as well.
231 00:18:37.680 ⇒ 00:18:38.500 Speaker 1 (Elizabeth Conference Room): Cool.
232 00:18:40.700 ⇒ 00:18:45.479 Robert Tseng: So maybe we’ll just exclude…
233 00:18:48.830 ⇒ 00:18:50.580 Robert Tseng: Enterprise…
234 00:18:52.330 ⇒ 00:18:56.269 Speaker 1 (Elizabeth Conference Room): We might need to group these plans together, like the startups.
235 00:18:56.920 ⇒ 00:18:58.140 Speaker 1 (Elizabeth Conference Room): Yeah, because there’s, like, so many.
236 00:18:58.140 ⇒ 00:19:04.780 Robert Tseng: Yeah, well, we could just kind of go one at a time first, so let’s just look at this, startups.
237 00:19:07.950 ⇒ 00:19:10.289 Robert Tseng: Let’s just say these are all startup.
238 00:19:16.780 ⇒ 00:19:23.089 Robert Tseng: So ideally, I would, I mean, just because… yeah, why? I think this would probably look cleaner.
239 00:19:23.700 ⇒ 00:19:26.870 Robert Tseng: We just excluded these…
240 00:19:29.570 ⇒ 00:19:33.049 Speaker 1 (Elizabeth Conference Room): So either I was telling you about his junk all this time.
241 00:19:33.260 ⇒ 00:19:36.030 Robert Tseng: The one here, that was just business…
242 00:19:51.740 ⇒ 00:19:54.690 Robert Tseng: Oh, I can’t hide. Okay, well…
243 00:19:55.000 ⇒ 00:20:03.249 Robert Tseng: Well, I was just trying to group all the ones that are… should be startup, all the ones that should be business, so you could look at them separately, but,
244 00:20:03.650 ⇒ 00:20:11.380 Robert Tseng: But I forgot that you can’t hide segments in Amplitude. That’s a Mixpanel feature. So that wasn’t the best view.
245 00:20:13.190 ⇒ 00:20:20.849 Robert Tseng: I guess, like, in a dashboard, I would just create this chart twice. I would do one segmented by disclosure code by another, so… .
246 00:20:21.730 ⇒ 00:20:22.259 Speaker 4 (Elizabeth Conference Room): Cool. Yeah.
247 00:20:22.260 ⇒ 00:20:24.970 Robert Tseng: I mean, did you want to look at it now, or…
248 00:20:25.110 ⇒ 00:20:30.549 Speaker 1 (Elizabeth Conference Room): We don’t have to do that now, I think it will be a follow-up ask from the group. Yeah.
249 00:20:30.750 ⇒ 00:20:31.290 Speaker 1 (Elizabeth Conference Room): Cool.
250 00:20:31.290 ⇒ 00:20:34.049 Robert Tseng: Yeah, okay, those are small adjustments we can make.
251 00:20:34.200 ⇒ 00:20:39.889 Robert Tseng: Yeah, if I were to do revenue, yeah. Yeah, I mean, I…
252 00:20:41.150 ⇒ 00:20:49.499 Robert Tseng: if we wanted to say revenue of those who hit the limit in the startup plan, then they have, like, a… they have a revenue, or ARPU, kind of, like, revenue…
253 00:20:49.500 ⇒ 00:20:49.930 Speaker 3 (Elizabeth Conference Room): Yeah.
254 00:20:49.930 ⇒ 00:20:54.500 Robert Tseng: kind of, like, chart here. I probably use that on people who
255 00:20:55.880 ⇒ 00:21:03.060 Robert Tseng: yeah, like, you know, by A… by A, B, and C, and, like, you’d be able to, like, we can just put that as, like, three, like…
256 00:21:03.440 ⇒ 00:21:06.230 Robert Tseng: We’ve got another, like, a tile buff, so you can say…
257 00:21:06.350 ⇒ 00:21:15.329 Robert Tseng: So you’d be able to see average revenue per user for those that did hit the usage limit versus who didn’t, or whatever. We can kind of figure out what those cuts are.
258 00:21:17.970 ⇒ 00:21:18.540 Robert Tseng: Yeah.
259 00:21:18.540 ⇒ 00:21:21.710 Speaker 2 (Elizabeth Conference Room): Is that… is that how it’s set up, that if you hit the limit, you…
260 00:21:22.030 ⇒ 00:21:27.039 Speaker 2 (Elizabeth Conference Room): You’re not required to pay, you just, like… Yeah, yeah, it’s like a second trigger. Right.
261 00:21:27.040 ⇒ 00:21:31.919 Speaker 1 (Elizabeth Conference Room): I think it’s, like, it’ll be good for us to go to the workflow there, now that we have so much.
262 00:21:32.340 ⇒ 00:21:33.390 Speaker 1 (Elizabeth Conference Room): Right.
263 00:21:34.280 ⇒ 00:21:34.910 Robert Tseng: Okay.
264 00:21:35.030 ⇒ 00:21:46.770 Robert Tseng: I mean, I will say that, like, based on the other AI, like, I don’t know if Phoebe was here when I was kind of sharing the AI feature exploration, but some of our initial, like, work there was basically showing
265 00:21:49.050 ⇒ 00:21:57.540 Robert Tseng: Yeah, I mean, AI feature usage is pretty up and down, in terms of, like, share of users is not very high, but then, it also doesn’t…
266 00:21:57.730 ⇒ 00:22:00.160 Robert Tseng: Drive… wait.
267 00:22:01.350 ⇒ 00:22:10.260 Robert Tseng: That’s not one. AI… Try to not save the right one.
268 00:22:15.900 ⇒ 00:22:19.779 Robert Tseng: Okay, sorry, I think I… I… there was…
269 00:22:19.960 ⇒ 00:22:29.400 Robert Tseng: Okay, well, compared to, like, your core features, like, like API docs, which, like, if a user is using API docs.
270 00:22:29.660 ⇒ 00:22:47.710 Robert Tseng: they’re… they’re converting at a much higher rate than… than average, versus, like, the AI… whether or not they’re using AI or not, it’s… it’s more like a amplifier to the other, features. It doesn’t have, like, just isolating AI users versus non-AI users, the converting rate.
271 00:22:47.710 ⇒ 00:22:48.100 Speaker 5 (Elizabeth Conference Room): That’s about.
272 00:22:48.100 ⇒ 00:22:48.830 Robert Tseng: insane.
273 00:22:48.990 ⇒ 00:22:49.800 Elizabeth Conference Room: Yeah.
274 00:22:49.800 ⇒ 00:22:52.190 Speaker 2 (Elizabeth Conference Room): But it’s also, I think it’s hard to say, because…
275 00:22:52.460 ⇒ 00:22:57.710 Speaker 2 (Elizabeth Conference Room): We’re not currently allowing people to trial
276 00:22:58.000 ⇒ 00:23:11.090 Speaker 2 (Elizabeth Conference Room): like, you have to be a paid user to add the AI booster. We are, like, manually, putting people on AI trials, but I’m… it wouldn’t surprise me if AI isn’t helping
277 00:23:11.220 ⇒ 00:23:16.869 Speaker 2 (Elizabeth Conference Room): New user conversion, because they’re not able to easily, like, trial it.
278 00:23:18.230 ⇒ 00:23:20.969 Speaker 2 (Elizabeth Conference Room): So I don’t even… it’s like, I don’t even know… I see.
279 00:23:20.970 ⇒ 00:23:22.650 Robert Tseng: Yeah, this is what I was trying to talk about, yeah.
280 00:23:22.650 ⇒ 00:23:23.280 Speaker 3 (Elizabeth Conference Room): Yeah.
281 00:23:23.280 ⇒ 00:23:30.829 Robert Tseng: So yeah, you’re saying these are so close because you have to end up… I mean, in order to use the AI features, you need to be a paid user anyway. Okay, so…
282 00:23:31.030 ⇒ 00:23:31.420 Speaker 4 (Elizabeth Conference Room): Hold on.
283 00:23:31.420 ⇒ 00:23:33.299 Robert Tseng: There is no AI trial. Yeah.
284 00:23:33.300 ⇒ 00:23:40.190 Speaker 2 (Elizabeth Conference Room): Yeah. As opposed to, like… we’re manually turning them on for people when they write in to support, but that’s probably…
285 00:23:40.350 ⇒ 00:23:41.000 Speaker 2 (Elizabeth Conference Room): the…
286 00:23:41.000 ⇒ 00:23:47.270 Robert Tseng: I see. So that could explain the difference here. These are also just people that you’re just manually, adding. Okay.
287 00:23:47.270 ⇒ 00:23:47.940 Speaker 3 (Elizabeth Conference Room): Yeah.
288 00:23:48.420 ⇒ 00:23:49.160 Robert Tseng: Understood.
289 00:23:49.490 ⇒ 00:23:53.919 Speaker 2 (Elizabeth Conference Room): But, but it’s… I think it’s a point that’s come up a couple times internally of, like.
290 00:23:54.260 ⇒ 00:23:55.919 Speaker 2 (Elizabeth Conference Room): Who are we…
291 00:23:56.400 ⇒ 00:24:08.159 Speaker 2 (Elizabeth Conference Room): who do we want to be using the AI features? Is it just existing customers, or… I would think it would be a big draw for new customers, and if that’s the case, we should enable it for a trial.
292 00:24:08.710 ⇒ 00:24:18.180 Speaker 2 (Elizabeth Conference Room): I mean, I don’t think the data here tells us what to do, that’s more of, like, a business decision. But people… Sure. What we would want to know, I guess, is, like, demand.
293 00:24:18.360 ⇒ 00:24:24.559 Speaker 2 (Elizabeth Conference Room): Is that something we have? Of people, like, who are clicking the booster button?
294 00:24:24.560 ⇒ 00:24:26.330 Robert Tseng: Yeah, yeah, we do, yeah.
295 00:24:28.740 ⇒ 00:24:30.210 Speaker 1 (Elizabeth Conference Room): We should’ve got…
296 00:24:30.210 ⇒ 00:24:31.350 Robert Tseng: their reports.
297 00:24:31.350 ⇒ 00:24:37.399 Speaker 1 (Elizabeth Conference Room): And then you had also started work on, like, top feature usage tied to different paid plans. I think we should bring that back.
298 00:24:37.400 ⇒ 00:24:37.900 Robert Tseng: Yes.
299 00:24:37.900 ⇒ 00:24:42.929 Speaker 1 (Elizabeth Conference Room): I can continue to iterate on that, but just look at the baseline for now. I know that there was an ask around, like.
300 00:24:43.080 ⇒ 00:24:47.940 Speaker 1 (Elizabeth Conference Room): combinations, but I think before we even make hypotheses, it would be good to just look at it again.
301 00:24:49.000 ⇒ 00:24:50.270 Speaker 2 (Elizabeth Conference Room): Oh, okay.
302 00:24:50.270 ⇒ 00:24:50.880 Robert Tseng: Yes.
303 00:24:50.880 ⇒ 00:24:54.089 Speaker 2 (Elizabeth Conference Room): And they will ask that. Yeah, this is interesting to me.
304 00:24:54.090 ⇒ 00:25:00.760 Robert Tseng: Yeah, so, I mean, this is kind of inflated, because this one… I’ll just remove it temporarily.
305 00:25:01.440 ⇒ 00:25:03.139 Robert Tseng: But,
306 00:25:03.310 ⇒ 00:25:17.610 Robert Tseng: Yeah, these are all the AI-related features, so it’s like, okay, well, weekly, people are asking to enable AI. It’s not as high as you would think. You would expect that maybe it’s the highest, and then everything else is kind of, it’s downstream of that.
307 00:25:18.020 ⇒ 00:25:22.459 Robert Tseng: But maybe a linter shouldn’t even be in this list. I forgot why I included it in this list.
308 00:25:22.460 ⇒ 00:25:22.940 Speaker 3 (Elizabeth Conference Room): It was.
309 00:25:22.940 ⇒ 00:25:24.380 Robert Tseng: Maybe this was just, like, a new feature.
310 00:25:24.750 ⇒ 00:25:25.450 Speaker 1 (Elizabeth Conference Room): It was cute.
311 00:25:25.450 ⇒ 00:25:26.260 Robert Tseng: Yeah, that’s why.
312 00:25:26.290 ⇒ 00:25:28.029 Speaker 1 (Elizabeth Conference Room): Yeah. Wait.
313 00:25:28.035 ⇒ 00:25:36.705 Speaker 2 (Elizabeth Conference Room): The Enable Ask AI… I don’t know if that is the AI booster. I think that’s the Albert.
314 00:25:37.415 ⇒ 00:25:38.885 Speaker 2 (Elizabeth Conference Room): Ask that.
315 00:25:38.890 ⇒ 00:25:40.100 Robert Tseng: That’s just Albert thing.
316 00:25:40.100 ⇒ 00:25:41.630 Speaker 2 (Elizabeth Conference Room): Yeah. Oh, I see.
317 00:25:41.630 ⇒ 00:25:43.399 Robert Tseng: Okay. Out, outbox.
318 00:25:44.240 ⇒ 00:25:44.600 Robert Tseng: Yeah.
319 00:25:44.600 ⇒ 00:25:49.419 Speaker 1 (Elizabeth Conference Room): Ai Booster… we’re not tracking anything on our pricing page, right?
320 00:25:49.590 ⇒ 00:25:51.219 Speaker 1 (Elizabeth Conference Room): Probably not. Well…
321 00:25:51.390 ⇒ 00:25:53.440 Speaker 2 (Elizabeth Conference Room): Yeah, so when you…
322 00:25:53.730 ⇒ 00:26:01.099 Speaker 2 (Elizabeth Conference Room): When you go to our Manage Plan page, which you can navigate to, via the launch button… sorry, I also…
323 00:26:01.660 ⇒ 00:26:07.210 Speaker 2 (Elizabeth Conference Room): premise. That’s where you add… Basically, what we want.
324 00:26:07.210 ⇒ 00:26:08.179 Robert Tseng: Oh, I see.
325 00:26:08.180 ⇒ 00:26:15.230 Speaker 2 (Elizabeth Conference Room): The equivalent of, like, attempted launch, but it would be, like, attempted… AI Booster Edition.
326 00:26:15.410 ⇒ 00:26:16.330 Speaker 2 (Elizabeth Conference Room): But I think that’s fair.
327 00:26:16.330 ⇒ 00:26:20.879 Robert Tseng: Yeah, do you know the text of that button? Just, like, lightly, we could just, like, add it in.
328 00:26:20.880 ⇒ 00:26:23.860 Speaker 2 (Elizabeth Conference Room): Because you’re on a free enterprise.
329 00:26:24.210 ⇒ 00:26:25.359 Robert Tseng: add AI Booster.
330 00:26:25.360 ⇒ 00:26:27.510 Speaker 4 (Elizabeth Conference Room): Yeah, because it’s on my phone. Sure.
331 00:26:27.510 ⇒ 00:26:28.650 Robert Tseng: Okay, let’s see.
332 00:26:28.650 ⇒ 00:26:30.739 Speaker 4 (Elizabeth Conference Room): After, yep. You found it?
333 00:26:30.740 ⇒ 00:26:33.580 Robert Tseng: Okay, yeah, so I changed it, yeah, that makes.
334 00:26:33.580 ⇒ 00:26:34.020 Speaker 2 (Elizabeth Conference Room): Cool.
335 00:26:34.020 ⇒ 00:26:36.809 Robert Tseng: to me. It’s pretty close to… yeah.
336 00:26:38.130 ⇒ 00:26:39.439 Speaker 2 (Elizabeth Conference Room): Okay, so wait, sorry, wait.
337 00:26:39.440 ⇒ 00:26:39.920 Robert Tseng: Excellent, yeah.
338 00:26:40.150 ⇒ 00:26:42.109 Speaker 2 (Elizabeth Conference Room): Add AI pack, I see the blue line.
339 00:26:46.320 ⇒ 00:26:47.430 Speaker 3 (Elizabeth Conference Room): She’s really cute.
340 00:26:47.540 ⇒ 00:26:49.580 Speaker 3 (Elizabeth Conference Room): Tracking the chances of those people are doing.
341 00:26:49.580 ⇒ 00:26:57.989 Robert Tseng: Seems like, yeah, everything kind of spiked during the launch, and then it kind of, like, kind of… Petered off. It seems like the AI, yeah, petered off. It’s still kind of there.
342 00:26:58.910 ⇒ 00:27:02.889 Speaker 2 (Elizabeth Conference Room): Okay, so people are still clicking that button.
343 00:27:03.010 ⇒ 00:27:06.120 Speaker 2 (Elizabeth Conference Room): Not at as high-rate as when it launched, but…
344 00:27:06.120 ⇒ 00:27:09.990 Speaker 1 (Elizabeth Conference Room): Now, I want a starter plan right now, and why can’t I.
345 00:27:09.990 ⇒ 00:27:12.839 Speaker 6 (Elizabeth Conference Room): It’s only available for business.
346 00:27:13.380 ⇒ 00:27:14.670 Speaker 6 (Elizabeth Conference Room): Oh, I think so.
347 00:27:15.120 ⇒ 00:27:21.010 Speaker 2 (Elizabeth Conference Room): And that was on our plan page, so… Hmm… Okay.
348 00:27:21.010 ⇒ 00:27:24.979 Robert Tseng: It sounds like we want to go further on the top feature usage thing.
349 00:27:24.980 ⇒ 00:27:27.480 Speaker 2 (Elizabeth Conference Room): Which I think, like…
350 00:27:27.830 ⇒ 00:27:28.550 Speaker 3 (Elizabeth Conference Room): Yeah.
351 00:27:28.550 ⇒ 00:27:31.849 Robert Tseng: I would be down to pick that back up.
352 00:27:32.480 ⇒ 00:27:37.029 Speaker 2 (Elizabeth Conference Room): I think it’d be helpful also, Robert, just to tell you, like, so our,
353 00:27:37.220 ⇒ 00:27:55.969 Speaker 2 (Elizabeth Conference Room): three focuses over the next couple of months. The team is focused on, as you know, we launched, like, README Refactored a year ago, but more than 50% of our revenue base is still not on the new, product. So, there’s a whole bunch of people focused on migrating enterprise customers to README Refactor.
354 00:27:55.970 ⇒ 00:27:58.769 Speaker 2 (Elizabeth Conference Room): That doesn’t touch, really, any of this. Then…
355 00:27:58.770 ⇒ 00:27:59.400 Robert Tseng: Okay.
356 00:27:59.730 ⇒ 00:28:06.759 Speaker 2 (Elizabeth Conference Room): Then, there’s also a ton of, like, general… oh, hey! Then there’s also a ton…
357 00:28:06.760 ⇒ 00:28:07.610 Uttam Kumaran: Listening in.
358 00:28:07.790 ⇒ 00:28:22.489 Speaker 2 (Elizabeth Conference Room): I was just talking through previous priorities. So, migrations, that’s like, we don’t need a ton of data to do that work, we know who needs to be migrated, and the issues they run into also. Then, there’s a general bucket of things around performance.
359 00:28:22.580 ⇒ 00:28:42.270 Speaker 2 (Elizabeth Conference Room): usability, reliability, we’ve run into a lot of issues recently, just on, like, even, like, loading states taking forever, or the site going down. Yeah. There’s a group of people internally focused on that. Not a ton of data needed there, although there might be some stuff around… and I don’t know if this is an amplitude thing, but…
360 00:28:42.270 ⇒ 00:28:44.380 Uttam Kumaran: Amplitude does have… Amplitude does have a lot.
361 00:28:44.380 ⇒ 00:28:52.350 Speaker 2 (Elizabeth Conference Room): a lot of stuff on loading, and… Oh, cool. Okay, so that’s… let’s table that for a second, but that’s something we might be interested in. And then the third is this…
362 00:28:52.420 ⇒ 00:28:57.419 Speaker 2 (Elizabeth Conference Room): All the growth-related activities around conversion.
363 00:28:57.420 ⇒ 00:29:16.770 Speaker 2 (Elizabeth Conference Room): I would also throw, like, self-serve retention in there, just, like, everything, and driving more leads and traffic to the site, so it’s, like, all of the stuff that fits into there, and we feel we’re still flying, like, pretty blind there on why people aren’t converting. I imagine it’s, in some ways, like, death by a thousand paper cuts, but there’s also probably a couple, like.
364 00:29:16.960 ⇒ 00:29:25.229 Speaker 2 (Elizabeth Conference Room): not-so-smart things we’re doing, like, for example, the AI trial not being accessible to people on multiple plans. You know, there’s, like, there’s stuff.
365 00:29:25.230 ⇒ 00:29:40.490 Speaker 2 (Elizabeth Conference Room): But I feel like we really need to establish a baseline of where we’re currently at before we can then start going in and implementing these experiments, tracking, iterating, moving quickly. So, like, the main, main focus, really, across the company on that growth
366 00:29:40.570 ⇒ 00:29:49.370 Speaker 2 (Elizabeth Conference Room): bucket is, like, current state, understanding current state, of the funnel. Yeah. And then, we’ll layer on the, like.
367 00:29:49.740 ⇒ 00:29:52.100 Speaker 2 (Elizabeth Conference Room): Data splunking of…
368 00:29:52.430 ⇒ 00:30:04.510 Speaker 2 (Elizabeth Conference Room): okay, well, people who launch have this sort of experience, so, like, how can we replicate that? You know what I mean? It’s like, that’s the part we layer on after. So to get to current state.
369 00:30:04.960 ⇒ 00:30:09.269 Speaker 2 (Elizabeth Conference Room): Really, what you all need up from us is… To improve defense.
370 00:30:10.430 ⇒ 00:30:13.669 Speaker 2 (Elizabeth Conference Room): To improve those. Better tracking on subscription, sign up.
371 00:30:13.960 ⇒ 00:30:22.109 Speaker 2 (Elizabeth Conference Room): and better tracking on… Free launch. Yeah, just like, I guess, plan types by launch.
372 00:30:22.880 ⇒ 00:30:23.430 Speaker 1 (Elizabeth Conference Room): land.
373 00:30:23.430 ⇒ 00:30:24.110 Robert Tseng: Yep.
374 00:30:24.110 ⇒ 00:30:27.479 Speaker 1 (Elizabeth Conference Room): I think it’s any project I’ve launched is not currently enough.
375 00:30:27.800 ⇒ 00:30:28.690 Speaker 1 (Elizabeth Conference Room): Right.
376 00:30:32.415 ⇒ 00:30:33.745 Speaker 2 (Elizabeth Conference Room): Just, just free.
377 00:30:34.290 ⇒ 00:30:37.360 Speaker 1 (Elizabeth Conference Room): Yeah, but that’s, like, that’s basically, like, any project, right?
378 00:30:37.735 ⇒ 00:30:40.594 Speaker 2 (Elizabeth Conference Room): Well, but the ones that are on paid plans are in there.
379 00:30:40.880 ⇒ 00:30:46.349 Speaker 1 (Elizabeth Conference Room): Isn’t that, like, a… isn’t paid a subset of the 227, or is it separate? Like, is it, like, almost… It is.
380 00:30:46.350 ⇒ 00:30:47.469 Robert Tseng: It’s a subset, yeah.
381 00:30:47.470 ⇒ 00:30:49.509 Speaker 1 (Elizabeth Conference Room): It’s a supplement, so then.
382 00:30:49.510 ⇒ 00:30:50.010 Robert Tseng: Yeah.
383 00:30:50.010 ⇒ 00:30:53.780 Speaker 1 (Elizabeth Conference Room): However we want to define it, but… The projects that launched.
384 00:30:55.090 ⇒ 00:30:58.450 Speaker 1 (Elizabeth Conference Room): Right? Like, that’s relaunched, basically. Projects that launch.
385 00:30:59.580 ⇒ 00:31:00.180 Robert Tseng: Yes.
386 00:31:00.180 ⇒ 00:31:01.200 Speaker 4 (Elizabeth Conference Room): On the free plan.
387 00:31:01.950 ⇒ 00:31:02.660 Speaker 4 (Elizabeth Conference Room): on.
388 00:31:02.990 ⇒ 00:31:13.749 Robert Tseng: Yeah, because every project… every project will launch first, and then you can switch it to a paid launch. You can switch it to paid, or whatever. Like, you’re not, like, going… yeah, like, paid is a subset of the free launch, yeah.
389 00:31:13.750 ⇒ 00:31:16.130 Speaker 7 (Elizabeth Conference Room): Cool. Yes.
390 00:31:17.710 ⇒ 00:31:21.230 Speaker 4 (Elizabeth Conference Room): User sign up, project sign-up. Oh, I see.
391 00:31:23.760 ⇒ 00:31:24.890 Robert Tseng: You see what I mean? Like, before we.
392 00:31:24.890 ⇒ 00:31:25.470 Speaker 3 (Elizabeth Conference Room): we’re trying to get.
393 00:31:25.470 ⇒ 00:31:28.850 Robert Tseng: guess at free launch, because we use this proxy of attempted launch.
394 00:31:28.850 ⇒ 00:31:29.330 Speaker 3 (Elizabeth Conference Room): Yeah.
395 00:31:29.330 ⇒ 00:31:37.899 Robert Tseng: is actually broken, it doesn’t actually flow through well to paid. Well, the tracking is working if we just really view it as, like, a subset of all.
396 00:31:37.900 ⇒ 00:31:46.260 Speaker 1 (Elizabeth Conference Room): comes from project created, do we want to do, like, two, like, a fork or a funnel? Yeah. Which I don’t have a strong… I don’t either, yeah.
397 00:31:46.260 ⇒ 00:32:03.440 Speaker 2 (Elizabeth Conference Room): Okay, sorry, I have to jump to this other meeting. But we will follow up. We have end resources now that are really interested in helping, so we’ll follow up on tracking those two events. But again, again, like, appreciate all the work. I think the biggest thing we can do right now, like, I don’t really want to focus on any of the other stuff until we do…
398 00:32:03.770 ⇒ 00:32:05.910 Speaker 2 (Elizabeth Conference Room): Baseline, where we’re currently at.
399 00:32:06.350 ⇒ 00:32:09.390 Speaker 2 (Elizabeth Conference Room): But that’s on us to get to. Okay. Okay, thanks.
400 00:32:10.590 ⇒ 00:32:11.720 Robert Tseng: Alright, thank you.
401 00:32:13.600 ⇒ 00:32:14.660 Uttam Kumaran: How’d it go?
402 00:32:15.520 ⇒ 00:32:18.930 Robert Tseng: I mean, you heard it, it’s a broken record, they’re saying the same thing over and over again.
403 00:32:18.930 ⇒ 00:32:26.660 Uttam Kumaran: Well, what do you think, like, we’re not doing well? Like, I… should I have Mustafa? Like, I just basically said he should just create a dashboard for all those priorities, maybe, like…
404 00:32:27.480 ⇒ 00:32:32.009 Robert Tseng: No, what we did was… what Mustafa did was the best that we could do for now.
405 00:32:32.010 ⇒ 00:32:34.849 Uttam Kumaran: So, like, I basically showed that… Yeah, I started racing.
406 00:32:34.850 ⇒ 00:32:35.500 Robert Tseng: table.
407 00:32:35.680 ⇒ 00:32:37.039 Robert Tseng: So, like, kind of like…
408 00:32:37.040 ⇒ 00:32:44.280 Uttam Kumaran: should you take? Because, like, what… for me, my question to them was, like, even if I got you, what can you do easily? Like, can you make
409 00:32:44.540 ⇒ 00:32:50.280 Uttam Kumaran: product decisions? Can you make pricing decisions? Can you make copy changes? Like… It sounds like they just.
410 00:32:50.280 ⇒ 00:32:53.939 Robert Tseng: I think they’re thinking about it the wrong way, like, yeah.
411 00:32:54.270 ⇒ 00:32:55.500 Robert Tseng: Might,
412 00:32:55.780 ⇒ 00:33:04.149 Robert Tseng: I understand that they’re trying to set up baseline. I mean, when you’re trying to, like, wait for the story to, like, get better, it’s not great. Like, I think…
413 00:33:04.400 ⇒ 00:33:10.459 Robert Tseng: you trust the projects that are created, you trust the launches, that’s… that’s enough. Like, sign up is, like, kind of a…
414 00:33:10.650 ⇒ 00:33:11.930 Robert Tseng: Whatever, like, I…
415 00:33:12.140 ⇒ 00:33:17.630 Robert Tseng: I mean, I think what we’ve learned from working with all of these… all our clients, I think
416 00:33:17.740 ⇒ 00:33:21.139 Robert Tseng: The growth funnel is really kind of, like.
417 00:33:22.140 ⇒ 00:33:35.889 Robert Tseng: three… three parts. There’s, like, the CRO side, where there’s some intake or sign up… intake for a CPG telehealth company, or, like, a sign-up process for a SaaS company.
418 00:33:36.300 ⇒ 00:33:53.219 Robert Tseng: You can run a lot of fast experiments there, it’s just, like, random UI changes for, you know, changing form answers or whatever, trying to, like, get the right… you can do some optimization there. Then there’s the onboarding part, like, how do you get the people who have signed up to take that first, like, meaningful action?
419 00:33:53.730 ⇒ 00:34:10.189 Robert Tseng: And then you can… then the… then, like, the… then the third part is, like, like, what is keeping them coming… coming back? Or, like, how do they actually become, like, high-value, like, users, or whatever? So, I do think that these are 3 parallel workstreams.
420 00:34:10.360 ⇒ 00:34:23.299 Robert Tseng: CRO should not be so inhibited, like, which is, like, that is the biggest… that’s the leaky bucket for them, like, they’re just… that’s… that’s where it is, and I don’t know why they’re moving so slowly there, like, they just need to run a lot of experiments there.
421 00:34:24.040 ⇒ 00:34:36.380 Uttam Kumaran: So if you were to pick one of those areas, and we were to drive them towards a change, do you see one… I mean, you can evaluate in multiple criterias, right? Something that could work faster, easiest for them to implement.
422 00:34:36.389 ⇒ 00:34:36.779 Robert Tseng: Yeah.
423 00:34:36.780 ⇒ 00:34:41.099 Uttam Kumaran: like, if we were to pick one, and I was to, in my mind, be like, okay, let’s drive towards
424 00:34:41.389 ⇒ 00:34:46.639 Uttam Kumaran: Getting the data in one place and saying, you should make these decisions, like, which part of that would it be?
425 00:34:48.590 ⇒ 00:34:55.300 Robert Tseng: So I’m looking… their CRO right now is, like…
426 00:34:56.000 ⇒ 00:34:59.009 Robert Tseng: Assignment excess and created 50% low.
427 00:34:59.420 ⇒ 00:35:04.390 Uttam Kumaran: I mean, I feel like the easiest, in my mind, is you have… you have people that have already signed up.
428 00:35:04.680 ⇒ 00:35:10.280 Uttam Kumaran: Who are not using it, focus on Notifications, emails, like…
429 00:35:11.730 ⇒ 00:35:25.830 Robert Tseng: Yeah, there’s all marketing levers to get them to come on, but, you know, this group of… their product and BizOps, like, yeah, I don’t think they… I don’t think they know anything about top of funnel, which is why they’re just, like, spinning their wheels there.
430 00:35:26.060 ⇒ 00:35:33.550 Robert Tseng: do they even have a product marketing person who has, like, the resources to be able to go turn it on? But yeah, like, that would be it. Just, like, you…
431 00:35:38.650 ⇒ 00:35:40.329 Uttam Kumaran: Wait, your mic’s… your mic’s cutting out.
432 00:35:43.750 ⇒ 00:35:45.330 Uttam Kumaran: Oh, you’re back. You’re back, you’re back.
433 00:35:45.940 ⇒ 00:35:46.790 Robert Tseng: Okay, okay.
434 00:35:46.930 ⇒ 00:35:49.860 Robert Tseng: I was gonna say, I mean, I was saying,
435 00:35:50.350 ⇒ 00:36:03.319 Robert Tseng: Yeah, they have 3,500 people signing up for their product, and less than 2,000 of them creating projects. Like, just getting those signups to go back in to create projects is, like, one way to just fill their funnel.
436 00:36:04.810 ⇒ 00:36:14.380 Robert Tseng: like, getting… solving the paid problem seems like that’s… that’s the harder problem to solve, though. And, like, that’s what… that’s the more meaningful problem to solve.
437 00:36:14.940 ⇒ 00:36:18.969 Uttam Kumaran: But dude, you can’t move people to paid if they’re not even using your free product.
438 00:36:20.450 ⇒ 00:36:23.280 Robert Tseng: Yeah, I mean, yeah, if you’re only getting…
439 00:36:23.510 ⇒ 00:36:28.940 Robert Tseng: 200 people trying to launch their product, and 30 of them are converting. Like, that’s…
440 00:36:29.190 ⇒ 00:36:35.609 Robert Tseng: you know, it’s above 10%, it’s not, like, insignificant. Yeah, you just need more people trying to do that. So, I mean, it’s… it’s…
441 00:36:35.920 ⇒ 00:36:40.369 Robert Tseng: I think it is a top-of-funnel problem for the size that they’re at, like…
442 00:36:40.370 ⇒ 00:36:45.410 Uttam Kumaran: But top of funnel is in they have bad… like, they’re not getting enough, or they… there’s not…
443 00:36:46.080 ⇒ 00:36:48.960 Uttam Kumaran: Great customers, they’re not great. Yeah.
444 00:36:48.960 ⇒ 00:36:51.999 Robert Tseng: Sorry, top of funnel’s not the right way to put it. Just like,
445 00:36:52.200 ⇒ 00:36:57.960 Robert Tseng: getting signed-up users to, like, do a meaningful action. Like, that’s… that’s, like, the biggest drop-off.
446 00:36:57.960 ⇒ 00:36:59.140 Uttam Kumaran: Yeah, okay, okay.
447 00:36:59.340 ⇒ 00:37:13.490 Uttam Kumaran: Yeah, I agree. I think it’s that, and then the last part. So, what I’m gonna kind of do is maybe we try to drive… I wanna… I want there to be… I don’t want there to be conversations on, like, we need a baseline, we need a baseline. So, we’ll get them to a baseline view.
448 00:37:13.600 ⇒ 00:37:15.719 Robert Tseng: Yeah. But then I want to be, like.
449 00:37:15.720 ⇒ 00:37:19.499 Uttam Kumaran: Here are, like, 3 things you should try, like, what’s stopping us from trying that?
450 00:37:19.890 ⇒ 00:37:31.610 Uttam Kumaran: And if they’re not… if they’re not willing to try, then there’s nothing we can do. And, like, that’s how we should push these folks. I don’t think we’re gonna be able to get to their marketing stuff.
451 00:37:31.770 ⇒ 00:37:38.930 Uttam Kumaran: I think if… she said she has some engineering resources, so if we can give them clear engineering or product changes to consider.
452 00:37:39.250 ⇒ 00:37:44.519 Uttam Kumaran: It may be looking at the data and then kind of going through the product and coming up with some ideas, but…
453 00:37:45.310 ⇒ 00:37:45.990 Robert Tseng: Yeah.
454 00:37:45.990 ⇒ 00:37:46.600 Uttam Kumaran: Yeah.
455 00:37:48.520 ⇒ 00:37:49.030 Robert Tseng: Yep.
456 00:37:49.030 ⇒ 00:37:51.289 Uttam Kumaran: So, free to… so it’s basically…
457 00:37:51.400 ⇒ 00:37:58.520 Uttam Kumaran: Free to paid, and then also new user to… Just generally, like, active user.
458 00:37:59.490 ⇒ 00:38:00.070 Robert Tseng: Yeah.
459 00:38:00.430 ⇒ 00:38:00.980 Uttam Kumaran: Okay.
460 00:38:03.620 ⇒ 00:38:05.590 Uttam Kumaran: Okay, cool.
461 00:38:07.160 ⇒ 00:38:08.340 Uttam Kumaran: How’s everything, dude?
462 00:38:09.850 ⇒ 00:38:11.019 Robert Tseng: Dude, I’m tired, I’m glad.
463 00:38:11.020 ⇒ 00:38:13.540 Uttam Kumaran: Alright, alright, alright.
464 00:38:13.540 ⇒ 00:38:15.599 Robert Tseng: Yeah, I’ll talk to you later.
465 00:38:15.600 ⇒ 00:38:17.410 Uttam Kumaran: Alright, I’ll talk to you later. Alright, bye.