Meeting Title: ReadMe <> Brainforge Check-In Date: 2025-10-06 Meeting participants: Robert Tseng, Alicia Shin, Justin Breshears, Henry Zhao
WEBVTT
1 00:02:09.220 ⇒ 00:02:10.660 Alicia Shin: Hey, can you hear me?
2 00:02:13.200 ⇒ 00:02:15.420 Robert Tseng: Hey, Alicia! Can you hear me?
3 00:02:15.560 ⇒ 00:02:16.180 Alicia Shin: Yes.
4 00:02:16.390 ⇒ 00:02:18.629 Robert Tseng: Sorry, I was muting… I muted myself.
5 00:02:19.200 ⇒ 00:02:20.050 Alicia Shin: You’re good.
6 00:02:27.220 ⇒ 00:02:41.940 Robert Tseng: I have Justin and Henry on my team joining. I’ll intro them when they jump… when they jump on, but… oh, I guess Justin’s here. So Justin is, yeah, just our lead PM, and so he’s… yeah, he just likes to hover around all of our
7 00:02:42.040 ⇒ 00:02:50.650 Robert Tseng: our meetings and kind of help us get organized, especially as we’re trying to get ramped up again. And then Henry is just, he’s on our…
8 00:02:50.820 ⇒ 00:02:57.240 Robert Tseng: I guess you’ll be the other analyst working with me, just so we have some redundancy, we can kind of keep momentum going.
9 00:02:57.390 ⇒ 00:03:03.819 Robert Tseng: Yeah, and then, I don’t know, I guess, guys, this is Alicia, Alicia’s…
10 00:03:04.070 ⇒ 00:03:12.260 Robert Tseng: our main stakeholder for this phase of, engagement. I guess maybe a couple of you have met Phoebe before, but not Alicia, so…
11 00:03:12.400 ⇒ 00:03:18.879 Robert Tseng: I guess today we’re just gonna read out on Jenna’s some work that we kicked off late last week, and then
12 00:03:19.040 ⇒ 00:03:23.030 Robert Tseng: Yeah, just kind of see what adjustments we need to make to kind of keep going from here.
13 00:03:24.400 ⇒ 00:03:25.479 Robert Tseng: Does that sound good?
14 00:03:27.560 ⇒ 00:03:40.740 Robert Tseng: Yeah, Alicia, I guess I’ll let you kind of, any questions that you wanted to ask first, we can kind of take note of that, and then I can open up both docs that I shared with you, and kind of dig into the details a bit more, if that’s helpful.
15 00:03:41.290 ⇒ 00:03:43.569 Robert Tseng: Yeah, no, I mean, everything looked…
16 00:03:43.570 ⇒ 00:03:49.380 Alicia Shin: Great, as I was going through, I think… My one reaction was…
17 00:03:49.610 ⇒ 00:04:07.009 Alicia Shin: Or I guess my question was, could customers be taking longer than one month to convert to paid? Yeah. We’re seeing, like, that drop-off, but I know this is just, like, an initial, pull to make sure we’re measuring things correctly before we dive in deeper. Yeah.
18 00:04:07.020 ⇒ 00:04:20.419 Alicia Shin: I feel like we’re pretty close. I’m happy to dig into some of the questions that you had to make sure we feel, like, great about the data, and if I don’t know the answers offhand, I will take the action item to go internal.
19 00:04:20.820 ⇒ 00:04:22.310 Robert Tseng: Okay, that sounds good.
20 00:04:23.210 ⇒ 00:04:34.970 Robert Tseng: I will kinda just… Share my screen… Let’s see… Hopefully this works.
21 00:04:40.600 ⇒ 00:04:50.249 Robert Tseng: Yeah, so just to kind of… you know, I just kind of took your initial requirements, I put it into a structure that helps me to kind of outline my thinking a bit more, and then I… I just…
22 00:04:50.360 ⇒ 00:05:01.349 Robert Tseng: when we don’t have a clear dashboard, I typically use amplitude notebooks to kind of get started. So yeah, I already kind of did like the readout. Seems like you already read through things. You’re right, I made the 30-day assumption here.
23 00:05:01.500 ⇒ 00:05:09.390 Robert Tseng: kind of question… I mean, I did kind of, like, look at… I expanded that range and didn’t really see too… too much of an adjustment, but…
24 00:05:09.610 ⇒ 00:05:19.370 Robert Tseng: I wonder, for users of different plan types, like, would you expect there to be different behavior? Because that’s kind of… that’s one of the things that I was noticing that
25 00:05:19.380 ⇒ 00:05:33.720 Robert Tseng: Seems like free plan users are kind of speeding through the process, whereas, like, maybe the enterprise or business plan users seem to be taking longer. I wasn’t sure if that was a data quality issue, or if that’s something that, anecdotally, you’ve actually, you know, you’ve seen before.
26 00:05:36.770 ⇒ 00:05:50.969 Alicia Shin: I actually don’t know if Enterprise is in scope for this, so I can confirm that, but yeah, generally, I would expect that, like, free… free users will, will go through quickly, and then, like, if they don’t want to pay, they’ll probably just stay on that free plan, and so they’ll convert.
27 00:05:51.330 ⇒ 00:05:53.689 Alicia Shin: To… to free or downgrade.
28 00:05:53.690 ⇒ 00:05:54.360 Robert Tseng: Yeah.
29 00:05:54.620 ⇒ 00:06:11.489 Alicia Shin: And then, yeah, I would expect paying customers to go more. Right now, the way we track our funnel today is more of, like, a screenshot point in time, and so we’re not doing that cohorted view. I don’t know what the reaction will be, but if this is, like, what the data’s telling us, then this is what we have to go off of.
30 00:06:12.050 ⇒ 00:06:12.750 Robert Tseng: Okay.
31 00:06:12.990 ⇒ 00:06:24.800 Robert Tseng: Cool. Well then I’ll kind of double-click into that a bit more. I know you had… you highlighted a few funnel stages. It’s a little bit different from what Phoebe had originally asked me to outline, so I kind of just took some views that we had before and I repurposed them.
32 00:06:24.800 ⇒ 00:06:41.029 Robert Tseng: I did clarify kind of, like, why these stages were kind of the way they are. I will say we can get downgrade from trial, I just didn’t build it out. There’s a little bit… it’s a little bit more involved to get there, so, I was just kind of trying to get something up quickly so we could start to chat directionally about what we’re seeing.
33 00:06:41.100 ⇒ 00:06:44.069 Robert Tseng: But yeah, I think, kind of, as I had mentioned.
34 00:06:44.110 ⇒ 00:06:51.480 Robert Tseng: I do think that the… I mean, you know, for each of these drop-off points, I would just want to be able to confirm it with this source of truth.
35 00:06:51.510 ⇒ 00:07:08.249 Robert Tseng: So, last time we talked, we talked, we mentioned, be able to map sign-ups to actually what you see in, kind of, the production database, and then we talked about, you know, we didn’t really have, like, the exact payment event, because Stripe, payments were not flowing into amplitude.
36 00:07:08.250 ⇒ 00:07:18.199 Robert Tseng: So we’re basically trying to use a proxy, which, obviously, attempted launch is not the best, because I show later on that 95% of attempted launches don’t actually turn into subscriptions.
37 00:07:18.230 ⇒ 00:07:22.120 Robert Tseng: And even if you get closer in that,
38 00:07:22.570 ⇒ 00:07:28.880 Robert Tseng: You know, there’s a few steps, you know, after attempted launch, where they’re viewing, like, a plan, then they’re actually logging the change.
39 00:07:29.270 ⇒ 00:07:40.220 Robert Tseng: This is probably the closest event we could get to before the pavement event. But yeah, even this is still, you know, like, a significant drop-off before, you know,
40 00:07:40.760 ⇒ 00:07:45.819 Robert Tseng: A successful payment, a plan change actually gets registered.
41 00:07:45.920 ⇒ 00:08:02.970 Robert Tseng: So I just kind of wanted to call out that there’s probably, like, a better proxy that we could get to here, and then wanting to make sure, obviously, that this lines up with what you’re seeing in Stripe. So I would say those are the checks that I would want to run in order to make sure that this funnel is actually what we’re seeing here.
42 00:08:03.700 ⇒ 00:08:09.319 Alicia Shin: And then the green versus pink, can you just voice over what previous users means?
43 00:08:09.320 ⇒ 00:08:28.400 Robert Tseng: Yeah, I wish I could edit this, but this is really a previous time period, so I know I’m only looking at pink as, like, the month of September, and so you could think of green as, like, August. Yeah, I think… I wasn’t really sure on the timing of the pricing changes and the feature launches, and figured we should just use, like,
44 00:08:28.540 ⇒ 00:08:41.719 Robert Tseng: like, a more recent, time period, and I just chose monthly, just to start. So, if you would prefer to kind of cut these periods however, like, in a different way, just, you know, let us know, we can make those changes.
45 00:08:41.720 ⇒ 00:08:56.279 Alicia Shin: Okay. I think just being able to filter on different time periods is what we would look for, so no worries there. Okay. So, for the ability to, like, use Stripe, how would that work? Like, do we need to integrate Stripe with Amplitude? Like, what was the blocker there?
46 00:08:56.560 ⇒ 00:09:09.129 Robert Tseng: Yeah, so you can… I don’t believe that Stripe does a direct integration with Amplitude, so what we’ve done with other clients is, yeah, typically, you know, a successful payment
47 00:09:09.270 ⇒ 00:09:18.190 Robert Tseng: You know, that event fires into your… wherever you’re logging things in your database, and then you can basically send that event into,
48 00:09:18.190 ⇒ 00:09:31.600 Robert Tseng: into amplitude. So, from an instrumentation perspective, right now, you have, like, a JavaScript-based kind of, like, auto tracker setup, where, you know, pretty much every page view, click, and kind of those types of events are all being logged.
49 00:09:31.600 ⇒ 00:09:45.720 Robert Tseng: But you’re not necessarily, like, passing in server events from your server. So, I think that’s typically what you would want to do if you want, like, higher fidelity data on, like, those key, milestones.
50 00:09:46.800 ⇒ 00:09:50.480 Alicia Shin: Okay, so what I’m hearing is that…
51 00:09:51.170 ⇒ 00:10:05.920 Alicia Shin: to get the subscription data, like, the most accurate data, you would want our database… Correct. …to be pulling from Amplitude? Okay. And that Stripe, there is no way to pull that from Stripe, like, with existing integrations.
52 00:10:06.130 ⇒ 00:10:08.630 Robert Tseng: Yeah, I have not seen it done.
53 00:10:09.110 ⇒ 00:10:10.889 Robert Tseng: Between Stripe and Amplitude.
54 00:10:11.020 ⇒ 00:10:11.620 Alicia Shin: Correct.
55 00:10:11.890 ⇒ 00:10:18.689 Alicia Shin: I will take that. I don’t know who would give access or how we would do that, but I will take the action item.
56 00:10:20.700 ⇒ 00:10:21.220 Robert Tseng: Okay.
57 00:10:23.110 ⇒ 00:10:23.700 Alicia Shin: Cool.
58 00:10:24.500 ⇒ 00:10:25.200 Robert Tseng: Yeah.
59 00:10:27.110 ⇒ 00:10:36.669 Robert Tseng: And then kind of just walking through these other pieces. So this one, I was just kind of going through each step of the funnel, so from user sign-ups to when the first project is created.
60 00:10:37.810 ⇒ 00:10:44.369 Robert Tseng: Or I guess I didn’t limit it to first time, so maybe that’s a filter that I need to… I need to go in and change.
61 00:10:44.370 ⇒ 00:10:57.990 Robert Tseng: But basically, you know, same thing, everything was just been over… conversion windows are all… actually, this is only 7 days. I mean, I limited this one more, because if they’re signing up and not creating a project within their first 7 days, like.
62 00:10:58.010 ⇒ 00:11:08.779 Robert Tseng: I don’t really know how much that, you know, really… if that’s actually a real user or not. But yeah, so I think, you know, you can… you can see there’s some… some takeaways here about…
63 00:11:08.780 ⇒ 00:11:26.390 Robert Tseng: Yeah, like, once again, the free users are kind of the lower plans, lower tier plans, they’re… they’re sprinting through this process, or at least they’re getting to their first project, whereas, like, the enterprise or business users seem to be slower, and not really, you know, taking… taking these steps within the first week.
64 00:11:27.550 ⇒ 00:11:45.260 Robert Tseng: Yeah, so that’s all that this was saying. I had a couple questions here, seems like Enterprise is out of scope, and then I was wondering, okay, is there a different flow for them, possibly, where, maybe it’s an account-level sign-up, and then projects are automatically created for their users?
65 00:11:45.460 ⇒ 00:11:54.559 Robert Tseng: I’m not sure, because I haven’t been on the Enterprise product, or I’m not really sure if it’s any different. So I was just kind of throwing out some, like, thoughts there.
66 00:11:54.950 ⇒ 00:12:15.130 Alicia Shin: Yeah, I think we definitely want to track, like, clicking chat with sales, or any sort of, like, upsell, like, behavior, but in terms of tracking, like, Enterprise has their own funnel that we go through that involves, sort of, the sales team, and so it’s… it feels like superimposing the self-serve funnel into Enterprise might not make sense here. Yeah.
67 00:12:15.490 ⇒ 00:12:30.110 Alicia Shin: And then, to your second question, I believe our enterprise customer team does create, like, a group, and then works, like, hand-in-hand with the customer. Okay. So, but I… that’s not… that shouldn’t be the case for business. They should just be going through the self-serve flow.
68 00:12:30.560 ⇒ 00:12:37.209 Robert Tseng: Okay, got it. Well, these are all the different plan types that we have as well, so is there anything here that kind of pops out as, like.
69 00:12:37.440 ⇒ 00:12:50.150 Robert Tseng: surprising to you, or, like, we were missing a plan, perhaps, but we can exclude Enterprise from this analysis in the future. But, like, free, free launch, none, seems like these are all the same thing, not really sure why they’re labeled differently.
70 00:12:50.460 ⇒ 00:12:56.709 Alicia Shin: Yeah, so that’s… that’s 1,800 people with no plan that were created in September.
71 00:12:57.020 ⇒ 00:12:58.030 Robert Tseng: Correct.
72 00:12:58.210 ⇒ 00:13:02.020 Robert Tseng: Or signed up, signed up to create… oh yeah, you’re right, to create it, yep.
73 00:13:02.350 ⇒ 00:13:03.840 Alicia Shin: Yeah, that’s surprising to me.
74 00:13:04.050 ⇒ 00:13:04.620 Robert Tseng: Okay.
75 00:13:04.620 ⇒ 00:13:12.290 Alicia Shin: I wonder if they should just be marked as free. I’ll flag that, or try to look into that. I’m not sure if I’ll have an answer this week, but…
76 00:13:12.400 ⇒ 00:13:13.830 Alicia Shin: I will look into it.
77 00:13:14.270 ⇒ 00:13:14.900 Robert Tseng: Okay.
78 00:13:17.190 ⇒ 00:13:22.169 Alicia Shin: I feel like they should automatically be free or free launch, so… I don’t know what’s going on there.
79 00:13:23.720 ⇒ 00:13:30.130 Robert Tseng: Yeah, I believe when I created mine, I automatically was assigned to…
80 00:13:30.890 ⇒ 00:13:32.539 Robert Tseng: Maybe it was because I was…
81 00:13:33.250 ⇒ 00:13:47.089 Robert Tseng: well, for whatever reason, when… on my accounts, which I’m sorry, I didn’t actually send it to you, I know you were gonna give me… provision me some access to something else, I had to, like, downgrade into, like, a… and to get back into no plan, but
82 00:13:47.160 ⇒ 00:13:54.189 Robert Tseng: Not really sure. I guess I could… I could just go through the workflow again, and I’ll send it to you if that was… if that’s not really fair.
83 00:13:54.420 ⇒ 00:13:55.930 Alicia Shin: Okay, sounds good.
84 00:13:56.550 ⇒ 00:14:05.330 Robert Tseng: Yeah, so then the next piece is, you know, a project created to, like, launch, and kind of trying to understand, like, you know, what, what is…
85 00:14:05.830 ⇒ 00:14:23.179 Robert Tseng: how much intent is in, like, the attempted launch, and then, like, every step afterwards. I mean, it seems like, you know, 20% of people are clicking attempted launch. I think that’s just really people just testing out different features. It’s… I don’t really think there’s high intent, obviously.
86 00:14:23.180 ⇒ 00:14:25.320 Alicia Shin: I think this view is interesting.
87 00:14:25.490 ⇒ 00:14:34.060 Robert Tseng: This is… I was cohorting it by week, so users who created a project and attempted launch within their first week, so it’s about 4%,
88 00:14:34.410 ⇒ 00:14:52.859 Robert Tseng: And then, we… we tried it in week two, then we tried… tried it in week three, and then week 4, or whatever. So, yeah, you kind of flagged that, like, there… this dip in August kind of did not surprise you, because there was something… something that went… some feature that was off or something.
89 00:14:53.350 ⇒ 00:15:01.859 Alicia Shin: I think it was, end of August and caught in September, like, mid-September, so I’m not sure how the rest of this chart will start looking, but,
90 00:15:01.860 ⇒ 00:15:02.530 Robert Tseng: I see.
91 00:15:02.530 ⇒ 00:15:06.020 Alicia Shin: Yeah, in any case, there was, like, a… there was a product issue.
92 00:15:06.260 ⇒ 00:15:07.580 Robert Tseng: Okay, got it.
93 00:15:09.190 ⇒ 00:15:21.860 Robert Tseng: Yeah, well, so, I think the story is consistent if you kind of just change, like, the conversion event from attempted launch to, like, an actual successful subscription. It, you know, suffers a dip around the same time.
94 00:15:22.050 ⇒ 00:15:30.480 Robert Tseng: doesn’t look like things are stable, necessarily. Like, I don’t really know why it’s so volatile week to week, but I assume…
95 00:15:30.690 ⇒ 00:15:42.709 Robert Tseng: yeah, I don’t know if these are tied to any promotions that you guys are doing, like, I’m not really sure what the levers are for this right now. I think we probably have to drill this… drill into it a bit more to see, like, why are people taking, you know.
96 00:15:42.710 ⇒ 00:15:54.150 Robert Tseng: four weeks to actually go through this flow versus the first week, and, yeah, I don’t really have that level of, detail in what I’ve seen so far.
97 00:15:54.600 ⇒ 00:16:12.180 Alicia Shin: And then, just to make sure I’m understanding this chart correctly, so this is about, like, 4-5% of people who create a project are then completing the payment workflow, and so they’re not downgrading free, they’re selecting startup or business, or how are you defining subscription success?
98 00:16:12.380 ⇒ 00:16:17.729 Robert Tseng: Yeah, I guess we should filter this by… type…
99 00:16:18.500 ⇒ 00:16:24.260 Robert Tseng: Maybe we should just limit it to non-free. This might… actually, that might change the story.
100 00:16:24.660 ⇒ 00:16:25.350 Alicia Shin: Okay.
101 00:16:27.780 ⇒ 00:16:38.400 Robert Tseng: Yeah, no, I’m saying it’s the same thing. So, a subscription success event, I know, this wouldn’t fire unless somebody changed from a free to a paid subscription, like, so…
102 00:16:38.790 ⇒ 00:16:43.110 Robert Tseng: Yeah, I think that’s why, even if you don’t specify the plant type, it looks exactly the same.
103 00:16:43.230 ⇒ 00:17:00.300 Robert Tseng: So, yeah, I mean, but that could also mean, which maybe that’s being conflated in here, people who are on a paid plan switching to a different paid plan, but I don’t know how often that’s actually happening. So, I mean, I would say it’s a fair assumption to assume that it’s mostly free to paid
104 00:17:00.420 ⇒ 00:17:02.090 Robert Tseng: Users here, yeah.
105 00:17:02.090 ⇒ 00:17:04.400 Alicia Shin: Okay, cool. This, this is a helpful one.
106 00:17:05.020 ⇒ 00:17:05.619 Robert Tseng: Cool.
107 00:17:05.890 ⇒ 00:17:15.120 Robert Tseng: But yeah, I think you were kind of saying maybe these weekly views are too… too short. I mean, we could expand this range, we could look at monthly, and we could look at it past 6 months.
108 00:17:15.220 ⇒ 00:17:18.280 Robert Tseng: I just didn’t want to look at it too much.
109 00:17:18.819 ⇒ 00:17:24.699 Robert Tseng: Yeah, I mean, nothing that surprising there, so maybe this is still the best view.
110 00:17:25.029 ⇒ 00:17:26.179 Alicia Shin: Okay.
111 00:17:27.089 ⇒ 00:17:27.929 Robert Tseng: Okay.
112 00:17:31.029 ⇒ 00:17:35.149 Robert Tseng: And then, yeah, so moving on…
113 00:17:35.259 ⇒ 00:17:49.149 Robert Tseng: Yeah, I guess we’re now talking about the specific, like, launch to subscription success, kind of what are the drop-offs there. So, obviously, everybody who attempts launch, they at least check out the different plans. Some begin the change workflow.
114 00:17:49.239 ⇒ 00:17:58.039 Robert Tseng: And then, like, very few of them actually go and actually convert. So, I mean, I kind of just threw out some, like, stream of consciousness ideas, like.
115 00:17:58.279 ⇒ 00:18:06.469 Robert Tseng: You know, once users are getting to plans, like, how, you know, what kind of intervention can we make to basically,
116 00:18:06.639 ⇒ 00:18:21.909 Robert Tseng: you know, impact this drop-off, because this is, like, where… this is where the most significant piece is happening. So, yeah, I don’t know if that’s a priority, or if that’s part of the pricing changes that the team is working on. Yeah.
117 00:18:22.990 ⇒ 00:18:24.760 Alicia Shin: Yeah, I think,
118 00:18:26.000 ⇒ 00:18:38.210 Alicia Shin: the answer is, like, yes, and this will allow us to start monitoring, like, some of the initial features that we start rolling out, and some of the changes. We’ll be able to see, like, is there any impact? I’m not really…
119 00:18:38.360 ⇒ 00:18:57.550 Alicia Shin: I mean, I’m not surprised that there’s, like, a drop-off towards the end. I just double-checked with the team, so they’re changing our customer billing hub, but that won’t have any impact to, like, this workflow. So really, like, the changes that I would expect to impact this would be, like, the rollout of the, the trial.
120 00:18:57.550 ⇒ 00:19:11.580 Alicia Shin: we have, like, an AI launch that’s coming out tomorrow, and, like, a couple other things that will start to be rolled out over the next few weeks. So I think those will be the things that we can, like, monitor, like, impact week over week, that we have this in place.
121 00:19:12.120 ⇒ 00:19:20.969 Robert Tseng: Okay, cool. So, yeah, that flat launch is happening next week, you said? Yeah, maybe we’ll just work… kind of take note of that launch date, and we’ll kind of measure.
122 00:19:21.160 ⇒ 00:19:31.840 Alicia Shin: It’s actually tomorrow, I think 1PM Eastern is, it’s, like, a AI booster pack that I think we’re, like, formally rolling out, so it would be, like, an add-on.
123 00:19:31.840 ⇒ 00:19:44.350 Robert Tseng: Okay, very cool. I mean, I did see that in the pricing page, that, like, hey, this feature is coming soon, or whatever. I know it wasn’t live, so, okay, so I’ll make a note of that, here as, like, possibly another segment, or, like, just a different review.
124 00:19:44.470 ⇒ 00:19:45.680 Alicia Shin: Awesome.
125 00:19:46.140 ⇒ 00:19:51.840 Robert Tseng: Yeah, and then we kind of had some thoughts about, like, okay, well, what about, not just looking at
126 00:19:52.000 ⇒ 00:20:07.460 Robert Tseng: segmentation by plant type, but by feature usage. So, this was just the first product I created. We can go much deeper into this. This is just, like, a random… or it lists all the ones that I kind of checked off. I know you specifically asked for GitBiddy, but I didn’t really know what that was, to be honest.
127 00:20:07.460 ⇒ 00:20:07.819 Alicia Shin: Oh my god.
128 00:20:08.000 ⇒ 00:20:08.680 Robert Tseng: Yeah.
129 00:20:08.680 ⇒ 00:20:11.090 Alicia Shin: It’s the bi-directional sync feature.
130 00:20:11.090 ⇒ 00:20:22.439 Robert Tseng: The bi-directional sync feature. Okay. Well, I mean, these are all just, like, different, you know, tabs or modules within the free plan that, like, a user could go through. It does seem like, yeah, like.
131 00:20:22.890 ⇒ 00:20:42.699 Robert Tseng: users that save changelogs, like, they actually, seems like, you know, they’re converting at a much higher percentage than… I think the average is something, like, below 2% or whatever. So, like, I mean, any usage of these other features beyond the initial, kind of, just, like, yeah, finished onboarding, does seem to, kind of.
132 00:20:43.530 ⇒ 00:20:53.869 Robert Tseng: you know, it’s a… it indexes positively compared to the average, but yeah, I mean, this is… maybe we can kind of expand on this list, but any reactions to this first?
133 00:20:54.310 ⇒ 00:21:18.199 Alicia Shin: I think this type of view is very helpful. We do have, like, a broader list of, like, features and stuff that, I think this is where getting you guys access to, like, the god mode will be helpful, just so you can see, like, everything, and probably that combined with, our database, like, essentially, like, any field that we have that’s, like, a feature, like, we should be able to sort of, like, track the impact of, so… Yep.
134 00:21:18.200 ⇒ 00:21:20.619 Alicia Shin: The ask would just be, like, let’s expand upon this.
135 00:21:20.900 ⇒ 00:21:21.590 Robert Tseng: Okay.
136 00:21:21.760 ⇒ 00:21:31.770 Robert Tseng: That sounds good. Yeah, I mean, with these charts, probably limited to, like, 6 to 8 different tiles before it gets too crazy, so we could just… I mean, I’ll organize it and…
137 00:21:32.130 ⇒ 00:21:50.330 Robert Tseng: we can visualize… visualize it better. But yeah, so I will… I’ll also get you the account for the… for the God Mode. It’s really just tied to my Brainforge email, so that’s that. Yeah, so that’s… that’s it. I think that’s, kind of what we were able to get done, starting on Friday, so…
138 00:21:50.740 ⇒ 00:21:56.729 Robert Tseng: yeah, I guess outstanding kind of things from where you had originally left off.
139 00:22:00.400 ⇒ 00:22:05.939 Robert Tseng: yeah, the AI Booster Pack that’s gonna go live tomorrow, so we should be able to take… answer that by next… by next week.
140 00:22:06.520 ⇒ 00:22:22.429 Robert Tseng: I’ll add the downgrade piece so that we can better understand… oh, right, I had this question. So, trial users. I think that was a bit of an unclear concept to me. Seems like everybody starts in trial, but, don’t really know how long the trial lasts, so maybe we could get some clarity around that.
141 00:22:22.650 ⇒ 00:22:29.529 Alicia Shin: What are you seeing on your end? Because my understanding is, right now, we don’t have a trial. We’re rolling it out in, like, 2 weeks.
142 00:22:29.780 ⇒ 00:22:30.230 Robert Tseng: Okay.
143 00:22:30.230 ⇒ 00:22:42.250 Alicia Shin: had it in the past, it turned off in November of last year, and we’re reintroducing now a year later. So I guess, like, my first question would be, like, are you seeing trial users, or what are you seeing on your end?
144 00:22:42.250 ⇒ 00:22:48.500 Robert Tseng: There is a user profile, like, thing that’s,
145 00:22:50.400 ⇒ 00:22:53.139 Robert Tseng: the way I get there? Once we buy…
146 00:22:59.600 ⇒ 00:23:04.835 Robert Tseng: Projugating… No…
147 00:23:07.610 ⇒ 00:23:15.400 Robert Tseng: Okay, maybe it was… maybe it wasn’t a property, but I had this conversation with Phoebe before, and she just told me, like, just assume that
148 00:23:15.850 ⇒ 00:23:27.530 Robert Tseng: first 14-day period was trial, so I kind of just interpreted it that way when he talked to me about it last time. But I don’t know if that’s the best way to consider that.
149 00:23:27.530 ⇒ 00:23:32.939 Alicia Shin: Yeah, I guess, like, it is a 14-day trial, and then at 14 days, it will be
150 00:23:33.260 ⇒ 00:23:40.030 Alicia Shin: asked to either downgrade or, like, select a paid plan. I guess what I don’t know
151 00:23:40.200 ⇒ 00:23:47.680 Alicia Shin: is… are they… like, what happens if they don’t do either action? You know what I mean? Like, what happens on that 15th day? Like, I…
152 00:23:47.680 ⇒ 00:23:48.020 Robert Tseng: Yeah.
153 00:23:48.020 ⇒ 00:23:53.739 Alicia Shin: That’s a question for the business, I guess. I can take that back, but I’m just not sure. Okay.
154 00:23:55.030 ⇒ 00:24:06.190 Robert Tseng: Yeah, well, considering I didn’t downgrade until much later than 14 days, like, I feel like I wasn’t forced to. Maybe I got an email notification or something, but I just probably ignored it or something. I’m not… I’m not really sure, but…
155 00:24:06.190 ⇒ 00:24:25.800 Alicia Shin: I’m guessing what happens is you lose, like, the feature access, because essentially on the trial, you get access to, like, a really, like, I think business-level feature. Yeah. And my guess is that 14 days, it just, like, it probably limits the feature set back to, like, what’s available for free. But let me… let me double-check that, like, if no… if there’s no user action, then what happens?
156 00:24:26.270 ⇒ 00:24:26.960 Robert Tseng: Okay.
157 00:24:27.230 ⇒ 00:24:35.529 Robert Tseng: All right, that sounds good. I think that gives me an idea of how to build out the flow for the post-14 day period, so we’ll add in the downgrade piece.
158 00:24:35.530 ⇒ 00:24:36.370 Alicia Shin: Okay, cool.
159 00:24:37.570 ⇒ 00:24:51.789 Alicia Shin: And then for, like, the, the ability to sort of, like, monitor this over time, so is this notebook intended to be, like, the dashboard where we, like, click into each metric, or is… I just, like, I haven’t used Amplitude as much, so I just need to familiarize myself with it, too.
160 00:24:51.790 ⇒ 00:25:10.180 Robert Tseng: Yeah, so I did save this into kind of the, larger, like, the department folder or whatever, self-serve conversion. You can copy it as a dashboard and strip out all the text if that’s noisy to you. Yeah, it’s kind of just… you can use it as this, it’ll just… I mean, I already pres… I preset these to September, so…
161 00:25:10.180 ⇒ 00:25:22.420 Robert Tseng: We’ll probably just have to go in and, make, like, kind of make the date ranges fuzzy so they’re not fixed. But yeah, otherwise, you know, you could just keep monitoring this over time.
162 00:25:22.890 ⇒ 00:25:41.739 Alicia Shin: Cool. Okay, yeah, I think let’s definitely pull out the date range as one filter, and then our head of product is out of office right now, but I’ll just… once we feel, like, good about the actual underlying data and do, like, a QA, I’ll share this with her, because ultimately she’s, like, the stakeholder who’s, like, running the pricing changes, and so she’ll.
163 00:25:41.740 ⇒ 00:25:42.380 Robert Tseng: Great.
164 00:25:42.380 ⇒ 00:25:49.019 Alicia Shin: interested, and I’ll either pull her into one of our meetings, or, have her review it, and then we can do it async.
165 00:25:49.680 ⇒ 00:25:57.600 Robert Tseng: Okay, that sounds good. So yeah, I guess we kind of exchanged notes on… we’ll send out another message on, like, what
166 00:25:57.940 ⇒ 00:26:03.649 Robert Tseng: Access we want from you, and then kind of the adjustments that we’ll make to this,
167 00:26:04.110 ⇒ 00:26:07.489 Robert Tseng: Notebook, and then we can kind of take it from there.
168 00:26:07.490 ⇒ 00:26:09.910 Alicia Shin: Awesome. Yeah, no, that sounds great.
169 00:26:10.910 ⇒ 00:26:13.270 Robert Tseng: Okay, cool. Any, any other questions?
170 00:26:13.990 ⇒ 00:26:19.080 Alicia Shin: No, this was… this was great, good work for… for less than a week, so…
171 00:26:19.080 ⇒ 00:26:25.050 Robert Tseng: Okay, good to hear. I’m curious, team-wise, like, kind of, I know people have shifted around.
172 00:26:25.050 ⇒ 00:26:26.709 Alicia Shin: Kind of.
173 00:26:27.550 ⇒ 00:26:28.440 Robert Tseng: I mean.
174 00:26:28.630 ⇒ 00:26:42.189 Robert Tseng: I’m assuming we’re not making any instrumentation changes right now. I’m, like, noting where, like, there are tracking gaps, but I know that’s not necessarily a priority for these first two weeks. But would like to know, like, is… who’s the person for, kind of.
175 00:26:42.570 ⇒ 00:26:56.359 Robert Tseng: Are you still using the same engineering kind of firm with… I guess I was working with a guy named Falco before, and then just, like, any other internal, like, kind of team changes that we should know about in the past couple months, probably, since I haven’t really been active on this since, like, August.
176 00:26:56.360 ⇒ 00:27:00.489 Alicia Shin: Yeah, I don’t even know what you, how you were working with FACO before.
177 00:27:00.490 ⇒ 00:27:01.100 Robert Tseng: Okay.
178 00:27:01.100 ⇒ 00:27:13.820 Alicia Shin: But we still work with him, he’s just, like, not in this, doing this body of work, so we have a pool of engineers that we, like, could potentially work with. How were you working with him before, just so I’m aware? Or, like.
179 00:27:14.620 ⇒ 00:27:16.730 Alicia Shin: Engineering partner for, specifically?
180 00:27:16.730 ⇒ 00:27:25.000 Robert Tseng: Yeah, so, like, to set up some of this amplitude tracking before, like, I guess I put… I put together, like, a tracking plan, and kind of was just…
181 00:27:25.250 ⇒ 00:27:38.769 Robert Tseng: checking to make sure that he was implementing it correctly. We only did, like, a piece of that work and didn’t get the whole thing done at the time, and then I think maybe put POS on that project. So, yeah, I wasn’t really sure where we left off there.
182 00:27:38.770 ⇒ 00:27:43.819 Alicia Shin: So this is, like, actually getting Mongo, like, hooked up into Amplitude correctly, is that correct?
183 00:27:43.820 ⇒ 00:27:50.489 Robert Tseng: Yeah, so I mentioned trying to get, yeah, like, server data into Amplitude. I didn’t end up doing that, and then…
184 00:27:50.750 ⇒ 00:27:55.340 Robert Tseng: Well, yeah, so that, that was, that was the main, that was the main step.
185 00:27:55.850 ⇒ 00:28:02.380 Alicia Shin: Do you see that as, like, a necessary activity for this work? Like, is it required, or…
186 00:28:03.120 ⇒ 00:28:07.990 Robert Tseng: I don’t… I don’t necessarily think so. I think once we get… I mean, we’ll check all the…
187 00:28:08.440 ⇒ 00:28:12.290 Robert Tseng: numbers for, like, the support funnel. If we’re off by a lot, then, like.
188 00:28:12.560 ⇒ 00:28:24.299 Robert Tseng: maybe we should, otherwise I don’t want to mislead you guys. But yeah, I guess we’ll… we’ll just have to see, like, what, what… how off we are right now, I guess.
189 00:28:24.300 ⇒ 00:28:35.780 Alicia Shin: Okay, sounds good. I will… I think the most important thing is I need to figure out who can get you guys some sort of access to our database. So you… you didn’t have any access before, like, Falco was just coordinating that on his end?
190 00:28:35.960 ⇒ 00:28:36.580 Robert Tseng: Yep.
191 00:28:36.900 ⇒ 00:28:47.939 Alicia Shin: Okay, and is it necessary for you to have the access, or is that… does that type of model work for you? I just… I don’t know what the response is going to be in terms of getting you guys access, but it’s helpful to know.
192 00:28:47.940 ⇒ 00:28:53.779 Robert Tseng: Yeah, I mean, I just… it limits my ability to QA if I don’t have it, so…
193 00:28:54.100 ⇒ 00:28:54.640 Alicia Shin: well.
194 00:28:54.770 ⇒ 00:28:57.029 Alicia Shin: I will look into that today.
195 00:28:58.080 ⇒ 00:28:58.830 Robert Tseng: Okay.
196 00:28:59.040 ⇒ 00:28:59.980 Robert Tseng: Cool.
197 00:28:59.980 ⇒ 00:29:06.250 Alicia Shin: Cool. No other major team changes on our end. I don’t think anyone that you’re working with
198 00:29:06.870 ⇒ 00:29:11.330 Alicia Shin: has, like, shifted in their role, but, I’ll keep you posted if that changes.
199 00:29:11.560 ⇒ 00:29:12.690 Robert Tseng: Okay, sounds good.
200 00:29:12.710 ⇒ 00:29:13.530 Alicia Shin: Alright.
201 00:29:13.530 ⇒ 00:29:14.590 Robert Tseng: Thanks, Alicia.
202 00:29:14.590 ⇒ 00:29:15.759 Alicia Shin: I’ll see you guys, bye!
203 00:29:15.760 ⇒ 00:29:16.820 Robert Tseng: Yep, bye.