Meeting Title: ReadMe <> Brainforge Check-In Date: 2025-12-18 Meeting participants: Greg Stoutenburg, Phoebe Miller
WEBVTT
1 00:00:12.760 ⇒ 00:00:13.979 Greg Stoutenburg: There we go.
2 00:00:14.910 ⇒ 00:00:16.449 Phoebe Miller: Hey, sorry about that.
3 00:00:16.810 ⇒ 00:00:24.310 Greg Stoutenburg: Hey, that’s okay. I thought it was on our side, I saw that Robert wasn’t going to be in today, and so I thought, oh shoot, maybe he’s the host.
4 00:00:24.310 ⇒ 00:00:25.270 Phoebe Miller: Oh, yeah.
5 00:00:25.270 ⇒ 00:00:27.150 Greg Stoutenburg: Guys, what do we do?
6 00:00:27.290 ⇒ 00:00:33.499 Phoebe Miller: It looks like he’s feeling under the weather, maybe. I was… I just recovered from something, so it’s going around New York.
7 00:00:33.500 ⇒ 00:00:43.120 Greg Stoutenburg: Yeah, my, I mean, I’m not in New York, but my youngest had influenza last week, and my oldest got it today. So, yes, stuff is very much going around.
8 00:00:43.390 ⇒ 00:00:44.469 Phoebe Miller: Where are you based?
9 00:00:44.820 ⇒ 00:00:49.490 Greg Stoutenburg: I’m in York, Pennsylvania, which is about a half hour south of Harrisburg and an hour north of Baltimore.
10 00:00:49.690 ⇒ 00:00:50.470 Phoebe Miller: Cool, nice.
11 00:00:50.570 ⇒ 00:00:53.970 Greg Stoutenburg: Yep. I can get to New York, but it’s a train ride from Lancaster, so…
12 00:00:53.970 ⇒ 00:00:54.690 Phoebe Miller: Yeah.
13 00:00:54.690 ⇒ 00:00:56.660 Greg Stoutenburg: Yeah. From Michigan originally.
14 00:00:56.990 ⇒ 00:01:00.330 Phoebe Miller: Michigan. Okay, so not… not a stranger to the cold, then.
15 00:01:00.330 ⇒ 00:01:03.009 Greg Stoutenburg: not a stranger. I’ve actually been telling people I love it.
16 00:01:03.010 ⇒ 00:01:03.630 Phoebe Miller: You’re lazy.
17 00:01:03.630 ⇒ 00:01:11.700 Greg Stoutenburg: This is not even, like, zip-up-all-the-way weather for me. Wow. And so I was out skiing for opening weekend last Sunday, so having a good time.
18 00:01:11.700 ⇒ 00:01:13.380 Phoebe Miller: What’s your mountain by you.
19 00:01:13.380 ⇒ 00:01:17.949 Greg Stoutenburg: So, we call it a mountain, and thank you for recognizing how we feel about it.
20 00:01:17.950 ⇒ 00:01:19.380 Phoebe Miller: More of a hill, a big hill.
21 00:01:19.680 ⇒ 00:01:29.339 Greg Stoutenburg: it is much more of a hill. I mean, if you… if you get to the right hill, you can see it in the distance, but it’s called Round Top. It’s one of many now Vail-owned resorts.
22 00:01:29.340 ⇒ 00:01:30.289 Phoebe Miller: Oh, really? Interesting.
23 00:01:30.310 ⇒ 00:01:37.050 Greg Stoutenburg: Yeah, yeah, yeah, no, it’s good. With the kids, I get a 7-day Epic Pass, and, you know, we figure it out. We have a good time.
24 00:01:37.050 ⇒ 00:01:42.889 Phoebe Miller: Yeah, we have, there’s someone in our office who’s from… I think it’s called Stroudsburg?
25 00:01:43.330 ⇒ 00:01:48.169 Greg Stoutenburg: Yeah, I know where that is. Yeah, 45 minutes or so outside of Philadelphia.
26 00:01:48.170 ⇒ 00:01:48.700 Phoebe Miller: Yeah.
27 00:01:48.700 ⇒ 00:01:49.430 Greg Stoutenburg: Yeah.
28 00:01:49.430 ⇒ 00:01:53.240 Phoebe Miller: Not far, and then they ski, so… Yeah. Ski? Snowbird?
29 00:01:53.240 ⇒ 00:02:02.709 Greg Stoutenburg: Ski. Yep, ski. Yep, joined Ski Club in 8th grade, on a whim, because the promise was, hey, for free, we’ll take you out and give you lessons, and see if you like it, and…
30 00:02:02.840 ⇒ 00:02:04.900 Greg Stoutenburg: I did. Great.
31 00:02:04.900 ⇒ 00:02:06.200 Phoebe Miller: Yeah, yeah.
32 00:02:06.340 ⇒ 00:02:09.300 Phoebe Miller: I grew up skiing a little bit,
33 00:02:09.620 ⇒ 00:02:13.360 Phoebe Miller: less… even though I grew up on… in New Jersey and, like.
34 00:02:13.700 ⇒ 00:02:29.160 Phoebe Miller: to the Northeast skiing. I skied up… grew up skiing more on the West Coast, and then I went to school in New Hampshire, so I did a little bit of skiing up there. I like it, it’s just, I am not like you, I don’t like the cold, so even though I was hardened by a few New Hampshire winters,
35 00:02:29.780 ⇒ 00:02:31.359 Phoebe Miller: something I never want to experience ever.
36 00:02:31.360 ⇒ 00:02:32.599 Greg Stoutenburg: Never got used to it? No.
37 00:02:32.600 ⇒ 00:02:35.680 Phoebe Miller: So, but I love, I love skiing, it’s just…
38 00:02:36.190 ⇒ 00:02:45.010 Phoebe Miller: if, like, that’s why I like West Coast skiing, I think, because you can have those, like, warm days in Colorado where the skiing is incredible, or Utah, or whatever it is, you know, so…
39 00:02:45.010 ⇒ 00:02:46.029 Greg Stoutenburg: Yeah, yeah.
40 00:02:46.030 ⇒ 00:02:48.240 Phoebe Miller: Got a ton of that in the Northeast.
41 00:02:48.540 ⇒ 00:02:53.499 Greg Stoutenburg: You don’t get any of that in the Northeast now. The moment it warms up, it’s slush or ice.
42 00:02:53.500 ⇒ 00:02:54.889 Phoebe Miller: Right, and then it gets dangerous.
43 00:02:54.890 ⇒ 00:03:08.479 Greg Stoutenburg: And dangerous, yeah. I know, when I see these, and then we can actually, you know, I actually do have a deck, but… When I see these specials they run, like, for college students, like, oh, after 5pm, it’s, you know, 20 bucks, and they’re like, are you trying to kill them?
44 00:03:08.480 ⇒ 00:03:08.840 Phoebe Miller: Right.
45 00:03:08.840 ⇒ 00:03:10.120 Greg Stoutenburg: Because, yeah, either.
46 00:03:10.120 ⇒ 00:03:11.859 Phoebe Miller: The mountain is torn up, yeah.
47 00:03:11.860 ⇒ 00:03:21.520 Greg Stoutenburg: Right, yeah, either it’s wet, and everyone’s gonna tear an ACL no matter their age, or it’s a sheet of ice, and they’re gonna go into something. So, anyway.
48 00:03:21.520 ⇒ 00:03:22.500 Phoebe Miller: Yeah. Yeah.
49 00:03:22.500 ⇒ 00:03:26.350 Greg Stoutenburg: No, jealous. Never had never gotten to enjoy West Coast skiing, unfortunately.
50 00:03:26.800 ⇒ 00:03:29.589 Phoebe Miller: Yeah, if you ever get a chance, it’s definitely a good one.
51 00:03:29.590 ⇒ 00:03:32.060 Greg Stoutenburg: Love to go, yeah.
52 00:03:32.630 ⇒ 00:03:50.700 Greg Stoutenburg: So, I saw your comments on the deck that I shared last week, thanks for that. I did revise the, I changed the one chart that was including an upgrade to a free plan, so good call. I saw that there were several. I was sort of going by intuitive judgment as to which plans
53 00:03:50.700 ⇒ 00:04:02.040 Greg Stoutenburg: counted as upgrades and got that one wrong, so thanks for calling that out. And then the other question about the median time to conversion, I think that is accurately the median time conversion to conversion, like, you know, 30…
54 00:04:02.080 ⇒ 00:04:05.989 Greg Stoutenburg: what was it, 30? Seconds? 30 minutes, yeah, and that the best.
55 00:04:05.990 ⇒ 00:04:09.270 Phoebe Miller: Oh, was it minutes or seconds? Because minutes make sense to me, but seconds does not.
56 00:04:09.270 ⇒ 00:04:12.430 Greg Stoutenburg: Let me take a look at what it was. Oh, I’m looking at the wrong…
57 00:04:12.880 ⇒ 00:04:14.850 Greg Stoutenburg: Get this other thing teed up over here.
58 00:04:28.160 ⇒ 00:04:45.390 Greg Stoutenburg: Seconds, yeah, is that these are users who are coming in and very quickly going to the brightest button on the page. That’s my hypothesis. We would need to, you know, come up with video recordings or something like that to validate it thoroughly. I see that it looks like you don’t currently have session replays.
59 00:04:45.390 ⇒ 00:04:57.560 Phoebe Miller: No, we use full story. Okay. But then, I guess, just a follow-up question on that, and then we don’t have to go, like, data point by data point, we can just do, like, a larger high-level overview, but,
60 00:04:57.790 ⇒ 00:05:04.030 Phoebe Miller: I want to confirm that subscription success is not just… Hitting that button that says
61 00:05:04.170 ⇒ 00:05:15.409 Phoebe Miller: upgrade, or it previously said launch, but in fact, selecting a plan and putting in your credit card, like, to me, it’s like, that would take longer than 30 seconds, you know what I mean?
62 00:05:16.560 ⇒ 00:05:28.750 Greg Stoutenburg: Yes… That sounds right, but this experiment is… Median time to project launch.
63 00:05:29.200 ⇒ 00:05:33.159 Greg Stoutenburg: So if project launch is just getting over and hitting that button.
64 00:05:33.160 ⇒ 00:05:37.349 Phoebe Miller: Oh, okay, then that’s probably the misalignment. I would want to do, two…
65 00:05:38.060 ⇒ 00:05:43.369 Phoebe Miller: Well, that also doesn’t make sense, though, because to launch your project…
66 00:05:43.510 ⇒ 00:05:45.549 Phoebe Miller: Is it just hitting that button? Is that…
67 00:05:46.070 ⇒ 00:06:02.159 Greg Stoutenburg: I believe that that is what it is. Got it, okay. As I looked at these… so some of this was just building on existing work, so if this is what you’ve been looking at before, then I use the same events. I am able to go and validate that that’s the right event to be looking at, if that would be valuable.
68 00:06:02.160 ⇒ 00:06:14.270 Phoebe Miller: No, no, I feel confident that it’s not the right event to be looking at, because, like, maybe… maybe it’d be helpful if I just walked you through quickly what this looks like. So…
69 00:06:17.390 ⇒ 00:06:22.330 Phoebe Miller: So when you… this is a project that I’m currently on our free plan for.
70 00:06:22.330 ⇒ 00:06:22.690 Greg Stoutenburg: -
71 00:06:22.690 ⇒ 00:06:34.870 Phoebe Miller: And like, let’s just imagine I just signed up. This is the button. If we’re only measuring the amount of time from sign up to pressing this button, then I can understand 30 seconds.
72 00:06:34.870 ⇒ 00:06:35.830 Greg Stoutenburg: That is the case.
73 00:06:35.830 ⇒ 00:06:43.740 Phoebe Miller: Oh, okay, because what I’d be more interested in is, like, okay, time from sign-up to pressing this button to selecting a plan.
74 00:06:45.320 ⇒ 00:06:51.720 Phoebe Miller: to inputting credit card information to hitting confirm. That is when,
75 00:06:52.830 ⇒ 00:06:55.680 Phoebe Miller: You actually become a subscription customer.
76 00:06:55.680 ⇒ 00:07:06.919 Greg Stoutenburg: Yes. So, for anything that says conversion or conversion to paid, I use the event subscription success, which is supposed to be that event that you just described. So, hitting confirm after details are in.
77 00:07:06.970 ⇒ 00:07:11.770 Phoebe Miller: Okay, great. And if that… if that’s what we’re using, that is, in fact, the correct…
78 00:07:11.970 ⇒ 00:07:15.359 Phoebe Miller: event, and that is 30 seconds of that.
79 00:07:15.630 ⇒ 00:07:24.529 Greg Stoutenburg: No, that’s 30 seconds to project launch, so I believe that that’s just there. They’re scrolling over, here’s my CTA, right, there’s a bright blue button, let me hit that one.
80 00:07:24.530 ⇒ 00:07:27.460 Phoebe Miller: Okay, great. Then yeah, subscription success would be the correct one there.
81 00:07:27.460 ⇒ 00:07:28.120 Greg Stoutenburg: Okay.
82 00:07:34.850 ⇒ 00:07:40.629 Greg Stoutenburg: Okay, I’ll see what we have on that. Any other comments or thoughts about what I shared last week?
83 00:07:41.240 ⇒ 00:07:45.620 Phoebe Miller: I think just generally,
84 00:07:45.720 ⇒ 00:07:48.640 Phoebe Miller: I found it compelling, particularly the retention stuff.
85 00:07:50.760 ⇒ 00:08:08.649 Phoebe Miller: And I acknowledge that a lot of the experiments haven’t had an impact yet. I have, like, slightly different takeaways from… like, I have my own deck that I’ve put together, and I have, like, slightly different takeaways, that, I don’t know, I might want to just chat through with.
86 00:08:08.940 ⇒ 00:08:14.940 Phoebe Miller: you with, trap through with you, that’s what I wanted to say. Yeah, that’s it.
87 00:08:15.510 ⇒ 00:08:22.270 Phoebe Miller: And I think that’s fine. I anticipate that, like, people will interpret data in different ways.
88 00:08:22.270 ⇒ 00:08:22.660 Greg Stoutenburg: Sure.
89 00:08:24.240 ⇒ 00:08:37.270 Phoebe Miller: what… I guess, and this is not necessarily about the data you sent over, but, like, as I think about what comes next, we’re in, like, a little bit of a waiting period, because there’s probably no new experiments launching for a couple of weeks,
90 00:08:37.280 ⇒ 00:08:48.080 Phoebe Miller: definitely not over break, and then in January, our team’s plate is pretty full with, like, other stuff they’re working on, so there won’t be necessarily any, like, self-serve.
91 00:08:48.440 ⇒ 00:09:03.669 Phoebe Miller: conversion work occurring probably in the month of January, at least in early January. And so, just want to talk about, you know, how we can use that time, or if, you know, if BreenForge is the best, vendor to continue, like.
92 00:09:03.950 ⇒ 00:09:10.019 Phoebe Miller: you know, banging their heads against walls with me on this stuff, or I know you guys, you know, have… you have… you have an idea of, like, how you’re…
93 00:09:10.210 ⇒ 00:09:15.849 Phoebe Miller: firm works and wants to work strategically, and it just might not be a good fit. So, we can talk about that, too.
94 00:09:15.850 ⇒ 00:09:24.349 Greg Stoutenburg: Yeah, yeah, that sounds good. So, I’m, again, you know, new to this, I’m within this one, and so, you know, we can talk about what next steps might look like, and, you know.
95 00:09:24.350 ⇒ 00:09:24.760 Phoebe Miller: Yeah.
96 00:09:24.760 ⇒ 00:09:26.799 Greg Stoutenburg: Gonna come back to the team and consult with Robert and see what we
97 00:09:27.290 ⇒ 00:09:42.250 Greg Stoutenburg: I, I would like, as we think about what that does look like, I would like to, share what I put together today. Great. And, and maybe we, you know, maybe we can jam on that, and, yeah, as in, jam on what your, what you’ve prepared as well.
98 00:09:42.250 ⇒ 00:09:42.710 Phoebe Miller: Yeah.
99 00:09:42.710 ⇒ 00:09:51.549 Greg Stoutenburg: Let me just share this as a sort of, like, let’s just think of this as, like, a part two, to what I shared last week, and my thought being, you know, I understand everything that…
100 00:09:52.100 ⇒ 00:09:55.970 Greg Stoutenburg: Okay, do you see it? Okay.
101 00:09:55.970 ⇒ 00:10:14.609 Greg Stoutenburg: I understand everything that you said about, you know, maybe… maybe now is not the right time for a full strategic takedown, you know, what else can we do to be moving the needle? And so I wanted to build on what I shared last week as a… and propose something that you might use to quickly verify an idea, and if this
102 00:10:15.030 ⇒ 00:10:28.889 Greg Stoutenburg: if we can validate this idea, then that might justify getting design and engineering on board and creating a totally new onboarding flow. So, you know, the kind of experiment that can be run in two weeks, and then go from there, so…
103 00:10:29.960 ⇒ 00:10:46.460 Phoebe Miller: Cool. And like, and I… I… like, there are prob… there are… there are… I have a lot of ideas, I imagine you all probably have a lot of ideas on how to improve README, and I don’t think it hurts to just, like, make that list, and then when we do free up engine design resources, like, people… people can start to work on those things.
104 00:10:46.460 ⇒ 00:10:47.090 Greg Stoutenburg: Yeah.
105 00:10:47.220 ⇒ 00:10:52.199 Greg Stoutenburg: Yeah, yeah. So this is… this is familiar. This was the retention chart from before.
106 00:10:52.450 ⇒ 00:10:56.500 Phoebe Miller: And I dropped that in, by the way, to my, internal, because I thought it was interesting.
107 00:10:56.740 ⇒ 00:11:09.960 Greg Stoutenburg: Yeah, great, yeah. Yeah, I mean, what users do varies… impacts us significantly. And I did, in the other deck, link to a dashboard that I put together for you, where you can see these, and if there’s anything you want to edit or play around with or follow up on, you know, feel free to.
108 00:11:10.130 ⇒ 00:11:20.750 Greg Stoutenburg: This was a repeat as well, and this just broke out. Anyone who completed forms. This one used 5 forms. They had to complete some form 5 times.
109 00:11:20.860 ⇒ 00:11:33.320 Greg Stoutenburg: But when I brought it down to 1, the results are pretty much the same, which is significant, in my mind, because it says anyone who’s actually starting off in README by building documentation is much more likely to pay. That’s the takeaway.
110 00:11:33.320 ⇒ 00:11:35.710 Phoebe Miller: And, like, again, makes a ton of sense to me, like…
111 00:11:35.710 ⇒ 00:11:38.230 Greg Stoutenburg: That’s the idea. Yeah.
112 00:11:38.230 ⇒ 00:11:42.470 Phoebe Miller: Yeah, it’s like, people who use the product are more likely to convert, if that makes sense.
113 00:11:42.470 ⇒ 00:11:45.279 Greg Stoutenburg: There’s… that’s the takeaway in bold in the middle of the page.
114 00:11:45.280 ⇒ 00:11:47.589 Phoebe Miller: Yeah, I hate… I hate this.
115 00:11:48.710 ⇒ 00:11:50.120 Greg Stoutenburg: Oh, the Choose Your Own Adventure?
116 00:11:50.970 ⇒ 00:11:53.440 Greg Stoutenburg: I confess, I got confused. I spent some time on this.
117 00:11:53.440 ⇒ 00:11:54.020 Phoebe Miller: Yeah.
118 00:11:54.320 ⇒ 00:12:13.920 Greg Stoutenburg: what I tried to do is think about, okay, now, I’m not an expert on this, this is definitely the sort of thing that, you know, your team would go, alright, we know what people need to do to create API documentation. I followed each of those paths and just tried to think to myself, what’s, like, the minimum necessary number of things someone would need to do to have created, like, basic documentation?
119 00:12:13.920 ⇒ 00:12:22.769 Greg Stoutenburg: By analogy, what’s the minimum number of steps a person would have to do to say they wrote a paper, if this were, like, a paper writing program? Probably something like a detailed outline.
120 00:12:22.860 ⇒ 00:12:28.150 Greg Stoutenburg: And then, you know, that’s your first draft, something like that. So I was trying to think in that kind of way. And this is what I found.
121 00:12:28.270 ⇒ 00:12:32.280 Greg Stoutenburg: A user who did any of these things,
122 00:12:32.520 ⇒ 00:12:36.500 Greg Stoutenburg: Run the linter, run docs audit, complete a form, an E form.
123 00:12:37.190 ⇒ 00:12:56.340 Greg Stoutenburg: converts at 6.5%, and that’s compared to 1.9 overall in the last year. And I wanted to make sure that this was a pretty big group of users, rather than, you know, someone who did all the magical things or something. So there’s a pretty minimal number of steps, but if someone did any of those things, they converted at triple the rate.
124 00:12:57.140 ⇒ 00:13:07.230 Greg Stoutenburg: And that’s within 14 days. And so at 6.5 in 14 days, if you bring it forward to 1 day, then it only goes down to 6. I think, 2%.
125 00:13:07.340 ⇒ 00:13:21.889 Greg Stoutenburg: Which makes a lot of sense, because as you see in the… in the box there that’s hovered over, users are doing that in 9 hours… nine and a half hours, on average. So the users who are getting in and building something are converting.
126 00:13:22.500 ⇒ 00:13:30.500 Greg Stoutenburg: So the hypothesis that I wanted to share is that a more opinionated onboarding path, something that really guides users to do those things.
127 00:13:30.530 ⇒ 00:13:39.030 Greg Stoutenburg: will bring up your conversion to… will double to 2.5x, your conversion. Why not more?
128 00:13:39.030 ⇒ 00:13:50.990 Greg Stoutenburg: Because why not more to just, like, take what I found on the previous page? Because I think you’ll also lose more users who just aren’t in the mood for, or feeling ready to create docs today. But then, you know, if they’re the right users, they’ll come back.
129 00:13:51.520 ⇒ 00:14:07.949 Greg Stoutenburg: Yeah. So then, this is the last slide. Proposal, then, is use some lightweight methods, and we can talk about what that is, you know, that might be tooltips, there might be some minimal product tour, to point those things out to users, and maybe serve as, like, a checklist. Like, hey, to create docs, do this and this and this.
130 00:14:07.970 ⇒ 00:14:20.079 Greg Stoutenburg: and then go to launch. I think if we can discourage users from doing what we saw in the previous, chart, where they’re just rushing over to launch and hitting go, then that would be good for your metrics.
131 00:14:21.000 ⇒ 00:14:32.630 Phoebe Miller: Yeah, and I 100% agree. Like, so, the thing that I’m most focused on, or want to be most focused on, is how do we encourage new signups to explore more of the product? Yeah.
132 00:14:32.920 ⇒ 00:14:34.340 Phoebe Miller: my… my…
133 00:14:34.770 ⇒ 00:14:48.129 Phoebe Miller: I don’t love lengthy onboarding, so part of me… and I know I hate our current onboarding altogether, but part of me is, like, I would love if all you have to do is give for sign-up, username, password.
134 00:14:48.150 ⇒ 00:15:03.119 Phoebe Miller: or whatever, name, username, password, and then you just land in the product. Because I don’t really… I never really understand onboarding, that it, like, lives outside of the product, unless you’re trying to, get data from that user.
135 00:15:03.220 ⇒ 00:15:09.580 Greg Stoutenburg: So that you can land them in the correct place, right? Like, the… and that’s all that that…
136 00:15:09.580 ⇒ 00:15:13.010 Phoebe Miller: choose your own adventure is, like, meant to do. It’s, like.
137 00:15:13.010 ⇒ 00:15:13.350 Greg Stoutenburg: Right?
138 00:15:13.350 ⇒ 00:15:24.929 Phoebe Miller: have us better understand the user so we can drop them in the right place in the product, but then they’re still likely gonna explore other parts of the product, and it’s, like, unguided. Right.
139 00:15:24.930 ⇒ 00:15:25.820 Greg Stoutenburg: Not guided, yeah.
140 00:15:25.820 ⇒ 00:15:30.890 Phoebe Miller: So… I’d prefer we just kind of remove the idea of, like.
141 00:15:31.480 ⇒ 00:15:35.860 Greg Stoutenburg: onboarding, and instead… I mean, we can still call it onboarding, but it’s like… Yeah.
142 00:15:36.130 ⇒ 00:15:48.920 Phoebe Miller: remove those steps where you live outside of the product, and this is what I proposed to the team. And instead, like, land directly in the product, and then… and I don’t love coach marks, but there’s, like, some version of…
143 00:15:49.050 ⇒ 00:15:53.589 Phoebe Miller: onboarding, where you’re taken through the product, more like,
144 00:15:54.570 ⇒ 00:15:58.390 Phoebe Miller: Like, in some sort of journey format, is where my head’s at.
145 00:15:58.390 ⇒ 00:16:08.979 Greg Stoutenburg: Yep, yep. Yeah, I agree. I think if you did something like, you know, so just speaking in really ideal terms, rather than… I suggested validating this first through things like tooltips, and then see what happens, right?
146 00:16:08.980 ⇒ 00:16:09.700 Phoebe Miller: Yeah.
147 00:16:09.700 ⇒ 00:16:29.560 Greg Stoutenburg: If I were to just go, alright, skip that step, I would have the user go to readme.com, click get started, and then they’re just in the product, and it says, you know, build out your docs, and it’s like, you know, step one, make the docs, and then, you know, do it yourself, or upload the OAS file, or, you know, whatever. Two, check their quality.
148 00:16:29.560 ⇒ 00:16:32.980 Greg Stoutenburg: 3, set up an endpoint. If you’ve done those things, then…
149 00:16:33.270 ⇒ 00:16:46.509 Greg Stoutenburg: That’s it. Now you can launch. And maybe even, I mean, I like this way of thinking about it, because again, it’s who’s the target user? Don’t even let them click the launch button until they’ve done those things. What would you be launching? You don’t have docs, so you’re not launching docs. Right.
150 00:16:47.190 ⇒ 00:16:47.970 Greg Stoutenburg: Yeah.
151 00:16:48.360 ⇒ 00:16:59.959 Phoebe Miller: Yeah, I totally agree, and something I have spent time on, which it would be very valuable to hear your perspective, is our main competitor, Mintlify.
152 00:16:59.960 ⇒ 00:17:00.420 Greg Stoutenburg: Okay.
153 00:17:00.420 ⇒ 00:17:07.540 Phoebe Miller: E-L-I-F-Y. I really like their onboarding. I think there’s a few, like.
154 00:17:08.000 ⇒ 00:17:18.210 Phoebe Miller: Crucial steps that they require users to do, or encourage users to do, that make the product more sticky, or just, like, more enticing?
155 00:17:18.210 ⇒ 00:17:31.890 Phoebe Miller: But I’d be interested to hear… what I’m… what I’m hoping to do is, like, go to our team with a strategic recommendation on the improvements we can make, between, like, sign-up to conversion.
156 00:17:31.930 ⇒ 00:17:41.549 Phoebe Miller: And I think, data is helpful in, like, me making that argument, but also, like, competitive intelligence is helpful. Okay.
157 00:17:41.880 ⇒ 00:17:51.779 Phoebe Miller: And yeah, so I’ve gone through Mintlify’s onboarding a couple times, but if you have… if you go through it and you have takeaways, I’d be curious to hear those.
158 00:17:51.780 ⇒ 00:18:00.129 Greg Stoutenburg: Okay, alright, that’s… that’s a good follow-up. Would it be helpful if I added to the deck that I just presented, and included some stuff about Mintlify?
159 00:18:00.130 ⇒ 00:18:00.510 Phoebe Miller: Yeah.
160 00:18:00.510 ⇒ 00:18:01.880 Greg Stoutenburg: would give some perspective. Okay.
161 00:18:01.880 ⇒ 00:18:06.809 Phoebe Miller: awesome, yeah, and like, I… I can send over, I… I recorded myself, like, on a loom.
162 00:18:06.810 ⇒ 00:18:10.149 Greg Stoutenburg: Like, calling out some things that I found interesting, okay.
163 00:18:10.150 ⇒ 00:18:14.239 Phoebe Miller: But… but just, yeah, however, however you… ultimately, like.
164 00:18:15.280 ⇒ 00:18:21.139 Phoebe Miller: having conviction around these things is just where I’m trying to get, so all the inputs that we can have, the better.
165 00:18:21.310 ⇒ 00:18:25.729 Greg Stoutenburg: Sounds good. Yep, that sounds good. Okay, I can expand on that.
166 00:18:27.700 ⇒ 00:18:37.360 Greg Stoutenburg: something else. I mean, now, yeah, we don’t know when our next meeting is, so, down the road, I noticed that there aren’t web analytics measuring
167 00:18:37.560 ⇒ 00:18:42.339 Greg Stoutenburg: measuring between page landing and sign-up, I think that that’s something that we should track as well.
168 00:18:42.420 ⇒ 00:18:44.140 Phoebe Miller: It’s not an amplitude?
169 00:18:44.610 ⇒ 00:18:47.640 Phoebe Miller: No, we use, like, Google Analytics, you mean?
170 00:18:48.280 ⇒ 00:18:54.089 Greg Stoutenburg: And is that… and you’re measuring, then, the step from landing to sign up? Okay, great.
171 00:18:54.090 ⇒ 00:19:02.100 Phoebe Miller: Yeah, so we measure in HubSpot, traffic, Source to, like, acquisition source.
172 00:19:02.100 ⇒ 00:19:02.680 Greg Stoutenburg: Okay.
173 00:19:02.680 ⇒ 00:19:03.550 Phoebe Miller: to sign up.
174 00:19:03.920 ⇒ 00:19:04.880 Greg Stoutenburg: Okay, alright.
175 00:19:04.880 ⇒ 00:19:10.820 Phoebe Miller: But if you all… if that’s something, like, down the road that you all want access to, I’m happy to facilitate that.
176 00:19:10.820 ⇒ 00:19:11.570 Greg Stoutenburg: Okay.
177 00:19:12.920 ⇒ 00:19:16.069 Phoebe Miller: Because I think there’s, like, a larger question.
178 00:19:16.180 ⇒ 00:19:20.199 Phoebe Miller: Around, like, high intent folks writing.
179 00:19:20.200 ⇒ 00:19:20.970 Greg Stoutenburg: Yes, sir.
180 00:19:20.970 ⇒ 00:19:22.180 Phoebe Miller: So…
181 00:19:22.800 ⇒ 00:19:37.690 Greg Stoutenburg: That’s exactly my thought, because… so, you know, I can create my segments in amplitude based on usage, but for all I know… for all I know, the real story of why the retention bar is real high for some users is it’s the ones who want to create some particular kind of docs. Right, came from a certain.
182 00:19:37.690 ⇒ 00:19:38.320 Phoebe Miller: Yeah.
183 00:19:38.320 ⇒ 00:19:41.100 Greg Stoutenburg: Yeah, but I can’t see that right now, so…
184 00:19:41.100 ⇒ 00:19:45.980 Phoebe Miller: Amplitude have a Google Analytics integration, or how do they… how do.
185 00:19:45.980 ⇒ 00:19:50.409 Greg Stoutenburg: They typically do. Let me find out. I don’t.
186 00:19:50.730 ⇒ 00:19:53.420 Phoebe Miller: Yeah, I’m curious if it weren’t, because, like…
187 00:19:54.090 ⇒ 00:19:56.979 Phoebe Miller: How would Amplitude do the acquisition?
188 00:19:56.980 ⇒ 00:19:57.800 Greg Stoutenburg: Yeah.
189 00:19:57.800 ⇒ 00:19:58.150 Phoebe Miller: data.
190 00:19:58.150 ⇒ 00:20:17.750 Greg Stoutenburg: I mean, yeah, I don’t… I don’t know. I know when I’ve used Amplitude for other projects, just web analytics are included, so you can see things like source, you can see things like number of visitors to which page, and things like that. And for me, what I’ve always found one of the most insightful steps to be, user lands at readme.com.
191 00:20:17.750 ⇒ 00:20:24.740 Greg Stoutenburg: And then they click sign up, and then what do they do? To, you know, just get a more complete picture of the funnel?
192 00:20:26.060 ⇒ 00:20:26.630 Greg Stoutenburg: Nope.
193 00:20:26.780 ⇒ 00:20:36.759 Greg Stoutenburg: Okay. So, alright, it is 12.52, so 8 minutes left. Do you want to share what you have already, or maybe…
194 00:20:36.760 ⇒ 00:20:37.450 Phoebe Miller: Yeah, that’d be great.
195 00:20:37.450 ⇒ 00:20:38.160 Greg Stoutenburg: I can review it.
196 00:20:38.160 ⇒ 00:20:41.640 Phoebe Miller: And okay.
197 00:20:42.200 ⇒ 00:20:48.389 Phoebe Miller: So… And, mine is more, like.
198 00:20:48.620 ⇒ 00:20:50.909 Phoebe Miller: slightly more narrative, because I’m trying to set the stage of, like.
199 00:20:50.910 ⇒ 00:20:51.510 Greg Stoutenburg: Sure.
200 00:20:51.970 ⇒ 00:20:56.670 Phoebe Miller: This year, our traffic
201 00:20:57.480 ⇒ 00:21:05.300 Phoebe Miller: rose and fell in relation to spend. So, like, not super concerned about that, because ultimately…
202 00:21:05.390 ⇒ 00:21:19.199 Phoebe Miller: we ended up with the same number of customers, kind of, like, regardless of how much traffic we had. Sign-ups from traffic converted around 9 or 10 or 11%, so, like, again, pretty standard.
203 00:21:19.340 ⇒ 00:21:21.440 Phoebe Miller: And then…
204 00:21:22.170 ⇒ 00:21:39.109 Phoebe Miller: sign-ups converted or created projects, again, kind of, like, consistently between 50 and 60%. Okay. So, like, across all of these funnel stages, traffic, signups, creating projects, those all have remained unchanged.
205 00:21:39.110 ⇒ 00:21:48.539 Phoebe Miller: But what does vary pretty significantly, is the conversion to customer. And so, what I got super curious about is, like.
206 00:21:48.950 ⇒ 00:22:03.840 Phoebe Miller: So, the amount of money we spend basically dictates those three parts of the funnel, right? How many people come to the site, how many people sign up, how many create projects, but then around the same quantity of people are becoming customers. Why is that?
207 00:22:04.380 ⇒ 00:22:06.820 Phoebe Miller: And so then I wa… I… I…
208 00:22:07.600 ⇒ 00:22:17.929 Phoebe Miller: was curious about, like, why specifically in the last two months have we seen higher conversion than we saw all year, besides January? And what I can point to is, like.
209 00:22:18.070 ⇒ 00:22:23.340 Phoebe Miller: These experiments that we’ve been running, these, like, product launches and experiments that we’ve been running.
210 00:22:23.720 ⇒ 00:22:28.089 Phoebe Miller: Less so experiments, more like product launches. And so…
211 00:22:28.420 ⇒ 00:22:32.910 Phoebe Miller: My takeaway is, like, okay, new product launches help improve conversion rate.
212 00:22:33.570 ⇒ 00:22:37.629 Phoebe Miller: So then I was like, okay.
213 00:22:37.900 ⇒ 00:22:43.569 Phoebe Miller: what are the product launches that we’ve done? Because there’s, like… so then I broke our experiments out into, like, 3 categories.
214 00:22:43.780 ⇒ 00:22:44.160 Greg Stoutenburg: Okay.
215 00:22:44.160 ⇒ 00:22:50.260 Phoebe Miller: There’s things that we do that are, like, onboarding initiatives, right? So, like, we did that OAS
216 00:22:50.820 ⇒ 00:22:55.040 Phoebe Miller: upload in onboarding the Choose Your Own Adventure, and, like, that didn’t…
217 00:22:55.270 ⇒ 00:22:59.770 Phoebe Miller: improve onboarding completion. So, like, I consider that, like, a failed experiment.
218 00:22:59.780 ⇒ 00:23:09.080 Phoebe Miller: Yeah. We, added managed plan… oh, sorry, let me zoom in a little, this is a little small. We added that managed plan… so, like, that page where you convert.
219 00:23:09.080 ⇒ 00:23:27.150 Phoebe Miller: It’s, like, a slightly nicer page, right? Like, we saw a small lift there month over month, but, like, I doubt that that’s the reason. And then now, we’ve added back the trial, and I… and we’ve seen early signs that, like, this is improving conversion, which, again, isn’t rocket science to me, like, we’ve re…
220 00:23:27.190 ⇒ 00:23:35.199 Phoebe Miller: time-bound, the amount of time you can use the product. I would expect a couple more people to convert. But, like, none of these things are product launches, right? They’re, like.
221 00:23:35.200 ⇒ 00:23:35.830 Greg Stoutenburg: Right.
222 00:23:35.830 ⇒ 00:23:37.990 Phoebe Miller: onboarding improvements, I would call this.
223 00:23:37.990 ⇒ 00:23:38.890 Greg Stoutenburg: Yeah.
224 00:23:40.550 ⇒ 00:23:44.590 Phoebe Miller: this is, like, the early success of the trial reintroduction that we talked about. Yeah.
225 00:23:45.260 ⇒ 00:24:00.980 Phoebe Miller: what is more interesting to me are these product launches. And so, in October, we launched our AI product. We already had some AI features, but this kind of, like, bundled them.
226 00:24:01.010 ⇒ 00:24:02.989 Greg Stoutenburg: And we saw…
227 00:24:02.990 ⇒ 00:24:11.420 Phoebe Miller: In Q3, 21 people add the AI booster, but then in Q4, 52 people add the AI booster, and I was like, okay, that’s super interesting to me.
228 00:24:11.670 ⇒ 00:24:12.110 Greg Stoutenburg: Nope.
229 00:24:12.110 ⇒ 00:24:16.140 Phoebe Miller: But then, I’m like, The next step here is, like, okay.
230 00:24:16.210 ⇒ 00:24:35.510 Phoebe Miller: of the people adding those 52 people, how many of those were new signups, right? Right. Like, I imagine a lot of these are existing customers, and so that’s actually not… while it’s helpful, because that’s revenue, it’s not helping improve conversion. So, like, that’s my next step. It was, like, the number of new customers adding the AI booster, and then we also launched
231 00:24:36.790 ⇒ 00:24:44.880 Phoebe Miller: there’s a… the way that README works is, most of our customers are working in either GitHub or GitLab.
232 00:24:44.880 ⇒ 00:24:58.429 Phoebe Miller: where developers do their work, and we have a bi-directional sync between README and those providers. We initially only had GitHub support. We recently launched GitLab support, so, like, in theory, that unlocked
233 00:24:58.430 ⇒ 00:25:01.510 Phoebe Miller: a bunch of new customers. Right.
234 00:25:01.570 ⇒ 00:25:18.300 Phoebe Miller: again, like, this data isn’t super compelling to me. Prior to launch, we had zero customers. After launch, we have 35 customers. Right. What’s interesting to me is how many of those 35 are new customers. Right. Like, some of them might have been existing, who now enabled the…
235 00:25:18.460 ⇒ 00:25:23.390 Phoebe Miller: GitLab, bi-directional. So, that’s where my brain was going, and then…
236 00:25:23.870 ⇒ 00:25:37.120 Phoebe Miller: And then what… the final piece, which I haven’t done enough work on, but I need to, is, like, these upsell moments of people moving from one plan to another, so, like, a plan. And I don’t know what I want to measure there.
237 00:25:37.120 ⇒ 00:25:38.300 Greg Stoutenburg: Yeah, okay.
238 00:25:38.300 ⇒ 00:25:57.369 Phoebe Miller: And then there’s a whole bunch of, like, experiments on deck, that we don’t need to talk about right now, and then I dropped in, gear retention tables, because I’m interested, like, this is the… the, like… and I had asked Robert to do this, like, early on in our engagement, but this is the, like.
239 00:25:58.650 ⇒ 00:26:02.210 Greg Stoutenburg: Data splunking, that’s super interesting to me, of, like.
240 00:26:02.600 ⇒ 00:26:08.319 Phoebe Miller: Okay, most of our users don’t come back, that sucks, but the people who do come back.
241 00:26:08.340 ⇒ 00:26:13.400 Greg Stoutenburg: People who use the product. And it’s like, how do we get.
242 00:26:13.430 ⇒ 00:26:25.490 Phoebe Miller: more people to use the product is, like, what I would want to encourage our team to do, or to think through. Precisely, yeah. And then here, it’s like, okay,
243 00:26:25.700 ⇒ 00:26:33.329 Phoebe Miller: who converts? And it’s, like, people using these parts of our product, so how do we make those more prominent?
244 00:26:33.440 ⇒ 00:26:37.850 Phoebe Miller: And so that’s… those… that’s what I have.
245 00:26:38.260 ⇒ 00:26:48.810 Greg Stoutenburg: Yeah, great. Yeah, I like it. Now, one thing I was curious about was the AI package. I saw so many new customers, gave that one a green. Was that a,
246 00:26:49.150 ⇒ 00:27:03.480 Greg Stoutenburg: Oh, that was a… yeah, that was a brand new feature. Okay, so what I was trying to think is, does that represent the increased spend that widened the funnel and all those relationships hold until, conversion? But…
247 00:27:03.480 ⇒ 00:27:17.089 Phoebe Miller: So that’s the question, right? So it’s like, right now, I think this is a little bit misleading, because product… we launched these two new products, this and this, but I haven’t pulled the data yet to know… to know if…
248 00:27:17.560 ⇒ 00:27:23.099 Phoebe Miller: The reason conversion is up is because these people are using these things.
249 00:27:23.100 ⇒ 00:27:23.680 Greg Stoutenburg: Right.
250 00:27:23.680 ⇒ 00:27:25.080 Phoebe Miller: That’s the final piece of, like.
251 00:27:25.080 ⇒ 00:27:25.800 Greg Stoutenburg: Okay.
252 00:27:26.090 ⇒ 00:27:31.720 Greg Stoutenburg: That should be a one-chart answer, can I just screenshot that?
253 00:27:31.720 ⇒ 00:27:32.910 Phoebe Miller: Yeah, I’ll just send it to you.
254 00:27:32.910 ⇒ 00:27:35.419 Greg Stoutenburg: Yeah, okay, great. Yeah, perfect.
255 00:27:35.800 ⇒ 00:27:48.270 Greg Stoutenburg: Okay, so knowing that we’re basically out of time, and I’ll be… I’ll be out for the next two weeks, through the end of December, but I’ll circle back with the team. So, tell me, I mean, tell me if this is basically right. You don’t know if you have…
256 00:27:48.270 ⇒ 00:27:57.269 Greg Stoutenburg: the capacity on the team to do new onboarding stuff for at least the first half of January, and but would be interested in it.
257 00:27:57.940 ⇒ 00:28:00.329 Phoebe Miller: Yeah, 100%. Like, I…
258 00:28:00.880 ⇒ 00:28:16.969 Phoebe Miller: this is why it’s, like, it was, like, you know, not upsetting or whatever, frustrating. I’m like, I want to do strategic work with you all, we’re just not there yet, so it just might not be a good fit, if, like, that’s all you are interested in at this time. Like, a lot of what I am gonna be doing is more of, like, this type of work, which is, like.
259 00:28:16.970 ⇒ 00:28:20.640 Greg Stoutenburg: Verifying hypotheses, narratives.
260 00:28:20.640 ⇒ 00:28:24.779 Phoebe Miller: Getting people on board, so that when resources do free up, they’re, like, ready to go.
261 00:28:24.780 ⇒ 00:28:29.960 Greg Stoutenburg: Yeah. Well, okay, so to, are we okay to go over a couple minutes?
262 00:28:30.080 ⇒ 00:28:34.789 Phoebe Miller: Potentially… No, actually, sorry, I have to jump.
263 00:28:34.790 ⇒ 00:28:39.579 Greg Stoutenburg: Okay, I’ll write to you in Slack, and if you can send that over. Thanks, talk soon.
264 00:28:39.580 ⇒ 00:28:41.290 Phoebe Miller: Yeah, really appreciate it, and happy holidays.
265 00:28:41.290 ⇒ 00:28:42.440 Greg Stoutenburg: Yep, you too. See ya.