Meeting Title: ReadMe <> Brainforge Check-In Date: 2025-12-11 Meeting participants: Robert Tseng, Greg Stoutenburg, Elizabeth Conference Room, Mustafa Raja
WEBVTT
1 00:01:13.050 ⇒ 00:01:13.960 Robert Tseng: Hey, Greg.
2 00:01:16.950 ⇒ 00:01:19.259 Greg Stoutenburg: I keep a green coffee by my bedside.
3 00:01:19.470 ⇒ 00:01:21.280 Greg Stoutenburg: That’s my middle of the night water.
4 00:01:22.350 ⇒ 00:01:24.580 Robert Tseng: Oh, a green bottle of water, you said?
5 00:01:24.580 ⇒ 00:01:26.330 Greg Stoutenburg: Green Nalgreen, just like that, yep.
6 00:01:26.330 ⇒ 00:01:27.400 Robert Tseng: Oh, no way!
7 00:01:28.650 ⇒ 00:01:31.969 Greg Stoutenburg: It’s at 2 AM, when I’m like, oh no, I haven’t had water in a day.
8 00:01:31.970 ⇒ 00:01:37.150 Robert Tseng: Oh, yeah. It’s so dry these days when it’s so cold in New York. Oh, man.
9 00:01:37.150 ⇒ 00:01:38.229 Greg Stoutenburg: It is so cold.
10 00:01:38.230 ⇒ 00:01:38.830 Robert Tseng: Yeah.
11 00:01:39.190 ⇒ 00:01:40.220 Elizabeth Conference Room: Hey there.
12 00:01:40.220 ⇒ 00:01:41.539 Robert Tseng: Hey, hey, baby!
13 00:01:43.340 ⇒ 00:01:44.439 Speaker 1 (Elizabeth Conference Room): How’s it going?
14 00:01:44.870 ⇒ 00:01:46.130 Robert Tseng: Good, how are you?
15 00:01:46.820 ⇒ 00:01:48.949 Speaker 1 (Elizabeth Conference Room): Yeah, doing alright. Nice to meet you, Greg.
16 00:01:49.180 ⇒ 00:01:50.150 Greg Stoutenburg: Nice to meet you.
17 00:01:50.590 ⇒ 00:01:51.640 Greg Stoutenburg: Glad to be on.
18 00:01:53.060 ⇒ 00:01:54.260 Elizabeth Conference Room: Cool.
19 00:01:54.260 ⇒ 00:01:56.159 Robert Tseng: Are we waiting for anyone else on your side?
20 00:01:56.670 ⇒ 00:01:58.239 Speaker 1 (Elizabeth Conference Room): Nope, just me.
21 00:01:58.470 ⇒ 00:01:59.140 Robert Tseng: Okay.
22 00:02:00.550 ⇒ 00:02:10.029 Robert Tseng: All right, well, yeah, I know we’ve had some back and forth on Slack. I guess maybe, you know, we could still kind of read out on, like, kind of the… some of the updates.
23 00:02:10.370 ⇒ 00:02:14.200 Robert Tseng: Mustafa tweaked a couple things, and then Greg has some stuff he wants to share.
24 00:02:14.440 ⇒ 00:02:20.079 Robert Tseng: But yeah, I mean, we’d also love to spend some time kind of talking about, like, some of the things that we brought up on, like.
25 00:02:20.400 ⇒ 00:02:25.949 Robert Tseng: how we think we could make things work. Yeah, I don’t know, does that sound okay?
26 00:02:27.200 ⇒ 00:02:29.860 Speaker 1 (Elizabeth Conference Room): Yeah, I guess, like.
27 00:02:31.880 ⇒ 00:02:45.199 Speaker 1 (Elizabeth Conference Room): It’s very possible we walk out of this meeting and we decide just to part ways. Yeah. It sounds like you all are frustrated with the way that README works. I can understand that. I’m also frustrated.
28 00:02:45.200 ⇒ 00:02:45.720 Robert Tseng: Okay.
29 00:02:47.470 ⇒ 00:03:01.669 Speaker 1 (Elizabeth Conference Room): But ultimately, like, what I am hoping to achieve is conviction around our self-serve funnel and the experiments that we’ve run over the last couple of months.
30 00:03:01.790 ⇒ 00:03:04.889 Speaker 1 (Elizabeth Conference Room): And I’m… I’m looking to you all as…
31 00:03:05.000 ⇒ 00:03:10.929 Speaker 1 (Elizabeth Conference Room): A partner on that, both on the data front, and also on, like, the narrative-building front.
32 00:03:11.000 ⇒ 00:03:27.139 Speaker 1 (Elizabeth Conference Room): And I think you’ve done a good job on the data front, despite, you know, the various roadblocks that you called out, Robert, around, like, stakeholder turnover and data access. As you know, like, Ruby underwent a RIF
33 00:03:27.300 ⇒ 00:03:32.289 Speaker 1 (Elizabeth Conference Room): 4 weeks ago, so, like, we’re still coming out of that.
34 00:03:33.540 ⇒ 00:03:48.460 Speaker 1 (Elizabeth Conference Room): But… but what I… what I would urge you all to, like, look inward on is, can you confidently answer the questions that I put into Slack? Like, would you get into a meeting with a bunch of executives and, like.
35 00:03:48.460 ⇒ 00:03:58.199 Speaker 1 (Elizabeth Conference Room): stand up and say, this is what I know about our funnel, this is what the experiments have done to the funnel, because if I don’t feel conviction around that, like.
36 00:03:58.500 ⇒ 00:04:11.419 Speaker 1 (Elizabeth Conference Room): I don’t… this isn’t… hasn’t been successful, right? And I know there are a lot of reports that I can look to, but I don’t feel confident about any of them at this time. So, I guess that… that’s where I’m at.
37 00:04:13.180 ⇒ 00:04:16.820 Robert Tseng: Okay, yeah, no, I think, yeah, that’s… that’s fair, we’ve…
38 00:04:17.240 ⇒ 00:04:32.450 Robert Tseng: like I… like I’ve kind of mentioned, like, we’ve gone through many iterations, even now, like, I don’t think the data is, like, perfect. Like, I think it’s… I think there’s a… the level of confidence, like, I think maybe whatever you feel like you need for, like.
39 00:04:32.490 ⇒ 00:04:38.920 Robert Tseng: that level of confidence. Like, I mean, I would… I guess my pers… my perspective is just that, like.
40 00:04:39.020 ⇒ 00:04:50.860 Robert Tseng: I think what we’ve built, I would get up and talk about, like, that, yeah, directionally, this is… this is correct. Like, whether or not it’s, like, 0.6% versus 0.7%, like, I don’t really think matters, to be honest, but, like, the entire.
41 00:04:50.860 ⇒ 00:04:51.350 Speaker 1 (Elizabeth Conference Room): academic experience.
42 00:04:51.350 ⇒ 00:04:57.520 Robert Tseng: experiments, if it, like, went… was better or worse, like, we… we can’t… we can’t say that.
43 00:04:57.560 ⇒ 00:05:13.479 Robert Tseng: maybe, like, we’re still, like, kind of, like, tweaking the, exclude this person, don’t exclude that person, and, like, I mean, I don’t know. I think those are just, like, kind of litigating small, small details, given that the volumes are so low on… but yeah, I don’t really think that these types of
44 00:05:13.570 ⇒ 00:05:24.209 Robert Tseng: adjustments that we’re making are, like, changing the narrative more. I just think that we’ve been looking at, like, a very limited lens, and that’s why, like, you know, I have…
45 00:05:24.310 ⇒ 00:05:29.610 Robert Tseng: I brought in Greg, and this past week, we’ve had many conversations about, like.
46 00:05:29.610 ⇒ 00:05:52.279 Robert Tseng: look, like, what are the different things that we need to do to, like, provide more context? Like, it’s got to be a triangulation of, like, the event tracking that we have, but then also, obviously, the strategic piece from your side, Phoebe, and, like, the roadmap that’s coming, or obviously you’ve shared some extent of that. And then also from Greg’s piece, like, I think, like, the strategy that he’s launched to be able to kind of,
47 00:05:52.280 ⇒ 00:06:00.609 Robert Tseng: kind of growth hack in this, like, PLG, like, motion. So, I do think that, you know, it’s gonna be this type of, like.
48 00:06:00.630 ⇒ 00:06:15.990 Robert Tseng: it’s gonna have to take these multiple, like, these parallel efforts in order to, build a narrative that you have good conviction over. Like, I don’t think looking at the amplitude reports is gonna give you the type of confidence that you want, yeah.
49 00:06:16.400 ⇒ 00:06:29.569 Speaker 1 (Elizabeth Conference Room): Sure. I agree with most of that on paper, but I guess my question to you all is, like, why you’re so eager to move on from conversions when I’m actively telling you, like, I don’t have conviction. So maybe, like.
50 00:06:29.570 ⇒ 00:06:37.949 Speaker 1 (Elizabeth Conference Room): But I would, like, can you all help me have conviction? Like, can you answer for me those 3 questions, or 4 questions that I’ve posed in Slack?
51 00:06:39.500 ⇒ 00:06:41.559 Robert Tseng: Yeah, I think we can answer those questions.
52 00:06:41.790 ⇒ 00:07:01.139 Speaker 1 (Elizabeth Conference Room): And, like, I’m not asking you to point me towards a reporter, as you just mentioned, like, just looking at amplitude isn’t the full picture, right? But it’s like, when you think about REB’s business as a whole, and let’s just say, you know, 2025, it’s like, what is the narrative? Sign-ups are up or down. Conversion is up or down. We did these…
53 00:07:01.220 ⇒ 00:07:21.079 Speaker 1 (Elizabeth Conference Room): three experiments, they impacted conversions positively or negatively. Like, I won’t know the answers to those questions, right? And if you all do, and we can put that into something that I can go back to my team with and present, then sure, let’s move on to the next thing. But, like, I’m not there yet. And I think there’s fault on both sides on that front.
54 00:07:22.460 ⇒ 00:07:25.200 Greg Stoutenburg: Phoebe, can I ask for clarification on that? So, I’m the new guy, I’ve been here forever.
55 00:07:25.200 ⇒ 00:07:25.680 Speaker 1 (Elizabeth Conference Room): Yeah.
56 00:07:25.680 ⇒ 00:07:30.679 Greg Stoutenburg: I signed into Amplitude for the first time last Friday. So,
57 00:07:30.880 ⇒ 00:07:48.179 Greg Stoutenburg: My question is, so, I’ve been able to look through past calls between BrainForge and README, look at documentation that’s been provided, things like that. From your perspective, when you say, you know, we really have the confidence to be able to go to leadership and say, here’s how things are with respect to
58 00:07:48.370 ⇒ 00:07:59.839 Greg Stoutenburg: these various pricing experiments and things like that. What do you feel is the piece that’s missing that’s not been provided in Brainforge’s reporting in the form of amplitude charts and things like that?
59 00:08:02.170 ⇒ 00:08:16.900 Speaker 1 (Elizabeth Conference Room): I just… I often get pointed towards a report that… and maybe part of this is that we asked for this, but it’s, like, it’s a very narrow view, you know, maybe looking at one month compared to the month after an experiment was launched.
60 00:08:16.900 ⇒ 00:08:26.389 Speaker 1 (Elizabeth Conference Room): And it’s like, to me, that’s too narrow of a view to understand how that’s impacting, like, our business as a whole. And Robert and I talked a lot about, it’s like.
61 00:08:26.680 ⇒ 00:08:40.830 Speaker 1 (Elizabeth Conference Room): conversions is the kind of through-line metric that we want to use. And then, yes, there are specific experiments where we want to be, use metrics per… use a metric per experiment that’s more hyper-specific, but…
62 00:08:41.080 ⇒ 00:08:47.759 Speaker 1 (Elizabeth Conference Room): it doesn’t… what I’m missing here is, like, the overall narrative,
63 00:08:47.940 ⇒ 00:08:51.459 Speaker 1 (Elizabeth Conference Room): And I guess, looking at all of these holistically.
64 00:08:51.610 ⇒ 00:08:56.130 Speaker 1 (Elizabeth Conference Room): as they relate to our conversion improvements, or I guess non-improvements. Like, I…
65 00:08:57.140 ⇒ 00:09:06.179 Speaker 1 (Elizabeth Conference Room): what would be most helpful to me, and maybe you all just want to, like, put a presentation together, if you all feel confident about it, like, that’s great, but I…
66 00:09:06.500 ⇒ 00:09:12.209 Speaker 1 (Elizabeth Conference Room): I… I can’t go to my team with what I currently have, because I don’t know what I’m looking at.
67 00:09:13.010 ⇒ 00:09:22.770 Greg Stoutenburg: Okay, so you feel like Brainforge… is this accurate? Please tell me if this is accurate. You feel that Brainforge has not provided adequate context to be able to put together that kind of story, where you could go to…
68 00:09:22.870 ⇒ 00:09:24.950 Greg Stoutenburg: Others in leadership, and say.
69 00:09:25.110 ⇒ 00:09:29.359 Greg Stoutenburg: These aren’t moving the needle, or these are moving the needle, or things like that.
70 00:09:29.880 ⇒ 00:09:37.320 Speaker 1 (Elizabeth Conference Room): That’s right, yeah, not… I don’t have, accurate context or conviction around the data points.
71 00:09:37.440 ⇒ 00:09:40.779 Speaker 1 (Elizabeth Conference Room): Because every time I open a report, it feels like…
72 00:09:41.090 ⇒ 00:09:44.720 Speaker 1 (Elizabeth Conference Room): the filters are set in a way that I don’t agree with.
73 00:09:45.790 ⇒ 00:10:01.629 Greg Stoutenburg: I see. Okay, so some of this does come down to that fine-grade detail type stuff. The question you want to answer is, did my experiment increase conversion, right? And then what you’re handed is something with, you know, a member is not in this or this or this, and here’s the… okay, I understand what you’re saying.
74 00:10:01.650 ⇒ 00:10:19.939 Speaker 1 (Elizabeth Conference Room): Yeah, I think it’s, like, overall, like, did this increase conversion and conversions, and then there’s a specific metric per experiment that we’re looking at, like, you know, are more AI boosters being added? And my idea is that more AI boosters being added would increase conversions, right? Like, the two would be related.
75 00:10:19.940 ⇒ 00:10:30.590 Speaker 1 (Elizabeth Conference Room): And I think you guys have done a lot of great work. I don’t want to under-represent that. It’s just not the work that I need to have the conviction around
76 00:10:30.670 ⇒ 00:10:32.020 Speaker 1 (Elizabeth Conference Room): this process.
77 00:10:34.550 ⇒ 00:10:35.800 Greg Stoutenburg: I see, that’s helpful.
78 00:10:39.300 ⇒ 00:10:41.830 Elizabeth Conference Room: maybe, Robert, you feel differently, but, like.
79 00:10:43.160 ⇒ 00:10:52.999 Robert Tseng: Yeah, no, I mean, I think, it’s, like, like I said, kind of going back to the analogy, like, I… I feel like we’re, we’re, like, looking at this from just, like, a, like, a model lens, and, like.
80 00:10:53.140 ⇒ 00:11:05.270 Robert Tseng: I feel like we’ve kind of maximized what we can. Sure, like, the… I mean, I don’t want to, like, nip… I don’t want to litigate too many details, like, on terms of, like, this filter is not working or whatever, like, I feel like these.
81 00:11:05.270 ⇒ 00:11:05.700 Speaker 1 (Elizabeth Conference Room): Yeah.
82 00:11:05.700 ⇒ 00:11:18.989 Robert Tseng: or, like, cosmetic changes, like, I don’t… I still, like, I… we believe that something was said last week that we kind of made that change, and then, like, it got reversed the change. Like, I don’t know, like, I think there’s a thing… I don’t really have anything to say about… about that.
83 00:11:19.040 ⇒ 00:11:33.890 Robert Tseng: I do think, like, the selection of reports that we’ve given you, like, the structure, even, like, starting first, like, with more free-form, like, notebook style, like, telling you all the assumptions, kind of, like, how we were structuring the problem, then we narrowed down into these specific questions.
84 00:11:34.120 ⇒ 00:11:48.559 Robert Tseng: Like, yeah, I think… I think I understand the questions. Like, I… I think, like, did AI Booster really, like, impact conversion? I think it’s… it’s, like, it’s… it’s not, like, a firm yes or no. Like, I think it’s… yeah, it, it drove… like, we can say… we can say.
85 00:11:48.560 ⇒ 00:11:56.290 Robert Tseng: for users who used AI… who toggled AI, even, like, even that has nuance, right? Like, what was the adoption of AI Booster?
86 00:11:56.290 ⇒ 00:12:08.930 Robert Tseng: first, like, I came to you with, like, AI booster means that they actually used AI features, and so I created a cohort that had, like, anybody that used any of those AI-related events.
87 00:12:08.930 ⇒ 00:12:19.719 Robert Tseng: And, we see that it doesn’t necessarily have, like, a… like, it’s like a marginal difference in impact on conversion. All it did was kind of drive engagement up, right? That was, like, that was the story there.
88 00:12:19.720 ⇒ 00:12:34.360 Robert Tseng: And then we kind of backtracked and was like, okay, well, what if we change that to actually just toggling the AI booster? And in fact, we’re adding, like, more… we’re changing the placement of where the AI booster is happening. So, like, that becomes a moving target, like, right? And, like.
89 00:12:34.360 ⇒ 00:12:51.579 Robert Tseng: it’s a more expansive definition, sure, like, but, like, yeah, I think, that to me is, like, a… that’s a different direction, and, like, we did iterate on that. So, like, I do feel like we’ve tried to, like, meet all of… and we understand that this is an iterative process, like.
90 00:12:52.000 ⇒ 00:13:09.829 Robert Tseng: I try… when you ask us a question, I’m telling you with more detail, this is what my answer is actually answering, or this is the question that I’m actually answering, and it… it’s a limited scope. Like, it can only… it’s… it’s only specific because I defined the events, and I, like, the range, whatever, in this particular way.
91 00:13:09.830 ⇒ 00:13:13.499 Robert Tseng: Like, I think that’s… that’s, like, you know, that… I think that’s… that’s…
92 00:13:13.500 ⇒ 00:13:22.659 Robert Tseng: That’s both the benefit and also the disadvantage of being… of doing this type of, like, bottoms-up approach to analysis, where… Yeah.
93 00:13:22.670 ⇒ 00:13:35.800 Robert Tseng: And so, I think it’s just hard to, like, come to a middle point or a point of agreement, maybe because you feel like I’m not really seeing things from your vantage point, and then, like, I feel like
94 00:13:35.920 ⇒ 00:13:41.509 Robert Tseng: Like, the nuance is, like, not really kind of, like, being,
95 00:13:41.940 ⇒ 00:13:51.069 Robert Tseng: it’s just, like, it’s just a constant moving target, like, we’re just, like, yeah, like, and that’s why we’re not having a meeting of the minds, like, I guess is what I would say.
96 00:13:53.350 ⇒ 00:14:03.539 Speaker 1 (Elizabeth Conference Room): I guess, from my… what I have to do is go into a room with people on my team and tell them that their experiments did or didn’t work, right? Yeah. And…
97 00:14:03.540 ⇒ 00:14:16.560 Speaker 1 (Elizabeth Conference Room): you may think that’s, I don’t know, now reminded of us, but, like, that’s what I have to do. And if you all aren’t the right firm to collaborate with on, to, like, arm you with that data, then…
98 00:14:16.580 ⇒ 00:14:21.019 Speaker 1 (Elizabeth Conference Room): I think it’s probably best we don’t continue working together.
99 00:14:21.270 ⇒ 00:14:33.839 Speaker 1 (Elizabeth Conference Room): I 100% agree with you all that I would love to work more strategically, but we’re just not there yet. That’s all I’m saying, just not there yet. And I, again.
100 00:14:33.900 ⇒ 00:14:47.820 Speaker 1 (Elizabeth Conference Room): I take a lot of the blame, too. Like, we’re still figuring things out. But what I’m asking for, I think, is pretty simple, which is why I’m confused, why it feels so difficult. And Robert, I know I’ve walked you through, like, this…
101 00:14:48.110 ⇒ 00:14:51.630 Speaker 1 (Elizabeth Conference Room): presentation, like, many times, but maybe, Craig?
102 00:14:51.630 ⇒ 00:14:57.750 Audio shared by Elizabeth Conference Room: you haven’t seen it, like, all I want to do is go into a meeting and be like, these are…
103 00:14:58.810 ⇒ 00:15:03.170 Audio shared by Elizabeth Conference Room: These are the 5 or 6 experiments that we’ve run this year.
104 00:15:03.610 ⇒ 00:15:11.229 Audio shared by Elizabeth Conference Room: this was the metric for success, this was what it was before launch, this is what it was after launch. And then I want to look at the funnel.
105 00:15:11.660 ⇒ 00:15:12.899 Audio shared by Elizabeth Conference Room: Is this not?
106 00:15:13.650 ⇒ 00:15:33.369 Audio shared by Elizabeth Conference Room: I want to look at this every couple of weeks, and be like, conversions are improving or not improving. That’s, like, that’s all I want to do, at least to start, because I have to establish just a baseline with these people that I know what I’m talking about, right? And I… I can’t do that yet. So, for you all to ask, like, let’s move on to.
107 00:15:33.370 ⇒ 00:15:40.669 Speaker 1 (Elizabeth Conference Room): revenue retention, like, it’s like… I’m still at the restaurant, you know? Like, we can’t do that yet.
108 00:15:41.030 ⇒ 00:15:56.679 Greg Stoutenburg: Yeah, I hear you. So, from my perspective, so again, one week, is when I look through previous reporting that’s been delivered to README, when I look through client calls, things like that, what I found myself not experiencing is the why behind
109 00:15:57.050 ⇒ 00:16:15.989 Greg Stoutenburg: behind many things. I saw a request, a report, send it, a Slack message. Very transactional, very, you know, here’s your chart. Missing that context. And then when I went into the README product, I, you know, did things like navigate the website, work through the funnel, things like that. Thought about what would be my experience as a user.
110 00:16:15.990 ⇒ 00:16:30.760 Greg Stoutenburg: if I show up with certain expectations, what problems do I think this tier will solve, or that the AI booster will solve, and things like that. And found myself being unsure what the answers to some of those questions were. So the recommendation, hey, why don’t we step back a little bit from some of the
111 00:16:31.200 ⇒ 00:16:47.630 Greg Stoutenburg: transactional relationship in, you know, delivering reports, that’s from me, because of missing that context. So I am in agreement with you that the, you know, missing the broader story is a critical piece here, and I’m aligned on that.
112 00:16:47.860 ⇒ 00:17:04.329 Greg Stoutenburg: If I could follow up on one thing that you said there. So, looking at that slide deck, and I… Robert shared that with me, I’ve seen that slide deck before. If you go into a meeting and on, you know, row, you know, row whatever, before launch is N, after launch is N+.
113 00:17:04.609 ⇒ 00:17:11.650 Greg Stoutenburg: Is that… is that all you need to be able to say this experiment was successful? I just want to see this happened on this date.
114 00:17:11.900 ⇒ 00:17:14.890 Greg Stoutenburg: The user used it, or some part of it, or they bought it.
115 00:17:15.170 ⇒ 00:17:17.260 Greg Stoutenburg: the number went up. That’s all?
116 00:17:17.569 ⇒ 00:17:32.749 Speaker 1 (Elizabeth Conference Room): Some of them are that binary. For example, like, the change we made to onboarding, we… I sent an image in Slack. Like, I want to revert that change, because I don’t feel confident that it improved conversion or project creation.
117 00:17:33.329 ⇒ 00:17:35.969 Speaker 1 (Elizabeth Conference Room): onboarding completion, those are all synonymous. Like.
118 00:17:36.059 ⇒ 00:17:55.569 Speaker 1 (Elizabeth Conference Room): that would be enough for us to be like, hey, I don’t think this change is working. But people on our team have said, like, if the stuff we’re launching isn’t moving the needle, or making things worse, like, we would revert it, or we would make a change. They don’t know. They have no idea if the work they’re doing is impactful. I don’t know. I can’t tell them.
119 00:17:56.260 ⇒ 00:17:56.960 Greg Stoutenburg: Right.
120 00:17:57.100 ⇒ 00:17:57.980 Greg Stoutenburg: Okay.
121 00:17:58.440 ⇒ 00:17:59.960 Greg Stoutenburg: So you’re wearing the measurement hat.
122 00:18:00.390 ⇒ 00:18:01.710 Greg Stoutenburg: You need to inform the team.
123 00:18:02.420 ⇒ 00:18:05.050 Speaker 1 (Elizabeth Conference Room): At this stage. At this stage, yes.
124 00:18:05.300 ⇒ 00:18:09.620 Speaker 1 (Elizabeth Conference Room): I’d like it to go beyond that, but at this stage, yes.
125 00:18:11.150 ⇒ 00:18:18.090 Greg Stoutenburg: If I can ask, what’s your level of confidence that they are shipping features that are of value to the right user segments?
126 00:18:18.820 ⇒ 00:18:25.510 Speaker 1 (Elizabeth Conference Room): Not super high. I’d be happy to go to the meeting and say that, too.
127 00:18:25.510 ⇒ 00:18:26.100 Greg Stoutenburg: Yeah.
128 00:18:26.340 ⇒ 00:18:31.150 Speaker 1 (Elizabeth Conference Room): And maybe you all have these answers in the reports that you’ve shared with me.
129 00:18:33.700 ⇒ 00:18:42.629 Speaker 1 (Elizabeth Conference Room): I don’t know if that’s the case, but maybe you all know this, and if that’s the case, then please tell me, and then we can all move past this.
130 00:18:45.910 ⇒ 00:18:52.800 Greg Stoutenburg: My take, looking at some of the recent work that’s been delivered is that, the things that have shipped don’t have an impact on conversion.
131 00:18:53.270 ⇒ 00:19:06.620 Greg Stoutenburg: At least not an immediate one. Now, in some cases, that’s going to be things like, for example, we want to look at who is using the AI booster since last Friday. Only 6 people bought it. So, even if 100% of them had
132 00:19:07.090 ⇒ 00:19:16.449 Greg Stoutenburg: you know, performed further follow-up actions, that might not be that useful of a data point. So in some cases, it’s things like that, and then in some others, I think it’s that they… they don’t have
133 00:19:16.730 ⇒ 00:19:30.249 Greg Stoutenburg: any obvious impact on conversion. Maybe they will over time, but I think that it would have to be part of a more strategic takedown and, you know, subsequent look at who we’re trying to deliver value to, what fundamental problems they have.
134 00:19:31.140 ⇒ 00:19:39.699 Greg Stoutenburg: Now, again, I’ve got two hats on here. From the reporting side, I think that we can provide more holistic data storytelling, and that is a capability of the team.
135 00:19:41.330 ⇒ 00:19:42.830 Speaker 1 (Elizabeth Conference Room): Cool.
136 00:19:43.060 ⇒ 00:19:52.570 Speaker 1 (Elizabeth Conference Room): that would be totally fine for me to go into a meeting and say, as long as I have the data to back that up, and then we could move forward. But it’s…
137 00:19:52.990 ⇒ 00:20:11.990 Speaker 1 (Elizabeth Conference Room): It’s like, I can’t… I can’t go into a meeting, say what you just said, Greg, which is what I believe, but without having the data to back it up. And then there’s a whole bunch of experiments and things we want to do in 2026, but we can’t do them… the team feels we can’t do them until we have conviction around things.
138 00:20:12.080 ⇒ 00:20:15.510 Speaker 1 (Elizabeth Conference Room): We already did, on whether or not they were impactful.
139 00:20:19.440 ⇒ 00:20:29.249 Robert Tseng: Okay, well, I mean, I think, like, one way that we can just basically repackage, like, some of the stuff that we’ve already built, like, I don’t really think there’s any… anything is a net new request. There’s nothing that we haven’t answered.
140 00:20:29.260 ⇒ 00:20:41.279 Robert Tseng: But is, like, yeah, I mean, I think we need to storytell better, and I don’t think the dashboard format does it, nor does a notebook. If you need to put it into slides, and, like, you know, I’ve, you know, I’ve been a consult… I mean, I’ve been.
141 00:20:41.280 ⇒ 00:20:41.690 Speaker 1 (Elizabeth Conference Room): Yeah.
142 00:20:41.690 ⇒ 00:20:46.059 Robert Tseng: we do a bunch of slides, and we could put it in any format. We could… we could use the whole…
143 00:20:46.950 ⇒ 00:21:01.309 Robert Tseng: headlines, like, you know, throw call-out bubbles on everything, just… and just try to, like, put… throw all the context onto that. Like, I think, you know, that’s… that’s our… that’s the best thing that we can do to, like, serve it to you in terms of, like, data storytelling.
144 00:21:01.310 ⇒ 00:21:07.129 Speaker 1 (Elizabeth Conference Room): I’m not necessarily even asking for that, because I… I can put slides together, but it’s like…
145 00:21:08.550 ⇒ 00:21:18.390 Speaker 1 (Elizabeth Conference Room): I guess I’m looking to you all to maybe have the conviction that I currently lack. Like, when you’re putting together these reports and dashboards, like, what’s the…
146 00:21:18.500 ⇒ 00:21:28.710 Speaker 1 (Elizabeth Conference Room): and Greg, you know, made an assumption around the narrative, but, like, do you have… Robert, since you’ve been close to it, like, do you feel confident about what the data is telling you about reading this business?
147 00:21:29.360 ⇒ 00:21:46.650 Robert Tseng: Yeah, I mean, I agree with… I agree with Bret’s perspective, and I also, like, kind of, like I kind of alluded to in Slack, it’s like, these changes are just, like, it’s… we’re doing death by a thousand cuts, but it’s, like, not the appropriate stage in the PLG, like, maturity, like, life cycle. Like, I… you know, we work with other clients that are…
148 00:21:46.670 ⇒ 00:21:58.819 Robert Tseng: like, in the early stage, like, they’re batching a lot of big changes at once, like, you want to significantly move the needle. You have to take big swings in order to really see, like, the impact. But, like, kind of…
149 00:21:59.070 ⇒ 00:22:19.019 Robert Tseng: moving a button there, like, adding a little tile there, like, those types of cosmetic things, like, it’s just… even if you see a lift, like, it’s just… there’s so many conflicting, conflating things. Maybe it’s just because, like, it’s just like a flash in the pan, like, the lift only happens for a week, and then it goes back to normal. So, like, it’s not, like, a substantial enough of a change to really, like.
150 00:22:19.390 ⇒ 00:22:20.939 Robert Tseng: Show that it, you know.
151 00:22:21.140 ⇒ 00:22:27.019 Robert Tseng: For… with any material impact in the long term, like, made things better or worse, like…
152 00:22:27.380 ⇒ 00:22:29.360 Robert Tseng: That’s… that’s what… that’s what I think.
153 00:22:29.800 ⇒ 00:22:45.029 Speaker 1 (Elizabeth Conference Room): And I agree with you. I guess maybe the context that you all lack is for the last 10 years, Readme has made, you know, massive changes with zero tracking, and the business is declining, growth is declining, you know, from
154 00:22:45.770 ⇒ 00:23:09.669 Speaker 1 (Elizabeth Conference Room): since I’ve been here at the highest, you know, 40% year-over-year revenue growth, ARR growth, to 5% in Q4. So much so that we let go a third of our team, right? So, like, people want answers, they want to know that the work they’re doing is moving the needle. And it’s… it’s on the minute button, like you said, it’s like, but we maybe have over-corrected.
155 00:23:09.670 ⇒ 00:23:11.990 Speaker 1 (Elizabeth Conference Room): It’s like, that’s where I’m at.
156 00:23:12.110 ⇒ 00:23:20.680 Speaker 1 (Elizabeth Conference Room): And, like, that’s what I’m asking you all for, and if you don’t… if you can’t work, like, within that style, I understand, because it’s not…
157 00:23:21.200 ⇒ 00:23:34.430 Speaker 1 (Elizabeth Conference Room): necessarily the best use of anyone’s time, but it’s where we’re at. That’s, you know, that’s where we’re at. There’s no other way around it. And I have to get myself out of that hole by showing up with
158 00:23:34.590 ⇒ 00:23:39.289 Speaker 1 (Elizabeth Conference Room): Answers every single week before we can move beyond.
159 00:23:40.560 ⇒ 00:23:40.940 Robert Tseng: Yeah.
160 00:23:41.450 ⇒ 00:23:41.770 Greg Stoutenburg: Yeah.
161 00:23:41.770 ⇒ 00:23:59.230 Robert Tseng: I… I… I understand… I guess I can see the position that you’re in, and yeah, I guess the way that we’re set up, just, like, it’s not that we don’t want to help, I think it’s just, like, it’s just not, like, I don’t know, we have the limits of, like, you know, there’s only so much that we can do with the limited budget and hours we have. So,
162 00:23:59.530 ⇒ 00:24:00.300 Robert Tseng: I think, like…
163 00:24:00.300 ⇒ 00:24:02.029 Speaker 1 (Elizabeth Conference Room): Where do you want to go from here?
164 00:24:03.220 ⇒ 00:24:12.239 Robert Tseng: Well, I mean, we have you for the rest of the month, and I want to, like, do what we can to give you the conviction to tell that story, so, like, you know, it’s…
165 00:24:12.400 ⇒ 00:24:13.820 Robert Tseng: Yeah, I, I think…
166 00:24:14.810 ⇒ 00:24:23.540 Robert Tseng: you know, I think it’s gonna sound similar to what we’ve already described, and if it’s just about, kind of, like, packaging it up differently, like, we can’t… we can do that.
167 00:24:23.920 ⇒ 00:24:38.000 Robert Tseng: And I think we will do that. I think we should put a deck in front of you, that’s… that’s got a lot more con… got a lot more context written into it, so that you can… so that you can tell the story, and if you feel good in sharing that with the team, then that’s…
168 00:24:38.000 ⇒ 00:24:51.610 Robert Tseng: that’s what we can do. Like, but if we’re not really going to kind of, like, expand the scope, then, like, I don’t know, there’s not really… like, I don’t really have any other suggestions at this point, you know? So, like, I… I think that’s… that’s all… that’s all we can do for now.
169 00:24:52.720 ⇒ 00:24:53.490 Speaker 1 (Elizabeth Conference Room): Like…
170 00:24:56.900 ⇒ 00:24:59.940 Robert Tseng: Yeah. Yeah, I guess,
171 00:25:01.240 ⇒ 00:25:09.400 Robert Tseng: So yeah, I get that that’s… yeah, that… I think that’s… that’s what we’ll do, like, probably… probably… and we… by tomorrow, we’ll… we’ll just kind of…
172 00:25:09.580 ⇒ 00:25:28.649 Robert Tseng: repackage what we have in a way that maybe it’s not going to be a dashboard that sits in, like, the amplitude space, because nobody’s, like, obviously going in there. We just… we’ll do it more creatively with some screenshots and stuff, and try to… and try to, like, basically tell… tell the same story. And we’ll… we’ll see if that… if you feel… if you feel better with that.
173 00:25:28.730 ⇒ 00:25:32.159 Robert Tseng: Yeah, I think… I think that’s… that… that… that’s what we can do.
174 00:25:32.720 ⇒ 00:25:37.739 Speaker 1 (Elizabeth Conference Room): Cool. That, that sounds good. I guess my, my last thing before.
175 00:25:37.740 ⇒ 00:25:41.019 Audio shared by Elizabeth Conference Room: you do that, and I hear you that the…
176 00:25:41.430 ⇒ 00:25:54.340 Audio shared by Elizabeth Conference Room: you’re not super concerned on, like, the .1 or 2 percentage points on things. Yeah. But something that every time I look at this that is, like, quite jarring for me is, like,
177 00:25:55.200 ⇒ 00:26:04.539 Audio shared by Elizabeth Conference Room: the volume is low, right, of sign-ups. We sign up, like, 1,500 people every two weeks, about. Yeah. And then, to see, like.
178 00:26:04.790 ⇒ 00:26:15.240 Audio shared by Elizabeth Conference Room: a 12% decrease in the amount of people, or I guess in this case it was an increase in the amount of people creating projects, like, I’m gonna have to… or, you know, a…
179 00:26:16.000 ⇒ 00:26:25.169 Audio shared by Elizabeth Conference Room: 6% increase, like, I’m gonna have to go… I’m gonna have to justify every single, like, why that happened every single time, right? I’m not gonna have to justify 4 versus 6.
180 00:26:25.350 ⇒ 00:26:42.499 Audio shared by Elizabeth Conference Room: Sure. Subscription successes, but it’s like, in every… people are gonna have questions. They’re like, whoa, that’s a big swing. Why are… why did 200 more people create projects in the… in this week than in the previous time period? Or, like, why did, 100 more people try to launch? You know, it’s like.
181 00:26:42.500 ⇒ 00:26:54.180 Speaker 1 (Elizabeth Conference Room): Every little detail is gonna be asked… people are gonna want answers, and maybe… maybe that’s kind of, like, the crux of the issue. It’s like, you all have provided reports
182 00:26:54.340 ⇒ 00:27:09.710 Speaker 1 (Elizabeth Conference Room): Because we’ve asked for them, but then I’m going in and trying to, through a mix of, like, QA and trying to build a narrative, it’s like I, myself, am having to justify what the reports mean for our business.
183 00:27:10.210 ⇒ 00:27:10.990 Speaker 1 (Elizabeth Conference Room): Yeah.
184 00:27:11.550 ⇒ 00:27:24.939 Robert Tseng: I think just session tracking heat maps, like, something qualitative for you to be able to, like, make an observa… like a… like a visual observation, like, is… is, like, what… what… what you could do. Like, I think that’s for… for… for something like that, like…
185 00:27:25.070 ⇒ 00:27:32.839 Robert Tseng: Yeah, I don’t really think the data… I know you’re coming back to us to ask, like, what… what happened.
186 00:27:32.840 ⇒ 00:27:52.890 Robert Tseng: we can’t, you know, obviously we’ll only be able to track between these steps. To fill in those blanks, you need… you need the qualitative look. You need to be able to hover over the shoulder of the user to observe them. Like, I think that’s… that’s, like, table stakes for us when we’re trying to, like, answer the question of, like, what’s… what happened in that drop-off, right? So…
187 00:27:53.920 ⇒ 00:27:54.670 Robert Tseng: Yeah.
188 00:27:54.980 ⇒ 00:27:58.659 Speaker 1 (Elizabeth Conference Room): Oh, okay,
189 00:27:59.770 ⇒ 00:28:13.659 Speaker 1 (Elizabeth Conference Room): I guess what it makes me question is, like, if the data is accurate, because I wouldn’t expect to see such a massive swing over a one-month period. I’m happy to add additional
190 00:28:13.780 ⇒ 00:28:20.790 Speaker 1 (Elizabeth Conference Room): like, data points in the qualitative form, but I guess, greg has a chart.
191 00:28:21.170 ⇒ 00:28:37.519 Greg Stoutenburg: Yeah, I think this is… I just… I just threw this together, but I think this is in line with the sort of thing that you’re thinking, right? So if you see at… I think the week showed November 30th, right, that there’s… there was some kind of… there was a difference in that week versus the week prior.
192 00:28:37.520 ⇒ 00:28:47.219 Greg Stoutenburg: when it comes to more holistic storytelling, you might like to see something like this along with it, and this is just a chart of everybody who signed up week over week, unique users. So if I find
193 00:28:47.270 ⇒ 00:29:04.729 Greg Stoutenburg: you know, when you have to answer those follow-up questions, and you might wonder, well, why is there a difference in one week versus another, right? Well, here, there were 789 signups, a week later, there were 768. So part of that context is there was, not by a lot, but, you know, negative 3%.
194 00:29:05.030 ⇒ 00:29:05.760 Greg Stoutenburg: growth.
195 00:29:05.760 ⇒ 00:29:29.050 Speaker 1 (Elizabeth Conference Room): Yeah, and that’s fine, like, I think… but I think what you’re getting at, Craig, is exactly right. It’s like, we have to tell the story across every piece of the funnel. So it’s like, alright, looking at that chart, sign-up’s relatively flat over the last quarter. That’s… that’s fine. A small increase in maybe the last couple of weeks, it looks like. But it’s like, okay, then we have to do the same thing on the project creation level, the same thing in maybe the
196 00:29:29.050 ⇒ 00:29:36.689 Speaker 1 (Elizabeth Conference Room): Not rubber on, like, the attempted launch level, and then the same thing on, like, the subscription success level. It’s like, establishing a baseline there.
197 00:29:36.710 ⇒ 00:29:49.899 Speaker 1 (Elizabeth Conference Room): never happened. And then… and that’s on… on my… that’s my bad as well. And then layering on, like, the experiments that we ran, and if… and why… it’s like, I think all that triangulation is what I’m missing.
198 00:29:50.330 ⇒ 00:29:51.230 Greg Stoutenburg: I see.
199 00:29:52.390 ⇒ 00:29:53.070 Robert Tseng: Okay.
200 00:29:54.000 ⇒ 00:30:05.459 Robert Tseng: Yeah, no, I… I think… I think the slide… I think the slide deck is the best way to present that. I don’t think amplitude… like, a dashboard is going to be able to help you triangulate that, so we will… we will deliver that, yeah.
201 00:30:05.460 ⇒ 00:30:24.830 Speaker 1 (Elizabeth Conference Room): Okay. We’ll chat again. Yeah, I appreciate it, and again, like, I’m sorry that this engagement hasn’t gone the way that you hoped, and ideally we can get things back on track, but if not, like, you know, it’s all good. And, we’ll see where we land.
202 00:30:25.150 ⇒ 00:30:27.219 Robert Tseng: Okay, yeah, thanks, Phoebe. Appreciate your time.
203 00:30:27.220 ⇒ 00:30:28.770 Speaker 1 (Elizabeth Conference Room): Fair. Thanks. Alright.
204 00:30:29.080 ⇒ 00:30:29.560 Greg Stoutenburg: Thank you.