Meeting Title: Brainforge x Stackblitz Weekly Check Date: 2025-02-07 Meeting participants: Luke Daque, Mitchell Wright, Uttam Kumaran
WEBVTT
1 00:00:24.080 ⇒ 00:00:25.030 Luke Daque: I’m Mitch.
2 00:00:25.490 ⇒ 00:00:27.010 Mitchell Wright: Hey? How you doing.
3 00:00:27.140 ⇒ 00:00:28.520 Luke Daque: Doing well, how are you.
4 00:00:28.870 ⇒ 00:00:29.850 Mitchell Wright: Good.
5 00:00:36.000 ⇒ 00:00:40.854 Mitchell Wright: let’s see. Okay, I’m finally opening up the real dashboard.
6 00:00:41.710 ⇒ 00:00:44.892 Mitchell Wright: I have not looked at that quite yet.
7 00:00:45.460 ⇒ 00:00:51.750 Mitchell Wright: And also I know that I owe you a bunch of stuff with like this segment connection, and.
8 00:00:52.440 ⇒ 00:00:54.959 Luke Daque: Yeah, but that’s fine.
9 00:00:55.200 ⇒ 00:00:59.039 Luke Daque: I know you’ve been like busy, and you were like out yesterday. I believe.
10 00:00:59.700 ⇒ 00:01:03.840 Mitchell Wright: Yeah, yeah. Yesterday, okay, I just requested access to the.
11 00:01:04.459 ⇒ 00:01:04.949 Luke Daque: Oh, it’s.
12 00:01:04.950 ⇒ 00:01:09.879 Mitchell Wright: Bill, so I think will need to give me.
13 00:01:10.790 ⇒ 00:01:18.880 Luke Daque: Yeah, okay, I can show you. Well, we don’t have access yet.
14 00:01:25.960 ⇒ 00:01:29.559 Luke Daque: Yeah, maybe let’s wait a bit for Utana. I think he’s joining.
15 00:01:29.880 ⇒ 00:01:30.519 Mitchell Wright: Okay. Cool.
16 00:01:33.680 ⇒ 00:01:35.919 Luke Daque: But yeah, just to give you like.
17 00:01:37.393 ⇒ 00:01:56.810 Luke Daque: like updates, I did like try to work on, because we already have the postgres data in Snowflake. So I didn’t try to work on like a users model. And the organization’s model, at least to whatever data that we had already in snowflake. But there’s still a couple here that
18 00:01:58.250 ⇒ 00:02:06.480 Luke Daque: I don’t see directly, so we might need some logic like like 1st Hobby start date in organizations. For example like, How do we know.
19 00:02:06.480 ⇒ 00:02:12.397 Mitchell Wright: Yeah, I need to. I some of that stuff, because there’s there’s some stuff that is
20 00:02:12.930 ⇒ 00:02:18.730 Mitchell Wright: yeah, it’s like, just like, I need to. Yeah, make some changes to that. So.
21 00:02:18.730 ⇒ 00:02:19.350 Luke Daque: Cool.
22 00:02:20.008 ⇒ 00:02:21.280 Mitchell Wright: I can go through.
23 00:02:21.390 ⇒ 00:02:26.861 Mitchell Wright: Yeah. I wanna say today, but I actually don’t want to commit to that, because I don’t know that I’ll have time.
24 00:02:27.320 ⇒ 00:02:28.920 Luke Daque: Sure no worries. Yeah.
25 00:02:29.340 ⇒ 00:02:35.390 Uttam Kumaran: I think even Monday or Monday or Tuesday is okay. As long as like we, we’re just gonna keep sort of walking.
26 00:02:35.390 ⇒ 00:02:41.229 Mitchell Wright: Yeah, anything that doesn’t make sense or that, like you, you’re like, I don’t have a thing just like ignore it, for now and then.
27 00:02:41.230 ⇒ 00:02:47.710 Uttam Kumaran: That’s basically what we’re doing. Yeah, and then look, do you want to ask about the messages?
28 00:02:48.570 ⇒ 00:02:54.089 Luke Daque: Yeah, sure. Yeah, we did try to load the Csv files as like seeds in Dbt.
29 00:02:54.589 ⇒ 00:02:59.199 Luke Daque: and like the chat mid message messages is like pretty huge. And there’s like
30 00:02:59.420 ⇒ 00:03:03.370 Luke Daque: fields that are other than message. The content basically is very
31 00:03:03.570 ⇒ 00:03:07.349 Luke Daque: huge. It’s getting into errors. I’m not sure if, like.
32 00:03:08.067 ⇒ 00:03:16.690 Luke Daque: are we gonna be using any metrics that would use the message content? Or are we like more into just counting the number of messages per user or stuff.
33 00:03:16.690 ⇒ 00:03:20.759 Mitchell Wright: Yeah, so I don’t think we’re gonna be.
34 00:03:22.310 ⇒ 00:03:37.240 Mitchell Wright: I don’t think that there’s necessarily any analytics that we’re going to run off of the message content. So you could probably pull that out. But there’s definitely some analysis that we’re gonna wanna do off of the message content. So
35 00:03:39.000 ⇒ 00:03:42.800 Uttam Kumaran: Like, what can you? Yeah, give us a couple of anecdotes because we will.
36 00:03:43.490 ⇒ 00:03:45.879 Uttam Kumaran: We will. I mean, there’s 2 strategies, one.
37 00:03:46.020 ⇒ 00:03:52.539 Uttam Kumaran: we just need to split it up even further or second, we need to preprocess any other.
38 00:03:52.540 ⇒ 00:03:59.332 Mitchell Wright: Probably need to do that. So cause the what what we want to do a little bit of is
39 00:04:00.830 ⇒ 00:04:03.800 Mitchell Wright: Do some analysis on
40 00:04:04.090 ⇒ 00:04:14.629 Mitchell Wright: trying to like bucket the types of apps people are building. And so a lot of that’s gonna be based off of messages. It, that so that. But that could
41 00:04:15.310 ⇒ 00:04:20.689 Mitchell Wright: I need to look at the data again, I haven’t looked at it in a second, but I think chats might also have
42 00:04:21.430 ⇒ 00:04:27.320 Mitchell Wright: we? We might be able to just use chats, because I think that there’s like a description or something like that where we.
43 00:04:27.320 ⇒ 00:04:32.499 Uttam Kumaran: Lou, can you pull up one of those bodies that we looked especially the large one that that we basically was looking at.
44 00:04:36.220 ⇒ 00:04:38.270 Luke Daque: Yeah, sure. Can you see my screen?
45 00:04:38.580 ⇒ 00:04:41.189 Uttam Kumaran: Yeah, yeah. Maybe you want to zoom in a little bit.
46 00:04:42.007 ⇒ 00:04:44.589 Luke Daque: I don’t know if I can zoom in, but.
47 00:04:44.590 ⇒ 00:04:46.460 Mitchell Wright: I think I’m good I’ve got. I’ve got a big monitor.
48 00:04:46.460 ⇒ 00:04:49.420 Uttam Kumaran: Okay, cool. I have a big monitor, too. Okay, cool. Great. They were fine.
49 00:04:49.730 ⇒ 00:04:59.039 Luke Daque: So yeah, let me open up like, I open this, the Csv for the chat messages like, here’s like examples of what they look like, and.
50 00:04:59.930 ⇒ 00:05:04.870 Luke Daque: This is, we might need to like, do some pre-processing, because, like it has, like
51 00:05:05.040 ⇒ 00:05:09.029 Luke Daque: page breaks and stuff like that with which we can’t parse, as Json.
52 00:05:09.640 ⇒ 00:05:10.259 Mitchell Wright: Figure it out.
53 00:05:10.260 ⇒ 00:05:13.350 Luke Daque: Like, remove this and stuff like cleaning this up.
54 00:05:14.830 ⇒ 00:05:15.890 Luke Daque: Yeah, it’s pretty.
55 00:05:16.050 ⇒ 00:05:22.949 Mitchell Wright: For now for now let’s yeah. Let’s pull the content out. And just don’t put that in.
56 00:05:23.280 ⇒ 00:05:27.470 Mitchell Wright: and we can decide if we want to
57 00:05:27.970 ⇒ 00:05:30.979 Mitchell Wright: do that at a later point in time. But yeah, that’s.
58 00:05:31.500 ⇒ 00:05:37.210 Uttam Kumaran: Do you have the example of the one, Luke? That was like the really large one cause then I I guess I’m trying to see
59 00:05:37.740 ⇒ 00:05:40.746 Uttam Kumaran: if we can, if we should just make like
60 00:05:41.530 ⇒ 00:05:46.059 Uttam Kumaran: if we just split it up one layer further where it’s just the back and forth, or every row.
61 00:05:47.770 ⇒ 00:05:49.100 Uttam Kumaran: Cause. I mean, we’re it’s like.
62 00:05:49.300 ⇒ 00:05:52.800 Uttam Kumaran: yeah, that way. We can get that sort of granularity.
63 00:05:55.410 ⇒ 00:05:56.680 Uttam Kumaran: Or something like that.
64 00:05:56.930 ⇒ 00:06:03.189 Luke Daque: Yeah, I tried to split it like per message, and it’s getting pretty huge as well.
65 00:06:03.350 ⇒ 00:06:08.310 Luke Daque: But yeah, like, body of more.
66 00:06:09.080 ⇒ 00:06:12.209 Uttam Kumaran: Go to the body of like this one that’s like.
67 00:06:12.800 ⇒ 00:06:15.419 Uttam Kumaran: or whichever field was like breaking. Oh, there’s a.
68 00:06:15.420 ⇒ 00:06:15.800 Mitchell Wright: Content.
69 00:06:15.800 ⇒ 00:06:22.280 Luke Daque: This one message, yeah, something like this,
70 00:06:25.080 ⇒ 00:06:25.480 Mitchell Wright: Yeah.
71 00:06:25.480 ⇒ 00:06:27.749 Uttam Kumaran: Mitch like. What are we looking at in this
72 00:06:29.370 ⇒ 00:06:32.419 Uttam Kumaran: cause? There’s I guess we can. We can run some cleanup.
73 00:06:32.920 ⇒ 00:06:33.940 Mitchell Wright: Yeah, yeah.
74 00:06:33.940 ⇒ 00:06:34.700 Uttam Kumaran: These, but.
75 00:06:35.000 ⇒ 00:06:37.719 Mitchell Wright: Let’s just leave this out for now. I don’t think.
76 00:06:38.280 ⇒ 00:06:41.080 Uttam Kumaran: And I guess how is this different than the than the chat?
77 00:06:41.340 ⇒ 00:06:44.179 Uttam Kumaran: Because the the but the chat is the higher.
78 00:06:44.180 ⇒ 00:06:50.999 Mitchell Wright: Chat is, and chat is like the overall. Right this is. This is a chat. Has chat messages right? So
79 00:06:53.190 ⇒ 00:06:58.360 Mitchell Wright: in in chat there is.
80 00:06:58.920 ⇒ 00:07:00.030 Mitchell Wright: I mean, I think I feel.
81 00:07:00.030 ⇒ 00:07:05.149 Uttam Kumaran: I feel like we should do the message back and forth as rose
82 00:07:05.915 ⇒ 00:07:10.600 Uttam Kumaran: and we can take a look. Because, yeah, I mean, I feel like it’s it’s really valuable.
83 00:07:11.150 ⇒ 00:07:11.500 Mitchell Wright: Yeah.
84 00:07:11.500 ⇒ 00:07:13.590 Uttam Kumaran: Clean. We’ll clean up a lot of the.
85 00:07:13.970 ⇒ 00:07:16.242 Uttam Kumaran: I will clean up a lot of this stuff. But
86 00:07:17.120 ⇒ 00:07:22.159 Uttam Kumaran: I mean, I don’t know. I think it could be really interesting summarization and stuff and extraction.
87 00:07:22.340 ⇒ 00:07:35.730 Mitchell Wright: Yeah. So I think there there is value in the messages. I don’t know that there’s value like right now, because we don’t, we don’t have the resources to to do much. I think the description here was probably gonna be enough.
88 00:07:36.521 ⇒ 00:07:40.729 Mitchell Wright: or probably what we need. So yeah.
89 00:07:40.730 ⇒ 00:07:43.760 Luke Daque: Description would be like the the title of the message, right like.
90 00:07:44.880 ⇒ 00:07:45.260 Uttam Kumaran: I interfere.
91 00:07:45.260 ⇒ 00:07:46.420 Luke Daque: Like a chat. Yeah.
92 00:07:46.740 ⇒ 00:07:48.580 Uttam Kumaran: What the chat like. Yeah, the whatever the.
93 00:07:49.790 ⇒ 00:07:52.985 Luke Daque: Stack with inspired website with both.
94 00:07:54.090 ⇒ 00:07:55.649 Mitchell Wright: Yeah, some of these aren’t great.
95 00:07:56.060 ⇒ 00:07:58.360 Mitchell Wright: I think some of them are pretty good.
96 00:07:59.320 ⇒ 00:08:07.150 Uttam Kumaran: I mean, what? How about we just like? Look, let’s just move forward. As long as you have the Id. We’ll just move forward with counts, and then we’ll come back, and
97 00:08:07.150 ⇒ 00:08:07.659 Uttam Kumaran: I’ll do.
98 00:08:07.660 ⇒ 00:08:11.160 Uttam Kumaran: We’ll do a string cleanup. We’ll see if, like
99 00:08:11.660 ⇒ 00:08:20.930 Uttam Kumaran: that gets the Max length down. If not, then we can consider breaking up messages. And think about the data model there.
100 00:08:21.704 ⇒ 00:08:27.500 Uttam Kumaran: Yeah, I I think one. I think we could. You could. You should totally use AI to parse
101 00:08:28.212 ⇒ 00:08:33.520 Uttam Kumaran: a couple of pieces of information. And second, I mean, we could definitely do it like the Dumber way, and just sort of parse for.
102 00:08:33.520 ⇒ 00:08:35.650 Mitchell Wright: Yeah, even even if it’s
103 00:08:37.100 ⇒ 00:08:39.989 Uttam Kumaran: If you want to look at positive chats, negative chat like.
104 00:08:40.200 ⇒ 00:08:44.700 Mitchell Wright: Well, yeah, and even even like the very 1st message, I feel like, yeah.
105 00:08:44.700 ⇒ 00:08:57.370 Mitchell Wright: I’m pretty sure that’s what we’re basing the description off of like on the product side. But even the very 1st message for every chat will probably be pretty descriptive of what they’re trying to build. I would say
106 00:08:57.610 ⇒ 00:08:58.490 Mitchell Wright: so.
107 00:08:59.250 ⇒ 00:08:59.880 Luke Daque: Okay. Great.
108 00:08:59.880 ⇒ 00:09:01.249 Luke Daque: This one, for example.
109 00:09:01.380 ⇒ 00:09:04.980 Luke Daque: Well, doesn’t look like they’re building anything for this one. But.
110 00:09:06.090 ⇒ 00:09:10.530 Mitchell Wright: Yeah, yeah, yeah, that’s that’s just like part. That’s a chat message that’s like, you know.
111 00:09:10.650 ⇒ 00:09:13.470 Mitchell Wright: in the middle of the chat, where they’re just fixing.
112 00:09:13.830 ⇒ 00:09:14.260 Luke Daque: And.
113 00:09:14.260 ⇒ 00:09:23.009 Mitchell Wright: The look of the actual of the actual app versus, you know. Order ordering
114 00:09:24.074 ⇒ 00:09:27.710 Mitchell Wright: like window function, right group by chat. Id.
115 00:09:28.830 ⇒ 00:09:39.370 Mitchell Wright: The 1st chat message would be like right. But this and this is also not the full database. Right? So we’ll probably be the 1st Id and a lot of these, because it just is a
116 00:09:39.370 ⇒ 00:09:41.780 Mitchell Wright: yeah randomly grab of.
117 00:09:42.050 ⇒ 00:09:45.020 Luke Daque: The message, order per user, Id and project.
118 00:09:45.020 ⇒ 00:09:53.539 Uttam Kumaran: Oh, okay, I mean. So then, honestly, it’s like, let’s if we’re only if there’s only like a few rows there that basically, they’re either like putting a book in there, or something.
119 00:09:53.740 ⇒ 00:09:55.140 Luke Daque: Then we can.
120 00:09:55.400 ⇒ 00:09:59.730 Uttam Kumaran: We can just like Luke. What we should do is we could just pre process.
121 00:09:59.890 ⇒ 00:10:01.210 Luke Daque: In python.
122 00:10:01.330 ⇒ 00:10:08.789 Uttam Kumaran: Null out the messages that are larger than some size, because some of these are small, and we can still leverage them like a lot of these don’t look like they’re gonna
123 00:10:08.930 ⇒ 00:10:10.159 Uttam Kumaran: they’re going to be rough.
124 00:10:10.510 ⇒ 00:10:10.910 Luke Daque: Yeah.
125 00:10:11.231 ⇒ 00:10:17.670 Uttam Kumaran: I just wanna we just wanna make sure that for the ones that are larger character count that we
126 00:10:18.050 ⇒ 00:10:20.920 Uttam Kumaran: you could just kill them before they enter, and then we can come back
127 00:10:21.290 ⇒ 00:10:25.040 Uttam Kumaran: to figure out how to get those. But like, there’s only like 10 or 20 like.
128 00:10:25.600 ⇒ 00:10:26.710 Uttam Kumaran: Yeah, we’re good. Okay.
129 00:10:26.710 ⇒ 00:10:29.830 Mitchell Wright: Okay, cool. I think that sounds like a good strategy.
130 00:10:30.430 ⇒ 00:10:34.205 Uttam Kumaran: Okay? And then we we also put together a real dashboard.
131 00:10:34.520 ⇒ 00:10:38.250 Mitchell Wright: Yeah, I requested access. I think someone has to approve.
132 00:10:38.660 ⇒ 00:10:41.820 Luke Daque: Yeah, I think it’s going to uta stacks.com.
133 00:10:41.820 ⇒ 00:10:43.179 Uttam Kumaran: Oh, let me. Let me do that.
134 00:10:44.710 ⇒ 00:10:45.340 Uttam Kumaran: Okay.
135 00:10:47.250 ⇒ 00:10:50.868 Luke Daque: But yeah, this is like what it looks like. Currently, I have.
136 00:10:51.920 ⇒ 00:10:57.909 Luke Daque: there’s like 3 dashboards. At the moment the subscriptions coming from stripe.
137 00:10:58.630 ⇒ 00:11:02.409 Luke Daque: The users coming from postgres and organizations as well.
138 00:11:02.700 ⇒ 00:11:10.650 Luke Daque: So subscriptions look something like this like, we have the total subscriptions. Total. First, st subscriptions.
139 00:11:11.050 ⇒ 00:11:14.579 Luke Daque: yeah, these are just like metrics matrices that
140 00:11:15.500 ⇒ 00:11:24.270 Luke Daque: added here, like all the dimensions like the subscription item, plan, strike plan name.
141 00:11:24.420 ⇒ 00:11:25.350 Luke Daque: So it’s literally.
142 00:11:25.350 ⇒ 00:11:32.460 Uttam Kumaran: Do you want to give? Maybe like a 30 second overview of real? I mean, Mitch is Mitch is like, maybe just give like a
143 00:11:32.780 ⇒ 00:11:36.419 Uttam Kumaran: just start at the homepage and sort of just give like the gist. And then also.
144 00:11:36.796 ⇒ 00:11:39.520 Uttam Kumaran: last time we went over some of the code. But yeah, we
145 00:11:40.200 ⇒ 00:11:41.530 Uttam Kumaran: you could just go over everything.
146 00:11:42.286 ⇒ 00:11:46.299 Luke Daque: Yeah, I don’t like running this locally, but let me see if I can.
147 00:11:48.520 ⇒ 00:11:54.579 Luke Daque: But yeah, I think we can do this. I I won’t be able to see the home page. This is the current.
148 00:11:54.580 ⇒ 00:11:55.640 Uttam Kumaran: Oh, okay. Okay.
149 00:11:56.190 ⇒ 00:11:58.120 Luke Daque: But thank you.
150 00:11:58.120 ⇒ 00:12:00.230 Uttam Kumaran: I have it up, too. I can. I can share.
151 00:12:01.070 ⇒ 00:12:01.730 Luke Daque: Okay.
152 00:12:02.370 ⇒ 00:12:03.490 Uttam Kumaran: Okay, let me do that.
153 00:12:04.640 ⇒ 00:12:08.755 Uttam Kumaran: Yeah. So sort of the reason why
154 00:12:09.620 ⇒ 00:12:15.809 Uttam Kumaran: we like real is, it’s bi as code primarily, meaning we can.
155 00:12:15.870 ⇒ 00:12:40.149 Uttam Kumaran: We can deploy have different environments and also make sure that the logic stays in Dbt and the bi layer really just maintains the representations and the availability of that in the tool. But when you get into real you’ll see this. You have access to one dashboard that we’ve created here. The nice thing is this should be sort of similar, and look and feel to stripe, except like
156 00:12:40.240 ⇒ 00:12:50.929 Uttam Kumaran: sort of on steroids. In that you have a lot of availability, for, like parameterized filter mechanisms. So what I mean by that is, you can quickly go to say like, Oh, I want to look at like the last 4 weeks.
157 00:12:51.050 ⇒ 00:12:55.659 Uttam Kumaran: I want to do a comparison to the previous period. You can select what those periods are.
158 00:12:56.890 ⇒ 00:13:00.639 Uttam Kumaran: And it pre-calculates a lot of this.
159 00:13:00.800 ⇒ 00:13:29.459 Uttam Kumaran: a lot of these on the fly in your browser, which means, you know, like, if we were to do comparisons and then build not only what the differences, but then what the percentages are. Those are 4 additional metrics that we have to calculate for every single combination. Typically, the way you would do that in Dbt is like you would have every combination of like comparing a day, and then on the fly. Your your Bi. Would then say, cool, what are the 2 values? Let me calculate it. They do a lot of that for you off off Rep.
160 00:13:30.590 ⇒ 00:13:38.810 Uttam Kumaran: And the nice thing is, you can quickly be like cool. I want to filter that. You can click. And the mechanism is actually a little bit nicer to do those sorts of things.
161 00:13:39.438 ⇒ 00:13:44.320 Uttam Kumaran: Of course you can subscribe and share these dashboards.
162 00:13:45.250 ⇒ 00:13:46.060 Uttam Kumaran: And
163 00:13:46.210 ⇒ 00:14:04.529 Uttam Kumaran: they have a couple of nice out of the box features the other thing that they’re building. And maybe this is what Luke, you can show. And when you actually develop these real dashboards. Locally there’s they’re now out pushing out this process where there is going to be a ui based
164 00:14:04.620 ⇒ 00:14:16.208 Uttam Kumaran: like development process. That’s we’re getting a demo of that next week, actually. And that should go live. So in case you have analysts that still want like a ui based, they’re adding that
165 00:14:16.990 ⇒ 00:14:25.550 Uttam Kumaran: But, Luke, I don’t know if you want to just show, sort of yeah, just like, sort of what the dashboard creation process it looks like, and how we actually
166 00:14:25.750 ⇒ 00:14:30.470 Uttam Kumaran: basically go from Dbt model to do the real dashboard.
167 00:14:31.390 ⇒ 00:14:32.300 Luke Daque: Yeah, sure.
168 00:14:32.410 ⇒ 00:14:35.384 Luke Daque: So for basically in real, we just
169 00:14:36.860 ⇒ 00:14:49.339 Luke Daque: we we install rail in locally. And then we have, like our own like folder in this in the repository for real and just, there’s like a lot of real commands.
170 00:14:49.640 ⇒ 00:15:00.340 Luke Daque: like a real start would be building a project from scratch like deploying the A, real, the real project. So basically, it’s like Dvt, where it’s like all
171 00:15:00.690 ⇒ 00:15:02.649 Luke Daque: it can be code based. At most.
172 00:15:03.080 ⇒ 00:15:05.010 Luke Daque: it’s like you have like sources.
173 00:15:05.428 ⇒ 00:15:07.979 Luke Daque: What we’re doing here is we’re just connecting
174 00:15:08.120 ⇒ 00:15:14.019 Luke Daque: to Snowflake and selecting from the March models that we created via Dbt.
175 00:15:14.320 ⇒ 00:15:16.750 Luke Daque: so this becomes the source of the dashboard.
176 00:15:17.110 ⇒ 00:15:22.139 Luke Daque: and then we can also create models from stack up from rail itself.
177 00:15:22.290 ⇒ 00:15:28.280 Luke Daque: like, if ever we need to do any additional calculations.
178 00:15:28.540 ⇒ 00:15:31.719 Luke Daque: But ideally, we should already do this in DVD, that’s why we have.
179 00:15:31.720 ⇒ 00:15:34.950 Uttam Kumaran: Yeah. And the and the reasons they have they have that is
180 00:15:35.370 ⇒ 00:15:47.100 Uttam Kumaran: real supports, multiple database sources. But let’s say you want to select from a Csv. From a Postgres instance, from Click house and from Snowflake you can bring that all in a source and then combine it
181 00:15:47.250 ⇒ 00:15:54.429 Uttam Kumaran: like. Of course, this is great because we’d only have one. So typically for us on the model side, we just pass through great selects.
182 00:15:55.590 ⇒ 00:15:58.539 Mitchell Wright: Because we’re already dealing all the transformation. Dbt.
183 00:15:58.540 ⇒ 00:15:59.910 Mitchell Wright: yeah, yeah, that makes total sense.
184 00:16:00.240 ⇒ 00:16:07.830 Luke Daque: But yeah. And then this, this one’s like new. But the BA, basically the metrics part is where the dashboard
185 00:16:09.135 ⇒ 00:16:13.620 Luke Daque: configuration happens like whatever the name of the dashboard is.
186 00:16:13.870 ⇒ 00:16:40.969 Uttam Kumaran: Yeah, and a couple and a couple of really great stuff. Here is one, you you basically can just start to do the display names, descriptions, things like that. But also you can have your measures. Have certain functionality. For example, if it’s an average, don’t want it to. Like, if if it’s like a, it’s average, you may not want it to be summed up. So you don’t want to have that as a percent of total, for example. So you can adjust some of those things formats the expression
187 00:16:41.281 ⇒ 00:16:51.340 Uttam Kumaran: things like that. But this is really just like the the final semantic layer. You could say, most of these will just be like selects, or some aggregation on top of the dimensions that we give.
188 00:16:51.768 ⇒ 00:16:54.039 Uttam Kumaran: Whether it’s the account. Min max,
189 00:16:55.120 ⇒ 00:17:13.069 Uttam Kumaran: But again, 1 1 thing this helps is it doesn’t help to bloat the the final marks models with each of these like reporting metrics, and instead, you can sort of keep the dimensions the same and build sort of like. Okay, what is the actual final end? Columns and sort of have that all here really nicely.
190 00:17:13.390 ⇒ 00:17:14.949 Mitchell Wright: Yeah, yeah, no. This.
191 00:17:14.950 ⇒ 00:17:16.940 Uttam Kumaran: And all in the repository, like.
192 00:17:16.940 ⇒ 00:17:17.550 Luke Daque: Yeah.
193 00:17:17.550 ⇒ 00:17:22.240 Mitchell Wright: Nice to have like 1 1 repo. And then, yeah, like, kind of have the semantic layer like you said.
194 00:17:22.240 ⇒ 00:17:22.740 Uttam Kumaran: Yeah.
195 00:17:22.849 ⇒ 00:17:24.459 Mitchell Wright: Loading those models.
196 00:17:24.460 ⇒ 00:17:49.670 Uttam Kumaran: And to take that one step further. We will. You can actually deploy. So real is a deployment. And so ideally, we will have as a part of the Cicd. So you can verify that new model. Changes aren’t affecting your deployment of your Bi tool, which always happens. It’s always an issue that we face where you change a column name or something, and Looker doesn’t give you like a hey? This is broken.
197 00:17:49.690 ⇒ 00:18:03.530 Uttam Kumaran: It breaks somewhere, and you have to do, you have to run like a content validation or something? We would. We run a real validate as part of our Cicd process. So you basically have, like the end to end, which is really really nice.
198 00:18:04.230 ⇒ 00:18:05.000 Mitchell Wright: Yeah.
199 00:18:05.390 ⇒ 00:18:08.330 Uttam Kumaran: And it’s so simple like, I don’t know I like, I really feel like
200 00:18:08.460 ⇒ 00:18:11.899 Uttam Kumaran: you can get away with not having like a full time
201 00:18:12.020 ⇒ 00:18:17.299 Uttam Kumaran: person just building dashboards. It pushes a lot of this work back onto the Ae. Who really
202 00:18:17.580 ⇒ 00:18:21.250 Uttam Kumaran: also has a lot of the knowledge about like the metrics and things like that.
203 00:18:23.130 ⇒ 00:18:23.970 Uttam Kumaran: So.
204 00:18:27.320 ⇒ 00:18:34.439 Mitchell Wright: Yeah, this is great. Cause I cause right now, my, my thought is, we’re probably gonna hire 2
205 00:18:34.960 ⇒ 00:18:42.789 Mitchell Wright: data engineer type people, but they’ll also probably be doing some of the dashboard analytics stuff as well. So
206 00:18:42.900 ⇒ 00:18:45.665 Mitchell Wright: I think they’ll they’ll probably like this.
207 00:18:46.060 ⇒ 00:18:46.660 Uttam Kumaran: Yeah.
208 00:18:47.040 ⇒ 00:18:49.820 Mitchell Wright: Because because of all the reasons we just talked about.
209 00:18:50.130 ⇒ 00:18:50.790 Uttam Kumaran: Yeah.
210 00:18:51.100 ⇒ 00:19:00.519 Uttam Kumaran: I think again, it’s like some at some level. I think executives may be like, I want a tableau or something, but in, I think, in terms of speed, and making sure that, like
211 00:19:00.990 ⇒ 00:19:06.739 Uttam Kumaran: you don’t have, you don’t have the delays that happen when a model has to go through a de and ae and an analyst.
212 00:19:06.960 ⇒ 00:19:12.869 Uttam Kumaran: And then like that happens. For even like small column changes and stuff, we found that this is really really effective.
213 00:19:14.270 ⇒ 00:19:18.159 Uttam Kumaran: You know, and that way we can just continue to sprint. You still get dashboard outputs.
214 00:19:19.820 ⇒ 00:19:25.229 Uttam Kumaran: Which are with, and the dashboard functionality in my mind is enough. Some people, of course, want like
215 00:19:25.900 ⇒ 00:19:28.500 Uttam Kumaran: they want a lot of different things in their dashboard, so I can’t.
216 00:19:28.500 ⇒ 00:19:35.739 Mitchell Wright: Yeah, yeah, we’ll we’ll kind of see. But I think, for now it’s probably pretty good for
217 00:19:36.110 ⇒ 00:19:44.030 Mitchell Wright: just having something up and running and and starting to do some analysis based on
218 00:19:44.210 ⇒ 00:19:50.470 Mitchell Wright: kind of what we’ve already got. So yeah, cool.
219 00:19:51.872 ⇒ 00:19:55.287 Mitchell Wright: Awesome. Yeah. This looks great.
220 00:19:56.430 ⇒ 00:19:59.119 Mitchell Wright: love, that we’re getting some of the
221 00:19:59.300 ⇒ 00:20:10.410 Mitchell Wright: modeling done. Would love to just go over what? Where? I’m blocking you guys, so I can make sure that I get some stuff on.
222 00:20:10.880 ⇒ 00:20:11.680 Uttam Kumaran: Yeah.
223 00:20:11.680 ⇒ 00:20:12.280 Mitchell Wright: View.
224 00:20:13.365 ⇒ 00:20:17.730 Uttam Kumaran: I think it would be great to to one get segment in.
225 00:20:17.980 ⇒ 00:20:18.360 Mitchell Wright: Right.
226 00:20:18.623 ⇒ 00:20:26.539 Uttam Kumaran: So then we can sprint. I think we’ve delivered all of the info you need. But if you wanna hop on and do that together, and we can schedule that.
227 00:20:26.540 ⇒ 00:20:28.279 Mitchell Wright: We can just look at that right now, while we’re.
228 00:20:28.280 ⇒ 00:20:29.090 Uttam Kumaran: Yeah, okay.
229 00:20:31.640 ⇒ 00:20:35.940 Uttam Kumaran: You should have it, should some. Some stuff should be in the slack thread. But let me.
230 00:20:36.150 ⇒ 00:20:38.390 Mitchell Wright: Yeah, let me find the thread.
231 00:20:51.420 ⇒ 00:20:54.540 Mitchell Wright: Do you have a link to that?
232 00:20:55.451 ⇒ 00:20:57.136 Uttam Kumaran: Yes, let me
233 00:21:06.360 ⇒ 00:21:08.770 Uttam Kumaran: I swear I saw a thread about it. But
234 00:21:08.770 ⇒ 00:21:13.780 Uttam Kumaran: yeah, I’ve I literally was been search. I every day I do this. So I go like, where was that message I sent
235 00:21:20.510 ⇒ 00:21:21.270 Mitchell Wright: Son.
236 00:21:23.830 ⇒ 00:21:25.310 Luke Daque: Yes, we need to use the
237 00:21:25.540 ⇒ 00:21:31.060 Luke Daque: PIN, the channel functionality. But I think that only gives one message right?
238 00:21:32.550 ⇒ 00:21:36.330 Uttam Kumaran: Yeah, I oh, yeah. Okay. Here.
239 00:21:36.700 ⇒ 00:21:37.370 Luke Daque: Yeah.
240 00:21:41.010 ⇒ 00:21:42.329 Uttam Kumaran: You get that notification.
241 00:21:44.340 ⇒ 00:21:49.308 Mitchell Wright: Yes, I think. So that up perfect. Okay,
242 00:21:50.990 ⇒ 00:21:54.760 Mitchell Wright: okay, one password. Pull this up.
243 00:21:57.310 ⇒ 00:22:02.300 Mitchell Wright: Okay. So let me share my screen and we’ll just walk through this.
244 00:22:05.850 ⇒ 00:22:12.730 Mitchell Wright: Okay, so let me log in here
245 00:22:18.160 ⇒ 00:22:20.576 Mitchell Wright: and let me grab.
246 00:22:21.580 ⇒ 00:22:22.750 Mitchell Wright: Let’s go.
247 00:22:38.060 ⇒ 00:22:39.629 Mitchell Wright: No, that’s not it.
248 00:22:40.090 ⇒ 00:22:41.930 Mitchell Wright: Wait. Why is it logging me into.
249 00:22:43.640 ⇒ 00:22:45.230 Mitchell Wright: That’s not what I want.
250 00:22:47.537 ⇒ 00:22:49.442 Uttam Kumaran: Have this problem times a hundred.
251 00:22:51.170 ⇒ 00:22:52.359 Uttam Kumaran: There’s like 10.
252 00:22:52.360 ⇒ 00:22:53.510 Mitchell Wright: Yeah, it’s.
253 00:22:54.056 ⇒ 00:22:56.240 Uttam Kumaran: You want to know.
254 00:22:57.230 ⇒ 00:23:03.280 Mitchell Wright: Okay. So for some reason, it keeps trying to log me into my personal account. One.
255 00:23:03.280 ⇒ 00:23:05.310 Luke Daque: Maybe it’s it’s cached or something.
256 00:23:05.310 ⇒ 00:23:06.440 Uttam Kumaran: Yeah.
257 00:23:06.440 ⇒ 00:23:06.960 Luke Daque: Yeah.
258 00:23:09.610 ⇒ 00:23:12.440 Uttam Kumaran: You could inspect them and sort of delete everything I don’t know.
259 00:23:12.760 ⇒ 00:23:13.989 Uttam Kumaran: Do what I try to do.
260 00:23:14.730 ⇒ 00:23:15.320 Mitchell Wright: I.
261 00:23:15.320 ⇒ 00:23:17.230 Uttam Kumaran: Or or incognito.
262 00:23:17.916 ⇒ 00:23:20.000 Mitchell Wright: Yeah, we’ll just do that.
263 00:23:23.530 ⇒ 00:23:28.239 Mitchell Wright: But I can’t even remember the last time I logged into segment under that account, anyway. So that’s bizarre.
264 00:23:30.562 ⇒ 00:23:37.799 Mitchell Wright: Let’s see log in mitchell@stackblitz.com
265 00:23:44.110 ⇒ 00:23:50.480 Mitchell Wright: password Leo. Oh, see, that’s
266 00:23:51.320 ⇒ 00:23:52.959 Mitchell Wright: oh, I want segment! That’s right.
267 00:23:53.510 ⇒ 00:24:01.540 Mitchell Wright: Oh, but what the oh, that’s why my stack Blitz got logged out, I guess.
268 00:24:03.260 ⇒ 00:24:05.310 Mitchell Wright: Oh, this is freaking, annoying.
269 00:24:05.460 ⇒ 00:24:06.440 Mitchell Wright: Okay.
270 00:24:17.100 ⇒ 00:24:23.199 Mitchell Wright: okay. I think we might be able to do it here now.
271 00:24:39.760 ⇒ 00:24:41.549 Mitchell Wright: No, okay, we’ll do it over here.
272 00:24:52.280 ⇒ 00:24:54.410 Mitchell Wright: now got code
273 00:25:01.750 ⇒ 00:25:09.170 Mitchell Wright: back. Blitz. Okay, let me reshare my screen and share this window.
274 00:25:13.290 ⇒ 00:25:25.768 Mitchell Wright: Okay, so, okay, so what were the steps I need to do now?
275 00:25:26.060 ⇒ 00:25:28.150 Uttam Kumaran: I think you go to destination.
276 00:25:32.690 ⇒ 00:25:34.199 Mitchell Wright: I don’t have access to add.
277 00:25:34.200 ⇒ 00:25:39.900 Uttam Kumaran: Maybe go to the left. Maybe it’s maybe this ui is like another connection
278 00:25:41.020 ⇒ 00:25:41.880 Luke Daque: Oh, yeah.
279 00:25:43.753 ⇒ 00:25:50.410 Mitchell Wright: Okay, so this is, this is good. This is good.
280 00:25:50.410 ⇒ 00:25:51.834 Uttam Kumaran: We got close.
281 00:25:52.520 ⇒ 00:25:53.060 Mitchell Wright: But.
282 00:25:53.060 ⇒ 00:25:54.590 Uttam Kumaran: That’s on the event side.
283 00:25:55.860 ⇒ 00:26:00.679 Mitchell Wright: Let me just send a message to the CTO. Real quick. Can I get access
284 00:26:00.890 ⇒ 00:26:06.609 Mitchell Wright: segment to add a new destination? I need to add
285 00:26:06.720 ⇒ 00:26:18.520 Mitchell Wright: Snowflake. So we can get our events there. Okay, cool, cool, cool, cool, cool.
286 00:26:20.300 ⇒ 00:26:26.919 Mitchell Wright: So once he gives me that I can set that up. But so connecting segment is one piece.
287 00:26:28.080 ⇒ 00:26:30.210 Mitchell Wright: What else did you all need from me?
288 00:26:31.318 ⇒ 00:26:37.750 Uttam Kumaran: I want to maybe grab time early next week to just review stripe metrics like in detail. I think.
289 00:26:37.750 ⇒ 00:26:38.180 Mitchell Wright: Like, maybe yeah.
290 00:26:38.180 ⇒ 00:26:41.349 Uttam Kumaran: Doing it, doing it back and forth, I think, is gonna be.
291 00:26:41.760 ⇒ 00:26:42.320 Mitchell Wright: Yeah.
292 00:26:42.320 ⇒ 00:26:44.120 Uttam Kumaran: And I think we’ll I think we’ll lose some signal.
293 00:26:44.120 ⇒ 00:26:49.819 Mitchell Wright: To give you an idea of what we’re kind of.
294 00:26:50.260 ⇒ 00:26:54.869 Uttam Kumaran: Or yeah, if you even have like a a screenshot version of what you’re even looking at today, then.
295 00:26:54.870 ⇒ 00:27:02.909 Mitchell Wright: So right now. This, I would say, is probably like the main thing that our CEO logs into, and looks at every day.
296 00:27:02.910 ⇒ 00:27:03.520 Uttam Kumaran: Okay.
297 00:27:05.150 ⇒ 00:27:14.900 Mitchell Wright: So the things the things that I I kind of look at are the
298 00:27:18.510 ⇒ 00:27:21.220 Mitchell Wright: new customers per day.
299 00:27:23.203 ⇒ 00:27:30.279 Mitchell Wright: you know. That’s that’s 1 that I care about active customers, and that’s like just a total
300 00:27:30.450 ⇒ 00:27:41.470 Mitchell Wright: active customers. And then, looking at what it was, day over day, churned customers per day.
301 00:27:41.970 ⇒ 00:27:42.600 Uttam Kumaran: Yes.
302 00:27:42.600 ⇒ 00:27:49.930 Mitchell Wright: Also another metric that I really care about. Then the other one is this other revenue
303 00:27:50.530 ⇒ 00:27:54.050 Mitchell Wright: which I’m not sure if this is.
304 00:27:54.580 ⇒ 00:28:01.270 Mitchell Wright: I use this basically as a way to look at number, the amount of token reloads we’re doing on any given day.
305 00:28:01.657 ⇒ 00:28:07.770 Mitchell Wright: And I’m not a hundred percent. I’m pretty sure that’s all that’s falling in this other revenue. But it would be nice.
306 00:28:07.770 ⇒ 00:28:08.790 Uttam Kumaran: See the products, yeah.
307 00:28:08.790 ⇒ 00:28:13.263 Mitchell Wright: Yeah, exactly so. And just get the token reloads.
308 00:28:13.910 ⇒ 00:28:18.640 Mitchell Wright: cause. This is another one where we really care about this number.
309 00:28:19.040 ⇒ 00:28:23.539 Mitchell Wright: Yeah, just as far as like value from the product. And and what people are.
310 00:28:24.560 ⇒ 00:28:27.379 Mitchell Wright: if they’re actually using all of their tokens. Kind of thing.
311 00:28:27.490 ⇒ 00:28:29.580 Mitchell Wright: Okay? And then
312 00:28:29.970 ⇒ 00:28:47.760 Mitchell Wright: I I’ll talk to Eric. I mean, yeah, like the Mrr number is, is important just to know where we’re at. And then that annualized is also great. So whatever the arr is also a number of the subscriptions, plus
313 00:28:47.910 ⇒ 00:28:51.380 Mitchell Wright: the token reloads annualized.
314 00:28:51.830 ⇒ 00:29:01.584 Mitchell Wright: that is, also one that would be nice to look at. So anyway, we can. We can get some separate time to go over all this if we want.
315 00:29:01.910 ⇒ 00:29:14.790 Uttam Kumaran: The 1st thing is also we’ll we’ll you’ll now be able to. Once. I just want to make sure the 1st thing will basically be, we validate the numbers, but in parallel, as soon as we do that to join on customer Id to everything else is available, so.
316 00:29:14.790 ⇒ 00:29:15.380 Mitchell Wright: Yeah, click.
317 00:29:15.380 ⇒ 00:29:17.610 Uttam Kumaran: Be able to segment by product.
318 00:29:17.770 ⇒ 00:29:23.776 Mitchell Wright: Yes, exactly. Cause. That’s 1 thing right now, that is like, really, actually not super easy is
319 00:29:25.930 ⇒ 00:29:32.230 Mitchell Wright: being able to to look at. For example, like token reloads and then
320 00:29:32.720 ⇒ 00:29:43.380 Mitchell Wright: splitting this out by the plan that the people are subscribed to like. There’s there’s actually not really any way for me to do that which is super annoying, because I’d like to see like.
321 00:29:44.040 ⇒ 00:29:59.350 Mitchell Wright: are people just subscribing to the lower tier plan and doing token reloads? Is it actually like our highest tier plan that they’re using their, you know, 100 million tokens, and then still adding token reloads? So we need to put them on an enterprise deal or whatever. So
322 00:29:59.450 ⇒ 00:30:12.810 Mitchell Wright: that’s the kind of stuff. Yeah, that’s gonna be awesome. And I think something like real will be fantastic for us to really just like I. I think the the main one is just gonna take any of these metrics and cut them by plan that people are subscribed to.
323 00:30:13.190 ⇒ 00:30:13.780 Uttam Kumaran: Right.
324 00:30:14.150 ⇒ 00:30:14.490 Mitchell Wright: That’ll be.
325 00:30:14.490 ⇒ 00:30:20.370 Uttam Kumaran: Yeah, I think once we give you that, you’ll probably then end up being like, Okay, cool, we need to build some more segments or stuff like that.
326 00:30:20.840 ⇒ 00:30:24.479 Uttam Kumaran: But I think we can drive. We’ll just drive towards that. So I think,
327 00:30:25.880 ⇒ 00:30:30.559 Uttam Kumaran: look, we have the we have the subscriptions and stuff coming in from.
328 00:30:30.840 ⇒ 00:30:31.980 Luke Daque: From stripe, yeah.
329 00:30:33.115 ⇒ 00:30:33.460 Uttam Kumaran: Cool
330 00:30:34.368 ⇒ 00:30:37.769 Uttam Kumaran: just trying to think if there’s anything else we need on the on the postgres side to make.
331 00:30:38.100 ⇒ 00:30:45.070 Mitchell Wright: Oh, yeah, one thing that’s a little bit wonky with the subscriptions. And with that subscription data is.
332 00:30:45.250 ⇒ 00:30:49.460 Mitchell Wright: let me find it. There was they don’t have.
333 00:30:50.760 ⇒ 00:30:52.820 Mitchell Wright: Do I not have that open anymore?
334 00:30:52.990 ⇒ 00:31:02.910 Mitchell Wright: I have to find the spreadsheet. But basically they just have, like a like a number like 1, 2, 3, 4, I think, in one of the databases. It might. That might be in the bolt database.
335 00:31:03.390 ⇒ 00:31:13.859 Mitchell Wright: I can’t remember now, but as far as like what subscription they’re on. And I think the stripe subscription names are kind of weird. So there’s like some mapping that we might have to just throw a
336 00:31:13.860 ⇒ 00:31:18.469 Mitchell Wright: yeah for for that. I don’t know if you ran into that, Luke, but.
337 00:31:18.470 ⇒ 00:31:21.586 Luke Daque: Yeah, I did like like the subscription meetings were like.
338 00:31:22.510 ⇒ 00:31:26.399 Luke Daque: not the same as like what you had in the spreadsheet. I believe.
339 00:31:26.560 ⇒ 00:31:35.420 Mitchell Wright: Yeah, yeah, yeah. So I’ll I’ll find that spreadsheet and give you that information. So
340 00:31:36.780 ⇒ 00:31:42.809 Luke Daque: Like the the Astronaut Monthly, 9, astronaut, yearly 99. So I don’t really know which one is like.
341 00:31:43.040 ⇒ 00:31:47.769 Mitchell Wright: Yeah, yeah. And they don’t. They don’t have it in a database anywhere. It’s pretty fun.
342 00:31:47.770 ⇒ 00:31:48.390 Luke Daque: Yeah.
343 00:31:49.080 ⇒ 00:31:54.119 Mitchell Wright: Anyway, I have to hop on another call, but I’ll make a note.
344 00:31:54.820 ⇒ 00:32:00.189 Uttam Kumaran: Yeah. And you can even throw it into bolt metrics, or we can create it. If you want us to create that mapping, we can. But.
345 00:32:00.470 ⇒ 00:32:00.870 Luke Daque: Yeah.
346 00:32:01.730 ⇒ 00:32:02.360 Mitchell Wright: Yeah.
347 00:32:03.192 ⇒ 00:32:28.469 Mitchell Wright: Okay, cool. Yeah. Yeah. I’ll I’ll get that. And I’ll talk to them about what they want to do there. But I think for now, we might just have to throw it in as a seed and throw it in and do that there. Okay, cool. Thanks so much, guys I’ll get segment connected and let you guys know. And then if you want to just grab some time for us to go over dashboard stuff if you feel like you need that, or if I give you enough like whatever, let me know I’m I’m flexible.
348 00:32:28.470 ⇒ 00:32:33.749 Uttam Kumaran: I might put it on for Tuesday, and then if we get to Tuesday and we’re like pretty good, then we’ll we’ll mix it. Okay.
349 00:32:33.750 ⇒ 00:32:34.230 Mitchell Wright: It’s good.
350 00:32:34.230 ⇒ 00:32:34.730 Uttam Kumaran: Sounds good.
351 00:32:34.960 ⇒ 00:32:36.520 Mitchell Wright: Thanks. Guys, have a good day.
352 00:32:36.960 ⇒ 00:32:38.240 Luke Daque: At the value of the lab.