Meeting Title: DBT-Structure-Documentation Date: 2024-08-30 Meeting participants: Patrick Trainer, Uttam Kumaran


WEBVTT

1 00:00:07.870 00:00:08.660 Uttam Kumaran: Okay.

2 00:00:08.660 00:00:09.600 Patrick Trainer: No.

3 00:00:09.910 00:00:11.360 Uttam Kumaran: Yo.

4 00:00:11.980 00:00:13.749 Patrick Trainer: So I downloaded the

5 00:00:14.510 00:00:16.560 Patrick Trainer: like the new beta

6 00:00:16.780 00:00:21.097 Patrick Trainer: like for the ipad OS, and I’ve got it for like

7 00:00:21.630 00:00:22.489 Patrick Trainer: my phone, too.

8 00:00:22.490 00:00:25.969 Uttam Kumaran: I have the Beta 2 on my phone, too. Yeah, but I didn’t get any of the AI stuff.

9 00:00:26.640 00:00:27.730 Patrick Trainer: Do you have

10 00:00:28.000 00:00:30.739 Patrick Trainer: in the iphone 15.

11 00:00:31.000 00:00:32.200 Uttam Kumaran: I have a 14.

12 00:00:32.390 00:00:35.520 Patrick Trainer: I think it’s like the 15, or you need to

13 00:00:35.610 00:00:39.919 Patrick Trainer: the AI stuff. You need the 18.1, Beta.

14 00:00:40.470 00:00:42.670 Uttam Kumaran: Oh, okay, but.

15 00:00:42.670 00:00:53.869 Patrick Trainer: Like I’ve got the 18.1 beta on my ipad. I’ve got a 12 iphone. I’ve had this for a while, but I’ve got it the 18 one beta on my ipad.

16 00:00:54.190 00:00:59.030 Patrick Trainer: and one like super sick thing. So it’s like I’m in notes right.

17 00:00:59.030 00:01:00.530 Uttam Kumaran: Yeah, I mean.

18 00:01:00.530 00:01:02.010 Patrick Trainer: Here, I’m going to start a new note.

19 00:01:02.160 00:01:03.910 Patrick Trainer: You can do like.

20 00:01:06.530 00:01:07.570 Patrick Trainer: Let’s go

21 00:01:11.650 00:01:13.109 Patrick Trainer: So we’ve got like

22 00:01:15.080 00:01:16.040 Patrick Trainer: 10

23 00:01:16.210 00:01:17.130 Patrick Trainer: plus

24 00:01:17.250 00:01:18.210 Patrick Trainer: 10.

25 00:01:18.210 00:01:20.770 Uttam Kumaran: Oh, I know. I know what the demo is. Yeah.

26 00:01:20.770 00:01:22.789 Patrick Trainer: Yeah, and and like, it’ll

27 00:01:23.970 00:01:25.749 Patrick Trainer: I need to delete that.

28 00:01:26.650 00:01:27.150 Uttam Kumaran: Sick.

29 00:01:27.150 00:01:27.900 Patrick Trainer: But it’ll

30 00:01:28.660 00:01:31.929 Patrick Trainer: yeah, it’ll solve it. And then you can also like

31 00:01:32.540 00:01:36.480 Patrick Trainer: change stuff, like at the same. Can you? Can you? Are you able to see this.

32 00:01:36.805 00:01:41.030 Uttam Kumaran: It’s kind of like, it’s a little bit bright. But yeah, yeah, yeah.

33 00:01:41.690 00:01:42.410 Patrick Trainer: Yeah.

34 00:01:44.750 00:01:45.480 Patrick Trainer: Down. Yeah.

35 00:01:45.480 00:01:48.319 Uttam Kumaran: Yeah. Oh, and it did it in your handwriting.

36 00:01:48.520 00:01:49.920 Patrick Trainer: Yeah, yeah, yeah.

37 00:01:49.920 00:01:50.779 Uttam Kumaran: That’s kind of.

38 00:01:50.780 00:01:51.560 Patrick Trainer: Like

39 00:01:51.670 00:01:52.670 Patrick Trainer: it’s

40 00:01:53.290 00:01:56.240 Patrick Trainer: even really good, like you can do like

41 00:01:57.100 00:01:59.489 Patrick Trainer: square root of like

42 00:01:59.980 00:02:01.730 Patrick Trainer: 12 over

43 00:02:02.280 00:02:04.280 Patrick Trainer: 3 raised

44 00:02:04.500 00:02:06.059 Patrick Trainer: to the 3, rd

45 00:02:06.470 00:02:07.520 Patrick Trainer: minus

46 00:02:07.590 00:02:09.600 Patrick Trainer: 5 equals.

47 00:02:17.600 00:02:18.370 Uttam Kumaran: Yeah.

48 00:02:18.370 00:02:20.600 Patrick Trainer: Like it’s, it’s really good.

49 00:02:21.270 00:02:29.890 Uttam Kumaran: I wonder if it’s like, I wonder if at some point you can be able to do all those diagrams in a meeting? Send it to some, send it to AI to like brand it basically, or like.

50 00:02:29.890 00:02:30.500 Patrick Trainer: Right.

51 00:02:30.500 00:02:30.919 Uttam Kumaran: Bit better.

52 00:02:30.920 00:02:31.720 Patrick Trainer: Right.

53 00:02:32.040 00:02:37.599 Patrick Trainer: And then I’ve also like you can also do like a equals. 4

54 00:02:37.970 00:02:40.709 Patrick Trainer: B equals 6,

55 00:02:40.950 00:02:44.470 Patrick Trainer: and then you can do like a plus B

56 00:02:44.740 00:02:45.490 Patrick Trainer: pools.

57 00:02:45.490 00:02:47.430 Uttam Kumaran: Some sort of variable storage.

58 00:02:53.200 00:02:53.950 Patrick Trainer: Yeah.

59 00:02:54.700 00:02:56.610 Uttam Kumaran: Wow! That’s crazy.

60 00:02:56.610 00:03:01.930 Patrick Trainer: And so like, I’ve been playing with it, trying to see like the boundaries of it. And basically, like.

61 00:03:02.070 00:03:04.930 Patrick Trainer: I’ve been trying to write code like

62 00:03:04.940 00:03:07.080 Patrick Trainer: in like a weird kind of like

63 00:03:08.030 00:03:12.440 Patrick Trainer: just variable expansion way of like like through

64 00:03:12.540 00:03:14.750 Patrick Trainer: words. It’s

65 00:03:14.970 00:03:21.020 Patrick Trainer: it, mess. It messes up a little bit. It’s not quite there, but like it’s really close. And there’s like.

66 00:03:21.020 00:03:21.830 Uttam Kumaran: Wow!

67 00:03:22.230 00:03:29.820 Patrick Trainer: And it’s it’s it’s crazy, too, like. So if you if you’re here, you can like tap on the 6 right.

68 00:03:30.130 00:03:32.390 Patrick Trainer: and then you can like move it.

69 00:03:33.420 00:03:34.740 Uttam Kumaran: No way.

70 00:03:35.030 00:03:37.620 Patrick Trainer: Yeah. And it all updates, like, in real time.

71 00:03:38.060 00:03:38.949 Uttam Kumaran: What the fuck.

72 00:03:38.950 00:03:41.810 Patrick Trainer: It’s it’s fucking wild.

73 00:03:42.960 00:03:44.979 Patrick Trainer: It’s absolutely wild.

74 00:03:46.430 00:03:47.120 Uttam Kumaran: Damn.

75 00:03:48.390 00:03:55.030 Uttam Kumaran: yeah, I I wish I got some of the AI stuff on my phone. But I think I have a 14. So it’s not gonna give me some of them, but.

76 00:03:55.030 00:03:55.800 Patrick Trainer: Yeah.

77 00:03:55.970 00:04:03.689 Patrick Trainer: yeah, I mean the coolest thing on the phone. I think, like, we’ll see, you can solve equations like, if you type out like

78 00:04:03.820 00:04:07.630 Patrick Trainer: 10 plus 10 equals in in notes, or whatever

79 00:04:07.720 00:04:10.539 Patrick Trainer: it’ll, it’ll solve the equation for you.

80 00:04:11.257 00:04:12.509 Patrick Trainer: Which is cool.

81 00:04:12.540 00:04:17.236 Patrick Trainer: But the I think the killer feature with with the on on the phone is

82 00:04:18.760 00:04:21.240 Patrick Trainer: scheduling like text messages

83 00:04:21.500 00:04:22.190 Patrick Trainer: and stuff.

84 00:04:22.190 00:04:23.610 Uttam Kumaran: Yeah, yeah, yeah, yeah.

85 00:04:23.610 00:04:24.729 Patrick Trainer: I really like that.

86 00:04:25.120 00:04:27.622 Uttam Kumaran: I tried. I did one on my girlfriend the other day.

87 00:04:29.360 00:04:30.030 Patrick Trainer: Nice.

88 00:04:30.030 00:04:31.320 Uttam Kumaran: Just to see if it worked.

89 00:04:32.010 00:04:32.810 Patrick Trainer: Nice.

90 00:04:34.130 00:04:37.100 Patrick Trainer: Yeah, if and then like.

91 00:04:37.350 00:04:41.229 Patrick Trainer: I mean, they’ve always had the shortcuts and stuff. But

92 00:04:41.500 00:04:45.189 Patrick Trainer: like those capabilities are like super powerful.

93 00:04:48.140 00:04:59.369 Uttam Kumaran: Yeah, I try. I added a couple of new shortcuts for talking to to github. Actually, I mean not talking to get up talking to a, to a coke, to a

94 00:04:59.710 00:05:01.330 Uttam Kumaran: open AI like.

95 00:05:01.330 00:05:01.830 Patrick Trainer: Nice.

96 00:05:01.830 00:05:04.859 Uttam Kumaran: From your homepage. You’ll just immediately open like the

97 00:05:05.070 00:05:06.819 Uttam Kumaran: the discussion thing.

98 00:05:06.820 00:05:07.690 Patrick Trainer: Oh, yeah.

99 00:05:07.690 00:05:08.540 Uttam Kumaran: Yeah.

100 00:05:08.760 00:05:11.499 Patrick Trainer: I’ve I’ve made one that like

101 00:05:12.450 00:05:16.620 Patrick Trainer: it allows me like it’ll take notes that I’ve written.

102 00:05:16.810 00:05:19.590 Patrick Trainer: and it’ll commit the notes to Github.

103 00:05:19.960 00:05:20.890 Uttam Kumaran: Hmm.

104 00:05:20.890 00:05:22.610 Patrick Trainer: Because you can like, you can.

105 00:05:22.610 00:05:23.490 Uttam Kumaran: Yeah, yeah.

106 00:05:23.490 00:05:28.050 Patrick Trainer: Yeah, they’ve got like the like. You can send like post and get requests.

107 00:05:28.510 00:05:29.529 Uttam Kumaran: Oh, no way.

108 00:05:29.530 00:05:33.600 Patrick Trainer: And so like you can literally like, do

109 00:05:34.060 00:05:38.020 Patrick Trainer: I mean, it’s its own little kind of it’s its own programming language. And.

110 00:05:38.020 00:05:39.164 Uttam Kumaran: Yeah, just kind of

111 00:05:39.450 00:05:41.829 Patrick Trainer: Drag and drop everything. It’s hard to

112 00:05:42.220 00:05:52.240 Patrick Trainer: hard to kind of mess about or get used to it, but like I’ve seen I’ve seen somebody do like advent of code in apple shortcuts.

113 00:05:53.320 00:05:54.750 Patrick Trainer: absolutely wild.

114 00:05:54.750 00:05:55.305 Uttam Kumaran: Wow.

115 00:05:56.710 00:05:57.836 Uttam Kumaran: yeah, I,

116 00:05:59.170 00:06:04.260 Uttam Kumaran: yeah, I I, I, I subscribe to this shortcut subreddit. So I kind of like, agree.

117 00:06:04.260 00:06:05.270 Patrick Trainer: Oh, really.

118 00:06:05.270 00:06:07.480 Uttam Kumaran: Yeah, there’s a lot on there. Actually.

119 00:06:09.920 00:06:15.420 Uttam Kumaran: they should go on there and check is actually you can, you can get links to shortcuts and like download them. So people share.

120 00:06:15.870 00:06:16.450 Patrick Trainer: Yeah.

121 00:06:16.450 00:06:17.919 Uttam Kumaran: Up there all the time.

122 00:06:17.920 00:06:18.570 Patrick Trainer: Yeah.

123 00:06:23.720 00:06:25.069 Uttam Kumaran: Okay, cool.

124 00:06:25.791 00:06:28.770 Uttam Kumaran: Yeah. I guess today I wanted to, just mainly.

125 00:06:28.790 00:06:35.468 Uttam Kumaran: Nick said he felt sick today. So he’s not gonna join. But I wanted to just walk through

126 00:06:36.430 00:06:40.509 Uttam Kumaran: like how we want to structure Dbt projects. I think we can have a conversation.

127 00:06:40.780 00:06:42.520 Uttam Kumaran: Write some stuff down

128 00:06:42.540 00:06:44.859 Uttam Kumaran: and then toss it to him to

129 00:06:45.490 00:06:47.606 Uttam Kumaran: like, draw some ideas down.

130 00:06:48.030 00:06:48.730 Patrick Trainer: Right.

131 00:06:49.530 00:06:51.930 Uttam Kumaran: I, there is a Doc.

132 00:06:55.700 00:06:58.160 Uttam Kumaran: There is a dock here on Dbt, so.

133 00:06:58.160 00:07:02.529 Patrick Trainer: I mean, the biggest change that I would say that like needs to be made is like

134 00:07:02.720 00:07:05.410 Patrick Trainer: we need to get away from the structure of like

135 00:07:06.200 00:07:09.590 Patrick Trainer: source, intermediate source, intermediate, like

136 00:07:09.958 00:07:11.750 Patrick Trainer: in the sense that, like

137 00:07:14.570 00:07:20.769 Patrick Trainer: the shopify here and then intermediate shopify marks, shopify.

138 00:07:21.070 00:07:22.700 Uttam Kumaran: Yeah, I mean, we’ve got you. I think.

139 00:07:22.700 00:07:27.540 Patrick Trainer: I like going all the way across the board, and it’s like we’re not actually making

140 00:07:27.740 00:07:28.900 Patrick Trainer: models

141 00:07:29.260 00:07:31.379 Patrick Trainer: like modular at all.

142 00:07:31.880 00:07:38.182 Uttam Kumaran: Yeah, let’s let’s like, I wanna just go through and list like the current challenges.

143 00:07:41.520 00:07:44.860 Uttam Kumaran: let’s just call this Doc critique.

144 00:07:45.620 00:07:48.170 Uttam Kumaran: I like having change logs for these docs.

145 00:07:48.890 00:07:49.530 Patrick Trainer: Yeah.

146 00:07:51.430 00:07:55.719 Uttam Kumaran: But okay, so I want to talk about a couple of challenges kind of, I think

147 00:07:55.990 00:07:57.949 Uttam Kumaran: I think it’s gonna be helpful

148 00:07:58.130 00:08:01.609 Uttam Kumaran: for this exercise. If you take the engineer side.

149 00:08:01.650 00:08:09.159 Uttam Kumaran: and I’m going to take the client side, because I do think that there’s probably some competing

150 00:08:09.310 00:08:10.240 Uttam Kumaran: like

151 00:08:10.923 00:08:22.129 Uttam Kumaran: needs there. And I think for the most. For the most part, I think I’ll probably agree with you on the engineering side. But the this client side thing is something that I’ve been thinking about as we have clients where I actually want this artifact

152 00:08:22.230 00:08:28.500 Uttam Kumaran: to be something that we hand over. So I’ll kind of like. Just list out some

153 00:08:28.940 00:08:32.649 Uttam Kumaran: client challenges with the current structure. One.

154 00:08:32.789 00:08:43.090 Uttam Kumaran: It’s not easy, and I’ll we’ll make them vague, for now and then I’ll kind of boil it down so it’s not easy to find documentation

155 00:08:43.270 00:08:44.220 Uttam Kumaran: on.

156 00:08:45.460 00:08:50.409 Uttam Kumaran: not easy to find sequel definition for a table

157 00:08:54.660 00:08:57.232 Uttam Kumaran: like, let’s just leave it there. Right?

158 00:08:58.510 00:09:01.709 Uttam Kumaran: given snowflake table location. Right?

159 00:09:02.120 00:09:04.359 Uttam Kumaran: The second thing is.

160 00:09:08.560 00:09:16.449 Uttam Kumaran: the second thing is within a SQL. File. It’s not easy to understand

161 00:09:18.690 00:09:19.790 Uttam Kumaran: how

162 00:09:20.370 00:09:22.380 Uttam Kumaran: column is

163 00:09:24.230 00:09:28.739 Uttam Kumaran: or the sources, and that may that may get solved here and may not be.

164 00:09:28.820 00:09:31.155 Uttam Kumaran: The 3rd thing is

165 00:09:38.650 00:09:40.760 Uttam Kumaran: When I change

166 00:09:40.900 00:09:45.420 Uttam Kumaran: is necessary to a file, it’s

167 00:09:45.770 00:09:49.610 Uttam Kumaran: not clear how to articulate

168 00:09:49.890 00:09:52.190 Uttam Kumaran: what the problem is.

169 00:09:52.910 00:09:55.309 Uttam Kumaran: what is required.

170 00:09:55.950 00:10:03.840 Uttam Kumaran: The brain forged team. Right again. This is maybe a Pm issue. This is this is more like, how do they speak our language?

171 00:10:04.010 00:10:08.979 Uttam Kumaran: And how do we kind of remove any translation issues between the client and the engineer?

172 00:10:09.160 00:10:09.510 Patrick Trainer: Right.

173 00:10:11.750 00:10:13.687 Uttam Kumaran: Okay? So then let’s talk about

174 00:10:15.050 00:10:18.030 Uttam Kumaran: like engineering engineer challenges right now.

175 00:10:20.146 00:10:21.600 Patrick Trainer: I’d say

176 00:10:23.420 00:10:25.999 Patrick Trainer: so. Things aren’t modular.

177 00:10:26.150 00:10:28.629 Patrick Trainer: so there’s a lot of like code reuse

178 00:10:31.010 00:10:32.880 Patrick Trainer: as well as

179 00:10:35.280 00:10:41.080 Patrick Trainer: similar fields, or like the same underlying column will be named differently

180 00:10:41.480 00:10:44.410 Patrick Trainer: in different models, like even in downstream models.

181 00:10:44.720 00:10:45.970 Patrick Trainer: And so

182 00:10:46.390 00:10:51.199 Patrick Trainer: essentially like, we’re, we’re providing aliases on top of aliases.

183 00:10:51.200 00:10:51.640 Uttam Kumaran: Yeah.

184 00:10:51.640 00:10:53.530 Patrick Trainer: And it just gets confusing.

185 00:10:53.610 00:10:54.625 Patrick Trainer: So

186 00:10:56.120 00:11:01.020 Patrick Trainer: like, we should only rename columns once, and that should be at the source.

187 00:11:01.100 00:11:03.729 Patrick Trainer: and then everything downstream like.

188 00:11:03.820 00:11:07.479 Patrick Trainer: unless you’re making a new column like

189 00:11:07.490 00:11:09.069 Patrick Trainer: it shouldn’t be

190 00:11:09.120 00:11:10.190 Patrick Trainer: aliased.

191 00:11:12.180 00:11:15.317 Patrick Trainer: and I think that goes the same for

192 00:11:18.560 00:11:23.389 Patrick Trainer: like tables and like joins, and even just like semantics in there, like.

193 00:11:23.870 00:11:25.930 Patrick Trainer: instead of like calling

194 00:11:26.750 00:11:29.309 Patrick Trainer: like the customer table. C,

195 00:11:30.310 00:11:34.730 Patrick Trainer: just call just type out customers. It’s easy. It doesn’t take that long.

196 00:11:34.730 00:11:35.510 Uttam Kumaran: Yeah.

197 00:11:36.870 00:11:38.279 Patrick Trainer: Like. Words are cheap.

198 00:11:42.510 00:11:43.780 Patrick Trainer: things

199 00:11:44.370 00:11:46.230 Patrick Trainer: like our models

200 00:11:46.840 00:11:51.119 Patrick Trainer: aren’t like they’re not modeling anything, really. They’re just

201 00:11:51.590 00:11:55.700 Patrick Trainer: it’s we’re just writing sequel in them, and they’re just sequel.

202 00:11:57.050 00:12:00.280 Patrick Trainer: so it’s like we’re kind of like straying from

203 00:12:02.980 00:12:03.650 Patrick Trainer: like

204 00:12:04.010 00:12:07.579 Patrick Trainer: good code. Hygiene is kind of how I see it.

205 00:12:10.960 00:12:12.500 Patrick Trainer: and

206 00:12:22.230 00:12:32.110 Uttam Kumaran: I think even on the engineering side it’s really hard to know where things are like if I open like, if I open the Stella Repo. I’m not sure where to go to find stuff.

207 00:12:32.260 00:12:33.360 Uttam Kumaran: and

208 00:12:34.700 00:12:38.889 Uttam Kumaran: I can’t look at Snowflake, and that doesn’t help me at all

209 00:12:38.950 00:12:42.250 Uttam Kumaran: right, right. There has to be some link between

210 00:12:42.320 00:12:47.910 Uttam Kumaran: where the thing lands in Snowflake and how the Dbt structure is organized because you can’t interact.

211 00:12:48.040 00:12:54.179 Uttam Kumaran: You’re not interacting with the Dbt structure. You’re gonna interact with the table and find an issue. And then you’re going to go

212 00:12:54.300 00:12:56.490 Uttam Kumaran: into the repo to make a fix.

213 00:12:56.540 00:12:59.420 Uttam Kumaran: Right? So Snowflake

214 00:13:01.410 00:13:10.450 Uttam Kumaran: and repo need to to have some logical math pattern, basically, or something like that. Right?

215 00:13:11.918 00:13:18.699 Patrick Trainer: But yeah, it’s hard to understand where a change should be made or where you should go and look for it, like

216 00:13:19.390 00:13:20.919 Patrick Trainer: if you need to.

217 00:13:20.990 00:13:22.869 Patrick Trainer: If there’s like a new request.

218 00:13:23.240 00:13:29.020 Patrick Trainer: I feel like, now, really, what’s being done is we’re just like going to the final table

219 00:13:29.280 00:13:30.310 Patrick Trainer: and then

220 00:13:31.180 00:13:33.609 Patrick Trainer: adding on something there instead.

221 00:13:33.610 00:13:34.330 Uttam Kumaran: Yeah.

222 00:13:34.330 00:13:40.040 Patrick Trainer: Oh, just going down like down the stack, and adding the change there, and like.

223 00:13:40.040 00:13:40.510 Uttam Kumaran: Yeah.

224 00:13:40.510 00:13:41.600 Patrick Trainer: Precipitate on

225 00:13:43.200 00:13:45.659 Patrick Trainer: So there’s like a lot of ad hoc like

226 00:13:46.350 00:13:52.210 Patrick Trainer: we’ll start out with like a convention. And then but adhering to that convention, or

227 00:13:52.260 00:13:54.870 Patrick Trainer: like mindset of maintainability.

228 00:13:54.890 00:13:56.989 Patrick Trainer: just kind of like, gets forgotten about.

229 00:13:57.160 00:13:57.780 Uttam Kumaran: Yeah.

230 00:13:58.880 00:14:00.030 Uttam Kumaran: that makes sense.

231 00:14:02.480 00:14:05.614 Uttam Kumaran: I think let’s like, I wanna just take. Let’s take the

232 00:14:07.870 00:14:10.359 Uttam Kumaran: Let’s take this dollar repo, as like, I mean

233 00:14:10.390 00:14:11.500 Uttam Kumaran: example, here.

234 00:14:13.910 00:14:16.210 Uttam Kumaran: just gonna pull it up on my side.

235 00:14:28.640 00:14:29.970 Uttam Kumaran: So

236 00:14:35.610 00:14:37.810 Uttam Kumaran: within Dbt, project

237 00:14:38.720 00:14:41.200 Uttam Kumaran: again, we have like a bunch of project files.

238 00:14:41.740 00:14:44.280 Uttam Kumaran: These aren’t getting leveraged.

239 00:14:44.560 00:14:45.240 Patrick Trainer: Right.

240 00:14:45.900 00:14:47.429 Uttam Kumaran: So one is like.

241 00:14:48.210 00:14:50.500 Uttam Kumaran: I mean, I’m just gonna take notes on the side.

242 00:15:00.220 00:15:06.909 Uttam Kumaran: So snapshots, seeds, macros, analyses are all unused.

243 00:15:08.590 00:15:10.217 Patrick Trainer: We’re also like

244 00:15:11.100 00:15:13.439 Patrick Trainer: at the like, the file level.

245 00:15:13.940 00:15:19.110 Patrick Trainer: defining like writing config there, instead of writing the config at the higher level.

246 00:15:19.410 00:15:21.819 Patrick Trainer: like in the Dbt. Project, or

247 00:15:22.810 00:15:23.390 Patrick Trainer: in.

248 00:15:23.390 00:15:25.150 Uttam Kumaran: In what situation.

249 00:15:25.150 00:15:26.819 Patrick Trainer: Like, if you go into the models

250 00:15:28.260 00:15:32.959 Patrick Trainer: and then go into, yeah, I guess, like, just choose any random

251 00:15:34.420 00:15:36.350 Patrick Trainer: file, something like that.

252 00:15:38.030 00:15:41.200 Patrick Trainer: Yeah. So we got like a config up there like.

253 00:15:41.220 00:15:43.350 Patrick Trainer: why are we defining the config

254 00:15:43.670 00:15:46.260 Patrick Trainer: for that as a table like right there.

255 00:15:46.560 00:15:51.849 Uttam Kumaran: So you’re saying there should be a default config, and then, if you need to override it, you do here.

256 00:15:51.850 00:15:55.360 Patrick Trainer: Right? Yeah, yeah, like, that’s that’s how it’s supposed to.

257 00:15:55.360 00:15:56.110 Uttam Kumaran: Yeah, yeah.

258 00:15:56.110 00:16:00.665 Patrick Trainer: Like you have that like higher level abstraction, and then

259 00:16:01.340 00:16:02.900 Patrick Trainer: Going down there.

260 00:16:07.310 00:16:10.030 Uttam Kumaran: Okay? I mean, yeah, that makes sense. That’s all good.

261 00:16:10.501 00:16:14.269 Patrick Trainer: And like, there’s examples of that like everywhere.

262 00:16:14.270 00:16:15.040 Uttam Kumaran: Yeah.

263 00:16:15.040 00:16:18.159 Patrick Trainer: And so it it just be. It just becomes hard to

264 00:16:18.490 00:16:19.980 Patrick Trainer: kind of like grok about.

265 00:16:20.280 00:16:24.019 Patrick Trainer: And then, if we like, also look at, look at these 2 like

266 00:16:24.670 00:16:26.140 Patrick Trainer: we have these

267 00:16:26.550 00:16:27.260 Patrick Trainer: like

268 00:16:27.510 00:16:30.569 Patrick Trainer: users or company location like

269 00:16:30.660 00:16:33.079 Patrick Trainer: that. Those could be their own

270 00:16:34.570 00:16:36.350 Patrick Trainer: models themselves.

271 00:16:36.540 00:16:37.809 Patrick Trainer: Stuff like that.

272 00:16:40.350 00:16:42.520 Patrick Trainer: know, if that’s like the best example. But

273 00:16:46.510 00:16:48.440 Uttam Kumaran: I think out the get go

274 00:16:50.840 00:16:51.780 Uttam Kumaran: like

275 00:16:53.420 00:16:56.760 Uttam Kumaran: there’s source level folders.

276 00:16:56.890 00:16:58.739 Uttam Kumaran: and then there’s parts.

277 00:17:01.190 00:17:04.160 Uttam Kumaran: and then within each

278 00:17:05.040 00:17:06.569 Uttam Kumaran: there’s another layer.

279 00:17:06.579 00:17:08.299 Patrick Trainer: Right, right.

280 00:17:08.599 00:17:10.939 Uttam Kumaran: And same within models. Here

281 00:17:11.349 00:17:14.889 Uttam Kumaran: there’s another layer, so it’s it’s at a source level.

282 00:17:14.989 00:17:18.369 Uttam Kumaran: But the one thing I don’t understand I don’t understand.

283 00:17:18.439 00:17:21.339 Uttam Kumaran: And this little thing we’re having back and forth on is

284 00:17:21.369 00:17:25.089 Uttam Kumaran: when you, when you have tables that are

285 00:17:25.539 00:17:34.209 Uttam Kumaran: combining sources or combining downstream tables, where does that end up exactly right. That’s why I I would rather have

286 00:17:35.049 00:17:37.129 Uttam Kumaran: like source cleanup.

287 00:17:37.399 00:17:45.669 Uttam Kumaran: Then I would rather have staging. And then artists right? And that’s very clear is that source, cleanup after source cleanup

288 00:17:45.999 00:17:46.969 Uttam Kumaran: like you never.

289 00:17:46.970 00:17:47.550 Patrick Trainer: The source.

290 00:17:47.550 00:17:49.330 Uttam Kumaran: You don’t touch the sources

291 00:17:49.380 00:17:56.320 Uttam Kumaran: right? And so it’s fair to like. Take what’s, for example, you take this hot spot thing, and then you take this dimensions like.

292 00:17:58.550 00:18:02.850 Patrick Trainer: Yeah, it’s like, why would we have a a dim Hubspot company

293 00:18:03.250 00:18:05.319 Patrick Trainer: when we’re combining that

294 00:18:05.500 00:18:07.230 Patrick Trainer: somewhere down the road with

295 00:18:07.660 00:18:10.120 Patrick Trainer: dim shopify company like.

296 00:18:10.120 00:18:12.980 Uttam Kumaran: Well, it’s like this should be a staging table.

297 00:18:13.200 00:18:18.779 Uttam Kumaran: right? And unless you need this to show up in March, it just gets used for modeling

298 00:18:18.810 00:18:21.899 Uttam Kumaran: right? So in this situation I would rather have

299 00:18:22.550 00:18:24.829 Uttam Kumaran: models I would rather have.

300 00:18:26.700 00:18:36.079 Uttam Kumaran: I mean, it’s like. And then again, like I want, I would. I would prefer it to match the schemas so that we can just use the folder names for the schema.

301 00:18:36.080 00:18:36.810 Patrick Trainer: Exactly.

302 00:18:36.810 00:18:45.720 Uttam Kumaran: Things, and then it it’s 1. It’s like a super great match, and the config works really well where it’s like you’re only constrained to 3 folders. Here you have

303 00:18:45.950 00:18:50.070 Uttam Kumaran: raw. What do we have raw staging in March, or like? I don’t know what.

304 00:18:50.070 00:18:51.390 Patrick Trainer: Raw, intermediate.

305 00:18:51.660 00:18:53.419 Patrick Trainer: final, that anything.

306 00:18:54.260 00:18:56.919 Uttam Kumaran: I like. I like marts.

307 00:18:57.500 00:19:01.270 Uttam Kumaran: I think staging indicates that there is a production.

308 00:19:01.850 00:19:05.110 Uttam Kumaran: So I’m not particularly like a fan of of that

309 00:19:06.121 00:19:09.738 Uttam Kumaran: because those are not staged. Technically, those are like

310 00:19:11.275 00:19:11.760 Uttam Kumaran: like.

311 00:19:11.760 00:19:13.710 Patrick Trainer: I’ve even. I’ve seen it done as like

312 00:19:13.730 00:19:15.770 Patrick Trainer: bronze, silver, gold.

313 00:19:15.770 00:19:19.388 Uttam Kumaran: Yeah, that one is excellent. That sounds just so, Corny.

314 00:19:19.750 00:19:26.830 Patrick Trainer: Yeah. But I mean, it’s it’s basically it’s it’s just whatever to signal the like where it is. But

315 00:19:26.880 00:19:35.889 Patrick Trainer: like, I really like, how like 5 grand structures, their projects. And what’s what’s nice about them, too, is like, they’re

316 00:19:37.430 00:19:39.870 Patrick Trainer: Yeah, there we go, like they’re

317 00:19:40.720 00:19:43.878 Patrick Trainer: structured the same for every single one

318 00:19:44.420 00:19:46.000 Patrick Trainer: for every single project.

319 00:19:48.430 00:19:52.420 Uttam Kumaran: So these guys have staging intermediate marts for us. It’s just gonna be

320 00:19:53.330 00:19:54.350 Uttam Kumaran: raw

321 00:19:55.030 00:20:00.649 Uttam Kumaran: intermediate marks. And this is where I’m going to take the business side of things, because raw means raw

322 00:20:00.950 00:20:02.849 Uttam Kumaran: intermediate means intermediate

323 00:20:03.050 00:20:04.140 Uttam Kumaran: marts.

324 00:20:04.220 00:20:06.039 Uttam Kumaran: I’m still a little bit like

325 00:20:06.460 00:20:09.450 Uttam Kumaran: I know. Only 1st time I heard this word is where it’s like

326 00:20:09.510 00:20:12.949 Uttam Kumaran: in our world. So I’m even like.

327 00:20:13.300 00:20:18.050 Uttam Kumaran: I don’t know whether this will stick longer term. But again, that’s like, that’s the fight we’re gonna have.

328 00:20:18.050 00:20:18.570 Patrick Trainer: Right.

329 00:20:18.570 00:20:23.800 Uttam Kumaran: Staging. I think raw intermediate marts is something that we can all agree on.

330 00:20:23.800 00:20:24.560 Patrick Trainer: Right.

331 00:20:27.590 00:20:29.230 Uttam Kumaran: 80th March.

332 00:20:36.930 00:20:41.089 Uttam Kumaran: and then even within here, look, they have models intermediate finance.

333 00:20:41.770 00:20:45.780 Uttam Kumaran: So they. So I wanna have an understanding of when we do

334 00:20:46.160 00:20:46.700 Uttam Kumaran: well.

335 00:20:46.700 00:20:55.489 Patrick Trainer: So. So if you read into like in their one bullet point, they’ve got like staging intermediate march there staging is like

336 00:20:56.420 00:20:59.079 Patrick Trainer: the smallest units.

337 00:20:59.100 00:21:03.759 Patrick Trainer: and then, like their intermediate, is like building, putting those units together.

338 00:21:07.180 00:21:09.999 Patrick Trainer: like I like. I think this system makes

339 00:21:10.300 00:21:17.039 Patrick Trainer: a lot of sense in the sense that like. So with intermediate, you’d have finance unit. The the unit of finance would be like

340 00:21:17.960 00:21:23.819 Patrick Trainer: related to to sales or related to like dollars or payments stuff like that.

341 00:21:24.250 00:21:26.239 Patrick Trainer: And then

342 00:21:26.990 00:21:28.889 Patrick Trainer: in the intermediate layer

343 00:21:28.940 00:21:31.570 Patrick Trainer: you would take some of the financial

344 00:21:31.880 00:21:38.639 Patrick Trainer: units. You take some of the customer units from somewhere else, and then then you would have, like your

345 00:21:39.400 00:21:40.480 Patrick Trainer: revenue

346 00:21:41.590 00:21:42.210 Patrick Trainer: portion.

347 00:21:42.210 00:21:42.550 Uttam Kumaran: Yeah.

348 00:21:42.550 00:21:44.940 Patrick Trainer: It’s like, it’s like.

349 00:21:45.790 00:21:48.859 Patrick Trainer: think of them as Lego pieces. Basically.

350 00:21:48.860 00:21:55.709 Uttam Kumaran: And that’s why, actually, like on the mart side for the folders, I could see the argument being

351 00:21:57.650 00:21:59.009 Uttam Kumaran: like, Hey.

352 00:22:00.070 00:22:04.170 Uttam Kumaran: we actually just need Hubspot data in a hubspot marts folder like

353 00:22:04.300 00:22:07.700 Uttam Kumaran: we don’t. I don’t want to call it sales. I want to call it Hubspot. That’s fine.

354 00:22:08.670 00:22:11.523 Uttam Kumaran: or it’s not fine. I don’t know, because, like.

355 00:22:11.840 00:22:12.220 Patrick Trainer: Right.

356 00:22:12.220 00:22:13.299 Uttam Kumaran: Or it’s a

357 00:22:13.330 00:22:16.629 Uttam Kumaran: we do it by business sector. Yeah. Well, that’s the.

358 00:22:16.630 00:22:23.849 Patrick Trainer: The idea behind the mart is like it. It’s the the data. Mark the Inman model of like where you’re

359 00:22:24.290 00:22:28.279 Patrick Trainer: grouping your reports or tables by domain

360 00:22:28.310 00:22:30.889 Patrick Trainer: that they’re relevant to to the business.

361 00:22:31.550 00:22:36.719 Uttam Kumaran: But is that something we need to fix? Or is that gonna be like up to the person? Right? Because

362 00:22:37.150 00:22:43.740 Uttam Kumaran: these guys have finance marketing like, what would would doing a Hubspot schema in March be okay.

363 00:22:43.740 00:22:45.189 Patrick Trainer: That doesn’t make any sense.

364 00:22:46.090 00:22:47.659 Uttam Kumaran: But I guess, like

365 00:22:48.460 00:22:50.519 Uttam Kumaran: I’m trying to play the other side like

366 00:22:55.260 00:22:58.319 Patrick Trainer: Unless it’s unless they just want to see like

367 00:22:58.750 00:23:00.830 Patrick Trainer: only Hubspot.

368 00:23:01.090 00:23:04.553 Uttam Kumaran: So you say, just think of a generic like, for example, these guys have like a

369 00:23:05.000 00:23:11.019 Uttam Kumaran: they have a product database like a postgres database. Yeah. But in that situation, let’s say they just want to see.

370 00:23:11.950 00:23:14.329 Uttam Kumaran: It would just be application, maybe, or like.

371 00:23:14.590 00:23:15.190 Patrick Trainer: Yeah.

372 00:23:16.910 00:23:22.720 Uttam Kumaran: This is where I feel like Nick’s gonna be like, we’re gonna have a try to think about some structure because I don’t.

373 00:23:22.910 00:23:28.430 Uttam Kumaran: Well, I mean, I don’t know. I you know it’s clear we have reporting here. We have something called Upl.

374 00:23:28.940 00:23:29.950 Uttam Kumaran: Right? If we go.

375 00:23:32.340 00:23:33.810 Uttam Kumaran: Looks like we have.

376 00:23:34.040 00:23:38.160 Uttam Kumaran: too. But even like this, even like dimensions and intermediate here.

377 00:23:38.220 00:23:40.820 Uttam Kumaran: I would rather all of this.

378 00:23:41.940 00:23:45.040 Uttam Kumaran: the okay. So like. Let’s take this example right.

379 00:23:45.640 00:23:48.979 Uttam Kumaran: These dimensions should get moved to

380 00:23:51.470 00:23:54.059 Uttam Kumaran: These dimensions should get moved to

381 00:23:54.830 00:23:55.740 Uttam Kumaran: raw.

382 00:23:57.920 00:24:01.100 Uttam Kumaran: Should these intermediate files just be tossed

383 00:24:01.200 00:24:03.380 Uttam Kumaran: into the intermediate

384 00:24:03.590 00:24:06.820 Uttam Kumaran: like, what is a subfolder and intermediate for this?

385 00:24:07.030 00:24:08.660 Uttam Kumaran: It it would match.

386 00:24:11.080 00:24:13.030 Uttam Kumaran: But I guess that’s my question is like

387 00:24:14.530 00:24:18.700 Uttam Kumaran: what I don’t know what the subfolder name within intermediate would be.

388 00:24:19.870 00:24:22.350 Uttam Kumaran: because not you don’t necessarily

389 00:24:23.100 00:24:25.419 Uttam Kumaran: doesn’t necessarily need to

390 00:24:25.890 00:24:27.240 Uttam Kumaran: match marts

391 00:24:27.560 00:24:28.800 Uttam Kumaran: like this is a pretty safe.

392 00:24:28.800 00:24:29.360 Patrick Trainer: No, it doesn’t.

393 00:24:29.360 00:24:30.070 Uttam Kumaran: Sample.

394 00:24:31.220 00:24:34.449 Patrick Trainer: It doesn’t. And you also, like you don’t have to.

395 00:24:34.750 00:24:37.459 Patrick Trainer: You don’t need those like intermediate

396 00:24:38.200 00:24:40.300 Patrick Trainer: files to really match

397 00:24:40.510 00:24:46.090 Patrick Trainer: anything like, I think those intermediate like. If you have sub directories like, that’s more for

398 00:24:46.560 00:24:50.799 Patrick Trainer: your organization, like, if you’re if you’re combining

399 00:24:51.080 00:24:53.230 Patrick Trainer: like I’m thinking of, like you

400 00:24:53.280 00:24:57.769 Patrick Trainer: combine the pieces with an implied like result.

401 00:24:59.390 00:25:05.809 Patrick Trainer: grouping those make sense together, and then, like, apply your subdirectory ask like.

402 00:25:07.580 00:25:09.350 Uttam Kumaran: I want to. I want to enforce.

403 00:25:09.350 00:25:10.299 Patrick Trainer: Pertains to.

404 00:25:10.480 00:25:12.800 Uttam Kumaran: I want to enforce only one layer

405 00:25:12.980 00:25:15.549 Uttam Kumaran: like one layer, deep meaning.

406 00:25:15.550 00:25:16.190 Patrick Trainer: Okay.

407 00:25:16.460 00:25:22.770 Uttam Kumaran: I don’t want. There’s I don’t see a world where you need a layer, another one within finance, unless.

408 00:25:22.770 00:25:23.740 Patrick Trainer: I mean, I agree.

409 00:25:23.900 00:25:27.159 Uttam Kumaran: Unless you have like a fat project. But like, I don’t see it. Cause

410 00:25:27.670 00:25:30.620 Uttam Kumaran: even then. Yeah, like, you have one separate.

411 00:25:30.620 00:25:31.560 Patrick Trainer: Agree too.

412 00:25:32.530 00:25:34.409 Uttam Kumaran: You get models, you get

413 00:25:34.580 00:25:50.490 Uttam Kumaran: the database. You get the schema right, because the reason why is like I wanted to match when you go into Snowflake and you search for a table. I want there to be like cool. Let me go find that within intermediate folder, and the and the finance, schema, and those match.

414 00:25:50.490 00:25:51.499 Patrick Trainer: Oh, yeah, right?

415 00:25:51.500 00:25:55.660 Uttam Kumaran: That’s I want that nav to really match. And it’s this always happens where

416 00:25:56.090 00:26:02.290 Uttam Kumaran: even and this is, this is gonna work both ways. Right. This is something where even the engine new engineer is going to be like, where is this thing?

417 00:26:03.040 00:26:05.849 Uttam Kumaran: And then also, I’ll

418 00:26:06.060 00:26:11.120 Uttam Kumaran: a business person is also gonna be like I selected from this table. But I want to go see the sequel.

419 00:26:11.510 00:26:15.429 Uttam Kumaran: Let me. Where is it? You know. So I I do want to enforce

420 00:26:15.800 00:26:20.190 Uttam Kumaran: the same constraints that you have on Snowflake here, which is just one layer deep.

421 00:26:20.490 00:26:22.990 Uttam Kumaran: or 2 layers from the David, from the yeah.

422 00:26:25.170 00:26:26.930 Uttam Kumaran: Let’s do that, too.

423 00:26:47.680 00:26:50.040 Uttam Kumaran: Is there any way to enforce that.

424 00:26:53.910 00:26:54.590 Patrick Trainer: Like.

425 00:26:56.000 00:26:58.555 Uttam Kumaran: Using like some sort of like

426 00:26:59.900 00:27:01.280 Uttam Kumaran: winter, or something.

427 00:27:01.280 00:27:02.260 Patrick Trainer: Wow!

428 00:27:02.760 00:27:03.690 Patrick Trainer: Maybe

429 00:27:04.320 00:27:05.410 Patrick Trainer: probably

430 00:27:07.470 00:27:09.720 Patrick Trainer: I don’t know how we would enforce it.

431 00:27:10.770 00:27:11.660 Patrick Trainer: for sure.

432 00:27:12.330 00:27:14.860 Uttam Kumaran: Well, I just yeah, I mean, I I we can ask

433 00:27:14.920 00:27:17.129 Uttam Kumaran: Claude. But basically, I just want to

434 00:27:17.620 00:27:19.529 Uttam Kumaran: like, I just want that to get caught.

435 00:27:20.490 00:27:21.050 Patrick Trainer: Yeah.

436 00:27:24.800 00:27:29.170 Uttam Kumaran: Because the game here is gonna be like codifying these roles into.

437 00:27:30.480 00:27:34.579 Uttam Kumaran: It’s like, I don’t think the game. I don’t think the game we’re playing is

438 00:27:36.810 00:27:37.830 Uttam Kumaran: is like

439 00:27:39.610 00:27:42.460 Uttam Kumaran: trust. That everybody adheres to. This, I think, is like

440 00:27:43.110 00:27:45.810 Uttam Kumaran: we. It almost just needs to be

441 00:27:46.260 00:27:59.759 Uttam Kumaran: like you. You’re you’re just prevented from doing anything else, basically as much as possible. I’m not saying that like people, I think that we’re just gonna have situations where there’s gonna be like, I just need to do this thing, or we just did this thing really quick. And

442 00:27:59.860 00:28:03.979 Uttam Kumaran: I would rather the system be like, you can’t do another thing.

443 00:28:04.010 00:28:08.210 Uttam Kumaran: Okay, cool, and we’ll just dodge the sprawl early on.

444 00:28:08.210 00:28:08.920 Patrick Trainer: Right.

445 00:28:10.245 00:28:14.524 Uttam Kumaran: And the other thing is like, we’ll, we’ll create

446 00:28:15.320 00:28:17.140 Uttam Kumaran: We’ll create a template repo.

447 00:28:17.170 00:28:18.670 Uttam Kumaran: So basically, it’s like

448 00:28:19.150 00:28:30.520 Uttam Kumaran: we’ll create a template repo with real and with dbt, that, like every new project you just bring in, it has the connection related information, or, like all that stuff, is easy to handle.

449 00:28:30.650 00:28:37.709 Uttam Kumaran: and then again, we’ll have like a little bit of like a how to start a new repo sort of like what secrets? So that’ll all like, get mirrored in.

450 00:28:41.130 00:28:44.509 Uttam Kumaran: Okay? I mean, I think this is pretty good. We have it. We didn’t talk

451 00:28:45.580 00:28:49.170 Uttam Kumaran: much about naming conventions for

452 00:28:49.580 00:28:51.830 Uttam Kumaran: tables or for columns.

453 00:28:53.710 00:28:55.679 Patrick Trainer: Yeah, I think we can get there.

454 00:28:56.272 00:29:01.099 Patrick Trainer: I mean, right now, the like. The high structure is what matters.

455 00:29:16.900 00:29:17.560 Uttam Kumaran: Okay.

456 00:29:21.270 00:29:21.980 Uttam Kumaran: Yeah.

457 00:29:22.340 00:29:24.970 Patrick Trainer: Part of the benefit or part of doing. This, too, is like.

458 00:29:25.880 00:29:32.870 Patrick Trainer: and especially with, like newer engineers like, I think is like this is really intuitive to like you and me, because we’ve done

459 00:29:33.110 00:29:36.799 Patrick Trainer: like, however many dbt projects in our lifetime, but like

460 00:29:37.210 00:29:40.250 Patrick Trainer: trying to convey this to I mean, I don’t know how many.

461 00:29:40.610 00:29:41.839 Uttam Kumaran: No, you’re right, I mean, look.

462 00:29:41.840 00:29:47.660 Patrick Trainer: To to like Nico or to even Robert. It’s like they’ve never

463 00:29:48.800 00:29:51.549 Patrick Trainer: like. They’ve never seen what a really

464 00:29:51.700 00:30:00.549 Patrick Trainer: fucked up sequel code base or like polluted database looks like or really understand, like the the deep seated problems in that.

465 00:30:00.550 00:30:00.890 Uttam Kumaran: Yeah.

466 00:30:00.890 00:30:02.840 Patrick Trainer: How that comes about.

467 00:30:02.910 00:30:12.019 Patrick Trainer: And it’s like. And they also haven’t seen the like. What? What is a really good like structured project or like code base look like

468 00:30:12.328 00:30:17.230 Patrick Trainer: and so it’s like it’s hard to see, like or reason about both sides of that coin.

469 00:30:17.320 00:30:18.760 Patrick Trainer: And so

470 00:30:19.690 00:30:20.769 Patrick Trainer: and so like

471 00:30:20.780 00:30:22.340 Patrick Trainer: trying to like

472 00:30:23.370 00:30:30.690 Patrick Trainer: tell that story is is difficult, because it’s like, what do you base it on? Yeah, like. And so

473 00:30:30.760 00:30:34.629 Patrick Trainer: I think it is good that it’s like, we got to a point where it’s like

474 00:30:34.660 00:30:36.559 Patrick Trainer: fuck. Everything is like.

475 00:30:36.900 00:30:39.169 Patrick Trainer: really difficult to manage now.

476 00:30:39.320 00:30:43.500 Patrick Trainer: And like, I think that puts like that context of like.

477 00:30:44.260 00:30:49.399 Patrick Trainer: okay, this Xyz isn’t working. So we need to like change gears.

478 00:30:49.567 00:30:50.070 Uttam Kumaran: For sure, like.

479 00:30:50.070 00:30:56.960 Patrick Trainer: Show showed like the good side of the world. And then it’s like, okay. And then then it starts to become intuitive.

480 00:30:57.330 00:31:05.960 Uttam Kumaran: And look a lot of people. This is the thing. It’s like a lot of people don’t care. But if you ask for opinion they’ll throw an opinion out instead if we’re like. This is the way it’s done.

481 00:31:06.710 00:31:11.929 Uttam Kumaran: I’m I’m down to discuss this with people who are like at our level or like, I wanna

482 00:31:12.060 00:31:15.690 Uttam Kumaran: argue one way or another. But most part

483 00:31:15.700 00:31:29.300 Uttam Kumaran: it’s not gonna happen that way. Also, because in this situation there’s also a business reason. Typically, this is just like internal engineering drama of like, how do we want to do things in this situation? This is this is something I want to be an artifact

484 00:31:29.480 00:31:36.389 Uttam Kumaran: that is set to the client right? Or the clients interact with the repository. There’s a different layer. There’s a different.

485 00:31:36.390 00:31:37.050 Patrick Trainer: Yeah, yeah.

486 00:31:37.050 00:31:37.810 Uttam Kumaran: Amount of

487 00:31:37.940 00:31:38.900 Uttam Kumaran: like.

488 00:31:39.640 00:31:40.900 Patrick Trainer: That becomes a feature.

489 00:31:41.040 00:31:59.829 Uttam Kumaran: Exactly. No, everything we do. I want it to be marketed and sold as something we’re developing. And you just have to assume another. There’s almost another layer of like. I have no idea what I’m looking at that we really have to guard against here, which is the business user. Right. If I’m if I’m like yo go look in the repo

490 00:32:00.630 00:32:05.210 Uttam Kumaran: that it’s like that’s that’s like a real thing. I want to be sending people because.

491 00:32:05.210 00:32:05.850 Patrick Trainer: Right.

492 00:32:06.200 00:32:12.988 Uttam Kumaran: Companies are not gonna do that like. That’s something I want us to be really proud of and like, be able to send out right and

493 00:32:13.450 00:32:18.680 Patrick Trainer: I mean, that’s partially why, like I mentioned the like, the 5 trend Dbt packages.

494 00:32:19.280 00:32:20.130 Uttam Kumaran: Yeah, like.

495 00:32:20.130 00:32:26.489 Patrick Trainer: They’re all set up the same exact way, not just like for their sanity, but for like

496 00:32:26.600 00:32:29.100 Patrick Trainer: every person that’s going to use that package.

497 00:32:31.420 00:32:34.260 Patrick Trainer: and so yeah, if you go into it. Like all of them.

498 00:32:34.270 00:32:35.869 Patrick Trainer: they’re all structured the same.

499 00:32:36.790 00:32:37.330 Patrick Trainer: They.

500 00:32:37.330 00:32:37.949 Uttam Kumaran: They do?

501 00:32:37.950 00:32:48.660 Patrick Trainer: A little bit different in that. Like they import all of their packages together, which is its own thing, but, like the structure of like in their model directory.

502 00:32:49.010 00:32:51.260 Patrick Trainer: Every single one is the same.

503 00:32:51.280 00:32:53.220 Patrick Trainer: and it’s like

504 00:32:54.010 00:33:00.649 Patrick Trainer: they’ve got the the way they do their sources like the way they do their intermediate models. Like they’re all the same.

505 00:33:00.900 00:33:03.490 Patrick Trainer: it’s it. It just works

506 00:33:04.930 00:33:12.280 Patrick Trainer: And so there’s that predictability. And that predictability becomes the feature in it of itself. It’s like a meta meta meta feature.

507 00:33:12.440 00:33:15.309 Uttam Kumaran: Yeah, no, you’re you’re you’re spot on, like.

508 00:33:15.980 00:33:20.139 Uttam Kumaran: I mean, I just, yeah, I agree that it’s it’s very consistent. And

509 00:33:20.310 00:33:24.579 Uttam Kumaran: look again, this is their like public facing product, right? And so

510 00:33:24.890 00:33:28.090 Uttam Kumaran: for them. This is like what we should be like, kind of.

511 00:33:28.250 00:33:29.070 Patrick Trainer: Striving towards yeah.

512 00:33:29.070 00:33:29.780 Uttam Kumaran: Yeah.

513 00:33:30.260 00:33:31.040 Uttam Kumaran: I mean, even.

514 00:33:31.040 00:33:33.370 Patrick Trainer: Down to like. They put 2.

515 00:33:33.370 00:33:33.910 Uttam Kumaran: Understand.

516 00:33:33.910 00:33:36.490 Patrick Trainer: Scores between, yeah, they.

517 00:33:36.490 00:33:37.080 Uttam Kumaran: Source, end.

518 00:33:37.080 00:33:42.920 Patrick Trainer: Scores between, like the the entity, and then then they have, like the

519 00:33:43.150 00:33:50.120 Patrick Trainer: I mean, adjective and verb, and those are only separated by one underscore like, it’s those sorts of things that like

520 00:33:50.640 00:33:51.510 Patrick Trainer: that.

521 00:33:51.660 00:33:53.320 Patrick Trainer: like those those types.

522 00:33:53.320 00:33:54.829 Uttam Kumaran: Like an attention to detail, thing.

523 00:33:54.830 00:33:57.439 Patrick Trainer: Yeah, those yeah, that that really matter.

524 00:33:59.490 00:34:00.450 Uttam Kumaran: Yeah.

525 00:34:05.020 00:34:07.770 Uttam Kumaran: Oh, and they have the Dbt Docs hosted here, too.

526 00:34:07.990 00:34:08.590 Patrick Trainer: You know.

527 00:34:09.770 00:34:14.670 Patrick Trainer: Yeah. And if you click between project and database like you can see how it’s

528 00:34:15.300 00:34:17.130 Patrick Trainer: like. And then everything just

529 00:34:17.429 00:34:18.600 Patrick Trainer: gets flattened.

530 00:34:19.350 00:34:20.400 Uttam Kumaran: Yeah.

531 00:34:22.159 00:34:25.889 Patrick Trainer: And so it’s like, you’re not going through all these different.

532 00:34:26.260 00:34:29.690 Uttam Kumaran: So see, they do. They do like almost

533 00:34:31.850 00:34:34.230 Uttam Kumaran: like layer source.

534 00:34:35.090 00:34:36.300 Uttam Kumaran: table name.

535 00:34:36.730 00:34:37.440 Patrick Trainer: Yeah.

536 00:34:37.469 00:34:40.789 Uttam Kumaran: And then at the end they do. They just do the source table name.

537 00:34:41.010 00:34:43.349 Uttam Kumaran: I don’t. Yeah, I mean, like.

538 00:34:43.350 00:34:45.670 Patrick Trainer: I mean, that’s just for the package.

539 00:34:45.670 00:34:46.750 Uttam Kumaran: For the package. Yeah.

540 00:34:46.750 00:34:53.919 Patrick Trainer: That’s just for the package. And like, I don’t think that makes sense like as you go further down, but like putting it into those

541 00:34:54.380 00:35:00.799 Patrick Trainer: like initial layers, I think like makes sense. But as you get towards the like, closer to the bi tool.

542 00:35:01.240 00:35:01.780 Uttam Kumaran: Then it’s.

543 00:35:01.780 00:35:07.420 Patrick Trainer: And like, it’s everything’s going to start pairing back and just be like your table name, essentially.

544 00:35:07.420 00:35:08.250 Uttam Kumaran: Yeah, you’re, you’re.

545 00:35:08.250 00:35:09.020 Patrick Trainer: Entity.

546 00:35:13.530 00:35:17.759 Uttam Kumaran: Okay, great. Okay? I think stocks pretty good. I’m going to.

547 00:35:19.950 00:35:25.100 Uttam Kumaran: We we already have a dock that’s basically like folder structure staging layer.

548 00:35:25.130 00:35:30.220 Uttam Kumaran: I’m going to use this transcript and our notes to kind of redo the doc.

549 00:35:30.340 00:35:31.639 Uttam Kumaran: And then

550 00:35:32.070 00:35:36.339 Uttam Kumaran: I want to send this to us to Brian and Nick.

551 00:35:36.870 00:35:45.380 Uttam Kumaran: So, and then if we all get the thumbs up, then we’ll have 2 tickets out, one for pool parts, one for Stella to just re-arc everything.

552 00:35:45.881 00:35:48.600 Uttam Kumaran: And then we’ll kind of close shop on this one.

553 00:35:48.600 00:35:49.310 Patrick Trainer: Right.

554 00:35:49.430 00:35:50.770 Patrick Trainer: no sounds good.

555 00:35:52.640 00:36:01.309 Uttam Kumaran: Okay, cool. I have a couple of things I need to do. But you should. We maybe meet you now we’ll hop on like later today.

556 00:36:02.260 00:36:04.299 Patrick Trainer: So I’m down for whatever. Just let me know.

557 00:36:04.300 00:36:04.990 Uttam Kumaran: Okay.

558 00:36:06.090 00:36:06.790 Uttam Kumaran: if.

559 00:36:07.040 00:36:09.780 Patrick Trainer: If not, we can just type it out, but.

560 00:36:10.540 00:36:11.520 Uttam Kumaran: Yeah, I, like.

561 00:36:11.520 00:36:12.850 Patrick Trainer: I mean, just play it by your.

562 00:36:13.000 00:36:17.629 Uttam Kumaran: Yeah, I just I wanna see if we can send one email and like, have it like.

563 00:36:17.700 00:36:22.129 Uttam Kumaran: go from Paula to Hubspot and like, just see one of them go. We can even just like

564 00:36:22.260 00:36:26.020 Uttam Kumaran: try to figure it out on the call. So cool.

565 00:36:26.665 00:36:28.800 Uttam Kumaran: Let’s just see if we can.

566 00:36:30.060 00:36:30.560 Uttam Kumaran: Same.

567 00:36:30.560 00:36:34.619 Patrick Trainer: So with with what you said earlier about like just wanting to

568 00:36:34.740 00:36:36.110 Patrick Trainer: continue

569 00:36:37.330 00:36:38.210 Patrick Trainer: with

570 00:36:38.360 00:36:39.530 Patrick Trainer: notion.

571 00:36:39.730 00:36:40.770 Uttam Kumaran: Oh, yeah.

572 00:36:40.770 00:36:42.090 Patrick Trainer: Yeah, how is.

573 00:36:42.090 00:36:46.230 Uttam Kumaran: Well, like dude. I tried to use Hubspot deal

574 00:36:46.840 00:36:58.700 Uttam Kumaran: and I could see it being nice because it basically like centralizes it like attaches all the emails you had to a lead, a lead as a contact. And like a deal stage, it’s just like

575 00:36:59.480 00:37:04.070 Uttam Kumaran: it’s just so many properties. And I have to do that in.

576 00:37:04.830 00:37:08.980 Uttam Kumaran: I’ll have to do that in Hubspot at the moment. All of our like

577 00:37:09.490 00:37:12.530 Uttam Kumaran: the Zoom Meetings are getting put into there.

578 00:37:12.530 00:37:13.120 Patrick Trainer: Yeah.

579 00:37:13.120 00:37:13.830 Uttam Kumaran: Like.

580 00:37:14.070 00:37:17.009 Uttam Kumaran: And basically, I’m already running between

581 00:37:17.764 00:37:20.580 Uttam Kumaran: email, figma and notion.

582 00:37:21.640 00:37:23.099 Patrick Trainer: A lot of context switch.

583 00:37:23.100 00:37:26.590 Uttam Kumaran: It’s a lot of contact phishing. And we’re not dealing with like we’re only.

584 00:37:26.840 00:37:28.029 Patrick Trainer: Totally. Agree. Yeah.

585 00:37:28.030 00:37:33.319 Uttam Kumaran: Yeah, so so I. But I also think like not using the deals is actually okay, because

586 00:37:33.360 00:37:47.240 Uttam Kumaran: we’re only focused on getting to the call stage. Once the call stage happens, then people move into a notion deal. If when we move, when we move beyond habit, when we move beyond that, everything, for example, having that

587 00:37:47.540 00:37:49.120 Uttam Kumaran: conversion rate.

588 00:37:49.210 00:37:50.279 Uttam Kumaran: or like

589 00:37:50.890 00:37:54.799 Uttam Kumaran: when it actually matters to know that someone has gone beyond

590 00:37:54.930 00:38:05.499 Uttam Kumaran: or when we’re like, Oh, this person isn’t good. We should like re up them in 6 months. That’s where I could see us like coming back to Hubspot, being like we should do this just a little bit. Od, right now for me

591 00:38:06.177 00:38:08.199 Uttam Kumaran: to like migrate the process so.

592 00:38:08.200 00:38:09.330 Patrick Trainer: Yeah. Fine with me.

593 00:38:09.330 00:38:13.220 Uttam Kumaran: Yeah, I mean, look, I think Hubspot is gonna work. The only other

594 00:38:14.570 00:38:20.219 Uttam Kumaran: I in about a year I think we should go talk to these guys because I’ve been following them

595 00:38:22.120 00:38:26.889 Uttam Kumaran: on Twitter. And I think they’re basically like trying to just like be like a Hubspot killer.

596 00:38:27.270 00:38:32.360 Uttam Kumaran: which is great because it’s I think, guys, I’ll send you on zoom chat.

597 00:38:38.140 00:38:39.480 Patrick Trainer: At TO.

598 00:38:47.830 00:38:50.038 Uttam Kumaran: They’re a bit cheaper, and

599 00:38:51.540 00:38:54.340 Uttam Kumaran: well, they’re a little bit. They’re a little bit cheaper. But

600 00:38:54.530 00:38:58.439 Uttam Kumaran: I I’ve been seeing really good things on Twitter about them.

601 00:38:58.440 00:38:59.130 Patrick Trainer: Okay.

602 00:39:00.810 00:39:02.430 Patrick Trainer: There’s another one that I’ve been

603 00:39:05.377 00:39:08.782 Patrick Trainer: that I’ve been following as well.

604 00:39:09.560 00:39:13.230 Patrick Trainer: it actually looks really similar to this.

605 00:39:17.280 00:39:18.040 Patrick Trainer: like.

606 00:39:18.190 00:39:20.109 Patrick Trainer: like scarily similar.

607 00:39:30.700 00:39:31.750 Uttam Kumaran: Oh, nice.

608 00:39:33.630 00:39:39.309 Uttam Kumaran: I mean, dude like I actually like, that’s so. That’s why in the tools database I started reporting on the cost, too.

609 00:39:39.430 00:39:42.899 Uttam Kumaran: because once we get stuff running I’ll then be able to be like

610 00:39:43.890 00:39:47.080 Uttam Kumaran: cool like this. Bill is like a hundred a month, or 300 a month like.

611 00:39:47.080 00:39:47.790 Patrick Trainer: Right.

612 00:39:48.740 00:39:50.760 Uttam Kumaran: Can we get feature parity somewhere else.

613 00:39:50.760 00:39:51.380 Patrick Trainer: Right.

614 00:39:53.390 00:39:55.100 Uttam Kumaran: But this looks sick, too.

615 00:39:57.660 00:40:02.060 Uttam Kumaran: The only thing is I’m I don’t like when github stars anymore, because.

616 00:40:02.569 00:40:03.590 Patrick Trainer: Yeah. No.

617 00:40:04.100 00:40:06.340 Uttam Kumaran: One big people farm but 2.

618 00:40:06.430 00:40:10.199 Uttam Kumaran: It’s like dude, like, these are developers starring in Github like

619 00:40:10.650 00:40:14.439 Uttam Kumaran: developers aren’t using the Crm like, it’s a bit. Yeah.

620 00:40:16.530 00:40:21.840 Uttam Kumaran: but it will see what matters. So the only thing I’ve heard about in hubs. Everybody has been like

621 00:40:21.950 00:40:26.019 Uttam Kumaran: dude. I was trying to cheap out. And then I ended up just like doing Hubspot and like.

622 00:40:26.580 00:40:28.309 Uttam Kumaran: just they’re the best

623 00:40:28.370 00:40:32.180 Uttam Kumaran: which I don’t typically like doing. But this is one area

624 00:40:32.330 00:40:36.199 Uttam Kumaran: people like just do Hubspot or or salesforce like.

625 00:40:36.200 00:40:36.930 Patrick Trainer: Yeah.

626 00:40:37.320 00:40:39.100 Patrick Trainer: yeah, no, I

627 00:40:39.620 00:40:41.959 Patrick Trainer: I’m with you in in that, too.

628 00:40:42.510 00:40:44.240 Patrick Trainer: It’s it’s 1 of those like.

629 00:40:44.880 00:40:46.050 Patrick Trainer: I don’t want to

630 00:40:46.810 00:40:49.860 Patrick Trainer: succumb to the beast, you know, and they’re both.

631 00:40:49.860 00:40:50.660 Uttam Kumaran: Yeah.

632 00:40:50.660 00:40:51.620 Patrick Trainer: At that point.

633 00:40:51.820 00:40:54.840 Patrick Trainer: I really don’t want to fuck with salesforce. I hate salesforce.

634 00:40:54.840 00:40:56.819 Uttam Kumaran: I don’t either. And

635 00:40:56.980 00:41:02.430 Uttam Kumaran: and again the whole thing through the whole process. I want everything workflows like I don’t want to be sitting on Hubspot

636 00:41:02.940 00:41:08.200 Uttam Kumaran: like and like clicking things. And that’s what they really want us to be doing. And it’s not.

637 00:41:08.200 00:41:11.519 Patrick Trainer: I mean, that’s kind of like. So I’ve been setting up the

638 00:41:11.700 00:41:15.150 Patrick Trainer: like the the points and shit in Hubspot.

639 00:41:15.400 00:41:16.760 Patrick Trainer: And it’s just like

640 00:41:16.900 00:41:18.740 Patrick Trainer: click, click.

641 00:41:18.820 00:41:21.420 Patrick Trainer: type, type, save click

642 00:41:21.430 00:41:23.889 Patrick Trainer: clay. It’s just it’s so boring.

643 00:41:24.260 00:41:25.390 Patrick Trainer: And it’s like, I hate.

644 00:41:25.390 00:41:26.240 Uttam Kumaran: Parking.

645 00:41:26.390 00:41:26.860 Patrick Trainer: Yeah.

646 00:41:26.860 00:41:29.675 Uttam Kumaran: It’s just like, yeah, it’s just like,

647 00:41:31.326 00:41:33.470 Uttam Kumaran: Yeah, I I just.

648 00:41:34.680 00:41:43.580 Uttam Kumaran: I don’t know all these mark do, because these marketing tools are built for marketers, and they’re like how to do anything except for like tying these like they need like these editors. And I’m like.

649 00:41:43.580 00:41:44.190 Patrick Trainer: Yeah.

650 00:41:44.190 00:41:44.796 Uttam Kumaran: We’ve been.

651 00:41:46.130 00:41:46.800 Patrick Trainer: Yeah.

652 00:41:47.900 00:41:50.059 Uttam Kumaran: But you know, that’s what it is.

653 00:41:50.380 00:41:55.970 Uttam Kumaran: Okay, cool. Well, I’ll message you later today. And then, yeah, okay, cool.

654 00:41:56.570 00:41:57.290 Patrick Trainer: Right on.

655 00:41:57.590 00:41:59.438 Uttam Kumaran: Thanks. Matt, later, peace.