Meeting Title: Analytics Engineering Daily Sync Date: 2025-02-17 Meeting participants: Luke Daque, Uttam Kumaran, Awaish Kumar, Caio
WEBVTT
1 00:04:14.970 ⇒ 00:04:15.640 Caio: Yes.
2 00:04:49.410 ⇒ 00:04:50.340 Uttam Kumaran: Hey, guys.
3 00:04:52.730 ⇒ 00:04:53.860 Caio: Hey! Hello!
4 00:04:53.860 ⇒ 00:04:54.909 Luke Daque: Hello! Hello!
5 00:04:55.560 ⇒ 00:05:01.775 Uttam Kumaran: A cool yeah, I wanted to get sort of the
6 00:05:02.610 ⇒ 00:05:06.999 Uttam Kumaran: The ae grew together and sort of talk through what each of us.
7 00:05:07.300 ⇒ 00:05:11.200 Uttam Kumaran: you know, kind of have on our plate, and just kind of start to.
8 00:05:11.370 ⇒ 00:05:16.149 Uttam Kumaran: you know. Maybe do a daily sync between the 4 of us on how we’re, gonna
9 00:05:16.750 ⇒ 00:05:20.550 Uttam Kumaran: you know, push Dbt models and sort of take care of stuff for clients.
10 00:05:21.143 ⇒ 00:05:26.990 Uttam Kumaran: This is supposed to be kind of a little bit open like I. I do want to spend a little bit of time talking about
11 00:05:27.652 ⇒ 00:05:31.309 Uttam Kumaran: maybe some tech debt. But I also want to talk about
12 00:05:32.920 ⇒ 00:05:36.470 Uttam Kumaran: You know the few items that I I mentioned.
13 00:05:37.112 ⇒ 00:05:42.429 Uttam Kumaran: I guess. Yeah. Anything else we wanna chat about, or we can just dive right in.
14 00:05:46.490 ⇒ 00:05:50.180 Luke Daque: Yeah, I think we’re good. We can like maybe dive right in.
15 00:05:51.110 ⇒ 00:05:54.050 Uttam Kumaran: Cool. So yeah, I think,
16 00:05:54.780 ⇒ 00:05:58.090 Uttam Kumaran: so one thing you know, we we reviewed all the
17 00:05:58.350 ⇒ 00:06:10.520 Uttam Kumaran: the core data modeling clients. Today. I think, one, it’s clear that there’s still a lot of work to be done on basically like core data marts for Javi even and stack Blitz
18 00:06:11.149 ⇒ 00:06:16.080 Uttam Kumaran: and so that’s 1 of the things that maybe I want to talk about was, I think now, between
19 00:06:16.250 ⇒ 00:06:22.310 Uttam Kumaran: the 3 of you guys and me, sort of assisting. I think we basically will have one person
20 00:06:23.183 ⇒ 00:06:25.350 Uttam Kumaran: on each client. I guess.
21 00:06:25.510 ⇒ 00:06:29.519 Uttam Kumaran: I think Eden is probably where I want to talk a wish like.
22 00:06:29.720 ⇒ 00:06:34.979 Uttam Kumaran: how do we wanna set up the data mark there like, how can I be helpful? I think
23 00:06:35.140 ⇒ 00:06:43.779 Uttam Kumaran: originally I was gonna sort of help to organize that. I haven’t done it yet. But like, we basically want to start to create a mark across the key data sources
24 00:06:43.980 ⇒ 00:06:47.669 Uttam Kumaran: like, what do you think is helpful for us to like kick that off.
25 00:06:50.340 ⇒ 00:06:58.470 Awaish Kumar: I think we build a like groundwork with the architecture diagram and the dB design.
26 00:06:58.580 ⇒ 00:07:03.459 Awaish Kumar: So we know, like, what are the major forces and what tables.
27 00:07:04.030 ⇒ 00:07:11.120 Awaish Kumar: And like, we are looking to create. And then, yeah, we can like build on top of that
28 00:07:11.740 ⇒ 00:07:20.240 Awaish Kumar: that for each business domain, like, for example marketing. And you can like.
29 00:07:21.027 ⇒ 00:07:23.650 Awaish Kumar: think of, maybe get help from
30 00:07:24.500 ⇒ 00:07:27.194 Awaish Kumar: analysts to see what kind of metrics
31 00:07:27.950 ⇒ 00:07:36.800 Awaish Kumar: or dashboards we want to create and based on that. We can like build a effect and dimension tables.
32 00:07:39.550 ⇒ 00:07:47.380 Uttam Kumaran: Yeah. So taking a look at like what we have here.
33 00:07:51.990 ⇒ 00:07:55.599 Uttam Kumaran: So this is roughly like the prod, the architecture diagram
34 00:07:55.800 ⇒ 00:07:59.549 Uttam Kumaran: for for Eden, right? So basically.
35 00:08:00.330 ⇒ 00:08:11.979 Awaish Kumar: I think there is another one. Actually, I I put it in a separate file, so you cannot see in this one. If you go back to the files. Maybe there is.
36 00:08:16.440 ⇒ 00:08:18.350 Awaish Kumar: There’s 1 called Aiden.
37 00:08:23.480 ⇒ 00:08:29.020 Uttam Kumaran: I just have this one, or it’s this one.
38 00:08:30.540 ⇒ 00:08:31.899 Awaish Kumar: I don’t know why.
39 00:08:34.510 ⇒ 00:08:38.330 Uttam Kumaran: This is sort of the same one, though I think.
40 00:08:40.400 ⇒ 00:08:43.990 Awaish Kumar: No like the Dbt. Models are separated out in that one.
41 00:08:44.290 ⇒ 00:08:49.689 Uttam Kumaran: Oh, yeah, if you can find it, let me know, and I can put it here.
42 00:08:50.200 ⇒ 00:08:50.745 Awaish Kumar: Okay.
43 00:08:53.800 ⇒ 00:08:54.510 Uttam Kumaran: Yes.
44 00:08:55.237 ⇒ 00:09:03.270 Caio: No 1 1 question that I had before. Actually, that that’s actually really really nice to see these divers. Because when I was looking at. Note in ocean.
45 00:09:03.715 ⇒ 00:09:07.850 Caio: I noted that there was no diagrams there to understand. Like the data flow.
46 00:09:07.850 ⇒ 00:09:08.290 Uttam Kumaran: Yeah.
47 00:09:08.290 ⇒ 00:09:12.009 Caio: Everything, so they’re probably is somewhere else, as I see. You know.
48 00:09:12.590 ⇒ 00:09:22.459 Uttam Kumaran: Yeah, I was literally about to say that for context, we are, we just started working on these diagrams for every single client. They’re not exactly like
49 00:09:22.870 ⇒ 00:09:27.880 Uttam Kumaran: super technical er diagrams, but I think for our clients it’s helpful to see
50 00:09:28.150 ⇒ 00:09:31.820 Uttam Kumaran: all the sources, how it’s coming into the warehouse.
51 00:09:32.050 ⇒ 00:09:39.189 Uttam Kumaran: I think in the warehouse we’ll start to add a little bit more like which database and stuff. But then, basically, what models and what dashboards.
52 00:09:40.470 ⇒ 00:09:44.750 Uttam Kumaran: So I’ll I’ll I’ll share this one with you. So you have it.
53 00:09:45.330 ⇒ 00:09:48.040 Caio: Great. No, that’s that looks really nice, actually.
54 00:09:58.980 ⇒ 00:10:10.130 Uttam Kumaran: So I think a waste like thinking through this. I think the biggest thing we sort of need to work from is like which ones of these we want to like keep versus, like, basically deprecate. Right?
55 00:10:12.630 ⇒ 00:10:17.269 Uttam Kumaran: I think we have the product sales summary.
56 00:10:17.610 ⇒ 00:10:23.079 Uttam Kumaran: We have orders. We have some stuff on transactions.
57 00:10:23.210 ⇒ 00:10:27.830 Uttam Kumaran: So I guess I want to start to think about, how do we go from domain to domain to domain and sort of
58 00:10:28.600 ⇒ 00:10:33.170 Uttam Kumaran: make sure we can build the underlying tables and start to deprecate some of these.
59 00:10:36.980 ⇒ 00:10:39.629 Awaish Kumar: Yeah, I’ve shared the link in the chat.
60 00:10:40.480 ⇒ 00:10:41.220 Uttam Kumaran: Oh, okay.
61 00:10:51.210 ⇒ 00:10:54.000 Uttam Kumaran: okay, I’m just gonna copy this one over. That’s okay.
62 00:10:54.730 ⇒ 00:10:55.730 Awaish Kumar: Okay.
63 00:11:02.060 ⇒ 00:11:04.457 Caio: And for this one, even, for example.
64 00:11:05.080 ⇒ 00:11:14.820 Caio: did you guys build the whole architecture for them? Or you’re just consuming from well, their sources or their data warehouse.
65 00:11:15.070 ⇒ 00:11:16.860 Caio: and then liberty.
66 00:11:17.440 ⇒ 00:11:21.189 Uttam Kumaran: So we came in to a situation that they had a bunch of stuff
67 00:11:21.330 ⇒ 00:11:25.900 Uttam Kumaran: basically like 80% of these are things that they already had
68 00:11:26.487 ⇒ 00:11:29.340 Uttam Kumaran: but like it’s like super messy sequel.
69 00:11:30.108 ⇒ 00:11:38.709 Uttam Kumaran: They sort of hired someone who came in and sort of did like half baked everything. But like, it’s really hard to maintain. It was really hard for them to basically identify
70 00:11:38.970 ⇒ 00:11:42.380 Uttam Kumaran: how to change things or what to implement new logic.
71 00:11:42.510 ⇒ 00:11:49.499 Uttam Kumaran: So one goal is like on our team is basically to start to deprecate their models and build out the data mark from our side.
72 00:11:51.170 ⇒ 00:11:54.580 Uttam Kumaran: And so part of the task
73 00:11:54.990 ⇒ 00:11:57.350 Uttam Kumaran: or a waste here is just to think about.
74 00:11:57.590 ⇒ 00:12:00.600 Uttam Kumaran: okay, how do we consolidate all of these into like.
75 00:12:00.760 ⇒ 00:12:08.459 Uttam Kumaran: okay, we need orders, transactions. Right? We need a clear sort of set of like mark tables that can answer all of these questions.
76 00:12:09.203 ⇒ 00:12:20.530 Uttam Kumaran: Some of these are related to marketing customer customer support right? But we just wanna make sure that we can start to pull from our data sources. And then we can start to have the bi tools pull from the new ones.
77 00:12:22.440 ⇒ 00:12:27.179 Uttam Kumaran: So I don’t know. I guess a wish like, do you think it’s best? I think we should start with.
78 00:12:27.330 ⇒ 00:12:32.870 Uttam Kumaran: I think we should start with like sales and marketing first, st basically
79 00:12:33.380 ⇒ 00:12:35.620 Uttam Kumaran: through, how do we like deprecate?
80 00:12:36.600 ⇒ 00:12:38.040 Uttam Kumaran: Basically like.
81 00:12:38.470 ⇒ 00:12:46.049 Uttam Kumaran: I mean, ideally, we end up with something that’s like. Let me put a comment like ideally, we end up with like orders, right
82 00:12:46.360 ⇒ 00:12:52.170 Uttam Kumaran: transactions, customers, products.
83 00:12:53.840 ⇒ 00:12:58.160 Uttam Kumaran: We may need some notion of like bundles, maybe
84 00:13:01.900 ⇒ 00:13:04.110 Uttam Kumaran: right like, but beyond this like.
85 00:13:04.520 ⇒ 00:13:06.520 Uttam Kumaran: and then we may. We may need shipments.
86 00:13:24.990 ⇒ 00:13:30.460 Caio: And we are not materializing everything to their data. Warehouse.
87 00:13:31.640 ⇒ 00:13:37.420 Uttam Kumaran: Yeah. So we’re so they’re using bigquery. And yeah, we’re they’re they’re materializing everything in there right now.
88 00:13:37.780 ⇒ 00:13:44.620 Caio: Okay. And you know, if they follow any, for example, the the standard medal architecture, or something like that.
89 00:13:45.150 ⇒ 00:13:50.670 Uttam Kumaran: No, they they we’re we are sort of coming in and and basically setting all that up.
90 00:13:50.780 ⇒ 00:13:53.231 Uttam Kumaran: So we came in and implemented
91 00:13:53.980 ⇒ 00:13:57.079 Uttam Kumaran: Dbt core. So they didn’t have Dbt at all.
92 00:13:57.250 ⇒ 00:14:03.280 Uttam Kumaran: So we came in and basically implemented Dbt, they had a bunch of scheduled queries in bigquery.
93 00:14:03.400 ⇒ 00:14:06.249 Uttam Kumaran: And so we sort of one moved everything to Dvt.
94 00:14:07.076 ⇒ 00:14:12.199 Uttam Kumaran: And then second, right now, we’re sort of building out the core marts models.
95 00:14:12.720 ⇒ 00:14:17.200 Uttam Kumaran: and some of these are not ours like. We only built like maybe 5 or 6 of these.
96 00:14:17.500 ⇒ 00:14:22.779 Uttam Kumaran: And then slowly, we’re like deprecating some of the ones they built and replacing it with, like our models.
97 00:14:23.561 ⇒ 00:14:28.630 Uttam Kumaran: Similarly with sources. So it took us about a month to sort of even get to this point, which was like
98 00:14:29.000 ⇒ 00:14:34.750 Uttam Kumaran: all the sources organized, and then sort of have a good sense of how what the existing
99 00:14:35.210 ⇒ 00:14:38.750 Uttam Kumaran: architecture is. And then, now we’re moving into.
100 00:14:40.800 ⇒ 00:14:45.150 Uttam Kumaran: we’re moving into. Okay, can we start to build a core data mark for each category.
101 00:14:45.290 ⇒ 00:14:51.880 Uttam Kumaran: And then right now. So on our side, we typically do. We’ll do raw intermediate. And Martz.
102 00:14:53.500 ⇒ 00:14:56.989 Uttam Kumaran: And we, we basically our typical structure.
103 00:14:57.536 ⇒ 00:15:00.980 Uttam Kumaran: and in notion, you’ll see that there is. There is actually data
104 00:15:02.960 ⇒ 00:15:06.079 Uttam Kumaran: like how we structure a Dbt thing here.
105 00:15:07.030 ⇒ 00:15:11.669 Uttam Kumaran: which is basically we structure things like this, which is like models raw.
106 00:15:11.990 ⇒ 00:15:15.305 Uttam Kumaran: And each of the sort of business category
107 00:15:16.650 ⇒ 00:15:22.330 Uttam Kumaran: so we’re basically trying to build like this sort of marts for them.
108 00:15:29.320 ⇒ 00:15:34.270 Caio: And as I understand, we are doing this, let’s say organization. And then
109 00:15:34.972 ⇒ 00:15:52.759 Caio: migrating the models. Are we also, for example, I don’t know. What is, what is the deal with them? But are we upselling something like, Oh, do you wanna also do some incremental strategy or something that would, you know, consume less from your data warehouse, or something like that?
110 00:15:53.690 ⇒ 00:15:55.400 Caio: Or is that already part of the.
111 00:15:55.400 ⇒ 00:15:56.190 Uttam Kumaran: Where’s
112 00:15:56.670 ⇒ 00:16:06.009 Uttam Kumaran: yeah? Like, in terms of the scope, we’re basically as like, how do we discussed in the meeting this morning? We sort of have several dashboards that we’re looking to build for them.
113 00:16:06.633 ⇒ 00:16:17.769 Uttam Kumaran: So right now, we’re sort of our team is really just building like the A team is basically just making sure that the reporting models are there so that the analy analyst team can sort of build that
114 00:16:18.030 ⇒ 00:16:21.520 Uttam Kumaran: ultimately, like, I don’t think we’re at the stage even to consider
115 00:16:22.480 ⇒ 00:16:26.839 Uttam Kumaran: cost saving strategies or anything we’re we’re still at the basic level of like
116 00:16:27.040 ⇒ 00:16:32.640 Uttam Kumaran: is the data accurate? And is it up to date, you know, in the warehouse. Yeah.
117 00:16:33.240 ⇒ 00:16:33.790 Caio: Perfect.
118 00:16:36.410 ⇒ 00:16:39.200 Uttam Kumaran: So I think a wish like, I don’t know. What do you think about this list
119 00:16:39.520 ⇒ 00:16:43.319 Uttam Kumaran: like? Maybe we drive towards this, and we can start to deprecate these.
120 00:16:44.640 ⇒ 00:16:49.310 Uttam Kumaran: We can also try to go look at stuff right now and and see what we’re like seeing in there.
121 00:16:53.890 ⇒ 00:16:54.500 Awaish Kumar: Yeah.
122 00:17:03.160 ⇒ 00:17:05.290 Uttam Kumaran: So like, product bundles.
123 00:17:06.480 ⇒ 00:17:08.589 Uttam Kumaran: Yeah, this is some sort of view.
124 00:17:09.000 ⇒ 00:17:14.659 Uttam Kumaran: So I don’t really know, even like where I don’t know where that’s coming from product mappings.
125 00:17:15.540 ⇒ 00:17:18.639 Uttam Kumaran: Similarly, I’m not exactly sure where it’s coming from.
126 00:17:23.420 ⇒ 00:17:26.999 Uttam Kumaran: So I think we need to see like whether we can just deprecate these
127 00:17:28.760 ⇒ 00:17:29.920 Uttam Kumaran: or what do you want to do.
128 00:17:32.440 ⇒ 00:17:37.539 Awaish Kumar: Yeah, like, we need to like, understand, like, where there these are being used or
129 00:17:38.720 ⇒ 00:17:46.969 Awaish Kumar: like any any metrics, dashboard reports like how they are using it like, even if.
130 00:17:47.700 ⇒ 00:17:53.570 Awaish Kumar: like, maybe even, they are not even like utilizing them somewhere. So.
131 00:17:53.960 ⇒ 00:17:57.040 Uttam Kumaran: Is it easy in Looker to see, because this is their core?
132 00:17:57.490 ⇒ 00:17:59.500 Uttam Kumaran: These are their core dashboards.
133 00:17:59.750 ⇒ 00:18:00.780 Uttam Kumaran: So
134 00:18:01.280 ⇒ 00:18:07.899 Uttam Kumaran: how? How can we see if one of these dashboards are pulling from that? Or how can I look? How can we see the usage actually.
135 00:18:08.040 ⇒ 00:18:10.040 Luke Daque: Click on edit and then
136 00:18:10.510 ⇒ 00:18:18.169 Luke Daque: click on one of the visuals, I guess. Oh, yeah, the at the right side. There’s like the blue button with the search.
137 00:18:19.725 ⇒ 00:18:20.350 Luke Daque: In.
138 00:18:20.350 ⇒ 00:18:20.830 Uttam Kumaran: Oh! Hey!
139 00:18:20.830 ⇒ 00:18:27.310 Luke Daque: Yeah. Oh, yeah. And the click, the like, user summary, for example, click on that blue icon.
140 00:18:27.550 ⇒ 00:18:29.820 Luke Daque: yeah, that one edit data source.
141 00:18:30.020 ⇒ 00:18:32.989 Luke Daque: And then edit connection at the left
142 00:18:33.790 ⇒ 00:18:35.970 Luke Daque: you should be able to see. Oh.
143 00:18:36.070 ⇒ 00:18:37.310 Uttam Kumaran: I see what you mean.
144 00:18:37.920 ⇒ 00:18:40.279 Luke Daque: Yeah, click authorize. I guess you should be able to
145 00:18:40.610 ⇒ 00:18:46.880 Luke Daque: see the like. What? What table? It’s sourcing.
146 00:18:50.490 ⇒ 00:18:53.189 Uttam Kumaran: So yeah, I see, like, there’s some of these.
147 00:18:53.190 ⇒ 00:18:56.380 Luke Daque: Yeah, it looks like, maybe they just named it the same. So like
148 00:18:56.510 ⇒ 00:19:00.260 Luke Daque: product sales. Summary, for example, would be the product sales summary table.
149 00:19:00.260 ⇒ 00:19:05.400 Uttam Kumaran: But is there anywhere in the warehouse to go see? Like if anyone is queried, for example, like.
150 00:19:06.430 ⇒ 00:19:11.040 Uttam Kumaran: how can I see if anyone’s queried this like product. Offerings.
151 00:19:11.740 ⇒ 00:19:13.340 Luke Daque: In bigquery.
152 00:19:13.520 ⇒ 00:19:14.170 Uttam Kumaran: Yeah.
153 00:19:15.548 ⇒ 00:19:18.040 Luke Daque: I guess, in in logs.
154 00:19:18.210 ⇒ 00:19:21.512 Luke Daque: If you go to bigquery, I’m not very sure.
155 00:19:21.880 ⇒ 00:19:25.420 Awaish Kumar: Same like Snowflake. Here also you have information, history.
156 00:19:26.130 ⇒ 00:19:26.779 Uttam Kumaran: Oh, really.
157 00:19:27.060 ⇒ 00:19:32.220 Luke Daque: The job history at the bottom, right? You can like
158 00:19:33.720 ⇒ 00:19:37.850 Uttam Kumaran: Oh, is is there? Is it in the is it in the where, in the warehouse? Here.
159 00:19:40.072 ⇒ 00:19:47.059 Awaish Kumar: No like basically, it’s it needs to be enabled by the admin.
160 00:19:49.540 ⇒ 00:19:51.019 Uttam Kumaran: Oh, where do I go to do that?
161 00:19:52.923 ⇒ 00:19:58.220 Awaish Kumar: Like from like this week.
162 00:19:58.570 ⇒ 00:20:02.020 Awaish Kumar: We cannot do it directly in here. We have to.
163 00:20:03.270 ⇒ 00:20:04.390 Awaish Kumar: Maybe.
164 00:20:05.620 ⇒ 00:20:09.230 Uttam Kumaran: There’s this job. There’s this jobs and monitoring.
165 00:20:10.669 ⇒ 00:20:14.289 Awaish Kumar: Not this one like these. The the jobs which are coming.
166 00:20:14.290 ⇒ 00:20:16.410 Uttam Kumaran: Query. I want to see query, history, basically.
167 00:20:16.900 ⇒ 00:20:17.509 Luke Daque: Go to.
168 00:20:17.510 ⇒ 00:20:17.980 Awaish Kumar: That would be.
169 00:20:17.980 ⇒ 00:20:19.020 Luke Daque: Query again.
170 00:20:20.660 ⇒ 00:20:27.140 Luke Daque: And then at the bottom, there’s job history at the bottom. Yeah, that one at the yeah. Try to open that.
171 00:20:27.330 ⇒ 00:20:33.260 Luke Daque: And yeah, you should be able to see it there. But you have to do some sort of query. Like, yeah, created
172 00:20:34.210 ⇒ 00:20:38.890 Luke Daque: owner. I guess if it’s coming from a service account or something.
173 00:20:46.340 ⇒ 00:20:47.070 Uttam Kumaran: Yeah.
174 00:20:47.230 ⇒ 00:20:50.309 Luke Daque: It’s a good they should need to query. But yeah.
175 00:20:52.970 ⇒ 00:20:57.389 Uttam Kumaran: Okay, yeah, I guess we’ll have to see. I don’t know. Waste. Do you have the instructions on how to do that? Because.
176 00:20:57.610 ⇒ 00:21:01.650 Uttam Kumaran: yeah, I would love if you could just go through and see which ones are not being used.
177 00:21:01.650 ⇒ 00:21:06.080 Awaish Kumar: I will send you, maybe the after after that, like how to do that.
178 00:21:06.530 ⇒ 00:21:15.350 Uttam Kumaran: Okay? Cause? Yeah, we should just let’s just like, cut as much of this as possible or archive it basically. And then we can start to delete the actual
179 00:21:16.664 ⇒ 00:21:20.559 Uttam Kumaran: tables from bigquery. Additionally, there’s like stuff like this, which is like
180 00:21:20.960 ⇒ 00:21:24.700 Uttam Kumaran: this should probably be in a shipment stable or something right like ours to ship
181 00:21:28.000 ⇒ 00:21:31.089 Uttam Kumaran: alright. This looks like order, and like something which is like
182 00:21:32.660 ⇒ 00:21:34.340 Uttam Kumaran: how long it took to ship.
183 00:21:34.730 ⇒ 00:21:35.730 Uttam Kumaran: Basically.
184 00:21:40.700 ⇒ 00:21:44.769 Uttam Kumaran: So do you need me to help organize the stuff around this or like, what do you think.
185 00:21:51.707 ⇒ 00:21:55.609 Awaish Kumar: like we like, I, yeah, I I
186 00:21:56.890 ⇒ 00:22:01.729 Awaish Kumar: I might need the help in like overall design, like
187 00:22:01.870 ⇒ 00:22:07.619 Awaish Kumar: the reviews, or something like how, what kind of tables are being used. And
188 00:22:09.034 ⇒ 00:22:16.950 Awaish Kumar: how we are going to build this march. But yeah, like, in that process, we can brainstorm together.
189 00:22:17.270 ⇒ 00:22:17.690 Uttam Kumaran: Okay.
190 00:22:18.210 ⇒ 00:22:18.730 Awaish Kumar: Yep.
191 00:22:19.600 ⇒ 00:22:23.590 Uttam Kumaran: Okay, great. It would be great if you could leave. But then, yeah, you have as much time for me as you need.
192 00:22:23.740 ⇒ 00:22:27.779 Uttam Kumaran: Let’s ideally like push towards these, to start with.
193 00:22:28.410 ⇒ 00:22:32.279 Awaish Kumar: Like how we want to move it, like the way
194 00:22:32.490 ⇒ 00:22:36.730 Awaish Kumar: we want to move it forward is that we 1st
195 00:22:36.900 ⇒ 00:22:44.600 Awaish Kumar: filter out like, what, how they are using and what what tables are being used frequently, and then
196 00:22:44.910 ⇒ 00:22:52.049 Awaish Kumar: start from there, or just like start from building the Mods from the.
197 00:22:52.050 ⇒ 00:22:56.529 Uttam Kumaran: I would start by basically taking a look at what exists here currently.
198 00:22:57.470 ⇒ 00:23:00.849 Uttam Kumaran: Because these are all the the tables they’re digesting now
199 00:23:01.220 ⇒ 00:23:07.500 Uttam Kumaran: so ideally. If there’s anything related to shipments, we should find it, and then make sure it ends up in one clean shipments table.
200 00:23:07.740 ⇒ 00:23:18.449 Uttam Kumaran: Right? For example, this this logic is, is just one part of shipping, but doesn’t have like any other information, I’m sure, on the shipment like, where is it going? When was it shipped? The status?
201 00:23:19.038 ⇒ 00:23:24.280 Uttam Kumaran: So we should start to say, like anything that goes into there needs to get
202 00:23:24.610 ⇒ 00:23:33.019 Uttam Kumaran: pushed into a shipments model. And then we basically find out which dashboard is pulling from that. And then we should we switch it to pull from shipments.
203 00:23:35.160 ⇒ 00:23:40.500 Awaish Kumar: Okay, so like, basically, you want to start with building marks.
204 00:23:40.880 ⇒ 00:23:43.920 Awaish Kumar: So yes, we start with the sales model.
205 00:23:43.920 ⇒ 00:23:44.290 Uttam Kumaran: Yeah.
206 00:23:44.290 ⇒ 00:23:56.589 Awaish Kumar: And then see what tables in there are. Part can be part of the sales, and just try to move them in there, or maybe remodel them.
207 00:23:57.530 ⇒ 00:24:02.340 Awaish Kumar: and same goes then for marketing and other marks.
208 00:24:03.310 ⇒ 00:24:03.950 Uttam Kumaran: That’s correct.
209 00:24:05.170 ⇒ 00:24:07.350 Awaish Kumar: Yep, okay.
210 00:24:08.110 ⇒ 00:24:09.080 Awaish Kumar: -Oh.
211 00:24:12.440 ⇒ 00:24:13.550 Awaish Kumar: No.
212 00:24:15.630 ⇒ 00:24:18.550 Uttam Kumaran: So yeah, I wanna start with basically with sales and marketing
213 00:24:18.830 ⇒ 00:24:24.400 Uttam Kumaran: on the marketing side, we will need something that’s around each
214 00:24:24.690 ⇒ 00:24:32.270 Uttam Kumaran: sales platform. They’re currently spend on like Google, Facebook, etcetera.
215 00:24:33.220 ⇒ 00:24:41.930 Uttam Kumaran: And then we also want to create like a digital like a basically like digital an ad spend table
216 00:24:42.680 ⇒ 00:24:52.030 Uttam Kumaran: which has ad spend across ads campaigns ad sets fine.
217 00:24:53.560 ⇒ 00:24:56.520 Uttam Kumaran: I don’t know what this budget Pacer is, though
218 00:25:03.080 ⇒ 00:25:06.020 Uttam Kumaran: I have. I have no idea what this is doing.
219 00:25:07.540 ⇒ 00:25:08.310 Awaish Kumar: Hmm, hmm.
220 00:25:18.300 ⇒ 00:25:22.130 Uttam Kumaran: But we should see. Yeah, if you could send me instructions. I’ll get make sure that
221 00:25:22.270 ⇒ 00:25:25.920 Uttam Kumaran: where history ends up there, and we can sort of look how people are using this right now.
222 00:25:31.070 ⇒ 00:25:33.060 Uttam Kumaran: Okay, cool.
223 00:25:33.643 ⇒ 00:25:37.749 Uttam Kumaran: I think maybe let’s talk about a similar thing for Javi. And I think
224 00:25:38.110 ⇒ 00:25:46.759 Uttam Kumaran: overall. I think. Kyle, I think for you. You’ll probably be closer with me on both Joby and with urban stems.
225 00:25:47.020 ⇒ 00:25:51.266 Uttam Kumaran: Urban stems is a client. We kind of just started with today. But we haven’t.
226 00:25:51.670 ⇒ 00:25:56.569 Uttam Kumaran: basically, we’re coming in just for a month to sort of audit their entire system.
227 00:25:56.760 ⇒ 00:26:00.320 Uttam Kumaran: all their sources, all of their models.
228 00:26:01.013 ⇒ 00:26:02.720 Uttam Kumaran: And so I’m sort of meeting with
229 00:26:03.490 ⇒ 00:26:05.849 Uttam Kumaran: a bunch of people from their company this week.
230 00:26:06.409 ⇒ 00:26:18.619 Uttam Kumaran: But I’m gonna add you to the slack channel. And basically, we have a few deliverables that we’re working on for them, which is a data diagram like this, some suggestions on how to improve different processes.
231 00:26:20.080 ⇒ 00:26:23.180 Uttam Kumaran: But ideally, me and you can work together on that this month.
232 00:26:24.125 ⇒ 00:26:26.780 Uttam Kumaran: That way. It’s like we have some redundancy there.
233 00:26:26.950 ⇒ 00:26:32.430 Uttam Kumaran: And then also you can come in and and take some of the work for Javi. I guess a wish like
234 00:26:32.890 ⇒ 00:26:37.259 Uttam Kumaran: I. Originally I had. You were on both Javi and Eden, but it seems like Eden
235 00:26:37.850 ⇒ 00:26:45.520 Uttam Kumaran: requires like a ton of time, right like, how do you feel overall? I kinda wanna make sure that job. You can keep moving forward, so I may bring in.
236 00:26:45.520 ⇒ 00:26:50.690 Awaish Kumar: I I think, like up until now, like I have
237 00:26:51.550 ⇒ 00:26:57.529 Awaish Kumar: covered all the task for for both the clients, and I don’t think there’s anything pending
238 00:26:58.360 ⇒ 00:27:03.360 Awaish Kumar: like you’re lagging behind the but I don’t know how much
239 00:27:04.700 ⇒ 00:27:09.060 Awaish Kumar: More workload will be in the future, but so far it has been manageable.
240 00:27:10.140 ⇒ 00:27:14.629 Uttam Kumaran: Yeah, there’s still a lot of there’s still like a bunch of stuff, I think, on Javi, like.
241 00:27:15.213 ⇒ 00:27:18.649 Awaish Kumar: I’m working with them on cleaning up the product names.
242 00:27:18.810 ⇒ 00:27:22.060 Uttam Kumaran: Like they just it just needs like a a little bit more help.
243 00:27:22.240 ⇒ 00:27:33.209 Uttam Kumaran: I’m sort of thinking of maybe having Kyle help me here as well. And then over time, I think we’ll see like, how much effort both of these need. Eden definitely needs a ton of work. So I’m almost like.
244 00:27:33.570 ⇒ 00:27:38.750 Uttam Kumaran: I think I would like how as much of your time as possible, focus there on building out the mark for them.
245 00:27:39.030 ⇒ 00:27:41.259 Uttam Kumaran: because they have so much stuff they need.
246 00:27:42.305 ⇒ 00:27:42.935 Awaish Kumar: It’s okay.
247 00:27:48.180 ⇒ 00:27:50.850 Awaish Kumar: But what about the product names like?
248 00:27:51.420 ⇒ 00:27:54.659 Awaish Kumar: Is there any other information from them on this.
249 00:27:55.710 ⇒ 00:27:56.880 Uttam Kumaran: For joby.
250 00:28:00.810 ⇒ 00:28:01.760 Uttam Kumaran: Yeah, I mean, in general.
251 00:28:01.760 ⇒ 00:28:02.090 Awaish Kumar: Forward!
252 00:28:02.090 ⇒ 00:28:06.369 Uttam Kumaran: Talking in the slack channel right now about like cleaning up the product names
253 00:28:06.680 ⇒ 00:28:10.450 Uttam Kumaran: and matching order lines to orders.
254 00:28:10.570 ⇒ 00:28:13.640 Uttam Kumaran: like, basically all the work this morning, but like this
255 00:28:13.810 ⇒ 00:28:22.680 Uttam Kumaran: I don’t know. I feel like if you take both of like this. Seems like this is so much work to do. So I, this has to happen like as soon as possible. Basically. So
256 00:28:22.910 ⇒ 00:28:26.179 Uttam Kumaran: I just wanna make sure that you can focus on just the Eden stuff.
257 00:28:26.440 ⇒ 00:28:29.130 Uttam Kumaran: Because we I really want to push this this forward.
258 00:28:29.130 ⇒ 00:28:31.380 Awaish Kumar: Okay, no, no no worries.
259 00:28:35.723 ⇒ 00:28:39.849 Uttam Kumaran: And then, yeah, Kyle, so I think you got a little bit of a of a
260 00:28:39.950 ⇒ 00:28:43.569 Uttam Kumaran: intro into sort of what we’re doing for
261 00:28:43.890 ⇒ 00:28:51.040 Uttam Kumaran: for Javi. But basically, we have a data mart already. This is all stuff that we developed where we have data
262 00:28:51.370 ⇒ 00:28:59.100 Uttam Kumaran: across sales, customer support logistics and some summary tables. Our initial goal right now is just finishing 2 core.
263 00:28:59.220 ⇒ 00:29:01.889 Uttam Kumaran: 2 core
264 00:29:02.441 ⇒ 00:29:16.479 Uttam Kumaran: dashboards. And next we’re moving on to answering some questions around a few other data sources that they have, particularly. If you go here and you go to backlog, you’ll see that there are
265 00:29:16.590 ⇒ 00:29:23.859 Uttam Kumaran: 2 things around gorgeous reporting which is the requirements are coming in as well as
266 00:29:24.410 ⇒ 00:29:27.899 Uttam Kumaran: for Oquendo, I believe. Let me just like double check.
267 00:29:41.380 ⇒ 00:29:49.100 Uttam Kumaran: yeah, I need to find the other one. But basically, we’re working on. We just brought in 2 new data sources, and we’re waiting on
268 00:29:50.360 ⇒ 00:29:57.779 Uttam Kumaran: sort of the business requirements on what we’re gonna answer, for them. I think it’s actually in.
269 00:29:58.495 ⇒ 00:29:58.850 Uttam Kumaran: So
270 00:30:08.820 ⇒ 00:30:17.020 Uttam Kumaran: yeah, so there’s some needs here that are, basically, oh.
271 00:30:19.650 ⇒ 00:30:26.069 Uttam Kumaran: yeah, I may have to find the link. But basically, we got like 5 or 10 questions from them that they want to build a dashboard to answer
272 00:30:26.170 ⇒ 00:30:28.259 Uttam Kumaran: for Oquendo and for gorgeous.
273 00:30:28.410 ⇒ 00:30:32.289 Uttam Kumaran: So basically, we’re gonna have some requirements coming in on modeling for that.
274 00:30:32.480 ⇒ 00:30:34.839 Uttam Kumaran: So I’m gonna make sure that you’re in snowflake
275 00:30:35.312 ⇒ 00:30:38.559 Uttam Kumaran: and then you can take on some of those tickets that may be building
276 00:30:38.770 ⇒ 00:30:46.040 Uttam Kumaran: some models on top of gorgeous data which is gorgeous is their their their review. Platform
277 00:30:46.629 ⇒ 00:30:50.589 Uttam Kumaran: and then, okendo, I kind of forgot what exactly opendo is.
278 00:30:50.920 ⇒ 00:30:54.149 Uttam Kumaran: Yeah, it’s their loy customer loyalty platform.
279 00:30:55.024 ⇒ 00:31:02.410 Uttam Kumaran: So probably nothing very immediate there. But I’m just gonna make sure you’re in Snowflake, and then those tickets are available for you to work on.
280 00:31:04.600 ⇒ 00:31:05.770 Caio: Okay. Cool.
281 00:31:07.714 ⇒ 00:31:13.689 Uttam Kumaran: And there, yeah, you’ll if you go into if you go into Github and you go into Java
282 00:31:14.281 ⇒ 00:31:18.770 Uttam Kumaran: you’ll see the repo here, which basically has the
283 00:31:18.960 ⇒ 00:31:21.690 Uttam Kumaran: all of our core models that we’re working on so definitely.
284 00:31:21.950 ⇒ 00:31:26.359 Uttam Kumaran: You can see if you can be a little bit familiar with all the stuff that we’re doing.
285 00:31:26.690 ⇒ 00:31:27.849 Uttam Kumaran: We’re doing here.
286 00:31:28.429 ⇒ 00:31:35.170 Uttam Kumaran: Also, I I just wanna make sure that you can get your local environment set up and everything. And you can connect and run.
287 00:31:35.670 ⇒ 00:31:41.510 Uttam Kumaran: Run. Dbt, so maybe when you log back in tomorrow, see if you can test that out, and
288 00:31:41.660 ⇒ 00:31:45.149 Uttam Kumaran: that our group is here to help debug anything so.
289 00:31:46.850 ⇒ 00:31:48.339 Caio: Okay, perfect. Go ahead.
290 00:31:48.730 ⇒ 00:31:49.449 Uttam Kumaran: Okay, cool.
291 00:31:50.800 ⇒ 00:32:00.489 Uttam Kumaran: Great, I think. Also, let’s talk about for for stock, Blitz, the diagram is here. Right?
292 00:32:01.060 ⇒ 00:32:04.059 Uttam Kumaran: Is there another diagram? Or is this the existing one?
293 00:32:04.220 ⇒ 00:32:05.120 Uttam Kumaran: Brian.
294 00:32:06.112 ⇒ 00:32:10.429 Luke Daque: That’s still yeah, that’s still in progress. There’s nothing there essentially for now.
295 00:32:10.700 ⇒ 00:32:16.099 Luke Daque: But yeah, I’ll add, the is that I’ll have to like split the ones coming from.
296 00:32:18.090 ⇒ 00:32:23.649 Luke Daque: So portable was it, and the the ones that are direct
297 00:32:24.320 ⇒ 00:32:27.039 Luke Daque: like segment, for example, and stuff like that. So yeah.
298 00:32:27.990 ⇒ 00:32:34.989 Uttam Kumaran: Yeah. So I think here, we want to talk about building out this data mark, we want to have users, subscriptions.
299 00:32:34.990 ⇒ 00:32:35.730 Luke Daque: Solutions.
300 00:32:35.950 ⇒ 00:32:39.269 Uttam Kumaran: Are there any other like, what are? Yeah, what are the core ones here?
301 00:32:39.997 ⇒ 00:32:42.979 Luke Daque: We have organizations as well. And
302 00:32:45.290 ⇒ 00:32:51.670 Luke Daque: yeah, if we, if we just base it on the bulk metrics, there should be sign ups
303 00:32:52.660 ⇒ 00:32:55.380 Luke Daque: sessions and page views.
304 00:32:56.210 ⇒ 00:32:59.370 Uttam Kumaran: Okay. So there’s there’s probably some like 4 events.
305 00:32:59.920 ⇒ 00:33:03.225 Uttam Kumaran: Yeah, sign ups, sessions.
306 00:33:10.280 ⇒ 00:33:12.050 Luke Daque: Yeah, I guess that.
307 00:33:12.050 ⇒ 00:33:14.590 Uttam Kumaran: Create one product events model, probably for that.
308 00:33:16.390 ⇒ 00:33:17.520 Luke Daque: Hmm, yeah.
309 00:33:20.020 ⇒ 00:33:24.650 Uttam Kumaran: And then, yeah, I guess we started looking at.
310 00:33:35.170 ⇒ 00:33:35.920 Uttam Kumaran: Yeah.
311 00:33:43.400 ⇒ 00:33:44.800 Luke Daque: And all of those events.
312 00:33:45.180 ⇒ 00:33:50.020 Uttam Kumaran: Yeah, I want to have yeah, like messages, chats.
313 00:33:50.860 ⇒ 00:33:51.560 Luke Daque: Hmm.
314 00:33:51.940 ⇒ 00:33:53.790 Uttam Kumaran: Tokens, templates.
315 00:33:56.330 ⇒ 00:34:02.615 Luke Daque: I guess I I guess the the mark model would be all events right then there’s just like
316 00:34:03.810 ⇒ 00:34:07.289 Luke Daque: like an event type work or something that’s like, yeah.
317 00:34:07.290 ⇒ 00:34:07.940 Uttam Kumaran: Exactly.
318 00:34:21.190 ⇒ 00:34:24.599 Uttam Kumaran: Yeah. So ideally, we can start to basically build out,
319 00:34:27.900 ⇒ 00:34:30.530 Uttam Kumaran: build out those. How has it been like looking at segment?
320 00:34:31.670 ⇒ 00:34:34.449 Luke Daque: Yeah, it should be. It’s pretty
321 00:34:35.179 ⇒ 00:34:37.650 Luke Daque: fine. There’s just like maybe a couple
322 00:34:38.199 ⇒ 00:34:46.769 Luke Daque: transformations needed for the fields that are in Json format. So maybe just extract the fields from the Json. And then.
323 00:34:46.900 ⇒ 00:34:49.690 Luke Daque: yeah, I guess should be good so far.
324 00:34:54.070 ⇒ 00:34:59.539 Uttam Kumaran: Yeah, I think, I think ideally, we create one.
325 00:34:59.910 ⇒ 00:35:05.320 Uttam Kumaran: We grab one event. And then we basically source out each
326 00:35:06.310 ⇒ 00:35:10.689 Uttam Kumaran: table from the Json, right like, what do you? How are you gonna approach modeling these?
327 00:35:11.550 ⇒ 00:35:14.540 Uttam Kumaran: Are you gonna union at first, st and then sort of rip it out.
328 00:35:15.680 ⇒ 00:35:19.900 Luke Daque: No, I guess we have to have. We have to rip out the Json stuff
329 00:35:20.340 ⇒ 00:35:25.160 Luke Daque: it early on, like, maybe in the staging model or something.
330 00:35:27.420 ⇒ 00:35:29.003 Luke Daque: Yeah, maybe we can even
331 00:35:30.260 ⇒ 00:35:32.179 Luke Daque: yeah, that way. We don’t have to like
332 00:35:32.810 ⇒ 00:35:39.779 Luke Daque: rip it out when the the models already very huge, something like that. But yeah.
333 00:35:58.800 ⇒ 00:36:10.639 Uttam Kumaran: Yeah, ideally. Okay. So I guess, like, if you’re if you start working on this, just send me the code before. And I can like, take a look and review. But ideally, we can get to a March model that like looks like we have all of these basically.
334 00:36:11.500 ⇒ 00:36:17.310 Luke Daque: Right, we should already have the the user subscription and organizations.
335 00:36:18.640 ⇒ 00:36:19.260 Uttam Kumaran: Okay.
336 00:36:19.780 ⇒ 00:36:24.683 Luke Daque: Yeah, we just there’s just a couple of fields that would need Mitch’s
337 00:36:26.690 ⇒ 00:36:34.629 Luke Daque: input like what the logic is or like, how, if there’s any kind of mapping like the one that he provided for the
338 00:36:36.070 ⇒ 00:36:37.110 Luke Daque: subscription.
339 00:36:37.520 ⇒ 00:36:39.830 Luke Daque: Okay, so yeah.
340 00:36:40.620 ⇒ 00:36:42.860 Luke Daque: But as other than that, all the like
341 00:36:43.300 ⇒ 00:36:49.060 Luke Daque: initial fields that are needed that are available should be there in the March model.
342 00:36:54.270 ⇒ 00:37:00.790 Uttam Kumaran: And then, okay, I think that’s perfect, I guess. How do you guys feel about like our naming conventions right now, like
343 00:37:02.690 ⇒ 00:37:10.010 Uttam Kumaran: like, I don’t know, I guess, between Kyle Luke Oish like, how how do we? How should we start to name tables?
344 00:37:10.875 ⇒ 00:37:13.904 Uttam Kumaran: And like, do you guys have any strong opinions on that.
345 00:37:15.560 ⇒ 00:37:20.930 Uttam Kumaran: I mean, we know we talked about dim versus fact tables like, how should we do things?
346 00:37:21.800 ⇒ 00:37:26.469 Uttam Kumaran: Honestly? I just want to try to make sure everything is aligned on at it. For every client, basically.
347 00:37:29.230 ⇒ 00:37:35.430 Awaish Kumar: Kelly for facts. We start, we we are right now also start with facts.
348 00:37:35.810 ⇒ 00:37:39.480 Awaish Kumar: fact, underscore name. And that looks
349 00:37:39.930 ⇒ 00:37:44.179 Awaish Kumar: okay to me. And for the dimensions we can choose, like between
350 00:37:44.620 ⇒ 00:37:47.050 Awaish Kumar: having a dim prefix or not, like.
351 00:37:48.810 ⇒ 00:37:52.480 Uttam Kumaran: Dim products or just products also shows the dimensions.
352 00:37:55.920 ⇒ 00:38:05.970 Uttam Kumaran: So like, let’s take it. Let’s take like orders, for example, like what would you? You would do fact orders. And so and then you would basically have, like dim customer that would join into fact orders.
353 00:38:11.816 ⇒ 00:38:12.929 Awaish Kumar: Yeah. Like.
354 00:38:16.271 ⇒ 00:38:18.020 Uttam Kumaran: Mean for Joby. We have like.
355 00:38:18.830 ⇒ 00:38:21.692 Uttam Kumaran: I guess we do have like fact orders here.
356 00:38:22.360 ⇒ 00:38:26.959 Uttam Kumaran: and then we don’t. But we have like we don’t have. We do have dim customers.
357 00:38:27.260 ⇒ 00:38:32.390 Uttam Kumaran: But there are probably like some product related stuff where we’re just pulling from orders.
358 00:38:32.570 ⇒ 00:38:37.130 Uttam Kumaran: So you’re basically saying we should pull all that out and have like a specific product table.
359 00:38:41.790 ⇒ 00:38:45.270 Awaish Kumar: Yeah, normally, that’s how we do right
360 00:38:45.610 ⇒ 00:38:48.205 Awaish Kumar: in a in a star schema or something
361 00:38:51.840 ⇒ 00:38:53.300 Awaish Kumar: facts. Then
362 00:38:54.730 ⇒ 00:39:03.170 Awaish Kumar: in the final table, like, if we create a summary table or something, then all this will be joined together at the end.
363 00:39:03.170 ⇒ 00:39:06.640 Uttam Kumaran: Like this is, this is a some. This is like a good example of, like a summary table where we’re pulling.
364 00:39:06.640 ⇒ 00:39:07.310 Awaish Kumar: Yeah.
365 00:39:07.760 ⇒ 00:39:10.849 Uttam Kumaran: So what would you? What like? What should we? What would we name this
366 00:39:14.630 ⇒ 00:39:17.959 Uttam Kumaran: like? Underscore summary? Or, yeah, go ahead.
367 00:39:20.860 ⇒ 00:39:27.879 Caio: I was. I was just just gonna say that usually on the intermediate. It’s them, or DM or
368 00:39:28.040 ⇒ 00:39:33.370 Caio: fact, or fc, usually, that’s what I see, and at the end.
369 00:39:34.000 ⇒ 00:39:40.319 Caio: usually people put the underline or at the end as well. But it it makes sense for me.
370 00:39:42.350 ⇒ 00:39:48.460 Uttam Kumaran: So right now, in terms of intermediate, we’re doing like this where it’s like int review attributes.
371 00:39:50.470 ⇒ 00:39:58.120 Uttam Kumaran: So this. But for intermediate, I guess we’re we’re just saying like, okay, it doesn’t matter. But like for the march, let’s do dim fact and then act.
372 00:39:59.060 ⇒ 00:39:59.900 Caio: Okay.
373 00:40:00.620 ⇒ 00:40:04.299 Uttam Kumaran: Is there anything beyond dim fact? And, Ag that we want to do.
374 00:40:06.055 ⇒ 00:40:06.510 Awaish Kumar: No.
375 00:40:07.220 ⇒ 00:40:10.740 Uttam Kumaran: Probably not right, and Ag is just a summary. Tables.
376 00:40:12.660 ⇒ 00:40:13.740 Awaish Kumar: Yes.
377 00:40:18.260 ⇒ 00:40:24.450 Uttam Kumaran: Okay, cool. So I think, Luke, probably for your case, let’s start to make sure that stack Blitz.
378 00:40:24.820 ⇒ 00:40:29.280 Uttam Kumaran: It sort of arranged that way. I’m gonna I’m gonna keep updating.
379 00:40:29.870 ⇒ 00:40:38.790 Uttam Kumaran: This file. This is sort of like, gonna be our Bible for how we structure. Dbt, I’m gonna make sure that this is up to date with
380 00:40:39.090 ⇒ 00:40:47.229 Uttam Kumaran: our decision on, like, okay, we’re gonna do dim fact and for everything in March.
381 00:40:50.090 ⇒ 00:40:51.330 Uttam Kumaran: So that’s perfect.
382 00:40:55.400 ⇒ 00:41:02.059 Uttam Kumaran: I think the only other thing I just pushed is for Javi. I just pushed the 2,
383 00:41:03.240 ⇒ 00:41:06.389 Uttam Kumaran: the 2 dvt. Not for Jabby for
384 00:41:09.313 ⇒ 00:41:12.699 Uttam Kumaran: for for Eden. I just pushed the
385 00:41:14.030 ⇒ 00:41:24.870 Uttam Kumaran: to Github actions. And Kyle, I think I’m we. I may have explained this to you, but basically we’re not using Dbt cloud. We’re using github actions for everything.
386 00:41:25.150 ⇒ 00:41:32.909 Uttam Kumaran: And so we have like, these, these github action workflow files that basically run.
387 00:41:34.890 ⇒ 00:41:39.610 Uttam Kumaran: They’re actually here. If you go into code and you go to Github.
388 00:41:39.890 ⇒ 00:41:46.950 Uttam Kumaran: Workflows run. Dbt, you’ll see that this is actually like our run. Dbt, script.
389 00:41:47.180 ⇒ 00:41:51.080 Uttam Kumaran: And so this get this gets run. So whenever you. You open a pull request
390 00:41:51.930 ⇒ 00:41:58.320 Uttam Kumaran: for example, like this one, it will actually. Well, I guess we didn’t get. We may not have had one since I,
391 00:41:58.650 ⇒ 00:42:13.089 Uttam Kumaran: yeah, okay, here’s a good one. So I just pushed. This is actually in order to make these workflows exist. And you can see it actually has, it actually gets run on. Dbt, so you can make sure that all of our code that we’re getting pushed is valid.
392 00:42:14.600 ⇒ 00:42:18.770 Uttam Kumaran: so this is maybe something. I think later this week or next week, I can kind of walk through with everybody.
393 00:42:18.900 ⇒ 00:42:22.219 Uttam Kumaran: But ideally, everybody’s familiar, like when we push a Pr.
394 00:42:22.370 ⇒ 00:42:26.779 Uttam Kumaran: it’s gonna go into staging, and so we can test, or whatever there and then
395 00:42:26.950 ⇒ 00:42:29.220 Uttam Kumaran: it’ll go into production. Once we merge.
396 00:42:31.800 ⇒ 00:42:36.739 Uttam Kumaran: Did you work with like any sort of development environments before Kyle, or or just like kind of like.
397 00:42:37.250 ⇒ 00:42:38.370 Caio: Oh, not really.
398 00:42:38.520 ⇒ 00:42:39.000 Uttam Kumaran: Okay,
399 00:42:40.750 ⇒ 00:42:46.559 Uttam Kumaran: So basically, the gist is that like, this is, this is Jabby. For example, we have, like.
400 00:42:46.920 ⇒ 00:42:49.624 Uttam Kumaran: we have intermediate.
401 00:42:50.790 ⇒ 00:43:01.959 Uttam Kumaran: don’t worry about this intermediate. We have prod intermediate. We’ll have prod staging and dev intermediate prod staging and dev marts, and then we’ll also have
402 00:43:02.070 ⇒ 00:43:08.159 Uttam Kumaran: just raw that way when we push code. When you’re working locally, you could just run on development
403 00:43:08.330 ⇒ 00:43:18.950 Uttam Kumaran: and you can go quickly, check things when you push a Pr. It’ll get run in staging, and so it’ll run the entire pipeline. And then once you push, once your code gets pushed it’ll run in production.
404 00:43:19.676 ⇒ 00:43:23.819 Uttam Kumaran: That way. It’s like we don’t sort of step over ourselves.
405 00:43:23.930 ⇒ 00:43:34.160 Uttam Kumaran: and it sort of solves like, probably a bunch of those issues where you like. You want to test something. But you want to run the whole pipeline and test it. And we can’t do that unless it’s in production. So we have these like 3 different
406 00:43:34.330 ⇒ 00:43:36.500 Uttam Kumaran: phases that we, we typically do.
407 00:43:37.930 ⇒ 00:43:38.560 Caio: Okay.
408 00:43:41.740 ⇒ 00:43:43.539 Uttam Kumaran: But I think you’ll get the hang of it. So.
409 00:43:44.200 ⇒ 00:43:45.469 Caio: Okay. Don’t worry.
410 00:43:48.670 ⇒ 00:43:56.941 Uttam Kumaran: Okay. So I will make sure that those tickets are ready for you, Kyle. So when you’re up again you can take those, and I’ll make sure you’re in snowflake and stuff.
411 00:43:57.620 ⇒ 00:44:03.880 Uttam Kumaran: and yeah, I think a waste. Just let me know about those your sort of plan for that march, and
412 00:44:04.050 ⇒ 00:44:10.100 Uttam Kumaran: I think, continue to just do any sort of like planning like that in big jam. And then.
413 00:44:10.380 ⇒ 00:44:15.850 Uttam Kumaran: yeah, I think hopefully, we make can make some decisions. And it’s just a lot of like sequel, basically, for the next few weeks.
414 00:44:16.950 ⇒ 00:44:17.970 Caio: Okay, perfect.
415 00:44:17.970 ⇒ 00:44:18.470 Awaish Kumar: Okay.
416 00:44:19.160 ⇒ 00:44:19.899 Caio: Sounds good.
417 00:44:21.360 ⇒ 00:44:22.110 Uttam Kumaran: Okay.
418 00:44:22.240 ⇒ 00:44:29.820 Uttam Kumaran: Awesome. Well, thanks, guys, good 1st meeting. And yeah, I’ll probably put I’ll put time on for for us to chat sometime earlier in the day.
419 00:44:30.212 ⇒ 00:44:36.210 Uttam Kumaran: But yeah, I’m excited that we finally have a squad together. We can keep moving on stuff on the Dbt side. So.
420 00:44:36.810 ⇒ 00:44:43.060 Luke Daque: And tomorrow, before we have a tech dips call, I’ll I’ll add Kyo. There.
421 00:44:43.600 ⇒ 00:44:44.979 Uttam Kumaran: Yeah, that that’d be perfect.
422 00:44:45.180 ⇒ 00:44:45.890 Caio: Thank you.
423 00:44:46.210 ⇒ 00:44:47.060 Luke Daque: That’s good.
424 00:44:48.790 ⇒ 00:44:50.900 Uttam Kumaran: Okay, thanks. Guys, appreciate it.
425 00:44:50.900 ⇒ 00:44:52.630 Caio: Thank you. Thank you. Bye-bye.