Meeting Title: Analytics Engineering Daily Sync Date: 2025-02-27 Meeting participants: Luke Daque, Uttam Kumaran, Awaish Kumar, Caio Velasco
WEBVTT
1 00:00:20.220 ⇒ 00:00:22.059 Caio Velasco: Hey, Luke, how’s it going.
2 00:00:23.140 ⇒ 00:00:25.630 Luke Daque: Hi, Kyle, how’s it going? I’m doing well.
3 00:00:26.060 ⇒ 00:00:26.960 Luke Daque: How are you.
4 00:00:27.707 ⇒ 00:00:31.980 Caio Velasco: Do this. Now I’m going with.
5 00:00:33.980 ⇒ 00:00:34.490 Awaish Kumar: Hello!
6 00:00:35.830 ⇒ 00:00:36.350 Caio Velasco: But.
7 00:00:36.350 ⇒ 00:00:38.020 Luke Daque: How’s everything on your end?
8 00:00:41.690 ⇒ 00:00:43.590 Caio Velasco: I am learning a lot.
9 00:00:44.790 ⇒ 00:00:45.920 Luke Daque: Nice.
10 00:00:46.880 ⇒ 00:00:50.599 Caio Velasco: Yeah, I’m feeling that I’m really learning a lot that’s good.
11 00:00:55.270 ⇒ 00:00:57.419 Luke Daque: How about you? A wish? How’s how’s it going.
12 00:00:58.557 ⇒ 00:01:01.359 Awaish Kumar: It’s all good. How about you?
13 00:01:02.250 ⇒ 00:01:03.699 Luke Daque: Yeah, all good as well.
14 00:01:05.170 ⇒ 00:01:08.090 Luke Daque: Like, are you loving what you’re doing here so far.
15 00:01:09.200 ⇒ 00:01:14.259 Awaish Kumar: Yeah, it’s it’s so it’s been amazing. I
16 00:01:14.490 ⇒ 00:01:18.740 Awaish Kumar: have have had chance to work with a lot of different kinds of data.
17 00:01:19.170 ⇒ 00:01:20.380 Luke Daque: Yeah, nice.
18 00:01:20.550 ⇒ 00:01:20.895 Awaish Kumar: Yeah.
19 00:02:08.820 ⇒ 00:02:09.550 Uttam Kumaran: Hey, guys.
20 00:02:12.400 ⇒ 00:02:13.830 Luke Daque: Hey? Whatam! How’s it going.
21 00:02:13.830 ⇒ 00:02:14.450 Uttam Kumaran: Hi.
22 00:02:15.950 ⇒ 00:02:16.820 Caio Velasco: Morning.
23 00:02:17.320 ⇒ 00:02:26.750 Uttam Kumaran: Cool. Sorry about the delay. You guys can start without me. I’m gonna be the I’m gonna be the I’m gonna win the late to class award all the time.
24 00:02:27.310 ⇒ 00:02:28.600 Uttam Kumaran: Sorry about that.
25 00:02:29.410 ⇒ 00:02:31.779 Luke Daque: Like, what time is it on your end? At the moment.
26 00:02:31.780 ⇒ 00:02:40.869 Uttam Kumaran: It’s 8. It’s 8 30. But I have a meeting before this and that sometimes goes over. And then I try to just like, get a coffee and get ready.
27 00:02:41.020 ⇒ 00:02:59.500 Uttam Kumaran: And then, like, Yeah, I because you know, we’re we’re global. I I think mornings is gonna be the best for us to get everything done. But like, I’m preparing for other stuff like, basically, I meet with the AI team we meet as a team, and then we go through client work?
28 00:03:00.084 ⇒ 00:03:05.400 Uttam Kumaran: And so I try. I basically just stack 4 meetings back to back. And then
29 00:03:05.600 ⇒ 00:03:09.320 Uttam Kumaran: we have another thing after this for a few hours. And then.
30 00:03:09.780 ⇒ 00:03:14.720 Uttam Kumaran: yeah, just like the day, just like starts immediately. So
31 00:03:15.300 ⇒ 00:03:19.602 Uttam Kumaran: I think yesterday I wasn’t up too late working
32 00:03:20.320 ⇒ 00:03:25.180 Uttam Kumaran: But yeah, that’s fine. Yeah, I guess today, I really just wanted to.
33 00:03:25.430 ⇒ 00:03:26.596 Uttam Kumaran: I think,
34 00:03:27.400 ⇒ 00:03:55.520 Uttam Kumaran: maybe, Kyle, it’s best we go through. We could go through your Pr as a group. Maybe and maybe we can go through the same review process. I think probably Luke and Away shall probably look to you guys for feedback, since I think this will be helpful to kind of have people with outside context. And then, yeah, I guess we can just talk about work. In flight. Around. What’s next for the 3 clients we’re working on. So.
35 00:03:57.220 ⇒ 00:03:59.269 Caio Velasco: Perfect, confirm sound, good.
36 00:04:00.440 ⇒ 00:04:05.520 Luke Daque: There’s also like I have Prs and stafflets. Maybe we can also like, look and look into those.
37 00:04:05.850 ⇒ 00:04:10.490 Uttam Kumaran: Okay, yeah. So yeah, whoever wants to go first, st you can just share screen, and then we can just walk through.
38 00:04:11.118 ⇒ 00:04:12.910 Uttam Kumaran: Can just walk through those.
39 00:04:14.100 ⇒ 00:04:17.230 Luke Daque: Sure maybe Cayo can can go first.st
40 00:04:17.800 ⇒ 00:04:18.560 Uttam Kumaran: And then back.
41 00:04:20.089 ⇒ 00:04:23.119 Caio Velasco: Should I share my screen from Github.
42 00:04:23.580 ⇒ 00:04:24.370 Uttam Kumaran: Yes.
43 00:04:28.250 ⇒ 00:04:29.100 Caio Velasco: Finally.
44 00:04:44.000 ⇒ 00:04:51.960 Caio Velasco: there’s always like the button is green and big that I always can’t find. But I just found okay.
45 00:04:52.170 ⇒ 00:04:55.249 Uttam Kumaran: Yeah, I feel like zoom is the best out of everything. But
46 00:04:55.750 ⇒ 00:05:02.369 Uttam Kumaran: I don’t know. It’s like, we only need like 3 features. They have like a hundred features. I can’t like. Turn the other ones off.
47 00:05:05.350 ⇒ 00:05:07.969 Caio Velasco: Okay, let me share my screen and let’s.
48 00:05:07.970 ⇒ 00:05:12.120 Luke Daque: The slack one like the slack huddle.
49 00:05:12.790 ⇒ 00:05:15.520 Luke Daque: It’s also pretty good, because, like you can.
50 00:05:15.650 ⇒ 00:05:18.249 Luke Daque: like, everybody can like annotate or something.
51 00:05:18.930 ⇒ 00:05:20.924 Uttam Kumaran: Slack is good, but then I lose
52 00:05:21.350 ⇒ 00:05:28.919 Uttam Kumaran: I lose a recording. And then I I wanna also go into slack. And then sometimes my computer sort of freezes up. I don’t know.
53 00:05:29.320 ⇒ 00:05:29.770 Luke Daque: Yeah.
54 00:05:32.500 ⇒ 00:05:33.650 Caio Velasco: And I see.
55 00:05:34.290 ⇒ 00:05:35.020 Uttam Kumaran: Yes.
56 00:05:35.170 ⇒ 00:05:36.150 Caio Velasco: Yeah, perfect.
57 00:05:37.880 ⇒ 00:05:46.279 Caio Velasco: If it’s too loud. Here, let me know, because there’s always happening. There’s a lot of wind, and it’s raining a lot. So there’s a lot of noise. But just I mean.
58 00:06:01.490 ⇒ 00:06:02.730 Caio Velasco: gonna have you.
59 00:06:03.430 ⇒ 00:06:07.490 Uttam Kumaran: Yeah, I mean, we could. Yeah, we could just probably walk through your pr,
60 00:06:08.360 ⇒ 00:06:10.669 Uttam Kumaran: but there’s no there’s no noise. We hear you fine.
61 00:06:10.880 ⇒ 00:06:12.550 Caio Velasco: Okay, okay.
62 00:06:14.690 ⇒ 00:06:16.809 Caio Velasco: Where? Where did you first? st
63 00:06:27.410 ⇒ 00:06:30.600 Caio Velasco: Where to start? Here in the description.
64 00:06:32.310 ⇒ 00:06:36.099 Uttam Kumaran: Yeah, I think, imagine, like, I mean for for for a wish.
65 00:06:36.220 ⇒ 00:06:39.209 Uttam Kumaran: And, Luke, they have probably very little context.
66 00:06:39.360 ⇒ 00:06:48.749 Uttam Kumaran: So I think it would be helpful. Yeah, to talk through just like what the goal was. And then maybe we can just walk through the core file that that you created. That would probably be best.
67 00:06:50.440 ⇒ 00:06:51.710 Caio Velasco: Okay, perfect.
68 00:06:52.580 ⇒ 00:06:56.529 Caio Velasco: So I think, though, this was the ticket.
69 00:06:56.640 ⇒ 00:07:04.730 Caio Velasco: So we we wanna build a dashboard or their operational things, as far as I understand
70 00:07:06.280 ⇒ 00:07:10.490 Caio Velasco: And basically they have some questions here
71 00:07:10.600 ⇒ 00:07:16.239 Caio Velasco: which macros are being used. The most which ticket fields are being used. The most which agents
72 00:07:16.520 ⇒ 00:07:28.279 Caio Velasco: interact with the market the most. And then another one related to subscription and refunds. That I understood comes from another source that I haven’t budgeted yet with charts.
73 00:07:29.600 ⇒ 00:07:34.710 Caio Velasco: So yes, initially. I started working.
74 00:07:35.371 ⇒ 00:07:49.219 Caio Velasco: Well, I was talking with some people to see how this could be done best, and then, understanding where should I put the the models? If it’s in March or something else? I don’t have a lot of experience, and it’s still learning on the job.
75 00:07:49.914 ⇒ 00:07:55.315 Caio Velasco: But at the end of the day I understand that you have to look at the whole picture
76 00:07:55.850 ⇒ 00:08:16.220 Caio Velasco: and well, if you have agents like some. Let’s say some categories, right? Like some entities like you have the agents you have. Macros, you have tickets, you have messages. You have many different things. So at the end of the day you are looking at everything at the same time. But you have to start from somewhere because you don’t know that a priori
77 00:08:16.848 ⇒ 00:08:25.740 Caio Velasco: and and then I think that well, if you wanna understand something related to Macros, you need the macros
78 00:08:26.288 ⇒ 00:08:31.730 Caio Velasco: which is just with well, Macro details. Macro is basically an automated response.
79 00:08:33.120 ⇒ 00:09:02.769 Caio Velasco: and that’s that. If you, if a customer opens a ticket this customer might be asking XY or Z, and then I think a macro would be used to just automate that reply. That’s basically what a macro is. Initially, there is a big tickets a table which is usually like tickets from customer support when they open a ticket to solve something. And I thought that if you go to the tickets
80 00:09:03.120 ⇒ 00:09:04.759 Caio Velasco: tables, for example.
81 00:09:05.516 ⇒ 00:09:23.589 Caio Velasco: Here, I think I I thought that I I was gonna be able to find the macros here and then just get the information. But for some reason the macros are not coming in the tickets. Table they’re coming in the message table. There’s a Mac
82 00:09:24.570 ⇒ 00:09:30.690 Caio Velasco: here, there’s a micro Json that can have one or multiple macros associated to a message.
83 00:09:30.870 ⇒ 00:09:39.489 Caio Velasco: and then I have to understand that messages come within tickets and many tickets can have many macros. So it’s a
84 00:09:39.630 ⇒ 00:09:43.130 Caio Velasco: many to many. Relationship. May. Yes.
85 00:09:43.870 ⇒ 00:09:48.200 Caio Velasco: many tickets can have many messages, many message can have many micros, something like that.
86 00:09:48.941 ⇒ 00:09:59.249 Caio Velasco: And then the and then well, we have also a micro source here. So it’s usually it’s it’s easy to get the dim macros from this one.
87 00:09:59.430 ⇒ 00:10:07.460 Caio Velasco: And it’s also interesting that there is a My
88 00:10:08.450 ⇒ 00:10:34.859 Caio Velasco: sorry, a usage column for the Macro stable, which, when I was going on the gorgeous website documentation to understand about it. Apparently whenever a macro is used in a ticket, there is only 89 unique macros. Just so that, you know. So anytime a macro is used, apparently the accounting and updating this table on their end.
89 00:10:35.400 ⇒ 00:10:42.189 Caio Velasco: But when I was using the macros from the messages which you can see
90 00:10:47.680 ⇒ 00:10:50.200 Caio Velasco: here. Here, see
91 00:10:50.420 ⇒ 00:10:58.139 Caio Velasco: this. In this case there are 2 macros for this for the same message. When I was counting the macros from the messages
92 00:10:58.795 ⇒ 00:11:02.120 Caio Velasco: the number was different from the usage. One
93 00:11:03.898 ⇒ 00:11:06.840 Uttam Kumaran: Is the back and forth.
94 00:11:10.870 ⇒ 00:11:14.980 Caio Velasco: Good question the messages. Let me see if I can.
95 00:11:15.150 ⇒ 00:11:16.040 Caio Velasco: Good question
96 00:11:17.963 ⇒ 00:11:24.316 Caio Velasco: A ticket message represents a message in the mail under normalized format. They’re outgoing, incoming internal notes.
97 00:11:24.690 ⇒ 00:11:25.170 Uttam Kumaran: Yeah.
98 00:11:25.170 ⇒ 00:11:34.350 Caio Velasco: So, yeah, so yeah. So that was the 1st thing that I was not completely sure.
99 00:11:36.035 ⇒ 00:11:39.729 Caio Velasco: So my mind got like this? Okay?
100 00:11:40.303 ⇒ 00:12:00.000 Caio Velasco: So then, I was also like, why is the number different? So let let’s assume that in the in the message table I could count all macros, and it is different from the user number in the macros table. But then I assume that well, maybe we are not getting all the messages. Then I count. Then I try to find the minimal date for the messages
101 00:12:00.535 ⇒ 00:12:09.389 Caio Velasco: and compared to the minimum date across all date columns, because I didn’t know what column means, what? So I just tried to get the minimum from from all of them.
102 00:12:10.021 ⇒ 00:12:17.359 Caio Velasco: And then I saw that. Yes, Macros is much more. There’s much more ears previously to messages.
103 00:12:17.550 ⇒ 00:12:25.780 Caio Velasco: So we are getting there’s this seems to be more information in Macros which makes sense. That’s why the usage is large, it’s it’s greater.
104 00:12:25.890 ⇒ 00:12:29.100 Caio Velasco: But then, when I was well back to the
105 00:12:30.760 ⇒ 00:12:37.780 Caio Velasco: is it this? Yes, back here. So then I start with the D Macros, and then
106 00:12:38.920 ⇒ 00:13:02.139 Caio Velasco: I was trying to get this, the the exploding, the Macro J. Zone into the message tables. And then I build a fact message. Macro stables just way be able to link both. And then there was also, after I understood that we also had to understand the agent interaction with all this, I pulled also an agent information from the message table.
107 00:13:03.025 ⇒ 00:13:10.199 Caio Velasco: Yeah. So at the end of the day, and then I hear with the aggregate the tables. That’s the thing. I don’t know if we need them.
108 00:13:10.813 ⇒ 00:13:23.490 Caio Velasco: Because I assume that analysts can join whatever they need here. But I also did this just to be sure that we can find the information that they would need at the end of the day to answer
109 00:13:23.650 ⇒ 00:13:25.709 Caio Velasco: these questions, which were.
110 00:13:25.710 ⇒ 00:13:29.173 Uttam Kumaran: So I guess, like, yeah, my question here would be
111 00:13:30.440 ⇒ 00:13:37.999 Uttam Kumaran: what’s the difference between the macro usage ag and the agent? Macro? Usage ag like can’t, are they? Do they have like duplicate
112 00:13:38.640 ⇒ 00:13:42.640 Uttam Kumaran: things in them and like, can you find the same answer in in both.
113 00:13:44.560 ⇒ 00:13:46.680 Caio Velasco: Questions. Let me.
114 00:13:46.680 ⇒ 00:13:48.750 Uttam Kumaran: Cause, like, yeah, like, for example.
115 00:13:48.750 ⇒ 00:13:50.020 Caio Velasco: I believe you can.
116 00:13:50.020 ⇒ 00:13:55.609 Uttam Kumaran: If it’s all macros by all agents, then you they can just sum the macro usage from there right.
117 00:13:59.580 ⇒ 00:14:01.940 Caio Velasco: You get from the agents. Yes, I.
118 00:14:01.940 ⇒ 00:14:03.599 Uttam Kumaran: Call. Left. Joins. Yeah.
119 00:14:03.600 ⇒ 00:14:06.990 Caio Velasco: Yes, exactly. And the macro usage.
120 00:14:07.440 ⇒ 00:14:09.861 Uttam Kumaran: Yeah. So maybe we just keep just keep the
121 00:14:10.380 ⇒ 00:14:13.380 Uttam Kumaran: just keep the agent Macro usage ag.
122 00:14:13.920 ⇒ 00:14:15.160 Caio Velasco: Yes, the mark will be.
123 00:14:15.160 ⇒ 00:14:20.059 Uttam Kumaran: And then that would be my thing. And then, I guess, like question for maybe awaii
124 00:14:20.180 ⇒ 00:14:24.340 Uttam Kumaran: like, what what do we think about like naming conventions for summary tables.
125 00:14:26.790 ⇒ 00:14:33.240 Awaish Kumar: Yeah, I was also thinking about this, I think, somebody looks
126 00:14:34.030 ⇒ 00:14:39.199 Awaish Kumar: like, at least for me. I was like more inclined towards writing it as a summary table.
127 00:14:43.130 ⇒ 00:14:44.699 Awaish Kumar: because it’s a summary of.
128 00:14:45.760 ⇒ 00:14:48.420 Uttam Kumaran: Just change ag to summary basically.
129 00:14:49.085 ⇒ 00:14:49.570 Awaish Kumar: Yes.
130 00:14:50.010 ⇒ 00:14:50.640 Uttam Kumaran: Text.
131 00:14:51.420 ⇒ 00:14:55.480 Caio Velasco: So this would be macro usage summary at the end. For some.
132 00:14:56.270 ⇒ 00:14:57.979 Uttam Kumaran: Yeah, just summary is fine.
133 00:14:58.270 ⇒ 00:15:03.639 Uttam Kumaran: I think summary is fine again, I think for us, we know dim fact analysts again. They’re gonna
134 00:15:04.020 ⇒ 00:15:07.159 Uttam Kumaran: they’re not gonna really get it. I think Summary is perfect.
135 00:15:08.360 ⇒ 00:15:11.169 Caio Velasco: Okay, and you prefer to put at the end or in the beginning.
136 00:15:11.710 ⇒ 00:15:13.109 Uttam Kumaran: At the at the end.
137 00:15:14.325 ⇒ 00:15:16.810 Caio Velasco: Okay, okay, cool.
138 00:15:17.100 ⇒ 00:15:29.239 Caio Velasco: And and then I I believe there are others, things that we can definitely do better in ticket fields and like how we’re gonna explode into different columns because no like ticket fields, they just so that you guys know
139 00:15:30.940 ⇒ 00:15:31.550 Caio Velasco: for example.
140 00:15:31.550 ⇒ 00:15:31.990 Awaish Kumar: Hi.
141 00:15:31.990 ⇒ 00:15:33.960 Caio Velasco: If you go to ticket details.
142 00:15:37.210 ⇒ 00:15:42.609 Caio Velasco: Everything is stored. Basically, in this column, it’s like a huge with, it’s crazy.
143 00:15:42.610 ⇒ 00:15:43.210 Uttam Kumaran: Yeah.
144 00:15:43.730 ⇒ 00:15:48.010 Awaish Kumar: Yeah, actually, I. I did also some work on this.
145 00:15:48.120 ⇒ 00:15:56.380 Awaish Kumar: And I like, there is some queries you can find in Java where I’m trying to get the data from these nested fields and all.
146 00:15:57.826 ⇒ 00:16:00.179 Caio Velasco: Think I took a look as well.
147 00:16:00.781 ⇒ 00:16:11.369 Caio Velasco: So. And then, I think, said something on the email about of separating to columns like 2 or 3 columns just to get different. I didn’t know exactly what he meant, but I think.
148 00:16:11.370 ⇒ 00:16:12.030 Uttam Kumaran: Yeah, that’s the
149 00:16:12.030 ⇒ 00:16:16.610 Uttam Kumaran: analysts they’re gonna tell. See, this is the thing. It’s gonna be a balance. They’re gonna say, bring everything in.
150 00:16:17.030 ⇒ 00:16:29.230 Uttam Kumaran: I’m gonna say, don’t bring anything in until they tell us. So we’re gonna be in the middle, right? So bring some stuff that we know. And then, now that you know how to bring new stuff in. If they need it, we’ll continue to. We’ll bring it in. So that’s it like
151 00:16:29.835 ⇒ 00:16:44.164 Uttam Kumaran: cause. Otherwise they’re gonna make. They’re just gonna keep being like, oh, I could use this. I could maybe use this like, I don’t wanna get into that rabbit hole so ideally. I think the way it is. I think it’s perfect. We can just ship it. If you just make those 2 changes, I’ll
152 00:16:44.897 ⇒ 00:16:46.329 Awaish Kumar: I have one.
153 00:16:46.780 ⇒ 00:16:47.380 Uttam Kumaran: Yeah, yeah, yeah.
154 00:16:47.665 ⇒ 00:16:50.240 Awaish Kumar: I’ve been looking at the PR.
155 00:16:50.480 ⇒ 00:16:58.875 Awaish Kumar: And I found like I I see this there are. There’s a table called Dim Ticket fields and
156 00:17:01.750 ⇒ 00:17:12.249 Awaish Kumar: and like there is no way to join it with tickets so like there’s no ticket id in this table. So how like, what, what, how we are going to use this table.
157 00:17:15.659 ⇒ 00:17:17.029 Caio Velasco: Good good catch.
158 00:17:17.159 ⇒ 00:17:19.879 Caio Velasco: Yes, I should put that ticket building for sure.
159 00:17:21.040 ⇒ 00:17:21.569 Awaish Kumar: And.
160 00:17:21.579 ⇒ 00:17:28.859 Uttam Kumaran: Either. Yeah, either. The field needs. Yeah, yeah, you’re totally right. Yeah. There has to be some other key somewhere. A foreign key for that.
161 00:17:29.410 ⇒ 00:17:35.139 Awaish Kumar: Like this table. Actually, there is a table called Fact Ticket Fields. And
162 00:17:35.690 ⇒ 00:17:42.900 Awaish Kumar: it is doing same thing as the dem ticket fields in in a in a cte
163 00:17:43.586 ⇒ 00:17:48.589 Awaish Kumar: and kind of you can just reuse that table in there.
164 00:17:50.020 ⇒ 00:17:57.430 Caio Velasco: In here. Yes, yes, you could select from that one perfect. Yeah. Yeah. So then I would add the ticket from there here.
165 00:17:57.650 ⇒ 00:18:03.459 Caio Velasco: Oh, here the ticket is so I think I probably did there, and I forgot to do in the other place no nice, perfect.
166 00:18:04.040 ⇒ 00:18:05.689 Caio Velasco: and one other comment
167 00:18:06.151 ⇒ 00:18:16.629 Caio Velasco: then. Well, also, what do you you said you thought yesterday was also was really helpful, because that’s exactly my point that I was thinking from yesterday to today that at the end of the day
168 00:18:16.790 ⇒ 00:18:21.186 Caio Velasco: we did, we had to do. Not we have to do. But we should do
169 00:18:21.970 ⇒ 00:18:35.709 Caio Velasco: the minimum sufficient they need, so that we don’t get into that rabbit road. You said it makes total sense now that I have a better picture that makes no sense for me to lose time into opening 30, Jason, if they don’t even know if they need it.
170 00:18:36.260 ⇒ 00:18:55.010 Uttam Kumaran: You’re you’re gonna find that that’s gonna be the case, and also, like, I’m gonna push on them for the same thing which is like, I want to. Let’s see the business case. And basically, this just helps us like, prevent tech debt from building up from the start, you know. And there’s other ways for us to find this like.
171 00:18:55.090 ⇒ 00:19:14.429 Uttam Kumaran: you know, I’m we’ll think about a tech debt process where we we scan what columns are being accessed, and we do that. But I think this is a good process. There’s going to be times where it’s just like I don’t know. Bring bring what you think is fair, there’s going to be times where it’s really clear. So yeah, this is a I think you went through almost the full exercise here.
172 00:19:15.900 ⇒ 00:19:21.899 Caio Velasco: And just 2 more things. First, st do you guys have any preference, or even
173 00:19:22.030 ⇒ 00:19:24.990 Caio Velasco: something that you could teach me in terms of?
174 00:19:25.150 ⇒ 00:19:39.390 Caio Velasco: Should this all these tables be here. Should them, some of them be inside in. I know that you can send me the the thing from Dbt. I I read it, but at the end of the day it seems that it’s a mayor of defining to be honest.
175 00:19:39.390 ⇒ 00:19:40.520 Uttam Kumaran: Yes,
176 00:19:42.650 ⇒ 00:19:45.820 Uttam Kumaran: I don’t know. So what is fact? Messages new.
177 00:19:47.480 ⇒ 00:19:47.930 Awaish Kumar: And.
178 00:19:47.930 ⇒ 00:19:50.340 Caio Velasco: It’s the.
179 00:19:51.970 ⇒ 00:19:54.760 Awaish Kumar: When, like we have to do lot of
180 00:19:55.470 ⇒ 00:20:07.940 Awaish Kumar: calculated fields. And then when we have to make lot of cities to to actually create effect table. I think, then, we have to move towards creating intermediate tables where we
181 00:20:08.100 ⇒ 00:20:13.680 Awaish Kumar: put those calculated fields in there, and just just reuse that in the fact table.
182 00:20:14.600 ⇒ 00:20:22.880 Luke Daque: Yeah, I was about to say the same thing like one like one of the ctes earlier, like the ticket field values or something
183 00:20:24.086 ⇒ 00:20:27.700 Luke Daque: since it’s being the fact ticket fields.
184 00:20:28.303 ⇒ 00:20:36.530 Luke Daque: Since it’s being used to more than one March model, and maybe we’ll be even using it again. Maybe we can put
185 00:20:37.210 ⇒ 00:20:43.060 Luke Daque: that to. I think when I worked on this before, I I created the raw
186 00:20:44.300 ⇒ 00:20:53.300 Luke Daque: folder which was just like extracting the Json fields, I believe I’m not sure if it’s still there. Yeah, like that. So we can.
187 00:20:54.270 ⇒ 00:20:56.569 Luke Daque: Yeah, like, like, shopify order line.
188 00:20:56.570 ⇒ 00:21:01.070 Uttam Kumaran: You can move the new, any of the underscore new. You can probably move it here.
189 00:21:02.110 ⇒ 00:21:07.870 Luke Daque: Yeah, that’s we can like reuse it in the March models any.
190 00:21:08.500 ⇒ 00:21:09.569 Caio Velasco: Like any.
191 00:21:09.570 ⇒ 00:21:28.119 Uttam Kumaran: Any time, any any file where you’re basically like selecting from source. And there’s like, Very, there’s maybe like changing column types or like string string changes, you can throw that in raw. And then, yeah, anytime, where you have a file with like a large number of Ctes.
192 00:21:28.660 ⇒ 00:21:33.140 Uttam Kumaran: or really complicated logic that can go into int.
193 00:21:33.250 ⇒ 00:21:34.060 Uttam Kumaran: In this case.
194 00:21:34.060 ⇒ 00:21:34.550 Caio Velasco: It’s.
195 00:21:34.550 ⇒ 00:21:36.189 Uttam Kumaran: I don’t know. Maybe there isn’t
196 00:21:36.550 ⇒ 00:21:47.819 Uttam Kumaran: anything with a serious amount of that. I definitely think you can move the fact messages new and fact message and fact tickets new. You can move that to raw as like raw.
197 00:21:48.775 ⇒ 00:21:51.940 Uttam Kumaran: Yeah, you can probably move those for sure.
198 00:21:52.590 ⇒ 00:21:55.100 Caio Velasco: Yeah, yeah, those are just because.
199 00:21:55.670 ⇒ 00:21:56.390 Luke Daque: Yeah. Yeah.
200 00:21:56.390 ⇒ 00:21:57.080 Luke Daque: Go ahead.
201 00:21:58.240 ⇒ 00:22:09.329 Caio Velasco: I was. I was just gonna say that they, they’re just new because I had to rename it because someone else did this, and I don’t know if we’re gonna keep this, or or some or different, because this one
202 00:22:09.640 ⇒ 00:22:13.770 Caio Velasco: comes from gorgeous. I think I’m not. Oh, no, portable gorgeous as well.
203 00:22:13.770 ⇒ 00:22:20.219 Uttam Kumaran: Yeah, so I would just I would just ditch all that like as part of your Pr, I would just probably delete those.
204 00:22:20.750 ⇒ 00:22:23.979 Caio Velasco: The issue, and the old ones here.
205 00:22:24.300 ⇒ 00:22:30.740 Uttam Kumaran: Yeah, unless you’re unless you’re using them. I don’t think there’s anybody using them right now.
206 00:22:32.540 ⇒ 00:22:35.410 Caio Velasco: Okay, okay, perfect. So I’ll delete this and the others.
207 00:22:37.230 ⇒ 00:22:43.750 Luke Daque: Yeah, I was just saying, like, any being anytime like, we’re just extracting fields from a Json or a variant
208 00:22:44.400 ⇒ 00:22:50.580 Luke Daque: from the source. Maybe we can put that in raw, because we might be like reusing that several times
209 00:22:50.690 ⇒ 00:22:53.480 Luke Daque: in several models.
210 00:22:55.050 ⇒ 00:23:00.952 Caio Velasco: Cool. Okay, so I’ll take notes from all of this, and then I’ll update also on on notion
211 00:23:01.480 ⇒ 00:23:09.920 Caio Velasco: that. We had all this idea over there and then. Last last question is, now that we approach it like demon sex. Here
212 00:23:10.600 ⇒ 00:23:15.100 Caio Velasco: is there, I mean, I don’t know if we should lose time with this. But should we draw
213 00:23:15.310 ⇒ 00:23:21.690 Caio Velasco: like an entity relationship diagram, something somewhere, or we don’t need this now.
214 00:23:22.140 ⇒ 00:23:35.108 Uttam Kumaran: I don’t think it’s necessary now I think we will. We will have it. We’ll have it in the documentation, both in like the spreadsheets, which is like, what are the primary keys?
215 00:23:35.990 ⇒ 00:23:43.050 Uttam Kumaran: But then I think we we will eventually move. We’ll start to document Martz as well in fig jam. But I think, for now it’s okay.
216 00:23:43.790 ⇒ 00:23:48.159 Caio Velasco: Perfect. Okay, I’ll make the changes. Thank you. Guys appreciate.
217 00:23:48.160 ⇒ 00:23:56.690 Uttam Kumaran: Okay? Great. So yeah, just to recap. So it’s the summary. It’s moving stuff to raw. It’s consolidating just the 2 fact tables.
218 00:24:01.010 ⇒ 00:24:07.940 Awaish Kumar: I think, like we do. We even need this fact ticket fields, because I don’t see any metrics in there
219 00:24:08.560 ⇒ 00:24:10.949 Awaish Kumar: like it can be just a dimension table.
220 00:24:12.070 ⇒ 00:24:16.329 Awaish Kumar: Okay, it can be a dim ticket fields.
221 00:24:16.790 ⇒ 00:24:22.480 Uttam Kumaran: Yeah, dim ticket fields. Cause, yeah, fact is, like, if there are, if there’s like a time series, basically.
222 00:24:23.010 ⇒ 00:24:23.550 Luke Daque: And.
223 00:24:27.420 ⇒ 00:24:31.160 Uttam Kumaran: I think, wishes right. We could just have this be dim ticket fields.
224 00:24:34.690 ⇒ 00:24:36.270 Caio Velasco: Okay, cool. Bye.
225 00:24:39.300 ⇒ 00:24:41.980 Caio Velasco: Okay, I’ll stop sharing.
226 00:24:46.660 ⇒ 00:24:47.649 Luke Daque: Okay, maybe I can.
227 00:24:47.650 ⇒ 00:24:49.080 Uttam Kumaran: Yeah. Go ahead.
228 00:24:49.080 ⇒ 00:24:54.760 Luke Daque: Well, we have 5 min, but maybe let me just share what I have, for
229 00:24:56.340 ⇒ 00:25:04.199 Luke Daque: I got 2 Prs open for stats, so the 1st one is adding the fact events. So I already changed this. This I was like
230 00:25:04.990 ⇒ 00:25:13.990 Luke Daque: the naming convention. But basically what what I did here was, there’s we’re getting the events tables from segment
231 00:25:14.910 ⇒ 00:25:27.550 Luke Daque: for staff bits. And then, basically, I consolidated all the events for the chat events, message events.
232 00:25:28.240 ⇒ 00:25:31.260 Luke Daque: I created intermediate models, for, like
233 00:25:31.800 ⇒ 00:25:39.920 Luke Daque: each group of events like chat events. For example, when a chat created event chat, canceled event.
234 00:25:40.700 ⇒ 00:25:46.490 Uttam Kumaran: Chat mode used there. So every all the chat related events, and then message related events.
235 00:25:47.753 ⇒ 00:25:51.690 Luke Daque: Template and token events. And then I created the final
236 00:25:51.890 ⇒ 00:25:57.790 Luke Daque: pack Events model, which is just a consolidation of all all of these in models.
237 00:25:58.675 ⇒ 00:26:02.370 Luke Daque: So we have, like one like
238 00:26:02.800 ⇒ 00:26:06.180 Luke Daque: 1 1 big events model for for this.
239 00:26:07.050 ⇒ 00:26:07.680 Uttam Kumaran: Okay.
240 00:26:07.680 ⇒ 00:26:14.610 Luke Daque: And other changes I did was, yeah, I should have separated this, I guess, ideally, in a separate Pr
241 00:26:15.327 ⇒ 00:26:18.449 Luke Daque: but these are the updates that I did for
242 00:26:18.760 ⇒ 00:26:29.859 Luke Daque: the other March models based on the discussion we had with last time where where he clarified the calculated fields and some of the dimensions that
243 00:26:30.040 ⇒ 00:26:38.700 Luke Daque: where where the sources are coming from for these like for fact. If subscriptions, for example, where the organizations created up
244 00:26:38.970 ⇒ 00:26:40.780 Luke Daque: need that other coming from?
245 00:26:42.140 ⇒ 00:26:44.160 Luke Daque: Yeah. Like how to determine
246 00:26:44.770 ⇒ 00:26:47.230 Luke Daque: the organization. Id and stuff like that.
247 00:26:47.460 ⇒ 00:26:51.020 Luke Daque: So same with the organ organization side.
248 00:26:56.510 ⇒ 00:27:02.170 Luke Daque: wait, I think this is incorrect. Oh, yeah, this is events.
249 00:27:04.110 ⇒ 00:27:07.069 Luke Daque: Yeah, it’s same with with the users and
250 00:27:09.540 ⇒ 00:27:13.729 Luke Daque: organizations, basically just based on the discussion
251 00:27:13.880 ⇒ 00:27:16.820 Luke Daque: we had with mitch for those calculated fields.
252 00:27:17.400 ⇒ 00:27:18.050 Uttam Kumaran: Okay.
253 00:27:22.030 ⇒ 00:27:26.470 Uttam Kumaran: okay, what do you think is best? For the review.
254 00:27:32.360 ⇒ 00:27:35.840 Luke Daque: I’m not sure at this point.
255 00:27:37.210 ⇒ 00:27:41.459 Luke Daque: like, Are you asking like, what what needs to be reviewed here? I guess.
256 00:27:46.660 ⇒ 00:27:50.939 Uttam Kumaran: okay, I can just go. I can go through all of it. Yeah. Sorry. I didn’t know how many files support.
257 00:27:51.920 ⇒ 00:27:52.329 Luke Daque: Yeah, I guess.
258 00:27:52.330 ⇒ 00:27:57.100 Uttam Kumaran: Is there any? Is there any particular file with like a lot of logic that you want to?
259 00:27:58.017 ⇒ 00:28:00.590 Uttam Kumaran: Really get like a thorough review on.
260 00:28:00.740 ⇒ 00:28:05.370 Uttam Kumaran: I would also like I mean, I don’t know. I think it’s helpful for all 4 of us to sort of go in and look at
261 00:28:06.130 ⇒ 00:28:09.620 Uttam Kumaran: Prs maybe I can see
262 00:28:10.570 ⇒ 00:28:19.620 Uttam Kumaran: I can see like I I’m just not sure if I can add everyone to every repo. But let me see if I can try to make that happen. I do think it’s helpful to have everyone.
263 00:28:19.620 ⇒ 00:28:19.990 Luke Daque: Look at that!
264 00:28:19.990 ⇒ 00:28:20.530 Uttam Kumaran: Nice.
265 00:28:20.700 ⇒ 00:28:27.290 Luke Daque: And maybe maybe you’ll have everyone do reviews instead of just maybe you doing all the reviews. It’s.
266 00:28:28.920 ⇒ 00:28:31.429 Uttam Kumaran: Yeah, basically, I think, yeah, exactly.
267 00:28:31.840 ⇒ 00:28:32.550 Luke Daque: Yeah.
268 00:28:35.870 ⇒ 00:28:40.960 Luke Daque: sure, I I think for the logic. I think it should be pretty simple for
269 00:28:41.170 ⇒ 00:28:50.159 Luke Daque: for this. Pr, I don’t think there’s any complicated ones, but the other Pr. Might be a bit more complicated. That was the Mrr.
270 00:28:51.080 ⇒ 00:28:52.260 Luke Daque: A. Pr.
271 00:28:52.690 ⇒ 00:28:58.259 Luke Daque: So this is creating a subscription overview. I’m I’m I’m not sure if this is the best
272 00:28:58.690 ⇒ 00:29:01.110 Luke Daque: name for this model.
273 00:29:01.270 ⇒ 00:29:05.090 Luke Daque: Basically, it’s the Mrr. And Arr, and.
274 00:29:05.090 ⇒ 00:29:06.930 Uttam Kumaran: Why don’t we just do fax subscriptions.
275 00:29:09.480 ⇒ 00:29:16.340 Luke Daque: So, yeah, I already had that on the previous Pr, which is the both metrics subscriptions.
276 00:29:16.730 ⇒ 00:29:17.480 Luke Daque: So maybe I can.
277 00:29:17.480 ⇒ 00:29:17.990 Uttam Kumaran: Oh, okay.
278 00:29:17.990 ⇒ 00:29:18.840 Luke Daque: Diminishing.
279 00:29:19.400 ⇒ 00:29:21.419 Luke Daque: Yeah, yeah, maybe that’s.
280 00:29:21.420 ⇒ 00:29:21.869 Uttam Kumaran: Maybe I can.
281 00:29:21.870 ⇒ 00:29:22.670 Luke Daque: They mentioned
282 00:29:23.090 ⇒ 00:29:29.380 Luke Daque: the other one, and then this one would be fact table, because it’s it’s a date spine with
283 00:29:30.680 ⇒ 00:29:36.649 Luke Daque: yeah, with the subscription id customer Id and the Mrr. Arr, as well as like
284 00:29:38.830 ⇒ 00:29:48.600 Luke Daque: like a Boolean value, for whether it’s new or it was cancelled, canceled, Mrr, and yeah, annualized
285 00:29:49.540 ⇒ 00:29:51.030 Luke Daque: revenue as well
286 00:29:56.060 ⇒ 00:29:57.469 Luke Daque: remaining revenue.
287 00:29:58.850 ⇒ 00:30:01.229 Luke Daque: So yeah, maybe I might. I need
288 00:30:01.500 ⇒ 00:30:03.880 Luke Daque: somebody to double check the logic
289 00:30:04.070 ⇒ 00:30:06.630 Luke Daque: that I have in here. And
290 00:30:07.250 ⇒ 00:30:10.090 Luke Daque: yeah, that’s that’s basically it. I might.
291 00:30:10.090 ⇒ 00:30:10.620 Uttam Kumaran: Okay.
292 00:30:10.620 ⇒ 00:30:15.060 Luke Daque: Create another Pr for a real dashboard out of this model.
293 00:30:16.230 ⇒ 00:30:16.760 Luke Daque: Yeah.
294 00:30:18.920 ⇒ 00:30:19.690 Uttam Kumaran: Okay? Awesome.
295 00:30:20.310 ⇒ 00:30:23.150 Uttam Kumaran: Okay? And then I think, on my side.
296 00:30:23.794 ⇒ 00:30:28.335 Uttam Kumaran: yeah, I pushed a Pr for Javi on
297 00:30:29.480 ⇒ 00:30:37.660 Uttam Kumaran: product lists and and getting product categories in I think, probably moving forward. I’ll start to tag
298 00:30:38.217 ⇒ 00:30:41.399 Uttam Kumaran: maybe me, I think on Javi. I think
299 00:30:41.580 ⇒ 00:30:49.759 Uttam Kumaran: between Kyle and me we can take. I think, Eden a wish. I’ll just make sure that you approve any Prs, and then stack Blitz just tag me
300 00:30:50.400 ⇒ 00:30:54.030 Uttam Kumaran: I ideally we have one
301 00:30:55.670 ⇒ 00:31:05.454 Uttam Kumaran: one ae, and then one reviewer on every single account. So I think that’s a perfect strategy. And I think maybe, as we go further on Javi, I think a wish. I’ll
302 00:31:06.540 ⇒ 00:31:14.170 Uttam Kumaran: I’ll see if you can start to take over a little bit for reviews. Is there anything on Eden that we need to chat about. I know we have the next meeting. But
303 00:31:14.390 ⇒ 00:31:17.479 Uttam Kumaran: just as this group, if there’s anything we need to talk about, there.
304 00:31:17.680 ⇒ 00:31:21.730 Awaish Kumar: No, there’s nothing.
305 00:31:21.730 ⇒ 00:31:23.230 Uttam Kumaran: Okay. Okay.
306 00:31:24.940 ⇒ 00:31:38.446 Uttam Kumaran: okay, cool. Alright. I think this is good meeting. I think tomorrow we will do our we’ll do our retro. So I think we’ll have some ability to talk through a little bit of our process and things like that. But yeah, I guess
307 00:31:39.288 ⇒ 00:31:43.399 Uttam Kumaran: let me know if there’s any questions. And yeah, I guess we’ll talk in the next 2 meetings.
308 00:31:45.160 ⇒ 00:31:46.619 Caio Velasco: Thank you. Thank you. Appreciate.
309 00:31:46.620 ⇒ 00:31:48.269 Uttam Kumaran: Okay, thanks guys.
310 00:31:48.530 ⇒ 00:31:49.270 Luke Daque: Thanks.