Meeting Title: Testing-Doc-Review Date: 2024-07-09 Meeting participants: Brian Pei, Ryan Luke Daque, Nicolas Sucari, Uttam Kumaran
WEBVTT
1 00:03:55.820 ⇒ 00:03:56.620 Nicolas Sucari: Hey, Ryan.
2 00:03:59.290 ⇒ 00:04:00.240 Ryan Luke Daque: Hi, Nicholas.
3 00:04:01.960 ⇒ 00:04:03.290 Ryan Luke Daque: how’s everything?
4 00:04:05.230 ⇒ 00:04:06.130 Nicolas Sucari: Oh, dude!
5 00:04:06.300 ⇒ 00:04:07.369 Nicolas Sucari: A bunch here!
6 00:04:07.960 ⇒ 00:04:09.759 Ryan Luke Daque: Yeah, everything’s doing. Well.
7 00:04:41.460 ⇒ 00:04:43.690 Nicolas Sucari: Hey? About that soft umphia?
8 00:04:44.250 ⇒ 00:04:45.049 Ryan Luke Daque: Yeah, 9.
9 00:04:45.810 ⇒ 00:04:47.149 Nicolas Sucari: What we need is
10 00:04:47.280 ⇒ 00:04:53.179 Nicolas Sucari: really keep there out out for smoke. But I don’t know what’s going on with her
11 00:04:53.320 ⇒ 00:04:54.680 Nicolas Sucari: access there.
12 00:04:55.000 ⇒ 00:04:58.849 Nicolas Sucari: Probably she just need to create like a new key.
13 00:04:59.270 ⇒ 00:04:59.710 Ryan Luke Daque: Yeah.
14 00:05:00.150 ⇒ 00:05:13.009 Nicolas Sucari: Shared that you would, Patrick, but I don’t know what is the issue is having, unless I don’t have windows on Patrick either. Like if you can click on that, one would be super useful.
15 00:05:13.230 ⇒ 00:05:14.430 Nicolas Sucari: Thank you for that.
16 00:05:14.670 ⇒ 00:05:16.049 Ryan Luke Daque: Sure. I’ll
17 00:05:16.150 ⇒ 00:05:19.660 Ryan Luke Daque: try to capture tomorrow like early
18 00:05:20.020 ⇒ 00:05:22.379 Ryan Luke Daque: correct tomorrow. But I think she’s like.
19 00:05:22.700 ⇒ 00:05:23.530 Ryan Luke Daque: early. Yeah.
20 00:05:23.530 ⇒ 00:05:24.059 Nicolas Sucari: Yeah, yeah.
21 00:05:24.386 ⇒ 00:05:25.039 Ryan Luke Daque: And yeah.
22 00:05:26.410 ⇒ 00:05:29.809 Nicolas Sucari: Yeah, it was too late today to figure that out with that.
23 00:05:30.560 ⇒ 00:05:32.129 Ryan Luke Daque: Yeah. Cool. No problem.
24 00:05:33.380 ⇒ 00:05:34.530 Ryan Luke Daque: Hey? Brian?
25 00:05:35.090 ⇒ 00:05:35.980 Brian Pei: What’s up?
26 00:05:36.220 ⇒ 00:05:36.950 Ryan Luke Daque: How’s it going.
27 00:05:36.950 ⇒ 00:05:37.610 Nicolas Sucari: And right.
28 00:05:38.710 ⇒ 00:05:43.749 Brian Pei: Oh, man, my camera sucks it’s good. I just went for a run, so I’m.
29 00:05:43.890 ⇒ 00:05:45.050 Ryan Luke Daque: Oh, cool!
30 00:05:45.050 ⇒ 00:05:45.750 Brian Pei: Tanked up.
31 00:05:46.280 ⇒ 00:05:47.420 Ryan Luke Daque: Nice
32 00:05:49.260 ⇒ 00:05:49.820 Ryan Luke Daque: Eric!
33 00:05:49.820 ⇒ 00:05:54.262 Nicolas Sucari: I think it’s like, I know. There you go. Wow! What a change.
34 00:05:55.130 ⇒ 00:05:59.422 Brian Pei: Yeah. Stopped using my finger for once, since she’s an actual napkin.
35 00:06:03.580 ⇒ 00:06:04.180 Nicolas Sucari: Buddy.
36 00:06:07.470 ⇒ 00:06:08.169 Brian Pei: No man.
37 00:06:15.100 ⇒ 00:06:17.210 Nicolas Sucari: Let me ask the team if he’s joining.
38 00:06:17.420 ⇒ 00:06:19.200 Nicolas Sucari: If not, we can start.
39 00:06:19.630 ⇒ 00:06:20.449 Ryan Luke Daque: Yeah, sure.
40 00:06:20.880 ⇒ 00:06:21.460 Brian Pei: Sure.
41 00:06:59.090 ⇒ 00:07:06.219 Nicolas Sucari: The idea of this meeting is to go through everything about testing and see how we can set evidence in all of the projects.
42 00:07:06.440 ⇒ 00:07:07.780 Nicolas Sucari: do actual
43 00:07:07.920 ⇒ 00:07:11.300 Nicolas Sucari: something that when we receive a alarm
44 00:07:11.390 ⇒ 00:07:13.759 Nicolas Sucari: we can do something about that right.
45 00:07:14.510 ⇒ 00:07:15.150 Ryan Luke Daque: Right.
46 00:07:15.770 ⇒ 00:07:16.480 Brian Pei: Yep.
47 00:07:17.270 ⇒ 00:07:17.890 Nicolas Sucari: Go ahead.
48 00:07:19.060 ⇒ 00:07:24.269 Nicolas Sucari: Were you able to take a look at what was what we had?
49 00:07:25.320 ⇒ 00:07:27.409 Nicolas Sucari: elementary Brian.
50 00:07:29.060 ⇒ 00:07:35.260 Brian Pei: Yeah, I’m gonna wait to see if we Tom can join. I have a document prepared. I’ll share my screen.
51 00:07:37.780 ⇒ 00:07:38.410 Nicolas Sucari: Great.
52 00:07:40.360 ⇒ 00:07:40.820 Brian Pei: Probably do.
53 00:07:40.820 ⇒ 00:07:41.239 Nicolas Sucari: One in 10.
54 00:07:41.240 ⇒ 00:07:45.740 Brian Pei: I think a lot of it is directed at him. A lot of it. Our opinions directed at him.
55 00:07:46.620 ⇒ 00:07:47.820 Brian Pei: Yeah, so we’ll see.
56 00:07:48.690 ⇒ 00:07:50.910 Nicolas Sucari: I know he’s with a bunch of stuff.
57 00:07:51.790 ⇒ 00:07:54.580 Nicolas Sucari: He’s with a lot of things on the sales side.
58 00:07:54.979 ⇒ 00:07:58.210 Nicolas Sucari: But yeah, if we need him, we should probably.
59 00:07:58.570 ⇒ 00:07:59.230 Brian Pei: Yeah.
60 00:08:00.050 ⇒ 00:08:02.629 Brian Pei: I think we give him a couple of minutes. I looked at his calendar.
61 00:08:04.410 ⇒ 00:08:07.060 Brian Pei: Might be might be open. We’ll see.
62 00:08:07.560 ⇒ 00:08:08.970 Brian Pei: I’ll
63 00:08:10.350 ⇒ 00:08:12.429 Brian Pei: see. It’s 2 30
64 00:08:23.860 ⇒ 00:08:29.520 Brian Pei: I highly recommend. I just. I got this a couple of weeks ago. It’s a notebook that has
65 00:08:29.760 ⇒ 00:08:31.119 Brian Pei: time of the day. Every.
66 00:08:31.120 ⇒ 00:08:31.890 Ryan Luke Daque: Oh, wow.
67 00:08:32.360 ⇒ 00:08:34.120 Ryan Luke Daque: yeah, that’s that’s cool.
68 00:08:34.539 ⇒ 00:08:37.699 Brian Pei: I used to just have like either
69 00:08:37.889 ⇒ 00:08:42.029 Brian Pei: notes on Mac or sticky notes.
70 00:08:42.079 ⇒ 00:08:44.859 Brian Pei: and I think I also have add.
71 00:08:44.879 ⇒ 00:08:48.469 Brian Pei: so I just like forget about all the notes, and I just like
72 00:08:48.989 ⇒ 00:08:50.689 Brian Pei: I don’t know. I lose track of time.
73 00:08:51.529 ⇒ 00:08:54.999 Brian Pei: And so with this, I try to plan out every hour of my day.
74 00:08:55.872 ⇒ 00:08:58.169 Brian Pei: So that I can actually like
75 00:08:58.599 ⇒ 00:08:59.919 Brian Pei: get shit done.
76 00:08:59.920 ⇒ 00:09:03.244 Ryan Luke Daque: Yeah, like old school. Writing always works right?
77 00:09:04.770 ⇒ 00:09:08.910 Brian Pei: I, it’s so interesting. How different it is to old school writing than like.
78 00:09:09.540 ⇒ 00:09:17.412 Brian Pei: I’ve tried everything technology wise like reminders on my phone alerts to, you know, switch over to do something else
79 00:09:18.060 ⇒ 00:09:21.829 Brian Pei: and nothing. Nothing has worked except for going back to pen and paper.
80 00:09:21.830 ⇒ 00:09:26.039 Ryan Luke Daque: Yeah. Still, the best way, like, I guess, like writing it like
81 00:09:26.270 ⇒ 00:09:27.450 Ryan Luke Daque: probably
82 00:09:27.550 ⇒ 00:09:30.530 Ryan Luke Daque: embeds it to your brain or something. I don’t know.
83 00:09:31.450 ⇒ 00:09:46.469 Brian Pei: I think it is. Yeah. And then having something physical, it’s just like, every time I’m confused of like, Wait, what do I have to do today. I could at least pick up this Nope notebook and be like, Oh, yeah, I have to do Xyz, I even. And I have I. And I write down like, Take a break, go on a run
84 00:09:47.290 ⇒ 00:09:49.729 Brian Pei: like don’t freak out about life.
85 00:09:49.730 ⇒ 00:09:50.440 Nicolas Sucari: I think
86 00:09:50.630 ⇒ 00:09:53.490 Nicolas Sucari: when you write it down like things
87 00:09:53.760 ⇒ 00:09:59.760 Nicolas Sucari: you probably don’t forgot to match about those things, because you’re writing it down means that you’re like
88 00:10:00.070 ⇒ 00:10:03.230 Nicolas Sucari: even your attention to write that down. Probably you keep.
89 00:10:03.300 ⇒ 00:10:05.690 Nicolas Sucari: That’s for later, too.
90 00:10:05.720 ⇒ 00:10:09.959 Nicolas Sucari: instead of forgetting all of the time when you rode on slack, or any other place.
91 00:10:11.210 ⇒ 00:10:16.799 Brian Pei: Yeah, if I type it out in slack cause I sometimes I use slack like I send messages to myself.
92 00:10:16.940 ⇒ 00:10:18.290 Nicolas Sucari: Yeah, I use it a lot.
93 00:10:18.500 ⇒ 00:10:21.189 Brian Pei: I just won’t read it. I just won’t read it. I’ll forget
94 00:10:21.260 ⇒ 00:10:23.420 Brian Pei: if I type it down in notepad. I’m gonna forget.
95 00:10:25.600 ⇒ 00:10:26.970 Brian Pei: No gotta be something new.
96 00:10:29.750 ⇒ 00:10:32.069 Brian Pei: oh, Boo! Tom said, he’ll join so.
97 00:10:32.300 ⇒ 00:10:32.690 Ryan Luke Daque: Cool.
98 00:10:32.690 ⇒ 00:10:34.029 Brian Pei: Give him some
99 00:10:34.400 ⇒ 00:10:37.720 Brian Pei: full time.
100 00:10:42.550 ⇒ 00:10:45.060 Brian Pei: This quit this.
101 00:10:59.790 ⇒ 00:11:02.060 Brian Pei: if that makes sense.
102 00:11:02.060 ⇒ 00:11:03.179 Uttam Kumaran: Yes. Sorry.
103 00:11:04.413 ⇒ 00:11:05.660 Uttam Kumaran: Slick them
104 00:11:06.410 ⇒ 00:11:08.030 Uttam Kumaran: wild meeting.
105 00:11:09.640 ⇒ 00:11:11.747 Uttam Kumaran: Okay. Let’s
106 00:11:12.450 ⇒ 00:11:13.559 Brian Pei: Oh, we can go.
107 00:11:14.030 ⇒ 00:11:15.180 Uttam Kumaran: Yes, let’s go.
108 00:11:15.530 ⇒ 00:11:15.974 Brian Pei: Alright.
109 00:11:16.890 ⇒ 00:11:18.640 Brian Pei: Go! I’m going in
110 00:11:19.100 ⇒ 00:11:30.690 Brian Pei: alright. I’m gonna read this almost word, word for word. It is an extension of notion. I just I like doing it on my own 1st before it goes into notion, because sometimes I get colorful with my language.
111 00:11:31.417 ⇒ 00:11:37.960 Brian Pei: Yeah, and it’s meant to also kind of like be a discussion as well.
112 00:11:39.640 ⇒ 00:11:40.820 Brian Pei: so
113 00:11:40.980 ⇒ 00:11:42.430 Brian Pei: please feel free.
114 00:11:42.480 ⇒ 00:11:45.934 Brian Pei: especially you, Tom, to to let me know what you think this is.
115 00:11:46.300 ⇒ 00:12:12.264 Brian Pei: Opinions by me may not be shared by Brain Forge. I think a lot of this is what I think of things, and I’m happy to get pushback. But let me go through. I I created sections, for I basically, I watched a 30 min tutorial on elementary. I looked through what we have for elementary. I looked through what the alerts look like in our slack channel, and I tried to do
116 00:12:12.840 ⇒ 00:12:22.520 Brian Pei: a more taking taking a step back and kind of like writing out what I think about the whole thing and how to make it most effective. So let me start
117 00:12:23.008 ⇒ 00:12:33.590 Brian Pei: after deep diving is all the features that elementary provides, as well as how full parts go, is currently using elementary test. I’ve come to following conclusions. It’s felt like I was writing an an essay in college
118 00:12:33.970 ⇒ 00:12:47.479 Brian Pei: number one. The more quantity of tests the less important the alerts become. Think everyone agrees with this. Test. Alerts become a nuisance of every column. Oh, also, like U. Tom, you have some of this in your doc, too, but I think I I might have added.
119 00:12:47.480 ⇒ 00:12:48.320 Uttam Kumaran: Yeah, yeah, no. Problem.
120 00:12:48.320 ⇒ 00:12:48.980 Brian Pei: Back to it.
121 00:12:49.531 ⇒ 00:13:15.910 Brian Pei: Test. Alerts become a nuisance if every column has a basic Dbt test on it. This makes future actual model breaking tests less impactful. I wrote Boy who cries Wolf situation, because even for me, like, when I 1st started, I I saw the alerts at first, st and I was like, Oh, and I was like tagging people. I was like, do we have to worry about this? Do we have to worry about this, and some of them are. No, some of them are. Yes, but then, recently, because we’re getting like 2 pings a day in that channel, nobody
122 00:13:15.950 ⇒ 00:13:24.108 Brian Pei: responds to them. And so when something does actually break, I will probably miss it because I get like 3 messages a day in that slack channel. Nobody wants that
123 00:13:24.811 ⇒ 00:13:41.690 Brian Pei: and then under that I wrote, especially if, since we’re a client based business, we need to hand this off at some point in the future. And again, this is my opinion. I recommend no tests with the severity being less than high, and each elementary test other than full model failure or sequel. Error like that’s the
124 00:13:41.690 ⇒ 00:13:54.117 Brian Pei: the main one is like, if there’s a sequel error. Obviously, yes. Being reviewed as absolutely necessary to resolve as soon as the Alert hits the slack channel. And I’ll talk about Sla’s and other stuff there, but just from a high level
125 00:13:54.700 ⇒ 00:14:16.900 Brian Pei: I I have been at places where, you know, there’s a there’s a Dbt unique and not null test on every single column, but it floods the the slack channel and our emails with this column is not null. And then nobody really looks at it. And then, you know, when something actually comes, it’s game breaking it it gets. There’s too much noise in there.
126 00:14:17.447 ⇒ 00:14:25.149 Brian Pei: Another potentially hurtful opinion. This is again my opinion. I personally have not found the ui useful
127 00:14:25.421 ⇒ 00:14:47.969 Brian Pei: in fact, the more product ui, that I feel like I need to interact with on a daily basis the angrier I I am as a person. This is true. I am in Snowflake a lot. I think Snowflake is where I get a lot of stuff done. And so for elementary. What I found is the most useful is this table dbt, elementary dbt. Run results instead of going to the ui and seeing like pie charts and stuff which
128 00:14:47.970 ⇒ 00:14:56.359 Brian Pei: I do say later, like it could be cool for clients to see that it’s a nice feature to have. But me as an engineer I,
129 00:14:57.120 ⇒ 00:15:06.509 Brian Pei: there is so much going on. That this for me. Le, let me look at all of the run results where the message doesn’t say success in it.
130 00:15:06.620 ⇒ 00:15:22.800 Brian Pei: and I can. I can get a lot of work done just looking at this just looking at the model name. What the message was when it when it ran, and this is a this is a test. Fail. But over here it has.
131 00:15:23.568 ⇒ 00:15:27.079 Brian Pei: Wait! Let me look at a different one.
132 00:15:28.070 ⇒ 00:15:30.969 Brian Pei: Let me look at at a real failure
133 00:15:32.380 ⇒ 00:15:35.150 Brian Pei: where there’s compiled. Okay?
134 00:15:35.280 ⇒ 00:15:42.349 Brian Pei: Yeah. So if there’s a real failure in here. That’s not a test failure. The compiled code is here, too. So this is great. I don’t.
135 00:15:42.350 ⇒ 00:15:44.479 Uttam Kumaran: What does root failure? I mean, what does that mean?
136 00:15:45.760 ⇒ 00:15:46.569 Brian Pei: What is, what.
137 00:15:46.940 ⇒ 00:15:48.740 Uttam Kumaran: What does real failure mean?
138 00:15:49.200 ⇒ 00:15:52.970 Brian Pei: Oh, sorry as in it’s it’s a test.
139 00:15:54.600 ⇒ 00:16:02.009 Brian Pei: Actually, this shouldn’t be test a real failure to me means it. It wasn’t a test. It wasn’t a column level test. It’s like the sequel was broken.
140 00:16:02.220 ⇒ 00:16:06.088 Brian Pei: But I guess here, I guess there are sequential level tests
141 00:16:06.950 ⇒ 00:16:13.710 Brian Pei: hard failures. That aren’t warnings which to me a warning is the yellow severity.
142 00:16:14.141 ⇒ 00:16:33.028 Brian Pei: But when something fails that is like a hard failure, meaning like the model didn’t run. It’s not a message that was like, Hey, everything ran. But this column has dupes. It’s like, Hey, this does. This didn’t actually run. And here’s the code. That didn’t run. This was really nice, because I’m already in Snowflake.
143 00:16:33.360 ⇒ 00:16:51.650 Brian Pei: I can look at the last couple of failures. I can probably do more filtering on it. And make a view out of it. And then I can copy paste this code in Snowflake. I don’t have to go into virtual environment. I don’t have to go to Dpd cloud. I can copy, paste this, and then I can run it here and figure out, oh, this sorry, this compiled code. This was a
144 00:16:52.360 ⇒ 00:16:58.590 Brian Pei: see extra sequel. Dbt, test, right? But if there’s a
145 00:16:58.610 ⇒ 00:17:04.619 Brian Pei: let me do a resource type and resource type, not in test should be like
146 00:17:07.930 ⇒ 00:17:13.643 Brian Pei: something actually broke. Okay, database error. Okay, so this is a model
147 00:17:15.170 ⇒ 00:17:18.830 Brian Pei: failure. This is what I meant more. The model
148 00:17:19.460 ⇒ 00:17:21.930 Brian Pei: did not run. There’s an error in the model.
149 00:17:22.446 ⇒ 00:17:26.810 Brian Pei: I can copy paste the compiled code of the entire model.
150 00:17:28.670 ⇒ 00:17:31.019 Brian Pei: And I assume if I run this
151 00:17:32.160 ⇒ 00:17:35.420 Brian Pei: yep, and it’s gonna be like, oh, here’s the here’s the error.
152 00:17:35.430 ⇒ 00:17:37.609 Brian Pei: Very quickly I can go.
153 00:17:37.700 ⇒ 00:17:58.349 Brian Pei: I can copy paste the compiled code in Snowflake. I’m already there. I know it’s a failure, and then I can find what was broken and then change it in Github. 30 seconds don’t need to go to the Ui don’t need to do anything crazy. I like that a lot. That’s probably the coolest thing that elementary has shown me. And yeah, look at how much compiled code is here. That’s great love, that
154 00:17:59.880 ⇒ 00:18:01.880 Brian Pei: cool. So
155 00:18:02.420 ⇒ 00:18:04.010 Brian Pei: let me do this, too.
156 00:18:04.810 ⇒ 00:18:06.250 Brian Pei: And
157 00:18:06.800 ⇒ 00:18:09.529 Brian Pei: what was that called resource, type.
158 00:18:09.630 ⇒ 00:18:11.109 Brian Pei: resource, type
159 00:18:11.300 ⇒ 00:18:12.820 Brian Pei: and test
160 00:18:14.400 ⇒ 00:18:40.857 Brian Pei: alright, an engineer’s full time. Job should not be checking a ui or a dashboard every morning to make sure things look like they are running, especially if no alerts or errors were triggered the day before. I don’t want people to get into a habit of checking, just because something is pretty to look at every morning. Save your time. If it’s not broken, it’s fine. Elementary should be. Something is broken. I need to do this in 3 h or less, or I don’t hit the sla it’s cool. I like the Ui. We like the documentation. You don’t have to check it every single day
161 00:18:41.190 ⇒ 00:18:54.610 Brian Pei: let the alerts be the reason I should check the ui. But if I get 2 alerts a day, I I’m over explaining, but it lessens the value of the whole reason to go to the Ui. In the 1st place, I don’t want to get comfortable in the Ui. I don’t wanna have to go there unless something’s really broken, and I shouldn’t write bad code. In the 1st place.
162 00:18:55.000 ⇒ 00:19:05.809 Brian Pei: if I have to open the ui and and check it, it should feel like a red alert. I have to drop everything and check it. That being said, I do think tracking anomalies is interesting. That’s 1 of the things that I’ve never seen before. That elementary does.
163 00:19:06.090 ⇒ 00:19:30.140 Brian Pei: but I only find it personally interesting if it’s associated with a 10 x bump in storage or cost to run tiny anomalies. I don’t really give a shit. But if if it tells me that hey, yesterday you th! This model created 10 x more rows than I did the day before, I should definitely look at that, or Hey, the compute cost for this was 10 x to the day before. I should. I should look at that as well.
164 00:19:30.150 ⇒ 00:19:47.391 Brian Pei: So I think anomalies are cool in certain situations, but not necessarily for every single little tiny thing. Again, I don’t wanna go to an anomaly checker every single day and find work for myself. I think this should be helpful for big anomalies that affect clients. Performance
165 00:19:48.435 ⇒ 00:20:01.800 Brian Pei: I like. Elementary is documentation. That’s fine, especially when handing off a project handing a project off to a client when brain force leaves, I think the documentation and the lineage is something good to present to clients. It’s nice
166 00:20:01.820 ⇒ 00:20:25.770 Brian Pei: I like it. It should be quick and easy documentation should also. I don’t think it should be iterative, I think, like we spend maybe a couple business days or one week, and just like, get everything in there and never have to think about it again. So I actually wrote that, for engineers never look at it again when it’s finished, unless an analyst or the client has a question about it, or thinks that a definition is too big and wants to add more specific definition.
167 00:20:26.125 ⇒ 00:20:48.949 Brian Pei: I don’t. I don’t like getting bogged down every time we have a new model to put in time for a ticket to document the model. I I think it should be a 1 and done because from what I’ve seen, also, like documentation is cool, and clients sometimes think it’s cool, and you have one presentation where you’re like. Look at this documentation lineage, and the client never looks at it again. But they like that. It’s there
168 00:20:49.143 ⇒ 00:20:56.126 Brian Pei: in their brain. They’re like, Oh, it’s nice that I have documentation. Usually they never look at it again. So I don’t wanna spend too much time on something that they’re never gonna look at again.
169 00:20:57.630 ⇒ 00:21:03.790 Brian Pei: no disrespect. The people who love documentation. I like it, too. It makes it neat and compact, but
170 00:21:04.130 ⇒ 00:21:09.960 Brian Pei: they check it like when they on board a new person. They check it. It’s it’s nice to have yeah.
171 00:21:10.783 ⇒ 00:21:14.490 Brian Pei: My last thing that I brainstormed once the.
172 00:21:14.490 ⇒ 00:21:18.309 Uttam Kumaran: I think I guess I guess before we go to like what we want to do.
173 00:21:18.680 ⇒ 00:21:19.100 Brian Pei: Yeah, yeah.
174 00:21:19.100 ⇒ 00:21:20.980 Uttam Kumaran: Let’s talk about
175 00:21:21.000 ⇒ 00:21:25.980 Uttam Kumaran: these. So yeah. For I think I I agree. This I didn’t know.
176 00:21:26.110 ⇒ 00:21:33.880 Uttam Kumaran: So actually, that’s great. I don’t. I don’t use the ui at all. I think it’s probably 80 to show off.
177 00:21:34.090 ⇒ 00:21:36.570 Uttam Kumaran: especially because it’s free for us.
178 00:21:37.017 ⇒ 00:21:41.500 Uttam Kumaran: I actually care way more about that. The issues get resolved.
179 00:21:43.010 ⇒ 00:21:45.149 Uttam Kumaran: so let’s start from the top. So
180 00:21:45.240 ⇒ 00:21:49.799 Uttam Kumaran: recommend no test. I agree, like, I want, I’d rather everything be binary.
181 00:21:50.950 ⇒ 00:21:55.330 Uttam Kumaran: even we. You know, we basically do this with our issues to like.
182 00:21:55.560 ⇒ 00:22:04.010 Uttam Kumaran: basically, there’s like a high priority. And there’s like low priority. Or there’s like, Drop everything sort of issues. I don’t want there to be like 5 different categories.
183 00:22:04.583 ⇒ 00:22:12.989 Uttam Kumaran: I want the decision and the understanding of what’s important to be super clear. So I agree. I don’t know right now that there are tests
184 00:22:13.300 ⇒ 00:22:15.160 Uttam Kumaran: that are not
185 00:22:15.590 ⇒ 00:22:17.229 Uttam Kumaran: less than high.
186 00:22:19.240 ⇒ 00:22:23.460 Uttam Kumaran: so I would love. So I guess one thing we could talk about is like, are there tests that are
187 00:22:23.520 ⇒ 00:22:26.970 Uttam Kumaran: not high right now, and like, what? What are they and like? Why.
188 00:22:27.760 ⇒ 00:22:29.639 Uttam Kumaran: like, why, even have those?
189 00:22:31.770 ⇒ 00:22:33.580 Uttam Kumaran: But I agree with this
190 00:22:34.168 ⇒ 00:22:36.359 Uttam Kumaran: so in terms of this.
191 00:22:36.420 ⇒ 00:22:39.969 Uttam Kumaran: So if if an alert comes up
192 00:22:40.460 ⇒ 00:22:45.259 Uttam Kumaran: and I wanna think about like what we want, I also want an action here to be
193 00:22:45.370 ⇒ 00:22:49.040 Uttam Kumaran: like what the actual alert is and like what it has basically. So
194 00:22:49.070 ⇒ 00:22:53.019 Uttam Kumaran: it could, you guys, can, you guys basically have exactly where you need to go.
195 00:22:53.350 ⇒ 00:22:57.249 Uttam Kumaran: Is this something that you would just basically go run and just go look directly.
196 00:22:58.931 ⇒ 00:23:06.320 Brian Pei: Yeah, it’s like, if it’s a test that failed. Which is like dupes or nulls
197 00:23:07.880 ⇒ 00:23:09.000 Brian Pei: you would.
198 00:23:09.050 ⇒ 00:23:15.040 Brian Pei: You can find everything in here. But yes, in order like separating it, it would. It’s mostly it’s this
199 00:23:15.676 ⇒ 00:23:18.760 Brian Pei: check the table in Snowflake.
200 00:23:18.880 ⇒ 00:23:20.010 Brian Pei: And
201 00:23:20.575 ⇒ 00:23:23.100 Brian Pei: you know, figure it out basically and resolve
202 00:23:23.780 ⇒ 00:23:25.126 Brian Pei: and get
203 00:23:25.850 ⇒ 00:23:28.420 Brian Pei: There is what’s the name? It’s a
204 00:23:29.230 ⇒ 00:23:42.041 Brian Pei: custom sequel test is what we saw earlier. It’s not a duper null. It’s like. It’s a business logic sequel, where, when it runs, it should return null, and if it doesn’t return null, it sends us a warning.
205 00:23:43.069 ⇒ 00:23:49.989 Brian Pei: Needs to have a comment on like, if it’s this one.
206 00:23:50.600 ⇒ 00:23:57.550 Brian Pei: this, this was one of them for Walmart orders. Oh, I guess this. Never mind, this isn’t custom sequel. This is basically
207 00:23:57.760 ⇒ 00:23:59.060 Brian Pei: unique fields.
208 00:24:02.670 ⇒ 00:24:05.489 Brian Pei: let me not get too crazy. If it’s an error.
209 00:24:06.500 ⇒ 00:24:10.350 Brian Pei: the action would be hit the table.
210 00:24:10.760 ⇒ 00:24:13.439 Brian Pei: find the row that has the error.
211 00:24:13.840 ⇒ 00:24:16.790 Brian Pei: Copy the compiled sequel, and Snowflake
212 00:24:18.200 ⇒ 00:24:22.329 Brian Pei: run it. It tells you the line, the error
213 00:24:22.890 ⇒ 00:24:24.040 Brian Pei: is on.
214 00:24:24.170 ⇒ 00:24:25.870 Brian Pei: Resolve in Snowflake.
215 00:24:26.130 ⇒ 00:24:27.229 Brian Pei: Make it run.
216 00:24:28.260 ⇒ 00:24:29.949 Brian Pei: and then fix and Github.
217 00:24:31.130 ⇒ 00:24:33.590 Uttam Kumaran: Okay. So this is what the alert is. Right. Now.
218 00:24:34.430 ⇒ 00:24:36.259 Uttam Kumaran: what do we want this to be.
219 00:24:38.080 ⇒ 00:24:43.021 Brian Pei: This one is. Let me go to it. This is the most recent one, right.
220 00:24:43.500 ⇒ 00:24:44.130 Uttam Kumaran: Yeah.
221 00:24:48.890 ⇒ 00:24:51.400 Uttam Kumaran: I can send to slack whatever we need.
222 00:24:53.760 ⇒ 00:24:58.319 Nicolas Sucari: So if if we can split on those like 2 categories of
223 00:24:58.360 ⇒ 00:25:04.760 Nicolas Sucari: tests and errors like, they have, like different workflows to do some, some of those
224 00:25:04.820 ⇒ 00:25:05.880 Nicolas Sucari: add
225 00:25:05.950 ⇒ 00:25:14.749 Nicolas Sucari: like, when we receive each of those alerts, we should probably split that 2 in the message of stack, so that we know what we need to trigger after we receive each of them right.
226 00:25:16.540 ⇒ 00:25:16.990 Brian Pei: Yeah.
227 00:25:16.990 ⇒ 00:25:18.710 Nicolas Sucari: From what Brian was saying.
228 00:25:19.365 ⇒ 00:25:26.280 Nicolas Sucari: yeah, from what you were saying, yeah, the the tests I need. We need like to reduce a lot. The
229 00:25:26.370 ⇒ 00:25:35.779 Nicolas Sucari: probably change the logic, how we are receiving those alerts so that we receive fewer, and when we receive those test alerts, we do need to do something
230 00:25:36.211 ⇒ 00:25:41.370 Nicolas Sucari: but the errors are fine, I think, like the process we need to follow is the one that you
231 00:25:41.390 ⇒ 00:25:43.010 Nicolas Sucari: just showed us right.
232 00:25:45.010 ⇒ 00:25:54.579 Brian Pei: Yeah, the only thing, the one that I just showed. I forgot that this was a category. The the one that failed most recently, what is a freshness test?
233 00:25:54.870 ⇒ 00:25:59.820 Brian Pei: And we do get those a lot. It’s just that this table hasn’t been updated in a long time.
234 00:25:59.980 ⇒ 00:26:06.339 Brian Pei: although this is a Google sheet. So I, some of them are static. So I don’t know if this is actually stale.
235 00:26:06.520 ⇒ 00:26:07.420 Brian Pei: But
236 00:26:08.170 ⇒ 00:26:09.109 Brian Pei: you know.
237 00:26:09.270 ⇒ 00:26:13.510 Brian Pei: part of my whole thing is that there are. There are 69 tests in here.
238 00:26:14.290 ⇒ 00:26:15.990 Brian Pei: Do we need to run all of them?
239 00:26:16.783 ⇒ 00:26:18.679 Brian Pei: If this is stale.
240 00:26:19.210 ⇒ 00:26:20.409 Brian Pei: there should be some.
241 00:26:20.410 ⇒ 00:26:28.370 Uttam Kumaran: The actual problem is like the one thing you’re noticing here, Brian, is the Dbt test step didn’t run because the source freshest failed.
242 00:26:29.800 ⇒ 00:26:31.230 Uttam Kumaran: So if you scroll down
243 00:26:31.350 ⇒ 00:26:32.879 Uttam Kumaran: on this page.
244 00:26:33.430 ⇒ 00:26:34.849 Brian Pei: Yeah. So it doesn’t need to skip.
245 00:26:34.850 ⇒ 00:26:35.630 Uttam Kumaran: The next.
246 00:26:36.280 ⇒ 00:26:37.929 Uttam Kumaran: So that’s an issue.
247 00:26:38.750 ⇒ 00:26:39.590 Brian Pei: It doesn’t.
248 00:26:40.020 ⇒ 00:26:41.320 Brian Pei: let’s say.
249 00:26:42.160 ⇒ 00:26:44.146 Uttam Kumaran: It should run the next step right.
250 00:26:44.600 ⇒ 00:26:46.309 Brian Pei: It should, it should run the next day.
251 00:26:46.600 ⇒ 00:26:51.730 Brian Pei: The source freshness. This is this, this is an elementary right. These are like freshness tests.
252 00:26:52.070 ⇒ 00:26:54.070 Ryan Luke Daque: No, that’s just dbt.
253 00:26:54.070 ⇒ 00:26:55.460 Brian Pei: Yeah, this is just dbt.
254 00:26:55.950 ⇒ 00:26:58.250 Brian Pei: so I forgot about
255 00:26:58.550 ⇒ 00:27:00.889 Brian Pei: this. I was just looking at elementary.
256 00:27:01.530 ⇒ 00:27:02.310 Brian Pei: But
257 00:27:03.770 ⇒ 00:27:06.970 Brian Pei: I think in the sources you you basically auto
258 00:27:07.790 ⇒ 00:27:15.580 Brian Pei: put in freshness for for everything you want to make sure that everything is has been updated from 5 Tran within like 24 h.
259 00:27:20.010 ⇒ 00:27:31.119 Brian Pei: I’m okay with freshness tests these, but these shouldn’t fail every single day, because then it goes into what I said. Like, if if I get a freshness error 3 times a day.
260 00:27:31.360 ⇒ 00:27:35.180 Brian Pei: The next time a real error comes along it, it gets lost in the noise.
261 00:27:36.060 ⇒ 00:27:36.720 Ryan Luke Daque: Yeah.
262 00:27:36.910 ⇒ 00:27:42.140 Brian Pei: So the message should just say, like stale data. And I I don’t know how
263 00:27:42.340 ⇒ 00:27:44.390 Brian Pei: specific those alerts can get.
264 00:27:44.450 ⇒ 00:27:45.839 Brian Pei: But if you know.
265 00:27:45.840 ⇒ 00:27:49.450 Uttam Kumaran: Just think about what, think about what the most ideal state would be.
266 00:27:49.800 ⇒ 00:27:52.180 Uttam Kumaran: and then I’ll figure out for for.
267 00:27:52.180 ⇒ 00:27:55.627 Brian Pei: Freshness in slack, it should say, Yeah.
268 00:27:56.970 ⇒ 00:27:59.719 Brian Pei: freshness failed for this table.
269 00:27:59.870 ⇒ 00:28:01.060 Brian Pei: and then
270 00:28:03.420 ⇒ 00:28:05.793 Brian Pei: I don’t know how this would be resolved.
271 00:28:06.340 ⇒ 00:28:07.490 Brian Pei: go to 5 Chan and.
272 00:28:07.490 ⇒ 00:28:10.670 Uttam Kumaran: Show you what it’s just. It should literally show you this link.
273 00:28:10.670 ⇒ 00:28:11.120 Brian Pei: This lie.
274 00:28:11.120 ⇒ 00:28:11.670 Uttam Kumaran: I.
275 00:28:11.920 ⇒ 00:28:16.420 Brian Pei: Yep, I don’t wanna see everything else I I wanna see this line.
276 00:28:19.140 ⇒ 00:28:19.790 Uttam Kumaran: Okay.
277 00:28:23.990 ⇒ 00:28:27.340 Brian Pei: So that’s fine freshness check. 5 tran
278 00:28:28.889 ⇒ 00:28:32.289 Brian Pei: or if the table
279 00:28:34.010 ⇒ 00:28:37.180 Brian Pei: Aka static Google sheet
280 00:28:38.700 ⇒ 00:28:41.599 Brian Pei: doesn’t need a freshness test.
281 00:28:41.760 ⇒ 00:28:43.820 Brian Pei: remove it from the repo.
282 00:28:44.960 ⇒ 00:28:48.100 Brian Pei: I think a lot of the Google sheets are static. So I don’t.
283 00:28:48.100 ⇒ 00:28:54.770 Ryan Luke Daque: I think, for that specific one that failed. It’s something that I had to like update once a week every Monday.
284 00:28:54.770 ⇒ 00:28:56.030 Brian Pei: Oh, never mind! Once a week.
285 00:28:56.030 ⇒ 00:28:56.400 Ryan Luke Daque: Updates.
286 00:28:56.400 ⇒ 00:28:58.420 Brian Pei: Yeah, I think that that’s fine. Then.
287 00:28:58.640 ⇒ 00:29:00.340 Ryan Luke Daque: But the freshness was like also.
288 00:29:00.643 ⇒ 00:29:02.770 Uttam Kumaran: Other test should also run right like.
289 00:29:02.770 ⇒ 00:29:09.200 Ryan Luke Daque: Right? Yeah. Like the Dbt test should still run, even if the source freshness test failed. Right?
290 00:29:10.120 ⇒ 00:29:10.440 Uttam Kumaran: Yeah.
291 00:29:11.160 ⇒ 00:29:11.790 Brian Pei: Yeah.
292 00:29:12.610 ⇒ 00:29:13.290 Uttam Kumaran: Okay.
293 00:29:14.010 ⇒ 00:29:20.030 Uttam Kumaran: okay, so there’s some changes on this. But I think we basically, we have 3 situations.
294 00:29:20.240 ⇒ 00:29:21.520 Uttam Kumaran: Again, like.
295 00:29:21.890 ⇒ 00:29:23.040 Uttam Kumaran: I,
296 00:29:23.570 ⇒ 00:29:27.829 Uttam Kumaran: I think we just put all this basically into like a run book. That’s just like
297 00:29:28.370 ⇒ 00:29:30.489 Uttam Kumaran: situations and how to handle.
298 00:29:31.044 ⇒ 00:29:40.739 Uttam Kumaran: Yeah, I will. I agree that the ui is like, kind of useless for this. I would just go straight to here. And I agree. I basically think every
299 00:29:40.890 ⇒ 00:29:44.000 Uttam Kumaran: tests that fails needs to be high.
300 00:29:44.480 ⇒ 00:29:49.410 Uttam Kumaran: I will say, I’m pretty sure that they all have been high, so I don’t know how far we.
301 00:29:49.670 ⇒ 00:29:50.699 Brian Pei: Carrier. Yeah.
302 00:29:50.700 ⇒ 00:29:53.909 Uttam Kumaran: I don’t. I don’t know that we are not doing this.
303 00:29:54.160 ⇒ 00:29:57.480 Uttam Kumaran: I just think what you said, which is basically
304 00:29:59.190 ⇒ 00:30:07.039 Uttam Kumaran: what you said, which is like the more quantity of tests the less important they become. I guess, like I don’t. If every
305 00:30:07.610 ⇒ 00:30:10.509 Uttam Kumaran: it’s just like not every table is gonna have tests.
306 00:30:10.700 ⇒ 00:30:18.900 Uttam Kumaran: But the biggest thing is like, I want the downstream reporting tables. We have tests that look at ids, we test to look at, know we have tests that look at sums that match.
307 00:30:19.930 ⇒ 00:30:24.889 Uttam Kumaran: I think we have a we have a backlog of failures now that we’ll get through.
308 00:30:25.010 ⇒ 00:30:31.520 Uttam Kumaran: but I also want to make sure that like, yeah, you’re right, that the test that exists now they’re all red, alert, level stuff.
309 00:30:32.560 ⇒ 00:30:44.559 Brian Pei: It’s a yeah. It’s a culture thing, right? Because if we get 3 failing tests today and they’re all important, then they should be 3 red alerts a day, where all hands are on deck like, why did this fail and this fail? And this fail?
310 00:30:45.070 ⇒ 00:30:48.839 Brian Pei: But some tests that I noticed that some people just ignore like
311 00:30:49.288 ⇒ 00:30:56.540 Brian Pei: this, like this field should not have nulls, and sometimes it has nulls. And then people are just like, you know what sometimes that field has nulls.
312 00:30:57.030 ⇒ 00:30:58.170 Brian Pei: What are you gonna do.
313 00:30:58.570 ⇒ 00:30:59.949 Uttam Kumaran: And we get rid of the test
314 00:31:00.470 ⇒ 00:31:01.320 Uttam Kumaran: or each other.
315 00:31:01.320 ⇒ 00:31:01.750 Brian Pei: And get.
316 00:31:02.146 ⇒ 00:31:08.090 Uttam Kumaran: Basically, basically, my expectation would be that if the alert comes up there’s some resolution.
317 00:31:08.190 ⇒ 00:31:12.859 Uttam Kumaran: Either the test was wrong with move the test, or we fix the logic.
318 00:31:13.230 ⇒ 00:31:14.710 Uttam Kumaran: Those are the only 2
319 00:31:14.860 ⇒ 00:31:17.010 Uttam Kumaran: appropriate outcomes. Right.
320 00:31:17.740 ⇒ 00:31:18.390 Brian Pei: Yeah.
321 00:31:18.780 ⇒ 00:31:20.190 Brian Pei: I’ve written here.
322 00:31:21.280 ⇒ 00:31:24.088 Brian Pei: let me just read through the the rest of these.
323 00:31:25.370 ⇒ 00:31:29.860 Brian Pei: but yeah. So once the elementary tester simplified. Well, we just talked about this.
324 00:31:30.267 ⇒ 00:31:39.152 Brian Pei: Either have a goalie system where each alert needs to be addressed and fixed within 3 h, or have each client have a dedicated goalie. We can talk about that later.
325 00:31:39.740 ⇒ 00:31:56.260 Brian Pei: to triage or yeah triage to the right engineer or fix it on the spot. Have a culture where every time there is an alert comment in slack under the alert or I’ve also seen in in slack bots. There’s like a button that says resolve, and you can click, resolve, and add a comment.
326 00:31:57.107 ⇒ 00:32:07.269 Brian Pei: You talked about this in the other Doc. Implement an sla in the slack channel with a time to resolve, and at whoever the goalie would be, or you know, just
327 00:32:07.720 ⇒ 00:32:19.219 Brian Pei: these should all be. There should be a comment within 3 h, reasonably like, if it’s in the middle of the night and like nobody’s freaking, the client isn’t freaking out. It’s fine. But reasonably.
328 00:32:19.380 ⇒ 00:32:21.399 Brian Pei: I think 3 h is okay.
329 00:32:23.040 ⇒ 00:32:32.359 Brian Pei: and then recommendation for the current state of pull parts to go. Have another meeting to go through the last week of alerts you just mentioned this so that I wanted to
330 00:32:32.500 ⇒ 00:32:38.540 Brian Pei: read that I I thought about that. We should just go alert by alert and figure out, is this important
331 00:32:38.962 ⇒ 00:32:45.770 Brian Pei: why isn’t important? Should we get rid of it, or should we just fix it right now, so that if we fix that alert now
332 00:32:45.840 ⇒ 00:32:47.950 Brian Pei: we’ll have less alerts in the future.
333 00:32:48.425 ⇒ 00:32:51.975 Brian Pei: Yeah, run book. You mentioned this, too, which is great.
334 00:32:52.560 ⇒ 00:33:01.619 Brian Pei: So when we do have that meeting, I think. Let’s have a document. We’ll take each alert. Take a picture of it in a run book, put under it what
335 00:33:01.780 ⇒ 00:33:13.830 Brian Pei: me or Ryan, or whoever what we would do triage it so in the future. If you know, if we have a new engineer, or whoever, and they get an alert, they can go to the run book and be like, oh, this is what somebody did for for this.
336 00:33:14.381 ⇒ 00:33:19.808 Brian Pei: So that’s great start cutting down on quantity. I don’t think this is like a red alert. I I.
337 00:33:20.080 ⇒ 00:33:26.051 Uttam Kumaran: I see I see what you mean. I see what you mean by that, so let’s try it. I just don’t know where I don’t.
338 00:33:26.571 ⇒ 00:33:29.099 Brian Pei: Through some of them some of them might be
339 00:33:29.650 ⇒ 00:33:36.080 Brian Pei: dumb to have. I don’t know. We. We should have a meeting to go through every to go through as many as we can.
340 00:33:36.431 ⇒ 00:33:46.860 Brian Pei: And oh! And then this, if there is a test in our repo, I think there should be a comment in the file that clearly states why we have this test. We don’t have that right now.
341 00:33:47.188 ⇒ 00:34:00.460 Brian Pei: And why it’s important to resolve from business perspective. So, for example, when you have something like this or unique, just have a comment. If this column is not unique, then Mega, important reporting table joins on this Id, and I’ll cause duplicates.
342 00:34:00.930 ⇒ 00:34:03.210 Brian Pei: Just have a little little blurb.
343 00:34:03.820 ⇒ 00:34:13.760 Brian Pei: Comments are great, especially with tests, because if somebody’s out, or whatever, and I have to go to a test that I didn’t implement, and I go to the repo. I have no context on why that test is there in the 1st place, so
344 00:34:14.090 ⇒ 00:34:15.330 Brian Pei: I think that’d be helpful.
345 00:34:16.290 ⇒ 00:34:24.020 Uttam Kumaran: So for the for the anomalies. I honestly don’t care. I I’m game to just remove them until we figure this out.
346 00:34:24.389 ⇒ 00:34:29.429 Uttam Kumaran: I think there’s a difference between anomalies from like an analyst perspective, which is like
347 00:34:29.449 ⇒ 00:34:32.699 Uttam Kumaran: sales are down last 30 days.
348 00:34:33.239 ⇒ 00:34:38.390 Uttam Kumaran: you know I and there’s a things that difference for the ones on the Ae. Side, which is like
349 00:34:39.040 ⇒ 00:34:43.440 Uttam Kumaran: there’s like a shit ton of rose like those may mean different things.
350 00:34:45.120 ⇒ 00:34:48.619 Uttam Kumaran: but if we don’t have a good sense on what we want to do there.
351 00:34:48.840 ⇒ 00:34:51.269 Uttam Kumaran: I think we remove it.
352 00:34:51.469 ⇒ 00:34:55.639 Uttam Kumaran: I’m more interested, I think, by having the
353 00:34:56.100 ⇒ 00:35:00.749 Uttam Kumaran: I’m interested in looking at like the row count anomalies. But I don’t know. I don’t like
354 00:35:00.990 ⇒ 00:35:02.780 Uttam Kumaran: I don’t know when that’s ever been like
355 00:35:03.400 ⇒ 00:35:04.990 Uttam Kumaran: useful, really.
356 00:35:05.900 ⇒ 00:35:09.400 Uttam Kumaran: and I don’t even know if we’re at the place with cool cards where
357 00:35:09.620 ⇒ 00:35:13.099 Uttam Kumaran: we can look at like oh, stuff is trending lower or higher.
358 00:35:13.220 ⇒ 00:35:15.430 Uttam Kumaran: and I think that has to run through Jacob.
359 00:35:15.510 ⇒ 00:35:16.630 Uttam Kumaran: because
360 00:35:16.960 ⇒ 00:35:24.140 Uttam Kumaran: this team is not going to be on the hook, for why sales lower this team will be on the hook for is sales wrong
361 00:35:24.520 ⇒ 00:35:36.720 Uttam Kumaran: like. Is it accurate? Right? If it’s lower than I? That doesn’t mean anything. So I we should just simplify and remove all that, and then come back at it through from the analyst side.
362 00:35:37.299 ⇒ 00:35:40.150 Uttam Kumaran: Or consider it. If you guys want to add those tests.
363 00:35:41.092 ⇒ 00:35:43.270 Uttam Kumaran: In terms of like the goalie.
364 00:35:45.430 ⇒ 00:35:46.790 Uttam Kumaran: yeah, I don’t know.
365 00:35:48.050 ⇒ 00:35:49.390 Uttam Kumaran: I think, like.
366 00:35:49.460 ⇒ 00:35:50.730 Uttam Kumaran: probably
367 00:35:52.780 ⇒ 00:35:55.319 Uttam Kumaran: like, Nick can handle everything for
368 00:35:56.036 ⇒ 00:35:59.200 Uttam Kumaran: Stella. And then maybe on this side, we just rotate
369 00:35:59.340 ⇒ 00:36:00.479 Uttam Kumaran: or something.
370 00:36:00.610 ⇒ 00:36:01.289 Uttam Kumaran: I think that.
371 00:36:01.290 ⇒ 00:36:10.110 Brian Pei: The dedicated person. The only reason I put that down is because we have, like 4 people who can do something in pull parts. But if nobody’s targeted the 4 people.
372 00:36:10.110 ⇒ 00:36:11.030 Uttam Kumaran: Yeah, just wait.
373 00:36:12.350 ⇒ 00:36:16.390 Uttam Kumaran: So I think we should switch like on a weekly basis.
374 00:36:17.770 ⇒ 00:36:20.659 Uttam Kumaran: which is pretty common. You have like on call.
375 00:36:20.960 ⇒ 00:36:21.880 Brian Pei: Yeah, that’s pretty common.
376 00:36:21.880 ⇒ 00:36:24.300 Uttam Kumaran: I would take. I would take it if it was once a week.
377 00:36:24.330 ⇒ 00:36:26.950 Uttam Kumaran: if it was once a month. I don’t mind that
378 00:36:27.030 ⇒ 00:36:32.179 Uttam Kumaran: cause, then, cause I can do that. It’s like I could go through and spend 30 min and figure it out. But
379 00:36:32.430 ⇒ 00:36:34.880 Uttam Kumaran: it can’t be all 3 of us every day.
380 00:36:35.050 ⇒ 00:36:35.580 Ryan Luke Daque: Yeah.
381 00:36:36.310 ⇒ 00:36:37.910 Uttam Kumaran: So that’s an
382 00:36:38.000 ⇒ 00:36:47.429 Uttam Kumaran: yeah, and then all. And then, so that’s the thing I think we do is maybe we just have. We do like a rotation between, like the Aes on an account.
383 00:36:47.640 ⇒ 00:36:48.810 Uttam Kumaran: And
384 00:36:49.720 ⇒ 00:36:53.729 Uttam Kumaran: that’s it, right cause? Then it saves everybody having to look in issues.
385 00:36:53.770 ⇒ 00:36:59.539 Uttam Kumaran: and then so we can bring each other in if we have other questions. But there’s like a known understanding that
386 00:37:00.460 ⇒ 00:37:05.830 Uttam Kumaran: within 24 h of an alert coming someone, one of us. Whoever’s assigned is going to jump on it. So.
387 00:37:07.020 ⇒ 00:37:07.450 Nicolas Sucari: Yeah.
388 00:37:07.450 ⇒ 00:37:12.719 Brian Pei: Yeah, I think that’s cool. And before we do that, maybe on Friday, I think we should pair
389 00:37:12.850 ⇒ 00:37:15.619 Brian Pei: we, Tommy? You might not have to be there, but we should pair for an hour
390 00:37:15.870 ⇒ 00:37:19.169 Brian Pei: as a team, and just go through the last 10, the last 20.
391 00:37:19.560 ⇒ 00:37:21.739 Brian Pei: So we get used to what the alerts are.
392 00:37:23.310 ⇒ 00:37:25.369 Nicolas Sucari: Adam. Is there a way on
393 00:37:25.380 ⇒ 00:37:35.620 Nicolas Sucari: like linking these alerts into a list in slack, or the list can get resolved. Each, like each alert is kind of a new entry on that list, and we can like
394 00:37:35.920 ⇒ 00:37:37.528 Nicolas Sucari: it’s hard to mark them.
395 00:37:37.850 ⇒ 00:37:42.759 Uttam Kumaran: What I’m what I’m probably gonna do is somehow try to send this log
396 00:37:43.120 ⇒ 00:37:45.490 Uttam Kumaran: from wherever the failure is.
397 00:37:45.740 ⇒ 00:37:49.329 Uttam Kumaran: and like get the at least a log into the thread or something.
398 00:37:50.016 ⇒ 00:37:55.280 Uttam Kumaran: But I I think, like again, that’s only 2 clicks away, basically, which is.
399 00:37:55.480 ⇒ 00:37:56.900 Uttam Kumaran: you know, a lot. But
400 00:37:56.950 ⇒ 00:38:04.339 Uttam Kumaran: I’m I think more of the thing that’s gonna help you is one having like the on call. And then, second, basically, all of us being like
401 00:38:04.420 ⇒ 00:38:07.379 Uttam Kumaran: we’re good. We’re confident that the test there
402 00:38:07.430 ⇒ 00:38:09.040 Uttam Kumaran: are the ones that matter.
403 00:38:09.510 ⇒ 00:38:16.600 Uttam Kumaran: That is, gonna we’re all gonna buy in and basically understand, like, Hey, the test we have there, we’re comfortable with. And so if there’s alerts.
404 00:38:16.690 ⇒ 00:38:17.899 Uttam Kumaran: they get resolved.
405 00:38:18.290 ⇒ 00:38:22.959 Uttam Kumaran: Then it’s like, it’s up to us to basically be like, Okay, cool. Actually, Patrick, that’s your problem, or like
406 00:38:23.280 ⇒ 00:38:28.280 Uttam Kumaran: or we dish it out between us. But I think the on call is gonna solve a lot of problems.
407 00:38:28.836 ⇒ 00:38:34.340 Uttam Kumaran: and then reducing and taking, removing as many alerts as possible, or again, having like
408 00:38:35.530 ⇒ 00:38:43.320 Uttam Kumaran: having more well defined alerts like, if there’s something where there’s a freshness set that’s only one day when we know there’s delays, we should extend it.
409 00:38:43.460 ⇒ 00:38:46.839 Uttam Kumaran: We’re gonna have to audit and basically walk through. I think we should do an hour on that
410 00:38:47.250 ⇒ 00:38:51.959 Uttam Kumaran: and just walk through and clear up, clear up what exists. And then basically start.
411 00:38:52.460 ⇒ 00:38:57.979 Uttam Kumaran: And then what I’m gonna do is have Nick is gonna own. This a similar process on the Stella side.
412 00:39:01.600 ⇒ 00:39:02.300 Brian Pei: Make, sense.
413 00:39:02.300 ⇒ 00:39:03.010 Nicolas Sucari: Great.
414 00:39:04.687 ⇒ 00:39:06.419 Uttam Kumaran: So the so
415 00:39:06.530 ⇒ 00:39:07.990 Uttam Kumaran: I think.
416 00:39:09.170 ⇒ 00:39:15.269 Uttam Kumaran: I mean, let’s just leave this, Doc, until we like. Figure this out. We should do another meeting on Friday.
417 00:39:17.820 ⇒ 00:39:19.070 Uttam Kumaran: and then
418 00:39:19.240 ⇒ 00:39:23.669 Uttam Kumaran: the meeting on Friday will basically be like, let’s audit and just go through and clear out
419 00:39:24.000 ⇒ 00:39:25.490 Uttam Kumaran: the stuff that exists.
420 00:39:26.341 ⇒ 00:39:32.259 Uttam Kumaran: And then we’ll basically from like next week on, we’ll we’ll try to aim to have one of us
421 00:39:33.300 ⇒ 00:39:41.260 Uttam Kumaran: start and like kind of in this process, like, build a run book. The nice thing is also is we can. It’ll just be passed around like a.
422 00:39:41.810 ⇒ 00:39:46.870 Uttam Kumaran: It’ll just mainly like we’ll. We’ll just have an issue on everybody’s board for the week. That, like, you’re the
423 00:39:47.180 ⇒ 00:39:52.499 Uttam Kumaran: person on call basically that way. It’s a reminder. And then that’ll just be a percentage of work.
424 00:39:55.240 ⇒ 00:39:58.599 Uttam Kumaran: you know. And then the goal, of course, is like less issues over time.
425 00:39:58.630 ⇒ 00:40:20.459 Uttam Kumaran: So ideally, that’s it’s like non-existent. But, as we know, like, there’s going to be issues that are caused by new prs and things like that. So there’s also processing for hey? If we need to run a bunch of tests before things get merged. There are process things that we should push to Patrick, basically to be like going with these. So let’s take Friday and like, walk through this. I think this is great, Brian. I think all these we can basically do.
426 00:40:21.250 ⇒ 00:40:21.575 Brian Pei: Thanks.
427 00:40:24.010 ⇒ 00:40:28.450 Brian Pei: Thanks. I got really angry reading, not our stuff reading like the Internet, like Internet stuff.
428 00:40:28.620 ⇒ 00:40:38.079 Uttam Kumaran: No, I think like I think we’ve done this many times before, and this is exactly you’re right, and I don’t care about like what it should be. I care about like what works
429 00:40:39.498 ⇒ 00:40:50.609 Uttam Kumaran: and this is what works like. I I don’t want people like I don’t want all 3 of us looking at alerts, and I especially don’t want to be like something’s failing. And we’re gonna lose the client because we fuck something out
430 00:40:50.650 ⇒ 00:40:58.620 Uttam Kumaran: every day. So like, I don’t want to have to think about that. So I think for everybody. I think this is a good process. So let’s talk on Friday.
431 00:40:58.978 ⇒ 00:41:03.459 Uttam Kumaran: If you have time this week to take a look at some issues before then that’d be great.
432 00:41:04.012 ⇒ 00:41:06.350 Uttam Kumaran: But apart from that, we’ll go through it all on Friday.
433 00:41:07.870 ⇒ 00:41:08.590 Brian Pei: Sweet.
434 00:41:10.150 ⇒ 00:41:10.610 Uttam Kumaran: Okay.
435 00:41:10.610 ⇒ 00:41:11.260 Nicolas Sucari: Raised.
436 00:41:11.590 ⇒ 00:41:16.453 Ryan Luke Daque: Yeah, just a quick note. I just like fix the source freshness issue. And I just like
437 00:41:16.810 ⇒ 00:41:19.239 Ryan Luke Daque: recrigger the workflow. So we’ll have
438 00:41:19.870 ⇒ 00:41:25.939 Ryan Luke Daque: the Dbt test. Dbd tests are running, so we’ll have some other tests to to look at.
439 00:41:26.219 ⇒ 00:41:31.530 Uttam Kumaran: Yeah, before Friday I’ll I’ll try and spend like an hour on making the alert a little bit better.
440 00:41:33.720 ⇒ 00:41:37.510 Uttam Kumaran: I built that little thing, so I’ll try to go make it a little bit better.
441 00:41:37.530 ⇒ 00:41:40.319 Uttam Kumaran: But otherwise I think we’re pretty solid.
442 00:41:43.700 ⇒ 00:41:45.480 Uttam Kumaran: Okay, dope.
443 00:41:47.250 ⇒ 00:41:48.380 Uttam Kumaran: But thank you, excellent.
444 00:41:48.380 ⇒ 00:41:49.053 Ryan Luke Daque: Thanks, Brian.
445 00:41:49.600 ⇒ 00:41:51.000 Brian Pei: Thanks a lot. Thanks guys.
446 00:41:51.010 ⇒ 00:41:51.639 Uttam Kumaran: And Sam.
447 00:41:51.640 ⇒ 00:41:52.350 Ryan Luke Daque: Thanks.