Meeting Title: PP2G-Standup Date: 2024-06-14 Meeting participants: Nicolas Sucari, Ryan Luke Daque, Patrick Trainer
WEBVTT
1 00:01:07.470 ⇒ 00:01:08.419 Ryan Luke Daque: Hello! Hello!
2 00:01:12.180 ⇒ 00:01:12.740 Patrick Trainer: Go.
3 00:01:15.150 ⇒ 00:01:17.650 Ryan Luke Daque: Hey, man! Nice, nice, nice one, with a
4 00:01:18.030 ⇒ 00:01:20.157 Ryan Luke Daque: training period of thing.
5 00:01:20.860 ⇒ 00:01:21.650 Patrick Trainer: Yeah, it’s, it’s.
6 00:01:21.650 ⇒ 00:01:25.329 Ryan Luke Daque: The other one, though, like the detection period. So it’s.
7 00:01:25.330 ⇒ 00:01:25.870 Patrick Trainer: Said that.
8 00:01:25.870 ⇒ 00:01:26.769 Ryan Luke Daque: Looks like.
9 00:01:26.770 ⇒ 00:01:28.780 Patrick Trainer: The detection period.
10 00:01:29.460 ⇒ 00:01:30.190 Ryan Luke Daque: Get.
11 00:01:30.190 ⇒ 00:01:30.990 Patrick Trainer: It’s
12 00:01:32.310 ⇒ 00:01:36.700 Patrick Trainer: it’s pretty confusing, honestly like.
13 00:01:37.690 ⇒ 00:01:40.830 Patrick Trainer: cause I like it’ll use
14 00:01:41.050 ⇒ 00:01:42.910 Patrick Trainer: the last week.
15 00:01:43.310 ⇒ 00:01:48.569 Patrick Trainer: Ask the bounds of like of finding the anomaly right.
16 00:01:48.570 ⇒ 00:01:49.820 Ryan Luke Daque: Right so, and then the.
17 00:01:49.820 ⇒ 00:01:53.140 Patrick Trainer: Training period is how far it looks back
18 00:01:53.320 ⇒ 00:01:57.329 Patrick Trainer: to understand. Like the the yeah, the pattern.
19 00:01:57.330 ⇒ 00:01:58.979 Ryan Luke Daque: Calculate the upper lower.
20 00:01:58.980 ⇒ 00:02:02.279 Patrick Trainer: Yeah, and then, and then it counts each row
21 00:02:03.360 ⇒ 00:02:05.629 Patrick Trainer: in into a time bucket.
22 00:02:06.840 ⇒ 00:02:07.650 Ryan Luke Daque: Right.
23 00:02:08.289 ⇒ 00:02:09.409 Patrick Trainer: So it’s
24 00:02:09.869 ⇒ 00:02:10.429 Patrick Trainer: like.
25 00:02:10.430 ⇒ 00:02:12.170 Ryan Luke Daque: Shouldn’t. Shouldn’t we? Like.
26 00:02:12.910 ⇒ 00:02:14.010 Ryan Luke Daque: yeah, yeah, go ahead.
27 00:02:14.010 ⇒ 00:02:16.539 Patrick Trainer: So I was, gonna say, the 3 are
28 00:02:16.770 ⇒ 00:02:18.110 Patrick Trainer: really like
29 00:02:18.790 ⇒ 00:02:20.330 Patrick Trainer: interconnected.
30 00:02:20.500 ⇒ 00:02:22.889 Patrick Trainer: And it’s like, if you change one.
31 00:02:22.920 ⇒ 00:02:24.929 Patrick Trainer: it kind of changes
32 00:02:25.370 ⇒ 00:02:28.979 Patrick Trainer: the like. The the other 2,
33 00:02:29.100 ⇒ 00:02:30.909 Patrick Trainer: or at least it it’s
34 00:02:31.660 ⇒ 00:02:32.730 Patrick Trainer: there’s
35 00:02:33.990 ⇒ 00:02:36.240 Patrick Trainer: I’m thinking of it as like
36 00:02:38.000 ⇒ 00:02:40.550 Patrick Trainer: there’s 3 variables in this function.
37 00:02:40.900 ⇒ 00:02:41.540 Patrick Trainer: and.
38 00:02:41.540 ⇒ 00:02:42.010 Ryan Luke Daque: Right.
39 00:02:42.010 ⇒ 00:02:43.350 Patrick Trainer: The output
40 00:02:44.400 ⇒ 00:02:52.159 Patrick Trainer: which is like a standard deviation, right is going to be impacted by one of those 3 variables.
41 00:02:52.160 ⇒ 00:02:53.050 Ryan Luke Daque: Right.
42 00:02:53.050 ⇒ 00:02:54.219 Patrick Trainer: And so
43 00:02:54.970 ⇒ 00:03:00.370 Patrick Trainer: we have the time buckets which is going, which is pretty much like
44 00:03:00.520 ⇒ 00:03:02.829 Patrick Trainer: the line, and like.
45 00:03:02.830 ⇒ 00:03:03.420 Nicolas Sucari: Guys.
46 00:03:03.960 ⇒ 00:03:04.420 Patrick Trainer: Hey.
47 00:03:04.740 ⇒ 00:03:06.980 Patrick Trainer: yeah, it’s yeah. It’s the
48 00:03:07.090 ⇒ 00:03:10.339 Patrick Trainer: the X axis. Yeah, where it’s plotted on the X,
49 00:03:10.650 ⇒ 00:03:13.330 Patrick Trainer: and then the training period
50 00:03:13.630 ⇒ 00:03:15.060 Patrick Trainer: is
51 00:03:16.889 ⇒ 00:03:22.159 Patrick Trainer: irrelevant to that. Yeah, it’s it’s just like how how long
52 00:03:22.640 ⇒ 00:03:26.809 Patrick Trainer: the time buck or or how long they’re going to bucket time
53 00:03:28.290 ⇒ 00:03:32.690 Patrick Trainer: and then the detection period is, it’s it’s like a moving
54 00:03:33.428 ⇒ 00:03:36.430 Patrick Trainer: like a moving period of 7 days.
55 00:03:37.040 ⇒ 00:03:40.059 Patrick Trainer: or like a moving period of X amount of
56 00:03:40.450 ⇒ 00:03:43.359 Patrick Trainer: days, weeks, months, etc.
57 00:03:43.650 ⇒ 00:03:46.190 Patrick Trainer: And so like, if you
58 00:03:46.540 ⇒ 00:03:49.099 Patrick Trainer: increase the detection period.
59 00:03:51.730 ⇒ 00:03:54.559 Patrick Trainer: it’s going to be less sensitive
60 00:03:55.220 ⇒ 00:03:56.400 Patrick Trainer: to
61 00:03:56.720 ⇒ 00:03:58.090 Patrick Trainer: anomalies.
62 00:03:58.960 ⇒ 00:04:00.910 Patrick Trainer: Because, yeah, that.
63 00:04:01.780 ⇒ 00:04:02.880 Patrick Trainer: And
64 00:04:03.940 ⇒ 00:04:05.520 Patrick Trainer: and then the
65 00:04:07.400 ⇒ 00:04:16.819 Patrick Trainer: right. And so then the anomaly sensitivity is where you’re setting either 1, 2, or 3 standard deviations away.
66 00:04:16.940 ⇒ 00:04:19.889 Patrick Trainer: So that’s gonna set the like. The gray.
67 00:04:20.170 ⇒ 00:04:20.980 Ryan Luke Daque: Right.
68 00:04:21.500 ⇒ 00:04:25.140 Patrick Trainer: Okay, I need to. I need to draw this out. I think.
69 00:04:25.140 ⇒ 00:04:26.920 Ryan Luke Daque: Yeah, I, think I I think.
70 00:04:26.920 ⇒ 00:04:30.300 Patrick Trainer: Figured it out, or at least I understand it now.
71 00:04:31.000 ⇒ 00:04:36.679 Ryan Luke Daque: The detection period. Looks like I’m I’m reading like what the it’s saying here. So like, currently
72 00:04:36.710 ⇒ 00:04:39.759 Ryan Luke Daque: you moved it up from 2 days to 7 days. Right?
73 00:04:40.080 ⇒ 00:04:41.359 Ryan Luke Daque: So it does. Detection.
74 00:04:41.360 ⇒ 00:04:45.440 Patrick Trainer: Yeah. Or I moved it to one week. So it’s always good. Yeah.
75 00:04:45.460 ⇒ 00:04:47.829 Patrick Trainer: it’s always gonna move one week at a time.
76 00:04:48.580 ⇒ 00:04:53.689 Ryan Luke Daque: Yeah, but that would mean that if there was any an anomaly within that week.
77 00:04:53.960 ⇒ 00:04:55.679 Ryan Luke Daque: then it would always be
78 00:04:56.280 ⇒ 00:04:58.350 Ryan Luke Daque: flagged as an anomaly.
79 00:04:59.220 ⇒ 00:05:02.370 Ryan Luke Daque: right, even even if today was within the bounds.
80 00:05:02.976 ⇒ 00:05:05.400 Patrick Trainer: So. So let’s say
81 00:05:05.440 ⇒ 00:05:06.620 Patrick Trainer: it’s
82 00:05:07.230 ⇒ 00:05:09.659 Patrick Trainer: like we’ll start the week on
83 00:05:09.820 ⇒ 00:05:16.667 Patrick Trainer: one and go through 7 right? If we’re only on day 2 or day. 3.
84 00:05:17.500 ⇒ 00:05:19.020 Patrick Trainer: It will
85 00:05:19.240 ⇒ 00:05:20.500 Patrick Trainer: do like
86 00:05:20.760 ⇒ 00:05:27.160 Patrick Trainer: day, minus one, minus 2, minus 3, until you get a period of one week.
87 00:05:27.160 ⇒ 00:05:28.410 Ryan Luke Daque: 7, right.
88 00:05:28.410 ⇒ 00:05:29.140 Patrick Trainer: Right?
89 00:05:30.050 ⇒ 00:05:35.380 Patrick Trainer: And so and so there’s kind of like a hierarchy. So it’s gonna go.
90 00:05:35.440 ⇒ 00:05:36.930 Patrick Trainer: what training
91 00:05:37.960 ⇒ 00:05:38.940 Patrick Trainer: period?
92 00:05:39.050 ⇒ 00:05:41.970 Patrick Trainer: That’s kinda like the the the top level.
93 00:05:42.290 ⇒ 00:05:44.020 Patrick Trainer: And then it’s gonna go
94 00:05:44.580 ⇒ 00:05:46.160 Patrick Trainer: detection period
95 00:05:47.890 ⇒ 00:05:49.000 Patrick Trainer: detect.
96 00:05:49.320 ⇒ 00:05:52.019 Patrick Trainer: And then and then it’s going to go
97 00:05:53.460 ⇒ 00:05:55.039 Patrick Trainer: to the time bucket.
98 00:05:58.510 ⇒ 00:05:59.859 Patrick Trainer: I think that’s how it’s
99 00:06:00.120 ⇒ 00:06:00.930 Patrick Trainer: yeah.
100 00:06:01.980 ⇒ 00:06:03.360 Patrick Trainer: And so
101 00:06:03.690 ⇒ 00:06:09.670 Patrick Trainer: I’m I’m like drawing on a sticky note right now, and and also thinking out loud. So
102 00:06:10.210 ⇒ 00:06:13.359 Patrick Trainer: yeah, the training period relates to
103 00:06:13.950 ⇒ 00:06:16.410 Patrick Trainer: the actual data in the table.
104 00:06:17.480 ⇒ 00:06:18.300 Patrick Trainer: Brian.
105 00:06:18.510 ⇒ 00:06:20.740 Patrick Trainer: And then the time bucket.
106 00:06:22.330 ⇒ 00:06:25.020 Patrick Trainer: Okay? So yeah, there’s like.
107 00:06:25.400 ⇒ 00:06:28.840 Patrick Trainer: training period is over the entire thing.
108 00:06:29.030 ⇒ 00:06:31.089 Patrick Trainer: And then there’s the
109 00:06:31.240 ⇒ 00:06:32.510 Patrick Trainer: the plots
110 00:06:32.970 ⇒ 00:06:36.020 Patrick Trainer: on the days like the circles.
111 00:06:40.370 ⇒ 00:06:43.649 Patrick Trainer: And then so that’s the time bucket.
112 00:06:44.250 ⇒ 00:06:44.920 Ryan Luke Daque: Right.
113 00:06:45.060 ⇒ 00:06:46.250 Patrick Trainer: And then
114 00:06:47.790 ⇒ 00:06:49.749 Patrick Trainer: we have the
115 00:06:50.170 ⇒ 00:06:51.520 Patrick Trainer: but anomaly
116 00:06:51.940 ⇒ 00:06:55.260 Patrick Trainer: sensitivity, which is like the gray.
117 00:06:55.260 ⇒ 00:06:57.350 Ryan Luke Daque: Standard. That’s standard TV.
118 00:06:57.350 ⇒ 00:07:01.040 Patrick Trainer: Yeah, the standard deviation. So like the acceptable
119 00:07:01.430 ⇒ 00:07:03.330 Patrick Trainer: range of values.
120 00:07:03.330 ⇒ 00:07:03.775 Ryan Luke Daque: Yeah.
121 00:07:04.780 ⇒ 00:07:06.430 Patrick Trainer: And then the
122 00:07:06.440 ⇒ 00:07:08.179 Patrick Trainer: look back period.
123 00:07:08.390 ⇒ 00:07:10.679 Patrick Trainer: The detection period
124 00:07:12.210 ⇒ 00:07:12.890 Patrick Trainer: is
125 00:07:13.010 ⇒ 00:07:15.270 Patrick Trainer: that, like orange
126 00:07:17.330 ⇒ 00:07:19.449 Patrick Trainer: kind of like highlighted area.
127 00:07:19.450 ⇒ 00:07:20.140 Ryan Luke Daque: Right.
128 00:07:20.380 ⇒ 00:07:21.310 Patrick Trainer: So
129 00:07:22.460 ⇒ 00:07:23.590 Patrick Trainer: I don’t know
130 00:07:24.210 ⇒ 00:07:26.070 Patrick Trainer: how well you can
131 00:07:26.500 ⇒ 00:07:29.929 Patrick Trainer: see this. Yeah. But like that.
132 00:07:30.120 ⇒ 00:07:31.280 Patrick Trainer: that’s it.
133 00:07:32.500 ⇒ 00:07:38.160 Patrick Trainer: So here. And I’m gonna label it. Now, wait. I gotta draw some
134 00:07:39.050 ⇒ 00:07:41.270 Patrick Trainer: lines on there.
135 00:07:48.990 ⇒ 00:07:52.690 Ryan Luke Daque: Says here that the detection peer, if the detection period
136 00:07:52.810 ⇒ 00:07:58.550 Ryan Luke Daque: is set to 2 days, only data points in the last 2 days will be included in the detection
137 00:07:58.930 ⇒ 00:08:01.599 Ryan Luke Daque: period, and could be flagged, anomalous.
138 00:08:01.600 ⇒ 00:08:02.570 Patrick Trainer: Right.
139 00:08:02.570 ⇒ 00:08:05.359 Ryan Luke Daque: So if it’s 7 days, or if it’s 1 week, then.
140 00:08:06.080 ⇒ 00:08:08.439 Patrick Trainer: It’s actually giving a chance to look
141 00:08:09.460 ⇒ 00:08:18.260 Patrick Trainer: over the past 7 days. So like, I’m kind of thinking of it like a moving average. So art is is the moving average is the moving average.
142 00:08:18.260 ⇒ 00:08:18.850 Ryan Luke Daque: -
143 00:08:18.850 ⇒ 00:08:26.529 Patrick Trainer: In the last 2 days? Or is the moving average in the last 7 days? And where are we in relation to that moving moving average.
144 00:08:26.530 ⇒ 00:08:27.270 Ryan Luke Daque: Versions.
145 00:08:27.520 ⇒ 00:08:30.530 Ryan Luke Daque: So it’s not the exact point within the
146 00:08:31.080 ⇒ 00:08:33.230 Ryan Luke Daque: within the range, but the
147 00:08:33.799 ⇒ 00:08:35.800 Ryan Luke Daque: average of the last 7 days.
148 00:08:35.809 ⇒ 00:08:37.019 Patrick Trainer: Right, right.
149 00:08:37.020 ⇒ 00:08:38.139 Ryan Luke Daque: Give me the link
150 00:08:39.299 ⇒ 00:08:40.250 Ryan Luke Daque: makes sense.
151 00:08:40.250 ⇒ 00:08:41.120 Patrick Trainer: Right?
152 00:08:41.440 ⇒ 00:08:43.210 Patrick Trainer: Yeah, because, like.
153 00:08:44.220 ⇒ 00:08:45.990 Patrick Trainer: if it used
154 00:08:49.810 ⇒ 00:08:52.700 Patrick Trainer: the and I’m trying to think of like the
155 00:08:52.950 ⇒ 00:08:58.139 Patrick Trainer: the, the the counter to it, like, if it used the entire data set.
156 00:09:03.030 ⇒ 00:09:04.520 Ryan Luke Daque: It’s always, if you really use.
157 00:09:04.520 ⇒ 00:09:05.170 Patrick Trainer: Yeah, it.
158 00:09:05.170 ⇒ 00:09:06.839 Ryan Luke Daque: It’s it’s always the mean right.
159 00:09:06.840 ⇒ 00:09:09.549 Patrick Trainer: Yeah, it’s it. Yeah, it’s always gonna be in the middle. Yeah.
160 00:09:09.550 ⇒ 00:09:10.220 Ryan Luke Daque: Yeah.
161 00:09:10.760 ⇒ 00:09:15.130 Patrick Trainer: Yeah, because you’re not able to compare any period to
162 00:09:15.250 ⇒ 00:09:16.650 Patrick Trainer: another period.
163 00:09:17.030 ⇒ 00:09:18.280 Patrick Trainer: Perfect sense.
164 00:09:18.310 ⇒ 00:09:19.450 Patrick Trainer: So
165 00:09:20.460 ⇒ 00:09:21.720 Patrick Trainer: training.
166 00:09:23.330 ⇒ 00:09:24.120 Patrick Trainer: I’m late
167 00:09:24.250 ⇒ 00:09:26.220 Patrick Trainer: thinking out rubber ducking
168 00:09:26.690 ⇒ 00:09:27.470 Patrick Trainer: and
169 00:09:27.640 ⇒ 00:09:28.470 Patrick Trainer: all
170 00:09:29.840 ⇒ 00:09:30.610 Patrick Trainer: donna
171 00:09:31.693 ⇒ 00:09:32.719 Patrick Trainer: and then
172 00:09:33.430 ⇒ 00:09:34.530 Patrick Trainer: the
173 00:09:37.020 ⇒ 00:09:38.190 Patrick Trainer: detection.
174 00:09:39.920 ⇒ 00:09:42.494 Nicolas Sucari: It was a tricky one to figure out right.
175 00:09:42.940 ⇒ 00:09:47.970 Patrick Trainer: Yeah, there, like, I thought their docs were good.
176 00:09:48.790 ⇒ 00:09:51.615 Patrick Trainer: But they’re they’re actually not.
177 00:09:52.180 ⇒ 00:09:52.950 Nicolas Sucari: Half level.
178 00:09:52.950 ⇒ 00:09:58.790 Patrick Trainer: Like some some of them are, but some of them are, are actually really bad.
179 00:09:59.450 ⇒ 00:10:02.609 Ryan Luke Daque: Training the parameters is like, not as
180 00:10:03.910 ⇒ 00:10:05.109 Ryan Luke Daque: where I am.
181 00:10:05.500 ⇒ 00:10:10.569 Patrick Trainer: Right? So let’s there’s like a crazy example. It was.
182 00:10:12.570 ⇒ 00:10:14.520 Patrick Trainer: I think it’s detection period.
183 00:10:14.930 ⇒ 00:10:15.890 Patrick Trainer: Laugh.
184 00:10:18.166 ⇒ 00:10:20.713 Patrick Trainer: Yeah, how it works
185 00:10:21.770 ⇒ 00:10:27.106 Patrick Trainer: detection period defines the detection period like Oh, no, like
186 00:10:29.338 ⇒ 00:10:31.609 Patrick Trainer: you don’t say like.
187 00:10:32.140 ⇒ 00:10:32.670 Ryan Luke Daque: Thanks.
188 00:10:33.405 ⇒ 00:10:33.780 Patrick Trainer: Thing
189 00:10:36.860 ⇒ 00:10:38.200 Patrick Trainer: honestly like
190 00:10:39.775 ⇒ 00:10:43.570 Patrick Trainer: wanna like raise an issue on that and be like this
191 00:10:43.880 ⇒ 00:10:45.290 Patrick Trainer: sucks.
192 00:10:50.630 ⇒ 00:10:54.470 Nicolas Sucari: Okay, I don’t think anyone else is gonna join
193 00:10:54.640 ⇒ 00:10:56.150 Nicolas Sucari: to this meeting today.
194 00:10:56.901 ⇒ 00:11:05.208 Nicolas Sucari: But just to understand Ryan, you already fixed like Kim’s report right like that. She’s working fine. I can see the tables.
195 00:11:05.990 ⇒ 00:11:16.703 Nicolas Sucari: so that’s working perfectly. I can. Probably we can mark that task that we have there in progress? Can we move it to done? Or, yeah, right?
196 00:11:17.100 ⇒ 00:11:18.709 Ryan Luke Daque: We can move that to done.
197 00:11:19.100 ⇒ 00:11:20.310 Nicolas Sucari: Home which is done
198 00:11:20.620 ⇒ 00:11:22.419 Nicolas Sucari: and the one that you have.
199 00:11:22.460 ⇒ 00:11:31.009 Nicolas Sucari: Patrick, about these errors. What do you want to do about that? Are you still like. Try trying to send something, you know, for request, or it’s already.
200 00:11:31.010 ⇒ 00:11:35.420 Patrick Trainer: Yeah, I’ve I’ve got a pull request open and I’m just like
201 00:11:35.870 ⇒ 00:11:38.390 Patrick Trainer: making the changes on
202 00:11:39.260 ⇒ 00:11:45.620 Patrick Trainer: all of the the volume and not or volume tests that are already there, so that, like
203 00:11:45.650 ⇒ 00:11:48.380 Patrick Trainer: that should be done in like the next half hour.
204 00:11:49.230 ⇒ 00:11:50.380 Nicolas Sucari: Excellent. Okay.
205 00:11:50.680 ⇒ 00:12:05.239 Nicolas Sucari: great and then I don’t know if you have the chance. Yesterday to grab some time with each of that with each of us. Well, I don’t need product to like set that key authorization for Snowflake. I think it was.
206 00:12:06.430 ⇒ 00:12:07.229 Nicolas Sucari: Thanks for what it was.
207 00:12:07.230 ⇒ 00:12:08.940 Patrick Trainer: Yes, I
208 00:12:10.290 ⇒ 00:12:18.839 Patrick Trainer: we’ll probably just reach out to you, you and Ryan today. The rest I can hit like Monday or Tuesday.
209 00:12:19.510 ⇒ 00:12:20.130 Patrick Trainer: Okay.
210 00:12:20.130 ⇒ 00:12:21.650 Nicolas Sucari: Depending, depending on.
211 00:12:21.650 ⇒ 00:12:23.700 Patrick Trainer: Yeah, depending on when people are back.
212 00:12:23.700 ⇒ 00:12:27.159 Nicolas Sucari: This is for Snowflake, or for what is that? Do we need.
213 00:12:27.160 ⇒ 00:12:29.729 Patrick Trainer: Yeah, it’s for Snowflake. It’s like, instead of
214 00:12:30.738 ⇒ 00:12:34.030 Patrick Trainer: like, when you’re developing like, Dbt.
215 00:12:34.450 ⇒ 00:12:34.830 Nicolas Sucari: Yeah.
216 00:12:34.830 ⇒ 00:12:39.149 Patrick Trainer: Meeting to connect to to Snowflake. You’ll use
217 00:12:40.250 ⇒ 00:12:47.190 Patrick Trainer: a a key pair like a a graphic key instead of a password. It’s just more secure.
218 00:12:48.010 ⇒ 00:12:51.051 Nicolas Sucari: Perfect. Okay, I don’t think.
219 00:12:52.290 ⇒ 00:12:56.260 Nicolas Sucari: yeah, I don’t know if I’m gonna need. I’m I mean, like, I’m using
220 00:12:56.430 ⇒ 00:13:01.310 Nicolas Sucari: Utam’s user to snowflake because I I don’t have like a snowflake user. I think.
221 00:13:01.310 ⇒ 00:13:04.770 Patrick Trainer: Oh, you don’t! I’ll create. I’ll create you user. You should.
222 00:13:04.770 ⇒ 00:13:05.460 Nicolas Sucari: Yeah, okay.
223 00:13:05.460 ⇒ 00:13:06.559 Patrick Trainer: Yeah, you should have your own.
224 00:13:06.896 ⇒ 00:13:09.590 Nicolas Sucari: I was using you them because I was.
225 00:13:09.590 ⇒ 00:13:09.940 Patrick Trainer: Yeah.
226 00:13:09.940 ⇒ 00:13:14.420 Nicolas Sucari: Recording or doing anything yet. So, but yeah, if you can create me a user, and it doesn’t like.
227 00:13:14.550 ⇒ 00:13:15.379 Nicolas Sucari: yeah, yeah.
228 00:13:15.380 ⇒ 00:13:16.689 Patrick Trainer: Yeah. Will you?
229 00:13:16.690 ⇒ 00:13:17.040 Nicolas Sucari: Or something.
230 00:13:17.040 ⇒ 00:13:20.439 Patrick Trainer: Create like just a a ticket for that.
231 00:13:20.710 ⇒ 00:13:22.490 Patrick Trainer: So I don’t lose it.
232 00:13:23.960 ⇒ 00:13:24.935 Nicolas Sucari: Totally.
233 00:13:26.050 ⇒ 00:13:27.920 Nicolas Sucari: I’m gonna add even backlog
234 00:13:28.880 ⇒ 00:13:32.070 Nicolas Sucari: brain with disease brain for objections. Probably
235 00:13:40.020 ⇒ 00:13:41.210 Nicolas Sucari: we. Okay.
236 00:13:42.070 ⇒ 00:13:43.250 Nicolas Sucari: and then.
237 00:13:45.000 ⇒ 00:13:48.604 Nicolas Sucari: yeah, I’m gonna assign to you, but
238 00:13:51.260 ⇒ 00:13:52.590 Nicolas Sucari: excellent.
239 00:13:53.780 ⇒ 00:14:02.389 Nicolas Sucari: Yeah. And then I don’t. I don’t know if we have like anything else for now, Ryan. I don’t know if you have like any other thing that you need to be working on.
240 00:14:03.090 ⇒ 00:14:05.150 Nicolas Sucari: Are you working on something else?
241 00:14:05.520 ⇒ 00:14:08.150 Ryan Luke Daque: I’m I’m like, Yeah, I’m working on the
242 00:14:08.290 ⇒ 00:14:09.260 Ryan Luke Daque: slack.
243 00:14:10.440 ⇒ 00:14:10.970 Nicolas Sucari: Yeah, okay.
244 00:14:10.970 ⇒ 00:14:17.149 Ryan Luke Daque: Files integration. I’m still like trying to research like what the best way to do that. So I was like thinking, maybe
245 00:14:17.870 ⇒ 00:14:27.050 Ryan Luke Daque: creating a lambda function or something, just to load all the Csp data into Snowflake. But yeah, that’s I’m still researching, basically the best
246 00:14:27.700 ⇒ 00:14:33.110 Ryan Luke Daque: way to do that. Unless maybe, Patrick, you have any ideas like what? What the best way to do that would be.
247 00:14:33.684 ⇒ 00:14:38.029 Patrick Trainer: Sorry I missed the i i i missed the question.
248 00:14:38.030 ⇒ 00:14:42.189 Ryan Luke Daque: Oh, no problem. It’s like the I’m starting to research on
249 00:14:42.270 ⇒ 00:14:48.390 Ryan Luke Daque: like integrating the slack files that we are getting. The Csv files excel, S files into
250 00:14:49.665 ⇒ 00:14:50.589 Ryan Luke Daque: snowflake.
251 00:14:50.890 ⇒ 00:14:53.229 Ryan Luke Daque: Create that data pipeline for that one.
252 00:14:53.230 ⇒ 00:14:53.650 Patrick Trainer: Right
253 00:14:54.490 ⇒ 00:14:59.119 Ryan Luke Daque: Like thinking, maybe just create a lambda function, or whatever be below that. But.
254 00:14:59.450 ⇒ 00:15:02.089 Patrick Trainer: So where are the
255 00:15:02.160 ⇒ 00:15:04.140 Patrick Trainer: files like?
256 00:15:04.630 ⇒ 00:15:05.999 Patrick Trainer: Where did those live?
257 00:15:06.220 ⇒ 00:15:08.330 Ryan Luke Daque: They’re in the Channel. They they’re
258 00:15:08.490 ⇒ 00:15:11.060 Ryan Luke Daque: and internal.
259 00:15:11.060 ⇒ 00:15:18.790 Nicolas Sucari: Process today is, I think, like full parts. We are receiving these from post pilot. They already like they.
260 00:15:19.060 ⇒ 00:15:26.520 Nicolas Sucari: There is like an automatic email that it is sent to us, and we receive that Csv file through a Channel slack channel. So we have that.
261 00:15:26.520 ⇒ 00:15:28.790 Patrick Trainer: Yeah, yeah, there’s it’s an email. That’s right.
262 00:15:28.790 ⇒ 00:15:29.130 Nicolas Sucari: Yeah.
263 00:15:29.465 ⇒ 00:15:35.160 Patrick Trainer: Can we? Can we, from post pilot? Can we just dump those into an S. 3 bucket.
264 00:15:35.800 ⇒ 00:15:42.159 Ryan Luke Daque: I’m not sure if the all of these are from post Pilot. Maybe these are. Some of these are like scheduled reports.
265 00:15:43.030 ⇒ 00:15:43.520 Ryan Luke Daque: February.
266 00:15:43.520 ⇒ 00:15:45.010 Nicolas Sucari: Where they are coming from.
267 00:15:45.460 ⇒ 00:15:47.418 Ryan Luke Daque: Like one is from attentive
268 00:15:48.210 ⇒ 00:15:51.560 Patrick Trainer: Right? I mean, we can like it
269 00:15:52.800 ⇒ 00:15:56.040 Patrick Trainer: regardless of like the the specifics like
270 00:15:56.120 ⇒ 00:16:00.060 Patrick Trainer: is, can we get those? If we can get those Csv’s
271 00:16:00.110 ⇒ 00:16:10.879 Patrick Trainer: into an S. 3 bucket we can set up snow pipe, or just like a an S. 3 integration, and then, whenever anything is loaded.
272 00:16:12.330 ⇒ 00:16:16.489 Patrick Trainer: or or like dropped into into S. 3. It’ll trigger.
273 00:16:16.490 ⇒ 00:16:16.890 Nicolas Sucari: Create it.
274 00:16:16.890 ⇒ 00:16:19.789 Patrick Trainer: Snow pipe, and it’ll load it. Load it into snowflake.
275 00:16:20.225 ⇒ 00:16:23.240 Patrick Trainer: That’s like that’s like snowflake bread and butter.
276 00:16:23.670 ⇒ 00:16:31.349 Ryan Luke Daque: Yeah, that’s what I was thinking, maybe because there’s no easy way to get like this from slack to an S 3 bucket. So maybe I’ll have to create this.
277 00:16:31.350 ⇒ 00:16:32.910 Patrick Trainer: They’re they’re they’re
278 00:16:33.600 ⇒ 00:16:34.520 Patrick Trainer: might
279 00:16:35.530 ⇒ 00:16:38.132 Patrick Trainer: the like. There’s
280 00:16:45.640 ⇒ 00:16:47.840 Patrick Trainer: There might be like either.
281 00:16:48.970 ⇒ 00:16:59.409 Nicolas Sucari: But like like the the best way to do this is like to understand if we can do it directly from post pipelot, attentive, or any other tool directly into Snowflake. Right like that’s like the.
282 00:16:59.410 ⇒ 00:17:03.210 Patrick Trainer: Yeah, yeah, that’s that’s that’s that’s the right way to do it. Yeah.
283 00:17:03.490 ⇒ 00:17:04.020 Patrick Trainer: yeah,
284 00:17:05.390 ⇒ 00:17:07.421 Ryan Luke Daque: Yeah, otherwise, yeah, we there’s.
285 00:17:07.760 ⇒ 00:17:11.769 Patrick Trainer: Like, I imagine. I imagine there’s some like aws.
286 00:17:12.089 ⇒ 00:17:13.040 Patrick Trainer: the.
287 00:17:13.490 ⇒ 00:17:14.740 Nicolas Sucari: Trick, yeah, yeah.
288 00:17:14.740 ⇒ 00:17:19.720 Patrick Trainer: Yeah, like service. Like, I was thinking, like ses, like, simple email service.
289 00:17:19.720 ⇒ 00:17:20.919 Ryan Luke Daque: Or like the queue.
290 00:17:20.920 ⇒ 00:17:21.920 Patrick Trainer: New service.
291 00:17:22.069 ⇒ 00:17:24.820 Patrick Trainer: or even SMS,
292 00:17:27.869 ⇒ 00:17:30.989 Patrick Trainer: or we can just like hack something together that
293 00:17:31.030 ⇒ 00:17:35.140 Patrick Trainer: horses emails. And then just like forwards
294 00:17:36.880 ⇒ 00:17:49.868 Patrick Trainer: like, I love using app script for these sorts of things, but like, if it just, we send it to a gmail inbox that only receives those we can filter that, and then.
295 00:17:50.554 ⇒ 00:17:50.870 Ryan Luke Daque: Mean.
296 00:17:51.270 ⇒ 00:17:54.720 Patrick Trainer: And and then like route it to to S. 3.
297 00:17:54.720 ⇒ 00:17:55.840 Ryan Luke Daque: S. 3. Because that.
298 00:17:55.840 ⇒ 00:17:58.240 Patrick Trainer: Like, that’s yeah. That’s pretty simple.
299 00:17:59.430 ⇒ 00:18:01.939 Patrick Trainer: Yeah, we just like post it to S, 3.
300 00:18:02.890 ⇒ 00:18:04.090 Nicolas Sucari: And and without.
301 00:18:04.490 ⇒ 00:18:07.729 Patrick Trainer: Plan B or plan C, so.
302 00:18:08.490 ⇒ 00:18:13.529 Ryan Luke Daque: But then we’ll have to do some data transformation to make sure there’s no duplicates and stuff like that right?
303 00:18:13.690 ⇒ 00:18:15.858 Ryan Luke Daque: Or we do that in Snowflake, in in.
304 00:18:16.130 ⇒ 00:18:20.500 Patrick Trainer: Yeah, yeah, I think we would do that and stuff like, yeah, we we dump it raw.
305 00:18:20.620 ⇒ 00:18:22.959 Patrick Trainer: And then, yeah, and then stage it.
306 00:18:23.690 ⇒ 00:18:24.340 Ryan Luke Daque: Okay.
307 00:18:25.010 ⇒ 00:18:26.019 Ryan Luke Daque: yeah, I’ll I’ll.
308 00:18:26.020 ⇒ 00:18:26.450 Patrick Trainer: Or.
309 00:18:26.450 ⇒ 00:18:27.080 Ryan Luke Daque: And mark.
310 00:18:27.250 ⇒ 00:18:28.290 Ryan Luke Daque: or.
311 00:18:29.370 ⇒ 00:18:32.539 Patrick Trainer: If we could send it to 5 Tran, because there’s a.
312 00:18:32.820 ⇒ 00:18:34.237 Ryan Luke Daque: Oh, right! Connector.
313 00:18:35.450 ⇒ 00:18:39.150 Patrick Trainer: Ye? Oh, yeah, like we can. There’s a connector, and we can
314 00:18:39.550 ⇒ 00:18:40.940 Patrick Trainer: like if there’s a post.
315 00:18:41.880 ⇒ 00:18:47.819 Patrick Trainer: If there’s a post pilot connector, and then we can set like an SS. 3 as a destination.
316 00:18:50.460 ⇒ 00:19:03.000 Nicolas Sucari: So we need to understand, like the documentation from post pilots and intent to understand, if that is possible, right like, let’s try to do it like the right way directly from both pilots attentive to free and then to Snowflake.
317 00:19:03.170 ⇒ 00:19:10.280 Nicolas Sucari: And if that’s not possible, we can go with Plan B, and see how we can do it in Google Drive, or something like that. What do you think.
318 00:19:10.800 ⇒ 00:19:11.340 Patrick Trainer: Right.
319 00:19:11.340 ⇒ 00:19:12.100 Nicolas Sucari: Yeah, there’s also.
320 00:19:12.100 ⇒ 00:19:13.580 Patrick Trainer: Some other products.
321 00:19:15.290 ⇒ 00:19:18.750 Patrick Trainer: that we can go through. But yeah, let’s let’s start with that step
322 00:19:19.100 ⇒ 00:19:21.399 Patrick Trainer: before we yeah get ahead of ourselves.
323 00:19:22.750 ⇒ 00:19:23.409 Nicolas Sucari: Like sense.
324 00:19:23.990 ⇒ 00:19:24.840 Nicolas Sucari: Okay.
325 00:19:26.270 ⇒ 00:19:40.850 Nicolas Sucari: cool. I don’t think we have like anything else for now, probably for next week, where we need to start working on having the teams report Ryan on real. Probably like we need to start understanding, if
326 00:19:41.040 ⇒ 00:19:43.920 Nicolas Sucari: is something possible to do. Like to have like the
327 00:19:43.990 ⇒ 00:19:48.320 Nicolas Sucari: light dash report, but instead of using light dash, move everything to real.
328 00:19:48.640 ⇒ 00:19:49.110 Ryan Luke Daque: You mean.
329 00:19:49.110 ⇒ 00:19:49.800 Nicolas Sucari: That happened.
330 00:19:49.800 ⇒ 00:19:51.310 Ryan Luke Daque: Weekly report right.
331 00:19:51.660 ⇒ 00:20:00.559 Nicolas Sucari: Yeah, so that we can show her how we are using real and like, if she has all the information there, probably she will start using it.
332 00:20:01.420 ⇒ 00:20:02.340 Ryan Luke Daque: Sounds good.
333 00:20:02.740 ⇒ 00:20:05.589 Ryan Luke Daque: Yeah, we can. We can do that for next week, I guess.
334 00:20:06.040 ⇒ 00:20:06.760 Ryan Luke Daque: Yeah.
335 00:20:07.110 ⇒ 00:20:10.518 Nicolas Sucari: Excellent. Okay, and then,
336 00:20:11.210 ⇒ 00:20:12.460 Nicolas Sucari: yeah, I think.
337 00:20:13.048 ⇒ 00:20:17.429 Nicolas Sucari: Patrick, we have, like 3 or 4 tickets about security
338 00:20:17.470 ⇒ 00:20:21.869 Nicolas Sucari: on ready this week. Probably we should like continue with that right.
339 00:20:23.604 ⇒ 00:20:27.240 Patrick Trainer: Yeah, I’m gonna reach out to you and Ryan.
340 00:20:27.460 ⇒ 00:20:28.420 Patrick Trainer: Excellent.
341 00:20:30.820 ⇒ 00:20:31.500 Nicolas Sucari: Okay.
342 00:20:31.700 ⇒ 00:20:32.729 Patrick Trainer: Should be today.
343 00:20:34.120 ⇒ 00:20:41.410 Nicolas Sucari: Perfect. Okay, thank you, guys, I don’t have like anything else, for now Jacob is out today. Utam, I really don’t know.
344 00:20:42.920 ⇒ 00:20:47.670 Nicolas Sucari: I sent him a message before, but he didn’t answer, so I don’t know if he’s around or not.
345 00:20:48.340 ⇒ 00:20:55.230 Nicolas Sucari: and we are having a meeting with Ben and Dan later today. So probably we’ll gather some more requirements or something. Okay.
346 00:20:55.660 ⇒ 00:20:56.350 Nicolas Sucari: cool.
347 00:20:58.110 ⇒ 00:21:01.434 Nicolas Sucari: excellent. Thank you. Guys talk to you.
348 00:21:01.910 ⇒ 00:21:02.600 Ryan Luke Daque: Someday.
349 00:21:02.710 ⇒ 00:21:03.550 Ryan Luke Daque: See, y’all.
350 00:21:03.550 ⇒ 00:21:04.300 Nicolas Sucari: Bye, bye.