Meeting Title: MatterMore x Brainforge | Standup Date: 2025-04-25 Meeting participants: Annie Yu, Trevor Cohen, Luke Daque, Uttam Kumaran, Amber Lin
WEBVTT
1 00:01:44.020 ⇒ 00:01:44.830 Trevor Cohen: Hello!
2 00:01:46.880 ⇒ 00:01:48.689 Trevor Cohen: Just us and the note takers.
3 00:01:49.150 ⇒ 00:01:50.700 Uttam Kumaran: Yes, yeah.
4 00:01:51.200 ⇒ 00:01:52.570 Uttam Kumaran: Yeah. 2 of them.
5 00:01:53.870 ⇒ 00:01:54.960 Trevor Cohen: Out of control.
6 00:01:56.550 ⇒ 00:01:57.600 Trevor Cohen: Annie.
7 00:01:58.240 ⇒ 00:02:00.300 Annie Yu: Hello, Trevor! How’s it going.
8 00:02:00.300 ⇒ 00:02:01.800 Trevor Cohen: I’m pretty good. How are you?
9 00:02:01.960 ⇒ 00:02:04.327 Annie Yu: Good. It’s it’s Friday.
10 00:02:06.480 ⇒ 00:02:09.560 Trevor Cohen: Forgetting any, any big plans. This weekend.
11 00:02:10.963 ⇒ 00:02:16.800 Annie Yu: Not really just probably wanna go shopping retail therapy.
12 00:02:17.070 ⇒ 00:02:18.150 Trevor Cohen: Nice like.
13 00:02:18.150 ⇒ 00:02:20.329 Uttam Kumaran: Are we talking clothes? Or what? Yeah.
14 00:02:20.330 ⇒ 00:02:33.319 Annie Yu: Yeah, most likely clothes. I found this like vintage store. I like, I’m really bad thrifting or like vintage shopping, but I’m like trying to get into it. So I’m gonna check it out.
15 00:02:33.320 ⇒ 00:02:35.010 Uttam Kumaran: Yeah, there’s nothing to be good at there.
16 00:02:35.010 ⇒ 00:02:40.170 Annie Yu: I just find anything that I like in my size. And it’s it’s.
17 00:02:40.170 ⇒ 00:02:43.669 Uttam Kumaran: That is tough. You have to really like I I used to go to like
18 00:02:43.810 ⇒ 00:02:57.302 Uttam Kumaran: goodwill a lot. I still go to like goodwill and 7. There’s some other thrift stores. But I go and like, put on a podcast and I like rack, rack, rack. I just go every rack. And I’m like I have to look at every single one.
19 00:02:57.590 ⇒ 00:03:01.699 Trevor Cohen: I was. Gonna say, I’m big into. I’m big into vintage shopping also, and
20 00:03:01.870 ⇒ 00:03:04.429 Trevor Cohen: going to goodwill is a real commitment, because.
21 00:03:04.430 ⇒ 00:03:06.199 Uttam Kumaran: Yeah, some stuff is really bad.
22 00:03:06.340 ⇒ 00:03:23.769 Trevor Cohen: Yeah, I have. I’m really lucky. I have this great vintage store right near my apartment. That is like, it’s a really small section, highly curated, and they just have. They have all these different styles. But, as a friend of mine put it, it’s like, everything is a style like everything is like interesting. And every time I go I get something. It’s really
23 00:03:23.770 ⇒ 00:03:25.090 Trevor Cohen: nice. Yeah.
24 00:03:27.330 ⇒ 00:03:31.380 Trevor Cohen: So I support you. Make sure. On Monday you can show us your fines. Yeah.
25 00:03:31.380 ⇒ 00:03:36.829 Uttam Kumaran: Yeah, yeah. You know, I got this great like, I got this double Xl Carhart Hoodie.
26 00:03:37.290 ⇒ 00:03:41.380 Uttam Kumaran: Great, fine, huge like it. And it’s so warm.
27 00:03:41.640 ⇒ 00:03:43.539 Uttam Kumaran: Got great sweaters.
28 00:03:44.900 ⇒ 00:03:46.720 Uttam Kumaran: I got a great lakers.
29 00:03:47.450 ⇒ 00:03:49.550 Uttam Kumaran: Champion sweatshirt.
30 00:03:50.236 ⇒ 00:03:54.030 Uttam Kumaran: Just like that’s great. Here, I’ll I’ll show you my favorite through. Fine one. Sec.
31 00:03:54.030 ⇒ 00:03:55.120 Uttam Kumaran: Okay, okay.
32 00:03:56.860 ⇒ 00:03:59.908 Annie Yu: But do you need warm clothes down there?
33 00:04:00.290 ⇒ 00:04:05.540 Annie Yu: I like wearing it to like dinner, but then you have to take it off like, yeah.
34 00:04:05.950 ⇒ 00:04:08.370 Trevor Cohen: Okay, here. It’s an embroidered jacket.
35 00:04:08.370 ⇒ 00:04:09.760 Uttam Kumaran: Wow!
36 00:04:09.760 ⇒ 00:04:10.510 Trevor Cohen: 5.
37 00:04:11.040 ⇒ 00:04:12.189 Uttam Kumaran: Oh, my! Gosh!
38 00:04:12.190 ⇒ 00:04:13.949 Trevor Cohen: It’s Sag Harbor. It’s really fun.
39 00:04:13.950 ⇒ 00:04:15.800 Uttam Kumaran: No way. Let’s go.
40 00:04:15.840 ⇒ 00:04:17.640 Trevor Cohen: Yeah, it’s really good.
41 00:04:17.640 ⇒ 00:04:19.350 Trevor Cohen: Yeah, that’s my recipes.
42 00:04:25.360 ⇒ 00:04:27.189 Trevor Cohen: Alright, that’s it all I got for you.
43 00:04:29.150 ⇒ 00:04:46.430 Trevor Cohen: Oh, you’ll appreciate this. So I had to get this. I had to get this like new glass for this frame. This like vintage frame I got, and it’s a you know. It’s a fancy pigeon. And so I took it to this guy like, I’ve been going to frame bridge. And I just like, I don’t. Wanna. This is like, not that good also don’t wanna support them. And so.
44 00:04:46.430 ⇒ 00:04:47.330 Uttam Kumaran: Yeah, yeah, yeah.
45 00:04:47.330 ⇒ 00:05:01.440 Trevor Cohen: I went to a local guy and he looks, and he’s like, Oh, that’s a that’s like a passenger, or like whatever pigeon it was like, that’s whatever pigeon I was like. Oh, do you know about pigeons? He’s like, yeah, back in Bangladesh I’ve got 1,200 pigeons. I have a pigeon farm.
46 00:05:01.440 ⇒ 00:05:02.560 Trevor Cohen: They show me all the time
47 00:05:02.560 ⇒ 00:05:10.470 Trevor Cohen: pictures of his pigeons, and it was just so cute. And he’s like I’m gonna do this for you right away, and it just I don’t know it was. It’s exhaustive. Why, you guys, shop local.
48 00:05:10.470 ⇒ 00:05:17.300 Uttam Kumaran: Wow, that’s incredible. Yeah, I need to go to frames here. But like, I’m just gonna go to target, I guess, like.
49 00:05:17.300 ⇒ 00:05:18.020 Trevor Cohen: Yeah.
50 00:05:18.300 ⇒ 00:05:23.709 Uttam Kumaran: Here. It’s like dude here. It’s like really lacking in terms of like small shops for stuff.
51 00:05:23.850 ⇒ 00:05:24.820 Trevor Cohen: Where are you? Again.
52 00:05:24.820 ⇒ 00:05:25.880 Uttam Kumaran: I’m in Austin.
53 00:05:25.880 ⇒ 00:05:28.200 Trevor Cohen: Oh, in Austin, really, that’s surprising. I feel like.
54 00:05:28.200 ⇒ 00:05:33.940 Uttam Kumaran: Good. No, there’s good. I mean I could go. But like target is like right here.
55 00:05:34.230 ⇒ 00:05:34.690 Trevor Cohen: Makes sense.
56 00:05:34.690 ⇒ 00:05:36.920 Uttam Kumaran: And yeah, I don’t know.
57 00:05:37.420 ⇒ 00:05:41.940 Trevor Cohen: I go to? I go to blink, is it? Or whatever like the.
58 00:05:42.375 ⇒ 00:05:43.680 Uttam Kumaran: Yeah. Yeah. Yeah.
59 00:05:43.680 ⇒ 00:05:45.329 Trevor Cohen: This is usually pretty good, but.
60 00:05:45.330 ⇒ 00:05:46.390 Uttam Kumaran: Yeah. Yeah.
61 00:05:48.660 ⇒ 00:05:49.960 Uttam Kumaran: Nice. Okay.
62 00:05:50.610 ⇒ 00:05:52.900 Trevor Cohen: We got. Let’s back to work.
63 00:05:55.022 ⇒ 00:05:58.349 Uttam Kumaran: Let me. I’m I guess I can just pull up
64 00:05:58.490 ⇒ 00:06:02.810 Uttam Kumaran: linear. I don’t think we’ve completely added all of the
65 00:06:02.920 ⇒ 00:06:08.209 Uttam Kumaran: I think we’ve added some of the core tickets, but I don’t think we’ve scoped everything out entirely.
66 00:06:08.210 ⇒ 00:06:08.630 Trevor Cohen: Okay.
67 00:06:08.804 ⇒ 00:06:11.239 Uttam Kumaran: But I don’t know if you had a chance to take a look at
68 00:06:11.690 ⇒ 00:06:13.180 Uttam Kumaran: any of the tickets in there.
69 00:06:14.550 ⇒ 00:06:17.399 Trevor Cohen: Let’s take a look. I did not start last call.
70 00:06:19.930 ⇒ 00:06:21.430 Uttam Kumaran: Wait! Wait! What.
71 00:06:22.580 ⇒ 00:06:24.389 Trevor Cohen: I? I said, not since our last call.
72 00:06:24.390 ⇒ 00:06:26.140 Uttam Kumaran: Oh, not since last call. Okay, okay, cool.
73 00:06:26.610 ⇒ 00:06:32.750 Uttam Kumaran: So I think that we’re just gonna make sure that everything ends up in there. So that’s why.
74 00:06:32.750 ⇒ 00:06:35.209 Trevor Cohen: Any tickets. Now, for some reason.
75 00:06:35.210 ⇒ 00:06:37.619 Uttam Kumaran: Click on all issues in the top left.
76 00:06:41.770 ⇒ 00:06:43.380 Uttam Kumaran: instead of just active.
77 00:06:45.240 ⇒ 00:06:45.810 Trevor Cohen: M.
78 00:06:51.920 ⇒ 00:06:56.000 Trevor Cohen: Oh, got it cool, got it? Got it? Okay, sweet.
79 00:06:57.990 ⇒ 00:07:02.790 Uttam Kumaran: So I think I’m just gonna work with amber to get that scoped for Monday, and then we can.
80 00:07:04.540 ⇒ 00:07:10.769 Trevor Cohen: let’s just let’s just make sure to add them all to the project. So what is it called Xyz analytic? I’ll see that right now.
81 00:07:10.770 ⇒ 00:07:11.959 Uttam Kumaran: Oh, okay. Okay.
82 00:07:17.060 ⇒ 00:07:22.879 Trevor Cohen: Cool. Alright! Alright, that’s great. Let me see, I’m just reading the table. Schemas. Yep.
83 00:07:25.050 ⇒ 00:07:30.399 Trevor Cohen: cool. I’ll just make a task for myself. Set up. Dbt, cause that’s what I’m up to right now.
84 00:07:30.610 ⇒ 00:07:31.240 Uttam Kumaran: Okay.
85 00:07:34.812 ⇒ 00:07:39.460 Uttam Kumaran: and then I know, Annie, you sent the success factors.
86 00:07:41.740 ⇒ 00:07:43.769 Uttam Kumaran: Field. Do we want to talk about that?
87 00:07:44.677 ⇒ 00:07:52.632 Annie Yu: Yeah, I was. What I did was I went through that. What’s that? Not work, Doc, but that doc?
88 00:07:53.280 ⇒ 00:07:56.540 Annie Yu: And then, based on those fields. I found
89 00:07:57.020 ⇒ 00:08:00.890 Annie Yu: the kind of the field name in success factors.
90 00:08:01.220 ⇒ 00:08:05.789 Annie Yu: But I think there are some that I can’t find, which
91 00:08:06.080 ⇒ 00:08:13.299 Annie Yu: I’m just not sure if they will be coming from success factors, or that’s some manual work that the client will Will
92 00:08:13.520 ⇒ 00:08:17.570 Annie Yu: would dealt with. But I think we, I think either way, we can
93 00:08:17.770 ⇒ 00:08:23.840 Annie Yu: just use these like this dog as the fields to synthesize.
94 00:08:24.250 ⇒ 00:08:24.880 Uttam Kumaran: Okay.
95 00:08:26.010 ⇒ 00:08:36.620 Annie Yu: So yeah, things like CEO, direct leader, hierarchy and distance to CEO. Like I, I can find those in success factors. But I mean.
96 00:08:36.914 ⇒ 00:08:38.089 Trevor Cohen: From open source. Then.
97 00:08:38.090 ⇒ 00:08:43.859 Annie Yu: Yeah. But then for synthetic data, we can get them all right.
98 00:08:43.860 ⇒ 00:08:44.490 Uttam Kumaran: Yeah.
99 00:08:44.490 ⇒ 00:08:47.040 Annie Yu: Doesn’t matter where they will be coming from.
100 00:08:47.920 ⇒ 00:08:52.820 Annie Yu: So I think with that we can, we can move forward with the synthetic.
101 00:08:53.200 ⇒ 00:09:02.440 Annie Yu: Yeah. Oh, maybe we need to spend more time on the graph to decide what fields we want. Specifically.
102 00:09:04.310 ⇒ 00:09:06.270 Trevor Cohen: Yeah, I mean, I think that
103 00:09:06.560 ⇒ 00:09:32.079 Trevor Cohen: part of the thing for doing the synthetic data is figuring out what the queries will be to create the models and like the like, the transform tables that we want. But if we don’t know exactly what the source tables will look like, we can always just kind of start from a table that is joined, you know, with with at least some of the fields that we want. And then, once we figure out, you know, we could just like backtrack and do that that joining thing in Dbt. Later.
104 00:09:32.300 ⇒ 00:09:32.910 Uttam Kumaran: Okay.
105 00:09:33.180 ⇒ 00:09:36.500 Annie Yu: How will we join data from graph
106 00:09:36.930 ⇒ 00:09:42.230 Annie Yu: to success factors? That’s 1 thing I can’t seem to figure out.
107 00:09:42.230 ⇒ 00:09:51.089 Trevor Cohen: Probably on on email. I would guess because everything will be anonymized. But in the same way. So that join queue will still work like, yeah.
108 00:09:51.460 ⇒ 00:09:53.229 Uttam Kumaran: Okay. I’ll just comment that here.
109 00:09:55.900 ⇒ 00:09:59.469 Trevor Cohen: I don’t think there’s like a different employee, Id or anything. So I think email is the way to go.
110 00:10:00.172 ⇒ 00:10:01.380 Annie Yu: That makes sense.
111 00:10:05.300 ⇒ 00:10:05.950 Uttam Kumaran: Okay.
112 00:10:07.080 ⇒ 00:10:14.059 Trevor Cohen: And you know, if you want like, and it’s going to be a hash. But you know, like in the synthetic data, it could just be email. It’s fine.
113 00:10:14.060 ⇒ 00:10:14.670 Uttam Kumaran: Okay.
114 00:10:15.180 ⇒ 00:10:27.130 Annie Yu: So the email just to make sure that email address here from success factors, not people’s personal email. But that would be like outlook.com or so like the work email.
115 00:10:27.700 ⇒ 00:10:34.420 Trevor Cohen: Yeah, I mean, it’s gonna be people’s work email across all the systems. Cause that, you know, like within Microsoft graph to the login with their work email.
116 00:10:34.670 ⇒ 00:10:35.290 Annie Yu: Yeah.
117 00:10:36.160 ⇒ 00:10:36.770 Trevor Cohen: Yeah.
118 00:10:41.130 ⇒ 00:10:46.590 Uttam Kumaran: Okay. So then I think on probably Monday, I think, Luke, you’re on the call.
119 00:10:46.957 ⇒ 00:10:49.099 Uttam Kumaran: Oh, wait! Maybe he’s not on the call.
120 00:10:49.536 ⇒ 00:10:55.359 Uttam Kumaran: I think I’ll I’ll talk to Luke about starting to get the synthetic scripts running on Monday.
121 00:10:55.360 ⇒ 00:11:17.880 Trevor Cohen: Okay, sweet, alright, that’s awesome. So yeah, so I think that like making synthetic scripts generate the data in the schema that we’re gonna receive it in is good. And then, you know, I guess you guys will start writing the queries to create the models that we want from there, and I don’t think I’m a blocker to you with the Dbc setup to do that, because, you know, you can always just run the queries like directly in
122 00:11:18.010 ⇒ 00:11:24.080 Trevor Cohen: in a big query, but as soon as that’s set up I’ll let you know, and we can start, like, you know, like
123 00:11:24.500 ⇒ 00:11:27.774 Trevor Cohen: like porting those queries into Dbt.
124 00:11:28.850 ⇒ 00:11:34.900 Uttam Kumaran: So in linear. I mark Ryan, and we can say, that’s to do so. That’s fine.
125 00:11:35.754 ⇒ 00:11:39.649 Uttam Kumaran: Define core dimensions. So yeah, I’ll assign this thing as well.
126 00:11:42.710 ⇒ 00:11:44.020 Uttam Kumaran: These.
127 00:11:44.210 ⇒ 00:11:45.580 Uttam Kumaran: Okay.
128 00:11:51.850 ⇒ 00:11:53.919 Uttam Kumaran: okay, the audit logs.
129 00:11:57.640 ⇒ 00:11:58.979 Uttam Kumaran: And then.
130 00:11:59.490 ⇒ 00:12:00.729 Trevor Cohen: What about audit logs.
131 00:12:01.400 ⇒ 00:12:03.440 Uttam Kumaran: We just have a ticket for
132 00:12:04.530 ⇒ 00:12:07.080 Uttam Kumaran: pulling audit and sign in logs from.
133 00:12:07.420 ⇒ 00:12:08.100 Trevor Cohen: Yeah, cool.
134 00:12:08.100 ⇒ 00:12:12.380 Uttam Kumaran: Yeah, that’s that tool set up.
135 00:12:13.150 ⇒ 00:12:15.519 Uttam Kumaran: Yep, okay. So Trevor is working on that
136 00:12:18.080 ⇒ 00:12:22.469 Uttam Kumaran: and then define core dimensions. Okay, this is also to do.
137 00:12:25.260 ⇒ 00:12:30.539 Uttam Kumaran: And then, yeah, table table schemas. This will just mainly be. Well, I’m gonna do this as
138 00:12:32.840 ⇒ 00:12:39.799 Uttam Kumaran: Dbt initialization. Oh, so you’re initializing Dbt, so can I? I still owe you a zip of.
139 00:12:41.020 ⇒ 00:12:45.360 Trevor Cohen: Yeah, is it that different than just doing Dvt init, and like what it gives you.
140 00:12:45.660 ⇒ 00:12:48.180 Uttam Kumaran: You can do that, and then we’ll just change. We can. Yeah, just do that.
141 00:12:48.180 ⇒ 00:12:51.891 Trevor Cohen: Oh, yeah, that’s what I did already. And I figured if you guys want to make any
142 00:12:53.920 ⇒ 00:12:59.430 Trevor Cohen: but it, yeah, I mean, if it’s if it’s different in that. And it’s easier to just start from that. If you.
143 00:12:59.430 ⇒ 00:12:59.830 Uttam Kumaran: That’s fine!
144 00:12:59.830 ⇒ 00:13:02.850 Trevor Cohen: I haven’t done much, so I can just replace it.
145 00:13:03.070 ⇒ 00:13:05.610 Uttam Kumaran: Okay. I’ll just export it now and.
146 00:13:05.610 ⇒ 00:13:09.520 Trevor Cohen: Okay, I just don’t have much of a sense of how different it is from the default.
147 00:13:09.520 ⇒ 00:13:15.540 Uttam Kumaran: Maybe just like, just keep running, and just make sure it’s connected to Bq and it like you’re you’re able to run like a select
148 00:13:15.700 ⇒ 00:13:19.330 Uttam Kumaran: one or whatever in the query issues, and then we’ll take it from there.
149 00:13:19.680 ⇒ 00:13:34.850 Trevor Cohen: Okay, great, great. All right, I’ll do that. And yeah, yeah, I think there’s a way to like, you know, it’ll have to operate across data sets, because, like by default, it has a default data set. But I’m just, I think, in the models you can like define a different data set per like per
150 00:13:35.610 ⇒ 00:13:39.270 Trevor Cohen: per set of model. Well, I don’t know what all the terminology is.
151 00:13:39.270 ⇒ 00:13:47.899 Uttam Kumaran: So data set. Oh, yeah, so, oh, yeah, exactly. So. The data set is just what we’ll just it’s basically just the the database equivalent.
152 00:13:48.260 ⇒ 00:13:53.680 Trevor Cohen: Right. But what I’m saying is that like for right now for what we’re doing now. Hey, Luke.
153 00:13:55.860 ⇒ 00:13:56.560 Annie Yu: Hello, Luke!
154 00:13:57.130 ⇒ 00:13:58.680 Trevor Cohen: Hey? How’s it going.
155 00:13:59.230 ⇒ 00:14:00.210 Luke Daque: Doing well.
156 00:14:00.210 ⇒ 00:14:01.180 Trevor Cohen: Welcome!
157 00:14:01.390 ⇒ 00:14:02.739 Trevor Cohen: Welcome to the team!
158 00:14:04.460 ⇒ 00:14:08.799 Uttam Kumaran: We’re we started the call talking about vintage clothing, Luke. I don’t know if you.
159 00:14:09.160 ⇒ 00:14:12.979 Uttam Kumaran: if you go thrift shopping, or you do, or you do any vintage shopping.
160 00:14:15.850 ⇒ 00:14:18.200 Luke Daque: I’m not sure I’m familiar.
161 00:14:19.920 ⇒ 00:14:22.940 Uttam Kumaran: It’s just like, you know, it’s just buying old clothes.
162 00:14:23.290 ⇒ 00:14:23.810 Trevor Cohen: Old.
163 00:14:23.810 ⇒ 00:14:24.560 Luke Daque: Oh, you live!
164 00:14:25.270 ⇒ 00:14:26.000 Uttam Kumaran: Yeah, yeah.
165 00:14:26.000 ⇒ 00:14:30.530 Luke Daque: Yeah, gotcha, yeah, we have a lot that
166 00:14:30.670 ⇒ 00:14:33.890 Luke Daque: of that here, basically like vintage clothing.
167 00:14:34.070 ⇒ 00:14:35.120 Luke Daque: Oh, okay.
168 00:14:35.450 ⇒ 00:14:36.450 Trevor Cohen: Where are you, Lou?
169 00:14:37.140 ⇒ 00:14:40.090 Luke Daque: I’m half a world away like in the Philippines.
170 00:14:40.220 ⇒ 00:14:41.440 Luke Daque: Oh, cool! That’s nice!
171 00:14:42.310 ⇒ 00:14:43.030 Trevor Cohen: Sweet.
172 00:14:44.046 ⇒ 00:14:57.099 Trevor Cohen: It also can go bad sometimes. I just got this like 19 nineties mets a mets fan 19 nineties mets like warm up jacket from a guy in Ukraine, and he said it was a large and it is, but it’s a youth, large.
173 00:14:57.500 ⇒ 00:15:00.140 Uttam Kumaran: Oh, yeah, I’ve done that, too.
174 00:15:00.140 ⇒ 00:15:01.360 Trevor Cohen: It smell on me.
175 00:15:01.500 ⇒ 00:15:04.359 Uttam Kumaran: Yeah, I’ve always usually. Now I go like 2 sizes up.
176 00:15:05.492 ⇒ 00:15:07.499 Uttam Kumaran: I found this great
177 00:15:07.630 ⇒ 00:15:13.270 Uttam Kumaran: Tommy Hilfiger pull over. I’ll have to. I’ll bring it on next on Monday. I’ll I’ll bring some. I’ll.
178 00:15:13.270 ⇒ 00:15:14.499 Trevor Cohen: We’ll do a fashion show.
179 00:15:14.500 ⇒ 00:15:16.300 Uttam Kumaran: We do. We can do a fashion, show.
180 00:15:16.300 ⇒ 00:15:17.149 Trevor Cohen: That sounds great.
181 00:15:17.150 ⇒ 00:15:17.690 Trevor Cohen: Yeah.
182 00:15:19.664 ⇒ 00:15:22.449 Trevor Cohen: Okay, cool. We already showed off our art. We can do fashion there.
183 00:15:22.450 ⇒ 00:15:23.540 Uttam Kumaran: Yes, yes.
184 00:15:25.164 ⇒ 00:15:30.290 Uttam Kumaran: Luke, we’re just talking about yeah. Dbt, setup. Oh, yeah. So yeah, continue on the data set piece.
185 00:15:30.290 ⇒ 00:15:42.669 Trevor Cohen: Yeah. So anyway, for this for what we’re doing now, it’s all gonna operate inside that synthetic data set. It’s call synthetic in in bigquery, but, like in general, the way we want to set up the
186 00:15:43.067 ⇒ 00:15:51.100 Trevor Cohen: the app is to be able to run across data sets because we’ll have a separate one for each client. And so I think there’s just it looks like there’s a simple way to do it.
187 00:15:51.100 ⇒ 00:15:53.320 Uttam Kumaran: Yes, you could do that in in Dbt. Pretty easily. So.
188 00:15:53.320 ⇒ 00:16:06.290 Trevor Cohen: Organize into. Yeah, cool. Yeah. So that’s what I’m up to. And yeah, I’ll add, I’m adding, logging log, everything. Instruct log centralized. So that’s where, like airflow logs also. So we can
189 00:16:06.740 ⇒ 00:16:16.150 Trevor Cohen: follow all that. All of I’m just looking at the tool set of things. So what we have done is, yeah. Airflow dags for orchestration. That’s like almost done.
190 00:16:16.625 ⇒ 00:16:20.885 Trevor Cohen: So I’ll just I’ll make a separate subtask to like finalize that
191 00:16:21.240 ⇒ 00:16:21.910 Uttam Kumaran: Okay.
192 00:16:21.910 ⇒ 00:16:26.330 Trevor Cohen: Finalize airflow guys.
193 00:16:27.910 ⇒ 00:16:29.260 Trevor Cohen: Astro cloud.
194 00:16:32.040 ⇒ 00:16:34.473 Trevor Cohen: Okay, that’s that. And then
195 00:16:35.853 ⇒ 00:16:39.369 Trevor Cohen: store credentials. And yeah, so we already, we got that. That’s good.
196 00:16:40.370 ⇒ 00:16:42.820 Trevor Cohen: And we’re using secret manager.
197 00:16:44.940 ⇒ 00:16:46.370 Trevor Cohen: Yeah, I am.
198 00:16:47.680 ⇒ 00:16:49.020 Trevor Cohen: That’s what I’m up to.
199 00:16:50.220 ⇒ 00:16:56.042 Uttam Kumaran: Okay. So I think next week, probably by mid next week, we should have, like the synthetics ready. I think, Luke.
200 00:16:57.220 ⇒ 00:16:59.500 Uttam Kumaran: you have access to the.
201 00:17:00.330 ⇒ 00:17:04.220 Uttam Kumaran: We have access to the repo now. So maybe if you want to start on like those
202 00:17:04.589 ⇒ 00:17:10.630 Uttam Kumaran: reissue like, we can take those synthetic strips, map it to the these schemas, the new basically news
203 00:17:10.910 ⇒ 00:17:13.260 Uttam Kumaran: the new fields that we want to do and
204 00:17:13.470 ⇒ 00:17:16.090 Uttam Kumaran: could try to just run it again. See what we get.
205 00:17:16.740 ⇒ 00:17:17.760 Trevor Cohen: Which Repo.
206 00:17:18.376 ⇒ 00:17:21.310 Uttam Kumaran: Do we have like the dbt repo for.
207 00:17:21.670 ⇒ 00:17:22.000 Trevor Cohen: Yes.
208 00:17:22.000 ⇒ 00:17:22.720 Uttam Kumaran: Yeah.
209 00:17:22.720 ⇒ 00:17:26.547 Trevor Cohen: I have. I have it now, I mean I haven’t. It’s just empty right now, but I’ll share.
210 00:17:27.235 ⇒ 00:17:32.120 Trevor Cohen: I’ll ping you as soon as I push it, but I think you can start writing the queries before you have access to it. Right.
211 00:17:32.120 ⇒ 00:17:36.499 Uttam Kumaran: Yeah. And we can. We can just add our synthetic scripts there so that they live somewhere.
212 00:17:36.500 ⇒ 00:17:39.030 Trevor Cohen: Awesome. Oh, yeah, yeah, perfect. That’s good.
213 00:17:40.747 ⇒ 00:17:48.770 Uttam Kumaran: Luke, any questions there? I know we have some stuff in this document and in that spreadsheet I think ideally, I want to centralize all the columns.
214 00:17:49.737 ⇒ 00:17:53.620 Uttam Kumaran: The column column column, name the
215 00:17:53.830 ⇒ 00:17:58.020 Uttam Kumaran: the type and ex like sort of some expected values.
216 00:17:58.615 ⇒ 00:18:05.560 Uttam Kumaran: As I basically create this, the schema there and then you can use that to build the to modify the python script.
217 00:18:07.760 ⇒ 00:18:09.243 Luke Daque: Yeah, I’ll take a look.
218 00:18:10.150 ⇒ 00:18:15.250 Luke Daque: India, if I have any questions I’ll just like messaging set up, or something.
219 00:18:15.680 ⇒ 00:18:16.250 Uttam Kumaran: Okay.
220 00:18:19.090 ⇒ 00:18:19.730 Uttam Kumaran: Okay.
221 00:18:19.996 ⇒ 00:18:25.329 Annie Yu: One more thing is the bigquery. I still can’t access. I’m not sure if what time you got a chance.
222 00:18:25.330 ⇒ 00:18:28.811 Uttam Kumaran: Oh, I did not get a chance to do it yesterday. Okay, let me.
223 00:18:30.100 ⇒ 00:18:35.429 Uttam Kumaran: alright. I will spend the next. If we’re done, I’ll spend the next 12 min figuring that out.
224 00:18:35.430 ⇒ 00:18:37.389 Trevor Cohen: Okay, I’ll make a task for you. An airflow soon.
225 00:18:37.390 ⇒ 00:18:40.200 Uttam Kumaran: Yes, yes, please make a task for me. Sorry.
226 00:18:40.350 ⇒ 00:18:49.100 Trevor Cohen: No worries. debug bigquery access issues.
227 00:18:49.740 ⇒ 00:18:50.430 Trevor Cohen: Boom.
228 00:18:51.580 ⇒ 00:18:53.749 Trevor Cohen: Alright, you’re assigned no excuses.
229 00:18:53.750 ⇒ 00:18:54.505 Uttam Kumaran: Okay.
230 00:18:55.777 ⇒ 00:18:56.812 Trevor Cohen: Okay, cool.
231 00:18:57.840 ⇒ 00:19:01.109 Trevor Cohen: What else? I gonna say, that might be it.
232 00:19:01.731 ⇒ 00:19:03.840 Trevor Cohen: Yeah. Well, that’s all I got.
233 00:19:03.840 ⇒ 00:19:11.739 Amber Lin: Awesome, and we’re all planning to get this done by Monday is that is that our goal like, what is the timeline we’re looking at here.
234 00:19:11.910 ⇒ 00:19:12.790 Uttam Kumaran: For? What?
235 00:19:12.950 ⇒ 00:19:14.809 Amber Lin: For all these, all this stuff.
236 00:19:14.990 ⇒ 00:19:20.710 Uttam Kumaran: Oh, no way Monday is like Oh, Monday is like the next meeting.
237 00:19:20.710 ⇒ 00:19:22.820 Trevor Cohen: Yeah, we got vintage shopping over the weekend. We don’t.
238 00:19:22.820 ⇒ 00:19:23.460 Uttam Kumaran: Yeah, yeah.
239 00:19:23.460 ⇒ 00:19:24.720 Luke Daque: It is okay.
240 00:19:25.707 ⇒ 00:19:30.482 Uttam Kumaran: I would. I would probably ask Luke once he gets a sense for
241 00:19:31.340 ⇒ 00:19:39.299 Uttam Kumaran: like the the scripts. And what’s what? Basically what he needs to generate for a timeline on that. And then
242 00:19:40.490 ⇒ 00:19:43.240 Uttam Kumaran: I should. I’m gonna figure out this Google thing today.
243 00:19:43.723 ⇒ 00:19:48.639 Uttam Kumaran: And then we have a couple of tickets. So I would just maybe on Monday we can set some due dates for everything.
244 00:19:48.640 ⇒ 00:19:49.160 Amber Lin: Okay.
245 00:19:49.160 ⇒ 00:19:49.609 Uttam Kumaran: But I probably.
246 00:19:49.610 ⇒ 00:19:50.100 Amber Lin: I’m just kidding.
247 00:19:50.100 ⇒ 00:19:51.710 Uttam Kumaran: Directly in the channel. Yeah.
248 00:19:51.710 ⇒ 00:19:52.600 Amber Lin: Okay. Good.
249 00:19:52.970 ⇒ 00:20:00.140 Uttam Kumaran: Probably by Friday I think we should have a we should have everything loaded. I think the deep. Once once we have this, the raw data. The Dbt setup should be pretty quick.
250 00:20:01.670 ⇒ 00:20:08.820 Trevor Cohen: Cool. Alright. So for the I just added to the repo like a script, synthetic data folder. So you can just throw all in there when I give you access.
251 00:20:08.980 ⇒ 00:20:09.580 Uttam Kumaran: Okay.
252 00:20:16.020 ⇒ 00:20:18.330 Uttam Kumaran: okay, cool. That’s all. I had.
253 00:20:19.730 ⇒ 00:20:21.040 Amber Lin: Okay. Okay.
254 00:20:21.340 ⇒ 00:20:23.099 Trevor Cohen: Alright, thanks, crew have a great day.
255 00:20:23.100 ⇒ 00:20:24.490 Uttam Kumaran: Have a good weekend.
256 00:20:25.050 ⇒ 00:20:25.809 Annie Yu: See you.
257 00:20:26.180 ⇒ 00:20:27.150 Amber Lin: Sounds good. Thank you.
258 00:20:27.150 ⇒ 00:20:28.430 Luke Daque: Thanks, Matt, bye-bye.