Meeting Title: MatterMore x Brainforge | Standup Date: 2025-04-28 Meeting participants: Annie Yu, Luke Daque, Trevor Cohen, Amber Lin
WEBVTT
1 00:00:25.270 ⇒ 00:00:25.865 Trevor Cohen: Amber.
2 00:00:27.410 ⇒ 00:00:28.760 Amber Lin: Hi! There!
3 00:00:29.190 ⇒ 00:00:30.130 Trevor Cohen: Morning.
4 00:00:30.710 ⇒ 00:00:32.009 Amber Lin: Good morning!
5 00:00:32.850 ⇒ 00:00:33.469 Trevor Cohen: Let’s do it.
6 00:00:33.730 ⇒ 00:00:37.292 Amber Lin: I’m pretty good. We’re waiting on.
7 00:00:39.490 ⇒ 00:00:41.970 Amber Lin: Okay, quite a few people.
8 00:00:42.730 ⇒ 00:00:44.119 Amber Lin: He’s gonna come today.
9 00:00:44.450 ⇒ 00:00:44.885 Trevor Cohen: Who.
10 00:00:45.620 ⇒ 00:00:47.060 Amber Lin: Do you know if Matthew’s gonna come.
11 00:00:47.394 ⇒ 00:00:50.400 Trevor Cohen: I don’t. I don’t know not sure, probably not.
12 00:00:52.046 ⇒ 00:00:53.019 Amber Lin: That’s okay.
13 00:00:54.050 ⇒ 00:00:58.910 Trevor Cohen: He said he might come like once a week or to Mondays, but I think he probably won’t.
14 00:00:59.400 ⇒ 00:01:03.620 Amber Lin: Okay, I see. Let me ping our people.
15 00:01:03.810 ⇒ 00:01:04.690 Trevor Cohen: Sounds good.
16 00:01:12.190 ⇒ 00:01:17.200 Luke Daque: How was your weekend, Trevor, was it? Did you? Were you able to buy it late stuff.
17 00:01:18.200 ⇒ 00:01:23.158 Trevor Cohen: I ended up. Not, I think. Yeah. Annie was the one who was gonna go thrifting
18 00:01:23.640 ⇒ 00:01:24.750 Luke Daque: Oh, for everything. Yeah.
19 00:01:24.750 ⇒ 00:01:30.969 Trevor Cohen: Yeah, I’m pretty aired like to get vintage clothes. Yeah, I’m pretty stocks wardrobe wise at the moment.
20 00:01:33.090 ⇒ 00:01:33.790 Luke Daque: Cool.
21 00:01:34.120 ⇒ 00:01:34.760 Trevor Cohen: Yeah.
22 00:01:35.550 ⇒ 00:01:36.940 Trevor Cohen: How’s yours, Luke?
23 00:01:38.210 ⇒ 00:01:43.379 Luke Daque: Yeah, it’s pretty fine it was pretty chill. Nothing much going on, basically.
24 00:01:44.710 ⇒ 00:01:46.219 Trevor Cohen: Good chills, nice.
25 00:01:46.510 ⇒ 00:01:48.569 Amber Lin: Yeah, I will buy your weekend traveler.
26 00:01:49.440 ⇒ 00:01:50.393 Trevor Cohen: Good. I
27 00:01:51.030 ⇒ 00:01:56.020 Trevor Cohen: I went dancing on Saturday night to this new club, and, like, you know, hip fine.
28 00:01:56.020 ⇒ 00:02:00.150 Trevor Cohen: So they put a sticker on my front camera. That’s how you know, it’s really.
29 00:02:00.150 ⇒ 00:02:04.089 Amber Lin: Oh, what music was it.
30 00:02:04.430 ⇒ 00:02:04.940 Trevor Cohen: It was.
31 00:02:04.940 ⇒ 00:02:09.679 Trevor Cohen: It was a it was like his house music. It was like sort of like, Hip hop, Disco house.
32 00:02:09.940 ⇒ 00:02:10.460 Trevor Cohen: That is right?
33 00:02:10.460 ⇒ 00:02:12.390 Trevor Cohen: Oh, interesting.
34 00:02:12.890 ⇒ 00:02:19.140 Trevor Cohen: Yeah, it was great. I just that’s that’s the like electronic music that I like. Like, I’m just not really a techno fan, like just all day.
35 00:02:19.140 ⇒ 00:02:19.650 Amber Lin: Okay.
36 00:02:20.211 ⇒ 00:02:23.579 Trevor Cohen: Need some like melodic elements and.
37 00:02:23.580 ⇒ 00:02:26.350 Amber Lin: Okay, I’m very deep in the techno space.
38 00:02:26.350 ⇒ 00:02:28.140 Trevor Cohen: Your techno. Okay, I respect it.
39 00:02:28.140 ⇒ 00:02:33.729 Amber Lin: Else. I’m like. Goodbye in my house, but not at a club.
40 00:02:34.310 ⇒ 00:02:35.599 Trevor Cohen: Not at a club.
41 00:02:35.850 ⇒ 00:02:40.609 Amber Lin: Yeah, if it’s a like, if it’s at a club, I I kind of want it either to be
42 00:02:40.930 ⇒ 00:02:54.760 Amber Lin: very melodic, like hip, hop, or pop, essentially, or as Techno. So that’s the 2 things I will go somewhere to hear. Otherwise I would just play at home.
43 00:02:55.130 ⇒ 00:02:55.789 Trevor Cohen: Listen, I.
44 00:02:56.306 ⇒ 00:02:58.373 Amber Lin: Also listen to Edm.
45 00:02:58.890 ⇒ 00:03:00.330 Trevor Cohen: So Annie does.
46 00:03:01.630 ⇒ 00:03:02.220 Trevor Cohen: Cool.
47 00:03:02.590 ⇒ 00:03:03.290 Annie Yu: Wait, what.
48 00:03:03.950 ⇒ 00:03:04.605 Amber Lin: Oh!
49 00:03:07.277 ⇒ 00:03:10.053 Trevor Cohen: Remember, and you said you also go to
50 00:03:10.670 ⇒ 00:03:14.449 Annie Yu: Yeah, yeah, I’m I don’t listen to much.
51 00:03:14.760 ⇒ 00:03:26.990 Annie Yu: I guess. House. I do like, Edm. I’m more basic like your millennium slender. But I’m also like kind of into like dubsteps. So subtronics. Let’s go.
52 00:03:26.990 ⇒ 00:03:27.550 Amber Lin: Wow!
53 00:03:27.550 ⇒ 00:03:31.369 Trevor Cohen: Cool. That’s cool. We we’re that’s that’s a good good place to start.
54 00:03:32.091 ⇒ 00:03:33.800 Amber Lin: And I was gonna say.
55 00:03:33.800 ⇒ 00:03:47.449 Trevor Cohen: Say amber. I fully recognize that it’s because I’m not evolved enough that I don’t enjoy techno like, I know, a lot of people who are like techno sucks, and then, like went to the right club, or, like honestly like went to Berlin or something. And they’re like Techno is amazing.
56 00:03:49.370 ⇒ 00:03:53.199 Amber Lin: Yeah, I only liked it after I went to Berlin, so I can’t complain.
57 00:03:53.200 ⇒ 00:03:54.370 Trevor Cohen: Is that true.
58 00:03:54.370 ⇒ 00:04:02.860 Amber Lin: It. It is true I was very skeptical, I was like, why do you listen to beats like? Why do you? Why do you listen to noise. And then I was like, Oh, yeah.
59 00:04:03.330 ⇒ 00:04:05.919 Trevor Cohen: Where’d you go in Berlin? Did you go to Bergen?
60 00:04:05.920 ⇒ 00:04:10.289 Amber Lin: Didn’t get, didn’t go, didn’t want to get in line for 2 h, and then get rejected.
61 00:04:10.290 ⇒ 00:04:11.629 Trevor Cohen: Get rejected. Yeah.
62 00:04:11.630 ⇒ 00:04:13.770 Amber Lin: Protecting my ego, so I just didn’t go.
63 00:04:13.770 ⇒ 00:04:19.670 Trevor Cohen: I know I I feel the same. I feel like I would be rejected, but I don’t.
64 00:04:19.670 ⇒ 00:04:24.350 Trevor Cohen: No, I have like my friends who have gone are like 3 for 3. So.
65 00:04:24.890 ⇒ 00:04:25.880 Amber Lin: Wow!
66 00:04:25.880 ⇒ 00:04:27.370 Trevor Cohen: Maybe they’re just cooler than us.
67 00:04:27.950 ⇒ 00:04:32.749 Amber Lin: I see. I just. I think you just need to look German, and I don’t look very German.
68 00:04:32.750 ⇒ 00:04:41.580 Trevor Cohen: You have to. You have to like. Try to speak some German. You can’t look too excited, but you have to look like excited enough
69 00:04:41.580 ⇒ 00:04:44.779 Trevor Cohen: like their specific niche. Yeah.
70 00:04:45.770 ⇒ 00:04:51.219 Amber Lin: My listen to some. We went on a hike. Sorry I let me pull up linear and share my life.
71 00:04:51.650 ⇒ 00:05:02.680 Amber Lin: went on a. I went on, a pretty nice hike with one of my friends here, and then we on the way back we listen to Techno and ran. So I I felt really good.
72 00:05:03.220 ⇒ 00:05:05.000 Trevor Cohen: That sounds great. That’s that’s.
73 00:05:05.000 ⇒ 00:05:05.480 Amber Lin: Yeah.
74 00:05:05.480 ⇒ 00:05:06.500 Trevor Cohen: Good. Yeah.
75 00:05:06.980 ⇒ 00:05:08.159 Amber Lin: Anyways what I.
76 00:05:08.160 ⇒ 00:05:09.210 Trevor Cohen: Yeah, it’s very good.
77 00:05:09.210 ⇒ 00:05:09.790 Amber Lin: Oh, Lord!
78 00:05:09.950 ⇒ 00:05:11.249 Trevor Cohen: Yeah, please.
79 00:05:12.435 ⇒ 00:05:18.979 Amber Lin: How is everything in cycle? What are some updates? Can I move some of this to done.
80 00:05:19.170 ⇒ 00:05:22.990 Trevor Cohen: Yeah, I mean, have you guys gotten figure out the bigquery access thing yet?
81 00:05:25.065 ⇒ 00:05:36.450 Amber Lin: Would try to figure it out, but it didn’t seem to work, so we might need to just create a shared account. And for you to just share it. That account.
82 00:05:36.770 ⇒ 00:05:41.700 Trevor Cohen: Yeah, happy to do that. It doesn’t make any sense to me why, Utam would get access. But you wouldn’t.
83 00:05:41.950 ⇒ 00:05:53.260 Amber Lin: Yeah, I think it’s more on our end of like permissions on our end. But probably just to create a like brain forge@modelmore.ai type of.
84 00:05:53.610 ⇒ 00:05:55.589 Trevor Cohen: Alright, I will. I will take care of that.
85 00:05:59.160 ⇒ 00:06:04.269 Trevor Cohen: Cool. Yeah, actually hold on. Let’s just do it right now, before, while we’re here.
86 00:06:04.560 ⇒ 00:06:07.409 Amber Lin: Yeah, let me just share it to
87 00:06:07.670 ⇒ 00:06:14.880 Amber Lin: change it to you and then comment this for your access.
88 00:06:18.680 ⇒ 00:06:19.370 Amber Lin: Cool?
89 00:06:25.730 ⇒ 00:06:34.980 Trevor Cohen: And alright created the oh, what? Oh, oh, wait!
90 00:06:36.080 ⇒ 00:06:37.130 Trevor Cohen: Oh, I see.
91 00:06:44.440 ⇒ 00:06:54.580 Trevor Cohen: Okay. I created one brain forge user, and then what else.
92 00:06:56.672 ⇒ 00:07:00.129 Amber Lin: Bigquery share share it to that email. Perhaps.
93 00:07:01.740 ⇒ 00:07:02.819 Trevor Cohen: Right? Okay.
94 00:07:02.820 ⇒ 00:07:05.750 Amber Lin: And then share the password.
95 00:07:06.365 ⇒ 00:07:10.640 Trevor Cohen: Cool, cool. Cool. Okay. Hold on. Let me. Yeah. Let me share the password first.st
96 00:07:10.840 ⇒ 00:07:11.430 Amber Lin: Hmm.
97 00:07:20.630 ⇒ 00:07:23.309 Amber Lin: Annie, how’s it? Progress on your end?
98 00:07:26.079 ⇒ 00:07:30.140 Annie Yu: Let me look at. I’m not sure if my thing is
99 00:07:30.710 ⇒ 00:07:34.407 Annie Yu: tied to any ticket here, but I was
100 00:07:36.300 ⇒ 00:07:46.149 Annie Yu: through the data fields and then get to the data type. And then what the expected value looks like for Microsoft Graph.
101 00:07:46.970 ⇒ 00:07:56.179 Annie Yu: And I saw your message, Trevor, about the success factors. I know that we might just have to assume the data types which I think it
102 00:07:56.390 ⇒ 00:08:02.280 Annie Yu: probably want to be hard to mom be too hard.
103 00:08:02.470 ⇒ 00:08:09.310 Trevor Cohen: Yeah, I think that’s probably right. But did you see, there also is documentation on the sap, like.
104 00:08:09.310 ⇒ 00:08:15.540 Annie Yu: Did. But I just don’t find anything relevant
105 00:08:15.680 ⇒ 00:08:21.190 Annie Yu: like telling me the data type, or like a sample value.
106 00:08:21.190 ⇒ 00:08:23.420 Trevor Cohen: Well, here, okay, hold on. Can I share my screen?
107 00:08:24.340 ⇒ 00:08:25.590 Amber Lin: Yeah, everyone could join.
108 00:08:25.590 ⇒ 00:08:27.979 Amber Lin: Let me find the thing. 1.st One. Sec.
109 00:08:27.980 ⇒ 00:08:28.550 Amber Lin: Yeah.
110 00:08:37.549 ⇒ 00:08:38.299 Trevor Cohen: Okay.
111 00:08:44.870 ⇒ 00:08:45.830 Trevor Cohen: so
112 00:08:46.900 ⇒ 00:08:55.149 Trevor Cohen: it’s really annoying, because they have all these different Apis, which again, I’m glad we don’t have to integrate with them. But okay, like, for example.
113 00:09:01.790 ⇒ 00:09:06.770 Trevor Cohen: like user management, I would expect. And then if you go to model view
114 00:09:07.290 ⇒ 00:09:13.380 Trevor Cohen: user, then it has all the data fields and then tells you what the I have it here.
115 00:09:14.720 ⇒ 00:09:18.050 Annie Yu: I think this might be what we need.
116 00:09:19.510 ⇒ 00:09:25.951 Trevor Cohen: Yeah, I think so. And then, like, I think, just for that, there are gonna be other tables that we need. Also here, let me pull up.
117 00:09:28.870 ⇒ 00:09:30.950 Trevor Cohen: The the document you put together.
118 00:09:35.080 ⇒ 00:09:38.780 Trevor Cohen: do you see? Do you see the doc? The spreadsheet.
119 00:09:39.170 ⇒ 00:09:39.840 Annie Yu: Yeah, yeah.
120 00:09:39.840 ⇒ 00:09:43.452 Trevor Cohen: Okay, cool. Alright? So yeah, right position. And then I was just looking through
121 00:09:46.550 ⇒ 00:09:50.440 Trevor Cohen: the 5 trend docs and.
122 00:09:51.430 ⇒ 00:09:53.540 Annie Yu: So this is where I looked.
123 00:09:53.540 ⇒ 00:09:57.839 Trevor Cohen: Yeah, yeah, I think this is the right place to like, find what we want. Cause it’s just easier to see
124 00:10:00.710 ⇒ 00:10:02.689 Trevor Cohen: But then, like
125 00:10:03.270 ⇒ 00:10:10.869 Trevor Cohen: they’re gonna be I? There’s so many freaking tables. But I scrolled through them, and there are other ones that I think right position. I agree with you. I think we want
126 00:10:14.390 ⇒ 00:10:17.190 Trevor Cohen: probably employment. Maybe.
127 00:10:18.760 ⇒ 00:10:19.770 Annie Yu: Okay.
128 00:10:19.990 ⇒ 00:10:20.505 Annie Yu: I.
129 00:10:21.340 ⇒ 00:10:26.999 Trevor Cohen: It seems like there’s some like duplicate information across tables. I don’t know, but just like 1st date worked and end date.
130 00:10:27.760 ⇒ 00:10:29.780 Trevor Cohen: I think, would be good. So.
131 00:10:31.560 ⇒ 00:10:40.760 Annie Yu: 1st date. Okay? So in users, there’s higher date. And is that so? That might be different than 1st date.
132 00:10:41.540 ⇒ 00:10:44.569 Trevor Cohen: Yeah, I don’t know. Yeah. Good. Good point. Let’s see.
133 00:10:46.130 ⇒ 00:10:47.670 Trevor Cohen: Right? They’ve got.
134 00:10:56.290 ⇒ 00:10:58.860 Trevor Cohen: I would. Maybe it’s also possible that
135 00:10:58.960 ⇒ 00:11:08.270 Trevor Cohen: people come off the user table when they leave. So maybe like just having, maybe employment would be historical. I don’t know. But my guess is that
136 00:11:10.630 ⇒ 00:11:11.710 Trevor Cohen: not fair.
137 00:11:11.710 ⇒ 00:11:14.980 Annie Yu: But employment would be another good one, too.
138 00:11:15.540 ⇒ 00:11:20.510 Trevor Cohen: I think, though, I’m not. Wait, I’m not seeing. Start, date and user. Now, hold on. I’m probably just missing it.
139 00:11:20.510 ⇒ 00:11:21.539 Annie Yu: I think I saw.
140 00:11:21.930 ⇒ 00:11:27.879 Trevor Cohen: I thought I thought, also org hire, date or ridge hire date. Yeah.
141 00:11:27.880 ⇒ 00:11:28.660 Annie Yu: No.
142 00:11:28.660 ⇒ 00:11:31.199 Trevor Cohen: Seems like employment probably has more information.
143 00:11:33.520 ⇒ 00:11:45.250 Trevor Cohen: I’ll go. I I’ll take a look at going through. Yeah, I if it would actually be good if we both do this. Probably Annie, like, just go through and kind of compare it to the data that we want that we got asked for on that document on the.
144 00:11:45.820 ⇒ 00:11:48.309 Trevor Cohen: And then you know, this guy.
145 00:11:48.310 ⇒ 00:11:51.811 Annie Yu: Can you show me again how you navigate that
146 00:11:52.480 ⇒ 00:11:55.340 Annie Yu: The other documentation that you just showed.
147 00:11:55.340 ⇒ 00:11:56.639 Trevor Cohen: Yeah, yeah, so.
148 00:11:56.640 ⇒ 00:11:57.410 Annie Yu: So.
149 00:11:57.610 ⇒ 00:12:03.599 Trevor Cohen: So this was the. This is just the list of all the Apis. So basically, what I did was just just scan through and like.
150 00:12:03.730 ⇒ 00:12:05.400 Trevor Cohen: look for
151 00:12:05.630 ⇒ 00:12:12.449 Trevor Cohen: things that like correspond to the tables that we care about. Right? Like there’s a job offer table. It looks like there’s a job offer endpoint.
152 00:12:13.180 ⇒ 00:12:15.060 Trevor Cohen: etc. So
153 00:12:16.120 ⇒ 00:12:21.050 Trevor Cohen: yeah, the one I looked at was user management. But like, let’s see if there’s another one that we can find hold on.
154 00:12:24.880 ⇒ 00:12:27.520 Trevor Cohen: Yeah. Probably employee profile. Maybe.
155 00:12:28.690 ⇒ 00:12:37.360 Trevor Cohen: And then I went. So anyway. So I clicked on on that particular endpoint, whatever Api layer, and then went to Model view.
156 00:12:37.600 ⇒ 00:12:42.829 Trevor Cohen: So there’s Api reference, which I guess like shows you all the fields. But then model view
157 00:12:43.270 ⇒ 00:12:47.449 Trevor Cohen: seems to give you the actual data type. So like.
158 00:12:50.150 ⇒ 00:12:52.039 Trevor Cohen: I don’t know where this would be.
159 00:12:57.230 ⇒ 00:12:59.469 Trevor Cohen: This doesn’t seem like what we want. Actually.
160 00:12:59.620 ⇒ 00:13:02.890 Trevor Cohen: yeah, it’s it’s annoying and and honestly like, if this
161 00:13:03.190 ⇒ 00:13:06.580 Trevor Cohen: this is pretty annoying, and it might not be worth the time
162 00:13:06.820 ⇒ 00:13:14.229 Trevor Cohen: like, it’s possible that just just assuming data types would be better. Because I think, for the most part, we can do that.
163 00:13:15.200 ⇒ 00:13:19.540 Annie Yu: Yeah. And I, I think actually, users covered
164 00:13:19.850 ⇒ 00:13:25.359 Annie Yu: well, at least half of the fields. So I I would definitely the the users that you just
165 00:13:26.490 ⇒ 00:13:27.270 Annie Yu: demonstrated.
166 00:13:27.570 ⇒ 00:13:29.389 Trevor Cohen: Okay, cool. Yeah. That was on.
167 00:13:29.740 ⇒ 00:13:32.059 Trevor Cohen: Oh, here, employment information, maybe.
168 00:13:32.060 ⇒ 00:13:35.049 Annie Yu: I think you were in user management.
169 00:13:35.210 ⇒ 00:13:42.710 Trevor Cohen: Yeah, sorry, I was just looking at additional ones. But yeah, user was on page 2 here, user management.
170 00:13:45.890 ⇒ 00:13:47.440 Trevor Cohen: Let’s drop that in slack.
171 00:13:50.540 ⇒ 00:13:54.869 Trevor Cohen: So, yeah, I think this gives us a lot of what we want. You’re right.
172 00:13:55.440 ⇒ 00:14:03.845 Trevor Cohen: But yeah, I just, I, I want to probably err on the side of being comprehensive. Yeah, I mean, I guess it’s like, if we can get get everything from user that’s great.
173 00:14:05.100 ⇒ 00:14:09.630 Trevor Cohen: and then like, you know.
174 00:14:10.570 ⇒ 00:14:13.389 Trevor Cohen: I don’t. I don’t know if levels there, like
175 00:14:13.820 ⇒ 00:14:17.759 Trevor Cohen: definitely like a manager, information, I think, would be on a different table.
176 00:14:18.120 ⇒ 00:14:25.919 Annie Yu: Yeah, I I tried to search manager, and I just can’t find anything that’s related. So if if you
177 00:14:26.130 ⇒ 00:14:28.849 Annie Yu: now have to be there, it will be great.
178 00:14:28.850 ⇒ 00:14:32.529 Trevor Cohen: Do you? Wanna do you? Wanna is it? Okay? We just take a little time and look through this together. I think that.
179 00:14:32.530 ⇒ 00:14:33.810 Amber Lin: For sure.
180 00:14:33.810 ⇒ 00:14:46.899 Amber Lin: I interject really quickly, me and Luke have to jump to another meeting. So I wanna I wanna update this room for you guys to you can just use this room to talk about it. I just want to get a sense of the tickets. So
181 00:14:47.170 ⇒ 00:15:06.300 Amber Lin: what do we need to get to a point where we can start generating output. Are we stuck on the schemas and the documentation? Or is there a way for Luke to start generating the synthetic data or start looking at the Dbt models like, what? What are we
182 00:15:06.790 ⇒ 00:15:07.599 Amber Lin: here on.
183 00:15:07.870 ⇒ 00:15:12.870 Trevor Cohen: I think that Luke can get started, Luke, you can get started on doing the art of the
184 00:15:13.270 ⇒ 00:15:27.573 Trevor Cohen: data generation for all the Microsoft team stuff, because I think, like Andy put together those schemas. And I looked at them. They look good. So I think we can start on that. And then, hopefully, by the time you’re done, Luke, we’ll have success factors ready to go.
185 00:15:27.860 ⇒ 00:15:28.510 Amber Lin: Okay.
186 00:15:28.510 ⇒ 00:15:31.310 Trevor Cohen: It’s not. Yeah. I don’t think like none of that is dependent on
187 00:15:31.960 ⇒ 00:15:37.959 Trevor Cohen: on the Dbt stuff set up that like, you know, Dbt, honestly can be last, because it’s just about.
188 00:15:38.370 ⇒ 00:15:38.780 Amber Lin: Oh!
189 00:15:38.940 ⇒ 00:15:43.060 Trevor Cohen: Like setting up the infrastructure to continuously run our.
190 00:15:43.330 ⇒ 00:15:43.740 Amber Lin: Okay.
191 00:15:43.740 ⇒ 00:16:02.900 Trevor Cohen: Queries afterwards. So I think, like generating this, the synthetic data, and then also writing the queries that we want to 1st like create the models that we want, like the join tables and stuff, and then even start to the the queries to generate the metrics we care about. We can do that all before any of the infrastructure stuff is in place.
192 00:16:03.770 ⇒ 00:16:04.200 Amber Lin: Okay.
193 00:16:04.200 ⇒ 00:16:15.989 Luke Daque: And just to clarify the the synthetic data that I can start working on would be the sheets over here right like, get messages, for example, would be like, yeah, sure.
194 00:16:17.140 ⇒ 00:16:21.329 Luke Daque: this would be some kind of raw data, essentially right? That.
195 00:16:21.330 ⇒ 00:16:22.279 Trevor Cohen: Yeah, so I think.
196 00:16:22.280 ⇒ 00:16:22.850 Luke Daque: Good.
197 00:16:23.220 ⇒ 00:16:29.960 Trevor Cohen: It looks like all of these these Microsoft graph ones already. So get message.
198 00:16:30.860 ⇒ 00:16:31.980 Trevor Cohen: List. Events.
199 00:16:31.980 ⇒ 00:16:32.730 Luke Daque: Cents!
200 00:16:34.290 ⇒ 00:16:35.360 Luke Daque: Gotcha.
201 00:16:35.360 ⇒ 00:16:39.550 Trevor Cohen: Yeah. So I think all like these 4 tables. I I think we’re ready to go.
202 00:16:39.760 ⇒ 00:16:47.669 Annie Yu: And, Trevor, I if I’m not wrong, you already moved something that we won’t need right from from these tables.
203 00:16:47.670 ⇒ 00:16:53.889 Trevor Cohen: Yeah, I removed all. So the the big thing is that we don’t want. We don’t want actual like content, like.
204 00:16:54.306 ⇒ 00:16:55.139 Annie Yu: Like bot.
205 00:16:55.140 ⇒ 00:16:55.750 Trevor Cohen: Pardon me.
206 00:16:56.450 ⇒ 00:17:14.860 Trevor Cohen: yeah, we just want metadata. So content would be like for an email, you know, like the body or the subject of the email for a message, the the body of the message for an event like the name of the event or the description. So like, we don’t want any of that stuff. We just want to know, like what time it was at like, who was between stuff like that.
207 00:17:15.800 ⇒ 00:17:16.569 Annie Yu: Yeah, so.
208 00:17:16.579 ⇒ 00:17:20.299 Trevor Cohen: So. Yeah, so I delete. I deleted the the ones that were like content related.
209 00:17:20.300 ⇒ 00:17:20.890 Annie Yu: Yeah.
210 00:17:23.723 ⇒ 00:17:28.349 Amber Lin: I’ll send you in our external client channel of
211 00:17:28.450 ⇒ 00:17:37.219 Amber Lin: the ticket, and if you can, would you help add in, say what tables you want Luke to generate synthetic data on, like what tables are ready.
212 00:17:38.250 ⇒ 00:17:39.470 Trevor Cohen: Oh, yeah. Yes.
213 00:17:39.990 ⇒ 00:17:41.540 Amber Lin: To see.
214 00:17:43.590 ⇒ 00:17:47.410 Annie Yu: And me and look are meeting actually in 2 h.
215 00:17:47.590 ⇒ 00:17:48.070 Amber Lin: Oh, great!
216 00:17:48.576 ⇒ 00:17:50.094 Annie Yu: I’m not sure.
217 00:17:51.530 ⇒ 00:17:58.149 Annie Yu: Yeah, so we’ll pair. We’ll pair on that. But Luke had experience with this.
218 00:17:58.400 ⇒ 00:17:59.080 Amber Lin: Awesome.
219 00:17:59.410 ⇒ 00:18:00.440 Luke Daque: So
220 00:18:02.230 ⇒ 00:18:15.999 Amber Lin: We’ll see because we want to have the data things, the setup we want to have the synthetic data. Is there anything else that we want like to define the join logic, and all of that.
221 00:18:16.810 ⇒ 00:18:26.680 Trevor Cohen: Yeah, yeah, I think that that’s definitely. Once we get all the synthetic data, then define the join logic. And like, just, you know, in general figuring it we might want like
222 00:18:26.960 ⇒ 00:18:31.220 Trevor Cohen: a variety of different in between tables. I think that’ll just that’ll come.
223 00:18:31.220 ⇒ 00:18:31.540 Amber Lin: Okay.
224 00:18:31.540 ⇒ 00:18:33.840 Trevor Cohen: Honestly, the easiest thing. Probably
225 00:18:35.070 ⇒ 00:18:43.450 Trevor Cohen: it depends like there might be just some straightforward things that we want to start joining, like, you know.
226 00:18:44.180 ⇒ 00:18:47.959 Trevor Cohen: The good thing about bigquery is that you can have nested records. And so
227 00:18:48.630 ⇒ 00:19:09.200 Trevor Cohen: creating, you can in theory create this enormous complete user table that has, like nested records of all of their emails and all of their chat history and all their events. But it might not make sense like it might that, because then you have to unnest it. And whatever. And so my guess is that there’ll be like
228 00:19:11.130 ⇒ 00:19:11.830 Trevor Cohen: it.
229 00:19:12.040 ⇒ 00:19:39.170 Trevor Cohen: It might. It might be easier to start with the actual metrics we want to generate. And then, you know, like, write the complete query from the raw source tables of like what that would look like. And then, once you’ve done a few metrics, then you can start to see what the commonalities are. And you’re like, okay, I’m always joining these tables in this way, in order to make this faster and easier, we should probably create a model that Pre joins these. And then that way, you know, it makes all the analytics easier.
230 00:19:39.300 ⇒ 00:19:41.690 Amber Lin: I see so like more like staging.
231 00:19:42.300 ⇒ 00:19:48.200 Trevor Cohen: Yeah, like, that’s sort of what I’ve done in the past. But yeah, I think for sure. We’ll want to
232 00:19:48.380 ⇒ 00:19:55.109 Trevor Cohen: have some intermediate. Some like joined models that like, I don’t think, I think that we won’t.
233 00:19:56.180 ⇒ 00:20:00.760 Trevor Cohen: We’ll rarely be like querying, or I don’t know. Rare it’s it’s hard to know, but
234 00:20:01.110 ⇒ 00:20:08.780 Trevor Cohen: not not necessarily querying just from the underlying tables. The way that we get it from the Api, like we definitely want to dump the
235 00:20:08.860 ⇒ 00:20:31.220 Trevor Cohen: the tables from the Api exactly as is into the database, but then from there it often makes sense to like, do some join tables and stuff and have. And that’s the point of dbt, right? Like those those tables then regenerate every time. We don’t have to just use views, either because views can be slower because it has to like, join every time and scan all the records. So yeah, I think that’s that’s up to you guys like what?
236 00:20:32.790 ⇒ 00:20:37.779 Trevor Cohen: But but I but both of those things, both like creating the the new models, and then
237 00:20:38.090 ⇒ 00:20:41.659 Trevor Cohen: then driving analytics. From that I think, like are on.
238 00:20:41.800 ⇒ 00:20:44.129 Trevor Cohen: We’re ready to move forward with that on your end.
239 00:20:44.480 ⇒ 00:20:58.549 Amber Lin: Sounds good. I’ll leave this room to you guys to figure some more things out. It’ll be great if you can update the tickets on the requirements. And I know Luke and Annie, you’ll you guys will meet so you can relay that information, too.
240 00:20:58.750 ⇒ 00:21:00.849 Amber Lin: I’m gonna hop to a different call.
241 00:21:00.850 ⇒ 00:21:01.940 Trevor Cohen: Okay, thanks. Amber.
242 00:21:02.920 ⇒ 00:21:04.079 Trevor Cohen: Alright, bye, Luke.
243 00:21:08.880 ⇒ 00:21:10.409 Trevor Cohen: Okay. Alright.
244 00:21:12.770 ⇒ 00:21:14.739 Trevor Cohen: Oh, how did you go vintage shopping?
245 00:21:17.370 ⇒ 00:21:33.919 Annie Yu: I did. I did go. I only got this top from which is like a cute stripe, long sleeve but it’s not like the coolest, you know, but I like it.
246 00:21:34.380 ⇒ 00:21:37.410 Annie Yu: So okay, very fine.
247 00:21:37.630 ⇒ 00:21:41.359 Trevor Cohen: So it was a good find, but not like the not like a life changing one.
248 00:21:41.360 ⇒ 00:21:42.670 Annie Yu: Yeah, yeah, okay.
249 00:21:42.670 ⇒ 00:21:43.960 Trevor Cohen: Okay, cool. That sounds like that’s a.