Meeting Title: Annie - Luke Pairing Session 2 - Synthetic Data Date: 2025-04-29 Meeting participants: Annie Yu, Luke Daque


WEBVTT

1 00:00:27.900 00:00:29.150 Annie Yu: Hello, Luke!

2 00:00:30.230 00:00:31.930 Luke Daque: Welcome back!

3 00:00:31.930 00:00:35.032 Annie Yu: You got a chance to take a break.

4 00:00:37.485 00:00:38.280 Luke Daque: Even.

5 00:00:39.460 00:00:42.739 Annie Yu: I think I got that figured out. So

6 00:00:42.870 00:00:52.109 Annie Yu: actually, Review, okay, let me screen share review, actually should be the same as

7 00:00:52.330 00:00:59.700 Annie Yu: I think it was right. Originally it should be the same as from, because the review really means.

8 00:00:59.700 00:01:00.550 Luke Daque: I don’t know.

9 00:01:00.550 00:01:01.170 Annie Yu: Is.

10 00:01:01.640 00:01:02.790 Luke Daque: I mean the reply.

11 00:01:03.060 00:01:05.800 Annie Yu: Yeah, who who will?

12 00:01:06.630 00:01:12.099 Annie Yu: What will the email address be if we want to reply to.

13 00:01:12.520 00:01:13.890 Luke Daque: I see.

14 00:01:14.400 00:01:15.020 Annie Yu: So.

15 00:01:15.020 00:01:15.910 Luke Daque: Make, sense.

16 00:01:16.120 00:01:20.270 Annie Yu: Yeah, that’s why I think that’s why, initially it

17 00:01:21.230 00:01:23.129 Annie Yu: it was the same as from.

18 00:01:23.380 00:01:27.549 Annie Yu: And I did ask to just

19 00:01:28.130 00:01:33.649 Annie Yu: just refine again. That’s just like from earlier. And then.

20 00:01:33.780 00:01:36.220 Annie Yu: from what I figured out, and also like.

21 00:01:36.550 00:01:39.680 Luke Daque: Asked it to match. Microsoft Graph.

22 00:01:40.814 00:01:46.339 Annie Yu: And I think that I I think I think this one now is is

23 00:01:48.530 00:01:52.689 Annie Yu: should be fine. But yeah.

24 00:01:52.980 00:01:57.700 Annie Yu: we we can. We can go through one just to make sure.

25 00:01:57.850 00:01:58.610 Luke Daque: Okay.

26 00:02:00.020 00:02:01.110 Annie Yu: So here.

27 00:02:02.180 00:02:09.860 Annie Yu: And it’s a subject we’re interesting. So this, this is having a different subject.

28 00:02:16.030 00:02:17.800 Luke Daque: Is it the same? Id.

29 00:02:19.210 00:02:21.240 Annie Yu: So oh, no!

30 00:02:21.240 00:02:22.230 Luke Daque: Oh, it’s a different.

31 00:02:23.640 00:02:24.240 Luke Daque: Yeah.

32 00:02:25.080 00:02:26.750 Annie Yu: Okay, nice.

33 00:02:27.090 00:02:32.499 Annie Yu: So, Brian, Brian. So this will be Brian, too. Yeah.

34 00:02:37.930 00:02:42.310 Annie Yu: I think it. It looks. It looks okay for me. Now.

35 00:02:42.820 00:02:44.020 Luke Daque: Yeah, nice.

36 00:02:48.550 00:02:55.999 Annie Yu: yeah, I think I I’m I’m just trying to be like as close as we can be. But I I feel like it’s

37 00:02:56.110 00:03:03.499 Annie Yu: also like, okay and expected that once we get the real data, we we do have

38 00:03:04.520 00:03:07.980 Annie Yu: like some differences. I think I think that’s okay.

39 00:03:08.220 00:03:19.679 Luke Daque: Yeah, that’s that should be fine. I think the I think Trevor’s main goal here is to at least get the correct schema and like data types. And then that way, we can

40 00:03:19.890 00:03:25.850 Luke Daque: start working on maybe a a data model that would like join 2 tables or something.

41 00:03:25.850 00:03:26.540 Annie Yu: Yeah, yeah.

42 00:03:26.540 00:03:27.610 Luke Daque: Yeah.

43 00:03:27.860 00:03:39.140 Luke Daque: yeah. So it doesn’t necessarily need to be perfect in terms of like the the emails and stuff like that. As long as like, the schema is correct. So yeah, I think we should be good.

44 00:03:39.300 00:03:39.820 Luke Daque: Good with.

45 00:03:39.820 00:03:40.570 Annie Yu: Okay.

46 00:03:40.740 00:03:45.080 Annie Yu: So I think I’ll save this. And then this could be the

47 00:03:45.480 00:03:47.850 Annie Yu: the output for that 1st one.

48 00:03:48.430 00:03:49.070 Luke Daque: Yeah.

49 00:03:49.490 00:03:54.970 Annie Yu: And we do have 3 more, do you wanna

50 00:03:55.220 00:03:58.120 Annie Yu: do you? Wanna we can stay on, and then.

51 00:03:59.100 00:03:59.650 Luke Daque: You know.

52 00:03:59.650 00:04:01.990 Annie Yu: It in parallel.

53 00:04:02.160 00:04:08.029 Luke Daque: Yeah, sure I can. I can do the list events, and maybe you can do the next one or something.

54 00:04:08.710 00:04:09.460 Annie Yu: Yeah.

55 00:04:09.650 00:04:10.530 Luke Daque: Yeah, let me do.

56 00:04:10.790 00:04:12.350 Annie Yu: And I did

57 00:04:13.200 00:04:23.999 Annie Yu: pasted this all the assumptions that I used earlier, but I also am not sure, because I I’m pretty sure in other tables. We don’t have

58 00:04:24.300 00:04:28.499 Annie Yu: conversation. Id, because there are 4 different things.

59 00:04:28.910 00:04:29.750 Luke Daque: So.

60 00:04:30.050 00:04:35.840 Annie Yu: So this list events is for calendar and then

61 00:04:35.960 00:04:39.590 Annie Yu: get all messages. It’s across apps.

62 00:04:40.170 00:04:44.210 Annie Yu: And then list call Ids teams

63 00:04:44.360 00:04:48.390 Annie Yu: call. So I yeah, just noted that

64 00:04:48.630 00:04:58.320 Annie Yu: we can reference this. But I feel like for each of the table. We will have to give probably a little different assumption.

65 00:04:59.000 00:04:59.860 Luke Daque: Yeah.

66 00:05:01.840 00:05:02.610 Luke Daque: Yeah.

67 00:05:06.080 00:05:06.880 Luke Daque: Yeah.

68 00:05:08.600 00:05:11.266 Luke Daque: Okay. Let me try doing the list events.

69 00:05:11.600 00:05:16.730 Annie Yu: Okay. Then I’m gonna do the get all list get all messages.

70 00:05:17.450 00:05:18.040 Luke Daque: Okay.

71 00:05:18.040 00:05:19.310 Annie Yu: While you’re doing that.

72 00:05:23.650 00:05:26.480 Luke Daque: And maybe we can review together the.

73 00:05:26.480 00:05:29.769 Annie Yu: Yeah, I think that would be great. Yeah.

74 00:05:33.070 00:05:41.610 Annie Yu: So we then should how do I tell cursor to? I guess.

75 00:05:43.076 00:05:43.590 Luke Daque: You can.

76 00:05:44.230 00:05:44.870 Annie Yu: File.

77 00:05:45.410 00:05:50.500 Luke Daque: You can create a new chat, you just click on the plus button at the top.

78 00:05:50.990 00:05:51.650 Annie Yu: Oh, okay.

79 00:05:51.650 00:05:53.189 Luke Daque: Way you can start fresh.

80 00:05:53.920 00:05:55.200 Annie Yu: Got it. Got it.

81 00:05:55.980 00:06:00.660 Luke Daque: Yeah, cause it might get confused with like both.

82 00:06:00.990 00:06:02.000 Luke Daque: So yeah.

83 00:06:07.630 00:06:09.749 Luke Daque: okay, looks like.

84 00:06:16.520 00:06:19.577 Luke Daque: actually, one more thing I can show you.

85 00:06:20.410 00:06:21.570 Annie Yu: Yeah, sure.

86 00:06:22.700 00:06:24.349 Luke Daque: But yeah, let me share my screen.

87 00:06:28.710 00:06:30.259 Luke Daque: Yeah. Can you see my screen?

88 00:06:32.830 00:06:37.490 Luke Daque: So another thing that we can actually do in cursor, like in this case.

89 00:06:38.010 00:06:40.230 Luke Daque: looking at pick installer requirement. Sorry

90 00:06:40.690 00:06:46.419 Luke Daque: I already did it. But like, for example, this one. So you don’t have to type, you can just click on, run.

91 00:06:47.140 00:06:47.940 Annie Yu: Oh!

92 00:06:47.940 00:06:49.820 Luke Daque: And it’s type sheets for you.

93 00:06:49.820 00:06:50.870 Annie Yu: Oh, nice!

94 00:06:51.534 00:06:52.199 Luke Daque: Yeah.

95 00:06:52.340 00:06:55.480 Annie Yu: That’s pretty cool. Thank you so much for sharing.

96 00:06:55.610 00:06:56.085 Luke Daque: Yeah.

97 00:07:06.660 00:07:07.320 Luke Daque: cool.

98 00:09:12.170 00:09:16.950 Luke Daque: is there in in the list events? Do you know, if there’s like any nuances in terms of like

99 00:09:17.190 00:09:21.469 Luke Daque: the I calendar, new id should be

100 00:09:23.020 00:09:27.750 Luke Daque: like. There’s like many to one like one id would have multiple calendar ids.

101 00:09:29.471 00:09:34.930 Annie Yu: Let me try to think through this different and.

102 00:09:35.440 00:09:39.870 Luke Daque: I need identify for an event across calendar.

103 00:09:41.050 00:09:41.580 Annie Yu: They’re free.

104 00:09:44.750 00:09:46.460 Luke Daque: In a recurrencies.

105 00:09:49.290 00:09:51.130 Annie Yu: Does it mean?

106 00:09:52.960 00:09:59.419 Annie Yu: So it means, if we both okay, we do have. We have a meeting.

107 00:10:00.280 00:10:08.399 Annie Yu: So this I call UID should be the same

108 00:10:09.890 00:10:15.069 Annie Yu: for but then how does that differ from Id? Let me see.

109 00:10:16.830 00:10:19.580 Luke Daque: Yeah, and I think it’s.

110 00:10:23.130 00:10:24.200 Annie Yu: Events.

111 00:10:26.100 00:10:32.800 Luke Daque: I think the series Master Id could be in multiple right. If it’s a recurring Id.

112 00:10:33.010 00:10:34.660 Luke Daque: I mean recurring meeting.

113 00:10:35.790 00:10:38.800 Annie Yu: I think. So. Yeah, yeah, you’re right. Good catch.

114 00:10:40.190 00:10:41.490 Luke Daque: Suggested.

115 00:10:41.490 00:10:43.010 Luke Daque: It’s recurring.

116 00:10:48.140 00:10:48.910 Luke Daque: See?

117 00:10:57.300 00:10:59.209 Annie Yu: I’m gonna share this.

118 00:11:00.220 00:11:06.790 Annie Yu: This is there is some description and data type.

119 00:11:07.470 00:11:08.419 Luke Daque: In this.

120 00:11:08.540 00:11:15.600 Annie Yu: Website. I’m I’m looking at the ical uid. But I just I don’t really know what that means.

121 00:11:16.670 00:11:18.999 Annie Yu: I think I I do know.

122 00:11:21.350 00:11:23.689 Annie Yu: I think it just means one.

123 00:11:24.500 00:11:28.790 Annie Yu: Let’s think of it as one meeting that will happen.

124 00:11:29.010 00:11:33.370 Annie Yu: That would be the same across your and my calendar.

125 00:11:35.430 00:11:36.300 Luke Daque: And.

126 00:11:51.250 00:11:58.700 Annie Yu: Different for each occurrence. Yeah, I think so. So. Let me think. And what’s that, Master.

127 00:11:59.060 00:12:01.170 Luke Daque: See if necessary. I do.

128 00:12:03.110 00:12:08.470 Annie Yu: Id for recurrence series.

129 00:12:32.230 00:12:32.930 Annie Yu: Yeah.

130 00:12:58.140 00:13:00.220 Annie Yu: yeah, I think that I

131 00:13:12.210 00:13:17.710 Annie Yu: which one are you? Verifying.

132 00:13:20.950 00:13:23.120 Luke Daque: I’m just trying to like figure out like.

133 00:13:25.330 00:13:27.689 Luke Daque: if there could be like multiple

134 00:13:29.240 00:13:33.380 Luke Daque: ids that are the same. So it looks like calendar. Id should be unique.

135 00:13:33.730 00:13:39.980 Luke Daque: But I think master can be multiple if it’s like recurring.

136 00:13:40.470 00:13:46.220 Luke Daque: But I’m not sure if, like, we should be showing all the recurring meetings here like.

137 00:13:47.030 00:13:52.419 Luke Daque: if this is recurring weekly, and then it like, it’s.

138 00:13:53.450 00:13:58.740 Luke Daque: there’s 5 weeks worth of meetings, and we’ll make sure the same.

139 00:13:59.775 00:14:00.570 Annie Yu: Sure.

140 00:14:00.740 00:14:03.250 Annie Yu: Yeah, I feel like, I feel like, that’s

141 00:14:03.870 00:14:05.900 Annie Yu: that should be the case right.

142 00:14:07.020 00:14:09.260 Luke Daque: So he.

143 00:14:09.580 00:14:14.979 Annie Yu: All these 5 ikuid is in the same series.

144 00:14:21.960 00:14:22.820 Luke Daque: Yeah.

145 00:14:24.380 00:14:28.080 Annie Yu: Okay, the yeah. I think we can assume that.

146 00:14:30.280 00:14:34.819 Luke Daque: It has to be a recurring meeting to have a series, Master Id.

147 00:14:36.080 00:14:41.139 Luke Daque: So we will be able to know if it’s recurring. Because of this recurrence. I guess

148 00:14:41.710 00:14:43.730 Luke Daque: so. This doesn’t make sense right

149 00:14:44.040 00:14:48.649 Luke Daque: because it has an id, but it’s not a recurring meeting.

150 00:14:52.640 00:15:01.330 Annie Yu: Yeah, that’s yeah. So it means, if it has. If something has a series, master Id.

151 00:15:02.410 00:15:05.760 Annie Yu: there should be values for recurrence.

152 00:15:07.050 00:15:08.219 Annie Yu: Is that it.

153 00:15:08.520 00:15:14.960 Luke Daque: I think. So, yeah, I’m trying to figure out as well that what that is

154 00:15:16.230 00:15:20.180 Luke Daque: yeah, this is gonna recurring 10,

155 00:15:36.440 00:15:37.980 Luke Daque: see it’s missing.

156 00:16:08.240 00:16:17.340 Annie Yu: Yeah, I think I think if a role has master id it, it should have recurrence value. Yeah.

157 00:16:20.580 00:16:21.839 Luke Daque: Let me start.

158 00:16:29.810 00:16:30.600 Luke Daque: Hmm.

159 00:16:58.410 00:17:06.300 Luke Daque: so I guess that would also mean if it’s a recurring anything, then the type would be.

160 00:17:10.470 00:17:12.639 Annie Yu: Wait. The type. Oh, yes.

161 00:17:13.950 00:17:17.083 Luke Daque: Series matches this type.

162 00:17:18.669 00:17:20.129 Luke Daque: It makes sense.

163 00:18:32.929 00:18:33.679 Luke Daque: check.

164 00:22:12.489 00:22:16.539 Luke Daque: Are you liking cursor now? Because in this.

165 00:22:18.890 00:22:19.989 Annie Yu: I feel like

166 00:22:20.870 00:22:29.889 Annie Yu: I I was still still getting used to it. I think I like how it’s integrated, so we don’t have to go back and forth, but I think.

167 00:22:30.550 00:22:35.229 Annie Yu: Oh, I I’m like I don’t know how

168 00:22:35.660 00:22:40.589 Annie Yu: how smart it is in terms of like assuming context.

169 00:22:41.100 00:22:44.939 Luke Daque: Yeah, like, you have to be

170 00:22:45.690 00:22:51.190 Luke Daque: it like forces you to be better at yeah, precise at your and

171 00:22:52.560 00:22:56.000 Luke Daque: yeah, like a query, or what they call that.

172 00:24:07.110 00:24:08.190 Luke Daque: do you?

173 00:24:20.670 00:24:22.920 Luke Daque: Yeah, I think this makes sense. Now.

174 00:24:25.480 00:24:27.490 Annie Yu: I mean the list, events.

175 00:24:28.020 00:24:29.759 Luke Daque: Yeah, I’ll show you.

176 00:24:30.100 00:24:30.890 Annie Yu: Okay.

177 00:24:38.290 00:24:43.680 Luke Daque: So, so we have the Id. It should be in.

178 00:24:44.180 00:24:49.620 Luke Daque: And then another calendar id, which also would be unique for each instance.

179 00:24:50.060 00:24:54.089 Luke Daque: And then, if it’s a recurring meeting, then it will have

180 00:24:54.430 00:25:03.490 Luke Daque: experience. Master Id. So these, like how many 6 7 meetings, would be the same meeting. It’s just recurring.

181 00:25:03.890 00:25:04.610 Annie Yu: Yeah.

182 00:25:04.610 00:25:05.860 Luke Daque: And then.

183 00:25:06.360 00:25:12.849 Luke Daque: yeah, you’ll see here the 1st one would be series, master, and then the succeeding ones would be occurrence.

184 00:25:13.090 00:25:14.210 Annie Yu: Oh!

185 00:25:14.210 00:25:15.260 Luke Daque: And then the non.

186 00:25:15.290 00:25:16.400 Annie Yu: I meant.

187 00:25:17.880 00:25:22.520 Luke Daque: Yeah, the non, recurring meetings are just like single instance.

188 00:25:24.400 00:25:31.699 Luke Daque: And then, yeah, only the series. Master will have the recurrence pattern, which is basically

189 00:25:32.200 00:25:35.530 Luke Daque: yeah, the pattern of the meeting, whether it’s like weekly

190 00:25:35.880 00:25:42.270 Luke Daque: monthly, but it looks like it’s like this looks like this. So I’m not quite sure.

191 00:25:42.270 00:25:45.529 Annie Yu: And the recurrence. Let me see.

192 00:25:46.030 00:25:54.400 Luke Daque: This is, it’s exactly yeah. It should. I guess it should be.

193 00:25:56.280 00:25:57.770 Annie Yu: Oh, so maybe, idiot!

194 00:25:58.110 00:26:02.269 Annie Yu: I need to give it like a better example.

195 00:26:04.460 00:26:14.510 Luke Daque: Yes, if this looks aside from the recurrence pattern, I think this looks pretty good.

196 00:26:14.890 00:26:21.430 Annie Yu: Yeah, is the star time of, and a series?

197 00:26:21.960 00:26:24.540 Annie Yu: Are they like.

198 00:26:27.060 00:26:29.190 Luke Daque: Well, I, I think we need

199 00:26:39.780 00:26:41.150 Luke Daque: yeah. The.

200 00:26:41.440 00:26:43.820 Annie Yu: Are different, like nice.

201 00:26:43.820 00:26:49.220 Luke Daque: Specific series it would be every week. It looks like every 7 days.

202 00:26:49.510 00:26:49.980 Annie Yu: Yeah.

203 00:26:49.980 00:26:52.119 Luke Daque: Man that makes sense.

204 00:26:52.650 00:26:53.719 Annie Yu: That makes sense.

205 00:26:56.000 00:26:57.790 Luke Daque: But the create time would be.

206 00:26:58.260 00:27:01.680 Luke Daque: I guess, all the same, because they would be creating the same.

207 00:27:02.480 00:27:03.630 Annie Yu: Also makes sense.

208 00:27:05.760 00:27:06.470 Luke Daque: Nice.

209 00:27:07.080 00:27:08.000 Annie Yu: Nice.

210 00:27:08.970 00:27:15.480 Luke Daque: Let me see this for now, but I guess we can still change the recurrence.

211 00:27:15.880 00:27:16.760 Luke Daque: My betting.

212 00:27:17.370 00:27:20.010 Annie Yu: Yeah, I’m I’m trying to find

213 00:27:22.330 00:27:25.749 Annie Yu: an example. If there is one.

214 00:27:27.870 00:27:28.540 Luke Daque: Yeah.

215 00:27:51.210 00:27:53.239 Annie Yu: I see, I think.

216 00:27:54.270 00:27:59.886 Annie Yu: Yeah, recurrence looks a bit complicated for me.

217 00:28:01.580 00:28:09.770 Annie Yu: so I guess that’s a Json. But then, within that, Json, that pattern is also a Json.

218 00:28:10.090 00:28:10.970 Luke Daque: And.

219 00:28:10.970 00:28:13.970 Annie Yu: Range is another. Json.

220 00:28:16.890 00:28:17.860 Luke Daque: I see.

221 00:28:18.860 00:28:20.369 Luke Daque: Are you seeing it clear?

222 00:28:21.230 00:28:22.170 Luke Daque: This one.

223 00:28:22.610 00:28:23.510 Annie Yu: Yeah.

224 00:28:23.780 00:28:24.550 Luke Daque: Let me!

225 00:28:28.300 00:28:33.280 Annie Yu: And then you can click on. Here’s also like property, type and description.

226 00:28:34.460 00:28:35.120 Annie Yu: So.

227 00:28:40.430 00:28:41.792 Annie Yu: How would that?

228 00:28:49.540 00:28:52.880 Annie Yu: I guess we we can.

229 00:29:00.120 00:29:01.619 Luke Daque: It should be something like.

230 00:29:06.650 00:29:10.150 Annie Yu: So this. But this Json is for

231 00:29:10.860 00:29:15.440 Annie Yu: I’m gonna go back to the Google sheet and see how.

232 00:29:17.940 00:29:19.620 Annie Yu: So that means.

233 00:29:23.160 00:29:25.869 Annie Yu: does that mean something like this.

234 00:29:29.720 00:29:36.950 Annie Yu: and then for range, there is also.

235 00:29:37.330 00:29:38.060 Luke Daque: Yeah.

236 00:29:43.040 00:29:44.080 Annie Yu: Like this.

237 00:29:46.640 00:29:52.630 Annie Yu: I’m I’m like, super uncomfortable with Json. I don’t really know like how

238 00:29:53.000 00:29:57.620 Annie Yu: like how to like flatten it. And like, I don’t.

239 00:29:57.800 00:30:01.260 Annie Yu: Yeah. But does this?

240 00:30:01.260 00:30:02.710 Annie Yu: Thanks for you.

241 00:30:02.710 00:30:04.330 Luke Daque: Yeah, that makes sense.

242 00:30:04.720 00:30:15.310 Annie Yu: But that also means okay. So I guess with all the Json objects, we have to do some other things to flatten it. Once, I guess.

243 00:30:15.730 00:30:18.259 Annie Yu: Loaded in bigquery.

244 00:30:19.160 00:30:21.339 Luke Daque: Yeah, and it depends. Like.

245 00:30:21.750 00:30:29.979 Luke Daque: yeah, it depends on like, what ingestion tool we are using. Are we using any ingestion tool for matter more like 5 Tran, for example.

246 00:30:31.080 00:30:36.250 Luke Daque: or are they like doing it themselves through the Api, or something.

247 00:30:36.470 00:30:52.879 Annie Yu: We did have this discussion, but I forgot. I think recommended 5 train to them. I think actually, they wanted to write something themselves, and I’m not sure if it’s already written, or they are considering

248 00:30:53.020 00:30:54.610 Annie Yu: using 5 train.

249 00:30:54.830 00:31:02.279 Luke Daque: I see, because from what I am, from what I based on my experience, 5 grand already unneces everything.

250 00:31:02.530 00:31:08.950 Luke Daque: So what’s going to happen if it’s coming from 5 Tran is, there would be a currents

251 00:31:10.720 00:31:14.649 Luke Daque: table, and then there will be another table for recurrence pattern.

252 00:31:14.900 00:31:15.900 Annie Yu: Oh!

253 00:31:15.900 00:31:22.150 Luke Daque: Just these. So they were. They’re already unnested. And there’s another table that’s recurrence range

254 00:31:22.590 00:31:29.170 Luke Daque: something like that. So it’s already in unnested or UN or flattened, basically.

255 00:31:29.410 00:31:29.900 Annie Yu: Yeah.

256 00:31:29.900 00:31:35.949 Luke Daque: But but for like, if they do it directly through Api, and they don’t

257 00:31:36.130 00:31:41.100 Luke Daque: do any flattening, then it would show up like this in bigquery, and it would be.

258 00:31:41.100 00:31:41.800 Annie Yu: Hmm.

259 00:31:41.800 00:31:47.050 Luke Daque: Either a Json object or a nested? Repeated Field.

260 00:31:47.720 00:31:48.340 Luke Daque: Where?

261 00:31:49.490 00:31:52.290 Luke Daque: Yeah? Like, yeah, where it’s like

262 00:31:52.430 00:31:59.849 Luke Daque: nested in bigquery itself. And we’ll have to do the unnesting during data modeling, or something like that.

263 00:32:01.190 00:32:03.830 Luke Daque: So I think, okay.

264 00:32:05.210 00:32:09.449 Annie Yu: I? If I yeah, if I’m not wrong, I think, Trevor said.

265 00:32:09.600 00:32:15.140 Annie Yu: would be unnested in bigquery. But it’s a little vague

266 00:32:15.753 00:32:17.569 Annie Yu: in my head now, I

267 00:32:18.140 00:32:23.519 Annie Yu: forgot. But but I think, yeah, we we you can talk about this.

268 00:32:24.750 00:32:30.760 Luke Daque: Yeah, sure, I guess if we if we use what we have in this

269 00:32:31.830 00:32:40.289 Luke Daque: Csv, and we upload it in bigquery, I think this would be an object. So basically

270 00:32:42.790 00:32:43.690 Annie Yu: Oh!

271 00:32:44.440 00:32:45.460 Luke Daque: Yeah, they.

272 00:32:45.460 00:32:46.989 Annie Yu: Yeah, yeah.

273 00:32:47.690 00:32:51.780 Luke Daque: I don’t know what term it is. But yeah, it’s going to be an object in big pay.

274 00:32:52.390 00:32:56.159 Luke Daque: And this would be a list just like it has like this, like bracket.

275 00:32:56.667 00:33:01.579 Luke Daque: It’s going to be a an array in the query. Basically, there could be like multiple

276 00:33:03.110 00:33:08.159 Luke Daque: rows. And under the same Id, or something like that.

277 00:33:11.250 00:33:11.990 Luke Daque: Yeah.

278 00:33:13.360 00:33:16.901 Annie Yu: Okay, maybe worry about that later.

279 00:33:17.510 00:33:18.120 Luke Daque: So.

280 00:33:18.460 00:33:24.309 Annie Yu: But do you think we need to update that recurrence? Or you think it’s not gonna.

281 00:33:24.680 00:33:29.409 Luke Daque: Yeah, I’ll I’ll update that, and then and then I think that should be good. Then.

282 00:33:30.030 00:33:38.899 Annie Yu: Okay, okay, I’m looking through. The file that I God for get all messages.

283 00:33:56.510 00:34:01.880 Annie Yu: Yeah, I think this one get all messages. I’m I’m this is

284 00:34:02.530 00:34:09.899 Annie Yu: probably one thing that I’m the least familiar with cause. I think it means.

285 00:34:14.770 00:34:16.070 Luke Daque: It means.

286 00:34:17.150 00:34:24.239 Annie Yu: All it’s saying, like all messages across chat.

287 00:34:25.210 00:34:25.949 Luke Daque: In.

288 00:34:26.659 00:34:29.559 Annie Yu: Which? Let’s see.

289 00:34:39.839 00:34:42.829 Annie Yu: man messages across all chats.

290 00:34:48.609 00:34:49.279 Annie Yu: Okay?

291 00:36:25.560 00:36:26.850 Annie Yu: Hmm.

292 00:36:27.980 00:36:33.400 Annie Yu: Are you familiar with? The documentation and how to navigate it?

293 00:36:34.510 00:36:36.469 Luke Daque: Which documentation.

294 00:36:37.004 00:36:39.770 Annie Yu: I’m I’m in that good.

295 00:36:40.890 00:36:51.219 Annie Yu: Okay, I’m in this. Get all messages. I’m trying to find that similar view that we have for

296 00:36:51.330 00:36:53.630 Annie Yu: like earlier for messages and.

297 00:37:02.310 00:37:04.410 Annie Yu: Chat, but it’s not.

298 00:37:04.540 00:37:05.969 Annie Yu: It’s not chat.

299 00:37:10.554 00:37:12.610 Luke Daque: Wait this is for.

300 00:37:13.510 00:37:15.389 Luke Daque: Get all messages soon.

301 00:37:15.660 00:37:16.430 Annie Yu: Yeah.

302 00:37:18.210 00:37:21.799 Annie Yu: And I’m trying to go to the page where I can see it.

303 00:37:22.750 00:37:24.160 Annie Yu: The properties.

304 00:37:26.390 00:37:27.520 Luke Daque: I think.

305 00:37:37.860 00:37:41.172 Luke Daque: No, this looks different than what you should

306 00:37:57.330 00:37:58.370 Luke Daque: requested.

307 00:38:15.840 00:38:29.329 Annie Yu: Yeah, it doesn’t look like it’s chat. But when I earlier, I just googled it. But when I say message, we would go to that message we we got for mail.

308 00:38:30.950 00:38:31.820 Luke Daque: Yeah.

309 00:38:39.730 00:38:41.350 Luke Daque: 1912,

310 00:38:44.480 00:38:45.920 Luke Daque: message that.

311 00:38:58.080 00:39:00.869 Annie Yu: Oh, no, I think I found wait.

312 00:39:02.870 00:39:04.278 Annie Yu: I think I find it.

313 00:39:06.150 00:39:08.530 Annie Yu: Chat message.

314 00:39:09.080 00:39:09.870 Annie Yu: Okay.

315 00:39:11.930 00:39:23.430 Luke Daque: It’s different, like I’m looking at. Are you looking at the Json representation, or something, or just the properties.

316 00:39:25.290 00:39:28.080 Annie Yu: So yeah, so

317 00:39:28.340 00:39:36.829 Annie Yu: this is it right? But I was trying to see. Okay, reply, Id, what’s the property? But I think I just found it. This chat

318 00:39:37.410 00:39:38.330 Annie Yu: message.

319 00:39:38.330 00:39:38.890 Luke Daque: It’s.

320 00:39:39.330 00:39:43.150 Annie Yu: Which there should be a reply to.

321 00:39:43.450 00:39:44.260 Luke Daque: Hmm.

322 00:39:45.160 00:39:47.840 Annie Yu: Yeah, so this looks like it.

323 00:39:57.420 00:40:01.740 Annie Yu: Yeah, for this, get all messages. I feel like

324 00:40:12.220 00:40:18.260 Annie Yu: I’m I’m just trying to verify. If there’s any more assumption. We need to feed it. But

325 00:40:18.950 00:40:23.380 Annie Yu: I think for this one, maybe not.

326 00:40:36.300 00:40:38.480 Luke Daque: Try to id message time.

327 00:41:00.980 00:41:11.010 Annie Yu: This, Id is saying, unique within a chat, slash channel, slash reply to message, but might be duplicated

328 00:41:11.260 00:41:13.270 Annie Yu: in other chats.

329 00:41:21.000 00:41:21.600 Luke Daque: And.

330 00:41:26.040 00:41:27.609 Annie Yu: What does that mean?

331 00:41:29.230 00:41:33.070 Annie Yu: So should we have duplicates? Or

332 00:41:48.000 00:41:50.160 Annie Yu: I think we can maybe go with this.

333 00:41:51.190 00:41:54.449 Luke Daque: Yeah, I think that I don’t see any

334 00:41:55.110 00:41:58.860 Luke Daque: potential issues so far. So yeah, maybe we can go with that.

335 00:41:59.530 00:42:00.150 Annie Yu: Okay.

336 00:42:14.350 00:42:17.470 Annie Yu: okay, is there? There’s 1 more.

337 00:42:18.430 00:42:24.569 Luke Daque: Yeah, I’m trying. I’m trying to generate one now for this call, like records.

338 00:42:24.950 00:42:26.080 Annie Yu: Okay. Okay.

339 00:42:27.310 00:42:32.010 Luke Daque: I already. Do you want me to send you the list?

340 00:42:32.500 00:42:33.650 Luke Daque: Events.

341 00:42:36.090 00:42:39.859 Luke Daque: So you can consolidate everything, I guess.

342 00:42:42.560 00:42:49.169 Annie Yu: Sure. Where. Where should we restore this before we get the bigquery?

343 00:42:50.270 00:42:54.259 Annie Yu: Do you think they are too big? If we put them in a drive.

344 00:42:55.810 00:42:57.749 Luke Daque: Oh, yeah, that’s a good question.

345 00:42:58.190 00:43:02.483 Annie Yu: Or should we just dump them in slack.

346 00:43:04.120 00:43:10.419 Luke Daque: Yeah, maybe we can dump within the client. Slack channel client platform more.

347 00:43:11.570 00:43:13.300 Annie Yu: Yeah, yeah, sure.

348 00:43:15.670 00:43:18.670 Luke Daque: That way. We can like just come back to it.

349 00:43:32.690 00:43:38.120 Luke Daque: So running the script now for call records.

350 00:43:40.120 00:43:42.660 Luke Daque: And let’s see what it looks like.

351 00:43:53.410 00:43:59.130 Luke Daque: yeah, I think this should be pretty straightforward. I don’t think there’s any like nuances here.

352 00:44:00.780 00:44:02.380 Annie Yu: The call records.

353 00:44:03.780 00:44:05.799 Luke Daque: Yeah, let me share my screen.

354 00:44:06.080 00:44:06.740 Annie Yu: Okay.

355 00:44:08.240 00:44:11.729 Luke Daque: So based on the sheet. It’s just

356 00:44:13.310 00:44:20.590 Luke Daque: it’s just id type, start date, last modified, and the organizer. So I don’t think there’s any.

357 00:44:21.970 00:44:26.079 Luke Daque: What do you any nuances that we need to.

358 00:44:27.450 00:44:27.880 Annie Yu: Yeah.

359 00:44:27.880 00:44:29.390 Luke Daque: Take into account right.

360 00:44:29.870 00:44:32.860 Annie Yu: I think so. Yeah, this one’s pretty straightforward.

361 00:44:33.240 00:44:33.880 Luke Daque: Yeah.

362 00:44:35.380 00:44:39.780 Luke Daque: So as long as the end date is after, then start date

363 00:44:40.580 00:44:46.610 Luke Daque: and then modified time could be, you know, any date, I guess, but

364 00:44:46.720 00:44:50.260 Luke Daque: and it can be the same as the end. Date, completion meeting.

365 00:44:50.620 00:44:51.620 Annie Yu: Yes.

366 00:44:51.620 00:44:52.300 Luke Daque: Fine.

367 00:44:52.300 00:44:58.605 Annie Yu: Oh, and I’m actually not seeing your screen. But I I did see your your

368 00:45:00.520 00:45:01.310 Luke Daque: Yeah.

369 00:45:01.710 00:45:03.749 Annie Yu: Movement to the Google Sheet.

370 00:45:04.810 00:45:06.032 Luke Daque: That’s fine. Then

371 00:45:06.620 00:45:07.050 Annie Yu: Yeah.

372 00:45:07.050 00:45:14.619 Luke Daque: Yeah, I can. That’s that’s we can save this in in the slack channel.

373 00:45:15.290 00:45:22.430 Annie Yu: Okay, should you send it, or should I send it? I mean, either way, we can copy.

374 00:45:25.367 00:45:26.970 Luke Daque: Yeah, let me send it, then.

375 00:45:27.930 00:45:34.779 Annie Yu: Okay, that I will share. How do you name them?

376 00:45:35.110 00:45:40.380 Luke Daque: I just named them. The like list call records synthetic data, something like that.

377 00:45:40.530 00:45:44.010 Annie Yu: Okay, let me.

378 00:45:49.280 00:45:52.009 Annie Yu: These are the the 2.

379 00:46:10.460 00:46:14.719 Annie Yu: So only I’m looking through the 4. So

380 00:46:15.620 00:46:23.190 Annie Yu: I guess only wait. Only list events has email right.

381 00:46:24.770 00:46:25.380 Luke Daque: All right.

382 00:46:27.660 00:46:30.162 Annie Yu: Yeah, I’m thinking through how we

383 00:46:31.060 00:46:34.560 Annie Yu: and one thing. Oh, no way.

384 00:46:35.070 00:46:40.510 Annie Yu: I think one thing we missed, and I don’t know if that will affect is we want.

385 00:46:40.510 00:46:41.170 Luke Daque: Oh!

386 00:46:41.170 00:46:42.690 Annie Yu: Be able to join.

387 00:46:43.060 00:46:44.000 Luke Daque: Yeah.

388 00:46:44.380 00:46:45.710 Annie Yu: Them together.

389 00:46:46.250 00:46:47.439 Luke Daque: Yeah, that’s what I need.

390 00:46:51.470 00:46:55.509 Luke Daque: Yeah, I can. I can work on that that piece.

391 00:46:55.990 00:47:00.360 Luke Daque: I guess, for now, at least, there’s something we can show them, and then.

392 00:47:00.360 00:47:07.250 Annie Yu: Okay. And okay, can you guide me through? Walk me through how to do that? Just because

393 00:47:07.758 00:47:19.490 Annie Yu: I think I, if we are to say, like, we want to based on these email address, because I, we still have to do the synthetic data for success factors. So.

394 00:47:19.490 00:47:19.969 Luke Daque: I think it.

395 00:47:19.970 00:47:29.459 Annie Yu: Do the same. I I don’t really know the right steps. But if you are saying like, Okay, let’s stick with these emails. I will grab those and then

396 00:47:29.620 00:47:36.010 Annie Yu: generate also the the synthetic data for success factors.

397 00:47:42.090 00:47:46.370 Luke Daque: Yeah, I’m not sure yet, like how we can

398 00:47:49.670 00:47:50.850 Luke Daque: do this.

399 00:47:51.630 00:47:53.760 Annie Yu: Okay, and.

400 00:47:53.760 00:47:55.349 Luke Daque: I wonder if

401 00:48:00.770 00:48:01.700 Annie Yu: Ape.

402 00:48:06.100 00:48:11.829 Luke Daque: I wonder if the cursor can do that? Because, like, if we put all the files in the same

403 00:48:12.760 00:48:19.290 Luke Daque: folder, it should have context, and then maybe we can ask it to update all the email.

404 00:48:20.640 00:48:24.859 Luke Daque: Addresses, or it depends, like what we need to join. Right? Is it.

405 00:48:25.980 00:48:40.620 Annie Yu: Yeah, I think we do want to join all 4 together, but also be able to join them with success factors, I think, with success factors. We, we utilize email address. But then.

406 00:48:40.620 00:48:41.220 Luke Daque: Yeah.

407 00:48:56.240 00:48:57.920 Annie Yu: Okay, I think it’s

408 00:49:02.580 00:49:06.570 Annie Yu: okay. I think it’s okay. We shared them for now.

409 00:49:10.160 00:49:12.905 Luke Daque: But yeah, that’s still something that we need to do

410 00:49:13.860 00:49:18.500 Luke Daque: like in order to, because we can’t join anything. If that nothing matches.

411 00:49:19.750 00:49:21.830 Annie Yu: Yeah, exactly.

412 00:49:23.060 00:49:28.370 Luke Daque: Even like, if you look at list call records. There’s a in the organizer.

413 00:49:31.020 00:49:34.739 Annie Yu: Yes, that one has personal info.

414 00:49:35.060 00:49:44.230 Luke Daque: Yeah, we should be able to join this user Id to list messages, for example. So we know what messages this user has, like.

415 00:49:44.410 00:49:47.860 Luke Daque: what events, this user, as.

416 00:49:48.300 00:49:53.310 Annie Yu: Okay, and what about the others? So get all messages.

417 00:49:55.720 00:49:56.120 Annie Yu: Brown.

418 00:49:56.120 00:49:58.040 Luke Daque: It’s.

419 00:50:01.410 00:50:01.910 Annie Yu: Row.

420 00:50:02.350 00:50:08.279 Luke Daque: Yeah, yeah, we can use the front because it also has user. Id you, sir.

421 00:50:08.280 00:50:13.689 Annie Yu: Wait, user it, but not email, right?

422 00:50:14.720 00:50:17.230 Luke Daque: It. Yeah, it would be great if I mean

423 00:50:18.720 00:50:25.319 Luke Daque: ideally, we should be joining ids and not like emails. Right? So should be the user. Id.

424 00:50:25.820 00:50:29.919 Annie Yu: So that means this one, the the Id.

425 00:50:30.940 00:50:34.960 Luke Daque: Not really cause like. That’s the idea of the message.

426 00:50:34.960 00:50:35.890 Annie Yu: Yeah.

427 00:50:36.540 00:50:37.060 Luke Daque: What?

428 00:50:38.920 00:50:43.420 Luke Daque: So like, let me list, wait.

429 00:50:49.240 00:50:55.670 Luke Daque: Yeah, I’m I’m confused now, like, what’s the difference between list message and get all message.

430 00:50:56.120 00:51:02.199 Annie Yu: I think list messages is only for mail, for I guess outlook and.

431 00:51:02.200 00:51:02.920 Luke Daque: Let’s see.

432 00:51:02.920 00:51:05.159 Annie Yu: Get all messages is across.

433 00:51:06.250 00:51:08.349 Annie Yu: I think teams.

434 00:51:09.320 00:51:11.450 Luke Daque: So chats and stuff like that.

435 00:51:11.450 00:51:12.270 Annie Yu: Yeah.

436 00:51:15.210 00:51:15.940 Luke Daque: Alright!

437 00:51:19.440 00:51:23.259 Annie Yu: Do you think this is something we should? We can like?

438 00:51:23.850 00:51:26.550 Annie Yu: Ask out, Holmes, help.

439 00:51:28.110 00:51:30.209 Luke Daque: Yeah, we can. We can definitely do that.

440 00:51:30.880 00:51:36.679 Luke Daque: But yeah, for sure, we’ll be joining, using the Id

441 00:51:37.020 00:51:41.320 Luke Daque: like, we would have a list of all the users. And then

442 00:51:41.950 00:51:51.540 Luke Daque: for each user, we will be able to know what outlook emails in that user pass.

443 00:51:51.670 00:51:58.470 Luke Daque: whether it’s coming from the stick, whether he’s the sender or the recipient, or whatever.

444 00:51:58.630 00:52:04.790 Luke Daque: And then we also get all the messages from teams for that user.

445 00:52:05.280 00:52:05.970 Annie Yu: Yeah.

446 00:52:05.970 00:52:13.400 Luke Daque: And then also, like calendar events for that user. So but I think that’s like what

447 00:52:14.300 00:52:15.929 Luke Daque: we want to achieve.

448 00:52:16.290 00:52:24.280 Annie Yu: So we we ideally would have another. I guess another table that shows user id wait, or

449 00:52:25.970 00:52:27.460 Luke Daque: Yeah.

450 00:52:27.460 00:52:30.250 Annie Yu: Like a massive table. No, I’m I’m I’m so comfortable.

451 00:52:30.250 00:52:30.930 Luke Daque: She’s not.

452 00:52:32.560 00:52:38.340 Luke Daque: Yeah. I think we should have a user table. I don’t know if that exists in Microsoft. Let me check

453 00:52:39.010 00:52:41.990 Luke Daque: users. Yeah, there’s users.

454 00:52:41.990 00:52:46.180 Annie Yu: Answers from success. Success Factors.

455 00:52:48.580 00:52:52.390 Luke Daque: Oh, that then? Yeah, I think 20.

456 00:52:54.080 00:52:57.570 Luke Daque: Yeah, we need. That would be like the master

457 00:52:57.760 00:53:02.039 Luke Daque: file for the users and their ids and their emails and stuff.

458 00:53:07.030 00:53:09.990 Luke Daque: And then, ideally, all the messages, events.

459 00:53:10.240 00:53:16.650 Luke Daque: teams and calendars would be coming from that list of user ids and emails.

460 00:53:19.270 00:53:19.980 Annie Yu: Yeah.

461 00:53:21.950 00:53:26.050 Luke Daque: Yeah, we can. Yeah, I’ll just send out the

462 00:53:26.570 00:53:28.449 Luke Daque: these in slack, and then we’ll just.

463 00:53:28.450 00:53:28.860 Annie Yu: Yeah.

464 00:53:28.860 00:53:30.909 Luke Daque: Put a note that these are.

465 00:53:30.910 00:53:33.850 Annie Yu: We probably need to have another session.

466 00:53:35.890 00:53:43.359 Annie Yu: And and even before that we wouldn’t know how to change our our files right? These 4

467 00:53:43.630 00:53:47.200 Annie Yu: before we we really figure out how to join them.

468 00:53:47.980 00:53:49.340 Luke Daque: Yeah, I don’t

469 00:53:49.730 00:53:57.200 Luke Daque: think so. Yeah, we well, we can always like try to ask any like the AI to see if it can

470 00:53:58.250 00:54:07.910 Luke Daque: do it. But yes should be possible, though. Yeah, we just need to be specific or something.

471 00:54:08.500 00:54:10.529 Annie Yu: Like a have a good prom.

472 00:54:11.780 00:54:16.530 Annie Yu: Yeah, all right.

473 00:54:16.770 00:54:17.620 Luke Daque: So I think.

474 00:54:17.620 00:54:21.962 Annie Yu: We’ll share these, for now and then we’ll we’ll have to.

475 00:54:22.750 00:54:25.150 Annie Yu: probably have another session, but I think.

476 00:54:25.150 00:54:25.660 Luke Daque: Thank you.

477 00:54:29.260 00:54:32.670 Annie Yu: Yeah, I’m I’m I’m like, not super

478 00:54:33.110 00:54:47.419 Annie Yu: knowledgeable in terms of like data modeling. So I I don’t know where like, when you think we should bring in, or or you are like comfortable that we we can do it.

479 00:54:49.270 00:54:50.270 Luke Daque: Yeah, I think.

480 00:54:51.630 00:55:00.310 Luke Daque: yeah, I think we can. We can try to take a stab at it like we’ll try to do it. And then, if we really can’t get anything done, then maybe we’ll have the time.

481 00:55:01.110 00:55:01.560 Annie Yu: Okay.

482 00:55:01.560 00:55:03.289 Luke Daque: We can get help when we come here.

483 00:55:03.660 00:55:04.690 Annie Yu: Yeah, yeah.

484 00:55:05.440 00:55:15.650 Annie Yu: okay, I’ll think through these 2. But in the meantime, do do I need to go ahead and then generate synthetic data for 6 X factors.

485 00:55:16.110 00:55:20.280 Luke Daque: Yeah, that’d be great. If you can start with that, then, yeah, we can.

486 00:55:20.740 00:55:21.930 Luke Daque: We can do that.

487 00:55:22.260 00:55:32.050 Annie Yu: Yeah. And I think I’ll use based on the 1st file that we generate today is the list messages. I think I’ll grab those emails and then.

488 00:55:32.660 00:55:33.310 Luke Daque: Yeah.

489 00:55:33.310 00:55:36.710 Annie Yu: Use those emails for 6 6 factors. Okay?

490 00:55:36.710 00:55:37.300 Luke Daque: Yeah.

491 00:55:37.490 00:55:39.200 Annie Yu: Nice, nice, alright.

492 00:55:39.200 00:55:39.760 Luke Daque: Okay.

493 00:55:40.500 00:55:41.219 Annie Yu: Oh, it’s good.

494 00:55:41.220 00:55:42.540 Annie Yu: Thank you so much. Okay.

495 00:55:42.540 00:55:44.730 Luke Daque: Thanks, Amy, have a nice rest of your day.

496 00:55:44.900 00:55:46.340 Annie Yu: You too, bye.