Meeting Title: Daily AI Team Sync Date: 2025-03-19 Meeting participants: Janna Wong, Amber Lin, Uttam, Miguel De Veyra, Casie Aviles
WEBVTT
1 00:02:30.120 ⇒ 00:02:31.230 Miguel de Veyra: Hello, Hello!
2 00:03:04.820 ⇒ 00:03:08.749 Miguel de Veyra: He seems like he’s good for you.
3 00:03:10.740 ⇒ 00:03:14.750 Miguel de Veyra: What? The Fuck is that, smart oh, shit.
4 00:03:45.820 ⇒ 00:03:46.570 Miguel de Veyra: Oh.
5 00:03:52.440 ⇒ 00:03:53.310 Miguel de Veyra: oh.
6 00:04:06.560 ⇒ 00:04:10.810 Miguel de Veyra: guys are, even by now, so I don’t know.
7 00:04:11.800 ⇒ 00:04:12.515 Miguel de Veyra: Alright!
8 00:08:38.350 ⇒ 00:08:39.979 Miguel de Veyra: Right? Hello, voice!
9 00:08:41.350 ⇒ 00:08:42.380 Miguel de Veyra: And that was an admin
10 00:08:42.380 ⇒ 00:08:42.740 Casie Aviles: Hey?
11 00:08:44.640 ⇒ 00:08:45.480 Casie Aviles: Another book.
12 00:08:45.780 ⇒ 00:08:51.210 Miguel de Veyra: Man. I love painting so much with 5,000.
13 00:08:51.760 ⇒ 00:08:54.810 Miguel de Veyra: What my thoughts on here hold on today.
14 00:09:04.530 ⇒ 00:09:05.300 Miguel de Veyra: Okay.
15 00:09:11.550 ⇒ 00:09:12.370 Casie Aviles: You’re welcome.
16 00:09:12.720 ⇒ 00:09:13.760 Casie Aviles: Nice
17 00:09:13.760 ⇒ 00:09:17.370 Miguel de Veyra: I will turn this on nice.
18 00:09:21.110 ⇒ 00:09:24.300 Miguel de Veyra: and you don’t oh, better like eventually you get it.
19 00:09:25.100 ⇒ 00:09:27.129 Miguel de Veyra: So I never paid that before. Right
20 00:09:27.470 ⇒ 00:09:33.680 Casie Aviles: Fine lang I mean
21 00:09:35.060 ⇒ 00:09:39.019 Miguel de Veyra: It’s but it’s like super thin
22 00:09:39.530 ⇒ 00:09:40.260 Casie Aviles: Hmm.
23 00:09:41.070 ⇒ 00:09:46.730 Miguel de Veyra: I was collecting this right. I have, like 20 pieces
24 00:09:48.120 ⇒ 00:09:48.650 Casie Aviles: Yeah.
25 00:09:48.970 ⇒ 00:09:51.400 Miguel de Veyra: And then I was like, but I can’t play with it.
26 00:09:51.700 ⇒ 00:09:52.770 Miguel de Veyra: Hey, Amber?
27 00:09:53.870 ⇒ 00:09:55.600 Miguel de Veyra: Good morning. Good night.
28 00:09:59.170 ⇒ 00:10:00.390 Miguel de Veyra: It’s messy.
29 00:10:02.000 ⇒ 00:10:02.850 Miguel de Veyra: Oh.
30 00:10:03.200 ⇒ 00:10:03.980 Casie Aviles: Hey! Amber.
31 00:10:05.660 ⇒ 00:10:10.979 Amber Lin: School, actually, because I I don’t know if I told you guys I’m getting my wisdom tooth withdrawn
32 00:10:10.980 ⇒ 00:10:11.710 Miguel de Veyra: A little bit later.
33 00:10:11.710 ⇒ 00:10:15.149 Amber Lin: Yeah, I’m trying to get full of them. Yes. So I have
34 00:10:15.270 ⇒ 00:10:20.590 Amber Lin: the weekly, the daily stand-ups for the different teams. And I’m going to go to the dentist
35 00:10:21.260 ⇒ 00:10:22.989 Miguel de Veyra: How how many will get removed?
36 00:10:25.710 ⇒ 00:10:26.590 Miguel de Veyra: Campaign
37 00:10:31.760 ⇒ 00:10:32.740 Casie Aviles: Please share that.
38 00:10:32.740 ⇒ 00:10:33.750 Miguel de Veyra: Oh, fair, fair.
39 00:10:43.670 ⇒ 00:10:45.359 Amber Lin: Am I cutting out right now
40 00:10:45.360 ⇒ 00:10:47.619 Miguel de Veyra: Yeah, we can’t hear you. Oh, there you go now, you just
41 00:10:47.620 ⇒ 00:10:51.502 Amber Lin: Let me get rid of the video to make it a little bit easier.
42 00:10:52.160 ⇒ 00:10:54.960 Amber Lin: Okie, dokie Hi, otam!
43 00:10:54.960 ⇒ 00:10:56.870 Miguel de Veyra: You got to remove. Oh, Hilton.
44 00:10:57.720 ⇒ 00:10:59.679 Amber Lin: Oh, he’s not here! Hi! Watam!
45 00:10:59.980 ⇒ 00:11:01.040 Uttam: Hey! Good morning!
46 00:11:01.770 ⇒ 00:11:03.329 Amber Lin: Good morning. Good morning.
47 00:11:06.720 ⇒ 00:11:08.389 Amber Lin: Miguel. What were you saying?
48 00:11:08.570 ⇒ 00:11:10.690 Miguel de Veyra: How many will you get removed today
49 00:11:11.239 ⇒ 00:11:21.779 Amber Lin: I’m aiming to get all 4 of them removed, because I don’t want to recover twice, and don’t want to be a chipmunk twice, but I think 4 is gonna be a little tough
50 00:11:21.780 ⇒ 00:11:23.719 Miguel de Veyra: Is it all impacted or just normal?
51 00:11:23.720 ⇒ 00:11:28.009 Amber Lin: Yes, it is. It is, in fact, all in the bone. So
52 00:11:28.010 ⇒ 00:11:29.000 Miguel de Veyra: Oh, good luck!
53 00:11:29.000 ⇒ 00:11:30.320 Amber Lin: A lot of drilling
54 00:11:33.330 ⇒ 00:11:34.460 Miguel de Veyra: And it’s awesome.
55 00:11:35.460 ⇒ 00:11:38.809 Miguel de Veyra: I had mine removed like last year, though the 2 above
56 00:11:39.180 ⇒ 00:11:40.140 Amber Lin: Oh dear!
57 00:11:40.140 ⇒ 00:11:43.369 Miguel de Veyra: Painful. I didn’t go for the bottom ones, I think
58 00:11:43.370 ⇒ 00:11:44.120 Amber Lin: Huh!
59 00:11:44.810 ⇒ 00:11:47.210 Amber Lin: Wow! I mean, if they’re not impacted
60 00:11:47.550 ⇒ 00:11:48.750 Miguel de Veyra: They are. I just didn’t want to.
61 00:11:48.750 ⇒ 00:11:50.080 Miguel de Veyra: Oh, do it again!
62 00:11:50.240 ⇒ 00:11:53.640 Amber Lin: Okay, that’s why 2, 4 at the same time.
63 00:11:54.720 ⇒ 00:11:55.410 Amber Lin: There are you guys
64 00:11:55.410 ⇒ 00:11:55.840 Miguel de Veyra: Thanks. Colin.
65 00:11:55.840 ⇒ 00:11:56.410 Amber Lin: Year.
66 00:11:56.740 ⇒ 00:11:59.660 Amber Lin: Sorry. Go go ahead.
67 00:11:59.840 ⇒ 00:12:09.139 Amber Lin: I’m gonna share my screen on linear my issues are not very groomed. That was my goal.
68 00:12:09.380 ⇒ 00:12:12.489 Amber Lin: But right now we have the
69 00:12:12.850 ⇒ 00:12:16.520 Amber Lin: all issues, and I have it per project.
70 00:12:16.720 ⇒ 00:12:25.080 Amber Lin: So I just wanted to check on the progress of everybody and see what we see what we have for
71 00:12:26.220 ⇒ 00:12:28.140 Amber Lin: for this week.
72 00:12:30.260 ⇒ 00:12:30.740 Amber Lin: So
73 00:12:32.629 ⇒ 00:12:39.860 Amber Lin: Miguel yes, I mean yesterday you already did. The most part, was the voice assistant. I know you also worked on the real dashboard right
74 00:12:49.250 ⇒ 00:12:54.480 Casie Aviles: Go for for real, I think. Yeah, I was.
75 00:12:55.000 ⇒ 00:12:57.130 Casie Aviles: I worked with Utam with that
76 00:13:01.450 ⇒ 00:13:07.019 Uttam: Yeah, are we? Is that ready to go like, are we ready to send that over? Or I guess my question yesterday was like.
77 00:13:07.500 ⇒ 00:13:14.360 Uttam: you know the question on Friday from the client was, where do we go to see the data like, can we consider that closed right now?
78 00:13:15.350 ⇒ 00:13:42.020 Amber Lin: Yeah, I showed you that when I met her on Monday, and her request was to add a few kpis, and I asked which one. And and she was like, Okay, I’ll send it when we have the Facebook proposal. So right now, we’re trying to add thumbs up and thumbs down data. That was something that we’re we’re talking about yesterday, but other than that we still, we could still add some other kpis, but they’ve seen it already.
79 00:13:42.430 ⇒ 00:13:44.899 Uttam: But do they have like? They have access to it?
80 00:13:45.790 ⇒ 00:13:49.399 Amber Lin: They have access to the one in the
81 00:13:50.170 ⇒ 00:13:52.919 Uttam: That’s I guess my my question is even earlier than that. Like.
82 00:13:53.300 ⇒ 00:13:56.249 Uttam: do they know that they can go see this data somewhere?
83 00:13:57.308 ⇒ 00:14:00.479 Amber Lin: That they though I’ve shown a bed, they have seen it
84 00:14:01.020 ⇒ 00:14:10.019 Uttam: Okay, so that we should, we should just get out the door. Because, as you know, like spending more time waiting on event is, gonna it’s gonna take a while
85 00:14:10.510 ⇒ 00:14:15.900 Uttam: So whatever we have, I wanna just get out the door because they asked for this on Friday.
86 00:14:16.010 ⇒ 00:14:20.949 Uttam: If we spend another week on metrics. Then it’s gonna
87 00:14:21.070 ⇒ 00:14:23.778 Uttam: it’s just gonna take time. So
88 00:14:25.290 ⇒ 00:14:32.220 Uttam: I think it’s best that like, can we? What can we? What can we get to them today in terms of like, Hey, here’s the dashboard. We’re continuing to make improvements
89 00:14:35.229 ⇒ 00:14:47.450 Amber Lin: Miguel, how’s the I know you sent a part in in the slack saying, now they can also see on the client, Hub, how’s what does that mean? Does it mean that we send them the
90 00:14:47.800 ⇒ 00:14:50.670 Miguel de Veyra: Yeah, I sent you the link in AI team chat
91 00:14:50.800 ⇒ 00:14:54.540 Miguel de Veyra: the demo dot brain forge, slash client ABC live
92 00:14:54.540 ⇒ 00:14:55.430 Amber Lin: No.
93 00:14:59.180 ⇒ 00:15:03.609 Uttam: Yeah, let’s just ship this, because otherwise we’re gonna lose another few days
94 00:15:03.610 ⇒ 00:15:03.950 Miguel de Veyra: Yeah.
95 00:15:03.950 ⇒ 00:15:04.560 Amber Lin: Okay.
96 00:15:04.920 ⇒ 00:15:08.720 Miguel de Veyra: I had to. Yeah, I had to fix this earlier, because.
97 00:15:08.840 ⇒ 00:15:12.630 Miguel de Veyra: yeah, Demo, that reinforce a different URL than the ones from Hero.
98 00:15:13.800 ⇒ 00:15:14.500 Miguel de Veyra: Yeah.
99 00:15:15.420 ⇒ 00:15:24.099 Amber Lin: Okay, sounds good. So that’s an important part. I put in the tickets to clean up
100 00:15:24.200 ⇒ 00:15:27.290 Amber Lin: to clean up the golden data sheet and
101 00:15:27.290 ⇒ 00:15:33.649 Uttam: I guess. Just look one more piece on the dashboard amber. Are you gonna own setting that out today?
102 00:15:34.400 ⇒ 00:15:35.800 Amber Lin: Yeah, I can send it up
103 00:15:36.440 ⇒ 00:15:37.870 Uttam: Okay, yeah, I,
104 00:15:37.870 ⇒ 00:15:42.310 Amber Lin: What is the what is the acceptance criteria for shipping it out
105 00:15:44.590 ⇒ 00:15:57.599 Uttam: I mean, I would like for me, like from my perspective, I think it’s ready, like I think their data is there. As long as Miguel. You signed off that the data is like accurate, and that everything is there.
106 00:15:58.035 ⇒ 00:16:03.719 Uttam: and like they can, the client can access it. Then it just needs to go into an email
107 00:16:03.720 ⇒ 00:16:04.090 Miguel de Veyra: Yeah.
108 00:16:04.090 ⇒ 00:16:07.885 Uttam: Maybe maybe even a quick loom about how to log in
109 00:16:08.900 ⇒ 00:16:12.889 Uttam: or like quickly move about how to log in and like what they’re looking at.
110 00:16:13.896 ⇒ 00:16:15.090 Uttam: That’s it.
111 00:16:16.020 ⇒ 00:16:21.719 Amber Lin: Okay. Sounds good. Yeah, cause you’ve already seen it, and they have the link.
112 00:16:22.101 ⇒ 00:16:27.439 Amber Lin: But I don’t think they have linked directly to the real. I’ll check that, and I’ll send it today.
113 00:16:27.670 ⇒ 00:16:31.399 Uttam: But I don’t know. Does like like. For example, does Steven know where he can go?
114 00:16:31.700 ⇒ 00:16:33.779 Uttam: Look at the data for the agent.
115 00:16:34.490 ⇒ 00:16:35.870 Uttam: If you were to ask him
116 00:16:37.290 ⇒ 00:16:37.920 Amber Lin: I see.
117 00:16:38.470 ⇒ 00:16:43.170 Amber Lin: Okay, I’ll send a separate email just outlining this issue. I think it’ll make it a little bit more clear
118 00:16:43.380 ⇒ 00:16:52.869 Uttam: Yeah, I I think like we could just close it out. We’re we’re gonna keep making changes. But the question on Friday was, Hey, where is this data? So that indicated to me that
119 00:16:53.020 ⇒ 00:16:59.050 Miguel de Veyra: They don’t know clearly where to go to get that so one piece is definitely on
120 00:16:59.580 ⇒ 00:17:03.120 Uttam: On getting that to them, you know, as soon as we can.
121 00:17:03.260 ⇒ 00:17:05.039 Uttam: and then that’ll be closed out
122 00:17:05.710 ⇒ 00:17:06.380 Amber Lin: Okay.
123 00:17:06.720 ⇒ 00:17:09.360 Amber Lin: So I’ll close that today.
124 00:17:09.940 ⇒ 00:17:14.280 Amber Lin: That’s dashboard, Miguel. I think this is this is done right
125 00:17:14.280 ⇒ 00:17:19.099 Miguel de Veyra: Yeah, yeah, this is one. There’s no done thing, I think, because I use the board
126 00:17:19.440 ⇒ 00:17:22.839 Amber Lin: Oh, I sweet audio
127 00:17:22.849 ⇒ 00:17:24.139 Miguel de Veyra: Oh, purple. Okay. Okay.
128 00:17:24.140 ⇒ 00:17:28.640 Amber Lin: Yeah, I think when you use a board, I think the done part is just hidden
129 00:17:28.830 ⇒ 00:17:29.940 Miguel de Veyra: Okay, okay. I see.
130 00:17:29.940 ⇒ 00:17:37.189 Amber Lin: Yeah. And also, I’m just checking in. Ha, have we updated the data yet of because yesterday we talked about it’s not completely updated
131 00:17:39.960 ⇒ 00:17:46.090 Miguel de Veyra: I think that’s the I think that’s the thumbs up thumbs down. But I think we’re gonna ship it first.st As Utah mentioned
132 00:17:46.760 ⇒ 00:17:49.819 Uttam: But is, is like is the data up to date. Otherwise
133 00:17:50.060 ⇒ 00:17:52.510 Miguel de Veyra: Like, do you see data from like yesterday?
134 00:17:52.870 ⇒ 00:17:53.930 Miguel de Veyra: Open the dashboard.
135 00:17:54.300 ⇒ 00:17:56.150 Miguel de Veyra: Yeah. I mean, we can check now
136 00:17:56.180 ⇒ 00:17:57.460 Uttam: Yeah, check.
137 00:17:57.460 ⇒ 00:17:58.700 Amber Lin: I’ll let you share your screen.
138 00:17:59.100 ⇒ 00:17:59.830 Miguel de Veyra: Okay.
139 00:18:00.880 ⇒ 00:18:09.600 Uttam: I guess while we’re on that. Yeah. My next question is, gonna be about evals, like, I checked the spreadsheet yesterday. It’s still sort of in the same
140 00:18:09.970 ⇒ 00:18:11.410 Uttam: state.
141 00:18:11.630 ⇒ 00:18:17.340 Uttam: So my feeling is that we haven’t done we we didn’t make much progress there right
142 00:18:17.800 ⇒ 00:18:18.400 Amber Lin: Oh!
143 00:18:18.400 ⇒ 00:18:20.700 Miguel de Veyra: They’re doing majority of the I’m sorry.
144 00:18:20.770 ⇒ 00:18:21.600 Miguel de Veyra: Go ahead.
145 00:18:21.600 ⇒ 00:18:22.999 Amber Lin: Sorry I cut you off
146 00:18:23.450 ⇒ 00:18:26.500 Miguel de Veyra: Yeah, they’re they’re prioritizing, I think, working here with them.
147 00:18:27.040 ⇒ 00:18:30.549 Miguel de Veyra: Like, as you can see, they’re like very active in this one. Not really that one
148 00:18:32.713 ⇒ 00:18:35.080 Uttam: Wait! What do you mean?
149 00:18:35.550 ⇒ 00:18:36.580 Amber Lin: Oh, no.
150 00:18:36.580 ⇒ 00:18:51.319 Amber Lin: okay, I think I get your point. So we, our team has not worked on a golden data set. The client has been updating the mostly the Google Doc. Janice has been occasionally updating the golden data sheet. But I get what you mean that we want
151 00:18:51.320 ⇒ 00:18:54.500 Uttam: But it’s it’s yeah, yeah, exactly like.
152 00:18:54.500 ⇒ 00:18:58.529 Amber Lin: By it, based on types so that they can navigate it easier right
153 00:18:58.530 ⇒ 00:19:06.180 Uttam: Exactly like our conversation on Friday, was question from Scott. About what questions are in the Evals.
154 00:19:06.300 ⇒ 00:19:10.620 Uttam: how hard are they, and how are we scoring right simple
155 00:19:11.600 ⇒ 00:19:12.850 Miguel de Veyra: Wait, let me do it.
156 00:19:15.180 ⇒ 00:19:18.590 Miguel de Veyra: Oh, yeah, really is not loading on my end.
157 00:19:21.700 ⇒ 00:19:27.340 Miguel de Veyra: because it’s not throwing in there. But let me just same training
158 00:19:36.590 ⇒ 00:19:40.270 Uttam: I guess. Let’s keep talking about the Evals while I have a sec. So yeah.
159 00:19:40.630 ⇒ 00:19:41.070 Miguel de Veyra: Sure.
160 00:19:41.070 ⇒ 00:19:43.390 Uttam: My my questions there were, just
161 00:19:44.010 ⇒ 00:20:01.959 Uttam: does the client like? The 1st thing is like, I think, the Eval data sheet? Yeah, it’s still in the same state. But really, there’s a multiple ways to solve this problem. It doesn’t necessarily need to be the Eval data sheet. But even for me, I still don’t know how we’re scoring on those, and that’s that wasn’t in real as of yesterday.
162 00:20:04.970 ⇒ 00:20:10.040 Uttam: So that’s a fun that’s like one. Those are the 2 biggest things from Friday that I still am like
163 00:20:10.830 ⇒ 00:20:11.949 Uttam: Where are we? On?
164 00:20:14.210 ⇒ 00:20:15.170 Amber Lin: I see.
165 00:20:16.006 ⇒ 00:20:21.549 Amber Lin: Let me put that into linear
166 00:20:23.390 ⇒ 00:20:35.689 Uttam: But this is like a 1 h thing. Guys like it’s just to clean up the Eval sheet. And then I mean, I’m going to do the Eval sheet if we can’t do it today. But where this like, are we getting the Brain trust scores into real
167 00:20:36.900 ⇒ 00:20:38.240 Uttam: Casey or Miguel
168 00:20:38.887 ⇒ 00:20:39.990 Miguel de Veyra: No, not yet
169 00:20:40.640 ⇒ 00:20:42.710 Uttam: What’s the what’s the blocker there?
170 00:20:47.231 ⇒ 00:20:51.210 Miguel de Veyra: What we decided was just to have the thumbs up, thumbs down
171 00:20:51.910 ⇒ 00:20:53.930 Casie Aviles: To add it, to add that 1st
172 00:20:55.840 ⇒ 00:21:01.029 Uttam: But that’s not a that’s not from brain trust. That’s from the client that’s from like.
173 00:21:02.560 ⇒ 00:21:05.831 Uttam: you guys see what I’m you guys, you guys see what I’m getting at. Right?
174 00:21:06.900 ⇒ 00:21:09.110 Uttam: You know what I’m gonna say.
175 00:21:09.390 ⇒ 00:21:21.680 Uttam: I want to know from the beginning of this project. All I want to know is how we’re scoring on the Evals, right? We spend so much time on the Evals. Still, we’re at the finish line. We’re like 10 feet from the finish line
176 00:21:22.080 ⇒ 00:21:29.350 Uttam: we just need, I just need to know, like I can’t tell them. We can’t tell the client accurately whether our answers are right or not.
177 00:21:31.310 ⇒ 00:21:34.840 Uttam: Nobody is looking at the Evals is what I’m hearing
178 00:21:37.440 ⇒ 00:21:45.860 Uttam: So what I I just like, what could be more important than that? Because we can’t. How can? How can like? If if I was playing the client
179 00:21:46.490 ⇒ 00:21:48.690 Uttam: I was gonna say, before rolling this out.
180 00:21:48.830 ⇒ 00:21:50.460 Uttam: how do you know it’s accurate?
181 00:21:51.830 ⇒ 00:21:55.379 Uttam: How can we answer like? So then that seems like the critical path right
182 00:21:56.640 ⇒ 00:21:57.400 Amber Lin: Oh, yeah.
183 00:21:58.970 ⇒ 00:21:59.780 Amber Lin: Okay.
184 00:22:00.040 ⇒ 00:22:00.770 Amber Lin: So
185 00:22:00.770 ⇒ 00:22:02.840 Uttam: I guess. Miguel Casey, what do you guys think
186 00:22:04.983 ⇒ 00:22:05.649 Miguel de Veyra: I mean
187 00:22:05.650 ⇒ 00:22:11.399 Uttam: You could push back. If it’s not, I mean I I’m I’m open to other options. But for me.
188 00:22:11.770 ⇒ 00:22:16.470 Uttam: looking on the outside, that’s that seems really really important.
189 00:22:17.087 ⇒ 00:22:22.329 Miguel de Veyra: Yeah, I mean, wasn’t it also discussed, like last Friday that we kind of put the
190 00:22:22.910 ⇒ 00:22:27.989 Miguel de Veyra: evil stuff behind first, st since they don’t, they’re more interested in like a thumbs up thumbs down.
191 00:22:28.960 ⇒ 00:22:33.249 Miguel de Veyra: That’s why we didn’t really work on like putting, you know the the
192 00:22:33.250 ⇒ 00:22:41.000 Uttam: I felt like they were interested in both, like Scott’s question was about the Evals directly, and like.
193 00:22:41.110 ⇒ 00:22:45.550 Uttam: I don’t, it’s it’s this is where I guess like for for us as engineers.
194 00:22:45.550 ⇒ 00:22:46.110 Miguel de Veyra: Yeah.
195 00:22:46.110 ⇒ 00:22:51.149 Uttam: Whether the thumbs up or thumbs down. Dude. You’re not going to rely on that for your system, are you
196 00:22:51.550 ⇒ 00:22:52.310 Miguel de Veyra: Yeah, yeah.
197 00:22:52.790 ⇒ 00:22:56.599 Uttam: So like we gotta do the evals, even for our sake.
198 00:22:57.270 ⇒ 00:23:01.390 Uttam: The evals are really gonna be the true measure of if we’re accurate or not.
199 00:23:03.290 ⇒ 00:23:13.910 Uttam: you know, but I don’t. I don’t think we’re that far, like I feel like we’re a few hours away from getting that data into Snowflake and into rail, and then cleaning up that spreadsheet like that’s like 3 h of work
200 00:23:14.390 ⇒ 00:23:18.680 Miguel de Veyra: Yeah, I mean, we’re we’re already logging everything on right.
201 00:23:18.810 ⇒ 00:23:20.209 Miguel de Veyra: Let me share screen
202 00:23:20.670 ⇒ 00:23:22.229 Amber Lin: Yeah, you can share screen
203 00:23:22.230 ⇒ 00:23:24.480 Casie Aviles: Yeah, it’s on brain trust. But
204 00:23:24.480 ⇒ 00:23:26.219 Miguel de Veyra: It. Yeah, I mean.
205 00:23:26.580 ⇒ 00:23:29.800 Casie Aviles: We have to send this to Snowflake somehow, I guess
206 00:23:29.800 ⇒ 00:23:31.780 Miguel de Veyra: You know, we can export, I think.
207 00:23:32.550 ⇒ 00:23:38.910 Miguel de Veyra: yeah, we can export it. But I’m not. We haven’t really explored. You know how we can
208 00:23:39.790 ⇒ 00:23:45.099 Miguel de Veyra: like how we can. How do you know? How do you say this like programmatically, or through an Api
209 00:23:45.480 ⇒ 00:23:49.100 Uttam: Okay, but you. But you, you know, it’s gonna take like a couple hours
210 00:23:49.100 ⇒ 00:23:50.030 Miguel de Veyra: Yeah, yeah.
211 00:23:50.240 ⇒ 00:23:51.910 Uttam: So how about let’s do today?
212 00:23:52.060 ⇒ 00:23:53.560 Miguel de Veyra: Okay. Yeah. Sure.
213 00:23:53.560 ⇒ 00:24:01.649 Uttam: So my 1st suggestion is just export the results straight Csv. Into Snowflake
214 00:24:01.900 ⇒ 00:24:07.960 Uttam: and make sure it ends up in real. I’ll show I’ll send you a little note on how to do that.
215 00:24:08.240 ⇒ 00:24:13.339 Uttam: You can go into Snowflake. You can say, insert data, and you can literally just upload a Csv
216 00:24:14.280 ⇒ 00:24:16.650 Miguel de Veyra: Do I need to create like the fields anymore? No.
217 00:24:16.650 ⇒ 00:24:22.349 Uttam: No, no, no, you literally just update. Just make sure that the Csv has the header row, and you literally can just upload it
218 00:24:23.040 ⇒ 00:24:24.349 Miguel de Veyra: You mind doing it now?
219 00:24:28.410 ⇒ 00:24:30.829 Miguel de Veyra: No, no, we’ll do it. We’ll do it later.
220 00:24:31.330 ⇒ 00:24:37.880 Miguel de Veyra: Okay, okay, yeah, then. And then, do you already know how like, how we measure this stuff?
221 00:24:39.610 ⇒ 00:24:42.341 Uttam: Say it. Wait ask me the question a different way.
222 00:24:43.210 ⇒ 00:24:47.110 Miguel de Veyra: Like, basically because, you know, factuality is basically comparing it.
223 00:24:47.300 ⇒ 00:24:48.329 Miguel de Veyra: And you know.
224 00:24:48.460 ⇒ 00:24:53.960 Miguel de Veyra: to an expected answer and stuff like that, do you know. I think we’ve shown you before. But have
225 00:24:54.590 ⇒ 00:24:56.120 Miguel de Veyra: are you? Do you remember
226 00:24:56.780 ⇒ 00:25:02.079 Uttam: Well, yeah, there’s a whole. I sent a document about all the different sort of question types.
227 00:25:02.541 ⇒ 00:25:07.169 Uttam: I can take that if you want. But this is a this is the thing is like, I don’t wanna
228 00:25:07.330 ⇒ 00:25:20.399 Uttam: any work that I take is going to be knowledge that stuck with me. So I would love to work with someone on the team on this. And I can work this afternoon on the An Eval data set with somebody
229 00:25:20.969 ⇒ 00:25:25.810 Amber Lin: I can work with you this afternoon. I just will not be able to talk at all.
230 00:25:27.670 ⇒ 00:25:30.420 Uttam: Well, I’ll work. I’ll call Casey this afternoon. That’s fine.
231 00:25:30.770 ⇒ 00:25:39.960 Amber Lin: I think it’s maybe a little bit late for Casey. I’ll try to join, if possible, because I also want to know, because I have to tell the client eventually. So maybe if it’s a little
232 00:25:39.960 ⇒ 00:25:42.459 Uttam: I mean we can record. I don’t. I don’t want you to be
233 00:25:42.460 ⇒ 00:25:44.960 Miguel de Veyra: Yes, you know, like a medical situation
234 00:25:46.160 ⇒ 00:25:46.670 Uttam: All right.
235 00:25:46.670 ⇒ 00:25:58.290 Uttam: Record it to me and Casey I mean. I don’t know, Casey. You can. You can go to bed early if you want, but we are up like Casey was up with me at like 3 working yesterday. I just think we’re like 2 h from this problem being solved.
236 00:25:58.900 ⇒ 00:26:03.190 Uttam: If we don’t solve it today, it’s gonna be another week
237 00:26:03.410 ⇒ 00:26:04.180 Miguel de Veyra: There you go!
238 00:26:04.180 ⇒ 00:26:10.170 Uttam: And that that means another week before we can confidently, because Scott is, gonna ask you about this on Friday.
239 00:26:10.280 ⇒ 00:26:13.089 Uttam: 100%. And I’m not going to play defense
240 00:26:13.440 ⇒ 00:26:13.880 Amber Lin: Okay.
241 00:26:14.760 ⇒ 00:26:19.639 Uttam: Let’s today, let’s work on like Scott, man, I’m like Scott, Mini
242 00:26:20.190 ⇒ 00:26:23.190 Amber Lin: Okay, don’t worry as we need. We need that. But you guys
243 00:26:23.190 ⇒ 00:26:23.940 Uttam: Hear me! Right!
244 00:26:23.940 ⇒ 00:26:27.300 Amber Lin: We’ll have a problem next week. So thank you for
245 00:26:27.300 ⇒ 00:26:40.699 Uttam: Yeah, let’s let’s get through today on the on the retro we can talk about, you know, okay, like, where, how do we? How do we? How do we approach this and like, how did we miss this and stuff like that? Let’s get this today out the door
246 00:26:41.120 ⇒ 00:26:41.440 Amber Lin: Okay.
247 00:26:41.440 ⇒ 00:26:43.730 Uttam: Kind of like critical paths are one.
248 00:26:44.381 ⇒ 00:27:09.670 Uttam: Miguel, I just want to make sure everything in real and real works. Just try it. A couple of different browsers. Just make sure that the iframe renders the second piece is yeah, if you can get an export of brain trust and just send it to the Channel. I will send over instructions. On how to upload it to Snowflake and Casey you you did you end up getting through on like how to do real stuff. Yesterday, after we talked
249 00:27:09.940 ⇒ 00:27:14.059 Casie Aviles: Yeah, I just didn’t push it anywhere. It’s locally for now
250 00:27:14.360 ⇒ 00:27:23.179 Uttam: Okay, okay, so we can, we can collaborate again later today on getting that over the line. And then we’ll just push it in real I my afternoon’s pretty free.
251 00:27:23.610 ⇒ 00:27:27.389 Uttam: So I will. I’ll spend an hour. We can get this over the line
252 00:27:28.450 ⇒ 00:27:29.719 Miguel de Veyra: Oh, yeah, I sent it now.
253 00:27:30.250 ⇒ 00:27:47.189 Uttam: So I think, Amber, it’s probably best to go ahead. If if the real thing works, it’s probably best to just send that as early as we can, maybe before you head out, and then we we’ll work on the Eval stuff and hopefully have that ready for you to send out later today, if you end up back
254 00:27:47.190 ⇒ 00:27:56.399 Amber Lin: Sounds good, and any updates in the real is that real time? So if I send it to the client, we don’t have to send them another code. They will just use the link right
255 00:27:56.400 ⇒ 00:27:58.020 Uttam: Yes, it’ll all be the link
256 00:27:58.020 ⇒ 00:27:59.260 Amber Lin: We’re hosting. Okay? Sounds good.
257 00:27:59.260 ⇒ 00:28:05.439 Amber Lin: Then I’ll send them that right after. Oh, I have another meeting, but I’ll send them that before I go to the dentist.
258 00:28:05.870 ⇒ 00:28:09.220 Amber Lin: So let me let me just note that down.
259 00:28:10.200 ⇒ 00:28:17.869 Amber Lin: Send client reel today. Okay, cleaning. So today a send client will update
260 00:28:18.060 ⇒ 00:28:27.590 Amber Lin: eval data from snow. Get it to Snowflake, get it into real hopefully, and then clean up the golden data sheet. That’s the 3 main things
261 00:28:27.970 ⇒ 00:28:28.950 Uttam: Yes.
262 00:28:28.950 ⇒ 00:28:29.760 Amber Lin: Do today
263 00:28:29.760 ⇒ 00:28:35.560 Uttam: Is there anything else? Because that’s all that’s on my mind? I don’t. I don’t. I have. Yeah.
264 00:28:35.820 ⇒ 00:28:45.600 Amber Lin: A few other things. So yesterday we had some really good progress on the voice, but we called it in the meeting, and it worked. So that’s really good.
265 00:28:45.790 ⇒ 00:28:51.430 Amber Lin: And for the let’s see for the document update.
266 00:28:51.870 ⇒ 00:28:58.249 Amber Lin: But I think we’re working on integrating a Google chat. And I want to hear from Casey how it is
267 00:28:59.895 ⇒ 00:29:04.730 Casie Aviles: Yeah, sure. So I managed to connect it to Nope
268 00:29:04.730 ⇒ 00:29:07.840 Casie Aviles: code. So, but the yeah, so I guess
269 00:29:08.130 ⇒ 00:29:12.790 Casie Aviles: the question is like, before I was. What my only concern is that before
270 00:29:12.950 ⇒ 00:29:17.550 Casie Aviles: sending that out, I think we should establish, like you know, the the
271 00:29:17.930 ⇒ 00:29:22.320 Casie Aviles: like without sending it to Tim all the time, like manually via email
272 00:29:22.320 ⇒ 00:29:23.730 Amber Lin: Hmm, yeah.
273 00:29:26.167 ⇒ 00:29:28.070 Casie Aviles: But yeah, there was. Yeah, I could show
274 00:29:28.070 ⇒ 00:29:31.319 Uttam: I think that’s gonna I think the Tim thing is gonna take
275 00:29:31.590 ⇒ 00:29:33.860 Uttam: another 2 weeks if I had to guess.
276 00:29:34.470 ⇒ 00:29:38.480 Uttam: So we’re gonna have to. We just have to keep going like I wouldn’t consider that a
277 00:29:38.880 ⇒ 00:29:40.080 Miguel de Veyra: A blocker.
278 00:29:40.450 ⇒ 00:29:42.929 Uttam: Okay, because that that’s like
279 00:29:43.070 ⇒ 00:29:50.020 Uttam: in technology, anything that involves security or like remote deployment, you can think is like the slowest.
280 00:29:50.420 ⇒ 00:30:03.790 Uttam: like worst part of the whole system. So I just if we rely on that as a critical path, we’re never gonna get this out for me. Like to be really honest. I’m like, okay, by next week. How can we deploy this to all 90 people?
281 00:30:04.190 ⇒ 00:30:10.019 Uttam: So I’m like trying to push to see like, okay, what is really left here, you know.
282 00:30:10.210 ⇒ 00:30:11.470 Uttam: because otherwise I
283 00:30:11.470 ⇒ 00:30:12.270 Amber Lin: Art, bot, or
284 00:30:12.270 ⇒ 00:30:14.170 Uttam: On the whole, on the whole, yeah. And the whole thing
285 00:30:14.170 ⇒ 00:30:15.460 Amber Lin: Yes. Okay.
286 00:30:15.460 ⇒ 00:30:16.519 Uttam: The whole thing
287 00:30:16.810 ⇒ 00:30:17.230 Amber Lin: Okay.
288 00:30:17.230 ⇒ 00:30:27.480 Uttam: Because otherwise we don’t have a deadline right every week. We’re gonna sort of keep trudging along slowly, like the voice agent is a good example of like that was like a side that was just like an ask from you that
289 00:30:28.370 ⇒ 00:30:42.560 Uttam: But I wanted I didn’t, I guess, like my point was that that was just like, if we have time to do it, let’s do it, but we still don’t have the Evals ready. We don’t have real real ready. So I want to make sure that all the the core pieces are done before we.
290 00:30:43.180 ⇒ 00:30:47.630 Uttam: so that we can ship this out, because right now nobody is using it except for a few people
291 00:30:48.170 ⇒ 00:30:48.510 Amber Lin: Hmm.
292 00:30:48.510 ⇒ 00:30:53.329 Uttam: That means we haven’t been successful. So I want to get this all the way out as fast as possible.
293 00:30:53.550 ⇒ 00:30:57.650 Amber Lin: I see. Can we split some ideas on the rollout plan?
294 00:30:58.912 ⇒ 00:31:04.639 Uttam: But this is again. I I can’t I can’t! We can’t! Just. We have to discuss that with Yvette and Steven.
295 00:31:04.640 ⇒ 00:31:05.370 Amber Lin: Okay.
296 00:31:05.370 ⇒ 00:31:13.829 Uttam: And, like the the Evals, are not done yet, so as like as the engineering lead. I can’t tell you that we’re we’re accurate yet I don’t know.
297 00:31:14.610 ⇒ 00:31:22.790 Uttam: so I can’t go to them and say we’re accurate. Everything’s ready. The data is ready. How do we roll this out? I can’t do that yet, like it’s I’m not confident yet.
298 00:31:24.380 ⇒ 00:31:25.140 Amber Lin: Okay.
299 00:31:25.980 ⇒ 00:31:27.149 Uttam: See what I mean. Guys.
300 00:31:27.360 ⇒ 00:31:27.900 Amber Lin: Seems like.
301 00:31:28.150 ⇒ 00:31:28.530 Uttam: Okay.
302 00:31:28.530 ⇒ 00:31:36.440 Amber Lin: The main blocker. And then we need that done hopefully today, tomorrow, and then
303 00:31:36.790 ⇒ 00:31:40.910 Amber Lin: we’ll sketch out a rollout plan and we’ll verify it in the meeting
304 00:31:41.230 ⇒ 00:31:44.210 Uttam: Okay, cool.
305 00:31:46.810 ⇒ 00:31:52.459 Uttam: So let’s let’s spend. Let’s spend time later, Casey, if you want to grab time on my calendar.
306 00:31:52.963 ⇒ 00:31:56.140 Uttam: Let’s spend time later. Today we can talk about
307 00:31:56.753 ⇒ 00:32:01.609 Uttam: the Evals. Or if you guys want to try it out between now and then, please
308 00:32:01.610 ⇒ 00:32:02.240 Miguel de Veyra: Sorry, not
309 00:32:02.580 ⇒ 00:32:05.789 Uttam: Go to. Yeah. Just search how to upload Csv into Snowflake.
310 00:32:06.543 ⇒ 00:32:12.769 Uttam: You’ll see the you’ll see the how to do it. Basically, it’s not that bad
311 00:32:13.000 ⇒ 00:32:20.450 Miguel de Veyra: Okay? And then I I’m assuming you. You and Casey discussed how to do the real stuff
312 00:32:20.450 ⇒ 00:32:25.110 Uttam: Yeah. So I need to do probably another 30 min with you guys to finalize that
313 00:32:25.110 ⇒ 00:32:25.590 Miguel de Veyra: Okay. Okay.
314 00:32:26.077 ⇒ 00:32:27.539 Uttam: So I will.
315 00:32:28.020 ⇒ 00:32:33.579 Uttam: Let’s let’s grab some time as soon as I’m free to do that, and then we should be good to go
316 00:32:34.200 ⇒ 00:32:37.949 Amber Lin: Can you guys also just send me the invite. I’ll join if I can
317 00:32:38.150 ⇒ 00:32:38.760 Uttam: Okay.
318 00:32:39.580 ⇒ 00:32:40.280 Amber Lin: Great.
319 00:32:40.740 ⇒ 00:32:43.439 Amber Lin: Okay? Oh, and let’s see.
320 00:32:44.120 ⇒ 00:32:50.930 Amber Lin: Oh, Jenna, I just also want to check on the progress, because I know I didn’t. We didn’t. I forgot about it yesterday.
321 00:32:51.330 ⇒ 00:32:55.010 Amber Lin: I know that. Are you still working on the stuff for Robert? Right
322 00:32:55.360 ⇒ 00:32:55.990 Janna Wong: Yeah.
323 00:32:56.250 ⇒ 00:32:57.899 Janna Wong: Still, stuck with that
324 00:33:00.140 ⇒ 00:33:00.610 Janna Wong: Oh!
325 00:33:00.610 ⇒ 00:33:17.210 Amber Lin: If there’s no progress. Do you want to help with the ABC. Client right now? And I’ll confirm with Robert cause I think the leads for it is. Isn’t. We have enough leads for mixed panel. And I know we also discussed. There’s also a list of other projects
326 00:33:17.630 ⇒ 00:33:18.340 Uttam: Yes.
327 00:33:18.340 ⇒ 00:33:27.399 Amber Lin: Maybe the clients a little bit urgent. Let’s plan out how we’re gonna do or what process we’re gonna do 1st internally, and then I think Shauna can work on it later.
328 00:33:27.400 ⇒ 00:33:37.359 Uttam: Okay. Yeah. I mean again, if there’s if there’s if there’s nothing there right now on linear for her, that’s fine, like it’ll it’ll be up to me and you, Amber, to plan that out
329 00:33:38.110 ⇒ 00:33:42.320 Uttam: Yeah, I mean, let’s my 100% focus right now is just getting this ABC thing done.
330 00:33:42.674 ⇒ 00:33:47.620 Uttam: I have a feeling that if we don’t do this this week. We’re gonna really start to lose a lot of time.
331 00:33:48.212 ⇒ 00:33:58.179 Uttam: We’re already a week past our deadline. So I really want to have this. I don’t. I don’t see there being any reason why we can’t get this fully out
332 00:33:58.330 ⇒ 00:34:02.154 Uttam: like even this week. If we pushed
333 00:34:02.560 ⇒ 00:34:02.950 Amber Lin: Okay.
334 00:34:02.950 ⇒ 00:34:10.110 Uttam: So I just wanna make sure that all of our resources are pointing towards getting this done. And we have a big meeting on Friday, where we’re like, we’re ready to go
335 00:34:10.859 ⇒ 00:34:11.309 Amber Lin: Okay.
336 00:34:11.739 ⇒ 00:34:22.639 Amber Lin: totally. So I think, Jonna, this for this week, you can also help with the ABC client, and we’ll see next week how we go on the internal tasks
337 00:34:23.219 ⇒ 00:34:30.739 Janna Wong: Oh, currently, I’m actually trying one more thing for Robert, if this would work. So yeah, I’ll update you also
338 00:34:31.429 ⇒ 00:34:39.649 Uttam: And if there’s anything that doesn’t work, I guess Miguel maybe check in on and check in on how Jana is doing the mixed channel scraping, and if there’s anything else we can try
339 00:34:40.362 ⇒ 00:34:44.280 Amber Lin: Think the other. It’s the other working
340 00:34:44.440 ⇒ 00:34:45.080 Janna Wong: Yeah.
341 00:34:48.239 ⇒ 00:34:50.789 Janna Wong: yeah, sure.
342 00:34:53.479 ⇒ 00:35:02.969 Amber Lin: Okay, today is very eval focused. And we’ll see how much progress we have tomorrow and tomorrow we will have different different set of tasks and pick up some other things
343 00:35:03.400 ⇒ 00:35:04.030 Uttam: Okay.
344 00:35:04.280 ⇒ 00:35:05.549 Miguel de Veyra: Okay. Thanks. Everyone.
345 00:35:05.860 ⇒ 00:35:06.340 Amber Lin: Alright!
346 00:35:06.570 ⇒ 00:35:07.030 Uttam: Thank you.
347 00:35:07.030 ⇒ 00:35:08.100 Amber Lin: Thank you guys. And thank you.
348 00:35:09.160 ⇒ 00:35:10.809 Amber Lin: It was really helpful
349 00:35:11.170 ⇒ 00:35:13.280 Uttam: Yeah. No problem. Good luck. Good luck, amber
350 00:35:13.280 ⇒ 00:35:16.050 Amber Lin: Oh, thank you. I appreciate that.
351 00:35:17.560 ⇒ 00:35:18.180 Amber Lin: Okay.
352 00:35:18.450 ⇒ 00:35:19.410 Amber Lin: Bye, everyone