Meeting Title: AI Team Retro/Weekly Close Date: 2025-02-14 Meeting participants: Janna Wong, Uttam Kumaran, Miguel De Veyra, Casie Aviles


WEBVTT

1 00:07:03.570 00:07:05.240 Miguel de Veyra: Hey? Guys, good evening.

2 00:07:07.540 00:07:08.150 Casie Aviles: Hey!

3 00:07:12.150 00:07:14.490 Miguel de Veyra: Yeah, I don’t know why I’m so sleepy.

4 00:07:19.600 00:07:22.808 Miguel de Veyra: Yeah. So how’s Valentine’s? Casey?

5 00:07:23.710 00:07:24.745 Casie Aviles: Without

6 00:07:36.960 00:07:37.949 Casie Aviles: gas, to see.

7 00:07:39.376 00:07:43.699 Miguel de Veyra: True. True, I can attest to that shit.

8 00:07:45.115 00:07:47.394 Casie Aviles: No, no, I’m sure. No.

9 00:07:58.790 00:08:01.379 Casie Aviles: Oh, nice onion! Yung!

10 00:08:03.130 00:08:05.079 Casie Aviles: What? What was it? Again?

11 00:08:10.128 00:08:11.399 Casie Aviles: Yes, yes, your Api.

12 00:08:11.410 00:08:16.420 Miguel de Veyra: Oh, go on

13 00:08:25.780 00:08:31.299 Miguel de Veyra: initial. So.

14 00:08:32.919 00:08:38.809 Miguel de Veyra: I.

15 00:08:39.559 00:08:40.419 Casie Aviles: Definitely.

16 00:08:40.429 00:08:41.879 Miguel de Veyra: Definitely there will.

17 00:08:43.240 00:08:45.060 Casie Aviles: So from low tech, one of Manila.

18 00:08:45.300 00:08:54.609 Miguel de Veyra: Oh, no, sir, eventually I think I’m getting service. Not indeed. Only you automate partner calls internally.

19 00:08:55.480 00:08:56.190 Casie Aviles: Hmm.

20 00:08:57.720 00:09:00.770 Miguel de Veyra: Making calls. So yeah, definitely

21 00:09:06.770 00:09:08.270 Miguel de Veyra: about it.

22 00:09:09.525 00:09:16.989 Miguel de Veyra: I’m in deep mood now before time again for diamond client meeting.

23 00:09:22.190 00:09:23.610 Casie Aviles: What time! A little.

24 00:09:24.349 00:09:24.909 Miguel de Veyra: 13.

25 00:09:24.910 00:09:26.449 Casie Aviles: 30, their time.

26 00:09:26.930 00:09:29.130 Miguel de Veyra: Or a time in client meeting.

27 00:10:14.670 00:10:15.356 Uttam Kumaran: Hi! Everyone.

28 00:10:15.960 00:10:16.500 Miguel de Veyra: Hey?

29 00:10:23.830 00:10:25.429 Miguel de Veyra: Your voice is cutting off.

30 00:10:26.020 00:10:33.889 Uttam Kumaran: Well, I just got an email. I just got an email out to ABC, about sort of what we want to go through later.

31 00:10:34.310 00:10:34.890 Miguel de Veyra: Yeah.

32 00:10:42.620 00:10:43.610 Miguel de Veyra: Oh, yes, sir.

33 00:11:11.740 00:11:13.339 Miguel de Veyra: okay. I just saw it now.

34 00:11:14.880 00:11:15.500 Uttam Kumaran: Okay.

35 00:11:49.470 00:11:49.835 Miguel de Veyra: Yeah,

36 00:11:50.640 00:11:57.369 Miguel de Veyra: So weekly closing, I think we’re pretty much prepared for the meeting later with ABC.

37 00:11:57.750 00:11:58.440 Uttam Kumaran: Okay.

38 00:12:00.040 00:12:06.559 Miguel de Veyra: Jana. Sorry can you share your screen? Especially about the evils? Because that part I’m not sure.

39 00:12:07.379 00:12:15.880 Miguel de Veyra: Like I I wasn’t able to, you know, look into that deeply. But put them what I did basically develop version of their

40 00:12:16.180 00:12:27.020 Miguel de Veyra: of the link that they have right now. They don’t have to change it, because, you know, they’re probably gonna mix it up. So what I do is if they go to the link they have, they’ll just direct to the new, to the newer one.

41 00:12:27.020 00:12:27.680 Uttam Kumaran: Okay.

42 00:12:28.210 00:12:30.649 Miguel de Veyra: So yeah, so there’s no problem.

43 00:12:31.320 00:12:31.690 Uttam Kumaran: Okay.

44 00:12:31.690 00:12:37.080 Miguel de Veyra: And it’s gonna all be connected to Snowflake, as we’ve seen yesterday. And then Jana worked on.

45 00:12:37.200 00:12:38.919 Miguel de Veyra: adding, what do you call it?

46 00:12:39.110 00:12:46.340 Miguel de Veyra: The testing stuff for Evals via valium. So she sent the video.

47 00:12:47.560 00:12:48.409 Miguel de Veyra: So yeah.

48 00:12:49.160 00:12:57.800 Janna Wong: For some reason it didn’t save. I think so. I can.

49 00:12:57.800 00:13:00.809 Uttam Kumaran: I haven’t watched the loom yet, but I could watch that too.

50 00:13:02.820 00:13:06.929 Miguel de Veyra: Yeah, cause we ideally, we wanna show part of it

51 00:13:07.080 00:13:13.819 Miguel de Veyra: if time permits, as we’ve mentioned, like later in the call. So if we could put something up really quickly.

52 00:13:14.910 00:13:17.109 Miguel de Veyra: just so we could show something later.

53 00:13:17.110 00:13:22.262 Janna Wong: Oh, yeah, it got it. Now. I can share my screen if you want.

54 00:13:22.630 00:13:23.235 Uttam Kumaran: Okay.

55 00:13:23.840 00:13:27.849 Janna Wong: Here’s what it looks like. Can you see my screen.

56 00:13:28.186 00:13:28.860 Miguel de Veyra: It’s loading.

57 00:13:30.095 00:13:52.164 Janna Wong: So can you. This is the evaluations that I did. So basically, this is like a test on different scenarios that we have. So basically, I got the test inputs from the n 8 n, from the old

58 00:13:52.890 00:14:15.540 Janna Wong: triggers. And then I added some for here. So what I did was that I just ran them and then looked at the output. So I found that they have like good output so far, and then, if it doesn’t know any answer. It will also say that it doesn’t have any access to this data and stuff. So if you can see here. But oh.

59 00:14:15.910 00:14:19.209 Janna Wong: oh, I’m sorry phone up here.

60 00:14:21.340 00:14:30.251 Janna Wong: You can see here I’m sorry you don’t have access to that specific data. So if it doesn’t know sorry, it’s kind of lagging on my end.

61 00:14:31.600 00:14:46.880 Janna Wong: okay, like that. Okay. So if it doesn’t know, it will answer, and then it will also like, relate to the human, if ever. So I’m sorry to hear about your pets. And then, yeah.

62 00:14:49.410 00:14:51.950 Uttam Kumaran: And then what? What sort of scores do we get.

63 00:14:52.885 00:15:03.909 Janna Wong: Not like a scoring. It’s like a test on different inputs. And up then this is the output whenever we run.

64 00:15:04.200 00:15:09.740 Janna Wong: So I’m not sure, if we can add like metrics here.

65 00:15:22.030 00:15:24.649 Miguel de Veyra: There’s some heavy background noise in you, Jenna.

66 00:15:25.030 00:15:26.310 Janna Wong: I’m sorry.

67 00:15:31.430 00:15:35.619 Miguel de Veyra: I think, for the scoring metric with them. That’s what we discussed yesterday. Now

68 00:15:36.060 00:15:38.549 Miguel de Veyra: for the quality stuff in Snowflake.

69 00:15:39.330 00:15:41.044 Uttam Kumaran: Yeah, take a look at that.

70 00:15:42.050 00:15:47.040 Uttam Kumaran: take a look at the link I sent in the email about support.

71 00:15:48.600 00:15:49.820 Uttam Kumaran: Thank you.

72 00:15:50.812 00:15:55.425 Uttam Kumaran: Take a look at the scoring that I sent in the email.

73 00:15:58.300 00:16:01.300 Uttam Kumaran: it’ll make a lot of sense from the ground.

74 00:16:03.870 00:16:05.099 Uttam Kumaran: This, and stuff.

75 00:16:09.990 00:16:16.159 Miguel de Veyra: Okay, sure, I’ll is it on the same thread with them, or is it a different one?

76 00:16:20.602 00:16:23.660 Uttam Kumaran: A new this is a new email.

77 00:16:39.560 00:16:41.069 Miguel de Veyra: Oh, there you go, good parameter.

78 00:16:46.590 00:16:54.359 Uttam Kumaran: Yes, if email is getting hard to manage, then we should think of something else, because they’re they. They only communicate on email.

79 00:16:56.080 00:16:59.719 Miguel de Veyra: Maybe we can propose we’ll create a slack channel for them with them.

80 00:17:00.140 00:17:01.220 Uttam Kumaran: They?

81 00:17:01.640 00:17:04.850 Uttam Kumaran: It’s gonna be hard because they have already have this internal it.

82 00:17:08.099 00:17:13.160 Uttam Kumaran: I mean, it’s probably it’s probably most hard that you’re just not checking email, probably right?

83 00:17:13.160 00:17:14.560 Miguel de Veyra: Yeah, yeah, okay.

84 00:17:14.560 00:17:17.750 Uttam Kumaran: Should we? Just maybe I could have it like, send to the slack channel

85 00:17:20.280 00:17:23.859 Uttam Kumaran: like, I’ll just. I’ll just add the slack channel

86 00:17:25.060 00:17:28.140 Uttam Kumaran: email to all my emails with them.

87 00:17:30.280 00:17:32.199 Miguel de Veyra: Oh, yeah, I was actually like

88 00:17:32.480 00:17:36.950 Miguel de Veyra: planning to do something like that where every code, either email just make a Zap.

89 00:17:38.070 00:17:40.555 Uttam Kumaran: Yeah, I mean, even easier. I’ll just

90 00:17:41.250 00:17:44.689 Uttam Kumaran: like I can generate a I can generate an email for

91 00:17:46.330 00:17:53.330 Uttam Kumaran: I can generate an email for the slack channel, and then I’ll just CC, that on everything other. Another thing we could do is

92 00:17:54.870 00:17:58.759 Uttam Kumaran: we can create an abc@brainforge.ai.

93 00:17:59.170 00:17:59.740 Miguel de Veyra: Yep.

94 00:17:59.740 00:18:00.630 Uttam Kumaran: Group.

95 00:18:01.140 00:18:05.660 Uttam Kumaran: And I can just CC that that way. It’s not like, there’s not like a weird, slack email.

96 00:18:06.080 00:18:08.990 Uttam Kumaran: And then I can.

97 00:18:09.880 00:18:13.470 Uttam Kumaran: how about I do that? And then I’ll just have it forward to the ABC. Client channel.

98 00:18:13.650 00:18:14.330 Miguel de Veyra: Okay.

99 00:18:15.110 00:18:20.030 Uttam Kumaran: Okay. But then you gotta look at that, because then I don’t. I don’t want there to be any excuses.

100 00:18:20.392 00:18:21.479 Miguel de Veyra: I don’t care.

101 00:18:21.785 00:18:32.170 Uttam Kumaran: Cause. I know you don’t like email. And that’s fine. I don’t like email, either. It’s kind of unfortunate. They don’t even have teams or anything. But yeah, okay, I’ll send it there.

102 00:18:32.712 00:18:33.727 Miguel de Veyra: Yeah, I saw it now.

103 00:18:36.460 00:18:40.110 Uttam Kumaran: I think the only other task is, you know how they have evolve and dream.

104 00:18:40.450 00:18:42.700 Miguel de Veyra: Yeah. Did they get back? No, no, they have.

105 00:18:42.700 00:18:44.140 Uttam Kumaran: Yes, we got

106 00:18:44.560 00:18:48.600 Uttam Kumaran: we got. We got some access. I haven’t set it up yet. Do you want to give it a shot?

107 00:18:48.790 00:18:50.930 Miguel de Veyra: Oh, yeah, sure. Wait.

108 00:18:51.260 00:18:56.020 Miguel de Veyra: Did they send it to you directly? Because I’ve been checking some of the emails? I haven’t seen any login today.

109 00:18:56.020 00:18:58.909 Uttam Kumaran: Yeah, let me let me send it to you.

110 00:19:16.200 00:19:17.922 Uttam Kumaran: Okay, I sent it. Yeah. Just

111 00:19:19.130 00:19:22.670 Uttam Kumaran: just make sure that. You add anything.

112 00:19:23.030 00:19:24.470 Uttam Kumaran: So one password.

113 00:19:24.700 00:19:25.210 Miguel de Veyra: Okay.

114 00:19:27.430 00:19:32.680 Uttam Kumaran: Yeah, I believe I already, I added the credentials already to

115 00:19:35.040 00:19:39.320 Uttam Kumaran: one password in the ABC. Home vault.

116 00:19:42.048 00:19:46.179 Uttam Kumaran: Cause I just forward you. You’ll see the link from Tim. He basically sent this like

117 00:19:46.670 00:19:53.909 Uttam Kumaran: this link that you can copy paste. I took copy paste that and already put it in our one password. I just haven’t logged into the

118 00:19:55.450 00:19:56.550 Uttam Kumaran: accounts yet.

119 00:19:56.980 00:19:58.580 Miguel de Veyra: Okay. Okay. I’ll.

120 00:19:58.580 00:20:03.819 Uttam Kumaran: So if you can log into those accounts, and then just make sure that you can just add them to

121 00:20:04.130 00:20:10.159 Uttam Kumaran: add them both to one password. And then, yeah, we should be good.

122 00:20:16.300 00:20:17.160 Uttam Kumaran: Okay?

123 00:20:17.550 00:20:20.900 Uttam Kumaran: And then, yeah, I think we’re gonna work on the Eval data set. But today.

124 00:20:21.410 00:20:23.839 Miguel de Veyra: Board here for all 8,

125 00:20:30.150 00:20:37.220 Miguel de Veyra: and then, I guess, for Casey, we’re still working on what do you call it?

126 00:20:39.880 00:20:41.069 Miguel de Veyra: Initiative Number One, now.

127 00:20:42.820 00:20:46.719 Casie Aviles: Yeah, it’s so it’s set up for Javi. So the next

128 00:20:47.040 00:20:51.570 Casie Aviles: thing is for the other client channels. I, yeah.

129 00:20:54.050 00:20:59.739 Uttam Kumaran: Yeah, I’m gonna talk. I’ll fall through the other ones today. And then.

130 00:20:59.850 00:21:07.750 Uttam Kumaran: yeah, I’ll show you how we can develop the actual like the dashboard for that as well.

131 00:21:09.630 00:21:10.350 Casie Aviles: Okay.

132 00:21:10.900 00:21:11.889 Uttam Kumaran: Please hold.

133 00:21:18.320 00:21:28.630 Uttam Kumaran: Yeah. And in case I was gonna for the loom thing, yeah, I’m gonna just try to upload a bunch of stuff there and delete them. But yeah, it’d be great if we can start uploading it all to loom as well.

134 00:21:29.140 00:21:34.120 Uttam Kumaran: Zoom is really shareable versus Google drive. But then I want to make sure we have a copy as well.

135 00:21:35.750 00:21:36.523 Uttam Kumaran: Okay, sure.

136 00:21:45.460 00:21:52.129 Casie Aviles: Yeah, I’ll look into the also deleting once we have and also exporting to zoom.

137 00:21:54.240 00:21:54.950 Uttam Kumaran: Okay.

138 00:21:54.950 00:21:56.950 Casie Aviles: Like the automation. Yeah.

139 00:22:02.020 00:22:07.460 Casie Aviles: Hmm. Other than that, I guess, for the ticket there.

140 00:22:08.070 00:22:12.560 Casie Aviles: starting to add, I mean, do some gradual changes. But.

141 00:22:13.180 00:22:15.729 Uttam Kumaran: Yeah. One more thing like, how, how? Where, like.

142 00:22:15.850 00:22:24.220 Uttam Kumaran: I know, we talked about the logs for each of the agent like, where are we gonna start to get that data from. It’s just like how often the agent is called.

143 00:22:27.733 00:22:30.279 Casie Aviles: Sorry were you saying like, how do we get

144 00:22:30.900 00:22:33.050 Casie Aviles: like? How frequently the agents are called.

145 00:22:33.710 00:22:34.370 Uttam Kumaran: Yes.

146 00:22:36.810 00:22:38.410 Uttam Kumaran: Exit here for gates.

147 00:22:38.410 00:22:38.810 Casie Aviles: Don’t worry.

148 00:22:38.810 00:22:40.809 Uttam Kumaran: Yeah, how? Frequently they’re called, yeah.

149 00:22:42.892 00:22:44.520 Casie Aviles: I haven’t released.

150 00:22:44.840 00:22:45.290 Uttam Kumaran: I’m just.

151 00:22:45.290 00:22:47.859 Casie Aviles: Started tracking them yet. I mean I haven’t.

152 00:22:48.830 00:22:52.811 Casie Aviles: I? I guess, like my only confusion. There is like

153 00:22:53.820 00:23:00.340 Casie Aviles: do I? Just if I if I could start like maybe I guess I could start with just, you know, sending it to

154 00:23:00.820 00:23:06.999 Casie Aviles: sheets like tracking, or or or or do we also want to have it on Snowflake.

155 00:23:08.420 00:23:10.820 Uttam Kumaran: Yeah, I would ideally like to be on Snowflake.

156 00:23:11.860 00:23:15.839 Casie Aviles: Okay. Yeah, yeah. That part is not yet. I haven’t set up that part yet.

157 00:23:16.240 00:23:21.760 Uttam Kumaran: So you’re gonna you’re just gonna create like an n 8 n node that trigger something.

158 00:23:23.133 00:23:23.920 Casie Aviles: Yes! Yes!

159 00:23:26.050 00:23:29.400 Uttam Kumaran: Will we also get like the question and and reply.

160 00:23:30.570 00:23:31.870 Casie Aviles: Yeah, yeah, we could do that.

161 00:23:32.340 00:23:38.819 Uttam Kumaran: Okay, that’s sick, cause that saves us from having to get the N. 8 and enterprise.

162 00:23:39.110 00:23:40.900 Uttam Kumaran: which is 8 50 a month.

163 00:23:41.250 00:23:43.349 Uttam Kumaran: I really don’t want to get it.

164 00:23:44.393 00:23:46.079 Uttam Kumaran: Okay, okay?

165 00:23:46.990 00:23:53.150 Uttam Kumaran: So yeah, like, ideally, we can, let’s just start by like

166 00:23:53.280 00:23:57.490 Uttam Kumaran: just writing it to a table if we can get the timestamp

167 00:23:58.330 00:24:01.079 Uttam Kumaran: if we get the timestamp, and

168 00:24:02.020 00:24:07.149 Uttam Kumaran: I mean I don’t know if we can get who sent it. Maybe we need to like merge the slack

169 00:24:07.630 00:24:11.880 Uttam Kumaran: bought with this eventually but ideally.

170 00:24:12.440 00:24:17.650 Uttam Kumaran: yeah, I just want to know, like what messages are sent, what’s and what’s the reply which agent was called.

171 00:24:17.960 00:24:23.730 Uttam Kumaran: and at what time that’ll give us like the core stuff. I think. Also, we want to look at.

172 00:24:24.177 00:24:30.629 Uttam Kumaran: We wanna look at response times. Eventually, we wanna we wanna look at like which tools are to use, etc.

173 00:24:30.750 00:24:32.740 Uttam Kumaran: But just in the short term.

174 00:24:33.120 00:24:37.076 Uttam Kumaran: just want to know the usage by agent

175 00:24:38.440 00:24:42.770 Uttam Kumaran: and if you can get the time response time for the entire workflow run.

176 00:24:43.170 00:24:48.209 Uttam Kumaran: That would be, that’ll be what do they say? Icing on the cake.

177 00:24:49.960 00:24:53.740 Casie Aviles: Yeah, I know. I know it’s possible on any 10. So.

178 00:24:54.016 00:25:02.020 Uttam Kumaran: Okay, okay, I mean, honestly, see what if you see what you can get, if if they give it all to you, shove all of it somewhere. That’s fine.

179 00:25:02.370 00:25:06.750 Casie Aviles: Okay, yeah, I’ll start picking tables for each agent. Then.

180 00:25:07.260 00:25:08.213 Uttam Kumaran: Okay, okay.

181 00:25:13.940 00:25:14.860 Uttam Kumaran: okay.

182 00:25:15.220 00:25:16.030 Uttam Kumaran: Great.

183 00:25:18.220 00:25:20.939 Uttam Kumaran: Anything else we want to discuss, Miguel.

184 00:25:22.260 00:25:29.860 Uttam Kumaran: I may have. I don’t. We don’t have like a architecture diagram necessarily yet for ABC. But I do want to kind of

185 00:25:29.990 00:25:33.110 Uttam Kumaran: see if you want to talk a little bit in that meeting.

186 00:25:33.210 00:25:35.040 Uttam Kumaran: What do you think.

187 00:25:35.980 00:25:38.139 Miguel de Veyra: In regards to architecture.

188 00:25:39.380 00:25:42.139 Uttam Kumaran: Yeah, or just like, I don’t know. I kind of want you to

189 00:25:42.320 00:25:51.429 Uttam Kumaran: also have a relationship with them and sort of talk through. What you noticed about the data. I mean. Honestly, I think it would be great for you to go through the

190 00:25:52.090 00:25:54.120 Uttam Kumaran: the Bible set up with them.

191 00:25:55.270 00:25:55.929 Miguel de Veyra: Okay. Yeah.

192 00:25:55.930 00:25:59.500 Uttam Kumaran: Maybe. Do you want to do that? Yeah, okay, I can take the yeah.

193 00:25:59.790 00:26:02.290 Uttam Kumaran: yeah, I can take the Prd and then

194 00:26:03.110 00:26:09.049 Uttam Kumaran: if if that when that comes up, if you can talk through that, that’d be super super great.

195 00:26:09.290 00:26:10.590 Miguel de Veyra: Okay. Okay. Sure. Sure.

196 00:26:11.180 00:26:13.250 Uttam Kumaran: Okay, I I may.

197 00:26:13.830 00:26:18.339 Uttam Kumaran: I’m gonna be on my way back from the airport. So if I can’t

198 00:26:19.250 00:26:22.330 Uttam Kumaran: share screen, I may ask you to

199 00:26:22.700 00:26:28.680 Uttam Kumaran: to to share and run it, but I should be on my I should be on zoom on Mobile.

200 00:26:28.680 00:26:30.140 Miguel de Veyra: No. Okay. Okay. Sure.

201 00:26:30.483 00:26:35.630 Uttam Kumaran: Okay? Awesome. Yeah. Otherwise I don’t know. What. How do we think? The week went.

202 00:26:37.960 00:26:52.430 Miguel de Veyra: I think this week was, yeah, we finally decided on what to use. So that’s very good. And then overall, I think everyone was pretty productive, and I don’t know. Everyone was setting into their roles, and you know.

203 00:26:54.418 00:26:55.882 Uttam Kumaran: Yeah, I think.

204 00:26:56.810 00:27:03.600 Uttam Kumaran: I think we did a good job. You guys meeting with Nico like, I spent some time with Nico. You mentioned that you guys are getting close with him. I think

205 00:27:04.160 00:27:11.459 Uttam Kumaran: ideally, we start to do that with everybody else in the company, to even someone from engineers, someone from operations, someone from

206 00:27:11.670 00:27:13.859 Uttam Kumaran: each of the core departments.

207 00:27:15.218 00:27:22.670 Uttam Kumaran: You know. And then but start with Nico. And yeah, I think, keep keep having cycles like that he’s really, really motivated. So.

208 00:27:23.060 00:27:24.200 Miguel de Veyra: Okay. Sure.

209 00:27:27.020 00:27:27.640 Casie Aviles: Yeah.

210 00:27:30.820 00:27:33.869 Uttam Kumaran: Okay, great, that’s all I had otherwise.

211 00:27:34.370 00:27:37.820 Miguel de Veyra: Okay, okay, thanks, guys. I’ll see you later.

212 00:27:37.890 00:27:38.770 Uttam Kumaran: Awesome.

213 00:27:38.770 00:27:39.299 Casie Aviles: Thank you.

214 00:27:40.060 00:27:40.900 Janna Wong: Thank you.