2026-03-27_ai_service_office_hours_block

Meeting Title: AI Service Office Hours Block Date: 2026-03-27 Meeting participants: Mustafa Raja, Samuel Roberts, Casie Aviles, Pranav

WEBVTT

1 00:00:31.910 ⇒ 00:00:32.590 Samuel Roberts: Hey.

2 00:00:33.900 ⇒ 00:00:35.140 Mustafa Raja: Hey, how are you?

3 00:00:36.580 ⇒ 00:00:38.440 Samuel Roberts: Good, good, okay, good.

4 00:00:40.960 ⇒ 00:00:44.990 Samuel Roberts: Okay, cool. I guess I wasn’t sure how long that would take, and yeah, I think we’ll get that.

5 00:00:44.990 ⇒ 00:00:46.140 Mustafa Raja: Yeah…

6 00:00:46.500 ⇒ 00:00:52.319 Mustafa Raja: I mean, if there aren’t any changes, it isn’t going to take any time, you know? But if there are…

7 00:00:52.800 ⇒ 00:00:53.440 Mustafa Raja: Excuse me.

8 00:00:53.660 ⇒ 00:00:55.880 Mustafa Raja: Then it takes a lot of time.

9 00:00:56.640 ⇒ 00:00:57.340 Samuel Roberts: Yeah.

10 00:00:58.700 ⇒ 00:00:59.510 Samuel Roberts: That’s…

11 00:01:00.970 ⇒ 00:01:02.849 Mustafa Raja: Yeah, there’s one issue, though.

12 00:01:03.460 ⇒ 00:01:04.140 Samuel Roberts: Sure.

13 00:01:04.560 ⇒ 00:01:06.469 Mustafa Raja: Let me hear mine.

14 00:01:07.390 ⇒ 00:01:10.719 Mustafa Raja: Screen… Give me a moment.

15 00:01:11.410 ⇒ 00:01:13.239 Mustafa Raja: I’m sure.

16 00:01:16.330 ⇒ 00:01:24.110 Mustafa Raja: Okay, UniversalDocs… Yeah, the ones that are this… so this is Word, Microsoft Word, right? These aren’t Word.

17 00:01:24.110 ⇒ 00:01:24.790 Samuel Roberts: Yes.

18 00:01:25.740 ⇒ 00:01:27.639 Samuel Roberts: Of course, okay. Is there…

19 00:01:27.800 ⇒ 00:01:28.180 Mustafa Raja: We’ll.

20 00:01:28.180 ⇒ 00:01:33.720 Samuel Roberts: I know… yeah, okay, I was gonna say, I know there’s a way to convert them, but I don’t know how that messes up formatting and stuff.

21 00:01:34.130 ⇒ 00:01:36.120 Samuel Roberts: And we probably…

22 00:01:36.960 ⇒ 00:01:41.930 Mustafa Raja: The other issue… I think we’ve already discussed that one, though.

23 00:01:43.170 ⇒ 00:01:46.940 Mustafa Raja: is that some of these documents, I just don’t have access.

24 00:01:48.170 ⇒ 00:01:50.700 Samuel Roberts: Oh, yeah, I saw there were a few in there that…

25 00:01:53.080 ⇒ 00:01:55.170 Mustafa Raja: Oh, this is mine.

26 00:01:56.650 ⇒ 00:02:01.070 Mustafa Raja: this… It’s still mine… yeah, this one. So, this…

27 00:02:02.230 ⇒ 00:02:05.010 Mustafa Raja: Yeah, quite a few ones. And then there’s this MP4.

28 00:02:05.140 ⇒ 00:02:07.670 Mustafa Raja: We just cannot add this.

29 00:02:07.670 ⇒ 00:02:11.720 Samuel Roberts: Yeah, I don’t even… okay, that’s… we’ll have to make sure that Printoff services that to them.

30 00:02:12.000 ⇒ 00:02:12.630 Mustafa Raja: Yeah.

31 00:02:12.630 ⇒ 00:02:13.290 Samuel Roberts: Okay.

32 00:02:13.990 ⇒ 00:02:14.940 Samuel Roberts: So, don’t…

33 00:02:14.940 ⇒ 00:02:18.169 Mustafa Raja: We need to surface, we need to convert this.

34 00:02:18.930 ⇒ 00:02:22.979 Mustafa Raja: Docs, and then we need access on additional modes.

35 00:02:24.150 ⇒ 00:02:27.560 Samuel Roberts: Okay, let me see if Pranav’s hopping on…

36 00:02:49.490 ⇒ 00:02:52.710 Samuel Roberts: Yeah, I think we’re probably okay to convert…

37 00:02:53.470 ⇒ 00:02:56.589 Samuel Roberts: I just don’t want to mess them up. Oops, my…

38 00:02:57.230 ⇒ 00:02:59.119 Mustafa Raja: Yeah, that was… yeah.

39 00:02:59.730 ⇒ 00:03:00.480 Mustafa Raja: Because I…

40 00:03:00.480 ⇒ 00:03:01.220 Samuel Roberts: Video.

41 00:03:01.220 ⇒ 00:03:09.340 Mustafa Raja: I see some logs coming in also, so I know that they might just be… You sing.

42 00:03:10.420 ⇒ 00:03:12.130 Samuel Roberts: Logs coming in for…

43 00:03:12.320 ⇒ 00:03:13.519 Mustafa Raja: For the Cloud Run, you know?

44 00:03:14.230 ⇒ 00:03:17.850 Samuel Roberts: Oh, okay, I didn’t know if you meant for the, those, those, documents.

45 00:03:18.120 ⇒ 00:03:20.409 Mustafa Raja: No, no, no, for the Cloud Run.

46 00:03:20.410 ⇒ 00:03:20.880 Samuel Roberts: That makes more sense.

47 00:03:20.880 ⇒ 00:03:21.940 Mustafa Raja: Using that.

48 00:03:23.050 ⇒ 00:03:23.680 Samuel Roberts: Okay.

49 00:03:24.650 ⇒ 00:03:26.790 Samuel Roberts: Yeah. Hyper enough.

50 00:03:28.380 ⇒ 00:03:29.120 Pranav: Hey, guys.

51 00:03:29.930 ⇒ 00:03:37.320 Samuel Roberts: So, Davao was just saying, the embeddings we ran, the…

52 00:03:37.520 ⇒ 00:03:42.010 Samuel Roberts: Couple issues there. There’s a few that are DocX, like, Word documents.

53 00:03:42.500 ⇒ 00:03:43.350 Pranav: Oh, okay.

54 00:03:44.020 ⇒ 00:03:53.649 Samuel Roberts: And those will have to… we’ll have to convert those to Google Docs, which I imagine is probably fine, I just don’t want to mess up their folder, so I don’t know.

55 00:03:53.760 ⇒ 00:03:54.390 Samuel Roberts: I guess we just.

56 00:03:54.390 ⇒ 00:03:57.110 Pranav: I think we’re copying these over anyways, right?

57 00:03:57.350 ⇒ 00:04:01.069 Mustafa Raja: Yeah, this one, we have that in,

58 00:04:01.240 ⇒ 00:04:06.250 Mustafa Raja: Yeah, we have copies in our own, drive also.

59 00:04:06.250 ⇒ 00:04:11.259 Samuel Roberts: Okay, how are we handling when they make… are they going to make changes to those ones, then? Like, if anything changes?

60 00:04:11.260 ⇒ 00:04:15.250 Mustafa Raja: I believe we want them to… those are our ones. Okay, good.

61 00:04:15.250 ⇒ 00:04:21.019 Samuel Roberts: I just want to make sure that that’s… that they know that. Okay, so that’s fine for the conversion, then. There’s also a video in there.

62 00:04:22.660 ⇒ 00:04:23.420 Pranav: Oh.

63 00:04:24.200 ⇒ 00:04:30.470 Samuel Roberts: So I don’t know what… how important that is for them, but definitely, like, make sure they know that that’s not getting included right now.

64 00:04:30.970 ⇒ 00:04:33.259 Pranav: Yep, yep, that’s totally fine.

65 00:04:33.630 ⇒ 00:04:36.649 Samuel Roberts: And then… What was the other thing, Mustafa?

66 00:04:37.330 ⇒ 00:04:40.570 Mustafa Raja: Yeah, some… for some videos, we just don’t have access.

67 00:04:40.570 ⇒ 00:04:43.570 Samuel Roberts: Oh, the access, yeah, there’s a few that, so I would say just…

68 00:04:44.050 ⇒ 00:04:46.890 Samuel Roberts: Probably just request the access.

69 00:04:47.280 ⇒ 00:04:52.390 Pranav: Yeah, I think a few of those, purposefully, they didn’t give us access.

70 00:04:52.390 ⇒ 00:04:54.520 Samuel Roberts: Oh, if that’s intentional, then that’s fine. Okay.

71 00:04:54.520 ⇒ 00:04:55.230 Pranav: Yeah.

72 00:04:56.900 ⇒ 00:04:57.820 Pranav: I believe so.

73 00:04:57.820 ⇒ 00:05:00.740 Samuel Roberts: Even better. Okay, then that’s… that solves that problem.

74 00:05:01.150 ⇒ 00:05:01.840 Pranav: Yep.

75 00:05:03.830 ⇒ 00:05:07.070 Samuel Roberts: Okay, so, it looks like we still need to then…

76 00:05:07.630 ⇒ 00:05:10.849 Samuel Roberts: convert and embed the DocX ones, or…

77 00:05:14.250 ⇒ 00:05:15.550 Samuel Roberts: Because we have to copy them.

78 00:05:18.820 ⇒ 00:05:22.070 Mustafa Raja: I can just con… Let me see if I can…

79 00:05:22.390 ⇒ 00:05:25.999 Samuel Roberts: There should be a way, like, I thought when you open it, it, like, does a…

80 00:05:26.000 ⇒ 00:05:32.620 Mustafa Raja: It opens in the Google Docs, but it’s still a DocX file, you know.

81 00:05:33.150 ⇒ 00:05:34.130 Samuel Roberts: Oh…

82 00:05:37.460 ⇒ 00:05:42.280 Samuel Roberts: Okay, so that’s the permission one. Where is there a dot? There it is, the disposition code?

83 00:05:42.280 ⇒ 00:05:44.959 Mustafa Raja: I think, use this one.

84 00:05:45.230 ⇒ 00:05:52.440 Mustafa Raja: This is the one that… the… sorry, I’m going to share a link in Zoom chat. Okay. These are the ones that are getting embedded.

85 00:05:52.760 ⇒ 00:05:53.889 Samuel Roberts: Oh, of course, yeah, I’m probably in the wrong.

86 00:05:53.890 ⇒ 00:05:58.300 Mustafa Raja: Because we copied the ones that we have access to in our…

87 00:05:58.640 ⇒ 00:06:00.349 Samuel Roberts: So it’s just those two…

88 00:06:00.870 ⇒ 00:06:02.520 Mustafa Raja: Yeah, yeah, these two.

89 00:06:03.800 ⇒ 00:06:05.100 Samuel Roberts: Okay…

90 00:06:05.560 ⇒ 00:06:08.639 Mustafa Raja: I’m seeing how… let me search it, how can I…

91 00:06:08.640 ⇒ 00:06:11.929 Samuel Roberts: Yeah, like, whenever I’ve done a DocX in Google.

92 00:06:13.290 ⇒ 00:06:18.160 Samuel Roberts: There’s usually a way to just say, like, convert, but I see it still has tagged with DocX.

93 00:06:18.490 ⇒ 00:06:19.850 Mustafa Raja: Yeah, yeah.

94 00:06:20.400 ⇒ 00:06:29.479 Samuel Roberts: Yeah, I… I honestly haven’t uploaded a Word document to Google Drive in literally years, so…

95 00:06:30.070 ⇒ 00:06:31.910 Samuel Roberts: I don’t know how that’s changed.

96 00:06:32.670 ⇒ 00:06:33.970 Mustafa Raja: Hmm…

97 00:06:39.920 ⇒ 00:06:40.810 Samuel Roberts: I mean, I suppose the.

98 00:06:40.810 ⇒ 00:06:42.340 Mustafa Raja: Yeah.

99 00:06:42.340 ⇒ 00:06:42.870 Samuel Roberts: what?

100 00:06:42.870 ⇒ 00:06:44.399 Mustafa Raja: Oh, same as Google Docs.

101 00:06:46.000 ⇒ 00:06:47.479 Samuel Roberts: They’re just treated the same?

102 00:06:48.270 ⇒ 00:06:50.590 Mustafa Raja: Yeah, there’s a button saved as Google Docs.

103 00:06:50.940 ⇒ 00:06:54.310 Samuel Roberts: That’s… okay, there we go, okay. Yep, there it is, I see it now, okay.

104 00:06:55.150 ⇒ 00:07:00.019 Mustafa Raja: Okay, I’ll have to update the drive, sorry, my code also then.

105 00:07:03.250 ⇒ 00:07:05.049 Mustafa Raja: Save as Google Docs…

106 00:07:10.010 ⇒ 00:07:15.560 Samuel Roberts: Yeah, I think it used to auto-convert when you upload, but they added that, like, ability to.

107 00:07:15.860 ⇒ 00:07:17.290 Samuel Roberts: Keep the formatting.

108 00:07:22.730 ⇒ 00:07:25.280 Samuel Roberts: Cool, okay, so that seems good.

109 00:07:25.280 ⇒ 00:07:29.490 Mustafa Raja: It’ll take about, 6-7 minutes.

110 00:07:29.700 ⇒ 00:07:32.069 Mustafa Raja: Because I’ll have to push code now.

111 00:07:34.320 ⇒ 00:07:37.569 Samuel Roberts: Wait, what’s changing with the code for that? Are these all unknown?

112 00:07:37.570 ⇒ 00:07:40.599 Mustafa Raja: These are IDs, in…

113 00:07:40.600 ⇒ 00:07:41.520 Samuel Roberts: Oh, okay.

114 00:07:42.010 ⇒ 00:07:43.740 Samuel Roberts: And we’re not just pulling from the, like.

115 00:07:43.870 ⇒ 00:07:46.049 Mustafa Raja: Yeah, yeah, we’re not pulling.

116 00:07:46.560 ⇒ 00:07:47.180 Samuel Roberts: Okay.

117 00:07:47.180 ⇒ 00:07:49.730 Mustafa Raja: We should be doing that, that’s the smarter way.

118 00:07:49.970 ⇒ 00:07:56.750 Samuel Roberts: It’s okay, I mean, yeah, there weren’t this many documents before, it was more specific, but… I understand.

119 00:08:03.110 ⇒ 00:08:07.319 Pranav: Okay, I’m just gonna share linear real quick, and let’s just go through…

120 00:08:07.920 ⇒ 00:08:14.659 Pranav: all just kind of, like, the pending tasks. Because, you know, a bunch kind of just populated in the last few days, and then there’s a few.

121 00:08:14.660 ⇒ 00:08:15.580 Samuel Roberts: Yeah, good call.

122 00:08:15.580 ⇒ 00:08:19.689 Pranav: So… Let’s just make sure we’re in a good spot now.

123 00:08:28.830 ⇒ 00:08:34.469 Mustafa Raja: Also, just for the team’s context, I’ll be, out of office from 1st April.

124 00:08:34.909 ⇒ 00:08:36.250 Mustafa Raja: Till 11th.

125 00:08:38.409 ⇒ 00:08:39.119 Pranav: Okay.

126 00:08:40.159 ⇒ 00:08:44.929 Pranav: That should be fine, yeah. Sam and I should be able to fill in and help Casey out.

127 00:08:45.660 ⇒ 00:08:46.600 Mustafa Raja: Thank you.

128 00:08:47.370 ⇒ 00:08:47.930 Pranav: Yep.

129 00:08:50.020 ⇒ 00:08:51.270 Pranav: Okay…

130 00:08:51.270 ⇒ 00:08:56.509 Samuel Roberts: Yeah, so, yeah, we’ll sort that out next week, I guess, make sure we have everything.

131 00:08:57.330 ⇒ 00:08:57.780 Pranav: Yup, right.

132 00:08:59.910 ⇒ 00:09:03.820 Pranav: Okay, can you guys see my screen? It’s a little bit weird, I could fix this, actually.

133 00:09:04.270 ⇒ 00:09:06.980 Samuel Roberts: Yeah, I see it a little narrow, but I see it.

134 00:09:07.860 ⇒ 00:09:10.710 Pranav: Okay, yeah, actually, give me one sec, let me make this a little bit better.

135 00:09:13.150 ⇒ 00:09:14.619 Mustafa Raja: Is this a vertical monitor?

136 00:09:16.520 ⇒ 00:09:21.040 Pranav: No, no, I’m just doing dual, like, sharing a specific tab, so I can…

137 00:09:21.040 ⇒ 00:09:21.840 Samuel Roberts: Yeah.

138 00:09:21.840 ⇒ 00:09:22.590 Pranav: Yeah.

139 00:09:40.340 ⇒ 00:09:40.900 Pranav: Okay.

140 00:09:47.870 ⇒ 00:09:53.859 Pranav: Okay, yeah, I just, there’s a ton of triage tickets now from all of the…

141 00:09:53.960 ⇒ 00:09:57.290 Pranav: Feedback that we got yesterday and the day before.

142 00:09:57.640 ⇒ 00:10:10.229 Pranav: So, that’s another issue. We’ll, I’m gonna talk to Janiece about that a little bit today, because she thinks that there’s probably, like, a lot of feedback in there that may not be actually accurate. So…

143 00:10:10.990 ⇒ 00:10:16.639 Pranav: We’ll… we’ll discuss that in my meeting later today, and I’ll… I’ll report back to you guys on that.

144 00:10:16.820 ⇒ 00:10:22.080 Pranav: But let’s just focus on these things for right now. So, let’s just start maybe with the internal review.

145 00:10:23.220 ⇒ 00:10:24.319 Pranav: Let’s see…

146 00:10:27.450 ⇒ 00:10:31.040 Pranav: are not happening for client ABC.

147 00:10:32.770 ⇒ 00:10:33.960 Casie Aviles: Yeah.

148 00:10:33.960 ⇒ 00:10:34.570 Pranav: Okay.

149 00:10:35.330 ⇒ 00:10:38.459 Pranav: This is, like, the chat history…

150 00:10:39.590 ⇒ 00:10:43.589 Pranav: Issue, right? Okay. And what was the resolution on this?

151 00:10:44.430 ⇒ 00:10:50.159 Casie Aviles: Basically, what I did there is I just, so there are two things. I updated the prompt,

152 00:10:50.640 ⇒ 00:10:57.460 Casie Aviles: To make sure that each call is… or, yeah, each time a query goes through the database agent.

153 00:10:57.620 ⇒ 00:10:59.830 Casie Aviles: It will… it will call the tool.

154 00:11:00.190 ⇒ 00:11:04.440 Casie Aviles: And then as another fallback, there’s…

155 00:11:04.870 ⇒ 00:11:11.220 Casie Aviles: it checks if the tool has been called. If it’s not, then it will just force the tool called.

156 00:11:11.770 ⇒ 00:11:14.379 Casie Aviles: So that’s what I added here.

157 00:11:14.890 ⇒ 00:11:22.510 Pranav: Oh, got you. Yay. And you documented it here, perfect. Sam, we feel pretty good about this, right? I can just, ship this.

158 00:11:22.850 ⇒ 00:11:23.250 Samuel Roberts: Yeah.

159 00:11:23.250 ⇒ 00:11:25.120 Pranav: Yeah, so… Yeah, cool.

160 00:11:25.120 ⇒ 00:11:26.020 Samuel Roberts: consultants, too.

161 00:11:28.150 ⇒ 00:11:36.659 Casie Aviles: What I’m a little… concerned about is this one, because earlier, I think I… we’re still getting some

162 00:11:37.300 ⇒ 00:11:44.860 Casie Aviles: Well, yesterday, it got lower, the execution times by the end of, you know, the day.

163 00:11:45.220 ⇒ 00:11:48.899 Casie Aviles: So… and I guess now, we’re still…

164 00:11:49.660 ⇒ 00:11:54.029 Casie Aviles: When I was testing just earlier, like, it was taking a little longer again.

165 00:11:54.320 ⇒ 00:11:55.200 Casie Aviles: So…

166 00:11:55.960 ⇒ 00:11:57.060 Samuel Roberts: I wonder if it’s, like…

167 00:11:57.260 ⇒ 00:12:01.540 Samuel Roberts: time of day with other… do we confirm at all if they’re using it for anything else?

168 00:12:02.790 ⇒ 00:12:03.990 Pranav: No.

169 00:12:03.990 ⇒ 00:12:04.730 Casie Aviles: No.

170 00:12:04.730 ⇒ 00:12:07.410 Pranav: I was not able to, but I’m not sure if on…

171 00:12:07.960 ⇒ 00:12:09.850 Pranav: Our end, are we able to see that?

172 00:12:10.130 ⇒ 00:12:15.910 Samuel Roberts: No, I think Tim was trying to get us access, because it’s a different… it’s not in GCP, it’s in a different,

173 00:12:17.370 ⇒ 00:12:18.810 Samuel Roberts: login, I guess.

174 00:12:20.810 ⇒ 00:12:23.150 Pranav: Okay, so it’s like a shared key, essentially.

175 00:12:23.810 ⇒ 00:12:26.480 Samuel Roberts: I… well, it might be, I’m just not sure. That’s…

176 00:12:26.480 ⇒ 00:12:26.949 Pranav: Could be, yeah.

177 00:12:26.950 ⇒ 00:12:32.159 Samuel Roberts: chat with him, he said it wasn’t something that he could just turn on in GCP for us to have access, but he was gonna…

178 00:12:32.980 ⇒ 00:12:34.849 Samuel Roberts: Try to…

179 00:12:42.860 ⇒ 00:12:44.630 Pranav: Gotcha. Okay, well…

180 00:12:44.630 ⇒ 00:12:46.290 Samuel Roberts: Yeah, I’m trying to find the chat now, but…

181 00:12:49.290 ⇒ 00:12:52.569 Samuel Roberts: Yeah, I don’t know where. Okay, I can try to follow up with that.

182 00:12:52.570 ⇒ 00:12:56.670 Pranav: Sam, do you actually want to maybe just… ping Tim about that in.

183 00:12:56.670 ⇒ 00:13:01.080 Samuel Roberts: That’s what I was just saying, yeah, I was just gonna, find that message and…

184 00:13:01.200 ⇒ 00:13:03.559 Samuel Roberts: Reference it for him.

185 00:13:09.680 ⇒ 00:13:10.890 Casie Aviles: Okay.

186 00:13:12.330 ⇒ 00:13:14.320 Casie Aviles: Yeah, I think I found it, though.

187 00:13:19.510 ⇒ 00:13:20.470 Casie Aviles: It’s here.

188 00:13:21.840 ⇒ 00:13:24.429 Samuel Roberts: Oh, you got it? Okay. I was just doing a quick search.

189 00:13:29.450 ⇒ 00:13:32.149 Samuel Roberts: Oh, there it is. Okay, cool, I’ve got it in here too. Yeah, okay, cool.

190 00:13:32.690 ⇒ 00:13:33.340 Samuel Roberts: Thank you.

191 00:13:33.340 ⇒ 00:13:34.860 Casie Aviles: Yeah, I was…

192 00:13:35.530 ⇒ 00:13:43.700 Casie Aviles: trying to access AI Studio. I could access it with our account, but not the Brainforge one. Like, I mean, the internal one that we have.

193 00:13:44.550 ⇒ 00:13:46.950 Samuel Roberts: Right, right, so we have ours, but not theirs, and so…

194 00:13:46.950 ⇒ 00:13:48.000 Casie Aviles: Yes, yes.

195 00:13:54.130 ⇒ 00:13:56.609 Pranav: And then for this one…

196 00:13:58.090 ⇒ 00:13:59.450 Casie Aviles: Oh,

197 00:13:59.840 ⇒ 00:14:10.059 Casie Aviles: Well, what was reported here was that there was nothing in the database, but we do have… we do have contents in the database, so I took

198 00:14:10.760 ⇒ 00:14:11.830 Casie Aviles: a look at.

199 00:14:12.460 ⇒ 00:14:21.599 Casie Aviles: the questions, and I tried to recreate, or, like, reproduce the issue. So for these, I was able to get an answer. It’s just this one.

200 00:14:22.380 ⇒ 00:14:28.580 Casie Aviles: That we need to probably… yeah, we need to update the database for that in order to get an answer.

201 00:14:29.630 ⇒ 00:14:30.040 Samuel Roberts: service.

202 00:14:30.040 ⇒ 00:14:32.950 Pranav: Gotcha, so this is probably something Janiece needs to do.

203 00:14:33.750 ⇒ 00:14:39.930 Casie Aviles: Yeah, I would say so. We just need to add… Whoever the service manager is.

204 00:14:41.610 ⇒ 00:14:48.990 Pranav: Okay, cool. Do you want to reach out to Janiece right now, and just… Maybe mention the specific…

205 00:14:49.540 ⇒ 00:14:56.159 Pranav: the specific prompt that was not returning, and how she can just update the DB to have it return the proper information.

206 00:14:57.180 ⇒ 00:14:58.180 Casie Aviles: Okay.

207 00:14:58.830 ⇒ 00:15:02.360 Pranav: Perfect, thanks. And then, I think we’re good to close this out then.

208 00:15:05.170 ⇒ 00:15:06.890 Pranav: And you can just do that in the channel.

209 00:15:09.830 ⇒ 00:15:19.610 Pranav: Thank you very much. Alright, integrate trainer’s feedback into central docs. Yeah, I think this is good to go, right, Mustafa? This is all we’re working on this morning.

210 00:15:21.770 ⇒ 00:15:31.070 Mustafa Raja: Yes, actually, this was, related to adding the universal info thing, you know? The first universal info document.

211 00:15:34.030 ⇒ 00:15:37.730 Pranav: Yeah, is there a… so I think we’re…

212 00:15:38.390 ⇒ 00:15:43.109 Pranav: Well, yeah, I guess you’re talking about, making the DocX changes right now, right?

213 00:15:45.060 ⇒ 00:15:51.539 Mustafa Raja: If you open this, so this ticket was related to adding this. Here are some templates that you need.

214 00:15:51.540 ⇒ 00:15:56.069 Pranav: Oh, right. Yeah, this is, like, that first dog. Yeah, yeah.

215 00:15:56.070 ⇒ 00:15:57.619 Mustafa Raja: Another dog that we added.

216 00:15:58.230 ⇒ 00:16:00.700 Mustafa Raja: So this is good, is what I’m trying to say. Yeah.

217 00:16:01.200 ⇒ 00:16:01.770 Pranav: Yup.

218 00:16:02.930 ⇒ 00:16:03.730 Pranav: Okay.

219 00:16:07.540 ⇒ 00:16:08.620 Pranav: What is this?

220 00:16:11.350 ⇒ 00:16:15.069 Pranav: Oh, okay, yeah, this is just, like, project plan stuff. I already did this.

221 00:16:19.360 ⇒ 00:16:23.920 Pranav: Okay, employment team and reviewed central doc update, report from triage data.

222 00:16:25.840 ⇒ 00:16:27.749 Pranav: Okay, this is kind of…

223 00:16:33.110 ⇒ 00:16:36.859 Pranav: Yeah, I’m just gonna… Delete this one. This is kind of…

224 00:16:37.410 ⇒ 00:16:40.279 Pranav: The whole point of our next project.

225 00:16:42.140 ⇒ 00:16:45.199 Pranav: And Sam, you and I will work on tickets to, like.

226 00:16:45.870 ⇒ 00:16:48.450 Samuel Roberts: Alright, let’s a message. Which one?

227 00:16:48.450 ⇒ 00:16:49.070 Pranav: units.

228 00:16:50.810 ⇒ 00:16:52.749 Samuel Roberts: I didn’t see the ticket, sorry, which one was it?

229 00:16:52.990 ⇒ 00:16:55.290 Pranav: Oh, no, you’re good. This is… I think,

230 00:16:55.500 ⇒ 00:16:57.810 Pranav: B made this, like, a while ago.

231 00:16:58.140 ⇒ 00:17:00.160 Pranav: Oh, yeah, it’s okay. Yeah, totally.

232 00:17:00.160 ⇒ 00:17:03.069 Samuel Roberts: Yeah, that’s good. I was looking at those tickets earlier for…

233 00:17:04.230 ⇒ 00:17:09.459 Pranav: Cool. Let’s go… Andy into… Yeah, this is…

234 00:17:12.230 ⇒ 00:17:15.959 Pranav: Okay. Yeah, I think I saw this other day, I think we can just…

235 00:17:16.230 ⇒ 00:17:18.619 Pranav: Put this into the backlog, like, this is…

236 00:17:18.760 ⇒ 00:17:22.150 Pranav: Obviously not important right now, so let’s just not even talk about it right now.

237 00:17:22.270 ⇒ 00:17:23.660 Pranav: Let’s just,

238 00:17:27.210 ⇒ 00:17:30.529 Pranav: Delete it maybe later, but we’ll just put it in the backlog for now.

239 00:17:30.720 ⇒ 00:17:34.999 Pranav: Okay, integrate trainer feedback and re-embed docs. Okay, this is already done.

240 00:17:40.760 ⇒ 00:17:44.420 Pranav: Okay… this…

241 00:17:51.100 ⇒ 00:17:53.509 Pranav: Yeah, put this into client review.

242 00:17:55.200 ⇒ 00:17:59.139 Pranav: Talk to James about that later. Template embeddings are not retrievable.

243 00:17:59.310 ⇒ 00:18:09.230 Pranav: In Andy… This was, I think, Mustafa, you confirmed that they are now, and you gave examples, right?

244 00:18:12.430 ⇒ 00:18:13.390 Mustafa Raja: Yes.

245 00:18:14.050 ⇒ 00:18:20.219 Mustafa Raja: Like, the, the issue, so this is related to the, issue with,

246 00:18:20.700 ⇒ 00:18:28.930 Mustafa Raja: They, they, they just weren’t getting the same information, right? And the issue was that, their, their,

247 00:18:29.720 ⇒ 00:18:33.989 Mustafa Raja: query was coming from. The PEST document, I think.

248 00:18:33.990 ⇒ 00:18:35.050 Pranav: Oh, right.

249 00:18:35.050 ⇒ 00:18:41.450 Mustafa Raja: We need to inform them that all of the information should live in one place, right?

250 00:18:42.410 ⇒ 00:18:43.529 Pranav: Yeah, so there’s conflicting information.

251 00:18:43.770 ⇒ 00:18:50.549 Mustafa Raja: Also, also, they should… I mean, the query that they made, they didn’t even mention template, I think, in the…

252 00:18:51.800 ⇒ 00:18:52.480 Pranav: How did they?

253 00:18:52.480 ⇒ 00:18:53.680 Samuel Roberts: phrase it. Okay.

254 00:18:53.680 ⇒ 00:18:55.830 Mustafa Raja: Yeah, the phrasing matters also, you know.

255 00:18:56.780 ⇒ 00:18:58.940 Samuel Roberts: We should probably catch that for a few seconds.

256 00:18:58.940 ⇒ 00:19:02.350 Mustafa Raja: Let me double-check if I am… Correct, dear?

257 00:19:03.930 ⇒ 00:19:05.010 Mustafa Raja: Let me…

258 00:19:10.300 ⇒ 00:19:15.149 Pranav: Yeah, if you want to add a little bit more clarity to this ticket, I’m gonna bring it up with Janice later.

259 00:19:15.510 ⇒ 00:19:16.240 Mustafa Raja: We’re good.

260 00:19:18.560 ⇒ 00:19:25.330 Pranav: Spike back in outage monitoring for Andy. Yeah, sam, maybe you and I can talk about this later.

261 00:19:26.170 ⇒ 00:19:26.870 Samuel Roberts: Go on.

262 00:19:27.140 ⇒ 00:19:31.070 Pranav: This one right here, this was because, like, you know, Andy…

263 00:19:31.300 ⇒ 00:19:37.099 Pranav: kind of went down, because of the Heroku, deprecation.

264 00:19:37.620 ⇒ 00:19:55.020 Pranav: And so, for us to just kind of get, like, some alerts when… when something like that is to happen, not, I guess, you know, just a whole service getting deprecated like that, like, we would be in the know for going forward, but say if there’s, like, some outage, things of that nature,

265 00:19:55.020 ⇒ 00:19:55.510 Samuel Roberts: Yeah.

266 00:19:55.510 ⇒ 00:19:59.510 Pranav: just having alerts maybe come into Slack, so, like, we’re not notified by the client on this.

267 00:19:59.770 ⇒ 00:20:00.500 Samuel Roberts: Totally.

268 00:20:00.810 ⇒ 00:20:02.760 Samuel Roberts: Yeah, yeah, we can, we can do that.

269 00:20:05.380 ⇒ 00:20:08.800 Pranav: Duplicate Azure and phase that on it in. Okay, we’ll do that in a sec.

270 00:20:08.990 ⇒ 00:20:13.120 Pranav: Do that in a sec, decide which I add on…

271 00:20:15.530 ⇒ 00:20:23.660 Pranav: Oh, this was already completed. This was the whole thing of, like, we had a Google… workspace add-on…

272 00:20:24.810 ⇒ 00:20:30.429 Pranav: Versus turning it into, like, a… using the Google Chat API. Yeah.

273 00:20:30.660 ⇒ 00:20:31.140 Casie Aviles: Yeah.

274 00:20:31.140 ⇒ 00:20:31.730 Pranav: Yeah.

275 00:20:34.580 ⇒ 00:20:37.029 Pranav: Yeah, so, okay, this is already done.

276 00:20:45.130 ⇒ 00:20:48.690 Pranav: Okay… Backfill missing triage tickets with…

277 00:20:49.540 ⇒ 00:20:54.230 Casie Aviles: So, let’s just mark that as done, that’s from a while ago, yeah.

278 00:20:54.410 ⇒ 00:20:54.840 Pranav: Okay.

279 00:20:54.840 ⇒ 00:20:55.740 Samuel Roberts: Oh, nice.

280 00:21:08.440 ⇒ 00:21:12.739 Pranav: Yup, Mustafa, you… you did this, right?

281 00:21:12.740 ⇒ 00:21:14.530 Mustafa Raja: Yeah, this is good.

282 00:21:15.020 ⇒ 00:21:15.750 Mustafa Raja: We have.

283 00:21:15.750 ⇒ 00:21:20.900 Pranav: Do you wanna leave the… Attach the report here.

284 00:21:20.900 ⇒ 00:21:22.799 Mustafa Raja: put it in Slack.

285 00:21:22.970 ⇒ 00:21:26.560 Mustafa Raja: Tag linear in our thread to update this.

286 00:21:27.370 ⇒ 00:21:28.350 Pranav: Yeah, perfect.

287 00:21:43.820 ⇒ 00:21:49.910 Pranav: Yeah, so… I think what I will do is…

288 00:21:51.290 ⇒ 00:21:53.370 Pranav: Kind of… when did I add this?

289 00:21:53.530 ⇒ 00:21:58.050 Pranav: ticket. Yeah, like, two and a half weeks ago, or something like that.

290 00:21:59.580 ⇒ 00:22:04.860 Pranav: This will help… this would have, like, helped us with, like, this template issue that we’re seeing today, or we saw yesterday.

291 00:22:05.320 ⇒ 00:22:11.250 Pranav: Just doing, like, a one-time search of maybe, like, each chunk.

292 00:22:11.690 ⇒ 00:22:17.159 Pranav: Across the entire… Central. All the central dock embeddings.

293 00:22:17.310 ⇒ 00:22:21.490 Pranav: And just assessing, like, is this conflicting or duplicate information?

294 00:22:22.890 ⇒ 00:22:24.890 Pranav: I think that would be, like, a good…

295 00:22:25.390 ⇒ 00:22:40.080 Pranav: good way for us to flag certain duplicate and conflicting stuff, and then have a report for Janice to look at to say, hey, these are all the flags and alerts we’re seeing for duplicate and conflicting stuff, can you make an assessment of

296 00:22:40.310 ⇒ 00:22:44.069 Pranav: what… Can be removed or updated.

297 00:22:44.300 ⇒ 00:22:45.729 Pranav: Does that… does that make sense?

298 00:22:50.580 ⇒ 00:22:54.200 Samuel Roberts: Isn’t that part of the, like, the project tickets we’re putting together?

299 00:22:55.570 ⇒ 00:23:01.470 Pranav: It actually is, yeah, so… This would be of just, like, kind of a historical search.

300 00:23:01.880 ⇒ 00:23:09.359 Samuel Roberts: Right, well, I’m just trying to think, like, if we could… like, if we do this ticket, that actually, like, might give us the logic we need to…

301 00:23:10.200 ⇒ 00:23:10.630 Pranav: Yes.

302 00:23:10.630 ⇒ 00:23:11.460 Samuel Roberts: consistently.

303 00:23:11.960 ⇒ 00:23:19.299 Pranav: So, what I’ll do is… I’ll leave this here. Sam, I’ll just assign it to you.

304 00:23:20.260 ⇒ 00:23:21.889 Pranav: And then…

305 00:23:23.150 ⇒ 00:23:29.249 Pranav: I think you can, like, I’ll just assign this to you, just so, like, we can figure out how we can, on our end.

306 00:23:29.430 ⇒ 00:23:31.140 Pranav: Create the linear tickets.

307 00:23:31.250 ⇒ 00:23:33.199 Pranav: To hit those milestones.

308 00:23:33.500 ⇒ 00:23:34.400 Samuel Roberts: Yeah, okay.

309 00:23:35.140 ⇒ 00:23:41.340 Samuel Roberts: Yeah, so, like, either this ticket will stay and we’ll do it, and then it’ll feed into that, or we’ll just fold it into the other ones.

310 00:23:41.660 ⇒ 00:23:44.680 Pranav: Yeah, yeah, and like, you can delete this, but it’s just kind of… it’s here.

311 00:23:44.680 ⇒ 00:23:46.829 Samuel Roberts: Yeah, they’ll leave it definitely for now, yeah, totally, totally.

312 00:23:46.830 ⇒ 00:23:47.400 Pranav: Yeah.

313 00:23:48.210 ⇒ 00:23:53.260 Pranav: Spike on Obsidian Bogulesu for versioning on central docks.

314 00:23:55.260 ⇒ 00:23:59.129 Pranav: Yeah, I don’t think this is… We’re good on this.

315 00:24:07.190 ⇒ 00:24:14.329 Pranav: Did we already do this, or is it even necessary? I know we were talking about doing this for, creating the dynamically sized chunks.

316 00:24:19.280 ⇒ 00:24:20.230 Samuel Roberts: Oh, yeah.

317 00:24:22.320 ⇒ 00:24:25.629 Pranav: But Mustafa, like, you’ve already done this, like, the…

318 00:24:26.000 ⇒ 00:24:34.100 Pranav: the dynamic chunk sizes, so… did you already put the numbers in, or did you not need to? Either way, this is probably not relevant anymore.

319 00:24:35.230 ⇒ 00:24:38.869 Mustafa Raja: Yeah, the dynamic junk size isn’t related to this.

320 00:24:38.980 ⇒ 00:24:44.090 Mustafa Raja: This is, much more related to… so…

321 00:24:45.440 ⇒ 00:24:49.120 Mustafa Raja: The agent would know, you know, the subsection,

322 00:24:49.410 ⇒ 00:24:54.100 Mustafa Raja: It is of a parent… of a parent, section, you know?

323 00:24:54.310 ⇒ 00:25:00.210 Mustafa Raja: And I haven’t added those in all of the documents, we need to do that. So let’s keep this.

324 00:25:01.180 ⇒ 00:25:06.190 Pranav: Okay, so this is… So this is for the subheadings.

325 00:25:06.550 ⇒ 00:25:08.909 Mustafa Raja: Yeah, this is for the capital, yeah, yeah, yeah.

326 00:25:09.090 ⇒ 00:25:15.579 Mustafa Raja: So subheadings, and so the agent would know, you know, which section belongs to what parent section.

327 00:25:20.680 ⇒ 00:25:23.749 Mustafa Raja: The dynamic tracking is already good.

328 00:25:26.490 ⇒ 00:25:27.639 Pranav: Sorry, what was that?

329 00:25:27.800 ⇒ 00:25:31.870 Mustafa Raja: Yeah, but the dynamic chunking is already good, so… Okay.

330 00:25:31.870 ⇒ 00:25:32.620 Pranav: Gotcha.

331 00:25:33.330 ⇒ 00:25:38.840 Pranav: Alright, so I’ll just leave it like this, to-do… Cool.

332 00:25:43.040 ⇒ 00:25:45.790 Pranav: Okay, yeah, this is for me to look into…

333 00:25:51.580 ⇒ 00:25:55.670 Pranav: Alright, enhanced reporting views… This is all old stuff.

334 00:25:58.330 ⇒ 00:26:02.910 Pranav: Okay, I think we’re pretty good then. Now… this…

335 00:26:03.760 ⇒ 00:26:11.489 Pranav: these 5 things up top here, or these 4 things actually right here, I think once we do the cutover right now, we’ll feel…

336 00:26:12.310 ⇒ 00:26:15.999 Pranav: we can just close these out, too. Or actually…

337 00:26:18.670 ⇒ 00:26:21.520 Pranav: This is one other thing that we wanted to work on.

338 00:26:22.790 ⇒ 00:26:23.690 Pranav: Yuck.

339 00:26:23.840 ⇒ 00:26:31.340 Pranav: Hmm. Wait, actually, is this a duplicate? I thought I assigned this to… Casey.

340 00:26:32.450 ⇒ 00:26:36.119 Casie Aviles: Yeah, I think this is a duplicate. I had a…

341 00:26:37.360 ⇒ 00:26:41.390 Casie Aviles: I had the ticket I just updated a few minutes earlier.

342 00:26:41.670 ⇒ 00:26:42.659 Casie Aviles: Let me see…

343 00:26:43.500 ⇒ 00:26:45.490 Pranav: Okay, oh, so is this already, like…

344 00:26:45.690 ⇒ 00:26:50.179 Pranav: I honestly haven’t checked in on this at all, so… kind of forgot about it. Did you already work on this?

345 00:26:51.370 ⇒ 00:26:53.570 Casie Aviles: Where is that? I think it’s this one.

346 00:26:53.820 ⇒ 00:26:54.580 Casie Aviles: Oh.

347 00:26:55.550 ⇒ 00:26:58.910 Casie Aviles: 2014, yeah, I think it’s that one.

348 00:27:00.460 ⇒ 00:27:02.100 Pranav: 2014…

349 00:27:02.100 ⇒ 00:27:03.710 Casie Aviles: A, B, C, this one.

350 00:27:04.320 ⇒ 00:27:05.460 Casie Aviles: Oh, okay.

351 00:27:10.670 ⇒ 00:27:20.019 Casie Aviles: That’s the one that got assigned to me. But it’s not on… it’s not unreal yet. Do we want it on real? Right now, it’s just… I just generated these CSVs.

352 00:27:21.130 ⇒ 00:27:29.559 Pranav: Okay, I mean… Yeah, on real would be optimal. You know, I think…

353 00:27:29.860 ⇒ 00:27:38.319 Pranav: with all the QA testing, like, it’s totally fine that it… it’s not in real right now. I’m not gonna close this out until it’s in real, but for today’s…

354 00:27:39.090 ⇒ 00:27:54.880 Pranav: I think once we do this cutover, Casey, if you have more time, like, to just add this to Rill, that would be great, and then if it’s done before my, meeting with them, then I can just demo it from Rill. If not, then I’ll just demo

355 00:27:55.130 ⇒ 00:27:56.820 Pranav: Just from this report right here.

356 00:27:57.320 ⇒ 00:27:57.990 Casie Aviles: Okay.

357 00:27:58.670 ⇒ 00:27:59.290 Pranav: Yeah.

358 00:28:00.130 ⇒ 00:28:02.210 Pranav: Okay, so I’ll leave this in progress, then.

359 00:28:02.210 ⇒ 00:28:03.420 Casie Aviles: It’s… it’s just…

360 00:28:03.770 ⇒ 00:28:10.380 Casie Aviles: Just to clarify if I got it right, like, it’s just generating, like, categories of what kinds of questions have been asked, right?

361 00:28:11.030 ⇒ 00:28:14.209 Pranav: Yeah, actually, let’s just look at this right now, just to make sure.

362 00:28:17.210 ⇒ 00:28:17.780 Casie Aviles: Okay.

363 00:28:22.870 ⇒ 00:28:30.350 Casie Aviles: Yeah, there’s, like, the summary one, I think. You could… you would be able to see, like, what kinds of categories have been generated.

364 00:28:31.080 ⇒ 00:28:31.909 Pranav: Oh, okay, cool.

365 00:28:46.210 ⇒ 00:28:52.290 Pranav: Okay, nice. And then what is this percent? Oh, percentage is, like, the percentage of the total… Questions?

366 00:28:52.740 ⇒ 00:29:02.980 Casie Aviles: this is for… 16th… the week of 16th, March 16th, until… Today, 27th.

367 00:29:05.410 ⇒ 00:29:09.739 Pranav: Could we… one other thing, and I don’t know if I added a description, this is probably on me.

368 00:29:10.130 ⇒ 00:29:16.039 Pranav: Just adding example questions per category.

369 00:29:17.220 ⇒ 00:29:20.919 Casie Aviles: Sure, I think one of these reports have examples.

370 00:29:22.200 ⇒ 00:29:23.230 Pranav: Oh, perfect.

371 00:29:24.000 ⇒ 00:29:26.029 Pranav: Oh, input category samples.

372 00:29:26.590 ⇒ 00:29:27.420 Casie Aviles: Yeah.

373 00:29:29.190 ⇒ 00:29:29.860 Pranav: Nice.

374 00:29:34.360 ⇒ 00:29:36.110 Pranav: Oh, okay. Cool, cool.

375 00:29:44.720 ⇒ 00:29:45.940 Pranav: So it’s like…

376 00:29:47.130 ⇒ 00:29:48.780 Samuel Roberts: We’re still seeing later, by the way.

377 00:29:49.430 ⇒ 00:29:50.330 Pranav: Yup, yep.

378 00:29:50.780 ⇒ 00:29:51.310 Samuel Roberts: Okay.

379 00:29:51.450 ⇒ 00:29:52.450 Samuel Roberts: I know we were supposed to.

380 00:29:52.450 ⇒ 00:29:53.900 Pranav: Yeah, I think this looks good.

381 00:29:54.190 ⇒ 00:29:54.910 Samuel Roberts: Okay.

382 00:29:54.910 ⇒ 00:29:59.840 Pranav: Yeah, so if we can just get all of that into Rel, that would be great, but for…

383 00:30:00.130 ⇒ 00:30:03.879 Pranav: for my, presentation today, this should be… this should be perfect. Thank you.

384 00:30:04.320 ⇒ 00:30:04.920 Casie Aviles: Okay.

385 00:30:05.420 ⇒ 00:30:06.000 Pranav: Yeah.

386 00:30:10.570 ⇒ 00:30:14.970 Pranav: Okay… Let’s see…

387 00:30:15.190 ⇒ 00:30:19.659 Pranav: Alright, so… yeah, Sam, do you want to kind of take over, and then we can,

388 00:30:20.170 ⇒ 00:30:21.630 Pranav: Hop into GCP.

389 00:30:24.820 ⇒ 00:30:29.560 Samuel Roberts: Yeah, we did. So those other… that embedding is…

390 00:30:30.220 ⇒ 00:30:34.259 Samuel Roberts: Presumably done at this point, it was 6-7 minutes. I’m guessing.

391 00:30:35.940 ⇒ 00:30:36.740 Pranav: Alright.

392 00:30:40.770 ⇒ 00:30:43.889 Pranav: Mustafa, the embedding pipeline is finished?

393 00:30:45.220 ⇒ 00:30:45.860 Mustafa Raja: Yeah.

394 00:30:46.160 ⇒ 00:30:50.170 Mustafa Raja: Okay. Actually, give me a moment…

395 00:30:50.400 ⇒ 00:30:51.240 Samuel Roberts: Okay.

396 00:30:52.280 ⇒ 00:30:56.159 Mustafa Raja: When we built again, I… I didn’t run it again.

397 00:31:00.900 ⇒ 00:31:02.989 Mustafa Raja: It’ll take a minute only.

398 00:31:04.060 ⇒ 00:31:04.830 Pranav: Yeah, that’s fine.

399 00:31:05.260 ⇒ 00:31:10.189 Samuel Roberts: Quick question, because I was just looking at Vertex Studio AI versus Studio.

400 00:31:10.340 ⇒ 00:31:14.980 Samuel Roberts: Or AI Studio stuff. I’m seeing conflicting things about which has higher rate limits.

401 00:31:15.790 ⇒ 00:31:20.060 Samuel Roberts: What I’m trying to figure out now…

402 00:31:20.620 ⇒ 00:31:22.840 Samuel Roberts: Before we actually, like, flip this switch.

403 00:31:23.070 ⇒ 00:31:28.800 Samuel Roberts: Is, when we tested the… the concurrency stuff, Mustafa.

404 00:31:29.270 ⇒ 00:31:29.980 Mustafa Raja: Yeah.

405 00:31:30.320 ⇒ 00:31:31.810 Samuel Roberts: What time of day was that?

406 00:31:34.450 ⇒ 00:31:39.820 Mustafa Raja: It was, after, it was after, end of day.

407 00:31:40.090 ⇒ 00:31:41.410 Samuel Roberts: End is okay.

408 00:31:41.410 ⇒ 00:31:45.299 Mustafa Raja: So, it was… let me, let me see…

409 00:31:45.960 ⇒ 00:31:48.630 Samuel Roberts: Okay, no, I’m just… cause I’m wondering if that is a…

410 00:31:50.380 ⇒ 00:31:53.649 Samuel Roberts: time-sensitive thing with the APIs we’re using.

411 00:31:56.020 ⇒ 00:32:02.500 Mustafa Raja: So you’re in, 50 PST.

412 00:32:05.740 ⇒ 00:32:07.770 Mustafa Raja: So it would be 9pm EST.

413 00:32:08.600 ⇒ 00:32:16.350 Samuel Roberts: Okay, yeah, so probably lower usage than when they were doing the QA and seeing issues yesterday.

414 00:32:16.970 ⇒ 00:32:17.800 Casie Aviles: Yeah.

415 00:32:20.100 ⇒ 00:32:22.409 Samuel Roberts: We may wanna… how long did that take to run?

416 00:32:24.200 ⇒ 00:32:30.459 Mustafa Raja: It didn’t take, much… time. Might have taken, 3-4 minutes.

417 00:32:31.160 ⇒ 00:32:31.790 Samuel Roberts: Okay.

418 00:32:38.440 ⇒ 00:32:43.359 Samuel Roberts: I’m just trying to think if it’s worth it to rerun that during the day now.

419 00:32:44.470 ⇒ 00:32:49.630 Samuel Roberts: And see if we hit the limits that we were seeing before, or the errors we were seeing before.

420 00:32:50.490 ⇒ 00:32:52.210 Samuel Roberts: during their QA testing.

421 00:32:56.740 ⇒ 00:33:02.309 Samuel Roberts: I mean, tell me if I’m overthinking it here, but… If… if we’re using the…

422 00:33:05.770 ⇒ 00:33:10.010 Samuel Roberts: Okay, hold on, wait a minute, Tim just responded to something I asked, so we’re good. Let’s see…

423 00:33:14.540 ⇒ 00:33:15.600 Samuel Roberts: Okay, cool.

424 00:33:20.360 ⇒ 00:33:24.600 Samuel Roberts: Okay, cool, that is a specific endpoint, so there’s nothing else. So, okay.

425 00:33:25.070 ⇒ 00:33:30.340 Samuel Roberts: That makes me more confused, but at least it’s a good thing.

426 00:33:33.890 ⇒ 00:33:37.240 Mustafa Raja: And, just for the context, we ran about…

427 00:33:37.470 ⇒ 00:33:40.849 Mustafa Raja: Let me double-check how many requests we ran.

428 00:33:41.870 ⇒ 00:33:49.870 Mustafa Raja: So we ran 15, 150, 300… 500… 500 requests, in, like, 3-4 minutes.

429 00:33:52.190 ⇒ 00:34:00.360 Mustafa Raja: And the response time was about 5-6 seconds for almost all of them. If you need detailed results, I can,

430 00:34:03.160 ⇒ 00:34:05.059 Mustafa Raja: And this is true.

431 00:34:06.350 ⇒ 00:34:07.710 Mustafa Raja: is…

432 00:34:07.710 ⇒ 00:34:13.850 Samuel Roberts: No, yeah, I saw that… I’m just trying to think if the time of day mattered, if there were other things, but Tim literally just confirmed that

433 00:34:14.300 ⇒ 00:34:18.980 Samuel Roberts: Nothing is hitting it programmatically, the AI Studio endpoint is specific to us.

434 00:34:23.940 ⇒ 00:34:26.770 Samuel Roberts: I’ll send in the screenshot… I guess.

435 00:34:30.750 ⇒ 00:34:31.610 Samuel Roberts: What the hell?

436 00:34:39.940 ⇒ 00:34:42.970 Samuel Roberts: We need 15 permissions? Alright, hold on.

437 00:34:46.670 ⇒ 00:34:50.469 Mustafa Raja: So we’re thinking to move to… Verdicts?

438 00:34:50.900 ⇒ 00:34:58.920 Samuel Roberts: I don’t want to do that right now. I’m just… I was just trying to figure out if there… this… this issue we saw with the QA was something that we will see again.

439 00:34:59.680 ⇒ 00:35:04.250 Samuel Roberts: And if it was a time of day thing, I didn’t know if it was a rate limit based on our endpoint or not.

440 00:35:20.110 ⇒ 00:35:21.770 Samuel Roberts: Suck, let me send this back.

441 00:35:27.930 ⇒ 00:35:30.249 Pranav: Are you able to see the monitoring data?

442 00:35:30.670 ⇒ 00:35:34.759 Samuel Roberts: That’s what he just shared, and I… we still need permissions for that, so I’m sending him what we need now.

443 00:35:35.290 ⇒ 00:35:35.990 Pranav: Yeah.

444 00:35:48.710 ⇒ 00:35:54.039 Samuel Roberts: Yeah, he had just responded to my message, and so… he sent me a page where we should be able to see it, but I don’t.

445 00:35:55.070 ⇒ 00:36:00.200 Samuel Roberts: But… If we’re the only ones using it, then that rate… Than the… or the…

446 00:36:00.840 ⇒ 00:36:04.590 Samuel Roberts: The message we saw, which seemed to imply a rate limit thing.

447 00:36:05.700 ⇒ 00:36:09.259 Samuel Roberts: I don’t know why we wouldn’t have seen it during the concurrency testing.

448 00:36:13.350 ⇒ 00:36:18.489 Samuel Roberts: Assuming Kugel’s consistent about that, which is another… Potential question mark.

449 00:36:22.360 ⇒ 00:36:30.869 Casie Aviles: Yeah, like, when I did my tests as well, I did… I sent them to the same deployed, URL.

450 00:36:32.360 ⇒ 00:36:33.809 Casie Aviles: Google Cloud, and…

451 00:36:34.030 ⇒ 00:36:40.999 Casie Aviles: I didn’t run into, like, 503s as much. I don’t think I did, yeah, and it was consistently, like.

452 00:36:42.610 ⇒ 00:36:43.500 Casie Aviles: Fast.

453 00:36:43.940 ⇒ 00:36:47.660 Casie Aviles: There were some spikes, but there were very few, so…

454 00:36:47.660 ⇒ 00:36:48.350 Samuel Roberts: Okay.

455 00:36:49.110 ⇒ 00:36:53.310 Samuel Roberts: Okay, so… If that’s the case.

456 00:36:53.310 ⇒ 00:36:56.129 Casie Aviles: for concurrency specifically, though.

457 00:36:56.130 ⇒ 00:37:02.670 Samuel Roberts: Right, but even so, there’s… there seems to be, like, some kind of other…

458 00:37:03.230 ⇒ 00:37:08.000 Samuel Roberts: Limits, there’s, like, tokens per minute, requests per minute, requests per day.

459 00:37:08.320 ⇒ 00:37:09.670 Samuel Roberts: From what I’m seeing.

460 00:37:12.740 ⇒ 00:37:13.470 Casie Aviles: Yeah.

461 00:37:13.470 ⇒ 00:37:14.279 Samuel Roberts: See you now.

462 00:37:17.960 ⇒ 00:37:22.390 Samuel Roberts: Okay, so I think I have it here now. This is the link, if y’all wanna…

463 00:37:22.660 ⇒ 00:37:25.170 Samuel Roberts: Take a quick look before we actually pull this trigger.

464 00:37:33.360 ⇒ 00:37:38.750 Samuel Roberts: Okay, traffic, so we saw a spike in the 10th.

465 00:37:39.890 ⇒ 00:37:43.610 Samuel Roberts: Spike of… this is the speed, and

466 00:37:43.840 ⇒ 00:37:46.120 Samuel Roberts: Where are we at today? 27th.

467 00:37:46.790 ⇒ 00:37:49.940 Samuel Roberts: Errors… Bye.

468 00:37:51.980 ⇒ 00:38:01.130 Samuel Roberts: So, yesterday, at 12pm, it says there was a 21.98% of the calls were errors.

469 00:38:02.700 ⇒ 00:38:09.580 Samuel Roberts: Overall latency looks fine. Latency by API method looks bad.

470 00:38:15.100 ⇒ 00:38:16.030 Samuel Roberts: Hmm.

471 00:38:18.650 ⇒ 00:38:22.560 Samuel Roberts: Let’s go with the last day real quick and see what that looks like.

472 00:38:24.110 ⇒ 00:38:27.750 Samuel Roberts: Yeah, P95 is high, P99 is high.

473 00:38:31.020 ⇒ 00:38:34.450 Samuel Roberts: But I don’t know what we… what we have control over to do anything about that with.

474 00:38:48.160 ⇒ 00:38:53.370 Samuel Roberts: Okay. At this point, I think we just need to… Flip it.

475 00:38:54.600 ⇒ 00:38:57.590 Samuel Roberts: Yeah, I think the QA testing was… was…

476 00:38:57.590 ⇒ 00:39:00.959 Pranav: Sam, just so, like, I understand, like, what we’re looking into right now is…

477 00:39:02.180 ⇒ 00:39:09.549 Pranav: we saw, like, kind of the increases in latency yesterday, like, or I should just say, more accurately, just the execution time was…

478 00:39:09.890 ⇒ 00:39:13.600 Pranav: Elevated. Yeah, when we dug into the…

479 00:39:14.450 ⇒ 00:39:15.350 Samuel Roberts: Yeah.

480 00:39:15.620 ⇒ 00:39:20.649 Samuel Roberts: there were, errors coming back from the Gemini API, like…

481 00:39:22.070 ⇒ 00:39:23.480 Pranav: Right, right, right, right.

482 00:39:23.480 ⇒ 00:39:28.949 Samuel Roberts: And so, that’s why I was trying to figure out, like, if we didn’t see this with the concurrency testing, what’s different?

483 00:39:30.530 ⇒ 00:39:30.930 Pranav: Yeah.

484 00:39:33.470 ⇒ 00:39:37.560 Pranav: Is there anything, like, a fallback that we can put? Like, right now?

485 00:39:37.560 ⇒ 00:39:41.950 Samuel Roberts: I mean…

486 00:39:47.540 ⇒ 00:39:50.600 Pranav: What would be the fallback? We need to get another API key?

487 00:39:51.940 ⇒ 00:39:52.660 Samuel Roberts: Yeah.

488 00:39:53.120 ⇒ 00:39:56.290 Pranav: previous OpenAI key that we can use as a fallback?

489 00:39:56.870 ⇒ 00:39:59.489 Samuel Roberts: We were using ours, which is the whole reason we’re moving over.

490 00:39:59.490 ⇒ 00:40:00.679 Pranav: The whole reason, my gosh.

491 00:40:01.110 ⇒ 00:40:07.820 Samuel Roberts: It was on our Azure, and we had issues with rate limits there, too, and we had to crank that up, but we had control over that.

492 00:40:08.000 ⇒ 00:40:09.280 Pranav: I’ll check out.

493 00:40:11.530 ⇒ 00:40:19.239 Samuel Roberts: I mean… The thing about it is, if it’s slower for certain calls, and much faster for others.

494 00:40:19.420 ⇒ 00:40:21.059 Samuel Roberts: It’s still an improvement.

495 00:40:24.710 ⇒ 00:40:27.090 Samuel Roberts: So unless we get just straight up, like…

496 00:40:28.820 ⇒ 00:40:32.209 Samuel Roberts: No response time, or no response at certain times.

497 00:40:32.780 ⇒ 00:40:33.570 Pranav: Yeah.

498 00:40:34.070 ⇒ 00:40:37.170 Samuel Roberts: then I think we should still… to it.

499 00:40:37.350 ⇒ 00:40:39.469 Samuel Roberts: Now. And then…

500 00:40:40.070 ⇒ 00:40:53.670 Samuel Roberts: I’ll have to do a little more digging. Again, I’m finding weird stuff for what people say, like, oh, I get better stuff with the, you know, Gemini Developer API, or I get better stuff with the AI Studio API. I don’t have a consistent understanding of it yet.

501 00:40:53.830 ⇒ 00:40:54.700 Samuel Roberts: So…

502 00:40:54.700 ⇒ 00:40:58.519 Pranav: For me, like, based on what Janice is saying, too, is that…

503 00:40:58.660 ⇒ 00:41:04.740 Pranav: you know, she’s giving a lot of feedback, but they did, like, a ton of testing yesterday, and so… That’s what I’m thinking.

504 00:41:04.740 ⇒ 00:41:09.749 Samuel Roberts: I feel like it’s… it’s probably… That was a high usage.

505 00:41:10.310 ⇒ 00:41:17.230 Pranav: Yeah, so Casey, is there, like, a real dashboard that you could just kind of show right now that just shows the amount of usage that they had?

506 00:41:17.940 ⇒ 00:41:22.150 Casie Aviles: Yeah, it’s… A lot when I last checked.

507 00:41:23.170 ⇒ 00:41:23.630 Pranav: Yeah.

508 00:41:23.630 ⇒ 00:41:32.459 Casie Aviles: I’m just worried, like, if we flip the switch now, and the CSRs will use it, it might… you know, it might be slow for them.

509 00:41:32.910 ⇒ 00:41:36.059 Samuel Roberts: That’s… yeah, but will it be slower than the current one?

510 00:41:37.900 ⇒ 00:41:40.170 Casie Aviles: Yeah, that…

511 00:41:40.170 ⇒ 00:41:40.779 Samuel Roberts: You know what I mean?

512 00:41:40.780 ⇒ 00:41:46.779 Casie Aviles: Case… Let’s see… Yeah, these are… this is one that I just tested earlier.

513 00:41:47.050 ⇒ 00:41:51.030 Samuel Roberts: Oh, wow, okay. But it was the… okay, interesting.

514 00:41:51.550 ⇒ 00:41:54.469 Casie Aviles: For this week, let’s see, custom…

515 00:41:56.240 ⇒ 00:41:58.339 Casie Aviles: Okay, I’ll just set it to…

516 00:42:01.160 ⇒ 00:42:06.030 Casie Aviles: Yeah, we have a lot of usage, like, compared to… last week.

517 00:42:06.140 ⇒ 00:42:08.029 Casie Aviles: So this is because of the dust.

518 00:42:08.550 ⇒ 00:42:11.809 Casie Aviles: Oh, look, yeah, look at this one, 26.

519 00:42:11.980 ⇒ 00:42:13.910 Samuel Roberts: What is our normal exchanges?

520 00:42:14.820 ⇒ 00:42:16.300 Samuel Roberts: on the NHN.

521 00:42:17.320 ⇒ 00:42:18.010 Pranav: We get 5.

522 00:42:18.010 ⇒ 00:42:18.960 Samuel Roberts: Or is this including that?

523 00:42:18.970 ⇒ 00:42:20.609 Pranav: $5.50 per week.

524 00:42:22.910 ⇒ 00:42:23.839 Samuel Roberts: Then we’re fine.

525 00:42:25.230 ⇒ 00:42:25.990 Samuel Roberts: Right.

526 00:42:28.240 ⇒ 00:42:35.709 Samuel Roberts: I’m just worried. I think I’m just… the testing threw me off, and the fact that we hit a rate limit then and not during the concurrency threw me off.

527 00:42:36.010 ⇒ 00:42:36.640 Samuel Roberts: So…

528 00:42:36.640 ⇒ 00:42:37.280 Pranav: Yeah.

529 00:42:37.600 ⇒ 00:42:38.079 Samuel Roberts: I think we’re.

530 00:42:38.080 ⇒ 00:42:38.800 Pranav: The thing is…

531 00:42:39.000 ⇒ 00:42:40.479 Samuel Roberts: We will… That’s true.

532 00:42:40.530 ⇒ 00:42:45.879 Pranav: we’re trying to kind of build up to this amount of exchanges, so… Of course.

533 00:42:45.880 ⇒ 00:42:51.889 Samuel Roberts: That might be a Vertex thing where we… we do want to switch. Mustafa, remind me, when we were talking about Vertex versus

534 00:42:52.150 ⇒ 00:42:55.589 Samuel Roberts: the developer API when Tim was asking.

535 00:42:57.830 ⇒ 00:43:02.669 Samuel Roberts: I think we saw that there were more options and stuff under the Vertex that we didn’t think we needed, but…

536 00:43:02.890 ⇒ 00:43:03.540 Mustafa Raja: Yeah, that is.

537 00:43:03.540 ⇒ 00:43:04.150 Samuel Roberts: I don’t remember…

538 00:43:04.150 ⇒ 00:43:05.720 Mustafa Raja: We also have…

539 00:43:05.930 ⇒ 00:43:12.489 Mustafa Raja: Yeah, Vertex just provides a lot more stuff, and we only needed the model, so that is why we… that is the first reason we…

540 00:43:12.490 ⇒ 00:43:15.139 Samuel Roberts: Right, we weren’t fine-tuning or doing anything like that.

541 00:43:15.140 ⇒ 00:43:24.250 Mustafa Raja: And then the other reason was, the general consensus is a vertex doesn’t play good when it comes to access and stuff.

542 00:43:24.370 ⇒ 00:43:25.240 Mustafa Raja: That’s, yeah.

543 00:43:25.240 ⇒ 00:43:26.810 Samuel Roberts: Yeah, it seemed like it’s more…

544 00:43:26.810 ⇒ 00:43:33.330 Mustafa Raja: from Vertex, and that is why we went with the developer API.

545 00:43:33.330 ⇒ 00:43:36.299 Samuel Roberts: That’s what I thought. Okay. Yeah, because even when I’m looking now, I’m seeing people.

546 00:43:36.300 ⇒ 00:43:43.229 Mustafa Raja: I’m just… I’m just wondering, are we just really not able to, adjust, limits?

547 00:43:43.230 ⇒ 00:43:43.950 Casie Aviles: limits.

548 00:43:43.950 ⇒ 00:43:44.950 Mustafa Raja: At all?

549 00:43:44.950 ⇒ 00:43:49.350 Samuel Roberts: I can’t see anything there, this is literally just,

550 00:43:51.480 ⇒ 00:43:54.079 Samuel Roberts: Quotas and system limits, let me see,

551 00:43:58.330 ⇒ 00:44:00.580 Samuel Roberts: It’s still loading, there we go.

552 00:44:02.410 ⇒ 00:44:05.619 Samuel Roberts: We have… oh, here we go. Oh, yeah, okay, let me…

553 00:44:06.140 ⇒ 00:44:08.470 Mustafa Raja: Can you share screen? I just want to see, you know…

554 00:44:09.110 ⇒ 00:44:11.159 Samuel Roberts: Oh, sure, yeah, if you want to go ahead.

555 00:44:16.070 ⇒ 00:44:19.659 Samuel Roberts: Yeah, I think… I think I just don’t see any knobs, you know?

556 00:44:26.980 ⇒ 00:44:30.509 Samuel Roberts: Let me… Sorry, were you sharing or stop sharing?

557 00:44:30.700 ⇒ 00:44:34.350 Mustafa Raja: Oh, I… I said, can you share?

558 00:44:34.920 ⇒ 00:44:45.160 Samuel Roberts: So I just heard you that. I was like, okay, yeah, yeah, let me, my bad, I was like, okay, so what I… so this is the metrics here that we were looking at for the last, like, two days, for example.

559 00:44:45.700 ⇒ 00:44:50.129 Samuel Roberts: So things are, like… This is a… I wonder if this was the…

560 00:44:51.770 ⇒ 00:44:54.959 Samuel Roberts: The 26th at 10pm, would that have been the concurrency test?

561 00:44:57.050 ⇒ 00:44:58.070 Pranav: That makes sense.

562 00:44:58.220 ⇒ 00:44:59.740 Mustafa Raja: Yeah, it…

563 00:44:59.740 ⇒ 00:45:04.790 Samuel Roberts: This is them yesterday getting a few, but this is… then this… the errors are what we saw.

564 00:45:07.700 ⇒ 00:45:10.249 Samuel Roberts: And the overall latency is also what we saw.

565 00:45:13.280 ⇒ 00:45:17.440 Samuel Roberts: Whereas, at this point, we didn’t see many errors.

566 00:45:18.940 ⇒ 00:45:24.709 Samuel Roberts: So I’m just having a hard time in my mind correlating that, and then when I go over to system, or quota and system limits.

567 00:45:25.540 ⇒ 00:45:26.310 Mustafa Raja: Yeah.

568 00:45:26.310 ⇒ 00:45:27.730 Samuel Roberts: We’re not near.

569 00:45:28.800 ⇒ 00:45:30.559 Samuel Roberts: Yeah, it doesn’t look like we’re near anything.

570 00:45:31.030 ⇒ 00:45:35.240 Mustafa Raja: Oh, yeah, the current usage percentage is very low.

571 00:45:35.570 ⇒ 00:45:41.620 Samuel Roberts: Yeah, so I don’t think we’re getting close to stuff, I just don’t know if it’s just a Google thing, where it’s like…

572 00:45:42.340 ⇒ 00:45:47.429 Samuel Roberts: You know, sometimes that happens when you’re, like, using cursor to try to hit a model, and there’s not much you can do about it, but…

573 00:45:49.100 ⇒ 00:45:49.900 Samuel Roberts: Yeah.

574 00:45:50.080 ⇒ 00:45:50.910 Samuel Roberts: Alright.

575 00:45:51.760 ⇒ 00:45:53.020 Pranav: I… so… I think…

576 00:45:53.510 ⇒ 00:45:54.180 Samuel Roberts: I think we’re here.

577 00:45:54.180 ⇒ 00:46:00.680 Pranav: One thing that I can think of, maybe this has nothing to do with it, is I believe when…

578 00:46:00.830 ⇒ 00:46:02.950 Pranav: Before we had this QA set up.

579 00:46:03.140 ⇒ 00:46:14.960 Pranav: Or maybe even… it was actually hitting this QA, like, whenever there would be a request directly to, like, the production Andy, it would also then pipe into the QA Andy, and maybe that happened at the same exact time.

580 00:46:14.960 ⇒ 00:46:24.850 Samuel Roberts: We did talk about that. I don’t… I think there’s probably some, but I don’t think that would have been… that doesn’t seem like it would have been enough concurrently to make this problem, in my mind.

581 00:46:24.850 ⇒ 00:46:26.749 Pranav: Yeah, I agree, yeah.

582 00:46:26.750 ⇒ 00:46:28.610 Samuel Roberts: The QA seemed like it was the bulk of it.

583 00:46:29.150 ⇒ 00:46:29.500 Pranav: Yup.

584 00:46:30.530 ⇒ 00:46:37.450 Samuel Roberts: So I would say I… let’s… let’s flip it, but here’s what we will not spin down NA to N yet.

585 00:46:39.060 ⇒ 00:46:40.790 Pranav: Yeah, in case we need to just flip that.

586 00:46:40.790 ⇒ 00:46:45.869 Samuel Roberts: already have been what we would have done anyway, because I don’t think, you know, we’re not ready to flip that switch until we know.

587 00:46:46.100 ⇒ 00:46:47.010 Samuel Roberts: everything’s running.

588 00:46:47.010 ⇒ 00:46:49.309 Pranav: Yeah, let’s give it, like, a week of them using it.

589 00:46:49.560 ⇒ 00:46:55.190 Samuel Roberts: Exactly, exactly. So I think we’re good to… Lip, then.

590 00:46:55.590 ⇒ 00:47:11.519 Pranav: Yeah, and then maybe, Sam, if… after we do this, if you just want to work on, like, a plan to just, like, in the case that, like, you know, this completely blows up, just for us to, like, switch back to N8in to have as little downtime for them, and then we can figure out things before we…

591 00:47:11.860 ⇒ 00:47:16.030 Pranav: do the cutover again. I think that would be great. And I’ll make a ticket right now for that.

592 00:47:16.560 ⇒ 00:47:27.100 Samuel Roberts: Okay, yeah, I think… and I’m gonna do some research into the actual limits in the vertex, and if we need to get Tim to help us move over there. But I feel like that’s not it, because it seems like we’re not even close to this stuff.

593 00:47:27.800 ⇒ 00:47:34.700 Samuel Roberts: But again, who knows? I’ve read different things about how people are experiencing this, so…

594 00:47:36.240 ⇒ 00:47:43.860 Samuel Roberts: Alright, so let’s go to… Maybe if you go to the Cloud Run… Brainforge, the backend invoker…

595 00:47:45.930 ⇒ 00:47:50.039 Samuel Roberts: And then, the current QA is the…

596 00:47:51.070 ⇒ 00:47:51.880 Mustafa Raja: Hmm.

597 00:47:53.290 ⇒ 00:47:55.590 Samuel Roberts: Sorry, I stopped, forgot, I stopped sharing.

598 00:48:00.370 ⇒ 00:48:06.380 Samuel Roberts: So I’ve got… I got… this one is where we want to change, right?

599 00:48:07.580 ⇒ 00:48:09.660 Samuel Roberts: This handle chat Python code.

600 00:48:09.660 ⇒ 00:48:10.690 Casie Aviles: Yes.

601 00:48:12.420 ⇒ 00:48:15.129 Samuel Roberts: Have we made other changes to this handler?

602 00:48:15.570 ⇒ 00:48:17.090 Samuel Roberts: Beyond just the URL?

603 00:48:19.460 ⇒ 00:48:22.740 Casie Aviles: Mmm… yeah, I think we… yeah, I believe we did.

604 00:48:22.740 ⇒ 00:48:29.630 Samuel Roberts: That’s… okay. So I basically want to… Go to… .

605 00:48:30.130 ⇒ 00:48:31.400 Casie Aviles: the development.

606 00:48:32.000 ⇒ 00:48:35.939 Samuel Roberts: And then there should be the chat handler dev.

607 00:48:36.840 ⇒ 00:48:39.289 Casie Aviles: Yeah, I think we just override it.

608 00:48:39.830 ⇒ 00:48:44.280 Samuel Roberts: Yep, that’s exactly what I was planning. Okay, good. I just want to make sure I had the right ones here. Okay.

609 00:48:44.590 ⇒ 00:48:50.670 Samuel Roberts: So we will, this is all pointing, let me just do a quick…

610 00:48:50.850 ⇒ 00:48:53.699 Samuel Roberts: Scan here, that is pointed to the right one…

611 00:48:54.870 ⇒ 00:48:59.300 Samuel Roberts: And this is all good… this is literally what the QA is right now, right?

612 00:49:03.700 ⇒ 00:49:04.400 Samuel Roberts: Alright.

613 00:49:05.770 ⇒ 00:49:08.659 Samuel Roberts: I’m going to take this, and we are going to edit it.

614 00:49:10.080 ⇒ 00:49:13.450 Samuel Roberts: Alright, and this is literally, I think, what makes the flip over, so are we… are we…

615 00:49:14.580 ⇒ 00:49:16.279 Samuel Roberts: Feel confident with this right now?

616 00:49:16.790 ⇒ 00:49:25.039 Casie Aviles: Let’s see… Let me just… I’m not sure if this has… If we can.

617 00:49:25.040 ⇒ 00:49:26.189 Samuel Roberts: Requirements?

618 00:49:27.130 ⇒ 00:49:31.080 Casie Aviles: Oh yeah, there are requirements, too. We might need to… We’ve made.

619 00:49:31.080 ⇒ 00:49:32.399 Samuel Roberts: Good call, could call.

620 00:49:32.400 ⇒ 00:49:33.519 Casie Aviles: Let’s see if we…

621 00:49:34.120 ⇒ 00:49:39.490 Samuel Roberts: 225330, 350, and request. Those are fine.

622 00:49:40.950 ⇒ 00:49:42.179 Samuel Roberts: Those look the same.

623 00:49:44.560 ⇒ 00:49:47.520 Samuel Roberts: Okay, and I mean, the revisions are here, so if I…

624 00:49:48.670 ⇒ 00:49:49.539 Casie Aviles: We can roll back.

625 00:49:49.910 ⇒ 00:49:51.859 Samuel Roberts: We should be able to…

626 00:49:53.440 ⇒ 00:49:58.050 Samuel Roberts: The revision is healthy, but not currently serving traffic. Yeah, we can reroute traff- manage traffic and stuff.

627 00:49:58.840 ⇒ 00:50:03.110 Samuel Roberts: This is the kind of stuff where I’m like, it’s powerful, and there’s just so much of it, compared to, like.

628 00:50:03.110 ⇒ 00:50:08.880 Pranav: Yeah, honestly, though, it seems, like, way less safe than railway.

629 00:50:09.170 ⇒ 00:50:16.500 Samuel Roberts: Well, that’s exactly why, like, Railway and Vercel and even Heroku exist, because, like, they’re just layers on top of this stuff.

630 00:50:16.500 ⇒ 00:50:17.430 Mustafa Raja: Yeah. Right.

631 00:50:18.030 ⇒ 00:50:22.259 Samuel Roberts: But you can break things so much easier here, and they, like, do a lot of the stuff for you.

632 00:50:22.380 ⇒ 00:50:22.939 Samuel Roberts: And then track.

633 00:50:22.940 ⇒ 00:50:23.340 Pranav: Yeah.

634 00:50:23.340 ⇒ 00:50:27.699 Samuel Roberts: So, like, it’s like, it’s one of those trade-offs, and this is the thing, like, GC… I feel a little…

635 00:50:27.910 ⇒ 00:50:31.989 Samuel Roberts: a little more comfortable in some other UIs than this one, even, but…

636 00:50:31.990 ⇒ 00:50:32.490 Pranav: Yeah.

637 00:50:32.490 ⇒ 00:50:40.239 Samuel Roberts: people… this is, like, famous for being the worst, like, console. And that’s saying something, because AWS is also ugly as shit.

638 00:50:40.240 ⇒ 00:50:40.870 Pranav: Yeah.

639 00:50:40.870 ⇒ 00:50:41.430 Samuel Roberts: Okay.

640 00:50:41.580 ⇒ 00:50:41.940 Pranav: It’s true.

641 00:50:41.940 ⇒ 00:50:43.400 Samuel Roberts: So, let’s…

642 00:50:43.400 ⇒ 00:50:43.890 Mustafa Raja: Just.

643 00:50:45.020 ⇒ 00:50:45.600 Mustafa Raja: I don’t…

644 00:50:45.600 ⇒ 00:50:45.930 Samuel Roberts: Sorry.

645 00:50:45.930 ⇒ 00:50:46.880 Mustafa Raja: Mmm…

646 00:50:51.410 ⇒ 00:50:52.030 Casie Aviles: Sorry?

647 00:50:52.900 ⇒ 00:51:01.140 Mustafa Raja: Yeah, I’m just wondering if we want to grab this one, or, us and the development chat one, because that is the one that’s being used, no?

648 00:51:03.750 ⇒ 00:51:06.769 Samuel Roberts: Oh, I thought it was this one. The chat one is the one that’s being used?

649 00:51:06.770 ⇒ 00:51:07.360 Casie Aviles: I, I thought…

650 00:51:07.360 ⇒ 00:51:13.450 Mustafa Raja: Oh, yeah, for sure, yeah, that just has, what’s it called, chat API, sorry, I thought the code.

651 00:51:13.450 ⇒ 00:51:14.849 Casie Aviles: Yeah, yeah.

652 00:51:15.600 ⇒ 00:51:16.170 Mustafa Raja: Yeah, we can…

653 00:51:16.170 ⇒ 00:51:17.529 Samuel Roberts: So you’re saying this one…

654 00:51:18.320 ⇒ 00:51:23.720 Casie Aviles: Yeah, it doesn’t have a chat handler, but it’s pointed to the… to the URL, right?

655 00:51:24.080 ⇒ 00:51:26.119 Mustafa Raja: Yeah, yeah, yeah, yeah, this is good, yeah, sorry.

656 00:51:26.590 ⇒ 00:51:27.479 Casie Aviles: Yeah.

657 00:51:27.800 ⇒ 00:51:29.069 Casie Aviles: Okay. All good.

658 00:51:29.830 ⇒ 00:51:30.580 Pranav: Oh, boy.

659 00:51:30.580 ⇒ 00:51:38.720 Samuel Roberts: The handler itself is fine. That’s just the entry point to this handler, to the code. It’s… yeah. Okay. GitKick, that’s fine. Okay. We’ll do this.

660 00:51:38.720 ⇒ 00:51:41.640 Pranav: And maybe let’s just send a few tests into…

661 00:51:41.770 ⇒ 00:51:45.800 Pranav: the… the Andy production, and then after that, we can…

662 00:51:46.010 ⇒ 00:51:47.639 Pranav: Send over my message to them.

663 00:51:51.380 ⇒ 00:51:54.229 Samuel Roberts: Yeah, so this is where we can flip it back, I think, if we need to. Okay, cool.

664 00:51:54.580 ⇒ 00:51:55.770 Pranav: Okay, cool.

665 00:51:57.680 ⇒ 00:51:58.460 Samuel Roberts: Something going on.

666 00:51:58.460 ⇒ 00:51:58.860 Pranav: Right.

667 00:51:58.860 ⇒ 00:51:59.570 Samuel Roberts: Of course.

668 00:52:01.750 ⇒ 00:52:06.309 Samuel Roberts: This is the scariest thing, doing this on a Friday. I love it. We’re living on the edge here.

669 00:52:06.970 ⇒ 00:52:08.509 Pranav: Oh my god.

670 00:52:08.820 ⇒ 00:52:09.240 Samuel Roberts: Yeah.

671 00:52:09.240 ⇒ 00:52:11.300 Pranav: I hate this process.

672 00:52:11.300 ⇒ 00:52:13.699 Samuel Roberts: Oh, I know, I do too, I do too, alright.

673 00:52:14.180 ⇒ 00:52:21.449 Samuel Roberts: Everyone saw me paste, you see the Monster app, I’m going to click this and hope no one’s literally in the middle of chatting with Andy right now, I guess?

674 00:52:21.880 ⇒ 00:52:23.130 Pranav: That’s fine.

675 00:52:23.690 ⇒ 00:52:25.190 Samuel Roberts: Yeah, alright, here we go.

676 00:52:25.390 ⇒ 00:52:27.960 Casie Aviles: Yeah, I’ll test this out as well.

677 00:52:29.010 ⇒ 00:52:29.350 Samuel Roberts: Okay.

678 00:52:29.350 ⇒ 00:52:30.270 Casie Aviles: presented to them.

679 00:52:30.720 ⇒ 00:52:32.969 Samuel Roberts: Yes, yes, yes, dude, please get ready.

680 00:52:43.290 ⇒ 00:52:47.999 Pranav: Yeah, in the future, if we ever have to do this, we’ll be like, okay, we need to do this after hours.

681 00:52:48.000 ⇒ 00:52:49.180 Samuel Roberts: 100%, 100%.

682 00:52:49.180 ⇒ 00:52:49.720 Pranav: Yeah.

683 00:52:50.590 ⇒ 00:52:51.209 Samuel Roberts: Like, the point.

684 00:52:51.210 ⇒ 00:52:52.910 Pranav: So, it’ll generally just be ready for that.

685 00:52:52.910 ⇒ 00:52:53.870 Samuel Roberts: on Monday.

686 00:52:54.520 ⇒ 00:52:59.600 Pranav: Yeah, so if Casey, like, tries to send, or if any of us try to send a message right now, it’ll just fail?

687 00:53:00.150 ⇒ 00:53:02.259 Samuel Roberts: I think it’s still going to the…

688 00:53:03.110 ⇒ 00:53:03.600 Casie Aviles: still good.

689 00:53:03.600 ⇒ 00:53:06.760 Samuel Roberts: The old one. It’s like Railway, where if a deploy fails, it still has the old.

690 00:53:06.760 ⇒ 00:53:07.210 Pranav: Oh, boy.

691 00:53:07.510 ⇒ 00:53:09.410 Samuel Roberts: I, I assume, like…

692 00:53:09.410 ⇒ 00:53:10.630 Mustafa Raja: Again.

693 00:53:11.220 ⇒ 00:53:13.919 Samuel Roberts: I can’t imagine why they’d do it any other way, is kind of my…

694 00:53:15.070 ⇒ 00:53:17.220 Pranav: Okay. Build completed.

695 00:53:17.220 ⇒ 00:53:27.469 Samuel Roberts: Rating revision, so that at some point, it will switch over, and traffic will get routed to the new one. But that… this is one nice thing, like, you can route traffic to different ones, so, like, Railway doesn’t have that, you know? Which…

696 00:53:27.470 ⇒ 00:53:28.130 Pranav: Yup.

697 00:53:28.130 ⇒ 00:53:34.260 Samuel Roberts: Some people need, some people don’t, but, like, I see where it’s powerful here. Okay, that looks like it’s deployed, it looks like all the traffic is set up.

698 00:53:34.800 ⇒ 00:53:35.859 Samuel Roberts: Give it a shot.

699 00:53:36.700 ⇒ 00:53:37.440 Pranav: Yeah.

700 00:53:41.730 ⇒ 00:53:43.060 Casie Aviles: Okay, yeah, it worked.

701 00:53:44.030 ⇒ 00:53:44.410 Samuel Roberts: Beautiful.

702 00:53:44.410 ⇒ 00:53:48.950 Pranav: Okay, did you… you want to test, like, some zip code questions, some central dock questions?

703 00:53:49.490 ⇒ 00:53:51.469 Casie Aviles: Sure, I can also share…

704 00:53:52.000 ⇒ 00:53:57.069 Samuel Roberts: Yeah, let me stop, because I want to double-check something else, too, that just popped in my head, and I forget why.

705 00:53:57.540 ⇒ 00:53:58.070 Casie Aviles: Tonight.

706 00:53:58.070 ⇒ 00:53:59.400 Samuel Roberts: That is good, that is good.

707 00:54:03.040 ⇒ 00:54:04.370 Samuel Roberts: Oops, didn’t even do that.

708 00:54:05.290 ⇒ 00:54:10.699 Pranav: And maybe let’s try to, like, test something that’s only in the new… In the new docs.

709 00:54:15.380 ⇒ 00:54:20.719 Pranav: So maybe, like, Mustafa, maybe, like, one of the prompts you’re using for the… the templates?

710 00:54:22.370 ⇒ 00:54:24.999 Mustafa Raja: Yeah… give me a moment.

711 00:54:25.600 ⇒ 00:54:26.160 Pranav: Yeah.

712 00:54:30.750 ⇒ 00:54:31.640 Casie Aviles: Okay.

713 00:54:31.840 ⇒ 00:54:33.770 Casie Aviles: That took a little longer, but…

714 00:54:33.770 ⇒ 00:54:39.300 Samuel Roberts: When I see the notification, it took, like, 21 seconds, because the… execute…

715 00:54:40.960 ⇒ 00:54:48.240 Samuel Roberts: with DB Agent on the flashlight preview took 16 seconds. This is what I’m talking about, like… Why?

716 00:54:52.140 ⇒ 00:54:54.400 Casie Aviles: Yeah, although it did return.

717 00:54:54.610 ⇒ 00:54:56.360 Casie Aviles: And so now, yeah, talk a little.

718 00:54:56.360 ⇒ 00:54:58.679 Samuel Roberts: I mean, like I said, like, it’s one of those things where…

719 00:54:59.240 ⇒ 00:55:01.949 Samuel Roberts: We have a lot more control now, so we can start to, like.

720 00:55:02.460 ⇒ 00:55:06.930 Samuel Roberts: tweak things as we need. If we need to move to Vertex, we can do that, but the…

721 00:55:07.450 ⇒ 00:55:11.310 Samuel Roberts: The improvement is still there, it’s just sometimes there’s these spikes, it seems.

722 00:55:12.870 ⇒ 00:55:13.640 Pranav: I’m good.

723 00:55:14.610 ⇒ 00:55:17.680 Casie Aviles: Are there any questions you guys want to test, or…

724 00:55:18.120 ⇒ 00:55:23.159 Pranav: Yeah, let’s do, like, a central dog question. Yeah.

725 00:55:27.080 ⇒ 00:55:29.679 Pranav: Let’s see, let me find one that Janiece posted.

726 00:55:30.370 ⇒ 00:55:31.230 Samuel Roberts: Yeah, perfect.

727 00:55:45.540 ⇒ 00:55:49.430 Mustafa Raja: Can we ask it for initial pest control templates?

728 00:55:50.140 ⇒ 00:55:50.690 Pranav: Yeah.

729 00:55:50.690 ⇒ 00:55:51.889 Samuel Roberts: Templates is a good one.

730 00:55:54.060 ⇒ 00:55:55.390 Pranav: This is perfect, yeah.

731 00:56:10.440 ⇒ 00:56:13.020 Mustafa Raja: Whoa, this is taking… Sometime.

732 00:56:17.880 ⇒ 00:56:18.859 Samuel Roberts: Don’t like…

733 00:56:18.860 ⇒ 00:56:19.500 Casie Aviles: Okay.

734 00:56:19.920 ⇒ 00:56:20.530 Casie Aviles: Oh, weird.

735 00:56:20.530 ⇒ 00:56:21.440 Samuel Roberts: seconds.

736 00:56:23.270 ⇒ 00:56:24.840 Samuel Roberts: Initial pest control.

737 00:56:27.360 ⇒ 00:56:28.180 Casie Aviles: room.

738 00:56:29.210 ⇒ 00:56:31.179 Casie Aviles: It returned these templates.

739 00:56:35.240 ⇒ 00:56:37.480 Mustafa Raja: Okay, this may be filtering to pest.

740 00:56:37.600 ⇒ 00:56:38.930 Mustafa Raja: document them.

741 00:56:39.680 ⇒ 00:56:42.000 Samuel Roberts: Oh, is that because we’re still… They haven’t resolved.

742 00:56:42.000 ⇒ 00:56:51.410 Mustafa Raja: Yeah, the, yeah, yeah, that, can we ask it to… Patch it from… Universal info only.

743 00:57:16.470 ⇒ 00:57:17.980 Pranav: Okay…

744 00:57:25.400 ⇒ 00:57:26.980 Pranav: Okay, this looks right.

745 00:57:29.970 ⇒ 00:57:31.540 Mustafa Raja: Do we have any other questions?

746 00:57:33.210 ⇒ 00:57:36.109 Pranav: No, I mean, I think this is good. I think we have, like.

747 00:57:36.220 ⇒ 00:57:49.389 Pranav: these are the main two things that we need to check, right? Just the new documents, the fact that I can pull from central docs, and then zip codes, but it is just taking a long time, like, I know, like, they’re… they’re gonna ask questions, they’re gonna be like, why is it taking.

748 00:57:49.390 ⇒ 00:57:53.330 Samuel Roberts: Yeah, I know, that’s… like, that’s not what we were seeing before, that’s why I’m like…

749 00:58:03.730 ⇒ 00:58:04.870 Samuel Roberts: 20.2…

750 00:58:10.220 ⇒ 00:58:14.640 Pranav: So I can just hold off on, like, giving them a, like, a thumbs up.

751 00:58:14.770 ⇒ 00:58:15.790 Pranav: on…

752 00:58:17.020 ⇒ 00:58:24.229 Pranav: if they ask me, I’ll be like, hey, yeah, we just completed it. We’re polishing up a few things, we’re noticing some delays, and…

753 00:58:24.450 ⇒ 00:58:25.810 Pranav: In response time?

754 00:58:25.960 ⇒ 00:58:28.900 Pranav: But it’s still ready to use.

755 00:58:30.020 ⇒ 00:58:35.299 Pranav: Sam, in the meantime, do you wanna kinda just do a spike on this?

756 00:58:35.540 ⇒ 00:58:36.050 Pranav: It’s like…

757 00:58:36.160 ⇒ 00:58:36.930 Samuel Roberts: Sunday.

758 00:58:36.930 ⇒ 00:58:42.310 Pranav: I know we kind of already did, and it’s a little bit confusing, but… we can…

759 00:58:43.020 ⇒ 00:58:45.190 Pranav: There has to be something that…

760 00:58:46.170 ⇒ 00:58:52.179 Pranav: is configurable, like you said, for us to, like, figure out, like, why this is taking so much time, or…

761 00:58:52.550 ⇒ 00:58:56.240 Pranav: maybe it’s something that Tim has to, like, do on his end.

762 00:58:56.810 ⇒ 00:59:01.860 Samuel Roberts: Yeah, I’ll do some digging and see what… I just… Really don’t. Yeah.

763 00:59:02.540 ⇒ 00:59:06.400 Samuel Roberts: understand, like, why I shouldn’t… I wasn’t, you know?

764 00:59:06.650 ⇒ 00:59:07.550 Casie Aviles: Yeah…

765 00:59:07.550 ⇒ 00:59:12.989 Pranav: Yeah, maybe it’s worth, like, doing a deep dive into the code, too. I feel like the code is probably not the…

766 00:59:13.230 ⇒ 00:59:18.910 Pranav: causing any bottleneck? I mean, there’s probably efficiency gains everywhere, but I don’t think that’s the main…

767 00:59:18.910 ⇒ 00:59:24.100 Samuel Roberts: Like, not at the scale, like, the model, like, you know, in the world of LLMs at this point, everything is…

768 00:59:25.170 ⇒ 00:59:28.779 Samuel Roberts: Right. Miscule compared to how long the responses can take.

769 00:59:29.250 ⇒ 00:59:30.000 Pranav: Yup.

770 00:59:35.540 ⇒ 00:59:42.860 Samuel Roberts: Let’s see, we go back for 1 hour here… Yeah, I’m seeing PigSpike there.

771 00:59:45.420 ⇒ 00:59:47.000 Samuel Roberts: 12, 20 minutes, better.

772 00:59:47.000 ⇒ 00:59:57.859 Pranav: If you guys stay on this meeting and figure that out, up to you. I have a… or if you guys want to, like, independently look into it and then get back onto a call, I have a meeting with Utam right now.

773 00:59:57.860 ⇒ 00:59:58.639 Samuel Roberts: I go for it.

774 00:59:59.030 ⇒ 01:00:02.400 Pranav: So… Yeah.

775 01:00:02.400 ⇒ 01:00:02.890 Mustafa Raja: on.

776 01:00:02.890 ⇒ 01:00:08.069 Pranav: Just, like, keep me… I’ll be on Slack, let me know if I can help in any way.

777 01:00:08.410 ⇒ 01:00:11.439 Pranav: But… Yeah, I think the main thing is, okay.

778 01:00:11.560 ⇒ 01:00:14.539 Pranav: This is something I think we’ll solve,

779 01:00:14.810 ⇒ 01:00:18.140 Pranav: the migration overall, though, I think went pretty well, like…

780 01:00:18.140 ⇒ 01:00:20.000 Samuel Roberts: Yeah, I mean, that’s all… like I said, it’s… it’s.

781 01:00:20.000 ⇒ 01:00:20.550 Pranav: up.

782 01:00:21.350 ⇒ 01:00:23.360 Samuel Roberts: Spiking bad, but…

783 01:00:24.740 ⇒ 01:00:29.239 Samuel Roberts: like, it was already doing that too, so, you know what I mean? Like, we can solve this, I feel like, more than we could have solved.

784 01:00:29.240 ⇒ 01:00:29.570 Pranav: Yeah.

785 01:00:29.570 ⇒ 01:00:32.049 Samuel Roberts: But these models, I just don’t know.

786 01:00:32.730 ⇒ 01:00:33.930 Samuel Roberts: What the hell?

787 01:00:34.850 ⇒ 01:00:41.080 Pranav: Yeah. And, like we said before, let’s keep N8N just, like, running in the background.

788 01:00:41.080 ⇒ 01:00:41.600 Samuel Roberts: Yeah.

789 01:00:42.060 ⇒ 01:00:50.090 Pranav: Also, can we stop the Andy QA? I just don’t want the trainers also routing traffic into there that could…

790 01:00:50.470 ⇒ 01:00:51.320 Pranav: You know.

791 01:00:52.450 ⇒ 01:00:53.450 Pranav: effect.

792 01:00:54.040 ⇒ 01:00:54.930 Casie Aviles: Yeah.

793 01:00:54.930 ⇒ 01:00:57.149 Pranav: The production and usage, so…

794 01:00:57.150 ⇒ 01:00:59.009 Samuel Roberts: Oh, yeah, yeah, good call.

795 01:00:59.340 ⇒ 01:01:00.460 Pranav: Yeah, let’s just halt it.

796 01:01:00.460 ⇒ 01:01:00.990 Casie Aviles: vote.

797 01:01:01.550 ⇒ 01:01:03.029 Pranav: Yeah, just disable that.

798 01:01:03.980 ⇒ 01:01:04.670 Samuel Roberts: Totally.

799 01:01:05.600 ⇒ 01:01:06.390 Pranav: Perfect.

800 01:01:07.090 ⇒ 01:01:09.370 Samuel Roberts: You got that, Casey, or should I do that? Are you in there?

801 01:01:09.370 ⇒ 01:01:10.590 Casie Aviles: Yeah, yeah, I can do it.

802 01:01:12.530 ⇒ 01:01:14.130 Pranav: Cool. Alright, guys.

803 01:01:14.990 ⇒ 01:01:17.480 Samuel Roberts: Alright. Well, yeah, we’ll dig into this.

804 01:01:18.470 ⇒ 01:01:19.200 Samuel Roberts: Perfect.

805 01:01:19.200 ⇒ 01:01:21.010 Pranav: Yeah, appreciate it, guys. I’ll talk soon.

806 01:01:21.010 ⇒ 01:01:22.059 Samuel Roberts: Alright, yep.

807 01:01:22.740 ⇒ 01:01:23.340 Pranav: See ya.

808 01:01:27.960 ⇒ 01:01:28.640 Samuel Roberts: Alright.

809 01:01:39.190 ⇒ 01:01:43.659 Samuel Roberts: So, when we were… Doing these tests…

810 01:01:47.680 ⇒ 01:01:49.390 Samuel Roberts: 15 seconds, Dane.

811 01:01:55.210 ⇒ 01:01:59.640 Samuel Roberts: Actually, I’m gonna have to put on flashlight preview was working before pretty well, though, right?

812 01:02:09.120 ⇒ 01:02:14.609 Samuel Roberts: And, like, not much has changed in terms of the actual… Prompts or anything, has it?

813 01:02:17.310 ⇒ 01:02:19.429 Samuel Roberts: Because we tweaked them a little bit today, right?

814 01:02:21.650 ⇒ 01:02:25.040 Casie Aviles: Alright, yeah, I did add just…

815 01:02:25.430 ⇒ 01:02:29.780 Casie Aviles: a couple of lines, so I don’t think that should be too major.

816 01:02:30.250 ⇒ 01:02:31.770 Samuel Roberts: No, I wouldn’t think so.

817 01:02:46.350 ⇒ 01:02:48.110 Samuel Roberts: See what else we can dig into here.

818 01:03:08.060 ⇒ 01:03:09.309 Samuel Roberts: Oh my goodness.

819 01:03:14.580 ⇒ 01:03:15.100 Mustafa Raja: What happened?

820 01:03:15.100 ⇒ 01:03:16.430 Casie Aviles: Did you find anything?

821 01:03:16.690 ⇒ 01:03:19.599 Samuel Roberts: No, no, I’m seeing the spike from the,

822 01:03:20.200 ⇒ 01:03:23.749 Samuel Roberts: Here, let me show you all this, screen share real quick.

823 01:03:25.110 ⇒ 01:03:30.900 Samuel Roberts: share… I was digging into the, like, limits and stuff.

824 01:03:32.100 ⇒ 01:03:34.350 Samuel Roberts: And I saw this spike, which I think was the concurrent.

825 01:03:34.350 ⇒ 01:03:35.320 Casie Aviles: testing, right?

826 01:03:35.960 ⇒ 01:03:39.890 Samuel Roberts: But that is where I would have expected things to break down then, you know?

827 01:03:41.540 ⇒ 01:03:53.879 Mustafa Raja: Yeah, and this pack doesn’t align with our peak usage at all, also. So you’d see that this is 1.7 million. Actually, this is above 3 million, but if you hover over it, it’s going to say 1.7 million.

828 01:03:54.200 ⇒ 01:03:58.819 Samuel Roberts: Yeah… I don’t know if there’s a better way to view this.

829 01:04:01.670 ⇒ 01:04:03.469 Samuel Roberts: the P&D, I don’t want the P&D, I want to…

830 01:04:04.550 ⇒ 01:04:06.809 Mustafa Raja: Yeah, I tried this, this is, this is…

831 01:04:06.810 ⇒ 01:04:09.140 Samuel Roberts: Oh, you did this already? Okay, yeah, what a weird…

832 01:04:11.810 ⇒ 01:04:14.999 Mustafa Raja: So, 1.3, but it is above 3, so…

833 01:04:15.720 ⇒ 01:04:16.220 Samuel Roberts: That’s…

834 01:04:16.220 ⇒ 01:04:18.620 Mustafa Raja: Very conflicting information.

835 01:04:19.110 ⇒ 01:04:20.440 Samuel Roberts: Google Man.

836 01:04:22.910 ⇒ 01:04:27.980 Mustafa Raja: And they should allow us to zoom into days that we need.

837 01:04:27.980 ⇒ 01:04:29.539 Samuel Roberts: Yeah, this one, this one doesn’t let you go.

838 01:04:29.540 ⇒ 01:04:38.669 Mustafa Raja: deeper. Yeah, I’m sure what’s happening is we may be, hovering… actually hovering over some other time than the spike itself.

839 01:04:38.820 ⇒ 01:04:44.860 Mustafa Raja: And then, what we should be able to do is Zoom and, you know… See what’s happening.

840 01:04:44.860 ⇒ 01:04:49.230 Samuel Roberts: Yeah, that’s the one… the one thing you can’t go, like, into the hour on, it looks like.

841 01:04:51.580 ⇒ 01:04:54.169 Samuel Roberts: Yeah, I can’t see anything token-related here.

842 01:04:55.520 ⇒ 01:04:56.300 Samuel Roberts: Oh my god.

843 01:05:02.310 ⇒ 01:05:03.170 Samuel Roberts: Like…

844 01:05:19.470 ⇒ 01:05:24.879 Samuel Roberts: Are we… we’re logging these execution times in real as well, right?

845 01:05:25.610 ⇒ 01:05:26.300 Casie Aviles: Yeah.

846 01:05:26.850 ⇒ 01:05:30.890 Samuel Roberts: But we’re not… we’re not… probably not breaking down the steps the way it is here in the workflow, right?

847 01:05:31.270 ⇒ 01:05:33.979 Casie Aviles: No, it’s just the overall.

848 01:05:34.450 ⇒ 01:05:36.089 Casie Aviles: Okay. Execution times.

849 01:05:41.240 ⇒ 01:05:43.659 Samuel Roberts: Is this the URL again? I can never remember this one.

850 01:05:43.960 ⇒ 01:05:45.690 Casie Aviles: Oh, yeah, I sent the Lincoln.

851 01:05:46.180 ⇒ 01:05:46.890 Samuel Roberts: Oh, did you?

852 01:05:46.890 ⇒ 01:05:48.849 Casie Aviles: Zoom chat, Zoom chat, yeah.

853 01:05:48.850 ⇒ 01:05:54.399 Samuel Roberts: Yeah, yeah, let me… hold on, I gotta find… there it is, okay, thank you. Oh, you just did, thank you.

854 01:05:55.480 ⇒ 01:05:57.040 Samuel Roberts: I just wanna see, like, is this…

855 01:06:02.560 ⇒ 01:06:08.229 Samuel Roberts: Like, we’re… part of it is we’re seeing all the bad ones. I want to make sure we’re at least getting some good ones, maybe? I don’t know.

856 01:06:34.630 ⇒ 01:06:37.540 Samuel Roberts: I wonder if the Monstra Studio would be helpful right now.

857 01:06:43.700 ⇒ 01:06:44.450 Samuel Roberts: That wasn’t it?

858 01:06:46.210 ⇒ 01:06:53.340 Samuel Roberts: to here… So if I go to the workflows, and I go to the CSR routing workflow…

859 01:06:56.820 ⇒ 01:06:58.170 Samuel Roberts: Failed one.

860 01:06:58.300 ⇒ 01:07:03.909 Samuel Roberts: Model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later. Yeah. Okay.

861 01:07:07.230 ⇒ 01:07:12.429 Samuel Roberts: This is… not good, but… Okay, so I just came in here to look at the workflow runs.

862 01:07:14.510 ⇒ 01:07:16.000 Casie Aviles: We’re still getting them.

863 01:07:16.480 ⇒ 01:07:22.110 Samuel Roberts: This is a… Literally 5.03 unavailable.

864 01:07:32.660 ⇒ 01:07:34.780 Casie Aviles: Oh yeah, that was my test earlier.

865 01:07:34.900 ⇒ 01:07:37.959 Samuel Roberts: That was your test, and it just… did it just fail, or did it just take forever?

866 01:07:38.820 ⇒ 01:07:40.769 Casie Aviles: It, it just failed here.

867 01:07:50.020 ⇒ 01:07:52.389 Samuel Roberts: Yeah. That’s not… hmm.

868 01:07:54.390 ⇒ 01:07:57.959 Samuel Roberts: I’m gonna do a little bit of digging here, I guess, and see if I can find anything else out.

869 01:08:00.300 ⇒ 01:08:02.889 Samuel Roberts: We’re using Gemini 3.1 Flashlight.

870 01:08:03.110 ⇒ 01:08:04.350 Samuel Roberts: Preview?

871 01:08:14.830 ⇒ 01:08:18.310 Samuel Roberts: Oh my god, yeah, a day ago, people were just complaining about it.

872 01:08:19.840 ⇒ 01:08:20.790 Casie Aviles: Oh, really?

873 01:08:21.189 ⇒ 01:08:22.889 Samuel Roberts: Yeah, here’s a Reddit post.

874 01:08:22.890 ⇒ 01:08:27.550 Mustafa Raja: Is it just an issue that… an ongoing issue right now?

875 01:08:28.210 ⇒ 01:08:31.619 Samuel Roberts: I don’t know, this was, like I said, 3 days ago, so this was…

876 01:08:33.979 ⇒ 01:08:38.470 Samuel Roberts: I’m trying to see if there’s any other, like, recent… as I search…

877 01:08:53.729 ⇒ 01:09:05.349 Casie Aviles: Yeah… That might be the case. That also happened with Lilo sometime before, but it was for the generative.

878 01:09:05.590 ⇒ 01:09:06.399 Casie Aviles: Yeah.

879 01:09:08.069 ⇒ 01:09:09.279 Samuel Roberts: You’re right, I wonder if, like.

880 01:09:10.909 ⇒ 01:09:13.019 Samuel Roberts: Gemini is down again, 12 hours ago.

881 01:09:13.490 ⇒ 01:09:14.390 Casie Aviles: -Oh.

882 01:09:14.750 ⇒ 01:09:16.189 Samuel Roberts: Jesus, ugh.

883 01:09:18.520 ⇒ 01:09:19.529 Samuel Roberts: Where is this?

884 01:09:19.710 ⇒ 01:09:21.669 Samuel Roberts: Where’s the status of this, actually?

885 01:09:25.200 ⇒ 01:09:32.849 Samuel Roberts: Status Google AI Studio. AI Studio… Partial outage, here it is.

886 01:09:33.490 ⇒ 01:09:38.279 Samuel Roberts: of a bitch. And yesterday, what time were they testing at? That’s February 20th. It’s all partial outages.

887 01:09:40.000 ⇒ 01:09:42.120 Samuel Roberts: Spend cap? What does that mean?

888 01:09:48.720 ⇒ 01:09:49.520 Samuel Roberts: Yeah.

889 01:09:50.050 ⇒ 01:09:51.100 Samuel Roberts: I’m sure this one used.

890 01:10:00.590 ⇒ 01:10:03.189 Samuel Roberts: Wait, is the spend cap the only thing not working? I wonder…

891 01:10:05.860 ⇒ 01:10:06.250 Casie Aviles: What is that?

892 01:10:06.250 ⇒ 01:10:08.260 Samuel Roberts: And mean that the spend cap isn’t working.

893 01:10:22.690 ⇒ 01:10:26.720 Casie Aviles: Hmm, but… Is that separate from the API?

894 01:10:26.830 ⇒ 01:10:28.429 Casie Aviles: They’re, they’re green.

895 01:10:28.430 ⇒ 01:10:31.660 Samuel Roberts: That’s why I’m trying to figure out, like, what that difference is, I don’t understand.

896 01:10:32.180 ⇒ 01:10:32.900 Casie Aviles: Yeah.

897 01:10:42.950 ⇒ 01:10:46.500 Samuel Roberts: I mean, that Reddit one seems to be… everyone’s like, yeah, of course it is.

898 01:10:48.260 ⇒ 01:10:49.490 Samuel Roberts: Son of a bitch.

899 01:10:50.260 ⇒ 01:10:52.130 Samuel Roberts: I guess we need the vertex, maybe.

900 01:10:58.150 ⇒ 01:11:00.490 Samuel Roberts: if that would fix it, I don’t know, you know?

901 01:11:07.120 ⇒ 01:11:08.290 Samuel Roberts: Crazy.

902 01:11:16.890 ⇒ 01:11:19.540 Samuel Roberts: Give them any constant 503 errors.

903 01:11:20.410 ⇒ 01:11:21.799 Samuel Roberts: One hour ago.

904 01:11:22.380 ⇒ 01:11:24.559 Samuel Roberts: Topic deleted by author, what the hell?

905 01:11:31.200 ⇒ 01:11:32.030 Samuel Roberts: Yeah.

906 01:11:32.250 ⇒ 01:11:34.949 Samuel Roberts: Someone literally just posted this an hour ago.

907 01:11:42.240 ⇒ 01:11:43.370 Samuel Roberts: That’s nut.

908 01:11:43.870 ⇒ 01:11:45.340 Samuel Roberts: Good.

909 01:11:51.120 ⇒ 01:11:55.070 Samuel Roberts: Can we fall back to another model, you think, or is that not gonna matter if it’s Gemini?

910 01:11:58.090 ⇒ 01:12:05.730 Casie Aviles: Yeah, I think I… hmm… No, I think it… it may work.

911 01:12:06.630 ⇒ 01:12:10.600 Casie Aviles: I did the same thing last time with the image gen.

912 01:12:11.660 ⇒ 01:12:12.119 Samuel Roberts: Oh, really?

913 01:12:12.120 ⇒ 01:12:19.840 Casie Aviles: I had it fall back to a different model. I think, yeah, the Nano Banana one.

914 01:12:20.210 ⇒ 01:12:21.430 Casie Aviles: Was having, like.

915 01:12:23.440 ⇒ 01:12:31.380 Casie Aviles: a lot of usage as well. Same 503 error, so we waited for a couple of hours until it died down, and…

916 01:12:31.630 ⇒ 01:12:32.440 Samuel Roberts: Okay.

917 01:12:34.010 ⇒ 01:12:35.059 Casie Aviles: But yeah, I did…

918 01:12:35.310 ⇒ 01:12:46.359 Casie Aviles: I was just… I’m just thinking, like, we have to… we won’t know, like, if there are any… if it will change the results, you know, if we switch to a fallback model.

919 01:12:47.700 ⇒ 01:12:53.880 Casie Aviles: Because we’ve already tested so much with… 3.1 flash.

920 01:12:54.600 ⇒ 01:12:55.120 Samuel Roberts: That’s true.

921 01:12:55.120 ⇒ 01:12:55.969 Casie Aviles: If the quality will.

922 01:12:55.970 ⇒ 01:12:56.510 Samuel Roberts: No, I hear you.

923 01:12:56.510 ⇒ 01:12:57.020 Casie Aviles: Good.

924 01:12:57.300 ⇒ 01:13:01.360 Samuel Roberts: Yeah, I don’t like just changing that if we… Haven’t tested it.

925 01:13:21.700 ⇒ 01:13:22.650 Casie Aviles: Hmm.

926 01:13:22.830 ⇒ 01:13:25.569 Casie Aviles: Do you think it would help if we…

927 01:13:25.830 ⇒ 01:13:33.129 Casie Aviles: if we try to use, I mean, just for locally or just testing, if we use our own API and see if we’re getting the same issue.

928 01:13:33.840 ⇒ 01:13:34.570 Casie Aviles: Or…

929 01:13:34.570 ⇒ 01:13:38.379 Samuel Roberts: Do we have a Gemini API?

930 01:13:39.320 ⇒ 01:13:40.810 Casie Aviles: I think so.

931 01:13:43.390 ⇒ 01:13:45.309 Casie Aviles: Yeah, let me look for it.

932 01:13:56.670 ⇒ 01:13:57.300 Samuel Roberts: Hmm.

933 01:14:01.980 ⇒ 01:14:03.459 Casie Aviles: Yeah, we, we do.

934 01:14:04.100 ⇒ 01:14:04.710 Samuel Roberts: Okay.

935 01:14:09.030 ⇒ 01:14:12.030 Casie Aviles: I can. Oh, I can probably test it.

936 01:14:12.470 ⇒ 01:14:16.659 Casie Aviles: If we get the same issue, so it’s probably Google-wide, if that’s the case.

937 01:14:16.660 ⇒ 01:14:17.689 Samuel Roberts: It seems it, yeah.

938 01:14:17.690 ⇒ 01:14:19.170 Casie Aviles: Or… We’re funny.

939 01:14:19.170 ⇒ 01:14:22.379 Samuel Roberts: How we handle… what’s the best way to handle this for them?

940 01:14:23.450 ⇒ 01:14:24.160 Casie Aviles: Yeah.

941 01:14:26.510 ⇒ 01:14:31.459 Samuel Roberts: I guess we need to test on another model, maybe? So that we can have a safe fallback?

942 01:14:33.070 ⇒ 01:14:33.770 Casie Aviles: Okay.

943 01:14:33.950 ⇒ 01:14:36.439 Casie Aviles: Yeah, that’s… that’s an option that we have.

944 01:14:36.440 ⇒ 01:14:42.930 Samuel Roberts: I th… yeah, I think, I think that’s the only…

945 01:14:45.300 ⇒ 01:14:51.569 Samuel Roberts: I mean, but again, if it’s a Google thing, like, who knows if it’s gonna affect the other models. Like, it might be fine, it might not, but I think that’s the only…

946 01:14:52.050 ⇒ 01:14:53.580 Samuel Roberts: Safe way to…

947 01:14:58.900 ⇒ 01:15:03.230 Samuel Roberts: Okay, yeah, let’s, let’s do that. Let’s leave it as it is.

948 01:15:03.990 ⇒ 01:15:08.389 Samuel Roberts: Do we get notices of, like, full-on failures to the logs?

949 01:15:12.520 ⇒ 01:15:14.150 Samuel Roberts: long execution times.

950 01:15:15.520 ⇒ 01:15:21.410 Casie Aviles: I’ve only… I’m not too sure. I only know that we have, like, yeah, just the execution times.

951 01:15:21.850 ⇒ 01:15:22.500 Samuel Roberts: Okay.

952 01:15:23.430 ⇒ 01:15:28.340 Samuel Roberts: By all means for… Phenomenos to let them know that, like, Google’s having issues.

953 01:15:28.660 ⇒ 01:15:30.300 Samuel Roberts: But we can…

954 01:15:35.570 ⇒ 01:15:41.069 Casie Aviles: Yeah, they’re, they’re getting… Very low execution times.

955 01:16:12.600 ⇒ 01:16:17.470 Samuel Roberts: I mean, it does… it is working, though, on some of them, when they’re not erroring, you know?

956 01:16:18.600 ⇒ 01:16:21.959 Samuel Roberts: Watching it work in the studio, it’s just taking a while, yeah.

957 01:16:23.810 ⇒ 01:16:25.790 Samuel Roberts: execute with vector agent is spinning.

958 01:16:27.630 ⇒ 01:16:29.740 Samuel Roberts: 13, 14… sorry, what?

959 01:16:31.660 ⇒ 01:16:33.969 Mustafa Raja: Need to drop to attend a call.

960 01:16:36.920 ⇒ 01:16:37.640 Casie Aviles: Dary?

961 01:16:38.730 ⇒ 01:16:40.279 Samuel Roberts: I think he had an issue, I don’t know.

962 01:16:40.280 ⇒ 01:16:42.289 Casie Aviles: Yeah, he had to drop, I think.

963 01:16:42.700 ⇒ 01:16:44.879 Samuel Roberts: Oh, okay, I didn’t catch what he said. Okay.

964 01:16:49.110 ⇒ 01:16:56.060 Samuel Roberts: I mean, I imagine… The long execution times and the 503s are… Related.

965 01:16:59.200 ⇒ 01:17:04.610 Casie Aviles: Yeah, in the first day of testing, we didn’t have that.

966 01:17:04.840 ⇒ 01:17:06.450 Casie Aviles: issue. It was…

967 01:17:06.810 ⇒ 01:17:14.839 Casie Aviles: Well, we would get logs, but it would just be a few milliseconds past 5, and then around 7 to 8, yeah.

968 01:17:14.840 ⇒ 01:17:17.219 Samuel Roberts: Yeah, that’s… that’s well within what I think.

969 01:17:17.450 ⇒ 01:17:18.680 Samuel Roberts: is reasonable.

970 01:17:19.000 ⇒ 01:17:20.770 Samuel Roberts: To expect at this point.

971 01:17:22.570 ⇒ 01:17:23.380 Samuel Roberts: Okay.

972 01:17:26.050 ⇒ 01:17:31.480 Samuel Roberts: I think, yeah, let’s… what are the other…

973 01:17:32.050 ⇒ 01:17:34.670 Samuel Roberts: Models that we have to test with.

974 01:17:38.980 ⇒ 01:17:41.370 Casie Aviles: Hmm, let’s see… I think there’s…

975 01:17:43.400 ⇒ 01:17:47.260 Casie Aviles: 2, I think, Gemini2Flash, is that the name?

976 01:17:48.810 ⇒ 01:17:49.420 Samuel Roberts: Okay.

977 01:17:49.770 ⇒ 01:17:53.729 Casie Aviles: 2.0, yeah. Gemini 2.0 Flash, I think we had that.

978 01:17:54.760 ⇒ 01:17:56.210 Casie Aviles: Without this.

979 01:17:56.440 ⇒ 01:17:58.140 Casie Aviles: If we could do a call.

980 01:18:04.930 ⇒ 01:18:10.850 Samuel Roberts: What did he say here, Gemini 3.1 Pro, 3 Flash 3.1 Flashlight. Did we get 3 flash?

981 01:18:13.000 ⇒ 01:18:13.550 Casie Aviles: Hmm.

982 01:18:13.550 ⇒ 01:18:19.119 Samuel Roberts: you know, these names, I don’t know the Gemini models nearly as well, but I feel the OpenAI and Cloud will…

983 01:18:21.610 ⇒ 01:18:24.659 Casie Aviles: I’ll check, I’ll check these models that we have.

984 01:18:25.620 ⇒ 01:18:29.200 Samuel Roberts: Yeah, if you can… Do that.

985 01:18:58.370 ⇒ 01:18:59.500 Casie Aviles: Just send.

986 01:19:13.520 ⇒ 01:19:14.289 Samuel Roberts: So it’s not.

987 01:19:26.710 ⇒ 01:19:27.910 Samuel Roberts: Of course it’s not.

988 01:19:28.190 ⇒ 01:19:30.649 Samuel Roberts: linking it properly to my 1Password app.

989 01:19:47.580 ⇒ 01:19:49.030 Samuel Roberts: Try this again now, it’s all.

990 01:20:12.290 ⇒ 01:20:13.760 Samuel Roberts: I mean, I suppose…

991 01:20:14.890 ⇒ 01:20:15.460 Casie Aviles: Yeah, so that works.

992 01:20:16.270 ⇒ 01:20:21.020 Samuel Roberts: Sorry? Oh, there you go, thank you. I’m still trying to get my password to load.

993 01:20:21.910 ⇒ 01:20:25.350 Samuel Roberts: Whoa, this is everything that we have access to through their GCP?

994 01:20:26.430 ⇒ 01:20:29.449 Casie Aviles: Yeah, I sent this request.

995 01:20:32.320 ⇒ 01:20:32.970 Samuel Roberts: Okay.

996 01:20:34.260 ⇒ 01:20:38.030 Casie Aviles: I sent this, and then the API key at the end.

997 01:20:38.680 ⇒ 01:20:40.020 Samuel Roberts: Oh, cool, okay.

998 01:20:51.540 ⇒ 01:20:53.989 Samuel Roberts: What is BO? Is that the video one?

999 01:20:54.300 ⇒ 01:20:56.060 Casie Aviles: Yeah, I think that’s a video one.

1000 01:20:56.390 ⇒ 01:20:57.080 Samuel Roberts: Okay.

1001 01:21:01.250 ⇒ 01:21:01.980 Samuel Roberts: Okay.

1002 01:21:05.090 ⇒ 01:21:09.560 Samuel Roberts: Yeah, Gemini 3 Flash Preview might be a good one to try, the one before 3.1, maybe?

1003 01:21:12.710 ⇒ 01:21:15.720 Samuel Roberts: Yeah. There’s also a 2.5 ring.

1004 01:21:20.930 ⇒ 01:21:26.789 Samuel Roberts: 2.5 flash, 2.5 flashlight… Yeah, I mean, we might want to…

1005 01:21:28.520 ⇒ 01:21:32.660 Samuel Roberts: I don’t think what’s the most systemic way to test a few of these as fallbacks.

1006 01:21:37.170 ⇒ 01:21:42.729 Samuel Roberts: I mean, we have a list of… we have some questions we can do some concurrency tests with, or tests like we did the concurrency with, right?

1007 01:21:46.040 ⇒ 01:21:47.159 Casie Aviles: I’ll… I’ll ask.

1008 01:21:47.310 ⇒ 01:21:48.400 Casie Aviles: Mustafa.

1009 01:21:48.810 ⇒ 01:21:56.609 Samuel Roberts: Yeah, even, even, I mean, I’m sure we have, like, some sample questions that we can run through different models and see how they compare like you were doing for the,

1010 01:21:56.820 ⇒ 01:21:58.529 Samuel Roberts: Before and after N8M stuff.

1011 01:21:59.610 ⇒ 01:22:06.710 Casie Aviles: Oh, yeah, yeah. That was a little limited. I didn’t test, like, all of the models. It was mainly.

1012 01:22:06.710 ⇒ 01:22:11.020 Samuel Roberts: But it’s something like that, if those questions, those similar kinds of questions.

1013 01:22:11.220 ⇒ 01:22:12.709 Casie Aviles: Yeah, yeah, we do.

1014 01:22:13.270 ⇒ 01:22:14.230 Samuel Roberts: Okay, yeah.

1015 01:22:14.730 ⇒ 01:22:21.270 Samuel Roberts: Yeah, if we could do that, and maybe… Yeah, testing with these different…

1016 01:22:21.580 ⇒ 01:22:26.469 Samuel Roberts: a few different models and just see which ones work best as a fallback, I suppose?

1017 01:22:29.450 ⇒ 01:22:32.430 Samuel Roberts: I would say, yeah, 3… I see.

1018 01:22:33.260 ⇒ 01:22:40.040 Samuel Roberts: We’re using… 3.1 flash… light.

1019 01:22:40.750 ⇒ 01:22:41.890 Samuel Roberts: Preview.

1020 01:22:45.570 ⇒ 01:22:52.300 Samuel Roberts: There’s 3 flash preview… Google’s naming convention is…

1021 01:22:54.560 ⇒ 01:22:58.620 Samuel Roberts: And then maybe 2.5 flash, and… Flashlight.

1022 01:23:00.490 ⇒ 01:23:01.130 Casie Aviles: Okay.

1023 01:23:01.830 ⇒ 01:23:04.109 Samuel Roberts: I would say give those a try and just see if they’re…

1024 01:23:04.550 ⇒ 01:23:06.899 Samuel Roberts: You know, capable of handling the fallback.

1025 01:23:07.970 ⇒ 01:23:12.159 Samuel Roberts: If they respond decently. I mean, what we might need to do is up it to a…

1026 01:23:13.760 ⇒ 01:23:16.340 Samuel Roberts: Smarter, slower model for the fallback?

1027 01:23:17.450 ⇒ 01:23:20.940 Samuel Roberts: like, an older, but not the Flash one kind of thing.

1028 01:23:21.800 ⇒ 01:23:27.629 Samuel Roberts: I don’t know how well those old ones, if they’re too fast, they might not be smart enough. I don’t know, we may have to just do a few tests that way.

1029 01:23:28.790 ⇒ 01:23:29.430 Casie Aviles: Okay.

1030 01:23:34.540 ⇒ 01:23:41.590 Samuel Roberts: Yeah, because if it’s gonna be a long time anyway, we might, you know… We definitely wanna…

1031 01:23:41.820 ⇒ 01:23:43.660 Samuel Roberts: Test those other ones. Okay.

1032 01:23:44.590 ⇒ 01:23:55.629 Samuel Roberts: Yeah, I think Gemini 3 flash preview, and maybe… 2.5… Lash light, I guess?

1033 01:23:58.270 ⇒ 01:24:00.480 Samuel Roberts: Is there a 2.5 flash? I don’t see.

1034 01:24:02.590 ⇒ 01:24:06.619 Samuel Roberts: Oh, there’s 2.5 flash at the very top, okay. Yeah, maybe give those ones a try.

1035 01:24:12.310 ⇒ 01:24:13.680 Samuel Roberts: As a fallback.

1036 01:24:14.020 ⇒ 01:24:15.080 Samuel Roberts: Does that work?

1037 01:24:15.760 ⇒ 01:24:20.099 Casie Aviles: Yes, yeah, I’ll try… I’ll try those out.

1038 01:24:22.750 ⇒ 01:24:27.349 Samuel Roberts: Alright, yeah, do you want me to… I can throw a linear ticket in there, too.

1039 01:24:54.950 ⇒ 01:25:05.939 Samuel Roberts: We are gonna call this… Doing cycle, I’m gonna assign this to you… That seems right… okay.

1040 01:25:08.260 ⇒ 01:25:17.610 Samuel Roberts: Change that go ahead. And then I’ll say, which ones did I just say? 2.5 flashed…

1041 01:25:18.510 ⇒ 01:25:22.709 Samuel Roberts: I mean, I’m not super familiar with these models, which is not… fun, but…

1042 01:25:23.210 ⇒ 01:25:24.270 Casie Aviles: Yeah, I do.

1043 01:25:26.300 ⇒ 01:25:29.130 Samuel Roberts: 2.5… Flashlight…

1044 01:25:38.430 ⇒ 01:25:45.600 Samuel Roberts: And then there’s a 3… Flash preview… I guess.

1045 01:25:45.750 ⇒ 01:25:51.599 Samuel Roberts: and a… I don’t know how different the Pro and the Flash are, but…

1046 01:25:52.630 ⇒ 01:25:57.379 Samuel Roberts: Let’s just try these flash ones first and see if they’re good, because the time is vertical here.

1047 01:25:58.940 ⇒ 01:25:59.700 Samuel Roberts: Okay.

1048 01:26:00.360 ⇒ 01:26:04.419 Samuel Roberts: I’m gonna create this ticket for you.

1049 01:26:05.330 ⇒ 01:26:06.910 Samuel Roberts: And then, okay, Sylvia.

1050 01:26:07.150 ⇒ 01:26:15.510 Samuel Roberts: I would say try to run at least a few basic test questions on those models and see if they handle it, and then we can go in and add that as a fallback.

1051 01:26:17.040 ⇒ 01:26:18.650 Casie Aviles: Okay, yeah, that sounds good.

1052 01:26:19.040 ⇒ 01:26:20.650 Samuel Roberts: Okay, cool, thank you.

1053 01:26:21.440 ⇒ 01:26:22.900 Samuel Roberts: Let me know how that goes.

1054 01:26:23.720 ⇒ 01:26:26.689 Casie Aviles: Okay, I’ll just let you guys know.

1055 01:26:28.670 ⇒ 01:26:29.510 Casie Aviles: Alright.

1056 01:26:31.000 ⇒ 01:26:31.570 Samuel Roberts: Okay.

1057 01:26:32.610 ⇒ 01:26:34.060 Samuel Roberts: Thanks, Casey.

1058 01:26:34.480 ⇒ 01:26:35.389 Casie Aviles: Thank you, Sam.

1059 01:26:35.690 ⇒ 01:26:37.360 Samuel Roberts: Let me know if you need anything from me for that.

1060 01:26:38.190 ⇒ 01:26:38.830 Casie Aviles: Sure.

1061 01:26:39.370 ⇒ 01:26:41.039 Samuel Roberts: Alright, good advice, talk to you later.

1062 01:26:42.550 ⇒ 01:26:43.600 Samuel Roberts: Bye.

Brainforge Knowledge

Explorer

2026-03-27_ai_service_office_hours_block_2248c070

Graph View