Meeting Title: AI Service Standup Date: 2026-01-19 Meeting participants: Uttam Kumaran, Mustafa Raja, Casie Aviles, Pranav Narahari, Samuel Roberts, Clarence Stone
WEBVTT
1 00:02:25.800 ⇒ 00:02:26.300 Mustafa Raja: Hey.
2 00:02:27.630 ⇒ 00:02:28.490 Uttam Kumaran: Hey!
3 00:04:24.050 ⇒ 00:04:26.479 Uttam Kumaran: Mustafi, am I messaging Sam?
4 00:04:27.800 ⇒ 00:04:31.249 Uttam Kumaran: And… worst case, we can get started. Anyways…
5 00:04:56.190 ⇒ 00:04:59.640 Mustafa Raja: Sam might be in another meeting, in the wrong meeting.
6 00:04:59.920 ⇒ 00:05:00.450 Uttam Kumaran: Oh, really?
7 00:05:00.890 ⇒ 00:05:03.480 Mustafa Raja: Yeah, let me share the link.
8 00:05:16.550 ⇒ 00:05:18.209 Samuel Roberts: That was very weird.
9 00:05:19.810 ⇒ 00:05:25.690 Samuel Roberts: My granola notification sent me a different link, I guess. I was sitting on another call, waiting for everyone. I was like, oh, why isn’t
10 00:05:26.950 ⇒ 00:05:28.539 Samuel Roberts: I’m the one at sleep.
11 00:05:28.540 ⇒ 00:05:29.240 Uttam Kumaran: All good.
12 00:05:29.240 ⇒ 00:05:29.900 Samuel Roberts: Everyone.
13 00:05:31.010 ⇒ 00:05:31.770 Uttam Kumaran: A…
14 00:05:34.250 ⇒ 00:05:35.779 Samuel Roberts: Everyone have a good weekend?
15 00:05:37.030 ⇒ 00:05:37.620 Mustafa Raja: Yeah.
16 00:05:37.620 ⇒ 00:05:42.719 Uttam Kumaran: Yeah, weekend was good. Had a friend visiting from New York, so it just…
17 00:05:43.160 ⇒ 00:05:45.390 Uttam Kumaran: Went to dinner, and had some…
18 00:05:45.880 ⇒ 00:05:50.680 Uttam Kumaran: Like, it’s… I don’t know if anyone watches football, but it’s, like, football playoffs right now, so…
19 00:05:50.680 ⇒ 00:05:53.179 Samuel Roberts: Oh yeah, yeah, I was watching the Pats yesterday.
20 00:05:53.650 ⇒ 00:05:55.050 Uttam Kumaran: Yeah, some good games.
21 00:05:55.050 ⇒ 00:05:55.920 Samuel Roberts: Yeah.
22 00:05:56.890 ⇒ 00:06:01.660 Samuel Roberts: Yeah, this is… I’m not, like, the biggest football fan, but playoffs I get excited for.
23 00:06:02.340 ⇒ 00:06:11.040 Samuel Roberts: Yeah, I’m the same way. I’m the same way. I tend to watch towards the end of the season. Exactly, exactly. I got… I did that with basketball last couple of seasons ago, and got really into it.
24 00:06:11.040 ⇒ 00:06:12.900 Uttam Kumaran: Are you a… are you a Cleveland fan?
25 00:06:13.000 ⇒ 00:06:14.350 Uttam Kumaran: For, like, the number of things?
26 00:06:14.350 ⇒ 00:06:18.660 Samuel Roberts: It’s complicated, so I’m more of a Boston fan than anything.
27 00:06:19.230 ⇒ 00:06:21.190 Samuel Roberts: But I’ve been engaged Almost 20.
28 00:06:21.190 ⇒ 00:06:22.359 Pranav Narahari: No, no, no.
29 00:06:22.360 ⇒ 00:06:23.890 Uttam Kumaran: Boohoo!
30 00:06:26.300 ⇒ 00:06:34.110 Samuel Roberts: So, like, I’m gonna root for any Boston teams, but I will also root for the Cavs or the Guardians now. I don’t really care about the Browns, so…
31 00:06:35.760 ⇒ 00:06:40.510 Uttam Kumaran: Celtics are good, Pats are, like… Anyway, keep winning.
32 00:06:40.510 ⇒ 00:06:41.799 Samuel Roberts: Crazy, yeah.
33 00:06:41.800 ⇒ 00:06:44.320 Uttam Kumaran: Totally overhauled the team, and they’re, like, back.
34 00:06:44.470 ⇒ 00:06:53.919 Samuel Roberts: It’s amazing what they did, yeah. I was, like, barely following the very beginning of the season, and all of a sudden I checked the standings, like, a month or two ago. I was just like, oh, I should be paying attention to them again.
35 00:06:55.540 ⇒ 00:06:56.170 Samuel Roberts: Fuck.
36 00:06:56.170 ⇒ 00:06:56.990 Uttam Kumaran: Nice.
37 00:06:58.240 ⇒ 00:07:01.669 Samuel Roberts: Oh, alright, well, are we missing anyone?
38 00:07:02.740 ⇒ 00:07:07.699 Samuel Roberts: Do we know if Gabe’s joining today, or… it’s early for Amber, so we’ll see if she pops on.
39 00:07:08.020 ⇒ 00:07:09.420 Uttam Kumaran: I’m not sure about Cade.
40 00:07:09.560 ⇒ 00:07:10.250 Samuel Roberts: Okay.
41 00:07:10.420 ⇒ 00:07:11.699 Uttam Kumaran: But we can get started.
42 00:07:11.700 ⇒ 00:07:17.210 Samuel Roberts: Yeah, so I guess, maybe let’s talk Lilo, in case Amber shows up. We’ll save you a good team.
43 00:07:19.780 ⇒ 00:07:29.210 Samuel Roberts: Yeah, update from me, I was trying to… I’m still working on some of the plan for, like, deployment strategy stuff, especially for the PRs.
44 00:07:29.340 ⇒ 00:07:44.280 Samuel Roberts: Cursor’s helping me put together, like, a list of scripts and everything that we can run to pull down the right environment variables, which I think will be helpful, not only for us, but if they’re gonna be in Cloud Code, they’ll have some easy scripts to run, and we’ll document all that.
45 00:07:44.520 ⇒ 00:07:51.720 Samuel Roberts: I was trying to do it Friday, but I don’t know if anyone was in Railway or Pranav, if you saw, like, the builds were, like, just spinning.
46 00:07:52.180 ⇒ 00:07:58.640 Samuel Roberts: it took, like, 45 minutes for the build to finally realize it wasn’t working.
47 00:07:58.640 ⇒ 00:07:59.179 Pranav Narahari: Weird, okay.
48 00:07:59.180 ⇒ 00:08:00.030 Uttam Kumaran: Oh, weird.
49 00:08:00.030 ⇒ 00:08:18.340 Samuel Roberts: Yeah, I think everything was working in terms of, like, railway, but I think the build environment, something went wrong, and I couldn’t even cancel it, I tried to redeploy, I just had, like, 3 builds going at once, and nothing was happening, and that was, like, the end of the day Friday, and I was like, well, I’m not gonna sit here waiting for all that, I’ll finish it Monday. But it seemed to have cleaned itself up, so I’m gonna…
50 00:08:18.510 ⇒ 00:08:20.190 Samuel Roberts: Hopefully get that sorted today.
51 00:08:20.620 ⇒ 00:08:22.689 Samuel Roberts: So the idea…
52 00:08:22.750 ⇒ 00:08:24.150 Uttam Kumaran: It’s very high level.
53 00:08:24.150 ⇒ 00:08:39.459 Samuel Roberts: quickly is we’ll have the production environment, which will be production. We’ll have the staging environment that’ll basically mirror that most of the time, except for when we’re testing new features. We can duplicate down to staging if we need to from production for the database, so everything will work
54 00:08:39.600 ⇒ 00:08:42.030 Samuel Roberts: for the MCPs,
55 00:08:42.309 ⇒ 00:08:47.580 Samuel Roberts: Then dev will be a little more flexible. We’ll still probably copy, like, staging down into dev.
56 00:08:47.730 ⇒ 00:08:54.269 Samuel Roberts: But what we’ll do is we’ll spin up the PR environments, we’ll use the dev database and the DevMCP servers by default.
57 00:08:54.670 ⇒ 00:08:57.710 Samuel Roberts: I think we might have talked about this Friday, I’m not sure, but…
58 00:08:58.210 ⇒ 00:09:02.779 Samuel Roberts: I just… the coordination of all that with the environment variables is a little bit of a…
59 00:09:03.350 ⇒ 00:09:04.270 Samuel Roberts: It’s not a trivial…
60 00:09:04.270 ⇒ 00:09:08.110 Uttam Kumaran: And the decision there is that, like, you would be…
61 00:09:08.570 ⇒ 00:09:13.339 Uttam Kumaran: testing, like, any changes in dev database, you want to reflect that up.
62 00:09:13.600 ⇒ 00:09:18.859 Uttam Kumaran: And then, basically, Yeah. Make the, and then… yeah, go ahead.
63 00:09:18.860 ⇒ 00:09:22.210 Samuel Roberts: Yeah, so basically, like, the… we’ll make… when the PR is made to…
64 00:09:22.390 ⇒ 00:09:39.860 Samuel Roberts: staging, from, like, a feature branch or something. It’ll spin up this environment, we can test it, make sure it’s, like, not breaking anything, merging the staging, do more QA, make sure it’s not breaking everything even further, but the PR thing is just like the Heroku thing, where if you want to share a feature before you’re really done with it.
65 00:09:40.330 ⇒ 00:09:51.919 Samuel Roberts: And not merge that into staging, right? So we could even merge into, like, a dev branch if we want, but I think, like, just having a dev branch that we keep pulling back down into staging to keep it in sync is good.
66 00:09:52.020 ⇒ 00:09:55.670 Samuel Roberts: Data-wise, all the migrations should be handled with Prisma.
67 00:09:56.790 ⇒ 00:10:02.510 Samuel Roberts: So, as long as we’re… like, managing the PR as well, everything should be pretty fine there.
68 00:10:02.640 ⇒ 00:10:03.650 Samuel Roberts: Okay.
69 00:10:04.180 ⇒ 00:10:12.909 Samuel Roberts: Yeah, it’s pretty good about, you know, when the schema changes, you just regenerate, or you generate a new migration, and it just goes into the migrations folder, and it can go up and down that way.
70 00:10:13.210 ⇒ 00:10:14.469 Uttam Kumaran: Oh, nice. Great.
71 00:10:15.000 ⇒ 00:10:27.210 Samuel Roberts: So, yeah, it’s pretty good. I tried to use it one time with Superbase, and that was a little more complicated, but this is nice, because it’s just a Postgres database underneath, so Prisma’s doing all the work there.
72 00:10:27.210 ⇒ 00:10:27.580 Uttam Kumaran: Cool.
73 00:10:27.580 ⇒ 00:10:33.389 Samuel Roberts: The reason we have to mirror everything to dev is so that all the environment variables and all the keys that make the MCP servers work.
74 00:10:33.670 ⇒ 00:10:36.239 Samuel Roberts: Are still useful when we’re testing.
75 00:10:36.530 ⇒ 00:10:41.920 Samuel Roberts: Otherwise, we just get blank… Database and have to reconnect everything.
76 00:10:42.290 ⇒ 00:10:47.810 Samuel Roberts: Or seed it all, which still means we have to manage all these secrets somewhere, so I figured this is the best way to do it.
77 00:10:48.170 ⇒ 00:10:51.390 Samuel Roberts: Especially if they’re going to be developing it, too.
78 00:10:53.080 ⇒ 00:11:10.039 Samuel Roberts: But yeah, that’s my update there. Hoping to get more of it done Friday, but like I said, I ran into that build issue, but Hersc has got a nice plan that I’m ready to execute on. I’m just still tweaking it a little bit to make sure everything’s the way we want it, and then hopefully by the end of the day, we’ll have a good set of scripts, and I can put together, like, a good AgentsMD file for it, so…
79 00:11:11.700 ⇒ 00:11:12.730 Samuel Roberts: Cool. Cool.
80 00:11:13.460 ⇒ 00:11:16.259 Samuel Roberts: I guess, Pranav, Casey, how we doing?
81 00:11:16.560 ⇒ 00:11:17.030 Pranav Narahari: Yeah.
82 00:11:17.030 ⇒ 00:11:18.060 Samuel Roberts: Any other updates?
83 00:11:18.240 ⇒ 00:11:25.989 Pranav Narahari: On my end, right after this meeting, I have a meeting with Bobby and Zach to get them set up with Clyde Code.
84 00:11:26.150 ⇒ 00:11:28.419 Pranav Narahari: So I’ll mention to them, like.
85 00:11:29.160 ⇒ 00:11:35.599 Pranav Narahari: probably we can figure out through Slack later, but about, like, the… maybe, like, the agent’s MD file might be changing to, like, help.
86 00:11:35.600 ⇒ 00:11:36.100 Samuel Roberts: Yeah.
87 00:11:36.100 ⇒ 00:11:40.639 Pranav Narahari: deploy process, or the PR process, I mean.
88 00:11:41.050 ⇒ 00:11:41.520 Samuel Roberts: Yeah.
89 00:11:41.520 ⇒ 00:11:44.629 Pranav Narahari: That shouldn’t be too complex for them, I think.
90 00:11:44.840 ⇒ 00:11:47.159 Pranav Narahari: What you can do…
91 00:11:47.160 ⇒ 00:11:50.070 Uttam Kumaran: is you can also set them up on GitHub Desktop.
92 00:11:50.960 ⇒ 00:11:56.710 Uttam Kumaran: And you can just show them, like, they can pull down the latest changes.
93 00:11:56.970 ⇒ 00:11:59.259 Uttam Kumaran: from Maine, ever so often.
94 00:12:01.710 ⇒ 00:12:05.029 Pranav Narahari: Yeah. Would that be less complex for them?
95 00:12:06.540 ⇒ 00:12:12.579 Uttam Kumaran: I guess, I don’t know, I haven’t done Cloud Code yet, so I’m not sure yet, like, how it pulls code down from the repo.
96 00:12:12.920 ⇒ 00:12:13.450 Samuel Roberts: Yeah, I mean.
97 00:12:13.450 ⇒ 00:12:18.190 Uttam Kumaran: I guess they could also literally say, Pull down the latest code.
98 00:12:18.690 ⇒ 00:12:20.930 Pranav Narahari: Yeah, I think that much easier.
99 00:12:20.930 ⇒ 00:12:21.910 Uttam Kumaran: Okay, okay, okay.
100 00:12:21.910 ⇒ 00:12:36.569 Pranav Narahari: I’m trying to set them up with just, like, just, like, kind of commands through Claude Code, so, like, they can use natural language, like, basically the entire time and not move around as much. And so I’m trying to make it set up in a way where, like.
101 00:12:36.860 ⇒ 00:12:39.330 Pranav Narahari: It’s, like, the most deterministic it can be.
102 00:12:39.590 ⇒ 00:12:41.590 Uttam Kumaran: Yeah, yeah, okay, okay, okay, okay, great.
103 00:12:41.590 ⇒ 00:12:50.859 Samuel Roberts: Yeah, I think… I think that’s fine, because they don’t really need to understand the ins and outs of Git too well. The only thing I want to make sure we do, if we’re going to let them go, kind of.
104 00:12:51.150 ⇒ 00:13:00.060 Samuel Roberts: hog wild with Cloud Code, is we will add the rules, I think I mentioned this in Slack or something, to staging into production.
105 00:13:00.330 ⇒ 00:13:13.659 Samuel Roberts: Yeah. So that no one can just push to it. I might have to get them to enable that on the org, but I’ll look at that today, too. Because then they can do whatever they want, as long as Cloud Code can’t just, like, merge or delete a branch on remote, we’re fine, so…
106 00:13:13.890 ⇒ 00:13:22.189 Pranav Narahari: Yeah, let me… I can probably look into that right after this meeting, so I can just, like, maybe have that set up before they’re, you know, messing around in clogged code.
107 00:13:22.400 ⇒ 00:13:30.870 Samuel Roberts: Yeah, actually, if you’re gonna be on with them, if they’re the ones who own the org, I think they might have to enable those rules, so go take a look at the rules for production and staging, at least.
108 00:13:31.100 ⇒ 00:13:33.640 Pranav Narahari: Perfect, yeah, yeah, I’ll get that set up then, by the end of the day.
109 00:13:33.640 ⇒ 00:13:41.360 Samuel Roberts: Yeah, let me know if you have any questions. I was digging into it the other day, so let me know if there’s anything. Okay, cool. I think we basically just want to make it so that you need to PR, and you need an approval from, like.
110 00:13:41.840 ⇒ 00:13:44.860 Samuel Roberts: I guess me, or you, or someone’s willing to make sure it’s not…
111 00:13:45.130 ⇒ 00:13:47.320 Samuel Roberts: Just them merging things on their own.
112 00:13:47.320 ⇒ 00:13:52.610 Pranav Narahari: Yeah, I’ll make it set up so, like, either of us can approve. Yeah, okay, sounds good.
113 00:13:53.210 ⇒ 00:13:58.649 Pranav Narahari: And then, yeah, so technically, like, today is the start of Phase 2.
114 00:13:59.060 ⇒ 00:14:03.030 Pranav Narahari: So… we…
115 00:14:03.150 ⇒ 00:14:10.180 Pranav Narahari: And then we also have the nana banana stuff, so I’ll let Casey talk about that, we should get that by Friday. Yeah, let’s talk that through.
116 00:14:10.330 ⇒ 00:14:15.020 Pranav Narahari: Yeah. But other than that, I’ll create the linear tickets for…
117 00:14:15.090 ⇒ 00:14:28.029 Pranav Narahari: what we need to do for, like, this week, and our next, like, milestone is 3 weeks from now, so not… so, like, yeah, 3 Fridays from now. So, yeah, we have a good amount of time, yeah, I feel like we’re… we’re…
118 00:14:28.030 ⇒ 00:14:37.729 Pranav Narahari: pretty on track. There’s some stuff with, Slack reports, I think, that still maybe need to be refined.
119 00:14:38.250 ⇒ 00:14:42.899 Pranav Narahari: And so, I’ll just basically, like, wrap that up this week, while also working on Phase 2.
120 00:14:45.740 ⇒ 00:14:49.849 Casie Aviles: Yeah, so for the Nano Banana app, I’ve…
121 00:14:50.480 ⇒ 00:14:57.399 Casie Aviles: connect… I’ve deployed the one I had on my branch, so I made changes to both the back end and the front end.
122 00:14:57.580 ⇒ 00:14:59.920 Casie Aviles: Okay. So that should be up on dev.
123 00:15:00.220 ⇒ 00:15:01.720 Casie Aviles: Yep, alright.
124 00:15:01.990 ⇒ 00:15:07.230 Casie Aviles: Yeah, I ended up using the buckets for storing the images.
125 00:15:07.690 ⇒ 00:15:12.519 Casie Aviles: And then I also added a new table in Postgres for…
126 00:15:12.800 ⇒ 00:15:13.430 Samuel Roberts: Perfect.
127 00:15:13.890 ⇒ 00:15:21.740 Casie Aviles: I think, yeah, there’s some logic going on there that needs, like, keys, or, like, being able to differentiate which store
128 00:15:22.000 ⇒ 00:15:25.990 Casie Aviles: We’re generating from, so… Yeah, made changes to the…
129 00:15:26.160 ⇒ 00:15:27.939 Casie Aviles: So that should be on dev.
130 00:15:28.170 ⇒ 00:15:32.239 Casie Aviles: I guess, before we move it to staging, I just want…
131 00:15:32.360 ⇒ 00:15:35.490 Casie Aviles: another look at it, in case… Yeah. Yeah.
132 00:15:36.230 ⇒ 00:15:37.249 Samuel Roberts: Yeah, I’ll take a look today.
133 00:15:37.250 ⇒ 00:15:38.800 Casie Aviles: Anything? Okay.
134 00:15:38.800 ⇒ 00:15:41.760 Samuel Roberts: See how that looks, and I’ll test it out as well.
135 00:15:41.910 ⇒ 00:15:47.400 Samuel Roberts: Is it all… is it all, like, working on the dev environment? Like, all the keys and everything are fine?
136 00:15:47.890 ⇒ 00:15:48.410 Samuel Roberts: H.
137 00:15:48.410 ⇒ 00:15:51.180 Casie Aviles: Yeah, so… there should be, like…
138 00:15:51.760 ⇒ 00:16:00.679 Casie Aviles: in the integrations settings, you should be able to, like, input the Gemini key, which is… I added it on 1Pass as well.
139 00:16:01.000 ⇒ 00:16:01.619 Samuel Roberts: Yeah, great.
140 00:16:02.000 ⇒ 00:16:03.660 Samuel Roberts: Yep, I’ll take a look.
141 00:16:04.650 ⇒ 00:16:04.970 Casie Aviles: Okay.
142 00:16:04.970 ⇒ 00:16:05.660 Samuel Roberts: some feedback there.
143 00:16:05.660 ⇒ 00:16:06.180 Casie Aviles: Thank you.
144 00:16:06.360 ⇒ 00:16:12.580 Samuel Roberts: Thank you, yeah, that’s good. I saw the deployments when I was in Railway this morning, so I figured you were… you looked like you were working hard on it, so…
145 00:16:12.730 ⇒ 00:16:13.599 Samuel Roberts: Thank you.
146 00:16:15.100 ⇒ 00:16:18.169 Samuel Roberts: Alright, cool. Any other thoughts on Lilo?
147 00:16:21.360 ⇒ 00:16:27.490 Samuel Roberts: Alright, cool. Yeah, I think we’re on a good track. They seem pretty happy, and I think getting them in the code will make them…
148 00:16:27.850 ⇒ 00:16:33.929 Samuel Roberts: Pretty happy too, so… Alright, let’s jump to Andy.
149 00:16:37.210 ⇒ 00:16:39.359 Samuel Roberts: How are you feeling there?
150 00:16:39.800 ⇒ 00:16:44.399 Casie Aviles: Yeah, I think I’ll just follow up on feedback. I think…
151 00:16:44.890 ⇒ 00:16:49.250 Casie Aviles: I need to… well, if I… I’m just going to…
152 00:16:50.550 ⇒ 00:16:58.629 Casie Aviles: try out or test, like, the database, if I can get that live already, since it’s gonna be blocking a lot of, like.
153 00:16:59.410 ⇒ 00:17:02.960 Casie Aviles: the triage tickets coming in? I guess,
154 00:17:04.310 ⇒ 00:17:17.270 Casie Aviles: Yeah, there’s… there’s going to… there’s a lot, like, coming in about the zip codes, and while the thing… while the admin thing, or the new database is not live yet, it’s gonna keep piling up.
155 00:17:17.460 ⇒ 00:17:19.019 Samuel Roberts: Okay. So I need to…
156 00:17:19.319 ⇒ 00:17:22.600 Casie Aviles: Yeah, I need to confirm with them.
157 00:17:24.400 ⇒ 00:17:29.010 Casie Aviles: Yeah, I think that that’s about it for… And D.
158 00:17:29.270 ⇒ 00:17:34.210 Casie Aviles: I think what we’re going to be working on next would be evals.
159 00:17:34.580 ⇒ 00:17:35.880 Samuel Roberts: Yes.
160 00:17:35.930 ⇒ 00:17:37.980 Casie Aviles: Yeah, at least for this week, so…
161 00:17:39.060 ⇒ 00:17:39.690 Mustafa Raja: Yeah.
162 00:17:40.500 ⇒ 00:17:48.199 Mustafa Raja: Yeah, so for evals, we talked, we talked on Friday, well, I talked on Friday with Sam.
163 00:17:49.900 ⇒ 00:18:00.149 Mustafa Raja: And we… I’m just going to locally set up, or configure, the current, evaluation dataset with,
164 00:18:00.710 ⇒ 00:18:05.589 Mustafa Raja: With the architecture that we have, with the workflows, and see how it performs.
165 00:18:06.160 ⇒ 00:18:09.479 Mustafa Raja: That’s pretty much it. I’ll be working on that today.
166 00:18:11.430 ⇒ 00:18:17.409 Samuel Roberts: Yes, so yeah, so, Uten, for your… for your reference, we figured… we found Monstra has, like, pretty good eval, like.
167 00:18:17.580 ⇒ 00:18:32.059 Samuel Roberts: It has some scores built in, but you can add your own, and some of them are pretty good for just matching against, like, the golden dataset, and it has a way to run, like, batch evals. So, we’ll probably use that ongoing to maybe store some scores
168 00:18:32.110 ⇒ 00:18:39.359 Samuel Roberts: From there, but also, like, Mustafa can just take that data set, run that locally on the master agent, and see how it compares.
169 00:18:39.480 ⇒ 00:18:42.260 Samuel Roberts: to the thumbs up, thumbs down stuff.
170 00:18:42.560 ⇒ 00:18:45.079 Samuel Roberts: Rather than set something up with blanket, is it…
171 00:18:45.600 ⇒ 00:18:50.619 Samuel Roberts: even if we were… even if Blackbeard hadn’t been bought, and who knows what’s gonna happen there,
172 00:18:51.210 ⇒ 00:19:00.189 Samuel Roberts: It’s still, like, it’s a little weird, because it’s, like, a workflow in Mastra, it’s not just a single prompt and output. It may not pair up as well, but I think Mastra is pretty set up for this, which is good.
173 00:19:00.960 ⇒ 00:19:01.570 Uttam Kumaran: Right.
174 00:19:02.830 ⇒ 00:19:06.679 Samuel Roberts: Cool, and then, I think Pranav, you…
175 00:19:07.180 ⇒ 00:19:15.819 Samuel Roberts: We’re supposed to be looking at the, metrics, the evaluation, like, metrics, so you may want to chat with Mustafa today.
176 00:19:15.820 ⇒ 00:19:16.470 Pranav Narahari: Yup.
177 00:19:17.270 ⇒ 00:19:24.830 Pranav Narahari: Sounds good. Yeah, I’m gonna write some stuff up in Notion, and then we can kinda discuss that Mustafa in another meeting, if you’d like.
178 00:19:25.680 ⇒ 00:19:32.219 Samuel Roberts: Yeah, there’s some… there’s some basic stuff there where we’re just getting feedback from them on whether or not the responses were good, thumbs up, thumbs down.
179 00:19:32.220 ⇒ 00:19:34.969 Pranav Narahari: Then there’s some, like.
180 00:19:34.970 ⇒ 00:19:36.270 Samuel Roberts: How long it took.
181 00:19:36.480 ⇒ 00:19:40.229 Samuel Roberts: to actually run, like, we want to focus on getting Andy to run faster.
182 00:19:40.380 ⇒ 00:19:42.159 Samuel Roberts: And then…
183 00:19:42.160 ⇒ 00:19:42.770 Pranav Narahari: Okay.
184 00:19:42.770 ⇒ 00:19:46.519 Samuel Roberts: Yeah, just kind of run with it from there, think of other things that we want to make sure we evaluate.
185 00:19:47.260 ⇒ 00:19:51.640 Pranav Narahari: Gotcha. In, that CSV that’s,
186 00:19:51.820 ⇒ 00:20:00.749 Pranav Narahari: Mustafa, I think, or Casey, that you added to the ticket, do we have, like, running times on there as a column?
187 00:20:01.110 ⇒ 00:20:03.100 Mustafa Raja: Yeah,
188 00:20:03.390 ⇒ 00:20:14.379 Mustafa Raja: I don’t know, I don’t know this history. I’ll link you a Google Sheets sheet that, with… that’s the evaluation dataset that we’ll be working on.
189 00:20:14.640 ⇒ 00:20:16.210 Mustafa Raja: That’s more relevant to me.
190 00:20:16.650 ⇒ 00:20:17.490 Pranav Narahari: Yep.
191 00:20:18.320 ⇒ 00:20:23.950 Pranav Narahari: Okay, that works. Yeah, I might be looking at a different dataset then, but yeah, if you can send me over that, that’d be great.
192 00:20:25.300 ⇒ 00:20:29.960 Casie Aviles: Yeah, I think the one I sent you, Pranav, was just the raw export from
193 00:20:30.140 ⇒ 00:20:32.399 Casie Aviles: Oh, yeah, there is an execution.
194 00:20:32.400 ⇒ 00:20:35.319 Samuel Roberts: I was gonna say, if it’s just the raw one, you definitely should have it, yeah.
195 00:20:35.690 ⇒ 00:20:37.819 Pranav Narahari: Okay, perfect, yeah. I’ll,
196 00:20:38.260 ⇒ 00:20:57.090 Pranav Narahari: I’ll think about that as well, then. I was just thinking about, like, thumbs up, thumbs down, and, like, the feedback. But yeah, I didn’t take into consideration the execution time, so… okay. I’ll, I’ll have something written up in, Notion. Yeah, Mustafa, just let me know, like, when you want to set up some time.
197 00:20:57.690 ⇒ 00:20:58.320 Mustafa Raja: Yeah.
198 00:20:58.320 ⇒ 00:21:02.290 Samuel Roberts: I would also say take a look at some of the stuff that Mazra has built in for scores.
199 00:21:02.550 ⇒ 00:21:05.290 Pranav Narahari: If there’s any, like, low-hanging fruit there that’s, like…
200 00:21:05.290 ⇒ 00:21:06.909 Samuel Roberts: worth including.
201 00:21:07.510 ⇒ 00:21:08.360 Pranav Narahari: Okay.
202 00:21:08.360 ⇒ 00:21:22.789 Samuel Roberts: there was, like, obviously there’s a similarity thing that we’re gonna be doing for the golden data set, but there were some other things that were, like, is this even a good answer to the question? Like, does this match the… like, you know, it’s obviously, like, LM is a judge kind of stuff, but it’s still, like, is this the right…
203 00:21:23.010 ⇒ 00:21:26.639 Samuel Roberts: type of answer, or things like that. Yeah. So take a look at those.
204 00:21:26.910 ⇒ 00:21:27.490 Pranav Narahari: Okay.
205 00:21:28.930 ⇒ 00:21:36.300 Samuel Roberts: Cool. And then, I’m trying to think if there’s anything else ABC-wise… Am I missing anything here?
206 00:21:36.980 ⇒ 00:21:37.650 Casie Aviles: Oh, I do.
207 00:21:37.650 ⇒ 00:21:38.180 Samuel Roberts: Hmm.
208 00:21:38.180 ⇒ 00:21:40.570 Casie Aviles: Just one question I had. Yeah.
209 00:21:40.780 ⇒ 00:21:44.520 Casie Aviles: For… for the BigQuery access, we’re all good there, right?
210 00:21:44.910 ⇒ 00:21:45.770 Mustafa Raja: Oh, yeah.
211 00:21:46.180 ⇒ 00:21:48.420 Casie Aviles: Okay, so, yeah, I’ll just…
212 00:21:48.420 ⇒ 00:21:53.969 Mustafa Raja: Civic account, we have a read and write and write permissions, and that’s what we could…
213 00:21:54.610 ⇒ 00:21:55.170 Samuel Roberts: Okay, great.
214 00:21:55.170 ⇒ 00:21:56.480 Casie Aviles: Okay, thank you.
215 00:21:58.460 ⇒ 00:22:00.990 Samuel Roberts: Cool. Yeah, I was just responding to…
216 00:22:00.990 ⇒ 00:22:05.200 Casie Aviles: Yvette’s message, asking about the status, so…
217 00:22:05.650 ⇒ 00:22:06.310 Samuel Roberts: Perfect.
218 00:22:07.820 ⇒ 00:22:08.690 Samuel Roberts: Alright.
219 00:22:09.420 ⇒ 00:22:11.069 Samuel Roberts: Any other ABC thoughts?
220 00:22:13.800 ⇒ 00:22:14.530 Samuel Roberts: Cool.
221 00:22:14.720 ⇒ 00:22:20.280 Samuel Roberts: Alright, I guess without game, we’re good on internal, unless anyone knows anything.
222 00:22:21.060 ⇒ 00:22:28.229 Samuel Roberts: what his current status was. I… we had a little bit of stuff with the Google Workspace MCP,
223 00:22:29.000 ⇒ 00:22:30.570 Samuel Roberts: It’s hard to get that running.
224 00:22:31.500 ⇒ 00:22:35.120 Samuel Roberts: With… there was some authentication loop that he was having issues with running in.
225 00:22:35.120 ⇒ 00:22:46.799 Uttam Kumaran: Yeah, I can follow up… I can follow up with him. I think there’s a couple of… now that we’re learning a little bit about, like, MCPs and, like, interactions Slack, there’s a couple things that the sales team is doing.
226 00:22:46.900 ⇒ 00:22:49.899 Uttam Kumaran: using HubSpot to, like, try to, like, for example.
227 00:22:50.070 ⇒ 00:23:06.110 Uttam Kumaran: one common task is, like, 15 days, 30 days, 60 days before a renewal, send a note into Slack. They’re using Zapier, I think, and basically sending a list of all the clients, and…
228 00:23:06.270 ⇒ 00:23:10.640 Uttam Kumaran: It’s, like, more, there’s just, like.
229 00:23:10.870 ⇒ 00:23:18.750 Uttam Kumaran: using Zapier to get all the clients and format a message. I think we can probably, honestly, just transition some of that to MCP.
230 00:23:19.070 ⇒ 00:23:25.219 Uttam Kumaran: So maybe I’m gonna… I’m just gonna have them, like, send a note in the AI team channel. They can use some help.
231 00:23:25.550 ⇒ 00:23:30.719 Samuel Roberts: Okay. It’s Ryan and Rico that are setting some of those up. Okay.
232 00:23:31.470 ⇒ 00:23:34.689 Uttam Kumaran: Yeah, and I think… I think what we’re gonna try to do is…
233 00:23:34.970 ⇒ 00:23:40.260 Uttam Kumaran: Get the… like, I’m gonna try this week a little bit more to see if I can get the Brain Forge bought.
234 00:23:40.640 ⇒ 00:23:50.620 Uttam Kumaran: like, deployed, and then folks can use the bot in multiple different ways, right? So, as long as it’s deployed, we can have it do a bunch of different workflows.
235 00:23:51.040 ⇒ 00:23:53.910 Uttam Kumaran: So that’s stuff on the internal side.
236 00:23:55.760 ⇒ 00:24:04.010 Uttam Kumaran: Similarly, I was gonna try to work on the… Zoom, Superbase, to GitHub workflow.
237 00:24:04.160 ⇒ 00:24:07.879 Uttam Kumaran: I guess my question there is, like, are we landing everything?
238 00:24:08.040 ⇒ 00:24:11.349 Uttam Kumaran: in S3, and then moving it to Superbase?
239 00:24:11.500 ⇒ 00:24:13.759 Uttam Kumaran: I forgot what the flow was.
240 00:24:15.170 ⇒ 00:24:18.290 Samuel Roberts: I believe so, yeah, I think the… go ahead, Casey, you probably know this better.
241 00:24:18.980 ⇒ 00:24:27.609 Casie Aviles: Yeah, it’s transferring the… what do you call this? The files, the videos, to S3, and then…
242 00:24:28.110 ⇒ 00:24:32.490 Casie Aviles: For, like, the metadata and then the transcript itself, it’s in Supabase.
243 00:24:33.630 ⇒ 00:24:34.270 Uttam Kumaran: Okay.
244 00:24:34.550 ⇒ 00:24:38.920 Uttam Kumaran: So what I could probably do is, like, Have some type of batch.
245 00:24:39.470 ⇒ 00:24:43.820 Uttam Kumaran: from Supabase to… GitHub.
246 00:24:44.240 ⇒ 00:24:46.970 Uttam Kumaran: And I run that through GitHub Action.
247 00:24:47.950 ⇒ 00:24:49.960 Uttam Kumaran: to move transcripts to the vault.
248 00:24:50.370 ⇒ 00:24:55.620 Uttam Kumaran: Because right now, one of the things that’s a bit manual is… People are,
249 00:24:56.370 ⇒ 00:24:59.750 Uttam Kumaran: Like, uncopying it and putting… committing it to repos.
250 00:24:59.860 ⇒ 00:25:03.819 Uttam Kumaran: Versus… I think I’m just gonna see if I can write something that does that.
251 00:25:04.110 ⇒ 00:25:05.970 Uttam Kumaran: I guess, alternatively.
252 00:25:06.340 ⇒ 00:25:13.280 Uttam Kumaran: like, I mean, one problem is I just don’t want this to live in N8N or windmill, so alternatively.
253 00:25:13.500 ⇒ 00:25:15.820 Uttam Kumaran: When I… wherever this, like, master…
254 00:25:15.970 ⇒ 00:25:19.269 Uttam Kumaran: sort of, like, S3 to wherever script ends up living.
255 00:25:19.800 ⇒ 00:25:22.300 Uttam Kumaran: We could weave this in later, I guess.
256 00:25:24.680 ⇒ 00:25:29.950 Samuel Roberts: Yeah, you could also… we could also do something in Supabase with, like, a onInsert update.
257 00:25:30.610 ⇒ 00:25:31.390 Uttam Kumaran: Oh, okay.
258 00:25:31.390 ⇒ 00:25:32.949 Samuel Roberts: do something, I don’t know.
259 00:25:33.450 ⇒ 00:25:38.900 Uttam Kumaran: Well, what I can do is I can… I can see how… I can see if… if, like, I can get Tupa Bass to send a,
260 00:25:39.010 ⇒ 00:25:40.360 Uttam Kumaran: A webhook?
261 00:25:40.360 ⇒ 00:25:40.950 Samuel Roberts: Yeah.
262 00:25:41.970 ⇒ 00:25:44.160 Uttam Kumaran: And then I’ll… That could be totally fine.
263 00:25:44.460 ⇒ 00:25:47.709 Samuel Roberts: Yeah, webhook from Superbase, or a function on Superbase that might…
264 00:25:48.130 ⇒ 00:25:49.060 Uttam Kumaran: Oh, god, yeah.
265 00:25:49.060 ⇒ 00:25:59.710 Samuel Roberts: do it right into the repo through GitHub or something, I don’t know. I’m not sure what GitHub has, like, available, it’s not just the CLI. I’m sure there’s an API you could probably tap into somehow with a function on Superbase.
266 00:26:00.150 ⇒ 00:26:01.659 Uttam Kumaran: Okay, so I can take a look at that.
267 00:26:01.660 ⇒ 00:26:03.540 Samuel Roberts: Yeah, there’s a few different ways you could probably do it.
268 00:26:03.910 ⇒ 00:26:13.709 Uttam Kumaran: And then, Sam, I have a… I don’t know if I’m… I maybe need to… this may even before you joined, or right when you started. There’s this company we were discussing with called Plan Medicare.
269 00:26:14.270 ⇒ 00:26:17.310 Uttam Kumaran: It’s like a Medicare…
270 00:26:17.580 ⇒ 00:26:28.899 Uttam Kumaran: agency, and I send you the SOW we put together for him. I have a call with him again on Thursday, wondering if you’d like to be there with me, and… Sure.
271 00:26:28.900 ⇒ 00:26:29.280 Samuel Roberts: Yeah.
272 00:26:29.280 ⇒ 00:26:40.309 Uttam Kumaran: I think he basically, when we first sent it, they were, like, kind of getting into busy season, because enrollment for Medicare, I think, just finished. So, he wants to kind of talk again, so…
273 00:26:40.630 ⇒ 00:26:43.050 Uttam Kumaran: Yeah, me and you can kind of probably tackle that and…
274 00:26:43.200 ⇒ 00:26:48.530 Uttam Kumaran: I feel like I need to go review what the SOW was again, but… Yeah, and okay, yeah.
275 00:26:48.910 ⇒ 00:26:50.690 Uttam Kumaran: Yeah, we could do that on Thursday.
276 00:26:52.710 ⇒ 00:26:53.260 Samuel Roberts: Okay.
277 00:26:53.460 ⇒ 00:26:56.970 Samuel Roberts: That sounds good. Yeah, where did… where would I find that right now? Where was that…
278 00:26:56.970 ⇒ 00:27:02.619 Uttam Kumaran: I will send that to you. I think it’s gonna be in drive, but… Okay. Yeah, I need to… I’ll send it to you.
279 00:27:03.160 ⇒ 00:27:03.720 Samuel Roberts: Okay.
280 00:27:04.460 ⇒ 00:27:12.519 Samuel Roberts: Sounds good. Oh, alright. I’m getting a call. I have to, jump real quick, because I’m supposed to be meeting a babysitter slash nanny to.
281 00:27:12.520 ⇒ 00:27:12.870 Uttam Kumaran: Okay.
282 00:27:12.870 ⇒ 00:27:18.919 Samuel Roberts: So, I’m getting a call from her. So, I think we’re good, though. If anything else comes up, message me on Slack, guys.
283 00:27:19.210 ⇒ 00:27:20.090 Uttam Kumaran: Okay, thank you.
284 00:27:20.260 ⇒ 00:27:21.000 Samuel Roberts: Yep.
285 00:27:21.120 ⇒ 00:27:22.699 Samuel Roberts: Alright, bye, y’all.
286 00:27:22.700 ⇒ 00:27:23.470 Casie Aviles: Thank you.