Meeting Title: Project Workflow and Evals Sync Date: 2025-09-10 Meeting participants: Mustafa Raja, Samuel Roberts
WEBVTT
1 00:01:58.380 ⇒ 00:01:59.300 Samuel Roberts: A…
2 00:02:01.800 ⇒ 00:02:02.900 Mustafa Raja: Hey, how are you?
3 00:02:03.910 ⇒ 00:02:05.090 Samuel Roberts: Doing alright.
4 00:02:05.090 ⇒ 00:02:06.839 Mustafa Raja: Yeah, sorry, sorry for rushing.
5 00:02:07.490 ⇒ 00:02:13.140 Samuel Roberts: No, no, you’re good, you’re good. I was in the middle of something, and I just wanted to get it done, but I think I’m making progress, so I did.
6 00:02:14.520 ⇒ 00:02:17.900 Mustafa Raja: Okay, so, couple of things.
7 00:02:18.020 ⇒ 00:02:22.009 Mustafa Raja: Let me share my screen.
8 00:02:25.120 ⇒ 00:02:26.240 Mustafa Raja: That is small.
9 00:02:26.980 ⇒ 00:02:27.730 Mustafa Raja: Okay.
10 00:02:28.350 ⇒ 00:02:30.350 Mustafa Raja: Hmm… Hmm.
11 00:02:31.750 ⇒ 00:02:34.829 Mustafa Raja: So, so this button is now working.
12 00:02:34.970 ⇒ 00:02:36.540 Mustafa Raja: We’re feeding days.
13 00:02:38.220 ⇒ 00:02:42.850 Mustafa Raja: Est-que… Yeah, so…
14 00:02:42.850 ⇒ 00:02:43.450 Samuel Roberts: Cool.
15 00:02:44.200 ⇒ 00:02:53.429 Mustafa Raja: Here we have it. And then the other thing was, it should only return us whatever we are editing, right? So, if we say…
16 00:02:56.850 ⇒ 00:03:02.950 Mustafa Raja: Let’s add more questions. Let’s add more… Questions…
17 00:03:06.530 ⇒ 00:03:11.260 Mustafa Raja: It’s going to return only what it’s going to edit.
18 00:03:11.850 ⇒ 00:03:15.219 Mustafa Raja: And then we… when we save it.
19 00:03:15.360 ⇒ 00:03:20.279 Mustafa Raja: We will see that the state was correctly stored.
20 00:03:20.420 ⇒ 00:03:21.520 Mustafa Raja: And updated.
21 00:03:22.050 ⇒ 00:03:25.589 Mustafa Raja: So here… hey, we see that only the questions appeared.
22 00:03:25.880 ⇒ 00:03:27.269 Mustafa Raja: If we save it again…
23 00:03:34.760 ⇒ 00:03:38.679 Mustafa Raja: Yeah, it’s the whole deck, and if we go to the questions…
24 00:03:38.890 ⇒ 00:03:41.939 Mustafa Raja: We’ll see that these are the new ones.
25 00:03:43.170 ⇒ 00:03:45.500 Mustafa Raja: accordingly. Okay.
26 00:03:45.690 ⇒ 00:03:52.519 Mustafa Raja: So, do you want to look into the workflow?
27 00:03:54.490 ⇒ 00:03:55.879 Mustafa Raja: Or you can skip it.
28 00:03:56.740 ⇒ 00:03:57.799 Samuel Roberts: No, it’s real quick.
29 00:03:59.420 ⇒ 00:04:09.700 Mustafa Raja: Okay. Yeah, so, so, so, two notes, this is for… at the end of the whole deck, just send the button, similar over here.
30 00:04:09.860 ⇒ 00:04:17.899 Mustafa Raja: Some changes over here, because, we wanted to maintain the text one, too, for the approval.
31 00:04:18.380 ⇒ 00:04:19.649 Mustafa Raja: Okay.
32 00:04:19.769 ⇒ 00:04:31.359 Mustafa Raja: So, some changes over here, and then this listens to the button. This webhook listens to the button. And yeah, some more changes over here to,
33 00:04:31.520 ⇒ 00:04:41.950 Mustafa Raja: update the state. This is to get the, get the deck from the state, these two, and this is to start the initial state.
34 00:04:43.050 ⇒ 00:04:43.610 Samuel Roberts: Great.
35 00:04:44.050 ⇒ 00:04:51.060 Mustafa Raja: Yeah, these are the… Quick updates for that. Let’s move to… what’s it called?
36 00:04:51.370 ⇒ 00:04:56.650 Mustafa Raja: The evals thing, because… Yes.
37 00:04:57.700 ⇒ 00:05:01.290 Mustafa Raja: Okay, so do you want to set it up on your end?
38 00:05:04.970 ⇒ 00:05:09.360 Samuel Roberts: I’m wood, my whole environment’s a little…
39 00:05:10.480 ⇒ 00:05:12.809 Mustafa Raja: Okay, I can run it on my end.
40 00:05:12.810 ⇒ 00:05:15.559 Samuel Roberts: Yeah, that’d be great, if you could just walk…
41 00:05:15.840 ⇒ 00:05:19.769 Mustafa Raja: Yeah, so this is from a previous run. Let’s… let’s do a new run.
42 00:05:20.220 ⇒ 00:05:20.840 Samuel Roberts: True.
43 00:05:23.370 ⇒ 00:05:24.250 Mustafa Raja: Hmm.
44 00:05:24.610 ⇒ 00:05:32.679 Mustafa Raja: Let’s see… Let’s do source… Let’s do these two.
45 00:05:36.430 ⇒ 00:05:37.210 Mustafa Raja: Okay.
46 00:05:41.130 ⇒ 00:05:42.860 Mustafa Raja: Yeah, this is how we run it.
47 00:05:42.860 ⇒ 00:05:43.620 Samuel Roberts: Cool.
48 00:05:43.620 ⇒ 00:05:50.029 Mustafa Raja: It’s now running, and it’s now going to tell us, okay, I have… Completed the run.
49 00:05:53.050 ⇒ 00:05:56.669 Mustafa Raja: We can look into the code also, if you want.
50 00:05:57.680 ⇒ 00:06:00.929 Samuel Roberts: Where’s the result over here? So this is…
51 00:06:00.930 ⇒ 00:06:05.699 Mustafa Raja: Yeah, so… so here we see the… the… so this is the scores.
52 00:06:06.340 ⇒ 00:06:06.890 Samuel Roberts: Yep.
53 00:06:07.100 ⇒ 00:06:16.069 Mustafa Raja: 0%, 80%, and 80%. Okay, let’s see the score. Are you in brain trust? I can also invite you if you’re not.
54 00:06:16.490 ⇒ 00:06:19.929 Samuel Roberts: I think I am, I just haven’t been in there in a minute.
55 00:06:19.930 ⇒ 00:06:20.950 Mustafa Raja: Okay.
56 00:06:21.480 ⇒ 00:06:22.119 Samuel Roberts: Okay, go ahead.
57 00:06:23.280 ⇒ 00:06:24.980 Mustafa Raja: Experiments…
58 00:06:26.950 ⇒ 00:06:30.500 Samuel Roberts: What’s… is it… we all have separate accounts for that, or is it one account?
59 00:06:30.690 ⇒ 00:06:35.580 Mustafa Raja: No, no, no, it’s the team one. We have unlimited members.
60 00:06:36.090 ⇒ 00:06:36.780 Samuel Roberts: Okay.
61 00:06:37.220 ⇒ 00:06:37.980 Mustafa Raja: Yeah.
62 00:06:37.980 ⇒ 00:06:41.920 Samuel Roberts: I just couldn’t remember. I’m always… I’m having a hard time keeping some of that straight, so let me make sure that this.
63 00:06:41.920 ⇒ 00:06:43.919 Mustafa Raja: Yeah, we have a lot.
64 00:06:43.930 ⇒ 00:06:45.800 Samuel Roberts: Yeah, exactly, exactly.
65 00:06:45.800 ⇒ 00:06:51.620 Mustafa Raja: Like, the extra, we have to log in via the engineering one, right?
66 00:06:51.620 ⇒ 00:06:55.929 Samuel Roberts: That’s what I keep forgetting, yeah. Okay, I think I’m in here now. I see one minute ago…
67 00:06:56.140 ⇒ 00:06:57.860 Samuel Roberts: recent experiments.
68 00:06:57.860 ⇒ 00:07:01.940 Mustafa Raja: Okay, okay. Oh, is it showing already on your side?
69 00:07:02.430 ⇒ 00:07:04.399 Samuel Roberts: Yeah. Cool.
70 00:07:04.400 ⇒ 00:07:06.340 Mustafa Raja: This is loading up.
71 00:07:06.800 ⇒ 00:07:07.790 Samuel Roberts: Oh, yeah.
72 00:07:08.150 ⇒ 00:07:09.679 Mustafa Raja: Bro, lord.
73 00:07:10.140 ⇒ 00:07:11.090 Samuel Roberts: Yeah.
74 00:07:12.840 ⇒ 00:07:17.570 Mustafa Raja: Yeah, my internet isn’t… isn’t too good today, so…
75 00:07:17.570 ⇒ 00:07:18.900 Samuel Roberts: Yeah, it seems like it.
76 00:07:21.000 ⇒ 00:07:24.620 Mustafa Raja: I’m actually on my… Mobile data.
77 00:07:24.880 ⇒ 00:07:26.290 Samuel Roberts: Oh, man, okay.
78 00:07:26.350 ⇒ 00:07:35.970 Mustafa Raja: Yeah, the Wi-Fi isn’t too good, and that seems also not too good. Anyways, I guess if you can see it on your side, that’s good.
79 00:07:35.970 ⇒ 00:07:37.500 Samuel Roberts: Yeah, I’ve seen it.
80 00:07:37.500 ⇒ 00:07:38.250 Mustafa Raja: Yeah.
81 00:07:38.580 ⇒ 00:07:48.110 Mustafa Raja: Yeah, so this is it. This is, where… where, the whole thing lives.
82 00:07:48.500 ⇒ 00:07:55.060 Mustafa Raja: So yeah, the idea is if we do not want to do it on dataset, we should just…
83 00:07:55.180 ⇒ 00:08:04.159 Mustafa Raja: Wait, is it fallback? Yeah, we should just return the output. And that output is going to be given…
84 00:08:04.350 ⇒ 00:08:09.190 Mustafa Raja: Like this, in JSON format only.
85 00:08:09.890 ⇒ 00:08:10.590 Samuel Roberts: Okay.
86 00:08:10.590 ⇒ 00:08:13.150 Mustafa Raja: Yeah, so, super simple.
87 00:08:14.090 ⇒ 00:08:15.559 Samuel Roberts: Okay, yeah, no, I think it’s good.
88 00:08:15.910 ⇒ 00:08:22.130 Mustafa Raja: Yeah, yeah, so the Bactel one, there is… oh, yeah, it’s a… or no.
89 00:08:22.130 ⇒ 00:08:22.790 Samuel Roberts: Oh, good, okay.
90 00:08:22.790 ⇒ 00:08:25.979 Mustafa Raja: Let’s do this one, let’s look.
91 00:08:26.180 ⇒ 00:08:33.869 Mustafa Raja: what the battle says. So, this data that I have over here is obviously synthetic, right?
92 00:08:34.610 ⇒ 00:08:35.280 Samuel Roberts: Right.
93 00:08:36.200 ⇒ 00:08:46.439 Mustafa Raja: So… funny enough, for the… If we go… to the reasoning…
94 00:08:47.940 ⇒ 00:08:49.729 Mustafa Raja: It does give us the reasoning, yeah.
95 00:08:51.440 ⇒ 00:08:52.630 Samuel Roberts: Oh, okay, cool.
96 00:08:52.630 ⇒ 00:08:58.080 Mustafa Raja: Yeah, so it’s, it says that the input does not really align with the.
97 00:08:58.080 ⇒ 00:08:58.770 Samuel Roberts: Oh, yeah.
98 00:08:58.770 ⇒ 00:09:00.400 Mustafa Raja: It’s for different companies.
99 00:09:00.820 ⇒ 00:09:01.430 Samuel Roberts: Yeah.
100 00:09:01.560 ⇒ 00:09:16.440 Mustafa Raja: Yeah, so, this is, I guess, when I set it up, I set it up for the wrong companies, so that’s why it’s coming up. So I believe in production, this battle should be somewhat realistic.
101 00:09:17.020 ⇒ 00:09:19.060 Samuel Roberts: I think so, yeah. Okay. Okay.
102 00:09:19.200 ⇒ 00:09:27.849 Mustafa Raja: Yeah, so, so if you approve it, I’ll, what I’ll do next is… Oh, yeah, this is where I need to…
103 00:09:28.290 ⇒ 00:09:40.989 Mustafa Raja: need your advice to, what do we want? Do we want it to run on… on, like, every iteration, or only when we are storing to Notion?
104 00:09:42.000 ⇒ 00:09:47.420 Samuel Roberts: My gut says only when we start in Notion.
105 00:09:47.720 ⇒ 00:09:51.409 Mustafa Raja: Okay, so… so only a call over here, right?
106 00:09:51.990 ⇒ 00:09:52.979 Samuel Roberts: I think so, yeah.
107 00:09:53.560 ⇒ 00:09:54.530 Mustafa Raja: Okay, okay.
108 00:09:54.680 ⇒ 00:10:00.570 Samuel Roberts: If we need to change that later, we can, but I think that’s enough to say, like, okay, this is the, you know, the final V1 kind of thing.
109 00:10:01.260 ⇒ 00:10:02.589 Mustafa Raja: Yeah, yeah.
110 00:10:02.710 ⇒ 00:10:07.149 Mustafa Raja: Okay, okay, okay, once I… once I merge it, I’ll…
111 00:10:07.250 ⇒ 00:10:13.070 Mustafa Raja: I’ll make sure to add it only when we store to the Notion thing.
112 00:10:13.200 ⇒ 00:10:22.000 Mustafa Raja: Yeah, this was pretty much it. One last thing. So this, for this.
113 00:10:22.000 ⇒ 00:10:22.770 Samuel Roberts: Oh, yeah.
114 00:10:22.770 ⇒ 00:10:25.150 Mustafa Raja: Yeah, Ryan… Ryan reacted.
115 00:10:25.440 ⇒ 00:10:27.589 Mustafa Raja: I don’t know if it’s a… if it’s…
116 00:10:27.590 ⇒ 00:10:28.930 Samuel Roberts: Yeah, I don’t know what that means.
117 00:10:29.000 ⇒ 00:10:30.190 Mustafa Raja: Yeah. Yeah.
118 00:10:30.450 ⇒ 00:10:34.440 Mustafa Raja: So, should I bump it up? Should I ask…
119 00:10:34.700 ⇒ 00:10:38.880 Mustafa Raja: If I should bump it up in internal channel, what should I do?
120 00:10:39.600 ⇒ 00:10:44.430 Samuel Roberts: Good question, yeah, I would say…
121 00:10:45.860 ⇒ 00:10:46.190 Mustafa Raja: Alright.
122 00:10:46.190 ⇒ 00:10:47.590 Samuel Roberts: I would say just respond to that.
123 00:10:48.240 ⇒ 00:10:54.099 Samuel Roberts: Yeah, I would say respond to it and just ask, like, is that, you know, reaction and approval to go ahead and do it?
124 00:10:54.100 ⇒ 00:10:56.940 Mustafa Raja: There’s I am…
125 00:10:57.170 ⇒ 00:10:58.360 Samuel Roberts: Yeah, is that funny?
126 00:10:58.820 ⇒ 00:10:59.590 Mustafa Raja: And so…
127 00:10:59.590 ⇒ 00:11:01.170 Samuel Roberts: But yeah, I think I would just say, like.
128 00:11:01.200 ⇒ 00:11:03.210 Mustafa Raja: Double checking…
129 00:11:03.210 ⇒ 00:11:04.170 Samuel Roberts: Perfect, yeah.
130 00:11:06.920 ⇒ 00:11:08.130 Mustafa Raja: I will take him.
131 00:11:08.280 ⇒ 00:11:13.240 Mustafa Raja: Is the reaction an approval, right? Is the… Shm.
132 00:11:13.990 ⇒ 00:11:25.610 Mustafa Raja: Yeah. Approval… 1, 2, 7… this often see this… Salesforce.
133 00:11:25.890 ⇒ 00:11:27.230 Mustafa Raja: Force, force.
134 00:11:27.830 ⇒ 00:11:30.420 Mustafa Raja: Is this good enough to sing?
135 00:11:30.620 ⇒ 00:11:32.050 Samuel Roberts: Sorry, honey.
136 00:11:33.670 ⇒ 00:11:34.700 Samuel Roberts: Yeah, go ahead.
137 00:11:36.620 ⇒ 00:11:37.540 Mustafa Raja: Hmm…
138 00:11:38.500 ⇒ 00:11:39.080 Samuel Roberts: Cool.
139 00:11:39.980 ⇒ 00:11:42.030 Mustafa Raja: Yeah.
140 00:11:42.330 ⇒ 00:11:46.940 Samuel Roberts: Okay, hopefully, hopefully by the morning, I’ll have the evals also set up.
141 00:11:47.480 ⇒ 00:11:51.529 Mustafa Raja: So that should be good. Okay, thank you. This was it.
142 00:11:51.530 ⇒ 00:11:52.230 Samuel Roberts: Alright.
143 00:11:52.500 ⇒ 00:11:54.309 Mustafa Raja: Let me know if you have any questions.
144 00:11:54.700 ⇒ 00:11:55.420 Mustafa Raja: Aww.
145 00:11:55.420 ⇒ 00:11:56.159 Samuel Roberts: Yeah, no, I think.
146 00:11:56.160 ⇒ 00:12:00.300 Mustafa Raja: also, okay, you proved it, right?
147 00:12:00.930 ⇒ 00:12:04.259 Mustafa Raja: I will go and approve it, yeah. Oh, yeah. Okay, okay.
148 00:12:04.260 ⇒ 00:12:04.730 Samuel Roberts: I’ll take care of it.
149 00:12:04.730 ⇒ 00:12:07.039 Mustafa Raja: Once you approve it, I’ll merge and deploy.
150 00:12:07.870 ⇒ 00:12:09.099 Samuel Roberts: Okay, sounds good.
151 00:12:09.100 ⇒ 00:12:12.490 Mustafa Raja: Yeah, okay, okay. Thank you.
152 00:12:13.170 ⇒ 00:12:14.289 Samuel Roberts: Yeah, thanks so much.
153 00:12:16.260 ⇒ 00:12:16.860 Mustafa Raja: Bye.