Meeting Title: Hannah <> Casie: MQS Case Study Date: 2025-08-11 Meeting participants: Casie Aviles, Hannah Wang
WEBVTT
1 00:01:49.310 ⇒ 00:01:50.470 Hannah Wang: Bye.
2 00:01:51.390 ⇒ 00:01:52.520 Casie Aviles: Hey, hey Anna.
3 00:01:53.660 ⇒ 00:01:54.690 Hannah Wang: How’s it going?
4 00:01:55.820 ⇒ 00:02:01.750 Casie Aviles: Yeah, doing good. Looks like a pretty, pretty busy Monday. Pretty busy weekend.
5 00:02:02.860 ⇒ 00:02:11.499 Hannah Wang: Mondays are… are always like that, for some reason. It’s always busy, and then it starts to get better on, like.
6 00:02:11.620 ⇒ 00:02:14.260 Hannah Wang: Thursday, or something.
7 00:02:14.260 ⇒ 00:02:16.720 Casie Aviles: Yeah, around Friday it starts to cool down.
8 00:02:16.720 ⇒ 00:02:18.740 Hannah Wang: Yeah, yeah.
9 00:02:18.910 ⇒ 00:02:24.819 Hannah Wang: Okay, so… yeah, this will be a similar drill to…
10 00:02:25.840 ⇒ 00:02:29.500 Hannah Wang: the Zoom platform case study that I walked…
11 00:02:29.760 ⇒ 00:02:45.180 Hannah Wang: with you through, like, a couple weeks ago. Yes. So, yeah, you can just share your screen and, give me, like, a high-level overview, and then, I’m gonna ask you questions after that.
12 00:02:45.180 ⇒ 00:02:50.109 Casie Aviles: Okay, I guess this might be a little too late, but…
13 00:02:50.350 ⇒ 00:02:56.059 Casie Aviles: I was just wondering, like, this… this work for, this, NQS?
14 00:02:56.950 ⇒ 00:03:01.789 Casie Aviles: work is… there’s… we don’t have a lot of, like, outcomes yet for this, so I was.
15 00:03:01.790 ⇒ 00:03:02.120 Hannah Wang: Okay.
16 00:03:02.120 ⇒ 00:03:05.959 Casie Aviles: if… is that fine? Like, it’s…
17 00:03:06.210 ⇒ 00:03:15.450 Casie Aviles: still kind of relatively early when we… when we… since we last great, worked on this, so we don’t really have a bunch of outcomes yet.
18 00:03:16.850 ⇒ 00:03:27.209 Hannah Wang: I think that’s okay. Hold on, let me just find… Because I know Utah mentioned…
19 00:03:31.190 ⇒ 00:03:37.859 Hannah Wang: Let’s see… Yeah, Ucham last Wednesday said.
20 00:03:38.190 ⇒ 00:03:42.110 Hannah Wang: MQS system can be a great case study, so…
21 00:03:42.280 ⇒ 00:03:51.130 Hannah Wang: I’m assuming there’s, like, enough work done where… Buchan thinks that it’s… good. To…
22 00:03:52.330 ⇒ 00:03:56.120 Hannah Wang: Have? Okay. I think it’s okay, yeah.
23 00:03:56.590 ⇒ 00:04:05.260 Casie Aviles: Alright Okay, so I guess just to give… Like, a background on… on this spike.
24 00:04:05.680 ⇒ 00:04:13.180 Casie Aviles: So, basically, … Utam was sharing with us that, you know, he… one of the goals was to…
25 00:04:13.340 ⇒ 00:04:20.480 Casie Aviles: Of course, the internal work that we do is to help improve, like, our internal processes.
26 00:04:21.029 ⇒ 00:04:23.449 Casie Aviles: And among those is…
27 00:04:23.660 ⇒ 00:04:33.299 Casie Aviles: Being able to kind of measure, like, the meeting quality that we… that… that takes… that we have, like, throughout, you know, throughout the week, …
28 00:04:33.550 ⇒ 00:04:38.710 Casie Aviles: So, the reason being is, you know, we have a lot of meetings, and
29 00:04:40.640 ⇒ 00:04:43.350 Casie Aviles: I think one of the things there is…
30 00:04:43.860 ⇒ 00:04:46.040 Casie Aviles: You know, it doesn’t mean, like.
31 00:04:46.160 ⇒ 00:04:51.030 Casie Aviles: If we met or did a meeting, then the results are already good, like…
32 00:04:51.550 ⇒ 00:04:58.470 Casie Aviles: Basically the idea is to measure if… how… how well the meeting went, like, were… were key…
33 00:04:59.470 ⇒ 00:05:01.579 Casie Aviles: Problems resolved, or…
34 00:05:01.850 ⇒ 00:05:10.660 Casie Aviles: you know, was there good progress, stuff like that, and so yeah, later on, we’ll be… I’ll be showing, like, which
35 00:05:10.980 ⇒ 00:05:19.559 Casie Aviles: dimensions, or we, we defined, like, where, where do we measure? You know, what do we measure, kind of, yeah.
36 00:05:19.680 ⇒ 00:05:24.889 Casie Aviles: So… Yeah, so it started out there, and …
37 00:05:25.320 ⇒ 00:05:31.230 Casie Aviles: Basically, what I did for this spike is… I just, …
38 00:05:32.110 ⇒ 00:05:36.590 Casie Aviles: Honestly, I just went to AI, and then I looked for… I asked for, like.
39 00:05:36.980 ⇒ 00:05:42.439 Casie Aviles: I had, like, a brainstorming with ChatGPT, or what could be some…
40 00:05:42.590 ⇒ 00:05:47.790 Casie Aviles: Things we needed to, measure, like… For example, …
41 00:05:48.870 ⇒ 00:05:52.799 Casie Aviles: like, what are these? So, like…
42 00:05:54.820 ⇒ 00:06:02.030 Casie Aviles: Engagement, yeah, like, one of those, engagement, you know, stuff like that. Whether, yeah, like.
43 00:06:02.620 ⇒ 00:06:07.550 Casie Aviles: I think I can just show you, like, what it looks like. We have a bunch of…
44 00:06:09.280 ⇒ 00:06:15.510 Casie Aviles: Yeah, for example, here… yeah, yeah, yeah. I don’t know why I’m trying to recite this at the top of my head, but…
45 00:06:16.590 ⇒ 00:06:21.490 Casie Aviles: These are… yeah, I could have just shown this, but, yeah, we have these meeting scores.
46 00:06:21.740 ⇒ 00:06:29.289 Casie Aviles: So we have the purpose-purpose alignment, preparation, participation, follow-through, and iteration feedback.
47 00:06:29.480 ⇒ 00:06:33.650 Casie Aviles: So I, I drafted, like… so before, it wasn’t these 5.
48 00:06:33.820 ⇒ 00:06:40.149 Casie Aviles: I just created, like, I drafted 5, like, and then I…
49 00:06:40.390 ⇒ 00:06:45.439 Casie Aviles: scheduled a call, basically, with Amber, and also asked, Alex.
50 00:06:45.790 ⇒ 00:06:49.549 Casie Aviles: For some feedback on what… how we want to approach this.
51 00:06:49.990 ⇒ 00:06:53.689 Casie Aviles: And, yeah, we, we basically, we, …
52 00:06:54.430 ⇒ 00:07:00.659 Casie Aviles: we agreed to do this. Of course, there were, like, Considerations that were kind of…
53 00:07:01.130 ⇒ 00:07:04.820 Casie Aviles: What’s the proper word? I guess, kind of…
54 00:07:05.320 ⇒ 00:07:10.150 Casie Aviles: There’s, like, trade-offs to keep in mind, like, …
55 00:07:11.060 ⇒ 00:07:14.959 Casie Aviles: Whether, like, do we do this on an aggregated weekly…
56 00:07:15.290 ⇒ 00:07:19.790 Casie Aviles: Kind of way where we produce scores.
57 00:07:20.020 ⇒ 00:07:21.369 Casie Aviles: After a week.
58 00:07:23.500 ⇒ 00:07:26.270 Casie Aviles: Or do we score it per meeting?
59 00:07:26.310 ⇒ 00:07:30.789 Hannah Wang: Right. There were those kinds of considerations, and then also….
60 00:07:31.380 ⇒ 00:07:32.710 Casie Aviles: What else?
61 00:07:35.400 ⇒ 00:07:41.349 Casie Aviles: Yeah, and then, like, for each meeting, like, not all meetings are the same, so how do we measure…
62 00:07:41.790 ⇒ 00:07:48.729 Casie Aviles: Different kinds of meetings, because, yeah, not all meetings would be, like, it would be the same, but…
63 00:07:48.850 ⇒ 00:07:52.449 Casie Aviles: Like, for us, a proof of concept, just to have something…
64 00:07:52.950 ⇒ 00:07:56.380 Casie Aviles: To put something out as, like, you know, a first version.
65 00:07:56.830 ⇒ 00:07:59.050 Casie Aviles: We decided to just go with
66 00:07:59.450 ⇒ 00:08:11.119 Casie Aviles: this simple scoring, where… because this was easier, because we could just have it on N810, where we already have an existing
67 00:08:11.850 ⇒ 00:08:18.190 Casie Aviles: workflow for this summary, right? … Yeah, which was on NA10.
68 00:08:18.720 ⇒ 00:08:21.100 Casie Aviles: I could… Let me just go there.
69 00:08:21.920 ⇒ 00:08:25.929 Casie Aviles: So, we already have this existing Zoom meeting summarizer.
70 00:08:26.200 ⇒ 00:08:30.050 Casie Aviles: So all we had to do was, attach a,
71 00:08:30.650 ⇒ 00:08:34.120 Casie Aviles: an AI step that, based on a given prompt.
72 00:08:36.679 ⇒ 00:08:42.240 Casie Aviles: it would generate the meeting score. Yeah, it’s… it’s… we have this step here, …
73 00:08:43.220 ⇒ 00:08:47.900 Casie Aviles: And we defined these, … Scoring definitions, like.
74 00:08:48.000 ⇒ 00:08:50.550 Casie Aviles: What does 0 mean? What does 5 mean?
75 00:08:51.280 ⇒ 00:08:54.809 Casie Aviles: And yeah, we also have, like, …
76 00:08:55.280 ⇒ 00:08:58.679 Casie Aviles: Yeah, the dimensions, the scoring dimensions that, ….
77 00:08:58.680 ⇒ 00:09:00.530 Hannah Wang: Me and Amber.
78 00:09:00.820 ⇒ 00:09:04.150 Casie Aviles: decided… on… yeah.
79 00:09:06.070 ⇒ 00:09:11.590 Casie Aviles: Okay, yeah, so far is everything clear? ….
80 00:09:12.400 ⇒ 00:09:14.590 Hannah Wang: Yeah, yeah, makes sense.
81 00:09:15.730 ⇒ 00:09:16.780 Casie Aviles: Okay, ….
82 00:09:17.000 ⇒ 00:09:18.180 Hannah Wang: Yeah, I think….
83 00:09:18.180 ⇒ 00:09:23.440 Casie Aviles: Yeah, that’s pretty much the… what I… what we’ve done so far on this, ….
84 00:09:23.750 ⇒ 00:09:24.810 Hannah Wang: Okay.
85 00:09:24.820 ⇒ 00:09:26.630 Casie Aviles: We haven’t done is…
86 00:09:26.870 ⇒ 00:09:34.240 Casie Aviles: we haven’t really… I haven’t really gotten back with Amber on whether this was helpful, or, like, this…
87 00:09:34.510 ⇒ 00:09:40.630 Casie Aviles: Can it be improved? And of course, other… another thing we could do here, like, as a next step is…
88 00:09:40.990 ⇒ 00:09:48.599 Casie Aviles: To store all these scores per meeting, and have it, generate, like, a dashboard for it, so we can
89 00:09:48.810 ⇒ 00:09:54.910 Casie Aviles: Kind of see and have some analysis, you know, for… Discourse, yeah.
90 00:09:55.480 ⇒ 00:09:58.049 Hannah Wang: I see. And is…
91 00:09:58.750 ⇒ 00:10:07.519 Hannah Wang: this… like, I know the Zoom summarizer is only on Slack, but I know that we have, like, the platform work.
92 00:10:07.660 ⇒ 00:10:18.320 Hannah Wang: So, would we also put, like, the scores in the platform per meeting? Or is that… is it, like, unrelated to that?
93 00:10:19.760 ⇒ 00:10:28.099 Casie Aviles: We haven’t really thought about that, but that’s actually not a bad idea. I mean, we could also have it on the platform, …
94 00:10:28.320 ⇒ 00:10:35.819 Casie Aviles: You know, to… I guess kind of have, like, a dedicated PM… for PMs, right, like…
95 00:10:36.650 ⇒ 00:10:46.600 Casie Aviles: Yeah, how we could help them better with… via the platform, but right now, it’s just on Slack, and … Okay. Yeah, we’re not storing it anywhere yet.
96 00:10:47.410 ⇒ 00:11:02.740 Hannah Wang: Okay, yeah, because I’m just thinking even on the platform, like, per meeting, on the side, you can just have, like, a little dashboard of the scores, but I think that’s, like, a… that’s, like, a nice-to-have. It’s not, like, a…
97 00:11:03.000 ⇒ 00:11:14.600 Hannah Wang: super urgent thing, but, well, I don’t know, like, what the PMs use, like, I don’t know if they look at Slack meeting summaries more, or the platform, but I personally…
98 00:11:15.130 ⇒ 00:11:30.369 Hannah Wang: use the platform a lot, and I just, like, interact with the chat bot next to the meeting video. So yeah, maybe adding it there could be helpful, but not, like, a high priority, I think.
99 00:11:30.610 ⇒ 00:11:31.380 Casie Aviles: Yeah, for sure.
100 00:11:31.380 ⇒ 00:11:39.160 Hannah Wang: That’s just, like, something for you to think about. So okay, kind of digging through the questions now. You might have already…
101 00:11:39.510 ⇒ 00:11:46.520 Hannah Wang: yeah, iterated, or talked about it, but I’m just gonna ask you for the sake of getting all the information, so…
102 00:11:47.150 ⇒ 00:11:48.370 Hannah Wang: I guess.
103 00:11:48.560 ⇒ 00:11:58.520 Hannah Wang: … yeah, like, what was the purpose of… these I guess.
104 00:11:59.570 ⇒ 00:12:10.860 Hannah Wang: scaling, score, like, what was the purpose of it? Was it for the PMs? Was it for team members? And, like, what… what would you hope that, like, the users of this
105 00:12:11.210 ⇒ 00:12:17.990 Hannah Wang: Meeting score… meeting quality score system? Like, how would they benefit from It.
106 00:12:18.300 ⇒ 00:12:25.970 Casie Aviles: Yeah, so for these scores, like, the primary, … I guess the primary…
107 00:12:27.280 ⇒ 00:12:30.410 Casie Aviles: What do you call this, like… The beneficiary?
108 00:12:30.410 ⇒ 00:12:30.730 Hannah Wang: I guess.
109 00:12:30.730 ⇒ 00:12:31.290 Casie Aviles: Yeah, yeah.
110 00:12:31.290 ⇒ 00:12:31.840 Hannah Wang: Yeah.
111 00:12:32.230 ⇒ 00:12:35.399 Casie Aviles: For this would be the PMs, of course.
112 00:12:35.670 ⇒ 00:12:43.140 Casie Aviles: And yeah, … So the idea is kind of, like, if… Although we did not, …
113 00:12:43.790 ⇒ 00:12:47.570 Casie Aviles: We don’t have something like that in place, or the mechanism, but…
114 00:12:48.230 ⇒ 00:12:54.510 Casie Aviles: What we were thinking is, if it reaches, like, … Bad, like, a poor score.
115 00:12:55.070 ⇒ 00:12:58.119 Casie Aviles: Below threshold, like… I think…
116 00:12:59.170 ⇒ 00:13:01.320 Casie Aviles: Probably when, when it’s around…
117 00:13:02.760 ⇒ 00:13:07.259 Casie Aviles: Let me… let me double check the prompt that we have here.
118 00:13:09.280 ⇒ 00:13:16.570 Casie Aviles: So, yeah, like, around here, when it, when it goes here, 0, 1, 2, I think the idea was…
119 00:13:16.970 ⇒ 00:13:19.450 Casie Aviles: We were going to send alerts, and…
120 00:13:19.660 ⇒ 00:13:24.850 Casie Aviles: So, you know, it basically helps, it should help the PMs
121 00:13:26.610 ⇒ 00:13:30.150 Casie Aviles: Understand why the meeting was bad, and how they could improve.
122 00:13:31.220 ⇒ 00:13:34.980 Casie Aviles: … Yeah, I think that was the main, main…
123 00:13:36.770 ⇒ 00:13:42.479 Casie Aviles: Kind of goal that we are trying to… Reach with these scores.
124 00:13:43.710 ⇒ 00:13:44.350 Hannah Wang: Okay.
125 00:13:44.350 ⇒ 00:13:45.010 Casie Aviles: Yeah.
126 00:13:46.020 ⇒ 00:13:54.080 Hannah Wang: And do you know what the PMs were doing before this to evaluate the quality of meetings, or were they just not…
127 00:13:54.430 ⇒ 00:13:56.390 Hannah Wang: Evaluating them at all.
128 00:13:56.560 ⇒ 00:14:02.930 Casie Aviles: Yeah, I believe there was, like, no actual evaluations happening. I mean, maybe there’s, like, …
129 00:14:04.480 ⇒ 00:14:06.040 Casie Aviles: I don’t know, …
130 00:14:06.730 ⇒ 00:14:14.049 Casie Aviles: It’s not, like, formal, like, maybe they would just have, oh, that meeting was bad, but they don’t really have, like, a system in place, or, like.
131 00:14:15.740 ⇒ 00:14:18.169 Casie Aviles: At least as far as I know, there was no…
132 00:14:18.920 ⇒ 00:14:22.729 Casie Aviles: There’s nothing like that in place at the moment.
133 00:14:23.170 ⇒ 00:14:23.920 Hannah Wang: Okay.
134 00:14:24.630 ⇒ 00:14:34.099 Hannah Wang: Cool. So, I guess the main problem that PMs were facing was An ability to evaluate
135 00:14:34.520 ⇒ 00:14:38.179 Hannah Wang: Whether a meeting was good or not. ….
136 00:14:38.610 ⇒ 00:14:40.019 Casie Aviles: Or I guess, yeah.
137 00:14:40.020 ⇒ 00:14:44.120 Hannah Wang: Is that the main, like, challenge that the PMs are facing?
138 00:14:45.600 ⇒ 00:14:48.080 Casie Aviles: Yeah, and also….
139 00:14:48.610 ⇒ 00:14:50.359 Hannah Wang: Let me think of how I….
140 00:14:52.250 ⇒ 00:14:59.089 Casie Aviles: Yeah, because, another one was, like, … Certain meetings were, like.
141 00:14:59.350 ⇒ 00:15:03.040 Casie Aviles: not all meetings are going to be good, so I think…
142 00:15:03.260 ⇒ 00:15:06.500 Casie Aviles: Yeah, being able to know if, …
143 00:15:07.090 ⇒ 00:15:10.379 Casie Aviles: Why it was not good, or how it could be improved.
144 00:15:10.620 ⇒ 00:15:15.819 Casie Aviles: is, like, you know, very… I guess, ideally very helpful for them, like.
145 00:15:16.890 ⇒ 00:15:19.669 Casie Aviles: If they could diagnose why it was not
146 00:15:19.780 ⇒ 00:15:25.449 Casie Aviles: Good based on how… how they would measure it, like, based on a…
147 00:15:26.130 ⇒ 00:15:32.119 Casie Aviles: You know, how they would say if it’s bad or not, based on these scoring dimensions.
148 00:15:32.810 ⇒ 00:15:34.520 Hannah Wang: Okay, got it.
149 00:15:34.650 ⇒ 00:15:44.489 Hannah Wang: So, PMs just want to improve meetings, right? Or see if… like, sometimes meetings aren’t even probably necessary, so they probably want to save people time by.
150 00:15:44.490 ⇒ 00:15:45.399 Casie Aviles: Yes, yes, exactly.
151 00:15:45.400 ⇒ 00:15:49.650 Hannah Wang: Getting rid of meetings that are useless, basically.
152 00:15:49.650 ⇒ 00:15:52.589 Casie Aviles: Yeah, yeah, that’s… yeah, that’s also one of the things, yeah.
153 00:15:52.590 ⇒ 00:15:53.330 Hannah Wang: Okay.
154 00:15:53.450 ⇒ 00:15:54.380 Hannah Wang: Cool.
155 00:15:54.530 ⇒ 00:15:55.620 Hannah Wang: …
156 00:15:55.940 ⇒ 00:16:05.760 Hannah Wang: So the solution was just attaching, like, a workflow to the existing Zoom summarizer, an ADAN workflow, and…
157 00:16:06.080 ⇒ 00:16:17.859 Hannah Wang: … basically… within the existing Zoom summary, just having, like, the… each category and scoring them, …
158 00:16:17.960 ⇒ 00:16:23.620 Hannah Wang: And just, like, posting that on the Slack thread. That’s the main solution, right?
159 00:16:23.850 ⇒ 00:16:38.590 Casie Aviles: Yeah, yeah, at the moment, that’s it. Yeah, like I mentioned, there were, like, other things planned, well, although we didn’t really move forward much here yet, I think most of the work was actually coming up with a prompt.
160 00:16:38.830 ⇒ 00:16:45.790 Casie Aviles: And… Thinking about how we want to measure it, like the meetings, like.
161 00:16:46.050 ⇒ 00:16:52.933 Casie Aviles: Not so much as the technical side, because… … Yeah, like… the
162 00:16:53.370 ⇒ 00:17:05.639 Casie Aviles: like, I… at the start of the spike, I already thought of, you know, just adding it as an AI step, so it’s more about, you know, yeah, most of the work was just, you know, thinking of what’s a good
163 00:17:06.000 ⇒ 00:17:07.199 Casie Aviles: Prompt, what’s so good.
164 00:17:07.200 ⇒ 00:17:08.040 Hannah Wang: I see.
165 00:17:08.900 ⇒ 00:17:10.929 Hannah Wang: Mostly prompt engineering, okay.
166 00:17:11.250 ⇒ 00:17:14.710 Hannah Wang: And what are good benchmarks for, ….
167 00:17:15.060 ⇒ 00:17:15.770 Casie Aviles: Yes.
168 00:17:15.770 ⇒ 00:17:16.730 Hannah Wang: the meetings.
169 00:17:16.869 ⇒ 00:17:26.490 Hannah Wang: … Okay. So, can you just list through the tools that you used? So, there’s N8N, there’s…
170 00:17:26.720 ⇒ 00:17:30.099 Hannah Wang: I’m assuming… SAIC Azure OpenAI.
171 00:17:30.230 ⇒ 00:17:31.150 Hannah Wang: ….
172 00:17:31.150 ⇒ 00:17:32.750 Casie Aviles: Yes, … What’s….
173 00:17:32.750 ⇒ 00:17:38.819 Hannah Wang: the parser, I’m assuming, is just, like, a JSON, or I don’t know, like, a file or something.
174 00:17:39.050 ⇒ 00:17:40.180 Casie Aviles: Yeah, it’s just…
175 00:17:40.780 ⇒ 00:17:47.819 Casie Aviles: It’s just there to make sure that, you know, we’re outputting the scores in a structured format, but.
176 00:17:47.820 ⇒ 00:17:48.720 Hannah Wang: Okay.
177 00:17:48.720 ⇒ 00:17:57.249 Casie Aviles: this is mainly the kind of… yeah, it’s mainly this AI step here, which is using, like, this Azure OpenAI model.
178 00:17:58.730 ⇒ 00:18:01.540 Casie Aviles: And then just, you know, Slack.
179 00:18:02.880 ⇒ 00:18:07.730 Casie Aviles: Slack API, I guess. We are using Slack in order to interface.
180 00:18:08.230 ⇒ 00:18:12.950 Casie Aviles: Or, like, send them here, to be able to send them here on Slack.
181 00:18:13.390 ⇒ 00:18:16.869 Casie Aviles: Yeah, at minimum, I think that’s it, …
182 00:18:17.190 ⇒ 00:18:24.709 Casie Aviles: And then I’m not sure if we need to add that, but we also use, like, the prompt library that we have, …
183 00:18:28.090 ⇒ 00:18:29.380 Casie Aviles: Yeah, the, ….
184 00:18:29.380 ⇒ 00:18:29.830 Hannah Wang: Oh, no.
185 00:18:29.830 ⇒ 00:18:32.199 Casie Aviles: Yeah, yeah.
186 00:18:32.430 ⇒ 00:18:38.700 Casie Aviles: We also have this prompt library, and I mainly use the prompt.
187 00:18:38.930 ⇒ 00:18:42.569 Casie Aviles: improver, I think, or a prompt optimizer, ….
188 00:18:44.200 ⇒ 00:18:47.980 Hannah Wang: Oh, yeah, I saw it. It’s at the top.
189 00:18:48.520 ⇒ 00:18:53.029 Hannah Wang: … right here… yeah, right there.
190 00:18:53.030 ⇒ 00:18:54.119 Casie Aviles: Yeah, this one.
191 00:18:54.300 ⇒ 00:18:54.760 Hannah Wang: Okay.
192 00:18:54.760 ⇒ 00:19:00.429 Casie Aviles: I mainly use this, and also I stored this prompt that I’m using here.
193 00:19:00.760 ⇒ 00:19:03.390 Casie Aviles: as well. Okay. I also have it here.
194 00:19:08.800 ⇒ 00:19:10.289 Casie Aviles: Yeah, that’s this one.
195 00:19:10.630 ⇒ 00:19:11.330 Hannah Wang: Okay.
196 00:19:11.580 ⇒ 00:19:12.529 Hannah Wang: I see.
197 00:19:14.150 ⇒ 00:19:22.950 Hannah Wang: Would you consider Zoom, or is Zoom too, like, far upstream that it doesn’t… count.
198 00:19:23.960 ⇒ 00:19:28.859 Casie Aviles: … Zoom is… We’re, we’re using Zoom…
199 00:19:29.630 ⇒ 00:19:33.530 Casie Aviles: Like, the transcript, so we’re using the transcript, so….
200 00:19:33.530 ⇒ 00:19:34.500 Hannah Wang: Oh, okay.
201 00:19:35.560 ⇒ 00:19:40.579 Casie Aviles: It’s not necessarily, like, … Yeah, it’s more like an input.
202 00:19:41.570 ⇒ 00:19:43.639 Casie Aviles: Okay. Yeah, not a tool.
203 00:19:43.700 ⇒ 00:19:44.600 Hannah Wang: Okay.
204 00:19:49.610 ⇒ 00:19:57.740 Hannah Wang: Okay, … So that’s the solution, and the results, …
205 00:19:58.070 ⇒ 00:20:03.889 Hannah Wang: I don’t know, did you get feedback from Rico or Amber or anyone else about the…
206 00:20:04.030 ⇒ 00:20:07.809 Hannah Wang: scoring system, or even UTARM, what they said about it.
207 00:20:10.000 ⇒ 00:20:16.520 Casie Aviles: Yeah, that’s the thing, I don’t have much feedback yet on the meeting scoring.
208 00:20:16.520 ⇒ 00:20:16.860 Hannah Wang: Okay.
209 00:20:16.860 ⇒ 00:20:19.280 Casie Aviles: It kind of got buried under a.
210 00:20:19.280 ⇒ 00:20:19.780 Hannah Wang: Okay.
211 00:20:20.350 ⇒ 00:20:20.780 Hannah Wang: Hahaha.
212 00:20:21.210 ⇒ 00:20:21.780 Casie Aviles: Yeah….
213 00:20:21.780 ⇒ 00:20:22.520 Hannah Wang: Okay.
214 00:20:22.750 ⇒ 00:20:25.090 Hannah Wang: … I guess…
215 00:20:25.450 ⇒ 00:20:32.109 Hannah Wang: For you, personally, do you think it’s helpful, or what did you… what do you think about it?
216 00:20:33.520 ⇒ 00:20:39.889 Casie Aviles: … I think there’s… it’s definite… there’s, like, there’s potential for it to be helpful.
217 00:20:40.030 ⇒ 00:20:43.930 Hannah Wang: Right now, I guess I don’t see, like….
218 00:20:44.180 ⇒ 00:20:46.570 Casie Aviles: I don’t see how it’s improving…
219 00:20:47.350 ⇒ 00:20:53.230 Casie Aviles: the PM’s work yet, and I guess that’s also on me, like, maybe I need to push them to, like.
220 00:20:53.550 ⇒ 00:20:55.000 Casie Aviles: Give feedback.
221 00:20:55.820 ⇒ 00:21:01.000 Casie Aviles: On this, and… but I do think there’s potential, especially, like, if…
222 00:21:01.980 ⇒ 00:21:05.069 Casie Aviles: You know, if the scores really reflect, like.
223 00:21:05.200 ⇒ 00:21:13.249 Casie Aviles: Their own judgment, or, like, maybe… maybe even have better judgment than what they would originally come up with.
224 00:21:13.680 ⇒ 00:21:14.570 Casie Aviles: …
225 00:21:14.720 ⇒ 00:21:23.300 Casie Aviles: And then if they could act… if it’s going to be actionable, you know, right now it’s not… there’s not, like, a very clear process for it being actionable, because…
226 00:21:23.650 ⇒ 00:21:28.230 Casie Aviles: like I did mention, we’re kind of lacking this mechanism where it
227 00:21:29.280 ⇒ 00:21:35.910 Casie Aviles: alerts the, EMs somehow, like, letting them know that
228 00:21:36.120 ⇒ 00:21:44.819 Casie Aviles: You need to improve on this meeting, because there’s just a lot of meetings happening, and they’re probably not going through all of these.
229 00:21:45.280 ⇒ 00:21:46.519 Casie Aviles: Right. At the moment.
230 00:21:47.820 ⇒ 00:21:52.789 Casie Aviles: Yeah, and if we’re able to establish something like that, like a clear…
231 00:21:53.200 ⇒ 00:21:57.420 Casie Aviles: I don’t know, would you call that a feedback loop?
232 00:21:57.960 ⇒ 00:22:04.310 Casie Aviles: Yeah, I guess that would be very… that’s where I would think that this meeting
233 00:22:04.640 ⇒ 00:22:08.540 Casie Aviles: scoring system would have some value, I believe, yeah?
234 00:22:08.670 ⇒ 00:22:10.040 Hannah Wang: Okay, right.
235 00:22:10.760 ⇒ 00:22:15.820 Hannah Wang: If there’s, like, actionable steps that happen because of the scores.
236 00:22:17.140 ⇒ 00:22:17.870 Casie Aviles: Yes.
237 00:22:19.370 ⇒ 00:22:21.000 Hannah Wang: Okay, …
238 00:22:22.090 ⇒ 00:22:30.620 Hannah Wang: So, in terms of, like, roadmap on the AI sprint, like, you worked on the initial spike, and…
239 00:22:30.980 ⇒ 00:22:47.260 Hannah Wang: Are there any, like, follow-up tickets? Like, are you gonna… are you planning to get back to this, or is it kinda just, oh, you built the MVP, and later on, if people need it, then we’ll make it… make, like, a V2?
240 00:22:47.390 ⇒ 00:22:54.000 Hannah Wang: Of it. Because I’m wondering, like, at what point should I make this case study? If there’s no, like, results.
241 00:22:54.000 ⇒ 00:22:54.340 Casie Aviles: Yeah.
242 00:22:54.340 ⇒ 00:23:09.919 Hannah Wang: or impact yet, like you mentioned… I know you mentioned that in the beginning, but I’m just thinking, oh, is it feasible for you to put it into your roadmap, or is it… or it’s just other stuff that came up, like client work and stuff?
243 00:23:10.840 ⇒ 00:23:18.689 Casie Aviles: Yeah, right now, we’re actually kind of, we’re slowing down on the internal work that we have.
244 00:23:18.690 ⇒ 00:23:19.560 Hannah Wang: Yeah.
245 00:23:20.450 ⇒ 00:23:26.409 Casie Aviles: Especially because, you know, I think there will be a few more AI clients coming in.
246 00:23:26.410 ⇒ 00:23:26.910 Hannah Wang: Right.
247 00:23:26.910 ⇒ 00:23:32.629 Casie Aviles: And, … yeah, there’s… we also have this concern about
248 00:23:32.920 ⇒ 00:23:40.899 Casie Aviles: Just, you know, putting something out very rapidly, but a lot of technical debt has been piling up, so that’s why we’re slowing down.
249 00:23:42.210 ⇒ 00:23:45.240 Casie Aviles: Although, after each spike, we do…
250 00:23:45.610 ⇒ 00:23:50.520 Casie Aviles: We do usually take it out, like, an implementation step.
251 00:23:51.080 ⇒ 00:23:53.029 Hannah Wang: For this.
252 00:23:53.310 ⇒ 00:23:57.360 Casie Aviles: I think maybe there’s… there could be, like…
253 00:23:57.590 ⇒ 00:24:02.870 Casie Aviles: In the future, I’m just not sure how… how urgent right now it is, but….
254 00:24:02.870 ⇒ 00:24:03.560 Hannah Wang: Okay.
255 00:24:05.030 ⇒ 00:24:12.940 Casie Aviles: Yeah, … if ever, I would, you know, … ask Amber for feedback, and…
256 00:24:14.410 ⇒ 00:24:22.180 Casie Aviles: Maybe we could see, like, if it has affected her at all, because this is just from my perspective, …
257 00:24:22.420 ⇒ 00:24:28.750 Casie Aviles: Like, if it has benefited her at all, or… there’s anything… else we could do.
258 00:24:28.890 ⇒ 00:24:34.249 Casie Aviles: To, like, you know, make it more… or make it bring more value.
259 00:24:34.370 ⇒ 00:24:36.470 Casie Aviles: to the PM, but yeah.
260 00:24:37.110 ⇒ 00:24:39.360 Casie Aviles: Yeah, that’s pretty much… Got it. Yeah.
261 00:24:40.290 ⇒ 00:24:56.230 Hannah Wang: Understood, okay. Yeah, this was helpful. I can totally make a case study and just kind of make up results, but I’ll… I’ll talk with Uten, or follow up with him about this case study, and then…
262 00:24:56.570 ⇒ 00:25:02.740 Hannah Wang: I’ll let you know if I need more information, but for now, I think this is… this is a good start.
263 00:25:02.980 ⇒ 00:25:04.060 Casie Aviles: Okay.
264 00:25:04.960 ⇒ 00:25:13.560 Hannah Wang: Yeah, and then if anyone ever does give feedback, just, like, tag me in it, or send me a screenshot of it, and then I’ll add it to…
265 00:25:13.760 ⇒ 00:25:16.710 Hannah Wang: the documentation that I’m gonna build for it.
266 00:25:17.590 ⇒ 00:25:26.239 Casie Aviles: Yes, yeah, I think I’ll try to get some feedback tomorrow as well. I think I have, like, weekly meetings with the PM team, so….
267 00:25:27.340 ⇒ 00:25:29.999 Hannah Wang: Okay, let me know how that goes, ….
268 00:25:30.000 ⇒ 00:25:30.650 Casie Aviles: Okay.
269 00:25:30.880 ⇒ 00:25:33.800 Hannah Wang: Are you just, like, Yeah, just…
270 00:25:34.250 ⇒ 00:25:41.290 Hannah Wang: send me the link to the meeting, and then I’ll look at the summarizer. … But okay, yeah.
271 00:25:41.670 ⇒ 00:25:50.570 Hannah Wang: Thanks, Casey. And if there’s any other, work… I guess it doesn’t… I guess you said internal work is slowing down, but…
272 00:25:50.740 ⇒ 00:25:57.310 Hannah Wang: … Yeah, I know you might start working on client ones, so…
273 00:25:57.440 ⇒ 00:26:05.759 Hannah Wang: Yeah, if there’s any one that you complete, just let me know, because always making case studies, is helpful, so…
274 00:26:05.870 ⇒ 00:26:13.159 Hannah Wang: Any other future ones with clients, you can just let me know, and then I’ll grab time with you, or…
275 00:26:13.280 ⇒ 00:26:18.670 Hannah Wang: Something like that, but… Okay, yeah, I think this is good for this meeting.
276 00:26:19.530 ⇒ 00:26:22.310 Casie Aviles: Alright, … Yeah, I think that’s it.
277 00:26:23.460 ⇒ 00:26:25.589 Casie Aviles: Thank you as well, Hannah.
278 00:26:25.990 ⇒ 00:26:30.620 Hannah Wang: Yeah. Yeah, thanks, Casey, for all your work. The AI stuff is super awesome.
279 00:26:30.860 ⇒ 00:26:43.609 Hannah Wang: I don’t know how to do any of it, so it’s cool that, you know… I mean, I do have a little bit of developer background, but I don’t know how much of that is needed in AI stuff, so it’s cool seeing your work.
280 00:26:43.750 ⇒ 00:26:44.990 Hannah Wang: So, thank you.
281 00:26:45.460 ⇒ 00:26:46.690 Casie Aviles: Yeah, thank you as well.
282 00:26:47.310 ⇒ 00:26:48.959 Hannah Wang: Alrighty, have a good day.
283 00:26:49.150 ⇒ 00:26:50.050 Hannah Wang: Bye.