Meeting Title: Brainforge Interview w- Sam Date: 2026-05-04 Meeting participants: Farhaj Shahid, Fireflies.ai Notetaker Farhaj Shahid, Samuel Roberts
WEBVTT
1 00:00:37.640 ⇒ 00:00:38.560 Samuel Roberts: Hello.
2 00:00:38.560 ⇒ 00:00:40.170 Farhaj Shahid: Hi, Sam, how’s it going?
3 00:00:40.170 ⇒ 00:00:43.050 Samuel Roberts: Good, good. How about yourself? You hear me alright?
4 00:00:43.270 ⇒ 00:00:45.540 Farhaj Shahid: Yes, doing well.
5 00:00:46.530 ⇒ 00:00:47.949 Samuel Roberts: Great, sorry, one sec, I’m not…
6 00:00:48.090 ⇒ 00:00:50.779 Samuel Roberts: Getting it on my headphones. Can you hear me now?
7 00:00:51.190 ⇒ 00:00:52.800 Farhaj Shahid: Yes, I can hear you.
8 00:00:52.800 ⇒ 00:00:56.189 Samuel Roberts: You bet. Okay, great. Yeah, well, thank you for taking the time.
9 00:00:56.210 ⇒ 00:01:10.700 Samuel Roberts: We, may have someone else joining to just shadow how these interviews go. 100%. So, if someone jumps on, hopefully it’s that person. But besides that, the way I usually run these is,
10 00:01:10.700 ⇒ 00:01:13.579 Samuel Roberts: Quick interview about myself, or a quick intro about myself.
11 00:01:13.580 ⇒ 00:01:30.819 Samuel Roberts: A little intro from you. I’ve got some questions, obviously. I like to get about halfway, and then switch over, for any questions you have, so we don’t get to the end and run out of time or anything like that. And then, yeah, we’ll just kind of, you know, chat and see. You know, nothing crazy.
12 00:01:30.820 ⇒ 00:01:40.200 Samuel Roberts: So, yeah. So, thank you for taking the time again. My name’s Sam Roberts, I’m the AI Tech Lead here at Brainforge. I’ve been at Brainforge since July.
13 00:01:40.550 ⇒ 00:01:55.340 Samuel Roberts: Previously, I was in a lot of startups, various product roles, a lot of web dev back in the day, a lot more AI now, as you can imagine. But yeah, that’s kind of my, you know, very, very quick elevator pitch, so I’d love to hear yours, and then we can get started.
14 00:01:55.560 ⇒ 00:02:05.939 Farhaj Shahid: 100%. So, I’m Fir Hodge, I’m a current senior at Buckley University. I’m studying computer science, and I’ll be graduating in less than 2 weeks with a minor in visual design as well.
15 00:02:06.330 ⇒ 00:02:07.590 Samuel Roberts: Congratulations.
16 00:02:07.820 ⇒ 00:02:26.460 Farhaj Shahid: Thank you. At my core, I’m a builder at heart that’s been shipping in and outside of the classroom. I worked at Prometric last summer as a full-stack engineering intern, and then I continued working my senior year as a software engineer, working on client-facing features, products, and production hotfixes.
17 00:02:26.460 ⇒ 00:02:34.510 Farhaj Shahid: I also co-founded my own, development agency, Arterio. I can dive into more projects that I did, there.
18 00:02:34.510 ⇒ 00:02:35.260 Samuel Roberts: Definitely will, yeah.
19 00:02:35.260 ⇒ 00:02:46.449 Farhaj Shahid: During college, I’ve had a bunch of different experiences and roles in regards to development and design. I have research experience, I also have teaching and TA experience.
20 00:02:46.450 ⇒ 00:02:57.010 Farhaj Shahid: And at my core, I’m someone that’s very excited about technology and AI, and I’m chronically online on tech Twitter, following Frontier Labs and researchers and seeing what they’re drawing, experimenting.
21 00:02:57.010 ⇒ 00:02:57.620 Samuel Roberts: Great.
22 00:02:58.120 ⇒ 00:02:59.770 Samuel Roberts: We’ll dig into that too, yeah.
23 00:03:01.130 ⇒ 00:03:09.379 Samuel Roberts: Okay, yeah, thank you so much. Love, love, all of that. That’s really exciting. Okay, so, let’s just start,
24 00:03:09.670 ⇒ 00:03:17.949 Samuel Roberts: Let’s dig into some of the experience. Can you talk about a LLM-based feature that you shipped to production, and the problem that it solved?
25 00:03:18.440 ⇒ 00:03:26.569 Farhaj Shahid: So, for one of my side projects, Fazel, I build a rack pipeline to automate grading, and…
26 00:03:26.810 ⇒ 00:03:48.999 Farhaj Shahid: But it uses… it used… I experimented with a bunch of different models, and I settled on using Gemini, Gemini 2.5 Pro at that time, because it’s a multi-modal model, and I wanted it to do parsing at the same time, and image extraction, because the exam PDFs that I was working with are very bit messy, because you couldn’t really just rely on just
27 00:03:49.000 ⇒ 00:03:56.959 Farhaj Shahid: plain text extraction, and so sometimes if that did not pass the threshold, it would just use, like, image recognition, and OCR.
28 00:03:56.960 ⇒ 00:03:57.580 Farhaj Shahid: To do stuff.
29 00:03:58.120 ⇒ 00:04:02.239 Farhaj Shahid: So that was one of the projects that I was interested in working in.
30 00:04:02.420 ⇒ 00:04:04.459 Samuel Roberts: Okay, and that was for grading, you said?
31 00:04:04.460 ⇒ 00:04:11.499 Farhaj Shahid: Yes, it was for automated grading. It was for an independent study I was doing in the full-stack and web development.
32 00:04:11.500 ⇒ 00:04:25.530 Samuel Roberts: Cool. Very cool. Okay, yeah, great. I guess, so sort of related to that, that’s obviously kind of a, you know, full-stack kind of thing, but what part of the AI stack have you spent the most time actually building versus, like, experimenting, or, you know, where do you feel most comfortable overall?
33 00:04:25.720 ⇒ 00:04:34.770 Farhaj Shahid: Honestly, I’m really comfortable working across the stack, like, front-end, back-end design, but I think, for me personally, like.
34 00:04:34.770 ⇒ 00:04:50.840 Farhaj Shahid: Being a design minor, I love to build technology that not just functions, but also feels intuitive for the user, because you can have the greatest technology, but if your target audience does not know how to really properly use it or take advantage from it, I think that needs serious work or readjustment.
35 00:04:51.110 ⇒ 00:05:06.980 Samuel Roberts: Yeah, totally, totally. That’s actually a really good segue to sort of my next question, which is, is there a time you can talk about that a user sort of misunderstood what the AI feature might have been capable of, and how, you know, how that resolved one way or the other?
36 00:05:08.630 ⇒ 00:05:12.940 Farhaj Shahid: Honestly, no, not… Okay.
37 00:05:12.940 ⇒ 00:05:13.510 Samuel Roberts: Okay.
38 00:05:13.510 ⇒ 00:05:15.200 Farhaj Shahid: Don’t have anything to speak on that.
39 00:05:15.200 ⇒ 00:05:20.490 Samuel Roberts: Cool, that’s cool. Okay, so then the other question is kind of sort of in the same area, I guess, is,
40 00:05:20.740 ⇒ 00:05:28.310 Samuel Roberts: you know, there’s so much news out there about AI and LLMs, and lots of non-technical people hear about all of this and think.
41 00:05:28.420 ⇒ 00:05:38.489 Samuel Roberts: All kinds of things. What are your thoughts, or how would you go about explaining, like, the limitations of LLM technology to a non-technical stakeholder, potentially?
42 00:05:39.140 ⇒ 00:05:58.110 Farhaj Shahid: Okay, that’s a good question. I think for non-technical people especially, I would first tell them that, A, it’s, the quality of the inputs, are going to be, I think, directly proportional to the quality of the outputs. If you put trash in, you’re gonna get trash out. At the same time, AI is, is…
43 00:05:58.110 ⇒ 00:06:11.399 Farhaj Shahid: Generative AI, especially, is basically, at its core, it’s predicting the next token. So it doesn’t have all the answers, and sometimes it can hallucinate, because it just really depends on the training data, and what is said, basically.
44 00:06:11.520 ⇒ 00:06:18.930 Farhaj Shahid: Moreover, however, there are techniques that you can use to leverage it to its full potential, like, for example,
45 00:06:19.070 ⇒ 00:06:26.860 Farhaj Shahid: using a lot of good prompting, and then at the same time, the context and harness around the AI also really matters.
46 00:06:26.890 ⇒ 00:06:40.010 Farhaj Shahid: And there are some serious, like, memory limitations as you work with AI, so, for example, if you… if you pass, like, I guess, the context window, I think in a recent talk, someone was saying that
47 00:06:40.140 ⇒ 00:06:45.010 Farhaj Shahid: 40 to 50% is… context usage is where the smart, is.
48 00:06:45.010 ⇒ 00:06:46.949 Samuel Roberts: Yep, the dumb zone, yeah.
49 00:06:46.950 ⇒ 00:07:00.330 Farhaj Shahid: After that, it just gets dumber and dumber, so you want to, like, either compact your chats or start new ones at the same time. So I guess that’s how I would start to explain, to our non-technical friends on, I guess, how to create using AI.
50 00:07:00.680 ⇒ 00:07:11.699 Samuel Roberts: Great, yeah. Have you… do you use a lot of AI coding agents? I am sort of getting a little off where I want to go, but I’m curious… we’ll get into that a little bit more, but I’m curious, have you experienced that dumb zone yourself when you’re working with these tools?
51 00:07:11.700 ⇒ 00:07:18.880 Farhaj Shahid: 100%, I’ve experienced it. I’ve been experimenting with a lot of coding agents, I’ve used Cloud Code, OpenCode.
52 00:07:18.880 ⇒ 00:07:19.770 Samuel Roberts: Okay, great.
53 00:07:19.770 ⇒ 00:07:36.799 Farhaj Shahid: Yep, and I’ve really experienced this after that 50% mark, like, the quality of the outputs that I’m getting is not what I want it to be, and I found myself, I guess, debugging more longer. And so after that, when I realized that
54 00:07:36.800 ⇒ 00:07:40.820 Farhaj Shahid: hey, maybe I don’t want to use the entire 1 million contacts window to its full potential.
55 00:07:40.820 ⇒ 00:07:41.619 Samuel Roberts: Oh, man, yeah.
56 00:07:41.620 ⇒ 00:07:58.720 Farhaj Shahid: So after that, I was then starting to be a bit more smarter around my prompting, and around, like, the amount of stuff I, I guess, fit into a prompt at the same time, because, you don’t want to have, fluff in there at all, you just want there to be
57 00:07:58.720 ⇒ 00:08:02.140 Farhaj Shahid: Concrete, instructions that they can follow.
58 00:08:02.540 ⇒ 00:08:05.629 Samuel Roberts: Totally. Good, good, alright, cool, cool.
59 00:08:06.460 ⇒ 00:08:17.009 Samuel Roberts: Let’s see, let’s, so I sort of mentioned a lot of how quickly things are moving, and, you know, news coming out all the time. So is there any kind of…
60 00:08:17.190 ⇒ 00:08:27.439 Samuel Roberts: LLM trend, or tool, or framework, or whatever it is, that you might have been excited about initially, but decided not to adopt for some reason.
61 00:08:27.840 ⇒ 00:08:31.519 Farhaj Shahid: Okay, I’ll be… okay, so OpenCloud was all the rage, a couple of.
62 00:08:31.520 ⇒ 00:08:32.370 Samuel Roberts: Sure, sure.
63 00:08:32.370 ⇒ 00:08:45.429 Farhaj Shahid: And I experimented with that, but I realized that it was burning too many tokens and not getting too much, and I wasn’t getting too much out of it. It ended up being a bit more expensive for me, like, especially being a college student, you know?
64 00:08:45.430 ⇒ 00:08:45.850 Samuel Roberts: Absolutely, yeah.
65 00:08:45.850 ⇒ 00:08:49.099 Farhaj Shahid: So I decided that,
66 00:08:49.100 ⇒ 00:09:14.030 Farhaj Shahid: I think it’s cool for people, I guess, if they want, like, a personal, you know, assistant on their phone that they can just hand off stuff to, and then just check back in. But if you’re doing some serious building, I think I prefer to be a bit more hands-on, and I would love to work on my laptop more, so I have more control over stuff. So I think it is good, but I think, like, the way it’s evolved, and I think, for example, the take on Hermes agents on top of that was really good, because it
67 00:09:14.030 ⇒ 00:09:20.370 Farhaj Shahid: was conditioned to be, not using Frontier, like, models for basic tool calling, you know?
68 00:09:20.370 ⇒ 00:09:20.890 Samuel Roberts: Yeah.
69 00:09:20.890 ⇒ 00:09:26.029 Farhaj Shahid: And I think that was, I think, a step in the right direction. Totally, totally. So, yeah,
70 00:09:26.230 ⇒ 00:09:30.359 Farhaj Shahid: I’m not a… not a fan of OpenClaw, but not a doctor, and I…
71 00:09:30.360 ⇒ 00:09:40.369 Samuel Roberts: Yeah, yeah. No, I had a very interesting, or similar experience, that I thought it was interesting, everyone was talking about it, I wanted to try it, but I… you know, most of my…
72 00:09:40.370 ⇒ 00:10:02.010 Samuel Roberts: AI experience as coding agents, not necessarily personal assistants, and I was real nervous, especially very early, about all the security issues people were having, and so I put it in a little sandbox VM on my server, and barely gave it anything that it could access, and every time I did, it was… it just became a pain that way, and I didn’t feel comfortable, you know, giving it tons of access to anything after I saw people were getting messages from their own
73 00:10:02.010 ⇒ 00:10:04.460 Samuel Roberts: Open claw and things, and seemed a little… yeah.
74 00:10:04.460 ⇒ 00:10:11.310 Farhaj Shahid: 100%. I… for me, personally, for that personal assistant vibe, like, I really enjoy using the perplexity stack.
75 00:10:11.310 ⇒ 00:10:11.680 Samuel Roberts: Okay.
76 00:10:11.680 ⇒ 00:10:14.259 Farhaj Shahid: Perplexity Campus Ambassador, this past year.
77 00:10:14.260 ⇒ 00:10:14.910 Samuel Roberts: Nice.
78 00:10:14.910 ⇒ 00:10:22.580 Farhaj Shahid: just, like, yeah, telling my friends about what’s happening. And I really enjoyed using, like, for example, the AI browser Comet as well. It’s really good for browser…
79 00:10:22.580 ⇒ 00:10:22.970 Samuel Roberts: Yeah.
80 00:10:22.970 ⇒ 00:10:27.290 Farhaj Shahid: and getting a lot of stuff done. That used to be a pain, I guess, before.
81 00:10:27.770 ⇒ 00:10:33.520 Farhaj Shahid: So I… so that’s what I use for not coding agents, but for, like, personal use.
82 00:10:33.520 ⇒ 00:10:38.100 Samuel Roberts: Totally, totally. Great, yeah. So, sort of in that… in this… the same,
83 00:10:38.410 ⇒ 00:10:46.969 Samuel Roberts: thread, I guess, how do you decide when a model or a framework or a tool is production ready?
84 00:10:48.650 ⇒ 00:11:09.389 Farhaj Shahid: I think it has to do with a lot of, you have to do a lot of research and evaluation before you ship it out to production. I think, for example, if you have a test set that you’ve already used for training, you need to have a completely new one that the model hasn’t seen before, for example, and you evaluate it based on that and how well it performs there.
85 00:11:09.390 ⇒ 00:11:21.740 Farhaj Shahid: And so, like, similar to, like, the RAG pipeline that I was mentioning about exam… exam grading, right? Like, I had a set where that were already graded by teachers, by hand, and I evaluated after the fact of
86 00:11:22.140 ⇒ 00:11:30.740 Farhaj Shahid: is the model hallucinating, or is it actually, like, giving a proper grade to these answers, and so…
87 00:11:30.740 ⇒ 00:11:31.380 Samuel Roberts: Right.
88 00:11:31.380 ⇒ 00:11:37.619 Farhaj Shahid: Yeah, and so there was a lot of back and forth after that moment because, yeah, like, some of the models were not it.
89 00:11:38.250 ⇒ 00:11:39.959 Samuel Roberts: Definitely, definitely, yeah, it’s, it’s…
90 00:11:40.880 ⇒ 00:11:46.730 Samuel Roberts: I feel that every time a new model’s out, it’s like, okay, we gotta really know what it does, what are its quirks, what are its…
91 00:11:46.730 ⇒ 00:11:47.450 Farhaj Shahid: You know.
92 00:11:47.450 ⇒ 00:11:52.409 Samuel Roberts: how much does it hallucinate, how much is it not, you know, and just general use before we even start messing with it.
93 00:11:52.410 ⇒ 00:12:04.710 Farhaj Shahid: And, like, online benchmarks are good, but at the same time, I feel like sometimes, like, companies can, like, skew them in one way or the other at the same time, which is why it’s always good to test it on, like, for example, your use cases, you know?
94 00:12:04.710 ⇒ 00:12:18.919 Samuel Roberts: Exactly, exactly. Yeah, we had a… we had a case where we were switching over, client work from… we had been kind of using our AWS instances, so it was mostly, like, you know, OpenAI, GPT 4.0, I think, at that point, and we were switching over to Gemini, and
95 00:12:18.920 ⇒ 00:12:28.790 Samuel Roberts: you know, there was a little bit of a push to do it, and I was like, we have to really make sure these answers are still good, because, you know, it’s such a different tool, potentially. Great, great. Yeah, yeah.
96 00:12:28.790 ⇒ 00:12:31.749 Farhaj Shahid: Was that because Gemini is just cheaper to use? Was that…
97 00:12:31.750 ⇒ 00:12:34.380 Samuel Roberts: You were actually moving everything onto their GCP.
98 00:12:34.380 ⇒ 00:12:35.430 Farhaj Shahid: Okay, okay.
99 00:12:35.430 ⇒ 00:12:50.650 Samuel Roberts: So it just became… that’s what we’ve got, and, you know, it made sense to, like, get into their… their cost center and deal with… and they could deal with it, that way, and they could, you know, tweak things a little bit more, but mostly it was… we were running it ourselves for testing and getting it going.
100 00:12:50.650 ⇒ 00:12:51.130 Farhaj Shahid: Okay.
101 00:12:51.130 ⇒ 00:12:59.640 Samuel Roberts: But at a certain point, you know, they want to own it, because we don’t like to lock people in. You know, some places will, you know, really kind of…
102 00:13:00.390 ⇒ 00:13:07.829 Samuel Roberts: Yeah, we worked with another client that was coming from another dev shop, and they had such proprietary code that it was hard to extract what their actual…
103 00:13:07.990 ⇒ 00:13:21.810 Samuel Roberts: product was out of it, and we… we came in, and we were like, okay, we will… the Git repo will be on your account, like, we will make sure to use all… it was just kind of… we… we were very okay with that, because that’s… we want to provide value, we don’t want to lock you in, necessarily, in a bad relationship, so…
104 00:13:21.810 ⇒ 00:13:29.920 Farhaj Shahid: And I really love that, that you don’t want to lock people in, because especially, like, as time goes on, models can degrade, people, like, companies also, like, degrade them on purpose at times.
105 00:13:29.920 ⇒ 00:13:31.230 Samuel Roberts: Yeah, yeah.
106 00:13:31.230 ⇒ 00:13:32.219 Farhaj Shahid: And out, you know, especially…
107 00:13:32.220 ⇒ 00:13:49.030 Samuel Roberts: I’m very scared of vendor lock-in in so many ways, and I have been historically, but with AI models, I’m always… I’m even more worried because, yeah, like, if you’re hitting OpenAI’s endpoints, you don’t know what could change, you know, you gotta keep on top of that, so yeah, that’s really good.
108 00:13:49.590 ⇒ 00:13:56.990 Samuel Roberts: Okay, great, let’s, what time are we at? Okay, we’re… I got a couple more questions, and then we’ll switch over. So let’s talk about,
109 00:13:57.600 ⇒ 00:14:07.739 Samuel Roberts: I like this one. So if you had 6 months, you know, you’re about to graduate, pretend you had 6 months of no obligations after that, what would, what would you work on? What would your, you know, what would you spend your time doing?
110 00:14:07.740 ⇒ 00:14:23.889 Farhaj Shahid: Yep, exactly. So, this is a good question, and, like, this is something that I’ve been thinking about for a while, especially towards the end of this semester. Like, I’m kind of still in the process of looking for my next obsession, you know, to pour myself into, but it definitely is, like, in technology and within the AI ecosystem, because
111 00:14:23.910 ⇒ 00:14:34.850 Farhaj Shahid: it’s just changing so rapidly, and I sort of like that change. I’m a fast learner, and I love to stay on top of things, and I think the potential to, for example.
112 00:14:35.490 ⇒ 00:14:41.080 Farhaj Shahid: make so much change, and, like, I guess 10X, I guess, your productivity at this.
113 00:14:41.080 ⇒ 00:14:41.810 Samuel Roberts: Same.
114 00:14:41.810 ⇒ 00:15:01.289 Farhaj Shahid: is so real when you’re working with AI, and I really think, like, in the coming future, like, the people that know how to harness its true power are gonna be in a much better position, I guess, than someone who’s completely non-technical, doesn’t know anything about it, you know? So, yeah, like, really gonna be diving more deeper into the AI ecosystem.
115 00:15:02.020 ⇒ 00:15:12.039 Samuel Roberts: Great, great. Let’s see. Is there something that you’ve stuck with outside of work or school, I guess, over multiple years? Is there, you know, a…
116 00:15:12.250 ⇒ 00:15:12.600 Farhaj Shahid: Yeah.
117 00:15:12.600 ⇒ 00:15:16.370 Samuel Roberts: Bobby could be a pro… anything, really, like, that, has been…
118 00:15:17.000 ⇒ 00:15:17.919 Samuel Roberts: term for you.
119 00:15:18.350 ⇒ 00:15:22.459 Farhaj Shahid: Okay, long term. Like, okay, so…
120 00:15:22.640 ⇒ 00:15:29.749 Farhaj Shahid: like, I guess based on my college career, like, long-term could be, I guess, multiple semesters, or multiple…
121 00:15:29.750 ⇒ 00:15:30.550 Samuel Roberts: Sure, yeah, yeah.
122 00:15:30.550 ⇒ 00:15:45.479 Farhaj Shahid: You know, yeah, so I’ve, I’ve been part of the digital accessibility research lab for two years. Yep, did a lot of work for them. For example, we worked with small businesses, acted as an accessibility consultant, and, for example, analyzing
123 00:15:45.480 ⇒ 00:16:01.150 Farhaj Shahid: For example, if a business has a predominantly, like, older target audience, and if their website is not as accessible to them, especially if people were impaired, we sort of analyze their tech stack, what they were doing wrong, what needs to be fixed, and doing it for them.
124 00:16:01.150 ⇒ 00:16:17.110 Farhaj Shahid: So that was pretty cool. At the same time, I’ve been a TA and a tutor for the CS department for a while, and just mainly dealing with data structures and Python tutoring. And I was also part of the Entrepreneurship Center here at Bucknell, for 2 years.
125 00:16:17.380 ⇒ 00:16:33.210 Farhaj Shahid: as a student ambassador and a founder, but that was a really great experience. They help students, for example, with subsidizing AI costs, for example. Like, they gave me a grant to, like, for example, experiment with the RAG models that I was building, you know?
126 00:16:33.260 ⇒ 00:16:37.710 Farhaj Shahid: And, and I was also part of the consulting club for a while.
127 00:16:37.710 ⇒ 00:16:38.230 Samuel Roberts: Typical.
128 00:16:38.230 ⇒ 00:16:51.680 Farhaj Shahid: I was part of the South Asian Students Association for all four years. I was the president, and recently just handed over to the junior class, so I think, yeah, that for sure is something that I’ve been part of my entirety of my four years here at college.
129 00:16:51.680 ⇒ 00:17:05.059 Samuel Roberts: Great. That sounds really good. Okay, yeah, we’re… I mean, I have more things I can ask, but we’re at halfway, so I want to switch around and answer any questions you have, whether that’s, you know, the company, the role, whatever you’ve got, yeah, so…
130 00:17:05.069 ⇒ 00:17:11.859 Farhaj Shahid: Okay, so one question I wanted to ask is, like, what’s been your personal most, like, interesting project that you guys have worked on so far?
131 00:17:12.000 ⇒ 00:17:15.719 Samuel Roberts: Oh, personally, the most interesting project.
132 00:17:15.970 ⇒ 00:17:21.570 Samuel Roberts: Okay, so there’s kind of two ways that I could go about answering this, because we’ve done…
133 00:17:21.569 ⇒ 00:17:39.280 Samuel Roberts: So the… a little background, Brainforge is kind of a data consultancy. You know, AI started growing using those tools internally, kind of spun out this AI team, not only for doing internal stuff, but also client stuff at that point. So we… we have kind of internal tools that we’ve built. We have a little platform that, like, ingests
134 00:17:39.280 ⇒ 00:17:49.910 Samuel Roberts: meetings, and you can find the transcripts, and look up, you know, what was the client talking about this day, that day. And so we’ve spent a little bit of time, you know, playing around with that stuff, and what’s fun about that is, since it’s not
135 00:17:50.420 ⇒ 00:18:00.299 Samuel Roberts: you know, client-facing. That question earlier about, like, production-ready, we have gotten to play with a lot of fun, you know, bleeding-edge kind of stuff that you might not want to, you know.
136 00:18:00.390 ⇒ 00:18:18.670 Samuel Roberts: lean on necessarily in production yet. So that’s… that’s fun. Some of the internal tooling, we get to, you know, play around with all kinds of things that way. The client work is, you know, we’ve done RAG pipelines, we’ve done chat, we’ve done some simple, you know, people were using Claude a certain way and copying and pasting, and basic automations.
137 00:18:18.760 ⇒ 00:18:21.470 Samuel Roberts: I’m trying to think, like, what I find the most interesting here.
138 00:18:23.100 ⇒ 00:18:30.929 Samuel Roberts: I think one of the things I’m doing right now is we actually have a client, and it’s a… we’ve built a chatbot for their customer service
139 00:18:31.340 ⇒ 00:18:37.749 Samuel Roberts: representatives when they’re on calls with people. So, like, when you call them, you’re still talking to a person, but they used to…
140 00:18:39.690 ⇒ 00:18:56.360 Samuel Roberts: have to jump around a ton of documents to figure out, okay, you know, you call to this department, you gotta assign someone to work, and so, like, we’ve built a whole system that knows, okay, you can… this person’s the person to assign, and, here are all the policies, and the chatbot has access to all that, and so instead of having to go through all these documents,
141 00:18:56.360 ⇒ 00:19:02.859 Samuel Roberts: What we’ve now started doing a little bit is they, you know, have transcripts of all these calls that, in addition to
142 00:19:02.900 ⇒ 00:19:18.679 Samuel Roberts: learning how the CSRs are doing, we can also then figure out, okay, where’s the chatbot most helpful, where is it not most helpful? And so we’re starting the process now of ingesting these thousands of calls a day that they get, because it’s a huge volume, as I learned recently when I tried to start pulling two weeks of data and got
143 00:19:18.680 ⇒ 00:19:38.440 Samuel Roberts: barely a day in before it was the end of the day while it was running. So, I think it’s really interesting that, you know, we can kind of come at it from a different angle now, because, you know, the right pipeline is good, and it works, and there’s, you know, things to fix as data changes and all that, but I think the tying that back in now and creating, like, a full feedback loop is really going to be,
144 00:19:38.820 ⇒ 00:19:47.009 Samuel Roberts: great for us, and great for them. You know, they can learn, you know, what are people canceling for, and how are those conversations going, where can we improve those?
145 00:19:47.010 ⇒ 00:19:47.400 Farhaj Shahid: 100%.
146 00:19:47.400 ⇒ 00:19:48.899 Samuel Roberts: But also, you know, yeah.
147 00:19:49.040 ⇒ 00:19:56.640 Farhaj Shahid: Is the company, like, or are you guys going to be exploring potential voice agents for customer service? Because there have been some really cool developments in that area.
148 00:19:56.640 ⇒ 00:20:03.130 Samuel Roberts: Yeah, yeah, we did a little bit of exploration there. I think their… big on…
149 00:20:03.460 ⇒ 00:20:04.829 Samuel Roberts: you know, the personal touch.
150 00:20:04.830 ⇒ 00:20:05.199 Farhaj Shahid: There’s some.
151 00:20:05.200 ⇒ 00:20:19.850 Samuel Roberts: And so, you know, I think there’s maybe some room for that, maybe, like, initial screening, but I… and I can appreciate that. You don’t necessarily always want to get on the phone and hear an automated voice or an AI voice now, and I think they… their customers specifically, they’re a…
152 00:20:19.850 ⇒ 00:20:26.600 Samuel Roberts: like, home and commercial services company. So, you know, their customers might not be the most technical people either, and sometimes they can get…
153 00:20:26.840 ⇒ 00:20:33.719 Samuel Roberts: frustrated or annoyed if it’s not working, they just need to talk to someone and say, I need someone to come out and deal with these pests or whatever.
154 00:20:33.720 ⇒ 00:20:49.670 Samuel Roberts: So I don’t know if that project specifically, but we have played around a little bit with some of that, just internally a little bit. We put together a… we do a lot of case studies based on our work, so that, you know, for sales and getting, you know, more clients, and we had put together an agent that,
155 00:20:49.860 ⇒ 00:21:04.199 Samuel Roberts: using a voice, you know, model would chat with someone rather than have to get the marketing team to set up time with an engineer and do all that, because it’s always the same questions, but that synchronous time was the bottleneck, so we’ve had some fun with that.
156 00:21:04.460 ⇒ 00:21:14.209 Farhaj Shahid: Interesting. Okay, and I guess the next question is, like, for our meeting, like, I looked you up on LinkedIn, I saw you were, like, Mechie, right?
157 00:21:14.210 ⇒ 00:21:16.759 Samuel Roberts: Yes, yes, oh, wow, yeah, yeah, totally, totally.
158 00:21:16.760 ⇒ 00:21:22.769 Farhaj Shahid: Switch, and, like, I think it must have been so interesting, I guess, like, diving into, like, then the CS and development side after that.
159 00:21:23.000 ⇒ 00:21:47.920 Samuel Roberts: Yeah, so it’s… it’s a long… I’m, you know, happy to dig in more, but, yeah, so in college, I… I studied mechanical engineering. My final majors were mechanical engineering, physics, and history and philosophy of Science. Oh, nice. And, yeah, it was one of those things where I had done engineering physics for a while. I came in with a lot of credits and was trying to figure out how I could do multiple things. In hindsight, I probably would have preferred to do mechanical and computer science, because I have always liked computers and programs.
160 00:21:47.920 ⇒ 00:21:53.600 Samuel Roberts: programming, but I also liked physics and kind of went down that route. And so when graduation came, I…
161 00:21:53.610 ⇒ 00:22:08.549 Samuel Roberts: had gone to a startup weekend with a friend, and we had kind of thrown together an idea, and ended up getting accepted to an accelerator in Cleveland. And so, in addition to that, at the same time, I was also doing a program called Venture for America.
162 00:22:08.610 ⇒ 00:22:22.780 Samuel Roberts: Which was, a program that would take recent grads and place them at startups around the country, and, you know, not New York or San Francisco, but, like, other cities kind of thing, and Cleveland was one of them, so I stuck around. And I learned a ton about Node.js, and this was back in
163 00:22:23.040 ⇒ 00:22:38.449 Samuel Roberts: 2013, so, you know, much newer technology. You know, I knew JavaScript, but it was, like, JavaScript on the server. It was, like, I didn’t have the whole web framework yet in my head, and so I learned a ton there. I was doing… that company was doing devices for
164 00:22:38.450 ⇒ 00:22:51.139 Samuel Roberts: doing energy audits of buildings. So, the node backend was managing all these sensors coming, the data coming in and everything, and so I did a bunch of that, but at the same time was, you know, calibrating sensors and doing more, you know.
165 00:22:51.140 ⇒ 00:22:59.600 Samuel Roberts: physical engineering stuff, I guess. And so, with that, and the other company I was doing, we had hired some devs, because I was kind of part-time on that while my friend was full-time.
166 00:22:59.600 ⇒ 00:23:13.459 Samuel Roberts: they didn’t work out great, and we were… he was an electrical engineer, and I was a mechanical engineer, and we were like, we can… we can learn this. And, we just went deep in it, and now we’re both kind of in… in tech still, so, it’s been… yeah, it’s been an interesting journey, so, yeah.
167 00:23:13.650 ⇒ 00:23:19.870 Farhaj Shahid: No, I love that when you said, like, we’re engineers, we can figure it out, you know? No problem is too big or small.
168 00:23:19.870 ⇒ 00:23:33.830 Samuel Roberts: Yeah, I mean, you know, and it’s interesting, because I’ve gone back and forth, you know, I did a hair care startup as well for a number of years, I don’t know if you saw that on there, but that was way less technical in this sense, it was a lot more operations. But what I missed about
169 00:23:33.910 ⇒ 00:23:37.519 Samuel Roberts: About coding was that the feedback loop is just so quick.
170 00:23:37.520 ⇒ 00:23:55.170 Samuel Roberts: You know, you build something, you model something for a project, and it’s fabricated, and it’s got to come back, and you gotta test it, whereas, you know, you can get immediate feedback whether or not your code worked, kind of, sometimes. So it’s… I like that feedback loop a lot, and I’ve… Nice. Yeah, it’s kind of why I’ve stuck around and checked this way, so… yeah.
171 00:23:55.400 ⇒ 00:23:59.820 Farhaj Shahid: Okay, yeah, I think those were all of my questions.
172 00:23:59.820 ⇒ 00:24:00.560 Samuel Roberts: Okay.
173 00:24:00.560 ⇒ 00:24:04.570 Farhaj Shahid: And I really enjoyed talking to you, Sam. It was, like, it’s fun.
174 00:24:04.570 ⇒ 00:24:11.779 Samuel Roberts: Yeah, no, definitely, I, you know, interviews can kind of go different ways, and this has been, this has been really enjoying to speak with you as well.
175 00:24:12.330 ⇒ 00:24:20.919 Samuel Roberts: Before we, you know, totally finish the rest of the process, I don’t know, what you’ve, you know, been told so far, but, you know, this is the first
176 00:24:20.950 ⇒ 00:24:40.630 Samuel Roberts: You passed the first gate of a screening, and now this is the first of three chats? Yes. Okay, and so the next one will be a little more technical, a little more role-focused. This was a little more, you know, cultural fit kind of thing. And so, after that second one, if you pass through that gate, there’s a take-home, you know, tech assessment kind of thing.
177 00:24:40.630 ⇒ 00:24:45.370 Samuel Roberts: And then a, like, presentation panel interview kind of step.
178 00:24:45.580 ⇒ 00:24:46.060 Farhaj Shahid: And so…
179 00:24:46.060 ⇒ 00:25:01.369 Samuel Roberts: So you would take the work you did there, you know, maybe put together a presentation, or slides, or however you would prefer to, you know, present it, because we do a lot of client-facing work, and we talk to clients a lot, so we like to make sure that people can do a bit of that. And then, yeah, you know, well, obviously, that
180 00:25:01.370 ⇒ 00:25:09.429 Samuel Roberts: presentation, plus the code, and, you know, and the other interviews, and then after that would be an offer or not. So, we like to move relatively quickly.
181 00:25:09.430 ⇒ 00:25:18.710 Samuel Roberts: I think I’ve been saying the biggest bottleneck is just getting synchronous time with, you know, people on the team. But besides that, you know, you should hear back pretty quickly one way or another.
182 00:25:18.890 ⇒ 00:25:20.340 Samuel Roberts: Yeah.
183 00:25:20.340 ⇒ 00:25:25.010 Farhaj Shahid: That sounds good. If you could tell me more, I guess, about the potential next round, like, the.
184 00:25:25.010 ⇒ 00:25:25.470 Samuel Roberts: Yeah.
185 00:25:25.480 ⇒ 00:25:30.899 Farhaj Shahid: what to sort of expect, because I know a lot of companies have, like, really different ways that they would like to talk.
186 00:25:30.900 ⇒ 00:25:37.699 Samuel Roberts: Totally. I… I don’t… I wouldn’t consider it, like, a technical interview. You’re not gonna be on…
187 00:25:37.810 ⇒ 00:25:44.820 Samuel Roberts: you know, coding live in front of anyone, anything like that. It’s more… questions will be a little more focused on…
188 00:25:46.770 ⇒ 00:25:55.679 Samuel Roberts: you know, where I asked a few things about LLMs and stuff, they’ll probably go in a little bit deeper about, like, what would you do in this situation? Or, you know, how do you make these trade-offs, or,
189 00:25:55.830 ⇒ 00:26:13.229 Samuel Roberts: you know, what… you know, basically trying to pull out a little bit more of, like, what you can do, and then the third one would be, okay, let’s actually give you a project and, you know, a few hours to work on it, and see what comes back. And that’s the technical side, but it’s not… not live in front of us or anything like that, you know, you have your own time to do that, so…
190 00:26:13.230 ⇒ 00:26:16.919 Farhaj Shahid: Okay, so the second one is more of an in-depth technical conversation.
191 00:26:16.920 ⇒ 00:26:24.740 Samuel Roberts: Yeah, yeah, we call it, like, the role-focused. This was, like, more cultural, but yeah, it’s not, you know, solve this problem in front of me kind of questions.
192 00:26:24.740 ⇒ 00:26:28.020 Farhaj Shahid: Yeah, because I was just wondering, like, do I expect a leakhood, or what?
193 00:26:28.020 ⇒ 00:26:31.269 Samuel Roberts: No, no, no, no, no, no. It’s… yeah, no, not that.
194 00:26:31.270 ⇒ 00:26:39.709 Farhaj Shahid: Yeah, and I guess, I guess what you’re… I guess the third round is, I guess, where you’re gonna be given a take-home assignment, and then you’re gonna present your findings.
195 00:26:39.710 ⇒ 00:26:56.429 Samuel Roberts: Correct. So, yeah, there won’t be any live coding or lead code questions in front of us or anything like that. And, you know, we use AI coding tools all the time, so, you know, we don’t expect you to hand code everything, but that’s sort of where the presentation comes in, because you still got to understand what’s being generated, so…
196 00:26:56.430 ⇒ 00:27:10.480 Farhaj Shahid: No, I think this is a really cool, interactive way of working in the morning, right? And, like, I think this is… I would love to obviously skip to the round three and just, like, do the take-home and present my findings, but yeah, I’m looking forward to talking to the team more.
197 00:27:10.480 ⇒ 00:27:34.840 Samuel Roberts: Yeah, yeah, and that’s the other part of it, is you get to, you know, talk to more people on the team, and more people on the team get to get time with you, and so we can really get a better evaluation of candidates that way, so yeah. So if that all sounds good, yeah, you should hear back, you know, hopefully in a day or two. I will say there’s a… I have a bunch of interviews in the next day or two, so it might be a minute before I get to process everything and get with the team and all that, but by the end of the week, I’m sure you’ll hear one where
198 00:27:34.840 ⇒ 00:27:37.610 Samuel Roberts: Another about, next steps from the recruitment team.
199 00:27:37.850 ⇒ 00:27:39.719 Farhaj Shahid: Thank you so much, Sam, it was a pleasure talking to you.
200 00:27:39.720 ⇒ 00:27:43.139 Samuel Roberts: Yeah, you as well. Good luck with graduation and everything, and yeah.
201 00:27:43.140 ⇒ 00:27:45.120 Farhaj Shahid: Thank you. Have a good one. Bye-bye.
202 00:27:45.120 ⇒ 00:27:45.860 Samuel Roberts: You as well.