Meeting Title: Brainforge AI Engineer Interview Date: 2026-04-20 Meeting participants: Samuel Roberts, Aarish
WEBVTT
1 00:00:19.360 ⇒ 00:00:20.370 Samuel Roberts: Hello.
2 00:00:21.500 ⇒ 00:00:23.989 Aarish: Hey, hi, Sam. Thanks.
3 00:00:24.390 ⇒ 00:00:26.070 Aarish: Who… who’s with me.
4 00:00:26.350 ⇒ 00:00:29.199 Samuel Roberts: Of course, sorry, this is gonna be… there we go, okay.
5 00:00:29.410 ⇒ 00:00:30.790 Samuel Roberts: How are you doing today?
6 00:00:31.350 ⇒ 00:00:32.690 Aarish: I’m good, how are you?
7 00:00:32.910 ⇒ 00:00:35.610 Samuel Roberts: Doing well, doing well. Yeah, thanks for taking the time as well.
8 00:00:36.050 ⇒ 00:00:53.039 Samuel Roberts: I appreciate it. I hope we can make the most of it. So, I think the plan is I have some questions. I imagine you have some questions, so I want to make sure we have time for that. So I’ll probably ask my questions to get about halfway through this call, leave you time for questions,
9 00:00:53.110 ⇒ 00:01:08.460 Samuel Roberts: If nothing, if there’s still questions, obviously. And then, I might have a few more, we can just chat, we’ll see how it goes, but I just want to make sure that we leave that kind of time for you. So, I think just to start, could you give me, just a quick intro to yourself, and, we can go from there?
10 00:01:08.680 ⇒ 00:01:11.589 Samuel Roberts: And then I’ll do the same for me, and then we’ll go, yeah.
11 00:01:11.760 ⇒ 00:01:20.569 Aarish: Sure, sure. Yes, so, I’m Arish. I’m an AI… I’m an AI engineer currently working at Scale Focus.
12 00:01:21.210 ⇒ 00:01:39.300 Aarish: So, most of my work, actually, over the last couple of years, have been around building agents and evaluating them across different frameworks, right? So I’ve built, like, flag bots, analytics co-pilots, etc, and then I’ve built a very good harness and an eval suit for all of these agents.
13 00:01:39.300 ⇒ 00:01:55.610 Aarish: So worked with pretty much every, EVY tooling which is there in the market as of now, which is LangFuse, or Icephoenix, or BrainTrust, actually, anything, and worked with also RAGAS and Deep Agents to evaluate these agents as well.
14 00:01:56.900 ⇒ 00:02:01.990 Aarish: So recently, I’ve been actually engaged in comparing different hardnesses.
15 00:02:02.110 ⇒ 00:02:19.550 Aarish: So be it, like, Hermes agent or OpenClaw, and decided that we wanted to actually try and create out our own harness as well. So, we kind of built our own harness, which is built on top of Cloud Code SDK. Okay. That is what I’ve been up to.
16 00:02:19.940 ⇒ 00:02:32.810 Samuel Roberts: Great, that’s exciting, that’s cool. Yeah, so a little bit about me. I’m Sam Roberts, I’m the AI tech lead here at Brainforge. I’ve been here since about July, and we’re growing, pretty…
17 00:02:33.010 ⇒ 00:02:37.960 Samuel Roberts: pretty quickly, looking to bring on other engineers, so I’m glad we were able to have this call. So,
18 00:02:37.960 ⇒ 00:02:53.740 Samuel Roberts: Yeah, if nothing else, I’d love to just jump into some questions. Thank you for that background context, that’s very helpful, actually, as well, and I’m really curious other things you found about harnesses, because I’ve been, you know, exploring things myself, and we’ve been exploring things here, so maybe we can chat about that if we get some time, so that’d be great.
19 00:02:53.740 ⇒ 00:03:02.149 Aarish: Yeah, definitely, yes. Okay, great. I’ll tell you my experience, why I started building up on my own hardness, right? That was my
20 00:03:03.370 ⇒ 00:03:16.229 Aarish: when OpenClaw was there, right, it had a lot of context pollution, so when I was replying it on thread and all, right, it was not able to pick up the current context, and that was the primary motivation for me to, go up and forge up a
21 00:03:16.230 ⇒ 00:03:23.960 Aarish: Harness for myself, but we kind of wanted to design it in a way that it’s not a personal agent, but it’s spread across an entire organization, so it’s a.
22 00:03:23.960 ⇒ 00:03:24.690 Samuel Roberts: Hmm, okay.
23 00:03:24.690 ⇒ 00:03:38.060 Aarish: agent for your workspace, right? For all of your company, for all of your employees, basically. So it lives inside Slack workspace, and it has, like, a common shared agent, which every employee can share.
24 00:03:38.520 ⇒ 00:03:40.569 Samuel Roberts: That’s cool. I like that. I like that.
25 00:03:40.970 ⇒ 00:03:51.790 Samuel Roberts: Great. Okay, so let’s… let’s actually dig into something like that. So, you know, you talked a little bit about, like, what you’ve built, but I would love to learn about, the kind of…
26 00:03:52.000 ⇒ 00:03:53.830 Samuel Roberts: more,
27 00:03:53.950 ⇒ 00:04:16.079 Samuel Roberts: nitty-gritty part of the, like, problems that it was solving, I guess? Think about it that way. So, like, you know, you’re building this agent. What kind of problems were you running into that other harnesses maybe weren’t accounting for? You kind of talk a little bit about the whole workspace, but I’m curious what you were able to do with that, and make it work better for a whole team.
28 00:04:17.120 ⇒ 00:04:22.419 Aarish: Right, so one thing which was important to us is, like, if you create a common shared agent, right.
29 00:04:22.430 ⇒ 00:04:44.449 Aarish: There’s one blocker that how do you restrict access to personal tools? So let’s suppose if I’m using Granola, right? Those meeting notes are personal to me, right? So if you and I are in our common workspace, or in our common channel, right? I don’t really want to expose my meeting notes to you, or if I’m asking a question, I would actually want my meetings to be referenced and not yours meetings, so it’s not possible to have a…
30 00:04:44.450 ⇒ 00:04:48.250 Aarish: Common, integration as per se, right?
31 00:04:48.250 ⇒ 00:04:52.890 Aarish: And all of the MCP integrations as well rely on the fact that you can only configure one
32 00:04:52.890 ⇒ 00:04:55.799 Aarish: account per bot, right?
33 00:04:55.840 ⇒ 00:05:10.489 Aarish: this was the most challenging part that we actually solved via something called as Credentials Proxy. So what Credentials Proxy basically does is that you create a common proxy layer for Granola. You send the… so, when I’m sending
34 00:05:10.490 ⇒ 00:05:24.400 Aarish: a message, right? I have my Slack user ID with me. So, at the time an agent queries the integration, may it be skill, ClaudeSkill, or any MCP integration, right? I have a proxy layer which is sitting.
35 00:05:24.400 ⇒ 00:05:33.219 Aarish: it’s going to hit the proxy layer with my Slack user ID, and that proxy server basically resolves my user ID to my credentials, my granola credential.
36 00:05:33.220 ⇒ 00:05:35.149 Samuel Roberts: Nice. Okay, that’s smart.
37 00:05:35.430 ⇒ 00:05:38.179 Aarish: That is… that is how this was wired up.
38 00:05:38.350 ⇒ 00:05:49.210 Aarish: On the bottom of my head, I, had some challenges which I did not see in Hermes Agent or OpenClaw, but around heartbeats and crunch. So that was…
39 00:05:49.300 ⇒ 00:06:06.399 Aarish: That was something that was actually quite difficult to build. On a surface level, it seems like it may not be difficult to build such kind of a system, but there were some hiccups on the road along… while building the heartbeat thing, and I think OpenClaw has done it very craftily, I would say that.
40 00:06:06.540 ⇒ 00:06:07.899 Aarish: The hard dual parts.
41 00:06:08.090 ⇒ 00:06:08.840 Aarish: Yep.
42 00:06:09.030 ⇒ 00:06:16.819 Samuel Roberts: Nice, nice. Alright, cool, cool. Let’s, let’s jump a little bit, and let’s talk about working with, non-technical stakeholders.
43 00:06:17.060 ⇒ 00:06:18.550 Samuel Roberts: So,
44 00:06:19.000 ⇒ 00:06:34.940 Samuel Roberts: Let’s say… yeah, so I’m curious about, you know, we work with these LLMs and these non-deterministic tools now, not… and everyone hears about what AI can do. I’m wondering, how do you communicate the limitations of some of these things to people that may not be technical?
45 00:06:36.000 ⇒ 00:06:37.010 Aarish: Right, so…
46 00:06:37.280 ⇒ 00:06:44.060 Aarish: Just for context, for the past year or so, I mean, I’ve, directly been involved in the client calls. In fact, I mean…
47 00:06:44.180 ⇒ 00:06:48.890 Aarish: I’ve been directly engaging with our clients and have my clients as well.
48 00:06:49.270 ⇒ 00:06:54.339 Aarish: If you have my resume, and you could see that I also worked at Future Path, there we were actually…
49 00:06:54.530 ⇒ 00:07:08.109 Aarish: dealing with Fortune 500 companies as well, so there was coal energy, and there was Kalyard as well. Primarily for non-technical stakeholders, right, we do want… we do not want to include technical jargon to them.
50 00:07:08.110 ⇒ 00:07:15.349 Aarish: We want to explain as easy as possible, so we mostly focus on the outcomes of the system and not how the
51 00:07:15.350 ⇒ 00:07:24.529 Aarish: thing is being… how the process is basically being translated, right? So possibly what we do is that we try to get out the expectations from them.
52 00:07:24.530 ⇒ 00:07:35.909 Aarish: After getting the expectations, we, focus entirely on the outcomes, and the input of the system, and how it can affect their business operations, basically.
53 00:07:37.360 ⇒ 00:07:37.920 Samuel Roberts: Great.
54 00:07:38.090 ⇒ 00:07:50.239 Samuel Roberts: Has there been a time when someone… a user misunderstood what the AI feature you were building could do? And… and how did you resolve that, kind of misunderstanding, if there was one?
55 00:07:50.250 ⇒ 00:07:54.000 Aarish: A lot of times that this has happened, so,
56 00:07:54.440 ⇒ 00:08:12.420 Aarish: basically, there are a lot of times when we are not able to converge on a point that this is necessary for this outcome to be delivered. So, for example, if, let’s suppose if we are working on something, let’s suppose there’s a target that we want to meet, so I’m going to take a metric for this. Let’s suppose if retention is something that we are focusing, right?
57 00:08:12.500 ⇒ 00:08:21.540 Aarish: Now we can have, we can shoot 10 ideas to improve that retention, right? But,
58 00:08:22.410 ⇒ 00:08:26.199 Aarish: the ideas that we want to implement, right?
59 00:08:26.240 ⇒ 00:08:47.149 Aarish: do we implement them one by one, or can we sort of build, like, a system which can… so, if you would have seen recent, tweet about Carpathy’s auto research tool, right? So that was interesting to us, and we kind of said that, hey, we want to leverage this so that this is, you know, usable to you in long term, and we don’t want to leave you hanging with
60 00:08:47.150 ⇒ 00:09:03.060 Aarish: There’s that, hey, we are doing one thing which improves your retention, but then our work is basically expired, right? It does not guarantee long-term, returns to you. So, we kind of wanted to do an… or build an auto-research system for them, which is going to take,
61 00:09:04.270 ⇒ 00:09:23.799 Aarish: any metric, like retention, right? And it’s going to sample out your production data, and then it’s going to change your production system to calibrate against that single matrix. So let’s suppose if Auto Research is able to come up with 10 different suggestions, and it can run it against an eval
62 00:09:23.800 ⇒ 00:09:31.690 Aarish: suit. Let’s suppose if I’m aiming for retention enough, I can define, retention as a function to some eval matrix. So let’s suppose.
63 00:09:32.090 ⇒ 00:09:50.100 Aarish: If I’m working with a consumer company, and retention is something which I can describe as a function of that quality of responses should be dramatically better, right? So, how do I also calibrate against quality of the responses is another problem, but you can… you can assume that this is some kind of a suit, so…
64 00:09:50.100 ⇒ 00:10:05.180 Aarish: Basically, my work research agent is going to come up with 10 different solutions to improving the quality of the responses, right? That’s definitely going to show on eval suit, and then you can run the simulations, basically, and then compare against different eval suits.
65 00:10:05.180 ⇒ 00:10:09.819 Aarish: So that is one way on how to approach the problem. The other way would be that you…
66 00:10:09.900 ⇒ 00:10:16.039 Aarish: Go and manually test out things according to your intuition and what is going to improve your retention score.
67 00:10:16.140 ⇒ 00:10:24.090 Aarish: So this was a challenge on us, basically, translating that what auto research could do for them, right? So, yeah.
68 00:10:24.360 ⇒ 00:10:32.629 Aarish: this is one problem. Another problem, I think, as far as the implementation is concerned, right? If,
69 00:10:33.030 ⇒ 00:10:39.820 Aarish: if implementation is basically scoped out, I think, I’ve had little problem to translate that into outcome, but yes.
70 00:10:40.870 ⇒ 00:10:41.360 Aarish: Yep.
71 00:10:41.360 ⇒ 00:10:48.439 Samuel Roberts: Okay, great. Let’s… let’s talk about, let’s talk about you a little bit. Let’s say,
72 00:10:48.890 ⇒ 00:10:58.949 Samuel Roberts: Is there something that you’ve stuck with outside of work over multiple years? Like, this could be, you know, anything kind of outside of work, but is there anything you can talk about like that?
73 00:11:00.580 ⇒ 00:11:18.960 Aarish: stuck outside of work. I’ve been trying to work on my fitness, and there’s, like, an on and off thing on my fitness thing, so I usually, like, pause after 3 months, or, like, lose motivation to work out after 3 months, but that is something which I’m constantly working on.
74 00:11:18.960 ⇒ 00:11:22.770 Aarish: But, yeah, healthy lifestyle is basically what Oh my god.
75 00:11:22.870 ⇒ 00:11:27.370 Samuel Roberts: Okay, okay, great. And then, is there,
76 00:11:27.990 ⇒ 00:11:38.359 Samuel Roberts: A little hypothetical, I guess, for you, kind of related. If you had 6 months with no obligations, what do you… what do you think you would work on? What would you… what would you spend that time doing?
77 00:11:39.300 ⇒ 00:11:45.509 Aarish: I think I’ve already talked about the credentials proxy layer, right? Yeah. I’ll go a little deeper in this, so…
78 00:11:46.100 ⇒ 00:11:56.190 Aarish: Just searching up about it. So, there’s a thing. So, the agents that I’ve been building, right, I built it under… these agents actually worked in a sandbox environment, E2B sandbox environments.
79 00:11:56.260 ⇒ 00:12:06.110 Aarish: So now what happens is credentials Proxy is a very… a great way to solve it, but it does not completely restrict certain actions. So, how do we restrict certain actions?
80 00:12:06.200 ⇒ 00:12:20.339 Aarish: is basically you intercept all of the requests inside that container, inside that sandboxed environment. So let’s suppose I’ve talked about hitting granola, right? So let’s suppose in my Docker container, or in my VM, right, sandboxed environment.
81 00:12:20.360 ⇒ 00:12:36.989 Aarish: If it sends a request to Granola, I’m going to intercept that network request, and then push the user’s credentials at that level. Things clean, and this is kind of a system which I think is currently lacking,
82 00:12:37.560 ⇒ 00:12:44.000 Aarish: in the current ecosystem. This is not something which somebody is actively working on. Sure.
83 00:12:44.000 ⇒ 00:13:05.920 Aarish: possibly like to work on something like this, and then there’s other problem statement as well, I think which I’ve talked about already, the auto-research loop, kind of a problem. So, let’s suppose if I want to make an agent self-healing, right? So, by self-healing, I mean that, learn from its mistakes, right? So, right now, we log all of the traces and all of the decisions taken by agent to.
84 00:13:06.210 ⇒ 00:13:17.719 Aarish: an observability platform such as Grafana, Langfuse, anything, right? So, I want to be able to build a system which can pull logs from these systems.
85 00:13:17.760 ⇒ 00:13:23.459 Aarish: Right, and with probability, a human in the loop, it can understand from what mistakes
86 00:13:23.480 ⇒ 00:13:39.789 Aarish: it has previously made, and then calibrate itself against those mistakes, right? So, it can be a simple thing, like adding to a system prompt, or it can be a larger thing, like adding to a persistent memory or something else, right? Not clearly thought it out, but this is.
87 00:13:39.790 ⇒ 00:13:40.140 Samuel Roberts: Yeah. Something else.
88 00:13:40.140 ⇒ 00:13:41.530 Aarish: So, which is…
89 00:13:41.530 ⇒ 00:13:45.570 Samuel Roberts: Very interesting stuff. Yeah, I’m really curious about that stuff myself, too.
90 00:13:45.680 ⇒ 00:14:03.769 Samuel Roberts: Okay, one more question, and then we’ll flip it, and you can start asking me questions if you have any questions about the role or Brainforge, but, let’s, we know things are changing rapidly in the AI LLM space. Has there been anything, a trend, model, a framework that you were initially excited about
91 00:14:03.770 ⇒ 00:14:08.090 Samuel Roberts: But decided not to adopt for some reason, and why.
92 00:14:09.940 ⇒ 00:14:18.830 Aarish: Yeah. So this is probably a little while back, but in 2023, I was excited about Lang Graph as a framework, but
93 00:14:19.030 ⇒ 00:14:33.690 Aarish: Langraph had consistency problems back then, but then in 2024, it kind of actually fixed all of its problems. 2020… late 2024, it kind of fixed all of its problems, and it was a we went out of, from Langraph as well.
94 00:14:33.810 ⇒ 00:14:50.280 Aarish: So they’re doing a very good job, actually, in creating state machine kind of systems in AI space, right? That’s something which is credible, but another SDK framework which I was using is Lama Index. I had high hopes from Lama Index, but now I don’t really use it.
95 00:14:51.330 ⇒ 00:15:00.749 Aarish: Okay. So, that is there, and for observability also, I was kind of excited to see what Phoenix is doing, or iThenix is doing, but now I’ve shifted to
96 00:15:00.750 ⇒ 00:15:11.430 Aarish: something better, which is, LangFuse, because I think LangFuse is very, product manager friendly, right? It’s, like, technical people can also use it, whereas
97 00:15:11.430 ⇒ 00:15:15.519 Aarish: As opposed to Lancuse, I mean, Phoenix is a little tech-heavy, kind of a thing.
98 00:15:15.520 ⇒ 00:15:28.179 Samuel Roberts: Definitely, yeah, I found… I found that similarly. Great, okay, yeah. So, yeah, we’re about halfway through, so, I’d love to answer whatever questions you have about the role, or Brainforge, or anything of the work we do, that kind of stuff.
99 00:15:28.620 ⇒ 00:15:42.639 Aarish: Yeah, definitely. So one thing I wanted to check in, like, what are the types of clients that you work with? And I’ve already read through all of your blogs, some of your blogs, and that you do work on Slackbots and everything else as well.
100 00:15:43.300 ⇒ 00:15:51.249 Aarish: Yeah, I just wanted to know, is there, like, a particular domain that you target, or is it, like, you have a wider, wider approach, basically?
101 00:15:51.250 ⇒ 00:16:13.099 Samuel Roberts: Yeah, I think that’s something we’re still kind of figuring out a little bit. We’ve had a lot of the… so, just for a little bit of context, you might know this from the blog post, but Brainforge started as much more of a data consultancy, and so there was a lot more data clients, and then the AI side kind of spun out of that for tools that we were building internally, as well as clients asking about things. And so, a lot of the data clients have been
102 00:16:13.100 ⇒ 00:16:19.800 Samuel Roberts: you know, CPG brands looking to understand all the data that they have from their site, and all the analytics that they have.
103 00:16:19.800 ⇒ 00:16:25.839 Samuel Roberts: And so, some of those sometimes become AI clients, but also other clients,
104 00:16:25.840 ⇒ 00:16:41.140 Samuel Roberts: come in separately, so we have some… they’re a bit spread out in the AI stuff. I think we’re still working on, kind of, identifying our ideal customer profile there. But we’ve worked on things, from… we have a, a big,
105 00:16:41.600 ⇒ 00:16:51.569 Samuel Roberts: home and commercial services company that we’ve built a RAG-based chatbot for their customer service agents to use when talking to customers, so they can look up
106 00:16:51.570 ⇒ 00:17:02.120 Samuel Roberts: procedures and policies and things much, much faster. We’re also working on, a kind of a command center for, like, a COO level of one of the,
107 00:17:02.120 ⇒ 00:17:15.230 Samuel Roberts: a health tech company, actually, so we have some of those, and he’s just looking to try to get a high-level view of things. We’ve worked with some other, kind of, agencies that do, kind of, ad work. We’ve done work with,
108 00:17:15.400 ⇒ 00:17:23.960 Samuel Roberts: Actually, another kind of design agency as well. So, right now in the AI space, we’re doing a lot of different things. The data side’s a little more…
109 00:17:24.589 ⇒ 00:17:33.600 Samuel Roberts: focused at this point, but we’re also trying to grow, so as you can imagine, we’re looking for, you know, bigger and bigger clients that way, so I think that’s shifted a little bit from, kind of.
110 00:17:33.700 ⇒ 00:17:39.609 Samuel Roberts: You know, was more startups, now is more, like, mid-range, and eventually hoping to get to, like, more enterprise stuff, so…
111 00:17:40.380 ⇒ 00:17:45.350 Aarish: Nice, nice. You mentioned something about building a RAG tooling, right?
112 00:17:45.350 ⇒ 00:17:45.730 Samuel Roberts: Yeah.
113 00:17:45.730 ⇒ 00:17:51.799 Aarish: I would just like to, like, know a little bit about how, basically, you…
114 00:17:52.200 ⇒ 00:17:58.029 Aarish: you build this rack system, because I, myself, have built, like, two to three rack systems using Ticket.
115 00:17:58.340 ⇒ 00:18:06.250 Aarish: long chunking rag, and corrective rag, etc. Would just love to know, like, how did you approach this problem?
116 00:18:06.650 ⇒ 00:18:17.030 Samuel Roberts: Yeah, so this is an interesting one, because some of it had been started before I was here, and so it was, we were using N8N initially for prototyping, and that grew into a much…
117 00:18:17.330 ⇒ 00:18:26.920 Samuel Roberts: bigger, workflow than I think N8N was kind of ideal for. You know, N8N is great for certain things, but, especially now…
118 00:18:26.920 ⇒ 00:18:29.069 Aarish: How to break under load, I mean, I was…
119 00:18:29.070 ⇒ 00:18:33.500 Samuel Roberts: That’s exactly what we were finding, yeah. So it was good for the initial, but I think by the time
120 00:18:33.500 ⇒ 00:18:51.790 Samuel Roberts: I joined, and maybe a little bit after we were seeing, it just wasn’t holding up. And so, we shifted that over. So this is… the client is a bit interesting, because a lot of their documents were spread out all over the place, so we worked with them to kind of centralize into a few main documents that we embed.
121 00:18:52.690 ⇒ 00:18:59.919 Samuel Roberts: And so, that all was embedded through NIDN as well, so we shifted some of that over. We’ve been using,
122 00:19:00.090 ⇒ 00:19:18.199 Samuel Roberts: Mostra, which is a TypeScript framework, if you’ve heard of that one. So, we’ve done a lot of TypeScript work that way. We do a little bit of Python here and there, but we’ve been building some UIs as well, so it just kind of made sense to stick with TypeScript full-stack that way. And so, for that, we kind of reimplemented a lot of the N8N stuff.
123 00:19:18.560 ⇒ 00:19:27.149 Samuel Roberts: With this master framework, which has, a number of tools, you know, kind of batteries included, out of the box there. And so, we were able to…
124 00:19:27.390 ⇒ 00:19:42.319 Samuel Roberts: migrate over the embedding pipeline, the actual agents. We actually reconstructed the way the architecture was, because the N8N had a number of different flows, but they were hard to change and hard to…
125 00:19:42.800 ⇒ 00:19:56.050 Samuel Roberts: you know, when we wanted to make some kind of architectural change, it was not easy. And so, with Monster, we’ve been able to do that a bit more easily, and so there were some things that were, for example, from these documents we needed to fetch, we also have a database of,
126 00:19:56.190 ⇒ 00:20:12.350 Samuel Roberts: assignments, so they know, okay, a customer’s calling from this zip code, and they want this service, who do I go to? Which is not in the documents, and that changes fairly frequently, so we put that into a database and have a, you know, kind of text-to-SQL flow there.
127 00:20:12.350 ⇒ 00:20:18.389 Samuel Roberts: then there’s some things that they want very specifically, and so we realize that the LLMs will take the stuff, and they’ll…
128 00:20:18.390 ⇒ 00:20:35.929 Samuel Roberts: explain what the document says sometimes, but sometimes they want an exact template for the customer service agent. So we have another flow that realizes that, and then just returns exact text rather than passing that through an LLM. So there’s a few different flows there, some of which are more rag-heavy, and some of which are not, but,
129 00:20:36.730 ⇒ 00:20:51.229 Samuel Roberts: Yeah, that’s kind of the main one that we’ve been working on for a while there, and a lot of their data’s changing a lot, so it’s… we have to build a whole UI for updating the database for the customer. Right now, it’s, you know, we’re working on a system for them to…
130 00:20:51.230 ⇒ 00:20:56.979 Samuel Roberts: Recognize when the document is out of date, and push changes that we can then auto,
131 00:20:56.980 ⇒ 00:21:03.999 Samuel Roberts: generate the update to the document for them and have them approve it. So there’s a lot of neat little bells and whistles we’re adding there.
132 00:21:04.850 ⇒ 00:21:06.800 Aarish: Nice, sounds, sounds really interesting.
133 00:21:06.800 ⇒ 00:21:11.180 Samuel Roberts: Yeah, yeah, it’s a cool project, and the client is pretty good, too, so…
134 00:21:13.590 ⇒ 00:21:16.370 Aarish: Cool. So how large is your team currently?
135 00:21:16.780 ⇒ 00:21:20.240 Samuel Roberts: Right now, it’s, it’s, we have sort of
136 00:21:20.270 ⇒ 00:21:37.529 Samuel Roberts: three, sort of four engineers. One of the engineers has kind of become more of the client, manager. So there’s… there’s me and two other engineers that work on most of the client work in the AI side, which is why we’re looking to hire, you know, more… more people. We’re…
137 00:21:37.860 ⇒ 00:21:50.559 Samuel Roberts: mostly on those two clients right now, but that changes as clients add, and sometimes churn, or the project finishes, or whatever, so we’ve had, you know, a few more projects here and there, but it’s hard to get the bandwidth with
138 00:21:50.560 ⇒ 00:22:00.429 Samuel Roberts: the limited number of people we have right now, even with the kind of agentic coding tools, which is something I wanted to ask about, because we’ve been trying to lean on those a bit more, which was another good reason to get off of,
139 00:22:00.430 ⇒ 00:22:07.950 Samuel Roberts: N8N and move to… to code, because we were able to scaffold that a lot more… much more quickly.
140 00:22:08.300 ⇒ 00:22:14.310 Aarish: I think a lot of stuff you do on Initane can be easily reproduced on Langgraph state machines, so that is…
141 00:22:14.310 ⇒ 00:22:20.719 Samuel Roberts: Yeah, yeah, and that’s… we… I was looking at line graph, because I had used that previously, but…
142 00:22:20.860 ⇒ 00:22:31.209 Samuel Roberts: when I kind of got interested in Maestra, and I saw what they were doing, and started following a little bit more, they’re kind of trying to do the same thing, but focused on TypeScript, whereas I know all the Lang…
143 00:22:31.210 ⇒ 00:22:43.720 Samuel Roberts: chain, like, graph, you know, is kind of Python-focused, but also has TypeScript, and I found, you know, if we were just going to be doing a full-stack app, we could plug Monster right into certain things, and it was, it was nice, so…
144 00:22:43.920 ⇒ 00:22:44.520 Aarish: Yeah.
145 00:22:44.530 ⇒ 00:22:50.970 Samuel Roberts: But I know exactly what you mean, yeah, like, that state… that state machine stuff is pretty… pretty nice for the line graph thing.
146 00:22:51.270 ⇒ 00:22:51.970 Aarish: Yeah.
147 00:22:53.030 ⇒ 00:22:54.550 Samuel Roberts: Other questions? Yeah.
148 00:22:55.630 ⇒ 00:23:05.569 Aarish: Yes, I think this might be the last, might be the last one. I just want to get an idea, like, how many, how many clients, basically, you work simultaneously with?
149 00:23:05.880 ⇒ 00:23:12.189 Samuel Roberts: Yeah, so, like, as a specific engineer, as one engineer, like, how many would you be on, or the company?
150 00:23:12.190 ⇒ 00:23:14.710 Aarish: No, specific to company, I mean…
151 00:23:14.710 ⇒ 00:23:17.599 Samuel Roberts: Okay, yeah, so we probably have,
152 00:23:18.400 ⇒ 00:23:37.580 Samuel Roberts: I have to think about it. So we have kind of two main projects in the AI team right now. We’ve had a few more here or there that have come and gone, depending on, you know, how things were with the clients. One of them is actually spun out from another of the data clients, so it’s kind of two very distinct projects, but one main client, but we kind of
153 00:23:37.610 ⇒ 00:23:44.139 Samuel Roberts: are answering to different people, as you can imagine, at different places. So, on the data side, I can think of at least another 3 or 4…
154 00:23:44.960 ⇒ 00:23:50.329 Samuel Roberts: that I’m… that I’m aware of. I’m not super up-to-date on the data side there.
155 00:23:50.330 ⇒ 00:23:50.840 Aarish: Go ahead.
156 00:23:50.840 ⇒ 00:23:54.200 Samuel Roberts: So we probably have… At any given time.
157 00:23:54.720 ⇒ 00:23:57.100 Samuel Roberts: What am I thinking? 1, 2, 3, 4, 5, 6…
158 00:23:57.240 ⇒ 00:24:09.939 Samuel Roberts: We have, like, 7 clients right now, but part of that is just what we can service right now, and so that’s why we’re looking to grow, because we’re… we have a lot of, kind of, deal flow happening on the sales side that we need to be able to
159 00:24:10.170 ⇒ 00:24:13.430 Samuel Roberts: To actually service, so… Looking to grow, yeah.
160 00:24:13.430 ⇒ 00:24:25.240 Aarish: How are you planning to basically solve for this? So, do you want the employees to take complete autonomy over one project, or is it a shared effort?
161 00:24:25.240 ⇒ 00:24:29.989 Samuel Roberts: It’s definitely a shared effort, and I think that will continue. You know, there may be…
162 00:24:29.990 ⇒ 00:24:47.959 Samuel Roberts: some technical knowledge that some people specifically have, but at this point, the way it’s mostly worked is we kind of tag-team a number of different things. There may be, you know, you may be more on certain projects or others, especially as you kind of develop the intuition and context for those projects, but,
163 00:24:47.960 ⇒ 00:24:48.659 Samuel Roberts: I don’t…
164 00:24:48.760 ⇒ 00:25:00.290 Samuel Roberts: I don’t love the idea of kind of one engineer on one project and single point of failure that way if someone’s out or someone’s sick. And so we try to kind of keep everyone in the loop that way as best as possible, even if you’re not
165 00:25:00.430 ⇒ 00:25:07.919 Samuel Roberts: on the project currently, like, there may be tickets that you’re better suited for at some point, kind of thing. And we… we love to…
166 00:25:08.150 ⇒ 00:25:18.750 Samuel Roberts: talk through the problems with each other, so, we’re kind of… we used to do, like, a daily stand-up, but we found that it wasn’t always necessary, so now I… we kind of do, like, an office hours, where we… we chat through
167 00:25:18.950 ⇒ 00:25:27.110 Samuel Roberts: you know, what are you working on? How are things… do you have any questions? Can we put a plan together and then talk about it, kind of stuff? Because I’m finding now, especially with,
168 00:25:27.410 ⇒ 00:25:38.889 Samuel Roberts: you know, agentic coding tools, like, the plan matters a lot before you pass it in, and so if we can kind of refine that as a group, I find that to be a much better, approach right now, so…
169 00:25:40.500 ⇒ 00:25:45.090 Aarish: And assuming that most of your communication happens via Slack.
170 00:25:45.090 ⇒ 00:25:56.229 Samuel Roberts: Yes, so we are, we are fully remote, all through Slack. We use Zoom for meetings, as you can see, and then, we have some internal tooling for,
171 00:25:56.750 ⇒ 00:26:18.669 Samuel Roberts: kind of the recordings of the meetings, and then we process the transcripts and surface things, so you can… we have a way to search old meetings and things like that, but… including Slack as well, so we ingest all the Slack messages, the meeting transcripts, linear tickets now, so we’re working on a lot of internal tooling as well, to kind of enable people, engineers and, you know, non-technical folks that are
172 00:26:19.400 ⇒ 00:26:28.230 Samuel Roberts: able to see the deals that are in process, and understand that, and understand the way the work is, and try to really, you know, give a high-level view for all the different roles, so…
173 00:26:28.550 ⇒ 00:26:30.039 Samuel Roberts: But yeah, very Slack-heavy.
174 00:26:30.590 ⇒ 00:26:31.170 Aarish: Yeah.
175 00:26:32.290 ⇒ 00:26:36.879 Aarish: Cole, I don’t think I have any more questions, but… Okay.
176 00:26:36.880 ⇒ 00:26:49.830 Samuel Roberts: Alright, well, we’re getting near time. I was gonna ask, just in general, so you mentioned harnesses and stuff, is there a particular one that you use on a daily basis in terms of, like, agentic coding stuff? Like, I know if you’re working on them, I’m wondering what your, kind of, preferred
177 00:26:49.960 ⇒ 00:26:52.369 Samuel Roberts: coding method is to this day, yeah.
178 00:26:52.370 ⇒ 00:27:00.169 Aarish: I, usually use Cloud Code for my dev experience. For anything…
179 00:27:00.480 ⇒ 00:27:05.310 Aarish: Other than dev experience, like, I have Hermes Agent integrated, so I just do, like…
180 00:27:05.310 ⇒ 00:27:05.840 Samuel Roberts: Yeah.
181 00:27:06.080 ⇒ 00:27:25.959 Aarish: meetings and all for… if I want a summary, or if I want a schedule. So one… one thing where I’m heavily using Homes Agent is basically for follow-ups and for schedules, so… Sure. I just… all the deliverables there, and then I keep a cron open that, hey, remind me of this task every 12 hours or every 24 hours, so that has been really…
182 00:27:26.180 ⇒ 00:27:29.149 Aarish: But yeah, for development, completely on Clotboard. Okay.
183 00:27:29.150 ⇒ 00:27:45.589 Samuel Roberts: Yeah, so we’ve been, using Cursor a lot, kind of going way back, and Cursor’s changed a bunch, and we’re looking at, kind of shifting to open code and a lot of our own hosted Azure models and things like that, but, you know, it’s… I’m just curious where you’re at with that, but…
184 00:27:46.090 ⇒ 00:27:58.619 Aarish: I think Cloud Code can be integrated with your Azure Anthropic integration. That is something which you can do, but if you use Cloud Code via API, it’s a lot more costlier.
185 00:27:58.620 ⇒ 00:28:13.900 Samuel Roberts: That’s exactly… yeah, exactly, and so that’s… that’s a big… and we’re… we’ve kind of got the whole company on cursor, so they’re drafting documents and things, so you can imagine our AI inference costs have really, really spiked, so we’re trying to figure out the best way to manage that as well, but, great.
186 00:28:14.090 ⇒ 00:28:26.980 Aarish: I’ve also tried Codex as well. So, codecs is slightly different than Cloud Code, but because I’ve gotten used to Claude Code so much that it’s, like, a very painful process for me, moving away from Claude Code.
187 00:28:26.980 ⇒ 00:28:34.340 Samuel Roberts: Yeah, yeah, no, I feel… I played with Codex, because I was… I was very… using… using cursor, and I shifted recently to kind of using the agent…
188 00:28:34.520 ⇒ 00:28:41.789 Samuel Roberts: focused mode instead of just the IDE mode, and that was a little bit of a… a mental pattern shift, but… because that was what Codex kind of…
189 00:28:42.060 ⇒ 00:28:58.029 Samuel Roberts: came out and had that nice UI as well, and the GUI. I haven’t been in the terminal nearly as much, but I might be getting back into it with open code now, so we’ll see how. It’s really fun playing with these different tools and figuring out what works best, so…
190 00:28:59.110 ⇒ 00:29:00.840 Samuel Roberts: Been a fun… been a fun place to work, so…
191 00:29:01.170 ⇒ 00:29:07.850 Aarish: Yeah, in fact, I’ve completely shifted from, you know, ID first to actually terminal first, because.
192 00:29:07.850 ⇒ 00:29:08.460 Samuel Roberts: Yeah.
193 00:29:08.950 ⇒ 00:29:14.420 Aarish: Not really sure what changed, but terminal-first experience feels more developer-friendly.
194 00:29:14.420 ⇒ 00:29:33.070 Samuel Roberts: Yeah, that’s… that’s… it’s funny, because I was… I… I was very terminal heavy, and then I got back to cursor a little bit, and then I’ve been using the cursor agent mode, which, you know, is nice to be able to shift between, but I’ve been using, CMUX as my terminal now, which helps me do the same thing.
195 00:29:33.170 ⇒ 00:29:40.720 Samuel Roberts: And so I’m probably gonna be jumping back to that now, but yeah. It’s interesting how these things shift and the trends come and go, so…
196 00:29:41.480 ⇒ 00:29:42.010 Aarish: Yeah.
197 00:29:42.320 ⇒ 00:29:52.490 Samuel Roberts: All right, well, I think we’re kind of at time. If there’s any other last questions, I’m happy to, you know, say anything else, but, besides that, I can, explain the follow-up kind of process, so…
198 00:29:52.950 ⇒ 00:30:03.179 Aarish: Yes, I think this was already… I think Kayla kind of actually also cleared this up, but I’ll just ask you again. So you’re looking for a full-time commitment, right?
199 00:30:03.820 ⇒ 00:30:05.350 Samuel Roberts: Yeah, so,
200 00:30:05.410 ⇒ 00:30:22.290 Samuel Roberts: Yes, that’s kind of what we’re looking for. I think there’s sometimes a ramp-up phase, just to, you know, kind of test out and make sure everything works with everyone culturally and that. But the idea, I think, is certainly, like, as long as, you know, if you get through this process and get an offer, it would be…
201 00:30:22.290 ⇒ 00:30:27.010 Samuel Roberts: full-time, or, you know, working two full-time pretty, pretty quickly. So,
202 00:30:27.010 ⇒ 00:30:36.039 Samuel Roberts: there might be, like, a slight probationary period sort of thing, just… but hours-wise, I think we’re looking full-time, because we just… we need more resources, and you know, as we get more clients, so, yeah.
203 00:30:36.900 ⇒ 00:30:45.270 Aarish: Understood. Cool. So, I think salary expectations and all have already filled up the form, so I just wanted to know that on the same page on that.
204 00:30:45.470 ⇒ 00:30:59.260 Samuel Roberts: Yeah, I think, I mean, I… as far as I’m concerned, I think you wouldn’t have made it to this point if it weren’t already aligned there. So, yeah, if you spoke with Kayla or, put that in somewhere, like, it’s definitely in there.
205 00:30:59.440 ⇒ 00:31:00.849 Samuel Roberts: So yeah, you should be good.
206 00:31:01.470 ⇒ 00:31:02.740 Aarish: Okay, good.
207 00:31:02.740 ⇒ 00:31:21.499 Samuel Roberts: Great. So yeah, the next… the next step, when I bring this back to the team, if you’re… if you plan to pass that gate, would be a little more of a role-focused technical chat, and then after that, we would have you do a little bit of a kind of take-home, challenge, and then a kind of presentation panel interview.
208 00:31:21.500 ⇒ 00:31:36.099 Samuel Roberts: With me and some other engineers, and then, and then an offer after that. And we… we try to move relatively quickly, so, you know, I keep saying the hardest thing is just scheduling the time with people, but, you should hear back one way or the other relatively soon, so…
209 00:31:36.520 ⇒ 00:31:38.790 Aarish: Got it. Yep. Great. Moving forward.
210 00:31:39.320 ⇒ 00:31:41.190 Samuel Roberts: Excellent, yeah, thank you so much for your time.
211 00:31:41.660 ⇒ 00:31:43.639 Aarish: Thanks, thanks, Emil. Yeah.
212 00:31:43.640 ⇒ 00:31:44.390 Samuel Roberts: Have a good one.
213 00:31:44.720 ⇒ 00:31:45.270 Samuel Roberts: Bye-bye.
214 00:31:45.270 ⇒ 00:31:46.299 Aarish: You too. Bye.