Meeting Title: Brainforge Interview w- Sam Date: 2026-03-06 Meeting participants: Mouhamad, Samuel Roberts
WEBVTT
1 00:01:04.129 ⇒ 00:01:05.950 Mouhamad: Hi, Sam, can you hear me?
2 00:01:10.290 ⇒ 00:01:11.740 Samuel Roberts: There we go, okay.
3 00:01:11.860 ⇒ 00:01:13.230 Samuel Roberts: Can you hear me alright?
4 00:01:13.710 ⇒ 00:01:15.060 Mouhamad: Yes, I can hear you, can you hear me?
5 00:01:15.060 ⇒ 00:01:19.640 Samuel Roberts: Okay. Yes, I can. Sorry, my audio didn’t automatically connect like it normally does.
6 00:01:20.570 ⇒ 00:01:21.120 Samuel Roberts: If I don’.
7 00:01:21.120 ⇒ 00:01:25.410 Mouhamad: No, that’s okay. It’s okay. Okay, I get it.
8 00:01:25.990 ⇒ 00:01:27.300 Samuel Roberts: Hello, how are you?
9 00:01:27.850 ⇒ 00:01:29.889 Mouhamad: I’m good, how are you? How’s everything?
10 00:01:29.890 ⇒ 00:01:30.480 Samuel Roberts: Good.
11 00:01:30.740 ⇒ 00:01:32.650 Samuel Roberts: Doing alright, doing alright.
12 00:01:33.240 ⇒ 00:01:35.210 Mouhamad: So you spoke…
13 00:01:35.210 ⇒ 00:01:37.210 Samuel Roberts: With Kayla, I believe?
14 00:01:37.660 ⇒ 00:01:38.320 Mouhamad: Yes.
15 00:01:38.320 ⇒ 00:01:39.460 Samuel Roberts: Okay.
16 00:01:39.800 ⇒ 00:01:40.670 Samuel Roberts: Great.
17 00:01:40.780 ⇒ 00:01:44.009 Samuel Roberts: So, I’ll just start quickly,
18 00:01:44.010 ⇒ 00:02:02.229 Samuel Roberts: My name is Sam Roberts, I’m the AI tech lead here at Brainforge. I think the way this will go is I’ll ask you to give a brief intro, and then I have some questions here. I’ll leave some time to make sure you have time to ask me questions, and yeah, we’ll just kind of see where it goes from there.
19 00:02:02.230 ⇒ 00:02:02.790 Mouhamad: Awesome.
20 00:02:03.180 ⇒ 00:02:05.350 Samuel Roberts: Yeah, so go ahead, if you can start a little bit.
21 00:02:05.930 ⇒ 00:02:11.980 Mouhamad: Yes, so, hello, my name is Muhammad, Muhammad Ibrahim. So, I am a…
22 00:02:12.160 ⇒ 00:02:17.509 Mouhamad: let’s say, senior AI slash lead AI engineer. I’m working here in the Middle East.
23 00:02:17.760 ⇒ 00:02:24.890 Mouhamad: I work with a company called Artifact. So, Artifact is basically a consultancy company here in the Middle East.
24 00:02:25.090 ⇒ 00:02:39.560 Mouhamad: And I am leading development in the use cases that we create, the products that we create, and everything. And sort of my journey, just in a nutshell. So, I started working as a data scientist at the beginning for about 3 years here in Lebanon.
25 00:02:39.670 ⇒ 00:02:50.969 Mouhamad: Then I went to London, and I pursued my master’s in AI, because I wanted to get into AI, and I wanted a degree with it. It was very tough, it was during COVID, so…
26 00:02:50.970 ⇒ 00:02:51.940 Samuel Roberts: Oh, yeah.
27 00:02:51.940 ⇒ 00:03:00.059 Mouhamad: locked down everything, so it was awful. Then I worked a bit over there, did some projects over there, then I went back to Lebanon.
28 00:03:00.110 ⇒ 00:03:13.029 Mouhamad: had a couple of jobs, but they didn’t continue because, like, the economic in Lebanon is not the best, so they closed doors. Then I went to Artifact, and yeah, now I’m leading development, and sort of what we do is…
29 00:03:13.090 ⇒ 00:03:16.310 Mouhamad: Consultancy. Yeah, consultancy,
30 00:03:16.390 ⇒ 00:03:35.679 Mouhamad: we have clients all over the GCC, in Saudi Arabia, in Dubai, in Qatar. Also, I work with clients, on, like, from Europe, and also from Asia sometimes, and in all sorts of fields, some sort of, like, would be aviation, some ministries.
31 00:03:37.470 ⇒ 00:03:47.589 Mouhamad: some retail, some… it’s just all over the place. Yeah, I’m leading development. And right now, I have currently two projects that I’m needing.
32 00:03:47.830 ⇒ 00:03:56.730 Mouhamad: And it sort of depends on the project, depends on the requirements. The team could be small, could be big, based on the requirements and, you know.
33 00:03:56.970 ⇒ 00:03:58.929 Mouhamad: That is, like, a brief overview.
34 00:03:59.960 ⇒ 00:04:07.890 Samuel Roberts: Okay, great, yeah. Thank you, that’s, okay, that’s very helpful. So I think I’ll probably just jump in then, probably ask about,
35 00:04:08.500 ⇒ 00:04:15.910 Samuel Roberts: Yeah, the first thing is actually telling me about a specific LLM-based feature that you shipped to production, and the problem that it solved.
36 00:04:16.649 ⇒ 00:04:24.329 Mouhamad: Yeah, sure. So, one of the projects is that we worked on is basically like a construct management system.
37 00:04:24.539 ⇒ 00:04:34.879 Mouhamad: So, the idea, basically, the problem that they had is they had a lot of contracts. They have a lot of contracts with, like, landlords, with B2B, with B2C, with multiple things.
38 00:04:34.999 ⇒ 00:04:41.919 Mouhamad: And it’s sort of… what they wanted is that they wanted a system where they can look at the contracts, where they can…
39 00:04:41.989 ⇒ 00:04:47.909 Mouhamad: Because they had PDFs. They always had PDFs scattered around, etc. They couldn’t look at it.
40 00:04:47.959 ⇒ 00:05:02.749 Mouhamad: So what we did is, a big system where basically, we did, first of all, the first asset is the pre-processing. So the preprocessing is, we take, like, let’s say a PDF, a Word document, or whatever, and…
41 00:05:02.989 ⇒ 00:05:08.119 Mouhamad: sort of, we do some pay processing to produce something called DQS, so the Document Quality Score.
42 00:05:08.299 ⇒ 00:05:24.039 Mouhamad: based on the SKU level, let’s say, the brightness, some sort of metrics. Based on these, we do some enhancement. If it’s slight enhancement or deep enhancement, so we add ink, or we do this Q, etc, so…
43 00:05:24.670 ⇒ 00:05:34.239 Mouhamad: Then, comes the part of the OCR. So for the OCR, one of the problems that we had is that we couldn’t use…
44 00:05:34.480 ⇒ 00:05:43.289 Mouhamad: cloud-based IOCR, like, for example, Document AI on GCP or anything, or they don’t have it. So we wanted something reliable. Also, they don’t have GPU.
45 00:05:43.380 ⇒ 00:06:00.510 Mouhamad: So what we did is that we do… so the architecture that I did is the… I created the architecture, and then we developed it. What we do is we OCR a line, each line by itself, using Tesseract, and Tesseract produces something called confidence level.
46 00:06:00.780 ⇒ 00:06:07.320 Mouhamad: So based on the confidence level, if the confidence level is below a threshold, I think the threshold that we put is, like, 80 or 85.
47 00:06:07.420 ⇒ 00:06:09.759 Mouhamad: it would revert to an LLM,
48 00:06:09.990 ⇒ 00:06:20.730 Mouhamad: will go down to an LLM, and the LLM will take the B box off the… of the line, etc, and it will OCR it itself, even if,
49 00:06:20.850 ⇒ 00:06:25.819 Mouhamad: Another thing, also, if there is a numeric or number in the line.
50 00:06:26.040 ⇒ 00:06:34.809 Mouhamad: there’s an LLM that goes on it automatically without Tesseract, because I did some testing, and it turns out Tesseract is awful with numbers. Doesn’t go well with numbers.
51 00:06:35.290 ⇒ 00:06:40.879 Mouhamad: And also we are working with Arabic and English, and not just English, so…
52 00:06:40.880 ⇒ 00:06:41.530 Samuel Roberts: Sure.
53 00:06:41.910 ⇒ 00:06:56.759 Mouhamad: the Arabic language is very tricky, because the point can be 0, can be 5, but the 5 is like a… like a small circle without the point. The Arabic is a bit tough, but that’s why, like.
54 00:06:56.760 ⇒ 00:06:57.160 Samuel Roberts: Okay.
55 00:06:57.160 ⇒ 00:07:03.100 Mouhamad: for the numbers, yes, LLM immediately without Tessarak, even though Tesseraq can handle, like, English, Arabic, etc.
56 00:07:03.100 ⇒ 00:07:03.590 Samuel Roberts: Sure.
57 00:07:04.520 ⇒ 00:07:12.419 Mouhamad: Then what we did after that is that we’re producing something called JSONL. So the JSON-L is basically the line with the confidence level, etc.
58 00:07:12.530 ⇒ 00:07:19.530 Mouhamad: Then we’re doing translation, because we also want it, like, in English, to save it in Mongo and in Postgres that we have, we have the system inside.
59 00:07:19.530 ⇒ 00:07:20.060 Samuel Roberts: Truth.
60 00:07:21.660 ⇒ 00:07:29.729 Samuel Roberts: Yeah, and then we produce it on the dashboard, on the front end, but also we are… we have Melvis, so Melvis is basically our…
61 00:07:29.830 ⇒ 00:07:38.119 Mouhamad: system that saves, the chunks with the embeddings, etc, and can produce. So it’s like a vector store, but would also.
62 00:07:38.120 ⇒ 00:07:38.790 Samuel Roberts: Okay.
63 00:07:38.790 ⇒ 00:07:48.300 Mouhamad: some retrieval metrics to it. And then, yeah, we have the system inside the front end, with a chatbot also next to it.
64 00:07:48.400 ⇒ 00:08:02.329 Mouhamad: And you can ask a question based on it, and you can see the contract with some alerts, like, let’s say the contract is about to expire, or anything. Also, recently, what we did is that we enhanced it even more. We created some…
65 00:08:02.330 ⇒ 00:08:16.790 Mouhamad: So, when we have a translation, and we have everything, there’s an… there’s a model, there’s an agent that we developed also, just to check some metrics. So, for example, if the start date is after the end date, if… some key aspects.
66 00:08:16.790 ⇒ 00:08:17.150 Samuel Roberts: Sure.
67 00:08:17.150 ⇒ 00:08:22.219 Mouhamad: And it will flag it immediately, so the people on the screen can actually fix it.
68 00:08:22.480 ⇒ 00:08:24.789 Mouhamad: I think this is one of the systems that we did.
69 00:08:24.790 ⇒ 00:08:42.319 Samuel Roberts: Very cool. Cool, cool. So when you’re building a system like that for, especially, like, as a consultancy, you know, you have clients, and they may not be technical, but a lot of people are excited about AI, and they hear all this stuff. How do you go about explaining the limitations of some of this stuff to non-technical stakeholders?
70 00:08:42.640 ⇒ 00:08:51.139 Mouhamad: Yeah, so I always… I always like to do it as a story for them. That’s one of the things, and I also like to give them examples. So let’s say…
71 00:08:51.340 ⇒ 00:08:54.859 Mouhamad: If someone, like, is asking me, like, oh,
72 00:08:55.040 ⇒ 00:09:14.789 Mouhamad: how does this, like, multi-agent, for example, work? So I try to make it as simple as, like, I say, okay, just imagine you have a bakery, and you have, like, the head baker, and you have some sort of people with him that will help him, and, like, the head baker is the one that gives orchestration, etc, etc. So I always try to…
73 00:09:14.890 ⇒ 00:09:27.949 Mouhamad: make… to give an example from real world, so they can understand it a bit. That’s one aspect. Second, also, I would try to also, for example, if I ask them, like, let’s say.
74 00:09:28.290 ⇒ 00:09:35.630 Mouhamad: Okay, you want an AI in here, you want an LLM in here, okay, do you have GPU? They’d be like, oh, no, they don’t, why do you need a GPU, for example?
75 00:09:35.930 ⇒ 00:09:36.450 Samuel Roberts: Right.
76 00:09:36.450 ⇒ 00:09:49.240 Mouhamad: Can you… can you dig a whale without spoon? With a spoon, for example? Can you do that? I try to make them… I try to do… to simplify it from real world, because they don’t understand, like, I cannot just tell them, like.
77 00:09:49.240 ⇒ 00:10:08.880 Mouhamad: okay, you need, like, the vector of the LLM, and then it comes out, and you need to… no, I cannot do that. So I try to give them a real-world example, and that usually, like, they’d be like, because I got a good feedback on this from a couple of clients, they’d be like, I’m very good at, like, translating the technical using this method, so I’ve been using it.
78 00:10:09.460 ⇒ 00:10:12.780 Samuel Roberts: Great. Oh, that’s smart. Yeah, I like that.
79 00:10:13.760 ⇒ 00:10:27.220 Samuel Roberts: Okay, let’s talk about when something, like, an AI feature might underperform. What, where do you take responsibility for that versus the model is weird? Like, how do you balance that with, like, you know, where…
80 00:10:27.350 ⇒ 00:10:33.980 Samuel Roberts: The responsibility is from the architectural standpoint versus the actual performance of the… feature.
81 00:10:34.480 ⇒ 00:10:36.220 Samuel Roberts: So, like…
82 00:10:36.540 ⇒ 00:10:49.669 Samuel Roberts: If you… if, like, the client is not satisfied with how it’s working, but it’s, you know, working to spec, or it’s underperforming, where do you step in and say, okay, yeah, we messed up, versus, okay, no, this is doing what it’s intended, we just didn’t do it right, or…
83 00:10:49.770 ⇒ 00:10:51.389 Samuel Roberts: Talk to me about that a little bit.
84 00:10:52.090 ⇒ 00:10:52.770 Mouhamad: Okay.
85 00:10:59.300 ⇒ 00:11:08.400 Mouhamad: Okay, so… let’s say… so… If we created this… okay.
86 00:11:09.130 ⇒ 00:11:15.430 Mouhamad: So, the thing is, for, let’s say, if,
87 00:11:16.510 ⇒ 00:11:20.219 Mouhamad: I’m just gathering my thoughts. Oh, okay.
88 00:11:20.220 ⇒ 00:11:20.930 Samuel Roberts: You’re good.
89 00:11:24.810 ⇒ 00:11:25.790 Mouhamad: Okay.
90 00:11:26.310 ⇒ 00:11:31.500 Mouhamad: So, the first thing is basically… diagnosis, so…
91 00:11:32.050 ⇒ 00:11:46.540 Mouhamad: Sure. So, let’s say, let’s say, I don’t know, they ask the system or anything, first of all, diagnosis, let’s say, any system. So, what I do is that I analyze where the future is happening in the pipeline. So, this could be, like, for example.
92 00:11:46.540 ⇒ 00:11:54.290 Mouhamad: Data quality issue, incorrect labeling or bias, model limitation, distribution shift, for example.
93 00:11:54.430 ⇒ 00:12:05.049 Mouhamad: Even as simple as, like, there’s, like, an integration issue, or, like, let’s say if you, like, if you have, like, MongoDB, and you didn’t use the right grid FS, for example, or etc.
94 00:12:05.260 ⇒ 00:12:14.619 Mouhamad: So, for this, I would rely, like, on monitoring, blogs, error analysis,
95 00:12:14.760 ⇒ 00:12:17.480 Mouhamad: Some evals, if needed, of anything.
96 00:12:19.080 ⇒ 00:12:30.580 Mouhamad: So, that’s… that’s the first part that I would do, this diagnosis. Then the second thing that I would go is the… would go deeper, so basically, root cause, let’s say, root cause analysis.
97 00:12:31.130 ⇒ 00:12:49.230 Mouhamad: So, depending on the finding, if it’s, like, let’s say, a problem in the dataset, I might do some augmenting, for example, the data set. Some other stuff that might be error from my side, for example, is that I didn’t do, like, correct labeling, for example.
98 00:12:49.320 ⇒ 00:12:53.250 Mouhamad: So, correcting the labeling is very important.
99 00:12:53.910 ⇒ 00:12:56.430 Mouhamad: some other stuff also that I might…
100 00:12:56.550 ⇒ 00:13:04.569 Mouhamad: be like, okay, myself, I didn’t do it also properly well, is that, maybe adjusting a feature or something in the architecture.
101 00:13:04.660 ⇒ 00:13:21.869 Mouhamad: That needs, happening. Other thing that the model is not maybe performing well is because it needs fine-tuning, so fine-tuning the parameter, fine-tuning, like, lower accuracy lower, etc, because a lot of the people think, like, oh, okay, this is an LLM, just put it, and it will work.
102 00:13:21.940 ⇒ 00:13:33.090 Mouhamad: Sometimes you need to fine-tune, if you have the capability, of course. After that, basically, is, just communicating with the threshold, with the stakeholders, so…
103 00:13:33.190 ⇒ 00:13:36.899 Mouhamad: So I keep the client, like, involved by explaining everything.
104 00:13:37.010 ⇒ 00:13:55.160 Mouhamad: Sir, it depends. So, some, some sort of stuff, like I said, like, from my side, like augmenting, labeling, adjusting a feature or architecture. One of the, one of the things that also I was working on is… I had this problem. I was working for the insurance authority.
105 00:13:55.160 ⇒ 00:14:02.180 Mouhamad: And we were doing a database, like a SQ… like a chatbot, but for the… for their BigQuery, so they had, like, over…
106 00:14:02.180 ⇒ 00:14:04.519 Mouhamad: 100 million, 200 million rows.
107 00:14:04.580 ⇒ 00:14:08.240 Mouhamad: There was an initial architecture.
108 00:14:08.330 ⇒ 00:14:23.839 Mouhamad: that a previous data science AI engineer did. I came in and I changed the architecture, and I communicated with the clients, like, okay… because the way that it was is that it’s throwing everything to the LLM, with all the columns, all the tables, everything.
109 00:14:23.990 ⇒ 00:14:34.010 Mouhamad: So I was like, okay, let me debuggle it, debuggle it a bit, and just give it, like, a, like, there’s, like, an initial filtering at the beginning, there is, like, entities.
110 00:14:34.250 ⇒ 00:14:46.609 Mouhamad: workspace, discovery at the beginning. So this is how I communicate with the client, okay, let’s say, look, okay, this is not working because the architecture is weak, let me fix it. So I would say, yeah, these are the things.
111 00:14:47.240 ⇒ 00:14:48.430 Samuel Roberts: Great, yeah, thank you.
112 00:14:48.540 ⇒ 00:14:53.000 Samuel Roberts: Okay, so let’s talk about, the, you know.
113 00:14:53.000 ⇒ 00:15:10.890 Samuel Roberts: the AI industry, I guess, is changing all the time, there’s new models, there’s new tools, there’s new frameworks, there’s new trends. One thing that’s nice about Brainforge is we get to play with a lot of those internally, but we don’t necessarily ship everything to the client, because things might not be ready, or they’re, you know.
114 00:15:10.890 ⇒ 00:15:22.419 Samuel Roberts: still, you know, getting the fine, fine-tuning the bugs and everything. What’s something that you’ve seen that you might have been excited about, but decided not to adopt for whatever reason?
115 00:15:22.440 ⇒ 00:15:24.369 Samuel Roberts: In the LLM AI space.
116 00:15:25.920 ⇒ 00:15:29.740 Mouhamad: I think on the…
117 00:15:33.600 ⇒ 00:15:35.400 Mouhamad: Okay,
118 00:15:39.450 ⇒ 00:15:43.849 Mouhamad: So I would say some of the things may be, like,
119 00:15:44.170 ⇒ 00:16:01.289 Mouhamad: like, like, let’s say in a tool, like, let’s say Langchain, LangGraph, or anything. Sure. Like, they do a lot of good work, and they’re very good. And also, like, whenever you discover it, like, for example, at the beginning, when I, when I was working, oh my god, this is amazing, this is working incredibly.
120 00:16:01.600 ⇒ 00:16:14.899 Mouhamad: But sometimes, also, like, a multi-system agent or anything, you create a system where it’s very complicated, and you feel like, I’ve been working on it, and I’m very happy with it, and I’m like.
121 00:16:15.920 ⇒ 00:16:25.650 Mouhamad: Yeah, maybe I shouldn’t ship it, because it’s already overcomplicated, it’s working, but maybe it will break later on if,
122 00:16:25.800 ⇒ 00:16:29.439 Mouhamad: If a new feature came, or the data and the real world changes.
123 00:16:29.750 ⇒ 00:16:36.490 Mouhamad: So, I would say, yeah, it’s the joy of discovering a new tool.
124 00:16:36.630 ⇒ 00:16:45.589 Mouhamad: That you are eager to… to use, and it will… do you think it will facilitate a lot of things in the project, and etc?
125 00:16:46.340 ⇒ 00:16:52.309 Mouhamad: But then, you take, like, a step back, and you analyze the actual problem that they’re trying to solve, and you’re like.
126 00:16:52.960 ⇒ 00:16:55.170 Mouhamad: It doesn’t require that much.
127 00:16:55.490 ⇒ 00:16:56.379 Samuel Roberts: Right, right.
128 00:16:56.380 ⇒ 00:17:02.740 Mouhamad: Plus, I’m over-complicating myself, I’m over-complicating the project, why am I doing this? And you know, like.
129 00:17:02.740 ⇒ 00:17:03.120 Samuel Roberts: Yeah.
130 00:17:03.120 ⇒ 00:17:11.700 Mouhamad: when you code something, and then later on, after, like, a couple of months, you go back and, like, who wrote this book? And you’re like, oh, I can’t.
131 00:17:11.700 ⇒ 00:17:12.269 Samuel Roberts: I don’t know what.
132 00:17:13.140 ⇒ 00:17:16.870 Mouhamad: Yeah. Definitely, definitely.
133 00:17:17.310 ⇒ 00:17:35.240 Samuel Roberts: Okay, so that’s… okay, so then let’s say, you know, what kind of, how do you decide when something is production-ready for, like, a tool, or a framework, or something that you might have seen and thought, this isn’t good yet, or it’s overcomplicating it now, but now it’s needed? Like, where is that…
134 00:17:35.400 ⇒ 00:17:38.279 Samuel Roberts: What’s the criteria you might look at there?
135 00:17:39.790 ⇒ 00:17:43.130 Mouhamad: Okay.
136 00:17:46.580 ⇒ 00:17:51.339 Mouhamad: So… It also depends on the tool, also depends on.
137 00:17:51.340 ⇒ 00:17:52.130 Samuel Roberts: Yeah, totally.
138 00:17:52.130 ⇒ 00:17:56.340 Mouhamad: They’re asking, okay.
139 00:17:56.570 ⇒ 00:17:57.580 Mouhamad: So…
140 00:17:59.240 ⇒ 00:18:12.339 Mouhamad: Okay, when… when we want to decide, like, whether, let’s say, a tool or a system is production ready, so usually what I do is I evaluate it across, like, several key criteria, like, let’s say.
141 00:18:12.510 ⇒ 00:18:18.270 Mouhamad: Based on performance, reliability, scalability, etc.
142 00:18:18.460 ⇒ 00:18:34.539 Mouhamad: So, the first thing, the first pillar, if we want to talk about this, is basically the performance and accuracy. So, if the model or the system must meet, like, the predefined success metric, because every client, like, has a success metric, or we define a success metric if they don’t have it.
143 00:18:34.670 ⇒ 00:18:51.980 Mouhamad: So, I would say, like, like, the beginning is basically, if the model or the system meets the success criteria, like, let’s say, based on, like, the accuracy, latency, recall, or business KPI, if the business has KPIs.
144 00:18:52.250 ⇒ 00:18:55.729 Mouhamad: Second is based on reliability,
145 00:18:55.910 ⇒ 00:18:59.860 Mouhamad: So, the system, basically, it should behave consistently, under any
146 00:19:00.260 ⇒ 00:19:18.169 Mouhamad: also what I asked, what I said, like, if the real-world data changes, still the system should be reliable enough to actually, work. And even if the data changes, it’s system, okay, yes, it will change a bit, will fine-tune a bit, but it will still need to function, at least.
147 00:19:18.170 ⇒ 00:19:18.590 Samuel Roberts: Sure.
148 00:19:18.590 ⇒ 00:19:27.489 Mouhamad: And from this also to handling, like, edge cases, noisy inputs, distribution shifts, etc.
149 00:19:27.630 ⇒ 00:19:32.959 Mouhamad: Third pillar, I would say, is scalability, as I said, and latency also, they go on and on.
150 00:19:33.050 ⇒ 00:19:49.080 Mouhamad: So, it must handle the expected load. So, let’s say, one of the things is the insurance authority, they will use it internally, and they have, like, let’s say, 300 people. Like, if it doesn’t handle 300 people, this is… this is bullocks, so to say.
151 00:19:49.330 ⇒ 00:19:49.670 Samuel Roberts: Sure.
152 00:19:49.670 ⇒ 00:19:56.390 Mouhamad: that shouldn’t go into production. So I evaluated, basically, if it can support concurrent requests,
153 00:19:56.390 ⇒ 00:19:56.750 Samuel Roberts: Yep.
154 00:19:56.750 ⇒ 00:20:01.640 Mouhamad: Other, like, inference latency is acceptable, whether the system can scale horizontally.
155 00:20:01.640 ⇒ 00:20:02.260 Samuel Roberts: Sure.
156 00:20:02.260 ⇒ 00:20:13.639 Mouhamad: If the user also grows, if it goes also more than this. After that, also, there are, like, stuff like monitoring and observability, because a lot of people, like, just…
157 00:20:13.790 ⇒ 00:20:26.550 Mouhamad: deploy it, and they’re like, just go, leave. No. Like, before deploying to production also, there must be, like, monitoring in place for, like, the model performance, data drift, etc, etc.
158 00:20:26.730 ⇒ 00:20:33.949 Mouhamad: And also, like, also to have… this is important, because I don’t see a lot of people do it, is to have loggings everywhere.
159 00:20:34.300 ⇒ 00:20:42.229 Mouhamad: It’s just so you can catch an error, like, don’t put, like, try accept onto the entire block of thing, and just, like…
160 00:20:42.230 ⇒ 00:20:42.770 Samuel Roberts: Yeah.
161 00:20:42.770 ⇒ 00:20:47.500 Mouhamad: Heather is here. No, just have, like, a proper logging for him.
162 00:20:47.500 ⇒ 00:20:47.900 Samuel Roberts: Yeah, yeah.
163 00:20:47.900 ⇒ 00:20:55.039 Mouhamad: The last couple of things, also, they can go, like, like, CICD integration, clear pipelines, etc, etc. These are, like, the pillars.
164 00:20:56.170 ⇒ 00:21:07.279 Samuel Roberts: Okay, yeah, no, good, thank you. So that covers most of what I had, so I want to leave some time here for you to ask any questions about, Brainforge, about the role, about the work, whatever questions you have.
165 00:21:07.450 ⇒ 00:21:09.500 Samuel Roberts: Yep. I’m happy to answer that now.
166 00:21:09.660 ⇒ 00:21:20.739 Mouhamad: So, one thing that Kayla said, that you have, like, an engagement with a client, for example, basically, based on, like, 3 months engagement, usually. Maybe below, maybe less.
167 00:21:20.810 ⇒ 00:21:34.959 Mouhamad: But how do you find time to actually, like, in 3 months, to be able to do everything, like the analysis of the client, of the data, of everything that you have into one thing, and to actually produce something that they will use?
168 00:21:35.600 ⇒ 00:21:45.920 Samuel Roberts: Yeah, it’s not, it’s not trivial, it’s definitely, so there’s the whole kind of, pre-engagement process.
169 00:21:45.990 ⇒ 00:22:00.820 Samuel Roberts: So that involves some of the, you know, statement of work and a little bit of research that way that goes into that. So it’s not… you know, we don’t just hit, you know, 3 months, okay, now tell us everything we need to know. It’s… it’s… we learn a little bit beforehand. So we have an idea of,
170 00:22:00.820 ⇒ 00:22:11.879 Samuel Roberts: what is reasonable in those 3 months. We work relatively quickly, I will say. It’s very much of a startup mentality, it’s very much a, you know, shipping kind of mentality.
171 00:22:13.130 ⇒ 00:22:20.269 Samuel Roberts: you know, biased towards getting things out, getting things in front of the client, getting feedback, very iterative that way.
172 00:22:20.530 ⇒ 00:22:22.720 Samuel Roberts: you know, I would also say we…
173 00:22:24.210 ⇒ 00:22:47.809 Samuel Roberts: we try to, you know, we’re working on sort of standardizing our service lines a little bit. So, you know, right now, in the AI side, so there’s kind of the AI side, and there’s the data side. You know, the data side is sort of how we started at Brainforge, was doing pipelines and ETLs and analysis, and we added kind of the AI automation team, because we were doing things internally. So some of the projects we’ve done have been a little more,
174 00:22:48.160 ⇒ 00:22:54.689 Samuel Roberts: like, ad hoc for different things, but we’re starting to standardize and say, okay, we’re… we’re getting pretty good at doing this rag over…
175 00:22:55.020 ⇒ 00:23:06.329 Samuel Roberts: you know, a bunch of data, and a chat bot that’s integrated into Google Chat. Like, we know how to do that now, because we’ve done it, so bringing that out is less work. You know, the 3-month engagement is not…
176 00:23:06.400 ⇒ 00:23:22.900 Samuel Roberts: you know, the idea is always to keep working with the client, right? So, we’re not looking to, be like, alright, 3 months is over, here’s your product, and leave. It’s like, here’s what we got done, this is what we targeted. There’s more to do, certainly, but we would want to ship something end-to-end.
177 00:23:22.920 ⇒ 00:23:35.019 Samuel Roberts: that is, you know, a fully featured thing. It might not be the final end result that they want at that point, because it’s only 3 months, but we still try to deliver something in a total package.
178 00:23:35.110 ⇒ 00:23:44.079 Samuel Roberts: You know, we don’t want to leave them hanging, we don’t want to leave them in a situation where they’re dependent on us, because I don’t think that’s good… like, it might be good business practice to make them dependent on you, but I don’.
179 00:23:44.080 ⇒ 00:23:44.720 Mouhamad: It would be…
180 00:23:44.720 ⇒ 00:23:45.340 Samuel Roberts: operate.
181 00:23:45.340 ⇒ 00:23:47.140 Mouhamad: It would be very good for us.
182 00:23:47.140 ⇒ 00:24:06.150 Samuel Roberts: Yeah, exactly. And we just had a project where we were working with someone, and the dev team they were working with had integrated it into this, like, custom backend that had all this other stuff, and we… a big part of our job was pulling it out of that and giving it to them in a way that they could take, and if they wanted to go to someone else, go to someone else. Like, it was all theirs. So we’re very big on that.
183 00:24:06.150 ⇒ 00:24:25.470 Samuel Roberts: But yeah, I think it’s important for us to define the work, you know, things change along the way, but as long as it’s well thought out at the beginning, and we know… we’re getting better at knowing what we’re capable of, and we’re also getting better at just moving faster. You know, we’re big, we use Cursor a lot, but we’ve played around with other AI
184 00:24:25.470 ⇒ 00:24:26.820 Samuel Roberts: coding-assisted tools.
185 00:24:27.000 ⇒ 00:24:35.969 Samuel Roberts: or AI-assisted coding tools. And so, you know, we’re constantly shipping new things, especially internally. We’re really good at experimenting with new things.
186 00:24:36.110 ⇒ 00:24:50.490 Samuel Roberts: Willing to try things. We have a whole bunch of, like, internal projects we’re working on in addition to the client work, and so we’re learning to move faster, we’re learning how to use these coding tools in more, you know,
187 00:24:51.140 ⇒ 00:24:52.850 Samuel Roberts: what’s the word I’m trying to think of here, like…
188 00:24:53.070 ⇒ 00:25:03.209 Samuel Roberts: It’s all agentic, but we’re trying to do the in-loop and outloop, if you’ve heard that, where sometimes, you know, you’re chatting with cursor, but sometimes you just let the cloud function go and see what it comes back with, and sometimes it’s really good, and that.
189 00:25:03.210 ⇒ 00:25:04.020 Mouhamad: Yeah, yeah, good.
190 00:25:04.100 ⇒ 00:25:13.470 Samuel Roberts: So there’s a lot of that, like, trying to, you know, build out more and more of what we can do as a small team. But I would say we move relatively quickly.
191 00:25:13.650 ⇒ 00:25:22.330 Samuel Roberts: we… we don’t have a ton of clients on the AI side compared to the data side, so we can focus a lot more, right now at least, but we’re growing, and so that’s sort of the reason we need more.
192 00:25:22.330 ⇒ 00:25:25.399 Mouhamad: How many people is in the AI team? Because Kayla’s just…
193 00:25:25.400 ⇒ 00:25:30.430 Samuel Roberts: There are, 3 engineers… 3 engineers and myself, so I’m sort of the… the…
194 00:25:30.580 ⇒ 00:25:41.069 Samuel Roberts: tech lead, but so there’s four of us engineers working on things. We have one or two main clients right now, plus the internal work, plus a few of the engineers are working on some other projects across the company.
195 00:25:41.350 ⇒ 00:25:41.870 Mouhamad: Got it.
196 00:25:41.870 ⇒ 00:25:54.979 Samuel Roberts: And part of that is just bandwidth, you know what I mean? Like, bringing on another 3-month contract for a client, like, we’ll need more… more bodies, more coders. So, you know, we can do a lot with the AI coding tools, but it’s still not, you know, we can’t just say.
197 00:25:54.980 ⇒ 00:25:56.740 Mouhamad: Start this project for me.
198 00:25:56.740 ⇒ 00:25:57.510 Samuel Roberts: Yeah, exactly.
199 00:25:57.510 ⇒ 00:26:02.320 Mouhamad: everyone thinks, like, oh, you have an IoTool now, you can do 100x.
200 00:26:02.320 ⇒ 00:26:03.239 Samuel Roberts: Oh my goodness.
201 00:26:03.240 ⇒ 00:26:04.059 Mouhamad: It’s not like that.
202 00:26:04.060 ⇒ 00:26:07.430 Samuel Roberts: I went to a… I went to a talk… not a talk, like a little…
203 00:26:07.610 ⇒ 00:26:19.889 Samuel Roberts: dinner kind of thing. This was maybe a year or two ago, and this business guy was saying that, you don’t need developers anymore, I can build my own app. And I was like, yeah, alright, you can, and have fun with it, and have fun debugging it, yeah.
204 00:26:19.890 ⇒ 00:26:20.520 Mouhamad: Exactly.
205 00:26:20.520 ⇒ 00:26:25.740 Samuel Roberts: Like, we’re getting closer to doing it, but it’s still… the technical knowledge is very helpful.
206 00:26:25.740 ⇒ 00:26:26.380 Mouhamad: Yes.
207 00:26:26.380 ⇒ 00:26:29.629 Samuel Roberts: No matter what. And the problem solving, and knowing…
208 00:26:29.670 ⇒ 00:26:52.060 Samuel Roberts: you know, when something goes wrong, where to look for it. So we’re… we’re finding that balance of what can the AI do on its own, and come back and test and show us that it did it, versus what do we need to do, where are our decisions relevant? But, yeah, I mean, to answer the question in a very long-winded way, like, we just… we work relatively quickly, we define the problems very early, we don’t… we try not to deviate too much unless the client really starts to…
209 00:26:52.060 ⇒ 00:26:58.880 Samuel Roberts: Yeah. Then it needs to be redefined, and then there’s a whole bunch of that, but yeah, we sort of… we don’t set out to, like, you know.
210 00:27:00.070 ⇒ 00:27:08.959 Samuel Roberts: you know, one bite at a time kind of thing, like, we’re not looking to solve their whole problem at once, we’re looking to find one solution to one problem and do it well, kind of thing.
211 00:27:09.550 ⇒ 00:27:15.009 Mouhamad: Hmm, get it. Yeah, awesome. I think that’s also Kayla also answered some questions, so I think I have left, yeah.
212 00:27:15.010 ⇒ 00:27:16.619 Samuel Roberts: Okay, alright, great, yeah.
213 00:27:16.960 ⇒ 00:27:27.319 Samuel Roberts: Yeah, alright, so then in that case, I think the next step would be a slightly more technical role-based interview, I believe a tech challenge, and then a panel interview with me and a couple of the other
214 00:27:27.500 ⇒ 00:27:40.560 Samuel Roberts: folks on the team, we like to move relatively quickly, you know, we’re not looking to drag out the process. I think… I’ve been saying the biggest hurdle is just scheduling this kind of time, but besides that, like, it shouldn’t be dragged out.
215 00:27:41.370 ⇒ 00:27:44.790 Samuel Roberts: And then, yeah, if you have any other questions, feel free to reach out. Yeah.
216 00:27:44.790 ⇒ 00:27:47.100 Mouhamad: Amazing. Okay, sounds good so far.
217 00:27:47.100 ⇒ 00:27:48.340 Samuel Roberts: Appreciate the time.
218 00:27:48.340 ⇒ 00:27:49.840 Mouhamad: Thank you so much, and thank you. Have a good day.
219 00:27:49.840 ⇒ 00:27:51.400 Samuel Roberts: Have a good one, Muhammad. Yep, bye.
220 00:27:51.500 ⇒ 00:27:52.190 Mouhamad: Yeah, but…