Meeting Title: Brainforge Interview w- Awaish Date: 2026-04-01 Meeting participants: Christina Knudson, Awaish Kumar
WEBVTT
1 00:00:40.120 ⇒ 00:00:40.920 Awaish Kumar: Hi.
2 00:00:41.360 ⇒ 00:00:42.190 Christina Knudson: Hello.
3 00:00:43.220 ⇒ 00:00:43.920 Awaish Kumar: Hi, I’m Christina.
4 00:00:44.850 ⇒ 00:00:46.179 Christina Knudson: Good! How are you doing?
5 00:00:46.830 ⇒ 00:00:48.889 Awaish Kumar: Yeah, I’m good as well.
6 00:00:49.260 ⇒ 00:00:53.420 Awaish Kumar: So, where are you located?
7 00:00:54.090 ⇒ 00:00:55.340 Christina Knudson: Minneapolis.
8 00:00:57.280 ⇒ 00:00:57.920 Awaish Kumar: Okay.
9 00:00:58.080 ⇒ 00:01:06.190 Awaish Kumar: So, yeah, I saw your… profile, and I have your profile as, data engineer.
10 00:01:07.770 ⇒ 00:01:12.479 Awaish Kumar: front of me, and I have seen most of your experience as data scientist.
11 00:01:12.610 ⇒ 00:01:15.479 Awaish Kumar: So my first question is, like, why…
12 00:01:15.900 ⇒ 00:01:18.380 Awaish Kumar: Are you choosing to be a data engineer?
13 00:01:19.610 ⇒ 00:01:27.200 Christina Knudson: So I’ve done a lot of, like, data engineer type work over the last year and a half, and I’ve found it really enjoyable.
14 00:01:27.650 ⇒ 00:01:29.090 Christina Knudson: Just, like, a really…
15 00:01:29.260 ⇒ 00:01:42.050 Christina Knudson: different, interesting kind of problem. Still uses a lot of the same tools as data science, like collaboration and understanding the problem and, like.
16 00:01:42.150 ⇒ 00:01:56.020 Christina Knudson: In both cases, you have some, like, restrictions and goals, and you’re trying to get from, like, one state to another state, and it’s just, like, kind of the details of how do you go about the problem, that’s different, so…
17 00:01:56.410 ⇒ 00:01:59.500 Christina Knudson: That’s kind of the process I’ve enjoyed.
18 00:02:00.970 ⇒ 00:02:03.570 Awaish Kumar: Okay, so since you have… you mentioned that you have…
19 00:02:03.770 ⇒ 00:02:07.249 Awaish Kumar: Since the last one and a half years you have been doing data engineering work.
20 00:02:07.710 ⇒ 00:02:17.349 Awaish Kumar: Maybe we can a little bit deep dive into what you have been doing as a data engineer, what kind of projects you have worked on, and…
21 00:02:19.010 ⇒ 00:02:24.460 Awaish Kumar: Yeah, specifically, if you have worked on any project end-to-end.
22 00:02:24.720 ⇒ 00:02:27.370 Awaish Kumar: Then I can… yeah, you can give that as an example.
23 00:02:28.140 ⇒ 00:02:35.840 Christina Knudson: Okay, so in my, current company, then, we have, like.
24 00:02:36.030 ⇒ 00:02:51.009 Christina Knudson: done a whole tech overhaul, like, changed everything with the tech stack, and part of that was, changing also, like, where the data is coming from, and so we’re getting data from a new place, and that means, like, it’s coming in a different raw format, and we still need to, like, get
25 00:02:51.010 ⇒ 00:03:07.909 Christina Knudson: all of our, you know, dashboards and machine learning models and stuff up and going with the new data. So, I was on a team of about 5 data people who were, rebuilding that data stack, so I helped to, like, architect it, and then we
26 00:03:08.240 ⇒ 00:03:16.599 Christina Knudson: kind of split it up so that there were different pieces that, like, each person owned, and so then I owned the, like, main,
27 00:03:16.710 ⇒ 00:03:21.540 Christina Knudson: Like, volume of products that we’re selling. And then, like.
28 00:03:21.720 ⇒ 00:03:31.879 Christina Knudson: did all the dbt work and everything to get that up and running and get, like, our new dashboards and machine learning models, like, all of the, like.
29 00:03:31.880 ⇒ 00:03:46.330 Christina Knudson: stakeholders for those, understanding, like, okay, now that we have this, like, new data, how do we plug it into that? What are differences? What are things to look out for? So I was able to do that, like, ahead of schedule, kind of
30 00:03:47.830 ⇒ 00:03:55.920 Christina Knudson: Just, like, was able to get that done, and get the results, like, really well matching what we had before, and
31 00:03:56.200 ⇒ 00:03:58.810 Christina Knudson: Yeah, we did that with dbt and Snowflake.
32 00:04:00.250 ⇒ 00:04:07.310 Awaish Kumar: So, like, what you moved from, like, what was the… The existing state and…
33 00:04:08.080 ⇒ 00:04:12.169 Awaish Kumar: Where you reached after the project was complete?
34 00:04:13.130 ⇒ 00:04:28.440 Christina Knudson: Yeah, so the existing state was receiving raw data from, like, one provider, and, so then we already had, like, the data transformations and, like, pipelines in place there for the old data, but then we stopped getting data from that. Oh, sorry.
35 00:04:29.510 ⇒ 00:04:39.130 Awaish Kumar: Yeah, I… when you talk about pipelines and all, like, I want to know all these things in detail, like, when the data lands somewhere, how…
36 00:04:39.360 ⇒ 00:04:43.580 Awaish Kumar: Where the… what the source is, how you land it into some
37 00:04:43.860 ⇒ 00:04:49.620 Awaish Kumar: Data warehouse or a database, and then how your transfer pipelines run.
38 00:04:49.950 ⇒ 00:04:59.130 Awaish Kumar: And what are those pipelines? Are they… like, what kind of code it executes? What is the orchestration tool, what is the cloud platform? And all these details.
39 00:04:59.630 ⇒ 00:05:05.360 Christina Knudson: Okay, yeah, so, when we… We, like…
40 00:05:05.490 ⇒ 00:05:17.090 Christina Knudson: get the data, the, like, raw data into Snowflake, and then use the dbt to, like, extract it and transform it, and, yeah, dbt was, like, the… the main…
41 00:05:17.350 ⇒ 00:05:18.200 Christina Knudson: thing.
42 00:05:19.260 ⇒ 00:05:23.609 Awaish Kumar: Yeah, but what was before this new infrastructure?
43 00:05:24.150 ⇒ 00:05:32.009 Christina Knudson: Oh, okay. Before this one, then we had the data, landing also in Snowflake, but…
44 00:05:32.550 ⇒ 00:05:35.800 Christina Knudson: It was modeled more with, like.
45 00:05:37.150 ⇒ 00:05:39.010 Christina Knudson: Well, some of it was with…
46 00:05:39.240 ⇒ 00:05:54.319 Christina Knudson: dbt, but, like, since most of the data stack was, like, predating dbt, then it was built, I think, just with, like, actually in Redshift with just, like, like, DDL, and, like.
47 00:05:55.240 ⇒ 00:06:01.260 Christina Knudson: Those sorts of jobs, or… Yeah, so then it was, like, in Redshift.
48 00:06:01.620 ⇒ 00:06:05.859 Christina Knudson: getting transformed, and then moved over to Snowflake.
49 00:06:06.150 ⇒ 00:06:07.940 Christina Knudson: And that’s just because, like.
50 00:06:08.360 ⇒ 00:06:15.589 Christina Knudson: that’s how, kind of, the, like, company started, like, 10 years ago, is that they, like, had it all going into Redshift first.
51 00:06:16.000 ⇒ 00:06:24.950 Christina Knudson: And then some of the, like, more new models, like in the last 5 years, then those are things that we had running in Snowflake in,
52 00:06:25.050 ⇒ 00:06:28.269 Christina Knudson: in… Running in dbt with Snowflake.
53 00:06:28.820 ⇒ 00:06:31.660 Awaish Kumar: Okay, how data lands in Snowflake right now?
54 00:06:31.870 ⇒ 00:06:39.679 Awaish Kumar: Like, what are the… What are the injection methods that you use today to ingest data to Snowflake?
55 00:06:41.040 ⇒ 00:06:45.060 Christina Knudson: Okay, so that part, I didn’t…
56 00:06:45.490 ⇒ 00:06:47.769 Christina Knudson: work on, so I don’t know that part.
57 00:06:49.450 ⇒ 00:06:50.180 Awaish Kumar: Okay.
58 00:06:50.460 ⇒ 00:06:57.030 Awaish Kumar: So, Nick, we have a different, work streams here in,
59 00:06:57.170 ⇒ 00:07:02.309 Awaish Kumar: in Brain Forge. So these work streams are, like, divided by…
60 00:07:04.080 ⇒ 00:07:08.760 Awaish Kumar: the kind of work, right? So, obviously, the first one is…
61 00:07:08.850 ⇒ 00:07:25.850 Awaish Kumar: The work you have been doing as a data scientist, data analyst, where you analyze more about analyzing the data, or trying to use machine learning models, or anything… something… something like that, and coming up with some answers for business
62 00:07:26.070 ⇒ 00:07:36.160 Awaish Kumar: Bi people, or the execs, right? The second workstream is where we are… we have analytics engineers.
63 00:07:36.290 ⇒ 00:07:46.160 Awaish Kumar: which are more sitting in between data engineers and data analysts, where you actually work with dbt,
64 00:07:46.620 ⇒ 00:07:55.379 Awaish Kumar: I’m writing SQL, and once the data isn’t in warehouse, it could be any, like Snowflake, BigQuery, Redshift, whatever it is.
65 00:07:55.480 ⇒ 00:07:59.890 Awaish Kumar: Once it is in a warehouse, then we have analytics engineers that sit
66 00:08:00.200 ⇒ 00:08:09.719 Awaish Kumar: and build these dbt models, and make sure that data is being transformed correctly for the end users, and
67 00:08:09.850 ⇒ 00:08:19.139 Awaish Kumar: Wrong… Make sure the data quality… data is reliable, and we are… we are maintaining data quality and integrity.
68 00:08:19.480 ⇒ 00:08:27.160 Awaish Kumar: The third part is data engineering, which is more like, Understanding of the infrastructure.
69 00:08:27.280 ⇒ 00:08:28.630 Awaish Kumar: So…
70 00:08:29.170 ⇒ 00:08:45.230 Awaish Kumar: the whole thing works, you can be pulled into doing anything, data, right? That means if somebody says my data is in some legacy AS400 server, and I need it in Snowflake.
71 00:08:45.360 ⇒ 00:08:49.950 Awaish Kumar: Then, as a data engineer, it’s your job to figure out the ways.
72 00:08:50.780 ⇒ 00:08:58.719 Awaish Kumar: either using different tools, writing Python, whatever you have to do to basically,
73 00:08:58.950 ⇒ 00:09:05.839 Awaish Kumar: Like, move their data, or… and it could be, like, batch streaming, or batch, or streaming, or…
74 00:09:06.090 ⇒ 00:09:09.460 Awaish Kumar: All kind of event streaming, whatever.
75 00:09:09.840 ⇒ 00:09:10.620 Awaish Kumar: And…
76 00:09:11.100 ⇒ 00:09:22.920 Awaish Kumar: So, yeah, these are kind of different workshops. I explore… I explain that because I really want to know where you see yourself moving, so I can ask, like, questions related to that.
77 00:09:23.370 ⇒ 00:09:25.130 Christina Knudson: Okay.
78 00:09:25.640 ⇒ 00:09:40.469 Christina Knudson: I see. So I… when I talked to the recruiter, somehow I got more of a impression that, like, analytics engineers were also doing, like, building dashboards, and that sort of work, so then…
79 00:09:40.850 ⇒ 00:09:43.700 Awaish Kumar: So, like, this… that is… that is, like,
80 00:09:43.780 ⇒ 00:09:48.730 Awaish Kumar: Being… that is the kind of being part of a startup company.
81 00:09:48.770 ⇒ 00:10:03.809 Awaish Kumar: and working really with slim teams, that you have to work, like, sometimes I’m data engineer, but I have to do analytics engineering, I also have to do dashboarding from time to time. That is just part of
82 00:10:04.100 ⇒ 00:10:20.679 Awaish Kumar: part of the, like, the job that week. Sometime, like, our analytics engineer is on leave, but we can’t say to the client that we can’t do that, right? Obviously, somebody has to fill for them. So, but the main point is that the
83 00:10:20.790 ⇒ 00:10:30.149 Awaish Kumar: The focus is giving you the work that you want to do, like, in that work stream, but obviously, there are times when you’re
84 00:10:30.350 ⇒ 00:10:33.980 Awaish Kumar: We have to go beyond what the normal day looks like.
85 00:10:34.750 ⇒ 00:10:39.179 Christina Knudson: Okay, I see. Well then, in that case, I think…
86 00:10:39.640 ⇒ 00:10:49.149 Christina Knudson: of the roles you’ve described, then probably analytics engineer sounds more, like, aligned with what my experience is.
87 00:10:50.150 ⇒ 00:10:59.039 Christina Knudson: And, like, I’m interested in understanding the, like, infrastructure better, but, like, that’s not stuff that I’ve done already.
88 00:11:00.160 ⇒ 00:11:00.830 Awaish Kumar: Okay.
89 00:11:02.410 ⇒ 00:11:07.550 Awaish Kumar: Yeah, then for data… the analytics engineers, We, like,
90 00:11:08.530 ⇒ 00:11:13.659 Awaish Kumar: Mostly, like, they sometimes can be… Involved in, like,
91 00:11:14.540 ⇒ 00:11:24.729 Awaish Kumar: infrastructure and things like that, but it’s not, like, the majority of work remains in the space of building models and supporting analysts.
92 00:11:24.890 ⇒ 00:11:30.749 Awaish Kumar: Whenever there are, like, modeling changes needed for supporting any dashboard.
93 00:11:30.880 ⇒ 00:11:37.080 Awaish Kumar: then, yeah, you will be on the hook to actually support them, and then… Okay.
94 00:11:37.270 ⇒ 00:11:42.699 Awaish Kumar: And majority of our modeling right now happening on dbt, because that’s the…
95 00:11:43.170 ⇒ 00:11:46.740 Awaish Kumar: One thing that is common in all of our clients, that…
96 00:11:47.040 ⇒ 00:12:04.849 Awaish Kumar: text state may vary. Ingestion… how we ingest the data may vary, but the transformation and the modeling kind of remains the same, that you have to use… once it lands in a warehouse, use dbt to model all the
97 00:12:05.310 ⇒ 00:12:11.759 Awaish Kumar: All the, like, the data. But it’s like, more of,
98 00:12:12.490 ⇒ 00:12:16.689 Awaish Kumar: Yeah, so… and then we use DVD for that, right? So, that’s the…
99 00:12:18.340 ⇒ 00:12:21.880 Awaish Kumar: And yeah, how much experience do you have with the DBT?
100 00:12:23.090 ⇒ 00:12:31.559 Christina Knudson: That’s, like, basically almost all that I did in 2025, and then, I’ve made some models also in, like.
101 00:12:32.360 ⇒ 00:12:45.259 Christina Knudson: the years before that, so I think I started with DBT in 2023, so a few months of experience in, like, 2023, a few months in 2024, and then, like, basically all of 2025, and, like.
102 00:12:45.260 ⇒ 00:12:45.910 Awaish Kumar: Yam.
103 00:12:45.910 ⇒ 00:12:46.760 Christina Knudson: This year.
104 00:12:47.570 ⇒ 00:12:50.659 Awaish Kumar: Okay, maybe you can then talk me through with the…
105 00:12:51.190 ⇒ 00:12:54.219 Awaish Kumar: like, how… how you rate yourself in DBT.
106 00:12:55.820 ⇒ 00:12:59.570 Christina Knudson: It’s… Oh, sorry.
107 00:13:00.290 ⇒ 00:13:05.180 Awaish Kumar: Yeah, I was just like, how would you rate yourself in DBDA, right? Yeah.
108 00:13:05.930 ⇒ 00:13:18.840 Christina Knudson: Okay, I think I’m pretty good, like, I am able to, like, make and debug the models myself, and, like, when something is breaking, then I’m able to, like.
109 00:13:19.170 ⇒ 00:13:26.529 Christina Knudson: you know, look at the error messages or the logs and figure out, like, what’s going on, even if it’s not a model that I made, like.
110 00:13:26.530 ⇒ 00:13:41.430 Christina Knudson: kind of similarly how your team is small, like, our team is also small, so yeah, if someone’s, like, out on vacation or something, then it’s like, well, if this is breaking, somebody needs to fix it, so I’m always happy to, like, jump in and see what’s going on and,
111 00:13:41.600 ⇒ 00:13:47.329 Christina Knudson: figure out a solution, even when it’s, like, not something that I had worked on previously.
112 00:13:47.970 ⇒ 00:13:54.440 Awaish Kumar: Okay, and okay, so maybe…
113 00:13:58.430 ⇒ 00:14:04.250 Awaish Kumar: How, like, what are the… what are different materialization techniques you have used in DBD?
114 00:14:04.750 ⇒ 00:14:17.279 Christina Knudson: I’ve materialized it as, like, a view, table, incremental table, yeah, those are the ones.
115 00:14:18.130 ⇒ 00:14:20.550 Christina Knudson: And then… like…
116 00:14:20.980 ⇒ 00:14:28.259 Christina Knudson: I set up… this is not quite what you asked, but then I’m also remembering, like, I set up, an exposure because…
117 00:14:28.400 ⇒ 00:14:35.070 Christina Knudson: That was just something we hadn’t had in place, and was something that… It sounded from…
118 00:14:35.650 ⇒ 00:14:44.279 Christina Knudson: like, in talking to a stakeholder, it sounded like that would be a useful thing for the model to have set up, so I set up an exposure, figured out how to do that.
119 00:14:44.440 ⇒ 00:14:58.760 Christina Knudson: By myself, and yeah, trying to think, like, what other DBT things… Would be useful to mention.
120 00:15:00.270 ⇒ 00:15:04.319 Christina Knudson: But I can’t think of any, so if you have other, like, Questions? .
121 00:15:05.960 ⇒ 00:15:10.830 Awaish Kumar: So, like, what are analytical functions in SQL?
122 00:15:12.050 ⇒ 00:15:13.609 Christina Knudson: analytical functions.
123 00:15:13.800 ⇒ 00:15:24.929 Christina Knudson: I guess… I don’t know that we actually used that term, so… I’m not sure.
124 00:15:25.130 ⇒ 00:15:32.439 Christina Knudson: Yeah, I don’t know… we haven’t used that word.
125 00:15:36.130 ⇒ 00:15:36.900 Awaish Kumar: Okay.
126 00:15:37.060 ⇒ 00:15:44.540 Awaish Kumar: Have you used, like, functions like… Some over-artition by…
127 00:15:45.110 ⇒ 00:15:45.990 Christina Knudson: Oh,
128 00:15:46.390 ⇒ 00:15:47.520 Awaish Kumar: Phone number. Yeah.
129 00:15:48.290 ⇒ 00:15:59.270 Christina Knudson: Yeah, like, row number, rank, partition, aggregate, qualify, Those sorts of things.
130 00:15:59.770 ⇒ 00:16:06.440 Awaish Kumar: No, but these row number, rank, these are the functions that… that are called analytical functions.
131 00:16:06.440 ⇒ 00:16:07.310 Christina Knudson: Oh, okay.
132 00:16:09.030 ⇒ 00:16:10.739 Awaish Kumar: Okay, and then what are the…
133 00:16:11.080 ⇒ 00:16:15.220 Awaish Kumar: concept of CTEs, and what is the benefit of using it?
134 00:16:16.040 ⇒ 00:16:27.980 Christina Knudson: Yeah, so a CT is really nice for making the code, like, modularized and readable, so instead of just having, like, one giant mess of a dbt model that’s
135 00:16:28.320 ⇒ 00:16:33.389 Christina Knudson: out of control to read. It’s like, okay, you have one CTE that has this particular goal.
136 00:16:33.950 ⇒ 00:16:48.760 Christina Knudson: and then you move to the next CTE, it has another goal, and each chunk is, like, doing one, like, understandable thing that’s pretty obvious from, like, maybe the naming of the CTE, or just, like, what’s going on in there. So it just, like.
137 00:16:48.860 ⇒ 00:16:55.200 Christina Knudson: Makes it so each piece is nicely modularized and organized, and then also it’s nice that you can, like.
138 00:16:55.500 ⇒ 00:17:00.069 Christina Knudson: debug more easily. So, like, if at the end
139 00:17:00.570 ⇒ 00:17:09.349 Christina Knudson: you see something unusual, then it’s really easy to, instead of saying, like, select star from final query, you can say select blah blah blah from, like.
140 00:17:09.650 ⇒ 00:17:20.470 Christina Knudson: whatever… whichever, you know, CTE you want to, so you can, like, zone in on where the… the bug or the weird thing is happening just a lot more easily.
141 00:17:22.020 ⇒ 00:17:28.950 Awaish Kumar: And, so… How… how you modularize code in dbt?
142 00:17:29.960 ⇒ 00:17:34.179 Christina Knudson: Happy to do it. So… I guess…
143 00:17:34.310 ⇒ 00:17:43.520 Awaish Kumar: dbt, right? So, like, there’s… There’s one way of doing it using CTEs, that’s when you are writing SQL.
144 00:17:43.660 ⇒ 00:17:46.169 Awaish Kumar: What else you can do if you are using dbt?
145 00:17:47.040 ⇒ 00:17:51.380 Christina Knudson: Yeah, you can, like, materialize it as,
146 00:17:52.420 ⇒ 00:17:57.329 Christina Knudson: like, I don’t remember what it’s called, but you can, like, basically make a model that doesn’t
147 00:17:57.720 ⇒ 00:18:00.279 Christina Knudson: Create a table in the…
148 00:18:00.520 ⇒ 00:18:08.070 Christina Knudson: in, like, Snowflake, but then you’re… it’s, like, basically a CTE, so that’s another thing you could do if it’s, like, maybe…
149 00:18:08.530 ⇒ 00:18:09.620 Christina Knudson: really…
150 00:18:10.550 ⇒ 00:18:19.889 Christina Knudson: if you’d have, like, a model that’s just, like, way too long of CTEs, then you could, like, break it up that way. That’s one way to modularize it. Or you can have, like.
151 00:18:20.070 ⇒ 00:18:28.669 Christina Knudson: If you do want to be able to see, like, the results from each step, then it could make sense to be like, okay, so the first model is doing…
152 00:18:29.090 ⇒ 00:18:40.920 Christina Knudson: like… One step of, like, maybe… Like, filtering the data, and… Like, aggregating it by…
153 00:18:41.380 ⇒ 00:18:45.270 Christina Knudson: User, and then the next one is calculating, like, okay, here’s, like.
154 00:18:45.970 ⇒ 00:18:49.640 Christina Knudson: this other step that’s happening.
155 00:18:49.810 ⇒ 00:18:55.490 Christina Knudson: Like, so for example, in the, like, models that I was making, then one of the upstream ones was, like.
156 00:18:55.930 ⇒ 00:19:03.649 Christina Knudson: Just an early step was, trying to just dedupe the records so that we would only have, like, one record for
157 00:19:03.760 ⇒ 00:19:15.450 Christina Knudson: user, or, like, per user per time period, and then that was, like, one big step, and then the next big step was, like, calculating some, like.
158 00:19:16.440 ⇒ 00:19:21.770 Christina Knudson: Like, just, like, a whole bunch of statistics per user, so, like, we had gotten the data that was…
159 00:19:22.330 ⇒ 00:19:25.139 Christina Knudson: Like… The record was…
160 00:19:26.010 ⇒ 00:19:37.859 Christina Knudson: like, each record… like, each user had, like, a lot of records, and so then we wanted to, like, collapse it down so that instead of having, like, each user have, like, 100 records, we just wanted to have, like, one user-level thing.
161 00:19:37.860 ⇒ 00:19:46.830 Christina Knudson: bunch of statistics, like monthly averages and this and that. So that’s, like, one thing you could do to modularize the code is, like.
162 00:19:47.400 ⇒ 00:19:49.250 Awaish Kumar: John, have you admitted?
163 00:19:49.540 ⇒ 00:19:51.159 Awaish Kumar: Have you used macros?
164 00:19:53.410 ⇒ 00:19:55.730 Awaish Kumar: Can you use it for categorizing code?
165 00:19:56.390 ⇒ 00:20:09.750 Christina Knudson: Yeah, so, like, for example, with those, like, monthly calculations, then that was really useful, just, like, writing a simple macro so that it was like, okay, so now we’re gonna have, like.
166 00:20:09.860 ⇒ 00:20:11.270 Christina Knudson: the volume…
167 00:20:12.660 ⇒ 00:20:27.050 Christina Knudson: so, like, we called it, like, usage, because it’s, like, electricity usage, so it’s, like, usage underscore 1, and then that’s, like, the monthly usage for the first month of the year. Usage underscore 2 is the monthly usage for, like, the second month of the year, and so, like, that would be really,
168 00:20:28.280 ⇒ 00:20:29.230 Christina Knudson: Kind of.
169 00:20:30.530 ⇒ 00:20:40.379 Christina Knudson: annoying to write and annoying to maintain. If you were just, like, usage 1 equals usage 2 equals and, like, really repetitive, but with a macro, then, that was really easy and just, like.
170 00:20:40.930 ⇒ 00:20:44.840 Christina Knudson: Pretty straightforward. And then, also, we had…
171 00:20:44.860 ⇒ 00:21:03.070 Christina Knudson: like, with those usage pipelines, like, it starts off as one, and then it kind of splits into two similar, but not quite the same pipelines, where it’s like, okay, for this pipeline, we want to have, like, all of these records, and then for this pipeline, we only want to have, like, a subset of those records, but we want to do the same, basically, like.
172 00:21:03.810 ⇒ 00:21:10.100 Christina Knudson: process to model those records, and so then, I used a macro to build those
173 00:21:10.130 ⇒ 00:21:23.839 Christina Knudson: like, processes that I wouldn’t have to change it in one place, and then forget to change it in the other, or like, you know, if someone else starts working on it, then it’s easier to understand, like, just fix it here, and it’ll help both.
174 00:21:24.860 ⇒ 00:21:30.959 Awaish Kumar: And, what is the medallion architecture?
175 00:21:32.040 ⇒ 00:21:34.540 Awaish Kumar: Have you used it in your projects?
176 00:21:35.830 ⇒ 00:21:38.710 Christina Knudson: So… I have…
177 00:21:39.410 ⇒ 00:21:53.060 Christina Knudson: benefited from it, but I have not, like, used it myself, so, like, what we used it for was, like, copying data from Redshift to Snowflake, and…
178 00:21:54.150 ⇒ 00:22:01.379 Christina Knudson: I think we just had, like, a timed job set up every day for copying over, like, the new records that came in.
179 00:22:02.600 ⇒ 00:22:03.110 Awaish Kumar: Ben, what are…
180 00:22:03.110 ⇒ 00:22:03.480 Christina Knudson: that.
181 00:22:03.480 ⇒ 00:22:04.340 Awaish Kumar: Yeah.
182 00:22:07.110 ⇒ 00:22:07.810 Christina Knudson: Oh, sorry.
183 00:22:08.660 ⇒ 00:22:12.570 Awaish Kumar: Yeah, my question is, like, what is Medellin architecture?
184 00:22:13.390 ⇒ 00:22:24.540 Christina Knudson: Okay, I guess I… Don’t really understand, like, what the question means.
185 00:22:26.020 ⇒ 00:22:26.630 Awaish Kumar: Okay.
186 00:22:26.630 ⇒ 00:22:27.090 Christina Knudson: Oh.
187 00:22:27.930 ⇒ 00:22:33.850 Awaish Kumar: There is an architecture when you… ingest data,
188 00:22:34.490 ⇒ 00:22:38.129 Awaish Kumar: Transform it, and… and make it, like,
189 00:22:38.640 ⇒ 00:22:44.159 Awaish Kumar: A golden dataset which is used by end users.
190 00:22:44.360 ⇒ 00:22:55.020 Awaish Kumar: So, there is a process, and which has a name, and that’s called Medellin architecture, but there’s some specifics that I didn’t, like.
191 00:22:55.720 ⇒ 00:23:04.979 Awaish Kumar: tell you, because I… that’s my question, I want to understand, like, if somebody… something, like, pops in… pops up in your mind, and I’m talking about, like, injection…
192 00:23:05.210 ⇒ 00:23:08.180 Awaish Kumar: Transformation, and we’re making a golden dataset.
193 00:23:10.300 ⇒ 00:23:12.160 Awaish Kumar: Anything that you recall, like…
194 00:23:17.850 ⇒ 00:23:18.769 Christina Knudson: Have you looked…
195 00:23:18.770 ⇒ 00:23:21.270 Awaish Kumar: How do you structure your DVT project?
196 00:23:22.870 ⇒ 00:23:26.529 Christina Knudson: So then we structure it with, like.
197 00:23:26.820 ⇒ 00:23:36.250 Christina Knudson: We have, like, the project, and then we have different, like, folders for each of the, kind of,
198 00:23:36.590 ⇒ 00:23:53.860 Christina Knudson: the, like, schemas, and then within each of those, then we have, like, a folder for each one of these, like, families of models, like usage, or, like, one about, like, accounts or customers or whatever. So we have, like, that nested architecture.
199 00:23:54.640 ⇒ 00:24:02.680 Awaish Kumar: So in the models, you might have raw, intermediate, staging, production, different parts, like, different layers.
200 00:24:02.940 ⇒ 00:24:03.360 Christina Knudson: Yeah.
201 00:24:03.360 ⇒ 00:24:07.130 Awaish Kumar: I… Basically, these layers are kind of…
202 00:24:07.270 ⇒ 00:24:11.539 Awaish Kumar: like, the Mediterranean architecture. So, the…
203 00:24:11.700 ⇒ 00:24:20.529 Awaish Kumar: You have all your odd data come, like the raw models, then you have a staging where you do some transformation.
204 00:24:21.380 ⇒ 00:24:27.699 Awaish Kumar: intermediate, and then March. So, basically, that… that’s what middle-in architecture sees.
205 00:24:28.550 ⇒ 00:24:34.249 Christina Knudson: Okay, I see. Yes, we have it like that. We have the raw, and then we have…
206 00:24:34.570 ⇒ 00:24:37.199 Christina Knudson: intermediate. We used to have…
207 00:24:37.800 ⇒ 00:24:43.040 Christina Knudson: I guess we do kind of have staging, but we treat that more as, like, the…
208 00:24:43.590 ⇒ 00:24:56.799 Christina Knudson: where all the, like, PR builds go, and then use that for, like, just kind of checking things before we merge it to main, and then, but, like, in production, it’s, like, raw, intermediate, and then the, like, marts.
209 00:24:57.640 ⇒ 00:24:58.350 Awaish Kumar: Okay.
210 00:25:00.900 ⇒ 00:25:05.690 Awaish Kumar: And, so, like, apart from that,
211 00:25:06.370 ⇒ 00:25:12.640 Awaish Kumar: Do you… are you familiar with the… With a dimensional modeling.
212 00:25:13.850 ⇒ 00:25:22.359 Christina Knudson: Dimensional modeling… I guess that word, I don’t… or that phrase, I don’t…
213 00:25:22.710 ⇒ 00:25:26.689 Christina Knudson: Know off the top of my head what it means, but…
214 00:25:27.540 ⇒ 00:25:32.190 Christina Knudson: it may be… I don’t know, if I was to guess, I’d either think it’s to do with, like.
215 00:25:35.290 ⇒ 00:25:38.379 Christina Knudson: Oh, dimensions as in, like, features and…
216 00:25:38.960 ⇒ 00:25:45.420 Christina Knudson: fields in the… okay, actually, you know, I don’t… I’m not really sure what, like, Dimensionally.
217 00:25:46.350 ⇒ 00:25:52.650 Awaish Kumar: Yeah, dimensional modeling is, is a way of… Model your data.
218 00:25:53.880 ⇒ 00:25:59.110 Awaish Kumar: When you are building mods, You create some dim and fact tables.
219 00:25:59.840 ⇒ 00:26:02.280 Awaish Kumar: That’s called dimension modeling.
220 00:26:03.070 ⇒ 00:26:08.799 Christina Knudson: Okay. I guess I have heard, like, dimension and fact,
221 00:26:09.090 ⇒ 00:26:17.080 Christina Knudson: a little bit when we were, like, setting up the architecture, but then I think because most of us weren’t actually familiar with those
222 00:26:17.270 ⇒ 00:26:23.090 Christina Knudson: Words, then we just, like… Kinda… stuck.
223 00:26:23.470 ⇒ 00:26:25.569 Christina Knudson: Talking like that, and then just…
224 00:26:25.690 ⇒ 00:26:29.960 Christina Knudson: I don’t know, talked more in terms of, like, the actual data that we had.
225 00:26:30.230 ⇒ 00:26:34.049 Christina Knudson: But I guess now that you’re mentioning it, I am remembering those are things I’ve heard.
226 00:26:35.100 ⇒ 00:26:39.140 Awaish Kumar: Okay, I think we are on time, we are just, like, 3 minutes…
227 00:26:41.080 ⇒ 00:26:43.560 Awaish Kumar: We just have 3 minutes, so yeah, I will leave.
228 00:26:44.260 ⇒ 00:26:46.989 Awaish Kumar: Leave the time for you to ask any questions.
229 00:26:47.670 ⇒ 00:26:50.660 Christina Knudson: Okay. So I’m wondering, like.
230 00:26:51.460 ⇒ 00:26:55.450 Christina Knudson: What have you enjoyed about, like, working at Brainforge?
231 00:26:57.350 ⇒ 00:27:00.980 Awaish Kumar: Rather than joy, flexibility, ownership
232 00:27:04.430 ⇒ 00:27:12.370 Awaish Kumar: Yeah, that’s what I… like, I have… Had an opportunity.
233 00:27:12.740 ⇒ 00:27:20.849 Awaish Kumar: to… To take ownership of the… Projects, the pipelines, and the…
234 00:27:21.300 ⇒ 00:27:26.980 Awaish Kumar: The, like, the infrastructure here, and then, basically, Point.
235 00:27:27.180 ⇒ 00:27:31.169 Awaish Kumar: I had the flexibility to also come up with my own solutions.
236 00:27:31.560 ⇒ 00:27:39.420 Awaish Kumar: But then, that comes with accountability and responsibility that, come with what I’ve been doing.
237 00:27:40.790 ⇒ 00:27:45.099 Awaish Kumar: So that’s what I enjoy here. So it’s a second thing is…
238 00:27:45.590 ⇒ 00:27:51.869 Awaish Kumar: Learning, like, the knowledge that you receive, the speed at which You received the knowledge?
239 00:27:53.330 ⇒ 00:28:00.559 Awaish Kumar: Like, it’s a fast-paced environment, and most of the startups are like that, but here it’s a little bit more.
240 00:28:00.830 ⇒ 00:28:08.090 Awaish Kumar: past-based, with the emergence of AI, obviously, in our day-to-day,
241 00:28:08.350 ⇒ 00:28:24.470 Awaish Kumar: workflows, so we are fast-paced, we’re learning at a very fast pace, and then we are using AI in our daily routine work, building models using AI and things like that. That really speeds up our development, but then
242 00:28:24.580 ⇒ 00:28:31.830 Awaish Kumar: That makes… give… maybe… Improves your… Productivity.
243 00:28:34.170 ⇒ 00:28:40.589 Awaish Kumar: And yeah, that way you can learn new stuff, you could maybe have some time for learning.
244 00:28:40.790 ⇒ 00:28:42.040 Awaish Kumar: Something else.
245 00:28:42.920 ⇒ 00:28:45.469 Christina Knudson: Yeah, I really appreciate that, because…
246 00:28:45.630 ⇒ 00:29:00.210 Christina Knudson: I really enjoy learning. I don’t want to just, like, do the same thing over and over without, like, you know, trying something better, especially when you know that there is something, like, more efficient or safer to do, but, like.
247 00:29:00.450 ⇒ 00:29:09.119 Christina Knudson: it’s nice that you are encouraged to do that, and you have, like, the flexibility to come up with your own solutions like that. Like, those…
248 00:29:09.590 ⇒ 00:29:12.809 Christina Knudson: But, like, flexibility and ownership, those are things that are, like.
249 00:29:13.140 ⇒ 00:29:16.120 Christina Knudson: Really important to me, too, so that’s really good to hear.
250 00:29:16.390 ⇒ 00:29:26.920 Christina Knudson: Cool. About, like, what kind of, like, challenges have the, like, company or team had recently, and, like, how did you handle those?
251 00:29:29.080 ⇒ 00:29:38.860 Awaish Kumar: Yeah, like, the challenges are, as I already mentioned, like, there… we are in a startup environment, in a startup environment, and also…
252 00:29:39.420 ⇒ 00:29:46.330 Awaish Kumar: We are a… like, we have a lot of clients that we serve, so while doing that, like,
253 00:29:46.610 ⇒ 00:29:57.160 Awaish Kumar: Obviously, we get, Ambiguous requirements, or… From the… Client, and then… We have,
254 00:29:57.810 ⇒ 00:30:05.989 Awaish Kumar: And learn, like… Like, a layer of people, like, that talk to the client, and then…
255 00:30:06.230 ⇒ 00:30:11.189 Awaish Kumar: Obviously, you are in… delivery team, and,
256 00:30:11.440 ⇒ 00:30:19.690 Awaish Kumar: you receive that requirement, work on that, but majority of the time, like, those requirements are unclear, so the challenges could be that,
257 00:30:20.440 ⇒ 00:30:22.919 Awaish Kumar: You have to figure out your requirements.
258 00:30:23.710 ⇒ 00:30:25.490 Awaish Kumar: And we don’t want to get stuck.
259 00:30:25.630 ⇒ 00:30:29.859 Awaish Kumar: By just saying that, okay, I’m logged on client, so…
260 00:30:29.970 ⇒ 00:30:34.900 Awaish Kumar: You keep hustling to get those requirements and actually delivering the work.
261 00:30:35.030 ⇒ 00:30:35.710 Awaish Kumar: Huh.
262 00:30:36.430 ⇒ 00:30:52.440 Awaish Kumar: And, like, we are coming with… coming up with processes, like, how to do that better, what… how can I grip that info… all of the requirements in one go, and things like that. But these are the kind of challenges, apart from that, like.
263 00:30:53.880 ⇒ 00:30:58.119 Awaish Kumar: I don’t think so, like, there’s any, like, obviously, new tool, new…
264 00:30:58.240 ⇒ 00:31:00.590 Awaish Kumar: Whenever a new client comes in, there’s a…
265 00:31:01.890 ⇒ 00:31:04.020 Awaish Kumar: New tech stake that they have.
266 00:31:04.190 ⇒ 00:31:12.649 Awaish Kumar: a complete different state of the data infrastructure they have, and then you have to take it and improve it. So it’s an always-a-
267 00:31:13.980 ⇒ 00:31:27.220 Awaish Kumar: challenge, but that makes… also keeps you exciting about, okay, now that you have something to do, like, it’s not a… you are not just on a maintenance mode all the time, you are actually…
268 00:31:27.360 ⇒ 00:31:35.329 Awaish Kumar: Trying to solve different things for different clients, and they are coming up, like, every 2-3 months, you…
269 00:31:35.880 ⇒ 00:31:42.669 Awaish Kumar: Once you are finished with One of the projects, and then you get the new one, and that’s, like.
270 00:31:43.310 ⇒ 00:31:52.079 Awaish Kumar: That’s… That… that can be challenging, but that also is what keeps us… Excited about the projects.
271 00:31:52.770 ⇒ 00:31:58.279 Christina Knudson: Right, yeah. You always get to see something new and different, and yeah, maintenance mode can be…
272 00:31:58.590 ⇒ 00:32:01.859 Christina Knudson: Like, definitely less exciting, less interesting, so…
273 00:32:02.390 ⇒ 00:32:07.790 Christina Knudson: Okay, cool. And then, like, the fuzzy end-user requirements, that sounds like
274 00:32:09.060 ⇒ 00:32:12.209 Christina Knudson: The… a normal challenge, so that makes sense.
275 00:32:13.370 ⇒ 00:32:14.080 Awaish Kumar: Okay.
276 00:32:14.460 ⇒ 00:32:15.070 Christina Knudson: Okay.
277 00:32:15.860 ⇒ 00:32:21.409 Christina Knudson: Well, thank you for talking with me and for, you know, teaching me a few new phrases.
278 00:32:22.450 ⇒ 00:32:23.010 Christina Knudson: Thank you.
279 00:32:23.010 ⇒ 00:32:27.400 Awaish Kumar: Yeah, yeah, no worries, and I think thank you for your time.
280 00:32:27.880 ⇒ 00:32:34.529 Awaish Kumar: we… yeah, once I get back to the team with the feedback, our recruiters will get back to you.
281 00:32:35.430 ⇒ 00:32:37.320 Awaish Kumar: And… And yeah.
282 00:32:37.790 ⇒ 00:32:42.430 Awaish Kumar: That’s… And from here, like, we have a process where…
283 00:32:42.940 ⇒ 00:32:47.320 Awaish Kumar: The process is typically, like, if whatever, based on our
284 00:32:47.430 ⇒ 00:32:51.939 Awaish Kumar: The feedback, if we move forward, then there will be a similar
285 00:32:52.290 ⇒ 00:32:54.739 Awaish Kumar: One-to-one session with one of my colleagues.
286 00:32:55.010 ⇒ 00:32:58.610 Awaish Kumar: And then maybe there will be some take-home assignment.
287 00:32:58.850 ⇒ 00:33:03.899 Awaish Kumar: And after that, A panel interview, which… where we discuss whatever
288 00:33:04.370 ⇒ 00:33:09.109 Awaish Kumar: We… whatever, like, the assignment was, and how did you…
289 00:33:10.500 ⇒ 00:33:14.120 Awaish Kumar: Implemented that, and then, yeah, that’s… that’s all.
290 00:33:14.920 ⇒ 00:33:16.240 Christina Knudson: Okay, sounds great.
291 00:33:16.440 ⇒ 00:33:17.020 Christina Knudson: Thank you.
292 00:33:17.020 ⇒ 00:33:17.390 Awaish Kumar: Okay.
293 00:33:17.390 ⇒ 00:33:18.440 Christina Knudson: Yeah.
294 00:33:18.610 ⇒ 00:33:20.290 Christina Knudson: Okay, have a good one.
295 00:33:20.690 ⇒ 00:33:21.470 Awaish Kumar: Right.