Meeting Title: Brainforge Data Engineer Interview Date: 2026-02-26 Meeting participants: Jayneel Shah, Awaish Kumar, Kaela Gallagher
WEBVTT
1 00:04:15.740 ⇒ 00:04:16.540 Jayneel Shah: Hello.
2 00:04:42.530 ⇒ 00:04:42.860 Awaish Kumar: Hi!
3 00:04:43.440 ⇒ 00:04:44.410 Jayneel Shah: Hi, Jimmy.
4 00:04:44.850 ⇒ 00:04:45.760 Jayneel Shah: How are you?
5 00:04:46.030 ⇒ 00:04:50.280 Awaish Kumar: Sorry, I was just, filling out my bike. Oh.
6 00:04:50.410 ⇒ 00:04:57.359 Awaish Kumar: Okay, yeah, let’s… let’s wait, maybe one of our… Lead recruiter will join, so…
7 00:04:57.360 ⇒ 00:04:58.699 Jayneel Shah: Oh, okay.
8 00:05:06.240 ⇒ 00:05:07.460 Awaish Kumar: By the way, how…
9 00:05:07.850 ⇒ 00:05:15.320 Awaish Kumar: Yeah, before she joins, I can just briefly introduce myself and the company. So, great to meet you.
10 00:05:15.620 ⇒ 00:05:19.640 Awaish Kumar: I’m Avish Kumar, and I’m kind of leading data engineering,
11 00:05:19.750 ⇒ 00:05:37.390 Awaish Kumar: service here at Bayforge, and we are a consultancy providing data and AI consultancy services to large-scale enterprises. Mostly, we are building, like, production-grade data systems for our clients.
12 00:05:38.250 ⇒ 00:05:45.600 Awaish Kumar: That usually means, like, high ownership, amicus requirements, and strong focus on.
13 00:05:45.660 ⇒ 00:05:48.820 Jayneel Shah: Like, reliability in production environments.
14 00:05:48.970 ⇒ 00:05:49.569 Awaish Kumar: Oh, no.
15 00:05:49.570 ⇒ 00:05:50.160 Jayneel Shah: the tool.
16 00:05:50.700 ⇒ 00:05:54.800 Awaish Kumar: So… For this interview today,
17 00:05:55.560 ⇒ 00:06:04.990 Awaish Kumar: We are just going to, like, bit deeper into your past experiences and what you have been doing, how you approach, different,
18 00:06:05.340 ⇒ 00:06:08.459 Awaish Kumar: Like, problem solving?
19 00:06:09.980 ⇒ 00:06:14.400 Awaish Kumar: How do you, like, basically approach different problems?
20 00:06:14.400 ⇒ 00:06:14.890 Jayneel Shah: That makes sense.
21 00:06:15.470 ⇒ 00:06:20.519 Awaish Kumar: And then we are just going to walk through some scenarios together. Sounds good?
22 00:06:20.810 ⇒ 00:06:21.940 Jayneel Shah: Yeah, sounds good.
23 00:06:22.780 ⇒ 00:06:23.550 Awaish Kumar: Okay.
24 00:06:25.220 ⇒ 00:06:31.019 Awaish Kumar: Let’s start, Like, with your brief overview.
25 00:06:31.170 ⇒ 00:06:32.150 Jayneel Shah: Sure, yeah.
26 00:06:33.060 ⇒ 00:06:49.780 Jayneel Shah: So, I’m a master’s student in computer science. I study at NC State. This is my last semester. I’ll be graduating in May 26. So, I mainly focus on data analytics and a bit of software engineering and machine learning, too.
27 00:06:49.780 ⇒ 00:07:07.439 Jayneel Shah: So, I have previously built data pipelines using Python and SQL, both in academic as well as industry settings. So, I’ve worked at PayPal before, my most recent internship was at Charles Aries Inc, which is here in Greensboro.
28 00:07:07.440 ⇒ 00:07:20.899 Jayneel Shah: So, there, too, I was kind of in charge of the data pipeline thing. It was more towards AI automation, but yep, there was a bit of data engineering here and there.
29 00:07:21.330 ⇒ 00:07:29.279 Jayneel Shah: And, I got inter… like, when I saw about Brainforge, there are many consultants, like, nowadays I’m seeing…
30 00:07:29.280 ⇒ 00:07:32.760 Awaish Kumar: What experience looks like, apart from internships?
31 00:07:32.760 ⇒ 00:07:47.579 Jayneel Shah: It is, like, I do not have any other work experience apart from internships. It is all based on that. Like, I graduated in 24 with my bachelor’s degree, and I came straight here to the US after that.
32 00:07:47.760 ⇒ 00:07:50.370 Jayneel Shah: So, I don’t have any full-time experience there.
33 00:07:51.540 ⇒ 00:08:08.769 Awaish Kumar: Okay, yeah, sounds good. I… yeah, since you already, once again mentioned PayPal experience, I already saw that in your Loom introduction. Yeah. Maybe we can just a little bit dive deeper into that project, in terms of what exactly you did. Okay.
34 00:08:09.500 ⇒ 00:08:12.349 Awaish Kumar: Like, yeah, like, you just don’t know.
35 00:08:12.350 ⇒ 00:08:24.039 Jayneel Shah: Yeah, like, a quick overview. Sounds good? Yeah. Yeah, so there, like, it was my first week where we, I was assigned to the merchant side of the things.
36 00:08:24.040 ⇒ 00:08:43.130 Jayneel Shah: So, we have the consumer and the merchant, so I was mainly with the merchant department. So, there, the… whatever feedback the merchant used to give us, it was logged into Salesforce, and as you know, in Salesforce, the data was kind of messy and unstructured, and it was scattered across categories.
37 00:08:43.130 ⇒ 00:08:59.620 Jayneel Shah: So, due to this, the teams that were handling the requests, they were missing the complaints, there were delayed responses, and there was kind of unsatisfaction within the merchants that we had, and there was a churn. So…
38 00:08:59.620 ⇒ 00:09:03.540 Jayneel Shah: My manager, he was in charge of this team.
39 00:09:03.650 ⇒ 00:09:16.059 Jayneel Shah: And as an intern, my job was to help him to build a dashboard to solve this exact issue. So, from this thing, I understood that the main issue was visibility, that…
40 00:09:16.060 ⇒ 00:09:28.170 Jayneel Shah: The people those who were handling those requests, they were mostly non-technical, because some requests could be not related to, like, getting one of the services up.
41 00:09:28.170 ⇒ 00:09:36.519 Jayneel Shah: Others could be more technical, like, they were not able to integrate the system into the checkout, or…
42 00:09:36.520 ⇒ 00:10:01.420 Jayneel Shah: things with the payments and the charges, etc. So, since there was no centralization, my task was to build a dashboard that could simplify this entire thing. So, I chose some KPIs, as in merchant size, verticals, issue categories, etc. So, once I identified this, I took the Salesforce, into, converted that to a
43 00:10:01.420 ⇒ 00:10:08.830 Jayneel Shah: Excel sheet, just for the basic prototyping. From the Excel sheet, I made a basic Tableau dashboard.
44 00:10:08.830 ⇒ 00:10:13.920 Awaish Kumar: Well, that… Were you only doing the prototyping, or did you actually build a…
45 00:10:13.920 ⇒ 00:10:16.909 Jayneel Shah: I did build it. Like, the first phase was…
46 00:10:16.910 ⇒ 00:10:17.980 Awaish Kumar: talked about?
47 00:10:18.150 ⇒ 00:10:26.769 Awaish Kumar: Let’s just talk about what you build, an end-to-end solution, how you ingested the data, how you transformed the data, and then…
48 00:10:27.060 ⇒ 00:10:40.490 Jayneel Shah: Okay, so, like, it was a kind of lightweight ETL, I would say, extracting the data from Salesforce, transforming them into categories, cleaning the fields, and then loading that into the Tableau.
49 00:10:42.590 ⇒ 00:10:48.489 Jayneel Shah: So, what… so, should I talk through the entire process, or.
50 00:10:48.940 ⇒ 00:10:52.690 Awaish Kumar: Can you just briefly give me an overview of what tools were used, how you ingested the data?
51 00:10:52.690 ⇒ 00:10:53.380 Jayneel Shah: Okay.
52 00:10:53.380 ⇒ 00:10:55.370 Awaish Kumar: How, you process the data.
53 00:10:55.370 ⇒ 00:10:56.080 Jayneel Shah: Oh…
54 00:10:56.310 ⇒ 00:10:58.819 Awaish Kumar: How have you structured your warehouse?
55 00:10:59.090 ⇒ 00:10:59.850 Awaish Kumar: Right?
56 00:11:01.020 ⇒ 00:11:06.770 Jayneel Shah: Okay, so, like, I did not go into that depth, because my internship was only for 3 months.
57 00:11:06.770 ⇒ 00:11:21.129 Jayneel Shah: So, what I’m describing right now is what we built towards the end of my internship, because once the prototyping stage was over, we had to get approvals from the technical and the non-technical stakeholders.
58 00:11:21.130 ⇒ 00:11:38.980 Jayneel Shah: whether they liked the design or not. But towards the end, while we were, like, in the process of connecting the Salesforce API directly to the Tableau, that was the time my internship was ending, but that was the time I was doing a separate project, which was based on SQL and Python.
59 00:11:39.530 ⇒ 00:11:56.440 Jayneel Shah: So, I don’t know about the warehousing part and the going into the depth of the things, but I can tell you the process that we had thought of at the time of, you know, starting with the prototypes.
60 00:11:57.970 ⇒ 00:11:58.770 Awaish Kumar: Okay.
61 00:11:59.140 ⇒ 00:12:06.540 Awaish Kumar: So let’s talk about how, you mentioned that you were involved in,
62 00:12:06.660 ⇒ 00:12:11.109 Awaish Kumar: prototyping. Obviously, you get, got some data, you…
63 00:12:11.400 ⇒ 00:12:19.969 Awaish Kumar: transformed it. So, after getting that data, I just want to know two things. One.
64 00:12:20.080 ⇒ 00:12:36.029 Awaish Kumar: how you transformed it, what kind of tools you used, apart from Python and SQL, if there were any used. Second thing, I just want to know how you basically shared your findings with the non-technical stakeholders.
65 00:12:36.390 ⇒ 00:12:49.269 Jayneel Shah: Okay, sounds good. So, firstly, transforming, it was mainly, Python and SQL, so we used… briefly used Teradata, but it was going to be discontinued,
66 00:12:49.270 ⇒ 00:13:08.479 Jayneel Shah: by the end of my internship, so we did not go deep into that. So it was basically Python scripts and SQL embedded into it. So these were the… mainly the two things that we used for transforming the data. So, the transformation, it included, like, all the data, the…
67 00:13:08.480 ⇒ 00:13:15.820 Jayneel Shah: merchant complaint that we had. We tried to extract, what business vertical it was affecting the most.
68 00:13:16.060 ⇒ 00:13:28.300 Jayneel Shah: And, if, say, it was a big merchant that was bringing us more revenue, so my task was to highlight them more, obviously because they’ll give us big revenue. So, yeah.
69 00:13:28.300 ⇒ 00:13:32.290 Awaish Kumar: How did you share your findings with the non-technical stakeholders?
70 00:13:32.290 ⇒ 00:13:47.640 Jayneel Shah: I’m getting to that. Yeah. So, that was the next part. So, since the non-technical stakeholders, they used to understand only 3 things, the business verticals, and the amount of revenue we used to get from the merchants.
71 00:13:47.640 ⇒ 00:14:02.679 Jayneel Shah: So, I, to convey it to them, I had to keep my, visualizations as simple as possible. So, sharing, those with them, I tried to build charts which were, you know, kind of bar charts.
72 00:14:02.680 ⇒ 00:14:09.230 Jayneel Shah: Which were focusing on the particular, business vertical, whether it was… What?
73 00:14:09.230 ⇒ 00:14:13.420 Awaish Kumar: Were there any tools used for the… for the editing the reports, or…
74 00:14:13.490 ⇒ 00:14:14.889 Jayneel Shah: It was all tableau.
75 00:14:15.570 ⇒ 00:14:16.350 Awaish Kumar: Okay.
76 00:14:16.760 ⇒ 00:14:21.979 Jayneel Shah: So, the transformation, like, once I got it from Salesforce.
77 00:14:21.980 ⇒ 00:14:42.019 Jayneel Shah: converted it to an Excel sheet, then I ingested it, using Python, like, pandas and everything, and the basic transformation was done there, and that clean data was used for creating the visualization in Tableau. So, the ingestion was done directly in Tableau, so there was no external tool which I used there.
78 00:14:42.020 ⇒ 00:14:44.110 Awaish Kumar: So, you didn’t use any database?
79 00:14:44.410 ⇒ 00:14:45.820 Jayneel Shah: No, we did not.
80 00:14:46.490 ⇒ 00:15:03.030 Jayneel Shah: Okay. Because, like, the plan was to, like, whatever I had built, that was Excel sheet, right? So, our plan was to directly connect the, Salesforce API into Tableau, and the data ingestion would take place directly from there.
81 00:15:03.830 ⇒ 00:15:08.850 Awaish Kumar: Okay, my next question is that you talked about data quality.
82 00:15:09.190 ⇒ 00:15:21.930 Awaish Kumar: How you handled during the… into Azure project, how do you ensure that all your transformations are correct and there’s no data quality issues, occurring?
83 00:15:22.250 ⇒ 00:15:24.010 Awaish Kumar: As a result.
84 00:15:24.970 ⇒ 00:15:31.150 Jayneel Shah: to, like, not getting, like, empty data, or if there’s.
85 00:15:31.150 ⇒ 00:15:34.020 Awaish Kumar: Whatever data quality checks you implemented.
86 00:15:34.590 ⇒ 00:15:46.230 Awaish Kumar: Second thing you mentioned about data observability, so only… I just want to know, like, how you implemented data quality checks, how did you ensure that, the… the…
87 00:15:46.440 ⇒ 00:15:49.870 Awaish Kumar: The, like, the teams have the observability.
88 00:15:49.870 ⇒ 00:16:01.110 Jayneel Shah: Okay, so, like, like, at the initial stage, we had certain KPIs that were given to us, so I used to check whether the, schemas and the types
89 00:16:01.110 ⇒ 00:16:14.320 Jayneel Shah: after transformation were similar to what we had before. I also tried to map it to categories, and wherever possible, have the non-null checks.
90 00:16:14.320 ⇒ 00:16:28.250 Jayneel Shah: So, like, these were the basic things that I used to do. Like, at every stage, I used to ensure whether the schemas and the data types of the things, they were matching what I had previously or not.
91 00:16:28.370 ⇒ 00:16:36.530 Jayneel Shah: And, if at any stage I used to get stuck, I used to, like, try to debug where it went wrong.
92 00:16:39.100 ⇒ 00:16:39.880 Awaish Kumar: Okay.
93 00:16:40.030 ⇒ 00:16:49.569 Awaish Kumar: Then my next question is, for example, you shared your filings with the stakeholders, and what if they…
94 00:16:49.870 ⇒ 00:16:53.039 Awaish Kumar: Don’t agree with your findings, or…
95 00:16:53.370 ⇒ 00:16:58.570 Awaish Kumar: visual analysis. So, like, how could you convey
96 00:17:00.170 ⇒ 00:17:02.589 Awaish Kumar: How would you convey your,
97 00:17:04.319 ⇒ 00:17:08.249 Awaish Kumar: your… your findings with analysis? Like, how will you convince them?
98 00:17:10.339 ⇒ 00:17:15.709 Jayneel Shah: Okay, so, like, whether they agree to that or not is your question, right?
99 00:17:15.710 ⇒ 00:17:22.480 Awaish Kumar: Yeah, for example, if we… we shared some analysis, but maybe execs knows from the…
100 00:17:22.670 ⇒ 00:17:34.500 Awaish Kumar: they look from, above, so they might not agree with your figures, findings, patterns, facts. So how would you then come up with what would you do, what things you will.
101 00:17:34.500 ⇒ 00:17:37.309 Jayneel Shah: To make them kind of share.
102 00:17:38.190 ⇒ 00:17:44.299 Jayneel Shah: make them believe, like, make them understand that this is the right thing. Is that your question?
103 00:17:44.470 ⇒ 00:17:45.260 Jayneel Shah: Oft!
104 00:17:45.260 ⇒ 00:17:45.590 Awaish Kumar: Yes.
105 00:17:45.590 ⇒ 00:18:02.750 Jayneel Shah: So, like, as I mentioned, that there were some key metrics and key trends that we were, after, in the entire analysis. So, since it is surely possible, like, in one of my first meetings.
106 00:18:02.750 ⇒ 00:18:16.360 Jayneel Shah: He was not able to understand, like, what chart went, or, like, what did the chart actually mean, and why was it important there. So, my first thought is always to,
107 00:18:16.360 ⇒ 00:18:39.550 Jayneel Shah: like, I like constructive feedback, so I understood. I tried to understand his point of view, why he was not understanding the chart. In the next iteration, I tried to make it more explicit in terms of what it was trying to convey. So, like, it was a bar chart which was highlighting the business vertical, where it is affecting the most.
108 00:18:39.550 ⇒ 00:18:45.549 Jayneel Shah: So I tried to tie it back to the key trends that we had observed in one of the other line charts.
109 00:18:45.550 ⇒ 00:18:58.850 Jayneel Shah: So, like, trying to make them understand that this is one part of the analysis, which indicates that, and this is the second part, which is what we are connecting it back to.
110 00:18:58.850 ⇒ 00:19:05.609 Jayneel Shah: So, trying to give them multiple evidences of why that particular.
111 00:19:13.400 ⇒ 00:19:14.110 Awaish Kumar: No.
112 00:19:15.330 ⇒ 00:19:17.220 Awaish Kumar: Kayla, can you hear me, or…
113 00:19:20.150 ⇒ 00:19:24.160 Kaela Gallagher: Yes, I can hear you just fine. I think we lost Janel.
114 00:19:25.710 ⇒ 00:19:29.970 Jayneel Shah: I think it’s… it went and came back, I’m not sure what happened.
115 00:19:30.230 ⇒ 00:19:31.900 Awaish Kumar: Okay, no worries. You can go…
116 00:19:31.900 ⇒ 00:19:51.160 Jayneel Shah: So, telling that, I used to focus, mainly on, giving them evidences, so it was mainly outcome-based storytelling, rather than just that this is the chart, this is what you need to focus on. So, giving them evidence, it
117 00:19:51.160 ⇒ 00:19:56.530 Jayneel Shah: always strengthens my claim on what I’m trying to tell them, so, yeah.
118 00:19:57.140 ⇒ 00:19:57.920 Awaish Kumar: Okay.
119 00:19:58.220 ⇒ 00:20:03.090 Awaish Kumar: We’re working on the metric, like you mentioned about the KPIs.
120 00:20:03.860 ⇒ 00:20:14.569 Awaish Kumar: Was working out on the KPIs and the definition of those KPIs, what they really mean? Was there any, like, did you try to understand that as part of your, exercise?
121 00:20:15.300 ⇒ 00:20:38.340 Jayneel Shah: So, like, yes, after, I think, when we built this first prototype, the next part was developing the entire dashboard. So I spent about 2 to 3 days understanding what each KPI meant, because I came from a computer science background, and it was… the KPIs that were defined were in a finance setting.
122 00:20:38.340 ⇒ 00:20:44.989 Jayneel Shah: So, I took some time understanding what each used… each KPA used to mean.
123 00:20:45.000 ⇒ 00:20:50.360 Jayneel Shah: I don’t think I recall any of the KPIs, but it was…
124 00:20:50.420 ⇒ 00:21:08.210 Jayneel Shah: Like, roughly, it was based on, main aspect was revenue, or what is the most amount of revenue a company can get us, because those would be our big clients, and they need to be, like, more focused on, so…
125 00:21:10.360 ⇒ 00:21:11.250 Awaish Kumar: Okay.
126 00:21:11.590 ⇒ 00:21:19.400 Awaish Kumar: So, yeah, my… Yeah, then I think I would just love to know, like, how,
127 00:21:19.880 ⇒ 00:21:27.030 Awaish Kumar: Do you, like, have you ever had a chance to learn a new skill on the job to deliver something, like,
128 00:21:27.670 ⇒ 00:21:29.930 Awaish Kumar: Like, something meaningful.
129 00:21:31.300 ⇒ 00:21:41.490 Jayneel Shah: Sure. So, I’d like to reference one of my, more recent experiences for that. So, like, when I came to NC State, it was…
130 00:21:41.490 ⇒ 00:21:53.319 Jayneel Shah: I think after my first semester that I was finding a research assistantship job here, and, that, I have some, GIS experience.
131 00:21:53.590 ⇒ 00:22:08.420 Jayneel Shah: And I am a very good software engineer and a data analyst. So, this role, it was kind of a blend of everything. It was a GIS analysis project. So, basically, GIS software engineering is what the role was called.
132 00:22:08.420 ⇒ 00:22:21.730 Jayneel Shah: it required me to learn, something known as GrassGIS. It was a new technology I’d never used before. I had used, more familiar ArcGIS or something like, those tools.
133 00:22:21.730 ⇒ 00:22:40.360 Jayneel Shah: So, this required me to learn that, and, like, to get selected for that position, I had to, learn that skill and implement one of the tasks that the professor had given me. So, like, I tried to divide it into, like, two days of going through the videos, documentation.
134 00:22:40.400 ⇒ 00:22:46.180 Jayneel Shah: Next two days, setting up the system, writing small codes.
135 00:22:46.660 ⇒ 00:23:04.659 Jayneel Shah: And I think I was able to do that in 10 days, a little more than what I was expected to complete. But since it was a holiday period, I did get the job, and after that, I worked there for 8 months. Like, my contract was for 8 months.
136 00:23:04.890 ⇒ 00:23:13.330 Jayneel Shah: And I helped solve over, 7 to 8 high vulnerabilities that were existing in the code that we had.
137 00:23:13.550 ⇒ 00:23:23.460 Jayneel Shah: And, there were multiple commits, and I got to know a lot more people through that network that I had built, in those 8 months.
138 00:23:24.940 ⇒ 00:23:25.620 Awaish Kumar: Okay.
139 00:23:26.670 ⇒ 00:23:40.190 Awaish Kumar: I think I have got enough on your current work experiences. I would like to, like, brainstorm for a few minutes on… we are just going to, like, talk about a scenario.
140 00:23:40.570 ⇒ 00:23:42.760 Awaish Kumar: And I’m just going to…
141 00:23:43.020 ⇒ 00:23:47.729 Awaish Kumar: Tell a story, and then we are going to just build on top of that.
142 00:23:48.500 ⇒ 00:23:49.020 Jayneel Shah: food.
143 00:23:49.250 ⇒ 00:23:55.539 Awaish Kumar: So, like, you mentioned that you have experience with Python and SQL, so we are just going to take SQL as an example.
144 00:23:55.820 ⇒ 00:24:02.050 Awaish Kumar: And, I just, we are just going to assume that we have a warehouse.
145 00:24:02.430 ⇒ 00:24:05.280 Awaish Kumar: Where we have a table, which is, like, really big.
146 00:24:05.520 ⇒ 00:24:14.189 Awaish Kumar: And the query we are running on top of it is really very slow, which gives us response maybe, for example, in 5 minutes.
147 00:24:14.450 ⇒ 00:24:21.359 Awaish Kumar: And, so can you talk… walk me through some of the optimization techniques?
148 00:24:21.480 ⇒ 00:24:25.830 Awaish Kumar: You may use, to optimize it.
149 00:24:28.100 ⇒ 00:24:35.189 Jayneel Shah: So, like, we have a data warehouse, it has a very large table, but the time it is returning the output…
150 00:24:35.190 ⇒ 00:24:51.899 Awaish Kumar: We have a table with millions of rows, and I have a query, I’m running on top of it, which is really slow. I’m not sure what the problem might be with my architecture. My database architecture can be a problem with my query itself.
151 00:24:52.030 ⇒ 00:24:57.559 Awaish Kumar: So, I just want to know your thoughts on how would you optimize that.
152 00:24:58.500 ⇒ 00:25:06.150 Jayneel Shah: Cool. So, like, as you mentioned, the first thing could be the query that we are writing is too long, there are.
153 00:25:21.650 ⇒ 00:25:22.969 Awaish Kumar: Hi, Kayla.
154 00:25:24.200 ⇒ 00:25:26.339 Awaish Kumar: I’m pronouncing your name correct.
155 00:25:26.930 ⇒ 00:25:29.180 Kaela Gallagher: Yeah, it looks like we lost him again.
156 00:25:29.950 ⇒ 00:25:33.909 Awaish Kumar: Yeah, I was just asking, like, am I pronouncing your name correct, or…
157 00:25:34.110 ⇒ 00:25:38.130 Kaela Gallagher: Kayla, yes, yep, you got it. And yours is a Wish?
158 00:25:38.490 ⇒ 00:25:39.200 Awaish Kumar: Yes, yes.
159 00:25:39.200 ⇒ 00:25:41.920 Kaela Gallagher: Okay, awesome, nice to meet you.
160 00:25:41.920 ⇒ 00:25:44.420 Awaish Kumar: I think we lost him.
161 00:25:44.910 ⇒ 00:25:47.370 Awaish Kumar: Let’s wait for a sec if he joins back.
162 00:25:47.730 ⇒ 00:25:50.670 Kaela Gallagher: Yeah, hopefully she should be able to hop back in.
163 00:25:57.230 ⇒ 00:26:01.840 Kaela Gallagher: Oh, it looks like you’re muted. I don’t know if you were trying to say something?
164 00:26:02.820 ⇒ 00:26:04.670 Awaish Kumar: I just muted myself.
165 00:26:04.910 ⇒ 00:26:06.490 Kaela Gallagher: Oh, okay, no worries.
166 00:27:43.200 ⇒ 00:27:44.929 Jayneel Shah: Am I audible?
167 00:27:47.250 ⇒ 00:27:48.480 Awaish Kumar: Yes, you are.
168 00:27:49.410 ⇒ 00:27:54.789 Jayneel Shah: I don’t know, there is some issue with the Wi-Fi, I’ve tried changing it, but not familiar.
169 00:27:54.790 ⇒ 00:28:01.580 Awaish Kumar: Okay, no worries, I think we can continue, like, I can just… Right, especially…
170 00:28:01.580 ⇒ 00:28:03.619 Jayneel Shah: Talking about the query, right?
171 00:28:04.050 ⇒ 00:28:07.100 Awaish Kumar: Yeah, we just have a table, which, we just optimize it, so…
172 00:28:07.100 ⇒ 00:28:21.080 Jayneel Shah: Okay, yeah, so, firstly, we’ll, as I was talking, that if we have a very long query that has many subqueries written into it, my first thought would be to break it down into smaller subqueries or CTEs.
173 00:28:21.080 ⇒ 00:28:28.069 Jayneel Shah: So that we can reuse them instead of using those same queries multiple times.
174 00:28:28.110 ⇒ 00:28:41.270 Jayneel Shah: If that does not solve the problem, the next thing, we can go to the tables. So, if we have a very big table, I’ll probably try to, get small chunks of it.
175 00:28:41.270 ⇒ 00:28:58.040 Jayneel Shah: So, if we have multiple chunks, so, something that we do in, big data analytics that we, like, do map and reduce, so I’ll break it down into, small queries, get those results, combine them, and get the output.
176 00:28:58.490 ⇒ 00:29:00.800 Awaish Kumar: Okay, how would you do that in a caddy?
177 00:29:01.290 ⇒ 00:29:01.910 Jayneel Shah: What?
178 00:29:02.210 ⇒ 00:29:05.940 Awaish Kumar: how will you… how would you do that in a skill carry?
179 00:29:06.280 ⇒ 00:29:08.269 Awaish Kumar: To divide the different…
180 00:29:09.370 ⇒ 00:29:11.630 Jayneel Shah: The data set…
181 00:29:11.630 ⇒ 00:29:13.410 Awaish Kumar: I’ll get it back, yeah.
182 00:29:13.690 ⇒ 00:29:17.140 Jayneel Shah: Yeah, so, like, probably,
183 00:29:18.240 ⇒ 00:29:36.440 Jayneel Shah: We can, like, divide the data, the… the table that we have, the table, we can divide the table itself into multiple, small tables, and, like, you know, not taking the entire table, but subparts of that particular table.
184 00:29:36.920 ⇒ 00:29:40.349 Jayneel Shah: and then combining the results using Union or something.
185 00:29:41.760 ⇒ 00:29:44.070 Awaish Kumar: Yeah, that’s my question. How would you divide the table?
186 00:29:46.140 ⇒ 00:29:55.709 Jayneel Shah: Okay, so, we’ll try to find, which of the, like, what is that one particular,
187 00:29:55.820 ⇒ 00:29:57.060 Jayneel Shah: recall that.
188 00:29:59.950 ⇒ 00:30:07.120 Jayneel Shah: one particular field that… on which we can divide the table on. So… What?
189 00:30:07.780 ⇒ 00:30:09.639 Awaish Kumar: Are you talking about primary key?
190 00:30:09.880 ⇒ 00:30:20.549 Jayneel Shah: Yeah. Yeah, like, yeah, maybe we can do that on primary key, that is the best way we can divide the table on. So, if we have the primary… yeah.
191 00:30:20.940 ⇒ 00:30:21.580 Jayneel Shah: yours.
192 00:30:21.580 ⇒ 00:30:23.559 Awaish Kumar: No, I’m just talking about,
193 00:30:23.880 ⇒ 00:30:29.039 Awaish Kumar: So is there… like, do you know anything about a partitioning strategy?
194 00:30:29.400 ⇒ 00:30:44.559 Jayneel Shah: Yes, but I have no refresher on that, so I’ll not be able to talk very freely on that. But, my idea would be to, of course, like, take the primary key if it is numbers.
195 00:30:44.870 ⇒ 00:30:50.499 Jayneel Shah: Maybe first 10, next 10, and then combine the results, so…
196 00:30:50.500 ⇒ 00:30:51.170 Awaish Kumar: Okay.
197 00:30:51.860 ⇒ 00:30:55.480 Awaish Kumar: I think…
198 00:30:56.090 ⇒ 00:31:03.739 Awaish Kumar: I think I have asked all the questions on my list. Do you have any questions for the Brainforge?
199 00:31:03.760 ⇒ 00:31:06.419 Jayneel Shah: Yeah, I do, like,
200 00:31:06.530 ⇒ 00:31:24.429 Jayneel Shah: what is that we do differently than the other consulting companies? So, is it only the end-to-end projects that we do, or is it, you know, they have the system set, but there is one particular part of it that we are tackling together?
201 00:31:25.060 ⇒ 00:31:31.470 Awaish Kumar: Yes, it is, Something like…
202 00:31:31.510 ⇒ 00:31:46.759 Awaish Kumar: you can say, both. Sometimes, you can have… we can have clients which are just, we have one project, and then we just do end-to-end, but normally, our, clients convert into being
203 00:31:46.760 ⇒ 00:31:56.409 Awaish Kumar: Getting a continuous data analytics services from us. So we have, like, a continuous engagement with them, where we are their data team.
204 00:31:56.590 ⇒ 00:31:59.290 Awaish Kumar: And basically, we continue to support whatever data
205 00:31:59.830 ⇒ 00:32:03.379 Awaish Kumar: And on the data analytics side, they need.
206 00:32:03.380 ⇒ 00:32:05.760 Jayneel Shah: engineering, analytics, or…
207 00:32:05.950 ⇒ 00:32:17.219 Awaish Kumar: reporting, whatever. So, it depends. It varies… goes… varies from client to client, but mostly we try to… we mostly have our clients which are, you know.
208 00:32:17.220 ⇒ 00:32:18.000 Jayneel Shah: Never line.
209 00:32:18.000 ⇒ 00:32:19.380 Awaish Kumar: Large engagements, yep.
210 00:32:19.380 ⇒ 00:32:22.009 Jayneel Shah: Oh, okay, sounds good. And,
211 00:32:22.130 ⇒ 00:32:27.179 Jayneel Shah: Like, what does your typical stack look like, or does it vary from…
212 00:32:27.180 ⇒ 00:32:33.990 Awaish Kumar: Yeah, it does vary from client to client, and we have a lot of say in designing those,
213 00:32:33.990 ⇒ 00:32:34.480 Jayneel Shah: Holden.
214 00:32:34.480 ⇒ 00:32:36.280 Awaish Kumar: mistakes, but…
215 00:32:36.330 ⇒ 00:32:54.899 Awaish Kumar: Sometimes, clients are in some contracts, which we then have to use them anyway, but mostly, if there is a flexibility, we try to come up with the best possible tech stack for the client’s use case, and then that’s what we are going to use.
216 00:32:55.580 ⇒ 00:32:56.190 Jayneel Shah: Yep.
217 00:32:56.570 ⇒ 00:32:59.509 Jayneel Shah: And the most popular ones, which we are using right now.
218 00:32:59.510 ⇒ 00:33:08.369 Awaish Kumar: Or, like, using, Polyatomic or Fivetrend as ETL tools, and then we have Snowflake, we use,
219 00:33:08.610 ⇒ 00:33:16.150 Awaish Kumar: Some… we are… we use AWS and GCP services for some of our clients, and then finally we…
220 00:33:16.310 ⇒ 00:33:21.009 Awaish Kumar: You… for BI, we may be using Tableau, or Omni, or…
221 00:33:21.440 ⇒ 00:33:21.830 Jayneel Shah: Lord.
222 00:33:21.830 ⇒ 00:33:26.050 Awaish Kumar: These are what, yeah, more popular for in our clients.
223 00:33:26.640 ⇒ 00:33:36.289 Jayneel Shah: Hmm, sounds good. Like, I have touched on Snowflake once, dashboarding, I probably have touched Tableau Power BI, and…
224 00:33:36.510 ⇒ 00:33:38.440 Jayneel Shah: Recently, Looker Studio.
225 00:33:38.810 ⇒ 00:33:45.969 Jayneel Shah: So, yeah, the stacks do align, so I’d be happy to… Maybe work in the future.
226 00:33:46.590 ⇒ 00:33:49.150 Jayneel Shah: I think that is all from my side.
227 00:33:50.320 ⇒ 00:33:55.149 Awaish Kumar: Okay, thank you, so much for your time. I think.
228 00:33:55.150 ⇒ 00:34:10.700 Jayneel Shah: Like, I do have one question which I forgot to ask, like, I am here, on an F1 visa, so, do you guys support, like, OPT for the purposes of…
229 00:34:11.590 ⇒ 00:34:20.729 Awaish Kumar: Yeah, I think, our recruiting and operations team will… can answer with those questions. I will let them know, and they are going to get back.
230 00:34:20.880 ⇒ 00:34:22.300 Awaish Kumar: How do you own that?
231 00:34:24.250 ⇒ 00:34:33.129 Awaish Kumar: But I think, we are at the end of the interview, so thank you for your time. And, yeah, Rico from our operations.
232 00:34:33.380 ⇒ 00:34:36.159 Awaish Kumar: Maybe, going to reach out to you.
233 00:34:36.400 ⇒ 00:34:37.300 Awaish Kumar: Oh, good.
234 00:34:37.300 ⇒ 00:34:37.630 Jayneel Shah: Excellent.
235 00:34:37.639 ⇒ 00:34:38.469 Awaish Kumar: Excuse me.
236 00:34:38.730 ⇒ 00:34:40.119 Jayneel Shah: Thank you so much. Have a good day.