Meeting Title: CTA DataOps PR Review Sync Date: 2026-02-13 Meeting participants: Ashwini Sharma, Kyle Wandel
WEBVTT
1 00:01:48.740 ⇒ 00:01:49.950 Ashwini Sharma: Hey, Kyle.
2 00:01:50.280 ⇒ 00:01:51.749 Kyle Wandel: Hey Srini, how you doing?
3 00:01:51.750 ⇒ 00:01:52.930 Ashwini Sharma: I’m good, how are you?
4 00:01:53.290 ⇒ 00:01:55.140 Kyle Wandel: Oh, good. Sorry I’m a little late.
5 00:01:55.350 ⇒ 00:01:56.270 Ashwini Sharma: No problem.
6 00:01:57.400 ⇒ 00:01:59.750 Ashwini Sharma: What time is it over the 9, right? This is 9.
7 00:01:59.750 ⇒ 00:02:02.820 Kyle Wandel: Oh, yeah, so… Yeah.
8 00:02:03.830 ⇒ 00:02:04.530 Kyle Wandel: But…
9 00:02:04.670 ⇒ 00:02:08.689 Kyle Wandel: It’s all good. Thank you for, meeting to join. It’s, nice to get this out of the way.
10 00:02:08.910 ⇒ 00:02:09.860 Ashwini Sharma: Oh, welcome.
11 00:02:10.410 ⇒ 00:02:13.300 Ashwini Sharma: Okay, so what do we want to do today?
12 00:02:13.950 ⇒ 00:02:22.619 Kyle Wandel: Yeah, so I think the biggest question is, so I have… I put that PR request in, and I just want to know, like, where… so…
13 00:02:23.420 ⇒ 00:02:25.979 Kyle Wandel: how does it go from, like, Wandel Dev…
14 00:02:26.140 ⇒ 00:02:39.320 Kyle Wandel: to ProdMarts? Is it… is it really just as committing that request and merging the PRs, or merging the branch with the main, and then it goes directly to ProdMarts, or do I have to do something else in terms of just, like.
15 00:02:39.630 ⇒ 00:02:41.940 Kyle Wandel: Updating the schema name or database name.
16 00:02:42.140 ⇒ 00:02:45.779 Ashwini Sharma: No, you don’t have to do anything, right? Once you put in a PR,
17 00:02:45.910 ⇒ 00:03:01.399 Ashwini Sharma: What I wanted to do was I wanted to add one more layer called SDG, if you’ve seen, SDG mods, right? And the purpose of that was to ensure people can do QA on that before we make changes in the production.
18 00:03:02.850 ⇒ 00:03:10.080 Ashwini Sharma: But right now, we just have one branch, and we are just going to merge it as it is, right?
19 00:03:10.460 ⇒ 00:03:18.159 Ashwini Sharma: But what I think, okay, hold on a second, let me open your GitHub. I think I’ve done something over there which I’m not able to recall right now.
20 00:03:19.020 ⇒ 00:03:30.030 Kyle Wandel: Yeah, I did, a bunch of cleaning and combining of, like, registration data from a history standpoint, from 2014 to 2016, and then…
21 00:03:30.050 ⇒ 00:03:39.580 Kyle Wandel: it looks like a Sweeney did something else, but on a different one. But… not a Sweeney, I can’t remember the other guy’s name. No, it was you, yeah.
22 00:03:39.580 ⇒ 00:03:40.500 Ashwini Sharma: Yeah,
23 00:03:40.500 ⇒ 00:03:48.380 Kyle Wandel: Yeah, but… Somebody closed it, or reviewed it, and then… and then accepted it.
24 00:03:48.380 ⇒ 00:03:51.260 Ashwini Sharma: I reviewed it, yeah, I reviewed your PR.
25 00:03:55.140 ⇒ 00:04:00.830 Kyle Wandel: And then once it gets reviewed, is it, like… Approved? Does it go to…
26 00:04:00.830 ⇒ 00:04:03.099 Ashwini Sharma: No, you have to merge it, right?
27 00:04:03.430 ⇒ 00:04:12.660 Ashwini Sharma: I see, your PR… Arthur is Kyle… yeah.
28 00:04:13.330 ⇒ 00:04:17.539 Kyle Wandel: Let me share my screen, right? I think that’ll be easier. Yeah. Yeah.
29 00:04:28.420 ⇒ 00:04:29.720 Ashwini Sharma: Yeah, can you see my screen?
30 00:04:30.720 ⇒ 00:04:31.290 Kyle Wandel: Dip.
31 00:04:31.800 ⇒ 00:04:33.739 Ashwini Sharma: Alright, so this was your PR, right?
32 00:04:34.140 ⇒ 00:04:34.800 Kyle Wandel: Yep.
33 00:04:35.600 ⇒ 00:04:39.250 Ashwini Sharma: Yeah, I reviewed it, and… who merged it?
34 00:04:39.720 ⇒ 00:04:44.709 Ashwini Sharma: Pull request… Okay, so you monstered yesterday, right?
35 00:04:44.710 ⇒ 00:04:45.160 Kyle Wandel: Yeah, I mean…
36 00:04:45.160 ⇒ 00:04:50.680 Ashwini Sharma: merged, right. So once it is merged, what happens is, like, if you look into this actions, right?
37 00:04:51.340 ⇒ 00:04:57.619 Ashwini Sharma: There’s this action, which runs, daily prod build, this one.
38 00:04:59.340 ⇒ 00:05:02.269 Ashwini Sharma: So, what this does is it’ll build it.
39 00:05:02.520 ⇒ 00:05:07.909 Ashwini Sharma: And this is going to run it against the prod mods. So, whatever changes you have pushed.
40 00:05:08.370 ⇒ 00:05:12.880 Ashwini Sharma: It will create those tables in the prod, databases.
41 00:05:14.140 ⇒ 00:05:25.640 Kyle Wandel: Okay. So it looks like it goes, like, on a… it’s, like, on a daily basis, basically. So, like, if I create something, let’s say, the day before… or, like, today, it won’t go to Prod Mart’s…
42 00:05:26.450 ⇒ 00:05:31.320 Kyle Wandel: or… and it gets committed today, like, merged. It won’t go to product march until the next day.
43 00:05:31.320 ⇒ 00:05:32.360 Ashwini Sharma: Yes, that’s right.
44 00:05:32.360 ⇒ 00:05:32.840 Kyle Wandel: Okay, cool.
45 00:05:32.840 ⇒ 00:05:43.249 Ashwini Sharma: We can change this. Yeah, I think Catherine mentioned some time back, I think she wanted to do it every 4 hours or so, because remember’s data changes every 4 hours.
46 00:05:43.350 ⇒ 00:05:48.449 Ashwini Sharma: So we can do that. We can change the cadence, right? Maybe if you look into this…
47 00:05:49.050 ⇒ 00:05:55.190 Ashwini Sharma: We have, in data ops, right, gitHub.
48 00:05:55.510 ⇒ 00:05:59.119 Ashwini Sharma: broad dbt build, so this is, like, once a daily.
49 00:05:59.940 ⇒ 00:06:05.149 Ashwini Sharma: Once daily, if we modify this… it can run… Every 4 hours.
50 00:06:05.150 ⇒ 00:06:05.720 Kyle Wandel: Whatever, yeah.
51 00:06:05.720 ⇒ 00:06:06.160 Ashwini Sharma: Yeah.
52 00:06:06.460 ⇒ 00:06:17.829 Kyle Wandel: That makes sense. Yeah, I don’t… I don’t… I mean, I think daily is okay right now. I mean, eventually, I think it will go to every 4 hours, I think, because that’s what Catherine wants. But it’s just good to know that it’s not, automatic, you know what I mean?
53 00:06:18.330 ⇒ 00:06:18.980 Ashwini Sharma: Yep.
54 00:06:19.440 ⇒ 00:06:20.710 Ashwini Sharma: Right, right.
55 00:06:21.730 ⇒ 00:06:26.560 Kyle Wandel: So then, I guess then, since it… since it went through the prod build.
56 00:06:27.300 ⇒ 00:06:29.170 Kyle Wandel: This mor- er, last night.
57 00:06:29.520 ⇒ 00:06:36.630 Kyle Wandel: if I go to, like, Snowflake, and then try to go to prod underscore Martz, and then try and find…
58 00:06:37.490 ⇒ 00:06:41.350 Kyle Wandel: the stuff I did, it should be there, right? Because it’s…
59 00:06:41.350 ⇒ 00:06:43.620 Ashwini Sharma: Yes, it should be there.
60 00:06:43.930 ⇒ 00:06:46.309 Kyle Wandel: Okay, and if I’m not seeing it…
61 00:06:47.610 ⇒ 00:06:50.240 Kyle Wandel: That’s something that… that’s another issue.
62 00:06:50.800 ⇒ 00:06:52.989 Ashwini Sharma: We can look away while you’re not able to see it.
63 00:06:52.990 ⇒ 00:06:57.379 Kyle Wandel: Oh, no, no, okay, so it went to default. Okay, so I found it. It just went to default. Okay.
64 00:06:57.380 ⇒ 00:07:05.910 Ashwini Sharma: Okay, yeah, I’ve changed that in a different PR. I didn’t realize that some of the data was going into default. Let me log into Okta.
65 00:07:06.390 ⇒ 00:07:12.589 Ashwini Sharma: And I have changed it in a different PR, so now things will move into, what do you call, correct schema.
66 00:07:13.050 ⇒ 00:07:13.820 Kyle Wandel: Okay.
67 00:07:14.200 ⇒ 00:07:29.059 Kyle Wandel: Cool, yeah, because I think it… the way… I think… I hoped, at least the way I set it up, it would have created the new schema CES registration, and then from there, I think it would have put it in there, but if it makes… if it just went to default first, that’s okay.
68 00:07:29.460 ⇒ 00:07:32.720 Kyle Wandel: But it’s nice that it updated, so that’s good to know.
69 00:07:32.720 ⇒ 00:07:38.760 Ashwini Sharma: Yeah, I’ll show you what was missing, right? Because, let me… let me go back to pull request.
70 00:07:39.420 ⇒ 00:07:40.870 Ashwini Sharma: Yes.
71 00:07:41.050 ⇒ 00:07:45.000 Ashwini Sharma: So if you see this, right, some of the…
72 00:07:45.170 ⇒ 00:07:49.769 Ashwini Sharma: Now, staging layer tables were going to default schema, right?
73 00:07:50.010 ⇒ 00:07:53.420 Ashwini Sharma: And… In dbtproject.yamo?
74 00:07:53.660 ⇒ 00:07:56.660 Ashwini Sharma: See, there’s this setting, right?
75 00:08:01.020 ⇒ 00:08:08.530 Ashwini Sharma: So here, I’m saying that this is the staging layer, and this is the remembers, so everything related to remembers is under this section.
76 00:08:08.750 ⇒ 00:08:16.919 Ashwini Sharma: Yeah. And then I’ve stated an explicit schema, saying that, okay, this is the schema under which it should go, all the accounting-related stuff should go into.
77 00:08:17.070 ⇒ 00:08:18.020 Ashwini Sharma: the schema.
78 00:08:18.270 ⇒ 00:08:27.110 Ashwini Sharma: And I hadn’t done that for this too, CS registration and member engagement, right? So now that I have added it, it will go into these schemas, the staging.
79 00:08:27.110 ⇒ 00:08:27.430 Kyle Wandel: Okay.
80 00:08:31.050 ⇒ 00:08:32.150 Kyle Wandel: And then…
81 00:08:32.150 ⇒ 00:08:32.899 Ashwini Sharma: Once this is a month.
82 00:08:32.900 ⇒ 00:08:40.390 Kyle Wandel: What about Prod March? Because I think it actually created it in the right staging, so then for Prod March, we’ll have to go in and do the similar thing?
83 00:08:40.390 ⇒ 00:08:43.450 Ashwini Sharma: No, prod is already in the correct directory, right?
84 00:08:43.450 ⇒ 00:08:43.900 Kyle Wandel: Okay.
85 00:08:43.900 ⇒ 00:08:47.689 Ashwini Sharma: Go to broad the project,
86 00:08:50.020 ⇒ 00:08:53.909 Ashwini Sharma: Smart. Smart, it’s, it’s just,
87 00:08:55.070 ⇒ 00:09:04.939 Ashwini Sharma: Okay, there is a CRM, there is a CAS, so they should just be created in their own directories. Let’s look at this one. Probably, I have done it only for CRM,
88 00:09:05.490 ⇒ 00:09:09.950 Ashwini Sharma: Let’s, let’s see… dbt Project Retina…
89 00:09:11.040 ⇒ 00:09:15.309 Ashwini Sharma: mods, right? So CRM and reports are going in their own schemas, right?
90 00:09:15.310 ⇒ 00:09:15.910 Kyle Wandel: Yep.
91 00:09:16.210 ⇒ 00:09:22.190 Ashwini Sharma: do we have anything? Like, for example, CES, CES is again going into its own schema, right?
92 00:09:23.430 ⇒ 00:09:26.810 Ashwini Sharma: Engagement. Engagement, there is nothing in the engagement.
93 00:09:27.100 ⇒ 00:09:31.150 Ashwini Sharma: But, this file, RPTCS registration history, this is.
94 00:09:31.150 ⇒ 00:09:31.840 Kyle Wandel: Oh, dude.
95 00:09:31.840 ⇒ 00:09:32.650 Ashwini Sharma: default.
96 00:09:33.190 ⇒ 00:09:41.009 Kyle Wandel: Yeah, no, that makes sense. Oh, so yeah, so then how do I change it so that it creates that CES schema, or is it just always going to go to default?
97 00:09:41.340 ⇒ 00:09:44.419 Ashwini Sharma: No, we can do that right now, like.
98 00:09:50.610 ⇒ 00:09:57.250 Kyle Wandel: And I’m starting to… I’m starting to learn all of the… the commands and stuff, so, it’s been nice playing around and getting used to it, so…
99 00:09:57.450 ⇒ 00:09:59.940 Ashwini Sharma: Yeah, yeah, true. It’ll take some time, but then it’s…
100 00:09:59.940 ⇒ 00:10:00.340 Kyle Wandel: Yes.
101 00:10:00.340 ⇒ 00:10:02.779 Ashwini Sharma: straightforward, right? Once you know it all, yeah.
102 00:10:02.780 ⇒ 00:10:03.450 Kyle Wandel: Yeah.
103 00:10:08.150 ⇒ 00:10:09.320 Ashwini Sharma: Policy, yes.
104 00:10:09.320 ⇒ 00:10:19.069 Kyle Wandel: Yeah, I’ve literally never used GitHub before, this year, so it’s been nice, and I have to do it for a school project as well, so it’s been nice to try and, like, figure it out, understand it more.
105 00:10:19.360 ⇒ 00:10:24.439 Ashwini Sharma: Alright, yeah. So, what version control were you using before this one?
106 00:10:25.040 ⇒ 00:10:48.660 Kyle Wandel: So… not anything. So, from, from a data… I just joined the DataOps team, like, right before this year, so, and before that, I’ve just never used GitHub from a version control standpoint. We’ve all… we’ve almost always… I’ve almost always played in a database that is prod, basically, or if it’s… if I’m creating and transforming stuff, we just treat it as prod. We’ve never done, really, version control or anything like that for my…
107 00:10:48.660 ⇒ 00:10:54.349 Kyle Wandel: From my previous history, so it’s really been… it’s actually really nice to be able to see it, basically, and kind of do it.
108 00:10:54.790 ⇒ 00:11:00.270 Ashwini Sharma: Right, right. So now, another, you know, good thing about dbt is, like, for example, let’s say
109 00:11:00.540 ⇒ 00:11:06.490 Ashwini Sharma: We changed this one, right? I changed some of the stuff in the staging layer, in the staging table.
110 00:11:07.020 ⇒ 00:11:14.839 Ashwini Sharma: So, initially, it was this one. I added some more stuff that I showed you, right? So, what that means is, some of the models that we have in
111 00:11:15.100 ⇒ 00:11:19.620 Ashwini Sharma: The staging layer moves from one schema to another schema, right?
112 00:11:20.190 ⇒ 00:11:23.780 Ashwini Sharma: So earlier it was in default, now it’ll be under its own schema.
113 00:11:23.920 ⇒ 00:11:29.599 Ashwini Sharma: But then, you are referring to these models from a higher layer, upstream models, right?
114 00:11:30.090 ⇒ 00:11:35.280 Ashwini Sharma: So, as long as you are using the syntax,
115 00:11:36.640 ⇒ 00:11:39.860 Ashwini Sharma: syntax like this, right? You’re referring to a model name.
116 00:11:40.470 ⇒ 00:11:41.010 Kyle Wandel: Yep.
117 00:11:41.010 ⇒ 00:11:45.300 Ashwini Sharma: You can move those models around in different schemas, and it will always
118 00:11:45.550 ⇒ 00:11:52.310 Ashwini Sharma: you know, resolved to the correct model. When it compiles, it will change it, change the schema accordingly.
119 00:11:53.950 ⇒ 00:11:57.910 Kyle Wandel: Perfect, and that’s based on the macros that you created earlier, right? The…
120 00:11:58.130 ⇒ 00:12:02.059 Ashwini Sharma: Yeah, some of the macros, but it’s the Jinja syntax, right? That will.
121 00:12:02.060 ⇒ 00:12:02.740 Kyle Wandel: Oh, good credit.
122 00:12:02.740 ⇒ 00:12:09.910 Ashwini Sharma: to, and, I just made the schema name as CES, is that okay, right? Okay.
123 00:12:09.910 ⇒ 00:12:21.330 Kyle Wandel: Yeah, I think so, because we’ll probably just do generic CS reports in that, and then probably have to have different staging levels for, like, attendees, exhibitor booths, and different stuff like that.
124 00:12:21.490 ⇒ 00:12:24.150 Ashwini Sharma: Cool. Alright, I’ll just push it then.
125 00:12:25.060 ⇒ 00:12:25.840 Kyle Wandel: Perfect.
126 00:12:30.240 ⇒ 00:12:31.470 Ashwini Sharma: I didn’t save it.
127 00:12:35.650 ⇒ 00:12:36.770 Ashwini Sharma: Oh, come on.
128 00:12:38.050 ⇒ 00:12:39.550 Ashwini Sharma: Did I already comment?
129 00:12:45.360 ⇒ 00:12:50.300 Ashwini Sharma: Oh, I was in… Yeah, I have multiple projects open, that’s, yeah.
130 00:12:50.690 ⇒ 00:12:54.530 Ashwini Sharma: I’m so keen…
131 00:12:56.740 ⇒ 00:13:03.719 Kyle Wandel: Yeah, it must be interesting, because I’m… do you work on multiple, like, with multiple clients, or do you use it… is it really just, like, one client at a time?
132 00:13:03.900 ⇒ 00:13:13.099 Ashwini Sharma: So at a time, it’s one client, but the time is split, right? For example, like, I work for 4 hours for CTA, and then another 2 hours for some other client.
133 00:13:13.180 ⇒ 00:13:18.499 Kyle Wandel: Okay, so yeah, you are doing multiple projects and clients per day, wow, that’s pretty crazy.
134 00:13:20.810 ⇒ 00:13:31.830 Ashwini Sharma: I mean, like, if there is enough volume of work for CTA, then I can be totally focused, but then sometimes, you know, I mean, it’s not such a huge volume, so I have to split it.
135 00:13:32.170 ⇒ 00:13:44.230 Kyle Wandel: Yeah, that makes sense. Yeah, thanks for us taking an initial look at the identity stitching methods. That really has helped out a lot. That’s really cool that you guys were able to find a lot of, matches.
136 00:13:45.170 ⇒ 00:13:49.429 Ashwini Sharma: Yeah, yeah, yeah, Avish added on the initial work that I had done for the
137 00:13:50.040 ⇒ 00:13:59.729 Ashwini Sharma: I think we can add some more stuff over there. There was one particular attribute which I saw missing, I’ll just let them know, Aesha and Utam, and then he can add
138 00:13:59.940 ⇒ 00:14:08.799 Ashwini Sharma: What was that, Let me see if I can… It’s a Samsung, right?
139 00:14:09.590 ⇒ 00:14:10.090 Kyle Wandel: Yeah.
140 00:14:10.090 ⇒ 00:14:11.520 Ashwini Sharma: Samsung as well.
141 00:14:11.970 ⇒ 00:14:12.560 Ashwini Sharma: 3rd?
142 00:14:12.560 ⇒ 00:14:19.169 Kyle Wandel: I mean, they’re the best… they’re the best test case, for sure. If you can figure them out, then, you figure everything out.
143 00:14:20.290 ⇒ 00:14:22.209 Ashwini Sharma: Some electronics.
144 00:14:25.230 ⇒ 00:14:26.690 Ashwini Sharma: What is Samson?
145 00:14:26.690 ⇒ 00:14:28.299 Kyle Wandel: I think it’s worth…
146 00:14:28.300 ⇒ 00:14:29.989 Ashwini Sharma: Radiolet is this one, yeah.
147 00:14:29.990 ⇒ 00:14:30.580 Kyle Wandel: Yep.
148 00:14:30.770 ⇒ 00:14:33.600 Kyle Wandel: Yeah, that’s the actual Samsung ID.
149 00:14:34.430 ⇒ 00:14:42.729 Ashwini Sharma: Right, yeah. And, and this has, what was it? This one. This, this, this attribute was not used anywhere.
150 00:14:42.850 ⇒ 00:14:47.680 Ashwini Sharma: for identity resolution, right? So I’ll just let Avesh know that we could utilize this one.
151 00:14:48.070 ⇒ 00:14:51.330 Ashwini Sharma: And maybe some more stuff can be resolved.
152 00:14:52.910 ⇒ 00:15:02.769 Kyle Wandel: Yeah, that would be… that should… that should match a lot, as well. Especially with the exhibitor booth, stuff. Yeah, I see that.
153 00:15:05.630 ⇒ 00:15:06.410 Kyle Wandel: Yeah.
154 00:15:06.440 ⇒ 00:15:25.729 Kyle Wandel: Yeah, the more you guys can match, the toughest thing would be matching an individual, and, like, tying that to the correct organization at the right time. But, I mean, this is a good start. I made a comment in terms of, like, how to match some of those individuals, basically, or getting started on that for the registration, but it’s gonna take some time to hammer that out as well.
155 00:15:25.980 ⇒ 00:15:26.560 Ashwini Sharma: Yeah.
156 00:15:27.390 ⇒ 00:15:30.370 Ashwini Sharma: Oh, so let me just push this thing,
157 00:15:30.370 ⇒ 00:15:30.890 Kyle Wandel: Yeah.
158 00:15:30.890 ⇒ 00:15:33.110 Ashwini Sharma: It does, yeah, yeah.
159 00:15:50.510 ⇒ 00:15:51.520 Ashwini Sharma: Oh…
160 00:16:10.410 ⇒ 00:16:11.420 Ashwini Sharma: Okay.
161 00:16:11.880 ⇒ 00:16:12.340 Kyle Wandel: Perfect.
162 00:16:12.340 ⇒ 00:16:12.890 Ashwini Sharma: Huh?
163 00:16:18.500 ⇒ 00:16:28.510 Ashwini Sharma: So, yeah, I mean, what I was thinking was, you know, how do we plan to do QA, right? As the team grows, and as you do a lot of
164 00:16:28.660 ⇒ 00:16:30.160 Ashwini Sharma: Data work, right?
165 00:16:30.940 ⇒ 00:16:35.809 Ashwini Sharma: I mean, what’s your thought, right? Let me hear your thought, and then I can…
166 00:16:36.090 ⇒ 00:16:47.179 Kyle Wandel: Yeah, that’s a good question. I mean, like, I’ll be the lead engineer, and so, like, I probably know, and I probably have the best institutional knowledge, and I know SQL pretty well. For the next foreseeable future.
167 00:16:47.350 ⇒ 00:17:02.699 Kyle Wandel: like, it might just be me. We may… we’re trying to add one more person. But for now, I mean, like, if you guys have anything that needs reviewed, like, you can send it either to me as well. I think that… I want to be able to start, like, commenting on,
168 00:17:02.860 ⇒ 00:17:24.500 Kyle Wandel: kind of, like, where to start and, like, kind of the changes to make. So, like, the Shopify downloads, I think I was gonna comment on making some changes, but, the changes that were made were perfect, so I took a look at that. I’ve taken a look at some of the other PR requests, so if you want me to start reviewing stuff, I would happily to and can commit the changes. My biggest thing was I just wanted to make sure that
169 00:17:24.500 ⇒ 00:17:37.919 Kyle Wandel: When I did commit a change that it did go to the correct schema, or create the correct schema in prod, basically. And so now that I’ve seen it done, and it’s one of those things where I’m a visual learner, so I know I can trust it more.
170 00:17:38.090 ⇒ 00:17:43.520 Kyle Wandel: So, I mean, like, I’m happy to do whatever you need based, but I think, to answer your question.
171 00:17:43.690 ⇒ 00:17:48.269 Kyle Wandel: I think when everything is up and running, when we no longer…
172 00:17:48.960 ⇒ 00:18:00.519 Kyle Wandel: and we have another engineer other than myself, like, they’ll do work, I’ll do work, and then, I’ll QA that person’s, and then they can maybe QA my per- my work, and then we’ll just commit it. But for now, I mean.
173 00:18:00.520 ⇒ 00:18:06.459 Ashwini Sharma: Oh, no, no, I think, I mean, what you’re talking about is sort of a peer review, right?
174 00:18:06.460 ⇒ 00:18:07.239 Kyle Wandel: Yeah, yeah.
175 00:18:07.240 ⇒ 00:18:10.579 Ashwini Sharma: That’s prior to the actual QA, so…
176 00:18:10.840 ⇒ 00:18:16.929 Ashwini Sharma: Okay, my bad, I think I wrong. What I was talking about is UAT, right? So, for example, like, let’s say
177 00:18:17.160 ⇒ 00:18:24.770 Ashwini Sharma: the… a business team, right? For example, finance or somebody, right? They ask for some models.
178 00:18:25.800 ⇒ 00:18:41.490 Ashwini Sharma: And, in order to solve their request, we create certain models. We are, you know, additional engineer, and you create certain models, right? And then you push it to production. Initially, everything is good, but after some time, they ask for certain changes, right?
179 00:18:41.700 ⇒ 00:18:46.430 Ashwini Sharma: And you make the changes, and then you give a… them.
180 00:18:46.600 ⇒ 00:18:51.670 Ashwini Sharma: Now… I mean, it’s not… Like, they could always say, you know, this is not correct.
181 00:18:55.210 ⇒ 00:18:57.900 Ashwini Sharma: Like, who’s gonna, like, manage that part in terms of, like, the business line?
182 00:18:57.900 ⇒ 00:19:13.120 Kyle Wandel: Yeah, I think Kai is going to be the main person that, like, interacts with the other departments, basically, in terms of understanding their needs and the requirements, and then she’ll hand it off to the engineer team to update that, basically.
183 00:19:13.120 ⇒ 00:19:29.430 Ashwini Sharma: No, no, no, I think I’m not addressing it correctly, right? So, let’s say business team is there, they are relying on certain dashboards, right? They make business decisions on top of that dashboard. And, you know, everything is going on smoothly. Now, they do not want to impact
184 00:19:29.590 ⇒ 00:19:34.070 Ashwini Sharma: That dashboard, because of some change that developers are pushing it, right?
185 00:19:34.600 ⇒ 00:19:49.499 Ashwini Sharma: So, some kind of a mechanism to isolate certain changes, show it to them on a different report, and once they say, yes, this is what we need, this looks good, then you make that change on the production tables, right?
186 00:19:50.330 ⇒ 00:19:59.360 Kyle Wandel: Interesting. Okay, no, we don’t have a process for that. I think that’s something that, hasn’t necessarily been thought of. Would you suggest doing, like, a…
187 00:19:59.910 ⇒ 00:20:14.240 Kyle Wandel: So, like, the two ways that I would think about doing it would be, one, it would be letting them have access to… no, I wouldn’t want them access to staging. So, I guess the only way I would do it would be, like, another version. I mean, but if there’s another ver… if there’s another way you have, like, I’m open to suggestions.
188 00:20:14.240 ⇒ 00:20:21.540 Ashwini Sharma: Yeah, so that’s why I had created it this way, right? See this thing… Look into data…
189 00:20:21.540 ⇒ 00:20:25.099 Kyle Wandel: It affects, like, the rule permissions that we gotta update as well.
190 00:20:25.270 ⇒ 00:20:32.960 Ashwini Sharma: Alright, so, you see this one, right? I created, 3 layers of this thing, right? One is the…
191 00:20:33.080 ⇒ 00:20:39.219 Ashwini Sharma: Broadmarth, right, which contains all the… all the tables from where dashboards will be built, right?
192 00:20:39.600 ⇒ 00:20:48.310 Ashwini Sharma: And then there is, SDG mods, right? This is… this is the layer that I intended for QA.
193 00:20:48.790 ⇒ 00:20:49.600 Kyle Wandel: Okay.
194 00:20:49.600 ⇒ 00:20:53.440 Ashwini Sharma: Right? And then there is, dev… dev mods, right?
195 00:20:53.600 ⇒ 00:20:59.900 Ashwini Sharma: This is DevMars, this is, you know, developers fooling around, with things, right? So…
196 00:21:00.030 ⇒ 00:21:09.029 Ashwini Sharma: That was my initial thought when I created these three layers, but I mean, we could always change it, right? Probably, you may never need this SDG.
197 00:21:09.390 ⇒ 00:21:11.650 Ashwini Sharma: SDG deals, right?
198 00:21:11.650 ⇒ 00:21:35.199 Kyle Wandel: Well, and I think one thing that I think Catherine was interested in, and I… maybe we can use… I think that we can use STGMarts for this, is the ability for the teams to build their own, like, reports, basically. And so maybe they can build the reports using their own SQL code, and not necessarily, like, using from a dbt standpoint, but just create it in SQL, like, create the table or create the review or something.
199 00:21:35.200 ⇒ 00:21:37.250 Kyle Wandel: And then we would go in and take that.
200 00:21:37.980 ⇒ 00:21:49.159 Kyle Wandel: and then create the actual model in ProdMarts. Maybe that’s one way we can do it as well. I mean, I think that… I get what you’re saying. I think, yeah, I think using SCG Marts and giving
201 00:21:49.480 ⇒ 00:21:57.690 Kyle Wandel: read access, or even maybe even write access to, certain users would help us understand and QA some of the reports.
202 00:21:58.230 ⇒ 00:21:59.190 Ashwini Sharma: Cool, yeah.
203 00:21:59.520 ⇒ 00:22:00.919 Ashwini Sharma: That could be done, okay.
204 00:22:01.700 ⇒ 00:22:04.010 Ashwini Sharma: Alright, yeah.
205 00:22:04.370 ⇒ 00:22:08.289 Ashwini Sharma: Cool. Any, any, any other stuff that you wanted to go through?
206 00:22:09.450 ⇒ 00:22:17.190 Kyle Wandel: No, I mean, I think that’s it. I think the… I guess the only other thing, and I’ll… and I was gonna bring this up at the, meeting at 10.30, but, I think…
207 00:22:17.190 ⇒ 00:22:35.740 Kyle Wandel: the biggest thing that we want to do is tie everything to DIM organizations instead of, like, the active report, active member report, because the active member report, it should be derived based on who is active members, but the DIM organizations level is probably the best level to tie to, and then build out from there.
208 00:22:36.050 ⇒ 00:22:37.140 Ashwini Sharma: Okay, cool.
209 00:22:37.650 ⇒ 00:22:42.070 Ashwini Sharma: Yeah, active member report derives out of DIM organization only, right?
210 00:22:42.070 ⇒ 00:22:50.710 Kyle Wandel: Yeah, yeah, it does. But so, like, what I think that we should do is have dim organization as, like, the center table, and then everything builds out from there.
211 00:22:50.710 ⇒ 00:22:51.390 Ashwini Sharma: Oh, okay.
212 00:22:51.580 ⇒ 00:22:53.439 Ashwini Sharma: Got it, yeah, that makes sense.
213 00:22:54.320 ⇒ 00:22:58.290 Ashwini Sharma: It’s always organization, it’s always involved, right?
214 00:22:58.290 ⇒ 00:22:58.870 Kyle Wandel: Yeah, pretty much.
215 00:22:58.870 ⇒ 00:22:59.300 Ashwini Sharma: That’s true.
216 00:22:59.300 ⇒ 00:23:11.940 Kyle Wandel: Pretty much. And even… even with the CS registration data, like, that’s tied to the individual, obviously, but, like, we’re not gonna have 99… like, 90% of those individuals in the,
217 00:23:12.200 ⇒ 00:23:13.440 Kyle Wandel: remembers data.
218 00:23:13.970 ⇒ 00:23:15.050 Ashwini Sharma: Alright, yeah.
219 00:23:15.320 ⇒ 00:23:24.780 Ashwini Sharma: Yeah, maybe, yeah, if we put more attributes to dim… dim this thing, organization, I think that is going to solve a lot of problems, a lot of reports, right? We don’t have to…
220 00:23:26.290 ⇒ 00:23:32.410 Ashwini Sharma: Maybe the active members report, right? Let’s take a look at this. This has the organization name, organization ID,
221 00:23:32.590 ⇒ 00:23:39.169 Ashwini Sharma: And then it has a membership name, which kind of links it to who is the primary member, primary contact.
222 00:23:39.560 ⇒ 00:23:40.440 Ashwini Sharma: Yeah.
223 00:23:42.080 ⇒ 00:23:58.279 Kyle Wandel: Yeah, but, like, for example, when tying up, what was I gonna say? The primary account information, or, like, the account manager information, I think it’d be interesting for membership to be able to filter to even non-members, and then see if there was ever an account manager previously assigned to that person, and then…
224 00:23:58.670 ⇒ 00:24:11.320 Kyle Wandel: either assign… re-sign that person, or whatever. I thought that just might be just an interesting little tidbit, because the… the member engagement kind of report won’t necessarily, in my head, be just members. It is…
225 00:24:11.370 ⇒ 00:24:19.879 Kyle Wandel: all organizations, and then you tell them if they’re members or not, or have they been involved previously, or XYZ.
226 00:24:21.610 ⇒ 00:24:22.250 Ashwini Sharma: Cool.
227 00:24:22.780 ⇒ 00:24:23.540 Ashwini Sharma: Yeah.
228 00:24:24.500 ⇒ 00:24:26.060 Ashwini Sharma: Alright.
229 00:24:26.560 ⇒ 00:24:43.280 Kyle Wandel: No, but this has been really helpful, and I really appreciate it, and it helps me to start, like, being more confident and, like, knowing that what I am building in WandelDev will go to ProdMarts eventually, and I don’t have to make any changes. It’s literally just building the DBT on my logo, pushing that to remote, and then it’ll be good to go.
230 00:24:43.560 ⇒ 00:24:45.770 Ashwini Sharma: Yeah, it does, it does.
231 00:24:46.070 ⇒ 00:24:50.590 Kyle Wandel: Yeah, that’s awesome. Thank you guys for so much for setting that up, and I know it’s been a lot of work, so…
232 00:24:50.590 ⇒ 00:24:53.190 Ashwini Sharma: Quick question, if you have 5 more minutes, right?
233 00:24:53.190 ⇒ 00:24:53.799 Kyle Wandel: Yeah, definitely.
234 00:24:54.820 ⇒ 00:25:01.239 Ashwini Sharma: I just wanted to, ask you about this thing. Yeah, about this particular ticket, right?
235 00:25:01.870 ⇒ 00:25:05.960 Ashwini Sharma: Can you provide a little bit more context on this one?
236 00:25:09.700 ⇒ 00:25:13.060 Ashwini Sharma: This is, this is ticket number…
237 00:25:13.840 ⇒ 00:25:27.689 Kyle Wandel: Yeah, yeah, oh, I think, actually, so I think we can close this one, because I… I’m pretty sure we’re good, because I looked at the… so it was on our end, we wanted to make sure that the, the Postgres DB, or…
238 00:25:28.400 ⇒ 00:25:36.540 Kyle Wandel: database was up-to-date with the correct logic. She wanted to include scan, like, success or fail, basically.
239 00:25:36.700 ⇒ 00:25:53.749 Kyle Wandel: and wanted to make sure that everything was included in that, and then when I took a look at, UTOM’s session code, it was okay. The one thing I don’t understand is what she’s talking about, the engagement session code mapping. Yeah, yeah, I get what you’re saying. Yeah, we don’t need that as a standalone prod Mart.
240 00:25:53.750 ⇒ 00:25:54.830 Ashwini Sharma: Just…
241 00:25:56.270 ⇒ 00:25:59.310 Kyle Wandel: Just the scan should be fine for the report.
242 00:25:59.480 ⇒ 00:26:05.650 Kyle Wandel: But use the mapping code and the listing code to create that final report, like… like you, Tom, did.
243 00:26:08.590 ⇒ 00:26:12.840 Ashwini Sharma: Utam has already created that report, I’m not aware of that.
244 00:26:12.840 ⇒ 00:26:13.769 Kyle Wandel: I believe so.
245 00:26:14.490 ⇒ 00:26:16.210 Kyle Wandel: session the…
246 00:26:16.870 ⇒ 00:26:18.540 Ashwini Sharma: This one, bad scans?
247 00:26:18.540 ⇒ 00:26:24.230 Kyle Wandel: Yeah, yeah, no, but the engagement report session scans at the bottom, that should be, I’m pretty sure it is.
248 00:26:24.550 ⇒ 00:26:33.279 Ashwini Sharma: Okay, this is STG, session scans, yeah, directly from session scans. So what we don’t need is this one, session code mapping and session listings, right?
249 00:26:33.500 ⇒ 00:26:50.750 Kyle Wandel: Yeah, you can… we can remove those, for sure. And then, like I said, I think… I’m pretty sure… I took a look at the SQL that UTOM created for the staging, or whatever, session scans, and it looks good. So, it looked like it was correct, and then, it made all the changes that we needed to. It included the columns that, we…
250 00:26:50.820 ⇒ 00:26:59.160 Kyle Wandel: that we want, including the ones that Catherine was, saying. Like, maybe the only… yeah, scan status, yep. So, I think… I think everything is good from that standpoint.
251 00:26:59.560 ⇒ 00:27:00.660 Ashwini Sharma: Okay, cool.
252 00:27:00.890 ⇒ 00:27:05.600 Ashwini Sharma: Alright, I’m going to remove those two models, and then delete the…
253 00:27:05.930 ⇒ 00:27:09.940 Ashwini Sharma: drop the table from Broadmart as well, so… Yeah.
254 00:27:10.480 ⇒ 00:27:13.529 Ashwini Sharma: Once the PR is merged, it won’t be created again. Alright.
255 00:27:14.280 ⇒ 00:27:15.820 Ashwini Sharma: Cool, thank you.
256 00:27:16.330 ⇒ 00:27:17.670 Kyle Wandel: Yeah, you’re welcome.
257 00:27:17.670 ⇒ 00:27:20.230 Ashwini Sharma: Let me know if you would need anything, right? Yeah.
258 00:27:20.230 ⇒ 00:27:26.520 Kyle Wandel: Yeah, definitely, thank you, and I really appreciate the time to kind of show me that, walk through me that final GitHub process, and that’s really… this is really helpful, so…
259 00:27:27.110 ⇒ 00:27:27.880 Ashwini Sharma: Cool.
260 00:27:28.490 ⇒ 00:27:29.140 Ashwini Sharma: All right.
261 00:27:29.140 ⇒ 00:27:31.740 Kyle Wandel: Alright, thank you, Sweeney. I’ll see you in a little bit.
262 00:27:31.740 ⇒ 00:27:32.310 Ashwini Sharma: Yep.
263 00:27:32.430 ⇒ 00:27:33.350 Ashwini Sharma: Bye.