Meeting Title: Shopify Data Integration Sync Date: 2026-02-03 Meeting participants: Samuel Roberts, Pranav Narahari


WEBVTT

1 00:00:48.300 00:00:49.790 Pranav Narahari: Hey, can you hear me?

2 00:00:49.790 00:00:51.080 Samuel Roberts: Yeah, can you hear me?

3 00:00:51.490 00:00:52.230 Pranav Narahari: Yep.

4 00:00:52.710 00:00:57.970 Samuel Roberts: Cool one, now the mic works here, it didn’t work before, whatever, okay. I don’t try to figure this out anymore.

5 00:00:59.400 00:01:03.299 Pranav Narahari: Okay, let me hop right into this. Yeah.

6 00:01:15.800 00:01:18.330 Pranav Narahari: One second… okay, here we are.

7 00:01:19.820 00:01:21.899 Pranav Narahari: Okay, you can see Shopify dashboard?

8 00:01:23.030 00:01:23.940 Samuel Roberts: Yes.

9 00:01:23.940 00:01:25.030 Pranav Narahari: Okay, perfect.

10 00:01:25.030 00:01:26.310 Samuel Roberts: Yes. Yes.

11 00:01:26.490 00:01:27.250 Samuel Roberts: So…

12 00:01:27.250 00:01:29.030 Pranav Narahari: Basically.

13 00:01:29.120 00:01:30.929 Samuel Roberts: the first thing that I’m trying to get.

14 00:01:30.930 00:01:34.150 Pranav Narahari: is, gross sales.

15 00:01:35.630 00:01:43.660 Pranav Narahari: And so, yeah, starting off, for whatever reason, like, because I feel like gross sales is the… is the value that…

16 00:01:44.630 00:01:48.240 Pranav Narahari: is, like, pre-calculations, you know? Like, net sales is the final thing that we want.

17 00:01:48.240 00:01:48.730 Samuel Roberts: again.

18 00:01:48.730 00:01:52.980 Pranav Narahari: But if we can get gross sales and then we can get discounts, then net sales.

19 00:01:52.980 00:01:53.880 Samuel Roberts: Right.

20 00:01:53.880 00:01:55.769 Pranav Narahari: Easy arithmetic, right?

21 00:01:55.770 00:01:56.850 Samuel Roberts: Yeah, total.

22 00:01:57.250 00:01:59.789 Pranav Narahari: That’s the idea there.

23 00:02:00.130 00:02:07.220 Pranav Narahari: now… now, that part is easy to explain, I think it’s easy to understand. How we’re going about doing that is…

24 00:02:07.570 00:02:10.779 Pranav Narahari: like this. So, you can see Mother Duck now, right?

25 00:02:11.050 00:02:11.700 Samuel Roberts: Yep.

26 00:02:11.890 00:02:14.249 Pranav Narahari: Okay, perfect. So…

27 00:02:14.850 00:02:22.910 Pranav Narahari: these queries don’t work right now, because Polytomic is, you know, doesn’t have any data in it anymore, but how we’re going about…

28 00:02:23.630 00:02:26.860 Pranav Narahari: One… actually, okay.

29 00:02:27.090 00:02:31.879 Pranav Narahari: One thing that I didn’t mention was that this is specifically for new customers.

30 00:02:32.100 00:02:32.820 Pranav Narahari: So… Yes.

31 00:02:32.820 00:02:36.259 Samuel Roberts: That’s right, okay. Yeah. So it’s their first order, right?

32 00:02:36.660 00:02:47.360 Pranav Narahari: Exactly, yeah, that’s just, like, another piece of complexity, that we just need to remember. So, how we’re going about doing that is… we’re going into…

33 00:02:48.620 00:02:54.799 Pranav Narahari: We’re basically checking For a field where order number equals 1,

34 00:02:54.910 00:02:58.449 Pranav Narahari: And so, yeah, basically what you said, like, that’s…

35 00:02:58.630 00:03:06.020 Pranav Narahari: Order number is a field that, casey found in the… orders table.

36 00:03:06.330 00:03:08.210 Pranav Narahari: And so, I don’t.

37 00:03:08.210 00:03:10.519 Samuel Roberts: Whoa, not on the… not on the customer?

38 00:03:11.030 00:03:18.479 Samuel Roberts: Oh, it’s like… It’s in the orders table which number order is for that customer.

39 00:03:19.740 00:03:21.360 Pranav Narahari: I believe so, yeah.

40 00:03:21.370 00:03:22.040 Samuel Roberts: Okay.

41 00:03:22.400 00:03:23.170 Pranav Narahari: Yeah.

42 00:03:23.860 00:03:29.920 Samuel Roberts: Okay, that makes sense. Yeah, I guess that makes sense. We don’t really care about the customers, we just care about orders of first customers. Yeah, that makes sense.

43 00:03:29.920 00:03:30.240 Pranav Narahari: Yeah.

44 00:03:30.240 00:03:30.970 Samuel Roberts: Like, okay.

45 00:03:32.590 00:03:33.840 Pranav Narahari: Yup, so…

46 00:03:35.280 00:03:41.079 Pranav Narahari: Yeah, there’s still a little bit more I need to dive into here, too, to understand, but all that…

47 00:03:41.240 00:03:45.329 Pranav Narahari: Casey used was the orders table and the customers table.

48 00:03:46.560 00:03:47.230 Samuel Roberts: Okay.

49 00:03:48.080 00:03:52.359 Pranav Narahari: Yeah, and so I believe everything should be there,

50 00:03:53.990 00:03:58.980 Pranav Narahari: I’m trying to think what else would be relevant to show right now. If I could, like, run this…

51 00:03:59.290 00:04:03.470 Samuel Roberts: Yeah, I mean, is it worth… so you have the air byte data?

52 00:04:04.540 00:04:08.169 Pranav Narahari: So, I’m not even using this earbite data, I’m using the one from your…

53 00:04:08.170 00:04:10.370 Samuel Roberts: Oh, the Postgres, sorry, you’re right, you’re right, you’re right, I forgot.

54 00:04:10.370 00:04:13.769 Pranav Narahari: Yeah, because this AirByte has not been re-synced, so…

55 00:04:13.770 00:04:14.929 Samuel Roberts: Bad one, yeah, okay.

56 00:04:14.930 00:04:15.410 Pranav Narahari: Yeah.

57 00:04:15.410 00:04:20.079 Samuel Roberts: I forgot, sorry, so many, so many things to juggle. Yeah, so the…

58 00:04:20.220 00:04:22.939 Samuel Roberts: So do we want to just try to start writing…

59 00:04:24.120 00:04:28.800 Samuel Roberts: queries against that Postgres database, then? Exactly, I think that is the best… Okay.

60 00:04:28.800 00:04:32.490 Pranav Narahari: order, like, of operations right now, like, let’s…

61 00:04:32.490 00:04:35.719 Samuel Roberts: I’m confident that if it’s in the Postgres table.

62 00:04:36.890 00:04:42.950 Pranav Narahari: That we will be able to transfer that to… like, that’ll also be available in,

63 00:04:43.710 00:04:51.810 Pranav Narahari: polyatomic, for sure. And because I feel like I saw that data, it was just off by a couple thousand, so maybe it was just…

64 00:04:52.480 00:04:58.200 Pranav Narahari: for whatever reason, like, I was pulling from the wrong thing, or, like, it was the wrong,

65 00:04:59.150 00:05:01.859 Pranav Narahari: Like, time zone, potentially, as well.

66 00:05:03.400 00:05:10.490 Samuel Roberts: Yeah, so there’s a lot of different things here. It’s not as cut and dry as I would hope for it to be.

67 00:05:10.490 00:05:15.299 Pranav Narahari: One thing that I still haven’t done yet is compare it to Shopify QL.

68 00:05:15.690 00:05:17.860 Pranav Narahari: So… That…

69 00:05:17.860 00:05:18.270 Samuel Roberts: Right.

70 00:05:18.270 00:05:25.030 Pranav Narahari: potentially help us as well. So, taking this query here, throwing it as,

71 00:05:25.140 00:05:31.070 Pranav Narahari: as input to the Shopify QL API should get us basically exactly this data.

72 00:05:31.980 00:05:33.110 Pranav Narahari: And so…

73 00:05:33.820 00:05:36.440 Samuel Roberts: Right, that’s what that’s doing, okay.

74 00:05:36.630 00:05:42.720 Pranav Narahari: Yeah. Now, I don’t know how much that’ll help us, But… It’s…

75 00:05:43.100 00:05:50.660 Pranav Narahari: something else to, like, discover, I know, like, if we… if we… looking into the API, like, how it translates to the…

76 00:05:51.070 00:05:53.920 Pranav Narahari: Just their, like, regular REST API?

77 00:05:56.870 00:05:57.860 Samuel Roberts: I’m sorry, repeat that?

78 00:05:58.430 00:06:06.410 Pranav Narahari: Like, so they have this, Shopify QL API, like, how this, like, compares to the other API that we’re using for…

79 00:06:06.850 00:06:09.149 Samuel Roberts: Right, oh, so we could do a side-by-side kind of thing.

80 00:06:09.150 00:06:09.900 Pranav Narahari: Yeah.

81 00:06:09.900 00:06:11.390 Samuel Roberts: Most grass, I see, yeah.

82 00:06:11.450 00:06:15.470 Pranav Narahari: Because… Yeah, this is just so complex for… I don’t know why.

83 00:06:15.880 00:06:19.140 Pranav Narahari: Yeah, I know, people complained about it as well, but yeah.

84 00:06:19.140 00:06:20.030 Samuel Roberts: Yeah.

85 00:06:21.970 00:06:22.820 Samuel Roberts: Okay.

86 00:06:24.510 00:06:25.760 Pranav Narahari: So…

87 00:06:28.250 00:06:30.389 Samuel Roberts: Let’s think about this. So we have the…

88 00:06:31.540 00:06:34.990 Samuel Roberts: Basically, we just want to kind of, like, run these queries

89 00:06:36.000 00:06:43.760 Samuel Roberts: Side by side to the post… let’s just… okay, maybe… Trying to think.

90 00:06:44.230 00:06:47.909 Samuel Roberts: How have you been running against the Postgres database? What… what are you using?

91 00:06:47.910 00:06:58.470 Pranav Narahari: Regarding that now, I haven’t run against the Postgres database yet. What I’m gonna do is, I have this tool, DBadmin, I don’t know if it’s the best one, but I’m just gonna use that.

92 00:06:58.670 00:07:02.609 Pranav Narahari: Db Visualizer, I mean.

93 00:07:03.950 00:07:04.730 Samuel Roberts: Okay.

94 00:07:04.730 00:07:08.499 Pranav Narahari: Yeah, so, like, in there, I can just, like, put in, like, the database URL, and then just, like, run.

95 00:07:08.500 00:07:12.979 Samuel Roberts: Yeah, perfect. Okay, that’s why I want to make sure there’s something good, otherwise we’re gonna have to, like, scaffold something out real quick, but…

96 00:07:13.290 00:07:18.630 Pranav Narahari: Yeah, I mean, you can also use PCQool if you want to use, like, the terminal, but…

97 00:07:19.790 00:07:25.140 Samuel Roberts: I was just trying to think what would be the easiest way to get, like, Cursor or Claude writing the, you know…

98 00:07:26.040 00:07:27.920 Pranav Narahari: Yeah, so that’s the thing, this deep visualization.

99 00:07:27.920 00:07:29.539 Samuel Roberts: Side by side, yeah.

100 00:07:29.540 00:07:30.110 Pranav Narahari: Yup.

101 00:07:30.550 00:07:36.640 Pranav Narahari: That’s what I was actually try it on here first, and just, like, you know, I can use, like…

102 00:07:36.760 00:07:40.360 Pranav Narahari: ChatGPT, Claude, Gemini, to, like, help.

103 00:07:40.820 00:07:45.760 Pranav Narahari: But, you know, now Chrome has… I don’t know if you ever use this, but it has, like, this.

104 00:07:46.260 00:07:49.490 Samuel Roberts: I just saw it today, actually, and it was like, I played with it a little bit, yeah.

105 00:07:49.490 00:07:54.440 Pranav Narahari: Yeah. So… I’ve been using that, actually, for Mother Duck.

106 00:07:54.730 00:07:58.719 Samuel Roberts: Okay. So, I’m gonna try to maybe find, like, a DB visualizing tool.

107 00:07:58.770 00:08:02.639 Pranav Narahari: on, like, like a website, like, that I can access via Chrome.

108 00:08:02.740 00:08:08.080 Pranav Narahari: And then, maybe then I can just use Gemini like this, or if not, you know, it’s just copy and pasting, like, it’s not a big deal.

109 00:08:08.080 00:08:09.190 Samuel Roberts: Yeah.

110 00:08:09.190 00:08:09.770 Pranav Narahari: Yeah.

111 00:08:09.770 00:08:14.869 Samuel Roberts: Yeah, there’s probably… Something you can even run locally that might just be able to set up a…

112 00:08:15.510 00:08:17.190 Samuel Roberts: a web UI for you.

113 00:08:17.800 00:08:18.420 Pranav Narahari: Yeah.

114 00:08:18.940 00:08:21.570 Samuel Roberts: I don’t know, I haven’t dug into that too much, I…

115 00:08:21.570 00:08:26.530 Pranav Narahari: Yeah, I might just do some research, and it’s like, what’s the best way to go about doing this, you know? Okay. Yeah.

116 00:08:26.690 00:08:28.069 Samuel Roberts: Yeah, I think.

117 00:08:28.070 00:08:29.550 Pranav Narahari: So let’s, let’s…

118 00:08:30.080 00:08:33.159 Samuel Roberts: Maybe the best thing to do would be…

119 00:08:36.200 00:08:41.410 Samuel Roberts: Let’s start, like… Back up a little bit, and go, like, basic, like.

120 00:08:42.039 00:08:44.839 Samuel Roberts: Trying… just try to match the Shopify QL to the…

121 00:08:46.570 00:08:50.799 Samuel Roberts: Postgres data, and figure out, like, where the discrepancies might pop up.

122 00:08:52.550 00:08:58.830 Samuel Roberts: Does that make sense? Like, let’s just look at, like, order revenue for a given day, right? Forget about new customers.

123 00:08:59.170 00:08:59.540 Pranav Narahari: Yeah.

124 00:08:59.540 00:09:05.450 Samuel Roberts: that’s any close. Like, then pull in the new customers, then pull in the… you know…

125 00:09:06.170 00:09:08.620 Samuel Roberts: Whatever other filters we need to start adding.

126 00:09:09.180 00:09:15.409 Samuel Roberts: And just see, like, how far off we… like, where it starts to build, maybe, and that might give us an insight into…

127 00:09:16.420 00:09:21.200 Samuel Roberts: one, how they’re doing their QL stuff and whatever modeling they have on top of it that we’re not seeing.

128 00:09:21.350 00:09:27.270 Samuel Roberts: And two, if it is, like, a time zone thing or whatever, like, we might find that too.

129 00:09:30.450 00:09:37.599 Samuel Roberts: Yeah… Let’s… let’s see… So are you… you’re using Cloud Code predominantly?

130 00:09:38.610 00:09:50.259 Pranav Narahari: Yeah, Cloud Code and Cursor. Mostly, I would say Cloud Code for, like, big tasks, for Cursor for more, like, research stuff, and, like, asking questions against the repo.

131 00:09:51.130 00:09:53.380 Samuel Roberts: Okay. I was trying to think if we could…

132 00:09:53.380 00:09:53.990 Pranav Narahari: Nope.

133 00:09:54.540 00:09:56.299 Samuel Roberts: Set up a quick, like…

134 00:09:59.310 00:10:02.300 Samuel Roberts: I’m just thinking, like, a simple… script.

135 00:10:03.180 00:10:07.149 Samuel Roberts: But when I, when I used to do something like this, I just, I go to Bun.

136 00:10:10.430 00:10:11.530 Samuel Roberts: And I think…

137 00:10:14.970 00:10:17.390 Samuel Roberts: Does it have a Postgres adapter built in?

138 00:10:20.200 00:10:26.710 Samuel Roberts: This is an SQL. Yeah, it has an SQL, so let’s see if it’s about screens.

139 00:10:26.850 00:10:36.199 Samuel Roberts: Yeah, okay, cool. So, my thought would be… the way I’m… I’m not necessarily telling you how to do it, I’m just trying to think, like, what is the way I would do this if this was…

140 00:10:36.520 00:10:37.560 Samuel Roberts: Me.

141 00:10:38.030 00:10:38.710 Pranav Narahari: Yeah.

142 00:10:38.890 00:10:43.080 Samuel Roberts: I would… Probably just start writing a couple quick scripts.

143 00:10:43.290 00:10:46.450 Samuel Roberts: that… You know, we’ll put the…

144 00:10:47.150 00:10:50.829 Samuel Roberts: the database URL in an NVAR, and then you can just run

145 00:10:51.560 00:10:56.490 Samuel Roberts: SQL queries right from, in this case, you know, a TS file.

146 00:10:57.020 00:10:57.420 Pranav Narahari: Yeah.

147 00:10:57.420 00:10:59.950 Samuel Roberts: While at the same time figuring out

148 00:11:00.730 00:11:07.920 Samuel Roberts: how to make the Shopify QL calls From that script as well.

149 00:11:10.880 00:11:14.530 Samuel Roberts: so that I could be chatting with Cursor or Claude.

150 00:11:15.200 00:11:25.989 Samuel Roberts: making sure that it knows we’re trying to keep these… they’re trying to see these similar… like, see what’s similar here. So when I run the script, it’ll do, like, you know, select all the orders, or count all the orders, for example, from…

151 00:11:26.430 00:11:32.409 Samuel Roberts: Whatever, like, you know, however… however recent the data we have is, you know, pick a date that we know is synced.

152 00:11:34.650 00:11:35.839 Samuel Roberts: And start, like…

153 00:11:36.670 00:11:44.269 Samuel Roberts: simply with… just, let’s count the orders, and then count the orders with the Shopify QL, and see if we can get that to run…

154 00:11:44.870 00:11:46.370 Samuel Roberts: In a quick script.

155 00:11:48.980 00:11:51.099 Samuel Roberts: And just dump that out and see, like.

156 00:11:51.320 00:11:53.729 Samuel Roberts: Are the order counts close, you know?

157 00:11:53.980 00:11:58.899 Pranav Narahari: Yeah, so ba- yeah, I see what you’re saying. That’s kind of how I went about, like,

158 00:11:59.940 00:12:04.359 Pranav Narahari: creating, like, that first query, that then Casey, like, refined.

159 00:12:04.430 00:12:22.780 Pranav Narahari: Yeah, just kind of getting one field to match, basically, right? Yeah. Yeah. I think that’s the best way to go about it. So, actually, yeah, I said gross sales first, but what I went for was, order number, because I was just like, okay, let’s just even… I just felt like that’s even more simpler to just match.

160 00:12:22.780 00:12:23.949 Samuel Roberts: Totally, yeah. Okay.

161 00:12:23.950 00:12:25.309 Pranav Narahari: And from there, it’s like…

162 00:12:25.750 00:12:34.510 Pranav Narahari: you’re likely going to be doing the same operations and just pulling in an additional field. So, everything else became much simpler, actually, to pull in, but yeah.

163 00:12:34.510 00:12:35.210 Samuel Roberts: I think…

164 00:12:35.520 00:12:40.330 Pranav Narahari: so what do you think is the best way to go about this? Because, like, I think we’re both…

165 00:12:40.850 00:12:43.869 Pranav Narahari: I need to rewrite this query, basically, to, like.

166 00:12:44.710 00:12:48.330 Pranav Narahari: Based on, the sequel, kind of.

167 00:12:48.660 00:12:51.379 Pranav Narahari: like, how things are set up in, Postgres?

168 00:12:51.650 00:12:56.740 Samuel Roberts: Okay. Versus, like, how things were… are set up in Mother Duck for the polyatomic test. Sure.

169 00:12:56.760 00:13:00.679 Pranav Narahari: So, I need to first do that, okay.

170 00:13:01.470 00:13:07.100 Pranav Narahari: I’m… yeah, so it’s kind of like a step backwards, since, the data is no longer in…

171 00:13:07.500 00:13:15.519 Pranav Narahari: in Polytomic, but I don’t want to wait, necessarily, for Polytomic to just update, because there also could be issues with that, I have no clue.

172 00:13:15.520 00:13:16.270 Samuel Roberts: Yeah.

173 00:13:16.660 00:13:17.080 Pranav Narahari: So…

174 00:13:17.080 00:13:18.219 Samuel Roberts: Yeah, I would say…

175 00:13:18.480 00:13:21.869 Samuel Roberts: do that, and then what’s the result of this query? Like, what comes out of this?

176 00:13:23.710 00:13:26.680 Samuel Roberts: The final, like, data that comes out is the…

177 00:13:27.580 00:13:32.000 Pranav Narahari: I wanted to show, yeah, exactly what was shown in that Shopify dashboard.

178 00:13:32.000 00:13:34.429 Samuel Roberts: So it’s basically that, like, a whole table like that?

179 00:13:34.430 00:13:35.070 Pranav Narahari: Yeah.

180 00:13:35.370 00:13:41.149 Samuel Roberts: Okay. Yeah, I would say work on that. Maybe I’ll jump in and just do a quick script and see if I can see, like.

181 00:13:41.280 00:13:46.940 Samuel Roberts: the Shopify QL from the API, like… call the Shopify QL.

182 00:13:47.600 00:13:48.000 Pranav Narahari: Yeah.

183 00:13:48.000 00:13:49.340 Samuel Roberts: A very simple thing.

184 00:13:49.920 00:13:55.369 Samuel Roberts: and just try to script it real quick, and see if I can even just point cursor to make some more, like, tests, and even see if I can…

185 00:13:55.560 00:13:56.170 Samuel Roberts: Identify.

186 00:13:56.170 00:13:56.550 Pranav Narahari: Yeah.

187 00:13:57.010 00:14:02.230 Pranav Narahari: I really think this is just, like, hacking at this, you know, so both of us will just…

188 00:14:02.550 00:14:05.439 Samuel Roberts: When we’re working, we’ll just be doing different approaches, and…

189 00:14:05.450 00:14:09.780 Pranav Narahari: we can just update each other, like, on Slack to say, like, where we’re at.

190 00:14:09.780 00:14:10.470 Samuel Roberts: Yeah.

191 00:14:10.650 00:14:14.300 Pranav Narahari: By end of day, maybe we can just do an update. I know you’ll be probably off in, like.

192 00:14:14.880 00:14:16.270 Pranav Narahari: Pretty soon, probably.

193 00:14:16.270 00:14:17.590 Samuel Roberts: Yeah, yeah.

194 00:14:17.590 00:14:22.440 Pranav Narahari: Yeah, so maybe, like, in an hour or less, like, you can give me an update of where you’re at, and then I can.

195 00:14:22.440 00:14:23.030 Samuel Roberts: Yeah.

196 00:14:23.480 00:14:34.420 Pranav Narahari: I’ll be working on mine, but if, like, you kind of find something out that I haven’t yet, then I can, like, integrate that into my query, and then continue from there.

197 00:14:35.110 00:14:43.530 Samuel Roberts: Okay, sounds good. Yeah, I’ll throw together some scripts, and I’ll just do it in, like, a temp folder or something, and then if anything works, I’ll drop it in the repo or something on a branch, maybe.

198 00:14:43.730 00:14:44.190 Pranav Narahari: Perfect.

199 00:14:44.190 00:14:47.159 Samuel Roberts: Or just share it, if it’s just a script, it doesn’t really matter that much, but…

200 00:14:47.160 00:14:47.720 Pranav Narahari: Yeah, if it’s.

201 00:14:47.720 00:14:49.540 Samuel Roberts: Yeah, I’ll see if I can do anything real quick.

202 00:14:49.540 00:14:50.120 Pranav Narahari: Yeah.

203 00:14:50.990 00:14:52.280 Samuel Roberts: Okay, sounds good.

204 00:14:52.910 00:14:54.750 Samuel Roberts: Perfect. Hopefully we hit the bottom of this.

205 00:14:55.130 00:14:55.670 Pranav Narahari: Yeah.

206 00:14:56.320 00:14:57.130 Pranav Narahari: Alright, see ya.