furyhawk/opencode
1
0
Fork 0
mirror of https://github.com/anomalyco/opencode.git synced 2026-07-21 10:16:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(data): cut athena stats sync cost

Browse Source 
Compute all stat dimensions and both grains in one GROUPING SETS query
so each sync pass scans the source table once instead of eight times.
Hourly passes now only recompute the current ISO week; a daily full pass
refreshes the whole display window. Restarted daemons resume the hourly
cadence from the last completed sync instead of immediately re-running a
pass, and the workgroup kills any query scanning more than 2 TB.
This commit is contained in:
Adam
2026-07-10 06:36:26 -05:00
parent 60c88336b9
commit 08a2a623dc
5 changed files with 122 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
infra/lake.ts
+4
View File
@@ -67,6 +67,10 @@ const athenaWorkgroup = new aws.athena.Workgroup("LakeAthenaWorkgroup", {
configuration: {
enforceWorkgroupConfiguration: true,
publishCloudwatchMetricsEnabled: true,
// Athena bills $5/TB scanned; kill any query that would scan more than 2 TB
// so a regression cannot silently burn money. Stats sync full passes scan
// ~250 GB as of 2026-07.
bytesScannedCutoffPerQuery: 2 * 1024 ** 4,
resultConfiguration: {
outputLocation: $interpolate`s3://${athenaResultsBucket.bucket}/`,
},