Dagster → Inline Migration Tracker
Owner: AI Team
Last updated: 2026-02-28
Scope: apps/dagster-pipelines/ to platform-native processing in apps/platform/
Goal
Consolidate orchestration context in one monorepo while migrating legacy Dagster jobs to inline/service execution patterns that are easier for agents to reason about end-to-end.
Workflow (use for each pipeline)
- Capture current Dagster behavior:
- trigger/schedule
- inputs and configuration
- external dependencies
- output destinations
- Define platform-native replacement and owner.
- Implement replacement in
apps/platform. - Validate parity on a bounded window (schema + row-level checks where feasible).
- Cut over schedule/trigger.
- Mark status in this tracker.
Status values:
dagster-active— still running in Dagsterin-migration— platform replacement in progressmigrated— production workload moved off Dagsterdeprecated— retired without replacement
Pipeline Inventory
| Dagster symbol | Source module | Status | Platform replacement | Notes |
|---|---|---|---|---|
github_to_s3 | pipelines.github_to_s3.github_to_s3_pipeline.github_to_s3 | dagster-active | TBD | |
slack_data_transformation_job | pipelines.slack_data_transformation.slack_data_transformation_pipeline.slack_data_transformation_job | dagster-active | TBD | |
client_data_to_s3 | pipelines.client_docs_to_s3.client_docs_to_s3.client_data_to_s3 | dagster-active | TBD | |
transform_job | pipelines.dagster_pre_retrieval.transform_pipeline.transform_job | dagster-active | TBD | |
embedding_job | pipelines.dagster_pre_retrieval.embedding_pipeline.embedding_job | dagster-active | TBD | |
clockify_to_sheets_job | pipelines.clockify_to_sheets.clockify_to_sheets_pipeline.clockify_to_sheets_job | dagster-active | TBD | |
abc_8_by_8_job | pipelines.abc_8_by_8_pipeline.abc_8_by_8_pipeline.report_pipeline | dagster-active | TBD | |
abc_8_by_8_transcript_job | pipelines.abc_8_by_8_transcript_pipeline.abc_8_by_8_transcript_pipeline.transcript_job | dagster-active | TBD | |
zoom_auto_delete_job | pipelines.zoom_auto_delete.zoom_auto_delete_pipeline.zoom_auto_delete_job | dagster-active | TBD | |
adspend_pipeline | pipelines.eden_offline_spend_pipeline.eden_offline_spend_pipeline.adspend_pipeline | dagster-active | TBD | |
vibeSpend_pipeline | pipelines.eden_update_vibe_spend_pipeline.eden_update_vibe_spend_pipeline.vibeSpend_pipeline | dagster-active | TBD | |
katalysSpend_pipeline | pipelines.eden_katalys_spend_pipeline.eden_katalys_spend_pipeline.katalysSpend_pipeline | dagster-active | TBD | |
theofferSpend_pipeline | pipelines.theoffer_daily_spend_pipeline.theoffer_daily_spend_pipeline.theofferSpend_pipeline | dagster-active | TBD | |
mntn_spend_pipeline | pipelines.eden_mntn_spend_pipeline.eden_mntn_spend_pipeline.mntn_spend_pipeline | dagster-active | TBD | |
eden_user_traits_audit_job | pipelines.eden_user_traits_model.eden_user_traits_model_pipeline.eden_user_traits_audit_job | dagster-active | TBD | |
event_scraper_trigger_pipeline | pipelines.event_scraper_trigger.event_scraper_trigger_pipeline.event_scraper_trigger_pipeline | dagster-active | TBD | |
operating_to_snowflake_job | pipelines.operating_to_snowflake.operating_to_snowflake_pipeline.operating_to_snowflake_job | dagster-active | TBD | |
clockify_to_snowflake_job | pipelines.clockify_to_snowflake.clockify_to_snowflake_pipeline.clockify_to_snowflake_job | dagster-active | TBD | |
clockify_to_operating_job | pipelines.clockify_to_operating.clockify_to_operating.clockify_to_operating_job | dagster-active | TBD | |
embed_zoom_transcripts_job | pipelines.zoom_transcript_embedding.zoom_transcript_embedding.embed_zoom_transcripts_job | dagster-active | TBD | |
braintrust_eval_job | pipelines.interlude_deck_evals.braintrust_eval_pipeline.braintrust_eval_job | dagster-active | TBD | |
braze_campaigns_job | pipelines.braze_to_sheets.braze_to_sheets.braze_campaigns_job | dagster-active | TBD | |
doordash_sponsored_listings_job | pipelines.doordash_ads_to_sheets.doordash_sponsored_listings_scraper.doordash_sponsored_listings_job | dagster-active | TBD | |
doordash_promotions_job | pipelines.doordash_promotions_to_sheets.doordash_promotions_scraper.doordash_promotions_job | dagster-active | TBD | |
uberads_scraping_job | pipelines.uberads_to_sheets.uberads_scraper.uberads_scraping_job | dagster-active | TBD | |
google_calendar_to_snowflake_job | pipelines.google_calendar_to_snowflake.google_calendar_to_snowflake_pipeline.google_calendar_to_snowflake_job | dagster-active | TBD | |
zoom_meeting_to_turbopuffer_job | pipelines.zoom_meeting_to_turbopuffer_pipeline.zoom_meeting_to_turbopuffer.zoom_meeting_to_turbopuffer_job | in-migration | TBD | Prior migration work exists in PR #47; confirm current production path before cutover |
ubereats_automation_job | pipelines.ubereats_automation_pipeline.ubereats_automation.ubereats_automation_job | dagster-active | TBD | |
ubereats_report_downloader_job | pipelines.ubereats_report_downloader_pipeline.ubereats_report_downloader.ubereats_report_downloader_job | dagster-active | TBD | |
katalys_orders_pipeline | pipelines.katalys_orders_pipeline.katalys_orders_pipeline.katalys_orders_pipeline | dagster-active | TBD | |
spend_data_to_northbeam_pipeline | pipelines.spend_data_to_northbeam_pipeline.spend_data_to_northbeam_pipeline.spend_data_to_northbeam_pipeline | dagster-active | TBD |
Notes
- Subtree import completed on branch
cursor/dagster-platform-merge-edd2. - Dagster source-of-truth path in this monorepo:
apps/dagster-pipelines/.