Data Engineering Roadmap

Air Quality Pulse Learning Path

This page is the single browser home for the tasks you have learned so far. Each task has its own goal, code, SQL, tests, and visual guide so you can trace it later.

Learned Tasks

01
Complete

Local Foundations and Batch Ingestion

Learned how to extract OpenAQ data, save raw JSON, transform API payloads into curated rows, validate fields, load SQLite, and serve a dashboard.

02
Complete

Relational Modeling

Learned what was new after ingestion: dimensions, facts, table grain, keys, joins, foreign-key constraints, indexes, and idempotent mart refreshes.

Current Architecture

SourceOpenAQ API or sample payloads
Rawdata/raw/*.json
Landingair_quality_measurements
Martdim_* and fact_*
ServingSQL workbook and Streamlit dashboard

How To Use This Home Page

When You Want ToOpen
Review what you learned in a taskThe task visual guide
Replay the task from terminal commandsThe task markdown guide
Practice SQL for the taskThe task SQL workbook
See the cumulative applicationhttp://localhost:8501