Back to Data Engineer Home View forgery project
AWS + Airflow Project

Street Crossing Analytics Pipeline

Computer vision data pipeline: ingest street video, run YOLO detection, and publish reliable crossing metrics through an AWS + Airflow workflow.

AWS-focused pipeline Airflow orchestration YOLO crossing count
Detected street crossing events with YOLO

YOLO detection and crossing event tracking

Pipeline Scope

Production-style pipeline that converts raw video files into structured crossing metrics.

Data engineering work included ingestion, ETL, workflow orchestration, cloud storage, and analysis-ready exports.

Source code is private due to internship data and cloud environment constraints.

Stack Used

AWS Airflow ETL Pipelines SQL Parquet Monitoring

Pipeline Schema

01. Ingestion

Collect video files and register metadata for tracked processing.

->
02. Preprocessing

Prepare frames with consistent format and quality checks.

->
03. YOLO Inference

Detect people and capture movement events per zone.

->
04. Metrics Build

Aggregate detections into crossing counts and timestamps.

->
05. Airflow Orchestration

Run scheduled DAGs with retries, logs, and rerun control.

->
06. AWS Outputs

Publish parquet/CSV outputs for SQL analysis and dashboards.