The DataDank 90-day plan
Master data engineering. Ace your interviews. Build your future.
A practical path to data engineering skills: clear tutorials, hands-on practice, interview questions, real-world projects, and structured roadmaps.
- Days
- 90
- Planned hours
- 270
- Topics tracked
- 1046
- Lessons
- 239
Your progress
Saved in this browser only, no account needed.Open a lesson or tick off an interview question and your progress appears here.
Continue where you left offThe roadmap
Three phases. Thirteen weeks. One goal.
Foundations first, then advanced topics and projects, then interview mastery. Each week starts with foundations, then adds tools after their prerequisites, with a weekly milestone.
Days 1–28 · 84 hours
Foundations
SQL, Python and easy DSA; data-engineering concepts and data modeling; start Spark; build a tested CSV-to-warehouse pipeline.
- Week 1Write filtered and aggregated SQL, use Python for files and functions, and solve easy array and hashing problems.
- Week 2Join tables safely, structure Python with tests and logging, and explain file formats and batch vs streaming.
- Week 3Design a star schema at the right grain, handle slowly changing dimensions, and load data idempotently.
- Week 4Write window-function SQL, explain Spark's execution model, and finish the CSV-to-warehouse project.
Target You can explain Python for data engineering, and you solved today's SQL and DSA problems without hints.
Open phase 1Days 29–63 · 105 hours
Core pipelines
Spark, Delta Lake, Airflow, dbt and Snowflake; medium DSA; batch and analytics projects; begin system design.
- Week 5Transform data with PySpark DataFrames, choose join strategies, and start system design with batch ingestion.
- Week 6Diagnose shuffles and skew in Spark, explain Delta Lake transactions, and finish the large-scale batch project.
- Week 7Write Airflow DAGs with safe retries and backfills, build dbt models, and design an ELT pipeline.
- Week 8Schedule with sensors and assets in Airflow, test dbt models, and build the e-commerce analytics project.
- Week 9Load and model data in Snowflake, use streams and tasks, and design analytics platforms.
Target You can explain Reading/writing Parquet, and you solved today's SQL and DSA problems without hints.
Open phase 2Days 64–90 · 81 hours
Streaming, design and interviews
Kafka, Spark streaming, AWS services and system design; graph and DP problems; a streaming project, mocks and final revision.
- Week 10Explain Kafka partitions, consumer groups and delivery guarantees; build the S3, PySpark and Snowflake pipeline.
- Week 11Explain Kafka replication and exactly-once, use AWS ingestion services, and design streaming systems.
- Week 12Build a watermarked Spark streaming job, design governed platforms, and run the streaming project end to end.
- Week 13Revise every track, solve hard DSA problems, and finish your project demo, résumé and mocks.
Target You can explain Clustering keys, and you solved today's SQL and DSA problems without hints.
Open phase 3
Courses
Pick a technology and start learning
Each course has ordered lessons, interview questions, projects and a cheat sheet.
- Languages & querySQLFree SQL course for Data Engineers: joins, window functions, CTEs, deduplication, SCD queries and query plans, with runnable examples and interview questions.
- Languages & queryPythonFree Python course for Data Engineering: generators, CSV and JSON files, APIs, pandas, error handling and pytest, with runnable code and interview questions.
- Distributed processingPySparkFree PySpark course for Data Engineers: DataFrames, joins, window functions, partitions, shuffles and skew, with runnable code and interview questions.
- Distributed processingApache SparkFree advanced Apache Spark course for data engineers: architecture, jobs and stages, shuffles and skew, caching, AQE, memory tuning and Structured Streaming.
- Streaming & orchestrationApache KafkaFree Apache Kafka course for Data Engineers: topics, partitions, consumer groups, delivery semantics and streaming pipelines, with code and interview questions.
- Streaming & orchestrationApache AirflowFree Apache Airflow course for data engineers, from a first DAG to production: scheduling, retries, sensors, XCom, executors, testing and CI/CD on Airflow 3.
- Data platformsDelta LakeFree Delta Lake course: ACID transactions, MERGE, schema evolution, time travel, OPTIMIZE and Z-ordering for the lakehouse, with code and interview questions.
- Data platformsData modelingFree data modeling and data warehouse course: star schemas, grain, facts and dimensions, SCD Type 2, Data Vault, incremental models and interview questions.
- Data platformsETL and ELTFree ETL and ELT course: how modern data pipelines extract, load and transform data, when each fits, idempotency and backfills, with interview questions.
- Data platformsDatabricksFree Databricks course for Data Engineers: workspace, Jobs, Auto Loader, declarative pipelines and Unity Catalog, with examples and interview questions.
- Data platformsSnowflakeFree Snowflake course for Data Engineers: virtual warehouses, micro-partitions, pruning, caching, Snowpipe, streams and cost control, with interview questions.
- CloudAWSFree AWS course for Data Engineers: S3, IAM, Glue, Athena, Redshift, EMR, Kinesis and Lambda as one data platform, with worked examples and interview questions.
- Interview foundationsDSAFree DSA course for Data Engineering interviews: arrays, hashing, strings, two pointers, sliding windows and more, with Python solutions and practice problems.
- Distributed processingdbtFree dbt course for Data Engineers: models, materializations, incremental models, tests, snapshots and CI, with runnable projects and interview questions.
- Interview foundationsDev toolsFree developer tools course for Data Engineers: command line, shell scripts, Git, Python environments and Docker, the toolkit every data pipeline job expects.
- Interview foundationsLLMsFree LLM course for AI engineers: how large language models work, tokens, embeddings, attention, generation and prompting, explained simply with runnable code.
- Interview foundationsRAGFree RAG course for AI engineers: how retrieval-augmented generation works, chunking, embeddings, hybrid search, reranking and evaluation, with runnable code.
- Interview foundationsAI AgentsFree AI agents course: how agents work, the agent loop, workflows vs agents, tools and MCP, evaluation and safety, explained simply with runnable Python code.
- Interview foundationsFDEFree forward deployed engineering course: discovery, scoping a proof of concept, API and data integration, and deploying for customers, with runnable code.
- Interview foundationsStatisticsFree statistics course for data science and analysis: averages and spread, distributions, the central limit theorem, hypothesis tests and A/B testing.
- Interview foundationsMachine LearningFree machine learning course: the ML workflow, regression, decision trees, random forests and boosting, evaluation metrics and data leakage, with runnable code.
- Interview foundationsDeep LearningFree deep learning course: how neural networks work and learn, backpropagation, learning rates, CNNs, RNNs, transformers and PyTorch basics, explained simply.
- Interview foundationsMLOpsFree MLOps course: the ML lifecycle, experiment tracking and model registries, batch and online serving, and monitoring for drift, explained with runnable code.
Roadmap
Data Engineer roadmap
From beginner foundations to interview-ready, in 13 ordered stages.
Interview
Three focused ways to prepare
- Technology interviewsInterview questionsDirect answers first, then reasoning, common mistakes and follow-ups.
- System designCase studiesRequirements, architecture, reliability and trade-offs for real data platforms.
- Company preparationCompany guidesPreparation that separates attributed information from practice questions.
Projects
Build something you can explain
- BeginnerCSV to Data Warehouse PipelineA small online shop exports orders as daily CSV files. Build a pipeline that loads them into a star schema so that sales can be reported reliably, even when files are resent, corrected or contain bad rows.
- IntermediateE-commerce Analytics Data PlatformAn online store has orders, customers, products, refunds and web sessions in separate systems. Build an ELT platform that models them into trusted marts for revenue, conversion and repeat customers, with every metric defined once and tested.
- AdvancedChange Data Capture PipelineReplicate an operational PostgreSQL table into a lakehouse table within minutes, including updates and deletes, so analysts query current data without touching the production database, and prove that replays, duplicates and out-of-order delivery cannot corrupt the copy.
Featured guides
Start with these
- Career guideBeginnerHow to Explain a Data Engineering Project in an InterviewA repeatable structure for explaining a data engineering project in an interview: problem, design, trade-offs, failure handling and honest results.
- ArticleBeginnerStar, Snowflake and Galaxy Schemas and the GrainBuild a star schema for an online shop, declare the grain, snowflake a dimension, and query a galaxy of fact tables without double counting. Verified SQL.
- ArticleBeginnerKafka Topics, Partitions and Consumer GroupsHow Kafka topics split into partitions, how keys decide ordering, what replication factor protects, and how consumer groups divide partitions between consumers.
- ArticleIntermediatePySpark Window Functions: Ranking, Lag and Running TotalsPySpark window functions for top-N per group, deduplication, lag and lead, running totals, gap filling and sessions, and avoiding single-partition windows.
How we write
Editorial principles
Reviewed technical content
Claims about tools are checked against official documentation, and each page shows when it was last reviewed.
Version-aware
Where behaviour depends on a version or platform release, the page says which one it describes.
Evidence-labelled interviews
Company material is labelled as attributed, reported or representative practice. We never present invented questions as real ones.
Structured paths
Every lesson sits in a course with a clear previous and next step.