Pipeline to Dashboardnihalsulaiman.com

Week 7 · Run it in production › Day 49

Day 49. Ship it and tell the story

⏱ 4h · README 60m · video 45m · write-up 60m · interview prep 75m🏷 allBefore: Day 48

By the end of today you can

  • Publish a portfolio-grade README and demo for the capstone
  • Turn the seven proofs into resume bullets, a LinkedIn post and a blog article
  • Answer the data-platform system-design interview question with structure
  • Plan the next 90 days of learning

1. The capstone README

  1. Title + one sentence + the health dashboard or architecture image.
  2. What it does in 5 bullets, with numbers (sources, rows/day, run time, cost/month).
  3. Demo video (2–3 minutes: the demo morning).
  4. Architecture diagram and the stack table with why for each choice (link the ADRs).
  5. Run it in 5 minutes (just demo).
  6. Design decisions: idempotency, business dates, quarantine, partitions, upsert patterns, SCD2, metric dictionary, missing ≠ zero.
  7. Reliability: SLOs, checks, triage reports, incident review.
  8. Cost per location per month for three stacks.
  9. What I'd do next.
  10. Links to the six proof repos.

2. Resume bullets (fill in your own measured numbers)

  • Built an end-to-end daily data platform (Python, Dagster, dbt, Azure SQL/Postgres, Power BI) ingesting email, SFTP, shared-drive and API sources for four synthetic businesses; reruns are idempotent and every report figure traces to source files.
  • Designed Kimball star schemas with SCD2 dimensions and [N] dbt tests, including a revenue-conservation test run on every build.
  • Cut full-pipeline runtime from [A] to [B] by making all layers incremental and compacting Parquet partitions.
  • Costed three platform options at 20–500 locations; the open-source stack ran at about $[X] per location per month.

Keep every number one you measured. Interviewers ask "how did you measure that?".

3. The system-design question

"Design a data platform for a 200-restaurant chain that wants daily sales dashboards by 8 a.m." Answer in this order (about 10 minutes):

  1. Clarify: sources and how they arrive, volumes, users, latency, budget, team size, existing licences.
  2. Size it: rows/day, history, growth (Fermi estimate, Day 22).
  3. Draw the flow: ingest → raw → clean → warehouse → model → BI, plus ops.
  4. Pick tools with reasons and one alternative each.
  5. Correctness: idempotency, business date, late data, dedup, reconciliation.
  6. Reliability: checks, SLOs, alerts, runbooks.
  7. Security and cost: identities, RLS, cost per location.
  8. Evolution: what changes at 10× (Day 47).
Maths and principle

Principle: structured answers beat brilliant fragments because interviewers score against a rubric.

Equation: if a rubric has \(k\) areas scored 0–2 and you cover all \(k\) at level 1 instead of \(k/2\) at level 2, you score \(k\) either way, but the full cover also shows judgment. Breadth first, then depth where they probe.

Analogy: a series expansion: get the leading terms everywhere before refining one term. Where it breaks: if they say "go deep on X", go deep.

Try it: 8 areas. You go deep on 3 (2 points each) and skip 5. Score? (6 of 16.) Cover all 8 at 1 point and 2 of them at 2? (10 of 16.)

4. Talking about the course itself

You learned this in seven weeks on synthetic data. Say exactly that, with confidence: "I built the full stack from scratch on synthetic data to understand every layer: ingestion from email, SFTP and APIs, Dagster, Azure SQL, dbt and Power BI, with open-source alternatives costed. The repos show every decision and test." Keep former employers, their systems and their data out of it, and don't imply production experience you don't have. The repos are the proof.

The 5-year lens

Write one blog post (LinkedIn article or your GitHub Pages site) about one hard problem, not the whole course: "Missing isn't zero: how a completeness matrix caught a silent store", or "Why my revenue doubled: fan-out and the grain sentence". Specific, short, with one chart. Hiring managers share posts like that; nobody shares "I finished a course".

Today

  1. Finish and polish the capstone README; record the video.
  2. Update your GitHub profile README with the seven projects (one line each).
  3. Write the resume bullets with your real numbers and add them to your resume files.
  4. Draft the blog post and a short LinkedIn post linking it.
  5. Do the system-design question aloud twice, recorded. Watch it back once.
  6. Write docs/next-90-days.md: one deeper topic per month (for example streaming, Spark/Databricks, advanced DAX/Fabric, ML in production) and one certification to consider (for example Microsoft PL-300 Power BI Data Analyst or DP-700 Fabric Data Engineer; check current Microsoft certification names).
  7. When you've chosen where to publish, ask for the repo and the site goes up alongside your projects.

Check yourself

1. First step in a data-platform system-design interview?

2. How should you describe your experience with this stack?

My notes