https://hk07qz.jp2haozu-naookg-okak.cc/Essays and field notes on data engineering.en https://hk07qz.jp2haozu-naookg-okak.cc/posts/small-data-is-underrated.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/small-data-is-underrated.htmlThu, 13 Aug 2026 09:00:00 +0000I've been working with Python for about 9 years now, and my opinions on backfills have changed more than I expected. https://hk07qz.jp2haozu-naookg-okak.cc/posts/data-contracts-without-the-hype.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/data-contracts-without-the-hype.htmlMon, 03 Aug 2026 09:00:00 +0000There is no shortage of articles about pandas. Most of them cover the happy path. This one is about what happened after the happy path ended. https://hk07qz.jp2haozu-naookg-okak.cc/posts/the-metric-everyone-defined-differently.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/the-metric-everyone-defined-differently.htmlSat, 25 Jul 2026 09:00:00 +0000This post started as a note to my future self. Every few months I relearn the same lessons about incremental models, so this time I'm writing them down. https://hk07qz.jp2haozu-naookg-okak.cc/posts/polars-vs-pandas-for-everyday-work.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/polars-vs-pandas-for-everyday-work.htmlMon, 13 Apr 2026 09:00:00 +0000For a long time I treated backfills as something to worry about later. Then a small change to our pandas setup turned into a week of debugging, and "later" arrived all at once. https://hk07qz.jp2haozu-naookg-okak.cc/posts/incremental-models-that-don-t-lie-to-you.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/incremental-models-that-don-t-lie-to-you.htmlSat, 03 Jan 2026 09:00:00 +0000A colleague asked me last week why our team cares so much about incremental models. I gave a rambling answer, so here is the longer, more organized version. https://hk07qz.jp2haozu-naookg-okak.cc/posts/testing-pipelines-like-application-code.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/testing-pipelines-like-application-code.htmlFri, 07 Nov 2025 09:00:00 +0000Every codebase has one area everyone is a little afraid to touch. On our team, it was everything related to late-arriving data. https://hk07qz.jp2haozu-naookg-okak.cc/posts/duckdb-replaced-half-of-our-data-stack.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/duckdb-replaced-half-of-our-data-stack.htmlMon, 20 Oct 2025 09:00:00 +0000For a long time I treated data quality checks as something to worry about later. Then a small change to our DuckDB setup turned into a week of debugging, and "later" arrived all at once. https://hk07qz.jp2haozu-naookg-okak.cc/posts/partitioning-strategies-i-wish-i-d-known-earlier.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/partitioning-strategies-i-wish-i-d-known-earlier.htmlWed, 13 Aug 2025 09:00:00 +0000There is no shortage of articles about DuckDB. Most of them cover the happy path. This one is about what happened after the happy path ended. https://hk07qz.jp2haozu-naookg-okak.cc/posts/dealing-with-late-arriving-data.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/dealing-with-late-arriving-data.htmlSun, 27 Jul 2025 09:00:00 +0000For a long time I treated late-arriving data as something to worry about later. Then a small change to our Spark setup turned into a week of debugging, and "later" arrived all at once. https://hk07qz.jp2haozu-naookg-okak.cc/posts/backfills-are-a-product-feature.htmlhttps://hk07qz.jp2haozu-naookg-okak.cc/posts/backfills-are-a-product-feature.htmlThu, 22 May 2025 09:00:00 +0000I've been working with Snowflake for about 7 years now, and my opinions on backfills have changed more than I expected.