Posts

All the articles I've posted.

• 21 MIN READ•May 22, 2026

  Building a Multicloud Agentic Lakehouse Reference Architecture

  A reference architecture for building an open, multicloud Data Lakehouse optimized for AI Agents using Apache Polaris, Apache Iceberg, and Dremio.

  data lakehouseagentic lakehouseapache polaris
• 21 MIN READ•May 22, 2026

  Common Misconceptions About Data Lakehouse and Apache Iceberg

  Addressing common search queries and reader confusion about Data Lakehouse architectures, Apache Iceberg catalogs, partitions, and lock-in.

  data lakehouseapache icebergdata engineering
• 7 MIN READ•Apr 29, 2026

  Migrating to Apache Iceberg: Strategies for Every Source System

  Migrate to Iceberg from Hive, data warehouses, or raw files using in-place migration, full rewrite, or the zero-downtime view swap pattern.

  migrating to Apache IcebergHive to Iceberg migrationIceberg migration strategy
• 7 MIN READ•Apr 29, 2026

  Hands-On with Apache Iceberg Using Dremio Cloud

  A practical walkthrough of creating, querying, and optimizing Iceberg tables on Dremio Cloud, from account setup to AI-powered analytics.

  Dremio Cloud Apache IcebergDremio Cloud tutorialIceberg hands-on
• 7 MIN READ•Apr 29, 2026

  Approaches to Streaming Data into Apache Iceberg Tables

  Stream data into Iceberg with Spark Structured Streaming, Flink, or Kafka Connect. Here is how each works and the trade-offs between latency and maintenance.

  streaming to Apache IcebergSpark Structured Streaming IcebergFlink Iceberg sink
• 7 MIN READ•Apr 29, 2026

  Using Apache Iceberg with Python and MPP Query Engines

  Access Iceberg tables from Python with PyIceberg, DuckDB, and Polars, or through MPP engines like Dremio, Spark, and Trino. Here is how each approach works.

  Python Apache IcebergPyIcebergDuckDB Iceberg
• 7 MIN READ•Apr 29, 2026

  Apache Iceberg Metadata Tables: Querying the Internals

  Iceberg metadata tables let you query snapshots, files, manifests, and partitions using SQL. Here is every metadata table and how to use them.

  Iceberg metadata tablestable snapshotstable files
• 7 MIN READ•Apr 29, 2026

  Maintaining Apache Iceberg Tables: Compaction, Expiry, and Cleanup

  Keep Iceberg tables fast with compaction, snapshot expiry, orphan cleanup, and manifest rewriting. Here is when and how to run each operation.

  Iceberg table maintenancecompactionsnapshot expiry
• 9 MIN READ•Apr 29, 2026

  Concurrency, Isolation, and MVCC: How Engines Handle Contention

  Databases handle concurrent access using locks, MVCC, or optimistic concurrency control. Here is how each approach works and what tradeoffs each creates.

  database concurrency controlMVCC databaseisolation levels
• 7 MIN READ•Apr 29, 2026

  How Data Lake Table Storage Degrades Over Time

  Iceberg tables degrade through small files, orphan files, metadata bloat, sort order decay, and partition skew. Here is how to diagnose each problem.

  Iceberg storage degradationsmall file problemorphan files
• 8 MIN READ•Apr 29, 2026

  Hash, Sort-Merge, Broadcast: How Distributed Joins Work

  Distributed joins move data across the network using shuffle, broadcast, or co-location strategies. Here is how each works and when engines choose which.

  distributed join algorithmsshuffle joinbroadcast join
• 7 MIN READ•Apr 29, 2026

  When Catalogs Are Embedded in Storage

  S3 Tables and MinIO AI Stor embed the Iceberg catalog directly in the storage layer. Here is when embedded catalogs make sense and when they do not.

  embedded Iceberg catalogS3 TablesMinIO AI Stor