Medallion Lakehouse Architecture

Replace {{PLACEHOLDERS}} with your environment values, then deploy to your stack.

# Medallion Lakehouse Architecture Framework

**Version:** {{FRAMEWORK_VERSION}}  
**Owner:** {{DATA_PLATFORM_TEAM}}  
**Last Updated:** {{LAST_UPDATED_DATE}}  
**Platforms:** AWS (S3 + Glue + Athena/EMR) and/or Snowflake

---

## Executive Summary

This framework defines the **Medallion Architecture** (Bronze → Silver → Gold) for {{ORGANIZATION_NAME}}'s lakehouse. It standardizes zone definitions, ingestion patterns, transformation rules, consumption interfaces, and SLAs so teams build consistent, governable data pipelines across {{PRIMARY_CLOUD}} and {{SNOWFLAKE_ACCOUNT}}.

---

## 1. Architecture Overview

```
                    SOURCES
    ┌──────────┬──────────┬──────────┬──────────┐
    │  OLTP/   │  SaaS    │  Events  │  Files   │
    │  CDC     │  APIs    │  Streams │  Batch   │
    └────┬─────┴────┬─────┴────┬─────┴────┬─────┘
         │          │          │          │
         ▼          ▼          ▼          ▼
    ┌─────────────────────────────────────────────┐
    │              BRONZE (Raw)                   │
    │  Immutable landing, source-faithful         │
    └─────────────────────┬───────────────────────┘
                          │ cleanse, dedupe, conform
                          ▼
    ┌─────────────────────────────────────────────┐
    │              SILVER (Curated)               │
    │  Entity-aligned, conformed dimensions       │
    └─────────────────────┬───────────────────────┘
                          │ aggregate, business rules
                          ▼
    ┌─────────────────────────────────────────────┐
    │              GOLD (Analytics-Ready)         │
    │  Data products, marts, features             │
    └─────────────────────┬───────────────────────┘
                          │
         ┌────────────────┼────────────────┐
         ▼                ▼                ▼
      BI / SQL         ML / Features    APIs / Reverse ETL
```

---

## 2. Zone Definitions

### 2.1 Bronze Layer

| Attribute | Standard |
|-----------|----------|
| **Purpose** | Persist raw source data exactly as received (plus metadata) |
| **Mutability** | Append-only; no in-place updates |
| **Retention** | {{BRONZE_RETENTION_DAYS}} days (regulatory holds override) |
| **Format** | JSON, CSV, Avro, or Parquet as landed; Iceberg optional |
| **Schema** | Source schema + `_ingested_at`, `_source_file`, `_run_id` |
| **Quality bar** | Schema validation only; no business rules |
| **Access** | Domain producers + platform; limited consumer access |

**AWS path:** `s3://{{LAKE_BUCKET}}/bronze/{source_system}/{entity}/`  
**Snowflake:** `{{BRONZE_DB}}.{SOURCE_SCHEMA}.{ENTITY}_RAW`

### 2.2 Silver Layer

| Attribute | Standard |
|-----------|----------|
| **Purpose** | Cleaned, deduplicated, conformed entity tables |
| **Mutability** | Merge/upsert (SCD Type 1/2 as designed) |
| **Retention** | {{SILVER_RETENTION_DAYS}} days active; archive to Glacier if needed |
| **Format** | **Apache Iceberg** (AWS) or native Snowflake tables |
| **Schema** | Canonical entity model per {{ENTERPRISE_DATA_MODEL}} |
| **Quality bar** | Critical + High DQ rules must pass |
| **Access** | Domain teams + approved cross-domain via governance |

**AWS path:** `s3://{{LAKE_BUCKET}}/silver/{domain}/{entity}/`  
**Snowflake:** `{{SILVER_DB}}.{DOMAIN_SCHEMA}.{ENTITY}`

// ... download full template for remaining code