Data Mesh Domain Data Product Workflow

Replace {{PLACEHOLDERS}} with your environment values, then deploy to your stack.

# Data Mesh Domain Data Product Workflow

> DE Architecture · {{ORGANIZATION_NAME}}

## Purpose

Workflow for domain teams publishing certified data products in a federated data mesh on {{CLOUD_PROVIDER}}.

## Actors

- **Domain Owner** - accountable for product quality and SLA
- **Platform Team** - enables self-serve infrastructure
- **Consumer** - discovers and consumes via catalog/API

## Data Product Lifecycle

```
┌──────────┐   design    ┌──────────┐   build     ┌──────────┐   certify   ┌──────────┐
│  Intent  │ ──────────▶ │  Build   │ ──────────▶ │  Test    │ ──────────▶ │ Publish  │
│  RFC     │             │  Pipeline│             │  DQ+SLA  │             │  Catalog │
└──────────┘             └──────────┘             └──────────┘             └────┬─────┘
                                                                                 │
                                                                                 ▼
                                                                          ┌──────────┐
                                                                          │ Consume  │
                                                                          │ + Feedback│
                                                                          └──────────┘
```

## Step-by-Step Workflow

### 1. Product intent (Week 0)
- Domain submits data product RFC: grain, SLA, consumers, classification
- Platform reviews for mesh compatibility and shared infra standards
- Assign product ID: `dp_{{DOMAIN}}_{{ENTITY}}`

### 2. Build (Weeks 1-3)
- Implement pipeline in domain account `{{DOMAIN_AWS_ACCOUNT}}`
- Output to curated zone with standard naming: `{{DOMAIN}}.{{ENTITY}}_v{{VERSION}}`
- Document schema, lineage, and sample queries

### 3. Quality certification
- Pass automated checks: freshness, completeness, uniqueness, schema stability
- Domain owner sign-off on business definitions
- Security review for PII / cross-domain sharing

### 4. Publish
- Register in enterprise catalog with tags: domain, owner, SLA, classification
- Configure Lake Formation / IAM policies for consumer roles
- Enable subscription workflow for dependent domains

### 5. Operate & evolve
- Monthly SLA review; version breaking changes via deprecation policy
- Consumer feedback loop in {{FEEDBACK_CHANNEL}}

## Data Product Contract Template

```yaml
product_id: dp_{{DOMAIN}}_{{ENTITY}}
owner: {{DOMAIN_OWNER_EMAIL}}
sla:
  freshness_hours: {{FRESHNESS_HOURS}}
  availability_pct: 99.5
schema:
  grain: one row per {{GRAIN}}
  primary_key: [{{PK_COLUMN}}]
interfaces:
  - type: table
    location: {{CATALOG}}.{{SCHEMA}}.{{TABLE}}
  - type: api
    endpoint: {{API_ENDPOINT}}
```

## Federation Controls

- Cross-account LF grants managed by platform
- No direct S3 access for consumers - catalog + IAM only
- Audit all access via CloudTrail / {{AUDIT_LOG}}