Skip to main content

Context Packs

Context packs are AI-ready bundles of project information used by AI commands to understand your project structure and generate accurate responses.

Purpose​

Context packs provide AI providers with:

  • Project structure and configuration
  • Asset metadata and schemas
  • Lineage relationships
  • Business glossary terms
  • Test definitions
  • Metrics and policies

Structure​

A context pack contains:

{
  "project": {
    "id": "my-project",
    "name": "My Project",
    "warehouse": "postgres",
    "default_schema": "public"
  },
  "assets": [
    {
      "id": "revenue",
      "path": "assets/revenue.msh",
      "blocks": {
        "ingest": {...},
        "transform": {...},
        "contract": {...}
      },
      "schema": {
        "columns": [
          {"name": "customer_id", "type": "integer"},
          {"name": "month", "type": "date"},
          {"name": "monthly_revenue", "type": "decimal"}
        ]
      }
    }
  ],
  "lineage": {
    "edges": [
      {
        "from": "stg_orders",
        "to": "revenue",
        "type": "transform"
      }
    ]
  },
  "glossary_terms": [
    {
      "id": "term.customer",
      "name": "Customer",
      "description": "A customer entity",
      "linked_assets": ["revenue"],
      "linked_columns": ["revenue.customer_id"]
    }
  ],
  "schemas": {...},
  "tests": [...],
  "metrics": [...],
  "policies": [...]
}

Generating Context Packs​

Project-Level Context Pack​

Generate context pack for entire project:

msh ai context

Asset-Focused Context Pack​

Generate context pack focused on specific asset:

msh ai context --asset revenue

Includes:

  • Target asset
  • Upstream dependencies
  • Downstream dependencies

With Additional Data​

Include tests and history:

msh ai context --asset revenue --include-tests --include-history

JSON Output​

Output as JSON for custom workflows:

msh ai context --asset revenue --json > context.json

Use Cases​

AI Commands​

Context packs are automatically generated and used by AI commands:

# Context pack generated automatically
msh ai explain assets/revenue.msh

# Context pack generated automatically
msh ai review assets/revenue.msh

Custom AI Workflows​

Generate context pack for custom AI workflows:

# Generate context pack
msh ai context --asset revenue --json > context.json

# Use in custom script
python my_ai_script.py context.json

Project Analysis​

Understand project structure:

# Generate context pack
msh ai context --json | jq '.assets | length'

# Analyze dependencies
msh ai context --json | jq '.lineage.edges'

Optimization​

Token Limits​

Context packs are optimized for AI token limits:

  • Relevant data only: Only necessary metadata included
  • Truncation: Large SQL queries are truncated
  • Summarization: Complex structures are summarized

PII Masking​

PII columns are masked based on glossary policies:

{
  "columns": [
    {"name": "customer_id", "type": "integer"},
    {"name": "email", "type": "string", "masked": true}
  ]
}

Focused Context​

Asset-focused context packs include only relevant assets:

# Only includes revenue and dependencies
msh ai context --asset revenue

Contents​

Project Information​

  • Project ID, name, warehouse, default schema
  • Configuration settings

Assets​

  • Asset metadata (ID, path, blocks)
  • Schema information
  • Transform SQL (truncated if large)

Lineage​

  • Upstream dependencies
  • Downstream dependencies
  • Dependency types (transform, reference)

Glossary Terms​

  • Business terms
  • Metrics
  • Dimensions
  • Policies

Schemas​

  • Flattened view of schemas per asset
  • Column types and constraints

Tests (Optional)​

  • Test definitions
  • Latest test statuses

History (Optional)​

  • Recent run history
  • Deployment history

Best Practices​

Keep Glossary Updated​

Link glossary terms for better context:

msh glossary link-term "Customer" --asset revenue --column customer_id

Generate After Changes​

Regenerate context pack after project changes:

# After adding assets
msh manifest
msh ai context

# After updating glossary
msh glossary add-term "Revenue" --description "..."
msh ai context

Use Asset Focus​

Use asset-focused context packs for better results:

# Better than project-level for specific asset
msh ai context --asset revenue