Skip to main content

Migration Guide

CLOUD

Migrate from OSS (Open Source) msh to Cloud Platform.

Overview​

If you're using the OSS version and want to migrate to Cloud Platform, follow these steps to migrate your metadata and glossary.

Step 1: Create Cloud Account​

  1. Sign up at https://app.msh.io
  2. Create a new project
  3. Generate an API token

Step 2: Export Local Metadata​

Export your local metadata:

# Generate context pack
msh ai context --json > context.json

# Export glossary
msh glossary export --json > glossary.json

Step 3: Upload Metadata​

Upload metadata to Cloud:

export MSH_API_TOKEN="your-token"
export MSH_PROJECT_ID=1

# Upload context pack
curl -X POST \
  -H "Authorization: Bearer $MSH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"context_pack\": $(cat context.json), \"source\": \"cli\"}" \
  https://api.msh.io/api/msh/metadata/projects/$MSH_PROJECT_ID/upload

Step 4: Import Glossary​

Import glossary terms:

# Read glossary JSON
GLOSSARY=$(cat glossary.json)

# Create terms
echo "$GLOSSARY" | jq -r '.terms[] | @json' | while read term; do
  curl -X POST \
    -H "Authorization: Bearer $MSH_API_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$term" \
    https://api.msh.io/api/msh/glossary/terms/
done

# Create metrics
echo "$GLOSSARY" | jq -r '.metrics[] | @json' | while read metric; do
  curl -X POST \
    -H "Authorization: Bearer $MSH_API_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$metric" \
    https://api.msh.io/api/msh/glossary/metrics/
done

Step 5: Set Up CI/CD Sync​

Set up automatic metadata sync:

# .github/workflows/msh-sync.yml
name: Sync msh Metadata

on:
  push:
    branches: [main]

jobs:
  sync-metadata:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install msh
        run: pip install msh-cli
      - name: Generate and upload metadata
        run: |
          msh ai context --json > context.json
          curl -X POST \
            -H "Authorization: Bearer ${{ secrets.MSH_API_TOKEN }}" \
            -H "Content-Type: application/json" \
            -d "{\"context_pack\": $(cat context.json), \"source\": \"webhook\"}" \
            https://api.msh.io/api/msh/metadata/projects/${{ secrets.MSH_PROJECT_ID }}/upload

Step 6: Update Workflows​

Update your workflows to use Cloud features:

# Instead of local search
msh ai explain assets/revenue.msh

# Use Cloud semantic search
curl -X POST \
  -H "Authorization: Bearer $MSH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Where is revenue calculated?",
    "project_id": '$MSH_PROJECT_ID',
    "entity_types": ["asset"]
  }' \
  https://api.msh.io/api/msh/ai/semantic-search

Use Cloud Query Console​

# Use Cloud query console for NL→SQL
curl -X POST \
  -H "Authorization: Bearer $MSH_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Show revenue by region",
    "project_id": '$MSH_PROJECT_ID',
    "warehouse_connection_id": 1
  }' \
  https://api.msh.io/api/msh/ai/query

Differences: OSS vs Cloud​

Metadata Storage​

  • OSS: Local .msh/ directory
  • Cloud: Centralized cloud storage

Glossary​

  • OSS: Local glossary.yaml file
  • Cloud: Centralized cloud glossary

AI Features​

  • OSS: Uses your API keys
  • Cloud: Cloud-hosted AI (no API keys needed)
  • OSS: Local file search
  • Cloud: Semantic search across projects

Troubleshooting​

Metadata Upload Fails​

Issue: Upload fails with authentication error

Solution: Check API token is valid and has correct permissions

Glossary Import Fails​

Issue: Terms fail to import

Solution: Check term IDs are unique and don't conflict

CI/CD Sync Fails​

Issue: Automated sync fails

Solution: Check secrets are set correctly in CI/CD platform