Skip to main content

Publish Command

Publish transformed data assets to external destinations (Reverse ETL). Moves data from your warehouse to operational systems like Salesforce, HubSpot, or other databases.

Overview​

The msh publish command exports an asset's currently active production version to an external destination. It uses the Blue/Green deployment hash to identify the correct table version, ensuring you're always publishing the live production data.

Usage​

msh publish <asset_name> --to <destination>

Arguments:

  • asset_name: Name of the asset to publish (must be deployed)

Options:

  • --to <destination>: Required. Target destination (e.g., salesforce, hubspot, postgres)

How It Works​

  1. Finds Active Version: Queries the warehouse to find the currently active Blue/Green hash for the asset
  2. Constructs Table Name: Uses the pattern model_{asset_name}_{active_hash}
  3. Exports Data: Calls the export engine to transfer data to the target destination

Security: Credentials Configuration​

Important: All destination credentials are read from environment variables in your .env file.

Format:

DESTINATION__<TARGET>__CREDENTIALS="<connection_string_or_config>"

Examples:

# Salesforce
DESTINATION__SALESFORCE__CREDENTIALS='{"username": "user@example.com", "password": "pass", "security_token": "token"}'

# HubSpot
DESTINATION__HUBSPOT__CREDENTIALS='{"api_key": "your-api-key"}'

# Postgres
DESTINATION__POSTGRES__CREDENTIALS="postgresql://user:pass@host:5432/dbname"

Security Best Practices:

  • Never commit .env files to version control
  • Use secret management tools in production (AWS Secrets Manager, HashiCorp Vault, etc.)
  • Rotate credentials regularly

Examples​

Publish to Salesforce​

msh publish customers --to salesforce

Output:

Publishing customers to salesforce...
Found active table: model_customers_a1b2c3
[OK] Exported 1,245 rows to Salesforce Contact object

Publish to HubSpot​

msh publish leads --to hubspot

Output:

Publishing leads to hubspot...
Found active table: model_leads_d4e5f6
[OK] Exported 892 rows to HubSpot Contacts

Publish to Postgres​

msh publish revenue --to postgres

Output:

Publishing revenue to postgres...
Found active table: model_revenue_f7g8h9
[OK] Exported 15,234 rows to analytics.revenue table

Error Handling​

Asset Not Deployed​

If the asset hasn't been deployed yet:

Error: Could not find active hash for asset 'customers'. Is it deployed?

Solution: Run msh run customers first to deploy the asset.

Missing Credentials​

If destination credentials are not configured:

Failed to publish: Missing DESTINATION__SALESFORCE__CREDENTIALS in environment

Solution: Add the required credentials to your .env file.

Connection Errors​

If the destination is unreachable:

Failed to publish: Connection refused to salesforce.com

Solution: Check network connectivity and firewall rules.

Use Cases​

1. Sync Customer Data to CRM​

Keep your CRM up-to-date with the latest customer data:

# Run daily sync
msh run customers
msh publish customers --to salesforce

2. Export Analytics to Operational DB​

Push transformed analytics back to operational systems:

msh run revenue_report
msh publish revenue_report --to postgres

3. Update Marketing Platforms​

Sync lead data to marketing automation tools:

msh run qualified_leads
msh publish qualified_leads --to hubspot

Integration with CI/CD​

Publish commands can be integrated into deployment pipelines:

# Example GitHub Actions workflow
- name: Deploy and Publish
  run: |
    msh run customers
    msh publish customers --to salesforce
  • msh run: Deploy assets before publishing
  • msh status: Check which version is currently active
  • msh versions: View deployment history

See Also​