Mandatum

Appearance

Versioning ​

Git-like version control for prompts and integration code.

How Versioning Works ​

Every time you update a prompt or integration, Mandatum automatically creates a new version. This allows you to:

  • Track changes over time
  • Rollback to previous versions
  • Compare versions side-by-side
  • Pin specific versions in production
  • Test new versions before deploying

Version Numbers ​

Versions are numbered sequentially starting from 1:

v1 → v2 → v3 → v4 (latest)

Each version includes:

  • Complete prompt template
  • Model configuration
  • Parameters (temperature, max_tokens, etc.)
  • Integration code (if any)
  • Timestamp and author

Creating Versions ​

Automatic Versioning ​

Versions are created automatically on every update:

python
from mandatum import Mandatum

client = Mandatum(api_key="your-api-key")

# Create prompt (v1)
client.prompts.create(
    name="greeting",
    template="Hello {name}!"
)

# Update prompt (creates v2)
client.prompts.update(
    name="greeting",
    template="Hi {name}! How are you?"
)

# Update again (creates v3)
client.prompts.update(
    name="greeting",
    template="Hey {name}! What's up?"
)

Manual Version Tags ​

Tag important versions:

python
# Tag a version for easy reference
client.prompts.tag_version(
    prompt_name="greeting",
    version=2,
    tag="stable"
)

# Tag current version as production
client.prompts.tag_version(
    prompt_name="greeting",
    version=3,
    tag="production"
)

# Run tagged version
response = client.prompts.run(
    prompt_name="greeting",
    version_tag="stable",
    input_variables={"name": "Alice"}
)

Viewing Version History ​

Via Dashboard ​

  1. Navigate to your prompt
  2. Click Version History tab
  3. View all versions with diffs

Version History

Via API ​

python
# List all versions
versions = client.prompts.list_versions(
    prompt_name="greeting"
)

for version in versions:
    print(f"v{version.number}: {version.created_at} by {version.author}")
    print(f"  Template: {version.template[:50]}...")
    print(f"  Model: {version.model}")

Get Specific Version ​

python
# Get version details
version = client.prompts.get_version(
    prompt_name="greeting",
    version=2
)

print(f"Template: {version.template}")
print(f"Model: {version.model}")
print(f"Parameters: {version.parameters}")

Comparing Versions ​

Side-by-Side Comparison ​

Compare two versions:

python
# Compare versions
diff = client.prompts.compare_versions(
    prompt_name="greeting",
    version_a=1,
    version_b=2
)

print("Changes:")
print(f"  Template: {diff.template_changed}")
print(f"  Model: {diff.model_changed}")
print(f"  Parameters: {diff.parameters_changed}")

# View diff
print("\nTemplate diff:")
for line in diff.template_diff:
    print(f"  {line}")

Diff Format ​

diff
- Hello {name}!
+ Hi {name}! How are you?

Running Specific Versions ​

By Version Number ​

python
# Run specific version
response = client.prompts.run(
    prompt_name="greeting",
    version=2,
    input_variables={"name": "Alice"}
)

By Tag ​

python
# Run tagged version
response = client.prompts.run(
    prompt_name="greeting",
    version_tag="production",
    input_variables={"name": "Alice"}
)

Latest Version (Default) ​

python
# Run latest version (default)
response = client.prompts.run(
    prompt_name="greeting",
    input_variables={"name": "Alice"}
)

Rollback ​

Rollback to Previous Version ​

Rollback creates a new version with previous content:

python
# Rollback to version 2
client.prompts.rollback(
    prompt_name="greeting",
    version=2
)

# This creates a new version (v4) with content from v2
# Version history: v1 → v2 → v3 → v4 (rollback to v2)

Rollback with Message ​

python
# Rollback with description
client.prompts.rollback(
    prompt_name="greeting",
    version=2,
    message="Reverted due to performance issues in v3"
)

Integration Code Versioning ​

Integration code is versioned alongside prompts:

python
# Create integration (v1)
client.integrations.create(
    prompt_name="classifier",
    code=code_v1
)

# Update integration (creates v2)
client.integrations.update(
    prompt_name="classifier",
    code=code_v2
)

# Run specific integration version
response = client.prompts.run(
    prompt_name="classifier",
    version=2,  # Uses integration v2
    input_variables={...}
)

Best Practices ​

Semantic Versioning ​

Use tags to mark major versions:

python
# Mark breaking changes
client.prompts.tag_version(
    prompt_name="classifier",
    version=5,
    tag="v2.0.0"  # Breaking change
)

# Mark minor updates
client.prompts.tag_version(
    prompt_name="classifier",
    version=6,
    tag="v2.1.0"  # New feature
)

# Mark patches
client.prompts.tag_version(
    prompt_name="classifier",
    version=7,
    tag="v2.1.1"  # Bug fix
)

Pin Production Versions ​

Always pin specific versions in production:

python
# Good: explicit version
response = client.prompts.run(
    prompt_name="classifier",
    version=5,  # Pinned to v5
    input_variables={...}
)

# Risky: latest version (may change unexpectedly)
response = client.prompts.run(
    prompt_name="classifier",
    input_variables={...}
)

Test Before Deploying ​

Test new versions before tagging as production:

python
# Test new version
test_response = client.prompts.run(
    prompt_name="classifier",
    version=6,  # New version
    input_variables={"message": "test"},
    metadata={"environment": "testing"}
)

# If successful, tag as production
if test_response.success:
    client.prompts.tag_version(
        prompt_name="classifier",
        version=6,
        tag="production"
    )

Document Changes ​

Add descriptions to major versions:

python
client.prompts.update(
    prompt_name="classifier",
    template=new_template,
    description="""
    v6 Changes:
    - Improved classification accuracy
    - Added support for multi-language
    - Reduced token usage by 30%
    """
)

Version Cleanup ​

Archive old versions:

python
# Archive versions older than 90 days
client.prompts.archive_old_versions(
    prompt_name="classifier",
    older_than_days=90,
    keep_tagged=True  # Keep tagged versions
)

Version Metadata ​

Track metadata with each version:

python
# Get version metadata
version = client.prompts.get_version(
    prompt_name="classifier",
    version=5
)

print(f"Author: {version.author}")
print(f"Created: {version.created_at}")
print(f"Request count: {version.request_count}")
print(f"Average cost: ${version.avg_cost_usd:.4f}")
print(f"Error rate: {version.error_rate:.2%}")

A/B Testing Versions ​

Test multiple versions simultaneously:

python
# Create A/B test
test = client.evaluations.create_ab_test(
    prompt_name="greeting",
    versions=[2, 3],
    traffic_split=[0.5, 0.5],  # 50/50 split
    metadata={"experiment": "greeting_tone_test"}
)

# Run test
response = client.prompts.run(
    prompt_name="greeting",
    ab_test_id=test.id,
    input_variables={"name": "Alice"}
)

# View results
results = client.evaluations.get_ab_test_results(test_id=test.id)
print(f"Version 2: {results.version_2.avg_latency_ms}ms, {results.version_2.success_rate:.1%}")
print(f"Version 3: {results.version_3.avg_latency_ms}ms, {results.version_3.success_rate:.1%}")

Exporting Version History ​

Export version history for analysis:

python
# Export to CSV
client.prompts.export_version_history(
    prompt_name="classifier",
    format="csv",
    output_file="classifier_versions.csv"
)

CSV format:

csv
version,created_at,author,model,template_preview,request_count,avg_cost_usd
1,2025-01-01T10:00:00Z,alice@example.com,gpt-4,"Classify...",1500,0.003
2,2025-01-05T14:30:00Z,bob@example.com,gpt-4-turbo,"Classify...",3200,0.002
3,2025-01-10T09:15:00Z,alice@example.com,gpt-4-turbo,"Enhanced...",5100,0.0018

Audit Trail ​

View complete audit trail:

python
# Get audit trail
audit = client.prompts.get_audit_trail(
    prompt_name="classifier",
    start_date="2025-01-01",
    end_date="2025-01-31"
)

for event in audit:
    print(f"{event.timestamp}: {event.action} by {event.user}")
    print(f"  Version: {event.version}")
    print(f"  Changes: {event.changes}")

Next Steps ​