Veriprompt Docs

Appearance

Setup Sherlock Wizard ​

Get your AI cost intelligence system up and running in minutes. The Setup Sherlock wizard walks you through configuring cost tracking, waste detection, budgets, and data exports — no technical expertise required.

Overview ​

The Setup Sherlock wizard is a guided 7-step onboarding experience that configures your Sherlock cost intelligence platform based on how your organization consumes AI services. Instead of manually configuring dozens of settings, you answer a few questions about your AI usage patterns, and the wizard applies intelligent defaults across cost tracking, waste detection, sandbox budgets, and reporting.

What you get:

  • Automatic configuration based on your organization type (startup, enterprise, agency)
  • Cost attribution tailored to your tracking needs (per user, project, or workflow)
  • Waste detection with configurable alerting or blocking
  • Sandbox budgets for safe testing and experimentation
  • Data exports to your preferred format and schedule
  • Integration setup with monitoring and alerting systems

Who should use it:

  • Finance/FinOps teams setting up AI cost tracking
  • Platform teams implementing cost controls
  • Account admins configuring Sherlock for the first time
  • Anyone who needs cost visibility into AI spend

When Does the Wizard Appear? ​

First-time Access ​

When you access Sherlock for the first time (/sherlock), and no configuration exists, VeriPrompt automatically redirects you to the Setup Sherlock wizard at /sherlock/setup.

You can:

  • Complete the wizard to configure Sherlock immediately
  • Skip for now to explore Sherlock first — a reminder banner will appear on the Sherlock dashboard so you can return to the wizard later

Manual Access ​

You can always run the wizard again from:

  • The banner on the Sherlock dashboard (if you skipped setup)
  • Settings → Reconfigure Sherlock (from /admin/company-settings)
  • Directly at /sherlock/setup

When to re-run:

  • Your organization's AI usage model changes (e.g., moving from subscriptions to BYOK)
  • You expand to new regions or departments
  • You need to update cost attribution or budget policies
  • You want to add new integrations or change export settings

The 7 Steps ​

Step 1: Welcome ​

An introduction to the wizard and what it will configure. No input required — this step explains the process and what to expect.

What you'll learn:

  • The purpose of each configuration step
  • What data Sherlock will track
  • How long setup typically takes (5-10 minutes)

Action: Click Next to begin.

Step 2: AI Usage Model ​

Tell Sherlock how your organization consumes AI services so it can apply appropriate cost tracking logic.

Select your primary model:

ModelWhat It MeansExample
Fixed SubscriptionMonthly/annual flat fee for unlimited or tiered usageChatGPT Plus, Claude Pro, GitHub Copilot
Pay-Per-Call (API)Metered billing based on API usage (tokens, requests) — billed by providerOpenAI API, Anthropic API, Gemini API
Bring Your Own Keys (BYOK)You own the API keys — VeriPrompt passes requests through, you pay the provider directlyEnterprise setups where each team has their own API keys
Credit/Prepaid SystemPre-purchased credits consumed per requestEnterprise AI platforms, internal prepaid systems
Self-Hosted ModelsYou run the models on your own infrastructureOpen-source models (Llama, Mistral) on your cloud or on-prem
Mixed / HybridCombination of multiple models aboveSubscriptions for employees + BYOK for production + self-hosted for sensitive workloads

Notes field: Optional free-text to record details about your setup (e.g., "OpenAI API for production, Claude for sensitive data, GPT-4 subscription for executives").

Why this matters: The usage model determines how Sherlock attributes costs:

  • Subscription: Tracks seat-based costs and usage patterns
  • Pay-Per-Call: Tracks token/request-level costs with real-time provider pricing
  • BYOK: Tracks passthrough costs; you see what your teams spent via your keys
  • Mixed: Enables multi-model tracking with separate attribution per source

Action: Select your model and click Next.

Step 3: Cost Tracking & Attribution ​

Configure how Sherlock organizes and attributes your AI spend.

Cost Attribution Preference ​

Choose your primary attribution dimension — this determines the default grouping in cost reports and dashboards.

PreferenceWhat It TracksBest For
User-PrimaryCosts per individual user firstChargeback, per-employee budgets, identifying high-usage individuals
Project-PrimaryCosts per project/initiativeProduct/feature cost allocation, R&D budgets, project-level ROI
Workflow-PrimaryCosts per workflow/use-caseProcess optimization, workflow ROI, identifying inefficient automations

Important: You can always pivot attribution in Sherlock dashboards (e.g., view user costs grouped by project) — this setting just determines the default view.

Waste Detection Mode ​

Choose how aggressively Sherlock should detect and act on wasteful AI usage patterns.

ModeBehaviorUse Case
Enabled (Block)Actively detect and block wasteful patternsProduction environments with strict cost controls
Alert OnlyDetect waste patterns and alert admins, but do not block requestsDevelopment/staging environments, or when you want visibility before enforcement
DisabledNo waste detection — all requests are allowedTest environments, early adoption phase

Waste patterns Sherlock detects:

  • Duplicate requests (same prompt sent multiple times in quick succession)
  • Over-provisioned models (using expensive models for simple tasks)
  • Retry storms (repeated failures to the same endpoint)
  • Token bloat (inefficient prompts with excessive context)

Waste Alerts ​

Enable email/Slack alerts when waste is detected (even in alert-only or disabled mode, you can track waste patterns for analysis).

Action: Select your attribution preference, waste detection mode, and alert settings. Click Next.

Step 4: Data Export & Reporting ​

Set up automated exports of cost data for external analytics, compliance, or integration with your finance systems.

Enable Data Export ​

Toggle on if you want automated exports; off if you only need in-app reporting.

Export Format ​

Choose the format that matches your analytics stack:

FormatBest For
CSVExcel, Google Sheets, simple reporting tools
JSONCustom scripts, APIs, data pipelines
ParquetBig data analytics (Spark, Athena, BigQuery)
XLSXFinance teams, enterprise reporting with formatting

Export Frequency ​

FrequencyWhen Exports Run
DailyEvery day at midnight UTC
WeeklyEvery Monday at midnight UTC
Monthly1st of each month at midnight UTC
Manual OnlyNo automated exports — use the UI export button when needed

Data Inclusion Options ​

  • Include User Data: Include user-level attribution (names, emails, roles) in exports. Disable for privacy-sensitive environments.
  • Include PII-Sanitized Data: Include sanitized prompts/responses (PII tokens removed). Useful for compliance audits.

Export Destination (Optional) ​

Configure a cloud storage destination for automated exports:

  • S3 Bucket: AWS S3 bucket name (e.g., my-company-sherlock-exports)
  • GCS Bucket: Google Cloud Storage bucket (e.g., gs://my-company-sherlock-exports)
  • Azure Container: Azure Blob Storage container

If no destination is configured, exports are stored locally in VeriPrompt and available for download from the UI.

Action: Configure export settings and click Next. You can skip this step if you only need in-app reporting.

Step 5: Test/Sandbox Budgets ​

Set up safe testing budgets for developers and teams to experiment with AI without impacting production costs.

Enable Sandbox Budgets ​

Toggle on to create sandbox budgets for testing environments.

Default Sandbox Amount (USD) ​

The default budget amount for each new sandbox (e.g., $10, $50, $100). Teams can request increases, but this is the starting amount.

Expiration Period (Days) ​

How long sandbox budgets remain valid before expiring (e.g., 7 days, 30 days). After expiration, unused budgets are reclaimed.

Why expiration matters: Sandbox budgets prevent cost runaway in test environments. A 7-day budget is typical for feature testing; 30 days works for longer R&D projects.

Auto-Create for New Users ​

  • Enabled: Every new user automatically gets a sandbox budget
  • Disabled: Admins manually create sandbox budgets per team/project

Warning: Auto-creating sandbox budgets can add up if you onboard many users. Enable this only if your onboarding volume is predictable.

Use cases:

  • Startup: Enable auto-create with $10 / 7 days — developers can test freely without asking for budgets
  • Enterprise: Disable auto-create — FinOps team assigns sandbox budgets per approved project
  • Agency: Disable — client projects get their own budgets, not individual developers

Action: Configure sandbox budget settings and click Next. You can skip this step if you don't need test budgets.

Step 6: Integration Setup ​

Connect Sherlock to your monitoring, alerting, and analytics tools so cost data flows where your teams already work.

Available Integrations ​

IntegrationWhat It Does
SlackSend cost alerts and daily/weekly reports to Slack channels
Microsoft TeamsSend cost alerts and reports to Teams channels
DatadogExport cost metrics to Datadog for unified monitoring and dashboards
PrometheusExpose cost metrics via Prometheus endpoint for scraping
Custom WebhookSend cost data to any custom webhook endpoint (JSON payload)

Configuring an Integration ​

For each integration you enable:

  1. Select the integration type (e.g., Slack)
  2. Enable the integration
  3. Provide required credentials:
    • Slack/Teams: Webhook URL
    • Datadog: API Key
    • Prometheus: Endpoint URL (auto-generated)
    • Custom Webhook: Endpoint URL and optional auth headers

Example: Slack Integration

  • Create a Slack incoming webhook at https://api.slack.com/messaging/webhooks
  • Paste the webhook URL (e.g., https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXX)
  • Sherlock will send daily cost summaries and alerts to your #finance-ai channel

Action: Add integrations as needed and click Next. You can skip this step if you only need in-app reporting.

Step 7: Review & Launch ​

Review your configuration and apply it to your organization.

What You'll See ​

A summary of all your selections:

  • AI usage model
  • Cost attribution and waste detection settings
  • Data export configuration (if enabled)
  • Sandbox budget policy (if enabled)
  • Configured integrations (if any)

Actions ​

  • Export Configuration as JSON: Download your configuration for documentation or to apply to another environment
  • Apply Configuration: Save all settings and launch Sherlock with your configuration

What Happens When You Click "Apply":

  1. Sherlock updates your company settings with the selected AI usage model and cost attribution preference
  2. Waste detection rules are applied company-wide
  3. If data export is enabled, an automated export template is created with your schedule
  4. If sandbox budgets are enabled, the default budget policy is saved (budgets are created on-demand, not in bulk)
  5. All integrations are enabled and credentials are stored securely
  6. You are redirected to the Sherlock dashboard at /sherlock

Action: Click Apply Configuration to complete setup.

Template Presets (Quick Start) ​

If you want to skip custom configuration, the wizard offers three template presets based on common organization types. These apply opinionated defaults across all steps.

Startup Template ​

Best for: Small teams (5-50 people), pay-per-call API usage, lean budgets.

What it configures:

  • AI Usage Model: Pay-Per-Call (API)
  • Cost Attribution: User-Primary (track per-person spend)
  • Waste Detection: Alert Only (visibility without blocking)
  • Data Export: Weekly CSV exports (disabled by default)
  • Sandbox Budgets: $10 per user, 7-day expiration, auto-create for new users
  • Integrations: None (manual reporting)

Use case: A startup using OpenAI/Anthropic APIs wants to track which team members are spending the most on AI, get alerts if someone runs up costs, and give each developer a $10 weekly sandbox for testing.

Enterprise Template ​

Best for: Large organizations (500+ people), mixed AI models, strict governance.

What it configures:

  • AI Usage Model: Mixed / Hybrid
  • Cost Attribution: Project-Primary (track per-initiative spend)
  • Waste Detection: Enabled (Block) — prevent wasteful patterns
  • Data Export: Daily Parquet exports to S3/GCS for analytics
  • Sandbox Budgets: $50 per project, 30-day expiration, manual approval
  • Integrations: Datadog + Slack alerts

Use case: A Fortune 500 using subscriptions, BYOK, and self-hosted models wants project-level cost allocation, automated daily exports to their data lake, strict waste blocking, and Datadog dashboards for cross-team visibility.

Agency Template ​

Best for: Agencies billing clients for AI usage, BYOK for client isolation.

What it configures:

  • AI Usage Model: Bring Your Own Keys (BYOK)
  • Cost Attribution: Project-Primary (track per-client spend)
  • Waste Detection: Alert Only (client-facing, non-blocking)
  • Data Export: Monthly CSV exports with PII-sanitized data (for client invoices)
  • Sandbox Budgets: Disabled (client projects get real budgets)
  • Integrations: None (manual client reporting)

Use case: A digital agency where each client brings their own OpenAI key wants per-client cost tracking, monthly billing exports, and no sandbox budgets (client projects use real budgets from day one).

After Setup ​

What Gets Created ​

Once the wizard completes, the following resources are created in your VeriPrompt organization:

ResourceDescription
Company SettingssherlockSetupComplete = true, AI usage model and cost attribution preference saved
Waste Detection RulesActive if enabled; monitored if alert-only
Export TemplateAutomated export schedule (if data export is enabled)
Sandbox Budget PolicyDefault sandbox amount and expiration (budgets created on-demand, not upfront)
IntegrationsConfigured webhooks/API keys for external systems

Next Steps ​

After completing the wizard, you can:

  1. Explore the Sherlock dashboard at /sherlock to see cost breakdowns and trends
  2. Set up budgets at /budgets for teams, projects, or workflows
  3. Review waste detection findings at /sherlock/waste (if enabled)
  4. Configure additional integrations from /admin/integrations
  5. Export cost data manually from /sherlock/exports

Common Questions ​

Can I change these settings later? ​

Yes. All settings configured in the wizard can be updated from the Sherlock settings page or by re-running the wizard at /sherlock/setup.

What if I skip the wizard? ​

If you skip the wizard, Sherlock will still function, but with no configuration — cost tracking will be generic (no attribution preference), waste detection will be off, and no integrations or exports will be set up. You can complete setup anytime from the banner on the Sherlock dashboard.

Do I need to configure everything in the wizard? ​

No. You can skip optional steps (data export, sandbox budgets, integrations) and still complete setup. Only Step 2 (AI Usage Model) and Step 3 (Cost Tracking) are required.

What happens if I run the wizard again? ​

Re-running the wizard updates your existing configuration — it does not delete previous data (cost history, budgets, etc.). For example:

  • Changing cost attribution preference updates the default view, but historical data remains
  • Enabling waste detection after setup applies rules going forward, not retroactively
  • Updating sandbox budget amounts only affects new budgets, not existing ones

Can I export my configuration? ​

Yes. On the Review & Launch step, click Export configuration as JSON to download a complete config file. This is useful for:

  • Documenting your setup for compliance/audit
  • Applying the same configuration to a different VeriPrompt environment (e.g., staging → production)
  • Sharing your setup with support for troubleshooting

Who can run the wizard? ​

The wizard requires cost viewer access — typically Account Owners, Account Admins, or users with the Cost Controller or Sherlock Analyst access profile. Standard users cannot run the wizard (they would not have permission to modify company-wide cost settings).

What if I need to configure different settings per department? ​

The wizard configures company-wide defaults. For per-department customization:

  • Set the company-wide default in the wizard (e.g., user-primary attribution, alert-only waste detection)
  • Override settings per department/project from /admin/budgets or /admin/projects

For example: company default is "user-primary attribution," but the R&D department can have "project-primary attribution" for their budgets.

Ready to get started? Head to /sherlock/setup or wait for the automatic redirect on your first Sherlock visit.