Util ElevenLabs Package

Overview

@zooly/util-elevenlabs is a server-side utility package that provides ElevenLabs API integration for voice cloning and text-to-speech functionality. It handles voice management operations and audio generation.

Package Details

• Package Name: @zooly/util-elevenlabs
• Location: packages/util-elevenlabs
• Type: ElevenLabs API integration library

Key Features

• Voice Cloning: Create voice clones from audio samples
• Voice Management: Add, delete, and retrieve voice information
• Text-to-Speech: Generate speech audio from text using cloned voices
• Voice Sample Creation: Create complete voice samples for likeness search (includes cloning, TTS generation, and S3 upload)
• Database Integration: High-level functions that combine ElevenLabs API calls with database persistence
• Error Handling: Comprehensive error handling for API responses

Core Functionality

Voice Operations

• addVoice - Create a voice clone in ElevenLabs from an audio blob
• deleteVoice - Delete a voice clone from ElevenLabs
• getVoice - Get voice details from ElevenLabs

Text-to-Speech

• generateVoiceForText - Generate TTS audio using an ElevenLabs voice clone
  • Supports custom model selection (default: eleven_multilingual_v2)
  • Returns MP3 audio as Buffer
  • Configurable output format

Voice Management (with Database Persistence)

High-level functions that combine ElevenLabs API operations with database persistence. These functions manage the full lifecycle of voice clones for accounts.

• createAndPersistVoice - Create a voice clone in ElevenLabs and persist it to the database

  • Uploads audio blob to ElevenLabs
  • Creates voice clone
  • Stores voice ID and settings in eleven_labs table
  • Returns persisted voice record

• createOrUpdateVoiceWithId - Create or update a voice record with an existing ElevenLabs voiceId

  • Useful for VIP voices or voices created directly in ElevenLabs
  • Optionally verifies the voice exists in ElevenLabs (default: true)
  • Persists voice ID and settings to eleven_labs table
  • Can be used to update existing voice records
  • Returns persisted voice record

• getAccountVoices - Get all voices for an account

  • Returns array of ElevenLabsVoice records from database

• getAccountVoice - Get a specific voice for an account

  • Returns voice record if found, null otherwise

• updateAccountVoiceExample - Update voice example URL

  • Updates the voiceExample field in database (typically S3 URL of TTS sample)

• deleteAccountVoice - Delete a voice clone (both from ElevenLabs and database)

  • Deletes voice from ElevenLabs API
  • Removes record from database
  • Handles errors gracefully (continues DB deletion even if API deletion fails)

• deleteAllAccountVoices - Delete all voices for an account

  • Deletes each voice from ElevenLabs API
  • Removes all records from database

• getVoiceDetails - Get voice details from ElevenLabs API

  • Fetches current voice information from ElevenLabs
  • Returns voice metadata (name, description, settings, etc.)

Voice Sample Creation (for Likeness Search)

• createVoiceSample - Create a complete voice sample for likeness search assets
  • Validates audio URL and content type
  • Fetches audio blob from URL
  • Creates ElevenLabs voice clone
  • Generates AI sample text based on voice characteristics
  • Creates TTS audio sample using the cloned voice
  • Uploads TTS sample to S3
  • Stores voiceSampleUrl in likeness_assets table
  • Persists voiceId to eleven_labs table
  • Returns voiceId and voiceSampleUrl
  • See packages/util-elevenlabs/src/createVoiceSample.ts for implementation

Dependencies

• @elevenlabs/elevenlabs-js - Official ElevenLabs JavaScript SDK
• @zooly/app-db - Database access layer
• @zooly/types - Shared types
• @zooly/util-srv - Server-side utilities (S3 operations)
• @zooly/likeness-search - Voice tag generation and audio validation utilities
• uuid - UUID generation for S3 keys

Usage

This package provides a clean interface for ElevenLabs operations. It's used by:

• Likeness Search: Creates voice samples and generates speech for voice-based likeness matching
• Voice Management: System-wide voice clone management (persona voices, marketplace previews, etc.)

Example: Create and Persist Voice

See packages/util-elevenlabs/src/voice-management.ts for createAndPersistVoice implementation.

Example: List Account Voices

See packages/util-elevenlabs/src/voice-management.ts for getAccountVoices implementation.

Example: Create/Update Voice with Existing ID (VIP Voices)

See packages/util-elevenlabs/src/voice-management.ts for createOrUpdateVoiceWithId implementation.

Example: Update Existing Voice Record

See packages/util-elevenlabs/src/voice-management.ts for createOrUpdateVoiceWithId implementation.

Example: Delete Voice

See packages/util-elevenlabs/src/voice-management.ts for deleteAccountVoice implementation.

Example: Create Voice Sample for Likeness Search

See packages/util-elevenlabs/src/createVoiceSample.ts for createVoiceSample implementation.

Environment Variables

The package requires the following environment variable:

• ELEVEN_LABS_API_KEY - ElevenLabs API key for authentication

API Features

• Voice cloning from audio blobs
• Voice metadata management
• Text-to-speech with multiple model support
• Error handling with detailed error messages
• Support for multilingual models
• Support for existing voice IDs (VIP voices, pre-created voices)
• Create and update operations for voice records