SuperCharged-Claude-Code-Upgrade/dexto/agents/music-agent

Music Creator Agent

A comprehensive AI agent for music creation, editing, and audio processing using the Music Creator MCP Server.

⚠️ Experimental Status: This agent is currently in experimental development. The tools have not been extensively tested in production environments and may have limitations or bugs. We're actively seeking feedback and improvements from users.

🧪 Experimental Features

• Limited Testing: Tools have been tested in controlled environments but may behave differently with various audio formats, file sizes, or system configurations
• Active Development: Features are being refined based on user feedback and real-world usage
• Feedback Welcome: We encourage users to report issues, suggest improvements, and share use cases
• Breaking Changes: API and tool behavior may change as we improve the implementation

Overview

This agent provides access to professional-grade music production tools through a clean conversational interface. Built with industry-standard libraries like librosa, pydub, and music21, it offers comprehensive audio processing capabilities using the published truffle-ai-music-creator-mcp package.

Features

🎵 Audio Analysis

• Tempo Detection: Automatically detect BPM and beat positions
• Key Detection: Identify musical key and mode
• Spectral Analysis: Analyze frequency spectrum, MFCC features, and audio characteristics
• Comprehensive Analysis: Get detailed audio information including duration, sample rate, and format

🎼 Music Generation

• Melody Creation: Generate melodies in any key and scale
• Chord Progressions: Create chord progressions using Roman numeral notation
• Drum Patterns: Generate drum patterns for rock, jazz, and funk styles
• MIDI Export: All generated music exports to MIDI format for further editing

🔊 Audio Processing

• Format Conversion: Convert between MP3, WAV, FLAC, OGG, M4A, AIFF, WMA
• Volume Control: Adjust audio levels with precise dB control
• Audio Normalization: Normalize audio to target levels
• Audio Trimming: Cut audio to specific time ranges
• Audio Effects: Apply reverb, echo, distortion, and filters

🎚️ Mixing & Arrangement

• Audio Merging: Combine multiple audio files with crossfade support
• Multi-track Mixing: Mix multiple audio tracks with individual volume control
• Batch Processing: Process multiple files with the same operation

Quick Start

Prerequisites

• Node.js 18+: For the Dexto framework
• Python 3.10+: Automatically managed by the MCP server
• FFmpeg: For audio processing (optional, but recommended)

Installation

1. Install FFmpeg (recommended):

   # macOS
   brew install ffmpeg
   
   # Ubuntu/Debian
   sudo apt update && sudo apt install ffmpeg
   

2. Run the Agent:

   # From the project root
   dexto --agent agents/music-agent/music-agent.yml
   

That's it! The MCP server will be automatically downloaded and installed via uvx on first run.

Usage Examples

Audio Analysis

"Analyze the tempo and key of my song.mp3"
"What's the BPM of this track?"
"What key is this song in?"

Music Generation

"Create a melody in G major at 140 BPM for 15 seconds"
"Create a I-IV-V-I chord progression in D major"
"Create a basic rock drum pattern"

Audio Processing

"Convert my song.wav to MP3 format"
"Convert my MIDI melody to WAV format"
"Increase the volume of my vocals by 3dB"
"Normalize my guitar track to -18dB"
"Trim my song from 30 seconds to 2 minutes"

Audio Effects

"Add reverb to my guitar with 200ms reverb time"
"Add echo to my vocals with 500ms delay and 0.7 decay"
"Add some distortion to my bass track"

Mixing & Playback

"Mix my vocals, guitar, and drums together with the vocals at +3dB"
"Mix a MIDI melody with an MP3 drum loop"
"Create a melody in G major and play it for 5 seconds"
"Play my song.mp3 starting from 30 seconds for 10 seconds"

Available Tools

Music Generation

• create_melody - Generate melodies in any key and scale
• create_chord_progression - Create chord progressions using Roman numerals
• create_drum_pattern - Generate drum patterns for different styles

Audio Analysis

• analyze_audio - Comprehensive audio analysis
• detect_tempo - Detect BPM and beat positions
• detect_key - Identify musical key and mode
• get_audio_info - Get detailed audio file information
• get_midi_info - Get detailed MIDI file information

Audio Processing

• convert_audio_format - Convert between audio formats
• convert_midi_to_audio - Convert MIDI files to high-quality audio format (WAV, 44.1kHz, 16-bit)
• adjust_volume - Adjust audio levels in dB
• normalize_audio - Normalize audio to target levels
• trim_audio - Cut audio to specific time ranges
• apply_audio_effect - Apply reverb, echo, distortion, filters

Mixing & Arrangement

• merge_audio_files - Combine multiple audio files
• mix_audio_files - Mix tracks with individual volume control (supports both audio and MIDI files)

Playback

• play_audio - Play audio files with optional start time and duration
• play_midi - Play MIDI files with optional start time and duration

Utility

• list_available_effects - List all audio effects
• list_drum_patterns - List available drum patterns

Supported Formats

Audio Formats

• MP3: Most common compressed format
• WAV: Uncompressed high-quality audio
• FLAC: Lossless compressed audio
• OGG: Open-source compressed format
• M4A: Apple's compressed format
• AIFF: Apple's uncompressed format
• WMA: Windows Media Audio

MIDI Formats

• MID: Standard MIDI files
• MIDI: Alternative MIDI extension

Configuration

Agent Configuration

The agent is configured to use the published MCP server:

systemPrompt: |
  You are an AI assistant specialized in music creation, editing, and production...

mcpServers:
  music_creator:
    type: stdio
    command: uvx
    args:
      - truffle-ai-music-creator-mcp
    connectionMode: strict

llm:
  provider: openai
  model: gpt-5-mini
  apiKey: $OPENAI_API_KEY

Environment Variables

Set your OpenAI API key:

export OPENAI_API_KEY="your-api-key-here"

Or create a .env file in the project root:

OPENAI_API_KEY=your-api-key-here

Use Cases

Music Production

• Create backing tracks and accompaniments
• Generate drum patterns for different genres
• Compose melodies and chord progressions
• Mix and master audio tracks

Audio Editing

• Clean up audio recordings
• Normalize volume levels
• Apply professional effects
• Convert between formats

Music Analysis

• Analyze existing music for tempo and key
• Extract musical features for machine learning
• Study musical patterns and structures
• Compare different audio files

Educational

• Learn about musical theory through generation
• Study different musical styles and patterns
• Experiment with composition techniques
• Understand audio processing concepts

MCP Server

This agent uses the Music Creator MCP Server, which is maintained separately at:

🔗 https://github.com/truffle-ai/mcp-servers/tree/main/src/music

The MCP server repository provides:

• Complete technical documentation
• Development and contribution guidelines
• Server implementation details
• Advanced configuration options

Troubleshooting

Common Issues

1. Server Installation

The MCP server will be automatically installed via uvx on first run. No manual setup required.

2. "FFmpeg not found" warnings

These warnings can be safely ignored. The agent includes fallback methods using librosa and soundfile for audio processing when FFmpeg is not available.

# Optional: Install FFmpeg for optimal performance
brew install ffmpeg  # macOS
sudo apt install ffmpeg  # Ubuntu/Debian

3. Large Audio Files

Consider trimming or converting to smaller formats for faster processing.

4. Memory Usage

Monitor system memory during heavy audio operations.

Performance Tips

1. Large Audio Files: Consider trimming or converting to smaller formats for faster processing
2. Memory Usage: Monitor system memory during heavy audio operations
3. Batch Processing: Use batch operations for multiple files to improve efficiency
4. FFmpeg: Install FFmpeg for optimal audio processing performance (optional - fallback methods available)

Technical Details

Dependencies

The MCP server uses industry-standard libraries:

• librosa: Audio analysis and music information retrieval
• pydub: Audio file manipulation and processing
• music21: Music notation and analysis
• pretty_midi: MIDI file handling
• numpy: Numerical computing
• scipy: Scientific computing
• matplotlib: Plotting and visualization

Architecture

The agent uses a Python-based MCP server that provides:

• Fast audio processing with optimized libraries
• Memory-efficient handling of large audio files
• Thread-safe operations for concurrent processing
• Comprehensive error handling and validation

Performance

• Supports audio files up to several hours in length
• Efficient processing of multiple file formats
• Optimized algorithms for real-time analysis
• Minimal memory footprint for batch operations

Getting Help

• MCP Server Issues: Report at the mcp-servers repository
• Agent Configuration: Report at the main Dexto repository
• Feature Requests: Use the mcp-servers repository for tool-related requests

License

This agent configuration is part of the Dexto AI Agent framework. The MCP server is distributed under the MIT license.