feat: Add complete Agentic Compaction & Pipeline System
- Context Compaction System with token counting and summarization - Deterministic State Machine for flow control (no LLM decisions) - Parallel Execution Engine (up to 12 concurrent sessions) - Event-Driven Coordination via Event Bus - Agent Workspace Isolation (tools, memory, identity, files) - YAML Workflow Integration (OpenClaw/Lobster compatible) - Claude Code integration layer - Complete demo UI with real-time visualization - Comprehensive documentation and README Components: - agent-system/: Context management, token counting, subagent spawning - pipeline-system/: State machine, parallel executor, event bus, workflows - skills/: AI capabilities (LLM, ASR, TTS, VLM, image generation, etc.) - src/app/: Next.js demo application Total: ~100KB of production-ready TypeScript code
This commit is contained in:
Z User
@@ -1 +1,235 @@
|
||||
Here are all the generated files.
|
||||
# Agent System - Complete Implementation
|
||||
|
||||
A comprehensive, open-source implementation of context compaction, agent orchestration, and subagent spawning.
|
||||
|
||||
## 📦 Package Contents
|
||||
|
||||
```
|
||||
agent-system/
|
||||
├── core/
|
||||
│ ├── token-counter.ts # Token counting & management
|
||||
│ ├── summarizer.ts # Conversation summarization (uses LLM)
|
||||
│ ├── context-manager.ts # Context compaction logic
|
||||
│ ├── orchestrator.ts # Agent orchestration system
|
||||
│ └── subagent-spawner.ts # Subagent spawning mechanism
|
||||
├── agents/
|
||||
│ ├── base-agent.ts # Base agent class
|
||||
│ └── task-agent.ts # Task-specific agent
|
||||
├── storage/
|
||||
│ └── memory-store.ts # Persistent file storage
|
||||
├── utils/
|
||||
│ └── helpers.ts # Utility functions
|
||||
└── index.ts # Main exports
|
||||
```
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### Installation
|
||||
|
||||
1. Copy the `agent-system` folder to your project
|
||||
2. Install dependencies:
|
||||
```bash
|
||||
bun add z-ai-web-dev-sdk
|
||||
```
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```typescript
|
||||
import {
|
||||
TokenCounter,
|
||||
ContextCompactor,
|
||||
AgentOrchestrator,
|
||||
SubagentSpawner,
|
||||
createAgent
|
||||
} from './agent-system';
|
||||
|
||||
// Token Counting
|
||||
const counter = new TokenCounter(128000);
|
||||
const result = counter.countText("Hello world");
|
||||
console.log(result.tokens); // ~3 tokens
|
||||
|
||||
// Context Compaction
|
||||
const compactor = new ContextCompactor({
|
||||
maxTokens: 100000,
|
||||
strategy: 'hybrid'
|
||||
});
|
||||
|
||||
if (compactor.needsCompaction(messages)) {
|
||||
const compacted = await compactor.compact(messages);
|
||||
console.log(`Saved ${compacted.tokensSaved} tokens`);
|
||||
}
|
||||
|
||||
// Agent Orchestration
|
||||
const orchestrator = new AgentOrchestrator();
|
||||
orchestrator.registerAgent({
|
||||
id: 'worker-1',
|
||||
name: 'Worker',
|
||||
type: 'worker',
|
||||
capabilities: ['process'],
|
||||
maxConcurrentTasks: 3,
|
||||
timeout: 60000
|
||||
});
|
||||
orchestrator.start();
|
||||
|
||||
// Subagent Spawning
|
||||
const spawner = new SubagentSpawner();
|
||||
const result = await spawner.executeWithSubagent(
|
||||
'researcher',
|
||||
'Research AI agents'
|
||||
);
|
||||
|
||||
// Create Custom Agent
|
||||
const agent = createAgent(
|
||||
'MyAgent',
|
||||
'You are a helpful assistant.'
|
||||
);
|
||||
await agent.initialize();
|
||||
const response = await agent.act('Hello!');
|
||||
```
|
||||
|
||||
## 📚 Components
|
||||
|
||||
### 1. Token Counter
|
||||
Estimates token counts using character-based approximation (~4 chars/token).
|
||||
|
||||
```typescript
|
||||
const counter = new TokenCounter(128000); // max tokens
|
||||
|
||||
// Count text
|
||||
counter.countText("text"); // { tokens, characters, words }
|
||||
|
||||
// Count conversation
|
||||
counter.countConversation(messages); // { total, breakdown[] }
|
||||
|
||||
// Check budget
|
||||
counter.getBudget(usedTokens); // { used, remaining, total, percentageUsed }
|
||||
```
|
||||
|
||||
### 2. Conversation Summarizer
|
||||
Creates intelligent summaries using LLM.
|
||||
|
||||
```typescript
|
||||
const summarizer = new ConversationSummarizer();
|
||||
|
||||
const result = await summarizer.summarize(messages, {
|
||||
maxSummaryTokens: 1000,
|
||||
extractKeyPoints: true,
|
||||
extractDecisions: true,
|
||||
extractActionItems: true
|
||||
});
|
||||
// result.summary, result.keyPoints[], result.decisions[], result.actionItems[]
|
||||
```
|
||||
|
||||
### 3. Context Compactor
|
||||
4 compaction strategies:
|
||||
- `sliding-window` - Keep recent messages only
|
||||
- `summarize-old` - Summarize older messages
|
||||
- `priority-retention` - Keep important messages
|
||||
- `hybrid` - Combine all strategies
|
||||
|
||||
```typescript
|
||||
const compactor = new ContextCompactor({
|
||||
maxTokens: 120000,
|
||||
targetTokens: 80000,
|
||||
strategy: 'hybrid',
|
||||
preserveRecentCount: 6,
|
||||
triggerThreshold: 80 // % of maxTokens
|
||||
});
|
||||
|
||||
const result = await compactor.compact(messages);
|
||||
// result.messages, result.tokensSaved, result.compressionRatio
|
||||
```
|
||||
|
||||
### 4. Agent Orchestrator
|
||||
Manages agent lifecycle and task routing.
|
||||
|
||||
```typescript
|
||||
const orchestrator = new AgentOrchestrator();
|
||||
|
||||
// Register agent
|
||||
orchestrator.registerAgent({
|
||||
id: 'agent-1',
|
||||
name: 'Worker',
|
||||
type: 'worker',
|
||||
capabilities: ['process', 'execute'],
|
||||
maxConcurrentTasks: 3,
|
||||
timeout: 60000
|
||||
});
|
||||
|
||||
// Create task
|
||||
orchestrator.createTask('process', 'Process data', { data: [...] });
|
||||
|
||||
// Events
|
||||
orchestrator.on('task_completed', (event) => { ... });
|
||||
|
||||
// Start processing
|
||||
orchestrator.start();
|
||||
```
|
||||
|
||||
### 5. Subagent Spawner
|
||||
Spawn and manage child agents for parallel execution.
|
||||
|
||||
```typescript
|
||||
const spawner = new SubagentSpawner();
|
||||
|
||||
// Single execution
|
||||
const result = await spawner.executeWithSubagent('explorer', 'Find files');
|
||||
|
||||
// Parallel execution
|
||||
const results = await spawner.executeParallel([
|
||||
{ type: 'explorer', input: 'Task 1' },
|
||||
{ type: 'coder', input: 'Task 2' }
|
||||
]);
|
||||
|
||||
// Pipeline execution (context passes between steps)
|
||||
const pipeline = await spawner.executePipeline([
|
||||
{ type: 'planner', input: 'Plan task' },
|
||||
{ type: 'coder', input: (prev) => `Implement: ${prev}` },
|
||||
{ type: 'reviewer', input: (prev) => `Review: ${prev}` }
|
||||
]);
|
||||
```
|
||||
|
||||
### 6. Predefined Subagent Types
|
||||
- `explorer` - Code exploration
|
||||
- `researcher` - Information gathering
|
||||
- `coder` - Code generation
|
||||
- `reviewer` - Code review
|
||||
- `planner` - Task planning
|
||||
- `executor` - Task execution
|
||||
- `custom` - Your custom type
|
||||
|
||||
## 🔧 Configuration Options
|
||||
|
||||
### CompactionConfig
|
||||
```typescript
|
||||
{
|
||||
maxTokens: 120000, // Maximum context tokens
|
||||
targetTokens: 80000, // Target after compaction
|
||||
strategy: 'hybrid', // Compaction strategy
|
||||
preserveRecentCount: 6, // Keep last N messages
|
||||
preserveSystemMessage: true, // Keep system message
|
||||
priorityKeywords: ['important', 'critical'],
|
||||
summaryMaxTokens: 2000, // Max tokens for summaries
|
||||
triggerThreshold: 80 // % to trigger compaction
|
||||
}
|
||||
```
|
||||
|
||||
### AgentConfig
|
||||
```typescript
|
||||
{
|
||||
id: 'agent-id',
|
||||
name: 'Agent Name',
|
||||
type: 'agent-type',
|
||||
capabilities: ['task1', 'task2'],
|
||||
maxConcurrentTasks: 3,
|
||||
timeout: 60000
|
||||
}
|
||||
```
|
||||
|
||||
## 📄 License
|
||||
|
||||
MIT License - Free to use, modify, and distribute.
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
Feel free to extend and customize for your needs!
|
||||
|
||||
Reference in New Issue
Block a user