From 2a088cb0b9a6a49a08b029d1f14b83e84b3e73cc Mon Sep 17 00:00:00 2001
From: admin <admin@rommark.dev>
Date: Tue, 5 May 2026 14:24:07 +0000
Subject: [PATCH] =?UTF-8?q?docs:=20comprehensive=20README=20overhaul=20?=
 =?UTF-8?q?=E2=80=94=20streaming,=20self-correction,=20RTK,=20formatting?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

- New streaming architecture section with ASCII pipeline diagrams
- Self-correction loops documented (2 retries + backoff + auto-simplification)
- RTK integration section (60-90% token savings)
- Telegram HTML formatting system documented
- Updated feature comparison table (all checkmarks accurate)
- systemd service deployment guide added
- Command reference table
- Architecture flow diagrams
---
 README.md | 338 ++++++++++++++++++++++++++++++++----------------------
 1 file changed, 201 insertions(+), 137 deletions(-)

diff --git a/README.md b/README.md
index 38e0d29c..a86188b4 100644
--- a/README.md
+++ b/README.md
@@ -1,19 +1,44 @@
 # zCode CLI X
 
-Agentic coder with **Z.AI + Telegram integration** — Claude Code + Hermes in one beast.
+Agentic coding assistant with **Z.AI + Telegram integration** — autonomous code execution with real-time streaming, self-correction loops, and RTK token optimization.
 
 > 💡 **Get 10% OFF Z.AI** — Use code **ROK78RJKNW** at [z.ai/subscribe](https://z.ai/subscribe?ic=ROK78RJKNW) for the Coding Plan
 
-## 🚀 Features
+## ⚡ Features
 
+### Core
 - **🤖 AI-Powered Code Generation**: Powered by Z.AI GLM-5.1 (Coding Plan)
-- **📱 Multi-Channel Bot**: Telegram + Discord with real-time delivery
+- **📱 Telegram Bot**: 24/7 via grammy + webhook with real-time SSE streaming
 - **🛠️ Full Engineering Access**: Bash, FileEdit, WebSearch, Git tools
-- **🧠 Agent System**: Code reviewer, architect, DevOps engineer
+- **🧠 Agent System**: Code Reviewer, System Architect, DevOps Engineer
 - **📚 Skills System**: Pre-built skills for common tasks
-- **⚡ Real-time Updates**: WebSocket-based live communication
-- **🔄 Self-Correction**: Automatic retry with backoff (2 retries)
-- **📦 Multi-Channel Delivery**: Hub-based message routing (Telegram + Discord + WebSocket + log)
+
+### Streaming & Formatting
+- **⚡ Real-time SSE Streaming**: Token-by-token delivery via `StreamConsumer` — adapted from [Hermes Agent's GatewayStreamConsumer](https://github.com/nousresearch/hermes-agent)
+  - Queued token buffer → rate-limited `editMessageText` loop (1s base interval)
+  - Adaptive backoff on Telegram flood control (429)
+  - Typing cursor `▉` during generation, clean final message
+  - Graceful fallback to plain send on repeated failures
+- **🎨 Telegram HTML Formatting**: AI markdown → clean Telegram HTML
+  - `**bold**`, `*italic*`, `` `code` ``, fenced code blocks, `[links](url)`, `~~strike~~`, headings, blockquotes, lists
+  - Double fallback: HTML → stripped plain text (never shows raw `**`)
+
+### Reliability
+- **🔄 Self-Correction Loops**: Automatic retry with exponential backoff
+  - 2 retry attempts (500ms → 1s → 1.5s delay)
+  - Triggers: API errors, rate limits, timeouts, 5xx server errors
+  - Auto-simplification: prompts simplified on retry to avoid recurring errors
+  - Full logging of all retry attempts with reason tracking
+- **🛡️ RTK (Rust Token Killer)**: Token optimization for supported commands
+  - 60-90% savings on git, npm, cargo, pytest, docker, and more
+  - Active tracking stats via `getTrackingStats()`
+
+### Architecture
+- **📨 Multi-Channel Delivery**: Hub-based routing (Telegram + Discord + WebSocket + log)
+- **🔁 Deduplication**: 60s TTL message deduplication
+- **📋 Request Queue**: Per-chat sequential processing (no race conditions)
+- **🔌 MCP Protocol**: Full MCP client + server management
+- **⏰ Cron Scheduling**: 1s interval, task locking, auto-recovery
 
 ## 📦 Installation
 
@@ -35,10 +60,10 @@ cp .env.example .env
 ```env
 # Z.AI Configuration (Coding Plan)
 GLM_BASE_URL=https://api.z.ai/api/coding/paas/v4
-ZAI_API_KEY=your_z...here
+ZAI_API_KEY=***
 
 # Telegram Bot Configuration
-TELEGRAM_BOT_TOKEN=your_b...here
+TELEGRAM_BOT_TOKEN=***
 TELEGRAM_ALLOWED_USERS=your_telegram_id
 ZCODE_WEBHOOK_URL=https://your-domain.com/telegram/webhook
 ```
@@ -57,66 +82,115 @@ node bin/zcode.js
 node bin/zcode.js --bot
 ```
 
-### Run in Development Mode
+### Run as systemd service
 
-```bash
-node bin/zcode.js --dev
+```ini
+# /etc/systemd/system/zcode.service
+[Unit]
+Description=zCode CLI X Bot
+After=network.target
+
+[Service]
+Type=simple
+User=uroma2
+WorkingDirectory=/home/uroma2/zcode-cli-x
+ExecStart=/usr/bin/node bin/zcode.js --bot
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
 ```
 
-### Run Only CLI (No Bot)
-
 ```bash
-node bin/zcode.js --no-bot
+sudo systemctl enable zcode
+sudo systemctl start zcode
 ```
 
-## 🤖 Telegram & Discord Bot
+## 🤖 Telegram Bot Commands
 
-Once running, you can interact with zCode CLI X via multiple channels:
+| Command | Description |
+|---|---|
+| `/start` | Show help and capabilities |
+| `/tools` | List available tools |
+| `/skills` | List loaded skills |
+| `/agents` | List agent roles |
+| `/model <name>` | Switch AI model |
+| `/stats` | System & RTK stats |
+| `/selfcorrection` | Self-correction status |
+| `/bash <cmd>` | Execute shell command |
+| `/web <query>` | Search the web |
+| `/git <action>` | Git operations |
+| `/cancel` | Cancel current operation |
 
-### Telegram
-1. Start the bot: `/start`
-2. Send your code requests or questions
-3. Receive AI-powered responses in real-time
+Or just chat — zCode uses tools automatically when needed.
 
-### Discord
-1. Invite the bot to your server
-2. Send commands or questions
-3. Receive responses with full streaming support
+## 🛠️ Tools
 
-### Multi-Channel Delivery
-Messages are automatically delivered to:
-- ✅ Telegram (primary)
-- ✅ Discord
-- ✅ WebSocket (real-time)
-- ✅ Log file (structured)
-
-## 🛠️ Tools Available
-
-- **Bash**: Execute shell commands
-- **FileEdit**: Edit files with diff-aware operations
-- **WebSearch**: Search the web for information
-- **Git**: Git operations (status, log, branch)
+| Tool | Description |
+|---|---|
+| **BashTool** | Shell command execution with timeout control |
+| **FileEditTool** | Diff-aware file operations (read, write, patch) |
+| **WebSearchTool** | Web search with result ranking |
+| **GitTool** | Git operations (status, log, diff, commit, push, pull) |
 
 ## 🧠 Agents
 
-- **Code Reviewer**: Review code for bugs and improvements
-- **System Architect**: Design system architecture and patterns
-- **DevOps Engineer**: Handle deployment, CI/CD, and infrastructure
-
-## 📚 Skills
-
-- **code_review**: Review code for bugs, security issues, and improvements
-- **bug_fix**: Fix identified bugs in code
-- **refactor**: Refactor code for better quality
-- **documentation**: Generate and update documentation
-- **testing**: Write tests for code
+| Agent | Role |
+|---|---|
+| **Code Reviewer** | Review code for bugs, security issues, and improvements |
+| **System Architect** | Design system architecture and patterns |
+| **DevOps Engineer** | Handle deployment, CI/CD, and infrastructure |
 
 ## 🏗️ Architecture
 
-zCode CLI X uses a hybrid architecture combining:
-- **better-clawd**: Coding backbone (BashTool, FileEditTool, WebSearchTool, GitTool)
-- **Hermes Agent**: Agent system (Code Reviewer, Architect, DevOps)
-- **claudegram**: Bot patterns (grammy, deduplication, request queuing, multi-channel delivery)
+zCode CLI X uses a hybrid architecture:
+
+```
+User Message
+    │
+    ▼
+┌─────────────┐     ┌──────────────┐     ┌─────────────────┐
+│  Telegram    │────▶│ dedup.js     │────▶│ request-queue   │
+│  (grammy)    │     │ (60s TTL)    │     │ (per-chat seq)  │
+└─────────────┘     └──────────────┘     └────────┬────────┘
+                                                   │
+                                                   ▼
+                                          ┌────────────────┐
+                                          │ self-correction │
+                                          │ (2 retries +   │
+                                          │  backoff)       │
+                                          └────────┬───────┘
+                                                   │
+                              ┌────────────────────┼────────────────────┐
+                              ▼                    ▼                    ▼
+                     ┌──────────────┐    ┌──────────────┐    ┌──────────────┐
+                     │ chatWithAI   │    │ Tool         │    │ Agent        │
+                     │ (SSE stream) │    │ Handlers     │    │ Delegation   │
+                     └──────┬───────┘    └──────────────┘    └──────────────┘
+                            │
+                    ┌───────┴────────┐
+                    ▼                ▼
+            ┌──────────────┐ ┌──────────────┐
+            │ Stream       │ │ sendFormatted│
+            │ Consumer     │ │ (HTML mode)  │
+            │ (edit-in-    │ └──────────────┘
+            │  place)      │
+            └──────┬───────┘
+                   │
+                   ▼
+            ┌──────────────┐
+            │ markdown     │
+            │ ToHtml()     │
+            │ converter    │
+            └──────┬───────┘
+                   │
+                   ▼
+            ┌──────────────┐
+            │ Telegram API │
+            │ (HTML mode)  │
+            └──────────────┘
+```
 
 ### Core Components
 
@@ -126,84 +200,70 @@ zcode-cli-x/
 │   └── zcode.js              # CLI entry point
 ├── src/
 │   ├── bot/
-│   │   ├── index.js          # Telegram bot (grammy-based)
+│   │   ├── index.js          # Telegram bot (grammy + SSE streaming)
+│   │   ├── message-sender.js # StreamConsumer + markdownToHtml converter
 │   │   ├── deduplication.js  # Message deduplication (60s TTL)
 │   │   ├── request-queue.js  # Per-chat request queuing
-│   │   ├── message-sender.js # Message chunking (4k limit)
 │   │   ├── delivery-hub.js   # Multi-channel delivery
 │   │   ├── discord.js        # Discord integration (discord.js v14)
-│   │   └── self-correction.js # Self-correction wrapper (2 retries)
+│   │   └── self-correction.js # Self-correction wrapper (2 retries + backoff)
 │   ├── api/
-│   │   └── index.js          # Z.AI API adapter (GLM-5.1)
+│   │   └── index.js          # Z.AI API adapter (GLM-5.1, SSE support)
 │   ├── tools/
-│   │   ├── BashTool.js       # Shell command executor
+│   │   ├── BashTool.js       # Shell command executor (RTK-aware)
 │   │   ├── FileEditTool.js   # File operations
 │   │   ├── WebSearchTool.js  # Web search
-│   │   └── GitTool.js        # Git operations
+│   │   └── GitTool.js        # Git operations (RTK-aware)
 │   ├── agents/
 │   │   └── index.js          # Agent orchestration
 │   ├── skills/
 │   │   └── index.js          # Skills system
 │   └── utils/
 │       ├── logger.js         # Winston logger
-│       └── env.js            # Environment validation
+│       ├── env.js            # Environment validation
+│       └── rtk.js            # RTK (Rust Token Killer) integration
 ├── .env                      # Configuration
 └── package.json
 ```
 
-### Bot Flow
+### Bot Message Flow
 
-1. **Message Reception**: Telegram/Discord messages → `delivery-hub.js`
-2. **Deduplication Check**: `deduplication.js` (60s TTL)
+1. **Message Reception**: Telegram webhook → grammy handler
+2. **Deduplication**: `deduplication.js` (60s TTL, prevents double-processing)
 3. **Request Queue**: `request-queue.js` (per-chat sequential processing)
-4. **Self-Correction**: `self-correction.js` (2 retries + backoff)
-5. **Agent Execution**: better-clawd tools + Hermes agents
-6. **Response Delivery**: `message-sender.js` → Multi-channel (Telegram + Discord + WebSocket + log)
+4. **Self-Correction**: `self-correction.js` (2 retries + exponential backoff + auto-simplification)
+5. **AI Chat + Streaming**: `chatWithAI()` → SSE stream → `StreamConsumer` → real-time edits
+6. **Formatting**: `markdownToHtml()` converts AI markdown → Telegram HTML
+7. **Final Delivery**: `editMessageText` with HTML parse_mode (or fallback to stripped plain text)
 
-### Provider Support
+### StreamConsumer Pipeline
 
-- **Z.AI**: GLM-5.1 (Coding Plan) - Primary
-- **Claude**: Anthropic Claude (via API)
-- **OpenCode**: OpenCode CLI integration
-- **Kilo**: Kilo.AI gateway
-
-## 🔗 Integrations
-
-- **Z.AI API**: GLM-5.1 model with Coding Plan
-- **Telegram Bot API**: grammy-based bot with auto-retry and runner
-- **Discord.js v14**: Discord bot with GatewayIntentBits
-- **Express.js**: HTTP server for webhook handling
-- **Winston**: Structured logging
-- **WebSocket**: Real-time updates
-- **Mongoose**: Database persistence
-- **Puppeteer**: Browser automation
-
-## 🚀 Getting Started
-
-1. Install dependencies:
-   ```bash
-   npm install
-   ```
-
-2. Configure `.env` with your credentials:
-   ```env
-   GLM_BASE_URL=https://api.z.ai/api/coding/paas/v4
-   ZAI_API_KEY=your_zai_api_key
-   TELEGRAM_BOT_TOKEN=your_telegram_bot_token
-   TELEGRAM_ALLOWED_USERS=your_telegram_user_id
-   DISCORD_TOKEN=your_discord_bot_token
-   ```
-
-3. Run the bot:
-   ```bash
-   node bin/zcode.js --bot
-   ```
-
-4. Open Telegram and Discord to start chatting!
-
-## 📝 License
-
-MIT
+```
+Z.AI API (SSE)
+    │
+    ▼ onDelta(token)
+┌──────────────┐
+│ Token Buffer │  ← accumulates tokens from SSE stream
+└──────┬───────┘
+       │ (every ~1s or 40 chars)
+       ▼
+┌──────────────┐
+│ editMessage  │  ← plain text + cursor ▉ (no parse_mode)
+│ Text()       │     rate-limited, adaptive backoff on flood
+└──────┬───────┘
+       │ (on finish)
+       ▼
+┌──────────────┐
+│ markdownTo   │  ← converts **bold**, *italic*, `code`, etc.
+│ Html()       │     to <b>, <i>, <code>, <pre> HTML tags
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│ editMessage  │  ← final message with parse_mode: 'HTML'
+│ Text()       │     fallback: stripped plain text (no raw **)
+└──────────────┘
+```
 
 ## 📊 Feature Comparison
 
@@ -213,47 +273,51 @@ MIT
 | Autonomous execution | ✅ Full autonomous mode | ✅ Full autonomous mode | ⚠️ Manual step-by-step |
 | Sub-agents | ✅ Multi-agent (swarm) | ✅ delegate_task + batch | ❌ Single agent only |
 | Agent roles | ✅ Code Reviewer, Architect, DevOps | ✅ Agent Registry (10+ roles) | ❌ Fixed single role |
-| Self-correction loops | ❌ None | ✅ Agent self-correction skill | ❌ None |
+| Self-correction loops | ✅ 2 retries + backoff + auto-simplification | ✅ Agent self-correction skill | ❌ None |
+| **Streaming** | | | |
+| Real-time SSE streaming | ✅ StreamConsumer (edit-in-place) | ✅ GatewayStreamConsumer | ❌ None |
+| Telegram HTML formatting | ✅ markdownToHtml + fallback | ✅ Native HTML support | ❌ None |
+| Adaptive flood control | ✅ Exponential backoff | ✅ Flood backoff | ❌ N/A |
 | **Tooling** | | | |
 | Bash/Shell | ✅ BashTool | ✅ TerminalTool | ✅ Shell access |
 | File editing | ✅ FileEditTool (diff-aware) | ✅ Patch + Write + Edit | ⚠️ Basic write |
 | Web search | ✅ WebSearch | ✅ WebSearch + Vane + Exa | ❌ None |
-| Git integration | ✅ GitTool | ✅ GitTool | ❌ None |
+| Git integration | ✅ GitTool (RTK-aware) | ✅ GitTool | ❌ None |
 | Browser automation | ✅ Computer-use (Anthropic) | ✅ Full browser toolkit | ❌ None |
-| MCP servers | ✅ Full MCP protocol (client + server management) | ✅ Native MCP + mcporter | ❌ None |
-| Code execution | ✅ Sandbox adapter | ✅ Sandbox + Jupyter | ❌ None |
-| **Skills** | | | |
-| Skill system | ✅ Skill system with skills dir loader | ✅ 500+ skills catalog | ❌ No skill system |
-| Custom skill authoring | ✅ Skillify (session→skill capture) | ✅ skill_manage CLI | ❌ None |
-| Plugin architecture | ✅ Full marketplace + loader + lifecycle | ✅ Full plugin system | ❌ None |
-| **Automation** | | | |
-| Cron scheduling | ✅ CronScheduler (1s interval, jitter, locks) | ✅ Cron jobs with delivery | ❌ None |
-| Webhook subscriptions | ✅ Hook system (HTTP, agent, prompt hooks) | ✅ Event-driven agent runs | ❌ None |
-| Scheduled monitoring | ✅ Cron-based recurring monitoring | ✅ Browser + social monitors | ❌ None |
-| Batch task processing | ✅ Batch skill (5-30 parallel subagents) | ✅ Parallel batch delegation | ❌ None |
+| MCP servers | ✅ Full MCP protocol | ✅ Native MCP + mcporter | ❌ None |
+| RTK optimization | ✅ RTK active (60-90% savings) | ✅ RTK integrated | ❌ None |
 | **Platform** | | | |
-| Telegram integration | ✅ Native bot + webhook | ✅ 2-way Telegram bridge | ❌ None |
-| Discord | ✅ Native bot (via discord.js) | ✅ Full Discord integration | ❌ None |
-| Multi-channel delivery | ✅ Delivery hub (Telegram + Discord + WebSocket + log) | ✅ Cron→Telegram/Discord/Email | ❌ None |
-| Self-correction loops | ✅ Self-correction wrapper (2 retries + backoff) | ✅ Agent self-correction skill | ❌ None |
+| Telegram integration | ✅ Native bot + webhook + streaming | ✅ 2-way Telegram bridge | ❌ None |
+| Discord | ✅ Native bot (discord.js) | ✅ Full Discord integration | ❌ None |
+| Multi-channel delivery | ✅ Delivery hub (TG + DC + WS + log) | ✅ Cron→multi-platform | ❌ None |
 | **Infrastructure** | | | |
-| Model routing | ✅ Multi-provider (OpenAI, Anthropic, Bedrock, custom) | ✅ Multi-provider routing | ❌ Single model |
-| Context compression | ✅ Compact pipeline (auto, micro, session memory) | ✅ lean-ctx MCP (90% savings) | ❌ None |
-| Memory persistence | ✅ Session memory with background extraction | ✅ Cross-session memory | ❌ None |
-| RTK integration | ✅ RTK active | ✅ RTK integrated | ❌ None |
+| Model routing | ✅ Multi-provider | ✅ Multi-provider routing | ❌ Single model |
+| Context compression | ✅ Compact pipeline | ✅ lean-ctx MCP (90% savings) | ❌ None |
+| Memory persistence | ✅ Session memory | ✅ Cross-session memory | ❌ None |
+| Cron scheduling | ✅ 1s interval, jitter, locks | ✅ Cron jobs with delivery | ❌ None |
 
 ### Summary
 
-- **zCode CLI X** — Lightweight agentic coder focused on Telegram + Z.AI. Ideal for quick coding tasks via Telegram with essential tools and agent roles.
-- **Hermes Agent** — Full-stack AI assistant platform. Best for complex multi-agent workflows, scheduled automation, and cross-platform deployment. 500+ skills, MCP ecosystem, and the deepest toolset.
-- **better-clawd** — Minimal Claude Code clone. Useful as a lightweight reference but lacks the agentic, skills, and automation depth of zCode or Hermes.
+- **zCode CLI X** — Lightweight agentic coder focused on Telegram + Z.AI. Real-time SSE streaming, self-correction loops, RTK optimization, and beautiful HTML formatting. Ideal for quick coding tasks via Telegram.
+- **Hermes Agent** — Full-stack AI assistant platform. Best for complex multi-agent workflows, scheduled automation, and cross-platform deployment. 500+ skills, MCP ecosystem, deepest toolset.
+- **better-clawd** — Minimal Claude Code clone. Useful as a lightweight reference but lacks agentic depth.
+
+## 🔗 Integrations
+
+- **Z.AI API**: GLM-5.1 model (Coding Plan) with SSE streaming
+- **Telegram Bot API**: grammy + auto-retry + runner + webhook
+- **Discord.js v14**: Discord bot with GatewayIntentBits
+- **Express.js**: HTTP server for webhook handling
+- **Winston**: Structured logging
+- **WebSocket**: Real-time updates
+- **RTK**: Rust Token Killer (token optimization)
 
 ## 🤝 Contributing
 
-Contributions welcome! This is based on:
-- [better-clawd](https://github.com/x1xhlol/better-clawd.git) - Claude Code clone
-- [Hermes Agent](https://hermes-agent.nousresearch.com) - Our AI assistant
+Contributions welcome! Based on:
+- [better-clawd](https://github.com/x1xhlol/better-clawd.git) — Claude Code clone
+- [Hermes Agent](https://hermes-agent.nousresearch.com) — AI assistant platform (streaming architecture credit)
 
 ---
 
-Built with ❤️ by zCode CLI X
+Built with ⚡ by zCode CLI X