perf: 3-tier conversation context with LRU cache, keyword relevance, debounced I/O
UPGRADE from naive JSON to production-grade conversation memory: Tier 1 — Compressed Summary (max 600 chars): Incrementally built from evicted messages. Preserves conversation topics across 100+ messages in a tiny budget. Tier 2 — Relevant Snippets (BM25-style keyword matching): Scores older messages against current query, injects top 3 matches. Zero external deps — keyword extraction is ~0.1ms. Tier 3 — Sliding Window (last 12 exchanges verbatim): Recent context preserved word-for-word, fitting within token budget. Performance optimizations: - In-memory Map cache with lazy-load from disk (0ms reads) - Debounced async disk writes (3s, non-blocking, never stalls response) - LRU eviction for cache (max 50 chats, prevents memory leak) - Keywords stripped before saving (smaller JSON files) - Backward-compatible: loads old format without keywords, backfills on load - Graceful shutdown flushes all pending saves to disk - Token-aware budget allocation: summary 15% + relevant 15% + recent 70%
This commit is contained in:
@@ -688,7 +688,7 @@ export async function initBot(config, api, tools, skills, agents) {
|
||||
|
||||
// ── Load conversation history for this chat ──
|
||||
const chatKey = conversation._key(ctx.chat.id, ctx.message?.message_thread_id);
|
||||
const history = await conversation.getContext(chatKey);
|
||||
const history = await conversation.getContext(chatKey, text);
|
||||
|
||||
// Create stream consumer for real-time edit-in-place
|
||||
const consumer = new StreamConsumer(ctx, { editInterval: 1000 });
|
||||
@@ -757,6 +757,15 @@ export async function initBot(config, api, tools, skills, agents) {
|
||||
logger.error('Unhandled rejection:', reason?.message || reason);
|
||||
});
|
||||
|
||||
// ── Graceful shutdown: flush conversation history ──
|
||||
const shutdown = async (signal) => {
|
||||
logger.info(`🛑 Shutting down (${signal})...`);
|
||||
await conversation.flush();
|
||||
process.exit(0);
|
||||
};
|
||||
process.on('SIGINT', () => shutdown('SIGINT'));
|
||||
process.on('SIGTERM', () => shutdown('SIGTERM'));
|
||||
|
||||
// ── Express + WebSocket server (keep for webhook compatibility) ──
|
||||
const app = express();
|
||||
app.use(express.json());
|
||||
|
||||
Reference in New Issue
Block a user