SuperCharged-Claude-Code-Upgrade/ROMAN_SESSION_FIX_DESIGN.md

# Project Roman Session Fix - Phase 2 Design Document

## Executive Summary
This document outlines the comprehensive fix for the Project Roman session issues. The solution addresses root causes: race conditions, multiple sources of truth, and insufficient error handling.

## Design Principles

### 1. Single Direction Data Flow
```
API (Source of Truth)
        ↓
  Frontend State (Derived)
        ↓
     UI (Computed)
```

### 2. Optimistic Updates with Rollback
- Update UI immediately on user action
- Verify with API in background
- Rollback on failure

### 3. Event-Driven Architecture
- All state changes emit events
- Components subscribe to relevant events
- No direct state manipulation

## Component 1: Enhanced State Management

### New Class: SessionStateManager
```javascript
class SessionStateManager extends EventEmitter {
    constructor() {
        super();
        this.state = {
            projects: new Map(),  // projectId -> Project
            sessions: new Map(),  // sessionId -> Session
            activeProjectId: null,
            activeSessionId: null,
            loading: false,
            error: null
        };
        this.initialized = false;
    }

    // ============================================================
    // CORE STATE OPERATIONS - All transactions go through here
    // ============================================================

    /**
     * Atomic state update with event emission
     * @param {string} operation - Operation name
     * @param {Function} updater - Function that updates state
     * @returns {Object} Result of operation
     */
    async transaction(operation, updater) {
        const transactionId = `${operation}-${Date.now()}`;
        console.log(`[State] Transaction START: ${transactionId}`);

        try {
            // Emit transaction start
            this.emit('transaction:start', { transactionId, operation });

            // Execute update
            const result = await updater(this.state);

            // Emit transaction success
            this.emit('transaction:success', { transactionId, operation, result });

            console.log(`[State] Transaction SUCCESS: ${transactionId}`);
            return result;
        } catch (error) {
            // Emit transaction failure
            this.emit('transaction:error', { transactionId, operation, error });
            console.error(`[State] Transaction ERROR: ${transactionId}`, error);
            throw error;
        }
    }

    /**
     * Initialize state from API
     */
    async initialize() {
        if (this.initialized) {
            console.log('[State] Already initialized');
            return;
        }

        return this.transaction('initialize', async (state) => {
            state.loading = true;
            this.emit('state:loading', { loading: true });

            // Fetch from API
            const response = await fetch('/claude/api/claude/sessions');
            if (!response.ok) throw new Error(`HTTP ${response.status}`);

            const data = await response.json();

            // Build state from API response
            const allSessions = [
                ...(data.active || []).map(s => ({...s, status: 'active'})),
                ...(data.historical || []).map(s => ({...s, status: 'historical'}))
            ];

            // Group sessions by project
            const grouped = this.groupSessionsByProject(allSessions);

            // Load manually created projects from localStorage (as metadata only)
            const manualProjects = this.loadManualProjectMetadata();

            // Merge manual projects with sessions
            this.mergeManualProjects(grouped, manualProjects);

            // Update state
            state.projects = grouped;
            allSessions.forEach(session => {
                state.sessions.set(session.id, session);
            });

            state.loading = false;
            this.initialized = true;

            this.emit('state:initialized', {
                projectCount: state.projects.size,
                sessionCount: state.sessions.size
            });

            return { projectCount: state.projects.size, sessionCount: state.sessions.size };
        });
    }

    /**
     * Create session with optimistic update
     */
    async createSession(workingDir, projectId, projectName) {
        // Optimistic: Add session to state immediately
        const tempSessionId = `temp-${Date.now()}`;
        const tempSession = {
            id: tempSessionId,
            workingDir,
            status: 'creating',
            metadata: { projectId, project: projectName },
            createdAt: new Date().toISOString(),
            _optimistic: true
        };

        this.transaction('createSession:optimistic', (state) => {
            // Add to sessions
            state.sessions.set(tempSessionId, tempSession);

            // Add to project
            const project = state.projects.get(projectId.replace('project-', ''));
            if (project) {
                project.sessions.unshift(tempSession);
                project.activeSessionId = tempSessionId;
            }

            this.emit('session:added:optimistic', { session: tempSession, projectId });
        });

        try {
            // Actual API call
            const response = await fetch('/claude/api/claude/sessions', {
                method: 'POST',
                headers: { 'Content-Type': 'application/json' },
                body: JSON.stringify({
                    workingDir,
                    metadata: { projectId, project: projectName }
                })
            });

            if (!response.ok) throw new Error(`HTTP ${response.status}`);

            const data = await response.json();
            const realSession = data.session || data;

            // Replace optimistic session with real session
            return this.transaction('createSession:commit', (state) => {
                // Remove temp session
                state.sessions.delete(tempSessionId);

                // Add real session
                state.sessions.set(realSession.id, realSession);

                // Update project
                const project = state.projects.get(projectId.replace('project-', ''));
                if (project) {
                    const tempIndex = project.sessions.findIndex(s => s.id === tempSessionId);
                    if (tempIndex !== -1) {
                        project.sessions[tempIndex] = realSession;
                    } else {
                        project.sessions.unshift(realSession);
                    }
                    project.activeSessionId = realSession.id;
                }

                this.emit('session:added:confirmed', { session: realSession, projectId });

                return realSession;
            });
        } catch (error) {
            // Rollback: Remove optimistic session
            this.transaction('createSession:rollback', (state) => {
                state.sessions.delete(tempSessionId);

                const project = state.projects.get(projectId.replace('project-', ''));
                if (project) {
                    project.sessions = project.sessions.filter(s => s.id !== tempSessionId);
                }

                this.emit('session:added:rollback', { tempSessionId, error });
            });

            throw error;
        }
    }

    /**
     * Group sessions by working directory (project)
     */
    groupSessionsByProject(sessions) {
        const grouped = new Map();

        sessions.forEach(session => {
            const dir = session.workingDir || 'default';
            let projectKey;

            // Handle virtual working directories
            if (dir.startsWith('/virtual/projects/')) {
                projectKey = dir.replace('/virtual/projects/', '');
            } else {
                projectKey = dir.replace(/\//g, '-').replace(/^-/, '') || 'default';
            }

            if (!grouped.has(projectKey)) {
                const projectName = dir.split('/').pop() || 'Default';
                grouped.set(projectKey, {
                    id: `project-${projectKey}`,
                    name: this.deduplicateProjectName(projectName, grouped),
                    workingDir: dir,
                    sessions: [],
                    activeSessionId: null,
                    manuallyCreated: dir.startsWith('/virtual/projects/'),
                    isVirtual: dir.startsWith('/virtual/projects/')
                });
            }

            const project = grouped.get(projectKey);
            project.sessions.push(session);
        });

        // Sort sessions and set active
        grouped.forEach(project => {
            project.sessions.sort((a, b) => {
                const dateA = new Date(a.lastActivity || a.createdAt || 0);
                const dateB = new Date(b.lastActivity || b.createdAt || 0);
                return dateB - dateA;
            });
            if (project.sessions.length > 0) {
                project.activeSessionId = project.sessions[0].id;
            }
        });

        return grouped;
    }

    /**
     * Load manual project metadata from localStorage
     */
    loadManualProjectMetadata() {
        try {
            const stored = localStorage.getItem('claude_ide_projects');
            if (!stored) return new Map();

            const projectsData = JSON.parse(stored);
            const metadata = new Map();

            projectsData.forEach(projectData => {
                const key = projectData.id.replace('project-', '');
                // Only store metadata, not sessions (sessions come from API)
                metadata.set(key, {
                    id: projectData.id,
                    name: projectData.name,
                    workingDir: projectData.workingDir,
                    manuallyCreated: true,
                    createdAt: projectData.createdAt
                });
            });

            return metadata;
        } catch (error) {
            console.error('[State] Error loading manual project metadata:', error);
            return new Map();
        }
    }

    /**
     * Merge manual projects with API-derived projects
     */
    mergeManualProjects(grouped, manualMetadata) {
        manualMetadata.forEach((meta, key) => {
            if (!grouped.has(key)) {
                // Project exists in metadata but not in API (no sessions yet)
                grouped.set(key, {
                    ...meta,
                    sessions: [],
                    activeSessionId: null
                });
            } else {
                // Project exists in both - prefer manual metadata for name/description
                const existing = grouped.get(key);
                existing.name = meta.name;
                existing.manuallyCreated = true;
            }
        });
    }

    deduplicateProjectName(name, existingProjects) {
        const names = Array.from(existingProjects.values()).map(p => p.name);
        let finalName = name;
        let counter = 2;

        while (names.includes(finalName)) {
            finalName = `${name} (${counter})`;
            counter++;
        }

        return finalName;
    }
}
```

## Component 2: Enhanced Real-Time Logger

### New File: real-time-logger.js
```javascript
/**
 * Real-Time Logger - Centralized logging with server integration
 * Provides debugging, monitoring, and alerting capabilities
 */

class RealTimeLogger {
    constructor() {
        this.logs = [];
        this.maxLogs = 1000;
        this.endpoint = '/claude/api/log-event';
        this.errorCounts = new Map();
        this.alertThreshold = 5; // Alert after 5 similar errors
    }

    /**
     * Log an event with context
     */
    log(category, level, message, data = {}) {
        const entry = {
            timestamp: Date.now(),
            isoTime: new Date().toISOString(),
            category,
            level,
            message,
            data,
            url: window.location.href,
            sessionId: window.attachedSessionId || null,
            projectId: window.projectManager?.activeProjectId || null
        };

        // Add to in-memory log
        this.logs.push(entry);
        if (this.logs.length > this.maxLogs) {
            this.logs.shift();
        }

        // Track error patterns
        if (level === 'error') {
            this.trackError(entry);
        }

        // Send to server
        this.sendToServer(entry);

        // Console output with color coding
        this.consoleOutput(entry);

        return entry;
    }

    trackError(entry) {
        const key = `${entry.category}:${entry.message}`;
        const count = (this.errorCounts.get(key) || 0) + 1;
        this.errorCounts.set(key, count);

        if (count >= this.alertThreshold) {
            this.alert(`Recurring error: ${entry.message}`, { count, entry });
        }
    }

    async sendToServer(entry) {
        try {
            // Batch multiple logs if possible
            if (this.pendingSend) {
                this.pendingBatch.push(entry);
                return;
            }

            this.pendingBatch = [entry];
            this.pendingSend = true;

            // Small delay to batch more logs
            await new Promise(resolve => setTimeout(resolve, 100));

            await fetch(this.endpoint, {
                method: 'POST',
                headers: { 'Content-Type': 'application/json' },
                body: JSON.stringify({
                    events: this.pendingBatch,
                    pageInfo: {
                        pathname: window.location.pathname,
                        search: window.location.search,
                        hash: window.location.hash
                    }
                })
            });
        } catch (error) {
            // Silent fail - don't log to avoid infinite loop
        } finally {
            this.pendingSend = false;
            this.pendingBatch = [];
        }
    }

    consoleOutput(entry) {
        const prefix = `[${entry.category}] ${entry.level.toUpperCase()}`;
        const styles = {
            info: 'color: #0066cc',
            warn: 'color: #ff9900',
            error: 'color: #cc0000',
            success: 'color: #00cc66',
            debug: 'color: #888888'
        };

        const style = styles[entry.level] || styles.info;

        if (entry.level === 'error') {
            console.error(`%c${prefix}`, style, entry.message, entry.data);
        } else if (entry.level === 'warn') {
            console.warn(`%c${prefix}`, style, entry.message, entry.data);
        } else {
            console.log(`%c${prefix}`, style, entry.message, entry.data);
        }
    }

    alert(message, data) {
        // Show visual alert
        this.showAlertIndicator(message);

        // Log critical
        this.log('alert', 'error', message, data);
    }

    showAlertIndicator(message) {
        const existing = document.getElementById('rt-logger-alert');
        if (existing) existing.remove();

        const alert = document.createElement('div');
        alert.id = 'rt-logger-alert';
        alert.innerHTML = `
            <div style="position:fixed; bottom:20px; right:20px; z-index:99999;
                        background:linear-gradient(135deg,#ff6b6b,#ee5a6f);
                        color:white; padding:16px 20px; border-radius:8px;
                        box-shadow:0 4px 12px rgba(0,0,0,0.3);
                        font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;
                        max-width:300px;">
                <div style="font-weight:600; margin-bottom:4px;">Recurring Error Detected</div>
                <div style="font-size:13px; opacity:0.9;">${message}</div>
                <button onclick="this.parentElement.remove()" style="margin-top:8px;
                           background:rgba(255,255,255,0.2); border:none;
                           color:white; padding:4px 12px; border-radius:4px;
                           cursor:pointer; font-size:12px;">Dismiss</button>
            </div>
        `;
        document.body.appendChild(alert);
    }

    // Convenience methods
    info(category, message, data) { return this.log(category, 'info', message, data); }
    warn(category, message, data) { return this.log(category, 'warn', message, data); }
    error(category, message, data) { return this.log(category, 'error', message, data); }
    success(category, message, data) { return this.log(category, 'success', message, data); }
    debug(category, message, data) { return this.log(category, 'debug', message, data); }

    /**
     * Get recent logs
     */
    getLogs(filter = {}) {
        let filtered = this.logs;

        if (filter.category) {
            filtered = filtered.filter(l => l.category === filter.category);
        }
        if (filter.level) {
            filtered = filtered.filter(l => l.level === filter.level);
        }
        if (filter.since) {
            filtered = filtered.filter(l => l.timestamp >= filter.since);
        }

        return filtered;
    }

    /**
     * Export logs for analysis
     */
    exportLogs() {
        return JSON.stringify({
            exportTime: new Date().toISOString(),
            url: window.location.href,
            totalLogs: this.logs.length,
            logs: this.logs,
            errorPatterns: Object.fromEntries(this.errorCounts)
        }, null, 2);
    }
}

// Global instance
window.rtLogger = new RealTimeLogger();

// Auto-export logs on error
window.addEventListener('error', () => {
    sessionStorage.setItem('rtLogger_logs', window.rtLogger.exportLogs());
});

// Export critical logs on unload
window.addEventListener('beforeunload', () => {
    const errorLogs = window.rtLogger.getLogs({ level: 'error' });
    if (errorLogs.length > 0) {
        sessionStorage.setItem('rtLogger_errors', JSON.stringify(errorLogs));
    }
});
```

## Component 3: Fixed SSE Client

### Modified: sse-client.js
```javascript
connect(sessionId) {
    if (this.eventSource) {
        this.disconnect();
    }

    this.currentSessionId = sessionId;
    const url = `/claude/api/session/${encodeURIComponent(sessionId)}/events`;

    window.rtLogger?.info('SSE', 'Connecting to SSE', { sessionId, url });

    try {
        this.eventSource = new EventSource(url);

        // CRITICAL FIX: Check if EventSource was successfully created
        if (!this.eventSource) {
            throw new Error('EventSource construction returned null');
        }

        // CRITICAL FIX: Check readyState before setting handlers
        if (this.eventSource.readyState === EventSource.CLOSED) {
            throw new Error('EventSource closed immediately');
        }

        // Now safe to set handlers
        this.eventSource.onopen = () => {
            window.rtLogger?.success('SSE', 'Connected', { sessionId });
            this.reconnectAttempts = 0;
            this.emit('connected', { sessionId });
        };

        this.eventSource.onerror = (error) => {
            window.rtLogger?.error('SSE', 'Connection error', {
                sessionId,
                readyState: this.eventSource?.readyState,
                error: error?.toString()
            });
            // ... rest of error handling
        };

        this.registerEventListeners();

    } catch (error) {
        window.rtLogger?.error('SSE', 'Failed to create EventSource', {
            sessionId,
            error: error.message
        });
        this.emit('error', { sessionId, error: error.message });
        this.handleReconnect();
    }
}
```

## Component 4: Defensive Array Operations

### Modified: chat-enhanced.js
```javascript
async function loadChatHistory(sessionsToRender = null) {
    try {
        const historyList = document.getElementById('chat-history-list');
        if (!historyList) {
            window.rtLogger?.warn('ChatHistory', 'Element not found', { element: 'chat-history-list' });
            return;
        }

        // CRITICAL FIX: Always ensure we have an array
        let allSessions = [];

        if (Array.isArray(sessionsToRender)) {
            // Use provided sessions
            allSessions = sessionsToRender;
            window.rtLogger?.info('ChatHistory', 'Rendering provided sessions', { count: allSessions.length });
        } else if (window.projectManager?.activeProjectId) {
            // Use active project sessions
            const activeProject = window.projectManager.projects.get(
                window.projectManager.activeProjectId.replace('project-', '')
            );

            if (activeProject?.sessions && Array.isArray(activeProject.sessions)) {
                allSessions = activeProject.sessions;
                window.rtLogger?.info('ChatHistory', 'Using project sessions', {
                    project: activeProject.name,
                    count: allSessions.length
                });
            }
        }

        // CRITICAL FIX: Validate array before sorting
        if (!Array.isArray(allSessions)) {
            window.rtLogger?.warn('ChatHistory', 'Sessions is not an array', { type: typeof allSessions });
            allSessions = [];
        }

        // Safe sort with validation
        allSessions.sort((a, b) => {
            const dateA = new Date(a?.createdAt || a?.created_at || 0);
            const dateB = new Date(b?.createdAt || b?.created_at || 0);
            return dateB - dateA;
        });

        // Safe render with validation
        if (allSessions.length === 0) {
            historyList.innerHTML = '<div class="chat-history-empty">No sessions in this project</div>';
            return;
        }

        historyList.innerHTML = allSessions.map(session => {
            // Validate session object
            if (!session?.id) {
                window.rtLogger?.warn('ChatHistory', 'Invalid session object', { session });
                return '';
            }

            // ... render code
        }).join('');

        window.rtLogger?.success('ChatHistory', 'Rendered sessions', { count: allSessions.length });

    } catch (error) {
        window.rtLogger?.error('ChatHistory', 'Failed to load history', { error: error.message });
    }
}
```

## Component 5: Virtual Working Directory Validator

### New: project-validator.js
```javascript
/**
 * Project Validator - Ensures project consistency
 */

class ProjectValidator {
    /**
     * Validate and fix project working directory
     */
    static validateProject(project) {
        const issues = [];
        const fixes = [];

        // Check if project is virtual but has wrong workingDir
        if (project.manuallyCreated && project.isVirtual) {
            const expectedWorkingDir = `/virtual/projects/${project.id.replace('project-', '')}`;

            if (project.workingDir !== expectedWorkingDir) {
                issues.push({
                    type: 'working_dir_mismatch',
                    project: project.name,
                    expected: expectedWorkingDir,
                    actual: project.workingDir
                });

                fixes.push({
                    type: 'fix_working_dir',
                    projectId: project.id,
                    workingDir: expectedWorkingDir
                });
            }
        }

        return { issues, fixes };
    }

    /**
     * Fix all project issues
     */
    static fixProjects(projects) {
        const results = [];

        projects.forEach((project, key) => {
            const validation = this.validateProject(project);

            if (validation.issues.length > 0) {
                window.rtLogger?.warn('ProjectValidator', 'Found project issues', {
                    project: project.name,
                    issues: validation.issues
                });

                validation.fixes.forEach(fix => {
                    if (fix.type === 'fix_working_dir') {
                        project.workingDir = fix.workingDir;
                        results.push({ project: project.name, fixed: fix.workingDir });
                    }
                });
            }
        });

        if (results.length > 0) {
            window.rtLogger?.success('ProjectValidator', 'Fixed projects', { results });
        }

        return results;
    }

    /**
     * Migrate existing localStorage projects
     */
    static migrateLocalStorage() {
        try {
            const stored = localStorage.getItem('claude_ide_projects');
            if (!stored) return { migrated: 0 };

            const projectsData = JSON.parse(stored);
            let migrated = 0;

            const fixed = projectsData.map(project => {
                if (project.manuallyCreated && !project.workingDir?.startsWith('/virtual/projects/')) {
                    const projectKey = project.id.replace('project-', '');
                    project.workingDir = `/virtual/projects/${projectKey}`;
                    project.isVirtual = true;
                    migrated++;

                    window.rtLogger?.info('ProjectValidator', 'Migrated project', {
                        name: project.name,
                        newWorkingDir: project.workingDir
                    });
                }
                return project;
            });

            localStorage.setItem('claude_ide_projects', JSON.stringify(fixed));

            return { migrated, projects: fixed };
        } catch (error) {
            window.rtLogger?.error('ProjectValidator', 'Migration failed', { error: error.message });
            return { migrated: 0, error: error.message };
        }
    }
}
```

## Implementation Order

### Phase 3: Implementation (Pending AI Engineer Approval)

1. **Step 1: Install Real-Time Logger** (Low Risk)
   - Add real-time-logger.js to index.html
   - Test logging functionality
   - Verify server endpoint exists

2. **Step 2: Fix SSE Client** (Low Risk)
   - Add null checks to sse-client.js
   - Test connection with valid session
   - Test connection with invalid session

3. **Step 3: Add Array Validation** (Low Risk)
   - Add guards to chat-enhanced.js
   - Add guards to project-manager.js
   - Test with empty sessions array
   - Test with undefined sessions

4. **Step 4: Add Project Validator** (Medium Risk)
   - Create project-validator.js
   - Run migration once
   - Validate existing projects

5. **Step 5: Implement State Manager** (High Risk)
   - Create session-state-manager.js
   - Migrate ProjectManager to use it
   - Test all state transitions
   - Verify optimistic updates work

## Testing Strategy

### Unit Tests
- SessionStateManager transactions
- ProjectValidator migration
- Array validation functions

### Integration Tests
- Session creation flow
- Project switching flow
- Session persistence flow

### Manual Tests
1. Create new project
2. Add session to project
3. Refresh page
4. Verify sessions persist
5. Check console for errors

## Rollback Plan

If any step fails:
1. Revert specific file changes
2. Keep existing logger for debugging
3. Document what failed and why
4. Adjust approach based on findings

---

**Status:** Phase 2 Complete - Awaiting AI Engineer Review
**Required Approval:** AI Engineer must review and approve before Phase 3 implementation
**Review Focus Areas:**
1. State management architecture
2. Optimistic update pattern
3. Error handling strategy
4. Migration safety