This is page 11 of 12. Use http://codebase.md/minipuft/claude-prompts-mcp?page={x} to view the full context.
# Directory Structure
```
├── .actrc
├── .gitattributes
├── .github
│ └── workflows
│ ├── ci.yml
│ ├── mcp-compliance.yml
│ └── pr-validation.yml
├── .gitignore
├── agent.md
├── assets
│ └── logo.png
├── CLAUDE.md
├── config
│ └── framework-state.json
├── docs
│ ├── architecture.md
│ ├── chain-modification-examples.md
│ ├── contributing.md
│ ├── enhanced-gate-system.md
│ ├── execution-architecture-guide.md
│ ├── installation-guide.md
│ ├── mcp-tool-usage-guide.md
│ ├── mcp-tools-reference.md
│ ├── prompt-format-guide.md
│ ├── prompt-management.md
│ ├── prompt-vs-template-guide.md
│ ├── README.md
│ ├── template-development-guide.md
│ ├── TODO.md
│ ├── troubleshooting.md
│ └── version-history.md
├── LICENSE
├── local-test.sh
├── plans
│ ├── nunjucks-dynamic-chain-orchestration.md
│ ├── outputschema-realtime-progress-and-validation.md
│ ├── parallel-conditional-execution-analysis.md
│ ├── sqlite-storage-migration.md
│ └── symbolic-command-language-implementation.md
├── README.md
├── scripts
│ ├── setup-windows-testing.sh
│ ├── test_server.js
│ ├── test-all-platforms.sh
│ └── windows-tests
│ ├── test-windows-paths.js
│ ├── test-windows-startup.sh
│ └── windows-env.sh
└── server
├── config
│ ├── framework-state.json
│ └── tool-descriptions.json
├── config.json
├── jest.config.cjs
├── LICENSE
├── package-lock.json
├── package.json
├── prompts
│ ├── analysis
│ │ ├── advanced_analysis_engine.md
│ │ ├── content_analysis.md
│ │ ├── deep_analysis.md
│ │ ├── deep_research.md
│ │ ├── markdown_notebook.md
│ │ ├── note_integration.md
│ │ ├── note_refinement.md
│ │ ├── notes.md
│ │ ├── progressive_research.md
│ │ ├── prompts.json
│ │ ├── query_refinement.md
│ │ └── review.md
│ ├── architecture
│ │ ├── prompts.json
│ │ └── strategic-system-alignment.md
│ ├── content_processing
│ │ ├── format_enhancement.md
│ │ ├── noteIntegration.md
│ │ ├── obsidian_metadata_optimizer.md
│ │ ├── prompts.json
│ │ ├── vault_related_notes_finder.md
│ │ └── video_notes_enhanced.md
│ ├── debugging
│ │ ├── analyze_logs.md
│ │ └── prompts.json
│ ├── development
│ │ ├── analyze_code_structure.md
│ │ ├── analyze_file_structure.md
│ │ ├── code_review_optimization_chain.md
│ │ ├── component_flow_analysis.md
│ │ ├── create_modularization_plan.md
│ │ ├── detect_code_issues.md
│ │ ├── detect_project_commands.md
│ │ ├── expert_code_implementation.md
│ │ ├── generate_comprehensive_claude_md.md
│ │ ├── prompts.json
│ │ ├── strategicImplement.md
│ │ ├── suggest_code_improvements.md
│ │ └── transform_code_to_modules.md
│ ├── documentation
│ │ ├── create_docs_chain.md
│ │ ├── docs-content-creation.md
│ │ ├── docs-content-planning.md
│ │ ├── docs-final-assembly.md
│ │ ├── docs-project-analysis.md
│ │ ├── docs-review-refinement.md
│ │ └── prompts.json
│ ├── education
│ │ ├── prompts.json
│ │ └── vault_integrated_notes.md
│ ├── general
│ │ ├── diagnose.md
│ │ └── prompts.json
│ ├── promptsConfig.json
│ └── testing
│ ├── final_verification_test.md
│ └── prompts.json
├── README.md
├── scripts
│ └── validate-dependencies.js
├── src
│ ├── api
│ │ └── index.ts
│ ├── chain-session
│ │ └── manager.ts
│ ├── config
│ │ └── index.ts
│ ├── Dockerfile
│ ├── execution
│ │ ├── context
│ │ │ ├── context-resolver.ts
│ │ │ ├── framework-injector.ts
│ │ │ └── index.ts
│ │ ├── index.ts
│ │ ├── parsers
│ │ │ ├── argument-parser.ts
│ │ │ ├── index.ts
│ │ │ └── unified-command-parser.ts
│ │ └── types.ts
│ ├── frameworks
│ │ ├── framework-manager.ts
│ │ ├── framework-state-manager.ts
│ │ ├── index.ts
│ │ ├── integration
│ │ │ ├── framework-semantic-integration.ts
│ │ │ └── index.ts
│ │ ├── methodology
│ │ │ ├── guides
│ │ │ │ ├── 5w1h-guide.ts
│ │ │ │ ├── cageerf-guide.ts
│ │ │ │ ├── react-guide.ts
│ │ │ │ └── scamper-guide.ts
│ │ │ ├── index.ts
│ │ │ ├── interfaces.ts
│ │ │ └── registry.ts
│ │ ├── prompt-guidance
│ │ │ ├── index.ts
│ │ │ ├── methodology-tracker.ts
│ │ │ ├── service.ts
│ │ │ ├── system-prompt-injector.ts
│ │ │ └── template-enhancer.ts
│ │ └── types
│ │ ├── index.ts
│ │ ├── integration-types.ts
│ │ ├── methodology-types.ts
│ │ └── prompt-guidance-types.ts
│ ├── gates
│ │ ├── constants.ts
│ │ ├── core
│ │ │ ├── gate-definitions.ts
│ │ │ ├── gate-loader.ts
│ │ │ ├── gate-validator.ts
│ │ │ ├── index.ts
│ │ │ └── temporary-gate-registry.ts
│ │ ├── definitions
│ │ │ ├── code-quality.json
│ │ │ ├── content-structure.json
│ │ │ ├── educational-clarity.json
│ │ │ ├── framework-compliance.json
│ │ │ ├── research-quality.json
│ │ │ ├── security-awareness.json
│ │ │ └── technical-accuracy.json
│ │ ├── gate-state-manager.ts
│ │ ├── guidance
│ │ │ ├── FrameworkGuidanceFilter.ts
│ │ │ └── GateGuidanceRenderer.ts
│ │ ├── index.ts
│ │ ├── intelligence
│ │ │ ├── GatePerformanceAnalyzer.ts
│ │ │ └── GateSelectionEngine.ts
│ │ ├── templates
│ │ │ ├── code_quality_validation.md
│ │ │ ├── educational_clarity_validation.md
│ │ │ ├── framework_compliance_validation.md
│ │ │ ├── research_self_validation.md
│ │ │ ├── security_validation.md
│ │ │ ├── structure_validation.md
│ │ │ └── technical_accuracy_validation.md
│ │ └── types.ts
│ ├── index.ts
│ ├── logging
│ │ └── index.ts
│ ├── mcp-tools
│ │ ├── config-utils.ts
│ │ ├── constants.ts
│ │ ├── index.ts
│ │ ├── prompt-engine
│ │ │ ├── core
│ │ │ │ ├── engine.ts
│ │ │ │ ├── executor.ts
│ │ │ │ ├── index.ts
│ │ │ │ └── types.ts
│ │ │ ├── index.ts
│ │ │ ├── processors
│ │ │ │ ├── response-formatter.ts
│ │ │ │ └── template-processor.ts
│ │ │ └── utils
│ │ │ ├── category-extractor.ts
│ │ │ ├── classification.ts
│ │ │ ├── context-builder.ts
│ │ │ └── validation.ts
│ │ ├── prompt-manager
│ │ │ ├── analysis
│ │ │ │ ├── comparison-engine.ts
│ │ │ │ ├── gate-analyzer.ts
│ │ │ │ └── prompt-analyzer.ts
│ │ │ ├── core
│ │ │ │ ├── index.ts
│ │ │ │ ├── manager.ts
│ │ │ │ └── types.ts
│ │ │ ├── index.ts
│ │ │ ├── operations
│ │ │ │ └── file-operations.ts
│ │ │ ├── search
│ │ │ │ ├── filter-parser.ts
│ │ │ │ └── prompt-matcher.ts
│ │ │ └── utils
│ │ │ ├── category-manager.ts
│ │ │ └── validation.ts
│ │ ├── shared
│ │ │ └── structured-response-builder.ts
│ │ ├── system-control.ts
│ │ ├── tool-description-manager.ts
│ │ └── types
│ │ └── shared-types.ts
│ ├── metrics
│ │ ├── analytics-service.ts
│ │ ├── index.ts
│ │ └── types.ts
│ ├── performance
│ │ ├── index.ts
│ │ └── monitor.ts
│ ├── prompts
│ │ ├── category-manager.ts
│ │ ├── converter.ts
│ │ ├── file-observer.ts
│ │ ├── hot-reload-manager.ts
│ │ ├── index.ts
│ │ ├── loader.ts
│ │ ├── promptUtils.ts
│ │ ├── registry.ts
│ │ └── types.ts
│ ├── runtime
│ │ ├── application.ts
│ │ └── startup.ts
│ ├── semantic
│ │ ├── configurable-semantic-analyzer.ts
│ │ └── integrations
│ │ ├── index.ts
│ │ └── llm-clients.ts
│ ├── server
│ │ ├── index.ts
│ │ └── transport
│ │ └── index.ts
│ ├── smithery.yaml
│ ├── text-references
│ │ ├── conversation.ts
│ │ └── index.ts
│ ├── types
│ │ └── index.ts
│ ├── types.ts
│ └── utils
│ ├── chainUtils.ts
│ ├── errorHandling.ts
│ ├── global-resource-tracker.ts
│ ├── index.ts
│ └── jsonUtils.ts
├── tests
│ ├── ci-startup-validation.js
│ ├── enhanced-validation
│ │ ├── contract-validation
│ │ │ ├── contract-test-suite.js
│ │ │ ├── interface-contracts.js
│ │ │ └── interface-contracts.ts
│ │ ├── environment-validation
│ │ │ ├── environment-parity-checker.js
│ │ │ └── environment-test-suite.js
│ │ ├── lifecycle-validation
│ │ │ ├── lifecycle-test-suite.js
│ │ │ └── process-lifecycle-validator.js
│ │ └── validation-orchestrator.js
│ ├── helpers
│ │ └── test-helpers.js
│ ├── integration
│ │ ├── mcp-tools.test.ts
│ │ ├── server-startup.test.ts
│ │ └── unified-parsing-integration.test.ts
│ ├── performance
│ │ ├── parsing-system-benchmark.test.ts
│ │ └── server-performance.test.ts
│ ├── scripts
│ │ ├── consolidated-tools.js
│ │ ├── establish-performance-baselines.js
│ │ ├── functional-mcp-validation.js
│ │ ├── integration-mcp-tools.js
│ │ ├── integration-routing-system.js
│ │ ├── integration-server-startup.js
│ │ ├── integration-unified-parsing.js
│ │ ├── methodology-guides.js
│ │ ├── performance-memory.js
│ │ ├── runtime-integration.js
│ │ ├── unit-conversation-manager.js
│ │ ├── unit-semantic-analyzer.js
│ │ └── unit-unified-parsing.js
│ ├── setup.ts
│ ├── test-enhanced-parsing.js
│ └── unit
│ ├── conversation-manager.test.ts
│ ├── semantic-analyzer-three-tier.test.ts
│ └── unified-parsing-system.test.ts
├── tsconfig.json
└── tsconfig.test.json
```
# Files
--------------------------------------------------------------------------------
/server/src/mcp-tools/index.ts:
--------------------------------------------------------------------------------
```typescript
/**
* MCP Tools Module - Consolidated Architecture
*
* This module provides 3 core MCP tools with framework-aware descriptions:
*
* CORE TOOLS:
* - prompt_engine: Universal execution engine with framework integration
* - prompt_manager: Complete prompt lifecycle management with intelligent filtering
* - system_control: Framework and system management with analytics
*
* ARCHITECTURE:
* - Framework-aware tool descriptions that change based on active methodology
* - Single source of truth for each functional area
* - Integrated ToolDescriptionManager for dynamic descriptions
* - Improved maintainability and clear separation of concerns
*/
import { z } from "zod";
import { ConfigManager } from "../config/index.js";
import { Logger } from "../logging/index.js";
import { PromptManager } from "../prompts/index.js";
import { Category, ConvertedPrompt, PromptData } from "../types/index.js";
// Gate evaluator removed - now using Framework methodology validation
import {
FrameworkManager,
createFrameworkManager,
} from "../frameworks/framework-manager.js";
import { FrameworkStateManager } from "../frameworks/framework-state-manager.js";
import { createContentAnalyzer } from "../semantic/configurable-semantic-analyzer.js";
import { createSemanticIntegrationFactory } from "../semantic/integrations/index.js";
import {
ConversationManager,
createConversationManager,
} from "../text-references/conversation.js";
import {
TextReferenceManager,
createTextReferenceManager,
} from "../text-references/index.js";
// REMOVED: ExecutionCoordinator and ChainOrchestrator - modular chain system removed
import { MetricsCollector, createMetricsCollector } from "../metrics/index.js";
// Consolidated tools
import {
ConsolidatedPromptEngine,
createConsolidatedPromptEngine,
} from "./prompt-engine/index.js";
import {
ConsolidatedPromptManager,
createConsolidatedPromptManager,
} from "./prompt-manager/index.js";
import {
ConsolidatedSystemControl,
createConsolidatedSystemControl,
} from "./system-control.js";
import { ToolDescriptionManager } from "./tool-description-manager.js";
// Gate system management integration
import {
GateSystemManager,
createGateSystemManager,
} from "../gates/gate-state-manager.js";
/**
* Consolidated MCP Tools Manager
*
* Manages 3 intelligent consolidated tools that replace the previous 24+ scattered tools
*/
export class ConsolidatedMcpToolsManager {
private logger: Logger;
private mcpServer: any;
private promptManager: PromptManager;
private configManager: ConfigManager;
// Consolidated tools (3 instead of 24+)
private promptEngine!: ConsolidatedPromptEngine;
private promptManagerTool!: ConsolidatedPromptManager;
private systemControl!: ConsolidatedSystemControl;
// Core tools: prompt engine, manager, and system control
// Shared components
private semanticAnalyzer!: ReturnType<typeof createContentAnalyzer>;
private frameworkStateManager?: FrameworkStateManager;
private frameworkManager?: FrameworkManager;
// REMOVED: chainOrchestrator - modular chain system removed
private conversationManager!: ConversationManager;
private textReferenceManager!: TextReferenceManager;
private toolDescriptionManager?: ToolDescriptionManager;
private gateSystemManager?: GateSystemManager;
private analyticsService!: MetricsCollector;
// Phase 3: Removed executionCoordinator - chains now use LLM-driven execution model
// Callback references
private onRestart?: (reason: string) => Promise<void>;
// Data references
private promptsData: PromptData[] = [];
private convertedPrompts: ConvertedPrompt[] = [];
private categories: Category[] = [];
// Pending analytics queue for initialization race condition
private pendingAnalytics: any[] = [];
constructor(
logger: Logger,
mcpServer: any,
promptManager: PromptManager,
configManager: ConfigManager
// Phase 3: Removed executionCoordinator parameter - using LLM-driven chain model
) {
this.logger = logger;
this.mcpServer = mcpServer;
this.promptManager = promptManager;
this.configManager = configManager;
// Phase 3: Removed executionCoordinator assignment - using LLM-driven chain model
}
/**
* Initialize the MCP tools with async configuration
*/
async initialize(
onRefresh: () => Promise<void>,
onRestart: (reason: string) => Promise<void>
): Promise<void> {
// Store callback references
this.onRestart = onRestart;
// Initialize shared components with configurable analysis
const analysisConfig = this.configManager.getSemanticAnalysisConfig();
const integrationFactory = createSemanticIntegrationFactory(this.logger);
this.semanticAnalyzer = await integrationFactory.createFromEnvironment(
analysisConfig
);
this.conversationManager = createConversationManager(this.logger);
this.textReferenceManager = createTextReferenceManager(this.logger);
this.analyticsService = createMetricsCollector(this.logger);
// Initialize gate system manager for runtime gate control
this.gateSystemManager = createGateSystemManager(this.logger, undefined);
await this.gateSystemManager.initialize();
// Integrate text reference manager with conversation manager
this.conversationManager.setTextReferenceManager(this.textReferenceManager);
this.logger.info(
`Configurable semantic analyzer initialized (mode: ${analysisConfig.mode})`
);
// Initialize consolidated tools
this.promptEngine = createConsolidatedPromptEngine(
this.logger,
this.mcpServer,
this.promptManager,
this.configManager,
this.semanticAnalyzer,
this.conversationManager,
this.textReferenceManager,
this // Pass manager reference for analytics data flow
// Phase 3: Removed executionCoordinator - chains now use LLM-driven execution
);
// Set gate system manager in prompt engine
this.promptEngine.setGateSystemManager(this.gateSystemManager);
this.promptManagerTool = createConsolidatedPromptManager(
this.logger,
this.mcpServer,
this.configManager,
this.semanticAnalyzer,
this.frameworkStateManager,
this.frameworkManager,
onRefresh,
onRestart
);
// Initialize 3 core consolidated tools
this.systemControl = createConsolidatedSystemControl(
this.logger,
this.mcpServer,
onRestart
);
// Set gate system manager in system control
this.systemControl.setGateSystemManager(this.gateSystemManager);
this.systemControl.setGateGuidanceRenderer(
this.promptEngine.getGateGuidanceRenderer()
);
// chainScaffolder removed - functionality consolidated into promptEngine
// Flush any pending analytics data that was queued during initialization
this.flushPendingAnalytics();
this.logger.info(
"Consolidated MCP Tools Manager initialized with 3 intelligent tools (chain management in prompt_engine)"
);
}
/**
* Set framework state manager (called after initialization)
*/
setFrameworkStateManager(frameworkStateManager: FrameworkStateManager): void {
this.frameworkStateManager = frameworkStateManager;
this.promptEngine.setFrameworkStateManager(frameworkStateManager);
this.systemControl.setFrameworkStateManager(frameworkStateManager);
this.promptManagerTool.setFrameworkStateManager?.(frameworkStateManager);
// FIXED: Synchronize Framework Manager with Framework State Manager to prevent injection duplication
if (this.frameworkManager) {
this.frameworkManager.setFrameworkStateManager(frameworkStateManager);
}
// Core tools handle framework state integration
}
/**
* Set tool description manager (called after initialization)
*/
setToolDescriptionManager(manager: ToolDescriptionManager): void {
this.toolDescriptionManager = manager;
this.promptEngine.setToolDescriptionManager(manager);
this.promptEngine.setAnalyticsService(this.analyticsService);
// promptManagerTool doesn't have setToolDescriptionManager method
this.systemControl.setToolDescriptionManager?.(manager);
this.systemControl.setAnalyticsService(this.analyticsService);
// Core tools integrated with framework-aware descriptions
// Set up hot-reload event listeners
this.setupToolDescriptionHotReload(manager);
this.logger.info(
"Tool description manager set for all MCP tools with hot-reload support"
);
}
/**
* Setup hot-reload event listeners for tool descriptions
*/
private setupToolDescriptionHotReload(manager: ToolDescriptionManager): void {
// Listen for description changes
manager.on("descriptions-changed", (stats) => {
this.logger.info(
`🔥 Tool descriptions hot-reloaded: ${stats.totalDescriptions} descriptions loaded`
);
this.handleToolDescriptionChange(stats);
});
// Listen for reload errors
manager.on("descriptions-error", (error) => {
this.logger.error(
`❌ Tool description reload failed: ${
error instanceof Error ? error.message : String(error)
}`
);
});
// Start file watching if not already watching
if (!manager.isWatchingFile()) {
manager.startWatching();
}
}
/**
* Handle tool description changes
*/
private async handleToolDescriptionChange(stats: any): Promise<void> {
try {
this.logger.info("🔄 Processing tool description changes...");
// Emit analytics update
this.updateAnalytics({
toolDescriptions: {
lastReload: new Date().toISOString(),
totalDescriptions: stats.totalDescriptions,
loadedFromFile: stats.loadedFromFile,
usingDefaults: stats.usingDefaults,
},
});
// Note: MCP SDK doesn't support dynamic tool updates
// The new descriptions will be loaded on next tool registration or server restart
this.logger.info("✅ Tool descriptions reloaded from file");
this.logger.info(
`📊 Stats: ${stats.totalDescriptions} total, using ${
stats.usingDefaults > 0 ? "defaults" : "external config"
}`
);
// Check if restart is configured for tool description changes
const restartOnChange =
this.configManager.getConfig().toolDescriptions?.restartOnChange ??
false;
if (restartOnChange) {
this.logger.info(
"🚨 Restart on tool description change is enabled - initiating server restart..."
);
// Use the existing restart mechanism
await this.onRestart?.(
"Tool descriptions updated - restart required for clients to see new descriptions"
);
} else {
this.logger.info(
"💡 Tip: New tool descriptions will be used for new client connections. For immediate effect, restart the server manually or enable 'restartOnChange' in config."
);
}
} catch (error) {
this.logger.error(
`Failed to handle tool description change: ${
error instanceof Error ? error.message : String(error)
}`
);
}
}
/**
* Initialize and set framework manager (called after framework state manager)
*/
async setFrameworkManager(
existingFrameworkManager?: FrameworkManager
): Promise<void> {
if (!this.frameworkManager) {
// Use provided framework manager or create a new one
this.frameworkManager =
existingFrameworkManager || (await createFrameworkManager(this.logger));
this.promptEngine.setFrameworkManager(this.frameworkManager);
this.systemControl.setFrameworkManager(this.frameworkManager);
this.promptManagerTool.setFrameworkManager?.(this.frameworkManager);
// Core tools integrated with framework management
// Set ConfigManager for system control config operations
this.systemControl.setConfigManager(this.configManager);
// Set MCPToolsManager reference for dynamic tool updates
this.systemControl.setMCPToolsManager(this);
// Enhanced tool delegation removed (Phase 1.2)
// Using core tools directly without delegation patterns
// REMOVED: ChainOrchestrator initialization - modular chain system removed
if (existingFrameworkManager) {
this.logger.info(
"Framework manager integrated with MCP tools (shared instance)"
);
} else {
this.logger.info(
"Framework manager initialized and integrated with MCP tools"
);
}
}
}
// REMOVED: wireExecutionCoordinator - ExecutionCoordinator removed
/**
* Register all consolidated MCP tools with the server (centralized registration)
*/
async registerAllTools(): Promise<void> {
this.logger.info(
"Registering consolidated MCP tools with server (centralized)..."
);
// Get current framework state for dynamic descriptions
const frameworkEnabled =
this.frameworkStateManager?.isFrameworkSystemEnabled() ?? false;
const activeFramework = this.frameworkStateManager?.getActiveFramework();
const activeMethodology =
activeFramework?.methodology ?? activeFramework?.id;
this.logger.info(`🔧 Registering tools with framework-aware descriptions:`);
this.logger.info(` Framework enabled: ${frameworkEnabled}`);
this.logger.info(` Active framework: ${activeFramework?.id || "none"}`);
this.logger.info(` Active methodology: ${activeMethodology || "none"}`);
this.logger.info(
` Tool description manager: ${
this.toolDescriptionManager ? "available" : "not available"
}`
);
// Register prompt_engine tool
try {
// Get dynamic description based on current framework state
const promptEngineDescription =
this.toolDescriptionManager?.getDescription(
"prompt_engine",
frameworkEnabled,
activeMethodology,
{ applyMethodologyOverride: true }
) ??
"🚀 PROMPT ENGINE [HOT-RELOAD]: Processes Nunjucks templates, returns executable instructions. WARNING: Output contains instructions YOU must execute (code gen, analysis, multi-step tasks) - not just information. IMPORTANT: When your arguments include newlines or structured blocks, wrap the call in JSON so the parser receives a single-line command shell.";
const getPromptEngineParamDescription = (
paramName: string,
fallback: string
) =>
this.toolDescriptionManager?.getParameterDescription(
"prompt_engine",
paramName,
frameworkEnabled,
activeMethodology,
{ applyMethodologyOverride: true }
) ?? fallback;
// Log which description source is being used for transparency
if (this.toolDescriptionManager) {
this.logger.info(
` prompt_engine: Using ToolDescriptionManager (framework: ${frameworkEnabled}, methodology: ${activeMethodology})`
);
} else {
this.logger.info(
` prompt_engine: Using fallback description (ToolDescriptionManager not available)`
);
}
this.mcpServer.registerTool(
"prompt_engine",
{
title: "Prompt Engine",
description: promptEngineDescription,
inputSchema: {
command: z
.string()
.min(1, "Command cannot be empty")
.describe(
getPromptEngineParamDescription(
"command",
"Prompt name and arguments to process. WARNING: Will return instructions for YOU to execute, not just information. SIMPLE: >>prompt_name content (single-line arguments only). MULTI-LINE / RICH FORMATTING: {\\\"command\\\": \\\" >>prompt_name\\\", \\\"args\\\":{...}} keeps payload intact. ADVANCED: JSON with execution options"
)
),
execution_mode: z
.enum(["auto", "prompt", "template", "chain"])
.optional()
.describe(
getPromptEngineParamDescription(
"execution_mode",
"Override intelligent auto-detection (default: auto). 'auto' intelligently detects execution type, 'prompt' for single execution, 'template' for variable substitution, 'chain' for multi-step workflows"
)
),
gate_validation: z
.boolean()
.optional()
.describe(
getPromptEngineParamDescription(
"gate_validation",
"Quality gate validation (MANDATORY for chains, auto-detected by default, see metadata sections for gate details)"
)
),
step_confirmation: z
.boolean()
.optional()
.describe(
getPromptEngineParamDescription(
"step_confirmation",
"Require confirmation between chain steps"
)
),
llm_driven_execution: z
.boolean()
.optional()
.describe(
getPromptEngineParamDescription(
"llm_driven_execution",
"Enable LLM-driven chain step coordination (requires semantic LLM integration)"
)
),
force_restart: z
.boolean()
.optional()
.describe(
getPromptEngineParamDescription(
"force_restart",
"Force restart chain from beginning, clearing all existing state"
)
),
session_id: z
.string()
.min(1, "Session ID cannot be empty if provided")
.regex(
/^[a-zA-Z0-9_-]+$/,
"Session ID must contain only alphanumeric characters, underscores, and hyphens"
)
.optional()
.describe(
getPromptEngineParamDescription(
"session_id",
"Specific session ID to use or resume"
)
),
chain_uri: z
.string()
.optional()
.describe(
getPromptEngineParamDescription(
"chain_uri",
"Full chain URI for precise session control (e.g., chain://research_pipeline/session-abc123?force_restart=true)"
)
),
timeout: z
.number()
.int()
.positive()
.optional()
.describe(
getPromptEngineParamDescription(
"timeout",
"Execution timeout in milliseconds"
)
),
options: z
.record(z.any())
.optional()
.describe(
getPromptEngineParamDescription(
"options",
"Additional execution options (key-value pairs). Supports framework-specific flags, debugging controls, and experimental features"
)
),
},
},
async (args: {
command: string;
execution_mode?: "auto" | "prompt" | "template" | "chain";
gate_validation?: boolean;
step_confirmation?: boolean;
llm_driven_execution?: boolean;
force_restart?: boolean;
session_id?: string;
chain_uri?: string;
timeout?: number;
options?: Record<string, any>;
}) => {
try {
const toolResponse = await this.promptEngine.executePromptCommand(
args,
{}
);
return {
content: toolResponse.content,
isError: toolResponse.isError,
...(toolResponse.structuredContent && {
structuredContent: toolResponse.structuredContent,
}),
};
} catch (error) {
this.logger.error(
`prompt_engine error: ${
error instanceof Error ? error.message : String(error)
}`
);
return {
content: [
{
type: "text",
text: `Error: ${
error instanceof Error ? error.message : String(error)
}`,
},
],
isError: true,
};
}
}
);
this.logger.debug("✅ prompt_engine tool registered successfully");
} catch (error) {
this.logger.error(
`❌ Failed to register prompt_engine tool: ${
error instanceof Error ? error.message : String(error)
}`
);
throw error;
}
// Register prompt_manager tool
try {
// Get dynamic description based on current framework state
const promptManagerDescription =
this.toolDescriptionManager?.getDescription(
"prompt_manager",
frameworkEnabled,
activeMethodology,
{ applyMethodologyOverride: true }
) ??
"🧰 PROMPT MANAGER: Create, update, delete, list, and analyze prompts. Supports gate configuration, temporary gates, and prompt-type migration.";
const getPromptManagerParamDescription = (
paramName: string,
fallback: string
) =>
this.toolDescriptionManager?.getParameterDescription(
"prompt_manager",
paramName,
frameworkEnabled,
activeMethodology,
{ applyMethodologyOverride: true }
) ?? fallback;
// Log which description source is being used for transparency
if (this.toolDescriptionManager) {
this.logger.info(
` prompt_manager: Using ToolDescriptionManager (framework: ${frameworkEnabled}, methodology: ${activeMethodology})`
);
} else {
this.logger.info(
` prompt_manager: Using fallback description (ToolDescriptionManager not available)`
);
}
this.mcpServer.registerTool(
"prompt_manager",
{
title: "Prompt Manager",
description: promptManagerDescription,
inputSchema: {
action: z
.enum([
"create",
"create_prompt",
"create_template",
"analyze_type",
"migrate_type",
"update",
"delete",
"modify",
"reload",
"list",
"analyze_gates",
"suggest_temporary_gates",
"create_with_gates",
"update_gates",
"add_temporary_gates",
])
.describe(
getPromptManagerParamDescription(
"action",
"Action to perform. Supported: create, create_prompt, create_template, create_with_gates, update, delete, modify, reload, list, analyze_type, migrate_type, analyze_gates, update_gates, add_temporary_gates, suggest_temporary_gates."
)
),
id: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"id",
"Prompt identifier. Required for create*, update, delete, modify, analyze_type, migrate_type, analyze_gates, update_gates, add_temporary_gates. Use letters, numbers, underscores, or hyphens."
)
),
name: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"name",
"Friendly prompt name. Required for create*, create_with_gates."
)
),
description: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"description",
"Short description of the prompt purpose. Required for create*, create_with_gates."
)
),
user_message_template: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"user_message_template",
"Prompt body with Nunjucks placeholders (e.g. 'Analyze {{input}}'). Required for create*, create_with_gates."
)
),
system_message: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"system_message",
"Optional system message to store with the prompt."
)
),
content: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"content",
"Full prompt content for create/update operations when not using templates."
)
),
category: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"category",
"Category label used for filtering and organization."
)
),
arguments: z
.array(
z.object({
name: z.string(),
type: z.string(),
description: z.string(),
})
)
.optional()
.describe(
getPromptManagerParamDescription(
"arguments",
"Array of argument definitions ({name, type, description}) for prompts with structured inputs."
)
),
suggested_gates: z
.array(
z.object({
type: z.enum(['validation', 'quality', 'approval', 'condition', 'guidance']),
name: z.string(),
description: z.string(),
criteria: z.array(z.string()).optional(),
})
)
.optional()
.describe(
getPromptManagerParamDescription(
"suggested_gates",
"Gate suggestions used by create_with_gates. Each entry should include type, name, description, and optional criteria."
)
),
gate_configuration: z
.object({
include: z.array(z.string()).optional(),
exclude: z.array(z.string()).optional(),
temporary_gates: z.array(z.any()).optional(),
framework_gates: z.boolean().optional(),
})
.optional()
.describe(
getPromptManagerParamDescription(
"gate_configuration",
"Explicit gate configuration (include/exclude lists, temporary gates, framework_gates flag)."
)
),
temporary_gates: z
.array(z.any())
.optional()
.describe(
getPromptManagerParamDescription(
"temporary_gates",
"Temporary gate definitions used by add_temporary_gates (include name, type, scope, description, guidance, pass_criteria)."
)
),
gate_scope: z
.enum(['execution', 'session', 'chain', 'step'])
.optional()
.describe(
getPromptManagerParamDescription(
"gate_scope",
"Scope for temporary gates (execution, session, chain, step)."
)
),
inherit_chain_gates: z
.boolean()
.optional()
.describe(
getPromptManagerParamDescription(
"inherit_chain_gates",
"When true, inherit gates from the parent chain (default true for add_temporary_gates)."
)
),
search_query: z
.string()
.optional()
.describe(
getPromptManagerParamDescription(
"search_query",
"Search expression for list (e.g. 'category:code type:chain')."
)
),
force: z
.boolean()
.optional()
.describe(
getPromptManagerParamDescription(
"force",
"Bypass confirmation prompts for supported actions."
)
),
},
},
async (args: {
action:
| "create"
| "create_prompt"
| "create_template"
| "analyze_type"
| "migrate_type"
| "update"
| "delete"
| "modify"
| "reload"
| "list"
| "analyze_gates"
| "suggest_temporary_gates"
| "create_with_gates"
| "update_gates"
| "add_temporary_gates";
[key: string]: any;
}) => {
try {
// Check if promptManagerTool exists
if (!this.promptManagerTool) {
this.logger.error(`ERROR: promptManagerTool is undefined!`);
return {
content: [
{
type: "text",
text: `Error: promptManagerTool is not initialized`,
},
],
isError: true,
};
}
// CRITICAL: Adding error-level logging to trace MCP tool invocation
this.logger.error(`[GATE-TRACE] 🔥 MCP TOOL INVOCATION: About to call promptManagerTool.handleAction for action: ${args.action}`);
this.logger.error(`[GATE-TRACE] 🔥 MCP args being passed:`, args);
const toolResponse = await this.promptManagerTool.handleAction(
args,
{}
);
this.logger.error(`[GATE-TRACE] 🔥 MCP TOOL RESPONSE: handleAction completed for action: ${args.action}`);
// Debug logging and validation
if (!toolResponse) {
this.logger.error(
`prompt_manager returned undefined response for action: ${args.action}`
);
return {
content: [
{
type: "text",
text: `Error: Tool returned undefined response`,
},
],
isError: true,
};
}
if (!toolResponse.content) {
this.logger.error(
`prompt_manager returned response with undefined content for action: ${args.action}`,
toolResponse
);
return {
content: [
{
type: "text",
text: `Error: Tool returned response with undefined content`,
},
],
isError: true,
};
}
return {
content: toolResponse.content,
isError: toolResponse.isError,
...(toolResponse.structuredContent && {
structuredContent: toolResponse.structuredContent,
}),
};
} catch (error) {
this.logger.error(
`prompt_manager error: ${
error instanceof Error ? error.message : String(error)
}`
);
return {
content: [
{
type: "text",
text: `Error: ${
error instanceof Error ? error.message : String(error)
}`,
},
],
isError: true,
};
}
}
);
this.logger.debug("✅ prompt_manager tool registered successfully");
} catch (error) {
this.logger.error(
`❌ Failed to register prompt_manager tool: ${
error instanceof Error ? error.message : String(error)
}`
);
throw error;
}
// Register system_control tool
try {
// Get dynamic description based on current framework state
const systemControlDescription =
this.toolDescriptionManager?.getDescription(
"system_control",
frameworkEnabled,
activeMethodology,
{ applyMethodologyOverride: true }
) ??
"⚙️ SYSTEM CONTROL: Unified interface for status, framework controls, gate management, analytics, configuration, and maintenance actions.";
// Log which description source is being used for transparency
if (this.toolDescriptionManager) {
this.logger.info(
` system_control: Using ToolDescriptionManager (framework: ${frameworkEnabled}, methodology: ${activeMethodology})`
);
} else {
this.logger.info(
` system_control: Using fallback description (ToolDescriptionManager not available)`
);
}
const getSystemControlParamDescription = (
paramName: string,
fallback: string
) =>
this.toolDescriptionManager?.getParameterDescription(
"system_control",
paramName,
frameworkEnabled,
activeMethodology,
{ applyMethodologyOverride: true }
) ?? fallback;
this.mcpServer.registerTool(
"system_control",
{
title: "System Control",
description: systemControlDescription,
inputSchema: {
action: z
.string()
.describe(
getSystemControlParamDescription(
"action",
"Top-level command. Supported values: status, framework, gates, analytics, config, maintenance."
)
),
operation: z
.string()
.optional()
.describe(
getSystemControlParamDescription(
"operation",
"Sub-command for the selected action (e.g. framework: switch|list|enable|disable, analytics: view|reset|history)."
)
),
framework: z
.string()
.optional()
.describe(
getSystemControlParamDescription(
"framework",
"Framework identifier when switching (CAGEERF, ReACT, 5W1H, SCAMPER)."
)
),
config_path: z
.string()
.optional()
.describe(
getSystemControlParamDescription(
"config_path",
"Configuration path or key for config operations."
)
),
config_value: z
.any()
.optional()
.describe(
getSystemControlParamDescription(
"config_value",
"Value to write when performing config updates."
)
),
restart_reason: z
.string()
.optional()
.describe(
getSystemControlParamDescription(
"restart_reason",
"Specific reason recorded for maintenance/restart operations."
)
),
reason: z
.string()
.optional()
.describe(
getSystemControlParamDescription(
"reason",
"Audit-friendly explanation for switches, config changes, or restarts."
)
),
include_history: z
.boolean()
.optional()
.describe(
getSystemControlParamDescription(
"include_history",
"Include historical entries (where supported)."
)
),
include_metrics: z
.boolean()
.optional()
.describe(
getSystemControlParamDescription(
"include_metrics",
"Include detailed metrics output (where supported)."
)
),
show_details: z
.boolean()
.optional()
.describe(
getSystemControlParamDescription(
"show_details",
"Request an expanded response for list/status style commands."
)
),
},
},
async (args: { action: string; [key: string]: any }) => {
try {
const toolResponse = await this.systemControl.handleAction(
args,
{}
);
return {
content: toolResponse.content,
isError: toolResponse.isError,
...(toolResponse.structuredContent && {
structuredContent: toolResponse.structuredContent,
}),
};
} catch (error) {
this.logger.error(
`system_control error: ${
error instanceof Error ? error.message : String(error)
}`
);
return {
content: [
{
type: "text",
text: `Error: ${
error instanceof Error ? error.message : String(error)
}`,
},
],
isError: true,
};
}
}
);
this.logger.debug("✅ system_control tool registered successfully");
} catch (error) {
this.logger.error(
`❌ Failed to register system_control tool: ${
error instanceof Error ? error.message : String(error)
}`
);
throw error;
}
this.logger.info("🎉 Centralized MCP tools registered successfully!");
this.logger.info("📊 Core Tools: 3 centrally managed tools");
this.logger.info(
"🚀 Active Tools: prompt_engine, prompt_manager, system_control"
);
// Log available tools for user reference
const toolSummary = [
"Available Core Tools (centralized registration):",
"🎯 prompt_engine - Centralized prompt execution engine",
"🎯 prompt_manager - Centralized prompt lifecycle management",
"⚙️ system_control - Centralized framework and system management",
].join("\n ");
this.logger.info(toolSummary);
}
/**
* Re-register all tools with updated descriptions (for framework switching) - centralized version
*/
async reregisterToolsWithUpdatedDescriptions(): Promise<void> {
this.logger.info(
"🔄 Re-registering tools with updated framework-aware descriptions..."
);
try {
// Simply call the centralized registration method which now uses dynamic descriptions
await this.registerAllTools();
// Notify MCP clients that tool list has changed
if (this.mcpServer?.server?.sendToolListChanged) {
await this.mcpServer.server.sendToolListChanged();
this.logger.info(
"✅ Sent tool list changed notification to MCP clients"
);
} else {
this.logger.warn(
"⚠️ MCP server does not support sendToolListChanged notification"
);
}
this.logger.info(
"🎉 Tools re-registered successfully with updated descriptions (centralized)!"
);
} catch (error) {
this.logger.error(
`Failed to re-register tools: ${
error instanceof Error ? error.message : String(error)
}`
);
throw error;
}
}
/**
* Update internal data references
*/
updateData(
promptsData: PromptData[],
convertedPrompts: ConvertedPrompt[],
categories: Category[]
): void {
this.promptsData = promptsData;
this.convertedPrompts = convertedPrompts;
this.categories = categories;
// Update all consolidated tools with new data
this.promptEngine.updateData(promptsData, convertedPrompts);
this.promptManagerTool.updateData(
promptsData,
convertedPrompts,
categories
);
// Core tools handle data updates directly
}
/**
* Update system analytics (from consolidated tools)
*/
updateAnalytics(analytics: any): void {
if (this.systemControl) {
this.systemControl.updateAnalytics(analytics);
} else {
// Queue analytics data until systemControl is initialized
this.pendingAnalytics.push(analytics);
this.logger.debug(
`SystemControl not yet initialized, queued analytics data (${this.pendingAnalytics.length} pending)`
);
}
}
/**
* Flush pending analytics data to systemControl after initialization
*/
private flushPendingAnalytics(): void {
if (this.systemControl && this.pendingAnalytics.length > 0) {
this.logger.debug(
`Flushing ${this.pendingAnalytics.length} pending analytics updates`
);
this.pendingAnalytics.forEach((analytics) => {
this.systemControl.updateAnalytics(analytics);
});
this.pendingAnalytics = [];
}
}
/**
* Shutdown all components and cleanup resources
*/
shutdown(): void {
this.logger.info("🛑 Shutting down MCP tools manager...");
// Shutdown tool description manager and stop file watching
if (this.toolDescriptionManager) {
this.toolDescriptionManager.shutdown();
this.logger.info("✅ Tool description manager shut down");
}
// Cleanup gate system manager
if (this.gateSystemManager) {
this.gateSystemManager.cleanup().catch((error) => {
this.logger.error("Error during gate system manager cleanup:", error);
});
this.logger.info("✅ Gate system manager cleanup initiated");
}
// Clear pending analytics
this.pendingAnalytics = [];
this.logger.info("✅ MCP tools manager shutdown completed");
}
}
/**
* Create consolidated MCP tools manager
*/
export async function createConsolidatedMcpToolsManager(
logger: Logger,
mcpServer: any,
promptManager: PromptManager,
configManager: ConfigManager,
onRefresh: () => Promise<void>,
onRestart: (reason: string) => Promise<void>
// Phase 3: Removed executionCoordinator parameter - using LLM-driven chain model
): Promise<ConsolidatedMcpToolsManager> {
const manager = new ConsolidatedMcpToolsManager(
logger,
mcpServer,
promptManager,
configManager
// Phase 3: Removed executionCoordinator parameter
);
await manager.initialize(onRefresh, onRestart);
return manager;
}
// Legacy compatibility - export the consolidated manager as the old name
export { ConsolidatedMcpToolsManager as McpToolsManager };
export const createMcpToolsManager = createConsolidatedMcpToolsManager;
```
--------------------------------------------------------------------------------
/server/src/runtime/application.ts:
--------------------------------------------------------------------------------
```typescript
/**
* Application Runtime Management
* Manages application lifecycle, module coordination, and system health
*
* This is the streamlined version of the original ApplicationOrchestrator,
* focused on runtime concerns while delegating execution to the execution engine.
*/
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { promises as fs } from "fs";
import path from "path";
import { fileURLToPath } from "url";
// Import all module managers
import { ApiManager, createApiManager } from "../api/index.js";
import { ConfigManager } from "../config/index.js";
import {
createFrameworkStateManager,
FrameworkStateManager,
} from "../frameworks/framework-state-manager.js";
import {
createLogger,
EnhancedLoggingConfig,
Logger,
} from "../logging/index.js";
import { createMcpToolsManager, McpToolsManager } from "../mcp-tools/index.js";
import { createToolDescriptionManager, ToolDescriptionManager } from "../mcp-tools/tool-description-manager.js";
import { PromptManager } from "../prompts/index.js";
import {
ServerManager,
startMcpServer,
createTransportManager,
TransportManager,
} from "../server/index.js";
import { TextReferenceManager } from "../text-references/index.js";
// Import execution modules
import {
ConversationManager,
createConversationManager,
} from "../text-references/conversation.js";
// REMOVED: ExecutionCoordinator and GateEvaluator imports - modular chain and gate systems removed
// Phase 1: Framework capabilities now integrated into base components
// No separate framework observers needed - functionality moved to enhanced FileObserver and HotReloadManager
// Import startup management
import { ServerRootDetector } from "./startup.js";
// Import types
import { Category, ConvertedPrompt, PromptData } from "../types/index.js";
// Import chain utilities
import { isChainPrompt } from "../utils/chainUtils.js";
/**
* Application Runtime class
* Coordinates all modules and manages application lifecycle
*/
export class Application {
private logger: Logger;
private configManager: ConfigManager;
private textReferenceManager: TextReferenceManager;
private conversationManager: ConversationManager;
private promptManager: PromptManager;
// REMOVED: executionCoordinator - modular chain system removed
// REMOVED: gateEvaluator - gate evaluation system removed
private mcpToolsManager: McpToolsManager;
private toolDescriptionManager: ToolDescriptionManager;
private frameworkStateManager: FrameworkStateManager;
private transportManager: TransportManager;
private apiManager?: ApiManager;
private serverManager?: ServerManager;
// Phase 1: Framework capabilities integrated into base components
// No separate framework observers needed
// MCP Server instance
private mcpServer: McpServer;
// Application data
private _promptsData: PromptData[] = [];
private _categories: Category[] = [];
private _convertedPrompts: ConvertedPrompt[] = [];
// Performance monitoring
private memoryOptimizationInterval?: NodeJS.Timeout;
// Server root detector
private serverRootDetector: ServerRootDetector;
// Debug output control
private debugOutput: boolean;
/**
* Conditional debug logging to prevent output flood during tests
*/
private debugLog(message: string, ...args: any[]): void {
if (this.debugOutput) {
console.error(`DEBUG: ${message}`, ...args);
}
}
constructor(logger?: Logger) {
// Will be initialized in startup() if not provided
this.logger = logger || (null as any);
this.configManager = null as any;
this.textReferenceManager = null as any;
this.conversationManager = null as any;
this.promptManager = null as any;
// REMOVED: executionCoordinator - modular chain system removed
// REMOVED: gateEvaluator - gate evaluation system removed
this.mcpToolsManager = null as any;
this.toolDescriptionManager = null as any;
this.frameworkStateManager = null as any;
this.transportManager = null as any;
this.mcpServer = null as any;
// Phase 1: Framework capabilities integrated into base components
// Phase 3 consensus observer removed
this.serverRootDetector = new ServerRootDetector();
// Initialize debug output control - suppress in test environments
this.debugOutput = !this.isTestEnvironment();
}
/**
* Detect if running in test environment to suppress debug output
*/
private isTestEnvironment(): boolean {
return (
process.env.NODE_ENV === 'test' ||
process.argv.includes('--suppress-debug') ||
process.argv.includes('--test-mode') ||
// Detect GitHub Actions CI environment
process.env.GITHUB_ACTIONS === 'true' ||
process.env.CI === 'true' ||
// Detect common test runner patterns
process.argv.some(arg => arg.includes('test') || arg.includes('jest') || arg.includes('mocha')) ||
// Detect if called from integration test scripts
process.argv[1]?.includes('tests/scripts/')
);
}
/**
* Initialize all modules in the correct order
*/
async startup(): Promise<void> {
try {
// Phase 1: Core Foundation
this.debugLog("Starting Phase 1 - Core Foundation...");
await this.initializeFoundation();
this.debugLog("Phase 1 completed successfully");
// Phase 2: Data Loading and Processing
this.debugLog("Starting Phase 2 - Data Loading and Processing...");
await this.loadAndProcessData();
this.debugLog("Phase 2 completed successfully");
// Phase 3: Module Initialization
this.debugLog("Starting Phase 3 - Module Initialization...");
await this.initializeModulesPrivate();
this.debugLog("Phase 3 completed successfully");
// Phase 4: Server Setup and Startup
this.debugLog("Starting Phase 4 - Server Setup and Startup...");
// Check if this is a startup test mode
const args = process.argv.slice(2);
const isStartupTest = args.includes("--startup-test");
await this.startServer(isStartupTest);
this.debugLog("Phase 4 completed successfully");
console.error(
"DEBUG: All startup phases completed, server should be running..."
);
this.logger.info("Application startup completed successfully");
} catch (error) {
if (this.logger) {
this.logger.error("Error during application startup:", error);
} else {
console.error("Error during application startup:", error);
}
throw error;
}
}
/**
* Public test methods for GitHub Actions compatibility
*/
async loadConfiguration(): Promise<void> {
await this.initializeFoundation();
}
async loadPromptsData(): Promise<void> {
await this.loadAndProcessData();
}
// Make initializeModules public for testing
async initializeModules(): Promise<void> {
return this.initializeModulesPrivate();
}
// Expose data for testing
get config() {
return this.configManager?.getConfig();
}
get promptsData() {
return this._promptsData;
}
get convertedPrompts() {
return this._convertedPrompts;
}
get categories() {
return this._categories;
}
/**
* Phase 1: Initialize foundation (configuration, logging, basic services)
*/
private async initializeFoundation(): Promise<void> {
// Determine server root directory robustly
const serverRoot = await this.serverRootDetector.determineServerRoot();
this.debugLog("Server root detected:", serverRoot);
// Initialize configuration manager using the detected server root
this.debugLog("About to call path.join with serverRoot:", serverRoot);
const CONFIG_FILE = path.join(serverRoot, "config.json");
this.debugLog("Config file path:", CONFIG_FILE);
this.debugLog("About to create ConfigManager with CONFIG_FILE:", CONFIG_FILE);
try {
this.configManager = new ConfigManager(CONFIG_FILE);
this.debugLog("ConfigManager created successfully");
} catch (error) {
this.debugLog("ConfigManager creation failed:", error);
throw error;
}
this.debugLog("About to load config");
try {
await this.configManager.loadConfig();
this.debugLog("Config loaded successfully");
} catch (error) {
this.debugLog("Config loading failed:", error);
throw error;
}
// Determine transport from command line arguments
const args = process.argv.slice(2);
this.debugLog("Args:", args);
const transport = TransportManager.determineTransport(
args,
this.configManager
);
this.debugLog("Transport determined:", transport);
// Check verbosity flags for conditional logging
const isVerbose =
args.includes("--verbose") || args.includes("--debug-startup");
const isQuiet = args.includes("--quiet");
this.debugLog("Verbose:", isVerbose, "Quiet:", isQuiet);
// Initialize enhanced logger with config-based settings
this.debugLog("About to create enhanced logger");
const loggingConfig = this.configManager.getLoggingConfig();
const logDirectory = path.isAbsolute(loggingConfig.directory)
? loggingConfig.directory
: path.resolve(serverRoot, loggingConfig.directory);
const logFile = path.join(logDirectory, "mcp-server.log");
// Ensure log directory exists
try {
await fs.mkdir(logDirectory, { recursive: true });
console.error(`DEBUG: Log directory ensured: ${logDirectory}`);
} catch (error) {
console.error(
`DEBUG: Failed to create log directory ${logDirectory}:`,
error
);
}
const enhancedLoggerConfig: EnhancedLoggingConfig = {
logFile,
transport,
enableDebug: isVerbose,
configuredLevel: loggingConfig.level,
};
this.logger = createLogger(enhancedLoggerConfig);
// Initialize log file
await (this.logger as any).initLogFile();
this.debugLog("Enhanced logger created and initialized");
// Only show startup messages if not in quiet mode
if (!isQuiet) {
this.debugLog("About to call logger.info - Starting MCP...");
this.logger.info("Starting MCP Claude Prompts Server...");
this.debugLog("First logger.info completed");
this.logger.info(`Transport: ${transport}`);
this.debugLog("Second logger.info completed");
}
// Verbose mode shows detailed configuration info
if (isVerbose) {
this.debugLog("About to call verbose logger.info calls");
this.logger.info(`Server root: ${serverRoot}`);
this.logger.info(`Config file: ${CONFIG_FILE}`);
this.logger.debug(`Command line args: ${JSON.stringify(args)}`);
this.logger.debug(`Process working directory: ${process.cwd()}`);
this.debugLog("Verbose logger.info calls completed");
}
// Initialize text reference manager
this.debugLog("About to create TextReferenceManager");
this.textReferenceManager = new TextReferenceManager(this.logger);
this.debugLog("TextReferenceManager created");
// Initialize conversation manager
this.debugLog("About to create ConversationManager");
try {
this.conversationManager = createConversationManager(this.logger);
this.debugLog("ConversationManager created successfully");
} catch (error) {
this.debugLog("ConversationManager creation failed:", error);
throw error;
}
this.debugLog("ConversationManager created");
// Create MCP server
this.debugLog("About to get config");
const config = this.configManager.getConfig();
this.debugLog("Config retrieved successfully");
this.debugLog("About to create McpServer");
this.mcpServer = new McpServer({
name: config.server.name,
version: config.server.version,
capabilities: {
prompts: { listChanged: true },
tools: { listChanged: true },
},
});
this.debugLog("McpServer created successfully");
// Only log completion in verbose mode
if (isVerbose) {
this.debugLog("About to log foundation initialized");
this.logger.info("Foundation modules initialized");
this.debugLog("Foundation initialized log completed");
}
this.debugLog("initializeFoundation completed successfully");
}
/**
* Phase 2: Load and process prompt data
*/
private async loadAndProcessData(): Promise<void> {
// Check verbosity flags for conditional logging
const args = process.argv.slice(2);
const isVerbose =
args.includes("--verbose") || args.includes("--debug-startup");
const isQuiet = args.includes("--quiet");
// Initialize prompt manager
this.promptManager = new PromptManager(
this.logger,
this.textReferenceManager,
this.configManager,
this.mcpServer
);
// Load and convert prompts with enhanced path resolution
const config = this.configManager.getConfig();
// ENHANCED: Allow direct prompts config path override via environment variable
// This bypasses server root detection issues entirely and is perfect for Claude Desktop
let PROMPTS_FILE: string;
if (process.env.MCP_PROMPTS_CONFIG_PATH) {
PROMPTS_FILE = process.env.MCP_PROMPTS_CONFIG_PATH;
if (isVerbose) {
this.logger.info(
"🎯 Using MCP_PROMPTS_CONFIG_PATH environment variable override"
);
}
} else {
// Fallback to ConfigManager's getPromptsFilePath() method
PROMPTS_FILE = this.configManager.getPromptsFilePath();
if (isVerbose) {
this.logger.info("📁 Using config-based prompts file path resolution");
}
}
// Enhanced logging for prompt loading pipeline (verbose mode only)
if (isVerbose) {
this.logger.info("=== PROMPT LOADING PIPELINE START ===");
this.logger.info(`Config prompts.file setting: "${config.prompts.file}"`);
if (process.env.MCP_PROMPTS_CONFIG_PATH) {
this.logger.info(
`🎯 MCP_PROMPTS_CONFIG_PATH override: "${process.env.MCP_PROMPTS_CONFIG_PATH}"`
);
} else {
this.logger.info(
`Config manager base directory: "${path.dirname(
this.configManager.getPromptsFilePath()
)}"`
);
}
this.logger.info(`✅ Final PROMPTS_FILE path: "${PROMPTS_FILE}"`);
// Add additional diagnostic information
this.logger.info("=== PATH RESOLUTION DIAGNOSTICS ===");
this.logger.info(`process.cwd(): ${process.cwd()}`);
this.logger.info(`process.argv[0]: ${process.argv[0]}`);
this.logger.info(`process.argv[1]: ${process.argv[1] || "undefined"}`);
this.logger.info(
`__filename equivalent: ${fileURLToPath(import.meta.url)}`
);
this.logger.info(
`Config file path: ${(this.configManager as any).configPath}`
);
this.logger.info(
`MCP_PROMPTS_CONFIG_PATH: ${
process.env.MCP_PROMPTS_CONFIG_PATH || "undefined"
}`
);
this.logger.info(
`MCP_SERVER_ROOT: ${process.env.MCP_SERVER_ROOT || "undefined"}`
);
this.logger.info(
`PROMPTS_FILE is absolute: ${path.isAbsolute(PROMPTS_FILE)}`
);
this.logger.info(
`PROMPTS_FILE normalized: ${path.normalize(PROMPTS_FILE)}`
);
}
// Validate that we're using absolute paths (critical for Claude Desktop)
if (!path.isAbsolute(PROMPTS_FILE)) {
if (isVerbose) {
this.logger.error(
`⚠️ CRITICAL: PROMPTS_FILE is not absolute: ${PROMPTS_FILE}`
);
this.logger.error(
`This will cause issues with Claude Desktop execution!`
);
}
// Convert to absolute path as fallback
// Use serverRoot which is determined earlier and more reliable for constructing the absolute path
const serverRoot = await this.serverRootDetector.determineServerRoot(); // Ensure serverRoot is available
const absolutePromptsFile = path.resolve(serverRoot, PROMPTS_FILE);
if (isVerbose) {
this.logger.info(
`🔧 Converting to absolute path: ${absolutePromptsFile}`
);
}
PROMPTS_FILE = absolutePromptsFile;
}
// Verify the file exists before attempting to load
try {
const fs = await import("fs/promises");
await fs.access(PROMPTS_FILE);
if (isVerbose) {
this.logger.info(
`✓ Prompts configuration file exists: ${PROMPTS_FILE}`
);
}
} catch (error) {
this.logger.error(
`✗ Prompts configuration file NOT FOUND: ${PROMPTS_FILE}`
);
if (isVerbose) {
this.logger.error(`File access error:`, error);
// Provide additional troubleshooting information
this.logger.error("=== TROUBLESHOOTING INFORMATION ===");
this.logger.error(`Is path absolute? ${path.isAbsolute(PROMPTS_FILE)}`);
this.logger.error(`Normalized path: ${path.normalize(PROMPTS_FILE)}`);
this.logger.error(`Path exists check: ${PROMPTS_FILE}`);
}
throw new Error(`Prompts configuration file not found: ${PROMPTS_FILE}`);
}
try {
this.logger.info("Initiating prompt loading and conversion...");
// Pass path.dirname(PROMPTS_FILE) as the basePath for resolving relative prompt file paths
const result = await this.promptManager.loadAndConvertPrompts(
PROMPTS_FILE,
path.dirname(PROMPTS_FILE)
);
this._promptsData = result.promptsData;
this._categories = result.categories;
this._convertedPrompts = result.convertedPrompts;
this.logger.info("=== PROMPT LOADING RESULTS ===");
this.logger.info(
`✓ Loaded ${this._promptsData.length} prompts from ${this._categories.length} categories`
);
this.logger.info(
`✓ Converted ${this._convertedPrompts.length} prompts to MCP format`
);
// Log category breakdown
if (this._categories.length > 0) {
this.logger.info("Categories loaded:");
this._categories.forEach((category) => {
const categoryPrompts = this._promptsData.filter(
(p) => p.category === category.id
);
this.logger.info(
` - ${category.name} (${category.id}): ${categoryPrompts.length} prompts`
);
});
} else {
this.logger.warn("⚠ No categories were loaded!");
}
this.logger.info("=== PROMPT LOADING PIPELINE END ===");
// Propagate updated data to other relevant managers
if (this.mcpToolsManager) {
this.mcpToolsManager.updateData(
this._promptsData,
this._convertedPrompts,
this.categories
);
}
// REMOVED: ExecutionCoordinator prompts update - modular chain system removed
if (this.apiManager) {
// apiManager might not exist for stdio
this.apiManager.updateData(
this._promptsData,
this._categories,
this.convertedPrompts
);
}
// CRUCIAL STEP: Re-register all prompts with the McpServer using the newly loaded data
if (this.promptManager && this.mcpServer) {
this.logger.info(
"🔄 Re-registering all prompts with MCP server after hot-reload..."
);
const registeredCount = await this.promptManager.registerAllPrompts(
this.convertedPrompts
);
this.logger.info(
`✅ Successfully re-registered ${registeredCount} prompts.`
);
} else {
this.logger.warn(
"⚠️ PromptManager or McpServer not available, skipping re-registration of prompts after hot-reload."
);
}
} catch (error) {
this.logger.error("✗ PROMPT LOADING FAILED:");
this.logger.error("Error details:", error);
this.logger.error(
"Stack trace:",
error instanceof Error ? error.stack : "No stack trace available"
);
throw error;
}
}
/**
* Phase 3: Initialize remaining modules with loaded data
*/
private async initializeModulesPrivate(): Promise<void> {
// Check verbosity flags for conditional logging
const args = process.argv.slice(2);
const isVerbose =
args.includes("--verbose") || args.includes("--debug-startup");
// REMOVED: Gate evaluator and ExecutionCoordinator initialization - modular systems removed
// Initialize Framework State Manager (for framework switching)
if (isVerbose)
this.logger.info("🔄 Initializing Framework State Manager...");
const frameworkStateRoot =
typeof this.configManager.getServerRoot === 'function'
? this.configManager.getServerRoot()
: path.dirname(this.configManager.getConfigPath());
this.frameworkStateManager = await createFrameworkStateManager(
this.logger,
frameworkStateRoot
);
// Validation: Ensure FrameworkStateManager was created successfully
if (!this.frameworkStateManager) {
throw new Error(
"Failed to initialize FrameworkStateManager - required for framework switching"
);
}
if (isVerbose)
this.logger.info("✅ FrameworkStateManager initialized successfully");
// Debug: Log chain prompt availability
const chainCount = this._convertedPrompts.filter((p) =>
isChainPrompt(p)
).length;
if (isVerbose)
this.logger.info(
`🔗 Chain prompts available: ${chainCount}/${this._convertedPrompts.length} total prompts`
);
if (chainCount > 0 && isVerbose) {
const chainNames = this._convertedPrompts
.filter((p) => isChainPrompt(p))
.map((p) => p.id)
.join(", ");
this.logger.info(`🔗 Available chains: ${chainNames}`);
}
// Phase 2: Workflow registration removed - chains handle all multi-step execution
// Initialize MCP tools manager
if (isVerbose) this.logger.info("🔄 Initializing MCP tools manager...");
this.mcpToolsManager = await createMcpToolsManager(
this.logger,
this.mcpServer,
this.promptManager,
this.configManager,
() => this.fullServerRefresh(),
(reason: string) => this.restartServer(reason)
// Phase 3: Removed executionCoordinator - chains now use LLM-driven execution
);
// Update MCP tools manager with current data
if (isVerbose) this.logger.info("🔄 Updating MCP tools manager data...");
this.mcpToolsManager.updateData(
this._promptsData,
this._convertedPrompts,
this.categories
);
// Connect Framework State Manager to MCP Tools Manager
if (isVerbose) this.logger.info("🔄 Connecting Framework State Manager...");
this.mcpToolsManager.setFrameworkStateManager(this.frameworkStateManager);
// Initialize and connect Framework Manager
if (isVerbose) this.logger.info("🔄 Initializing Framework Manager...");
await this.mcpToolsManager.setFrameworkManager();
// Initialize Tool Description Manager
if (isVerbose) this.logger.info("🔄 Initializing Tool Description Manager...");
this.toolDescriptionManager = createToolDescriptionManager(this.logger, this.configManager);
await this.toolDescriptionManager.initialize();
// Connect Tool Description Manager to MCP Tools Manager
if (isVerbose) this.logger.info("🔄 Connecting Tool Description Manager to MCP Tools...");
this.mcpToolsManager.setToolDescriptionManager(this.toolDescriptionManager);
// REMOVED: ConsolidatedPromptEngine to ExecutionCoordinator wiring - ExecutionCoordinator removed
// Register all MCP tools
if (isVerbose) this.logger.info("🔄 Registering all MCP tools...");
await this.mcpToolsManager.registerAllTools();
// Register all prompts
if (isVerbose) this.logger.info("🔄 Registering all prompts...");
await this.promptManager.registerAllPrompts(this._convertedPrompts);
this.logger.info("All modules initialized successfully");
// REMOVED: ExecutionCoordinator stats collection - modular chain system removed
}
// Phase 2: Workflow registration completely removed - chains handle all multi-step execution
/**
* Phase 4: Setup and start the server
*/
private async startServer(isStartupTest: boolean = false): Promise<void> {
console.error("DEBUG: startServer() - Determining transport...");
// Determine transport
const args = process.argv.slice(2);
const transport = TransportManager.determineTransport(
args,
this.configManager
);
console.error("DEBUG: startServer() - Transport determined:", transport);
console.error("DEBUG: startServer() - Creating transport manager...");
// Create transport manager
this.transportManager = createTransportManager(
this.logger,
this.configManager,
this.mcpServer,
transport
);
console.error("DEBUG: startServer() - Transport manager created");
console.error("DEBUG: startServer() - Checking if SSE transport...");
// Create API manager for SSE transport
if (this.transportManager.isSse()) {
console.error("DEBUG: startServer() - Creating API manager for SSE...");
this.apiManager = createApiManager(
this.logger,
this.configManager,
this.promptManager,
this.mcpToolsManager
);
console.error("DEBUG: startServer() - API manager created");
console.error("DEBUG: startServer() - Updating API manager data...");
// Update API manager with current data
this.apiManager.updateData(
this._promptsData,
this._categories,
this.convertedPrompts
);
console.error("DEBUG: startServer() - API manager data updated");
} else {
console.error(
"DEBUG: startServer() - Using STDIO transport (no API manager needed)"
);
}
// Phase 1: Framework capabilities integrated into base components
console.error(
"DEBUG: startServer() - Framework capabilities integrated into base components"
);
if (isStartupTest) {
console.error(
"DEBUG: startServer() - Skipping MCP server startup (test mode)"
);
// Create a mock server manager for health validation
this.serverManager = {
shutdown: () => console.error("DEBUG: Mock server shutdown"),
getStatus: () => ({ running: true, transport: "stdio" }),
isRunning: () => true,
} as any;
console.error("DEBUG: startServer() - Mock server manager created");
} else {
console.error("DEBUG: startServer() - About to start MCP server...");
// Start the server
this.serverManager = await startMcpServer(
this.logger,
this.configManager,
this.transportManager,
this.apiManager
);
console.error("DEBUG: startServer() - MCP server started");
}
this.logger.info("Server started successfully");
console.error("DEBUG: startServer() - Server startup completed");
}
/**
* Switch to a different framework by ID (CAGEERF, ReACT, 5W1H, etc.)
* Core functionality: Allow switching between frameworks to guide the system
*/
async switchFramework(
frameworkId: string
): Promise<{ success: boolean; message: string }> {
// Phase 1: Framework switching simplified - basic support only
try {
this.logger.info(
`Framework switching to ${frameworkId} (Phase 1 basic support)`
);
const result = {
success: true,
message: `Switched to ${frameworkId}`,
newFramework: frameworkId,
previousFramework: "basic",
};
if (result.success) {
this.logger.info(`🔄 Framework switched to: ${result.newFramework}`);
return {
success: true,
message: `Successfully switched from ${result.previousFramework} to ${result.newFramework}`,
};
} else {
this.logger.warn(`❌ Framework switch failed: ${result.message}`);
return {
success: false,
message: result.message || "Unknown error during framework switch",
};
}
} catch (error) {
this.logger.error("Framework switch error:", error);
return {
success: false,
message: `Error switching framework: ${
error instanceof Error ? error.message : String(error)
}`,
};
}
}
/**
* Get current framework information
*/
getCurrentFrameworkInfo(): {
id: string;
name: string;
availableFrameworks: string[];
isHealthy: boolean;
} {
// Phase 1: Framework status simplified - basic support only
const status = {
currentFramework: "basic",
currentFrameworkName: "Basic Framework",
isHealthy: true,
};
const available = ["basic"];
return {
id: status.currentFramework,
name: status.currentFrameworkName,
availableFrameworks: available,
isHealthy: status.isHealthy,
};
}
/**
* Graceful shutdown
*/
async shutdown(): Promise<void> {
try {
if (this.logger) {
this.logger.info("Initiating application shutdown...");
}
// Phase 1: Stop server and transport layers
if (this.serverManager) {
if (this.logger) {
this.logger.debug("Shutting down server manager...");
}
this.serverManager.shutdown();
}
// Phase 2: Stop transport layer (if it has shutdown method)
if (this.transportManager && 'shutdown' in this.transportManager && typeof (this.transportManager as any).shutdown === 'function') {
if (this.logger) {
this.logger.debug("Shutting down transport manager...");
}
try {
await (this.transportManager as any).shutdown();
} catch (error) {
this.logger?.warn("Error shutting down transport manager:", error);
}
}
// Phase 3: Stop monitoring and resource-intensive components (if they have shutdown method)
if (this.frameworkStateManager && 'shutdown' in this.frameworkStateManager && typeof (this.frameworkStateManager as any).shutdown === 'function') {
if (this.logger) {
this.logger.debug("Shutting down framework state manager...");
}
try {
await (this.frameworkStateManager as any).shutdown();
} catch (error) {
this.logger?.warn("Error shutting down framework state manager:", error);
}
}
// Phase 4: Stop file watchers and hot-reload systems (if they have shutdown method)
if (this.promptManager && 'shutdown' in this.promptManager && typeof (this.promptManager as any).shutdown === 'function') {
if (this.logger) {
this.logger.debug("Shutting down prompt manager...");
}
try {
await (this.promptManager as any).shutdown();
} catch (error) {
this.logger?.warn("Error shutting down prompt manager:", error);
}
}
// Phase 5: Stop API and MCP tools (if they have shutdown method)
if (this.apiManager && 'shutdown' in this.apiManager && typeof (this.apiManager as any).shutdown === 'function') {
if (this.logger) {
this.logger.debug("Shutting down API manager...");
}
try {
await (this.apiManager as any).shutdown();
} catch (error) {
this.logger?.warn("Error shutting down API manager:", error);
}
}
if (this.mcpToolsManager && 'shutdown' in this.mcpToolsManager && typeof (this.mcpToolsManager as any).shutdown === 'function') {
if (this.logger) {
this.logger.debug("Shutting down MCP tools manager...");
}
try {
await (this.mcpToolsManager as any).shutdown();
} catch (error) {
this.logger?.warn("Error shutting down MCP tools manager:", error);
}
}
// Phase 6: Stop conversation and text reference managers (if they have shutdown method)
if (this.conversationManager && 'shutdown' in this.conversationManager && typeof (this.conversationManager as any).shutdown === 'function') {
if (this.logger) {
this.logger.debug("Shutting down conversation manager...");
}
try {
await (this.conversationManager as any).shutdown();
} catch (error) {
this.logger?.warn("Error shutting down conversation manager:", error);
}
}
if (this.textReferenceManager && 'shutdown' in this.textReferenceManager && typeof (this.textReferenceManager as any).shutdown === 'function') {
if (this.logger) {
this.logger.debug("Shutting down text reference manager...");
}
try {
await (this.textReferenceManager as any).shutdown();
} catch (error) {
this.logger?.warn("Error shutting down text reference manager:", error);
}
}
// Phase 6: Clean up internal timers
this.cleanup();
if (this.logger) {
this.logger.info("Application shutdown completed successfully");
}
} catch (error) {
if (this.logger) {
this.logger.error("Error during shutdown:", error);
} else {
console.error("Error during shutdown (logger not available):", error);
}
throw error;
}
}
/**
* Perform a full server refresh (hot-reload).
* This reloads all prompts from disk and updates all relevant modules.
*/
public async fullServerRefresh(): Promise<void> {
this.logger.info(
"🔥 Application: Starting full server refresh (hot-reload)..."
);
try {
// Step 1: Reload all prompt data from disk by re-running the data loading phase.
// This updates the application's internal state with the latest file contents.
await this.loadAndProcessData();
this.logger.info("✅ Data reloaded and processed from disk.");
// Step 2: Framework hot-reload integration now handled by enhanced base components
this.logger.info(
"✅ Framework capabilities integrated into base components"
);
// Step 2.5: Phase 2 - Simple framework switching status check
const switchingStatus = {
currentFramework: "basic",
currentFrameworkName: "Basic Framework",
enabledFrameworks: 1,
availableFrameworks: 1,
};
this.logger.info(
`✅ Framework switching system ready: ${switchingStatus.currentFrameworkName} active ` +
`(${switchingStatus.enabledFrameworks}/${switchingStatus.availableFrameworks} frameworks available)`
);
// Phase 3 complexity removed - focusing on simple framework switching instead of multi-framework consensus
// Step 3: Propagate the new data to all dependent modules.
// This ensures all parts of the application are synchronized with the new state.
// REMOVED: ExecutionCoordinator prompts update - modular chain system removed
if (this.mcpToolsManager) {
this.mcpToolsManager.updateData(
this._promptsData,
this._convertedPrompts,
this.categories
);
this.logger.info("✅ McpToolsManager updated with new data.");
}
if (this.apiManager) {
// The API manager is only available for the SSE transport.
this.apiManager.updateData(
this._promptsData,
this._categories,
this.convertedPrompts
);
this.logger.info("✅ ApiManager updated with new data.");
}
// Step 4: Notify MCP clients that the prompt list has changed (proper hot-reload)
// This follows MCP protocol - clients will re-query the server for the updated list
await this.promptManager.notifyPromptsListChanged();
this.logger.info(
"✅ Prompts list_changed notification sent to MCP clients."
);
// Step 5: Phase 2 - Workflow registration removed
this.logger.info("🚀 Full server refresh completed successfully.");
} catch (error) {
this.logger.error("❌ Error during full server refresh:", error);
// Re-throw the error so the caller can handle it appropriately.
throw error;
}
}
/**
* Restart the application by shutting down and exiting with a restart code.
* Relies on a process manager (e.g., PM2) to restart the process.
*/
public async restartServer(reason: string = "Manual restart"): Promise<void> {
this.logger.info(`🚨 Initiating server restart. Reason: ${reason}`);
try {
// Ensure all current operations are gracefully shut down.
await this.shutdown();
this.logger.info(
"✅ Server gracefully shut down. Exiting with restart code."
);
} catch (error) {
this.logger.error("❌ Error during pre-restart shutdown:", error);
} finally {
// Exit with a specific code that a process manager can detect.
process.exit(100);
}
}
/**
* Get application status
*/
getStatus(): {
running: boolean;
transport?: string;
promptsLoaded: number;
categoriesLoaded: number;
serverStatus?: any;
executionCoordinator?: {
totalExecutions: number;
promptExecutions: number;
chainExecutions: number;
// workflowExecutions: number; // Phase 2: removed, workflows tracked as advanced chains
successRate: number;
};
} {
// REMOVED: ExecutionCoordinator status - providing default execution status
const executionCoordinatorStatus = {
totalExecutions: 0,
promptExecutions: 0,
chainExecutions: 0,
successRate: 1.0,
};
return {
running: this.serverManager?.isRunning() || false,
transport: this.transportManager?.getTransportType(),
promptsLoaded: this._promptsData.length,
categoriesLoaded: this._categories.length,
serverStatus: this.serverManager?.getStatus(),
executionCoordinator: executionCoordinatorStatus,
};
}
/**
* Get all module instances (for debugging/testing)
*/
getModules() {
return {
logger: this.logger,
configManager: this.configManager,
promptManager: this.promptManager,
textReferenceManager: this.textReferenceManager,
conversationManager: this.conversationManager,
// REMOVED: executionCoordinator and gateEvaluator - modular systems removed
mcpToolsManager: this.mcpToolsManager,
apiManager: this.apiManager,
serverManager: this.serverManager,
};
}
/**
* Validate application health - comprehensive health check
*/
validateHealth(): {
healthy: boolean;
modules: {
foundation: boolean;
dataLoaded: boolean;
modulesInitialized: boolean;
serverRunning: boolean;
};
details: {
promptsLoaded: number;
categoriesLoaded: number;
serverStatus?: any;
moduleStatus: Record<string, boolean>;
};
issues: string[];
} {
const issues: string[] = [];
const moduleStatus: Record<string, boolean> = {};
// Check foundation modules
const foundationHealthy = !!(
this.logger &&
this.configManager &&
this.textReferenceManager
);
moduleStatus.foundation = foundationHealthy;
if (!foundationHealthy) {
issues.push("Foundation modules not properly initialized");
}
// Check data loading
const dataLoaded =
this._promptsData.length > 0 && this._categories.length > 0;
moduleStatus.dataLoaded = dataLoaded;
if (!dataLoaded) {
issues.push("Prompt data not loaded or empty");
}
// Check module initialization
const modulesInitialized = !!(
this.promptManager &&
// REMOVED: this.executionCoordinator && this.gateEvaluator - modular systems removed
this.mcpToolsManager
);
moduleStatus.modulesInitialized = modulesInitialized;
moduleStatus.serverRunning = !!(
this.serverManager && this.transportManager
);
moduleStatus.configManager = !!this.configManager;
moduleStatus.logger = !!this.logger;
moduleStatus.promptManager = !!this.promptManager;
moduleStatus.textReferenceManager = !!this.textReferenceManager;
moduleStatus.conversationManager = !!this.conversationManager;
// REMOVED: moduleStatus for executionCoordinator and gateEvaluator - modular systems removed
moduleStatus.mcpToolsManager = !!this.mcpToolsManager;
moduleStatus.transportManager = !!this.transportManager;
moduleStatus.apiManager = !!this.apiManager;
moduleStatus.serverManager = !!this.serverManager;
// Check overall health
const isHealthy =
foundationHealthy &&
dataLoaded &&
modulesInitialized &&
moduleStatus.serverRunning &&
issues.length === 0;
return {
healthy: isHealthy,
modules: {
foundation: foundationHealthy,
dataLoaded,
modulesInitialized,
serverRunning: moduleStatus.serverRunning,
},
details: {
promptsLoaded: this._promptsData.length,
categoriesLoaded: this._categories.length,
serverStatus: this.serverManager?.getStatus(),
moduleStatus,
},
issues,
};
}
/**
* Get performance metrics for monitoring
*/
getPerformanceMetrics(): {
uptime: number;
memoryUsage: NodeJS.MemoryUsage;
process: {
pid: number;
nodeVersion: string;
platform: string;
arch: string;
};
application: {
promptsLoaded: number;
categoriesLoaded: number;
serverConnections?: number;
};
executionCoordinator?: {
statistics: any;
};
} {
// REMOVED: ExecutionCoordinator metrics - providing default metrics
const executionCoordinatorMetrics = {
statistics: {
totalExecutions: 0,
promptExecutions: 0,
chainExecutions: 0,
successRate: 1.0,
averageExecutionTime: 0,
failedExecutions: 0,
},
};
return {
uptime: process.uptime(),
memoryUsage: process.memoryUsage(),
process: {
pid: process.pid,
nodeVersion: process.version,
platform: process.platform,
arch: process.arch,
},
application: {
promptsLoaded: this._promptsData.length,
categoriesLoaded: this._categories.length,
serverConnections: this.transportManager?.isSse()
? this.transportManager.getActiveConnectionsCount()
: undefined,
},
executionCoordinator: executionCoordinatorMetrics,
};
}
/**
* Cleanup resources and stop timers
*/
cleanup(): void {
if (this.memoryOptimizationInterval) {
clearInterval(this.memoryOptimizationInterval);
this.memoryOptimizationInterval = undefined;
this.logger.debug("Memory optimization timer stopped");
}
}
/**
* Emergency diagnostic information for troubleshooting
*/
getDiagnosticInfo(): {
timestamp: string;
health: ReturnType<Application["validateHealth"]>;
performance: ReturnType<Application["getPerformanceMetrics"]>;
configuration: {
transport: string;
configLoaded: boolean;
};
errors: string[];
} {
const errors: string[] = [];
try {
// Collect any recent errors or issues
if (!this.mcpServer) {
errors.push("MCP Server instance not available");
}
if (this._promptsData.length === 0) {
errors.push("No prompts loaded");
}
if (this._categories.length === 0) {
errors.push("No categories loaded");
}
return {
timestamp: new Date().toISOString(),
health: this.validateHealth(),
performance: this.getPerformanceMetrics(),
configuration: {
transport: this.transportManager?.getTransportType() || "unknown",
configLoaded: !!this.configManager,
},
errors,
};
} catch (error) {
errors.push(
`Error collecting diagnostic info: ${
error instanceof Error ? error.message : String(error)
}`
);
return {
timestamp: new Date().toISOString(),
health: {
healthy: false,
modules: {
foundation: false,
dataLoaded: false,
modulesInitialized: false,
serverRunning: false,
},
details: { promptsLoaded: 0, categoriesLoaded: 0, moduleStatus: {} },
issues: ["Failed to collect health information"],
},
performance: {
uptime: process.uptime(),
memoryUsage: process.memoryUsage(),
process: {
pid: process.pid,
nodeVersion: process.version,
platform: process.platform,
arch: process.arch,
},
application: { promptsLoaded: 0, categoriesLoaded: 0 },
},
configuration: {
transport: "unknown",
configLoaded: false,
},
errors,
};
}
}
}
/**
* Create and configure an application runtime
*/
export function createApplication(): Application {
return new Application();
}
/**
* Main application entry point
*/
export async function startApplication(): Promise<Application> {
const application = createApplication();
await application.startup();
return application;
}
```
--------------------------------------------------------------------------------
/plans/symbolic-command-language-implementation.md:
--------------------------------------------------------------------------------
```markdown
# Symbolic Command Language Implementation Plan
**Status**: Planning Phase
**Created**: 2025-10-19
**Last Updated**: 2025-10-19
**Priority**: High
**Complexity**: Medium-High
## Executive Summary
Enhance the parser system to support intuitive symbolic operators for dynamic prompt composition, eliminating the need for explicit template files for common workflow patterns like chaining, quality gates, and framework selection.
**Core Vision**: Transform `>>content_analysis {{content}} --> deep_analysis --> notes = "comprehensive"` into a first-class command syntax.
## Problem Statement
### Current Limitations
1. **Template File Dependency**: Creating multi-step workflows requires:
- Creating markdown template file (e.g., `/prompts/analysis/notes.md`)
- Defining step-by-step instructions
- Manually managing context propagation
- Updating `prompts.json` registry
2. **Workflow Rigidity**:
- Fixed execution sequences
- No ad-hoc composition
- Limited dynamic behavior
- Verbose syntax for simple patterns
3. **Implicit Quality Control**:
- Gates must be predefined
- No inline validation criteria
- Difficult to express expected outcomes
4. **Framework Selection Friction**:
- Requires explicit `system_control` tool call
- Persists globally (can affect other executions)
- No scoped/temporary framework application
### User Pain Points
**Example Current Workflow**:
```bash
# Step 1: Create template file
cat > prompts/analysis/my_research.md <<EOF
# Research Workflow
{{steps definition...}}
EOF
# Step 2: Register in prompts.json
# Edit prompts/analysis/prompts.json manually
# Step 3: Execute
>>my_research topic="AI trends"
```
**Desired Workflow**:
```bash
# Single command - no template needed
>>web_search "AI trends" --> deep_analysis --> synthesis_report = "actionable insights"
```
## Proposed Solution: Symbolic Command Language
### Operator Catalog
#### 1. Chain Operator: `-->` (Sequential Execution)
**Purpose**: Compose prompts into sequential workflows without template files
**Syntax**:
```bash
>>prompt1 args --> prompt2 --> prompt3
```
**Examples**:
```bash
# Simple chain
>>content_analysis {{content}} --> deep_analysis --> note_refinement
# With arguments
>>web_search query="AI ethics" --> summarize depth="comprehensive" --> translate language="spanish"
# Multi-step research
>>literature_review topic="quantum computing" --> gap_analysis --> research_proposal
```
**Execution Behavior**:
- Each step receives previous step's output as input
- Automatic variable mapping: `step1_result`, `step2_result`, etc.
- Context preservation across chain
- Session state management via ChainSessionManager
- Gate validation at each step (if enabled)
**Implementation Mapping**:
- Integrates with existing `ChainExecutor`
- Dynamically generates chain step definitions
- Uses Nunjucks template variables for context passing
#### 2. Quality Gate Operator: `=` (Validation Criteria)
**Purpose**: Inline quality validation without predefined gate definitions
**Syntax**:
```bash
>>prompt args = "validation criteria"
```
**Examples**:
```bash
# Code quality validation
>>code_review target="app.ts" = "no type errors, 80% test coverage, follows style guide"
# Research depth validation
>>deep_research topic="climate change" = "minimum 10 academic sources, peer-reviewed, published after 2020"
# Content completeness
>>documentation topic="API" = "includes examples, covers error cases, has diagrams"
# Chain with final gate
>>analysis --> synthesis --> report = "executive summary included, data visualizations present"
```
**Execution Behavior**:
- Parses criteria string into validation requirements
- Creates temporary gate definition with LLM-based evaluation
- Applies gate after execution completes
- Returns pass/fail status with detailed reasoning
- Optional retry on failure (configurable)
**Implementation Details**:
```typescript
// Generated temporary gate structure
{
type: 'quality',
name: 'inline_gate_<hash>',
description: 'Inline validation criteria',
criteria: [
"no type errors",
"80% test coverage",
"follows style guide"
],
scope: 'execution',
temporary: true
}
```
#### 3. Framework Selector: `@` (Methodology)
**Purpose**: Apply specific framework/methodology for single execution
**Syntax**:
```bash
@FRAMEWORK >>prompt args
```
**Examples**:
```bash
# CAGEERF for strategic thinking
@CAGEERF >>product_roadmap feature="user_authentication"
# ReACT for debugging
@ReACT >>debug_issue error="memory_leak" logs="{{logs}}"
# 5W1H for requirements
@5W1H >>requirements_gathering stakeholder="product_manager"
# SCAMPER for innovation
@SCAMPER >>brainstorm_features product="smartwatch"
# Framework with chain
@CAGEERF >>strategic_analysis --> implementation_plan --> risk_assessment
```
**Execution Behavior**:
- Temporarily switches to specified framework
- Applies framework-specific system prompt enhancements
- Executes prompt with framework context
- Automatically reverts to default framework after execution
- Framework name case-insensitive (CAGEERF = cageerf = CaGeErF)
**Supported Frameworks**:
- `CAGEERF`: Context → Analysis → Goals → Execution → Evaluation → Refinement → Framework
- `ReACT`: Reasoning and Acting for systematic problem-solving
- `5W1H`: Who, What, When, Where, Why, How analysis
- `SCAMPER`: Substitute, Combine, Adapt, Modify, Put to other use, Eliminate, Reverse
#### 4. Parallel Execution: `+` (Concurrent Analysis)
**Purpose**: Execute multiple prompts concurrently and aggregate results
**Syntax**:
```bash
>>prompt1 args + prompt2 args + prompt3 args
```
**Examples**:
```bash
# Multi-perspective analysis
>>market_analysis product="X" + competitive_analysis product="X" + user_research product="X"
# Parallel research
>>academic_search topic="AI" + news_search topic="AI" + patent_search topic="AI"
# Diverse evaluations
>>code_review_security + code_review_performance + code_review_maintainability
```
**Execution Behavior**:
- Execute all prompts concurrently (true parallelization)
- Aggregate results into unified response
- Include execution time comparison
- Merge metadata from all executions
**Implementation Notes**:
- Uses `Promise.all()` for concurrent execution
- Each prompt gets independent execution context
- Results combined with section headers
- Useful for independent analyses that don't depend on each other
#### 5. Conditional Execution: `?` (Decision Branching)
**Purpose**: Branch execution based on output evaluation
**Syntax**:
```bash
>>prompt args ? "condition" : alternative_prompt
```
**Examples**:
```bash
# Error-aware workflow
>>code_analysis file="app.ts" ? "errors found" : comprehensive_review
# Complexity-based routing
>>analyze_complexity code="{{code}}" ? "complexity > 10" : simple_refactor : complex_redesign
# Adaptive research depth
>>initial_research topic="{{topic}}" ? "sufficient depth" : deep_research
```
**Execution Behavior**:
- Execute initial prompt
- LLM evaluates condition against output
- If condition true: execute conditional prompt
- If condition false: execute alternative (or stop)
- Return combined execution path metadata
**Implementation Complexity**: Phase 3 (advanced feature)
### Operator Precedence & Composition
#### Precedence Rules (Highest to Lowest)
1. **`@` Framework** - Applies to entire execution scope
2. **`?` Conditional** - Determines execution path
3. **`+` Parallel** - Execution strategy
4. **`-->` Chain** - Sequential composition
5. **`=` Gate** - Final validation
#### Complex Composition Examples
```bash
# Framework + Chain + Gate
@CAGEERF >>strategic_analysis topic="market_expansion" --> implementation_roadmap --> risk_assessment = "comprehensive coverage, realistic timelines"
# Parallel + Chain + Gate
>>market_research + competitive_analysis + user_survey --> synthesis_report = "actionable recommendations"
# Framework + Conditional + Chain
@ReACT >>diagnose_issue logs="{{logs}}" ? "root cause identified" : deep_investigation --> fix_implementation
# Full composition
@CAGEERF >>requirement_gathering + technical_feasibility --> architecture_design = "scalable, maintainable" --> implementation_plan
```
#### Parsing Order
```
Input: @CAGEERF >>analysis {{content}} --> synthesis --> notes = "comprehensive"
Parse Tree:
├── Framework Operator: @CAGEERF (scope: entire execution)
├── Chain Operator: --> (3 steps)
│ ├── Step 1: analysis {{content}}
│ ├── Step 2: synthesis
│ └── Step 3: notes
└── Gate Operator: = "comprehensive" (validation on final output)
```
## Technical Architecture
### Phase 1: Foundation - Symbolic Parser (Week 1)
#### New Files Structure
```
server/src/execution/parsers/
├── symbolic-command-parser.ts # Main symbolic parser
├── operators/
│ ├── index.ts # Operator exports
│ ├── chain-operator.ts # --> implementation
│ ├── gate-operator.ts # = implementation
│ ├── framework-operator.ts # @ implementation
│ ├── parallel-operator.ts # + implementation
│ └── conditional-operator.ts # ? implementation
└── types/
├── operator-types.ts # Operator interfaces
└── parse-result-extensions.ts # Extended CommandParseResult
```
#### Core Type Definitions
```typescript
// server/src/execution/parsers/types/operator-types.ts
/**
* Chain operator representing sequential execution
*/
export interface ChainOperator {
type: 'chain';
steps: ChainStep[];
contextPropagation: 'automatic' | 'manual';
}
export interface ChainStep {
promptId: string;
args: string;
position: number;
variableName: string; // e.g., "step1_result"
}
/**
* Quality gate operator for inline validation
*/
export interface GateOperator {
type: 'gate';
criteria: string;
parsedCriteria: string[]; // Split by commas/logical separators
scope: 'execution' | 'step' | 'chain';
retryOnFailure: boolean;
maxRetries: number;
}
/**
* Framework selector operator
*/
export interface FrameworkOperator {
type: 'framework';
frameworkId: string;
normalizedId: string; // Uppercased for validation
temporary: boolean; // Always true for @ operator
scopeType: 'execution' | 'chain';
}
/**
* Parallel execution operator
*/
export interface ParallelOperator {
type: 'parallel';
prompts: ParallelPrompt[];
aggregationStrategy: 'merge' | 'compare' | 'summarize';
}
export interface ParallelPrompt {
promptId: string;
args: string;
position: number;
}
/**
* Conditional execution operator
*/
export interface ConditionalOperator {
type: 'conditional';
condition: string;
conditionType: 'presence' | 'comparison' | 'pattern';
trueBranch: string; // Prompt ID for true path
falseBranch?: string; // Prompt ID for false path (optional)
}
/**
* Unified operator container
*/
export type SymbolicOperator =
| ChainOperator
| GateOperator
| FrameworkOperator
| ParallelOperator
| ConditionalOperator;
/**
* Operator detection result
*/
export interface OperatorDetectionResult {
hasOperators: boolean;
operatorTypes: string[]; // ['chain', 'gate', 'framework']
operators: SymbolicOperator[];
parseComplexity: 'simple' | 'moderate' | 'complex';
}
```
#### Extended CommandParseResult
```typescript
// server/src/execution/parsers/types/parse-result-extensions.ts
import { SymbolicOperator, OperatorDetectionResult } from './operator-types.js';
export interface SymbolicCommandParseResult extends CommandParseResult {
format: 'simple' | 'json' | 'symbolic' | 'chain';
operators?: OperatorDetectionResult;
executionPlan?: ExecutionPlan;
}
/**
* Execution plan generated from operators
*/
export interface ExecutionPlan {
steps: ExecutionStep[];
frameworkOverride?: string;
finalValidation?: GateOperator;
estimatedComplexity: number; // 1-10 scale
requiresSessionState: boolean;
}
export interface ExecutionStep {
stepNumber: number;
type: 'prompt' | 'gate' | 'framework_switch' | 'parallel_group';
promptId?: string;
args?: string;
dependencies: number[]; // Step numbers this depends on
outputVariable: string; // e.g., "step1_result"
}
```
#### Symbolic Parser Implementation
```typescript
// server/src/execution/parsers/symbolic-command-parser.ts
import { Logger } from "../../logging/index.js";
import { PromptData } from "../../types/index.js";
import { ValidationError } from "../../utils/index.js";
import {
SymbolicCommandParseResult,
OperatorDetectionResult,
SymbolicOperator
} from "./types/index.js";
export class SymbolicCommandParser {
private logger: Logger;
// Operator detection patterns
private readonly OPERATOR_PATTERNS = {
chain: /-->/g,
gate: /\s*=\s*["'](.+?)["']\s*$/,
framework: /^@([A-Za-z0-9]+)\s+/,
parallel: /\s*\+\s*/g,
conditional: /\s*\?\s*["'](.+?)["']\s*:\s*(\w+)/
};
constructor(logger: Logger) {
this.logger = logger;
}
/**
* Detect symbolic operators in command
*/
public detectOperators(command: string): OperatorDetectionResult {
const operators: SymbolicOperator[] = [];
const operatorTypes: string[] = [];
// Detect framework operator (@)
const frameworkMatch = command.match(this.OPERATOR_PATTERNS.framework);
if (frameworkMatch) {
operatorTypes.push('framework');
operators.push({
type: 'framework',
frameworkId: frameworkMatch[1],
normalizedId: frameworkMatch[1].toUpperCase(),
temporary: true,
scopeType: 'execution'
});
}
// Detect chain operator (-->)
const chainMatches = command.match(this.OPERATOR_PATTERNS.chain);
if (chainMatches && chainMatches.length > 0) {
operatorTypes.push('chain');
operators.push(this.parseChainOperator(command));
}
// Detect gate operator (=)
const gateMatch = command.match(this.OPERATOR_PATTERNS.gate);
if (gateMatch) {
operatorTypes.push('gate');
operators.push({
type: 'gate',
criteria: gateMatch[1],
parsedCriteria: this.parseCriteria(gateMatch[1]),
scope: 'execution',
retryOnFailure: true,
maxRetries: 1
});
}
// Detect parallel operator (+)
const parallelMatches = command.match(this.OPERATOR_PATTERNS.parallel);
if (parallelMatches && parallelMatches.length > 0 && !chainMatches) {
operatorTypes.push('parallel');
operators.push(this.parseParallelOperator(command));
}
// Detect conditional operator (?)
const conditionalMatch = command.match(this.OPERATOR_PATTERNS.conditional);
if (conditionalMatch) {
operatorTypes.push('conditional');
operators.push({
type: 'conditional',
condition: conditionalMatch[1],
conditionType: 'presence',
trueBranch: conditionalMatch[2],
falseBranch: undefined
});
}
const complexity = this.calculateComplexity(operators);
return {
hasOperators: operators.length > 0,
operatorTypes,
operators,
parseComplexity: complexity
};
}
/**
* Parse chain operator into structured steps
*/
private parseChainOperator(command: string): ChainOperator {
// Remove framework operator if present
let cleanCommand = command.replace(this.OPERATOR_PATTERNS.framework, '');
// Remove gate operator if present
cleanCommand = cleanCommand.replace(this.OPERATOR_PATTERNS.gate, '');
// Split by chain operator
const stepStrings = cleanCommand.split('-->').map(s => s.trim());
const steps = stepStrings.map((stepStr, index) => {
// Parse each step (format: >>prompt_name args or just prompt_name)
const stepMatch = stepStr.match(/^(?:>>)?([a-zA-Z0-9_-]+)(?:\s+(.*))?$/);
if (!stepMatch) {
throw new ValidationError(`Invalid chain step format: ${stepStr}`);
}
return {
promptId: stepMatch[1],
args: stepMatch[2] || '',
position: index,
variableName: `step${index + 1}_result`
};
});
return {
type: 'chain',
steps,
contextPropagation: 'automatic'
};
}
/**
* Parse parallel operator into concurrent prompts
*/
private parseParallelOperator(command: string): ParallelOperator {
const promptStrings = command.split('+').map(s => s.trim());
const prompts = promptStrings.map((promptStr, index) => {
const promptMatch = promptStr.match(/^(?:>>)?([a-zA-Z0-9_-]+)(?:\s+(.*))?$/);
if (!promptMatch) {
throw new ValidationError(`Invalid parallel prompt format: ${promptStr}`);
}
return {
promptId: promptMatch[1],
args: promptMatch[2] || '',
position: index
};
});
return {
type: 'parallel',
prompts,
aggregationStrategy: 'merge'
};
}
/**
* Parse gate criteria into individual requirements
*/
private parseCriteria(criteriaString: string): string[] {
// Split by common delimiters
return criteriaString
.split(/,|and|\||;/)
.map(c => c.trim())
.filter(c => c.length > 0);
}
/**
* Calculate parse complexity based on operators
*/
private calculateComplexity(operators: SymbolicOperator[]): 'simple' | 'moderate' | 'complex' {
if (operators.length === 0) return 'simple';
if (operators.length === 1) return 'simple';
if (operators.length === 2) return 'moderate';
return 'complex';
}
/**
* Generate execution plan from operators
*/
public generateExecutionPlan(
operators: OperatorDetectionResult,
basePromptId: string,
baseArgs: string
): ExecutionPlan {
const steps: ExecutionStep[] = [];
let frameworkOverride: string | undefined;
let finalValidation: GateOperator | undefined;
// Extract framework override
const frameworkOp = operators.operators.find(op => op.type === 'framework');
if (frameworkOp && frameworkOp.type === 'framework') {
frameworkOverride = frameworkOp.normalizedId;
}
// Extract final gate
const gateOp = operators.operators.find(op => op.type === 'gate');
if (gateOp && gateOp.type === 'gate') {
finalValidation = gateOp;
}
// Build execution steps based on operator types
const chainOp = operators.operators.find(op => op.type === 'chain');
if (chainOp && chainOp.type === 'chain') {
// Chain execution
chainOp.steps.forEach((step, index) => {
steps.push({
stepNumber: index + 1,
type: 'prompt',
promptId: step.promptId,
args: step.args,
dependencies: index > 0 ? [index] : [],
outputVariable: step.variableName
});
});
} else {
// Single prompt execution
steps.push({
stepNumber: 1,
type: 'prompt',
promptId: basePromptId,
args: baseArgs,
dependencies: [],
outputVariable: 'result'
});
}
return {
steps,
frameworkOverride,
finalValidation,
estimatedComplexity: operators.operators.length,
requiresSessionState: steps.length > 1
};
}
}
export function createSymbolicCommandParser(logger: Logger): SymbolicCommandParser {
return new SymbolicCommandParser(logger);
}
```
### Phase 2: Operator Executors (Week 2)
#### Chain Operator Execution
```typescript
// server/src/execution/operators/chain-executor.ts
import { ChainOperator, ChainStep } from "../parsers/types/operator-types.js";
import { ChainExecutor } from "../../mcp-tools/prompt-engine/core/executor.js";
export class ChainOperatorExecutor {
private chainExecutor: ChainExecutor;
constructor(chainExecutor: ChainExecutor) {
this.chainExecutor = chainExecutor;
}
/**
* Execute chain operator by generating dynamic chain definition
*/
async execute(
chainOp: ChainOperator,
initialContext: Record<string, any>
): Promise<any> {
// Convert chain operator to chain execution format
const chainSteps = chainOp.steps.map(step => ({
prompt: step.promptId,
arguments: this.parseStepArguments(step.args, initialContext)
}));
// Generate temporary chain prompt definition
const chainPrompt = this.generateChainPrompt(chainSteps);
// Execute using existing ChainExecutor
return await this.chainExecutor.executeChain({
promptId: '__dynamic_chain__',
promptArgs: initialContext,
convertedPrompt: chainPrompt,
chainGateIds: [],
chainExecutionId: `chain_${Date.now()}`
}, {
enableGates: true,
session_id: `symbolic_${Date.now()}`
});
}
private parseStepArguments(
argsString: string,
context: Record<string, any>
): Record<string, any> {
// Parse argument string into object
// Support both "key=value" and template variables "{{var}}"
const args: Record<string, any> = {};
// Simple parsing - can be enhanced
const kvPairs = argsString.match(/(\w+)="([^"]*)"/g);
if (kvPairs) {
kvPairs.forEach(pair => {
const [key, value] = pair.split('=');
args[key] = value.replace(/"/g, '');
});
} else if (argsString.trim()) {
// Single argument, map to 'content' or 'input'
args.content = argsString.trim();
}
return args;
}
private generateChainPrompt(steps: any[]): any {
// Generate ConvertedPrompt structure
return {
id: '__dynamic_chain__',
name: 'Dynamic Chain',
type: 'chain',
steps: steps,
metadata: {
generatedFrom: 'symbolic_operator',
timestamp: new Date().toISOString()
}
};
}
}
```
#### Gate Operator Execution
```typescript
// server/src/execution/operators/gate-executor.ts
import { GateOperator } from "../parsers/types/operator-types.js";
import { LightweightGateSystem } from "../../gates/core/index.js";
import type { TemporaryGateDefinition } from "../../gates/core/temporary-gate-registry.js";
export class GateOperatorExecutor {
private gateSystem: LightweightGateSystem;
constructor(gateSystem: LightweightGateSystem) {
this.gateSystem = gateSystem;
}
/**
* Execute gate operator by creating temporary gate and validating
*/
async execute(
gateOp: GateOperator,
executionResult: any,
executionId: string
): Promise<{
passed: boolean;
gateResults: any;
retryRequired: boolean;
}> {
// Generate temporary gate definition
const gateDefinition: TemporaryGateDefinition = {
name: `inline_gate_${Date.now()}`,
type: 'quality',
scope: gateOp.scope,
description: `Inline validation: ${gateOp.criteria}`,
guidance: this.generateGateGuidance(gateOp.parsedCriteria),
pass_criteria: gateOp.parsedCriteria
};
// Register temporary gate
const gateId = await this.gateSystem.registerTemporaryGate(
gateDefinition,
executionId
);
// Evaluate gate
const gateResults = await this.gateSystem.evaluateGates(
[gateId],
executionResult
);
const passed = gateResults.every((r: any) => r.passed);
const retryRequired = !passed && gateOp.retryOnFailure;
return {
passed,
gateResults,
retryRequired
};
}
private generateGateGuidance(criteria: string[]): string {
return `Evaluate the output against these criteria:\n${criteria.map((c, i) => `${i + 1}. ${c}`).join('\n')}`;
}
}
```
#### Framework Operator Execution
```typescript
// server/src/execution/operators/framework-executor.ts
import { FrameworkOperator } from "../parsers/types/operator-types.js";
import { FrameworkStateManager } from "../../frameworks/framework-state-manager.js";
export class FrameworkOperatorExecutor {
private frameworkStateManager: FrameworkStateManager;
private originalFramework: string | null = null;
constructor(frameworkStateManager: FrameworkStateManager) {
this.frameworkStateManager = frameworkStateManager;
}
/**
* Apply framework override before execution
*/
async applyFrameworkOverride(frameworkOp: FrameworkOperator): Promise<void> {
// Store original framework
const currentFramework = this.frameworkStateManager.getCurrentFramework();
this.originalFramework = currentFramework.frameworkId;
// Temporarily switch to requested framework
await this.frameworkStateManager.switchFramework(
frameworkOp.normalizedId,
`Symbolic operator: temporary framework switch`
);
}
/**
* Restore original framework after execution
*/
async restoreFramework(): Promise<void> {
if (this.originalFramework) {
await this.frameworkStateManager.switchFramework(
this.originalFramework,
`Restoring framework after symbolic operator execution`
);
this.originalFramework = null;
}
}
/**
* Execute with framework context (try-finally pattern)
*/
async executeWithFramework<T>(
frameworkOp: FrameworkOperator,
executionFn: () => Promise<T>
): Promise<T> {
try {
await this.applyFrameworkOverride(frameworkOp);
return await executionFn();
} finally {
await this.restoreFramework();
}
}
}
```
### Phase 3: Integration (Week 3)
#### UnifiedCommandParser Enhancement
```typescript
// Additions to server/src/execution/parsers/unified-command-parser.ts
import { SymbolicCommandParser } from './symbolic-command-parser.js';
import { SymbolicCommandParseResult } from './types/parse-result-extensions.js';
export class UnifiedCommandParser {
private symbolicParser: SymbolicCommandParser;
// ... existing code ...
private initializeStrategies(): ParsingStrategy[] {
return [
this.createSymbolicCommandStrategy(), // NEW - Highest priority
this.createSimpleCommandStrategy(),
this.createJsonCommandStrategy()
];
}
/**
* NEW: Symbolic command strategy (highest priority)
*/
private createSymbolicCommandStrategy(): ParsingStrategy {
return {
name: 'symbolic',
confidence: 0.95,
canHandle: (command: string) => {
// Detect any symbolic operators
return /-->|=\s*["']|^@[A-Za-z0-9]+|\+|[\?]/.test(command);
},
parse: (command: string): SymbolicCommandParseResult | null => {
try {
// Detect operators
const operators = this.symbolicParser.detectOperators(command);
if (!operators.hasOperators) {
return null; // Fallback to other strategies
}
// Extract base prompt ID (first prompt in chain/composition)
let basePromptId: string;
let baseArgs: string;
// Remove framework prefix if present
let cleanCommand = command.replace(/^@[A-Za-z0-9]+\s+/, '');
// Remove gate suffix if present
cleanCommand = cleanCommand.replace(/\s*=\s*["'].+["']\s*$/, '');
// Extract first prompt
const firstPromptMatch = cleanCommand.match(/^(?:>>)?([a-zA-Z0-9_-]+)(?:\s+([^-]*))?/);
if (!firstPromptMatch) {
return null;
}
basePromptId = firstPromptMatch[1];
baseArgs = firstPromptMatch[2] || '';
// Generate execution plan
const executionPlan = this.symbolicParser.generateExecutionPlan(
operators,
basePromptId,
baseArgs
);
return {
promptId: basePromptId,
rawArgs: baseArgs,
format: 'symbolic',
confidence: 0.95,
operators: operators,
executionPlan: executionPlan,
metadata: {
originalCommand: command,
parseStrategy: 'symbolic',
detectedFormat: `Symbolic (${operators.operatorTypes.join(', ')})`,
warnings: []
}
};
} catch (error) {
this.logger.error('Symbolic parsing failed:', error);
return null; // Fallback to other strategies
}
}
};
}
}
```
#### ConsolidatedPromptEngine Integration
```typescript
// Additions to server/src/mcp-tools/prompt-engine/core/engine.ts
import { ChainOperatorExecutor } from '../../../execution/operators/chain-executor.js';
import { GateOperatorExecutor } from '../../../execution/operators/gate-executor.js';
import { FrameworkOperatorExecutor } from '../../../execution/operators/framework-executor.js';
export class ConsolidatedPromptEngine {
private chainOperatorExecutor: ChainOperatorExecutor;
private gateOperatorExecutor: GateOperatorExecutor;
private frameworkOperatorExecutor: FrameworkOperatorExecutor;
// ... existing code ...
async execute(command: string, options: any = {}): Promise<ToolResponse> {
// Parse command (now includes symbolic operators)
const parseResult = await this.parsingSystem.commandParser.parseCommand(
command,
this.promptsData
);
// Check if symbolic operators detected
if (parseResult.format === 'symbolic' && parseResult.executionPlan) {
return await this.executeSymbolicCommand(parseResult, options);
}
// ... existing execution logic ...
}
/**
* NEW: Execute command with symbolic operators
*/
private async executeSymbolicCommand(
parseResult: SymbolicCommandParseResult,
options: any
): Promise<ToolResponse> {
const { executionPlan, operators } = parseResult;
let result: any;
try {
// Step 1: Apply framework override if present
const frameworkOp = operators?.operators.find(op => op.type === 'framework');
if (frameworkOp && frameworkOp.type === 'framework') {
return await this.frameworkOperatorExecutor.executeWithFramework(
frameworkOp,
async () => {
// Execute chain or single prompt within framework context
return await this.executeWithinFramework(parseResult, options);
}
);
}
// Step 2: Execute without framework override
result = await this.executeWithinFramework(parseResult, options);
return result;
} catch (error) {
this.logger.error('Symbolic command execution failed:', error);
throw error;
}
}
/**
* Execute chain or single prompt (with or without framework context)
*/
private async executeWithinFramework(
parseResult: SymbolicCommandParseResult,
options: any
): Promise<ToolResponse> {
const { executionPlan, operators } = parseResult;
let result: any;
// Check for chain operator
const chainOp = operators?.operators.find(op => op.type === 'chain');
if (chainOp && chainOp.type === 'chain') {
// Execute chain
result = await this.chainOperatorExecutor.execute(
chainOp,
parseResult.rawArgs ? { content: parseResult.rawArgs } : {}
);
} else {
// Execute single prompt (existing logic)
result = await this.executeSinglePrompt(parseResult, options);
}
// Step 3: Apply gate validation if present
const gateOp = operators?.operators.find(op => op.type === 'gate');
if (gateOp && gateOp.type === 'gate') {
const gateResult = await this.gateOperatorExecutor.execute(
gateOp,
result,
`exec_${Date.now()}`
);
// Attach gate results to response
result.gateValidation = gateResult;
// Handle retry if needed
if (gateResult.retryRequired) {
this.logger.warn('Gate validation failed, retry required');
// Could implement retry logic here
}
}
return result;
}
}
```
## Implementation Phases
### Phase 1: Foundation (Week 1) - Chain Operator Only
**Goals**:
- ✅ Implement symbolic parser foundation
- ✅ Chain operator (`-->`) detection and parsing
- ✅ Integration with existing ChainExecutor
- ✅ Comprehensive tests
**Deliverables**:
- `symbolic-command-parser.ts`
- `chain-operator.ts`
- `chain-executor.ts` (operator executor)
- Test suite for chain operator
- Documentation: basic chaining examples
**Success Criteria**:
```bash
# Must work:
>>content_analysis {{content}} --> deep_analysis --> notes
>>web_search "AI trends" --> summarize --> translate language="spanish"
```
### Phase 2: Quality Gates (Week 2) - Gate Operator
**Goals**:
- ✅ Gate operator (`=`) detection and parsing
- ✅ Temporary gate definition generation
- ✅ LLM-based validation execution
- ✅ Retry logic implementation
**Deliverables**:
- `gate-operator.ts`
- `gate-executor.ts`
- Gate criteria parsing logic
- Test suite for gate operator
- Documentation: inline quality validation
**Success Criteria**:
```bash
# Must work:
>>code_review target="app.ts" = "no type errors, 80% coverage"
>>deep_research topic="AI" --> synthesis = "comprehensive, actionable"
```
### Phase 3: Framework & Advanced (Week 3)
**Goals**:
- ✅ Framework operator (`@`) implementation
- ✅ Temporary framework switching
- ✅ Parallel operator (`+`) foundation
- ✅ Complex composition support
**Deliverables**:
- `framework-operator.ts`
- `framework-executor.ts`
- `parallel-operator.ts` (basic version)
- Combined operator tests
- Documentation: advanced compositions
**Success Criteria**:
```bash
# Must work:
@CAGEERF >>strategic_planning project="feature_x"
@ReACT >>debug_issue error="memory_leak" --> fix_implementation
>>market_analysis + competitive_analysis + user_research
```
### Phase 4: Polish & Documentation (Week 4)
**Goals**:
- ✅ Comprehensive documentation
- ✅ Error message improvements
- ✅ Performance optimization
- ✅ Example library
**Deliverables**:
- `/docs/symbolic-command-language.md`
- `/docs/operator-reference.md`
- `/docs/chaining-examples.md`
- Performance benchmarks
- User migration guide
## Testing Strategy
### Unit Tests
```typescript
// tests/unit/symbolic-parser.test.ts
describe('SymbolicCommandParser', () => {
describe('Chain Operator Detection', () => {
it('should detect simple chain', () => {
const parser = createSymbolicCommandParser(logger);
const result = parser.detectOperators('>>step1 --> step2 --> step3');
expect(result.hasOperators).toBe(true);
expect(result.operatorTypes).toContain('chain');
expect(result.operators[0].type).toBe('chain');
expect(result.operators[0].steps).toHaveLength(3);
});
it('should parse chain with arguments', () => {
const parser = createSymbolicCommandParser(logger);
const result = parser.detectOperators(
'>>analyze content="{{text}}" --> summarize depth="detailed"'
);
const chainOp = result.operators[0];
expect(chainOp.steps[0].promptId).toBe('analyze');
expect(chainOp.steps[0].args).toBe('content="{{text}}"');
expect(chainOp.steps[1].promptId).toBe('summarize');
expect(chainOp.steps[1].args).toBe('depth="detailed"');
});
});
describe('Gate Operator Detection', () => {
it('should detect gate criteria', () => {
const parser = createSymbolicCommandParser(logger);
const result = parser.detectOperators(
'>>code_review target="app.ts" = "no errors, 80% coverage"'
);
expect(result.operatorTypes).toContain('gate');
const gateOp = result.operators.find(op => op.type === 'gate');
expect(gateOp.parsedCriteria).toHaveLength(2);
expect(gateOp.parsedCriteria[0]).toBe('no errors');
expect(gateOp.parsedCriteria[1]).toBe('80% coverage');
});
});
describe('Framework Operator Detection', () => {
it('should detect framework selector', () => {
const parser = createSymbolicCommandParser(logger);
const result = parser.detectOperators('@CAGEERF >>strategic_planning');
expect(result.operatorTypes).toContain('framework');
const frameworkOp = result.operators.find(op => op.type === 'framework');
expect(frameworkOp.frameworkId).toBe('CAGEERF');
expect(frameworkOp.normalizedId).toBe('CAGEERF');
expect(frameworkOp.temporary).toBe(true);
});
it('should handle case-insensitive framework names', () => {
const parser = createSymbolicCommandParser(logger);
const result = parser.detectOperators('@react >>analyze');
const frameworkOp = result.operators.find(op => op.type === 'framework');
expect(frameworkOp.normalizedId).toBe('REACT');
});
});
describe('Complex Compositions', () => {
it('should parse framework + chain + gate', () => {
const parser = createSymbolicCommandParser(logger);
const result = parser.detectOperators(
'@CAGEERF >>analysis --> synthesis --> report = "comprehensive"'
);
expect(result.operatorTypes).toHaveLength(3);
expect(result.operatorTypes).toContain('framework');
expect(result.operatorTypes).toContain('chain');
expect(result.operatorTypes).toContain('gate');
});
});
});
```
### Integration Tests
```typescript
// tests/integration/symbolic-execution.test.ts
describe('Symbolic Command Execution', () => {
it('should execute simple chain', async () => {
const engine = createConsolidatedPromptEngine(/* ... */);
const result = await engine.execute(
'>>content_analysis "test content" --> deep_analysis'
);
expect(result).toBeDefined();
expect(result.metadata.executionType).toBe('chain');
expect(result.metadata.stepsExecuted).toBe(2);
});
it('should apply inline gate validation', async () => {
const engine = createConsolidatedPromptEngine(/* ... */);
const result = await engine.execute(
'>>code_review target="app.ts" = "no type errors"'
);
expect(result.gateValidation).toBeDefined();
expect(result.gateValidation.passed).toBeDefined();
});
it('should execute with framework override', async () => {
const engine = createConsolidatedPromptEngine(/* ... */);
const result = await engine.execute(
'@ReACT >>debug_issue error="memory leak"'
);
expect(result.metadata.frameworkUsed).toBe('ReACT');
});
});
```
## Documentation Plan
### User Documentation
**File**: `/docs/symbolic-command-language.md`
**Contents**:
1. Introduction to symbolic operators
2. Quick start examples
3. Operator reference with examples
4. Complex composition patterns
5. Best practices
6. Troubleshooting
**File**: `/docs/operator-reference.md`
**Contents**:
- Detailed operator specifications
- Syntax diagrams
- Parameter options
- Execution behavior
- Error handling
**File**: `/docs/chaining-examples.md`
**Contents**:
- Research workflows
- Code review workflows
- Content processing pipelines
- Multi-stage analysis patterns
- Quality-gated executions
### Developer Documentation
**Updates to**: `/CLAUDE.md`
**New Section**: "Symbolic Command Language"
- Parser architecture
- Operator implementation guide
- Adding new operators
- Testing requirements
## Migration Path
### Backward Compatibility
✅ **100% Backward Compatible**
- All existing command syntax continues to work
- No breaking changes to API
- Symbolic operators are additive enhancement
- Fallback to existing parsers if symbolic parsing fails
### Template File Migration (Optional)
**Automatic detection**: Templates with simple patterns can suggest symbolic equivalents
Example migration suggestions:
```
Current template: /prompts/analysis/research_chain.md
Symbolic equivalent: >>web_search --> deep_analysis --> synthesis_report
Consider using symbolic syntax for simpler workflows!
```
## Performance Considerations
### Parser Performance
**Operator Detection**: O(n) single pass through command string
- Regex matching for each operator type
- No backtracking
- Early termination on match
**Expected Overhead**: < 5ms for typical commands
### Execution Performance
**Chain Operator**: Same as existing ChainExecutor (no overhead)
**Gate Operator**: +100-500ms for LLM validation (one-time per gate)
**Framework Operator**: < 50ms for framework switching
### Optimization Strategies
1. **Operator Detection Caching**: Cache operator patterns per session
2. **Lazy Evaluation**: Only parse operators when needed
3. **Parallel Gate Evaluation**: Evaluate multiple gates concurrently
4. **Framework Switch Batching**: Batch multiple framework switches
## Security Considerations
### Input Validation
**Operator Injection Prevention**:
- Sanitize all operator inputs
- Validate framework names against whitelist
- Escape special characters in criteria strings
- Limit chain depth (max 10 steps)
**Gate Criteria Validation**:
- Prevent code injection in criteria
- Limit criteria length (max 500 chars)
- Sanitize LLM evaluation prompts
### Execution Limits
**Resource Limits**:
- Max chain depth: 10 steps
- Max parallel prompts: 5 concurrent
- Max gate retries: 3 attempts
- Execution timeout: 5 minutes
## Open Questions & Decisions
### Question 1: Escaping Special Characters
**Problem**: How to use literal `-->`, `=`, `@`, `+`, `?` in arguments?
**Options**:
- A) Quote entire argument: `content="text with --> arrow"`
- B) Escape characters: `content=text with \-\-> arrow`
- C) Raw string syntax: `content=r"text with --> arrow"`
**Decision**: **Option A** (quoting) - Most intuitive, aligns with shell conventions
### Question 2: Nested Chains
**Problem**: Should we support nested/grouped chains?
**Example**: `(>>a --> b) + (>>c --> d)`
**Options**:
- A) Phase 5 enhancement (not MVP)
- B) Implement with parentheses from start
- C) Never support (keep simple)
**Decision**: **Option A** - Defer to Phase 5, gather user feedback first
### Question 3: Gate Failure Handling
**Problem**: What happens when gate validation fails?
**Options**:
- A) Automatic retry (configurable max attempts)
- B) Return error, let user decide
- C) Prompt user for retry decision
**Decision**: **Option A** - Auto-retry once by default, configurable via gate operator parameters
### Question 4: Framework Persistence
**Problem**: Should `@FRAMEWORK` persist beyond single execution?
**Options**:
- A) Always temporary (revert after execution)
- B) Option to make persistent: `@@FRAMEWORK` (sticky)
- C) User setting controls default behavior
**Decision**: **Option A** - Keep simple, use `system_control` for persistent switching
### Question 5: Operator Aliases
**Problem**: Should we support Unicode operators as aliases?
**Examples**:
- `→` for `-->`
- `⚡` for `@`
- `✓` for `=`
**Options**:
- A) Support with deprecation warning
- B) Never support (ASCII only)
- C) Full support, no warnings
**Decision**: **Option B** - ASCII only for maximum compatibility
### Question 6: Parallel Aggregation Strategies
**Problem**: How to combine results from parallel execution?
**Options**:
- A) Simple concatenation (default)
- B) Explicit aggregator: `>>a + b + c | merge_strategy`
- C) LLM-based synthesis
**Decision**: **Option A** for MVP, **Option C** for Phase 4 enhancement
## Success Metrics
### Adoption Metrics
**Target Goals**:
- 30% of users try symbolic operators within 1 month
- 50% reduction in template file creation
- 80% satisfaction rating for symbolic syntax
**Tracking**:
- Operator usage statistics
- Template file creation trends
- User feedback surveys
### Performance Metrics
**Target Goals**:
- < 5ms parsing overhead
- < 50ms framework switching overhead
- 95% operator detection accuracy
**Monitoring**:
- Parser performance metrics
- Execution time tracking
- Error rate monitoring
### Quality Metrics
**Target Goals**:
- 0 breaking changes to existing functionality
- 90%+ test coverage for new code
- < 1% regression rate
**Validation**:
- Comprehensive test suite
- Backward compatibility tests
- Performance benchmarks
## Risk Assessment
### High Risk: Parser Complexity
**Risk**: Symbolic parser becomes too complex, hard to maintain
**Mitigation**:
- Modular operator design (one operator = one file)
- Comprehensive test coverage
- Progressive enhancement (one operator at a time)
- Clear documentation
### Medium Risk: User Confusion
**Risk**: Users confused by multiple command syntaxes
**Mitigation**:
- Clear documentation with examples
- Error messages suggest correct syntax
- Migration guide for template users
- Interactive tutorial
### Low Risk: Performance Degradation
**Risk**: Operator detection slows down parsing
**Mitigation**:
- Performance benchmarks
- Optimization strategies
- Caching mechanisms
- Lazy evaluation
## Next Steps
### Immediate Actions (This Week)
1. ✅ Review and approve plan
2. ⏳ Create feature branch: `feature/symbolic-command-language`
3. ⏳ Set up test infrastructure
4. ⏳ Begin Phase 1 implementation (chain operator)
### Phase 1 Kickoff (Week 1)
**Day 1-2**: Foundation
- Create symbolic parser file structure
- Implement operator detection framework
- Write basic type definitions
**Day 3-4**: Chain Operator
- Implement chain operator parsing
- Create chain executor
- Write unit tests
**Day 5**: Integration
- Integrate with UnifiedCommandParser
- Add to ConsolidatedPromptEngine
- Write integration tests
### Success Checkpoint
**Phase 1 Complete When**:
✅ Chain operator fully functional
✅ 90%+ test coverage
✅ Integration tests passing
✅ Documentation complete
✅ User feedback collected
---
## Appendix: Example Use Cases
### Research Workflow
```bash
# Literature review to publication pipeline
@CAGEERF >>literature_search topic="quantum computing" --> gap_analysis --> research_proposal = "novel contribution, feasible methodology"
```
### Code Review Pipeline
```bash
# Multi-stage code review with quality gates
>>static_analysis file="app.ts" --> security_review --> performance_review = "no vulnerabilities, optimized algorithms" --> approval
```
### Content Creation Workflow
```bash
# Multi-perspective content generation
>>outline_generator topic="AI ethics" + competitor_analysis + seo_research --> content_creation = "comprehensive, SEO-optimized, original"
```
### Debugging Workflow
```bash
# Systematic debugging with ReACT methodology
@ReACT >>error_analysis logs="{{logs}}" --> root_cause_identification --> fix_proposal ? "fix validated" : comprehensive_test_suite
```
### Market Analysis
```bash
# Parallel market research synthesis
>>market_trends + competitor_landscape + customer_feedback --> synthesis_report = "data-driven recommendations, risk assessment"
```
---
**End of Implementation Plan**
This plan will be updated as implementation progresses and new insights emerge.
```
--------------------------------------------------------------------------------
/plans/sqlite-storage-migration.md:
--------------------------------------------------------------------------------
```markdown
# SQLite Storage Layer - Comprehensive Implementation Plan
**Created**: 2025-01-19
**Status**: Planning
**Priority**: High - Foundation for User Data Management & Versioning
**Impact**: Enables change history, user data separation, scalable storage, no-fork user experience
## Executive Summary
**Strategic Goal**: Migrate from JSON file-based storage to SQLite database with versioning capabilities, while maintaining backwards compatibility and enabling user data separation from the server codebase.
**Core Requirements**:
- **Change history** - Track all modifications with rollback capability
- **User data separation** - User prompts/gates/frameworks stored in `~/.claude-prompts-mcp/` not in repo
- **Modification workflow** - Enhanced MCP tool integration for seamless CRUD operations
- **Scale to hundreds** - Handle 100-500 items efficiently
- **Profile support** - Built-in foundation for work/life/code separation (UI comes later)
- **Git-optional** - Users don't need to fork repo, but can export to JSON for Git backup
- **Zero breaking changes** - Gradual migration with adapter pattern
**Key Principle**: This is a **storage layer enhancement**, not a database system. Users never interact with SQL directly - all operations through existing MCP tools (`prompt_manager`, `system_control`).
---
## Table of Contents
1. [Current State Analysis](#current-state-analysis)
2. [Architecture Design](#architecture-design)
3. [Storage Abstraction Layer](#storage-abstraction-layer)
4. [SQLite Schema Design](#sqlite-schema-design)
- [Profile Support Architecture](#profile-support-architecture)
5. [Migration Strategy](#migration-strategy)
6. [User Data Separation](#user-data-separation)
7. [Versioning & History System](#versioning--history-system)
8. [MCP Tool Integration](#mcp-tool-integration)
9. [Implementation Phases](#implementation-phases)
10. [Testing Strategy](#testing-strategy)
11. [Rollback & Recovery](#rollback--recovery)
12. [Performance Considerations](#performance-considerations)
13. [Future Enhancements](#future-enhancements)
- [Profile Management Features (Phase 6+)](#profile-management-features-phase-6)
---
## Current State Analysis
### Existing Storage System
**Current Architecture**:
```
server/prompts/
├── promptsConfig.json # Main configuration
├── category-name/
│ ├── prompts.json # Category prompt registry
│ ├── prompt-1.md # Individual prompt files
│ └── prompt-2.md
└── another-category/
└── ...
Storage Mechanism:
- JSON files for metadata and registry
- Markdown files for prompt content
- Hot-reload via file watching
- Git version control for server defaults
```
**Current Statistics** (measured):
- **Files**: 48 total (JSON + Markdown)
- **Storage**: 300KB total
- **Categories**: 18 categories
- **Scale**: Dozens of items (appropriate for file-based)
**Pain Points Identified**:
1. **No change history** - Can't see previous versions or rollback changes
2. **Modification workflow** - Manual JSON editing required (though MCP tools help)
3. **User data in repo** - Users need to fork to customize, or modify in-place
4. **Limited querying** - File-based filtering is basic
5. **No audit trail** - Can't track who changed what and why
6. **No profile separation** - At 500+ items, need work/life/code separation (future concern)
### Framework & Gates Current Storage
**Frameworks**:
```typescript
// Location: server/src/frameworks/framework-manager.ts
// Storage: In-memory Map<string, FrameworkDefinition>
// Source: Generated from methodology guides (code-based)
// Future need: User-customizable frameworks stored persistently
```
**Gates**:
```typescript
// Location: server/src/gates/core/gate-loader.ts
// Storage: YAML/JSON files in server/src/gates/definitions/
// Caching: In-memory Map with file watching
// Future need: User-defined gates with versioning
```
### What Works Well (Keep)
1. **Hot-reload system** - File watching works great for development
2. **MCP tool interface** - `prompt_manager` provides good CRUD interface
3. **Category organization** - Logical grouping system is solid
4. **Markdown content** - Human-readable, easy to edit
5. **Git integration** - Works well for server defaults
### What Needs Enhancement
1. **Storage backend** - Add SQLite for versioning and querying
2. **User data location** - Move to `~/.claude-prompts-mcp/` for user customizations
3. **History tracking** - Automatic versioning on every change
4. **Migration path** - Smooth transition from JSON to SQLite
5. **Backup system** - Automatic backups before modifications
---
## Architecture Design
### High-Level Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ MCP Tools Layer │
│ (prompt_manager, system_control - User Interface) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Storage Abstraction Layer │
│ (IStorageAdapter) │
└─────────────────────────────────────────────────────────────┘
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────────┐
│ JSONStorageAdapter │ │ SQLiteStorageAdapter │
│ (Legacy/Defaults) │ │ (User Data/History) │
└──────────────────────┘ └──────────────────────────┘
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────────┐
│ JSON Files │ │ SQLite Database │
│ /server/defaults/ │ │ ~/.claude-prompts-mcp/ │
└──────────────────────┘ └──────────────────────────┘
```
### Directory Structure (Post-Migration)
```
Server Installation:
/server/
├── src/
│ ├── storage/ # NEW: Storage layer
│ │ ├── interfaces/
│ │ │ └── IStorageAdapter.ts # Storage interface
│ │ ├── adapters/
│ │ │ ├── json-adapter.ts # JSON implementation
│ │ │ └── sqlite-adapter.ts # SQLite implementation
│ │ ├── migrations/
│ │ │ ├── json-to-sqlite.ts # Migration utilities
│ │ │ └── schema-versions.ts # Schema versioning
│ │ ├── versioning/
│ │ │ ├── history-manager.ts # Change history
│ │ │ └── backup-manager.ts # Automatic backups
│ │ └── index.ts
│ └── ...
├── defaults/ # MOVED: Server default prompts
│ ├── prompts/
│ │ ├── analysis/
│ │ ├── development/
│ │ └── ...
│ ├── gates/
│ └── frameworks/
└── prompts/ # KEPT: Legacy location (symlink to defaults)
User Environment:
~/.claude-prompts-mcp/
├── storage.db # SQLite database (all entities)
├── config.json # User preferences
├── backups/ # Automatic backups
│ ├── storage-2025-01-19-143020.db
│ └── storage-2025-01-18-091533.db
└── exports/ # Optional JSON exports
└── prompts-export-2025-01-19.json
```
### Hybrid Storage Strategy
**Two-Tier Storage System**:
1. **Server Defaults** (JSON in Git)
- Location: `/server/defaults/`
- Format: JSON + Markdown (current structure)
- Purpose: Shipped defaults, tracked in Git
- Loaded: On server startup, imported to user DB if missing
- Modification: Via Git commits, server updates
2. **User Data** (SQLite in Home Directory)
- Location: `~/.claude-prompts-mcp/storage.db`
- Format: SQLite with full versioning
- Purpose: User customizations, overrides, new items
- Loaded: Hot-reload from database
- Modification: Via MCP tools with automatic history
**Resolution Order**:
```
User requests prompt "analyze_code"
→ Check user database first (customizations)
→ If not found, check server defaults
→ Merge defaults with user overrides
```
---
## Storage Abstraction Layer
### Interface Design
```typescript
// server/src/storage/interfaces/IStorageAdapter.ts
export interface IStorageAdapter {
// ==================== LIFECYCLE ====================
initialize(): Promise<void>;
close(): Promise<void>;
health(): Promise<StorageHealthStatus>;
// ==================== PROMPTS ====================
// Basic CRUD
getPrompt(id: string): Promise<PromptData | null>;
savePrompt(prompt: PromptData, reason?: string): Promise<void>;
updatePrompt(id: string, updates: Partial<PromptData>, reason?: string): Promise<void>;
deletePrompt(id: string, reason?: string): Promise<void>;
listPrompts(filter?: FilterOptions): Promise<PromptData[]>;
// Versioning
getPromptHistory(id: string): Promise<PromptVersion[]>;
getPromptVersion(id: string, version: number): Promise<PromptData | null>;
rollbackPrompt(id: string, version: number): Promise<void>;
// ==================== FRAMEWORKS ====================
// Future: When framework dynamic creation is implemented
getFramework(id: string): Promise<FrameworkDefinition | null>;
saveFramework(framework: FrameworkDefinition, reason?: string): Promise<void>;
listFrameworks(): Promise<FrameworkDefinition[]>;
getFrameworkHistory(id: string): Promise<FrameworkVersion[]>;
// ==================== GATES ====================
// Future: When gate dynamic creation is implemented
getGate(id: string): Promise<GateDefinition | null>;
saveGate(gate: GateDefinition, reason?: string): Promise<void>;
listGates(): Promise<GateDefinition[]>;
getGateHistory(id: string): Promise<GateVersion[]>;
// ==================== UTILITIES ====================
// Backup & Export
backup(reason?: string): Promise<string>; // Returns backup file path
exportToJSON(path: string): Promise<void>;
importFromJSON(path: string): Promise<ImportResult>;
// Statistics
getStats(): Promise<StorageStats>;
// Search & Query
search(query: string, options?: SearchOptions): Promise<SearchResult[]>;
}
// Supporting Types
export interface StorageHealthStatus {
healthy: boolean;
storageType: 'json' | 'sqlite';
location: string;
size: number;
itemCount: number;
lastBackup?: number;
errors?: string[];
}
export interface PromptVersion {
version: number;
prompt: PromptData;
changedAt: number;
changeReason?: string;
changedBy?: string; // Future: multi-user support
}
export interface FilterOptions {
category?: string;
type?: 'prompt' | 'template' | 'chain' | 'workflow';
tags?: string[];
searchText?: string;
createdAfter?: number;
updatedAfter?: number;
limit?: number;
offset?: number;
}
export interface StorageStats {
totalPrompts: number;
totalFrameworks: number;
totalGates: number;
totalVersions: number;
storageSize: number;
oldestItem: number;
newestItem: number;
categories: { [key: string]: number };
}
export interface ImportResult {
imported: number;
updated: number;
skipped: number;
errors: Array<{ item: string; error: string }>;
}
```
### Adapter Factory Pattern
```typescript
// server/src/storage/index.ts
import { IStorageAdapter } from './interfaces/IStorageAdapter.js';
import { JSONStorageAdapter } from './adapters/json-adapter.js';
import { SQLiteStorageAdapter } from './adapters/sqlite-adapter.js';
import { Logger } from '../logging/index.js';
export class StorageFactory {
static async createAdapter(
config: StorageConfig,
logger: Logger
): Promise<IStorageAdapter> {
const storageType = config.type || 'sqlite'; // Default to SQLite
switch (storageType) {
case 'json':
logger.info('Initializing JSON storage adapter (legacy mode)');
return new JSONStorageAdapter(config.jsonPath, logger);
case 'sqlite':
logger.info('Initializing SQLite storage adapter');
const sqliteAdapter = new SQLiteStorageAdapter(
config.sqlitePath || '~/.claude-prompts-mcp/storage.db',
logger
);
// Load server defaults on first run
if (config.defaultsPath && await sqliteAdapter.isEmpty()) {
logger.info('First run detected, importing server defaults');
await this.importDefaults(sqliteAdapter, config.defaultsPath);
}
return sqliteAdapter;
default:
throw new Error(`Unknown storage type: ${storageType}`);
}
}
private static async importDefaults(
adapter: IStorageAdapter,
defaultsPath: string
): Promise<void> {
// Import server defaults from JSON files
const jsonAdapter = new JSONStorageAdapter(defaultsPath);
await jsonAdapter.initialize();
const prompts = await jsonAdapter.listPrompts();
for (const prompt of prompts) {
await adapter.savePrompt(prompt, 'Imported from server defaults');
}
}
}
export interface StorageConfig {
type: 'json' | 'sqlite';
jsonPath?: string; // For JSON adapter
sqlitePath?: string; // For SQLite adapter
defaultsPath?: string; // Server defaults location
}
```
---
## SQLite Schema Design
### Database Schema
```sql
-- ==================== CORE TABLES ====================
-- Prompts table (current state)
CREATE TABLE prompts (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
category TEXT,
content TEXT NOT NULL, -- Full prompt content (markdown)
content_type TEXT DEFAULT 'markdown', -- 'markdown' | 'nunjucks'
-- Metadata
metadata TEXT, -- JSON: { arguments, systemMessage, tags, etc. }
-- Versioning
version INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL, -- Unix timestamp
updated_at INTEGER NOT NULL, -- Unix timestamp
-- Source tracking
source TEXT DEFAULT 'user', -- 'user' | 'server_default' | 'imported'
is_default BOOLEAN DEFAULT 0, -- From server defaults?
-- Status
deleted BOOLEAN DEFAULT 0, -- Soft delete
deleted_at INTEGER
);
-- Prompt history (all versions)
CREATE TABLE prompt_history (
prompt_id TEXT NOT NULL,
version INTEGER NOT NULL,
-- Snapshot of prompt at this version
name TEXT NOT NULL,
description TEXT,
category TEXT,
content TEXT NOT NULL,
metadata TEXT,
-- Change tracking
changed_at INTEGER NOT NULL,
change_reason TEXT,
previous_version INTEGER,
PRIMARY KEY (prompt_id, version),
FOREIGN KEY (prompt_id) REFERENCES prompts(id) ON DELETE CASCADE
);
-- Frameworks table (future)
CREATE TABLE frameworks (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
methodology TEXT NOT NULL, -- JSON: Full methodology guide
-- Metadata
metadata TEXT,
-- Versioning
version INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
-- Source tracking
source TEXT DEFAULT 'user',
is_default BOOLEAN DEFAULT 0,
-- Status
deleted BOOLEAN DEFAULT 0,
deleted_at INTEGER
);
-- Framework history
CREATE TABLE framework_history (
framework_id TEXT NOT NULL,
version INTEGER NOT NULL,
name TEXT NOT NULL,
description TEXT,
methodology TEXT NOT NULL,
metadata TEXT,
changed_at INTEGER NOT NULL,
change_reason TEXT,
previous_version INTEGER,
PRIMARY KEY (framework_id, version),
FOREIGN KEY (framework_id) REFERENCES frameworks(id) ON DELETE CASCADE
);
-- Gates table (future)
CREATE TABLE gates (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
gate_type TEXT NOT NULL, -- 'validation' | 'quality' | 'approval' | 'condition'
definition TEXT NOT NULL, -- JSON: Full gate definition
-- Metadata
metadata TEXT,
-- Versioning
version INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
-- Source tracking
source TEXT DEFAULT 'user',
is_default BOOLEAN DEFAULT 0,
-- Status
deleted BOOLEAN DEFAULT 0,
deleted_at INTEGER
);
-- Gate history
CREATE TABLE gate_history (
gate_id TEXT NOT NULL,
version INTEGER NOT NULL,
name TEXT NOT NULL,
description TEXT,
gate_type TEXT NOT NULL,
definition TEXT NOT NULL,
metadata TEXT,
changed_at INTEGER NOT NULL,
change_reason TEXT,
previous_version INTEGER,
PRIMARY KEY (gate_id, version),
FOREIGN KEY (gate_id) REFERENCES gates(id) ON DELETE CASCADE
);
-- ==================== INDEXES ====================
-- Prompts indexes
CREATE INDEX idx_prompts_category ON prompts(category);
CREATE INDEX idx_prompts_updated ON prompts(updated_at);
CREATE INDEX idx_prompts_source ON prompts(source);
CREATE INDEX idx_prompts_deleted ON prompts(deleted);
CREATE INDEX idx_prompts_name ON prompts(name);
-- History indexes
CREATE INDEX idx_prompt_history_changed ON prompt_history(changed_at);
CREATE INDEX idx_prompt_history_prompt ON prompt_history(prompt_id);
-- Frameworks indexes
CREATE INDEX idx_frameworks_updated ON frameworks(updated_at);
CREATE INDEX idx_frameworks_source ON frameworks(source);
CREATE INDEX idx_frameworks_deleted ON frameworks(deleted);
-- Gates indexes
CREATE INDEX idx_gates_type ON gates(gate_type);
CREATE INDEX idx_gates_updated ON gates(updated_at);
CREATE INDEX idx_gates_source ON gates(source);
CREATE INDEX idx_gates_deleted ON gates(deleted);
-- ==================== TRIGGERS ====================
-- Automatic history archiving on update
CREATE TRIGGER archive_prompt_on_update
AFTER UPDATE ON prompts
WHEN NEW.version > OLD.version
BEGIN
INSERT INTO prompt_history (
prompt_id, version, name, description, category,
content, metadata, changed_at, change_reason, previous_version
) VALUES (
OLD.id, OLD.version, OLD.name, OLD.description, OLD.category,
OLD.content, OLD.metadata, NEW.updated_at,
'Automatic archive on update', OLD.version
);
END;
-- Update timestamp trigger
CREATE TRIGGER update_prompt_timestamp
AFTER UPDATE ON prompts
BEGIN
UPDATE prompts SET updated_at = strftime('%s', 'now') * 1000
WHERE id = NEW.id;
END;
-- Similar triggers for frameworks and gates (future)
```
### Schema Versioning
```sql
-- Schema version tracking
CREATE TABLE schema_version (
version INTEGER PRIMARY KEY,
applied_at INTEGER NOT NULL,
description TEXT
);
-- Initial version
INSERT INTO schema_version (version, applied_at, description)
VALUES (1, strftime('%s', 'now') * 1000, 'Initial schema with prompts, frameworks, gates and profile support');
```
### Profile Support Architecture
**Design Decision**: Build profile support into schema from day one, enable UI features later.
**Rationale**: At 500+ items, users need work/life/code separation. Adding profiles later requires painful schema migration. Adding now costs +1 day but avoids future migration and enables seamless profile features when needed.
**Strategy**: Single database with `profile_id` column, not separate database files.
#### Enhanced Schema with Profiles
```sql
-- ==================== PROFILES MANAGEMENT ====================
-- Profiles table
CREATE TABLE profiles (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
description TEXT,
icon TEXT, -- Optional: emoji like 💼, 🏠, 💻
color TEXT, -- Optional: hex color for UI
settings TEXT, -- JSON: Profile-specific settings
is_default BOOLEAN DEFAULT 0,
is_active BOOLEAN DEFAULT 1,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
last_used_at INTEGER
);
-- Active profile tracking (single row table)
CREATE TABLE active_profile (
id INTEGER PRIMARY KEY DEFAULT 1,
profile_id TEXT NOT NULL,
changed_at INTEGER NOT NULL,
FOREIGN KEY (profile_id) REFERENCES profiles(id),
CHECK (id = 1)
);
-- Profile-aware prompts (enhanced)
CREATE TABLE prompts (
id TEXT PRIMARY KEY,
profile_id TEXT NOT NULL DEFAULT 'default', -- NEW: Profile association
name TEXT NOT NULL,
description TEXT,
category TEXT,
content TEXT NOT NULL,
content_type TEXT DEFAULT 'markdown',
metadata TEXT,
version INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
source TEXT DEFAULT 'user',
is_default BOOLEAN DEFAULT 0,
original_profile_id TEXT, -- NEW: If shared from another profile
deleted BOOLEAN DEFAULT 0,
deleted_at INTEGER,
FOREIGN KEY (profile_id) REFERENCES profiles(id) ON DELETE CASCADE
);
-- Enhanced indexes for profile-aware queries
CREATE INDEX idx_prompts_profile ON prompts(profile_id);
CREATE INDEX idx_prompts_profile_category ON prompts(profile_id, category);
CREATE INDEX idx_prompts_profile_deleted ON prompts(profile_id, deleted);
-- Profile-aware history
CREATE INDEX idx_prompt_history_profile ON prompt_history(profile_id);
-- Initial default profile
INSERT INTO profiles (id, name, description, is_default, is_active, created_at, updated_at, last_used_at)
VALUES (
'default',
'Default',
'Default profile for all prompts',
1,
1,
strftime('%s', 'now') * 1000,
strftime('%s', 'now') * 1000,
strftime('%s', 'now') * 1000
);
INSERT INTO active_profile (profile_id, changed_at)
VALUES ('default', strftime('%s', 'now') * 1000);
```
#### Profile-Aware Adapter Implementation
**Phase 1 Implementation** (default profile only):
```typescript
export class SQLiteStorageAdapter implements IStorageAdapter {
private activeProfileId: string = 'default';
async initialize(): Promise<void> {
this.initializeSchema(); // Creates profiles table + default profile
this.activeProfileId = await this.loadActiveProfile(); // 'default'
}
async getPrompt(id: string): Promise<PromptData | null> {
// Automatically scoped to active profile
const row = this.db.prepare(`
SELECT * FROM prompts
WHERE id = ? AND profile_id = ? AND deleted = 0
`).get(id, this.activeProfileId);
return row ? this.rowToPromptData(row) : null;
}
async savePrompt(prompt: PromptData, reason?: string): Promise<void> {
// Automatically associates with active profile
this.db.prepare(`
INSERT INTO prompts (
id, profile_id, name, description, category, content, metadata,
version, created_at, updated_at, source
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(
prompt.id,
this.activeProfileId, // Always 'default' in Phase 1
prompt.name,
// ... other fields
);
}
}
```
**User Experience in Phase 1**:
- Single "default" profile created automatically
- All queries scoped to default profile (invisible to users)
- Zero complexity, works exactly like no profiles
- Foundation ready for future profile features
**Server Defaults Handling**:
- Server defaults marked with `is_default = 1`
- Accessible from all profiles via SQL view or query filter
- Not duplicated across profiles (efficient)
```sql
-- View: Prompts for active profile + shared defaults
CREATE VIEW profile_prompts_with_defaults AS
SELECT p.*
FROM prompts p
WHERE p.profile_id = (SELECT profile_id FROM active_profile WHERE id = 1)
OR p.is_default = 1;
```
#### Future Profile Features (Phase 6+)
**MCP Tool Commands** (deferred until needed):
```bash
# Create profiles
>>system_control action="create_profile" profile='{"id":"work","name":"Work Projects","icon":"💼"}'
# List profiles
>>system_control action="list_profiles"
# Output: 💼 Work (127 prompts, active) | 🏠 Life (43 prompts) | 💻 Code (89 prompts)
# Switch profiles
>>system_control action="switch_profile" profile_id="work"
# Cross-profile operations
>>prompt_manager action="copy_to_profile" id="analyze_code" target_profile="work"
>>prompt_manager action="share_prompt" id="research_template" profiles='["work","code"]'
```
**Cost-Benefit**:
- **Cost Now**: +1 day to Phase 1 (add profile tables, scoped queries)
- **Cost Later**: 0 days (no migration needed)
- **Benefit**: Seamless profile features when users hit 500+ items
- **Risk**: None (hidden from users, default profile only)
---
## Migration Strategy
### Phase 1: Storage Abstraction Layer (Non-Breaking)
**Goal**: Introduce adapter pattern without changing storage backend
**Implementation**:
1. **Create interface and JSON adapter**:
```typescript
// server/src/storage/adapters/json-adapter.ts
export class JSONStorageAdapter implements IStorageAdapter {
// Wraps existing file-based system
// No changes to actual storage
// Just provides IStorageAdapter interface
}
```
2. **Update existing code to use adapter**:
```typescript
// server/src/prompts/loader.ts
// BEFORE:
const prompts = await loadPromptsFromJSON(configPath);
// AFTER:
const storage = await StorageFactory.createAdapter(config, logger);
const prompts = await storage.listPrompts();
```
3. **Update MCP tools**:
```typescript
// server/src/mcp-tools/prompt-manager.ts
// Replace direct file operations with storage adapter calls
const prompt = await storage.getPrompt(id);
await storage.savePrompt(updatedPrompt);
```
**Testing**:
- All existing tests should pass
- No user-visible changes
- Storage still JSON files
**Duration**: 2-3 days
---
### Phase 2: SQLite Adapter Implementation
**Goal**: Implement SQLite adapter with feature parity
**Implementation**:
1. **SQLite adapter skeleton**:
```typescript
// server/src/storage/adapters/sqlite-adapter.ts
import Database from 'better-sqlite3';
export class SQLiteStorageAdapter implements IStorageAdapter {
private db: Database.Database;
constructor(dbPath: string, logger: Logger) {
this.db = new Database(dbPath);
this.initializeSchema();
}
private initializeSchema(): void {
// Execute schema.sql
}
async getPrompt(id: string): Promise<PromptData | null> {
const row = this.db.prepare(
'SELECT * FROM prompts WHERE id = ? AND deleted = 0'
).get(id);
return row ? this.rowToPromptData(row) : null;
}
async savePrompt(prompt: PromptData, reason?: string): Promise<void> {
const exists = this.db.prepare(
'SELECT id FROM prompts WHERE id = ?'
).get(prompt.id);
if (exists) {
await this.updatePrompt(prompt.id, prompt, reason);
} else {
this.insertPrompt(prompt, reason);
}
}
private insertPrompt(prompt: PromptData, reason?: string): void {
this.db.prepare(`
INSERT INTO prompts (
id, name, description, category, content, metadata,
version, created_at, updated_at, source
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`).run(
prompt.id,
prompt.name,
prompt.description,
prompt.category,
prompt.content,
JSON.stringify(prompt.metadata),
1,
Date.now(),
Date.now(),
'user'
);
}
async updatePrompt(
id: string,
updates: Partial<PromptData>,
reason?: string
): Promise<void> {
// Archive current version first (manual, trigger handles auto)
const current = await this.getPrompt(id);
if (!current) throw new Error(`Prompt not found: ${id}`);
this.db.prepare(`
INSERT INTO prompt_history (
prompt_id, version, name, description, category,
content, metadata, changed_at, change_reason
) SELECT
id, version, name, description, category,
content, metadata, ?, ?
FROM prompts WHERE id = ?
`).run(Date.now(), reason || 'Update', id);
// Update with incremented version
this.db.prepare(`
UPDATE prompts SET
name = COALESCE(?, name),
description = COALESCE(?, description),
category = COALESCE(?, category),
content = COALESCE(?, content),
metadata = COALESCE(?, metadata),
version = version + 1,
updated_at = ?
WHERE id = ?
`).run(
updates.name,
updates.description,
updates.category,
updates.content,
updates.metadata ? JSON.stringify(updates.metadata) : null,
Date.now(),
id
);
}
async getPromptHistory(id: string): Promise<PromptVersion[]> {
const rows = this.db.prepare(`
SELECT * FROM prompt_history
WHERE prompt_id = ?
ORDER BY version DESC
`).all(id);
return rows.map(row => this.rowToPromptVersion(row));
}
async rollbackPrompt(id: string, version: number): Promise<void> {
// Get the historical version
const historical = this.db.prepare(`
SELECT * FROM prompt_history
WHERE prompt_id = ? AND version = ?
`).get(id, version);
if (!historical) {
throw new Error(`Version ${version} not found for prompt ${id}`);
}
// Archive current before rollback
await this.updatePrompt(id, {
name: historical.name,
description: historical.description,
category: historical.category,
content: historical.content,
metadata: JSON.parse(historical.metadata)
}, `Rollback to version ${version}`);
}
// ... other methods
}
```
2. **Configuration support**:
```typescript
// server/config.json
{
"storage": {
"type": "sqlite", // "json" for legacy
"sqlitePath": "~/.claude-prompts-mcp/storage.db",
"defaultsPath": "./defaults/prompts",
"backupOnStartup": true,
"autoBackup": {
"enabled": true,
"intervalHours": 24,
"keepCount": 7
}
}
}
```
**Testing**:
- Unit tests for all adapter methods
- Migration tests (JSON → SQLite)
- Performance benchmarks
- Concurrent access tests
**Duration**: 3-4 days
---
### Phase 3: User Data Separation
**Goal**: Move user data to home directory, keep server defaults in Git
**Implementation**:
1. **Data location setup**:
```typescript
// server/src/storage/paths.ts
import * as path from 'path';
import * as os from 'os';
import * as fs from 'fs/promises';
export class StoragePaths {
static async getUserDataPath(): Promise<string> {
const homeDir = os.homedir();
const dataDir = path.join(homeDir, '.claude-prompts-mcp');
// Ensure directory exists
await fs.mkdir(dataDir, { recursive: true });
await fs.mkdir(path.join(dataDir, 'backups'), { recursive: true });
await fs.mkdir(path.join(dataDir, 'exports'), { recursive: true });
return dataDir;
}
static async getStoragePath(): Promise<string> {
const dataDir = await this.getUserDataPath();
return path.join(dataDir, 'storage.db');
}
static getServerDefaultsPath(): string {
// Server installation defaults
return path.join(__dirname, '../../defaults');
}
}
```
2. **First-run initialization**:
```typescript
// server/src/storage/initialization.ts
export class StorageInitializer {
static async initializeUserStorage(logger: Logger): Promise<void> {
const dbPath = await StoragePaths.getStoragePath();
const defaultsPath = StoragePaths.getServerDefaultsPath();
// Check if database exists
const isFirstRun = !(await this.fileExists(dbPath));
if (isFirstRun) {
logger.info('First run detected, creating user database');
// Create SQLite database
const storage = new SQLiteStorageAdapter(dbPath, logger);
await storage.initialize();
// Import server defaults
logger.info('Importing server defaults to user database');
await this.importDefaults(storage, defaultsPath, logger);
logger.info('User storage initialized successfully');
}
}
private static async importDefaults(
storage: IStorageAdapter,
defaultsPath: string,
logger: Logger
): Promise<void> {
// Load defaults from JSON
const jsonAdapter = new JSONStorageAdapter(defaultsPath, logger);
await jsonAdapter.initialize();
const prompts = await jsonAdapter.listPrompts();
for (const prompt of prompts) {
await storage.savePrompt({
...prompt,
source: 'server_default',
isDefault: true
}, 'Imported from server defaults');
}
logger.info(`Imported ${prompts.length} default prompts`);
}
}
```
3. **Update server startup**:
```typescript
// server/src/runtime/application.ts
async startupPhase1_Foundation(): Promise<void> {
// ... existing code ...
// Initialize user storage
await StorageInitializer.initializeUserStorage(this.logger);
// Create storage adapter
const config: StorageConfig = {
type: 'sqlite',
sqlitePath: await StoragePaths.getStoragePath(),
defaultsPath: StoragePaths.getServerDefaultsPath()
};
this.storage = await StorageFactory.createAdapter(config, this.logger);
await this.storage.initialize();
}
```
**Migration for Existing Users**:
```typescript
// One-time migration script
export async function migrateExistingUserData(): Promise<void> {
const oldPath = path.join(__dirname, '../../prompts');
const newPath = await StoragePaths.getStoragePath();
// Check if user has modified prompts in old location
const hasCustomPrompts = await detectUserModifications(oldPath);
if (hasCustomPrompts) {
console.log('Migrating user prompts to ~/.claude-prompts-mcp/');
// Create SQLite database
const storage = new SQLiteStorageAdapter(newPath, logger);
await storage.initialize();
// Import existing prompts
const jsonAdapter = new JSONStorageAdapter(oldPath, logger);
const prompts = await jsonAdapter.listPrompts();
for (const prompt of prompts) {
await storage.savePrompt(prompt, 'Migrated from server location');
}
console.log('Migration complete!');
}
}
```
**Testing**:
- First-run experience
- Migration from old location
- Default import verification
- Path resolution cross-platform
**Duration**: 2 days
---
### Phase 4: Versioning & History Features
**Goal**: Expose history features through MCP tools
**Implementation**:
1. **Enhanced MCP tool commands**:
```typescript
// server/src/mcp-tools/prompt-manager.ts
// Add new actions to PromptManagerAction enum
export enum PromptManagerAction {
// ... existing actions ...
HISTORY = 'history',
GET_VERSION = 'get_version',
ROLLBACK = 'rollback',
COMPARE_VERSIONS = 'compare_versions'
}
// Implementation
async handleHistoryAction(id: string): Promise<ToolResponse> {
const history = await this.storage.getPromptHistory(id);
if (history.length === 0) {
return {
success: false,
message: `No history found for prompt: ${id}`
};
}
// Format for LLM consumption
const formatted = history.map(v => ({
version: v.version,
changedAt: new Date(v.changedAt).toISOString(),
reason: v.changeReason || 'No reason provided',
summary: this.summarizeChanges(v)
}));
return {
success: true,
data: {
promptId: id,
currentVersion: history[0].version,
totalVersions: history.length,
history: formatted
},
message: `Retrieved ${history.length} versions for prompt: ${id}`
};
}
async handleRollbackAction(
id: string,
version: number
): Promise<ToolResponse> {
try {
// Get current state
const current = await this.storage.getPrompt(id);
if (!current) {
return { success: false, message: `Prompt not found: ${id}` };
}
// Automatic backup before rollback
await this.storage.backup(`Before rollback of ${id} to v${version}`);
// Perform rollback
await this.storage.rollbackPrompt(id, version);
return {
success: true,
message: `Rolled back prompt "${id}" from v${current.version} to v${version}`,
data: {
previousVersion: current.version,
newVersion: version,
backupCreated: true
}
};
} catch (error) {
return {
success: false,
message: `Rollback failed: ${error.message}`
};
}
}
```
2. **Usage examples for LLM**:
```typescript
// Add to tool descriptions
const examples = `
View change history:
>>prompt_manager action="history" id="analyze_code"
Output:
Version 5 (2025-01-19 14:30) - Updated argument descriptions
Version 4 (2025-01-18 10:15) - Added error handling guidance
Version 3 (2025-01-17 09:00) - Refactored structure
Version 2 (2025-01-16 15:45) - Initial refinements
Version 1 (2025-01-15 11:20) - Initial creation
Rollback to previous version:
>>prompt_manager action="rollback" id="analyze_code" version="4"
Output:
✓ Rolled back "analyze_code" from v5 to v4
✓ Automatic backup created
Compare versions:
>>prompt_manager action="compare_versions" id="analyze_code" version1="4" version2="5"
Output:
Changes in v5:
+ Added argument: "error_handling_level"
~ Modified: System message instructions
- Removed: Legacy compatibility note
`;
```
3. **Automatic backup manager**:
```typescript
// server/src/storage/versioning/backup-manager.ts
export class BackupManager {
private backupDir: string;
private maxBackups: number;
async createBackup(
dbPath: string,
reason: string
): Promise<string> {
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
const backupName = `storage-${timestamp}.db`;
const backupPath = path.join(this.backupDir, backupName);
// Copy database file
await fs.copyFile(dbPath, backupPath);
// Write backup metadata
await fs.writeFile(
backupPath + '.meta.json',
JSON.stringify({
reason,
timestamp: Date.now(),
originalPath: dbPath
})
);
// Cleanup old backups
await this.cleanupOldBackups();
return backupPath;
}
private async cleanupOldBackups(): Promise<void> {
const backups = await this.listBackups();
if (backups.length > this.maxBackups) {
// Delete oldest backups
const toDelete = backups
.sort((a, b) => a.timestamp - b.timestamp)
.slice(0, backups.length - this.maxBackups);
for (const backup of toDelete) {
await fs.unlink(backup.path);
await fs.unlink(backup.path + '.meta.json');
}
}
}
}
```
**Testing**:
- History retrieval accuracy
- Rollback correctness
- Backup creation and restoration
- Concurrent modification handling
**Duration**: 2-3 days
---
## MCP Tool Integration
### Enhanced Prompt Manager Tool
**New Schema Extensions**:
```typescript
const promptManagerSchema = z.object({
action: z.enum([
'create', 'update', 'delete', 'list', 'analyze',
'history', 'get_version', 'rollback', 'compare_versions', // NEW
'export', 'import', 'backup' // NEW
]),
id: z.string().optional(),
version: z.number().optional(), // For version-specific operations
version1: z.number().optional(), // For comparisons
version2: z.number().optional(),
// ... existing fields ...
});
```
**Command Examples**:
```bash
# View history
>>prompt_manager action="history" id="analyze_code"
# Get specific version
>>prompt_manager action="get_version" id="analyze_code" version="3"
# Rollback
>>prompt_manager action="rollback" id="analyze_code" version="4"
# Compare versions
>>prompt_manager action="compare_versions" id="analyze_code" version1="4" version2="5"
# Create backup
>>prompt_manager action="backup" reason="Before major refactor"
# Export to JSON (for Git backup)
>>prompt_manager action="export" format="json" path="./my-prompts-backup/"
# Import from JSON
>>prompt_manager action="import" path="./my-prompts-backup/prompts.json"
```
### System Control Enhancements
**Storage status command**:
```typescript
// Add to system_control tool
async handleStorageStatus(): Promise<ToolResponse> {
const health = await this.storage.health();
const stats = await this.storage.getStats();
return {
success: true,
data: {
storage: {
type: health.storageType,
location: health.location,
size: this.formatBytes(health.size),
healthy: health.healthy,
lastBackup: health.lastBackup
? new Date(health.lastBackup).toISOString()
: 'Never'
},
statistics: {
totalPrompts: stats.totalPrompts,
totalFrameworks: stats.totalFrameworks,
totalGates: stats.totalGates,
totalVersions: stats.totalVersions,
categories: stats.categories
}
}
};
}
```
**Usage**:
```bash
>>system_control action="storage_status"
Output:
Storage Status:
Type: SQLite
Location: ~/.claude-prompts-mcp/storage.db
Size: 2.4 MB
Health: ✓ Healthy
Last Backup: 2025-01-19 14:30:00
Statistics:
Prompts: 127 (892 versions)
Frameworks: 4 (12 versions)
Gates: 23 (67 versions)
Categories: 18
```
---
## Implementation Phases
### Phase 1: Storage Abstraction (Week 1)
**Deliverables**:
- ✅ `IStorageAdapter` interface defined
- ✅ `JSONStorageAdapter` wrapping existing system
- ✅ `StorageFactory` for adapter creation
- ✅ Updated MCP tools to use adapter
- ✅ All existing tests passing
**Testing**:
- Unit tests for adapter interface
- Integration tests with existing system
- No regressions in functionality
**Success Criteria**:
- Zero breaking changes
- All existing features work
- Adapter pattern validated
---
### Phase 2: SQLite Implementation (Week 2)
**Deliverables**:
- ✅ `SQLiteStorageAdapter` fully implemented
- ✅ Schema creation with profile support (default profile only)
- ✅ Profile-aware CRUD operations (auto-scoped to default)
- ✅ Basic CRUD operations working
- ✅ Migration utility (JSON → SQLite)
- ✅ Configuration support
**Profile Support Note**:
- Profile tables created, default profile initialized
- All queries automatically scoped to 'default' profile
- Zero user-visible changes (single profile mode)
- Foundation ready for future multi-profile features
**Testing**:
- Unit tests for all SQL operations
- Profile-scoped query tests
- Migration tests
- Performance benchmarks (<10ms with profile filtering)
- Concurrent access tests
**Success Criteria**:
- Feature parity with JSON adapter
- <10ms query performance (with profile_id filter)
- Successful migrations from JSON
- Profile foundation validated
---
### Phase 3: User Data Separation (Week 3)
**Deliverables**:
- ✅ User data path management (`~/.claude-prompts-mcp/`)
- ✅ First-run initialization
- ✅ Server defaults import
- ✅ Migration script for existing users
- ✅ Documentation updated
**Testing**:
- First-run experience
- Migration from old location
- Cross-platform path resolution
- Default import accuracy
**Success Criteria**:
- User data isolated from server
- Server defaults tracked in Git
- Smooth upgrade path for existing users
---
### Phase 4: Versioning & History (Week 4)
**Deliverables**:
- ✅ History tracking implemented
- ✅ Version retrieval methods
- ✅ Rollback functionality
- ✅ Automatic backup system
- ✅ MCP tool history commands
**Testing**:
- History accuracy tests
- Rollback correctness
- Backup/restore verification
- Concurrent modification handling
**Success Criteria**:
- Full change history for all entities
- Safe rollback with automatic backups
- Performance: <50ms for history queries
---
### Phase 5: Polish & Documentation (Week 5)
**Deliverables**:
- ✅ Export/import features
- ✅ Storage statistics and health monitoring
- ✅ Comprehensive error handling
- ✅ User documentation
- ✅ Migration guide
**Testing**:
- End-to-end testing
- Performance validation
- Edge case coverage
- User acceptance testing
**Success Criteria**:
- All features documented
- Migration guide complete
- Performance targets met
---
## Testing Strategy
### Unit Tests
**Storage Adapter Tests**:
```typescript
// tests/storage/sqlite-adapter.test.ts
describe('SQLiteStorageAdapter', () => {
let adapter: SQLiteStorageAdapter;
let testDbPath: string;
beforeEach(async () => {
testDbPath = path.join(tmpdir(), `test-${Date.now()}.db`);
adapter = new SQLiteStorageAdapter(testDbPath, logger);
await adapter.initialize();
});
afterEach(async () => {
await adapter.close();
await fs.unlink(testDbPath);
});
describe('getPrompt', () => {
it('should retrieve existing prompt', async () => {
const prompt = createTestPrompt();
await adapter.savePrompt(prompt);
const retrieved = await adapter.getPrompt(prompt.id);
expect(retrieved).toEqual(prompt);
});
it('should return null for non-existent prompt', async () => {
const result = await adapter.getPrompt('non-existent');
expect(result).toBeNull();
});
});
describe('savePrompt', () => {
it('should create new prompt', async () => {
const prompt = createTestPrompt();
await adapter.savePrompt(prompt, 'Initial creation');
const retrieved = await adapter.getPrompt(prompt.id);
expect(retrieved).toBeTruthy();
expect(retrieved.version).toBe(1);
});
it('should update existing prompt and increment version', async () => {
const prompt = createTestPrompt();
await adapter.savePrompt(prompt);
const updated = { ...prompt, name: 'Updated Name' };
await adapter.savePrompt(updated, 'Name change');
const retrieved = await adapter.getPrompt(prompt.id);
expect(retrieved.name).toBe('Updated Name');
expect(retrieved.version).toBe(2);
});
});
describe('getPromptHistory', () => {
it('should track all versions', async () => {
const prompt = createTestPrompt();
await adapter.savePrompt(prompt);
// Make 3 updates
for (let i = 0; i < 3; i++) {
await adapter.updatePrompt(
prompt.id,
{ name: `Version ${i + 2}` },
`Update ${i + 1}`
);
}
const history = await adapter.getPromptHistory(prompt.id);
expect(history).toHaveLength(4); // Original + 3 updates
expect(history[0].version).toBe(4);
expect(history[3].version).toBe(1);
});
});
describe('rollbackPrompt', () => {
it('should restore previous version', async () => {
const prompt = createTestPrompt();
await adapter.savePrompt(prompt);
await adapter.updatePrompt(prompt.id, { name: 'V2' });
await adapter.updatePrompt(prompt.id, { name: 'V3' });
await adapter.rollbackPrompt(prompt.id, 1);
const current = await adapter.getPrompt(prompt.id);
expect(current.name).toBe(prompt.name);
});
});
});
```
### Integration Tests
**Migration Tests**:
```typescript
// tests/storage/migration.test.ts
describe('JSON to SQLite Migration', () => {
it('should migrate all prompts correctly', async () => {
// Setup JSON data
const jsonPath = setupTestJSONStorage();
const jsonAdapter = new JSONStorageAdapter(jsonPath, logger);
const originalPrompts = await jsonAdapter.listPrompts();
// Migrate to SQLite
const sqlitePath = path.join(tmpdir(), 'migrated.db');
const sqliteAdapter = new SQLiteStorageAdapter(sqlitePath, logger);
for (const prompt of originalPrompts) {
await sqliteAdapter.savePrompt(prompt, 'Migrated from JSON');
}
// Verify
const migratedPrompts = await sqliteAdapter.listPrompts();
expect(migratedPrompts).toHaveLength(originalPrompts.length);
for (const original of originalPrompts) {
const migrated = await sqliteAdapter.getPrompt(original.id);
expect(migrated).toMatchObject(original);
}
});
});
```
### Performance Tests
**Benchmark Suite**:
```typescript
// tests/storage/performance.test.ts
describe('Storage Performance', () => {
it('should retrieve prompts in <10ms', async () => {
const adapter = new SQLiteStorageAdapter(dbPath, logger);
// Insert test data
for (let i = 0; i < 100; i++) {
await adapter.savePrompt(createTestPrompt(`prompt-${i}`));
}
// Benchmark retrieval
const start = performance.now();
for (let i = 0; i < 100; i++) {
await adapter.getPrompt(`prompt-${i}`);
}
const duration = performance.now() - start;
const avgTime = duration / 100;
expect(avgTime).toBeLessThan(10);
});
it('should list prompts with filtering in <50ms', async () => {
// Insert 500 prompts across categories
for (let i = 0; i < 500; i++) {
await adapter.savePrompt({
...createTestPrompt(`prompt-${i}`),
category: `category-${i % 10}`
});
}
// Benchmark filtered listing
const start = performance.now();
const results = await adapter.listPrompts({
category: 'category-5',
searchText: 'test'
});
const duration = performance.now() - start;
expect(duration).toBeLessThan(50);
expect(results.length).toBeGreaterThan(0);
});
});
```
---
## Rollback & Recovery
### Automatic Backup System
**Backup Triggers**:
1. **Before major operations**:
- Rollback
- Bulk import
- Mass deletion
2. **Scheduled backups**:
- Daily at midnight
- Configurable interval
3. **Manual backups**:
- Via MCP tool command
- Before user-initiated migrations
**Implementation**:
```typescript
// server/src/storage/versioning/backup-manager.ts
export class AutomaticBackupSystem {
private backupInterval: NodeJS.Timer;
startAutoBackup(adapter: IStorageAdapter, intervalHours: number): void {
this.backupInterval = setInterval(
async () => {
await adapter.backup('Scheduled automatic backup');
},
intervalHours * 60 * 60 * 1000
);
}
async backupBeforeRiskyOperation(
adapter: IStorageAdapter,
operation: string
): Promise<string> {
return await adapter.backup(`Before ${operation}`);
}
}
```
### Recovery Procedures
**Restore from Backup**:
```typescript
async restoreFromBackup(backupPath: string): Promise<void> {
// Close current database connection
await this.storage.close();
// Get current database path
const dbPath = await StoragePaths.getStoragePath();
// Archive current database
const archivePath = `${dbPath}.before-restore-${Date.now()}`;
await fs.rename(dbPath, archivePath);
// Restore from backup
await fs.copyFile(backupPath, dbPath);
// Reinitialize storage
this.storage = new SQLiteStorageAdapter(dbPath, this.logger);
await this.storage.initialize();
this.logger.info(`Restored from backup: ${backupPath}`);
}
```
**MCP Tool Command**:
```bash
>>system_control action="list_backups"
# Shows available backups with timestamps and reasons
>>system_control action="restore_backup" backup="storage-2025-01-19-143020.db"
# Restores from specified backup
```
---
## Performance Considerations
### Query Optimization
**Indexed Queries**:
```sql
-- Fast category filtering (indexed)
SELECT * FROM prompts
WHERE category = ? AND deleted = 0;
-- Fast date range queries (indexed)
SELECT * FROM prompts
WHERE updated_at > ? AND deleted = 0;
-- Full-text search (consider FTS5 extension)
CREATE VIRTUAL TABLE prompts_fts USING fts5(
id, name, description, content,
content=prompts
);
```
**Prepared Statements**:
```typescript
// Pre-compile frequently used queries
class SQLiteStorageAdapter {
private preparedQueries = {
getPrompt: null as Database.Statement | null,
savePrompt: null as Database.Statement | null,
listByCategory: null as Database.Statement | null
};
initialize() {
// Compile once, reuse many times
this.preparedQueries.getPrompt = this.db.prepare(
'SELECT * FROM prompts WHERE id = ? AND deleted = 0'
);
this.preparedQueries.listByCategory = this.db.prepare(
'SELECT * FROM prompts WHERE category = ? AND deleted = 0'
);
}
async getPrompt(id: string): Promise<PromptData | null> {
// Use pre-compiled statement
const row = this.preparedQueries.getPrompt.get(id);
return row ? this.rowToPromptData(row) : null;
}
}
```
### Transaction Batching
**Bulk Operations**:
```typescript
async bulkSave(prompts: PromptData[]): Promise<void> {
// Use transaction for atomic bulk insert
const transaction = this.db.transaction((prompts: PromptData[]) => {
for (const prompt of prompts) {
this.insertPrompt(prompt);
}
});
transaction(prompts);
}
```
### Memory Management
**Streaming Large Results**:
```typescript
async *streamPrompts(filter?: FilterOptions): AsyncGenerator<PromptData> {
const stmt = this.db.prepare('SELECT * FROM prompts WHERE deleted = 0');
for (const row of stmt.iterate()) {
yield this.rowToPromptData(row);
}
}
// Usage
for await (const prompt of adapter.streamPrompts()) {
// Process one at a time, memory efficient
}
```
### Performance Targets
**Benchmarks**:
- **Single prompt retrieval**: <10ms
- **List 100 prompts**: <50ms
- **History query (50 versions)**: <30ms
- **Rollback operation**: <100ms
- **Backup creation**: <500ms for 10MB database
- **Database size**: <1KB per prompt with history
---
## Future Enhancements
### Phase 6+: Advanced Features
**1. Full-Text Search**:
```sql
-- SQLite FTS5 extension
CREATE VIRTUAL TABLE prompts_fts USING fts5(
id UNINDEXED,
name,
description,
content,
tokenize='porter unicode61'
);
-- Trigger to keep FTS in sync
CREATE TRIGGER prompts_fts_insert AFTER INSERT ON prompts BEGIN
INSERT INTO prompts_fts(id, name, description, content)
VALUES (new.id, new.name, new.description, new.content);
END;
```
**2. Change Diffs**:
```typescript
async getPromptDiff(
id: string,
version1: number,
version2: number
): Promise<PromptDiff> {
const v1 = await this.getPromptVersion(id, version1);
const v2 = await this.getPromptVersion(id, version2);
return {
contentDiff: diffLines(v1.content, v2.content),
metadataDiff: diffJSON(v1.metadata, v2.metadata),
summary: summarizeChanges(v1, v2)
};
}
```
**3. Remote Sync** (Optional):
```typescript
// Future: Sync user data across machines
interface IRemoteSyncProvider {
push(localDb: Database): Promise<void>;
pull(): Promise<Database>;
resolveConflicts(local: PromptData, remote: PromptData): PromptData;
}
```
**4. Profile Management Features (Phase 6+)**:
When users accumulate 500+ items and need organization:
```typescript
// MCP Tool Interface
interface ProfileManagement {
// Profile CRUD
createProfile(profile: Profile): Promise<void>;
listProfiles(): Promise<Profile[]>;
switchProfile(profileId: string): Promise<void>;
deleteProfile(profileId: string): Promise<void>;
// Cross-profile operations
copyPromptToProfile(promptId: string, targetProfileId: string): Promise<void>;
movePromptToProfile(promptId: string, targetProfileId: string): Promise<void>;
sharePromptBetweenProfiles(promptId: string, targetProfileIds: string[]): Promise<void>;
// Profile analytics
getProfileStats(profileId: string): Promise<ProfileStats>;
}
// Usage examples
>>system_control action="create_profile" profile='{"id":"work","name":"Work","icon":"💼"}'
>>system_control action="switch_profile" profile_id="work"
>>prompt_manager action="list" profile_id="*" // All profiles
>>prompt_manager action="copy_to_profile" id="template" target_profile="work"
```
**Implementation Ready**:
- Schema already supports profiles (Phase 2)
- Add MCP tool commands (2-3 days)
- Add profile switching UI (1-2 days)
- Zero migration needed (foundation built-in)
**5. Framework & Gate Storage**:
- Once dynamic creation is implemented, use same storage pattern
- Same versioning and history capabilities
- Same MCP tool integration patterns
- Same profile support
**6. Analytics & Insights**:
```typescript
interface StorageAnalytics {
mostModifiedPrompts: Array<{ id: string; modifications: number }>;
recentActivity: Array<{ date: string; changes: number }>;
categoryDistribution: { [category: string]: number };
averageVersionsPerPrompt: number;
}
```
---
## Appendix A: Configuration Reference
### Complete Configuration Example
```json
{
"storage": {
"type": "sqlite",
"sqlitePath": "~/.claude-prompts-mcp/storage.db",
"defaultsPath": "./defaults/prompts",
"backup": {
"enabled": true,
"automatic": {
"intervalHours": 24,
"keepCount": 7
},
"beforeRiskyOperations": true
},
"performance": {
"cacheSize": 100,
"prepareStatements": true,
"enableWAL": true
},
"features": {
"fullTextSearch": true,
"autoVersioning": true,
"softDelete": true
}
}
}
```
---
## Appendix B: Migration Checklist
**Pre-Migration**:
- [ ] Backup current JSON files
- [ ] Test SQLite adapter with sample data
- [ ] Document current prompt count and structure
- [ ] Prepare rollback plan
**Migration**:
- [ ] Create SQLite database in user directory
- [ ] Import server defaults
- [ ] Migrate existing user customizations
- [ ] Verify data integrity
- [ ] Test MCP tool functionality
**Post-Migration**:
- [ ] Verify all prompts accessible
- [ ] Test history functionality
- [ ] Validate backup system
- [ ] Update documentation
- [ ] Monitor performance
**Rollback Plan**:
- [ ] Keep JSON files for 30 days
- [ ] Provide export-to-JSON tool
- [ ] Document recovery procedure
- [ ] Test restoration process
---
## Appendix C: Performance Benchmarks
**Target Performance** (SQLite vs JSON):
| Operation | JSON | SQLite | Improvement |
|-----------|------|--------|-------------|
| Get single prompt | 5ms | 2ms | 2.5x faster |
| List 100 prompts | 80ms | 15ms | 5.3x faster |
| Filter by category | 100ms | 10ms | 10x faster |
| Get history | N/A | 25ms | New feature |
| Rollback | N/A | 80ms | New feature |
| Search text | 200ms | 30ms | 6.7x faster |
**Memory Usage**:
- JSON: ~2MB for 100 prompts (full in-memory)
- SQLite: ~500KB baseline + query cache (streaming capable)
---
## Conclusion
This implementation plan provides a comprehensive roadmap for migrating from JSON-based storage to SQLite while maintaining backwards compatibility, enabling versioning features, and separating user data from the server codebase.
**Key Success Factors**:
1. **Gradual migration** - No breaking changes, users opt-in
2. **Storage abstraction** - Clean adapter pattern enables flexibility
3. **User-centric design** - Data lives in user's home directory
4. **Git-optional** - Users don't need to fork, but can export to JSON
5. **Versioning built-in** - Automatic history tracking and rollback
6. **Performance optimized** - SQLite provides significant speed improvements
7. **Profile support** - Built-in from day one, enables work/life/code separation at scale
8. **Future-ready** - Extensible to frameworks and gates
**Timeline**: 5 weeks for full implementation
**Team Size**: 1-2 developers
**Risk Level**: Low (gradual migration, extensive testing, rollback capabilities)
---
**Next Steps**:
1. Review and approve this plan
2. Begin Phase 1: Storage Abstraction Layer
3. Set up testing infrastructure
4. Prepare user communication materials
5. Implement according to phased approach
**Questions for Consideration**:
- Should we support remote sync in the future? (Multi-machine)
- Do we need multi-user support? (Collaboration)
- Should we add prompt templates marketplace? (Import from community)
- Performance targets acceptable? (Can optimize further if needed)
```