This is page 5 of 47. Use http://codebase.md/doobidoo/mcp-memory-service?lines=true&page={x} to view the full context.
# Directory Structure
```
├── .claude
│ ├── agents
│ │ ├── amp-bridge.md
│ │ ├── amp-pr-automator.md
│ │ ├── code-quality-guard.md
│ │ ├── gemini-pr-automator.md
│ │ └── github-release-manager.md
│ ├── settings.local.json.backup
│ └── settings.local.json.local
├── .commit-message
├── .dockerignore
├── .env.example
├── .env.sqlite.backup
├── .envnn#
├── .gitattributes
├── .github
│ ├── FUNDING.yml
│ ├── ISSUE_TEMPLATE
│ │ ├── bug_report.yml
│ │ ├── config.yml
│ │ ├── feature_request.yml
│ │ └── performance_issue.yml
│ ├── pull_request_template.md
│ └── workflows
│ ├── bridge-tests.yml
│ ├── CACHE_FIX.md
│ ├── claude-code-review.yml
│ ├── claude.yml
│ ├── cleanup-images.yml.disabled
│ ├── dev-setup-validation.yml
│ ├── docker-publish.yml
│ ├── LATEST_FIXES.md
│ ├── main-optimized.yml.disabled
│ ├── main.yml
│ ├── publish-and-test.yml
│ ├── README_OPTIMIZATION.md
│ ├── release-tag.yml.disabled
│ ├── release.yml
│ ├── roadmap-review-reminder.yml
│ ├── SECRET_CONDITIONAL_FIX.md
│ └── WORKFLOW_FIXES.md
├── .gitignore
├── .mcp.json.backup
├── .mcp.json.template
├── .pyscn
│ ├── .gitignore
│ └── reports
│ └── analyze_20251123_214224.html
├── AGENTS.md
├── archive
│ ├── deployment
│ │ ├── deploy_fastmcp_fixed.sh
│ │ ├── deploy_http_with_mcp.sh
│ │ └── deploy_mcp_v4.sh
│ ├── deployment-configs
│ │ ├── empty_config.yml
│ │ └── smithery.yaml
│ ├── development
│ │ └── test_fastmcp.py
│ ├── docs-removed-2025-08-23
│ │ ├── authentication.md
│ │ ├── claude_integration.md
│ │ ├── claude-code-compatibility.md
│ │ ├── claude-code-integration.md
│ │ ├── claude-code-quickstart.md
│ │ ├── claude-desktop-setup.md
│ │ ├── complete-setup-guide.md
│ │ ├── database-synchronization.md
│ │ ├── development
│ │ │ ├── autonomous-memory-consolidation.md
│ │ │ ├── CLEANUP_PLAN.md
│ │ │ ├── CLEANUP_README.md
│ │ │ ├── CLEANUP_SUMMARY.md
│ │ │ ├── dream-inspired-memory-consolidation.md
│ │ │ ├── hybrid-slm-memory-consolidation.md
│ │ │ ├── mcp-milestone.md
│ │ │ ├── multi-client-architecture.md
│ │ │ ├── test-results.md
│ │ │ └── TIMESTAMP_FIX_SUMMARY.md
│ │ ├── distributed-sync.md
│ │ ├── invocation_guide.md
│ │ ├── macos-intel.md
│ │ ├── master-guide.md
│ │ ├── mcp-client-configuration.md
│ │ ├── multi-client-server.md
│ │ ├── service-installation.md
│ │ ├── sessions
│ │ │ └── MCP_ENHANCEMENT_SESSION_MEMORY_v4.1.0.md
│ │ ├── UBUNTU_SETUP.md
│ │ ├── ubuntu.md
│ │ ├── windows-setup.md
│ │ └── windows.md
│ ├── docs-root-cleanup-2025-08-23
│ │ ├── AWESOME_LIST_SUBMISSION.md
│ │ ├── CLOUDFLARE_IMPLEMENTATION.md
│ │ ├── DOCUMENTATION_ANALYSIS.md
│ │ ├── DOCUMENTATION_CLEANUP_PLAN.md
│ │ ├── DOCUMENTATION_CONSOLIDATION_COMPLETE.md
│ │ ├── LITESTREAM_SETUP_GUIDE.md
│ │ ├── lm_studio_system_prompt.md
│ │ ├── PYTORCH_DOWNLOAD_FIX.md
│ │ └── README-ORIGINAL-BACKUP.md
│ ├── investigations
│ │ └── MACOS_HOOKS_INVESTIGATION.md
│ ├── litestream-configs-v6.3.0
│ │ ├── install_service.sh
│ │ ├── litestream_master_config_fixed.yml
│ │ ├── litestream_master_config.yml
│ │ ├── litestream_replica_config_fixed.yml
│ │ ├── litestream_replica_config.yml
│ │ ├── litestream_replica_simple.yml
│ │ ├── litestream-http.service
│ │ ├── litestream.service
│ │ └── requirements-cloudflare.txt
│ ├── release-notes
│ │ └── release-notes-v7.1.4.md
│ └── setup-development
│ ├── README.md
│ ├── setup_consolidation_mdns.sh
│ ├── STARTUP_SETUP_GUIDE.md
│ └── test_service.sh
├── CHANGELOG-HISTORIC.md
├── CHANGELOG.md
├── claude_commands
│ ├── memory-context.md
│ ├── memory-health.md
│ ├── memory-ingest-dir.md
│ ├── memory-ingest.md
│ ├── memory-recall.md
│ ├── memory-search.md
│ ├── memory-store.md
│ ├── README.md
│ └── session-start.md
├── claude-hooks
│ ├── config.json
│ ├── config.template.json
│ ├── CONFIGURATION.md
│ ├── core
│ │ ├── memory-retrieval.js
│ │ ├── mid-conversation.js
│ │ ├── session-end.js
│ │ ├── session-start.js
│ │ └── topic-change.js
│ ├── debug-pattern-test.js
│ ├── install_claude_hooks_windows.ps1
│ ├── install_hooks.py
│ ├── memory-mode-controller.js
│ ├── MIGRATION.md
│ ├── README-NATURAL-TRIGGERS.md
│ ├── README-phase2.md
│ ├── README.md
│ ├── simple-test.js
│ ├── statusline.sh
│ ├── test-adaptive-weights.js
│ ├── test-dual-protocol-hook.js
│ ├── test-mcp-hook.js
│ ├── test-natural-triggers.js
│ ├── test-recency-scoring.js
│ ├── tests
│ │ ├── integration-test.js
│ │ ├── phase2-integration-test.js
│ │ ├── test-code-execution.js
│ │ ├── test-cross-session.json
│ │ ├── test-session-tracking.json
│ │ └── test-threading.json
│ ├── utilities
│ │ ├── adaptive-pattern-detector.js
│ │ ├── context-formatter.js
│ │ ├── context-shift-detector.js
│ │ ├── conversation-analyzer.js
│ │ ├── dynamic-context-updater.js
│ │ ├── git-analyzer.js
│ │ ├── mcp-client.js
│ │ ├── memory-client.js
│ │ ├── memory-scorer.js
│ │ ├── performance-manager.js
│ │ ├── project-detector.js
│ │ ├── session-tracker.js
│ │ ├── tiered-conversation-monitor.js
│ │ └── version-checker.js
│ └── WINDOWS-SESSIONSTART-BUG.md
├── CLAUDE.md
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── Development-Sprint-November-2025.md
├── docs
│ ├── amp-cli-bridge.md
│ ├── api
│ │ ├── code-execution-interface.md
│ │ ├── memory-metadata-api.md
│ │ ├── PHASE1_IMPLEMENTATION_SUMMARY.md
│ │ ├── PHASE2_IMPLEMENTATION_SUMMARY.md
│ │ ├── PHASE2_REPORT.md
│ │ └── tag-standardization.md
│ ├── architecture
│ │ ├── search-enhancement-spec.md
│ │ └── search-examples.md
│ ├── architecture.md
│ ├── archive
│ │ └── obsolete-workflows
│ │ ├── load_memory_context.md
│ │ └── README.md
│ ├── assets
│ │ └── images
│ │ ├── dashboard-v3.3.0-preview.png
│ │ ├── memory-awareness-hooks-example.png
│ │ ├── project-infographic.svg
│ │ └── README.md
│ ├── CLAUDE_CODE_QUICK_REFERENCE.md
│ ├── cloudflare-setup.md
│ ├── deployment
│ │ ├── docker.md
│ │ ├── dual-service.md
│ │ ├── production-guide.md
│ │ └── systemd-service.md
│ ├── development
│ │ ├── ai-agent-instructions.md
│ │ ├── code-quality
│ │ │ ├── phase-2a-completion.md
│ │ │ ├── phase-2a-handle-get-prompt.md
│ │ │ ├── phase-2a-index.md
│ │ │ ├── phase-2a-install-package.md
│ │ │ └── phase-2b-session-summary.md
│ │ ├── code-quality-workflow.md
│ │ ├── dashboard-workflow.md
│ │ ├── issue-management.md
│ │ ├── pr-review-guide.md
│ │ ├── refactoring-notes.md
│ │ ├── release-checklist.md
│ │ └── todo-tracker.md
│ ├── docker-optimized-build.md
│ ├── document-ingestion.md
│ ├── DOCUMENTATION_AUDIT.md
│ ├── enhancement-roadmap-issue-14.md
│ ├── examples
│ │ ├── analysis-scripts.js
│ │ ├── maintenance-session-example.md
│ │ ├── memory-distribution-chart.jsx
│ │ └── tag-schema.json
│ ├── first-time-setup.md
│ ├── glama-deployment.md
│ ├── guides
│ │ ├── advanced-command-examples.md
│ │ ├── chromadb-migration.md
│ │ ├── commands-vs-mcp-server.md
│ │ ├── mcp-enhancements.md
│ │ ├── mdns-service-discovery.md
│ │ ├── memory-consolidation-guide.md
│ │ ├── migration.md
│ │ ├── scripts.md
│ │ └── STORAGE_BACKENDS.md
│ ├── HOOK_IMPROVEMENTS.md
│ ├── hooks
│ │ └── phase2-code-execution-migration.md
│ ├── http-server-management.md
│ ├── ide-compatability.md
│ ├── IMAGE_RETENTION_POLICY.md
│ ├── images
│ │ └── dashboard-placeholder.md
│ ├── implementation
│ │ ├── health_checks.md
│ │ └── performance.md
│ ├── IMPLEMENTATION_PLAN_HTTP_SSE.md
│ ├── integration
│ │ ├── homebrew.md
│ │ └── multi-client.md
│ ├── integrations
│ │ ├── gemini.md
│ │ ├── groq-bridge.md
│ │ ├── groq-integration-summary.md
│ │ └── groq-model-comparison.md
│ ├── integrations.md
│ ├── legacy
│ │ └── dual-protocol-hooks.md
│ ├── LM_STUDIO_COMPATIBILITY.md
│ ├── maintenance
│ │ └── memory-maintenance.md
│ ├── mastery
│ │ ├── api-reference.md
│ │ ├── architecture-overview.md
│ │ ├── configuration-guide.md
│ │ ├── local-setup-and-run.md
│ │ ├── testing-guide.md
│ │ └── troubleshooting.md
│ ├── migration
│ │ └── code-execution-api-quick-start.md
│ ├── natural-memory-triggers
│ │ ├── cli-reference.md
│ │ ├── installation-guide.md
│ │ └── performance-optimization.md
│ ├── oauth-setup.md
│ ├── pr-graphql-integration.md
│ ├── quick-setup-cloudflare-dual-environment.md
│ ├── README.md
│ ├── remote-configuration-wiki-section.md
│ ├── research
│ │ ├── code-execution-interface-implementation.md
│ │ └── code-execution-interface-summary.md
│ ├── ROADMAP.md
│ ├── sqlite-vec-backend.md
│ ├── statistics
│ │ ├── charts
│ │ │ ├── activity_patterns.png
│ │ │ ├── contributors.png
│ │ │ ├── growth_trajectory.png
│ │ │ ├── monthly_activity.png
│ │ │ └── october_sprint.png
│ │ ├── data
│ │ │ ├── activity_by_day.csv
│ │ │ ├── activity_by_hour.csv
│ │ │ ├── contributors.csv
│ │ │ └── monthly_activity.csv
│ │ ├── generate_charts.py
│ │ └── REPOSITORY_STATISTICS.md
│ ├── technical
│ │ ├── development.md
│ │ ├── memory-migration.md
│ │ ├── migration-log.md
│ │ ├── sqlite-vec-embedding-fixes.md
│ │ └── tag-storage.md
│ ├── testing
│ │ └── regression-tests.md
│ ├── testing-cloudflare-backend.md
│ ├── troubleshooting
│ │ ├── cloudflare-api-token-setup.md
│ │ ├── cloudflare-authentication.md
│ │ ├── general.md
│ │ ├── hooks-quick-reference.md
│ │ ├── pr162-schema-caching-issue.md
│ │ ├── session-end-hooks.md
│ │ └── sync-issues.md
│ └── tutorials
│ ├── advanced-techniques.md
│ ├── data-analysis.md
│ └── demo-session-walkthrough.md
├── examples
│ ├── claude_desktop_config_template.json
│ ├── claude_desktop_config_windows.json
│ ├── claude-desktop-http-config.json
│ ├── config
│ │ └── claude_desktop_config.json
│ ├── http-mcp-bridge.js
│ ├── memory_export_template.json
│ ├── README.md
│ ├── setup
│ │ └── setup_multi_client_complete.py
│ └── start_https_example.sh
├── install_service.py
├── install.py
├── LICENSE
├── NOTICE
├── pyproject.toml
├── pytest.ini
├── README.md
├── run_server.py
├── scripts
│ ├── .claude
│ │ └── settings.local.json
│ ├── archive
│ │ └── check_missing_timestamps.py
│ ├── backup
│ │ ├── backup_memories.py
│ │ ├── backup_sqlite_vec.sh
│ │ ├── export_distributable_memories.sh
│ │ └── restore_memories.py
│ ├── benchmarks
│ │ ├── benchmark_code_execution_api.py
│ │ ├── benchmark_hybrid_sync.py
│ │ └── benchmark_server_caching.py
│ ├── database
│ │ ├── analyze_sqlite_vec_db.py
│ │ ├── check_sqlite_vec_status.py
│ │ ├── db_health_check.py
│ │ └── simple_timestamp_check.py
│ ├── development
│ │ ├── debug_server_initialization.py
│ │ ├── find_orphaned_files.py
│ │ ├── fix_mdns.sh
│ │ ├── fix_sitecustomize.py
│ │ ├── remote_ingest.sh
│ │ ├── setup-git-merge-drivers.sh
│ │ ├── uv-lock-merge.sh
│ │ └── verify_hybrid_sync.py
│ ├── hooks
│ │ └── pre-commit
│ ├── installation
│ │ ├── install_linux_service.py
│ │ ├── install_macos_service.py
│ │ ├── install_uv.py
│ │ ├── install_windows_service.py
│ │ ├── install.py
│ │ ├── setup_backup_cron.sh
│ │ ├── setup_claude_mcp.sh
│ │ └── setup_cloudflare_resources.py
│ ├── linux
│ │ ├── service_status.sh
│ │ ├── start_service.sh
│ │ ├── stop_service.sh
│ │ ├── uninstall_service.sh
│ │ └── view_logs.sh
│ ├── maintenance
│ │ ├── assign_memory_types.py
│ │ ├── check_memory_types.py
│ │ ├── cleanup_corrupted_encoding.py
│ │ ├── cleanup_memories.py
│ │ ├── cleanup_organize.py
│ │ ├── consolidate_memory_types.py
│ │ ├── consolidation_mappings.json
│ │ ├── delete_orphaned_vectors_fixed.py
│ │ ├── fast_cleanup_duplicates_with_tracking.sh
│ │ ├── find_all_duplicates.py
│ │ ├── find_cloudflare_duplicates.py
│ │ ├── find_duplicates.py
│ │ ├── memory-types.md
│ │ ├── README.md
│ │ ├── recover_timestamps_from_cloudflare.py
│ │ ├── regenerate_embeddings.py
│ │ ├── repair_malformed_tags.py
│ │ ├── repair_memories.py
│ │ ├── repair_sqlite_vec_embeddings.py
│ │ ├── repair_zero_embeddings.py
│ │ ├── restore_from_json_export.py
│ │ └── scan_todos.sh
│ ├── migration
│ │ ├── cleanup_mcp_timestamps.py
│ │ ├── legacy
│ │ │ └── migrate_chroma_to_sqlite.py
│ │ ├── mcp-migration.py
│ │ ├── migrate_sqlite_vec_embeddings.py
│ │ ├── migrate_storage.py
│ │ ├── migrate_tags.py
│ │ ├── migrate_timestamps.py
│ │ ├── migrate_to_cloudflare.py
│ │ ├── migrate_to_sqlite_vec.py
│ │ ├── migrate_v5_enhanced.py
│ │ ├── TIMESTAMP_CLEANUP_README.md
│ │ └── verify_mcp_timestamps.py
│ ├── pr
│ │ ├── amp_collect_results.sh
│ │ ├── amp_detect_breaking_changes.sh
│ │ ├── amp_generate_tests.sh
│ │ ├── amp_pr_review.sh
│ │ ├── amp_quality_gate.sh
│ │ ├── amp_suggest_fixes.sh
│ │ ├── auto_review.sh
│ │ ├── detect_breaking_changes.sh
│ │ ├── generate_tests.sh
│ │ ├── lib
│ │ │ └── graphql_helpers.sh
│ │ ├── quality_gate.sh
│ │ ├── resolve_threads.sh
│ │ ├── run_pyscn_analysis.sh
│ │ ├── run_quality_checks.sh
│ │ ├── thread_status.sh
│ │ └── watch_reviews.sh
│ ├── quality
│ │ ├── fix_dead_code_install.sh
│ │ ├── phase1_dead_code_analysis.md
│ │ ├── phase2_complexity_analysis.md
│ │ ├── README_PHASE1.md
│ │ ├── README_PHASE2.md
│ │ ├── track_pyscn_metrics.sh
│ │ └── weekly_quality_review.sh
│ ├── README.md
│ ├── run
│ │ ├── run_mcp_memory.sh
│ │ ├── run-with-uv.sh
│ │ └── start_sqlite_vec.sh
│ ├── run_memory_server.py
│ ├── server
│ │ ├── check_http_server.py
│ │ ├── check_server_health.py
│ │ ├── memory_offline.py
│ │ ├── preload_models.py
│ │ ├── run_http_server.py
│ │ ├── run_memory_server.py
│ │ ├── start_http_server.bat
│ │ └── start_http_server.sh
│ ├── service
│ │ ├── deploy_dual_services.sh
│ │ ├── install_http_service.sh
│ │ ├── mcp-memory-http.service
│ │ ├── mcp-memory.service
│ │ ├── memory_service_manager.sh
│ │ ├── service_control.sh
│ │ ├── service_utils.py
│ │ └── update_service.sh
│ ├── sync
│ │ ├── check_drift.py
│ │ ├── claude_sync_commands.py
│ │ ├── export_memories.py
│ │ ├── import_memories.py
│ │ ├── litestream
│ │ │ ├── apply_local_changes.sh
│ │ │ ├── enhanced_memory_store.sh
│ │ │ ├── init_staging_db.sh
│ │ │ ├── io.litestream.replication.plist
│ │ │ ├── manual_sync.sh
│ │ │ ├── memory_sync.sh
│ │ │ ├── pull_remote_changes.sh
│ │ │ ├── push_to_remote.sh
│ │ │ ├── README.md
│ │ │ ├── resolve_conflicts.sh
│ │ │ ├── setup_local_litestream.sh
│ │ │ ├── setup_remote_litestream.sh
│ │ │ ├── staging_db_init.sql
│ │ │ ├── stash_local_changes.sh
│ │ │ ├── sync_from_remote_noconfig.sh
│ │ │ └── sync_from_remote.sh
│ │ ├── README.md
│ │ ├── safe_cloudflare_update.sh
│ │ ├── sync_memory_backends.py
│ │ └── sync_now.py
│ ├── testing
│ │ ├── run_complete_test.py
│ │ ├── run_memory_test.sh
│ │ ├── simple_test.py
│ │ ├── test_cleanup_logic.py
│ │ ├── test_cloudflare_backend.py
│ │ ├── test_docker_functionality.py
│ │ ├── test_installation.py
│ │ ├── test_mdns.py
│ │ ├── test_memory_api.py
│ │ ├── test_memory_simple.py
│ │ ├── test_migration.py
│ │ ├── test_search_api.py
│ │ ├── test_sqlite_vec_embeddings.py
│ │ ├── test_sse_events.py
│ │ ├── test-connection.py
│ │ └── test-hook.js
│ ├── utils
│ │ ├── claude_commands_utils.py
│ │ ├── generate_personalized_claude_md.sh
│ │ ├── groq
│ │ ├── groq_agent_bridge.py
│ │ ├── list-collections.py
│ │ ├── memory_wrapper_uv.py
│ │ ├── query_memories.py
│ │ ├── smithery_wrapper.py
│ │ ├── test_groq_bridge.sh
│ │ └── uv_wrapper.py
│ └── validation
│ ├── check_dev_setup.py
│ ├── check_documentation_links.py
│ ├── diagnose_backend_config.py
│ ├── validate_configuration_complete.py
│ ├── validate_memories.py
│ ├── validate_migration.py
│ ├── validate_timestamp_integrity.py
│ ├── verify_environment.py
│ ├── verify_pytorch_windows.py
│ └── verify_torch.py
├── SECURITY.md
├── selective_timestamp_recovery.py
├── SPONSORS.md
├── src
│ └── mcp_memory_service
│ ├── __init__.py
│ ├── api
│ │ ├── __init__.py
│ │ ├── client.py
│ │ ├── operations.py
│ │ ├── sync_wrapper.py
│ │ └── types.py
│ ├── backup
│ │ ├── __init__.py
│ │ └── scheduler.py
│ ├── cli
│ │ ├── __init__.py
│ │ ├── ingestion.py
│ │ ├── main.py
│ │ └── utils.py
│ ├── config.py
│ ├── consolidation
│ │ ├── __init__.py
│ │ ├── associations.py
│ │ ├── base.py
│ │ ├── clustering.py
│ │ ├── compression.py
│ │ ├── consolidator.py
│ │ ├── decay.py
│ │ ├── forgetting.py
│ │ ├── health.py
│ │ └── scheduler.py
│ ├── dependency_check.py
│ ├── discovery
│ │ ├── __init__.py
│ │ ├── client.py
│ │ └── mdns_service.py
│ ├── embeddings
│ │ ├── __init__.py
│ │ └── onnx_embeddings.py
│ ├── ingestion
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── chunker.py
│ │ ├── csv_loader.py
│ │ ├── json_loader.py
│ │ ├── pdf_loader.py
│ │ ├── registry.py
│ │ ├── semtools_loader.py
│ │ └── text_loader.py
│ ├── lm_studio_compat.py
│ ├── mcp_server.py
│ ├── models
│ │ ├── __init__.py
│ │ └── memory.py
│ ├── server.py
│ ├── services
│ │ ├── __init__.py
│ │ └── memory_service.py
│ ├── storage
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── cloudflare.py
│ │ ├── factory.py
│ │ ├── http_client.py
│ │ ├── hybrid.py
│ │ └── sqlite_vec.py
│ ├── sync
│ │ ├── __init__.py
│ │ ├── exporter.py
│ │ ├── importer.py
│ │ └── litestream_config.py
│ ├── utils
│ │ ├── __init__.py
│ │ ├── cache_manager.py
│ │ ├── content_splitter.py
│ │ ├── db_utils.py
│ │ ├── debug.py
│ │ ├── document_processing.py
│ │ ├── gpu_detection.py
│ │ ├── hashing.py
│ │ ├── http_server_manager.py
│ │ ├── port_detection.py
│ │ ├── system_detection.py
│ │ └── time_parser.py
│ └── web
│ ├── __init__.py
│ ├── api
│ │ ├── __init__.py
│ │ ├── analytics.py
│ │ ├── backup.py
│ │ ├── consolidation.py
│ │ ├── documents.py
│ │ ├── events.py
│ │ ├── health.py
│ │ ├── manage.py
│ │ ├── mcp.py
│ │ ├── memories.py
│ │ ├── search.py
│ │ └── sync.py
│ ├── app.py
│ ├── dependencies.py
│ ├── oauth
│ │ ├── __init__.py
│ │ ├── authorization.py
│ │ ├── discovery.py
│ │ ├── middleware.py
│ │ ├── models.py
│ │ ├── registration.py
│ │ └── storage.py
│ ├── sse.py
│ └── static
│ ├── app.js
│ ├── index.html
│ ├── README.md
│ ├── sse_test.html
│ └── style.css
├── start_http_debug.bat
├── start_http_server.sh
├── test_document.txt
├── test_version_checker.js
├── tests
│ ├── __init__.py
│ ├── api
│ │ ├── __init__.py
│ │ ├── test_compact_types.py
│ │ └── test_operations.py
│ ├── bridge
│ │ ├── mock_responses.js
│ │ ├── package-lock.json
│ │ ├── package.json
│ │ └── test_http_mcp_bridge.js
│ ├── conftest.py
│ ├── consolidation
│ │ ├── __init__.py
│ │ ├── conftest.py
│ │ ├── test_associations.py
│ │ ├── test_clustering.py
│ │ ├── test_compression.py
│ │ ├── test_consolidator.py
│ │ ├── test_decay.py
│ │ └── test_forgetting.py
│ ├── contracts
│ │ └── api-specification.yml
│ ├── integration
│ │ ├── package-lock.json
│ │ ├── package.json
│ │ ├── test_api_key_fallback.py
│ │ ├── test_api_memories_chronological.py
│ │ ├── test_api_tag_time_search.py
│ │ ├── test_api_with_memory_service.py
│ │ ├── test_bridge_integration.js
│ │ ├── test_cli_interfaces.py
│ │ ├── test_cloudflare_connection.py
│ │ ├── test_concurrent_clients.py
│ │ ├── test_data_serialization_consistency.py
│ │ ├── test_http_server_startup.py
│ │ ├── test_mcp_memory.py
│ │ ├── test_mdns_integration.py
│ │ ├── test_oauth_basic_auth.py
│ │ ├── test_oauth_flow.py
│ │ ├── test_server_handlers.py
│ │ └── test_store_memory.py
│ ├── performance
│ │ ├── test_background_sync.py
│ │ └── test_hybrid_live.py
│ ├── README.md
│ ├── smithery
│ │ └── test_smithery.py
│ ├── sqlite
│ │ └── simple_sqlite_vec_test.py
│ ├── test_client.py
│ ├── test_content_splitting.py
│ ├── test_database.py
│ ├── test_hybrid_cloudflare_limits.py
│ ├── test_hybrid_storage.py
│ ├── test_memory_ops.py
│ ├── test_semantic_search.py
│ ├── test_sqlite_vec_storage.py
│ ├── test_time_parser.py
│ ├── test_timestamp_preservation.py
│ ├── timestamp
│ │ ├── test_hook_vs_manual_storage.py
│ │ ├── test_issue99_final_validation.py
│ │ ├── test_search_retrieval_inconsistency.py
│ │ ├── test_timestamp_issue.py
│ │ └── test_timestamp_simple.py
│ └── unit
│ ├── conftest.py
│ ├── test_cloudflare_storage.py
│ ├── test_csv_loader.py
│ ├── test_fastapi_dependencies.py
│ ├── test_import.py
│ ├── test_json_loader.py
│ ├── test_mdns_simple.py
│ ├── test_mdns.py
│ ├── test_memory_service.py
│ ├── test_memory.py
│ ├── test_semtools_loader.py
│ ├── test_storage_interface_compatibility.py
│ └── test_tag_time_filtering.py
├── tools
│ ├── docker
│ │ ├── DEPRECATED.md
│ │ ├── docker-compose.http.yml
│ │ ├── docker-compose.pythonpath.yml
│ │ ├── docker-compose.standalone.yml
│ │ ├── docker-compose.uv.yml
│ │ ├── docker-compose.yml
│ │ ├── docker-entrypoint-persistent.sh
│ │ ├── docker-entrypoint-unified.sh
│ │ ├── docker-entrypoint.sh
│ │ ├── Dockerfile
│ │ ├── Dockerfile.glama
│ │ ├── Dockerfile.slim
│ │ ├── README.md
│ │ └── test-docker-modes.sh
│ └── README.md
└── uv.lock
```
# Files
--------------------------------------------------------------------------------
/archive/docs-removed-2025-08-23/windows-setup.md:
--------------------------------------------------------------------------------
```markdown
1 | # Windows Setup Guide for MCP Memory Service
2 |
3 | This guide provides comprehensive instructions for setting up and running the MCP Memory Service on Windows systems, including handling common Windows-specific issues.
4 |
5 | ## Installation
6 |
7 | ### Prerequisites
8 | - Python 3.10 or newer
9 | - Git for Windows
10 | - Visual Studio Build Tools (for PyTorch)
11 |
12 | ### Recommended Installation (Using UV)
13 |
14 | 1. Install UV:
15 | ```bash
16 | pip install uv
17 | ```
18 |
19 | 2. Clone and setup:
20 | ```bash
21 | git clone https://github.com/doobidoo/mcp-memory-service.git
22 | cd mcp-memory-service
23 | uv venv
24 | .venv\Scripts\activate
25 | uv pip install -r requirements.txt
26 | uv pip install -e .
27 | ```
28 |
29 | ### Alternative: Windows-Specific Installation
30 |
31 | If you encounter issues with UV, use our Windows-specific installation script:
32 |
33 | ```bash
34 | python scripts/install_windows.py
35 | ```
36 |
37 | This script handles:
38 | 1. Detecting CUDA availability
39 | 2. Installing the correct PyTorch version
40 | 3. Setting up dependencies without conflicts
41 | 4. Verifying the installation
42 |
43 | ## Configuration
44 |
45 | ### Claude Desktop Configuration
46 |
47 | 1. Create or edit your Claude Desktop configuration file:
48 | - Location: `%APPDATA%\Claude\claude_desktop_config.json`
49 |
50 | 2. Add the following configuration:
51 | ```json
52 | {
53 | "memory": {
54 | "command": "python",
55 | "args": [
56 | "C:\\path\\to\\mcp-memory-service\\memory_wrapper.py"
57 | ],
58 | "env": {
59 | "MCP_MEMORY_CHROMA_PATH": "C:\\Users\\YourUsername\\AppData\\Local\\mcp-memory\\chroma_db",
60 | "MCP_MEMORY_BACKUPS_PATH": "C:\\Users\\YourUsername\\AppData\\Local\\mcp-memory\\backups"
61 | }
62 | }
63 | }
64 | ```
65 |
66 | ### Environment Variables
67 |
68 | Important Windows-specific environment variables:
69 | ```
70 | MCP_MEMORY_USE_DIRECTML=1 # Enable DirectML acceleration if CUDA is not available
71 | PYTORCH_ENABLE_MPS_FALLBACK=0 # Disable MPS (not needed on Windows)
72 | ```
73 |
74 | ## Common Windows-Specific Issues
75 |
76 | ### PyTorch Installation Issues
77 |
78 | If you see errors about PyTorch installation:
79 |
80 | 1. Use the Windows-specific installation script:
81 | ```bash
82 | python scripts/install_windows.py
83 | ```
84 |
85 | 2. Or manually install PyTorch with the correct index URL:
86 | ```bash
87 | pip install torch==2.1.0 torchvision==2.1.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu118
88 | ```
89 |
90 | ### JSON Parsing Errors
91 |
92 | If you see "Unexpected token" errors in Claude Desktop:
93 |
94 | **Symptoms:**
95 | ```
96 | Unexpected token 'U', "Using Chro"... is not valid JSON
97 | Unexpected token 'I', "[INFO] Star"... is not valid JSON
98 | ```
99 |
100 | **Solution:**
101 | - Update to the latest version which includes Windows-specific stream handling fixes
102 | - Use the memory wrapper script which properly handles stdout/stderr separation
103 |
104 | ### Recursion Errors
105 |
106 | If you encounter recursion errors:
107 |
108 | 1. Run the sitecustomize fix script:
109 | ```bash
110 | python scripts/fix_sitecustomize.py
111 | ```
112 |
113 | 2. Restart your Python environment
114 |
115 | ## Debugging Tools
116 |
117 | Windows-specific debugging tools:
118 |
119 | ```bash
120 | # Verify PyTorch installation
121 | python scripts/verify_pytorch_windows.py
122 |
123 | # Check environment compatibility
124 | python scripts/verify_environment_enhanced.py
125 |
126 | # Test the memory server
127 | python scripts/run_memory_server.py
128 | ```
129 |
130 | ## Log Files
131 |
132 | Important log locations on Windows:
133 | - Claude Desktop logs: `%APPDATA%\Claude\logs\mcp-server-memory.log`
134 | - Memory service logs: `%LOCALAPPDATA%\mcp-memory\logs\memory_service.log`
135 |
136 | ## Performance Optimization
137 |
138 | ### GPU Acceleration
139 |
140 | 1. CUDA (recommended if available):
141 | - Ensure NVIDIA drivers are up to date
142 | - CUDA toolkit is not required (bundled with PyTorch)
143 |
144 | 2. DirectML (alternative):
145 | - Enable with `MCP_MEMORY_USE_DIRECTML=1`
146 | - Useful for AMD GPUs or when CUDA is not available
147 |
148 | ### Memory Usage
149 |
150 | If experiencing memory issues:
151 | 1. Reduce batch size:
152 | ```bash
153 | set MCP_MEMORY_BATCH_SIZE=4
154 | ```
155 |
156 | 2. Use a smaller model:
157 | ```bash
158 | set MCP_MEMORY_MODEL_NAME=paraphrase-MiniLM-L3-v2
159 | ```
160 |
161 | ## Getting Help
162 |
163 | If you encounter Windows-specific issues:
164 | 1. Check the logs in `%APPDATA%\Claude\logs\`
165 | 2. Run verification tools mentioned above
166 | 3. Contact support via Telegram: t.me/doobeedoo
```
--------------------------------------------------------------------------------
/archive/docs-removed-2025-08-23/claude-code-quickstart.md:
--------------------------------------------------------------------------------
```markdown
1 | # Claude Code Commands - Quick Start Guide
2 |
3 | Get up and running with MCP Memory Service Claude Code commands in just 2 minutes!
4 |
5 | ## Prerequisites
6 |
7 | ✅ [Claude Code CLI](https://claude.ai/code) installed and working
8 | ✅ Python 3.10+ with pip
9 | ✅ 5 minutes of your time
10 |
11 | ## Step 1: Install MCP Memory Service with Commands
12 |
13 | ```bash
14 | # Clone and install with Claude Code commands
15 | git clone https://github.com/doobidoo/mcp-memory-service.git
16 | cd mcp-memory-service
17 | python install.py --install-claude-commands
18 | ```
19 |
20 | The installer will:
21 | - ✅ Detect your Claude Code CLI automatically
22 | - ✅ Install the memory service with optimal settings for your system
23 | - ✅ Install 5 conversational memory commands
24 | - ✅ Test everything to ensure it works
25 |
26 | ## Step 2: Test Your Installation
27 |
28 | ```bash
29 | # Check if everything is working
30 | claude /memory-health
31 | ```
32 |
33 | You should see a comprehensive health check interface. If you see the command description and interface, you're all set! 🎉
34 |
35 | ## Step 3: Store Your First Memory
36 |
37 | ```bash
38 | # Store something important
39 | claude /memory-store "I successfully set up MCP Memory Service with Claude Code commands on $(date)"
40 | ```
41 |
42 | ## Step 4: Try the Core Commands
43 |
44 | ```bash
45 | # Recall memories by time
46 | claude /memory-recall "what did I store today?"
47 |
48 | # Search by content
49 | claude /memory-search "MCP Memory Service"
50 |
51 | # Capture current session context
52 | claude /memory-context --summary "Initial setup and testing"
53 | ```
54 |
55 | ## 🎯 You're Done!
56 |
57 | That's it! You now have powerful memory capabilities integrated directly into Claude Code.
58 |
59 | ## Available Commands
60 |
61 | | Command | Purpose | Example |
62 | |---------|---------|---------|
63 | | `claude /memory-store` | Store information with context | `claude /memory-store "Important decision about architecture"` |
64 | | `claude /memory-recall` | Retrieve by time expressions | `claude /memory-recall "what did we decide last week?"` |
65 | | `claude /memory-search` | Search by tags or content | `claude /memory-search --tags "architecture,database"` |
66 | | `claude /memory-context` | Capture session context | `claude /memory-context --summary "Planning session"` |
67 | | `claude /memory-health` | Check service status | `claude /memory-health --detailed` |
68 |
69 | ## Next Steps
70 |
71 | ### Explore Advanced Features
72 | - **Context-aware operations**: Commands automatically detect your current project
73 | - **Smart tagging**: Automatic tag generation based on your work
74 | - **Time-based queries**: Natural language like "yesterday", "last week", "two months ago"
75 | - **Semantic search**: Find related information even with different wording
76 |
77 | ### Learn More
78 | - 📖 [**Full Integration Guide**](claude-code-integration.md) - Complete documentation
79 | - 🔧 [**Installation Master Guide**](../installation/master-guide.md) - Advanced installation options
80 | - ❓ [**Troubleshooting**](../troubleshooting/general.md) - Solutions to common issues
81 |
82 | ## Troubleshooting Quick Fixes
83 |
84 | ### Commands Not Working?
85 | ```bash
86 | # Check if Claude Code CLI is working
87 | claude --version
88 |
89 | # Check if commands are installed
90 | ls ~/.claude/commands/memory-*.md
91 |
92 | # Reinstall commands
93 | python scripts/claude_commands_utils.py
94 | ```
95 |
96 | ### Memory Service Not Connecting?
97 | ```bash
98 | # Check if service is running
99 | memory --help
100 |
101 | # Check service health
102 | claude /memory-health
103 |
104 | # Start the service if needed
105 | memory
106 | ```
107 |
108 | ### Need Help?
109 | - 💬 [GitHub Issues](https://github.com/doobidoo/mcp-memory-service/issues)
110 | - 📚 [Full Documentation](../README.md)
111 | - 🔍 [Search Existing Solutions](https://github.com/doobidoo/mcp-memory-service/issues?q=is%3Aissue)
112 |
113 | ---
114 |
115 | ## What Makes This Special?
116 |
117 | 🚀 **Zero Configuration**: No MCP server setup required
118 | 🧠 **Context Intelligence**: Understands your current project and session
119 | 💬 **Conversational Interface**: Natural, CCPlugins-compatible commands
120 | ⚡ **Instant Access**: Direct command-line memory operations
121 | 🛠️ **Professional Grade**: Enterprise-level capabilities through simple commands
122 |
123 | **Enjoy your enhanced Claude Code experience with persistent memory!** 🎉
```
--------------------------------------------------------------------------------
/tests/integration/test_mcp_memory.py:
--------------------------------------------------------------------------------
```python
1 | #\!/usr/bin/env python3
2 | """
3 | Test script for MCP Memory Service with Homebrew PyTorch.
4 | """
5 | import os
6 | import sys
7 | import asyncio
8 | import time
9 | from datetime import datetime
10 |
11 | # Configure environment variables
12 | os.environ["MCP_MEMORY_STORAGE_BACKEND"] = "sqlite_vec"
13 | os.environ["MCP_MEMORY_SQLITE_PATH"] = os.path.expanduser("~/Library/Application Support/mcp-memory/sqlite_vec.db")
14 | os.environ["MCP_MEMORY_BACKUPS_PATH"] = os.path.expanduser("~/Library/Application Support/mcp-memory/backups")
15 | os.environ["MCP_MEMORY_USE_ONNX"] = "1"
16 |
17 | # Import the MCP Memory Service modules
18 | try:
19 | from mcp_memory_service.storage.sqlite_vec import SqliteVecMemoryStorage
20 | from mcp_memory_service.models.memory import Memory
21 | from mcp_memory_service.utils.hashing import generate_content_hash
22 | except ImportError as e:
23 | print(f"Error importing MCP Memory Service modules: {e}")
24 | sys.exit(1)
25 |
26 | async def main():
27 | print("=== MCP Memory Service Test ===")
28 |
29 | # Initialize the storage
30 | db_path = os.environ["MCP_MEMORY_SQLITE_PATH"]
31 | print(f"Using SQLite-vec database at: {db_path}")
32 |
33 | storage = SqliteVecMemoryStorage(db_path)
34 | await storage.initialize()
35 |
36 | # Check database health
37 | print("\n=== Database Health Check ===")
38 | if storage.conn is None:
39 | print("Database connection is not initialized")
40 | else:
41 | try:
42 | cursor = storage.conn.execute('SELECT COUNT(*) FROM memories')
43 | memory_count = cursor.fetchone()[0]
44 | print(f"Database connected successfully. Contains {memory_count} memories.")
45 |
46 | cursor = storage.conn.execute("SELECT name FROM sqlite_master WHERE type='table'")
47 | tables = [row[0] for row in cursor.fetchall()]
48 | print(f"Database tables: {', '.join(tables)}")
49 |
50 | print(f"Embedding model availability: {storage.embedding_model is not None}")
51 | if not storage.embedding_model:
52 | print("No embedding model available. Limited functionality.")
53 |
54 | except Exception as e:
55 | print(f"Database error: {str(e)}")
56 |
57 | # Get database stats
58 | print("\n=== Database Stats ===")
59 | stats = storage.get_stats()
60 | import json
61 | print(json.dumps(stats, indent=2))
62 |
63 | # Store a test memory
64 | print("\n=== Creating Test Memory ===")
65 | test_content = f"MCP Test memory created at {datetime.now().isoformat()} with Homebrew PyTorch"
66 |
67 | test_memory = Memory(
68 | content=test_content,
69 | content_hash=generate_content_hash(test_content),
70 | tags=["mcp-test", "homebrew-pytorch"],
71 | memory_type="note",
72 | metadata={"source": "mcp_test_script"}
73 | )
74 | print(f"Memory content: {test_memory.content}")
75 | print(f"Content hash: {test_memory.content_hash}")
76 |
77 | success, message = await storage.store(test_memory)
78 | print(f"Store success: {success}")
79 | print(f"Message: {message}")
80 |
81 | # Try to retrieve the memory
82 | print("\n=== Retrieving by Tag ===")
83 | memories = await storage.search_by_tag(["mcp-test"])
84 |
85 | if memories:
86 | print(f"Found {len(memories)} memories with tag 'mcp-test'")
87 | for i, memory in enumerate(memories):
88 | print(f" Memory {i+1}: {memory.content[:60]}...")
89 | else:
90 | print("No memories found with tag 'mcp-test'")
91 |
92 | # Try semantic search
93 | print("\n=== Semantic Search ===")
94 | results = await storage.retrieve("test memory homebrew pytorch", n_results=5)
95 |
96 | if results:
97 | print(f"Found {len(results)} memories via semantic search")
98 | for i, result in enumerate(results):
99 | print(f" Result {i+1}:")
100 | print(f" Content: {result.memory.content[:60]}...")
101 | print(f" Score: {result.relevance_score}")
102 | else:
103 | print("No memories found via semantic search")
104 |
105 | print("\n=== Test Complete ===")
106 | storage.close()
107 |
108 | if __name__ == "__main__":
109 | asyncio.run(main())
110 |
```
--------------------------------------------------------------------------------
/scripts/pr/amp_generate_tests.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # scripts/pr/amp_generate_tests.sh - Generate pytest tests using Amp CLI
3 | #
4 | # Usage: bash scripts/pr/amp_generate_tests.sh <PR_NUMBER>
5 | # Example: bash scripts/pr/amp_generate_tests.sh 215
6 |
7 | set -e
8 |
9 | PR_NUMBER=$1
10 |
11 | if [ -z "$PR_NUMBER" ]; then
12 | echo "Usage: $0 <PR_NUMBER>"
13 | exit 1
14 | fi
15 |
16 | if ! command -v gh &> /dev/null; then
17 | echo "Error: GitHub CLI (gh) is not installed"
18 | exit 1
19 | fi
20 |
21 | echo "=== Amp CLI Test Generation for PR #$PR_NUMBER ==="
22 | echo ""
23 |
24 | # Ensure Amp directories exist
25 | mkdir -p .claude/amp/prompts/pending
26 | mkdir -p .claude/amp/responses/ready
27 | mkdir -p /tmp/amp_tests
28 |
29 | # Get changed Python files (excluding tests)
30 | echo "Fetching changed files from PR #$PR_NUMBER..."
31 | changed_files=$(gh pr diff $PR_NUMBER --name-only | grep '\.py$' | grep -v '^tests/' || echo "")
32 |
33 | if [ -z "$changed_files" ]; then
34 | echo "No Python files changed (excluding tests)."
35 | exit 0
36 | fi
37 |
38 | echo "Changed Python files (non-test):"
39 | echo "$changed_files"
40 | echo ""
41 |
42 | # Track UUIDs for all test generation tasks
43 | test_uuids=()
44 |
45 | for file in $changed_files; do
46 | if [ ! -f "$file" ]; then
47 | echo "Skipping $file (not found in working directory)"
48 | continue
49 | fi
50 |
51 | echo "Creating test generation prompt for: $file"
52 |
53 | # Generate UUID for this file's test generation
54 | test_uuid=$(uuidgen 2>/dev/null || cat /proc/sys/kernel/random/uuid)
55 | test_uuids+=("$test_uuid")
56 |
57 | # Determine if test file already exists
58 | base_name=$(basename "$file" .py)
59 | test_file="tests/test_${base_name}.py"
60 |
61 | if [ -f "$test_file" ]; then
62 | existing_tests=$(cat "$test_file")
63 | prompt_mode="append"
64 | prompt_text="Existing test file exists. Analyze the existing tests and new/changed code to suggest ADDITIONAL pytest test cases. Only output new test functions to append to the existing file.\n\nExisting tests:\n${existing_tests}\n\nNew/changed code:\n$(cat "$file")\n\nProvide only new test functions (complete pytest syntax) that cover new functionality not already tested."
65 | else
66 | prompt_mode="create"
67 | prompt_text="Generate comprehensive pytest tests for this Python module. Include: 1) Happy path tests, 2) Edge cases, 3) Error handling, 4) Async test cases if applicable. Output complete pytest test file.\n\nModule code:\n$(cat "$file")\n\nProvide complete test file content with imports, fixtures, and test functions."
68 | fi
69 |
70 | # Create test generation prompt
71 | cat > .claude/amp/prompts/pending/tests-${test_uuid}.json << EOF
72 | {
73 | "id": "${test_uuid}",
74 | "timestamp": "$(date -u +"%Y-%m-%dT%H:%M:%S.000Z")",
75 | "prompt": "${prompt_text}",
76 | "context": {
77 | "project": "mcp-memory-service",
78 | "task": "test-generation",
79 | "pr_number": "${PR_NUMBER}",
80 | "source_file": "${file}",
81 | "test_file": "${test_file}",
82 | "mode": "${prompt_mode}"
83 | },
84 | "options": {
85 | "timeout": 180000,
86 | "format": "python"
87 | }
88 | }
89 | EOF
90 |
91 | echo " ✅ Created prompt for ${file} (${prompt_mode} mode)"
92 | done
93 |
94 | echo ""
95 | echo "=== Created ${#test_uuids[@]} test generation prompts ==="
96 | echo ""
97 |
98 | # Show Amp commands to run
99 | echo "=== Run these Amp commands (can run in parallel) ==="
100 | for uuid in "${test_uuids[@]}"; do
101 | echo "amp @.claude/amp/prompts/pending/tests-${uuid}.json &"
102 | done
103 | echo ""
104 |
105 | echo "=== Or use this one-liner to run all in background ==="
106 | parallel_cmd=""
107 | for uuid in "${test_uuids[@]}"; do
108 | parallel_cmd+="(amp @.claude/amp/prompts/pending/tests-${uuid}.json > /tmp/amp-test-${uuid}.log 2>&1 &); "
109 | done
110 | parallel_cmd+="sleep 10 && bash scripts/pr/amp_collect_results.sh --timeout 300 --uuids '$(IFS=,; echo "${test_uuids[*]}")'"
111 | echo "$parallel_cmd"
112 | echo ""
113 |
114 | # Save UUIDs for later collection
115 | echo "$(IFS=,; echo "${test_uuids[*]}")" > /tmp/amp_test_generation_uuids_${PR_NUMBER}.txt
116 | echo "UUIDs saved to /tmp/amp_test_generation_uuids_${PR_NUMBER}.txt"
117 | echo ""
118 |
119 | echo "After Amp completes, tests will be in .claude/amp/responses/consumed/"
120 | echo "Extract test content and review before committing to tests/ directory"
121 |
```
--------------------------------------------------------------------------------
/scripts/run/run_mcp_memory.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # Run MCP Memory Service with Homebrew PyTorch Integration for use with MCP
3 |
4 | # Set paths
5 | SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
6 | DB_DIR="$HOME/Library/Application Support/mcp-memory"
7 |
8 | # Use environment variables if set, otherwise use defaults
9 | DB_PATH="${MCP_MEMORY_SQLITE_PATH:-$DB_DIR/sqlite_vec.db}"
10 | BACKUPS_PATH="${MCP_MEMORY_BACKUPS_PATH:-$DB_DIR/backups}"
11 |
12 | # Extract directory parts
13 | DB_DIR="$(dirname "$DB_PATH")"
14 | BACKUPS_DIR="$(dirname "$BACKUPS_PATH")"
15 |
16 | # Create directories if they don't exist
17 | mkdir -p "$DB_DIR"
18 | mkdir -p "$BACKUPS_DIR"
19 |
20 | # Set environment variables (only if not already set)
21 | export MCP_MEMORY_STORAGE_BACKEND="${MCP_MEMORY_STORAGE_BACKEND:-sqlite_vec}"
22 | export MCP_MEMORY_SQLITE_PATH="$DB_PATH"
23 | export MCP_MEMORY_BACKUPS_PATH="$BACKUPS_PATH"
24 | export MCP_MEMORY_USE_ONNX="${MCP_MEMORY_USE_ONNX:-1}"
25 | export MCP_MEMORY_USE_HOMEBREW_PYTORCH="${MCP_MEMORY_USE_HOMEBREW_PYTORCH:-1}"
26 |
27 | # Check if we're running in Claude Desktop (indicated by a special env var we'll set)
28 | if [ "${CLAUDE_DESKTOP_ENV:-}" = "1" ]; then
29 | echo "🖥️ Running in Claude Desktop environment, skipping Homebrew PyTorch check" >&2
30 | SKIP_HOMEBREW_CHECK=1
31 | else
32 | SKIP_HOMEBREW_CHECK=0
33 | fi
34 |
35 | # Check if Homebrew PyTorch is installed, unless skipped
36 | if [ "$SKIP_HOMEBREW_CHECK" = "0" ]; then
37 | if ! brew list | grep -q pytorch; then
38 | echo "❌ ERROR: PyTorch is not installed via Homebrew." >&2
39 | echo "Please install PyTorch first: brew install pytorch" >&2
40 | exit 1
41 | else
42 | echo "✅ Homebrew PyTorch found" >&2
43 | fi
44 | fi
45 |
46 | # Skip Homebrew-related checks if running in Claude Desktop
47 | if [ "$SKIP_HOMEBREW_CHECK" = "0" ]; then
48 | # Check if sentence-transformers is installed in Homebrew Python
49 | HOMEBREW_PYTHON="$(brew --prefix pytorch)/libexec/bin/python3"
50 | echo "Checking for sentence-transformers in $HOMEBREW_PYTHON..." >&2
51 |
52 | # Use proper Python syntax with newlines for the import check
53 | if ! $HOMEBREW_PYTHON -c "
54 | try:
55 | import sentence_transformers
56 | print('Success: sentence-transformers is installed')
57 | except ImportError as e:
58 | print(f'Error: {e}')
59 | exit(1)
60 | " 2>&1 | grep -q "Success"; then
61 | echo "⚠️ WARNING: sentence-transformers is not installed in Homebrew Python." >&2
62 | echo "Installing sentence-transformers in Homebrew Python..." >&2
63 | $HOMEBREW_PYTHON -m pip install sentence-transformers >&2
64 | else
65 | echo "✅ sentence-transformers is already installed in Homebrew Python" >&2
66 | fi
67 | else
68 | echo "🖥️ Skipping sentence-transformers check in Claude Desktop environment" >&2
69 | # Set a default Python path for reference in the log
70 | HOMEBREW_PYTHON="/usr/bin/python3"
71 | fi
72 |
73 | # Activate virtual environment if it exists
74 | if [ -d "$SCRIPT_DIR/venv" ]; then
75 | source "$SCRIPT_DIR/venv/bin/activate"
76 | echo "✅ Activated virtual environment" >&2
77 | else
78 | echo "⚠️ No virtual environment found at $SCRIPT_DIR/venv" >&2
79 | echo " Running with system Python" >&2
80 | fi
81 |
82 | # Redirect all informational output to stderr to avoid JSON parsing errors
83 | echo "========================================================" >&2
84 | echo " MCP Memory Service with Homebrew PyTorch Integration" >&2
85 | echo "========================================================" >&2
86 | echo "Storage backend: $MCP_MEMORY_STORAGE_BACKEND" >&2
87 | echo "SQLite-vec database: $MCP_MEMORY_SQLITE_PATH" >&2
88 | echo "Backups path: $MCP_MEMORY_BACKUPS_PATH" >&2
89 | echo "Homebrew Python: $HOMEBREW_PYTHON" >&2
90 | echo "ONNX Runtime enabled: ${MCP_MEMORY_USE_ONNX:-No}" >&2
91 | echo "Homebrew PyTorch enabled: ${MCP_MEMORY_USE_HOMEBREW_PYTORCH:-No}" >&2
92 | echo "========================================================" >&2
93 |
94 | # Ensure our source code is in the PYTHONPATH
95 | export PYTHONPATH="$SCRIPT_DIR:$SCRIPT_DIR/src:$PYTHONPATH"
96 | echo "PYTHONPATH: $PYTHONPATH" >&2
97 |
98 | # Start the memory server with Homebrew PyTorch integration
99 | echo "Starting MCP Memory Service..." >&2
100 | python -m mcp_memory_service.homebrew_server "$@"
```
--------------------------------------------------------------------------------
/scripts/benchmarks/benchmark_code_execution_api.py:
--------------------------------------------------------------------------------
```python
1 | #!/usr/bin/env python
2 | """
3 | Benchmark script for Code Execution Interface API.
4 |
5 | Measures token efficiency and performance of the new code execution API
6 | compared to traditional MCP tool calls.
7 |
8 | Usage:
9 | python scripts/benchmarks/benchmark_code_execution_api.py
10 | """
11 |
12 | import time
13 | import sys
14 | from pathlib import Path
15 |
16 | # Add parent directory to path
17 | sys.path.insert(0, str(Path(__file__).parent.parent.parent / "src"))
18 |
19 | from mcp_memory_service.api import search, store, health
20 |
21 |
22 | def estimate_tokens(text: str) -> int:
23 | """Rough token estimate: 1 token ≈ 4 characters."""
24 | return len(text) // 4
25 |
26 |
27 | def benchmark_search():
28 | """Benchmark search operation."""
29 | print("\n=== Search Operation Benchmark ===")
30 |
31 | # Store some test data
32 | for i in range(10):
33 | store(f"Test memory {i} for benchmarking", tags=["benchmark", "test"])
34 |
35 | # Warm up
36 | search("benchmark", limit=1)
37 |
38 | # Benchmark cold call
39 | start = time.perf_counter()
40 | results = search("benchmark test", limit=5)
41 | cold_ms = (time.perf_counter() - start) * 1000
42 |
43 | # Benchmark warm calls
44 | warm_times = []
45 | for _ in range(10):
46 | start = time.perf_counter()
47 | results = search("benchmark test", limit=5)
48 | warm_times.append((time.perf_counter() - start) * 1000)
49 |
50 | avg_warm_ms = sum(warm_times) / len(warm_times)
51 |
52 | # Estimate tokens
53 | result_str = str(results.memories)
54 | tokens = estimate_tokens(result_str)
55 |
56 | print(f"Results: {results.total} memories found")
57 | print(f"Cold call: {cold_ms:.1f}ms")
58 | print(f"Warm call (avg): {avg_warm_ms:.1f}ms")
59 | print(f"Token estimate: {tokens} tokens")
60 | print(f"MCP comparison: ~2,625 tokens (85% reduction)")
61 |
62 |
63 | def benchmark_store():
64 | """Benchmark store operation."""
65 | print("\n=== Store Operation Benchmark ===")
66 |
67 | # Warm up
68 | store("Warmup memory", tags=["warmup"])
69 |
70 | # Benchmark warm calls
71 | warm_times = []
72 | for i in range(10):
73 | start = time.perf_counter()
74 | hash_val = store(f"Benchmark memory {i}", tags=["benchmark"])
75 | warm_times.append((time.perf_counter() - start) * 1000)
76 |
77 | avg_warm_ms = sum(warm_times) / len(warm_times)
78 |
79 | # Estimate tokens
80 | param_str = "store('content', tags=['tag1', 'tag2'])"
81 | tokens = estimate_tokens(param_str)
82 |
83 | print(f"Warm call (avg): {avg_warm_ms:.1f}ms")
84 | print(f"Token estimate: {tokens} tokens")
85 | print(f"MCP comparison: ~150 tokens (90% reduction)")
86 |
87 |
88 | def benchmark_health():
89 | """Benchmark health operation."""
90 | print("\n=== Health Operation Benchmark ===")
91 |
92 | # Benchmark warm calls
93 | warm_times = []
94 | for _ in range(10):
95 | start = time.perf_counter()
96 | info = health()
97 | warm_times.append((time.perf_counter() - start) * 1000)
98 |
99 | avg_warm_ms = sum(warm_times) / len(warm_times)
100 |
101 | # Estimate tokens
102 | info = health()
103 | info_str = str(info)
104 | tokens = estimate_tokens(info_str)
105 |
106 | print(f"Status: {info.status}")
107 | print(f"Backend: {info.backend}")
108 | print(f"Count: {info.count}")
109 | print(f"Warm call (avg): {avg_warm_ms:.1f}ms")
110 | print(f"Token estimate: {tokens} tokens")
111 | print(f"MCP comparison: ~125 tokens (84% reduction)")
112 |
113 |
114 | def main():
115 | """Run all benchmarks."""
116 | print("=" * 60)
117 | print("Code Execution Interface API Benchmarks")
118 | print("=" * 60)
119 |
120 | try:
121 | benchmark_search()
122 | benchmark_store()
123 | benchmark_health()
124 |
125 | print("\n" + "=" * 60)
126 | print("Summary")
127 | print("=" * 60)
128 | print("✅ All benchmarks completed successfully")
129 | print("\nKey Findings:")
130 | print("- Search: 85%+ token reduction vs MCP tools")
131 | print("- Store: 90%+ token reduction vs MCP tools")
132 | print("- Health: 84%+ token reduction vs MCP tools")
133 | print("- Performance: <50ms cold, <10ms warm calls")
134 |
135 | except Exception as e:
136 | print(f"\n❌ Benchmark failed: {e}")
137 | import traceback
138 | traceback.print_exc()
139 | sys.exit(1)
140 |
141 |
142 | if __name__ == "__main__":
143 | main()
144 |
```
--------------------------------------------------------------------------------
/.github/workflows/SECRET_CONDITIONAL_FIX.md:
--------------------------------------------------------------------------------
```markdown
1 | # GitHub Actions Secret Conditional Logic Fix
2 |
3 | ## Critical Issue Resolved
4 | **Date**: 2024-08-24
5 | **Problem**: Workflows failing due to incorrect secret checking syntax in conditionals
6 |
7 | ### Root Cause
8 | GitHub Actions does not support checking if secrets are empty using `!= ''` or `== ''` in conditional expressions.
9 |
10 | ### Incorrect Syntax (BROKEN)
11 | ```yaml
12 | # ❌ This syntax doesn't work in GitHub Actions
13 | if: matrix.registry == 'docker.io' && secrets.DOCKER_USERNAME != '' && secrets.DOCKER_PASSWORD != ''
14 |
15 | # ❌ This also doesn't work
16 | if: matrix.registry == 'docker.io' && (secrets.DOCKER_USERNAME == '' || secrets.DOCKER_PASSWORD == '')
17 | ```
18 |
19 | ### Correct Syntax (FIXED)
20 | ```yaml
21 | # ✅ Check if secrets exist (truthy check)
22 | if: matrix.registry == 'docker.io' && secrets.DOCKER_USERNAME && secrets.DOCKER_PASSWORD
23 |
24 | # ✅ Check if secrets don't exist (falsy check)
25 | if: matrix.registry == 'docker.io' && (!secrets.DOCKER_USERNAME || !secrets.DOCKER_PASSWORD)
26 | ```
27 |
28 | ## Changes Applied
29 |
30 | ### 1. main-optimized.yml - Line 286
31 | **Before:**
32 | ```yaml
33 | - name: Log in to Docker Hub
34 | if: matrix.registry == 'docker.io' && secrets.DOCKER_USERNAME != '' && secrets.DOCKER_PASSWORD != ''
35 | ```
36 |
37 | **After:**
38 | ```yaml
39 | - name: Log in to Docker Hub
40 | if: matrix.registry == 'docker.io' && secrets.DOCKER_USERNAME && secrets.DOCKER_PASSWORD
41 | ```
42 |
43 | ### 2. main-optimized.yml - Line 313
44 | **Before:**
45 | ```yaml
46 | - name: Build and push Docker image
47 | if: matrix.registry == 'ghcr.io' || (matrix.registry == 'docker.io' && secrets.DOCKER_USERNAME != '' && secrets.DOCKER_PASSWORD != '')
48 | ```
49 |
50 | **After:**
51 | ```yaml
52 | - name: Build and push Docker image
53 | if: matrix.registry == 'ghcr.io' || (matrix.registry == 'docker.io' && secrets.DOCKER_USERNAME && secrets.DOCKER_PASSWORD)
54 | ```
55 |
56 | ### 3. main-optimized.yml - Line 332
57 | **Before:**
58 | ```yaml
59 | - name: Docker Hub push skipped
60 | if: matrix.registry == 'docker.io' && (secrets.DOCKER_USERNAME == '' || secrets.DOCKER_PASSWORD == '')
61 | ```
62 |
63 | **After:**
64 | ```yaml
65 | - name: Docker Hub push skipped
66 | if: matrix.registry == 'docker.io' && (!secrets.DOCKER_USERNAME || !secrets.DOCKER_PASSWORD)
67 | ```
68 |
69 | ## How GitHub Actions Handles Secrets in Conditionals
70 |
71 | ### Secret Behavior
72 | - **Exists**: `secrets.SECRET_NAME` evaluates to truthy
73 | - **Missing/Empty**: `secrets.SECRET_NAME` evaluates to falsy
74 | - **Cannot compare**: Direct string comparison with `!= ''` fails
75 |
76 | ### Recommended Patterns
77 | ```yaml
78 | # Check if secret exists
79 | if: secrets.MY_SECRET
80 |
81 | # Check if secret doesn't exist
82 | if: !secrets.MY_SECRET
83 |
84 | # Check multiple secrets exist
85 | if: secrets.SECRET1 && secrets.SECRET2
86 |
87 | # Check if any secret is missing
88 | if: !secrets.SECRET1 || !secrets.SECRET2
89 |
90 | # Combine with other conditions
91 | if: github.event_name == 'push' && secrets.MY_SECRET
92 | ```
93 |
94 | ## Impact
95 |
96 | ### Before Fix
97 | - ✗ Workflows failed immediately at conditional evaluation
98 | - ✗ Error: Invalid conditional syntax
99 | - ✗ No Docker Hub operations could run
100 |
101 | ### After Fix
102 | - ✅ Conditionals evaluate correctly
103 | - ✅ Docker Hub steps run when credentials exist
104 | - ✅ GHCR steps always run (no credentials needed)
105 | - ✅ Skip messages show when credentials missing
106 |
107 | ## Alternative Approaches
108 |
109 | ### Option 1: Environment Variable Check
110 | ```yaml
111 | env:
112 | HAS_DOCKER_CREDS: ${{ secrets.DOCKER_USERNAME != null && secrets.DOCKER_PASSWORD != null }}
113 | steps:
114 | - name: Login
115 | if: env.HAS_DOCKER_CREDS == 'true'
116 | ```
117 |
118 | ### Option 2: Continue on Error
119 | ```yaml
120 | - name: Log in to Docker Hub
121 | continue-on-error: true
122 | uses: docker/login-action@v3
123 | ```
124 |
125 | ### Option 3: Job-Level Conditional
126 | ```yaml
127 | jobs:
128 | docker-hub-publish:
129 | if: secrets.DOCKER_USERNAME && secrets.DOCKER_PASSWORD
130 | ```
131 |
132 | ## Testing
133 |
134 | All changes validated:
135 | - ✅ YAML syntax check passed
136 | - ✅ Conditional logic follows GitHub Actions standards
137 | - ✅ Both positive and negative conditionals fixed
138 |
139 | ## References
140 |
141 | - [GitHub Actions: Expressions](https://docs.github.com/en/actions/learn-github-actions/expressions)
142 | - [GitHub Actions: Contexts](https://docs.github.com/en/actions/learn-github-actions/contexts#secrets-context)
143 |
144 | Date: 2024-08-24
145 | Status: Fixed and ready for deployment
```
--------------------------------------------------------------------------------
/docs/technical/sqlite-vec-embedding-fixes.md:
--------------------------------------------------------------------------------
```markdown
1 | # SQLite-vec Embedding Fixes
2 |
3 | This document summarizes the fixes applied to resolve issue #64 where semantic search returns 0 results in the SQLite-vec backend.
4 |
5 | ## Root Causes Identified
6 |
7 | 1. **Missing Core Dependencies**: `sentence-transformers` and `torch` were in optional dependencies, causing silent failures
8 | 2. **Dimension Mismatch**: Vector table was created with hardcoded dimensions before model initialization
9 | 3. **Silent Failures**: Missing dependencies returned zero vectors without raising exceptions
10 | 4. **Database Integrity Issues**: Potential rowid misalignment between memories and embeddings tables
11 |
12 | ## Changes Made
13 |
14 | ### 1. Fixed Dependencies (pyproject.toml)
15 |
16 | - Moved `sentence-transformers>=2.2.2` from optional to core dependencies
17 | - Added `torch>=1.6.0` to core dependencies
18 | - This ensures embedding functionality is always available
19 |
20 | ### 2. Fixed Initialization Order (sqlite_vec.py)
21 |
22 | - Moved embedding model initialization BEFORE vector table creation
23 | - This ensures the correct embedding dimension is used for the table schema
24 | - Added explicit check for sentence-transformers availability
25 |
26 | ### 3. Improved Error Handling
27 |
28 | - Replaced silent failures with explicit exceptions
29 | - Added proper error messages for missing dependencies
30 | - Added embedding validation after generation (dimension check, finite values check)
31 |
32 | ### 4. Fixed Database Operations
33 |
34 | #### Store Operation:
35 | - Added try-catch for embedding generation with proper error propagation
36 | - Added fallback for rowid insertion if direct rowid insert fails
37 | - Added validation before storing embeddings
38 |
39 | #### Retrieve Operation:
40 | - Added check for empty embeddings table
41 | - Added debug logging for troubleshooting
42 | - Improved error handling for query embedding generation
43 |
44 | ### 5. Created Diagnostic Script
45 |
46 | - `scripts/test_sqlite_vec_embeddings.py` - comprehensive test suite
47 | - Tests dependencies, initialization, embedding generation, storage, and search
48 | - Provides clear error messages and troubleshooting guidance
49 |
50 | ## Key Code Changes
51 |
52 | ### sqlite_vec.py:
53 |
54 | 1. **Initialize method**:
55 | - Added sentence-transformers check
56 | - Moved model initialization before table creation
57 |
58 | 2. **_generate_embedding method**:
59 | - Raises exception instead of returning zero vector
60 | - Added comprehensive validation
61 |
62 | 3. **store method**:
63 | - Better error handling for embedding generation
64 | - Fallback for rowid insertion
65 |
66 | 4. **retrieve method**:
67 | - Check for empty embeddings table
68 | - Better debug logging
69 |
70 | ## Testing
71 |
72 | Run the diagnostic script to verify the fixes:
73 |
74 | ```bash
75 | python3 scripts/test_sqlite_vec_embeddings.py
76 | ```
77 |
78 | This will check:
79 | - Dependency installation
80 | - Storage initialization
81 | - Embedding generation
82 | - Memory storage with embeddings
83 | - Semantic search functionality
84 | - Database integrity
85 |
86 | ## Migration Notes
87 |
88 | For existing installations:
89 |
90 | 1. Update dependencies: `uv pip install -e .`
91 | 2. Use the provided migration tools to save existing memories:
92 |
93 | ### Option 1: Quick Repair (Try First)
94 | For databases with missing embeddings but correct schema:
95 |
96 | ```bash
97 | python3 scripts/repair_sqlite_vec_embeddings.py /path/to/your/sqlite_vec.db
98 | ```
99 |
100 | This will:
101 | - Analyze your database
102 | - Generate missing embeddings
103 | - Verify search functionality
104 |
105 | ### Option 2: Full Migration (If Repair Fails)
106 | For databases with dimension mismatches or schema issues:
107 |
108 | ```bash
109 | python3 scripts/migrate_sqlite_vec_embeddings.py /path/to/your/sqlite_vec.db
110 | ```
111 |
112 | This will:
113 | - Create a backup of your database
114 | - Extract all memories
115 | - Create a new database with correct schema
116 | - Regenerate all embeddings
117 | - Restore all memories
118 |
119 | **Important**: The migration creates a timestamped backup before making any changes.
120 |
121 | ## Future Improvements
122 |
123 | 1. ~~Add migration script for existing databases~~ ✓ Done
124 | 2. Add batch embedding generation for better performance
125 | 3. ~~Add embedding regeneration capability for existing memories~~ ✓ Done
126 | 4. Implement better rowid synchronization between tables
127 | 5. Add automatic detection and repair on startup
128 | 6. Add embedding model versioning to handle model changes
```
--------------------------------------------------------------------------------
/scripts/maintenance/fast_cleanup_duplicates_with_tracking.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # Fast duplicate cleanup using direct SQL with hash tracking for Cloudflare sync
3 | set -e
4 |
5 | # Platform-specific database path
6 | if [[ "$OSTYPE" == "darwin"* ]]; then
7 | DB_PATH="$HOME/Library/Application Support/mcp-memory/sqlite_vec.db"
8 | else
9 | DB_PATH="$HOME/.local/share/mcp-memory/sqlite_vec.db"
10 | fi
11 | HASH_FILE="$HOME/deleted_duplicates.txt"
12 |
13 | echo "🛑 Stopping HTTP server..."
14 | # Try to stop the HTTP server - use the actual PID method since systemd may not be available on macOS
15 | ps aux | grep -E "uvicorn.*8889" | grep -v grep | awk '{print $2}' | xargs kill 2>/dev/null || true
16 | sleep 2
17 |
18 | echo "📊 Analyzing duplicates and tracking hashes..."
19 |
20 | # Create Python script to find duplicates, save hashes, and delete
21 | python3 << 'PYTHON_SCRIPT'
22 | import sqlite3
23 | from pathlib import Path
24 | from collections import defaultdict
25 | import hashlib
26 | import re
27 | import os
28 |
29 | import platform
30 |
31 | # Platform-specific database path
32 | if platform.system() == "Darwin": # macOS
33 | DB_PATH = Path.home() / "Library/Application Support/mcp-memory/sqlite_vec.db"
34 | else: # Linux/Windows
35 | DB_PATH = Path.home() / ".local/share/mcp-memory/sqlite_vec.db"
36 |
37 | HASH_FILE = Path.home() / "deleted_duplicates.txt"
38 |
39 | def normalize_content(content):
40 | """Normalize content by removing timestamps."""
41 | normalized = content
42 | normalized = re.sub(r'\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z', 'TIMESTAMP', normalized)
43 | normalized = re.sub(r'\*\*Date\*\*: \d{2,4}[./]\d{2}[./]\d{2,4}', '**Date**: DATE', normalized)
44 | normalized = re.sub(r'Timestamp: \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}', 'Timestamp: TIMESTAMP', normalized)
45 | return normalized.strip()
46 |
47 | def get_normalized_hash(content):
48 | """Create a hash of normalized content."""
49 | normalized = normalize_content(content)
50 | return hashlib.md5(normalized.encode()).hexdigest()
51 |
52 | conn = sqlite3.connect(DB_PATH)
53 | cursor = conn.cursor()
54 |
55 | print("Analyzing memories...")
56 | cursor.execute("SELECT id, content_hash, content, created_at FROM memories ORDER BY created_at DESC")
57 | memories = cursor.fetchall()
58 |
59 | print(f"Total memories: {len(memories)}")
60 |
61 | # Group by normalized content
62 | content_groups = defaultdict(list)
63 | for mem_id, mem_hash, mem_content, created_at in memories:
64 | norm_hash = get_normalized_hash(mem_content)
65 | content_groups[norm_hash].append({
66 | 'id': mem_id,
67 | 'hash': mem_hash,
68 | 'created_at': created_at
69 | })
70 |
71 | # Find duplicates
72 | duplicates = {k: v for k, v in content_groups.items() if len(v) > 1}
73 |
74 | if not duplicates:
75 | print("✅ No duplicates found!")
76 | conn.close()
77 | exit(0)
78 |
79 | print(f"Found {len(duplicates)} duplicate groups")
80 |
81 | # Collect IDs and hashes to delete (keep newest, delete older)
82 | ids_to_delete = []
83 | hashes_to_delete = []
84 |
85 | for group in duplicates.values():
86 | for memory in group[1:]: # Keep first (newest), delete rest
87 | ids_to_delete.append(memory['id'])
88 | hashes_to_delete.append(memory['hash'])
89 |
90 | print(f"Deleting {len(ids_to_delete)} duplicate memories...")
91 |
92 | # Save hashes to file for Cloudflare cleanup
93 | print(f"Saving {len(hashes_to_delete)} content hashes to {HASH_FILE}...")
94 | with open(HASH_FILE, 'w') as f:
95 | for content_hash in hashes_to_delete:
96 | f.write(f"{content_hash}\n")
97 |
98 | print(f"✅ Saved hashes to {HASH_FILE}")
99 |
100 | # Delete from memories table
101 | placeholders = ','.join('?' * len(ids_to_delete))
102 | cursor.execute(f"DELETE FROM memories WHERE id IN ({placeholders})", ids_to_delete)
103 |
104 | # Note: Can't delete from virtual table without vec0 extension
105 | # Orphaned embeddings will be cleaned up on next regeneration
106 |
107 | conn.commit()
108 | conn.close()
109 |
110 | print(f"✅ Deleted {len(ids_to_delete)} duplicates from SQLite")
111 | print(f"📝 Content hashes saved for Cloudflare cleanup")
112 |
113 | PYTHON_SCRIPT
114 |
115 | echo ""
116 | echo "🚀 Restarting HTTP server..."
117 | nohup uv run python -m uvicorn mcp_memory_service.web.app:app --host 127.0.0.1 --port 8889 > /tmp/memory_http_server.log 2>&1 &
118 | sleep 3
119 |
120 | echo ""
121 | echo "✅ SQLite cleanup complete!"
122 | echo "📋 Next steps:"
123 | echo " 1. Review deleted hashes: cat $HASH_FILE"
124 | echo " 2. Delete from Cloudflare: uv run python scripts/maintenance/delete_cloudflare_duplicates.py"
125 | echo " 3. Verify counts match"
126 |
```
--------------------------------------------------------------------------------
/archive/docs-removed-2025-08-23/development/test-results.md:
--------------------------------------------------------------------------------
```markdown
1 | # FastAPI MCP Server Test Results
2 |
3 | ## Date: 2025-08-03
4 | ## Branch: feature/fastapi-mcp-native-v4
5 | ## Version: 4.0.0-alpha.1
6 |
7 | ## ✅ **SUCCESSFUL LOCAL TESTING**
8 |
9 | ### Server Startup Test
10 | - ✅ **FastAPI MCP Server starts successfully**
11 | - ✅ **Listening on localhost:8000**
12 | - ✅ **MCP protocol responding correctly**
13 | - ✅ **Streamable HTTP transport working**
14 | - ✅ **Session management functional**
15 |
16 | ### MCP Protocol Validation
17 | - ✅ **Server accepts MCP requests** (responds with proper JSON-RPC)
18 | - ✅ **Session ID handling** (creates transport sessions)
19 | - ✅ **Error handling** (proper error responses for invalid requests)
20 | - ✅ **Content-type requirements** (requires text/event-stream)
21 |
22 | ### Tools Implementation Status
23 | **✅ Implemented (5 core tools)**:
24 | 1. `store_memory` - Store memories with tags and metadata
25 | 2. `retrieve_memory` - Semantic search and retrieval
26 | 3. `search_by_tag` - Tag-based memory search
27 | 4. `delete_memory` - Delete specific memories
28 | 5. `check_database_health` - Health check and statistics
29 |
30 | ### Configuration Update
31 | - ✅ **Claude Code config updated** from Node.js bridge to FastAPI MCP
32 | - ✅ **Old config**: `node examples/http-mcp-bridge.js`
33 | - ✅ **New config**: `python test_mcp_minimal.py`
34 | - ✅ **Environment simplified** (no complex SSL/endpoint config needed)
35 |
36 | ## 🏗️ **ARCHITECTURE VALIDATION**
37 |
38 | ### Node.js Bridge Replacement
39 | - ✅ **Native MCP protocol** (no HTTP-to-MCP translation)
40 | - ✅ **Direct Python implementation** (using official MCP SDK)
41 | - ✅ **Simplified configuration** (no bridging complexity)
42 | - ✅ **Local SSL eliminated** (direct protocol, no HTTPS needed locally)
43 |
44 | ### Performance Observations
45 | - ✅ **Fast startup** (~2 seconds to ready state)
46 | - ✅ **Low memory usage** (minimal overhead vs Node.js bridge)
47 | - ✅ **Responsive** (immediate MCP protocol responses)
48 | - ✅ **Stable** (clean session management)
49 |
50 | ## 📊 **NEXT STEPS VALIDATION**
51 |
52 | ### ✅ Completed Phases
53 | 1. ✅ **Phase 1A**: Local server testing - SUCCESS
54 | 2. ✅ **Phase 1B**: Claude Code configuration - SUCCESS
55 | 3. 🚧 **Phase 1C**: MCP tools testing - PENDING (requires session restart)
56 |
57 | ### Ready for Next Phase
58 | - ✅ **Foundation proven** - FastAPI MCP architecture works
59 | - ✅ **Protocol compatibility** - Official MCP SDK integration successful
60 | - ✅ **Configuration working** - Claude Code can connect to new server
61 | - ✅ **Tool structure validated** - 5 core operations implemented
62 |
63 | ### Remaining Tasks
64 | 1. **Restart Claude Code session** to pick up new MCP server config
65 | 2. **Test 5 core MCP tools** with real Claude Code integration
66 | 3. **Validate SSL issues resolved** (vs Node.js bridge problems)
67 | 4. **Expand to full 22 tools** implementation
68 | 5. **Remote server deployment** planning
69 |
70 | ## 🎯 **SUCCESS INDICATORS**
71 |
72 | ### ✅ **Major Architecture Success**
73 | - **Problem**: Node.js SSL handshake failures with self-signed certificates
74 | - **Solution**: Native FastAPI MCP server eliminates SSL layer entirely
75 | - **Result**: Direct MCP protocol communication, no SSL issues possible
76 |
77 | ### ✅ **Implementation Success**
78 | - **FastMCP Framework**: Official MCP Python SDK working perfectly
79 | - **Streamable HTTP**: Correct transport for Claude Code integration
80 | - **Tool Structure**: All 5 core memory operations implemented
81 | - **Session Management**: Proper MCP session lifecycle handling
82 |
83 | ### ✅ **Configuration Success**
84 | - **Simplified Config**: No complex environment variables needed
85 | - **Direct Connection**: No intermediate bridging or translation
86 | - **Local Testing**: Immediate validation without remote dependencies
87 | - **Version Management**: Clean v4.0.0-alpha.1 progression
88 |
89 | ## 📝 **CONCLUSION**
90 |
91 | The **FastAPI MCP Server migration is fundamentally successful**. The architecture change from Node.js bridge to native Python MCP server resolves all SSL issues and provides a much cleaner, more maintainable solution.
92 |
93 | **Status**: Ready for full MCP tools integration testing
94 | **Confidence**: High - core architecture proven to work
95 | **Risk**: Low - fallback to Node.js bridge available if needed
96 |
97 | This validates our architectural decision and proves the FastAPI MCP approach will solve the remote memory access problems that users have been experiencing.
```
--------------------------------------------------------------------------------
/scripts/sync/litestream/pull_remote_changes.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # Enhanced remote sync with conflict awareness
3 | # Based on the working manual_sync.sh but with staging awareness
4 |
5 | DB_PATH="/Users/hkr/Library/Application Support/mcp-memory/sqlite_vec.db"
6 | STAGING_DB="/Users/hkr/Library/Application Support/mcp-memory/sqlite_vec_staging.db"
7 | REMOTE_BASE="http://narrowbox.local:8080/mcp-memory"
8 | BACKUP_PATH="/Users/hkr/Library/Application Support/mcp-memory/sqlite_vec.db.backup"
9 | TEMP_DIR="/tmp/litestream_pull_$$"
10 |
11 | echo "$(date): Starting enhanced pull from remote master..."
12 |
13 | # Create temporary directory
14 | mkdir -p "$TEMP_DIR"
15 |
16 | # Check if staging database exists
17 | if [ ! -f "$STAGING_DB" ]; then
18 | echo "$(date): WARNING: Staging database not found. Creating..."
19 | ./init_staging_db.sh
20 | fi
21 |
22 | # Get the latest generation ID
23 | GENERATION=$(curl -s "$REMOTE_BASE/generations/" | grep -o 'href="[^"]*/"' | sed 's/href="//;s/\/"//g' | head -1)
24 |
25 | if [ -z "$GENERATION" ]; then
26 | echo "$(date): ERROR: Could not determine generation ID"
27 | rm -rf "$TEMP_DIR"
28 | exit 1
29 | fi
30 |
31 | echo "$(date): Found remote generation: $GENERATION"
32 |
33 | # Get the latest snapshot
34 | SNAPSHOT_URL="$REMOTE_BASE/generations/$GENERATION/snapshots/"
35 | SNAPSHOT_FILE=$(curl -s "$SNAPSHOT_URL" | grep -o 'href="[^"]*\.snapshot\.lz4"' | sed 's/href="//;s/"//g' | tail -1)
36 |
37 | if [ -z "$SNAPSHOT_FILE" ]; then
38 | echo "$(date): ERROR: Could not find snapshot file"
39 | rm -rf "$TEMP_DIR"
40 | exit 1
41 | fi
42 |
43 | echo "$(date): Downloading snapshot: $SNAPSHOT_FILE"
44 |
45 | # Download and decompress snapshot
46 | curl -s "$SNAPSHOT_URL$SNAPSHOT_FILE" -o "$TEMP_DIR/snapshot.lz4"
47 |
48 | if ! command -v lz4 >/dev/null 2>&1; then
49 | echo "$(date): ERROR: lz4 command not found. Please install: brew install lz4"
50 | rm -rf "$TEMP_DIR"
51 | exit 1
52 | fi
53 |
54 | lz4 -d "$TEMP_DIR/snapshot.lz4" "$TEMP_DIR/remote_database.db" 2>/dev/null
55 |
56 | if [ ! -f "$TEMP_DIR/remote_database.db" ]; then
57 | echo "$(date): ERROR: Failed to decompress remote database"
58 | rm -rf "$TEMP_DIR"
59 | exit 1
60 | fi
61 |
62 | # Conflict detection: Check if we have staged changes that might conflict
63 | STAGED_COUNT=0
64 | if [ -f "$STAGING_DB" ]; then
65 | STAGED_COUNT=$(sqlite3 "$STAGING_DB" "SELECT COUNT(*) FROM staged_memories WHERE conflict_status = 'none';" 2>/dev/null || echo "0")
66 | fi
67 |
68 | if [ "$STAGED_COUNT" -gt 0 ]; then
69 | echo "$(date): WARNING: $STAGED_COUNT staged changes detected"
70 | echo "$(date): Checking for potential conflicts..."
71 |
72 | # Create a list of content hashes in staging
73 | sqlite3 "$STAGING_DB" "SELECT content_hash FROM staged_memories;" > "$TEMP_DIR/staged_hashes.txt"
74 |
75 | # Check if any of these hashes exist in the remote database
76 | # Note: This requires knowledge of the remote database schema
77 | # For now, we'll just warn about the existence of staged changes
78 | echo "$(date): Staged changes will be applied after remote pull"
79 | fi
80 |
81 | # Backup current database
82 | if [ -f "$DB_PATH" ]; then
83 | cp "$DB_PATH" "$BACKUP_PATH"
84 | echo "$(date): Created backup at $BACKUP_PATH"
85 | fi
86 |
87 | # Replace with remote database
88 | cp "$TEMP_DIR/remote_database.db" "$DB_PATH"
89 |
90 | if [ $? -eq 0 ]; then
91 | echo "$(date): Successfully pulled database from remote master"
92 |
93 | # Update staging database with sync timestamp
94 | if [ -f "$STAGING_DB" ]; then
95 | sqlite3 "$STAGING_DB" "
96 | UPDATE sync_status
97 | SET value = datetime('now'), updated_at = CURRENT_TIMESTAMP
98 | WHERE key = 'last_remote_sync';
99 | "
100 | fi
101 |
102 | # Remove backup on success
103 | rm -f "$BACKUP_PATH"
104 |
105 | # Show database info
106 | echo "$(date): Database size: $(du -h "$DB_PATH" | cut -f1)"
107 | echo "$(date): Database modified: $(stat -f "%Sm" "$DB_PATH")"
108 |
109 | if [ "$STAGED_COUNT" -gt 0 ]; then
110 | echo "$(date): NOTE: You have $STAGED_COUNT staged changes to apply"
111 | echo "$(date): Run ./apply_local_changes.sh to merge them"
112 | fi
113 | else
114 | echo "$(date): ERROR: Failed to replace database with remote version"
115 | # Restore backup on failure
116 | if [ -f "$BACKUP_PATH" ]; then
117 | mv "$BACKUP_PATH" "$DB_PATH"
118 | echo "$(date): Restored backup"
119 | fi
120 | rm -rf "$TEMP_DIR"
121 | exit 1
122 | fi
123 |
124 | # Cleanup
125 | rm -rf "$TEMP_DIR"
126 | echo "$(date): Remote pull completed successfully"
```
--------------------------------------------------------------------------------
/src/mcp_memory_service/api/__init__.py:
--------------------------------------------------------------------------------
```python
1 | # Copyright 2024 Heinrich Krupp
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # http://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
15 | """
16 | Code Execution API for MCP Memory Service.
17 |
18 | This module provides a lightweight, token-efficient interface for direct
19 | Python code execution, replacing verbose MCP tool calls with compact
20 | function calls and results.
21 |
22 | Token Efficiency Comparison:
23 | - Import: ~10 tokens (once per session)
24 | - search(5 results): ~385 tokens vs ~2,625 (85% reduction)
25 | - store(): ~15 tokens vs ~150 (90% reduction)
26 | - health(): ~20 tokens vs ~125 (84% reduction)
27 |
28 | Annual Savings (Conservative):
29 | - 10 users x 5 sessions/day x 365 days x 6,000 tokens = 109.5M tokens/year
30 | - At $0.15/1M tokens: $16.43/year per 10-user deployment
31 |
32 | Performance:
33 | - First call: ~50ms (includes storage initialization)
34 | - Subsequent calls: ~5-10ms (connection reused)
35 | - Memory overhead: <10MB
36 |
37 | Usage Example:
38 | >>> from mcp_memory_service.api import search, store, health
39 | >>>
40 | >>> # Search memories (20 tokens)
41 | >>> results = search("architecture decisions", limit=5)
42 | >>> for m in results.memories:
43 | ... print(f"{m.hash}: {m.preview[:50]}...")
44 | abc12345: Implemented OAuth 2.1 authentication for...
45 | def67890: Refactored storage backend to support...
46 | >>>
47 | >>> # Store memory (15 tokens)
48 | >>> hash = store("New memory", tags=["note", "important"])
49 | >>> print(f"Stored: {hash}")
50 | Stored: abc12345
51 | >>>
52 | >>> # Health check (5 tokens)
53 | >>> info = health()
54 | >>> print(f"Backend: {info.backend}, Count: {info.count}")
55 | Backend: sqlite_vec, Count: 1247
56 |
57 | Backward Compatibility:
58 | This API is designed to work alongside existing MCP tools without
59 | breaking changes. Users can gradually migrate from tool-based calls
60 | to code execution as needed.
61 |
62 | Implementation:
63 | - Phase 1 (Current): Core operations (search, store, health)
64 | - Phase 2: Extended operations (search_by_tag, recall, delete, update)
65 | - Phase 3: Advanced features (batch operations, streaming)
66 |
67 | For More Information:
68 | - Research: /docs/research/code-execution-interface-implementation.md
69 | - Documentation: /docs/api/code-execution-interface.md
70 | - Issue: https://github.com/doobidoo/mcp-memory-service/issues/206
71 | """
72 |
73 | from .types import (
74 | CompactMemory, CompactSearchResult, CompactHealthInfo,
75 | CompactConsolidationResult, CompactSchedulerStatus
76 | )
77 | from .operations import (
78 | search, store, health, consolidate, scheduler_status,
79 | _consolidate_async, _scheduler_status_async
80 | )
81 | from .client import close, close_async, set_consolidator, set_scheduler
82 |
83 | __all__ = [
84 | # Core operations
85 | 'search', # Semantic search with compact results
86 | 'store', # Store new memory
87 | 'health', # Service health check
88 | 'close', # Close and cleanup storage resources (sync)
89 | 'close_async', # Close and cleanup storage resources (async)
90 |
91 | # Consolidation operations
92 | 'consolidate', # Trigger memory consolidation
93 | 'scheduler_status', # Get consolidation scheduler status
94 |
95 | # Consolidation management (internal use by HTTP server)
96 | 'set_consolidator', # Set global consolidator instance
97 | 'set_scheduler', # Set global scheduler instance
98 |
99 | # Compact data types
100 | 'CompactMemory',
101 | 'CompactSearchResult',
102 | 'CompactHealthInfo',
103 | 'CompactConsolidationResult',
104 | 'CompactSchedulerStatus',
105 | ]
106 |
107 | # API version for compatibility tracking
108 | __api_version__ = "1.0.0"
109 |
110 | # Module metadata
111 | __doc_url__ = "https://github.com/doobidoo/mcp-memory-service/blob/main/docs/api/code-execution-interface.md"
112 | __issue_url__ = "https://github.com/doobidoo/mcp-memory-service/issues/206"
113 |
```
--------------------------------------------------------------------------------
/.github/workflows/docker-publish.yml:
--------------------------------------------------------------------------------
```yaml
1 | name: Docker Publish (Tags)
2 |
3 | on:
4 | push:
5 | tags:
6 | - 'v*.*.*'
7 | workflow_dispatch:
8 |
9 | env:
10 | REGISTRY: docker.io
11 | IMAGE_NAME: doobidoo/mcp-memory-service
12 |
13 | jobs:
14 | build:
15 | runs-on: ubuntu-latest
16 | permissions:
17 | contents: read
18 | packages: write
19 | id-token: write
20 | attestations: write
21 |
22 | steps:
23 | - name: Checkout repository
24 | uses: actions/checkout@v4
25 |
26 | - name: Set up Docker Buildx
27 | uses: docker/setup-buildx-action@v3
28 |
29 | - name: Debug - Check required files for Docker Hub build
30 | run: |
31 | echo "=== Checking required files for Docker Hub build ==="
32 | echo "Standard Dockerfile exists:" && ls -la tools/docker/Dockerfile
33 | echo "Slim Dockerfile exists:" && ls -la tools/docker/Dockerfile.slim
34 | echo "Source directory exists:" && ls -la src/
35 | echo "Entrypoint scripts exist:" && ls -la tools/docker/docker-entrypoint*.sh
36 | echo "Utils scripts exist:" && ls -la scripts/utils/
37 | echo "pyproject.toml exists:" && ls -la pyproject.toml
38 | echo "uv.lock exists:" && ls -la uv.lock
39 |
40 | - name: Log in to Docker Hub
41 | if: github.event_name != 'pull_request'
42 | uses: docker/login-action@v3
43 | with:
44 | registry: ${{ env.REGISTRY }}
45 | username: ${{ secrets.DOCKER_USERNAME }}
46 | password: ${{ secrets.DOCKER_PASSWORD }}
47 |
48 | - name: Extract metadata (Standard)
49 | id: meta
50 | uses: docker/metadata-action@v5
51 | with:
52 | images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
53 | tags: |
54 | type=ref,event=branch
55 | type=ref,event=pr
56 | type=semver,pattern={{version}}
57 | type=semver,pattern={{major}}.{{minor}}
58 | type=semver,pattern={{major}}
59 | type=raw,value=latest,enable={{is_default_branch}}
60 |
61 | - name: Extract metadata (Slim)
62 | id: meta-slim
63 | uses: docker/metadata-action@v5
64 | with:
65 | images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
66 | tags: |
67 | type=ref,event=branch,suffix=-slim
68 | type=ref,event=pr,suffix=-slim
69 | type=semver,pattern={{version}},suffix=-slim
70 | type=semver,pattern={{major}}.{{minor}},suffix=-slim
71 | type=semver,pattern={{major}},suffix=-slim
72 | type=raw,value=slim,enable={{is_default_branch}}
73 |
74 | - name: Build and push Standard Docker image
75 | id: build-and-push
76 | uses: docker/build-push-action@v5
77 | with:
78 | context: .
79 | file: ./tools/docker/Dockerfile
80 | platforms: linux/amd64,linux/arm64
81 | push: ${{ github.event_name != 'pull_request' }}
82 | tags: ${{ steps.meta.outputs.tags }}
83 | labels: ${{ steps.meta.outputs.labels }}
84 | cache-from: type=gha,scope=standard
85 | cache-to: type=gha,mode=max,scope=standard
86 | build-args: |
87 | SKIP_MODEL_DOWNLOAD=true
88 | outputs: type=registry
89 |
90 | - name: Build and push Slim Docker image
91 | id: build-and-push-slim
92 | uses: docker/build-push-action@v5
93 | with:
94 | context: .
95 | file: ./tools/docker/Dockerfile.slim
96 | platforms: linux/amd64,linux/arm64
97 | push: ${{ github.event_name != 'pull_request' }}
98 | tags: ${{ steps.meta-slim.outputs.tags }}
99 | labels: ${{ steps.meta-slim.outputs.labels }}
100 | cache-from: type=gha,scope=slim
101 | cache-to: type=gha,mode=max,scope=slim
102 | build-args: |
103 | SKIP_MODEL_DOWNLOAD=true
104 | outputs: type=registry
105 |
106 | - name: Generate artifact attestation (Standard)
107 | if: github.event_name != 'pull_request'
108 | uses: actions/attest-build-provenance@v1
109 | with:
110 | subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
111 | subject-digest: ${{ steps.build-and-push.outputs.digest }}
112 | push-to-registry: true
113 | continue-on-error: true # Don't fail the workflow if attestation fails
114 |
115 | - name: Generate artifact attestation (Slim)
116 | if: github.event_name != 'pull_request'
117 | uses: actions/attest-build-provenance@v1
118 | with:
119 | subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
120 | subject-digest: ${{ steps.build-and-push-slim.outputs.digest }}
121 | push-to-registry: true
122 | continue-on-error: true # Don't fail the workflow if attestation fails
```
--------------------------------------------------------------------------------
/src/mcp_memory_service/api/sync_wrapper.py:
--------------------------------------------------------------------------------
```python
1 | # Copyright 2024 Heinrich Krupp
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # http://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
15 | """
16 | Async-to-sync utilities for code execution interface.
17 |
18 | Provides lightweight wrappers to convert async storage operations into
19 | synchronous functions suitable for code execution contexts (e.g., hooks).
20 |
21 | Performance:
22 | - Cold call: ~50ms (includes event loop creation)
23 | - Warm call: ~5ms (reuses existing loop)
24 | - Overhead: <10ms compared to native async
25 |
26 | Design Philosophy:
27 | - Hide asyncio complexity from API users
28 | - Reuse event loops when possible for performance
29 | - Graceful error handling and cleanup
30 | - Zero async/await in public API
31 | """
32 |
33 | import asyncio
34 | from functools import wraps
35 | from typing import Callable, TypeVar, Any
36 | import logging
37 |
38 | logger = logging.getLogger(__name__)
39 |
40 | # Type variable for generic function wrapping
41 | T = TypeVar('T')
42 |
43 |
44 | def sync_wrapper(async_func: Callable[..., Any]) -> Callable[..., Any]:
45 | """
46 | Convert async function to synchronous with minimal overhead.
47 |
48 | This wrapper handles event loop management transparently:
49 | 1. Attempts to get existing event loop
50 | 2. Creates new loop if none exists
51 | 3. Runs async function to completion
52 | 4. Returns result or raises exception
53 |
54 | Performance:
55 | - Adds ~1-5ms overhead per call
56 | - Reuses event loop when possible
57 | - Optimized for repeated calls (e.g., in hooks)
58 |
59 | Args:
60 | async_func: Async function to wrap
61 |
62 | Returns:
63 | Synchronous wrapper function with same signature
64 |
65 | Example:
66 | >>> async def fetch_data(query: str) -> list:
67 | ... return await storage.retrieve(query, limit=5)
68 | >>> sync_fetch = sync_wrapper(fetch_data)
69 | >>> results = sync_fetch("architecture") # No await needed
70 |
71 | Note:
72 | This wrapper is designed for code execution contexts where
73 | async/await is not available or desirable. For pure async
74 | code, use the storage backend directly.
75 | """
76 | @wraps(async_func)
77 | def wrapper(*args: Any, **kwargs: Any) -> Any:
78 | try:
79 | # Try to get existing event loop
80 | loop = asyncio.get_event_loop()
81 | if loop.is_closed():
82 | # Loop exists but is closed, create new one
83 | loop = asyncio.new_event_loop()
84 | asyncio.set_event_loop(loop)
85 | except RuntimeError:
86 | # No event loop in current thread, create new one
87 | loop = asyncio.new_event_loop()
88 | asyncio.set_event_loop(loop)
89 |
90 | try:
91 | # Run async function to completion
92 | result = loop.run_until_complete(async_func(*args, **kwargs))
93 | return result
94 | except Exception as e:
95 | # Re-raise exception with context
96 | logger.error(f"Error in sync wrapper for {async_func.__name__}: {e}")
97 | raise
98 |
99 | return wrapper
100 |
101 |
102 | def run_async(coro: Any) -> Any:
103 | """
104 | Run a coroutine synchronously and return its result.
105 |
106 | Convenience function for running async operations in sync contexts
107 | without explicitly creating a wrapper function.
108 |
109 | Args:
110 | coro: Coroutine object to run
111 |
112 | Returns:
113 | Result of the coroutine
114 |
115 | Example:
116 | >>> result = run_async(storage.retrieve("query", limit=5))
117 | >>> print(len(result))
118 |
119 | Note:
120 | Prefer sync_wrapper() for repeated calls to the same function,
121 | as it avoids wrapper creation overhead.
122 | """
123 | try:
124 | loop = asyncio.get_event_loop()
125 | if loop.is_closed():
126 | loop = asyncio.new_event_loop()
127 | asyncio.set_event_loop(loop)
128 | except RuntimeError:
129 | loop = asyncio.new_event_loop()
130 | asyncio.set_event_loop(loop)
131 |
132 | return loop.run_until_complete(coro)
133 |
```
--------------------------------------------------------------------------------
/tests/unit/test_memory.py:
--------------------------------------------------------------------------------
```python
1 | #!/usr/bin/env python3
2 | """
3 | Simple test script to verify memory service functionality.
4 | """
5 | import asyncio
6 | import json
7 | import sys
8 | import os
9 | from datetime import datetime
10 |
11 | # Set environment variables for testing
12 | os.environ["MCP_MEMORY_STORAGE_BACKEND"] = "sqlite_vec"
13 | os.environ["MCP_MEMORY_SQLITE_PATH"] = os.path.expanduser("~/Library/Application Support/mcp-memory/sqlite_vec.db")
14 | os.environ["MCP_MEMORY_BACKUPS_PATH"] = os.path.expanduser("~/Library/Application Support/mcp-memory/backups")
15 | os.environ["MCP_MEMORY_USE_ONNX"] = "1"
16 |
17 | # Import our modules
18 | from mcp_memory_service.storage.sqlite_vec import SqliteVecMemoryStorage
19 | from mcp_memory_service.models.memory import Memory
20 | from mcp_memory_service.utils.db_utils import validate_database, get_database_stats
21 |
22 | async def main():
23 | print("=== MCP Memory Service Test ===")
24 |
25 | # Initialize the storage
26 | db_path = os.environ["MCP_MEMORY_SQLITE_PATH"]
27 | print(f"Using SQLite-vec database at: {db_path}")
28 |
29 | storage = SqliteVecMemoryStorage(db_path)
30 | await storage.initialize()
31 |
32 | # Run our own database health check
33 | print("\n=== Database Health Check ===")
34 | if storage.conn is None:
35 | print("Database connection is not initialized")
36 | else:
37 | try:
38 | cursor = storage.conn.execute('SELECT COUNT(*) FROM memories')
39 | memory_count = cursor.fetchone()[0]
40 | print(f"Database connected successfully. Contains {memory_count} memories.")
41 |
42 | cursor = storage.conn.execute("SELECT name FROM sqlite_master WHERE type='table'")
43 | tables = [row[0] for row in cursor.fetchall()]
44 | print(f"Database tables: {', '.join(tables)}")
45 |
46 | if not storage.embedding_model:
47 | print("No embedding model available. Limited functionality.")
48 |
49 | except Exception as e:
50 | print(f"Database error: {str(e)}")
51 |
52 | # Get database stats directly
53 | print("\n=== Database Stats ===")
54 | try:
55 | # Simple stats
56 | cursor = storage.conn.execute('SELECT COUNT(*) FROM memories')
57 | memory_count = cursor.fetchone()[0]
58 |
59 | # Get database file size
60 | db_path = storage.db_path
61 | file_size = os.path.getsize(db_path) if os.path.exists(db_path) else 0
62 |
63 | stats = {
64 | "backend": "sqlite-vec",
65 | "total_memories": memory_count,
66 | "database_size_bytes": file_size,
67 | "database_size_mb": round(file_size / (1024 * 1024), 2),
68 | "embedding_model": storage.embedding_model_name if hasattr(storage, 'embedding_model_name') else "none",
69 | "embedding_dimension": storage.embedding_dimension if hasattr(storage, 'embedding_dimension') else 0
70 | }
71 | print(json.dumps(stats, indent=2))
72 | except Exception as e:
73 | print(f"Error getting stats: {str(e)}")
74 |
75 | # Store a test memory
76 | print("\n=== Creating Test Memory ===")
77 | test_content = "This is a test memory created on " + datetime.now().isoformat()
78 |
79 | # Import the hash function
80 | from mcp_memory_service.utils.hashing import generate_content_hash
81 |
82 | test_memory = Memory(
83 | content=test_content,
84 | content_hash=generate_content_hash(test_content),
85 | tags=["test", "example"],
86 | memory_type="note",
87 | metadata={"source": "test_script"}
88 | )
89 | print(f"Memory content: {test_memory.content}")
90 | print(f"Content hash: {test_memory.content_hash}")
91 |
92 | success, message = await storage.store(test_memory)
93 | print(f"Store success: {success}")
94 | print(f"Message: {message}")
95 |
96 | # Try to retrieve the memory
97 | print("\n=== Retrieving Memories ===")
98 | results = await storage.retrieve("test memory", n_results=5)
99 |
100 | if results:
101 | print(f"Found {len(results)} memories")
102 | for i, result in enumerate(results):
103 | print(f" Result {i+1}:")
104 | print(f" Content: {result.memory.content}")
105 | print(f" Tags: {result.memory.tags}")
106 | print(f" Score: {result.relevance_score}")
107 | else:
108 | print("No memories found")
109 |
110 | print("\n=== Test Complete ===")
111 | storage.close()
112 |
113 | if __name__ == "__main__":
114 | asyncio.run(main())
```
--------------------------------------------------------------------------------
/scripts/pr/generate_tests.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # scripts/pr/generate_tests.sh - Auto-generate tests for new code in PR
3 | #
4 | # Usage: bash scripts/pr/generate_tests.sh <PR_NUMBER>
5 | # Example: bash scripts/pr/generate_tests.sh 123
6 |
7 | set -e
8 |
9 | PR_NUMBER=$1
10 |
11 | if [ -z "$PR_NUMBER" ]; then
12 | echo "Usage: $0 <PR_NUMBER>"
13 | exit 1
14 | fi
15 |
16 | if ! command -v gh &> /dev/null; then
17 | echo "Error: GitHub CLI (gh) is not installed"
18 | exit 1
19 | fi
20 |
21 | if ! command -v gemini &> /dev/null; then
22 | echo "Error: Gemini CLI is not installed"
23 | exit 1
24 | fi
25 |
26 | echo "=== Test Generation for PR #$PR_NUMBER ==="
27 | echo ""
28 |
29 | # Get changed Python files (exclude tests/)
30 | changed_files=$(gh pr diff $PR_NUMBER --name-only | grep '\.py$' | grep -v '^tests/' || echo "")
31 |
32 | if [ -z "$changed_files" ]; then
33 | echo "No Python source files changed (excluding tests/)"
34 | exit 0
35 | fi
36 |
37 | echo "Files to generate tests for:"
38 | echo "$changed_files"
39 | echo ""
40 |
41 | tests_generated=0
42 |
43 | # Process files safely (handle spaces in filenames)
44 | echo "$changed_files" | while IFS= read -r file; do
45 | if [ -z "$file" ]; then
46 | continue
47 | fi
48 |
49 | if [ ! -f "$file" ]; then
50 | echo "Skipping $file (not found in working directory)"
51 | continue
52 | fi
53 |
54 | echo "=== Processing: $file ==="
55 |
56 | # Extract basename for temp files
57 | base_name=$(basename "$file" .py)
58 |
59 | # Determine test file path (mirror source structure)
60 | # e.g., src/api/utils.py -> tests/api/test_utils.py
61 | test_dir="tests/$(dirname "${file#src/}")"
62 | mkdir -p "$test_dir"
63 | test_file="$test_dir/test_$(basename "$file")"
64 |
65 | if [ -f "$test_file" ]; then
66 | echo "Test file exists: $test_file"
67 | echo "Suggesting additional test cases..."
68 |
69 | # Read existing tests
70 | existing_tests=$(cat "$test_file")
71 |
72 | # Read source code
73 | source_code=$(cat "$file")
74 |
75 | # Generate additional tests
76 | additional_tests=$(gemini "Existing pytest test file:
77 | \`\`\`python
78 | $existing_tests
79 | \`\`\`
80 |
81 | Source code with new/changed functionality:
82 | \`\`\`python
83 | $source_code
84 | \`\`\`
85 |
86 | Task: Suggest additional pytest test functions to cover new/changed code that isn't already tested.
87 |
88 | Requirements:
89 | - Use pytest framework
90 | - Include async tests if source has async functions
91 | - Test happy paths and edge cases
92 | - Test error handling
93 | - Follow existing test style
94 | - Output ONLY the new test functions (no imports, no existing tests)
95 |
96 | Format: Complete Python test functions ready to append.")
97 |
98 | # Use mktemp for output file
99 | output_file=$(mktemp -t test_additions_${base_name}.XXXXXX)
100 | echo "$additional_tests" > "$output_file"
101 |
102 | echo "Additional tests generated: $output_file"
103 | echo ""
104 | echo "--- Preview ---"
105 | head -20 "$output_file"
106 | echo "..."
107 | echo "--- End Preview ---"
108 | echo ""
109 | echo "To append: cat $output_file >> $test_file"
110 |
111 | else
112 | echo "Creating new test file: $test_file"
113 |
114 | # Read source code
115 | source_code=$(cat "$file")
116 |
117 | # Generate complete test file
118 | new_tests=$(gemini "Generate comprehensive pytest tests for this Python module:
119 |
120 | \`\`\`python
121 | $source_code
122 | \`\`\`
123 |
124 | Requirements:
125 | - Complete pytest test file with imports
126 | - Test all public functions/methods
127 | - Include happy paths and edge cases
128 | - Test error handling and validation
129 | - Use pytest fixtures if appropriate
130 | - Include async tests for async functions
131 | - Follow pytest best practices
132 | - Add docstrings to test functions
133 |
134 | Format: Complete, ready-to-use Python test file.")
135 |
136 | # Use mktemp for output file
137 | output_file=$(mktemp -t test_new_${base_name}.XXXXXX)
138 | echo "$new_tests" > "$output_file"
139 |
140 | echo "New test file generated: $output_file"
141 | echo ""
142 | echo "--- Preview ---"
143 | head -30 "$output_file"
144 | echo "..."
145 | echo "--- End Preview ---"
146 | echo ""
147 | echo "To create: cp $output_file $test_file"
148 | fi
149 |
150 | tests_generated=$((tests_generated + 1))
151 | echo ""
152 | done
153 |
154 | echo "=== Test Generation Complete ==="
155 | echo "Files processed: $tests_generated"
156 | echo ""
157 | echo "Generated test files are in /tmp/"
158 | echo "Review and apply manually with the commands shown above."
159 | echo ""
160 | echo "After applying tests:"
161 | echo "1. Run: pytest $test_file"
162 | echo "2. Verify tests pass"
163 | echo "3. Commit: git add $test_file && git commit -m 'test: add tests for <feature>'"
164 |
```
--------------------------------------------------------------------------------
/.github/workflows/README_OPTIMIZATION.md:
--------------------------------------------------------------------------------
```markdown
1 | # GitHub Actions Optimization Guide
2 |
3 | ## Performance Issues Identified
4 |
5 | The current GitHub Actions setup takes ~33 minutes for releases due to:
6 |
7 | 1. **Redundant workflows** - Multiple workflows building the same Docker images
8 | 2. **Sequential platform builds** - Building linux/amd64 and linux/arm64 one after another
9 | 3. **Poor caching** - Not utilizing registry-based caching effectively
10 | 4. **Duplicate test runs** - Same tests running in multiple workflows
11 |
12 | ## Optimizations Implemented
13 |
14 | ### 1. New Consolidated Workflows
15 |
16 | - **`release-tag.yml`** - Replaces both `docker-publish.yml` and `publish-and-test.yml`
17 | - Uses matrix strategy for parallel platform builds
18 | - Implements registry-based caching
19 | - Builds platforms in parallel (2x faster)
20 | - Single test run shared across all jobs
21 |
22 | - **`main-optimized.yml`** - Optimized version of `main.yml`
23 | - Parallel test execution with matrix strategy
24 | - Shared Docker test build
25 | - Registry-based caching with GHCR
26 | - Conditional publishing only after successful release
27 |
28 | ### 2. Key Improvements
29 |
30 | #### Matrix Strategy for Parallel Builds
31 | ```yaml
32 | strategy:
33 | matrix:
34 | platform: [linux/amd64, linux/arm64]
35 | variant: [standard, slim]
36 | ```
37 | This runs 4 builds in parallel instead of sequentially.
38 |
39 | #### Registry-Based Caching
40 | ```yaml
41 | cache-from: |
42 | type=registry,ref=ghcr.io/doobidoo/mcp-memory-service:buildcache-${{ matrix.variant }}-${{ matrix.platform }}
43 | cache-to: |
44 | type=registry,ref=ghcr.io/doobidoo/mcp-memory-service:buildcache-${{ matrix.variant }}-${{ matrix.platform }},mode=max
45 | ```
46 | Uses GHCR as a cache registry for better cross-workflow cache reuse.
47 |
48 | #### Build Once, Push Everywhere
49 | - Builds images once with digests
50 | - Creates multi-platform manifests separately
51 | - Pushes to multiple registries without rebuilding
52 |
53 | ### 3. Migration Steps
54 |
55 | To use the optimized workflows:
56 |
57 | 1. **Test the new workflows first**:
58 | ```bash
59 | # Create a test branch
60 | git checkout -b test-optimized-workflows
61 |
62 | # Temporarily disable old workflows
63 | mv .github/workflows/docker-publish.yml .github/workflows/docker-publish.yml.bak
64 | mv .github/workflows/publish-and-test.yml .github/workflows/publish-and-test.yml.bak
65 | mv .github/workflows/main.yml .github/workflows/main.yml.bak
66 |
67 | # Rename optimized workflows
68 | mv .github/workflows/release-tag.yml .github/workflows/release-tag.yml
69 | mv .github/workflows/main-optimized.yml .github/workflows/main.yml
70 |
71 | # Push and test
72 | git add .
73 | git commit -m "test: optimized workflows"
74 | git push origin test-optimized-workflows
75 | ```
76 |
77 | 2. **Monitor the test run** to ensure everything works correctly
78 |
79 | 3. **If successful, merge to main**:
80 | ```bash
81 | git checkout main
82 | git merge test-optimized-workflows
83 | git push origin main
84 | ```
85 |
86 | 4. **Clean up old workflows**:
87 | ```bash
88 | rm .github/workflows/*.bak
89 | ```
90 |
91 | ### 4. Expected Performance Improvements
92 |
93 | | Metric | Before | After | Improvement |
94 | |--------|--------|-------|-------------|
95 | | Total Build Time | ~33 minutes | ~12-15 minutes | 55-60% faster |
96 | | Docker Builds | Sequential | Parallel (4x) | 4x faster |
97 | | Cache Hit Rate | ~30% | ~80% | 2.6x better |
98 | | Test Runs | 3x redundant | 1x shared | 66% reduction |
99 | | GitHub Actions Minutes | ~150 min/release | ~60 min/release | 60% cost reduction |
100 |
101 | ### 5. Additional Optimizations to Consider
102 |
103 | 1. **Use merge queues** for main branch to batch CI runs
104 | 2. **Implement path filtering** to skip workflows when only docs change
105 | 3. **Use larger runners** for critical jobs (2x-4x faster but costs more)
106 | 4. **Pre-build base images** weekly with all dependencies
107 | 5. **Implement incremental testing** based on changed files
108 |
109 | ### 6. Monitoring
110 |
111 | After implementing these changes, monitor:
112 | - Workflow run times in Actions tab
113 | - Cache hit rates in build logs
114 | - Failed builds due to caching issues
115 | - Registry storage usage (GHCR has limits)
116 |
117 | ### 7. Rollback Plan
118 |
119 | If issues occur, quickly rollback:
120 | ```bash
121 | # Restore original workflows
122 | git checkout main -- .github/workflows/main.yml
123 | git checkout main -- .github/workflows/docker-publish.yml
124 | git checkout main -- .github/workflows/publish-and-test.yml
125 |
126 | # Remove optimized versions
127 | rm .github/workflows/release-tag.yml
128 | rm .github/workflows/main-optimized.yml
129 |
130 | # Commit and push
131 | git commit -m "revert: rollback to original workflows"
132 | git push origin main
133 | ```
```
--------------------------------------------------------------------------------
/scripts/utils/generate_personalized_claude_md.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 |
3 | # Generate personalized CLAUDE.md with memory context for local network distribution
4 | # Usage: ./generate_personalized_claude_md.sh [target_machine_ip] [output_file]
5 |
6 | TARGET_IP="${1:-10.0.1.30}"
7 | OUTPUT_FILE="${2:-CLAUDE_PERSONALIZED.md}"
8 | MCP_ENDPOINT="https://${TARGET_IP}:8443/mcp"
9 | API_KEY="test-key-123"
10 |
11 | echo "Generating personalized CLAUDE.md for network distribution..."
12 | echo "Target: $TARGET_IP"
13 | echo "Output: $OUTPUT_FILE"
14 |
15 | # Create the personalized CLAUDE.md with embedded memory context
16 | cat > "$OUTPUT_FILE" << 'EOF'
17 | # CLAUDE.md - Personalized with Memory Context
18 |
19 | This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
20 | **This version includes pre-loaded memory context from your local MCP Memory Service.**
21 |
22 | ## Memory Context Integration
23 |
24 | Your local memory service contains essential project context. Here's the prompt to retrieve it:
25 |
26 | ```
27 | Load MCP Memory Service context for this project. Retrieve all memories tagged with 'claude-code-reference' and 'distributable-reference' from the following endpoint and incorporate the knowledge into your understanding of this codebase:
28 |
29 | Memory Service: https://TARGET_IP:8443/mcp
30 | Authorization: Bearer test-key-123
31 |
32 | Use this command to fetch context:
33 | curl -k -s -X POST https://TARGET_IP:8443/mcp \
34 | -H "Content-Type: application/json" \
35 | -H "Authorization: Bearer test-key-123" \
36 | -d '{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "retrieve_memory", "arguments": {"query": "claude-code-reference distributable-reference", "limit": 20}}}' \
37 | | jq -r '.result.content[0].text'
38 |
39 | This will provide you with:
40 | - Project structure and architecture details
41 | - Key commands for development, testing, and deployment
42 | - Environment variables and configuration options
43 | - Recent changes including v5.0.2 ONNX implementation
44 | - Issue management patterns and current status
45 | - Testing practices and platform-specific optimizations
46 | - Remote service deployment information
47 |
48 | After loading this context, you'll have comprehensive knowledge of the MCP Memory Service project without needing to discover the codebase structure through file reading.
49 | ```
50 |
51 | ## Quick Memory Retrieval Commands
52 |
53 | If memory context fails to load automatically, use these commands:
54 |
55 | ### Get All Project Context
56 | ```bash
57 | curl -k -s -X POST https://TARGET_IP:8443/mcp \
58 | -H "Content-Type: application/json" \
59 | -H "Authorization: Bearer test-key-123" \
60 | -d '{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "retrieve_memory", "arguments": {"query": "claude-code-reference", "limit": 20}}}' \
61 | | jq -r '.result.content[0].text'
62 | ```
63 |
64 | ### Check Memory Service Health
65 | ```bash
66 | curl -k -s -X POST https://TARGET_IP:8443/mcp \
67 | -H "Content-Type: application/json" \
68 | -H "Authorization: Bearer test-key-123" \
69 | -d '{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "check_database_health", "arguments": {}}}' \
70 | | jq -r '.result.content[0].text'
71 | ```
72 |
73 | ## Memory Categories Available
74 | - **Project Structure**: Server architecture, file locations, component relationships
75 | - **Key Commands**: Installation, testing, debugging, deployment commands
76 | - **Environment Variables**: Configuration options and platform-specific settings
77 | - **Recent Changes**: Version history, resolved issues, breaking changes
78 | - **Testing Practices**: Framework preferences, test patterns, validation steps
79 | - **Current Status**: Active issues, recent work, development context
80 |
81 | EOF
82 |
83 | # Replace TARGET_IP placeholder with actual IP
84 | sed -i "s/TARGET_IP/$TARGET_IP/g" "$OUTPUT_FILE"
85 |
86 | # Append the original CLAUDE.md content (without the memory section)
87 | echo "" >> "$OUTPUT_FILE"
88 | echo "## Original Project Documentation" >> "$OUTPUT_FILE"
89 | echo "" >> "$OUTPUT_FILE"
90 |
91 | # Extract content from original CLAUDE.md starting after memory section
92 | awk '/^## Overview/{print; getline; while(getline > 0) print}' CLAUDE.md >> "$OUTPUT_FILE"
93 |
94 | echo "✅ Personalized CLAUDE.md generated: $OUTPUT_FILE"
95 | echo ""
96 | echo "Distribution instructions:"
97 | echo "1. Copy $OUTPUT_FILE to target machines as CLAUDE.md"
98 | echo "2. Ensure target machines can access https://$TARGET_IP:8443"
99 | echo "3. Claude Code will automatically use memory context on those machines"
100 | echo ""
101 | echo "Network test command:"
102 | echo "curl -k -s https://$TARGET_IP:8443/api/health"
```
--------------------------------------------------------------------------------
/claude_commands/memory-search.md:
--------------------------------------------------------------------------------
```markdown
1 | # Search Memories by Tags and Content
2 |
3 | I'll help you search through your stored memories using tags, content keywords, and semantic similarity. This command is perfect for finding specific information across all your stored memories regardless of when they were created.
4 |
5 | ## What I'll do:
6 |
7 | 1. **Tag-Based Search**: I'll search for memories associated with specific tags, supporting both exact and partial tag matching.
8 |
9 | 2. **Content Search**: I'll perform semantic search across memory content using the same embedding model used for storage.
10 |
11 | 3. **Combined Queries**: I'll support complex searches combining tags, content, and metadata filters.
12 |
13 | 4. **Smart Ranking**: I'll rank results by relevance, considering both semantic similarity and tag match strength.
14 |
15 | 5. **Context Integration**: I'll highlight how found memories relate to your current project and session.
16 |
17 | ## Usage Examples:
18 |
19 | ```bash
20 | claude /memory-search --tags "architecture,database"
21 | claude /memory-search "SQLite performance optimization"
22 | claude /memory-search --tags "decision" --content "database backend"
23 | claude /memory-search --project "mcp-memory-service" --type "note"
24 | ```
25 |
26 | ## Implementation:
27 |
28 | I'll connect to your MCP Memory Service at `https://memory.local:8443/` and use its search API endpoints:
29 |
30 | 1. **Query Processing**: Parse your search criteria (tags, content, filters)
31 | 2. **Search Execution**: Use appropriate API endpoints:
32 | - `POST /api/search` - Semantic similarity search
33 | - `POST /api/search/by-tag` - Tag-based search (AND/OR matching)
34 | - `POST /api/search/by-time` - Time-based natural language queries
35 | - `GET /api/search/similar/{hash}` - Find similar memories
36 | 3. **Result Aggregation**: Process search responses with similarity scores
37 | 4. **Relevance Scoring**: Use returned similarity scores and match reasons
38 | 5. **Context Highlighting**: Show why each result matches your query
39 |
40 | All requests use curl with `-k` flag for HTTPS and proper JSON formatting.
41 |
42 | For each search result, I'll display:
43 | - **Content**: The memory content with search terms highlighted
44 | - **Tags**: All associated tags (with matching tags emphasized)
45 | - **Relevance Score**: How closely the memory matches your query
46 | - **Created Date**: When the memory was stored
47 | - **Project Context**: Associated project and file context
48 | - **Memory Type**: Classification (note, decision, task, etc.)
49 |
50 | ## Search Types:
51 |
52 | ### Tag Search
53 | - **Exact**: `--tags "architecture"` - memories with exact tag match
54 | - **Multiple**: `--tags "database,performance"` - memories with any of these tags
55 | - **Machine Source**: `--tags "source:machine-name"` - memories from specific machine
56 | - **Partial**: `--tags "*arch*"` - memories with tags containing "arch"
57 |
58 | ### Content Search
59 | - **Semantic**: Content-based similarity using embeddings
60 | - **Keyword**: Simple text matching within memory content
61 | - **Combined**: Both semantic and keyword matching
62 |
63 | ### Filtered Search
64 | - **Project**: `--project "name"` - memories from specific project
65 | - **Type**: `--type "decision"` - memories of specific type
66 | - **Date Range**: `--since "last week"` - memories within time range
67 | - **Author**: `--author "session"` - memories from specific session
68 |
69 | ## Arguments:
70 |
71 | - `$ARGUMENTS` - The search query (content or primary search terms)
72 | - `--tags "tag1,tag2"` - Search by specific tags
73 | - `--content "text"` - Explicit content search terms
74 | - `--project "name"` - Filter by project name
75 | - `--type "note|decision|task|reference"` - Filter by memory type
76 | - `--limit N` - Maximum results to return (default: 20)
77 | - `--min-score 0.X` - Minimum relevance score threshold
78 | - `--include-metadata` - Show full metadata for each result
79 | - `--export` - Export results to a file for review
80 |
81 | ## Advanced Features:
82 |
83 | - **Machine Source Tracking**: Search for memories by originating machine
84 | - **Fuzzy Matching**: Handle typos and variations in search terms
85 | - **Context Expansion**: Find related memories based on current project context
86 | - **Search History**: Remember recent searches for quick re-execution
87 | - **Result Grouping**: Group results by tags, projects, or time periods
88 |
89 | If no results are found, I'll suggest alternative search terms, check for typos, or recommend broadening the search criteria. I'll also provide statistics about the total number of memories in your database and suggest ways to improve future searches.
```
--------------------------------------------------------------------------------
/scripts/service/memory_service_manager.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # Memory Service Manager for Claude Code on Linux
3 | # Manages dual backend setup with Cloudflare primary and SQLite-vec backup
4 |
5 | SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
6 | PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
7 |
8 | # Service configuration
9 | CLOUDFLARE_ENV="$PROJECT_DIR/.env"
10 | SQLITE_ENV="$PROJECT_DIR/.env.sqlite"
11 |
12 | # Create SQLite-vec environment file if it doesn't exist
13 | if [ ! -f "$SQLITE_ENV" ]; then
14 | cat > "$SQLITE_ENV" << EOF
15 | # SQLite-vec Configuration for MCP Memory Service (Backup)
16 | MCP_MEMORY_STORAGE_BACKEND=sqlite_vec
17 | MCP_MEMORY_SQLITE_PATH="$HOME/.local/share/mcp-memory/primary_sqlite_vec.db"
18 | EOF
19 | echo "Created SQLite-vec environment configuration: $SQLITE_ENV"
20 | fi
21 |
22 | usage() {
23 | echo "Memory Service Manager for Claude Code"
24 | echo ""
25 | echo "Usage: $0 <command>"
26 | echo ""
27 | echo "Commands:"
28 | echo " start-cloudflare Start memory server with Cloudflare backend"
29 | echo " start-sqlite Start memory server with SQLite-vec backend"
30 | echo " status Show current backend and sync status"
31 | echo " sync-backup Backup Cloudflare → SQLite-vec"
32 | echo " sync-restore Restore SQLite-vec → Cloudflare"
33 | echo " sync-both Bidirectional sync"
34 | echo " stop Stop any running memory server"
35 | echo ""
36 | }
37 |
38 | start_memory_service() {
39 | local backend="$1"
40 | local env_file="$2"
41 |
42 | echo "Starting memory service with $backend backend..."
43 |
44 | # Stop any existing service
45 | pkill -f "memory server" 2>/dev/null || true
46 | sleep 2
47 |
48 | # Start new service
49 | cd "$PROJECT_DIR"
50 | if [ -f "$env_file" ]; then
51 | echo "Loading environment from: $env_file"
52 | set -a
53 | source "$env_file"
54 | set +a
55 | fi
56 |
57 | echo "Starting: uv run memory server"
58 | nohup uv run memory server > /tmp/memory-service-$backend.log 2>&1 &
59 |
60 | # Wait a moment and check if it started
61 | sleep 3
62 | if pgrep -f "memory server" > /dev/null; then
63 | echo "Memory service started successfully with $backend backend"
64 | echo "Logs: /tmp/memory-service-$backend.log"
65 |
66 | # Save active backend to state file for reliable detection
67 | echo "$backend" > /tmp/memory-service-backend.state
68 | else
69 | echo "Failed to start memory service"
70 | echo "Check logs: /tmp/memory-service-$backend.log"
71 | return 1
72 | fi
73 | }
74 |
75 | show_status() {
76 | echo "=== Memory Service Status ==="
77 |
78 | # Check if service is running
79 | if pgrep -f "memory server" > /dev/null; then
80 | echo "Service: Running"
81 |
82 | # Check which backend is active using state file
83 | if [ -f "/tmp/memory-service-backend.state" ]; then
84 | local active_backend=$(cat /tmp/memory-service-backend.state)
85 | echo "Active Backend: $active_backend (from state file)"
86 | else
87 | echo "Active Backend: Unknown (no state file found)"
88 | fi
89 | else
90 | echo "Service: Not running"
91 | # Clean up state file if service is not running
92 | [ -f "/tmp/memory-service-backend.state" ] && rm -f /tmp/memory-service-backend.state
93 | fi
94 |
95 | echo ""
96 | echo "=== Sync Status ==="
97 | cd "$PROJECT_DIR"
98 | uv run python scripts/claude_sync_commands.py status
99 | }
100 |
101 | sync_memories() {
102 | local direction="$1"
103 | echo "Syncing memories: $direction"
104 | cd "$PROJECT_DIR"
105 | uv run python scripts/claude_sync_commands.py "$direction"
106 | }
107 |
108 | stop_service() {
109 | echo "Stopping memory service..."
110 | pkill -f "memory server" 2>/dev/null || true
111 | sleep 2
112 | if ! pgrep -f "memory server" > /dev/null; then
113 | echo "Memory service stopped"
114 | # Clean up state file when service is stopped
115 | [ -f "/tmp/memory-service-backend.state" ] && rm -f /tmp/memory-service-backend.state
116 | else
117 | echo "Failed to stop memory service"
118 | return 1
119 | fi
120 | }
121 |
122 | # Main command handling
123 | case "$1" in
124 | start-cloudflare)
125 | start_memory_service "cloudflare" "$CLOUDFLARE_ENV"
126 | ;;
127 | start-sqlite)
128 | start_memory_service "sqlite" "$SQLITE_ENV"
129 | ;;
130 | status)
131 | show_status
132 | ;;
133 | sync-backup)
134 | sync_memories "backup"
135 | ;;
136 | sync-restore)
137 | sync_memories "restore"
138 | ;;
139 | sync-both)
140 | sync_memories "sync"
141 | ;;
142 | stop)
143 | stop_service
144 | ;;
145 | *)
146 | usage
147 | exit 1
148 | ;;
149 | esac
150 |
```
--------------------------------------------------------------------------------
/scripts/testing/test_cleanup_logic.py:
--------------------------------------------------------------------------------
```python
1 | #!/usr/bin/env python3
2 | """
3 | Test script for Docker Hub cleanup logic
4 | Tests the retention policy rules without actual API calls
5 | """
6 |
7 | import re
8 | from datetime import datetime, timedelta, timezone
9 |
10 | def should_keep_tag(tag_name, tag_date, keep_versions=5, cutoff_date=None):
11 | """Test version of the retention policy logic"""
12 | if cutoff_date is None:
13 | cutoff_date = datetime.now(timezone.utc) - timedelta(days=30)
14 |
15 | # Always keep these tags
16 | protected_tags = ["latest", "slim", "main", "stable"]
17 | if tag_name in protected_tags:
18 | return True, "Protected tag"
19 |
20 | # Keep semantic version tags (v1.2.3)
21 | if re.match(r'^v?\d+\.\d+\.\d+$', tag_name):
22 | return True, "Semantic version"
23 |
24 | # Keep major.minor tags (1.0, 2.1)
25 | if re.match(r'^v?\d+\.\d+$', tag_name):
26 | return True, "Major.minor version"
27 |
28 | # Delete buildcache tags older than cutoff
29 | if tag_name.startswith("buildcache-"):
30 | if tag_date < cutoff_date:
31 | return False, "Old buildcache tag"
32 | return True, "Recent buildcache tag"
33 |
34 | # Delete sha/digest tags older than cutoff
35 | if tag_name.startswith("sha256-") or (len(tag_name) == 7 and tag_name.isalnum()):
36 | if tag_date < cutoff_date:
37 | return False, "Old sha/digest tag"
38 | return True, "Recent sha/digest tag"
39 |
40 | # Delete test/dev tags older than cutoff
41 | if any(x in tag_name.lower() for x in ["test", "dev", "tmp", "temp"]):
42 | if tag_date < cutoff_date:
43 | return False, "Old test/dev tag"
44 | return True, "Recent test/dev tag"
45 |
46 | # Keep if recent
47 | if tag_date >= cutoff_date:
48 | return True, "Recent tag"
49 |
50 | return False, "Old tag"
51 |
52 | def test_retention_policy():
53 | """Test various tag scenarios"""
54 | now = datetime.now(timezone.utc)
55 | old_date = now - timedelta(days=40)
56 | recent_date = now - timedelta(days=10)
57 | cutoff = now - timedelta(days=30)
58 |
59 | test_cases = [
60 | # (tag_name, tag_date, expected_keep, expected_reason)
61 | ("latest", old_date, True, "Protected tag"),
62 | ("slim", old_date, True, "Protected tag"),
63 | ("main", old_date, True, "Protected tag"),
64 | ("stable", old_date, True, "Protected tag"),
65 |
66 | ("v6.6.0", old_date, True, "Semantic version"),
67 | ("6.6.0", old_date, True, "Semantic version"),
68 | ("v6.6", old_date, True, "Major.minor version"),
69 | ("6.6", old_date, True, "Major.minor version"),
70 |
71 | ("buildcache-linux-amd64", old_date, False, "Old buildcache tag"),
72 | ("buildcache-linux-amd64", recent_date, True, "Recent buildcache tag"),
73 |
74 | ("sha256-abc123", old_date, False, "Old sha/digest tag"),
75 | ("abc1234", old_date, False, "Old sha/digest tag"),
76 | ("sha256-abc123", recent_date, True, "Recent sha/digest tag"),
77 |
78 | ("test-feature", old_date, False, "Old test/dev tag"),
79 | ("dev-branch", old_date, False, "Old test/dev tag"),
80 | ("tmp-build", recent_date, True, "Recent test/dev tag"),
81 |
82 | ("feature-xyz", old_date, False, "Old tag"),
83 | ("feature-xyz", recent_date, True, "Recent tag"),
84 | ]
85 |
86 | print("Testing Docker Hub Cleanup Retention Policy")
87 | print("=" * 60)
88 | print(f"Cutoff date: {cutoff.strftime('%Y-%m-%d')}")
89 | print()
90 |
91 | passed = 0
92 | failed = 0
93 |
94 | for tag_name, tag_date, expected_keep, expected_reason in test_cases:
95 | should_keep, reason = should_keep_tag(tag_name, tag_date, cutoff_date=cutoff)
96 |
97 | # Format date for display
98 | date_str = tag_date.strftime('%Y-%m-%d')
99 | days_old = (now - tag_date).days
100 |
101 | # Check if test passed
102 | if should_keep == expected_keep and reason == expected_reason:
103 | status = "✓ PASS"
104 | passed += 1
105 | else:
106 | status = "✗ FAIL"
107 | failed += 1
108 |
109 | # Print result
110 | action = "KEEP" if should_keep else "DELETE"
111 | print(f"{status}: {tag_name:30} ({days_old:3}d old) -> {action:6} ({reason})")
112 |
113 | if status == "✗ FAIL":
114 | print(f" Expected: {action:6} ({expected_reason})")
115 |
116 | print()
117 | print("=" * 60)
118 | print(f"Results: {passed} passed, {failed} failed")
119 |
120 | return failed == 0
121 |
122 | if __name__ == "__main__":
123 | success = test_retention_policy()
124 | exit(0 if success else 1)
```
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/bug_report.yml:
--------------------------------------------------------------------------------
```yaml
1 | name: Bug Report
2 | description: Report a bug or unexpected behavior
3 | title: "[Bug]: "
4 | labels: ["bug", "triage"]
5 | body:
6 | - type: markdown
7 | attributes:
8 | value: |
9 | Thank you for reporting a bug! Please fill out the sections below to help us diagnose and fix the issue.
10 |
11 | - type: textarea
12 | id: description
13 | attributes:
14 | label: Bug Description
15 | description: A clear and concise description of what the bug is.
16 | placeholder: What happened?
17 | validations:
18 | required: true
19 |
20 | - type: textarea
21 | id: steps
22 | attributes:
23 | label: Steps to Reproduce
24 | description: Detailed steps to reproduce the behavior
25 | placeholder: |
26 | 1. Configure storage backend as...
27 | 2. Run command...
28 | 3. Observe error...
29 | value: |
30 | 1.
31 | 2.
32 | 3.
33 | validations:
34 | required: true
35 |
36 | - type: textarea
37 | id: expected
38 | attributes:
39 | label: Expected Behavior
40 | description: What you expected to happen
41 | placeholder: The memory should be stored successfully...
42 | validations:
43 | required: true
44 |
45 | - type: textarea
46 | id: actual
47 | attributes:
48 | label: Actual Behavior
49 | description: What actually happened (include error messages)
50 | placeholder: |
51 | Error: database is locked
52 | Traceback...
53 | validations:
54 | required: true
55 |
56 | - type: dropdown
57 | id: storage-backend
58 | attributes:
59 | label: Storage Backend
60 | description: Which storage backend are you using?
61 | options:
62 | - sqlite-vec (local)
63 | - cloudflare (remote)
64 | - hybrid (sqlite + cloudflare)
65 | - unsure
66 | validations:
67 | required: true
68 |
69 | - type: dropdown
70 | id: os
71 | attributes:
72 | label: Operating System
73 | options:
74 | - macOS
75 | - Windows
76 | - Linux
77 | - Docker
78 | - Other
79 | validations:
80 | required: true
81 |
82 | - type: input
83 | id: python-version
84 | attributes:
85 | label: Python Version
86 | description: Output of `python --version`
87 | placeholder: "Python 3.11.5"
88 | validations:
89 | required: true
90 |
91 | - type: input
92 | id: mcp-version
93 | attributes:
94 | label: MCP Memory Service Version
95 | description: Output of `uv run memory --version` or check `pyproject.toml`
96 | placeholder: "v8.17.0"
97 | validations:
98 | required: true
99 |
100 | - type: dropdown
101 | id: installation
102 | attributes:
103 | label: Installation Method
104 | options:
105 | - Source (git clone)
106 | - pip/uv install
107 | - Docker
108 | - Other
109 | validations:
110 | required: true
111 |
112 | - type: dropdown
113 | id: interface
114 | attributes:
115 | label: Interface Used
116 | description: How are you accessing the memory service?
117 | options:
118 | - Claude Desktop (MCP)
119 | - Claude Code (CLI)
120 | - HTTP API (dashboard)
121 | - Python API (direct import)
122 | - Other
123 | validations:
124 | required: true
125 |
126 | - type: textarea
127 | id: config
128 | attributes:
129 | label: Configuration
130 | description: Relevant parts of your `.env` file or Claude Desktop config (redact API keys)
131 | placeholder: |
132 | MCP_MEMORY_STORAGE_BACKEND=hybrid
133 | MCP_HTTP_ENABLED=true
134 | # Cloudflare credentials redacted
135 | render: shell
136 |
137 | - type: textarea
138 | id: logs
139 | attributes:
140 | label: Relevant Log Output
141 | description: Logs from server, MCP client, or error messages
142 | placeholder: Paste relevant logs here
143 | render: shell
144 |
145 | - type: textarea
146 | id: context
147 | attributes:
148 | label: Additional Context
149 | description: |
150 | Any other context about the problem:
151 | - Recent changes or upgrades
152 | - Concurrent usage (multiple clients)
153 | - Network conditions (if using remote backend)
154 | - Screenshots (if dashboard issue)
155 | placeholder: Any additional information that might help...
156 |
157 | - type: checkboxes
158 | id: checks
159 | attributes:
160 | label: Pre-submission Checklist
161 | description: Please verify you've completed these steps
162 | options:
163 | - label: I've searched existing issues and this is not a duplicate
164 | required: true
165 | - label: I'm using the latest version (or specified version above)
166 | required: true
167 | - label: I've included all required environment information
168 | required: true
169 | - label: I've redacted sensitive information (API keys, tokens)
170 | required: true
171 |
```
--------------------------------------------------------------------------------
/tests/integration/test_api_key_fallback.py:
--------------------------------------------------------------------------------
```python
1 | #!/usr/bin/env python3
2 | """
3 | Test script to verify API key authentication fallback works with OAuth enabled.
4 |
5 | This test verifies that existing API key authentication continues to work
6 | when OAuth is enabled, ensuring backward compatibility.
7 | """
8 |
9 | import asyncio
10 | import sys
11 | from pathlib import Path
12 |
13 | import httpx
14 |
15 | # Add src to path for standalone execution
16 | sys.path.insert(0, str(Path(__file__).parent.parent.parent / 'src'))
17 |
18 |
19 | async def test_api_key_fallback(base_url: str = "http://localhost:8000", api_key: str = None) -> bool:
20 | """
21 | Test API key authentication fallback with OAuth enabled.
22 |
23 | Returns:
24 | True if all tests pass, False otherwise
25 | """
26 | print(f"Testing API key fallback at {base_url}")
27 | print("=" * 50)
28 |
29 | if not api_key:
30 | print("❌ No API key provided - cannot test fallback")
31 | print(" Set MCP_API_KEY environment variable or pass as argument")
32 | return False
33 |
34 | async with httpx.AsyncClient() as client:
35 | try:
36 | # Test 1: API Key as Bearer Token (should work)
37 | print("1. Testing API Key as Bearer Token...")
38 |
39 | headers = {"Authorization": f"Bearer {api_key}"}
40 | response = await client.get(f"{base_url}/api/memories", headers=headers)
41 |
42 | if response.status_code == 200:
43 | print(f" ✅ API key authentication working")
44 | else:
45 | print(f" ❌ API key authentication failed: {response.status_code}")
46 | return False
47 |
48 | # Test 2: API Key for write operations
49 | print("\n2. Testing API Key for Write Operations...")
50 |
51 | memory_data = {
52 | "content": "Test memory for API key authentication",
53 | "tags": ["test", "api-key"],
54 | "memory_type": "test"
55 | }
56 |
57 | response = await client.post(f"{base_url}/api/memories", json=memory_data, headers=headers)
58 |
59 | if response.status_code == 200:
60 | print(f" ✅ API key write operation working")
61 | # Store content hash for cleanup
62 | memory_hash = response.json().get("content_hash")
63 | else:
64 | print(f" ❌ API key write operation failed: {response.status_code}")
65 | return False
66 |
67 | # Test 3: Invalid API Key (should fail)
68 | print("\n3. Testing Invalid API Key...")
69 |
70 | invalid_headers = {"Authorization": "Bearer invalid_key"}
71 | response = await client.get(f"{base_url}/api/memories", headers=invalid_headers)
72 |
73 | if response.status_code == 401:
74 | print(f" ✅ Invalid API key correctly rejected")
75 | else:
76 | print(f" ⚠️ Invalid API key test inconclusive: {response.status_code}")
77 |
78 | # Test 4: Cleanup - Delete test memory
79 | if memory_hash:
80 | print("\n4. Cleaning up test memory...")
81 | response = await client.delete(f"{base_url}/api/memories/{memory_hash}", headers=headers)
82 | if response.status_code == 200:
83 | print(f" ✅ Test memory cleaned up successfully")
84 | else:
85 | print(f" ⚠️ Cleanup failed: {response.status_code}")
86 |
87 | print("\n" + "=" * 50)
88 | print("🎉 API key fallback authentication tests passed!")
89 | print("✅ Backward compatibility maintained")
90 | return True
91 |
92 | except Exception as e:
93 | print(f"\n❌ Test failed with exception: {e}")
94 | return False
95 |
96 |
97 | async def main():
98 | """Main test function."""
99 | if len(sys.argv) > 1:
100 | base_url = sys.argv[1]
101 | else:
102 | base_url = "http://localhost:8000"
103 |
104 | # Try to get API key from command line or environment
105 | api_key = None
106 | if len(sys.argv) > 2:
107 | api_key = sys.argv[2]
108 | else:
109 | import os
110 | api_key = os.getenv('MCP_API_KEY')
111 |
112 | print("API Key Authentication Fallback Test")
113 | print("===================================")
114 | print(f"Target: {base_url}")
115 | print()
116 | print("This test verifies that API key authentication works")
117 | print("as a fallback when OAuth 2.1 is enabled.")
118 | print()
119 |
120 | success = await test_api_key_fallback(base_url, api_key)
121 |
122 | if success:
123 | print("\n🚀 API key fallback is working correctly!")
124 | sys.exit(0)
125 | else:
126 | print("\n💥 API key fallback tests failed")
127 | sys.exit(1)
128 |
129 |
130 | if __name__ == "__main__":
131 | asyncio.run(main())
```
--------------------------------------------------------------------------------
/scripts/sync/safe_cloudflare_update.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # Safe Cloudflare Update Script
3 | # Pushes corrected timestamps from local SQLite to Cloudflare
4 | # Run this AFTER timestamp restoration, BEFORE re-enabling hybrid on other machines
5 |
6 | set -e # Exit on error
7 |
8 | SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
9 | PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
10 |
11 | echo "================================================================================"
12 | echo "SAFE CLOUDFLARE UPDATE - Timestamp Recovery"
13 | echo "================================================================================"
14 | echo ""
15 | echo "This script will:"
16 | echo " 1. Verify local database has correct timestamps"
17 | echo " 2. Push corrected timestamps to Cloudflare"
18 | echo " 3. Verify Cloudflare update success"
19 | echo ""
20 | echo "⚠️ IMPORTANT: Run this BEFORE re-enabling hybrid sync on other machines!"
21 | echo ""
22 |
23 | # Check if we're in the right directory
24 | if [ ! -f "$PROJECT_ROOT/scripts/sync/sync_memory_backends.py" ]; then
25 | echo "❌ ERROR: Cannot find sync script. Are you in the project directory?"
26 | exit 1
27 | fi
28 |
29 | # Step 1: Verify local timestamps
30 | echo "================================================================================"
31 | echo "STEP 1: VERIFYING LOCAL TIMESTAMPS"
32 | echo "================================================================================"
33 | echo ""
34 |
35 | python3 << 'EOF'
36 | import sqlite3
37 | import sys
38 | from pathlib import Path
39 |
40 | # Add src to path
41 | sys.path.insert(0, str(Path(__file__).parent / "src"))
42 |
43 | try:
44 | from mcp_memory_service import config
45 | db_path = config.SQLITE_VEC_PATH
46 | except:
47 | db_path = str(Path.home() / "Library/Application Support/mcp-memory/sqlite_vec.db")
48 |
49 | conn = sqlite3.connect(db_path)
50 | cursor = conn.cursor()
51 |
52 | # Check total memories
53 | cursor.execute('SELECT COUNT(*) FROM memories')
54 | total = cursor.fetchone()[0]
55 |
56 | # Check corruption period (Nov 16-18)
57 | cursor.execute('''
58 | SELECT COUNT(*) FROM memories
59 | WHERE created_at_iso LIKE "2025-11-16%"
60 | OR created_at_iso LIKE "2025-11-17%"
61 | OR created_at_iso LIKE "2025-11-18%"
62 | ''')
63 | corrupted = cursor.fetchone()[0]
64 |
65 | corruption_pct = (corrupted * 100 / total) if total > 0 else 0
66 |
67 | print(f"Database: {db_path}")
68 | print(f"Total memories: {total}")
69 | print(f"Nov 16-18 dates: {corrupted} ({corruption_pct:.1f}%)")
70 | print()
71 |
72 | if corruption_pct < 10:
73 | print("✅ VERIFICATION PASSED: Timestamps look good")
74 | print(" Safe to proceed with Cloudflare update")
75 | conn.close()
76 | sys.exit(0)
77 | else:
78 | print("❌ VERIFICATION FAILED: Too many corrupted timestamps")
79 | print(f" Expected: <10%, Found: {corruption_pct:.1f}%")
80 | print()
81 | print("Run timestamp restoration first:")
82 | print(" python scripts/maintenance/restore_from_json_export.py --apply")
83 | conn.close()
84 | sys.exit(1)
85 | EOF
86 |
87 | if [ $? -ne 0 ]; then
88 | echo ""
89 | echo "❌ Local verification failed. Aborting."
90 | exit 1
91 | fi
92 |
93 | echo ""
94 | read -p "Continue with Cloudflare update? [y/N]: " -n 1 -r
95 | echo ""
96 |
97 | if [[ ! $REPLY =~ ^[Yy]$ ]]; then
98 | echo "Update cancelled."
99 | exit 0
100 | fi
101 |
102 | # Step 2: Push to Cloudflare
103 | echo ""
104 | echo "================================================================================"
105 | echo "STEP 2: PUSHING TO CLOUDFLARE"
106 | echo "================================================================================"
107 | echo ""
108 | echo "This will overwrite Cloudflare timestamps with your corrected local data."
109 | echo "Duration: 5-10 minutes (network dependent)"
110 | echo ""
111 |
112 | cd "$PROJECT_ROOT"
113 | python scripts/sync/sync_memory_backends.py --direction sqlite-to-cf
114 |
115 | if [ $? -ne 0 ]; then
116 | echo ""
117 | echo "❌ Cloudflare sync failed. Check logs above."
118 | exit 1
119 | fi
120 |
121 | # Step 3: Verify Cloudflare
122 | echo ""
123 | echo "================================================================================"
124 | echo "STEP 3: VERIFYING CLOUDFLARE UPDATE"
125 | echo "================================================================================"
126 | echo ""
127 |
128 | python scripts/sync/sync_memory_backends.py --status
129 |
130 | echo ""
131 | echo "================================================================================"
132 | echo "UPDATE COMPLETE ✅"
133 | echo "================================================================================"
134 | echo ""
135 | echo "Next steps:"
136 | echo " 1. Verify status output above shows expected memory counts"
137 | echo " 2. Check other machines are still offline (hybrid disabled)"
138 | echo " 3. When ready, sync other machines FROM Cloudflare:"
139 | echo " python scripts/sync/sync_memory_backends.py --direction cf-to-sqlite"
140 | echo ""
141 | echo "See TIMESTAMP_RECOVERY_CHECKLIST.md for detailed next steps."
142 | echo ""
143 |
```
--------------------------------------------------------------------------------
/tests/sqlite/simple_sqlite_vec_test.py:
--------------------------------------------------------------------------------
```python
1 | #!/usr/bin/env python3
2 | """
3 | Simple standalone test for sqlite-vec functionality.
4 | """
5 |
6 | import asyncio
7 | import os
8 | import tempfile
9 | import sys
10 | import sqlite3
11 |
12 | async def test_sqlite_vec_basic():
13 | """Test basic sqlite-vec functionality."""
14 | print("🔧 Testing basic SQLite-vec functionality...")
15 | print("=" * 50)
16 |
17 | try:
18 | # Test sqlite-vec import
19 | print("1. Testing sqlite-vec import...")
20 | import sqlite_vec
21 | from sqlite_vec import serialize_float32
22 | print(" ✅ sqlite-vec imported successfully")
23 |
24 | # Test basic database operations
25 | print("\n2. Testing database creation...")
26 | temp_dir = tempfile.mkdtemp()
27 | db_path = os.path.join(temp_dir, "test.db")
28 |
29 | conn = sqlite3.connect(db_path)
30 | conn.enable_load_extension(True)
31 | sqlite_vec.load(conn)
32 | print(" ✅ Database created and sqlite-vec loaded")
33 |
34 | # Create test table
35 | print("\n3. Creating test table...")
36 | conn.execute('''
37 | CREATE TABLE test_vectors (
38 | id INTEGER PRIMARY KEY,
39 | content TEXT,
40 | embedding BLOB
41 | )
42 | ''')
43 | print(" ✅ Test table created")
44 |
45 | # Test vector operations
46 | print("\n4. Testing vector operations...")
47 | test_vector = [0.1, 0.2, 0.3, 0.4, 0.5]
48 | serialized = serialize_float32(test_vector)
49 |
50 | conn.execute('''
51 | INSERT INTO test_vectors (content, embedding)
52 | VALUES (?, ?)
53 | ''', ("Test content", serialized))
54 |
55 | conn.commit()
56 | print(" ✅ Vector stored successfully")
57 |
58 | # Test retrieval
59 | print("\n5. Testing retrieval...")
60 | cursor = conn.execute('''
61 | SELECT content, embedding FROM test_vectors WHERE id = 1
62 | ''')
63 | row = cursor.fetchone()
64 |
65 | if row:
66 | content, stored_embedding = row
67 | print(f" Retrieved content: {content}")
68 | print(" ✅ Retrieval successful")
69 |
70 | # Cleanup
71 | conn.close()
72 | os.remove(db_path)
73 | os.rmdir(temp_dir)
74 |
75 | print("\n✅ Basic sqlite-vec test passed!")
76 | print("\n🚀 SQLite-vec is working correctly on your Ubuntu system!")
77 |
78 | return True
79 |
80 | except Exception as e:
81 | print(f" ❌ Test failed: {e}")
82 | import traceback
83 | traceback.print_exc()
84 | return False
85 |
86 | async def show_next_steps():
87 | """Show next steps for integration."""
88 | print("\n" + "=" * 60)
89 | print("🎯 Next Steps for Claude Code + VS Code Integration")
90 | print("=" * 60)
91 |
92 | print("\n1. 📦 Complete MCP Memory Service Setup:")
93 | print(" # Stay in your virtual environment")
94 | print(" source venv/bin/activate")
95 | print()
96 | print(" # Set the backend")
97 | print(" export MCP_MEMORY_STORAGE_BACKEND=sqlite_vec")
98 | print()
99 | print(" # Install remaining MCP dependencies")
100 | print(" pip install mcp")
101 |
102 | print("\n2. 🔧 Configure Claude Code Integration:")
103 | print(" The sqlite-vec backend is now ready!")
104 | print(" Your memory database will be stored at:")
105 | home = os.path.expanduser("~")
106 | print(f" {home}/.local/share/mcp-memory/sqlite_vec.db")
107 |
108 | print("\n3. 💻 For VS Code Integration:")
109 | print(" # Install VS Code MCP extension (if available)")
110 | print(" # Or use Claude Code directly in VS Code terminal")
111 |
112 | print("\n4. 🧪 Test the Setup:")
113 | print(" # Test that MCP Memory Service works with sqlite-vec")
114 | print(" python -c \"")
115 | print(" import os")
116 | print(" os.environ['MCP_MEMORY_STORAGE_BACKEND'] = 'sqlite_vec'")
117 | print(" # Your memory operations will now use sqlite-vec!")
118 | print(" \"")
119 |
120 | print("\n5. 🔄 Migration (if you have existing ChromaDB data):")
121 | print(" python migrate_to_sqlite_vec.py")
122 |
123 | print("\n✨ Benefits of SQLite-vec:")
124 | print(" • 75% less memory usage")
125 | print(" • Single file database (easy backup)")
126 | print(" • Faster startup times")
127 | print(" • Better for <100K memories")
128 |
129 | async def main():
130 | """Main test function."""
131 | success = await test_sqlite_vec_basic()
132 |
133 | if success:
134 | await show_next_steps()
135 | return 0
136 | else:
137 | print("\n❌ sqlite-vec test failed. Please install sqlite-vec:")
138 | print(" pip install sqlite-vec")
139 | return 1
140 |
141 | if __name__ == "__main__":
142 | sys.exit(asyncio.run(main()))
```
--------------------------------------------------------------------------------
/docs/DOCUMENTATION_AUDIT.md:
--------------------------------------------------------------------------------
```markdown
1 | # Documentation Audit Report
2 | **Date**: 2025-07-26
3 | **Branch**: feature/http-sse-sqlite-vec
4 | **Purpose**: Consolidation analysis for unified installer merge
5 |
6 | ## Current Documentation Inventory
7 |
8 | ### Installation-Related Documentation
9 | - `README.md` (root) - Main installation instructions, needs backend choice integration
10 | - `docs/guides/installation.md` - Detailed installation guide (12KB)
11 | - `docs/guides/windows-setup.md` - Windows-specific setup (4KB)
12 | - `docs/guides/UBUNTU_SETUP.md` - Ubuntu-specific setup
13 | - `docs/sqlite-vec-backend.md` - SQLite-vec backend guide
14 | - `MIGRATION_GUIDE.md` (root) - ChromaDB to SQLite-vec migration
15 | - `scripts/install_windows.py` - Windows installer script
16 | - `scripts/installation/install.py` - Alternative installer script
17 |
18 | ### Platform-Specific Documentation
19 | - `docs/integration/homebrew/` (7 files) - Homebrew PyTorch integration
20 | - `HOMEBREW_PYTORCH_README.md` - Main Homebrew integration
21 | - `HOMEBREW_PYTORCH_SETUP.md` - Setup instructions
22 | - `TROUBLESHOOTING_GUIDE.md` - Homebrew troubleshooting
23 | - `docs/guides/windows-setup.md` - Windows platform guide
24 | - `docs/guides/UBUNTU_SETUP.md` - Linux platform guide
25 |
26 | ### API and Technical Documentation
27 | - `docs/IMPLEMENTATION_PLAN_HTTP_SSE.md` - HTTP/SSE implementation plan
28 | - `docs/guides/claude_integration.md` - Claude Desktop integration
29 | - `docs/guides/invocation_guide.md` - Usage guide
30 | - `docs/technical/` - Technical implementation details
31 |
32 | ### Migration and Troubleshooting
33 | - `MIGRATION_GUIDE.md` - ChromaDB to SQLite-vec migration
34 | - `docs/guides/migration.md` - General migration guide
35 | - `docs/guides/troubleshooting.md` - General troubleshooting
36 | - `docs/integration/homebrew/TROUBLESHOOTING_GUIDE.md` - Homebrew-specific
37 |
38 | ## Documentation Gaps Identified
39 |
40 | ### 1. Master Installation Guide Missing
41 | - No single source of truth for installation
42 | - Backend selection guidance scattered
43 | - Hardware-specific optimization not documented coherently
44 |
45 | ### 2. Legacy Hardware Support Documentation
46 | - 2015 MacBook Pro scenario not explicitly documented
47 | - Older Intel Mac optimization path unclear
48 | - Homebrew PyTorch integration buried in subdirectory
49 |
50 | ### 3. Storage Backend Comparison
51 | - No comprehensive comparison between ChromaDB and SQLite-vec
52 | - Selection criteria not clearly documented
53 | - Migration paths not prominently featured
54 |
55 | ### 4. HTTP/SSE API Documentation
56 | - Implementation plan exists but user-facing API docs missing
57 | - Integration examples needed
58 | - SSE event documentation incomplete
59 |
60 | ## Consolidation Strategy
61 |
62 | ### Phase 1: Create Master Documents
63 | 1. **docs/guides/INSTALLATION_MASTER.md** - Comprehensive installation guide
64 | 2. **docs/guides/STORAGE_BACKENDS.md** - Backend comparison and selection
65 | 3. **docs/guides/HARDWARE_OPTIMIZATION.md** - Platform-specific optimizations
66 | 4. **docs/api/HTTP_SSE_API.md** - Complete API documentation
67 |
68 | ### Phase 2: Platform-Specific Consolidation
69 | 1. **docs/platforms/macos-intel-legacy.md** - Your 2015 MacBook Pro use case
70 | 2. **docs/platforms/macos-modern.md** - Recent Mac configurations
71 | 3. **docs/platforms/windows.md** - Consolidated Windows guide
72 | 4. **docs/platforms/linux.md** - Consolidated Linux guide
73 |
74 | ### Phase 3: Merge and Reorganize
75 | 1. Consolidate duplicate content
76 | 2. Create cross-references between related docs
77 | 3. Update README.md to point to new structure
78 | 4. Archive or remove obsolete documentation
79 |
80 | ## High-Priority Actions
81 |
82 | 1. ✅ Create this audit document
83 | 2. ⏳ Create master installation guide
84 | 3. ⏳ Consolidate platform-specific guides
85 | 4. ⏳ Document hardware intelligence matrix
86 | 5. ⏳ Create migration consolidation guide
87 | 6. ⏳ Update README.md with new structure
88 |
89 | ## Content Quality Assessment
90 |
91 | ### Good Documentation (Keep/Enhance)
92 | - `MIGRATION_GUIDE.md` - Well structured, clear steps
93 | - `docs/sqlite-vec-backend.md` - Comprehensive backend guide
94 | - `docs/integration/homebrew/HOMEBREW_PYTORCH_README.md` - Good Homebrew integration
95 |
96 | ### Needs Improvement
97 | - `README.md` - Lacks backend choice prominence
98 | - `docs/guides/installation.md` - Too generic, needs hardware-specific paths
99 | - Multiple troubleshooting guides need consolidation
100 |
101 | ### Duplicated Content (Consolidate)
102 | - Installation instructions repeated across multiple files
103 | - Windows setup scattered between guides and scripts
104 | - Homebrew integration documentation fragmented
105 |
106 | ## Next Steps
107 | 1. Begin creating master installation guide
108 | 2. Merge hardware-specific content from various sources
109 | 3. Create clear user journey documentation
110 | 4. Test documentation accuracy with actual installations
```
--------------------------------------------------------------------------------
/scripts/backup/backup_memories.py:
--------------------------------------------------------------------------------
```python
1 | #!/usr/bin/env python3
2 | # Copyright 2024 Heinrich Krupp
3 | #
4 | # Licensed under the Apache License, Version 2.0 (the "License");
5 | # you may not use this file except in compliance with the License.
6 | # You may obtain a copy of the License at
7 | #
8 | # http://www.apache.org/licenses/LICENSE-2.0
9 | #
10 | # Unless required by applicable law or agreed to in writing, software
11 | # distributed under the License is distributed on an "AS IS" BASIS,
12 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 | # See the License for the specific language governing permissions and
14 | # limitations under the License.
15 |
16 | """
17 | Backup script to export all memories from the database to a JSON file.
18 | This provides a safe backup before running migrations or making database changes.
19 | """
20 | import sys
21 | import os
22 | import json
23 | import asyncio
24 | import logging
25 | import datetime
26 | from pathlib import Path
27 |
28 | # Add parent directory to path so we can import from the src directory
29 | sys.path.insert(0, str(Path(__file__).parent.parent))
30 |
31 | from src.mcp_memory_service.storage.chroma import ChromaMemoryStorage
32 | from src.mcp_memory_service.config import CHROMA_PATH, BACKUPS_PATH
33 |
34 | # Configure logging
35 | logging.basicConfig(
36 | level=logging.INFO,
37 | format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
38 | )
39 | logger = logging.getLogger("memory_backup")
40 |
41 | async def backup_memories():
42 | """
43 | Export all memories from the database to a JSON file.
44 | """
45 | logger.info(f"Initializing ChromaDB storage at {CHROMA_PATH}")
46 | storage = ChromaMemoryStorage(CHROMA_PATH)
47 |
48 | # Create backups directory if it doesn't exist
49 | os.makedirs(BACKUPS_PATH, exist_ok=True)
50 |
51 | # Generate backup filename with timestamp
52 | timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
53 | backup_file = os.path.join(BACKUPS_PATH, f"memory_backup_{timestamp}.json")
54 |
55 | logger.info(f"Creating backup at {backup_file}")
56 |
57 | try:
58 | # Retrieve all memories from the database
59 | try:
60 | # First try with embeddings
61 | logger.info("Attempting to retrieve memories with embeddings")
62 | results = storage.collection.get(
63 | include=["metadatas", "documents"]
64 | )
65 | include_embeddings = False
66 | except Exception as e:
67 | logger.warning(f"Failed to retrieve with embeddings: {e}")
68 | logger.info("Falling back to retrieving memories without embeddings")
69 | # Fall back to no embeddings
70 | results = storage.collection.get(
71 | include=["metadatas", "documents"]
72 | )
73 | include_embeddings = False
74 |
75 | if not results["ids"]:
76 | logger.info("No memories found in database")
77 | return backup_file
78 |
79 | total_memories = len(results["ids"])
80 | logger.info(f"Found {total_memories} memories to backup")
81 |
82 | # Create backup data structure
83 | backup_data = {
84 | "timestamp": datetime.datetime.now().isoformat(),
85 | "total_memories": total_memories,
86 | "memories": []
87 | }
88 |
89 | # Process each memory
90 | for i, memory_id in enumerate(results["ids"]):
91 | metadata = results["metadatas"][i]
92 | document = results["documents"][i]
93 | embedding = None
94 | if include_embeddings and "embeddings" in results and results["embeddings"] is not None:
95 | if i < len(results["embeddings"]):
96 | embedding = results["embeddings"][i]
97 |
98 | memory_data = {
99 | "id": memory_id,
100 | "document": document,
101 | "metadata": metadata,
102 | "embedding": embedding
103 | }
104 |
105 | backup_data["memories"].append(memory_data)
106 |
107 | # Write backup to file
108 | with open(backup_file, 'w', encoding='utf-8') as f:
109 | json.dump(backup_data, f, indent=2, ensure_ascii=False)
110 |
111 | logger.info(f"Successfully backed up {total_memories} memories to {backup_file}")
112 | return backup_file
113 |
114 | except Exception as e:
115 | logger.error(f"Error creating backup: {str(e)}")
116 | raise
117 |
118 | async def main():
119 | """Main function to run the backup."""
120 | logger.info("=== Starting memory backup ===")
121 |
122 | try:
123 | backup_file = await backup_memories()
124 | logger.info(f"=== Backup completed successfully: {backup_file} ===")
125 | except Exception as e:
126 | logger.error(f"Backup failed: {str(e)}")
127 | sys.exit(1)
128 |
129 | if __name__ == "__main__":
130 | asyncio.run(main())
```
--------------------------------------------------------------------------------
/scripts/maintenance/scan_todos.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 | # scripts/maintenance/scan_todos.sh - Scan codebase for TODOs and prioritize
3 | #
4 | # Usage: bash scripts/maintenance/scan_todos.sh [DIRECTORY]
5 | # Example: bash scripts/maintenance/scan_todos.sh src/
6 |
7 | set -e
8 |
9 | SCAN_DIR=${1:-src}
10 |
11 | if ! command -v gemini &> /dev/null; then
12 | echo "Error: Gemini CLI is not installed"
13 | exit 1
14 | fi
15 |
16 | if [ ! -d "$SCAN_DIR" ]; then
17 | echo "Error: Directory not found: $SCAN_DIR"
18 | exit 1
19 | fi
20 |
21 | echo "=== TODO Scanner ==="
22 | echo "Scanning directory: $SCAN_DIR"
23 | echo ""
24 |
25 | # Extract all TODOs with file and line number
26 | echo "Finding TODO comments..."
27 | todos=$(find "$SCAN_DIR" -name '*.py' -exec grep -Hn "TODO\|FIXME\|HACK\|XXX" {} \; 2>/dev/null || echo "")
28 |
29 | if [ -z "$todos" ]; then
30 | echo "✅ No TODOs found in $SCAN_DIR"
31 | exit 0
32 | fi
33 |
34 | todo_count=$(echo "$todos" | wc -l)
35 | echo "Found $todo_count TODO comments"
36 | echo ""
37 |
38 | # Save to temp file using mktemp
39 | todos_raw=$(mktemp -t todos_raw.XXXXXX)
40 | echo "$todos" > "$todos_raw"
41 |
42 | echo "Analyzing and prioritizing TODOs with Gemini..."
43 | echo ""
44 |
45 | # Use Gemini to prioritize
46 | prioritized=$(gemini "Analyze these TODO/FIXME/HACK/XXX comments from a Python codebase and categorize by priority.
47 |
48 | Priority Levels:
49 | - **CRITICAL (P0)**: Security vulnerabilities, data corruption risks, blocking bugs, production-breaking issues
50 | - **HIGH (P1)**: Performance bottlenecks (>100ms), user-facing bugs, incomplete core features, API breaking changes
51 | - **MEDIUM (P2)**: Code quality improvements, minor optimizations, technical debt, convenience features
52 | - **LOW (P3)**: Documentation, cosmetic changes, nice-to-haves, future enhancements
53 |
54 | Consider:
55 | - Security impact (SQL injection, XSS, etc.)
56 | - Performance implications
57 | - Feature completeness
58 | - User impact
59 | - Technical debt accumulation
60 |
61 | TODO comments (format: file:line:comment):
62 | $(cat "$todos_raw")
63 |
64 | Output format (be concise):
65 | ## CRITICAL (P0)
66 | - file.py:123 - Brief description of issue
67 |
68 | ## HIGH (P1)
69 | - file.py:456 - Brief description
70 |
71 | ## MEDIUM (P2)
72 | - file.py:789 - Brief description
73 |
74 | ## LOW (P3)
75 | - file.py:012 - Brief description")
76 |
77 | todos_prioritized=$(mktemp -t todos_prioritized.XXXXXX)
78 | echo "$prioritized" > "$todos_prioritized"
79 |
80 | # Display results
81 | cat "$todos_prioritized"
82 | echo ""
83 |
84 | # Count actual items using awk (robust, order-independent parsing)
85 | # Pattern: count lines starting with '-' within each priority section
86 | critical_items=$(awk '/^## CRITICAL/,/^## [A-Z]/ {if (/^-/ && !/^## /) count++} END {print count+0}' "$todos_prioritized")
87 | high_items=$(awk '/^## HIGH/,/^## [A-Z]/ {if (/^-/ && !/^## /) count++} END {print count+0}' "$todos_prioritized")
88 | medium_items=$(awk '/^## MEDIUM/,/^## [A-Z]/ {if (/^-/ && !/^## /) count++} END {print count+0}' "$todos_prioritized")
89 | # LOW section: handle both cases (followed by another section or end of file)
90 | low_items=$(awk '/^## LOW/ {in_low=1; next} /^## [A-Z]/ && in_low {in_low=0} in_low && /^-/ {count++} END {print count+0}' "$todos_prioritized")
91 |
92 | echo "=== Summary ==="
93 | echo "Total TODOs: $todo_count"
94 | echo ""
95 | echo "By Priority:"
96 | echo " CRITICAL (P0): $critical_items"
97 | echo " HIGH (P1): $high_items"
98 | echo " MEDIUM (P2): $medium_items"
99 | echo " LOW (P3): $low_items"
100 | echo ""
101 |
102 | # Save to docs (optional)
103 | if [ -d "docs/development" ]; then
104 | echo "Saving to docs/development/todo-tracker.md..."
105 | cat > docs/development/todo-tracker.md << EOF
106 | # TODO Tracker
107 |
108 | **Last Updated:** $(date '+%Y-%m-%d %H:%M:%S')
109 | **Scan Directory:** $SCAN_DIR
110 | **Total TODOs:** $todo_count
111 |
112 | ## Summary
113 |
114 | | Priority | Count | Description |
115 | |----------|-------|-------------|
116 | | CRITICAL (P0) | $critical_items | Security, data corruption, blocking bugs |
117 | | HIGH (P1) | $high_items | Performance, user-facing, incomplete features |
118 | | MEDIUM (P2) | $medium_items | Code quality, optimizations, technical debt |
119 | | LOW (P3) | $low_items | Documentation, cosmetic, nice-to-haves |
120 |
121 | ---
122 |
123 | $(cat "$todos_prioritized")
124 |
125 | ---
126 |
127 | ## How to Address
128 |
129 | 1. **CRITICAL**: Address immediately, block releases if necessary
130 | 2. **HIGH**: Schedule for current/next sprint
131 | 3. **MEDIUM**: Add to backlog, address in refactoring sprints
132 | 4. **LOW**: Address opportunistically or when touching related code
133 |
134 | ## Updating This Tracker
135 |
136 | Run: \`bash scripts/maintenance/scan_todos.sh\`
137 | EOF
138 | echo "✅ Saved to docs/development/todo-tracker.md"
139 | fi
140 |
141 | # Clean up temp files
142 | rm -f "$todos_raw" "$todos_prioritized"
143 |
144 | # Exit with warning if critical TODOs found
145 | if [ $critical_items -gt 0 ]; then
146 | echo ""
147 | echo "⚠️ WARNING: $critical_items CRITICAL TODOs found!"
148 | echo "These should be addressed immediately."
149 | exit 1
150 | fi
151 |
152 | exit 0
153 |
```
--------------------------------------------------------------------------------
/docs/LM_STUDIO_COMPATIBILITY.md:
--------------------------------------------------------------------------------
```markdown
1 | # LM Studio Compatibility Guide
2 |
3 | ## Issue Description
4 |
5 | When using MCP Memory Service with LM Studio or Claude Desktop, you may encounter errors when operations are cancelled or timeout:
6 |
7 | ### Error Types
8 |
9 | 1. **Validation Error (LM Studio)**:
10 | ```
11 | pydantic_core._pydantic_core.ValidationError: 5 validation errors for ClientNotification
12 | ProgressNotification.method
13 | Input should be 'notifications/progress' [type=literal_error, input_value='notifications/cancelled', input_type=str]
14 | ```
15 |
16 | 2. **Timeout Error (Claude Desktop)**:
17 | ```
18 | Message from client: {"jsonrpc":"2.0","method":"notifications/cancelled","params":{"requestId":0,"reason":"McpError: MCP error -32001: Request timed out"}}
19 | Server transport closed unexpectedly, this is likely due to the process exiting early.
20 | ```
21 |
22 | These occur because:
23 | - LM Studio and Claude Desktop send non-standard `notifications/cancelled` messages
24 | - These messages aren't part of the official MCP (Model Context Protocol) specification
25 | - Timeouts can cause the server to exit prematurely on Windows systems
26 |
27 | ## Solution
28 |
29 | The MCP Memory Service now includes an automatic compatibility patch that handles LM Studio's non-standard notifications. This patch is applied automatically when the server starts.
30 |
31 | ### How It Works
32 |
33 | 1. **Automatic Detection**: The server detects when clients send `notifications/cancelled` messages
34 | 2. **Graceful Handling**: Instead of crashing, the server handles these gracefully:
35 | - Logs the cancellation reason (including timeouts)
36 | - Converts to harmless notifications that don't cause validation errors
37 | - Continues operation normally
38 | 3. **Platform Optimizations**:
39 | - **Windows**: Extended timeouts (30s vs 15s) due to security software interference
40 | - **Cross-platform**: Enhanced signal handling for graceful shutdowns
41 |
42 | ### What You Need to Do
43 |
44 | **Nothing!** The compatibility patch is applied automatically when you start the MCP Memory Service.
45 |
46 | ### Verifying the Fix
47 |
48 | You can verify the patch is working by checking the server logs. You should see:
49 |
50 | ```
51 | Applied enhanced LM Studio/Claude Desktop compatibility patch for notifications/cancelled
52 | ```
53 |
54 | When operations are cancelled or timeout, you'll see:
55 |
56 | ```
57 | Intercepted cancelled notification (ID: 0): McpError: MCP error -32001: Request timed out
58 | Operation timeout detected: McpError: MCP error -32001: Request timed out
59 | ```
60 |
61 | Instead of a crash, the server will continue running.
62 |
63 | ## Technical Details
64 |
65 | The compatibility layer is implemented in `src/mcp_memory_service/lm_studio_compat.py` and:
66 |
67 | 1. **Notification Patching**: Monkey-patches the MCP library's `ClientNotification.model_validate` method
68 | 2. **Timeout Detection**: Identifies and logs timeout scenarios vs regular cancellations
69 | 3. **Graceful Substitution**: Converts `notifications/cancelled` to valid `InitializedNotification` objects
70 | 4. **Platform Optimization**: Uses extended timeouts on Windows (30s vs 15s)
71 | 5. **Signal Handling**: Adds Windows-specific signal handlers for graceful shutdowns
72 | 6. **Alternative Patching**: Fallback approach modifies the session receive loop if needed
73 |
74 | ## Windows-Specific Improvements
75 |
76 | - **Extended Timeouts**: 30-second timeout for storage initialization (vs 15s on other platforms)
77 | - **Security Software Compatibility**: Accounts for Windows Defender and antivirus delays
78 | - **Signal Handling**: Enhanced SIGTERM/SIGINT handling for clean shutdowns
79 | - **Timeout Recovery**: Better recovery from initialization timeouts
80 |
81 | ## Limitations
82 |
83 | - **Workaround Nature**: This addresses non-standard client behavior, not a server issue
84 | - **Cancelled Operations**: Operations aren't truly cancelled server-side, just client notifications are handled
85 | - **Timeout Recovery**: While timeouts are handled gracefully, the original operation may still complete
86 |
87 | ## Future Improvements
88 |
89 | Ideally, this should be fixed in one of two ways:
90 |
91 | 1. **LM Studio Update**: LM Studio should follow the MCP specification and not send non-standard notifications
92 | 2. **MCP Library Update**: The MCP library could be updated to handle vendor-specific extensions gracefully
93 |
94 | ## Troubleshooting
95 |
96 | If you still experience issues:
97 |
98 | 1. Ensure you're using the latest version of MCP Memory Service
99 | 2. Check that the patch is being applied (look for the log message)
100 | 3. Report the issue with full error logs to the repository
101 |
102 | ## Related Issues
103 |
104 | - This is a known compatibility issue between LM Studio and the MCP protocol
105 | - Similar issues may occur with other non-standard MCP clients
106 | - The patch specifically handles LM Studio's behavior and may need updates for other clients
```
--------------------------------------------------------------------------------
/docs/technical/memory-migration.md:
--------------------------------------------------------------------------------
```markdown
1 | # Memory Migration Technical Documentation
2 |
3 | This document provides technical details about the memory migration process in the MCP Memory Service.
4 |
5 | ## Overview
6 |
7 | The memory migration process allows transferring memories between different ChromaDB instances, supporting both local and remote migrations. The process is handled by the `mcp-migration.py` script, which provides a robust and efficient way to move memories while maintaining data integrity.
8 |
9 | ## Migration Types
10 |
11 | ### 1. Local to Remote Migration
12 | - Source: Local ChromaDB instance
13 | - Target: Remote ChromaDB server
14 | - Use case: Moving memories from a development environment to production
15 |
16 | ### 2. Remote to Local Migration
17 | - Source: Remote ChromaDB server
18 | - Target: Local ChromaDB instance
19 | - Use case: Creating local backups or development environments
20 |
21 | ## Technical Implementation
22 |
23 | ### Environment Verification
24 | Before starting the migration, the script performs environment verification:
25 | - Checks Python version compatibility
26 | - Verifies required packages are installed
27 | - Validates ChromaDB paths and configurations
28 | - Ensures network connectivity for remote migrations
29 |
30 | ### Migration Process
31 | 1. **Connection Setup**
32 | - Establishes connections to both source and target ChromaDB instances
33 | - Verifies collection existence and creates if necessary
34 | - Sets up embedding functions for consistent vectorization
35 |
36 | 2. **Data Transfer**
37 | - Implements batch processing (default batch size: 10)
38 | - Includes delay between batches to prevent overwhelming the target
39 | - Handles duplicate detection to avoid data redundancy
40 | - Maintains metadata and document relationships
41 |
42 | 3. **Verification**
43 | - Validates successful transfer by comparing record counts
44 | - Checks for data integrity
45 | - Provides detailed logging of the migration process
46 |
47 | ## Error Handling
48 |
49 | The migration script includes comprehensive error handling for:
50 | - Connection failures
51 | - Collection access issues
52 | - Data transfer interruptions
53 | - Configuration errors
54 | - Environment incompatibilities
55 |
56 | ## Performance Considerations
57 |
58 | - **Batch Size**: Default 10 records per batch
59 | - **Delay**: 1 second between batches
60 | - **Memory Usage**: Optimized for minimal memory footprint
61 | - **Network**: Handles connection timeouts and retries
62 |
63 | ## Configuration Options
64 |
65 | ### Source Configuration
66 | ```json
67 | {
68 | "type": "local|remote",
69 | "config": {
70 | "path": "/path/to/chroma", // for local
71 | "host": "remote-host", // for remote
72 | "port": 8000 // for remote
73 | }
74 | }
75 | ```
76 |
77 | ### Target Configuration
78 | ```json
79 | {
80 | "type": "local|remote",
81 | "config": {
82 | "path": "/path/to/chroma", // for local
83 | "host": "remote-host", // for remote
84 | "port": 8000 // for remote
85 | }
86 | }
87 | ```
88 |
89 | ## Best Practices
90 |
91 | 1. **Pre-Migration**
92 | - Verify environment compatibility
93 | - Ensure sufficient disk space
94 | - Check network connectivity for remote migrations
95 | - Backup existing data
96 |
97 | 2. **During Migration**
98 | - Monitor progress through logs
99 | - Avoid interrupting the process
100 | - Check for error messages
101 |
102 | 3. **Post-Migration**
103 | - Verify data integrity
104 | - Check collection statistics
105 | - Validate memory access
106 |
107 | ## Troubleshooting
108 |
109 | Common issues and solutions:
110 |
111 | 1. **Connection Failures**
112 | - Verify network connectivity
113 | - Check firewall settings
114 | - Validate host and port configurations
115 |
116 | 2. **Data Transfer Issues**
117 | - Check disk space
118 | - Verify collection permissions
119 | - Monitor system resources
120 |
121 | 3. **Environment Issues**
122 | - Run environment verification
123 | - Check package versions
124 | - Validate Python environment
125 |
126 | ## Example Usage
127 |
128 | ### Command Line
129 | ```bash
130 | # Local to Remote Migration
131 | python scripts/mcp-migration.py \
132 | --source-type local \
133 | --source-config /path/to/local/chroma \
134 | --target-type remote \
135 | --target-config '{"host": "remote-host", "port": 8000}'
136 |
137 | # Remote to Local Migration
138 | python scripts/mcp-migration.py \
139 | --source-type remote \
140 | --source-config '{"host": "remote-host", "port": 8000}' \
141 | --target-type local \
142 | --target-config /path/to/local/chroma
143 | ```
144 |
145 | ### Programmatic Usage
146 | ```python
147 | from scripts.mcp_migration import migrate_memories
148 |
149 | # Local to Remote Migration
150 | migrate_memories(
151 | source_type='local',
152 | source_config='/path/to/local/chroma',
153 | target_type='remote',
154 | target_config={'host': 'remote-host', 'port': 8000}
155 | )
156 |
157 | # Remote to Local Migration
158 | migrate_memories(
159 | source_type='remote',
160 | source_config={'host': 'remote-host', 'port': 8000},
161 | target_type='local',
162 | target_config='/path/to/local/chroma'
163 | )
164 | ```
```