This is page 5 of 23. Use http://codebase.md/basicmachines-co/basic-memory?lines=true&page={x} to view the full context.
# Directory Structure
```
├── .claude
│ ├── agents
│ │ ├── python-developer.md
│ │ └── system-architect.md
│ └── commands
│ ├── release
│ │ ├── beta.md
│ │ ├── changelog.md
│ │ ├── release-check.md
│ │ └── release.md
│ ├── spec.md
│ └── test-live.md
├── .dockerignore
├── .github
│ ├── dependabot.yml
│ ├── ISSUE_TEMPLATE
│ │ ├── bug_report.md
│ │ ├── config.yml
│ │ ├── documentation.md
│ │ └── feature_request.md
│ └── workflows
│ ├── claude-code-review.yml
│ ├── claude-issue-triage.yml
│ ├── claude.yml
│ ├── dev-release.yml
│ ├── docker.yml
│ ├── pr-title.yml
│ ├── release.yml
│ └── test.yml
├── .gitignore
├── .python-version
├── CHANGELOG.md
├── CITATION.cff
├── CLA.md
├── CLAUDE.md
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── docker-compose.yml
├── Dockerfile
├── docs
│ ├── ai-assistant-guide-extended.md
│ ├── character-handling.md
│ ├── cloud-cli.md
│ └── Docker.md
├── justfile
├── LICENSE
├── llms-install.md
├── pyproject.toml
├── README.md
├── SECURITY.md
├── smithery.yaml
├── specs
│ ├── SPEC-1 Specification-Driven Development Process.md
│ ├── SPEC-10 Unified Deployment Workflow and Event Tracking.md
│ ├── SPEC-11 Basic Memory API Performance Optimization.md
│ ├── SPEC-12 OpenTelemetry Observability.md
│ ├── SPEC-13 CLI Authentication with Subscription Validation.md
│ ├── SPEC-14 Cloud Git Versioning & GitHub Backup.md
│ ├── SPEC-14- Cloud Git Versioning & GitHub Backup.md
│ ├── SPEC-15 Configuration Persistence via Tigris for Cloud Tenants.md
│ ├── SPEC-16 MCP Cloud Service Consolidation.md
│ ├── SPEC-17 Semantic Search with ChromaDB.md
│ ├── SPEC-18 AI Memory Management Tool.md
│ ├── SPEC-19 Sync Performance and Memory Optimization.md
│ ├── SPEC-2 Slash Commands Reference.md
│ ├── SPEC-3 Agent Definitions.md
│ ├── SPEC-4 Notes Web UI Component Architecture.md
│ ├── SPEC-5 CLI Cloud Upload via WebDAV.md
│ ├── SPEC-6 Explicit Project Parameter Architecture.md
│ ├── SPEC-7 POC to spike Tigris Turso for local access to cloud data.md
│ ├── SPEC-8 TigrisFS Integration.md
│ ├── SPEC-9 Multi-Project Bidirectional Sync Architecture.md
│ ├── SPEC-9 Signed Header Tenant Information.md
│ └── SPEC-9-1 Follow-Ups- Conflict, Sync, and Observability.md
├── src
│ └── basic_memory
│ ├── __init__.py
│ ├── alembic
│ │ ├── alembic.ini
│ │ ├── env.py
│ │ ├── migrations.py
│ │ ├── script.py.mako
│ │ └── versions
│ │ ├── 3dae7c7b1564_initial_schema.py
│ │ ├── 502b60eaa905_remove_required_from_entity_permalink.py
│ │ ├── 5fe1ab1ccebe_add_projects_table.py
│ │ ├── 647e7a75e2cd_project_constraint_fix.py
│ │ ├── 9d9c1cb7d8f5_add_mtime_and_size_columns_to_entity_.py
│ │ ├── a1b2c3d4e5f6_fix_project_foreign_keys.py
│ │ ├── b3c3938bacdb_relation_to_name_unique_index.py
│ │ ├── cc7172b46608_update_search_index_schema.py
│ │ └── e7e1f4367280_add_scan_watermark_tracking_to_project.py
│ ├── api
│ │ ├── __init__.py
│ │ ├── app.py
│ │ ├── routers
│ │ │ ├── __init__.py
│ │ │ ├── directory_router.py
│ │ │ ├── importer_router.py
│ │ │ ├── knowledge_router.py
│ │ │ ├── management_router.py
│ │ │ ├── memory_router.py
│ │ │ ├── project_router.py
│ │ │ ├── prompt_router.py
│ │ │ ├── resource_router.py
│ │ │ ├── search_router.py
│ │ │ └── utils.py
│ │ └── template_loader.py
│ ├── cli
│ │ ├── __init__.py
│ │ ├── app.py
│ │ ├── auth.py
│ │ ├── commands
│ │ │ ├── __init__.py
│ │ │ ├── cloud
│ │ │ │ ├── __init__.py
│ │ │ │ ├── api_client.py
│ │ │ │ ├── bisync_commands.py
│ │ │ │ ├── cloud_utils.py
│ │ │ │ ├── core_commands.py
│ │ │ │ ├── mount_commands.py
│ │ │ │ ├── rclone_config.py
│ │ │ │ ├── rclone_installer.py
│ │ │ │ ├── upload_command.py
│ │ │ │ └── upload.py
│ │ │ ├── command_utils.py
│ │ │ ├── db.py
│ │ │ ├── import_chatgpt.py
│ │ │ ├── import_claude_conversations.py
│ │ │ ├── import_claude_projects.py
│ │ │ ├── import_memory_json.py
│ │ │ ├── mcp.py
│ │ │ ├── project.py
│ │ │ ├── status.py
│ │ │ ├── sync.py
│ │ │ └── tool.py
│ │ └── main.py
│ ├── config.py
│ ├── db.py
│ ├── deps.py
│ ├── file_utils.py
│ ├── ignore_utils.py
│ ├── importers
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── chatgpt_importer.py
│ │ ├── claude_conversations_importer.py
│ │ ├── claude_projects_importer.py
│ │ ├── memory_json_importer.py
│ │ └── utils.py
│ ├── markdown
│ │ ├── __init__.py
│ │ ├── entity_parser.py
│ │ ├── markdown_processor.py
│ │ ├── plugins.py
│ │ ├── schemas.py
│ │ └── utils.py
│ ├── mcp
│ │ ├── __init__.py
│ │ ├── async_client.py
│ │ ├── project_context.py
│ │ ├── prompts
│ │ │ ├── __init__.py
│ │ │ ├── ai_assistant_guide.py
│ │ │ ├── continue_conversation.py
│ │ │ ├── recent_activity.py
│ │ │ ├── search.py
│ │ │ └── utils.py
│ │ ├── resources
│ │ │ ├── ai_assistant_guide.md
│ │ │ └── project_info.py
│ │ ├── server.py
│ │ └── tools
│ │ ├── __init__.py
│ │ ├── build_context.py
│ │ ├── canvas.py
│ │ ├── chatgpt_tools.py
│ │ ├── delete_note.py
│ │ ├── edit_note.py
│ │ ├── list_directory.py
│ │ ├── move_note.py
│ │ ├── project_management.py
│ │ ├── read_content.py
│ │ ├── read_note.py
│ │ ├── recent_activity.py
│ │ ├── search.py
│ │ ├── utils.py
│ │ ├── view_note.py
│ │ └── write_note.py
│ ├── models
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── knowledge.py
│ │ ├── project.py
│ │ └── search.py
│ ├── repository
│ │ ├── __init__.py
│ │ ├── entity_repository.py
│ │ ├── observation_repository.py
│ │ ├── project_info_repository.py
│ │ ├── project_repository.py
│ │ ├── relation_repository.py
│ │ ├── repository.py
│ │ └── search_repository.py
│ ├── schemas
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── cloud.py
│ │ ├── delete.py
│ │ ├── directory.py
│ │ ├── importer.py
│ │ ├── memory.py
│ │ ├── project_info.py
│ │ ├── prompt.py
│ │ ├── request.py
│ │ ├── response.py
│ │ ├── search.py
│ │ └── sync_report.py
│ ├── services
│ │ ├── __init__.py
│ │ ├── context_service.py
│ │ ├── directory_service.py
│ │ ├── entity_service.py
│ │ ├── exceptions.py
│ │ ├── file_service.py
│ │ ├── initialization.py
│ │ ├── link_resolver.py
│ │ ├── project_service.py
│ │ ├── search_service.py
│ │ └── service.py
│ ├── sync
│ │ ├── __init__.py
│ │ ├── background_sync.py
│ │ ├── sync_service.py
│ │ └── watch_service.py
│ ├── templates
│ │ └── prompts
│ │ ├── continue_conversation.hbs
│ │ └── search.hbs
│ └── utils.py
├── test-int
│ ├── BENCHMARKS.md
│ ├── cli
│ │ ├── test_project_commands_integration.py
│ │ ├── test_sync_commands_integration.py
│ │ └── test_version_integration.py
│ ├── conftest.py
│ ├── mcp
│ │ ├── test_build_context_underscore.py
│ │ ├── test_build_context_validation.py
│ │ ├── test_chatgpt_tools_integration.py
│ │ ├── test_default_project_mode_integration.py
│ │ ├── test_delete_note_integration.py
│ │ ├── test_edit_note_integration.py
│ │ ├── test_list_directory_integration.py
│ │ ├── test_move_note_integration.py
│ │ ├── test_project_management_integration.py
│ │ ├── test_project_state_sync_integration.py
│ │ ├── test_read_content_integration.py
│ │ ├── test_read_note_integration.py
│ │ ├── test_search_integration.py
│ │ ├── test_single_project_mcp_integration.py
│ │ └── test_write_note_integration.py
│ ├── test_db_wal_mode.py
│ ├── test_disable_permalinks_integration.py
│ └── test_sync_performance_benchmark.py
├── tests
│ ├── __init__.py
│ ├── api
│ │ ├── conftest.py
│ │ ├── test_async_client.py
│ │ ├── test_continue_conversation_template.py
│ │ ├── test_directory_router.py
│ │ ├── test_importer_router.py
│ │ ├── test_knowledge_router.py
│ │ ├── test_management_router.py
│ │ ├── test_memory_router.py
│ │ ├── test_project_router_operations.py
│ │ ├── test_project_router.py
│ │ ├── test_prompt_router.py
│ │ ├── test_relation_background_resolution.py
│ │ ├── test_resource_router.py
│ │ ├── test_search_router.py
│ │ ├── test_search_template.py
│ │ ├── test_template_loader_helpers.py
│ │ └── test_template_loader.py
│ ├── cli
│ │ ├── conftest.py
│ │ ├── test_bisync_commands.py
│ │ ├── test_cli_tools.py
│ │ ├── test_cloud_authentication.py
│ │ ├── test_cloud_utils.py
│ │ ├── test_ignore_utils.py
│ │ ├── test_import_chatgpt.py
│ │ ├── test_import_claude_conversations.py
│ │ ├── test_import_claude_projects.py
│ │ ├── test_import_memory_json.py
│ │ └── test_upload.py
│ ├── conftest.py
│ ├── db
│ │ └── test_issue_254_foreign_key_constraints.py
│ ├── importers
│ │ ├── test_importer_base.py
│ │ └── test_importer_utils.py
│ ├── markdown
│ │ ├── __init__.py
│ │ ├── test_date_frontmatter_parsing.py
│ │ ├── test_entity_parser_error_handling.py
│ │ ├── test_entity_parser.py
│ │ ├── test_markdown_plugins.py
│ │ ├── test_markdown_processor.py
│ │ ├── test_observation_edge_cases.py
│ │ ├── test_parser_edge_cases.py
│ │ ├── test_relation_edge_cases.py
│ │ └── test_task_detection.py
│ ├── mcp
│ │ ├── conftest.py
│ │ ├── test_obsidian_yaml_formatting.py
│ │ ├── test_permalink_collision_file_overwrite.py
│ │ ├── test_prompts.py
│ │ ├── test_resources.py
│ │ ├── test_tool_build_context.py
│ │ ├── test_tool_canvas.py
│ │ ├── test_tool_delete_note.py
│ │ ├── test_tool_edit_note.py
│ │ ├── test_tool_list_directory.py
│ │ ├── test_tool_move_note.py
│ │ ├── test_tool_read_content.py
│ │ ├── test_tool_read_note.py
│ │ ├── test_tool_recent_activity.py
│ │ ├── test_tool_resource.py
│ │ ├── test_tool_search.py
│ │ ├── test_tool_utils.py
│ │ ├── test_tool_view_note.py
│ │ ├── test_tool_write_note.py
│ │ └── tools
│ │ └── test_chatgpt_tools.py
│ ├── Non-MarkdownFileSupport.pdf
│ ├── repository
│ │ ├── test_entity_repository_upsert.py
│ │ ├── test_entity_repository.py
│ │ ├── test_entity_upsert_issue_187.py
│ │ ├── test_observation_repository.py
│ │ ├── test_project_info_repository.py
│ │ ├── test_project_repository.py
│ │ ├── test_relation_repository.py
│ │ ├── test_repository.py
│ │ ├── test_search_repository_edit_bug_fix.py
│ │ └── test_search_repository.py
│ ├── schemas
│ │ ├── test_base_timeframe_minimum.py
│ │ ├── test_memory_serialization.py
│ │ ├── test_memory_url_validation.py
│ │ ├── test_memory_url.py
│ │ ├── test_schemas.py
│ │ └── test_search.py
│ ├── Screenshot.png
│ ├── services
│ │ ├── test_context_service.py
│ │ ├── test_directory_service.py
│ │ ├── test_entity_service_disable_permalinks.py
│ │ ├── test_entity_service.py
│ │ ├── test_file_service.py
│ │ ├── test_initialization.py
│ │ ├── test_link_resolver.py
│ │ ├── test_project_removal_bug.py
│ │ ├── test_project_service_operations.py
│ │ ├── test_project_service.py
│ │ └── test_search_service.py
│ ├── sync
│ │ ├── test_character_conflicts.py
│ │ ├── test_sync_service_incremental.py
│ │ ├── test_sync_service.py
│ │ ├── test_sync_wikilink_issue.py
│ │ ├── test_tmp_files.py
│ │ ├── test_watch_service_edge_cases.py
│ │ ├── test_watch_service_reload.py
│ │ └── test_watch_service.py
│ ├── test_config.py
│ ├── test_db_migration_deduplication.py
│ ├── test_deps.py
│ ├── test_production_cascade_delete.py
│ └── utils
│ ├── test_file_utils.py
│ ├── test_frontmatter_obsidian_compatible.py
│ ├── test_parse_tags.py
│ ├── test_permalink_formatting.py
│ ├── test_utf8_handling.py
│ └── test_validate_project_path.py
├── uv.lock
├── v0.15.0-RELEASE-DOCS.md
└── v15-docs
├── api-performance.md
├── background-relations.md
├── basic-memory-home.md
├── bug-fixes.md
├── chatgpt-integration.md
├── cloud-authentication.md
├── cloud-bisync.md
├── cloud-mode-usage.md
├── cloud-mount.md
├── default-project-mode.md
├── env-file-removal.md
├── env-var-overrides.md
├── explicit-project-parameter.md
├── gitignore-integration.md
├── project-root-env-var.md
├── README.md
└── sqlite-performance.md
```
# Files
--------------------------------------------------------------------------------
/tests/cli/test_import_claude_conversations.py:
--------------------------------------------------------------------------------
```python
1 | """Tests for import_claude command (chat conversations)."""
2 |
3 | import json
4 |
5 | import pytest
6 | from typer.testing import CliRunner
7 |
8 | from basic_memory.cli.app import app
9 | from basic_memory.cli.commands import import_claude_conversations # noqa
10 | from basic_memory.config import get_project_config
11 |
12 | # Set up CLI runner
13 | runner = CliRunner()
14 |
15 |
16 | @pytest.fixture
17 | def sample_conversation():
18 | """Sample conversation data for testing."""
19 | return {
20 | "uuid": "test-uuid",
21 | "name": "Test Conversation",
22 | "created_at": "2025-01-05T20:55:32.499880+00:00",
23 | "updated_at": "2025-01-05T20:56:39.477600+00:00",
24 | "chat_messages": [
25 | {
26 | "uuid": "msg-1",
27 | "text": "Hello, this is a test",
28 | "sender": "human",
29 | "created_at": "2025-01-05T20:55:32.499880+00:00",
30 | "content": [{"type": "text", "text": "Hello, this is a test"}],
31 | },
32 | {
33 | "uuid": "msg-2",
34 | "text": "Response to test",
35 | "sender": "assistant",
36 | "created_at": "2025-01-05T20:55:40.123456+00:00",
37 | "content": [{"type": "text", "text": "Response to test"}],
38 | },
39 | ],
40 | }
41 |
42 |
43 | @pytest.fixture
44 | def sample_conversations_json(tmp_path, sample_conversation):
45 | """Create a sample conversations.json file."""
46 | json_file = tmp_path / "conversations.json"
47 | with open(json_file, "w", encoding="utf-8") as f:
48 | json.dump([sample_conversation], f)
49 | return json_file
50 |
51 |
52 | def test_import_conversations_command_file_not_found(tmp_path):
53 | """Test error handling for nonexistent file."""
54 | nonexistent = tmp_path / "nonexistent.json"
55 | result = runner.invoke(app, ["import", "claude", "conversations", str(nonexistent)])
56 | assert result.exit_code == 1
57 | assert "File not found" in result.output
58 |
59 |
60 | def test_import_conversations_command_success(tmp_path, sample_conversations_json, monkeypatch):
61 | """Test successful conversation import via command."""
62 | # Set up test environment
63 | monkeypatch.setenv("HOME", str(tmp_path))
64 |
65 | # Run import
66 | result = runner.invoke(
67 | app, ["import", "claude", "conversations", str(sample_conversations_json)]
68 | )
69 | assert result.exit_code == 0
70 | assert "Import complete" in result.output
71 | assert "Imported 1 conversations" in result.output
72 | assert "Containing 2 messages" in result.output
73 |
74 |
75 | def test_import_conversations_command_invalid_json(tmp_path):
76 | """Test error handling for invalid JSON."""
77 | # Create invalid JSON file
78 | invalid_file = tmp_path / "invalid.json"
79 | invalid_file.write_text("not json")
80 |
81 | result = runner.invoke(app, ["import", "claude", "conversations", str(invalid_file)])
82 | assert result.exit_code == 1
83 | assert "Error during import" in result.output
84 |
85 |
86 | def test_import_conversations_with_custom_folder(tmp_path, sample_conversations_json, monkeypatch):
87 | """Test import with custom conversations folder."""
88 | # Set up test environment
89 | config = get_project_config()
90 | config.home = tmp_path
91 | conversations_folder = "chats"
92 |
93 | # Run import
94 | result = runner.invoke(
95 | app,
96 | [
97 | "import",
98 | "claude",
99 | "conversations",
100 | str(sample_conversations_json),
101 | "--folder",
102 | conversations_folder,
103 | ],
104 | )
105 | assert result.exit_code == 0
106 |
107 | # Check files in custom folder
108 | conv_path = tmp_path / conversations_folder / "20250105-Test_Conversation.md"
109 | assert conv_path.exists()
110 |
111 |
112 | def test_import_conversation_with_attachments(tmp_path):
113 | """Test importing conversation with attachments."""
114 | # Create conversation with attachments
115 | conversation = {
116 | "uuid": "test-uuid",
117 | "name": "Test With Attachments",
118 | "created_at": "2025-01-05T20:55:32.499880+00:00",
119 | "updated_at": "2025-01-05T20:56:39.477600+00:00",
120 | "chat_messages": [
121 | {
122 | "uuid": "msg-1",
123 | "text": "Here's a file",
124 | "sender": "human",
125 | "created_at": "2025-01-05T20:55:32.499880+00:00",
126 | "content": [{"type": "text", "text": "Here's a file"}],
127 | "attachments": [
128 | {"file_name": "test.txt", "extracted_content": "Test file content"}
129 | ],
130 | }
131 | ],
132 | }
133 |
134 | json_file = tmp_path / "with_attachments.json"
135 | with open(json_file, "w", encoding="utf-8") as f:
136 | json.dump([conversation], f)
137 |
138 | config = get_project_config()
139 | # Set up environment
140 | config.home = tmp_path
141 |
142 | # Run import
143 | result = runner.invoke(app, ["import", "claude", "conversations", str(json_file)])
144 | assert result.exit_code == 0
145 |
146 | # Check attachment formatting
147 | conv_path = tmp_path / "conversations/20250105-Test_With_Attachments.md"
148 | content = conv_path.read_text(encoding="utf-8")
149 | assert "**Attachment: test.txt**" in content
150 | assert "```" in content
151 | assert "Test file content" in content
152 |
153 |
154 | def test_import_conversation_with_none_text_values(tmp_path):
155 | """Test importing conversation with None text values in content array (issue #236)."""
156 | # Create conversation with None text values
157 | conversation = {
158 | "uuid": "test-uuid",
159 | "name": "Test With None Text",
160 | "created_at": "2025-01-05T20:55:32.499880+00:00",
161 | "updated_at": "2025-01-05T20:56:39.477600+00:00",
162 | "chat_messages": [
163 | {
164 | "uuid": "msg-1",
165 | "text": None,
166 | "sender": "human",
167 | "created_at": "2025-01-05T20:55:32.499880+00:00",
168 | "content": [
169 | {"type": "text", "text": "Valid text here"},
170 | {"type": "text", "text": None}, # This caused the TypeError
171 | {"type": "text", "text": "More valid text"},
172 | ],
173 | },
174 | {
175 | "uuid": "msg-2",
176 | "text": None,
177 | "sender": "assistant",
178 | "created_at": "2025-01-05T20:55:40.123456+00:00",
179 | "content": [
180 | {"type": "text", "text": None}, # All None case
181 | {"type": "text", "text": None},
182 | ],
183 | },
184 | ],
185 | }
186 |
187 | json_file = tmp_path / "with_none_text.json"
188 | with open(json_file, "w", encoding="utf-8") as f:
189 | json.dump([conversation], f)
190 |
191 | config = get_project_config()
192 | config.home = tmp_path
193 |
194 | # Run import - should not fail with TypeError
195 | result = runner.invoke(app, ["import", "claude", "conversations", str(json_file)])
196 | assert result.exit_code == 0
197 |
198 | # Check that valid text is preserved and None values are filtered out
199 | conv_path = tmp_path / "conversations/20250105-Test_With_None_Text.md"
200 | assert conv_path.exists()
201 | content = conv_path.read_text(encoding="utf-8")
202 | assert "Valid text here" in content
203 | assert "More valid text" in content
204 |
```
--------------------------------------------------------------------------------
/src/basic_memory/file_utils.py:
--------------------------------------------------------------------------------
```python
1 | """Utilities for file operations."""
2 |
3 | import hashlib
4 | from pathlib import Path
5 | import re
6 | from typing import Any, Dict, Union
7 |
8 | import aiofiles
9 | import yaml
10 | import frontmatter
11 | from loguru import logger
12 |
13 | from basic_memory.utils import FilePath
14 |
15 |
16 | class FileError(Exception):
17 | """Base exception for file operations."""
18 |
19 | pass
20 |
21 |
22 | class FileWriteError(FileError):
23 | """Raised when file operations fail."""
24 |
25 | pass
26 |
27 |
28 | class ParseError(FileError):
29 | """Raised when parsing file content fails."""
30 |
31 | pass
32 |
33 |
34 | async def compute_checksum(content: Union[str, bytes]) -> str:
35 | """
36 | Compute SHA-256 checksum of content.
37 |
38 | Args:
39 | content: Content to hash (either text string or bytes)
40 |
41 | Returns:
42 | SHA-256 hex digest
43 |
44 | Raises:
45 | FileError: If checksum computation fails
46 | """
47 | try:
48 | if isinstance(content, str):
49 | content = content.encode()
50 | return hashlib.sha256(content).hexdigest()
51 | except Exception as e: # pragma: no cover
52 | logger.error(f"Failed to compute checksum: {e}")
53 | raise FileError(f"Failed to compute checksum: {e}")
54 |
55 |
56 | async def write_file_atomic(path: FilePath, content: str) -> None:
57 | """
58 | Write file with atomic operation using temporary file.
59 |
60 | Uses aiofiles for true async I/O (non-blocking).
61 |
62 | Args:
63 | path: Target file path (Path or string)
64 | content: Content to write
65 |
66 | Raises:
67 | FileWriteError: If write operation fails
68 | """
69 | # Convert string to Path if needed
70 | path_obj = Path(path) if isinstance(path, str) else path
71 | temp_path = path_obj.with_suffix(".tmp")
72 |
73 | try:
74 | # Use aiofiles for non-blocking write
75 | async with aiofiles.open(temp_path, mode="w", encoding="utf-8") as f:
76 | await f.write(content)
77 |
78 | # Atomic rename (this is fast, doesn't need async)
79 | temp_path.replace(path_obj)
80 | logger.debug("Wrote file atomically", path=str(path_obj), content_length=len(content))
81 | except Exception as e: # pragma: no cover
82 | temp_path.unlink(missing_ok=True)
83 | logger.error("Failed to write file", path=str(path_obj), error=str(e))
84 | raise FileWriteError(f"Failed to write file {path}: {e}")
85 |
86 |
87 | def has_frontmatter(content: str) -> bool:
88 | """
89 | Check if content contains valid YAML frontmatter.
90 |
91 | Args:
92 | content: Content to check
93 |
94 | Returns:
95 | True if content has valid frontmatter markers (---), False otherwise
96 | """
97 | if not content:
98 | return False
99 |
100 | content = content.strip()
101 | if not content.startswith("---"):
102 | return False
103 |
104 | return "---" in content[3:]
105 |
106 |
107 | def parse_frontmatter(content: str) -> Dict[str, Any]:
108 | """
109 | Parse YAML frontmatter from content.
110 |
111 | Args:
112 | content: Content with YAML frontmatter
113 |
114 | Returns:
115 | Dictionary of frontmatter values
116 |
117 | Raises:
118 | ParseError: If frontmatter is invalid or parsing fails
119 | """
120 | try:
121 | if not content.strip().startswith("---"):
122 | raise ParseError("Content has no frontmatter")
123 |
124 | # Split on first two occurrences of ---
125 | parts = content.split("---", 2)
126 | if len(parts) < 3:
127 | raise ParseError("Invalid frontmatter format")
128 |
129 | # Parse YAML
130 | try:
131 | frontmatter = yaml.safe_load(parts[1])
132 | # Handle empty frontmatter (None from yaml.safe_load)
133 | if frontmatter is None:
134 | return {}
135 | if not isinstance(frontmatter, dict):
136 | raise ParseError("Frontmatter must be a YAML dictionary")
137 | return frontmatter
138 |
139 | except yaml.YAMLError as e:
140 | raise ParseError(f"Invalid YAML in frontmatter: {e}")
141 |
142 | except Exception as e: # pragma: no cover
143 | if not isinstance(e, ParseError):
144 | logger.error(f"Failed to parse frontmatter: {e}")
145 | raise ParseError(f"Failed to parse frontmatter: {e}")
146 | raise
147 |
148 |
149 | def remove_frontmatter(content: str) -> str:
150 | """
151 | Remove YAML frontmatter from content.
152 |
153 | Args:
154 | content: Content with frontmatter
155 |
156 | Returns:
157 | Content with frontmatter removed, or original content if no frontmatter
158 |
159 | Raises:
160 | ParseError: If content starts with frontmatter marker but is malformed
161 | """
162 | content = content.strip()
163 |
164 | # Return as-is if no frontmatter marker
165 | if not content.startswith("---"):
166 | return content
167 |
168 | # Split on first two occurrences of ---
169 | parts = content.split("---", 2)
170 | if len(parts) < 3:
171 | raise ParseError("Invalid frontmatter format")
172 |
173 | return parts[2].strip()
174 |
175 |
176 | def dump_frontmatter(post: frontmatter.Post) -> str:
177 | """
178 | Serialize frontmatter.Post to markdown with Obsidian-compatible YAML format.
179 |
180 | This function ensures that tags are formatted as YAML lists instead of JSON arrays:
181 |
182 | Good (Obsidian compatible):
183 | ---
184 | tags:
185 | - system
186 | - overview
187 | - reference
188 | ---
189 |
190 | Bad (current behavior):
191 | ---
192 | tags: ["system", "overview", "reference"]
193 | ---
194 |
195 | Args:
196 | post: frontmatter.Post object to serialize
197 |
198 | Returns:
199 | String containing markdown with properly formatted YAML frontmatter
200 | """
201 | if not post.metadata:
202 | # No frontmatter, just return content
203 | return post.content
204 |
205 | # Serialize YAML with block style for lists
206 | yaml_str = yaml.dump(
207 | post.metadata, sort_keys=False, allow_unicode=True, default_flow_style=False
208 | )
209 |
210 | # Construct the final markdown with frontmatter
211 | if post.content:
212 | return f"---\n{yaml_str}---\n\n{post.content}"
213 | else:
214 | return f"---\n{yaml_str}---\n"
215 |
216 |
217 | def sanitize_for_filename(text: str, replacement: str = "-") -> str:
218 | """
219 | Sanitize string to be safe for use as a note title
220 | Replaces path separators and other problematic characters
221 | with hyphens.
222 | """
223 | # replace both POSIX and Windows path separators
224 | text = re.sub(r"[/\\]", replacement, text)
225 |
226 | # replace some other problematic chars
227 | text = re.sub(r'[<>:"|?*]', replacement, text)
228 |
229 | # compress multiple, repeated replacements
230 | text = re.sub(f"{re.escape(replacement)}+", replacement, text)
231 |
232 | return text.strip(replacement)
233 |
234 |
235 | def sanitize_for_folder(folder: str) -> str:
236 | """
237 | Sanitize folder path to be safe for use in file system paths.
238 | Removes leading/trailing whitespace, compresses multiple slashes,
239 | and removes special characters except for /, -, and _.
240 | """
241 | if not folder:
242 | return ""
243 |
244 | sanitized = folder.strip()
245 |
246 | if sanitized.startswith("./"):
247 | sanitized = sanitized[2:]
248 |
249 | # ensure no special characters (except for a few that are allowed)
250 | sanitized = "".join(
251 | c for c in sanitized if c.isalnum() or c in (".", " ", "-", "_", "\\", "/")
252 | ).rstrip()
253 |
254 | # compress multiple, repeated instances of path separators
255 | sanitized = re.sub(r"[\\/]+", "/", sanitized)
256 |
257 | # trim any leading/trailing path separators
258 | sanitized = sanitized.strip("\\/")
259 |
260 | return sanitized
261 |
```
--------------------------------------------------------------------------------
/src/basic_memory/services/initialization.py:
--------------------------------------------------------------------------------
```python
1 | """Shared initialization service for Basic Memory.
2 |
3 | This module provides shared initialization functions used by both CLI and API
4 | to ensure consistent application startup across all entry points.
5 | """
6 |
7 | import asyncio
8 | from pathlib import Path
9 |
10 | from loguru import logger
11 |
12 | from basic_memory import db
13 | from basic_memory.config import BasicMemoryConfig
14 | from basic_memory.models import Project
15 | from basic_memory.repository import (
16 | ProjectRepository,
17 | )
18 |
19 |
20 | async def initialize_database(app_config: BasicMemoryConfig) -> None:
21 | """Initialize database with migrations handled automatically by get_or_create_db.
22 |
23 | Args:
24 | app_config: The Basic Memory project configuration
25 |
26 | Note:
27 | Database migrations are now handled automatically when the database
28 | connection is first established via get_or_create_db().
29 | """
30 | # Trigger database initialization and migrations by getting the database connection
31 | try:
32 | await db.get_or_create_db(app_config.database_path)
33 | logger.info("Database initialization completed")
34 | except Exception as e:
35 | logger.error(f"Error initializing database: {e}")
36 | # Allow application to continue - it might still work
37 | # depending on what the error was, and will fail with a
38 | # more specific error if the database is actually unusable
39 |
40 |
41 | async def reconcile_projects_with_config(app_config: BasicMemoryConfig):
42 | """Ensure all projects in config.json exist in the projects table and vice versa.
43 |
44 | This uses the ProjectService's synchronize_projects method to ensure bidirectional
45 | synchronization between the configuration file and the database.
46 |
47 | Args:
48 | app_config: The Basic Memory application configuration
49 | """
50 | logger.info("Reconciling projects from config with database...")
51 |
52 | # Get database session - migrations handled centrally
53 | _, session_maker = await db.get_or_create_db(
54 | db_path=app_config.database_path,
55 | db_type=db.DatabaseType.FILESYSTEM,
56 | ensure_migrations=False,
57 | )
58 | project_repository = ProjectRepository(session_maker)
59 |
60 | # Import ProjectService here to avoid circular imports
61 | from basic_memory.services.project_service import ProjectService
62 |
63 | try:
64 | # Create project service and synchronize projects
65 | project_service = ProjectService(repository=project_repository)
66 | await project_service.synchronize_projects()
67 | logger.info("Projects successfully reconciled between config and database")
68 | except Exception as e:
69 | # Log the error but continue with initialization
70 | logger.error(f"Error during project synchronization: {e}")
71 | logger.info("Continuing with initialization despite synchronization error")
72 |
73 |
74 | async def initialize_file_sync(
75 | app_config: BasicMemoryConfig,
76 | ):
77 | """Initialize file synchronization services. This function starts the watch service and does not return
78 |
79 | Args:
80 | app_config: The Basic Memory project configuration
81 |
82 | Returns:
83 | The watch service task that's monitoring file changes
84 | """
85 |
86 | # delay import
87 | from basic_memory.sync import WatchService
88 |
89 | # Load app configuration - migrations handled centrally
90 | _, session_maker = await db.get_or_create_db(
91 | db_path=app_config.database_path,
92 | db_type=db.DatabaseType.FILESYSTEM,
93 | ensure_migrations=False,
94 | )
95 | project_repository = ProjectRepository(session_maker)
96 |
97 | # Initialize watch service
98 | watch_service = WatchService(
99 | app_config=app_config,
100 | project_repository=project_repository,
101 | quiet=True,
102 | )
103 |
104 | # Get active projects
105 | active_projects = await project_repository.get_active_projects()
106 |
107 | # Start sync for all projects as background tasks (non-blocking)
108 | async def sync_project_background(project: Project):
109 | """Sync a single project in the background."""
110 | # avoid circular imports
111 | from basic_memory.sync.sync_service import get_sync_service
112 |
113 | logger.info(f"Starting background sync for project: {project.name}")
114 | try:
115 | # Create sync service
116 | sync_service = await get_sync_service(project)
117 |
118 | sync_dir = Path(project.path)
119 | await sync_service.sync(sync_dir, project_name=project.name)
120 | logger.info(f"Background sync completed successfully for project: {project.name}")
121 | except Exception as e: # pragma: no cover
122 | logger.error(f"Error in background sync for project {project.name}: {e}")
123 |
124 | # Create background tasks for all project syncs (non-blocking)
125 | sync_tasks = [
126 | asyncio.create_task(sync_project_background(project)) for project in active_projects
127 | ]
128 | logger.info(f"Created {len(sync_tasks)} background sync tasks")
129 |
130 | # Don't await the tasks - let them run in background while we continue
131 |
132 | # Then start the watch service in the background
133 | logger.info("Starting watch service for all projects")
134 | # run the watch service
135 | try:
136 | await watch_service.run()
137 | logger.info("Watch service started")
138 | except Exception as e: # pragma: no cover
139 | logger.error(f"Error starting watch service: {e}")
140 |
141 | return None
142 |
143 |
144 | async def initialize_app(
145 | app_config: BasicMemoryConfig,
146 | ):
147 | """Initialize the Basic Memory application.
148 |
149 | This function handles all initialization steps:
150 | - Running database migrations
151 | - Reconciling projects from config.json with projects table
152 | - Setting up file synchronization
153 | - Starting background migration for legacy project data
154 |
155 | Args:
156 | app_config: The Basic Memory project configuration
157 | """
158 | logger.info("Initializing app...")
159 | # Initialize database first
160 | await initialize_database(app_config)
161 |
162 | # Reconcile projects from config.json with projects table
163 | await reconcile_projects_with_config(app_config)
164 |
165 | logger.info("App initialization completed (migration running in background if needed)")
166 |
167 |
168 | def ensure_initialization(app_config: BasicMemoryConfig) -> None:
169 | """Ensure initialization runs in a synchronous context.
170 |
171 | This is a wrapper for the async initialize_app function that can be
172 | called from synchronous code like CLI entry points.
173 |
174 | No-op if app_config.cloud_mode == True. Cloud basic memory manages it's own projects
175 |
176 | Args:
177 | app_config: The Basic Memory project configuration
178 | """
179 | # Skip initialization in cloud mode - cloud manages its own projects
180 | if app_config.cloud_mode_enabled:
181 | logger.debug("Skipping initialization in cloud mode - projects managed by cloud")
182 | return
183 |
184 | try:
185 | result = asyncio.run(initialize_app(app_config))
186 | logger.info(f"Initialization completed successfully: result={result}")
187 | except Exception as e: # pragma: no cover
188 | logger.exception(f"Error during initialization: {e}")
189 | # Continue execution even if initialization fails
190 | # The command might still work, or will fail with a
191 | # more specific error message
192 |
```
--------------------------------------------------------------------------------
/specs/SPEC-9 Signed Header Tenant Information.md:
--------------------------------------------------------------------------------
```markdown
1 | ---
2 | title: 'SPEC-9: Signed Header Tenant Information'
3 | type: spec
4 | permalink: specs/spec-9-signed-header-tenant-information
5 | tags:
6 | - authentication
7 | - tenant-isolation
8 | - proxy
9 | - security
10 | - mcp
11 | ---
12 |
13 | # SPEC-9: Signed Header Tenant Information
14 |
15 | ## Why
16 |
17 | WorkOS JWT templates don't work with MCP's dynamic client registration requirement, preventing us from getting tenant information directly in JWT tokens. We need an alternative secure method to pass tenant context from the Cloud Proxy Service to tenant instances.
18 |
19 | **Problem Context:**
20 | - MCP spec requires dynamic client registration
21 | - WorkOS JWT templates only apply to statically configured clients
22 | - Without tenant information, we can't properly route requests or isolate tenant data
23 | - Current JWT tokens only contain standard OIDC claims (sub, email, etc.)
24 |
25 | **Affected Areas:**
26 | - Cloud Proxy Service (`apps/cloud`) - request forwarding
27 | - Tenant API instances (`apps/api`) - tenant context validation
28 | - MCP Gateway (`apps/mcp`) - authentication flow
29 | - Overall tenant isolation security model
30 |
31 | ## What
32 |
33 | Implement HMAC-signed headers that the Cloud Proxy Service adds when forwarding requests to tenant instances. This provides secure, tamper-proof tenant information without relying on JWT custom claims.
34 |
35 | **Components:**
36 | - Header signing utility in Cloud Proxy Service
37 | - Header validation middleware in Tenant API instances
38 | - Shared secret configuration across services
39 | - Fallback mechanisms for development and error cases
40 |
41 | ## How (High Level)
42 |
43 | ### 1. Header Format
44 | Add these signed headers to all proxied requests:
45 | ```
46 | X-BM-Tenant-ID: {tenant_id}
47 | X-BM-Timestamp: {unix_timestamp}
48 | X-BM-Signature: {hmac_sha256_signature}
49 | ```
50 |
51 | ### 2. Signature Algorithm
52 | ```python
53 | # Canonical message format
54 | message = f"{tenant_id}:{timestamp}"
55 |
56 | # HMAC-SHA256 signature
57 | signature = hmac.new(
58 | key=shared_secret.encode('utf-8'),
59 | msg=message.encode('utf-8'),
60 | digestmod=hashlib.sha256
61 | ).hexdigest()
62 | ```
63 |
64 | ### 3. Implementation Flow
65 |
66 | #### Cloud Proxy Service (`apps/cloud`)
67 | 1. Extract `tenant_id` from authenticated user profile
68 | 2. Generate timestamp and canonical message
69 | 3. Sign message with shared secret
70 | 4. Add headers to request before forwarding to tenant instance
71 |
72 | #### Tenant API Instances (`apps/api`)
73 | 1. Middleware validates headers on all incoming requests
74 | 2. Extract tenant_id, timestamp from headers
75 | 3. Verify timestamp is within acceptable window (5 minutes)
76 | 4. Recompute signature and compare in constant time
77 | 5. If valid, make tenant context available to Basic Memory tools
78 |
79 | ### 4. Security Properties
80 | - **Authenticity**: Only services with shared secret can create valid signatures
81 | - **Integrity**: Header tampering invalidates signature
82 | - **Replay Protection**: Timestamp prevents reuse of old signatures
83 | - **Non-repudiation**: Each request is cryptographically tied to specific tenant
84 |
85 | ### 5. Configuration
86 | ```bash
87 | # Shared across Cloud Proxy and Tenant instances
88 | BM_TENANT_HEADER_SECRET=randomly-generated-256-bit-secret
89 |
90 | # Tenant API configuration
91 | BM_TENANT_HEADER_VALIDATION=true # true (production) | false (dev only)
92 | ```
93 |
94 | ## How to Evaluate
95 |
96 | ### Unit Tests
97 | - [ ] Header signing utility generates correct signatures
98 | - [ ] Header validation correctly accepts/rejects signatures
99 | - [ ] Timestamp validation within acceptable windows
100 | - [ ] Constant-time signature comparison prevents timing attacks
101 |
102 | ### Integration Tests
103 | - [ ] End-to-end request flow from MCP client → proxy → tenant
104 | - [ ] Tenant isolation verified with signed headers
105 | - [ ] Error handling for missing/invalid headers
106 | - [ ] Disabled validation in development environment
107 |
108 | ### Security Validation
109 | - [ ] Shared secret rotation procedure
110 | - [ ] Header tampering detection
111 | - [ ] Clock skew tolerance testing
112 | - [ ] Performance impact measurement
113 |
114 | ### Production Readiness
115 | - [ ] Logging and monitoring of header validation
116 | - [ ] Graceful degradation for header validation failures
117 | - [ ] Documentation for secret management
118 | - [ ] Deployment configuration templates
119 |
120 | ## Implementation Notes
121 |
122 | ### Shared Secret Management
123 | - Generate cryptographically secure 256-bit secret
124 | - Same secret deployed to Cloud Proxy and all Tenant instances
125 | - Consider secret rotation strategy for production
126 |
127 | ### Error Handling
128 | ```python
129 | # Strict mode (production)
130 | if not validate_headers(request):
131 | raise HTTPException(status_code=401, detail="Invalid tenant headers")
132 |
133 | # Fallback mode (development)
134 | if not validate_headers(request):
135 | logger.warning("Invalid headers, falling back to default tenant")
136 | tenant_id = "default"
137 | ```
138 |
139 | ### Performance Considerations
140 | - HMAC-SHA256 computation is fast (~microseconds)
141 | - Headers add ~200 bytes to each request
142 | - Validation happens once per request in middleware
143 |
144 | ## Benefits
145 |
146 | ✅ **Works with MCP dynamic client registration** - No dependency on JWT custom claims
147 | ✅ **Simple and reliable** - Standard HMAC signature approach
148 | ✅ **Secure by design** - Cryptographic authenticity and integrity
149 | ✅ **Infrastructure controlled** - No external service dependencies
150 | ✅ **Easy to implement** - Clear signature algorithm and validation
151 |
152 | ## Trade-offs
153 |
154 | ⚠️ **Shared secret management** - Need secure distribution and rotation
155 | ⚠️ **Clock synchronization** - Timestamp validation requires reasonably synced clocks
156 | ⚠️ **Header visibility** - Headers visible in logs (tenant_id not sensitive)
157 | ⚠️ **Additional complexity** - More moving parts in proxy forwarding
158 |
159 | ## Implementation Tasks
160 |
161 | ### Cloud Service (Header Signing)
162 | - [ ] Create `utils/header_signing.py` with HMAC-SHA256 signing function
163 | - [ ] Add `bm_tenant_header_secret` to Cloud service configuration
164 | - [ ] Update `ProxyService.forward_request()` to call signing utility
165 | - [ ] Add signed headers (X-BM-Tenant-ID, X-BM-Timestamp, X-BM-Signature)
166 |
167 | ### Tenant API (Header Validation)
168 | - [ ] Create `utils/header_validation.py` with signature verification
169 | - [ ] Add `bm_tenant_header_secret` to API service configuration
170 | - [ ] Create `TenantHeaderValidationMiddleware` class
171 | - [ ] Add middleware to FastAPI app (before other middleware)
172 | - [ ] Skip validation for `/health` endpoint
173 | - [ ] Store validated tenant_id in request.state
174 |
175 | ### Testing
176 | - [ ] Unit test for header signing utility
177 | - [ ] Unit test for header validation utility
178 | - [ ] Integration test for proxy → tenant flow
179 | - [ ] Test invalid/missing header handling
180 | - [ ] Test timestamp window validation
181 | - [ ] Test signature tampering detection
182 |
183 | ### Configuration & Deployment
184 | - [ ] Update `.env.example` with BM_TENANT_HEADER_SECRET
185 | - [ ] Generate secure 256-bit secret for production
186 | - [ ] Update Fly.io secrets for both services
187 | - [ ] Document secret rotation procedure
188 |
189 | ## Status
190 |
191 | - [x] **Specification Complete** - Design finalized and documented
192 | - [ ] **Implementation Started** - Header signing utility development
193 | - [ ] **Cloud Proxy Updated** - ProxyService adds signed headers
194 | - [ ] **Tenant Validation Added** - Middleware validates headers
195 | - [ ] **Testing Complete** - All validation criteria met
196 | - [ ] **Production Deployed** - Live with tenant isolation via headers
```
--------------------------------------------------------------------------------
/src/basic_memory/cli/commands/cloud/upload.py:
--------------------------------------------------------------------------------
```python
1 | """WebDAV upload functionality for basic-memory projects."""
2 |
3 | import os
4 | from pathlib import Path
5 |
6 | import aiofiles
7 | import httpx
8 |
9 | from basic_memory.ignore_utils import load_gitignore_patterns, should_ignore_path
10 | from basic_memory.mcp.async_client import get_client
11 | from basic_memory.mcp.tools.utils import call_put
12 |
13 |
14 | async def upload_path(
15 | local_path: Path,
16 | project_name: str,
17 | verbose: bool = False,
18 | use_gitignore: bool = True,
19 | dry_run: bool = False,
20 | ) -> bool:
21 | """
22 | Upload a file or directory to cloud project via WebDAV.
23 |
24 | Args:
25 | local_path: Path to local file or directory
26 | project_name: Name of cloud project (destination)
27 | verbose: Show detailed information about filtering and upload
28 | use_gitignore: If False, skip .gitignore patterns (still use .bmignore)
29 | dry_run: If True, show what would be uploaded without uploading
30 |
31 | Returns:
32 | True if upload succeeded, False otherwise
33 | """
34 | try:
35 | # Resolve path
36 | local_path = local_path.resolve()
37 |
38 | # Check if path exists
39 | if not local_path.exists():
40 | print(f"Error: Path does not exist: {local_path}")
41 | return False
42 |
43 | # Get files to upload
44 | if local_path.is_file():
45 | files_to_upload = [(local_path, local_path.name)]
46 | if verbose:
47 | print(f"Uploading single file: {local_path.name}")
48 | else:
49 | files_to_upload = _get_files_to_upload(local_path, verbose, use_gitignore)
50 |
51 | if not files_to_upload:
52 | print("No files found to upload")
53 | if verbose:
54 | print(
55 | "\nTip: Use --verbose to see which files are being filtered, "
56 | "or --no-gitignore to skip .gitignore patterns"
57 | )
58 | return True
59 |
60 | print(f"Found {len(files_to_upload)} file(s) to upload")
61 |
62 | # Calculate total size
63 | total_bytes = sum(file_path.stat().st_size for file_path, _ in files_to_upload)
64 |
65 | # If dry run, just show what would be uploaded
66 | if dry_run:
67 | print("\nFiles that would be uploaded:")
68 | for file_path, relative_path in files_to_upload:
69 | size = file_path.stat().st_size
70 | if size < 1024:
71 | size_str = f"{size} bytes"
72 | elif size < 1024 * 1024:
73 | size_str = f"{size / 1024:.1f} KB"
74 | else:
75 | size_str = f"{size / (1024 * 1024):.1f} MB"
76 | print(f" {relative_path} ({size_str})")
77 | else:
78 | # Upload files using httpx
79 | async with get_client() as client:
80 | for i, (file_path, relative_path) in enumerate(files_to_upload, 1):
81 | # Build remote path: /webdav/{project_name}/{relative_path}
82 | remote_path = f"/webdav/{project_name}/{relative_path}"
83 | print(f"Uploading {relative_path} ({i}/{len(files_to_upload)})")
84 |
85 | # Get file modification time
86 | file_stat = file_path.stat()
87 | mtime = int(file_stat.st_mtime)
88 |
89 | # Read file content asynchronously
90 | async with aiofiles.open(file_path, "rb") as f:
91 | content = await f.read()
92 |
93 | # Upload via HTTP PUT to WebDAV endpoint with mtime header
94 | # Using X-OC-Mtime (ownCloud/Nextcloud standard)
95 | response = await call_put(
96 | client, remote_path, content=content, headers={"X-OC-Mtime": str(mtime)}
97 | )
98 | response.raise_for_status()
99 |
100 | # Format total size based on magnitude
101 | if total_bytes < 1024:
102 | size_str = f"{total_bytes} bytes"
103 | elif total_bytes < 1024 * 1024:
104 | size_str = f"{total_bytes / 1024:.1f} KB"
105 | else:
106 | size_str = f"{total_bytes / (1024 * 1024):.1f} MB"
107 |
108 | if dry_run:
109 | print(f"\nTotal: {len(files_to_upload)} file(s) ({size_str})")
110 | else:
111 | print(f"✓ Upload complete: {len(files_to_upload)} file(s) ({size_str})")
112 |
113 | return True
114 |
115 | except httpx.HTTPStatusError as e:
116 | print(f"Upload failed: HTTP {e.response.status_code} - {e.response.text}")
117 | return False
118 | except Exception as e:
119 | print(f"Upload failed: {e}")
120 | return False
121 |
122 |
123 | def _get_files_to_upload(
124 | directory: Path, verbose: bool = False, use_gitignore: bool = True
125 | ) -> list[tuple[Path, str]]:
126 | """
127 | Get list of files to upload from directory.
128 |
129 | Uses .bmignore and optionally .gitignore patterns for filtering.
130 |
131 | Args:
132 | directory: Directory to scan
133 | verbose: Show detailed filtering information
134 | use_gitignore: If False, skip .gitignore patterns (still use .bmignore)
135 |
136 | Returns:
137 | List of (absolute_path, relative_path) tuples
138 | """
139 | files = []
140 | ignored_files = []
141 |
142 | # Load ignore patterns from .bmignore and optionally .gitignore
143 | ignore_patterns = load_gitignore_patterns(directory, use_gitignore=use_gitignore)
144 |
145 | if verbose:
146 | gitignore_path = directory / ".gitignore"
147 | gitignore_exists = gitignore_path.exists() and use_gitignore
148 | print(f"\nScanning directory: {directory}")
149 | print("Using .bmignore: Yes")
150 | print(f"Using .gitignore: {'Yes' if gitignore_exists else 'No'}")
151 | print(f"Ignore patterns loaded: {len(ignore_patterns)}")
152 | if ignore_patterns and len(ignore_patterns) <= 20:
153 | print(f"Patterns: {', '.join(sorted(ignore_patterns))}")
154 | print()
155 |
156 | # Walk through directory
157 | for root, dirs, filenames in os.walk(directory):
158 | root_path = Path(root)
159 |
160 | # Filter directories based on ignore patterns
161 | filtered_dirs = []
162 | for d in dirs:
163 | dir_path = root_path / d
164 | if should_ignore_path(dir_path, directory, ignore_patterns):
165 | if verbose:
166 | rel_path = dir_path.relative_to(directory)
167 | print(f" [IGNORED DIR] {rel_path}/")
168 | else:
169 | filtered_dirs.append(d)
170 | dirs[:] = filtered_dirs
171 |
172 | # Process files
173 | for filename in filenames:
174 | file_path = root_path / filename
175 |
176 | # Calculate relative path for display/remote
177 | rel_path = file_path.relative_to(directory)
178 | remote_path = str(rel_path).replace("\\", "/")
179 |
180 | # Check if file should be ignored
181 | if should_ignore_path(file_path, directory, ignore_patterns):
182 | ignored_files.append(remote_path)
183 | if verbose:
184 | print(f" [IGNORED] {remote_path}")
185 | continue
186 |
187 | if verbose:
188 | print(f" [INCLUDE] {remote_path}")
189 |
190 | files.append((file_path, remote_path))
191 |
192 | if verbose:
193 | print("\nSummary:")
194 | print(f" Files to upload: {len(files)}")
195 | print(f" Files ignored: {len(ignored_files)}")
196 |
197 | return files
198 |
```
--------------------------------------------------------------------------------
/test-int/mcp/test_build_context_validation.py:
--------------------------------------------------------------------------------
```python
1 | """Integration tests for build_context memory URL validation."""
2 |
3 | import pytest
4 | from fastmcp import Client
5 |
6 |
7 | @pytest.mark.asyncio
8 | async def test_build_context_valid_urls(mcp_server, app, test_project):
9 | """Test that build_context works with valid memory URLs."""
10 |
11 | async with Client(mcp_server) as client:
12 | # Create a test note to ensure we have something to find
13 | await client.call_tool(
14 | "write_note",
15 | {
16 | "project": test_project.name,
17 | "title": "URL Validation Test",
18 | "folder": "testing",
19 | "content": "# URL Validation Test\n\nThis note tests URL validation.",
20 | "tags": "test,validation",
21 | },
22 | )
23 |
24 | # Test various valid URL formats
25 | valid_urls = [
26 | "memory://testing/url-validation-test", # Full memory URL
27 | "testing/url-validation-test", # Relative path
28 | "testing/*", # Pattern matching
29 | ]
30 |
31 | for url in valid_urls:
32 | result = await client.call_tool(
33 | "build_context", {"project": test_project.name, "url": url}
34 | )
35 |
36 | # Should return a valid GraphContext response
37 | assert len(result.content) == 1
38 | response = result.content[0].text # pyright: ignore [reportAttributeAccessIssue]
39 | assert '"results"' in response # Should contain results structure
40 | assert '"metadata"' in response # Should contain metadata
41 |
42 |
43 | @pytest.mark.asyncio
44 | async def test_build_context_invalid_urls_fail_validation(mcp_server, app, test_project):
45 | """Test that build_context properly validates and rejects invalid memory URLs."""
46 |
47 | async with Client(mcp_server) as client:
48 | # Test cases: (invalid_url, expected_error_fragment)
49 | invalid_test_cases = [
50 | ("memory//test", "double slashes"),
51 | ("invalid://test", "protocol scheme"),
52 | ("notes<brackets>", "invalid characters"),
53 | ('notes"quotes"', "invalid characters"),
54 | ]
55 |
56 | for invalid_url, expected_error in invalid_test_cases:
57 | with pytest.raises(Exception) as exc_info:
58 | await client.call_tool(
59 | "build_context", {"project": test_project.name, "url": invalid_url}
60 | )
61 |
62 | error_message = str(exc_info.value).lower()
63 | assert expected_error in error_message, (
64 | f"URL '{invalid_url}' should fail with '{expected_error}' error"
65 | )
66 |
67 |
68 | @pytest.mark.asyncio
69 | async def test_build_context_empty_urls_fail_validation(mcp_server, app, test_project):
70 | """Test that empty or whitespace-only URLs fail validation."""
71 |
72 | async with Client(mcp_server) as client:
73 | # These should fail validation
74 | empty_urls = [
75 | "", # Empty string
76 | " ", # Whitespace only
77 | ]
78 |
79 | for empty_url in empty_urls:
80 | with pytest.raises(Exception) as exc_info:
81 | await client.call_tool(
82 | "build_context", {"project": test_project.name, "url": empty_url}
83 | )
84 |
85 | error_message = str(exc_info.value)
86 | # Should fail with validation error
87 | assert (
88 | "cannot be empty" in error_message
89 | or "empty or whitespace" in error_message
90 | or "value_error" in error_message
91 | or "should be non-empty" in error_message
92 | )
93 |
94 |
95 | @pytest.mark.asyncio
96 | async def test_build_context_nonexistent_urls_return_empty_results(mcp_server, app, test_project):
97 | """Test that valid but nonexistent URLs return empty results (not errors)."""
98 |
99 | async with Client(mcp_server) as client:
100 | # These are valid URL formats but don't exist in the system
101 | nonexistent_valid_urls = [
102 | "memory://nonexistent/note",
103 | "nonexistent/note",
104 | "missing/*",
105 | ]
106 |
107 | for url in nonexistent_valid_urls:
108 | result = await client.call_tool(
109 | "build_context", {"project": test_project.name, "url": url}
110 | )
111 |
112 | # Should return valid response with empty results
113 | assert len(result.content) == 1
114 | response = result.content[0].text # pyright: ignore [reportAttributeAccessIssue]
115 | assert '"results":[]' in response # Empty results
116 | assert '"total_results":0' in response # Zero count
117 | assert '"metadata"' in response # But should have metadata
118 |
119 |
120 | @pytest.mark.asyncio
121 | async def test_build_context_error_messages_are_helpful(mcp_server, app, test_project):
122 | """Test that validation error messages provide helpful guidance."""
123 |
124 | async with Client(mcp_server) as client:
125 | # Test double slash error message
126 | with pytest.raises(Exception) as exc_info:
127 | await client.call_tool(
128 | "build_context", {"project": test_project.name, "url": "memory//bad"}
129 | )
130 |
131 | error_msg = str(exc_info.value).lower()
132 | # Should contain validation error info
133 | assert (
134 | "double slashes" in error_msg
135 | or "value_error" in error_msg
136 | or "validation error" in error_msg
137 | )
138 |
139 | # Test protocol scheme error message
140 | with pytest.raises(Exception) as exc_info:
141 | await client.call_tool(
142 | "build_context", {"project": test_project.name, "url": "http://example.com"}
143 | )
144 |
145 | error_msg = str(exc_info.value).lower()
146 | assert (
147 | "protocol scheme" in error_msg
148 | or "protocol" in error_msg
149 | or "value_error" in error_msg
150 | or "validation error" in error_msg
151 | )
152 |
153 |
154 | @pytest.mark.asyncio
155 | async def test_build_context_pattern_matching_works(mcp_server, app, test_project):
156 | """Test that valid pattern matching URLs work correctly."""
157 |
158 | async with Client(mcp_server) as client:
159 | # Create multiple test notes
160 | test_notes = [
161 | ("Pattern Test One", "patterns", "# Pattern Test One\n\nFirst pattern test."),
162 | ("Pattern Test Two", "patterns", "# Pattern Test Two\n\nSecond pattern test."),
163 | ("Other Note", "other", "# Other Note\n\nNot a pattern match."),
164 | ]
165 |
166 | for title, folder, content in test_notes:
167 | await client.call_tool(
168 | "write_note",
169 | {
170 | "project": test_project.name,
171 | "title": title,
172 | "folder": folder,
173 | "content": content,
174 | },
175 | )
176 |
177 | # Test pattern matching
178 | result = await client.call_tool(
179 | "build_context", {"project": test_project.name, "url": "patterns/*"}
180 | )
181 |
182 | assert len(result.content) == 1
183 | response = result.content[0].text # pyright: ignore [reportAttributeAccessIssue]
184 |
185 | # Should find the pattern matches but not the other note
186 | assert '"total_results":2' in response or '"primary_count":2' in response
187 | assert "Pattern Test" in response
188 | assert "Other Note" not in response
189 |
```
--------------------------------------------------------------------------------
/tests/mcp/test_tool_resource.py:
--------------------------------------------------------------------------------
```python
1 | """Tests for resource tools that exercise the full stack with SQLite."""
2 |
3 | import io
4 | import base64
5 | from PIL import Image as PILImage
6 |
7 | import pytest
8 | from mcp.server.fastmcp.exceptions import ToolError
9 |
10 | from basic_memory.mcp.tools import read_content, write_note
11 | from basic_memory.mcp.tools.read_content import (
12 | calculate_target_params,
13 | resize_image,
14 | optimize_image,
15 | )
16 |
17 |
18 | @pytest.mark.asyncio
19 | async def test_read_file_text_file(app, synced_files, test_project):
20 | """Test reading a text file.
21 |
22 | Should:
23 | - Correctly identify text content
24 | - Return the content as text
25 | - Include correct metadata
26 | """
27 | # First create a text file via notes
28 | result = await write_note.fn(
29 | project=test_project.name,
30 | title="Text Resource",
31 | folder="test",
32 | content="This is a test text resource",
33 | tags=["test", "resource"],
34 | )
35 | assert result is not None
36 |
37 | # Now read it as a resource
38 | response = await read_content.fn("test/text-resource", project=test_project.name)
39 |
40 | assert response["type"] == "text"
41 | assert "This is a test text resource" in response["text"]
42 | assert response["content_type"].startswith("text/")
43 | assert response["encoding"] == "utf-8"
44 |
45 |
46 | @pytest.mark.asyncio
47 | async def test_read_content_file_path(app, synced_files, test_project):
48 | """Test reading a text file.
49 |
50 | Should:
51 | - Correctly identify text content
52 | - Return the content as text
53 | - Include correct metadata
54 | """
55 | # First create a text file via notes
56 | result = await write_note.fn(
57 | project=test_project.name,
58 | title="Text Resource",
59 | folder="test",
60 | content="This is a test text resource",
61 | tags=["test", "resource"],
62 | )
63 | assert result is not None
64 |
65 | # Now read it as a resource
66 | response = await read_content.fn("test/Text Resource.md", project=test_project.name)
67 |
68 | assert response["type"] == "text"
69 | assert "This is a test text resource" in response["text"]
70 | assert response["content_type"].startswith("text/")
71 | assert response["encoding"] == "utf-8"
72 |
73 |
74 | @pytest.mark.asyncio
75 | async def test_read_file_image_file(app, synced_files, test_project):
76 | """Test reading an image file.
77 |
78 | Should:
79 | - Correctly identify image content
80 | - Optimize the image
81 | - Return base64 encoded image data
82 | """
83 | # Get the path to the synced image file
84 | image_path = synced_files["image"].name
85 |
86 | # Read it as a resource
87 | response = await read_content.fn(image_path, project=test_project.name)
88 |
89 | assert response["type"] == "image"
90 | assert response["source"]["type"] == "base64"
91 | assert response["source"]["media_type"] == "image/jpeg"
92 |
93 | # Verify the image data is valid base64 that can be decoded
94 | img_data = base64.b64decode(response["source"]["data"])
95 | assert len(img_data) > 0
96 |
97 | # Should be able to open as an image
98 | img = PILImage.open(io.BytesIO(img_data))
99 | assert img.width > 0
100 | assert img.height > 0
101 |
102 |
103 | @pytest.mark.asyncio
104 | async def test_read_file_pdf_file(app, synced_files, test_project):
105 | """Test reading a PDF file.
106 |
107 | Should:
108 | - Correctly identify PDF content
109 | - Return base64 encoded PDF data
110 | """
111 | # Get the path to the synced PDF file
112 | pdf_path = synced_files["pdf"].name
113 |
114 | # Read it as a resource
115 | response = await read_content.fn(pdf_path, project=test_project.name)
116 |
117 | assert response["type"] == "document"
118 | assert response["source"]["type"] == "base64"
119 | assert response["source"]["media_type"] == "application/pdf"
120 |
121 | # Verify the PDF data is valid base64 that can be decoded
122 | pdf_data = base64.b64decode(response["source"]["data"])
123 | assert len(pdf_data) > 0
124 | assert pdf_data.startswith(b"%PDF") # PDF signature
125 |
126 |
127 | @pytest.mark.asyncio
128 | async def test_read_file_not_found(app, test_project):
129 | """Test trying to read a non-existent"""
130 | with pytest.raises(ToolError, match="Resource not found"):
131 | await read_content.fn("does-not-exist", project=test_project.name)
132 |
133 |
134 | @pytest.mark.asyncio
135 | async def test_read_file_memory_url(app, synced_files, test_project):
136 | """Test reading a resource using a memory:// URL."""
137 | # Create a text file via notes
138 | await write_note.fn(
139 | project=test_project.name,
140 | title="Memory URL Test",
141 | folder="test",
142 | content="Testing memory:// URL handling for resources",
143 | )
144 |
145 | # Read it with a memory:// URL
146 | memory_url = "memory://test/memory-url-test"
147 | response = await read_content.fn(memory_url, project=test_project.name)
148 |
149 | assert response["type"] == "text"
150 | assert "Testing memory:// URL handling for resources" in response["text"]
151 |
152 |
153 | @pytest.mark.asyncio
154 | async def test_image_optimization_functions(app):
155 | """Test the image optimization helper functions."""
156 | # Create a test image
157 | img = PILImage.new("RGB", (1000, 800), color="white")
158 |
159 | # Test calculate_target_params function
160 | # Small image
161 | quality, size = calculate_target_params(100000)
162 | assert quality == 70
163 | assert size == 1000
164 |
165 | # Medium image
166 | quality, size = calculate_target_params(800000)
167 | assert quality == 60
168 | assert size == 800
169 |
170 | # Large image
171 | quality, size = calculate_target_params(2000000)
172 | assert quality == 50
173 | assert size == 600
174 |
175 | # Test resize_image function
176 | # Image that needs resizing
177 | resized = resize_image(img, 500)
178 | assert resized.width <= 500
179 | assert resized.height <= 500
180 |
181 | # Image that doesn't need resizing
182 | small_img = PILImage.new("RGB", (300, 200), color="white")
183 | resized = resize_image(small_img, 500)
184 | assert resized.width == 300
185 | assert resized.height == 200
186 |
187 | # Test optimize_image function
188 | img_bytes = io.BytesIO()
189 | img.save(img_bytes, format="PNG")
190 | img_bytes.seek(0)
191 | content_length = len(img_bytes.getvalue())
192 |
193 | # In a small test image, optimization might make the image larger
194 | # because of JPEG overhead. Let's just test that it returns something
195 | optimized = optimize_image(img, content_length)
196 | assert len(optimized) > 0
197 |
198 |
199 | @pytest.mark.asyncio
200 | async def test_image_conversion(app, synced_files, test_project):
201 | """Test reading an image and verify conversion works.
202 |
203 | Should:
204 | - Handle image content correctly
205 | - Return optimized image data
206 | """
207 | # Use the synced image file that's already part of our test fixtures
208 | image_path = synced_files["image"].name
209 |
210 | # Test reading the resource
211 | response = await read_content.fn(image_path, project=test_project.name)
212 |
213 | assert response["type"] == "image"
214 | assert response["source"]["media_type"] == "image/jpeg"
215 |
216 | # Verify the image data is valid
217 | img_data = base64.b64decode(response["source"]["data"])
218 | img = PILImage.open(io.BytesIO(img_data))
219 | assert img.width > 0
220 | assert img.height > 0
221 | assert img.mode == "RGB" # Should be in RGB mode
222 |
223 |
224 | # Skip testing the large document size handling since it would require
225 | # complex mocking of internal logic. We've already tested the happy path
226 | # with the PDF file, and the error handling with our updated tool_utils tests.
227 | # We have 100% coverage of this code in read_file.py according to the coverage report.
228 |
```
--------------------------------------------------------------------------------
/tests/services/test_entity_service_disable_permalinks.py:
--------------------------------------------------------------------------------
```python
1 | """Tests for EntityService with disable_permalinks flag."""
2 |
3 | from textwrap import dedent
4 | import pytest
5 | import yaml
6 |
7 | from basic_memory.config import BasicMemoryConfig
8 | from basic_memory.schemas import Entity as EntitySchema
9 | from basic_memory.services import FileService
10 | from basic_memory.services.entity_service import EntityService
11 |
12 |
13 | @pytest.mark.asyncio
14 | async def test_create_entity_with_permalinks_disabled(
15 | entity_repository,
16 | observation_repository,
17 | relation_repository,
18 | entity_parser,
19 | file_service: FileService,
20 | link_resolver,
21 | ):
22 | """Test that entities created with disable_permalinks=True don't have permalinks."""
23 | # Create entity service with permalinks disabled
24 | app_config = BasicMemoryConfig(disable_permalinks=True)
25 | entity_service = EntityService(
26 | entity_parser=entity_parser,
27 | entity_repository=entity_repository,
28 | observation_repository=observation_repository,
29 | relation_repository=relation_repository,
30 | file_service=file_service,
31 | link_resolver=link_resolver,
32 | app_config=app_config,
33 | )
34 |
35 | entity_data = EntitySchema(
36 | title="Test Entity",
37 | folder="test",
38 | entity_type="note",
39 | content="Test content",
40 | )
41 |
42 | # Create entity
43 | entity = await entity_service.create_entity(entity_data)
44 |
45 | # Assert entity has no permalink
46 | assert entity.permalink is None
47 |
48 | # Verify file frontmatter doesn't contain permalink
49 | file_path = file_service.get_entity_path(entity)
50 | file_content, _ = await file_service.read_file(file_path)
51 | _, frontmatter, doc_content = file_content.split("---", 2)
52 | metadata = yaml.safe_load(frontmatter)
53 |
54 | assert "permalink" not in metadata
55 | assert metadata["title"] == "Test Entity"
56 | assert metadata["type"] == "note"
57 |
58 |
59 | @pytest.mark.asyncio
60 | async def test_update_entity_with_permalinks_disabled(
61 | entity_repository,
62 | observation_repository,
63 | relation_repository,
64 | entity_parser,
65 | file_service: FileService,
66 | link_resolver,
67 | ):
68 | """Test that entities updated with disable_permalinks=True don't get permalinks added."""
69 | # First create with permalinks enabled
70 | app_config_enabled = BasicMemoryConfig(disable_permalinks=False)
71 | entity_service_enabled = EntityService(
72 | entity_parser=entity_parser,
73 | entity_repository=entity_repository,
74 | observation_repository=observation_repository,
75 | relation_repository=relation_repository,
76 | file_service=file_service,
77 | link_resolver=link_resolver,
78 | app_config=app_config_enabled,
79 | )
80 |
81 | entity_data = EntitySchema(
82 | title="Test Entity",
83 | folder="test",
84 | entity_type="note",
85 | content="Original content",
86 | )
87 |
88 | # Create entity with permalinks enabled
89 | entity = await entity_service_enabled.create_entity(entity_data)
90 | assert entity.permalink is not None
91 | original_permalink = entity.permalink
92 |
93 | # Now create service with permalinks disabled
94 | app_config_disabled = BasicMemoryConfig(disable_permalinks=True)
95 | entity_service_disabled = EntityService(
96 | entity_parser=entity_parser,
97 | entity_repository=entity_repository,
98 | observation_repository=observation_repository,
99 | relation_repository=relation_repository,
100 | file_service=file_service,
101 | link_resolver=link_resolver,
102 | app_config=app_config_disabled,
103 | )
104 |
105 | # Update entity with permalinks disabled
106 | entity_data.content = "Updated content"
107 | updated = await entity_service_disabled.update_entity(entity, entity_data)
108 |
109 | # Permalink should remain unchanged (not removed, just not updated)
110 | assert updated.permalink == original_permalink
111 |
112 | # Verify file still has the original permalink
113 | file_path = file_service.get_entity_path(updated)
114 | file_content, _ = await file_service.read_file(file_path)
115 | assert "Updated content" in file_content
116 | assert f"permalink: {original_permalink}" in file_content
117 |
118 |
119 | @pytest.mark.asyncio
120 | async def test_create_entity_with_content_frontmatter_permalinks_disabled(
121 | entity_repository,
122 | observation_repository,
123 | relation_repository,
124 | entity_parser,
125 | file_service: FileService,
126 | link_resolver,
127 | ):
128 | """Test that content frontmatter permalinks are ignored when disabled."""
129 | # Create entity service with permalinks disabled
130 | app_config = BasicMemoryConfig(disable_permalinks=True)
131 | entity_service = EntityService(
132 | entity_parser=entity_parser,
133 | entity_repository=entity_repository,
134 | observation_repository=observation_repository,
135 | relation_repository=relation_repository,
136 | file_service=file_service,
137 | link_resolver=link_resolver,
138 | app_config=app_config,
139 | )
140 |
141 | # Content with frontmatter containing permalink
142 | content = dedent(
143 | """
144 | ---
145 | permalink: custom-permalink
146 | ---
147 | # Test Content
148 | """
149 | ).strip()
150 |
151 | entity_data = EntitySchema(
152 | title="Test Entity",
153 | folder="test",
154 | entity_type="note",
155 | content=content,
156 | )
157 |
158 | # Create entity
159 | entity = await entity_service.create_entity(entity_data)
160 |
161 | # Entity should not have a permalink set
162 | assert entity.permalink is None
163 |
164 | # Verify file doesn't have permalink in frontmatter
165 | file_path = file_service.get_entity_path(entity)
166 | file_content, _ = await file_service.read_file(file_path)
167 | _, frontmatter, doc_content = file_content.split("---", 2)
168 | metadata = yaml.safe_load(frontmatter)
169 |
170 | # The permalink from content frontmatter should not be present
171 | assert "permalink" not in metadata
172 |
173 |
174 | @pytest.mark.asyncio
175 | async def test_move_entity_with_permalinks_disabled(
176 | entity_repository,
177 | observation_repository,
178 | relation_repository,
179 | entity_parser,
180 | file_service: FileService,
181 | link_resolver,
182 | project_config,
183 | ):
184 | """Test that moving an entity with disable_permalinks=True doesn't update permalinks."""
185 | # First create with permalinks enabled
186 | app_config = BasicMemoryConfig(disable_permalinks=False, update_permalinks_on_move=True)
187 | entity_service = EntityService(
188 | entity_parser=entity_parser,
189 | entity_repository=entity_repository,
190 | observation_repository=observation_repository,
191 | relation_repository=relation_repository,
192 | file_service=file_service,
193 | link_resolver=link_resolver,
194 | app_config=app_config,
195 | )
196 |
197 | entity_data = EntitySchema(
198 | title="Test Entity",
199 | folder="test",
200 | entity_type="note",
201 | content="Test content",
202 | )
203 |
204 | # Create entity
205 | entity = await entity_service.create_entity(entity_data)
206 | original_permalink = entity.permalink
207 |
208 | # Now disable permalinks
209 | app_config_disabled = BasicMemoryConfig(disable_permalinks=True, update_permalinks_on_move=True)
210 |
211 | # Move entity
212 | moved = await entity_service.move_entity(
213 | identifier=entity.permalink,
214 | destination_path="new_folder/test_entity.md",
215 | project_config=project_config,
216 | app_config=app_config_disabled,
217 | )
218 |
219 | # Permalink should remain unchanged even though update_permalinks_on_move is True
220 | assert moved.permalink == original_permalink
221 |
```
--------------------------------------------------------------------------------
/tests/db/test_issue_254_foreign_key_constraints.py:
--------------------------------------------------------------------------------
```python
1 | """Test to verify that issue #254 is fixed.
2 |
3 | Issue #254: Foreign key constraint failures when deleting projects with related entities.
4 |
5 | The issue was that when migration 647e7a75e2cd recreated the project table,
6 | it did not re-establish the foreign key constraint from entity.project_id to project.id
7 | with CASCADE DELETE, causing foreign key constraint failures when trying to delete
8 | projects that have related entities.
9 |
10 | Migration a1b2c3d4e5f6 was created to fix this by adding the missing foreign key
11 | constraint with CASCADE DELETE behavior.
12 |
13 | This test file verifies that the fix works correctly in production databases
14 | that have had the migration applied.
15 | """
16 |
17 | import tempfile
18 | from datetime import datetime, timezone
19 | from pathlib import Path
20 |
21 | import pytest
22 |
23 | from basic_memory.services.project_service import ProjectService
24 |
25 |
26 | # @pytest.mark.skip(reason="Issue #254 not fully resolved yet - foreign key constraint errors still occur")
27 | @pytest.mark.asyncio
28 | async def test_issue_254_foreign_key_constraint_fix(project_service: ProjectService):
29 | """Test to verify issue #254 is fixed: project removal with foreign key constraints.
30 |
31 | This test reproduces the exact scenario from issue #254:
32 | 1. Create a project
33 | 2. Create entities, observations, and relations linked to that project
34 | 3. Attempt to remove the project
35 | 4. Verify it succeeds without "FOREIGN KEY constraint failed" errors
36 | 5. Verify all related data is properly cleaned up via CASCADE DELETE
37 |
38 | Once issue #254 is fully fixed, remove the @pytest.mark.skip decorator.
39 | """
40 | test_project_name = "issue-254-verification"
41 | with tempfile.TemporaryDirectory() as temp_dir:
42 | test_root = Path(temp_dir)
43 | test_project_path = str(test_root / "issue-254-verification")
44 |
45 | # Step 1: Create test project
46 | await project_service.add_project(test_project_name, test_project_path)
47 | project = await project_service.get_project(test_project_name)
48 | assert project is not None, "Project should be created successfully"
49 |
50 | # Step 2: Create related entities that would cause foreign key constraint issues
51 | from basic_memory.repository.entity_repository import EntityRepository
52 | from basic_memory.repository.observation_repository import ObservationRepository
53 | from basic_memory.repository.relation_repository import RelationRepository
54 |
55 | entity_repo = EntityRepository(
56 | project_service.repository.session_maker, project_id=project.id
57 | )
58 | obs_repo = ObservationRepository(
59 | project_service.repository.session_maker, project_id=project.id
60 | )
61 | rel_repo = RelationRepository(
62 | project_service.repository.session_maker, project_id=project.id
63 | )
64 |
65 | # Create entity
66 | entity_data = {
67 | "title": "Issue 254 Test Entity",
68 | "entity_type": "note",
69 | "content_type": "text/markdown",
70 | "project_id": project.id,
71 | "permalink": "issue-254-entity",
72 | "file_path": "issue-254-entity.md",
73 | "checksum": "issue254test",
74 | "created_at": datetime.now(timezone.utc),
75 | "updated_at": datetime.now(timezone.utc),
76 | }
77 | entity = await entity_repo.create(entity_data)
78 |
79 | # Create observation linked to entity
80 | observation_data = {
81 | "entity_id": entity.id,
82 | "content": "This observation should be cascade deleted",
83 | "category": "test",
84 | }
85 | observation = await obs_repo.create(observation_data)
86 |
87 | # Create relation involving the entity
88 | relation_data = {
89 | "from_id": entity.id,
90 | "to_name": "some-other-entity",
91 | "relation_type": "relates-to",
92 | }
93 | relation = await rel_repo.create(relation_data)
94 |
95 | # Step 3: Attempt to remove the project
96 | # This is where issue #254 manifested - should NOT raise "FOREIGN KEY constraint failed"
97 | try:
98 | await project_service.remove_project(test_project_name)
99 | except Exception as e:
100 | if "FOREIGN KEY constraint failed" in str(e):
101 | pytest.fail(
102 | f"Issue #254 not fixed - foreign key constraint error still occurs: {e}. "
103 | f"The migration a1b2c3d4e5f6 may not have been applied correctly or "
104 | f"the CASCADE DELETE constraint is not working as expected."
105 | )
106 | else:
107 | # Re-raise unexpected errors
108 | raise
109 |
110 | # Step 4: Verify project was successfully removed
111 | removed_project = await project_service.get_project(test_project_name)
112 | assert removed_project is None, "Project should have been removed"
113 |
114 | # Step 5: Verify related data was cascade deleted
115 | remaining_entity = await entity_repo.find_by_id(entity.id)
116 | assert remaining_entity is None, "Entity should have been cascade deleted"
117 |
118 | remaining_observation = await obs_repo.find_by_id(observation.id)
119 | assert remaining_observation is None, "Observation should have been cascade deleted"
120 |
121 | remaining_relation = await rel_repo.find_by_id(relation.id)
122 | assert remaining_relation is None, "Relation should have been cascade deleted"
123 |
124 |
125 | @pytest.mark.asyncio
126 | async def test_issue_254_reproduction(project_service: ProjectService):
127 | """Test that reproduces issue #254 to document the current state.
128 |
129 | This test demonstrates the current behavior and will fail until the issue is fixed.
130 | It serves as documentation of what the problem was.
131 | """
132 | test_project_name = "issue-254-reproduction"
133 | with tempfile.TemporaryDirectory() as temp_dir:
134 | test_root = Path(temp_dir)
135 | test_project_path = str(test_root / "issue-254-reproduction")
136 |
137 | # Create project and entity
138 | await project_service.add_project(test_project_name, test_project_path)
139 | project = await project_service.get_project(test_project_name)
140 |
141 | from basic_memory.repository.entity_repository import EntityRepository
142 |
143 | entity_repo = EntityRepository(
144 | project_service.repository.session_maker, project_id=project.id
145 | )
146 |
147 | entity_data = {
148 | "title": "Reproduction Entity",
149 | "entity_type": "note",
150 | "content_type": "text/markdown",
151 | "project_id": project.id,
152 | "permalink": "reproduction-entity",
153 | "file_path": "reproduction-entity.md",
154 | "checksum": "repro123",
155 | "created_at": datetime.now(timezone.utc),
156 | "updated_at": datetime.now(timezone.utc),
157 | }
158 | await entity_repo.create(entity_data)
159 |
160 | # This should eventually work without errors once issue #254 is fixed
161 | # with pytest.raises(Exception) as exc_info:
162 | await project_service.remove_project(test_project_name)
163 |
164 | # Document the current error for tracking
165 | # error_message = str(exc_info.value)
166 | # assert any(keyword in error_message for keyword in [
167 | # "FOREIGN KEY constraint failed",
168 | # "constraint",
169 | # "integrity"
170 | # ]), f"Expected foreign key or integrity constraint error, got: {error_message}"
171 |
```
--------------------------------------------------------------------------------
/src/basic_memory/mcp/resources/ai_assistant_guide.md:
--------------------------------------------------------------------------------
```markdown
1 | # AI Assistant Guide for Basic Memory
2 |
3 | Quick reference for using Basic Memory tools effectively through MCP.
4 |
5 | **For comprehensive coverage**: See the [Extended AI Assistant Guide](https://github.com/basicmachines-co/basic-memory/blob/main/docs/ai-assistant-guide-extended.md) with detailed examples, advanced patterns, and self-contained sections.
6 |
7 | ## Overview
8 |
9 | Basic Memory creates a semantic knowledge graph from markdown files. Focus on building rich connections between notes.
10 |
11 | - **Local-First**: Plain text files on user's computer
12 | - **Persistent**: Knowledge survives across sessions
13 | - **Semantic**: Observations and relations create a knowledge graph
14 |
15 | **Your role**: You're helping humans build enduring knowledge they'll own forever. The semantic graph (observations, relations, context) helps you provide better assistance by understanding connections and maintaining continuity. Think: lasting insights worth keeping, not disposable chat logs.
16 |
17 | ## Project Management
18 |
19 | All tools require explicit project specification.
20 |
21 | **Three-tier resolution:**
22 | 1. CLI constraint: `--project name` (highest priority)
23 | 2. Explicit parameter: `project="name"` in tool calls
24 | 3. Default mode: `default_project_mode=true` in config (fallback)
25 |
26 | ### Quick Setup Check
27 |
28 | ```python
29 | # Discover projects
30 | projects = await list_memory_projects()
31 |
32 | # Check if default_project_mode enabled
33 | # If yes: project parameter optional
34 | # If no: project parameter required
35 | ```
36 |
37 | ### Default Project Mode
38 |
39 | When `default_project_mode=true`:
40 | ```python
41 | # These are equivalent:
42 | await write_note("Note", "Content", "folder")
43 | await write_note("Note", "Content", "folder", project="main")
44 | ```
45 |
46 | When `default_project_mode=false` (default):
47 | ```python
48 | # Project required:
49 | await write_note("Note", "Content", "folder", project="main") # ✓
50 | await write_note("Note", "Content", "folder") # ✗ Error
51 | ```
52 |
53 | ## Core Tools
54 |
55 | ### Writing Knowledge
56 |
57 | ```python
58 | await write_note(
59 | title="Topic",
60 | content="# Topic\n## Observations\n- [category] fact\n## Relations\n- relates_to [[Other]]",
61 | folder="notes",
62 | project="main" # Required unless default_project_mode=true
63 | )
64 | ```
65 |
66 | ### Reading Knowledge
67 |
68 | ```python
69 | # By identifier
70 | content = await read_note("Topic", project="main")
71 |
72 | # By memory:// URL
73 | content = await read_note("memory://folder/topic", project="main")
74 | ```
75 |
76 | ### Searching
77 |
78 | ```python
79 | results = await search_notes(
80 | query="authentication",
81 | project="main",
82 | page_size=10
83 | )
84 | ```
85 |
86 | ### Building Context
87 |
88 | ```python
89 | context = await build_context(
90 | url="memory://specs/auth",
91 | project="main",
92 | depth=2,
93 | timeframe="1 week"
94 | )
95 | ```
96 |
97 | ## Knowledge Graph Essentials
98 |
99 | ### Observations
100 |
101 | Categorized facts with optional tags:
102 | ```markdown
103 | - [decision] Use JWT for authentication #security
104 | - [technique] Hash passwords with bcrypt #best-practice
105 | - [requirement] Support OAuth 2.0 providers
106 | ```
107 |
108 | ### Relations
109 |
110 | Directional links between entities:
111 | ```markdown
112 | - implements [[Authentication Spec]]
113 | - requires [[User Database]]
114 | - extends [[Base Security Model]]
115 | ```
116 |
117 | **Common relation types:** `relates_to`, `implements`, `requires`, `extends`, `part_of`, `contrasts_with`
118 |
119 | ### Forward References
120 |
121 | Reference entities that don't exist yet:
122 | ```python
123 | # Create note with forward reference
124 | await write_note(
125 | title="Login Flow",
126 | content="## Relations\n- requires [[OAuth Provider]]", # Doesn't exist yet
127 | folder="auth",
128 | project="main"
129 | )
130 |
131 | # Later, create referenced entity
132 | await write_note(
133 | title="OAuth Provider",
134 | content="# OAuth Provider\n...",
135 | folder="auth",
136 | project="main"
137 | )
138 | # → Relation automatically resolved
139 | ```
140 |
141 | ## Best Practices
142 |
143 | ### 1. Project Management
144 |
145 | **Single-project users:**
146 | - Enable `default_project_mode=true`
147 | - Simpler tool calls
148 |
149 | **Multi-project users:**
150 | - Keep `default_project_mode=false`
151 | - Always specify project explicitly
152 |
153 | **Discovery:**
154 | ```python
155 | # Start with discovery
156 | projects = await list_memory_projects()
157 |
158 | # Cross-project activity (no project param = all projects)
159 | activity = await recent_activity()
160 |
161 | # Or specific project
162 | activity = await recent_activity(project="main")
163 | ```
164 |
165 | ### 2. Building Rich Graphs
166 |
167 | **Always include:**
168 | - 3-5 observations per note
169 | - 2-3 relations per note
170 | - Meaningful categories and relation types
171 |
172 | **Search before creating:**
173 | ```python
174 | # Find existing entities to reference
175 | results = await search_notes(query="authentication", project="main")
176 | # Use exact titles in [[WikiLinks]]
177 | ```
178 |
179 | ### 3. Writing Effective Notes
180 |
181 | **Structure:**
182 | ```markdown
183 | # Title
184 |
185 | ## Context
186 | Background information
187 |
188 | ## Observations
189 | - [category] Fact with #tags
190 | - [category] Another fact
191 |
192 | ## Relations
193 | - relation_type [[Exact Entity Title]]
194 | ```
195 |
196 | **Categories:** `[idea]`, `[decision]`, `[fact]`, `[technique]`, `[requirement]`
197 |
198 | ### 4. Error Handling
199 |
200 | **Missing project:**
201 | ```python
202 | try:
203 | await search_notes(query="test") # Missing project parameter - will error
204 | except:
205 | # Show available projects
206 | projects = await list_memory_projects()
207 | # Then retry with project
208 | results = await search_notes(query="test", project=projects[0].name)
209 | ```
210 |
211 | **Forward references:**
212 | ```python
213 | # Check response for unresolved relations
214 | response = await write_note(
215 | title="New Topic",
216 | content="## Relations\n- relates_to [[Future Topic]]",
217 | folder="notes",
218 | project="main"
219 | )
220 | # Forward refs will resolve when target created
221 | ```
222 |
223 | ### 5. Recording Context
224 |
225 | **Ask permission:**
226 | > "Would you like me to save our discussion about [topic] to Basic Memory?"
227 |
228 | **Confirm when done:**
229 | > "I've saved our discussion to Basic Memory."
230 |
231 | **What to record:**
232 | - Decisions and rationales
233 | - Important discoveries
234 | - Action items and plans
235 | - Connected topics
236 |
237 | ## Common Patterns
238 |
239 | ### Capture Decision
240 |
241 | ```python
242 | await write_note(
243 | title="DB Choice",
244 | content="""# DB Choice\n## Decision\nUse PostgreSQL\n## Observations\n- [requirement] ACID compliance #reliability\n- [decision] PostgreSQL over MySQL\n## Relations\n- implements [[Data Architecture]]""",
245 | folder="decisions",
246 | project="main"
247 | )
248 | ```
249 |
250 | ### Link Topics & Build Context
251 |
252 | ```python
253 | # Link bidirectionally
254 | await write_note(title="API Auth", content="## Relations\n- part_of [[API Design]]", folder="api", project="main")
255 | await edit_note(identifier="API Design", operation="append", content="\n- includes [[API Auth]]", project="main")
256 |
257 | # Search and build context
258 | results = await search_notes(query="authentication", project="main")
259 | context = await build_context(url=f"memory://{results[0].permalink}", project="main", depth=2)
260 | ```
261 |
262 | ## Tool Quick Reference
263 |
264 | | Tool | Purpose | Key Params |
265 | |------|---------|------------|
266 | | `write_note` | Create/update | title, content, folder, project |
267 | | `read_note` | Read content | identifier, project |
268 | | `edit_note` | Modify existing | identifier, operation, content, project |
269 | | `search_notes` | Find notes | query, project |
270 | | `build_context` | Graph traversal | url, depth, project |
271 | | `recent_activity` | Recent changes | timeframe, project |
272 | | `list_memory_projects` | Show projects | (none) |
273 |
274 | ## memory:// URL Format
275 |
276 | - `memory://title` - By title
277 | - `memory://folder/title` - By folder + title
278 | - `memory://permalink` - By permalink
279 | - `memory://folder/*` - All in folder
280 |
281 | For full documentation: https://docs.basicmemory.com
282 |
283 | Built with ♥️ by Basic Machines
284 |
```
--------------------------------------------------------------------------------
/docs/character-handling.md:
--------------------------------------------------------------------------------
```markdown
1 | # Character Handling and Conflict Resolution
2 |
3 | Basic Memory handles various character encoding scenarios and file naming conventions to provide consistent permalink generation and conflict resolution. This document explains how the system works and how to resolve common character-related issues.
4 |
5 | ## Overview
6 |
7 | Basic Memory uses a sophisticated system to generate permalinks from file paths while maintaining consistency across different operating systems and character encodings. The system normalizes file paths and generates unique permalinks to prevent conflicts.
8 |
9 | ## Character Normalization Rules
10 |
11 | ### 1. Permalink Generation
12 |
13 | When Basic Memory processes a file path, it applies these normalization rules:
14 |
15 | ```
16 | Original: "Finance/My Investment Strategy.md"
17 | Permalink: "finance/my-investment-strategy"
18 | ```
19 |
20 | **Transformation process:**
21 | 1. Remove file extension (`.md`)
22 | 2. Convert to lowercase (case-insensitive)
23 | 3. Replace spaces with hyphens
24 | 4. Replace underscores with hyphens
25 | 5. Handle international characters (transliteration for Latin, preservation for non-Latin)
26 | 6. Convert camelCase to kebab-case
27 |
28 | ### 2. International Character Support
29 |
30 | **Latin characters with diacritics** are transliterated:
31 | - `ø` → `o` (Søren → soren)
32 | - `ü` → `u` (Müller → muller)
33 | - `é` → `e` (Café → cafe)
34 | - `ñ` → `n` (Niño → nino)
35 |
36 | **Non-Latin characters** are preserved:
37 | - Chinese: `中文/测试文档.md` → `中文/测试文档`
38 | - Japanese: `日本語/文書.md` → `日本語/文書`
39 |
40 | ## Common Conflict Scenarios
41 |
42 | ### 1. Hyphen vs Space Conflicts
43 |
44 | **Problem:** Files with existing hyphens conflict with generated permalinks from spaces.
45 |
46 | **Example:**
47 | ```
48 | File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
49 | File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug" (CONFLICT!)
50 | ```
51 |
52 | **Resolution:** The system automatically resolves this by adding suffixes:
53 | ```
54 | File 1: "basic memory bug.md" → permalink: "basic-memory-bug"
55 | File 2: "basic-memory-bug.md" → permalink: "basic-memory-bug-1"
56 | ```
57 |
58 | **Best Practice:** Choose consistent naming conventions within your project.
59 |
60 | ### 2. Case Sensitivity Conflicts
61 |
62 | **Problem:** Different case variations that normalize to the same permalink.
63 |
64 | **Example on macOS:**
65 | ```
66 | Directory: Finance/investment.md
67 | Directory: finance/investment.md (different on filesystem, same permalink)
68 | ```
69 |
70 | **Resolution:** Basic Memory detects case conflicts and prevents them during sync operations with helpful error messages.
71 |
72 | **Best Practice:** Use consistent casing for directory and file names.
73 |
74 | ### 3. Character Encoding Conflicts
75 |
76 | **Problem:** Different Unicode normalizations of the same logical character.
77 |
78 | **Example:**
79 | ```
80 | File 1: "café.md" (é as single character)
81 | File 2: "café.md" (e + combining accent)
82 | ```
83 |
84 | **Resolution:** Basic Memory normalizes Unicode characters using NFD normalization to detect these conflicts.
85 |
86 | ### 4. Forward Slash Conflicts
87 |
88 | **Problem:** Forward slashes in frontmatter or file names interpreted as path separators.
89 |
90 | **Example:**
91 | ```yaml
92 | ---
93 | permalink: finance/investment/strategy
94 | ---
95 | ```
96 |
97 | **Resolution:** Basic Memory validates frontmatter permalinks and warns about path separator conflicts.
98 |
99 | ## Error Messages and Troubleshooting
100 |
101 | ### "UNIQUE constraint failed: entity.file_path, entity.project_id"
102 |
103 | **Cause:** Two entities trying to use the same file path within a project.
104 |
105 | **Common scenarios:**
106 | 1. File move operation where destination is already occupied
107 | 2. Case sensitivity differences on macOS
108 | 3. Character encoding conflicts
109 | 4. Concurrent file operations
110 |
111 | **Resolution steps:**
112 | 1. Check for duplicate file names with different cases
113 | 2. Look for files with similar names but different character encodings
114 | 3. Rename conflicting files to have unique names
115 | 4. Run sync again after resolving conflicts
116 |
117 | ### "File path conflict detected during move"
118 |
119 | **Cause:** Enhanced conflict detection preventing potential database integrity violations.
120 |
121 | **What this means:** The system detected that moving a file would create a conflict before attempting the database operation.
122 |
123 | **Resolution:** Follow the specific guidance in the error message, which will indicate the type of conflict detected.
124 |
125 | ## Best Practices
126 |
127 | ### 1. File Naming Conventions
128 |
129 | **Recommended patterns:**
130 | - Use consistent casing (prefer lowercase)
131 | - Use hyphens instead of spaces for multi-word files
132 | - Avoid special characters that could conflict with path separators
133 | - Be consistent with directory structure casing
134 |
135 | **Examples:**
136 | ```
137 | ✅ Good:
138 | - finance/investment-strategy.md
139 | - projects/basic-memory-features.md
140 | - docs/api-reference.md
141 |
142 | ❌ Problematic:
143 | - Finance/Investment Strategy.md (mixed case, spaces)
144 | - finance/Investment Strategy.md (inconsistent case)
145 | - docs/API/Reference.md (mixed case directories)
146 | ```
147 |
148 | ### 2. Permalink Management
149 |
150 | **Custom permalinks in frontmatter:**
151 | ```yaml
152 | ---
153 | type: knowledge
154 | permalink: custom-permalink-name
155 | ---
156 | ```
157 |
158 | **Guidelines:**
159 | - Use lowercase permalinks
160 | - Use hyphens for word separation
161 | - Avoid path separators unless creating sub-paths
162 | - Ensure uniqueness within your project
163 |
164 | ### 3. Directory Structure
165 |
166 | **Consistent casing:**
167 | ```
168 | ✅ Good:
169 | finance/
170 | investment-strategies.md
171 | portfolio-management.md
172 |
173 | ❌ Problematic:
174 | Finance/ (capital F)
175 | investment-strategies.md
176 | finance/ (lowercase f)
177 | portfolio-management.md
178 | ```
179 |
180 | ## Migration and Cleanup
181 |
182 | ### Identifying Conflicts
183 |
184 | Use Basic Memory's built-in conflict detection:
185 |
186 | ```bash
187 | # Sync will report conflicts
188 | basic-memory sync
189 |
190 | # Check sync status for warnings
191 | basic-memory status
192 | ```
193 |
194 | ### Resolving Existing Conflicts
195 |
196 | 1. **Identify conflicting files** from sync error messages
197 | 2. **Choose consistent naming convention** for your project
198 | 3. **Rename files** to follow the convention
199 | 4. **Re-run sync** to verify resolution
200 |
201 | ### Bulk Renaming Strategy
202 |
203 | For projects with many conflicts:
204 |
205 | 1. **Backup your project** before making changes
206 | 2. **Standardize on lowercase** file and directory names
207 | 3. **Replace spaces with hyphens** in file names
208 | 4. **Use consistent character encoding** (UTF-8)
209 | 5. **Test sync after each batch** of changes
210 |
211 | ## System Enhancements
212 |
213 | ### Recent Improvements (v0.13+)
214 |
215 | 1. **Enhanced conflict detection** before database operations
216 | 2. **Improved error messages** with specific resolution guidance
217 | 3. **Character normalization utilities** for consistent handling
218 | 4. **File swap detection** for complex move scenarios
219 | 5. **Proactive conflict warnings** during permalink resolution
220 |
221 | ### Monitoring and Logging
222 |
223 | The system now provides detailed logging for conflict resolution:
224 |
225 | ```
226 | DEBUG: Detected potential file path conflicts for 'Finance/Investment.md': ['finance/investment.md']
227 | WARNING: File path conflict detected during move: entity_id=123 trying to move from 'old.md' to 'new.md'
228 | ```
229 |
230 | These logs help identify and resolve conflicts before they cause sync failures.
231 |
232 | ## Support and Resources
233 |
234 | If you encounter character-related conflicts not covered in this guide:
235 |
236 | 1. **Check the logs** for specific conflict details
237 | 2. **Review error messages** for resolution guidance
238 | 3. **Report issues** with examples of the conflicting files
239 | 4. **Consider the file naming best practices** outlined above
240 |
241 | The Basic Memory system is designed to handle most character conflicts automatically while providing clear guidance for manual resolution when needed.
```
--------------------------------------------------------------------------------
/src/basic_memory/cli/commands/cloud/rclone_installer.py:
--------------------------------------------------------------------------------
```python
1 | """Cross-platform rclone installation utilities."""
2 |
3 | import platform
4 | import shutil
5 | import subprocess
6 | from typing import Optional
7 |
8 | from rich.console import Console
9 |
10 | console = Console()
11 |
12 |
13 | class RcloneInstallError(Exception):
14 | """Exception raised for rclone installation errors."""
15 |
16 | pass
17 |
18 |
19 | def is_rclone_installed() -> bool:
20 | """Check if rclone is already installed and available in PATH."""
21 | return shutil.which("rclone") is not None
22 |
23 |
24 | def get_platform() -> str:
25 | """Get the current platform identifier."""
26 | system = platform.system().lower()
27 | if system == "darwin":
28 | return "macos"
29 | elif system == "linux":
30 | return "linux"
31 | elif system == "windows":
32 | return "windows"
33 | else:
34 | raise RcloneInstallError(f"Unsupported platform: {system}")
35 |
36 |
37 | def run_command(command: list[str], check: bool = True) -> subprocess.CompletedProcess:
38 | """Run a command with proper error handling."""
39 | try:
40 | console.print(f"[dim]Running: {' '.join(command)}[/dim]")
41 | result = subprocess.run(command, capture_output=True, text=True, check=check)
42 | if result.stdout:
43 | console.print(f"[dim]Output: {result.stdout.strip()}[/dim]")
44 | return result
45 | except subprocess.CalledProcessError as e:
46 | console.print(f"[red]Command failed: {e}[/red]")
47 | if e.stderr:
48 | console.print(f"[red]Error output: {e.stderr}[/red]")
49 | raise RcloneInstallError(f"Command failed: {e}") from e
50 | except FileNotFoundError as e:
51 | raise RcloneInstallError(f"Command not found: {' '.join(command)}") from e
52 |
53 |
54 | def install_rclone_macos() -> None:
55 | """Install rclone on macOS using Homebrew or official script."""
56 | # Try Homebrew first
57 | if shutil.which("brew"):
58 | try:
59 | console.print("[blue]Installing rclone via Homebrew...[/blue]")
60 | run_command(["brew", "install", "rclone"])
61 | console.print("[green]✓ rclone installed via Homebrew[/green]")
62 | return
63 | except RcloneInstallError:
64 | console.print(
65 | "[yellow]Homebrew installation failed, trying official script...[/yellow]"
66 | )
67 |
68 | # Fallback to official script
69 | console.print("[blue]Installing rclone via official script...[/blue]")
70 | try:
71 | run_command(["sh", "-c", "curl https://rclone.org/install.sh | sudo bash"])
72 | console.print("[green]✓ rclone installed via official script[/green]")
73 | except RcloneInstallError:
74 | raise RcloneInstallError(
75 | "Failed to install rclone. Please install manually: brew install rclone"
76 | )
77 |
78 |
79 | def install_rclone_linux() -> None:
80 | """Install rclone on Linux using package managers or official script."""
81 | # Try snap first (most universal)
82 | if shutil.which("snap"):
83 | try:
84 | console.print("[blue]Installing rclone via snap...[/blue]")
85 | run_command(["sudo", "snap", "install", "rclone"])
86 | console.print("[green]✓ rclone installed via snap[/green]")
87 | return
88 | except RcloneInstallError:
89 | console.print("[yellow]Snap installation failed, trying apt...[/yellow]")
90 |
91 | # Try apt (Debian/Ubuntu)
92 | if shutil.which("apt"):
93 | try:
94 | console.print("[blue]Installing rclone via apt...[/blue]")
95 | run_command(["sudo", "apt", "update"])
96 | run_command(["sudo", "apt", "install", "-y", "rclone"])
97 | console.print("[green]✓ rclone installed via apt[/green]")
98 | return
99 | except RcloneInstallError:
100 | console.print("[yellow]apt installation failed, trying official script...[/yellow]")
101 |
102 | # Fallback to official script
103 | console.print("[blue]Installing rclone via official script...[/blue]")
104 | try:
105 | run_command(["sh", "-c", "curl https://rclone.org/install.sh | sudo bash"])
106 | console.print("[green]✓ rclone installed via official script[/green]")
107 | except RcloneInstallError:
108 | raise RcloneInstallError(
109 | "Failed to install rclone. Please install manually: sudo snap install rclone"
110 | )
111 |
112 |
113 | def install_rclone_windows() -> None:
114 | """Install rclone on Windows using package managers."""
115 | # Try winget first (built into Windows 10+)
116 | if shutil.which("winget"):
117 | try:
118 | console.print("[blue]Installing rclone via winget...[/blue]")
119 | run_command(["winget", "install", "Rclone.Rclone"])
120 | console.print("[green]✓ rclone installed via winget[/green]")
121 | return
122 | except RcloneInstallError:
123 | console.print("[yellow]winget installation failed, trying chocolatey...[/yellow]")
124 |
125 | # Try chocolatey
126 | if shutil.which("choco"):
127 | try:
128 | console.print("[blue]Installing rclone via chocolatey...[/blue]")
129 | run_command(["choco", "install", "rclone", "-y"])
130 | console.print("[green]✓ rclone installed via chocolatey[/green]")
131 | return
132 | except RcloneInstallError:
133 | console.print("[yellow]chocolatey installation failed, trying scoop...[/yellow]")
134 |
135 | # Try scoop
136 | if shutil.which("scoop"):
137 | try:
138 | console.print("[blue]Installing rclone via scoop...[/blue]")
139 | run_command(["scoop", "install", "rclone"])
140 | console.print("[green]✓ rclone installed via scoop[/green]")
141 | return
142 | except RcloneInstallError:
143 | console.print("[yellow]scoop installation failed[/yellow]")
144 |
145 | # No package manager available
146 | raise RcloneInstallError(
147 | "Could not install rclone automatically. Please install a package manager "
148 | "(winget, chocolatey, or scoop) or install rclone manually from https://rclone.org/downloads/"
149 | )
150 |
151 |
152 | def install_rclone(platform_override: Optional[str] = None) -> None:
153 | """Install rclone for the current platform."""
154 | if is_rclone_installed():
155 | console.print("[green]rclone is already installed[/green]")
156 | return
157 |
158 | platform_name = platform_override or get_platform()
159 | console.print(f"[blue]Installing rclone for {platform_name}...[/blue]")
160 |
161 | try:
162 | if platform_name == "macos":
163 | install_rclone_macos()
164 | elif platform_name == "linux":
165 | install_rclone_linux()
166 | elif platform_name == "windows":
167 | install_rclone_windows()
168 | else:
169 | raise RcloneInstallError(f"Unsupported platform: {platform_name}")
170 |
171 | # Verify installation
172 | if not is_rclone_installed():
173 | raise RcloneInstallError("rclone installation completed but command not found in PATH")
174 |
175 | console.print("[green]✓ rclone installation completed successfully[/green]")
176 |
177 | except RcloneInstallError:
178 | raise
179 | except Exception as e:
180 | raise RcloneInstallError(f"Unexpected error during installation: {e}") from e
181 |
182 |
183 | def get_rclone_version() -> Optional[str]:
184 | """Get the installed rclone version."""
185 | if not is_rclone_installed():
186 | return None
187 |
188 | try:
189 | result = run_command(["rclone", "version"], check=False)
190 | if result.returncode == 0:
191 | # Parse version from output (format: "rclone v1.64.0")
192 | lines = result.stdout.strip().split("\n")
193 | for line in lines:
194 | if line.startswith("rclone v"):
195 | return line.split()[1]
196 | return "unknown"
197 | except Exception:
198 | return "unknown"
199 |
```
--------------------------------------------------------------------------------
/tests/markdown/test_markdown_plugins.py:
--------------------------------------------------------------------------------
```python
1 | """Tests for markdown plugins."""
2 |
3 | from textwrap import dedent
4 | from markdown_it import MarkdownIt
5 | from markdown_it.token import Token
6 |
7 | from basic_memory.markdown.plugins import (
8 | observation_plugin,
9 | relation_plugin,
10 | is_observation,
11 | is_explicit_relation,
12 | parse_relation,
13 | parse_inline_relations,
14 | )
15 |
16 |
17 | def test_observation_plugin():
18 | """Test observation plugin."""
19 | # Set up markdown-it instance
20 | md = MarkdownIt().use(observation_plugin)
21 |
22 | # Test basic observation with all features
23 | content = dedent("""
24 | - [design] Basic observation #tag1 #tag2 (with context)
25 | """)
26 |
27 | tokens = md.parse(content)
28 | token = [t for t in tokens if t.type == "inline"][0]
29 | assert "observation" in token.meta
30 | obs = token.meta["observation"]
31 | assert obs["category"] == "design"
32 | assert obs["content"] == "Basic observation #tag1 #tag2"
33 | assert set(obs["tags"]) == {"tag1", "tag2"}
34 | assert obs["context"] == "with context"
35 |
36 | # Test without category
37 | content = "- Basic observation #tag1 (context)"
38 | token = [t for t in md.parse(content) if t.type == "inline"][0]
39 | obs = token.meta["observation"]
40 | assert obs["category"] is None
41 | assert obs["content"] == "Basic observation #tag1"
42 | assert obs["tags"] == ["tag1"]
43 | assert obs["context"] == "context"
44 |
45 | # Test without tags
46 | content = "- [note] Basic observation (context)"
47 | token = [t for t in md.parse(content) if t.type == "inline"][0]
48 | obs = token.meta["observation"]
49 | assert obs["category"] == "note"
50 | assert obs["content"] == "Basic observation"
51 | assert obs["tags"] is None
52 | assert obs["context"] == "context"
53 |
54 |
55 | def test_observation_edge_cases():
56 | """Test observation parsing edge cases."""
57 | # Test non-inline token
58 | token = Token("paragraph", "", 0)
59 | assert not is_observation(token)
60 |
61 | # Test empty content
62 | token = Token("inline", "", 0)
63 | assert not is_observation(token)
64 |
65 | # Test markdown task
66 | token = Token("inline", "[ ] Task item", 0)
67 | assert not is_observation(token)
68 |
69 | # Test completed task
70 | token = Token("inline", "[x] Done task", 0)
71 | assert not is_observation(token)
72 |
73 | # Test in-progress task
74 | token = Token("inline", "[-] Ongoing task", 0)
75 | assert not is_observation(token)
76 |
77 |
78 | def test_observation_excludes_markdown_and_wiki_links():
79 | """Test that markdown links and wiki links are NOT parsed as observations.
80 |
81 | This test validates the fix for issue #247 where:
82 | - [text](url) markdown links were incorrectly parsed as observations
83 | - [[text]] wiki links were incorrectly parsed as observations
84 | """
85 | # Test markdown links are NOT observations
86 | token = Token("inline", "[Click here](https://example.com)", 0)
87 | assert not is_observation(token), "Markdown links should not be parsed as observations"
88 |
89 | token = Token("inline", "[Documentation](./docs/readme.md)", 0)
90 | assert not is_observation(token), "Relative markdown links should not be parsed as observations"
91 |
92 | token = Token("inline", "[Empty link]()", 0)
93 | assert not is_observation(token), "Empty markdown links should not be parsed as observations"
94 |
95 | # Test wiki links are NOT observations
96 | token = Token("inline", "[[SomeWikiPage]]", 0)
97 | assert not is_observation(token), "Wiki links should not be parsed as observations"
98 |
99 | token = Token("inline", "[[Multi Word Page]]", 0)
100 | assert not is_observation(token), "Multi-word wiki links should not be parsed as observations"
101 |
102 | # Test nested brackets are NOT observations
103 | token = Token("inline", "[[Nested [[Inner]] Link]]", 0)
104 | assert not is_observation(token), "Nested wiki links should not be parsed as observations"
105 |
106 | # Test valid observations still work (should return True)
107 | token = Token("inline", "[category] This is a valid observation", 0)
108 | assert is_observation(token), "Valid observations should still be parsed correctly"
109 |
110 | token = Token("inline", "[design] Valid observation #tag", 0)
111 | assert is_observation(token), "Valid observations with tags should still work"
112 |
113 | token = Token("inline", "Just some text #tag", 0)
114 | assert is_observation(token), "Tag-only observations should still work"
115 |
116 | # Test edge cases that should NOT be observations
117 | token = Token("inline", "[]Empty brackets", 0)
118 | assert not is_observation(token), "Empty category brackets should not be observations"
119 |
120 | token = Token("inline", "[category]No space after category", 0)
121 | assert not is_observation(token), "No space after category should not be valid observation"
122 |
123 |
124 | def test_relation_plugin():
125 | """Test relation plugin."""
126 | md = MarkdownIt().use(relation_plugin)
127 |
128 | # Test explicit relation with all features
129 | content = dedent("""
130 | - implements [[Component]] (with context)
131 | """)
132 |
133 | tokens = md.parse(content)
134 | token = [t for t in tokens if t.type == "inline"][0]
135 | assert "relations" in token.meta
136 | rel = token.meta["relations"][0]
137 | assert rel["type"] == "implements"
138 | assert rel["target"] == "Component"
139 | assert rel["context"] == "with context"
140 |
141 | # Test implicit relations in text
142 | content = "Some text with a [[Link]] and [[Another Link]]"
143 | token = [t for t in md.parse(content) if t.type == "inline"][0]
144 | rels = token.meta["relations"]
145 | assert len(rels) == 2
146 | assert rels[0]["type"] == "links to"
147 | assert rels[0]["target"] == "Link"
148 | assert rels[1]["target"] == "Another Link"
149 |
150 |
151 | def test_relation_edge_cases():
152 | """Test relation parsing edge cases."""
153 | # Test non-inline token
154 | token = Token("paragraph", "", 0)
155 | assert not is_explicit_relation(token)
156 |
157 | # Test empty content
158 | token = Token("inline", "", 0)
159 | assert not is_explicit_relation(token)
160 |
161 | # Test incomplete relation (missing target)
162 | token = Token("inline", "relates_to [[]]", 0)
163 | result = parse_relation(token)
164 | assert result is None
165 |
166 | # Test non-relation content
167 | token = Token("inline", "Just some text", 0)
168 | result = parse_relation(token)
169 | assert result is None
170 |
171 | # Test invalid inline link (empty target)
172 | assert not parse_inline_relations("Text with [[]] empty link")
173 |
174 | # Test nested links (avoid duplicates)
175 | result = parse_inline_relations("Text with [[Outer [[Inner]] Link]]")
176 | assert len(result) == 1
177 | assert result[0]["target"] == "Outer [[Inner]] Link"
178 |
179 |
180 | def test_combined_plugins():
181 | """Test both plugins working together."""
182 | md = MarkdownIt().use(observation_plugin).use(relation_plugin)
183 |
184 | content = dedent("""
185 | # Section
186 | - [design] Observation with [[Link]] #tag (context)
187 | - implements [[Component]] (details)
188 | - Just a [[Reference]] in text
189 |
190 | Some text with a [[Link]] reference.
191 | """)
192 |
193 | tokens = md.parse(content)
194 | inline_tokens = [t for t in tokens if t.type == "inline"]
195 |
196 | # First token has both observation and relation
197 | obs_token = inline_tokens[1]
198 | assert "observation" in obs_token.meta
199 | assert "relations" in obs_token.meta
200 |
201 | # Second token has explicit relation
202 | rel_token = inline_tokens[2]
203 | assert "relations" in rel_token.meta
204 | rel = rel_token.meta["relations"][0]
205 | assert rel["type"] == "implements"
206 |
207 | # Third token has implicit relation
208 | text_token = inline_tokens[4]
209 | assert "relations" in text_token.meta
210 | link = text_token.meta["relations"][0]
211 | assert link["type"] == "links to"
212 |
```
--------------------------------------------------------------------------------
/.claude/agents/python-developer.md:
--------------------------------------------------------------------------------
```markdown
1 | ---
2 | name: python-developer
3 | description: Python backend developer specializing in FastAPI, DBOS workflows, and API implementation. Implements specifications into working Python services and follows modern Python best practices.
4 | model: sonnet
5 | color: red
6 | ---
7 |
8 | You are an expert Python developer specializing in implementing specifications into working Python services and APIs. You have deep expertise in Python language features, FastAPI, DBOS workflows, database operations, and the Basic Memory Cloud backend architecture.
9 |
10 | **Primary Role: Backend Implementation Agent**
11 | You implement specifications into working Python code and services. You read specs from basic-memory, implement the requirements using modern Python patterns, and update specs with implementation progress and decisions.
12 |
13 | **Core Responsibilities:**
14 |
15 | **Specification Implementation:**
16 | - Read specs using basic-memory MCP tools to understand backend requirements
17 | - Implement Python services, APIs, and workflows that fulfill spec requirements
18 | - Update specs with implementation progress, decisions, and completion status
19 | - Document any architectural decisions or modifications needed during implementation
20 |
21 | **Python/FastAPI Development:**
22 | - Create FastAPI applications with proper middleware and dependency injection
23 | - Implement DBOS workflows for durable, long-running operations
24 | - Design database schemas and implement repository patterns
25 | - Handle authentication, authorization, and security requirements
26 | - Implement async/await patterns for optimal performance
27 |
28 | **Backend Implementation Process:**
29 | 1. **Read Spec**: Use `mcp__basic-memory__read_note` to get spec requirements
30 | 2. **Analyze Existing Patterns**: Study codebase architecture and established patterns before implementing
31 | 3. **Follow Modular Structure**: Create separate modules/routers following existing conventions
32 | 4. **Implement**: Write Python code following spec requirements and codebase patterns
33 | 5. **Test**: Create tests that validate spec success criteria
34 | 6. **Update Spec**: Document completion and any implementation decisions
35 | 7. **Validate**: Run tests and ensure integration works correctly
36 |
37 | **Technical Standards:**
38 | - Follow PEP 8 and modern Python conventions
39 | - Use type hints throughout the codebase
40 | - Implement proper error handling and logging
41 | - Use async/await for all database and external service calls
42 | - Write comprehensive tests using pytest
43 | - Follow security best practices for web APIs
44 | - Document functions and classes with clear docstrings
45 |
46 | **Codebase Architecture Patterns:**
47 |
48 | **CLI Structure Patterns:**
49 | - Follow existing modular CLI pattern: create separate CLI modules (e.g., `upload_cli.py`) instead of adding commands directly to `main.py`
50 | - Existing examples: `polar_cli.py`, `tenant_cli.py` in `apps/cloud/src/basic_memory_cloud/cli/`
51 | - Register new CLI modules using `app.add_typer(new_cli, name="command", help="description")`
52 | - Maintain consistent command structure and help text patterns
53 |
54 | **FastAPI Router Patterns:**
55 | - Create dedicated routers for logical endpoint groups instead of adding routes directly to main app
56 | - Place routers in dedicated files (e.g., `apps/api/src/basic_memory_cloud_api/routers/webdav_router.py`)
57 | - Follow existing middleware and dependency injection patterns
58 | - Register routers using `app.include_router(router, prefix="/api-path")`
59 |
60 | **Modular Organization:**
61 | - Always analyze existing codebase structure before implementing new features
62 | - Follow established file organization and naming conventions
63 | - Create separate modules for distinct functionality areas
64 | - Maintain consistency with existing architectural decisions
65 | - Preserve separation of concerns across service boundaries
66 |
67 | **Pattern Analysis Process:**
68 | 1. Examine similar existing functionality in the codebase
69 | 2. Identify established patterns for file organization and module structure
70 | 3. Follow the same architectural approach for consistency
71 | 4. Create new modules/routers following existing conventions
72 | 5. Integrate new code using established registration patterns
73 |
74 | **Basic Memory Cloud Expertise:**
75 |
76 | **FastAPI Service Patterns:**
77 | - Multi-app architecture (Cloud, MCP, API services)
78 | - Shared middleware for JWT validation, CORS, logging
79 | - Dependency injection for services and repositories
80 | - Proper async request handling and error responses
81 |
82 | **DBOS Workflow Implementation:**
83 | - Durable workflows for tenant provisioning and infrastructure operations
84 | - Service layer pattern with repository data access
85 | - Event sourcing for audit trails and business processes
86 | - Idempotent operations with proper error handling
87 |
88 | **Database & Repository Patterns:**
89 | - SQLAlchemy with async patterns
90 | - Repository pattern for data access abstraction
91 | - Database migration strategies
92 | - Multi-tenant data isolation patterns
93 |
94 | **Authentication & Security:**
95 | - JWT token validation and middleware
96 | - OAuth 2.1 flow implementation
97 | - Tenant-specific authorization patterns
98 | - Secure API design and input validation
99 |
100 | **Code Quality Standards:**
101 | - Clear, descriptive variable and function names
102 | - Proper docstrings for functions and classes
103 | - Handle edge cases and error conditions gracefully
104 | - Use context managers for resource management
105 | - Apply composition over inheritance
106 | - Consider security implications for all API endpoints
107 | - Optimize for performance while maintaining readability
108 |
109 | **Testing & Validation:**
110 | - Write pytest tests that validate spec requirements
111 | - Include unit tests for business logic
112 | - Integration tests for API endpoints
113 | - Test error conditions and edge cases
114 | - Use fixtures for consistent test setup
115 | - Mock external dependencies appropriately
116 |
117 | **Debugging & Problem Solving:**
118 | - Analyze error messages and stack traces methodically
119 | - Identify root causes rather than applying quick fixes
120 | - Use logging effectively for troubleshooting
121 | - Apply systematic debugging approaches
122 | - Document solutions for future reference
123 |
124 | **Basic Memory Integration:**
125 | - Use `mcp__basic-memory__read_note` to read specifications
126 | - Use `mcp__basic-memory__edit_note` to update specs with progress
127 | - Document implementation patterns and decisions
128 | - Link related services and database schemas
129 | - Maintain implementation history and troubleshooting guides
130 |
131 | **Communication Style:**
132 | - Focus on concrete implementation results and working code
133 | - Document technical decisions and trade-offs clearly
134 | - Ask specific questions about requirements and constraints
135 | - Provide clear status updates on implementation progress
136 | - Explain code choices and architectural patterns
137 |
138 | **Deliverables:**
139 | - Working Python services that meet spec requirements
140 | - Updated specifications with implementation status
141 | - Comprehensive tests validating functionality
142 | - Clean, maintainable, type-safe Python code
143 | - Proper error handling and logging
144 | - Database migrations and schema updates
145 |
146 | **Key Principles:**
147 | - Implement specifications faithfully and completely
148 | - Write clean, efficient, and maintainable Python code
149 | - Follow established patterns and conventions
150 | - Apply proper error handling and security practices
151 | - Test thoroughly and document implementation decisions
152 | - Balance performance with code clarity and maintainability
153 |
154 | When handed a specification via `/spec implement`, you will read the spec, understand the requirements, implement the Python solution using appropriate patterns and frameworks, create tests to validate functionality, and update the spec with completion status and any implementation notes.
```
--------------------------------------------------------------------------------
/tests/api/test_template_loader_helpers.py:
--------------------------------------------------------------------------------
```python
1 | """Tests for additional template loader helpers."""
2 |
3 | import pytest
4 | from datetime import datetime
5 |
6 | from basic_memory.api.template_loader import TemplateLoader
7 |
8 |
9 | @pytest.fixture
10 | def temp_template_dir(tmpdir):
11 | """Create a temporary directory for test templates."""
12 | template_dir = tmpdir.mkdir("templates").mkdir("prompts")
13 | return template_dir
14 |
15 |
16 | @pytest.fixture
17 | def custom_template_loader(temp_template_dir):
18 | """Return a TemplateLoader instance with a custom template directory."""
19 | return TemplateLoader(str(temp_template_dir))
20 |
21 |
22 | @pytest.mark.asyncio
23 | async def test_round_helper(custom_template_loader, temp_template_dir):
24 | """Test the round helper for number formatting."""
25 | # Create template file
26 | round_path = temp_template_dir / "round.hbs"
27 | round_path.write_text(
28 | "{{round number}} {{round number 0}} {{round number 3}}",
29 | encoding="utf-8",
30 | )
31 |
32 | # Test with various values
33 | result = await custom_template_loader.render("round.hbs", {"number": 3.14159})
34 | assert result == "3.14 3.0 3.142" or result == "3.14 3 3.142"
35 |
36 | # Test with non-numeric value
37 | result = await custom_template_loader.render("round.hbs", {"number": "not-a-number"})
38 | assert "not-a-number" in result
39 |
40 | # Test with insufficient args
41 | empty_path = temp_template_dir / "round_empty.hbs"
42 | empty_path.write_text("{{round}}", encoding="utf-8")
43 | result = await custom_template_loader.render("round_empty.hbs", {})
44 | assert result == ""
45 |
46 |
47 | @pytest.mark.asyncio
48 | async def test_date_helper_edge_cases(custom_template_loader, temp_template_dir):
49 | """Test edge cases for the date helper."""
50 | # Create template file
51 | date_path = temp_template_dir / "date_edge.hbs"
52 | date_path.write_text(
53 | "{{date timestamp}} {{date timestamp '%Y'}} {{date string_date}} {{date invalid_date}} {{date}}",
54 | encoding="utf-8",
55 | )
56 |
57 | # Test with various values
58 | result = await custom_template_loader.render(
59 | "date_edge.hbs",
60 | {
61 | "timestamp": datetime(2023, 1, 1, 12, 30),
62 | "string_date": "2023-01-01T12:30:00",
63 | "invalid_date": "not-a-date",
64 | },
65 | )
66 |
67 | assert "2023-01-01" in result
68 | assert "2023" in result # Custom format
69 | assert "not-a-date" in result # Invalid date passed through
70 | assert result.strip() != "" # Empty date case
71 |
72 |
73 | @pytest.mark.asyncio
74 | async def test_size_helper_edge_cases(custom_template_loader, temp_template_dir):
75 | """Test edge cases for the size helper."""
76 | # Create template file
77 | size_path = temp_template_dir / "size_edge.hbs"
78 | size_path.write_text(
79 | "{{size list}} {{size string}} {{size dict}} {{size null}} {{size}}",
80 | encoding="utf-8",
81 | )
82 |
83 | # Test with various values
84 | result = await custom_template_loader.render(
85 | "size_edge.hbs",
86 | {
87 | "list": [1, 2, 3, 4, 5],
88 | "string": "hello",
89 | "dict": {"a": 1, "b": 2, "c": 3},
90 | "null": None,
91 | },
92 | )
93 |
94 | assert "5" in result # List size
95 | assert "hello".find("5") == -1 # String size should be 5
96 | assert "3" in result # Dict size
97 | assert "0" in result # Null size
98 | assert result.count("0") >= 2 # At least two zeros (null and empty args)
99 |
100 |
101 | @pytest.mark.asyncio
102 | async def test_math_helper(custom_template_loader, temp_template_dir):
103 | """Test the math helper for basic arithmetic."""
104 | # Create template file
105 | math_path = temp_template_dir / "math.hbs"
106 | math_path.write_text(
107 | "{{math 5 '+' 3}} {{math 10 '-' 4}} {{math 6 '*' 7}} {{math 20 '/' 5}}",
108 | encoding="utf-8",
109 | )
110 |
111 | # Test basic operations
112 | result = await custom_template_loader.render("math.hbs", {})
113 | assert "8" in result # Addition
114 | assert "6" in result # Subtraction
115 | assert "42" in result # Multiplication
116 | assert "4" in result # Division
117 |
118 | # Test with invalid operator
119 | invalid_op_path = temp_template_dir / "math_invalid_op.hbs"
120 | invalid_op_path.write_text("{{math 5 'invalid' 3}}", encoding="utf-8")
121 | result = await custom_template_loader.render("math_invalid_op.hbs", {})
122 | assert "Unsupported operator" in result
123 |
124 | # Test with invalid numeric values
125 | invalid_num_path = temp_template_dir / "math_invalid_num.hbs"
126 | invalid_num_path.write_text("{{math 'not-a-number' '+' 3}}", encoding="utf-8")
127 | result = await custom_template_loader.render("math_invalid_num.hbs", {})
128 | assert "Math error" in result
129 |
130 | # Test with insufficient arguments
131 | insufficient_path = temp_template_dir / "math_insufficient.hbs"
132 | insufficient_path.write_text("{{math 5 '+'}}", encoding="utf-8")
133 | result = await custom_template_loader.render("math_insufficient.hbs", {})
134 | assert "Insufficient arguments" in result
135 |
136 |
137 | @pytest.mark.asyncio
138 | async def test_if_cond_helper(custom_template_loader, temp_template_dir):
139 | """Test the if_cond helper for conditionals."""
140 | # Create template file with true condition
141 | if_true_path = temp_template_dir / "if_true.hbs"
142 | if_true_path.write_text(
143 | "{{#if_cond (lt 5 10)}}True condition{{else}}False condition{{/if_cond}}",
144 | encoding="utf-8",
145 | )
146 |
147 | # Create template file with false condition
148 | if_false_path = temp_template_dir / "if_false.hbs"
149 | if_false_path.write_text(
150 | "{{#if_cond (lt 15 10)}}True condition{{else}}False condition{{/if_cond}}",
151 | encoding="utf-8",
152 | )
153 |
154 | # Test true condition
155 | result = await custom_template_loader.render("if_true.hbs", {})
156 | assert result == "True condition"
157 |
158 | # Test false condition
159 | result = await custom_template_loader.render("if_false.hbs", {})
160 | assert result == "False condition"
161 |
162 |
163 | @pytest.mark.asyncio
164 | async def test_lt_helper_edge_cases(custom_template_loader, temp_template_dir):
165 | """Test edge cases for the lt (less than) helper."""
166 | # Create template file
167 | lt_path = temp_template_dir / "lt_edge.hbs"
168 | lt_path.write_text(
169 | "{{#if_cond (lt 'a' 'b')}}String LT True{{else}}String LT False{{/if_cond}} "
170 | "{{#if_cond (lt 'z' 'a')}}String LT2 True{{else}}String LT2 False{{/if_cond}} "
171 | "{{#if_cond (lt)}}Missing args True{{else}}Missing args False{{/if_cond}}",
172 | encoding="utf-8",
173 | )
174 |
175 | # Test with string values and missing args
176 | result = await custom_template_loader.render("lt_edge.hbs", {})
177 | assert "String LT True" in result # 'a' < 'b' is true
178 | assert "String LT2 False" in result # 'z' < 'a' is false
179 | assert "Missing args False" in result # Missing args should return false
180 |
181 |
182 | @pytest.mark.asyncio
183 | async def test_dedent_helper_edge_case(custom_template_loader, temp_template_dir):
184 | """Test an edge case for the dedent helper."""
185 | # Create template with empty dedent block
186 | empty_dedent_path = temp_template_dir / "empty_dedent.hbs"
187 | empty_dedent_path.write_text("{{#dedent}}{{/dedent}}", encoding="utf-8")
188 |
189 | # Test empty block
190 | result = await custom_template_loader.render("empty_dedent.hbs", {})
191 | assert result == ""
192 |
193 | # Test with complex content including lists
194 | complex_dedent_path = temp_template_dir / "complex_dedent.hbs"
195 | complex_dedent_path.write_text(
196 | "{{#dedent}}\n {{#each items}}\n - {{this}}\n {{/each}}\n{{/dedent}}",
197 | encoding="utf-8",
198 | )
199 |
200 | result = await custom_template_loader.render("complex_dedent.hbs", {"items": [1, 2, 3]})
201 | assert "- 1" in result
202 | assert "- 2" in result
203 | assert "- 3" in result
204 |
```
--------------------------------------------------------------------------------
/src/basic_memory/markdown/plugins.py:
--------------------------------------------------------------------------------
```python
1 | """Markdown-it plugins for Basic Memory markdown parsing."""
2 |
3 | from typing import List, Any, Dict
4 | from markdown_it import MarkdownIt
5 | from markdown_it.token import Token
6 |
7 |
8 | # Observation handling functions
9 | def is_observation(token: Token) -> bool:
10 | """Check if token looks like our observation format."""
11 | import re
12 |
13 | if token.type != "inline": # pragma: no cover
14 | return False
15 | # Use token.tag which contains the actual content for test tokens, fallback to content
16 | content = (token.tag or token.content).strip()
17 | if not content: # pragma: no cover
18 | return False
19 | # if it's a markdown_task, return false
20 | if content.startswith("[ ]") or content.startswith("[x]") or content.startswith("[-]"):
21 | return False
22 |
23 | # Exclude markdown links: [text](url)
24 | if re.match(r"^\[.*?\]\(.*?\)$", content):
25 | return False
26 |
27 | # Exclude wiki links: [[text]]
28 | if re.match(r"^\[\[.*?\]\]$", content):
29 | return False
30 |
31 | # Check for proper observation format: [category] content
32 | match = re.match(r"^\[([^\[\]()]+)\]\s+(.+)", content)
33 | has_tags = "#" in content
34 | return bool(match) or has_tags
35 |
36 |
37 | def parse_observation(token: Token) -> Dict[str, Any]:
38 | """Extract observation parts from token."""
39 | import re
40 |
41 | # Use token.tag which contains the actual content for test tokens, fallback to content
42 | content = (token.tag or token.content).strip()
43 |
44 | # Parse [category] with regex
45 | match = re.match(r"^\[([^\[\]()]+)\]\s+(.+)", content)
46 | category = None
47 | if match:
48 | category = match.group(1).strip()
49 | content = match.group(2).strip()
50 | else:
51 | # Handle empty brackets [] followed by content
52 | empty_match = re.match(r"^\[\]\s+(.+)", content)
53 | if empty_match:
54 | content = empty_match.group(1).strip()
55 |
56 | # Parse (context)
57 | context = None
58 | if content.endswith(")"):
59 | start = content.rfind("(")
60 | if start != -1:
61 | context = content[start + 1 : -1].strip()
62 | content = content[:start].strip()
63 |
64 | # Extract tags and keep original content
65 | tags = []
66 | parts = content.split()
67 | for part in parts:
68 | if part.startswith("#"):
69 | if "#" in part[1:]:
70 | subtags = [t for t in part.split("#") if t]
71 | tags.extend(subtags)
72 | else:
73 | tags.append(part[1:])
74 |
75 | return {
76 | "category": category,
77 | "content": content,
78 | "tags": tags if tags else None,
79 | "context": context,
80 | }
81 |
82 |
83 | # Relation handling functions
84 | def is_explicit_relation(token: Token) -> bool:
85 | """Check if token looks like our relation format."""
86 | if token.type != "inline": # pragma: no cover
87 | return False
88 |
89 | # Use token.tag which contains the actual content for test tokens, fallback to content
90 | content = (token.tag or token.content).strip()
91 | return "[[" in content and "]]" in content
92 |
93 |
94 | def parse_relation(token: Token) -> Dict[str, Any] | None:
95 | """Extract relation parts from token."""
96 | # Remove bullet point if present
97 | # Use token.tag which contains the actual content for test tokens, fallback to content
98 | content = (token.tag or token.content).strip()
99 |
100 | # Extract [[target]]
101 | target = None
102 | rel_type = "relates_to" # default
103 | context = None
104 |
105 | start = content.find("[[")
106 | end = content.find("]]")
107 |
108 | if start != -1 and end != -1:
109 | # Get text before link as relation type
110 | before = content[:start].strip()
111 | if before:
112 | rel_type = before
113 |
114 | # Get target
115 | target = content[start + 2 : end].strip()
116 |
117 | # Look for context after
118 | after = content[end + 2 :].strip()
119 | if after.startswith("(") and after.endswith(")"):
120 | context = after[1:-1].strip() or None
121 |
122 | if not target: # pragma: no cover
123 | return None
124 |
125 | return {"type": rel_type, "target": target, "context": context}
126 |
127 |
128 | def parse_inline_relations(content: str) -> List[Dict[str, Any]]:
129 | """Find wiki-style links in regular content."""
130 | relations = []
131 | start = 0
132 |
133 | while True:
134 | # Find next outer-most [[
135 | start = content.find("[[", start)
136 | if start == -1: # pragma: no cover
137 | break
138 |
139 | # Find matching ]]
140 | depth = 1
141 | pos = start + 2
142 | end = -1
143 |
144 | while pos < len(content):
145 | if content[pos : pos + 2] == "[[":
146 | depth += 1
147 | pos += 2
148 | elif content[pos : pos + 2] == "]]":
149 | depth -= 1
150 | if depth == 0:
151 | end = pos
152 | break
153 | pos += 2
154 | else:
155 | pos += 1
156 |
157 | if end == -1:
158 | # No matching ]] found
159 | break
160 |
161 | target = content[start + 2 : end].strip()
162 | if target:
163 | relations.append({"type": "links to", "target": target, "context": None})
164 |
165 | start = end + 2
166 |
167 | return relations
168 |
169 |
170 | def observation_plugin(md: MarkdownIt) -> None:
171 | """Plugin for parsing observation format:
172 | - [category] Content text #tag1 #tag2 (context)
173 | - Content text #tag1 (context) # No category is also valid
174 | """
175 |
176 | def observation_rule(state: Any) -> None:
177 | """Process observations in token stream."""
178 | tokens = state.tokens
179 |
180 | for idx in range(len(tokens)):
181 | token = tokens[idx]
182 |
183 | # Initialize meta for all tokens
184 | token.meta = token.meta or {}
185 |
186 | # Parse observations in list items
187 | if token.type == "inline" and is_observation(token):
188 | obs = parse_observation(token)
189 | if obs["content"]: # Only store if we have content
190 | token.meta["observation"] = obs
191 |
192 | # Add the rule after inline processing
193 | md.core.ruler.after("inline", "observations", observation_rule)
194 |
195 |
196 | def relation_plugin(md: MarkdownIt) -> None:
197 | """Plugin for parsing relation formats:
198 |
199 | Explicit relations:
200 | - relation_type [[target]] (context)
201 |
202 | Implicit relations (links in content):
203 | Some text with [[target]] reference
204 | """
205 |
206 | def relation_rule(state: Any) -> None:
207 | """Process relations in token stream."""
208 | tokens = state.tokens
209 | in_list_item = False
210 |
211 | for idx in range(len(tokens)):
212 | token = tokens[idx]
213 |
214 | # Track list nesting
215 | if token.type == "list_item_open":
216 | in_list_item = True
217 | elif token.type == "list_item_close":
218 | in_list_item = False
219 |
220 | # Initialize meta for all tokens
221 | token.meta = token.meta or {}
222 |
223 | # Only process inline tokens
224 | if token.type == "inline":
225 | # Check for explicit relations in list items
226 | if in_list_item and is_explicit_relation(token):
227 | rel = parse_relation(token)
228 | if rel:
229 | token.meta["relations"] = [rel]
230 |
231 | # Always check for inline links in any text
232 | else:
233 | content = token.tag or token.content
234 | if "[[" in content:
235 | rels = parse_inline_relations(content)
236 | if rels:
237 | token.meta["relations"] = token.meta.get("relations", []) + rels
238 |
239 | # Add the rule after inline processing
240 | md.core.ruler.after("inline", "relations", relation_rule)
241 |
```
--------------------------------------------------------------------------------
/specs/SPEC-5 CLI Cloud Upload via WebDAV.md:
--------------------------------------------------------------------------------
```markdown
1 | ---
2 | title: 'SPEC-5: CLI Cloud Upload via WebDAV'
3 | type: spec
4 | permalink: specs/spec-5-cli-cloud-upload-via-webdav
5 | tags:
6 | - cli
7 | - webdav
8 | - upload
9 | - migration
10 | - poc
11 | ---
12 |
13 | # SPEC-5: CLI Cloud Upload via WebDAV
14 |
15 | ## Why
16 |
17 | Existing basic-memory users need a simple migration path to basic-memory-cloud. The web UI drag-and-drop approach outlined in GitHub issue #59, while user-friendly, introduces significant complexity for a proof-of-concept:
18 |
19 | - Complex web UI components for file upload and progress tracking
20 | - Browser file handling limitations and CORS complexity
21 | - Proxy routing overhead for large file transfers
22 | - Authentication integration across multiple services
23 |
24 | A CLI-first approach solves these issues by:
25 |
26 | - **Leveraging existing infrastructure**: Both cloud CLI and tenant API already exist with WorkOS JWT authentication
27 | - **Familiar user experience**: Basic-memory users are CLI-comfortable and expect command-line tools
28 | - **Direct connection efficiency**: Bypassing the MCP gateway/proxy for bulk file transfers
29 | - **Rapid implementation**: Building on existing `CLIAuth` and FastAPI foundations
30 |
31 | The fundamental problem is migration friction - users have local basic-memory projects but no path to cloud tenants. A simple CLI upload command removes this barrier immediately.
32 |
33 | ## What
34 |
35 | This spec defines a CLI-based project upload system using WebDAV for direct tenant connections.
36 |
37 | **Affected Areas:**
38 | - `apps/cloud/src/basic_memory_cloud/cli/main.py` - Add upload command to existing CLI
39 | - `apps/api/src/basic_memory_cloud_api/main.py` - Add WebDAV endpoints to tenant FastAPI
40 | - Authentication flow - Reuse existing WorkOS JWT validation
41 | - File transfer protocol - WebDAV for cross-platform compatibility
42 |
43 | **Core Components:**
44 |
45 | ### CLI Upload Command
46 | ```bash
47 | basic-memory-cloud upload <project-path> --tenant-url https://basic-memory-{tenant}.fly.dev
48 | ```
49 |
50 | ### WebDAV Server Endpoints
51 | - `GET/PUT/DELETE /webdav/*` - Standard WebDAV operations on tenant file system
52 | - Authentication via existing JWT validation
53 | - File operations preserve timestamps and directory structure
54 |
55 | ### Authentication Flow
56 | ```
57 | 1. User runs `basic-memory-cloud login` (existing)
58 | 2. CLI stores WorkOS JWT token (existing)
59 | 3. Upload command reads JWT from storage
60 | 4. WebDAV requests include JWT in Authorization header
61 | 5. Tenant API validates JWT using existing middleware
62 | ```
63 |
64 | ## How (High Level)
65 |
66 | ### Implementation Strategy
67 |
68 | **Phase 1: CLI Command**
69 | - Add `upload` command to existing Typer app
70 | - Reuse `CLIAuth` class for token management
71 | - Implement WebDAV client using `webdavclient3` or similar
72 | - Rich progress bars for transfer feedback
73 |
74 | **Phase 2: WebDAV Server**
75 | - Add WebDAV endpoints to existing tenant FastAPI app
76 | - Leverage existing `get_current_user` dependency for authentication
77 | - Map WebDAV operations to tenant file system
78 | - Preserve file modification times using `os.utime()`
79 |
80 | **Phase 3: Integration**
81 | - Direct connection bypasses MCP gateway and proxy
82 | - Simple conflict resolution: overwrite existing files
83 | - Error handling: fail fast with clear error messages
84 |
85 | ### Technical Architecture
86 |
87 | ```
88 | basic-memory-cloud CLI → WorkOS JWT → Direct WebDAV → Tenant FastAPI
89 | ↓
90 | Tenant File System
91 | ```
92 |
93 | **Key Libraries:**
94 | - CLI: `webdavclient3` for WebDAV client operations
95 | - API: `wsgidav` or FastAPI-compatible WebDAV server
96 | - Progress: `rich` library (already imported in CLI)
97 | - Auth: Existing WorkOS JWT infrastructure
98 |
99 | ### WebDAV Protocol Choice
100 |
101 | WebDAV provides:
102 | - **Cross-platform clients**: Native support in most operating systems
103 | - **Standardized protocol**: Well-defined for file operations
104 | - **HTTP-based**: Works with existing FastAPI and JWT auth
105 | - **Library support**: Good Python libraries for both client and server
106 |
107 | ### POC Constraints
108 |
109 | **Simplifications for rapid implementation:**
110 | - **Known tenant URLs**: Assume `https://basic-memory-{tenant}.fly.dev` format
111 | - **Upload only**: No download or bidirectional sync
112 | - **Overwrite conflicts**: No merge or conflict resolution prompting
113 | - **No fallbacks**: Fail fast if WebDAV connection issues occur
114 | - **Direct connection only**: No proxy fallback mechanism
115 |
116 | ## How to Evaluate
117 |
118 | ### Success Criteria
119 |
120 | **Functional Requirements:**
121 | - [ ] Transfer complete basic-memory project (100+ files) in < 30 seconds
122 | - [ ] Preserve directory structure exactly as in source project
123 | - [ ] Preserve file modification timestamps for proper sync behavior
124 | - [ ] Rich progress bars show real-time transfer status (files/MB transferred)
125 | - [ ] WorkOS JWT authentication validates correctly on WebDAV endpoints
126 | - [ ] Direct tenant connection bypasses MCP gateway successfully
127 |
128 | **Quality Requirements:**
129 | - [ ] Clear error messages for authentication failures
130 | - [ ] Graceful handling of network interruptions
131 | - [ ] CLI follows existing command patterns and help text standards
132 | - [ ] WebDAV endpoints integrate cleanly with existing FastAPI app
133 |
134 | **Performance Requirements:**
135 | - [ ] File transfer speed > 1MB/s on typical connections
136 | - [ ] Memory usage remains reasonable for large projects
137 | - [ ] No timeout issues with 500+ file projects
138 |
139 | ### Testing Procedure
140 |
141 | **Unit Testing:**
142 | 1. CLI command parsing and argument validation
143 | 2. WebDAV client connection and authentication
144 | 3. File timestamp preservation during transfer
145 | 4. JWT token validation on WebDAV endpoints
146 |
147 | **Integration Testing:**
148 | 1. End-to-end upload of test project
149 | 2. Direct tenant connection without proxy
150 | 3. File integrity verification after upload
151 | 4. Progress tracking accuracy during transfer
152 |
153 | **User Experience Testing:**
154 | 1. Upload existing basic-memory project from local installation
155 | 2. Verify uploaded files appear correctly in cloud tenant
156 | 3. Confirm basic-memory database rebuilds properly with uploaded files
157 | 4. Test CLI help text and error message clarity
158 |
159 | ### Validation Commands
160 |
161 | **Setup:**
162 | ```bash
163 | # Login to WorkOS
164 | basic-memory-cloud login
165 |
166 | # Upload project
167 | basic-memory-cloud upload ~/my-notes --tenant-url https://basic-memory-test.fly.dev
168 | ```
169 |
170 | **Verification:**
171 | ```bash
172 | # Check tenant health and file count via API
173 | curl -H "Authorization: Bearer $JWT" https://basic-memory-test.fly.dev/health
174 | curl -H "Authorization: Bearer $JWT" https://basic-memory-test.fly.dev/notes/search
175 | ```
176 |
177 | ### Performance Benchmarks
178 |
179 | **Target metrics for 100MB basic-memory project:**
180 | - Transfer time: < 30 seconds
181 | - Memory usage: < 100MB during transfer
182 | - Progress updates: Every 1MB or 10 files
183 | - Authentication time: < 2 seconds
184 |
185 | ## Observations
186 |
187 | - [implementation-speed] CLI approach significantly faster than web UI for POC development #rapid-prototyping
188 | - [user-experience] Basic-memory users already comfortable with CLI tools #user-familiarity
189 | - [architecture-benefit] Direct connection eliminates proxy complexity and latency #performance
190 | - [auth-reuse] Existing WorkOS JWT infrastructure handles authentication cleanly #code-reuse
191 | - [webdav-choice] WebDAV protocol provides cross-platform compatibility and standard libraries #protocol-selection
192 | - [poc-scope] Simple conflict handling and error recovery sufficient for proof-of-concept #scope-management
193 | - [migration-value] Removes primary barrier for local users migrating to cloud platform #business-value
194 |
195 | ## Relations
196 |
197 | - depends_on [[SPEC-1: Specification-Driven Development Process]]
198 | - enables [[GitHub Issue #59: Web UI Upload Feature]]
199 | - uses [[WorkOS Authentication Integration]]
200 | - builds_on [[Existing Cloud CLI Infrastructure]]
201 | - builds_on [[Existing Tenant API Architecture]]
```
--------------------------------------------------------------------------------
/tests/mcp/tools/test_chatgpt_tools.py:
--------------------------------------------------------------------------------
```python
1 | """Tests for ChatGPT-compatible MCP tools."""
2 |
3 | import json
4 | import pytest
5 | from unittest.mock import AsyncMock, patch
6 |
7 | from basic_memory.schemas.search import SearchResponse, SearchResult, SearchItemType
8 |
9 |
10 | @pytest.mark.asyncio
11 | async def test_search_successful_results():
12 | """Test search with successful results returns proper MCP content array format."""
13 | # Mock successful search results
14 | mock_results = SearchResponse(
15 | results=[
16 | SearchResult(
17 | title="Test Document 1",
18 | permalink="docs/test-doc-1",
19 | content="This is test content for document 1",
20 | type=SearchItemType.ENTITY,
21 | score=1.0,
22 | file_path="/test/docs/test-doc-1.md",
23 | ),
24 | SearchResult(
25 | title="Test Document 2",
26 | permalink="docs/test-doc-2",
27 | content="This is test content for document 2",
28 | type=SearchItemType.ENTITY,
29 | score=0.9,
30 | file_path="/test/docs/test-doc-2.md",
31 | ),
32 | ],
33 | current_page=1,
34 | page_size=10,
35 | )
36 |
37 | with patch(
38 | "basic_memory.mcp.tools.chatgpt_tools.search_notes.fn", new_callable=AsyncMock
39 | ) as mock_search:
40 | mock_search.return_value = mock_results
41 |
42 | # Import and call the actual function
43 | from basic_memory.mcp.tools.chatgpt_tools import search
44 |
45 | result = await search.fn("test query")
46 |
47 | # Verify MCP content array format
48 | assert isinstance(result, list)
49 | assert len(result) == 1
50 | assert result[0]["type"] == "text"
51 |
52 | # Parse the JSON content
53 | content = json.loads(result[0]["text"])
54 | assert "results" in content
55 | assert "query" in content
56 |
57 | # Verify result structure
58 | assert len(content["results"]) == 2
59 | assert content["query"] == "test query"
60 |
61 | # Verify individual result format
62 | result_item = content["results"][0]
63 | assert result_item["id"] == "docs/test-doc-1"
64 | assert result_item["title"] == "Test Document 1"
65 | assert result_item["url"] == "docs/test-doc-1"
66 |
67 |
68 | @pytest.mark.asyncio
69 | async def test_search_with_error_response():
70 | """Test search when underlying search_notes returns error string."""
71 | error_message = "# Search Failed - Invalid Syntax\n\nThe search query contains errors..."
72 |
73 | with patch(
74 | "basic_memory.mcp.tools.chatgpt_tools.search_notes.fn", new_callable=AsyncMock
75 | ) as mock_search:
76 | mock_search.return_value = error_message
77 |
78 | from basic_memory.mcp.tools.chatgpt_tools import search
79 |
80 | result = await search.fn("invalid query")
81 |
82 | # Verify MCP content array format
83 | assert isinstance(result, list)
84 | assert len(result) == 1
85 | assert result[0]["type"] == "text"
86 |
87 | # Parse the JSON content
88 | content = json.loads(result[0]["text"])
89 | assert content["results"] == []
90 | assert content["error"] == "Search failed"
91 | assert "error_details" in content
92 |
93 |
94 | @pytest.mark.asyncio
95 | async def test_fetch_successful_document():
96 | """Test fetch with successful document retrieval."""
97 | document_content = """# Test Document
98 |
99 | This is the content of a test document.
100 |
101 | ## Section 1
102 | Some content here.
103 |
104 | ## Observations
105 | - [observation] This is a test observation
106 |
107 | ## Relations
108 | - relates_to [[Another Document]]
109 | """
110 |
111 | with patch(
112 | "basic_memory.mcp.tools.chatgpt_tools.read_note.fn", new_callable=AsyncMock
113 | ) as mock_read:
114 | mock_read.return_value = document_content
115 |
116 | from basic_memory.mcp.tools.chatgpt_tools import fetch
117 |
118 | result = await fetch.fn("docs/test-document")
119 |
120 | # Verify MCP content array format
121 | assert isinstance(result, list)
122 | assert len(result) == 1
123 | assert result[0]["type"] == "text"
124 |
125 | # Parse the JSON content
126 | content = json.loads(result[0]["text"])
127 | assert content["id"] == "docs/test-document"
128 | assert content["title"] == "Test Document" # Extracted from markdown
129 | assert content["text"] == document_content
130 | assert content["url"] == "docs/test-document"
131 | assert content["metadata"]["format"] == "markdown"
132 |
133 |
134 | @pytest.mark.asyncio
135 | async def test_fetch_document_not_found():
136 | """Test fetch when document is not found."""
137 | error_content = """# Note Not Found: "nonexistent-doc"
138 |
139 | I couldn't find any notes matching "nonexistent-doc". Here are some suggestions:
140 |
141 | ## Check Identifier Type
142 | - If you provided a title, try using the exact permalink instead
143 | """
144 |
145 | with patch(
146 | "basic_memory.mcp.tools.chatgpt_tools.read_note.fn", new_callable=AsyncMock
147 | ) as mock_read:
148 | mock_read.return_value = error_content
149 |
150 | from basic_memory.mcp.tools.chatgpt_tools import fetch
151 |
152 | result = await fetch.fn("nonexistent-doc")
153 |
154 | # Verify MCP content array format
155 | assert isinstance(result, list)
156 | assert len(result) == 1
157 | assert result[0]["type"] == "text"
158 |
159 | # Parse the JSON content
160 | content = json.loads(result[0]["text"])
161 | assert content["id"] == "nonexistent-doc"
162 | assert content["text"] == error_content
163 | assert content["metadata"]["error"] == "Document not found"
164 |
165 |
166 | def test_format_search_results_for_chatgpt():
167 | """Test search results formatting."""
168 | from basic_memory.mcp.tools.chatgpt_tools import _format_search_results_for_chatgpt
169 |
170 | mock_results = SearchResponse(
171 | results=[
172 | SearchResult(
173 | title="Document One",
174 | permalink="docs/doc-one",
175 | content="Content for document one",
176 | type=SearchItemType.ENTITY,
177 | score=1.0,
178 | file_path="/test/docs/doc-one.md",
179 | ),
180 | SearchResult(
181 | title="", # Test empty title handling
182 | permalink="docs/untitled",
183 | content="Content without title",
184 | type=SearchItemType.ENTITY,
185 | score=0.8,
186 | file_path="/test/docs/untitled.md",
187 | ),
188 | ],
189 | current_page=1,
190 | page_size=10,
191 | )
192 |
193 | formatted = _format_search_results_for_chatgpt(mock_results)
194 |
195 | assert len(formatted) == 2
196 | assert formatted[0]["id"] == "docs/doc-one"
197 | assert formatted[0]["title"] == "Document One"
198 | assert formatted[0]["url"] == "docs/doc-one"
199 |
200 | # Test empty title handling
201 | assert formatted[1]["title"] == "Untitled"
202 |
203 |
204 | def test_format_document_for_chatgpt():
205 | """Test document formatting."""
206 | from basic_memory.mcp.tools.chatgpt_tools import _format_document_for_chatgpt
207 |
208 | content = "# Test Document\n\nThis is test content."
209 | result = _format_document_for_chatgpt(content, "docs/test")
210 |
211 | assert result["id"] == "docs/test"
212 | assert result["title"] == "Test Document"
213 | assert result["text"] == content
214 | assert result["url"] == "docs/test"
215 | assert result["metadata"]["format"] == "markdown"
216 |
217 |
218 | def test_format_document_error_handling():
219 | """Test document formatting with error content."""
220 | from basic_memory.mcp.tools.chatgpt_tools import _format_document_for_chatgpt
221 |
222 | error_content = '# Note Not Found: "missing-doc"\n\nDocument not found.'
223 | result = _format_document_for_chatgpt(error_content, "missing-doc", "Missing Doc")
224 |
225 | assert result["id"] == "missing-doc"
226 | assert result["title"] == "Missing Doc"
227 | assert result["text"] == error_content
228 | assert result["metadata"]["error"] == "Document not found"
229 |
```
--------------------------------------------------------------------------------
/src/basic_memory/importers/chatgpt_importer.py:
--------------------------------------------------------------------------------
```python
1 | """ChatGPT import service for Basic Memory."""
2 |
3 | import logging
4 | from datetime import datetime
5 | from typing import Any, Dict, List, Optional, Set
6 |
7 | from basic_memory.markdown.schemas import EntityFrontmatter, EntityMarkdown
8 | from basic_memory.importers.base import Importer
9 | from basic_memory.schemas.importer import ChatImportResult
10 | from basic_memory.importers.utils import clean_filename, format_timestamp
11 |
12 | logger = logging.getLogger(__name__)
13 |
14 |
15 | class ChatGPTImporter(Importer[ChatImportResult]):
16 | """Service for importing ChatGPT conversations."""
17 |
18 | async def import_data(
19 | self, source_data, destination_folder: str, **kwargs: Any
20 | ) -> ChatImportResult:
21 | """Import conversations from ChatGPT JSON export.
22 |
23 | Args:
24 | source_path: Path to the ChatGPT conversations.json file.
25 | destination_folder: Destination folder within the project.
26 | **kwargs: Additional keyword arguments.
27 |
28 | Returns:
29 | ChatImportResult containing statistics and status of the import.
30 | """
31 | try: # pragma: no cover
32 | # Ensure the destination folder exists
33 | self.ensure_folder_exists(destination_folder)
34 | conversations = source_data
35 |
36 | # Process each conversation
37 | messages_imported = 0
38 | chats_imported = 0
39 |
40 | for chat in conversations:
41 | # Convert to entity
42 | entity = self._format_chat_content(destination_folder, chat)
43 |
44 | # Write file
45 | file_path = self.base_path / f"{entity.frontmatter.metadata['permalink']}.md"
46 | await self.write_entity(entity, file_path)
47 |
48 | # Count messages
49 | msg_count = sum(
50 | 1
51 | for node in chat["mapping"].values()
52 | if node.get("message")
53 | and not node.get("message", {})
54 | .get("metadata", {})
55 | .get("is_visually_hidden_from_conversation")
56 | )
57 |
58 | chats_imported += 1
59 | messages_imported += msg_count
60 |
61 | return ChatImportResult(
62 | import_count={"conversations": chats_imported, "messages": messages_imported},
63 | success=True,
64 | conversations=chats_imported,
65 | messages=messages_imported,
66 | )
67 |
68 | except Exception as e: # pragma: no cover
69 | logger.exception("Failed to import ChatGPT conversations")
70 | return self.handle_error("Failed to import ChatGPT conversations", e) # pyright: ignore [reportReturnType]
71 |
72 | def _format_chat_content(
73 | self, folder: str, conversation: Dict[str, Any]
74 | ) -> EntityMarkdown: # pragma: no cover
75 | """Convert chat conversation to Basic Memory entity.
76 |
77 | Args:
78 | folder: Destination folder name.
79 | conversation: ChatGPT conversation data.
80 |
81 | Returns:
82 | EntityMarkdown instance representing the conversation.
83 | """
84 | # Extract timestamps
85 | created_at = conversation["create_time"]
86 | modified_at = conversation["update_time"]
87 |
88 | root_id = None
89 | # Find root message
90 | for node_id, node in conversation["mapping"].items():
91 | if node.get("parent") is None:
92 | root_id = node_id
93 | break
94 |
95 | # Generate permalink
96 | date_prefix = datetime.fromtimestamp(created_at).astimezone().strftime("%Y%m%d")
97 | clean_title = clean_filename(conversation["title"])
98 |
99 | # Format content
100 | content = self._format_chat_markdown(
101 | title=conversation["title"],
102 | mapping=conversation["mapping"],
103 | root_id=root_id,
104 | created_at=created_at,
105 | modified_at=modified_at,
106 | )
107 |
108 | # Create entity
109 | entity = EntityMarkdown(
110 | frontmatter=EntityFrontmatter(
111 | metadata={
112 | "type": "conversation",
113 | "title": conversation["title"],
114 | "created": format_timestamp(created_at),
115 | "modified": format_timestamp(modified_at),
116 | "permalink": f"{folder}/{date_prefix}-{clean_title}",
117 | }
118 | ),
119 | content=content,
120 | )
121 |
122 | return entity
123 |
124 | def _format_chat_markdown(
125 | self,
126 | title: str,
127 | mapping: Dict[str, Any],
128 | root_id: Optional[str],
129 | created_at: float,
130 | modified_at: float,
131 | ) -> str: # pragma: no cover
132 | """Format chat as clean markdown.
133 |
134 | Args:
135 | title: Chat title.
136 | mapping: Message mapping.
137 | root_id: Root message ID.
138 | created_at: Creation timestamp.
139 | modified_at: Modification timestamp.
140 |
141 | Returns:
142 | Formatted markdown content.
143 | """
144 | # Start with title
145 | lines = [f"# {title}\n"]
146 |
147 | # Traverse message tree
148 | seen_msgs: Set[str] = set()
149 | messages = self._traverse_messages(mapping, root_id, seen_msgs)
150 |
151 | # Format each message
152 | for msg in messages:
153 | # Skip hidden messages
154 | if msg.get("metadata", {}).get("is_visually_hidden_from_conversation"):
155 | continue
156 |
157 | # Get author and timestamp
158 | author = msg["author"]["role"].title()
159 | ts = format_timestamp(msg["create_time"]) if msg.get("create_time") else ""
160 |
161 | # Add message header
162 | lines.append(f"### {author} ({ts})")
163 |
164 | # Add message content
165 | content = self._get_message_content(msg)
166 | if content:
167 | lines.append(content)
168 |
169 | # Add spacing
170 | lines.append("")
171 |
172 | return "\n".join(lines)
173 |
174 | def _get_message_content(self, message: Dict[str, Any]) -> str: # pragma: no cover
175 | """Extract clean message content.
176 |
177 | Args:
178 | message: Message data.
179 |
180 | Returns:
181 | Cleaned message content.
182 | """
183 | if not message or "content" not in message:
184 | return ""
185 |
186 | content = message["content"]
187 | if content.get("content_type") == "text":
188 | return "\n".join(content.get("parts", []))
189 | elif content.get("content_type") == "code":
190 | return f"```{content.get('language', '')}\n{content.get('text', '')}\n```"
191 | return ""
192 |
193 | def _traverse_messages(
194 | self, mapping: Dict[str, Any], root_id: Optional[str], seen: Set[str]
195 | ) -> List[Dict[str, Any]]: # pragma: no cover
196 | """Traverse message tree iteratively to handle deep conversations.
197 |
198 | Args:
199 | mapping: Message mapping.
200 | root_id: Root message ID.
201 | seen: Set of seen message IDs.
202 |
203 | Returns:
204 | List of message data.
205 | """
206 | messages = []
207 | if not root_id:
208 | return messages
209 |
210 | # Use iterative approach with stack to avoid recursion depth issues
211 | stack = [root_id]
212 |
213 | while stack:
214 | node_id = stack.pop()
215 | if not node_id:
216 | continue
217 |
218 | node = mapping.get(node_id)
219 | if not node:
220 | continue
221 |
222 | # Process current node if it has a message and hasn't been seen
223 | if node["id"] not in seen and node.get("message"):
224 | seen.add(node["id"])
225 | messages.append(node["message"])
226 |
227 | # Add children to stack in reverse order to maintain conversation flow
228 | children = node.get("children", [])
229 | for child_id in reversed(children):
230 | stack.append(child_id)
231 |
232 | return messages
233 |
```
--------------------------------------------------------------------------------
/v15-docs/basic-memory-home.md:
--------------------------------------------------------------------------------
```markdown
1 | # BASIC_MEMORY_HOME Environment Variable
2 |
3 | **Status**: Existing (clarified in v0.15.0)
4 | **Related**: project-root-env-var.md
5 |
6 | ## What It Is
7 |
8 | `BASIC_MEMORY_HOME` specifies the location of your **default "main" project**. This is the primary directory where Basic Memory stores knowledge files when no other project is specified.
9 |
10 | ## Quick Reference
11 |
12 | ```bash
13 | # Default (if not set)
14 | ~/basic-memory
15 |
16 | # Custom location
17 | export BASIC_MEMORY_HOME=/Users/you/Documents/knowledge-base
18 | ```
19 |
20 | ## How It Works
21 |
22 | ### Default Project Location
23 |
24 | When Basic Memory initializes, it creates a "main" project:
25 |
26 | ```python
27 | # Without BASIC_MEMORY_HOME
28 | projects = {
29 | "main": "~/basic-memory" # Default
30 | }
31 |
32 | # With BASIC_MEMORY_HOME set
33 | export BASIC_MEMORY_HOME=/Users/you/custom-location
34 | projects = {
35 | "main": "/Users/you/custom-location" # Uses env var
36 | }
37 | ```
38 |
39 | ### Only Affects "main" Project
40 |
41 | **Important:** `BASIC_MEMORY_HOME` ONLY sets the path for the "main" project. Other projects are unaffected.
42 |
43 | ```bash
44 | export BASIC_MEMORY_HOME=/Users/you/my-knowledge
45 |
46 | # config.json will have:
47 | {
48 | "projects": {
49 | "main": "/Users/you/my-knowledge", # ← From BASIC_MEMORY_HOME
50 | "work": "/Users/you/work-notes", # ← Independently configured
51 | "personal": "/Users/you/personal-kb" # ← Independently configured
52 | }
53 | }
54 | ```
55 |
56 | ## Relationship with BASIC_MEMORY_PROJECT_ROOT
57 |
58 | These are **separate** environment variables with **different purposes**:
59 |
60 | | Variable | Purpose | Scope | Default |
61 | |----------|---------|-------|---------|
62 | | `BASIC_MEMORY_HOME` | Where "main" project lives | Single project | `~/basic-memory` |
63 | | `BASIC_MEMORY_PROJECT_ROOT` | Security boundary for ALL projects | All projects | None (unrestricted) |
64 |
65 | ### Using Together
66 |
67 | ```bash
68 | # Common containerized setup
69 | export BASIC_MEMORY_HOME=/app/data/basic-memory # Main project location
70 | export BASIC_MEMORY_PROJECT_ROOT=/app/data # All projects must be under here
71 | ```
72 |
73 | **Result:**
74 | - Main project created at `/app/data/basic-memory`
75 | - All other projects must be under `/app/data/`
76 | - Provides both convenience and security
77 |
78 | ### Comparison Table
79 |
80 | | Scenario | BASIC_MEMORY_HOME | BASIC_MEMORY_PROJECT_ROOT | Result |
81 | |----------|-------------------|---------------------------|---------|
82 | | **Default** | Not set | Not set | Main at `~/basic-memory`, projects anywhere |
83 | | **Custom main** | `/Users/you/kb` | Not set | Main at `/Users/you/kb`, projects anywhere |
84 | | **Containerized** | `/app/data/main` | `/app/data` | Main at `/app/data/main`, all projects under `/app/data/` |
85 | | **Secure SaaS** | `/app/tenant-123/main` | `/app/tenant-123` | Main at `/app/tenant-123/main`, tenant isolated |
86 |
87 | ## Use Cases
88 |
89 | ### Personal Setup (Default)
90 |
91 | ```bash
92 | # Use default location
93 | # BASIC_MEMORY_HOME not set
94 |
95 | # Main project created at:
96 | ~/basic-memory/
97 | ```
98 |
99 | ### Custom Location
100 |
101 | ```bash
102 | # Store in Documents folder
103 | export BASIC_MEMORY_HOME=~/Documents/BasicMemory
104 |
105 | # Main project created at:
106 | ~/Documents/BasicMemory/
107 | ```
108 |
109 | ### Synchronized Cloud Folder
110 |
111 | ```bash
112 | # Store in Dropbox/iCloud
113 | export BASIC_MEMORY_HOME=~/Dropbox/BasicMemory
114 |
115 | # Main project syncs via Dropbox:
116 | ~/Dropbox/BasicMemory/
117 | ```
118 |
119 | ### Docker Deployment
120 |
121 | ```bash
122 | # Mount volume for persistence
123 | docker run \
124 | -e BASIC_MEMORY_HOME=/app/data/basic-memory \
125 | -v $(pwd)/data:/app/data \
126 | basic-memory:latest
127 |
128 | # Main project persists at:
129 | ./data/basic-memory/ # (host)
130 | /app/data/basic-memory/ # (container)
131 | ```
132 |
133 | ### Multi-User System
134 |
135 | ```bash
136 | # Per-user isolation
137 | export BASIC_MEMORY_HOME=/home/$USER/basic-memory
138 |
139 | # Alice's main project:
140 | /home/alice/basic-memory/
141 |
142 | # Bob's main project:
143 | /home/bob/basic-memory/
144 | ```
145 |
146 | ## Configuration Examples
147 |
148 | ### Basic Setup
149 |
150 | ```bash
151 | # .bashrc or .zshrc
152 | export BASIC_MEMORY_HOME=~/Documents/knowledge
153 | ```
154 |
155 | ### Docker Compose
156 |
157 | ```yaml
158 | services:
159 | basic-memory:
160 | environment:
161 | BASIC_MEMORY_HOME: /app/data/basic-memory
162 | volumes:
163 | - ./data:/app/data
164 | ```
165 |
166 | ### Kubernetes
167 |
168 | ```yaml
169 | apiVersion: v1
170 | kind: ConfigMap
171 | metadata:
172 | name: basic-memory-config
173 | data:
174 | BASIC_MEMORY_HOME: "/app/data/basic-memory"
175 | ---
176 | apiVersion: v1
177 | kind: Pod
178 | spec:
179 | containers:
180 | - name: basic-memory
181 | envFrom:
182 | - configMapRef:
183 | name: basic-memory-config
184 | ```
185 |
186 | ### systemd Service
187 |
188 | ```ini
189 | [Service]
190 | Environment="BASIC_MEMORY_HOME=/var/lib/basic-memory"
191 | ExecStart=/usr/local/bin/basic-memory serve
192 | ```
193 |
194 | ## Migration
195 |
196 | ### Changing BASIC_MEMORY_HOME
197 |
198 | If you need to change the location:
199 |
200 | **Option 1: Move files**
201 | ```bash
202 | # Stop services
203 | bm sync --stop
204 |
205 | # Move data
206 | mv ~/basic-memory ~/Documents/knowledge
207 |
208 | # Update environment
209 | export BASIC_MEMORY_HOME=~/Documents/knowledge
210 |
211 | # Restart
212 | bm sync
213 | ```
214 |
215 | **Option 2: Copy and sync**
216 | ```bash
217 | # Copy to new location
218 | cp -r ~/basic-memory ~/Documents/knowledge
219 |
220 | # Update environment
221 | export BASIC_MEMORY_HOME=~/Documents/knowledge
222 |
223 | # Verify
224 | bm status
225 |
226 | # Remove old location once verified
227 | rm -rf ~/basic-memory
228 | ```
229 |
230 | ### From v0.14.x
231 |
232 | No changes needed - `BASIC_MEMORY_HOME` works the same way:
233 |
234 | ```bash
235 | # v0.14.x and v0.15.0+ both use:
236 | export BASIC_MEMORY_HOME=~/my-knowledge
237 | ```
238 |
239 | ## Common Patterns
240 |
241 | ### Development vs Production
242 |
243 | ```bash
244 | # Development (.bashrc)
245 | export BASIC_MEMORY_HOME=~/dev/basic-memory-dev
246 |
247 | # Production (systemd/docker)
248 | export BASIC_MEMORY_HOME=/var/lib/basic-memory
249 | ```
250 |
251 | ### Shared Team Setup
252 |
253 | ```bash
254 | # Shared network drive
255 | export BASIC_MEMORY_HOME=/mnt/shared/team-knowledge
256 |
257 | # Note: Use with caution, consider file locking
258 | ```
259 |
260 | ### Backup Strategy
261 |
262 | ```bash
263 | # Primary location
264 | export BASIC_MEMORY_HOME=~/basic-memory
265 |
266 | # Automated backup script
267 | rsync -av ~/basic-memory/ ~/Backups/basic-memory-$(date +%Y%m%d)/
268 | ```
269 |
270 | ## Verification
271 |
272 | ### Check Current Value
273 |
274 | ```bash
275 | # View environment variable
276 | echo $BASIC_MEMORY_HOME
277 |
278 | # View resolved config
279 | bm project list
280 | # Shows actual path for "main" project
281 | ```
282 |
283 | ### Verify Main Project Location
284 |
285 | ```python
286 | from basic_memory.config import ConfigManager
287 |
288 | config = ConfigManager().config
289 | print(config.projects["main"])
290 | # Shows where "main" project is located
291 | ```
292 |
293 | ## Troubleshooting
294 |
295 | ### Main Project Not at Expected Location
296 |
297 | **Problem:** Files not where you expect
298 |
299 | **Check:**
300 | ```bash
301 | # What's the environment variable?
302 | echo $BASIC_MEMORY_HOME
303 |
304 | # Where is main project actually?
305 | bm project list | grep main
306 | ```
307 |
308 | **Solution:** Set environment variable and restart
309 |
310 | ### Permission Errors
311 |
312 | **Problem:** Can't write to BASIC_MEMORY_HOME location
313 |
314 | ```bash
315 | $ bm sync
316 | Error: Permission denied: /var/lib/basic-memory
317 | ```
318 |
319 | **Solution:**
320 | ```bash
321 | # Fix permissions
322 | sudo chown -R $USER:$USER /var/lib/basic-memory
323 |
324 | # Or use accessible location
325 | export BASIC_MEMORY_HOME=~/basic-memory
326 | ```
327 |
328 | ### Conflicts with PROJECT_ROOT
329 |
330 | **Problem:** BASIC_MEMORY_HOME outside PROJECT_ROOT
331 |
332 | ```bash
333 | export BASIC_MEMORY_HOME=/Users/you/kb
334 | export BASIC_MEMORY_PROJECT_ROOT=/app/data
335 |
336 | # Error: /Users/you/kb not under /app/data
337 | ```
338 |
339 | **Solution:** Align both variables
340 | ```bash
341 | export BASIC_MEMORY_HOME=/app/data/basic-memory
342 | export BASIC_MEMORY_PROJECT_ROOT=/app/data
343 | ```
344 |
345 | ## Best Practices
346 |
347 | 1. **Use absolute paths:**
348 | ```bash
349 | export BASIC_MEMORY_HOME=/Users/you/knowledge # ✓
350 | # not: export BASIC_MEMORY_HOME=~/knowledge # ✗ (may not expand)
351 | ```
352 |
353 | 2. **Document the location:**
354 | - Add comment in shell config
355 | - Document for team if shared
356 |
357 | 3. **Backup regularly:**
358 | - Main project contains your primary knowledge
359 | - Automate backups of this directory
360 |
361 | 4. **Consider PROJECT_ROOT for security:**
362 | - Use both together in production/containers
363 |
364 | 5. **Test changes:**
365 | - Verify with `bm project list` after changing
366 |
367 | ## See Also
368 |
369 | - `project-root-env-var.md` - Security constraints for all projects
370 | - `env-var-overrides.md` - Environment variable precedence
371 | - Project management documentation
372 |
```