This is page 1 of 4. Use http://codebase.md/marianfoo/mcp-sap-docs?page={x} to view the full context.
# Directory Structure
```
├── .cursor
│ └── rules
│ ├── 00-overview.mdc
│ ├── 10-search-stack.mdc
│ ├── 20-tools-and-apis.mdc
│ ├── 30-tests-and-output.mdc
│ ├── 40-deploy.mdc
│ ├── 50-metadata-config.mdc
│ ├── 60-adding-github-sources.mdc
│ ├── 70-tool-usage-guide.mdc
│ └── 80-abap-integration.mdc
├── .cursorignore
├── .gitattributes
├── .github
│ ├── ISSUE_TEMPLATE
│ │ ├── config.yml
│ │ ├── missing-documentation.yml
│ │ └── new-documentation-source.yml
│ └── workflows
│ ├── deploy-mcp-sap-docs.yml
│ ├── test-pr.yml
│ └── update-submodules.yml
├── .gitignore
├── .gitmodules
├── .npmignore
├── .vscode
│ ├── extensions.json
│ └── settings.json
├── docs
│ ├── ABAP-INTEGRATION-SUMMARY.md
│ ├── ABAP-MULTI-VERSION-INTEGRATION.md
│ ├── ABAP-STANDARD-INTEGRATION.md
│ ├── ABAP-USAGE-GUIDE.md
│ ├── ARCHITECTURE.md
│ ├── COMMUNITY-SEARCH-IMPLEMENTATION.md
│ ├── CONTENT-SIZE-LIMITS.md
│ ├── CURSOR-SETUP.md
│ ├── DEV.md
│ ├── FTS5-IMPLEMENTATION-COMPLETE.md
│ ├── LLM-FRIENDLY-IMPROVEMENTS.md
│ ├── METADATA-CONSOLIDATION.md
│ ├── TEST-SEARCH.md
│ └── TESTS.md
├── ecosystem.config.cjs
├── index.html
├── LICENSE
├── package-lock.json
├── package.json
├── README.md
├── REMOTE_SETUP.md
├── scripts
│ ├── build-fts.ts
│ ├── build-index.ts
│ ├── check-version.js
│ └── summarize-src.js
├── server.json
├── setup.sh
├── src
│ ├── global.d.ts
│ ├── http-server.ts
│ ├── lib
│ │ ├── BaseServerHandler.ts
│ │ ├── communityBestMatch.ts
│ │ ├── config.ts
│ │ ├── localDocs.ts
│ │ ├── logger.ts
│ │ ├── metadata.ts
│ │ ├── sapHelp.ts
│ │ ├── search.ts
│ │ ├── searchDb.ts
│ │ ├── truncate.ts
│ │ ├── types.ts
│ │ └── url-generation
│ │ ├── abap.ts
│ │ ├── BaseUrlGenerator.ts
│ │ ├── cap.ts
│ │ ├── cloud-sdk.ts
│ │ ├── dsag.ts
│ │ ├── GenericUrlGenerator.ts
│ │ ├── index.ts
│ │ ├── README.md
│ │ ├── sapui5.ts
│ │ ├── utils.ts
│ │ └── wdi5.ts
│ ├── metadata.json
│ ├── server.ts
│ └── streamable-http-server.ts
├── test
│ ├── _utils
│ │ ├── httpClient.js
│ │ └── parseResults.js
│ ├── community-search.ts
│ ├── comprehensive-url-generation.test.ts
│ ├── performance
│ │ └── README.md
│ ├── prompts.test.ts
│ ├── quick-url-test.ts
│ ├── README.md
│ ├── tools
│ │ ├── run-tests.js
│ │ ├── sap_docs_search
│ │ │ ├── search-cap-docs.js
│ │ │ ├── search-cloud-sdk-ai.js
│ │ │ ├── search-cloud-sdk-js.js
│ │ │ └── search-sapui5-docs.js
│ │ ├── search-url-verification.js
│ │ ├── search.generic.spec.js
│ │ └── search.smoke.js
│ ├── url-status.ts
│ └── validate-urls.ts
├── test-community-search.js
├── test-search-interactive.ts
├── test-search.http
├── test-search.ts
├── tsconfig.json
└── vitest.config.ts
```
# Files
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
```
.DS_Store
node_modules
dist
data
sources
.cache
src_context.txt
test_context.txt
.mcpregistry_github_token
.mcpregistry_registry_token
```
--------------------------------------------------------------------------------
/.gitattributes:
--------------------------------------------------------------------------------
```
# Handle large files
*.sqlite binary
*.sqlite-shm binary
*.sqlite-wal binary
*.json -text
*.md text
*.ts text
*.js text
# Submodules
sources/ -text
```
--------------------------------------------------------------------------------
/.npmignore:
--------------------------------------------------------------------------------
```
# Development files
src/
test/
docs/
scripts/
.github/
# Large data files that shouldn't be in NPM package
sources/
data/
dist/data/
*.sqlite
*.db
*.sqlite-shm
*.sqlite-wal
# Development configuration
tsconfig.json
vitest.config.ts
.DS_Store
.cache
ecosystem.config.cjs
setup.sh
# Test and development files
test-*.ts
test-*.js
test-*.http
*_context.txt
server.json
# CI/CD and deployment
.github/
REMOTE_SETUP.md
index.html
# Keep these important files
!README.md
!LICENSE
!package.json
!dist/
```
--------------------------------------------------------------------------------
/.cursorignore:
--------------------------------------------------------------------------------
```
# Build output & caches
dist/**
node_modules/**
.cache/**
coverage/**
*.log
.npm/**
.pnpm-store/**
# Huge vendor docs & tests you don't want in context
sources/**/test/**
sources/openui5/**/test/**
sources/**/.git/**
sources/**/.github/**
sources/**/node_modules/**
sources/**/.cache/**
# Large generated search artifacts
dist/data/index.json
dist/data/*.sqlite
dist/data/*.db
# Test artifacts and temporary files
test-*.js
debug-*.js
*.tmp
*.temp
# IDE and editor files
.vscode/**
.idea/**
*.swp
*.swo
# OS generated files
.DS_Store
Thumbs.db
# Large documentation that's already indexed
sources/sapui5-docs/docs/**/*.md
sources/cap-docs/**/*.md
sources/openui5/docs/**/*.md
sources/wdi5/docs/**/*.md
# Submodule git directories
sources/**/.git
# Large binary or generated files
*.pdf
*.zip
*.tar.gz
*.tgz
```
--------------------------------------------------------------------------------
/.gitmodules:
--------------------------------------------------------------------------------
```
[submodule "sources/sapui5-docs"]
path = sources/sapui5-docs
url = https://github.com/SAP-docs/sapui5.git
branch = main
[submodule "sources/cap-docs"]
path = sources/cap-docs
url = https://github.com/cap-js/docs.git
branch = main
[submodule "sources/openui5"]
path = sources/openui5
url = https://github.com/SAP/openui5.git
branch = master
[submodule "sources/wdi5"]
path = sources/wdi5
url = https://github.com/ui5-community/wdi5.git
branch = main
[submodule "sources/ui5-tooling"]
path = sources/ui5-tooling
url = https://github.com/SAP/ui5-tooling.git
branch = main
[submodule "sources/cloud-mta-build-tool"]
path = sources/cloud-mta-build-tool
url = https://github.com/SAP/cloud-mta-build-tool.git
branch = master
[submodule "sources/ui5-webcomponents"]
path = sources/ui5-webcomponents
url = https://github.com/SAP/ui5-webcomponents.git
branch = main
[submodule "sources/cloud-sdk"]
path = sources/cloud-sdk
url = https://github.com/SAP/cloud-sdk.git
branch = main
[submodule "sources/cloud-sdk-ai"]
path = sources/cloud-sdk-ai
url = https://github.com/SAP/ai-sdk.git
branch = main
[submodule "sources/ui5-typescript"]
path = sources/ui5-typescript
url = https://github.com/UI5/typescript.git
branch = gh-pages
[submodule "sources/ui5-cc-spreadsheetimporter"]
path = sources/ui5-cc-spreadsheetimporter
url = https://github.com/spreadsheetimporter/ui5-cc-spreadsheetimporter.git
branch = main
[submodule "sources/abap-cheat-sheets"]
path = sources/abap-cheat-sheets
url = https://github.com/SAP-samples/abap-cheat-sheets.git
branch = main
[submodule "sources/sap-styleguides"]
path = sources/sap-styleguides
url = https://github.com/SAP/styleguides.git
branch = main
[submodule "sources/dsag-abap-leitfaden"]
path = sources/dsag-abap-leitfaden
url = https://github.com/1DSAG/ABAP-Leitfaden.git
branch = main
[submodule "sources/abap-fiori-showcase"]
path = sources/abap-fiori-showcase
url = https://github.com/SAP-samples/abap-platform-fiori-feature-showcase.git
branch = main
[submodule "sources/cap-fiori-showcase"]
path = sources/cap-fiori-showcase
url = https://github.com/SAP-samples/fiori-elements-feature-showcase.git
branch = main
[submodule "sources/abap-docs"]
path = sources/abap-docs
url = https://github.com/marianfoo/abap-docs.git
branch = main
```
--------------------------------------------------------------------------------
/test/performance/README.md:
--------------------------------------------------------------------------------
```markdown
# Performance Benchmarks
This directory contains performance tests for the hybrid search system.
## Reranker Benchmark
The `reranker-benchmark.js` script compares performance between:
- **BM25-only mode** (`RERANKER_MODEL=""`)
- **BGE reranker mode** (`RERANKER_MODEL="Xenova/bge-reranker-base"`)
### Usage
```bash
npm run benchmark:reranker
```
### What it measures
1. **Startup Time**: How long each server takes to become ready
2. **Request Times**: Average response time for various queries
3. **Memory Usage**: RAM consumption for each mode
### Test Queries
The benchmark tests these representative queries:
- `extensionAPI` - Simple term search
- `UI.Chart #SpecificationWidthColumnChart` - Complex annotation query
- `column micro chart sapui5` - Multi-term search
- `Use enums cql cap` - Technical documentation query
- `getting started with sap cloud sdk for ai` - Long descriptive query
### Expected Results
**BM25-only mode:**
- Startup: ~1-2 seconds
- Requests: ~50-150ms average
- Memory: ~200-400MB
**BGE reranker mode:**
- Startup: ~5-60 seconds (depending on model download)
- Requests: ~200-800ms average
- Memory: ~2-3GB additional
### Notes
- First run with BGE model will be slower due to model download
- Subsequent runs will be faster as model is cached
- Results may vary based on hardware and network conditions
- The benchmark uses different ports (3901, 3902) to avoid conflicts
```
--------------------------------------------------------------------------------
/test/README.md:
--------------------------------------------------------------------------------
```markdown
# URL Validation Testing Scripts
This directory contains comprehensive URL validation tools for the SAP Docs MCP server. These tools help ensure that generated documentation URLs are accurate and reachable.
## 🚀 Quick Start
```bash
# Check URL generation status for all sources
npm run test:urls:status
# Test random URLs from all sources (comprehensive)
npm run test:urls
# Quick test of specific sources
npm run test:urls:quick cloud-sdk 2
```
## 📋 Available Scripts
### 1. `npm run test:urls` - Comprehensive URL Validation
**File**: `validate-urls.ts`
Tests 5 random URLs from each documentation source in parallel. Provides detailed reporting including:
- Success/failure rates by source
- Response times and performance metrics
- Failed URL analysis
- Overall coverage statistics
```bash
npm run test:urls
```
**Sample Output**:
```
🔗 SAP Docs MCP - URL Validation Tool
Testing random URLs from each documentation source...
📚 Testing SAPUI5 (/sapui5)
✅ [200] View Gallery (221ms)
✅ [200] Ordering (190ms)
Results: ✅ 5 OK, ❌ 0 Failed
📊 SUMMARY REPORT
Overall Results:
Total URLs tested: 55
Successful: 34
Failed: 21
Success rate: 61.8%
```
### 2. `npm run test:urls:quick` - Quick Targeted Testing
**File**: `quick-url-test.ts`
Fast testing of specific sources with configurable sample size.
```bash
# Test 2 URLs from Cloud SDK sources
npm run test:urls:quick cloud-sdk 2
# Test 5 URLs from CAP
npm run test:urls:quick cap 5
# Test 1 URL from UI5 sources
npm run test:urls:quick ui5 1
# Test 3 URLs from all sources (default)
npm run test:urls:quick
```
**Sample Output**:
```
🔗 Quick URL Test
📚 Cloud SDK (JavaScript)
✅ [200] Getting Started
https://sap.github.io/cloud-sdk/docs/js/getting-started
215ms
```
### 3. `npm run test:urls:status` - URL Configuration Status
**File**: `url-status.ts`
Shows which sources have URL generation configured and provides system overview.
```bash
npm run test:urls:status
```
**Sample Output**:
```
🔗 URL Generation Status
✅ Sources with URL generation (11):
✅ Cloud SDK (JavaScript) (/cloud-sdk-js)
📄 394 documents
🌐 https://sap.github.io/cloud-sdk/docs/js
❌ Sources without URL generation (1):
❌ OpenUI5 Samples (/openui5-samples)
📊 Summary:
URL generation coverage: 92%
```
## 🎯 Use Cases
### Development Workflow
1. **Check status**: Run `npm run test:urls:status` to see URL configuration coverage
2. **Quick test**: Use `npm run test:urls:quick` to test specific sources you're working on
3. **Full validation**: Run `npm run test:urls` before releases to ensure URL accuracy
### CI/CD Integration
```bash
# Add to your CI pipeline
npm run test:urls
if [ $? -ne 0 ]; then
echo "URL validation failed"
exit 1
fi
```
### Debugging URL Issues
```bash
# Test specific problematic source
npm run test:urls:quick ui5-tooling 5
# Check if URL generation is configured
npm run test:urls:status
```
## 📊 Understanding Results
### Status Codes
- **✅ 200**: URL is valid and reachable
- **❌ 404**: URL not found (indicates URL generation issue)
- **❌ 0**: Network error or timeout
### Success Rates by Source Type
- **100%**: Well-configured sources (SAPUI5, CAP, wdi5)
- **60-80%**: Sources with some URL pattern issues (Cloud SDK variants)
- **0%**: Sources with systematic URL generation problems (UI5 Tooling, Web Components)
### Performance Metrics
- **<200ms**: Good response time
- **200-400ms**: Acceptable response time
- **>400ms**: Slow response (investigate server or network issues)
## 🔧 Features
### Parallel Testing
All URL tests run in parallel for maximum speed and efficiency.
### Error Handling
- Network timeouts (10 second default)
- Graceful failure handling
- Detailed error reporting
### Colored Output
- ✅ Green: Success
- ❌ Red: Failure
- ⚠️ Yellow: Warnings
- 🔵 Blue: Information
### Flexible Filtering
- Test specific sources by name or ID
- Configurable sample sizes
- Support for partial name matching
## 🚀 Advanced Usage
### Custom Source Testing
```bash
# Test only CAP documentation
npx tsx test/quick-url-test.ts cap 10
# Test all UI5-related sources
npx tsx test/quick-url-test.ts ui5 3
```
### Programmatic Usage
```typescript
import { generateDocumentationUrl } from '../src/lib/url-generation/index.js';
import { getDocUrlConfig } from '../src/lib/metadata.js';
const config = getDocUrlConfig('/cloud-sdk-js');
const url = generateDocumentationUrl('/cloud-sdk-js', 'guides/debug.mdx', content, config);
```
## 🐛 Troubleshooting
### Common Issues
1. **"Index not found" Error**
```bash
npm run build
```
2. **High Failure Rate**
- Check internet connection
- Verify URL patterns in `src/metadata.json`
- Check if documentation sites are accessible
3. **Timeout Errors**
- Network connectivity issues
- Server temporarily unavailable
- Consider increasing timeout in code
### Debugging Steps
1. Run status check to see configuration:
```bash
npm run test:urls:status
```
2. Test a small sample first:
```bash
npm run test:urls:quick problematic-source 1
```
3. Check URL generation for specific files:
```bash
# Look at the generated URLs in the output
npm run test:urls:quick source-name 1
```
## 📈 Metrics and Reporting
The comprehensive test provides detailed metrics:
- **Overall success rate**: Percentage of working URLs
- **Per-source breakdown**: Success rates for each documentation source
- **Performance analysis**: Average and maximum response times
- **Failed URL listing**: Complete list of broken URLs for investigation
These metrics help identify:
- Sources needing URL pattern fixes
- Documentation sites with accessibility issues
- Performance bottlenecks in URL validation
## 🎉 Success Stories
After implementing this URL validation system:
- **Fixed Cloud SDK URLs**: Correct frontmatter-based URL generation
- **Identified broken patterns**: Found UI5 Tooling URL configuration issues
- **Performance insights**: Average response time of 233ms across all sources
- **Coverage improvement**: 92% of sources now have URL generation configured
```
--------------------------------------------------------------------------------
/src/lib/url-generation/README.md:
--------------------------------------------------------------------------------
```markdown
# URL Generation System
This directory contains the URL generation system for the SAP Docs MCP server. It provides source-specific URL builders that generate accurate links to documentation based on content metadata and file paths.
## Architecture
The system is organized into source-specific modules with a centralized dispatcher:
```
src/lib/url-generation/
├── index.ts # Main entry point and dispatcher
├── utils.ts # Common utilities (frontmatter parsing, path handling)
├── cloud-sdk.ts # SAP Cloud SDK URL generation
├── sapui5.ts # SAPUI5/OpenUI5 URL generation
├── cap.ts # SAP CAP URL generation
├── wdi5.ts # wdi5 testing framework URL generation
└── README.md # This file
```
## Key Features
### 1. Frontmatter-Based URL Generation
The system extracts metadata from document frontmatter to generate accurate URLs:
```yaml
---
id: remote-debugging
title: Remotely debug an application on SAP BTP
slug: debug-guide
---
```
### 2. Source-Specific Handlers
Each documentation source has its own URL generation logic:
- **Cloud SDK**: Uses frontmatter `id` with section-based paths
- **SAPUI5**: Uses topic IDs and API control names
- **CAP**: Uses docsify-style URLs with section handling
- **wdi5**: Uses docsify-style URLs with testing-specific sections
### 3. Fallback Mechanism
If no source-specific handler is available, the system falls back to generic filename-based URL generation.
### 4. Anchor Generation
Automatically detects main headings in content and generates appropriate anchor fragments based on the documentation platform's anchor style.
## Usage
### Basic Usage
```typescript
import { generateDocumentationUrl } from './url-generation/index.js';
import { getDocUrlConfig } from '../metadata.js';
const config = getDocUrlConfig('/cloud-sdk-js');
const url = generateDocumentationUrl(
'/cloud-sdk-js',
'guides/debug-remote-app.mdx',
content,
config
);
// Result: https://sap.github.io/cloud-sdk/docs/js/guides/remote-debugging
```
### Using Utilities Directly
```typescript
import { parseFrontmatter, extractSectionFromPath, buildUrl } from './url-generation/utils.js';
// Parse document metadata
const frontmatter = parseFrontmatter(content);
// Extract section from file path
const section = extractSectionFromPath('guides/tutorial.mdx'); // '/guides/'
// Build clean URLs
const url = buildUrl('https://example.com', 'docs', 'guides', 'tutorial');
```
## Supported Sources
### SAP Cloud SDK (`/cloud-sdk-js`, `/cloud-sdk-java`, `/cloud-sdk-ai-js`, `/cloud-sdk-ai-java`)
- Uses frontmatter `id` field for URL generation
- Automatically detects sections: guides, features, tutorials, environments
- Example: `https://sap.github.io/cloud-sdk/docs/js/guides/remote-debugging`
### SAPUI5 (`/sapui5`, `/openui5-api`, `/openui5-samples`)
- **SAPUI5 Docs**: Uses topic IDs from frontmatter or filename
- **API Docs**: Uses control/namespace paths
- **Samples**: Uses sample-specific paths
- Example: `https://ui5.sap.com/#/topic/123e4567-e89b-12d3-a456-426614174000`
### CAP (`/cap`)
- Uses docsify-style URLs with `#/` fragments
- Supports frontmatter `id` and `slug` fields
- Handles CDS reference docs, tutorials, and guides
- Example: `https://cap.cloud.sap/docs/#/guides/getting-started`
### wdi5 (`/wdi5`)
- Uses docsify-style URLs for testing documentation
- Handles configuration, selectors, and usage guides
- Example: `https://ui5-community.github.io/wdi5/#/configuration/basic`
## Adding New Sources
To add support for a new documentation source:
1. **Create a new source file** (e.g., `my-source.ts`):
```typescript
import { parseFrontmatter, buildUrl } from './utils.js';
import { DocUrlConfig } from '../metadata.js';
export interface MySourceUrlOptions {
relFile: string;
content: string;
config: DocUrlConfig;
libraryId: string;
}
export function generateMySourceUrl(options: MySourceUrlOptions): string | null {
const { relFile, content, config } = options;
const frontmatter = parseFrontmatter(content);
// Your URL generation logic here
if (frontmatter.id) {
return buildUrl(config.baseUrl, 'docs', frontmatter.id);
}
// Fallback logic
return null;
}
```
2. **Register in the main dispatcher** (`index.ts`):
```typescript
import { generateMySourceUrl, MySourceUrlOptions } from './my-source.js';
const sourceGenerators: Record<string, (options: UrlGenerationOptions) => string | null> = {
// ... existing generators
'/my-source': (options) => generateMySourceUrl({
...options,
libraryId: options.libraryId
} as MySourceUrlOptions),
};
```
3. **Add tests** in `test/url-generation.test.ts`
4. **Export functions** at the bottom of `index.ts`
## Testing
The system includes comprehensive tests covering:
- Utility functions (frontmatter parsing, path handling, URL building)
- Source-specific URL generation
- Main dispatcher functionality
- Error handling
Run tests with:
```bash
npm test test/url-generation.test.ts
```
## Configuration
URL generation is configured through the metadata system. Each source should have:
```json
{
"id": "my-source",
"libraryId": "/my-source",
"baseUrl": "https://docs.example.com",
"pathPattern": "/{file}",
"anchorStyle": "github"
}
```
- `baseUrl`: Base URL for the documentation site
- `pathPattern`: Pattern for constructing paths (`{file}` is replaced with filename)
- `anchorStyle`: How to format anchor fragments (`github`, `docsify`, or `custom`)
## Best Practices
1. **Always use frontmatter when available** - It provides the most reliable URL generation
2. **Handle multiple content patterns** - Documents may have different structures
3. **Provide fallbacks** - Always have a fallback for when specialized logic fails
4. **Test thoroughly** - Include tests for different content patterns and edge cases
5. **Document special cases** - Add comments for source-specific URL patterns
## Troubleshooting
### URLs not generating correctly
1. Check if the source has proper metadata configuration
2. Verify frontmatter format in test documents
3. Ensure the source is registered in the dispatcher
4. Check console logs for fallback usage messages
### Tests failing
1. Verify expected URLs match the actual anchor generation behavior
2. Check that frontmatter parsing handles the content format correctly
3. Ensure path section extraction works with the file structure
### New source not working
1. Confirm the source ID matches between metadata and dispatcher
2. Verify the URL generation function is exported and imported correctly
3. Check that the function returns non-null values for valid inputs
```
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
```markdown
# SAP Documentation MCP Server
A fast, lightweight MCP server that provides unified access to official SAP documentation (SAPUI5, CAP, OpenUI5 APIs & samples, wdi5) using efficient BM25 full-text search.
Use it remotely (hosted URL) or run it locally.
**Public server (MCP Streamable HTTP)**: https://mcp-sap-docs.marianzeis.de/mcp
**Local Streamable HTTP (default: 3122, configurable via MCP_PORT)**: http://127.0.0.1:3122/mcp
**Local HTTP status**: http://127.0.0.1:3001/status
---
## Quick start
<details>
<summary><b>Use the hosted server (recommended)</b></summary>
Point your MCP client to the Streamable HTTP URL:
```
https://mcp-sap-docs.marianzeis.de/mcp
```
Verify from a shell:
```bash
# Should return JSON with api_last_activity
curl -sS https://mcp-sap-docs.marianzeis.de/status | jq .
# Should return HTTP 410 with migration info (SSE endpoint deprecated)
curl -i https://mcp-sap-docs.marianzeis.de/sse
```
</details>
<details>
<summary><b>Run it locally (STDIO + local HTTP status + Streamable HTTP)</b></summary>
```bash
# From repo root
npm ci
./setup.sh # execute this script to clone the github documentation submodules
npm run build
# Start the MCP server (STDIO)
node dist/src/server.js
# OR start the Streamable HTTP server
npm run start:streamable
```
**Local health checks**
```bash
# HTTP server
curl -sS http://127.0.0.1:3001/status | jq .
# Streamable HTTP server (local & deployment default)
curl -sS http://127.0.0.1:3122/health | jq .
```
</details>
---
## What you get
### 🔍 **Unified Documentation Search**
- **search** – Search across all official SAP documentation sources with intelligent filtering
- **fetch** – Retrieve complete documents/snippets with smart formatting
### 🌐 **Community & Help Portal**
- **sap_community_search** – Real-time SAP Community posts with full content of top 3 results (intelligently truncated to 75k chars if needed)
- **sap_help_search** – Comprehensive search across SAP Help Portal documentation
- **sap_help_get** – Retrieve complete SAP Help pages with metadata (intelligently truncated to 75k chars if needed)
---
## Connect from your MCP client
✅ **Remote URL**: use the public MCP Streamable HTTP endpoint
✅ **Local/STDIO**: run `node dist/src/server.js` and point the client to a command + args
✅ **Local/Streamable HTTP**: run `npm run start:streamable` and point the client to `http://127.0.0.1:3122/mcp`
Below are copy-paste setups for popular clients. Each block has remote, local, and streamable HTTP options.
---
## Claude (Desktop / Web "Connectors")
<details>
<summary><b>Remote (recommended) — add a custom connector</b></summary>
1. Open Claude Settings → Connectors → Add custom connector
2. Paste the URL:
```
https://mcp-sap-docs.marianzeis.de/mcp
```
3. Save; Claude will use the MCP Streamable HTTP protocol for communication.
**Docs**: Model Context Protocol ["Connect to Remote MCP Servers"](https://modelcontextprotocol.info/docs/clients/) (shows how Claude connects to MCP servers).
</details>
<details>
<summary><b>Local (STDIO) — add a local MCP server</b></summary>
Point Claude to the command and args:
```
command: node
args: ["<absolute-path-to-your-repo>/dist/src/server.js"]
```
Claude's [user quickstart](https://modelcontextprotocol.io/docs/tutorials/use-remote-mcp-server) shows how to add local servers by specifying a command/args pair.
</details>
<details>
<summary><b>Local (Streamable HTTP) — latest MCP protocol</b></summary>
For the latest MCP protocol (2025-03-26) with Streamable HTTP support:
1. Start the streamable HTTP server:
```bash
npm run start:streamable
```
2. Add a custom connector with the URL:
```
http://127.0.0.1:3122/mcp
```
This provides better performance and supports the latest MCP features including session management and resumability.
</details>
---
## Cursor
<details>
<summary><b>Remote (MCP Streamable HTTP)</b></summary>
Create or edit `~/.cursor/mcp.json`:
```json
{
"mcpServers": {
"sap-docs-remote": {
"url": "https://mcp-sap-docs.marianzeis.de/mcp"
}
}
}
```
</details>
<details>
<summary><b>Local (STDIO)</b></summary>
`~/.cursor/mcp.json`:
```json
{
"mcpServers": {
"sap-docs": {
"command": "node",
"args": ["/absolute/path/to/dist/src/server.js"]
}
}
}
```
</details>
---
## Eclipse (GitHub Copilot)
Eclipse users can integrate the SAP Docs MCP server with GitHub Copilot for seamless access to SAP development documentation.
<details>
<summary><b>Remote (recommended) — hosted server</b></summary>
### Prerequisites
- **Eclipse Version**: 2024-09 or higher
- **GitHub Copilot Extension**: Latest version from Eclipse Marketplace
- **GitHub Account**: With Copilot access
- **Note**: Full ABAP ADT integration is not yet supported
### Configuration Steps
1. **Install GitHub Copilot Extension**
- Download from [Eclipse Marketplace](https://marketplace.eclipse.org/content/github-copilot)
- Follow the installation instructions
2. **Open MCP Configuration**
- Click the Copilot icon (🤖) in the Eclipse status bar
- Select "Edit preferences" from the menu
- Expand "Copilot Chat" in the left panel
- Click on "MCP"
3. **Add SAP Docs MCP Server**
```json
{
"name": "SAP Docs MCP",
"description": "Comprehensive SAP development documentation with ABAP keyword documentation",
"url": "https://mcp-sap-docs.marianzeis.de/mcp"
}
```
4. **Verify Configuration**
- The server should appear in your MCP servers list
- Status should show as "Connected" when active
### Using SAP Docs in Eclipse
Once configured, you can use Copilot Chat in Eclipse with enhanced SAP documentation:
**Example queries:**
```
How do I implement a Wizard control in UI5?
What is the syntax for inline declarations in ABAP 7.58?
Show me best practices for RAP development
Find wdi5 testing examples for OData services
```
**Available Tools:**
- `search` - **Unified search** for all SAP development (UI5, CAP, ABAP, testing) with intelligent ABAP version filtering
- `fetch` - Retrieve complete documentation for any source
- `sap_community_search` - SAP Community integration
- `sap_help_search` - SAP Help Portal access
</details>
<details>
<summary><b>Local setup — for offline use</b></summary>
### Local MCP Server Configuration
```json
{
"name": "SAP Docs MCP (Local)",
"description": "Local SAP documentation server",
"command": "npm",
"args": ["start"],
"cwd": "/absolute/path/to/your/sap-docs-mcp",
"env": {
"NODE_ENV": "production"
}
}
```
**Prerequisites for local setup:**
1. Clone and build this repository locally
2. Run `npm run setup` to initialize all documentation sources
3. Ensure the server starts correctly with `npm start`
</details>
---
## VS Code (GitHub Copilot Chat)
<details>
<summary><b>Remote (recommended) — no setup required</b></summary>
**Prerequisites**: VS Code 1.102+ with MCP support enabled (enabled by default).
### Quick Setup
Create `.vscode/mcp.json` in your workspace:
```json
{
"servers": {
"sap-docs": {
"type": "http",
"url": "https://mcp-sap-docs.marianzeis.de/mcp"
}
}
}
```
### Using the Remote Server
1. Save the `.vscode/mcp.json` file in your workspace
2. VS Code will automatically detect and start the MCP server
3. Open Chat view and select **Agent mode**
4. Click **Tools** button to see available SAP documentation tools
5. Ask questions like "How do I implement authentication in SAPUI5?"
**Benefits**:
- ✅ No local installation required
- ✅ Always up-to-date documentation
- ✅ Automatic updates and maintenance
- ✅ Works across all your projects
**Note**: You'll be prompted to trust the remote MCP server when connecting for the first time.
</details>
<details>
<summary><b>Local setup — for offline use</b></summary>
### Local STDIO Server
```json
{
"servers": {
"sap-docs-local": {
"type": "stdio",
"command": "node",
"args": ["<absolute-path>/dist/src/server.js"]
}
}
}
```
### Local HTTP Server
```json
{
"servers": {
"sap-docs-http": {
"type": "http",
"url": "http://127.0.0.1:3122/mcp"
}
}
}
```
(Start local server with `npm run start:streamable` first)
### Alternative Setup Methods
- **Command Palette**: Run `MCP: Add Server` → choose server type → provide details → select scope
- **User Configuration**: Run `MCP: Open User Configuration` for global setup across all workspaces
See Microsoft's ["Use MCP servers in VS Code"](https://code.visualstudio.com/docs/copilot/chat/mcp-servers) for complete documentation.
</details>
---
## Raycast
<details>
<summary><b>Remote (MCP Streamable HTTP)</b></summary>
Open Raycast → Open Command "Manage Servers (MCP) → Import following JSON
```json
{
"mcpServers": {
"sap-docs": {
"command": "npx",
"args": ["mcp-remote@latest", "https://mcp-sap-docs.marianzeis.de/mcp"]
}
}
}
```
</details>
<details>
<summary><b>Local (STDIO)</b></summary>
Open Raycast → Open Command "Manage Servers (MCP) → Import following JSON
```json
{
"mcpServers": {
"sap-docs": {
"command": "node",
"args": ["/absolute/path/to/dist/src/server.js"]
}
}
}
```
</details>
Raycast by default asks to confirm each usage of an MCP tool. You can enable automatic confirmation:
Open Raycast → Raycast Settings → AI → Model Context Protocol → Check "Automatically confirm all tool calls"
---
## Features
### 🔍 Advanced Search Capabilities
- **Unified search** across all official SAP documentation with intelligent ABAP version filtering
- **BM25 full-text search** with SQLite FTS5 for fast, relevant results (~15ms average query time)
- **Context-aware scoring** with automatic stopword filtering and phrase detection
- **Version-specific filtering** - shows latest ABAP by default, specific versions only when requested
### 🌐 Real-time External Integration
- **SAP Community**: Full content retrieval using "Best Match" algorithm with engagement metrics
- **SAP Help Portal**: Direct API access to all SAP product documentation (S/4HANA, BTP, Analytics Cloud)
- **Efficient processing**: Batch content retrieval and intelligent caching for fast response times
### 💡 Smart Features
- **Automatic content enhancement**: Code highlighting and sample categorization
- **Intelligent ranking**: Context-aware scoring with source-specific weighting
- **Performance optimized**: Lightweight SQLite FTS5 with no external ML dependencies
---
## What's Included
This MCP server provides unified access to **comprehensive SAP development documentation** across multiple product areas. All sources are searched simultaneously through the `search` tool, with intelligent filtering and ranking.
### 📊 Documentation Coverage Overview
| Source Category | Sources | File Count | Description |
|-----------------|---------|------------|-------------|
| **ABAP Development** | 4 sources | 40,800+ files | Official ABAP keyword docs (8 versions), cheat sheets, Fiori showcase, community guidelines |
| **UI5 Development** | 6 sources | 12,000+ files | SAPUI5 docs, OpenUI5 APIs/samples, TypeScript, tooling, web components, custom controls |
| **CAP Development** | 2 sources | 250+ files | Cloud Application Programming model docs and Fiori Elements showcase |
| **Cloud & Deployment** | 3 sources | 500+ files | SAP Cloud SDK (JS/Java), Cloud SDK for AI, Cloud MTA Build Tool |
| **Testing & Quality** | 2 sources | 260+ files | wdi5 E2E testing framework, SAP style guides |
### 🔍 ABAP Development Sources
- **Official ABAP Keyword Documentation** (`/abap-docs`) - **40,761+ curated ABAP files** across 8 versions (7.52-7.58 + latest) with intelligent version filtering
📁 **GitHub**: [marianfoo/abap-docs](https://github.com/marianfoo/abap-docs)
- **ABAP Cheat Sheets** (`/abap-cheat-sheets`) - 32 comprehensive cheat sheets covering core ABAP concepts, SQL, OOP, RAP, and more
📁 **GitHub**: [SAP-samples/abap-cheat-sheets](https://github.com/SAP-samples/abap-cheat-sheets)
- **ABAP RAP Fiori Elements Showcase** (`/abap-fiori-showcase`) - Complete annotation reference for ABAP RESTful Application Programming (RAP)
📁 **GitHub**: [SAP-samples/abap-platform-fiori-feature-showcase](https://github.com/SAP-samples/abap-platform-fiori-feature-showcase)
- **DSAG ABAP Guidelines** (`/dsag-abap-leitfaden`) - German ABAP community best practices and development standards
📁 **GitHub**: [1DSAG/ABAP-Leitfaden](https://github.com/1DSAG/ABAP-Leitfaden)
### 🎨 UI5 Development Sources
- **SAPUI5 Documentation** (`/sapui5-docs`) - **1,485+ files** - Complete official developer guide, controls, and best practices
📁 **GitHub**: [SAP-docs/sapui5](https://github.com/SAP-docs/sapui5)
- **OpenUI5 Framework** (`/openui5`) - **20,000+ files** - Complete OpenUI5 source including 500+ control APIs with detailed JSDoc and 2,000+ working examples from demokit samples
📁 **GitHub**: [SAP/openui5](https://github.com/SAP/openui5)
- **UI5 TypeScript Integration** (`/ui5-typescript`) - Official TypeScript setup guides, type definitions, and migration documentation
📁 **GitHub**: [UI5/typescript](https://github.com/UI5/typescript)
- **UI5 Tooling** (`/ui5-tooling`) - Complete UI5 Tooling documentation for project setup, build, and development workflows
📁 **GitHub**: [SAP/ui5-tooling](https://github.com/SAP/ui5-tooling)
- **UI5 Web Components** (`/ui5-webcomponents`) - **4,500+ files** - Comprehensive web components documentation, APIs, and implementation examples
📁 **GitHub**: [SAP/ui5-webcomponents](https://github.com/SAP/ui5-webcomponents)
- **UI5 Custom Controls** (`/ui5-cc-spreadsheetimporter`) - Spreadsheet importer and other community custom control documentation
📁 **GitHub**: [spreadsheetimporter/ui5-cc-spreadsheetimporter](https://github.com/spreadsheetimporter/ui5-cc-spreadsheetimporter)
### ☁️ CAP Development Sources
- **CAP Documentation** (`/cap-docs`) - **195+ files** - Complete Cloud Application Programming model documentation for Node.js and Java
📁 **GitHub**: [cap-js/docs](https://github.com/cap-js/docs)
- **CAP Fiori Elements Showcase** (`/cap-fiori-showcase`) - Comprehensive annotation reference and examples for CAP-based Fiori Elements applications
📁 **GitHub**: [SAP-samples/fiori-elements-feature-showcase](https://github.com/SAP-samples/fiori-elements-feature-showcase)
### 🚀 Cloud & Deployment Sources
- **SAP Cloud SDK for JavaScript** (`/cloud-sdk`) - Complete SDK documentation, tutorials, and API references for JavaScript/TypeScript
📁 **GitHub**: [SAP/cloud-sdk](https://github.com/SAP/cloud-sdk)
- **SAP Cloud SDK for Java** (`/cloud-sdk`) - Comprehensive Java SDK documentation and integration guides
📁 **GitHub**: [SAP/cloud-sdk](https://github.com/SAP/cloud-sdk)
- **SAP Cloud SDK for AI** (`/cloud-sdk-ai`) - Latest AI capabilities integration documentation for both JavaScript and Java
📁 **GitHub**: [SAP/ai-sdk](https://github.com/SAP/ai-sdk)
- **Cloud MTA Build Tool** (`/cloud-mta-build-tool`) - Complete documentation for Multi-Target Application development and deployment
📁 **GitHub**: [SAP/cloud-mta-build-tool](https://github.com/SAP/cloud-mta-build-tool)
### ✅ Testing & Quality Sources
- **wdi5 Testing Framework** (`/wdi5`) - **225+ files** - End-to-end testing documentation, setup guides, and real-world examples
📁 **GitHub**: [ui5-community/wdi5](https://github.com/ui5-community/wdi5)
- **SAP Style Guides** (`/sap-styleguides`) - Official SAP coding standards, clean code practices, and development guidelines
📁 **GitHub**: [SAP/styleguides](https://github.com/SAP/styleguides)
---
## Example Prompts
Try these with any connected MCP client to explore the comprehensive documentation:
### 🔍 ABAP Development Queries
**ABAP Keyword Documentation (8 versions with intelligent filtering):**
- "What is the syntax for inline declarations in ABAP 7.58?"
- "How do I use SELECT statements with internal tables in ABAP 7.57?"
- "Show me exception handling with TRY-CATCH in modern ABAP"
- "What are constructor expressions for VALUE and CORRESPONDING?"
- "How do I implement ABAP Unit tests with test doubles?"
**ABAP Best Practices & Guidelines:**
- "What is Clean ABAP and how do I follow the style guide?"
- "Show me ABAP cheat sheet for internal tables operations"
- "Find DSAG ABAP guidelines for object-oriented programming"
- "How to implement RAP with EML in ABAP for Cloud?"
### 🎨 UI5 Development Queries
**SAPUI5 & OpenUI5:**
- "How do I implement authentication in SAPUI5?"
- "Find OpenUI5 button control examples with click handlers"
- "Show me fragment reuse patterns in UI5"
- "What are UI5 model binding best practices?"
**Modern UI5 Development:**
- "Show me TypeScript setup for UI5 development"
- "How do I configure UI5 Tooling for a new project?"
- "Find UI5 Web Components integration examples"
- "How to implement custom controls with UI5 Web Components?"
### ☁️ CAP & Cloud Development
**CAP Framework:**
- "How do I implement CDS views with calculated fields in CAP?"
- "Show me CAP authentication and authorization patterns"
- "Find CAP Node.js service implementation examples"
- "How to handle temporal data in CAP applications?"
**Cloud SDK & Deployment:**
- "How do I use SAP Cloud SDK for JavaScript with OData?"
- "Show me Cloud SDK for AI integration examples"
- "Find Cloud MTA Build Tool configuration for multi-target apps"
- "How to deploy CAP applications to SAP BTP?"
### ✅ Testing & Quality
**Testing Frameworks:**
- "Show me wdi5 testing examples for forms and tables"
- "How do I set up wdi5 for OData service testing?"
- "Find end-to-end testing patterns for Fiori Elements apps"
**Code Quality:**
- "What are SAP style guide recommendations for JavaScript?"
- "Show me clean code practices for ABAP development"
### 🌐 Community & Help Portal
**Community Knowledge (with full content):**
- "Find community examples of OData batch operations with complete implementation"
- "Search for RAP development tips and tricks from the community"
- "What are the latest CAP authentication best practices from the community?"
**SAP Help Portal:**
- "How to configure S/4HANA Fiori Launchpad?"
- "Find BTP integration documentation for Analytics Cloud"
- "Search for ABAP development best practices in S/4HANA"
---
## Troubleshooting
<details>
<summary><b>Claude says it can't connect</b></summary>
- Make sure you're using the modern MCP Streamable HTTP URL:
`https://mcp-sap-docs.marianzeis.de/mcp` (not /sse, which is deprecated).
- Test MCP endpoint from your machine:
```bash
curl -i https://mcp-sap-docs.marianzeis.de/mcp
```
You should see JSON indicating MCP protocol support.
</details>
<details>
<summary><b>VS Code wizard can't detect the server</b></summary>
- Try adding it as URL first. If there are connection issues, use your local server via command:
```
node <absolute-path>/dist/src/server.js
```
- Microsoft's ["Add an MCP server"](https://code.visualstudio.com/docs/copilot/chat/mcp-servers) guide shows both URL and command flows.
</details>
<details>
<summary><b>Local server runs, but the client can't find it</b></summary>
- Ensure you're pointing to the built entry:
```
node dist/src/server.js
```
- If using PM2/systemd, confirm it's alive:
```bash
pm2 status mcp-sap-http
pm2 status mcp-sap-proxy
curl -fsS http://127.0.0.1:3001/status | jq .
curl -fsS http://127.0.0.1:18080/status | jq .
```
</details>
---
## Development
### Build Commands
```bash
npm run build:tsc # Compile TypeScript
npm run build:index # Build search index from sources
npm run build:fts # Build FTS5 database
npm run build # Complete build pipeline (tsc + index + fts)
npm run setup # Complete setup (submodules + build)
```
### Server Commands
```bash
npm start # Start STDIO MCP server
npm run start:http # Start HTTP status server (port 3001)
npm run start:streamable # Start Streamable HTTP MCP server (port 3122)
```
### Local Setup
```bash
git clone https://github.com/marianfoo/mcp-sap-docs.git
cd mcp-sap-docs
npm ci # Install dependencies
npm run setup # Enhanced setup (optimized submodules + complete build)
```
The build process creates optimized search indices for fast offline access while maintaining real-time connectivity to the SAP Community API.
---
## Health & Status Monitoring
### Public Endpoints
```bash
# Check server status
curl -sS https://mcp-sap-docs.marianzeis.de/status | jq .
# Test MCP endpoint
curl -i https://mcp-sap-docs.marianzeis.de/mcp
```
### Local Endpoints
```bash
# HTTP server status
curl -sS http://127.0.0.1:3001/status | jq .
# MCP Streamable HTTP server status
curl -sS http://127.0.0.1:3122/health | jq .
```
---
## Deployment
### Automated Workflows
This project includes dual automated workflows:
1. **Main Deployment** (on push to `main` or manual trigger)
- SSH into server and pull latest code
- Run enhanced setup with optimized submodule handling
- Restart all PM2 processes (http, streamable) with health checks
2. **Daily Documentation Updates** (4 AM UTC)
- Update all documentation submodules to latest versions
- Rebuild search indices with fresh content using enhanced setup
- Restart services automatically
### Manual Updates
Trigger documentation updates anytime via GitHub Actions → "Update Documentation Submodules" workflow.
---
## Architecture
- **MCP Server** (Node.js/TypeScript) - Exposes Resources/Tools for SAP docs, community & help portal
- **Streamable HTTP Transport** (Latest MCP spec) - HTTP-based transport with session management and resumability
- **BM25 Search Engine** - SQLite FTS5 with optimized OR-logic queries for fast, relevant results
- **Optimized Submodules** - Shallow, single-branch clones with blob filtering for minimal bandwidth
### Technical Stack
- **Search Engine**: BM25 with SQLite FTS5 for fast full-text search with OR logic
- **Performance**: ~15ms average query time with optimized indexing
- **Transport**: Latest MCP protocol with HTTP Streamable transport and session management
```
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/config.yml:
--------------------------------------------------------------------------------
```yaml
blank_issues_enabled: true
```
--------------------------------------------------------------------------------
/.vscode/extensions.json:
--------------------------------------------------------------------------------
```json
{
"recommendations": [
"ms-vscode.vscode-typescript-next",
"esbenp.prettier-vscode",
"ms-vscode.vscode-json"
]
}
```
--------------------------------------------------------------------------------
/src/global.d.ts:
--------------------------------------------------------------------------------
```typescript
import { SapHelpSearchResult } from "./lib/types.js";
declare global {
var sapHelpSearchCache: Map<string, SapHelpSearchResult> | undefined;
}
export {};
```
--------------------------------------------------------------------------------
/test/tools/sap_docs_search/search-cap-docs.js:
--------------------------------------------------------------------------------
```javascript
// CAP documentation test cases
export default [
{
name: 'CAP CQL Enums - Enum definitions',
tool: 'search',
query: 'Use enums cql cap',
expectIncludes: ['/cap/cds/cql']
}
];
```
--------------------------------------------------------------------------------
/tsconfig.json:
--------------------------------------------------------------------------------
```json
{
"compilerOptions": {
"target": "ES2022",
"module": "ES2022",
"moduleResolution": "Node",
"outDir": "dist",
"rootDir": ".",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true
},
"include": ["src", "scripts"]
}
```
--------------------------------------------------------------------------------
/.vscode/settings.json:
--------------------------------------------------------------------------------
```json
{
"search.exclude": {
"**/sources/**": true,
"**/node_modules/**": true,
"**/dist/**": true,
"**/data/**": true
},
"files.watcherExclude": {
"**/sources/**": true,
"**/dist/**": true,
"**/data/**": true
},
"git.ignoredFolders": [
"sources"
],
"typescript.preferences.includePackageJsonAutoImports": "off"
}
```
--------------------------------------------------------------------------------
/test/tools/sap_docs_search/search-cloud-sdk-ai.js:
--------------------------------------------------------------------------------
```javascript
// SAP Cloud SDK for AI test cases
export default [
{
name: 'AI SDK error handling doc present',
tool: 'search',
query: 'how to access Error Information in the cloud sdk for ai',
expectIncludes: ['/cloud-sdk-ai-js/error-handling.mdx']
},
{
name: 'AI SDK getting started guide',
tool: 'search',
query: 'getting started with sap cloud sdk for ai',
expectIncludes: ['/cloud-sdk-ai-js/']
},
{
name: 'AI SDK FAQ section',
tool: 'search',
query: 'cloud sdk ai frequently asked questions',
expectIncludes: ['/cloud-sdk-ai-java/faq.mdx']
}
];
```
--------------------------------------------------------------------------------
/test/tools/sap_docs_search/search-cloud-sdk-js.js:
--------------------------------------------------------------------------------
```javascript
// SAP Cloud SDK (JavaScript) test cases
export default [
{
name: 'Cloud SDK JS remote debug guide present',
tool: 'search',
query: 'debug remote app cloud sdk',
expectIncludes: ['/cloud-sdk-js/guides/debug-remote-app.mdx']
},
{
name: 'Cloud SDK JS getting started',
tool: 'search',
query: 'getting started cloud sdk javascript',
expectIncludes: ['/cloud-sdk-js/']
},
{
name: 'Cloud SDK JS upgrade guide',
tool: 'search',
query: 'cloud sdk javascript upgrade version 4',
expectIncludes: ['/cloud-sdk-js/guides/upgrade-to-version-4.mdx']
}
];
```
--------------------------------------------------------------------------------
/test/tools/sap_docs_search/search-sapui5-docs.js:
--------------------------------------------------------------------------------
```javascript
// SAPUI5 documentation test cases
export default [
{
name: 'SAPUI5 Column Micro Chart',
tool: 'search',
query: 'column micro chart sapui5',
expectIncludes: ['/sapui5/06_SAP_Fiori_Elements/column-micro-chart-1a4ecb8']
},
{
name: 'SAPUI5 Column Micro Chart',
tool: 'search',
query: 'UI.Chart #SpecificationWidthColumnChart',
expectIncludes: ['/sapui5/06_SAP_Fiori_Elements/column-micro-chart-1a4ecb8']
},
{
name: 'SAPUI5 ExtensionAPI',
tool: 'search',
query: 'extensionAPI',
expectIncludes: ['/sapui5/06_SAP_Fiori_Elements/using-the-extensionapi-bd2994b']
}
];
```
--------------------------------------------------------------------------------
/server.json:
--------------------------------------------------------------------------------
```json
{
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-07-09/server.schema.json",
"name": "io.github.marianfoo/mcp-sap-docs",
"description": "Fast MCP server for unified SAP docs search (SAPUI5, CAP, OpenUI5, wdi5) with BM25 full-text search",
"status": "active",
"repository": {
"url": "https://github.com/marianfoo/mcp-sap-docs",
"source": "github"
},
"version": "0.3.15",
"packages": [
{
"registry_type": "npm",
"registry_base_url": "https://registry.npmjs.org",
"identifier": "mcp-sap-docs",
"version": "0.3.15",
"transport": {
"type": "stdio"
}
}
]
}
```
--------------------------------------------------------------------------------
/test/_utils/parseResults.js:
--------------------------------------------------------------------------------
```javascript
// Parse the formatted summary from http-server /mcp response
// Extracts top items and their numeric scores.
export function parseSummaryText(text) {
const items = [];
const lineRe = /^⭐️ \*\*(.+?)\*\* \(Score: ([\d.]+)\)/;
const lines = String(text || '').split('\n');
for (const line of lines) {
const m = line.match(lineRe);
if (m) {
items.push({
id: m[1],
finalScore: parseFloat(m[2]),
rerankerScore: 0, // Always 0 in BM25-only mode
});
}
}
const matchCandidates = text.match(/Found (\d+) results/);
const totalCandidates = matchCandidates ? parseInt(matchCandidates[1], 10) : null;
return { items, totalCandidates };
}
```
--------------------------------------------------------------------------------
/vitest.config.ts:
--------------------------------------------------------------------------------
```typescript
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
// Enable TypeScript support
typecheck: {
enabled: true
},
// Test environment
environment: 'node',
// Include patterns
include: [
'test/**/*.test.ts',
'test/**/*.spec.ts'
],
// Exclude patterns
exclude: [
'node_modules',
'dist',
'sources'
],
// Test timeout
testTimeout: 10000,
// Reporter
reporter: ['verbose', 'json'],
// Coverage (optional)
coverage: {
provider: 'v8',
reporter: ['text', 'json', 'html'],
exclude: [
'node_modules/',
'test/',
'dist/',
'sources/'
]
}
}
});
```
--------------------------------------------------------------------------------
/src/lib/config.ts:
--------------------------------------------------------------------------------
```typescript
// Central configuration for search system
export const CONFIG = {
// Default number of results to return
RETURN_K: Number(process.env.RETURN_K ||30),
// Database paths
DB_PATH: "dist/data/docs.sqlite",
METADATA_PATH: "src/metadata.json",
// Search behavior
USE_OR_LOGIC: true, // Use OR logic for better recall in BM25-only mode
// Excerpt lengths for different search types
EXCERPT_LENGTH_MAIN: 400, // Main search results excerpt length
EXCERPT_LENGTH_COMMUNITY: 600, // Community search results excerpt length
// Maximum content length for SAP Help and Community full content retrieval
// Limits help prevent token overflow and keep responses manageable (~18,750 tokens)
MAX_CONTENT_LENGTH: 75000, // 75,000 characters
};
```
--------------------------------------------------------------------------------
/test/tools/search.smoke.js:
--------------------------------------------------------------------------------
```javascript
// Simple smoke test for critical search behaviors
import { startServerHttp, waitForStatus, stopServer, docsSearch } from '../_utils/httpClient.js';
import { parseSummaryText } from '../_utils/parseResults.js';
import assert from 'node:assert/strict';
const QUERIES = [
{ q: 'UI5 column micro chart', expect: /Column Micro Chart|Micro.*Chart/i },
{ q: 'CAP CQL enums', expect: /Use enums|CQL/i },
{ q: 'Cloud SDK AI getting started', expect: /getting started|AI SDK/i },
{ q: 'ExtensionAPI', expect: /ExtensionAPI/i },
];
(async () => {
const child = startServerHttp();
try {
await waitForStatus();
for (const { q, expect } of QUERIES) {
const summary = await docsSearch(q);
const { items, totalCandidates } = parseSummaryText(summary);
assert.ok(items.length > 0, `no results for "${q}"`);
assert.ok(expect.test(summary), `expected hint missing in "${q}"`);
// Assert we're in BM25-only mode
assert.ok(items.every(i => i.rerankerScore === 0), 'reranker not zero');
}
console.log('✅ Smoke tests passed');
} finally {
await stopServer(child);
}
})();
```
--------------------------------------------------------------------------------
/src/lib/url-generation/GenericUrlGenerator.ts:
--------------------------------------------------------------------------------
```typescript
/**
* Generic URL Generator for sources without specialized logic
* Handles ui5-tooling, ui5-webcomponents, cloud-mta-build-tool, etc.
*/
import { BaseUrlGenerator, UrlGenerationContext } from './BaseUrlGenerator.js';
import { FrontmatterData } from './utils.js';
/**
* Generic URL Generator
* Uses configuration-based URL generation for sources without special requirements
*/
export class GenericUrlGenerator extends BaseUrlGenerator {
protected generateSourceSpecificUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
const identifier = this.getIdentifierFromFrontmatter(context.frontmatter);
// Use frontmatter ID if available
if (identifier) {
// Use the pathPattern to construct the URL properly
let url = this.config.pathPattern.replace('{file}', identifier);
url = this.config.baseUrl + url;
// Add anchor if available
if (context.anchor) {
url += this.getSeparator() + context.anchor;
}
return url;
}
return null; // Let fallback handle filename-based generation
}
}
```
--------------------------------------------------------------------------------
/src/lib/types.ts:
--------------------------------------------------------------------------------
```typescript
export interface SearchResult {
library_id: string;
topic: string;
id: string;
title: string;
url?: string;
snippet?: string;
score?: number;
metadata?: Record<string, any>;
// Legacy fields for backward compatibility
description?: string;
totalSnippets?: number;
source?: string; // "docs" | "community" | "help"
postTime?: string; // For community posts
author?: string; // For community posts - author name
likes?: number; // For community posts - number of likes/kudos
tags?: string[]; // For community posts - associated tags
}
export interface SearchResponse {
results: SearchResult[];
error?: string;
}
// SAP Help specific types
export interface SapHelpSearchResult {
loio: string;
title: string;
url: string;
productId?: string;
product?: string;
version?: string;
versionId?: string;
language?: string;
snippet?: string;
}
export interface SapHelpSearchResponse {
data?: {
results?: SapHelpSearchResult[];
};
}
export interface SapHelpMetadataResponse {
data?: {
deliverable?: {
id: string;
buildNo: string;
};
filePath?: string;
};
}
export interface SapHelpPageContentResponse {
data?: {
currentPage?: {
t?: string; // title
};
deliverable?: {
title?: string;
};
body?: string;
};
}
```
--------------------------------------------------------------------------------
/scripts/check-version.js:
--------------------------------------------------------------------------------
```javascript
#!/usr/bin/env node
// Simple script to check the version of deployed MCP server
import { readFileSync } from 'fs';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
console.log('🔍 SAP Docs MCP Version Check');
console.log('================================');
// Check local package.json version
try {
const packagePath = join(__dirname, '../package.json');
const packageInfo = JSON.parse(readFileSync(packagePath, 'utf8'));
console.log(`📦 Package Version: ${packageInfo.version}`);
} catch (error) {
console.log('❌ Could not read package.json');
}
// Try to check running servers
console.log('\n🌐 Checking Running Servers:');
const servers = [
{ name: 'HTTP Server', port: 3001, path: '/status' },
{ name: 'Streamable HTTP', port: 3122, path: '/health' }
];
for (const server of servers) {
try {
const response = await fetch(`http://127.0.0.1:${server.port}${server.path}`);
if (response.ok) {
const data = await response.json();
console.log(` ✅ ${server.name}: v${data.version || 'unknown'} (port ${server.port})`);
} else {
console.log(` ❌ ${server.name}: Server error ${response.status}`);
}
} catch (error) {
console.log(` ⚠️ ${server.name}: Not responding (port ${server.port})`);
}
}
console.log('\n✅ Version check complete');
```
--------------------------------------------------------------------------------
/test/_utils/httpClient.js:
--------------------------------------------------------------------------------
```javascript
// Simple HTTP client for testing MCP tools via /mcp endpoint
import { spawn } from 'node:child_process';
// ANSI color codes for logging
const colors = {
reset: '\x1b[0m',
dim: '\x1b[2m',
yellow: '\x1b[33m',
red: '\x1b[31m'
};
function colorize(text, color) {
return `${colors[color]}${text}${colors.reset}`;
}
const TEST_PORT = process.env.TEST_MCP_PORT || '43122';
const BASE_URL = `http://127.0.0.1:${TEST_PORT}`;
async function sleep(ms) {
return new Promise(r => setTimeout(r, ms));
}
export function startServerHttp() {
return spawn('node', ['dist/src/http-server.js'], {
env: { ...process.env, PORT: TEST_PORT },
stdio: 'ignore'
});
}
export async function waitForStatus(maxAttempts = 50, delayMs = 200) {
for (let i = 1; i <= maxAttempts; i++) {
try {
const res = await fetch(`${BASE_URL}/status`);
if (res.ok) return await res.json();
} catch (_) {}
await sleep(delayMs);
}
throw new Error(colorize('status endpoint did not become ready in time', 'red'));
}
export async function stopServer(child) {
try { child?.kill?.('SIGINT'); } catch (_) {}
await sleep(150);
}
export async function docsSearch(query) {
const res = await fetch(`${BASE_URL}/mcp`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ role: 'user', content: String(query) })
});
if (!res.ok) throw new Error(colorize(`http /mcp failed: ${res.status}`, 'red'));
const payload = await res.json();
return payload?.content || '';
}
```
--------------------------------------------------------------------------------
/test-search.ts:
--------------------------------------------------------------------------------
```typescript
#!/usr/bin/env tsx
// Simple search test script - can be run with: npx tsx test-search.ts [keyword]
import { searchLibraries } from './dist/src/lib/localDocs.js';
async function testSearch() {
// Get search keyword from command line arguments or use default
const keyword = process.argv[2] || 'wizard';
console.log(`🔍 Testing search for: "${keyword}"\n`);
try {
const startTime = Date.now();
const result = await searchLibraries(keyword);
const endTime = Date.now();
console.log(`⏱️ Search completed in ${endTime - startTime}ms\n`);
if (result.results.length > 0) {
console.log('✅ Search Results:');
console.log('='.repeat(50));
console.log(result.results[0].description);
console.log('='.repeat(50));
console.log(`\n📊 Summary:`);
console.log(`- Total snippets: ${result.results[0].totalSnippets || 0}`);
console.log(`- Result length: ${result.results[0].description.length} characters`);
} else {
console.log('❌ No results found');
if (result.error) {
console.log(`Error: ${result.error}`);
}
}
} catch (error) {
console.error('❌ Search failed:', error);
process.exit(1);
}
}
// Usage examples
if (process.argv.length === 2) {
console.log(`
🎯 SAP Docs Search Test
Usage: npx tsx test-search.ts [keyword]
Examples:
npx tsx test-search.ts wizard
npx tsx test-search.ts "cds entity"
npx tsx test-search.ts "wdi5 testing"
npx tsx test-search.ts button
npx tsx test-search.ts annotation
npx tsx test-search.ts service
Running with default keyword "wizard"...
`);
}
testSearch().catch(console.error);
```
--------------------------------------------------------------------------------
/test/tools/search.generic.spec.js:
--------------------------------------------------------------------------------
```javascript
// Generic semantic tests that use the harness server and docsSearch helper
import { parseSummaryText } from '../_utils/parseResults.js';
export default [
{
name: 'UI5 micro chart concept is discoverable',
tool: 'search',
query: 'UI.Chart #SpecificationWidthColumnChart',
validate: async ({ docsSearch }) => {
const summary = await docsSearch('UI.Chart #SpecificationWidthColumnChart');
const txt = String(summary).toLowerCase();
const ok = summary.includes('/sapui5/')
&& (txt.includes('chart') || txt.includes('micro'));
return { passed: ok, message: ok ? '' : 'No UI5 chart content in results' };
}
},
{
name: 'Cloud SDK AI getting started is discoverable',
tool: 'search',
query: 'getting started with sap cloud sdk for ai',
validate: async ({ docsSearch }) => {
const summary = await docsSearch('getting started with sap cloud sdk for ai');
const ok = summary.includes('cloud-sdk-ai')
&& /getting|start/i.test(summary);
return { passed: ok, message: ok ? '' : 'No Cloud SDK AI getting started content' };
}
},
{
name: 'CAP enums query finds enums sections',
tool: 'search',
query: 'Use enums cql cap',
validate: async ({ docsSearch }) => {
const summary = await docsSearch('Use enums cql cap');
const ok = summary.includes('/cap/') && /enum/i.test(summary);
return { passed: ok, message: ok ? '' : 'No CAP enums content' };
}
},
{
name: 'ExtensionAPI is discoverable in UI5',
tool: 'search',
query: 'extensionAPI',
validate: async ({ docsSearch }) => {
const summary = await docsSearch('extensionAPI');
const ok = summary.includes('/sapui5/') && /extension/i.test(summary);
return { passed: ok, message: ok ? '' : 'No ExtensionAPI content' };
}
}
];
```
--------------------------------------------------------------------------------
/src/lib/url-generation/dsag.ts:
--------------------------------------------------------------------------------
```typescript
/**
* DSAG ABAP Leitfaden URL Generator
* Handles GitHub Pages URLs for DSAG ABAP Guidelines
*/
import { BaseUrlGenerator, UrlGenerationContext } from './BaseUrlGenerator.js';
import { FrontmatterData } from './utils.js';
export interface DsagUrlOptions {
relFile: string;
content: string;
libraryId: string;
}
/**
* URL Generator for DSAG ABAP Leitfaden
*
* Transforms docs/path/file.md -> /path/file/
* Example: docs/clean-core/what-is-clean-core.md -> https://1dsag.github.io/ABAP-Leitfaden/clean-core/what-is-clean-core/
*/
export class DsagUrlGenerator extends BaseUrlGenerator {
protected generateSourceSpecificUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
// Transform the relative file path for GitHub Pages
// Remove docs/ prefix and .md extension, add trailing slash
let urlPath = context.relFile;
// Remove docs/ prefix if present
if (urlPath.startsWith('docs/')) {
urlPath = urlPath.substring(5);
}
// Remove .md extension
urlPath = urlPath.replace(/\.md$/, '');
// Build the final URL with trailing slash
let url = `${this.config.baseUrl}/${urlPath}/`;
// Add anchor if available
if (context.anchor) {
url += '#' + context.anchor;
}
return url;
}
}
/**
* Generate DSAG ABAP Leitfaden URL
* @param relFile - Relative file path (e.g., "docs/clean-core/what-is-clean-core.md")
* @param content - File content for extracting anchors
* @returns Generated GitHub Pages URL with proper path transformation
*/
export function generateDsagUrl(relFile: string, content: string): string {
const baseUrl = 'https://1dsag.github.io/ABAP-Leitfaden';
// Transform path: docs/clean-core/what-is-clean-core.md -> clean-core/what-is-clean-core/
let urlPath = relFile;
if (urlPath.startsWith('docs/')) {
urlPath = urlPath.substring(5);
}
urlPath = urlPath.replace(/\.md$/, '');
return `${baseUrl}/${urlPath}/`;
}
```
--------------------------------------------------------------------------------
/ecosystem.config.cjs:
--------------------------------------------------------------------------------
```
// PM2 configuration for SAP Docs MCP server
// Modern MCP streamable HTTP transport only (SSE proxy removed)
module.exports = {
apps: [
// HTTP status server on :3001 (pinned port for PM2)
{
name: "mcp-sap-http",
script: "node",
args: ["/opt/mcp-sap/mcp-sap-docs/dist/src/http-server.js"],
cwd: "/opt/mcp-sap/mcp-sap-docs",
env: {
NODE_ENV: "production",
PORT: "3001",
LOG_LEVEL: "DEBUG", // Enhanced for debugging
LOG_FORMAT: "json",
// BM25-only search configuration
RETURN_K: "30" // Centralized result limit (can override CONFIG.RETURN_K)
},
autorestart: true,
max_restarts: 10,
restart_delay: 2000,
// Enhanced logging configuration
log_file: "/opt/mcp-sap/logs/mcp-http-combined.log",
out_file: "/opt/mcp-sap/logs/mcp-http-out.log",
error_file: "/opt/mcp-sap/logs/mcp-http-error.log",
log_type: "json",
log_date_format: "YYYY-MM-DD HH:mm:ss Z",
// Log rotation
max_size: "10M",
retain: 10,
compress: true
},
// Streamable HTTP MCP server (latest MCP spec)
{
name: "mcp-sap-streamable",
script: "node",
args: ["/opt/mcp-sap/mcp-sap-docs/dist/src/streamable-http-server.js"],
cwd: "/opt/mcp-sap/mcp-sap-docs",
env: {
NODE_ENV: "production",
MCP_PORT: "3122",
LOG_LEVEL: "DEBUG", // Enhanced for debugging
LOG_FORMAT: "json",
// BM25-only search configuration
RETURN_K: "30" // Centralized result limit (can override CONFIG.RETURN_K)
},
autorestart: true,
max_restarts: 10,
restart_delay: 2000,
// Enhanced logging configuration
log_file: "/opt/mcp-sap/logs/mcp-streamable-combined.log",
out_file: "/opt/mcp-sap/logs/mcp-streamable-out.log",
error_file: "/opt/mcp-sap/logs/mcp-streamable-error.log",
log_type: "json",
log_date_format: "YYYY-MM-DD HH:mm:ss Z",
// Log rotation
max_size: "10M",
retain: 10,
compress: true
}
]
}
```
--------------------------------------------------------------------------------
/package.json:
--------------------------------------------------------------------------------
```json
{
"name": "mcp-sap-docs",
"version": "0.3.19",
"type": "module",
"main": "dist/server.js",
"scripts": {
"build:tsc": "tsc",
"build:index": "npm run build:tsc && node dist/scripts/build-index.js",
"build:fts": "npm run build:tsc && node dist/scripts/build-fts.js",
"build": "npm run build:tsc && npm run build:index && npm run build:fts",
"start": "node dist/src/server.js",
"start:http": "node dist/src/http-server.js",
"start:streamable": "node dist/src/streamable-http-server.js",
"inspect": "npx @modelcontextprotocol/inspector",
"test": "npm run test:url-generation && npm run test:integration",
"test:url-generation": "npm run build:tsc && npx vitest run test/comprehensive-url-generation.test.ts test/prompts.test.ts",
"test:url-generation:debug": "npm run build:tsc && DEBUG_TESTS=true npx vitest run test/comprehensive-url-generation.test.ts",
"test:mcp-urls": "npm run build:tsc && npx vitest run test/mcp-search-url-verification.test.ts",
"test:integration": "npm run build && node test/tools/run-tests.js",
"test:integration:urls": "npm run build && node test/tools/run-tests.js --spec search-url-verification.js",
"test:smoke": "node test/tools/search.smoke.js",
"test:community": "npm run build:tsc && node test/community-search.ts",
"test:urls:status": "npm run build:tsc && npx tsx test/url-status.ts",
"check-version": "node scripts/check-version.js",
"setup": "bash setup.sh",
"setup:submodules": "bash -lc 'git submodule sync --recursive && git submodule update --init --recursive --depth 1 && git submodule status --recursive'"
},
"ts-node": {
"esm": true
},
"dependencies": {
"@modelcontextprotocol/sdk": "^1.19.0",
"better-sqlite3": "^12.2.0",
"cors": "^2.8.5",
"express": "^4.21.2",
"fast-glob": "^3.3.2",
"gray-matter": "^4.0.3",
"zod": "^3.23.8"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.13",
"@types/cors": "^2.8.19",
"@types/express": "^4.17.23",
"@types/node": "^20.11.19",
"ts-node": "^10.9.2",
"typescript": "^5.4.4",
"vitest": "^2.1.5"
}
}
```
--------------------------------------------------------------------------------
/.github/workflows/test-pr.yml:
--------------------------------------------------------------------------------
```yaml
name: Test PR
on:
pull_request:
branches: [ main ]
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: recursive
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Setup submodules and build with database verification
run: |
export SKIP_NESTED_SUBMODULES=1
bash setup.sh
# Verify database was built correctly and is not corrupted
DB_PATH="./dist/data/docs.sqlite"
if [ -f "$DB_PATH" ]; then
echo "✅ Database file created: $DB_PATH"
DB_SIZE=$(du -h "$DB_PATH" | cut -f1)
echo "📏 Database size: $DB_SIZE"
# Test database integrity
if sqlite3 "$DB_PATH" "PRAGMA integrity_check;" | grep -q "ok"; then
echo "✅ Database integrity check passed"
else
echo "❌ Database integrity check failed"
exit 1
fi
# Test basic FTS functionality
if sqlite3 "$DB_PATH" "SELECT COUNT(*) FROM docs;" | grep -qE '^[1-9][0-9]*$'; then
echo "✅ Database contains indexed documents"
DOCUMENT_COUNT=$(sqlite3 "$DB_PATH" "SELECT COUNT(*) FROM docs;")
echo "📄 Document count: $DOCUMENT_COUNT"
else
echo "❌ Database appears empty or malformed"
exit 1
fi
else
echo "❌ Database file not found: $DB_PATH"
exit 1
fi
- name: Run tests with database health check
run: |
# Run the standard tests
npm run test
# Additional database health verification after tests
echo "==> Post-test database integrity check"
DB_PATH="./dist/data/docs.sqlite"
if sqlite3 "$DB_PATH" "PRAGMA integrity_check;" | grep -q "ok"; then
echo "✅ Database integrity maintained after tests"
else
echo "❌ Database corruption detected after tests"
exit 1
fi
```
--------------------------------------------------------------------------------
/src/server.ts:
--------------------------------------------------------------------------------
```typescript
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { logger } from "./lib/logger.js";
import { BaseServerHandler } from "./lib/BaseServerHandler.js";
function createServer() {
const serverOptions: NonNullable<ConstructorParameters<typeof Server>[1]> & {
protocolVersions?: string[];
} = {
protocolVersions: ["2025-07-09"],
capabilities: {
// resources: {}, // DISABLED: Causes 60,000+ resources which breaks Cursor
tools: {}, // Enable tools capability
prompts: {} // Enable prompts capability for 2025-07-09 protocol
}
};
const srv = new Server({
name: "Local SAP Docs",
description:
"Offline SAPUI5 & CAP documentation server with SAP Community, SAP Help Portal, and ABAP Keyword Documentation integration",
version: "0.1.0"
}, serverOptions);
// Configure server with shared handlers
BaseServerHandler.configureServer(srv);
return srv;
}
async function main() {
// Initialize search system with metadata
BaseServerHandler.initializeMetadata();
const srv = createServer();
// Log server startup
logger.info("MCP SAP Docs server starting up", {
nodeEnv: process.env.NODE_ENV,
logLevel: process.env.LOG_LEVEL,
logFormat: process.env.LOG_FORMAT
});
await srv.connect(new StdioServerTransport());
console.error("📚 MCP server ready (stdio) with Tools and Prompts support.");
// Log successful startup
logger.info("MCP SAP Docs server ready and connected", {
transport: "stdio",
pid: process.pid
});
// Set up performance monitoring (every 10 minutes for stdio servers)
const performanceInterval = setInterval(() => {
logger.logPerformanceMetrics();
}, 10 * 60 * 1000);
// Handle server shutdown
process.on('SIGINT', () => {
logger.info('Shutdown signal received, closing stdio server gracefully');
clearInterval(performanceInterval);
logger.info('Stdio server shutdown complete');
process.exit(0);
});
// Log the port if we're running in HTTP mode (for debugging)
if (process.env.PORT) {
console.error(`📚 MCP server configured for port: ${process.env.PORT}`);
}
}
main().catch((e) => {
console.error("Fatal:", e);
process.exit(1);
});
```
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/new-documentation-source.yml:
--------------------------------------------------------------------------------
```yaml
name: New Documentation Source Request
description: Suggest a new SAP-related documentation source to be indexed by the MCP server
title: "[NEW SOURCE]: "
assignees: []
body:
- type: markdown
attributes:
value: |
Thanks for suggesting a new documentation source! This helps expand our search coverage.
**Prerequisites:**
- The source must be SAP-related documentation
- **MANDATORY**: The source must be available on GitHub
- The documentation should be publicly accessible
- The content should be searchable and indexable
- type: input
id: github-repository
attributes:
label: GitHub Repository URL
description: "**REQUIRED**: Provide the full GitHub repository URL"
placeholder: "https://github.com/SAP/some-documentation-repo"
validations:
required: true
- type: textarea
id: justification
attributes:
label: Why should this source be added?
description: Explain the value this documentation would provide to SAP developers
placeholder: |
This documentation covers important SAP functionality that developers frequently need:
- Key use cases it addresses
- How it complements existing sources
- Developer pain points it would solve
validations:
required: true
- type: input
id: source-name
attributes:
label: Source Display Name
description: What should this source be called in search results?
placeholder: "SAP Business Application Studio"
validations:
required: true
- type: input
id: source-id
attributes:
label: Source ID (suggested)
description: Unique identifier for this source (lowercase, kebab-case, starts with /)
placeholder: "/business-application-studio"
validations:
required: false
- type: textarea
id: source-description
attributes:
label: Source Description
description: Brief description of what this documentation covers
placeholder: "Official documentation for SAP Business Application Studio development tools and features"
validations:
required: true
- type: input
id: documentation-path
attributes:
label: Documentation Directory Path
description: Path within the repository where documentation files are located
placeholder: "docs/ or documentation/ or README.md"
validations:
required: true
- type: textarea
id: additional-info
attributes:
label: Additional Information
description: Any other relevant details about this documentation source
validations:
required: false
```
--------------------------------------------------------------------------------
/src/lib/url-generation/cloud-sdk.ts:
--------------------------------------------------------------------------------
```typescript
/**
* URL generation for SAP Cloud SDK documentation sources
* Handles JavaScript, Java, and AI SDK variants
*/
import { BaseUrlGenerator, UrlGenerationContext } from './BaseUrlGenerator.js';
import { FrontmatterData } from './utils.js';
import { DocUrlConfig } from '../metadata.js';
export interface CloudSdkUrlOptions {
relFile: string;
content: string;
config: DocUrlConfig;
libraryId: string;
}
/**
* Cloud SDK URL Generator
* Handles JavaScript, Java, and AI SDK variants with specialized URL generation
*/
export class CloudSdkUrlGenerator extends BaseUrlGenerator {
protected generateSourceSpecificUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
const identifier = this.getIdentifierFromFrontmatter(context.frontmatter);
// Use frontmatter ID if available (preferred method)
if (identifier) {
// Special handling for AI SDK variants
if (this.isAiSdk()) {
return this.buildAiSdkUrl(context.relFile, identifier);
} else {
return this.buildUrl(this.config.baseUrl, context.section, identifier);
}
}
return null;
}
/**
* Check if this is an AI SDK variant
*/
private isAiSdk(): boolean {
return this.libraryId.includes('-ai-');
}
/**
* Build AI SDK specific URL with proper section handling
*/
private buildAiSdkUrl(relFile: string, identifier: string): string {
// Extract section from the file path for AI SDK
if (this.isInDirectory(relFile, 'langchain')) {
return this.buildUrl(this.config.baseUrl, 'langchain', identifier);
} else if (this.isInDirectory(relFile, 'getting-started')) {
return this.buildUrl(this.config.baseUrl, 'getting-started', identifier);
} else if (this.isInDirectory(relFile, 'examples')) {
return this.buildUrl(this.config.baseUrl, 'examples', identifier);
}
// Default behavior for other sections
const section = this.extractSection(relFile);
return this.buildUrl(this.config.baseUrl, section, identifier);
}
/**
* Override section extraction for Cloud SDK specific patterns
*/
protected extractSection(relFile: string): string {
// Check for Cloud SDK specific patterns first
if (this.isInDirectory(relFile, 'environments')) {
return '/environments/';
} else if (this.isInDirectory(relFile, 'getting-started')) {
return '/getting-started/';
}
// Use base implementation for common patterns
return super.extractSection(relFile);
}
}
// Convenience functions for backward compatibility and external use
/**
* Generate URL for Cloud SDK documentation using the class-based approach
*/
export function generateCloudSdkUrl(options: CloudSdkUrlOptions): string | null {
const generator = new CloudSdkUrlGenerator(options.libraryId, options.config);
return generator.generateUrl(options);
}
/**
* Generate AI SDK URL (now handled by the main generator)
*/
export function generateCloudSdkAiUrl(options: CloudSdkUrlOptions): string | null {
return generateCloudSdkUrl(options);
}
/**
* Main URL generator dispatcher for all Cloud SDK variants
*/
export function generateCloudSdkUrlForLibrary(options: CloudSdkUrlOptions): string | null {
return generateCloudSdkUrl(options);
}
```
--------------------------------------------------------------------------------
/test/prompts.test.ts:
--------------------------------------------------------------------------------
```typescript
import { describe, expect, it } from 'vitest';
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import {
GetPromptRequestSchema,
GetPromptResultSchema,
ListPromptsRequestSchema,
ListPromptsResultSchema
} from '@modelcontextprotocol/sdk/types.js';
import { BaseServerHandler } from '../src/lib/BaseServerHandler.js';
type RequestHandler = (request: unknown, extra?: unknown) => unknown;
function createTestServer() {
const server = new Server({
name: 'Test Server',
version: '1.0.0'
}, {
capabilities: {
tools: {},
prompts: {}
}
});
BaseServerHandler.configureServer(server);
return server;
}
function getHandler(server: Server, method: string): RequestHandler {
const handlers: Map<string, RequestHandler> = (server as unknown as { _requestHandlers: Map<string, RequestHandler> })._requestHandlers;
const handler = handlers.get(method);
if (!handler) {
throw new Error(`Handler not registered for method: ${method}`);
}
return handler;
}
describe('Prompt handlers', () => {
it('lists prompts with schema-compliant metadata', async () => {
const server = createTestServer();
const handler = getHandler(server, 'prompts/list');
const request = ListPromptsRequestSchema.parse({ method: 'prompts/list' });
const result = await handler(request);
const parsed = ListPromptsResultSchema.safeParse(result);
expect(parsed.success).toBe(true);
if (!parsed.success) return;
const prompts = parsed.data.prompts;
expect(prompts.length).toBeGreaterThan(0);
const searchPrompt = prompts.find(prompt => prompt.name === 'sap_search_help');
expect(searchPrompt).toBeDefined();
expect(searchPrompt?.title).toBe('SAP Documentation Search Helper');
expect(searchPrompt?.arguments?.some(arg => arg.name === 'domain')).toBe(true);
});
it('returns templated prompt content for sap_search_help', async () => {
const server = createTestServer();
const handler = getHandler(server, 'prompts/get');
const request = GetPromptRequestSchema.parse({
method: 'prompts/get',
params: {
name: 'sap_search_help',
arguments: {
domain: 'SAPUI5',
context: 'routing features'
}
}
});
const result = await handler(request);
const parsed = GetPromptResultSchema.parse(result);
expect(parsed.messages.length).toBeGreaterThan(0);
const [message] = parsed.messages;
expect(message.role).toBe('user');
expect(message.content.type).toBe('text');
expect(message.content.text).toContain('SAPUI5');
expect(message.content.text).toContain('routing features');
});
it('returns default guidance when optional arguments omitted', async () => {
const server = createTestServer();
const handler = getHandler(server, 'prompts/get');
const request = GetPromptRequestSchema.parse({
method: 'prompts/get',
params: {
name: 'sap_troubleshoot'
}
});
const result = await handler(request);
const parsed = GetPromptResultSchema.parse(result);
expect(parsed.description).toBe('Troubleshooting guide for SAP');
const [message] = parsed.messages;
expect(message.role).toBe('user');
expect(message.content.type).toBe('text');
expect(message.content.text).toContain("I'm experiencing an issue with SAP");
});
});
```
--------------------------------------------------------------------------------
/src/lib/url-generation/wdi5.ts:
--------------------------------------------------------------------------------
```typescript
/**
* URL generation for wdi5 testing framework documentation
* Handles testing guides, API docs, and examples
*/
import { BaseUrlGenerator, UrlGenerationContext } from './BaseUrlGenerator.js';
import { FrontmatterData } from './utils.js';
import { DocUrlConfig } from '../metadata.js';
export interface Wdi5UrlOptions {
relFile: string;
content: string;
config: DocUrlConfig;
libraryId: string;
}
/**
* wdi5 URL Generator
* Handles wdi5 testing framework documentation with docsify-style URLs
*/
export class Wdi5UrlGenerator extends BaseUrlGenerator {
protected generateSourceSpecificUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
const identifier = this.getIdentifierFromFrontmatter(context.frontmatter);
// Use frontmatter id for docsify-style URLs
if (identifier) {
const section = this.extractWdi5Section(context.relFile);
if (section) {
return this.buildDocsifyUrl(`${section}/${identifier}`);
}
return this.buildDocsifyUrl(identifier);
}
// Fallback to filename-based URL
const section = this.extractWdi5Section(context.relFile);
const fileName = this.getCleanFileName(context.relFile);
if (section) {
return this.buildDocsifyUrl(`${section}/${fileName}`);
}
// Simple filename-based URL with docsify fragment
const cleanFileName = fileName.replace(/\//g, '-').toLowerCase();
return this.buildDocsifyUrl(cleanFileName);
}
/**
* Extract wdi5-specific sections from file path
*/
private extractWdi5Section(relFile: string): string {
if (this.isInDirectory(relFile, 'configuration')) {
return 'configuration';
} else if (this.isInDirectory(relFile, 'usage')) {
return 'usage';
} else if (this.isInDirectory(relFile, 'selectors')) {
return 'selectors';
} else if (this.isInDirectory(relFile, 'locators')) {
return 'locators'; // Handle locators separately from selectors
} else if (this.isInDirectory(relFile, 'authentication')) {
return 'authentication';
} else if (this.isInDirectory(relFile, 'plugins')) {
return 'plugins';
} else if (this.isInDirectory(relFile, 'examples')) {
return 'examples';
} else if (this.isInDirectory(relFile, 'migration')) {
return 'migration';
} else if (this.isInDirectory(relFile, 'troubleshooting')) {
return 'troubleshooting';
}
return '';
}
/**
* Override to use wdi5-specific section extraction
*/
protected extractSection(relFile: string): string {
return this.extractWdi5Section(relFile);
}
}
// Convenience functions for backward compatibility
/**
* Generate URL for wdi5 documentation using the class-based approach
*/
export function generateWdi5Url(options: Wdi5UrlOptions): string | null {
const generator = new Wdi5UrlGenerator(options.libraryId, options.config);
return generator.generateUrl(options);
}
/**
* Generate URL for wdi5 configuration documentation
*/
export function generateWdi5ConfigUrl(options: Wdi5UrlOptions): string | null {
return generateWdi5Url(options); // Now handled by the main generator
}
/**
* Generate URL for wdi5 selector and locator documentation
*/
export function generateWdi5SelectorUrl(options: Wdi5UrlOptions): string | null {
return generateWdi5Url(options); // Now handled by the main generator
}
```
--------------------------------------------------------------------------------
/src/lib/url-generation/cap.ts:
--------------------------------------------------------------------------------
```typescript
/**
* URL generation for SAP CAP (Cloud Application Programming) documentation
* Handles CDS guides, reference docs, and tutorials
*/
import { BaseUrlGenerator, UrlGenerationContext } from './BaseUrlGenerator.js';
import { FrontmatterData } from './utils.js';
import { DocUrlConfig } from '../metadata.js';
export interface CapUrlOptions {
relFile: string;
content: string;
config: DocUrlConfig;
libraryId: string;
}
/**
* CAP URL Generator
* Handles CDS guides, reference docs, and tutorials with docsify-style URLs
*/
export class CapUrlGenerator extends BaseUrlGenerator {
protected generateSourceSpecificUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
const identifier = this.getIdentifierFromFrontmatter(context.frontmatter);
// Use frontmatter slug or id for URL generation
if (identifier) {
const section = this.extractCapSection(context.relFile);
if (section) {
return this.buildDocsifyUrl(`${section}/${identifier}`);
}
return this.buildDocsifyUrl(identifier);
}
// Fallback to filename-based URL
const fileName = this.getCleanFileName(context.relFile);
const section = this.extractCapSection(context.relFile);
if (section) {
return this.buildDocsifyUrl(`${section}/${fileName}`);
}
return this.buildDocsifyUrl(fileName);
}
/**
* Extract CAP-specific sections from file path
*/
private extractCapSection(relFile: string): string {
if (this.isInDirectory(relFile, 'guides')) {
return 'guides';
} else if (this.isInDirectory(relFile, 'cds')) {
return 'cds';
} else if (this.isInDirectory(relFile, 'node.js')) {
return 'node.js';
} else if (this.isInDirectory(relFile, 'java')) {
return 'java';
} else if (this.isInDirectory(relFile, 'plugins')) {
return 'plugins';
} else if (this.isInDirectory(relFile, 'advanced')) {
return 'advanced';
} else if (this.isInDirectory(relFile, 'get-started')) {
return 'get-started';
} else if (this.isInDirectory(relFile, 'tutorials')) {
return 'tutorials';
}
return '';
}
/**
* Override to use CAP-specific section extraction
*/
protected extractSection(relFile: string): string {
return this.extractCapSection(relFile);
}
/**
* Override to use CAP-specific docsify URL building
* CAP URLs have a /docs/ prefix before the # fragment
*/
protected buildDocsifyUrl(path: string): string {
const cleanPath = path.startsWith('/') ? path.slice(1) : path;
return `${this.config.baseUrl}/docs/#/${cleanPath}`;
}
}
// Convenience functions for backward compatibility
/**
* Generate URL for CAP documentation using the class-based approach
*/
export function generateCapUrl(options: CapUrlOptions): string | null {
const generator = new CapUrlGenerator(options.libraryId, options.config);
return generator.generateUrl(options);
}
/**
* Generate URL for CAP CDS reference documentation
*/
export function generateCapCdsUrl(options: CapUrlOptions): string | null {
return generateCapUrl(options); // Now handled by the main generator
}
/**
* Generate URL for CAP tutorials and getting started guides
*/
export function generateCapTutorialUrl(options: CapUrlOptions): string | null {
return generateCapUrl(options); // Now handled by the main generator
}
```
--------------------------------------------------------------------------------
/src/lib/url-generation/abap.ts:
--------------------------------------------------------------------------------
```typescript
/**
* URL Generator for ABAP Keyword Documentation
* Maps individual .md files to official SAP ABAP documentation URLs
*/
import { BaseUrlGenerator } from './BaseUrlGenerator.js';
import { DocUrlConfig } from '../metadata.js';
/**
* ABAP URL Generator for official SAP documentation
* Converts .md filenames to proper help.sap.com URLs
*/
export class AbapUrlGenerator extends BaseUrlGenerator {
generateSourceSpecificUrl(context: any): string | null {
// Extract filename without extension
let filename = context.relFile.replace(/\.md$/, '');
// Remove 'md/' prefix if present (from sources/abap-docs/docs/7.58/md/)
filename = filename.replace(/^md\//, '');
// Convert .md filename back to .html for SAP documentation
const htmlFile = filename + '.html';
// Get version from config or default to latest (which now points to cloud version)
const version = this.extractVersion() || 'latest';
// Build SAP help URL
const baseUrl = this.getAbapBaseUrl(version);
const fullUrl = `${baseUrl}/${htmlFile}`;
// Add anchor if provided
return context.anchor ? `${fullUrl}#${context.anchor}` : fullUrl;
}
/**
* Extract ABAP version from config
*/
private extractVersion(): string | null {
// Check if version is in the library ID or path pattern
const pathPattern = this.config.pathPattern || '';
const libraryId = this.libraryId || '';
// Handle "latest" version explicitly
if (libraryId.includes('latest') || pathPattern.includes('latest')) {
return 'latest';
}
// Try to extract version patterns: 7.58, 9.16, 8.10, etc.
const versionMatch = (libraryId + pathPattern).match(/\/(\d+\.\d+)\//);
if (versionMatch) {
return versionMatch[1];
}
// Try alternative patterns for cloud/new versions
const cloudMatch = (libraryId + pathPattern).match(/-(latest|cloud|916|916\w*|81\w*)-/);
if (cloudMatch) {
const match = cloudMatch[1];
if (match === 'latest' || match === 'cloud') return 'latest';
if (match.startsWith('916')) return '9.16';
if (match.startsWith('81')) return '8.10';
}
return null;
}
/**
* Get base URL for ABAP documentation based on version
*/
private getAbapBaseUrl(version: string): string {
// Handle latest version - use the newest cloud version
if (version === 'latest') {
return 'https://help.sap.com/doc/abapdocu_cp_index_htm/CLOUD/en-US';
}
const versionNum = parseFloat(version);
// Cloud versions (9.1x) - ABAP Cloud / SAP BTP
if (versionNum >= 9.1) {
return 'https://help.sap.com/doc/abapdocu_cp_index_htm/CLOUD/en-US';
}
// S/4HANA 2025 versions (8.1x)
if (versionNum >= 8.1) {
// Use the cloud pattern for S/4HANA 2025 as well, since they share the same doc structure
return 'https://help.sap.com/doc/abapdocu_cp_index_htm/CLOUD/en-US';
}
// Legacy versions (7.x) - keep existing pattern
const versionCode = version.replace('.', '');
return `https://help.sap.com/doc/abapdocu_${versionCode}_index_htm/${version}/en-US`;
}
}
/**
* Generate ABAP documentation URL
*/
export function generateAbapUrl(libraryId: string, relativeFile: string, config: DocUrlConfig, anchor?: string): string | null {
const generator = new AbapUrlGenerator(libraryId, config);
return generator.generateSourceSpecificUrl({
relFile: relativeFile,
content: '',
config,
libraryId,
anchor
});
}
```
--------------------------------------------------------------------------------
/setup.sh:
--------------------------------------------------------------------------------
```bash
#!/bin/bash
# SAP Documentation MCP Server Setup Script
echo "🚀 Setting up SAP Documentation MCP Server..."
# Install dependencies
echo "📦 Installing dependencies..."
npm install
# Initialize and update git submodules
echo "📚 Initializing documentation submodules..."
# Initialize/update submodules (including new ones) to latest
echo " → Syncing submodule configuration..."
git submodule sync --recursive
# Collect submodules from .gitmodules
echo " → Ensuring all top-level submodules are present (shallow, single branch)..."
while IFS= read -r line; do
name=$(echo "$line" | awk '{print $1}' | sed -E 's/^submodule\.([^ ]*)\.path$/\1/')
path=$(echo "$line" | awk '{print $2}')
branch=$(git config -f .gitmodules "submodule.${name}.branch" || echo main)
url=$(git config -f .gitmodules "submodule.${name}.url")
# Skip if missing required fields
[ -z "$path" ] && continue
[ -z "$url" ] && continue
echo " • $path (branch: $branch)"
if [ ! -d "$path/.git" ]; then
echo " - cloning shallow..."
GIT_LFS_SKIP_SMUDGE=1 git clone --filter=blob:none --no-tags --single-branch --depth 1 --branch "$branch" "$url" "$path" || {
echo " ! clone failed for $path, retrying with master"
GIT_LFS_SKIP_SMUDGE=1 git clone --filter=blob:none --no-tags --single-branch --depth 1 --branch master "$url" "$path" || true
}
else
echo " - updating shallow to latest $branch..."
# Limit origin to a single branch and fetch shallow
git -C "$path" config --unset-all remote.origin.fetch >/dev/null 2>&1 || true
git -C "$path" config remote.origin.fetch "+refs/heads/${branch}:refs/remotes/origin/${branch}"
git -C "$path" remote set-branches origin "$branch" || true
# configure partial clone + no-tags for smaller fetches
git -C "$path" config remote.origin.tagOpt --no-tags || true
git -C "$path" config remote.origin.promisor true || true
git -C "$path" config remote.origin.partialclonefilter blob:none || true
if ! GIT_LFS_SKIP_SMUDGE=1 git -C "$path" fetch --filter=blob:none --no-tags --depth 1 --prune origin "$branch"; then
echo " ! fetch failed for $branch, trying master"
git -C "$path" config remote.origin.fetch "+refs/heads/master:refs/remotes/origin/master"
git -C "$path" remote set-branches origin master || true
GIT_LFS_SKIP_SMUDGE=1 git -C "$path" fetch --filter=blob:none --no-tags --depth 1 --prune origin master || true
branch=master
fi
# Checkout/reset to the fetched tip
git -C "$path" checkout -B "$branch" "origin/$branch" 2>/dev/null || git -C "$path" checkout "$branch" || true
git -C "$path" reset --hard "origin/$branch" 2>/dev/null || true
# Compact local repository storage (keeps only the shallow pack)
git -C "$path" reflog expire --expire=now --all >/dev/null 2>&1 || true
git -C "$path" gc --prune=now --aggressive >/dev/null 2>&1 || true
fi
done < <(git config -f .gitmodules --get-regexp 'submodule\..*\.path')
if [ -n "$SKIP_NESTED_SUBMODULES" ]; then
echo " → Skipping nested submodule initialization (SKIP_NESTED_SUBMODULES=1)"
else
echo " → Initializing nested submodules to pinned commits (shallow)..."
git submodule update --init --recursive --depth 1 || true
fi
echo " → Current submodule status:"
git submodule status --recursive || true
# Build the search index
echo "🔍 Building search index..."
npm run build
echo "✅ Setup complete!"
echo ""
echo "To start the MCP server:"
echo " npm start"
echo ""
echo "To use in Cursor:"
echo "1. Open Cursor IDE"
echo "2. Go to Tools → Add MCP Server"
echo "3. Use command: npm start"
echo "4. Set working directory to: $(pwd)"
```
--------------------------------------------------------------------------------
/src/lib/search.ts:
--------------------------------------------------------------------------------
```typescript
// Simple BM25-only search using FTS5 with metadata-driven configuration
import { searchFTS } from "./searchDb.js";
import { CONFIG } from "./config.js";
import { loadMetadata, getSourceBoosts, expandQueryTerms, getAllLibraryMappings } from "./metadata.js";
export type SearchResult = {
id: string;
text: string;
bm25: number;
sourceId: string;
path: string;
finalScore: number;
};
// Helper to extract source ID from library_id or document path using metadata
function extractSourceId(libraryIdOrPath: string): string {
if (libraryIdOrPath.startsWith('/')) {
const parts = libraryIdOrPath.split('/');
if (parts.length > 1) {
const sourceId = parts[1];
// Use metadata-driven library mappings
const mappings = getAllLibraryMappings();
return mappings[sourceId] || sourceId;
}
}
return libraryIdOrPath;
}
export async function search(
query: string,
{ k = CONFIG.RETURN_K } = {}
): Promise<SearchResult[]> {
// Load metadata for boosts and query expansion
loadMetadata();
const sourceBoosts = getSourceBoosts();
// Expand query with synonyms and acronyms
const queryVariants = expandQueryTerms(query);
const seen = new Map<string, any>();
// Check if query contains specific ABAP version
const versionMatch = query.match(/\b(7\.\d{2}|latest)\b/i);
const requestedVersion = versionMatch ? versionMatch[1].toLowerCase() : null;
const requestedVersionId = requestedVersion ? requestedVersion.replace('.', '') : null;
// Search with all query variants (union approach)
for (const variant of queryVariants) {
try {
const rows = searchFTS(variant, {}, k);
for (const r of rows) {
if (!seen.has(r.id)) {
seen.set(r.id, r);
}
}
} catch (error) {
console.warn(`FTS query failed for variant "${variant}":`, error);
continue;
}
if (seen.size >= k) break; // enough candidates
}
let rows = Array.from(seen.values()).slice(0, k);
// Smart ABAP version filtering - only show latest unless version specified
if (!requestedVersion) {
// For general ABAP queries without version, aggressively filter out older versions
rows = rows.filter(r => {
const id = r.id || '';
// Keep all non-ABAP-docs sources
if (!id.includes('/abap-docs-')) return true;
// For ABAP docs, ONLY keep latest version for general queries
return id.includes('/abap-docs-latest/');
});
console.log(`Filtered to latest ABAP version only: ${rows.length} results`);
} else {
// For version-specific queries, ONLY show the requested version and non-ABAP sources
rows = rows.filter(r => {
const id = r.id || '';
// Keep all non-ABAP-docs sources (style guides, cheat sheets, etc.)
if (!id.includes('/abap-docs-')) return true;
// For ABAP docs, ONLY keep the specifically requested version
return id.includes(`/abap-docs-${requestedVersionId}/`);
});
console.log(`Filtered to ABAP version ${requestedVersion} only: ${rows.length} results`);
}
// Convert to consistent format with source boosts
const results = rows.map(r => {
const sourceId = extractSourceId(r.libraryId || r.id);
let boost = sourceBoosts[sourceId] || 0;
// Additional boost for version-specific queries
if (requestedVersionId && r.id.includes(`/abap-docs-${requestedVersionId}/`)) {
boost += 1.0; // Extra boost for requested version
}
return {
id: r.id,
text: `${r.title || ""}\n\n${r.description || ""}\n\n${r.id}`,
bm25: r.bm25Score,
sourceId,
path: r.id,
finalScore: (-r.bm25Score) * (1 + boost) // Convert to descending with boost
};
});
// Results are already filtered above, just sort them
// Sort by final score (higher = better)
return results.sort((a, b) => b.finalScore - a.finalScore);
}
```
--------------------------------------------------------------------------------
/scripts/build-fts.ts:
--------------------------------------------------------------------------------
```typescript
// Build pipeline step 2: Compiles dist/data/index.json into dist/data/docs.sqlite/FTS5 for fast search
import fs from "fs";
import path from "path";
import Database from "better-sqlite3";
type Doc = {
id: string;
relFile: string;
title: string;
description: string;
snippetCount: number;
type?: string;
controlName?: string;
namespace?: string;
keywords?: string[];
properties?: string[];
events?: string[];
aggregations?: string[];
};
type LibraryBundle = {
id: string; // "/sapui5" | "/cap" | "/openui5-api" | "/openui5-samples" | "/wdi5"
name: string;
description: string;
docs: Doc[];
};
const DATA_DIR = path.join(process.cwd(), "dist", "data");
const SRC = path.join(DATA_DIR, "index.json");
const DST = path.join(DATA_DIR, "docs.sqlite");
function libFromId(id: string): string {
// id looks like "/sapui5/..." etc.
const m = id.match(/^\/[^/]+/);
return m ? m[0] : "";
}
function safeText(x: unknown): string {
if (!x) return "";
if (Array.isArray(x)) return x.join(" ");
return String(x);
}
function main() {
if (!fs.existsSync(SRC)) {
throw new Error(`Missing ${SRC}. Run npm run build:index first.`);
}
console.log(`📖 Reading index from ${SRC}...`);
const raw = JSON.parse(fs.readFileSync(SRC, "utf8")) as Record<string, LibraryBundle>;
// Fresh DB
if (fs.existsSync(DST)) {
console.log(`🗑️ Removing existing ${DST}...`);
fs.unlinkSync(DST);
}
console.log(`🏗️ Creating FTS5 database at ${DST}...`);
const db = new Database(DST);
db.pragma("journal_mode = WAL");
db.pragma("synchronous = NORMAL");
db.pragma("temp_store = MEMORY");
// FTS5 schema: columns you want to search get indexed; metadata can be UNINDEXED
// We keep this simple - FTS is just for fast candidate filtering
db.exec(`
CREATE VIRTUAL TABLE docs USING fts5(
libraryId, -- indexed for filtering
type, -- markdown/jsdoc/sample (indexed for filtering)
title, -- strong signal for search
description, -- weaker signal for search
keywords, -- control tags and properties
controlName, -- e.g., Wizard, Button
namespace, -- e.g., sap.m, sap.f
id UNINDEXED, -- metadata (full path id)
relFile UNINDEXED, -- metadata
snippetCount UNINDEXED -- metadata
);
`);
console.log(`📝 Inserting documents into FTS5 index...`);
const ins = db.prepare(`
INSERT INTO docs (libraryId,type,title,description,keywords,controlName,namespace,id,relFile,snippetCount)
VALUES (?,?,?,?,?,?,?,?,?,?)
`);
let totalDocs = 0;
const tx = db.transaction(() => {
for (const lib of Object.values(raw)) {
for (const d of lib.docs) {
const libraryId = libFromId(d.id);
const keywords = safeText(d.keywords);
const props = safeText(d.properties);
const events = safeText(d.events);
const aggs = safeText(d.aggregations);
// Combine all searchable keywords
const keywordsAll = [keywords, props, events, aggs].filter(Boolean).join(" ");
ins.run(
libraryId,
d.type ?? "",
safeText(d.title),
safeText(d.description),
keywordsAll,
safeText(d.controlName),
safeText(d.namespace),
d.id,
d.relFile,
d.snippetCount ?? 0
);
totalDocs++;
}
}
});
tx();
console.log(`📊 Optimizing FTS5 index...`);
db.pragma("optimize");
// Get some stats
const rowCount = db.prepare("SELECT count(*) as n FROM docs").get() as { n: number };
db.close();
console.log(`✅ FTS5 index built successfully!`);
console.log(` 📄 Documents indexed: ${totalDocs}`);
console.log(` 📄 Rows in FTS table: ${rowCount.n}`);
console.log(` 💾 Database size: ${(fs.statSync(DST).size / 1024 / 1024).toFixed(2)} MB`);
console.log(` 📍 Location: ${DST}`);
}
// ES module equivalent of require.main === module
import { fileURLToPath } from 'url';
if (import.meta.url === `file://${process.argv[1]}`) {
try {
main();
} catch (error) {
console.error("❌ Error building FTS index:", error);
process.exit(1);
}
}
```
--------------------------------------------------------------------------------
/test/quick-url-test.ts:
--------------------------------------------------------------------------------
```typescript
#!/usr/bin/env node
/**
* Quick URL Test - Simple version for testing a few URLs from specific sources
*/
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
import fs from 'fs/promises';
import { existsSync } from 'fs';
import { generateDocumentationUrl } from '../src/lib/url-generation/index.js';
import { getDocUrlConfig, getSourcePath } from '../src/lib/metadata.js';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const PROJECT_ROOT = join(__dirname, '..');
const DATA_DIR = join(PROJECT_ROOT, 'dist', 'data');
// Simple colors
const c = {
green: '\x1b[32m', red: '\x1b[31m', yellow: '\x1b[33m',
blue: '\x1b[34m', bold: '\x1b[1m', reset: '\x1b[0m'
};
async function loadIndex() {
const indexPath = join(DATA_DIR, 'index.json');
if (!existsSync(indexPath)) {
throw new Error(`Index not found. Run: npm run build`);
}
const raw = await fs.readFile(indexPath, 'utf8');
return JSON.parse(raw);
}
async function getDocContent(libraryId: string, relFile: string): Promise<string> {
const sourcePath = getSourcePath(libraryId);
if (!sourcePath) return '# No content';
const fullPath = join(PROJECT_ROOT, 'sources', sourcePath, relFile);
if (!existsSync(fullPath)) return '# No content';
return await fs.readFile(fullPath, 'utf8');
}
async function testUrl(url: string): Promise<{ok: boolean, status: number, time: number}> {
const start = Date.now();
try {
const response = await fetch(url, { method: 'HEAD' });
return { ok: response.ok, status: response.status, time: Date.now() - start };
} catch {
return { ok: false, status: 0, time: Date.now() - start };
}
}
async function quickTest(sourceFilter?: string, count: number = 3) {
console.log(`${c.bold}${c.blue}🔗 Quick URL Test${c.reset}\n`);
const index = await loadIndex();
const sources = Object.values(index).filter((lib: any) => {
if (sourceFilter) {
return lib.id.includes(sourceFilter) || lib.name.toLowerCase().includes(sourceFilter.toLowerCase());
}
return getDocUrlConfig(lib.id) !== null; // Only test sources with URL configs
});
if (sources.length === 0) {
console.log(`${c.red}No sources found matching "${sourceFilter}"${c.reset}`);
return;
}
console.log(`Testing ${count} URLs from ${sources.length} source(s)...\n`);
for (const lib of sources) {
console.log(`${c.bold}📚 ${lib.name}${c.reset}`);
const randomDocs = lib.docs
.sort(() => 0.5 - Math.random())
.slice(0, count);
for (const doc of randomDocs) {
try {
const config = getDocUrlConfig(lib.id);
if (!config) {
console.log(`${c.yellow} ⚠️ No URL config for ${lib.id}${c.reset}`);
continue;
}
const content = await getDocContent(lib.id, doc.relFile);
const url = generateDocumentationUrl(lib.id, doc.relFile, content, config);
if (!url) {
console.log(`${c.yellow} ❌ Could not generate URL for: ${doc.title}${c.reset}`);
continue;
}
const result = await testUrl(url);
const statusColor = result.ok ? c.green : c.red;
const icon = result.ok ? '✅' : '❌';
console.log(` ${icon} ${statusColor}[${result.status}]${c.reset} ${doc.title.substring(0, 50)}${doc.title.length > 50 ? '...' : ''}`);
console.log(` ${c.blue}${url}${c.reset}`);
console.log(` ${result.time}ms\n`);
} catch (error) {
console.log(` ${c.red}❌ Error testing ${doc.title}: ${error}${c.reset}\n`);
}
}
}
}
// CLI usage
const args = process.argv.slice(2);
const sourceFilter = args[0];
const count = parseInt(args[1]) || 3;
console.log(`${c.bold}Usage:${c.reset} npx tsx test/quick-url-test.ts [source-filter] [count]`);
console.log(`${c.bold}Examples:${c.reset}`);
console.log(` npx tsx test/quick-url-test.ts cloud-sdk 2 # Test 2 URLs from Cloud SDK sources`);
console.log(` npx tsx test/quick-url-test.ts cap 5 # Test 5 URLs from CAP`);
console.log(` npx tsx test/quick-url-test.ts ui5 1 # Test 1 URL from UI5 sources`);
console.log(` npx tsx test/quick-url-test.ts # Test 3 URLs from all sources\n`);
quickTest(sourceFilter, count).catch(console.error);
```
--------------------------------------------------------------------------------
/test-community-search.js:
--------------------------------------------------------------------------------
```javascript
#!/usr/bin/env node
// Test script for the new community search functionality
// Tests both search and detailed post retrieval
import { searchCommunityBestMatch, getCommunityPostByUrl } from './dist/src/lib/communityBestMatch.js';
async function testCommunitySearch() {
console.log('🔍 Testing SAP Community Search with HTML Scraping\n');
const testQueries = [
'odata cache',
// 'CAP authentication',
// 'odata binding',
// 'fiori elements'
];
for (const query of testQueries) {
console.log(`\n📝 Testing query: "${query}"`);
console.log('=' .repeat(50));
try {
const results = await searchCommunityBestMatch(query, {
includeBlogs: true,
limit: 5,
userAgent: 'SAP-Docs-MCP-Test/1.0'
});
if (results.length === 0) {
console.log('❌ No results found');
continue;
}
console.log(`✅ Found ${results.length} results:`);
results.forEach((result, index) => {
console.log(`\n${index + 1}. ${result.title}`);
console.log(` URL: ${result.url}`);
console.log(` Author: ${result.author || 'Unknown'}`);
console.log(` Published: ${result.published || 'Unknown'}`);
console.log(` Likes: ${result.likes || 0}`);
console.log(` Snippet: ${result.snippet ? result.snippet.substring(0, 100) + '...' : 'No snippet'}`);
console.log(` Tags: ${result.tags?.join(', ') || 'None'}`);
console.log(` Post ID: ${result.postId || 'Not extracted'}`);
// Verify post ID extraction
if (result.postId) {
console.log(` ✅ Post ID extracted: ${result.postId}`);
} else {
console.log(` ⚠️ Post ID not extracted from URL: ${result.url}`);
}
});
// Test detailed post retrieval for the first result
if (results.length > 0) {
console.log(`\n🔎 Testing detailed post retrieval for: "${results[0].title}"`);
console.log('-'.repeat(50));
try {
const postContent = await getCommunityPostByUrl(results[0].url, 'SAP-Docs-MCP-Test/1.0');
if (postContent) {
console.log('✅ Successfully retrieved full post content:');
console.log(postContent.substring(0, 500) + '...\n');
} else {
console.log('❌ Failed to retrieve full post content');
}
} catch (error) {
console.log(`❌ Error retrieving post content: ${error.message}`);
}
}
} catch (error) {
console.log(`❌ Error searching for "${query}": ${error.message}`);
}
// Add delay between requests to be respectful
await new Promise(resolve => setTimeout(resolve, 2000));
}
}
async function testSpecificPost() {
console.log('\n🎯 Testing specific post retrieval');
console.log('=' .repeat(50));
// Test with the known SAP Community URL from your example
const testUrl = 'https://community.sap.com/t5/technology-blog-posts-by-sap/fiori-cache-maintenance/ba-p/13961398';
try {
console.log(`Testing URL: ${testUrl}`);
console.log(`Expected Post ID: 13961398`);
const content = await getCommunityPostByUrl(testUrl, 'SAP-Docs-MCP-Test/1.0');
if (content) {
console.log('✅ Successfully retrieved content:');
console.log(content.substring(0, 800) + '...');
// Verify the content contains expected elements
if (content.includes('FIORI Cache Maintenance')) {
console.log('✅ Title extraction successful');
}
if (content.includes('MarkNed')) {
console.log('✅ Author extraction successful');
}
if (content.includes('SMICM')) {
console.log('✅ Content extraction successful');
}
} else {
console.log('❌ No content retrieved');
}
} catch (error) {
console.log(`❌ Error: ${error.message}`);
}
}
async function main() {
console.log('🚀 Starting SAP Community Search Tests');
console.log('=====================================');
try {
await testCommunitySearch();
await testSpecificPost();
console.log('\n✅ All tests completed!');
} catch (error) {
console.error('❌ Test suite failed:', error);
process.exit(1);
}
}
// Handle graceful shutdown
process.on('SIGINT', () => {
console.log('\n👋 Test interrupted by user');
process.exit(0);
});
// Run the tests
main().catch(error => {
console.error('💥 Unexpected error:', error);
process.exit(1);
});
```
--------------------------------------------------------------------------------
/test/tools/search-url-verification.js:
--------------------------------------------------------------------------------
```javascript
// MCP Search URL Verification Test Cases
// Verifies that search results from the MCP server include proper documentation URLs
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
function loadAllowedUrlPrefixes() {
const metadataPath = join(__dirname, '..', '..', 'src', 'metadata.json');
const raw = readFileSync(metadataPath, 'utf8');
const metadata = JSON.parse(raw);
const prefixes = new Set();
for (const source of metadata?.sources || []) {
if (typeof source.baseUrl === 'string' && source.baseUrl.trim().length > 0) {
const normalized = source.baseUrl.replace(/\/$/, '');
prefixes.add(normalized);
}
}
return prefixes;
}
const allowedPrefixes = loadAllowedUrlPrefixes();
const allowedPrefixList = Array.from(allowedPrefixes);
function extractUrls(text) {
const regex = /🔗\s+(https?:\/\/[^\s]+)/g;
const urls = [];
let match;
while ((match = regex.exec(text)) !== null) {
urls.push(match[1]);
}
return urls;
}
function isAllowedDocumentationUrl(url) {
try {
const normalized = url.replace(/\/$/, '');
for (const prefix of allowedPrefixes) {
if (normalized === prefix || normalized.startsWith(`${prefix}/`) || normalized.startsWith(`${prefix}#`)) {
return true;
}
}
return false;
} catch (_) {
return false;
}
}
export default [
{
name: 'CAP CDS - Should include documentation URL',
tool: 'search',
query: 'cds query language',
skipIfNoResults: true,
expectIncludes: ['/cap/'],
expectContains: ['🔗'], // Should contain URL link emoji
expectUrlPattern: 'https://cap.cloud.sap/docs'
},
{
name: 'Cloud SDK JS - Should include documentation URL',
tool: 'search',
query: 'cloud sdk javascript remote debugging',
skipIfNoResults: true,
expectIncludes: ['/cloud-sdk-js/'],
expectContains: ['🔗'],
expectUrlPattern: 'https://sap.github.io/cloud-sdk/docs/js'
},
{
name: 'SAPUI5 - Should include documentation URL',
tool: 'search',
query: 'sapui5 button control',
skipIfNoResults: true,
expectIncludes: ['/sapui5/'],
expectContains: ['🔗'],
expectUrlPattern: 'https://ui5.sap.com'
},
{
name: 'wdi5 - Should include documentation URL',
tool: 'search',
query: 'wdi5 locators testing',
skipIfNoResults: true,
expectIncludes: ['/wdi5/'],
expectContains: ['🔗'],
expectUrlPattern: 'https://ui5-community.github.io/wdi5'
},
{
name: 'UI5 Tooling - Should include documentation URL',
tool: 'search',
query: 'ui5 tooling build',
skipIfNoResults: true,
expectIncludes: ['/ui5-tooling/'],
expectContains: ['🔗'],
expectUrlPattern: 'https://sap.github.io/ui5-tooling'
},
{
name: 'Search results should have consistent format with excerpts',
tool: 'search',
query: 'button',
skipIfNoResults: true,
expectIncludes: ['Score:', '🔗', 'Use in fetch'],
expectPattern: /⭐️\s+\*\*[^*]+\*\*\s+\(Score:\s+[\d.]+\)/
},
{
name: 'All returned documentation URLs should be HTTPS and match known sources',
async validate({ docsSearch }) {
const response = await docsSearch('sap');
if (/No results found/.test(response)) {
return {
skipped: true,
message: 'no documentation results available to validate'
};
}
const urls = extractUrls(response);
if (!urls.length) {
return {
passed: false,
message: 'No documentation URLs were found in the response.'
};
}
const invalidUrls = urls.filter(url => !/^https:\/\//.test(url) || !isAllowedDocumentationUrl(url));
if (invalidUrls.length) {
return {
passed: false,
message: `Found URLs that are not allowed or not HTTPS: ${invalidUrls.join(', ')}`
};
}
return { passed: true };
}
},
{
name: 'Metadata should expose base URLs for critical SAP documentation sources',
async validate() {
const requiredPrefixes = [
'https://cap.cloud.sap',
'https://sap.github.io/cloud-sdk',
'https://ui5.sap.com',
'https://ui5-community.github.io/wdi5'
];
const missing = requiredPrefixes.filter(prefix => {
return !allowedPrefixList.some(allowed => allowed === prefix || allowed.startsWith(prefix));
});
if (missing.length) {
return {
passed: false,
message: `Missing required base URL prefixes in metadata: ${missing.join(', ')}`
};
}
return { passed: true };
}
}
];
```
--------------------------------------------------------------------------------
/REMOTE_SETUP.md:
--------------------------------------------------------------------------------
```markdown
# Remote Server Setup Guide
This guide explains how to connect to the hosted SAP Documentation MCP Server for instant access to SAP documentation and community content without local setup.
## 🚀 Quick Setup (2 minutes)
### Step 1: Locate Your MCP Configuration File
The MCP configuration file location depends on your operating system:
| OS | Location |
|---|---|
| **macOS** | `~/.cursor/mcp.json` |
| **Linux** | `~/.cursor/mcp.json` |
| **Windows** | `%APPDATA%\Cursor\mcp.json` |
### Step 2: Create or Edit the Configuration File
If the file doesn't exist, create it. If it exists, add the new server to the existing configuration.
#### New Configuration File
```json
{
"mcpServers": {
"sap-docs-remote": {
"url": "https://mcp-sap-docs.marianzeis.de/mcp"
}
}
}
```
#### Adding to Existing Configuration
If you already have other MCP servers configured, add the new server:
```json
{
"mcpServers": {
"existing-server": {
"command": "some-command"
},
"sap-docs-remote": {
"url": "https://mcp-sap-docs.marianzeis.de/mcp"
}
}
}
```
### Step 3: Restart Cursor
Close and reopen Cursor to load the new MCP server configuration.
### Step 4: Test the Connection
Ask Cursor a SAP-related question to verify the connection:
- "How do I implement authentication in SAPUI5?"
- "Show me wdi5 testing examples"
- "What are CAP service best practices?"
## 🎯 What You Get
### Comprehensive Documentation Access
- **SAPUI5 Documentation**: 1,485+ files with complete developer guides
- **CAP Documentation**: 195+ files covering Cloud Application Programming
- **OpenUI5 API Documentation**: 500+ control APIs with detailed JSDoc
- **OpenUI5 Sample Code**: 2,000+ working examples
- **wdi5 Documentation**: End-to-end test framework docs
### Real-Time Community Content
- **SAP Community Posts**: Latest blog posts and solutions
- **High-Quality Content**: Engagement info (kudos) included when available; results follow SAP Community's Best Match ranking
- **Code Examples**: Real-world implementations from developers
- **Best Practices**: Community-tested approaches
## 🔧 Troubleshooting
### Server Not Responding
1. Check your internet connection
2. Verify the URL is correct: `https://mcp-sap-docs.marianzeis.de/mcp`
3. Restart Cursor
4. Check Cursor's MCP server status in settings
### Configuration Not Loading
1. Verify the JSON syntax is correct (use a JSON validator)
2. Ensure the file path is correct for your OS
3. Check file permissions (should be readable by your user)
4. Restart Cursor after any configuration changes
### No SAP Documentation in Responses
1. Try asking more specific SAP-related questions
2. Include keywords like "SAPUI5", "CAP", "wdi5", or "SAP Community"
3. Check if other MCP servers might be taking precedence
## 💡 Usage Tips
### Effective Prompts
Instead of generic questions, use specific SAP terminology:
✅ **Good:**
- "How do I implement SAPUI5 data binding with OData models?"
- "Show me wdi5 test examples for table interactions"
- "What are CAP service annotations for authorization?"
❌ **Less Effective:**
- "How do I bind data?"
- "Show me test examples"
- "What are service annotations?"
### Combining Local and Remote
You can use both local and remote MCP servers simultaneously:
```json
{
"mcpServers": {
"sap-docs-local": {
"command": "node",
"args": ["/path/to/local/server.js"]
},
"sap-docs-remote": {
"url": "https://mcp-sap-docs.marianzeis.de/mcp"
}
}
}
```
## 🌐 Server Information
- **URL**: https://mcp-sap-docs.marianzeis.de
- **Health Check**: https://mcp-sap-docs.marianzeis.de/status
- **Protocol**: MCP Streamable HTTP
- **Uptime**: Monitored 24/7 with automatic deployment
- **Updates**: Automatically synced with latest documentation
- **Environment**: Uses PM2 for process management (no Docker)
## 🖥️ Local Development
### VS Code Performance Optimization
The large documentation submodules are excluded from VS Code operations to prevent crashes:
- **Search Excluded**: `sources/`, `node_modules/`, `dist/`, `data/`
- **File Explorer Hidden**: Large submodule folders are hidden
- **Git Operations**: Submodules are excluded from local git tracking
### Local Setup
For local development without the large submodules:
```bash
# Quick setup (excludes large submodules)
npm run setup
# Or manually initialize submodules
npm run setup:submodules
```
## 📞 Support
If you encounter issues:
1. Check the [repository issues](https://github.com/marianfoo/mcp-sap-docs/issues)
2. Verify server status at the health check URL
3. Create a new issue with your configuration and error details
---
*The remote server provides the same comprehensive SAP documentation access as the local installation but with zero setup complexity and automatic updates.*
```
--------------------------------------------------------------------------------
/test/url-status.ts:
--------------------------------------------------------------------------------
```typescript
#!/usr/bin/env node
/**
* URL Status - Shows which sources have URL generation configured
*/
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';
import fs from 'fs/promises';
import { existsSync } from 'fs';
import { getDocUrlConfig, getSourcePath, getMetadata } from '../src/lib/metadata.js';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const PROJECT_ROOT = join(__dirname, '..');
const DATA_DIR = join(PROJECT_ROOT, 'dist', 'data');
// Colors
const c = {
green: '\x1b[32m', red: '\x1b[31m', yellow: '\x1b[33m',
blue: '\x1b[34m', bold: '\x1b[1m', reset: '\x1b[0m', dim: '\x1b[2m'
};
async function loadIndex() {
const indexPath = join(DATA_DIR, 'index.json');
if (!existsSync(indexPath)) {
throw new Error(`Index not found. Run: npm run build`);
}
const raw = await fs.readFile(indexPath, 'utf8');
return JSON.parse(raw);
}
async function showUrlStatus() {
console.log(`${c.bold}${c.blue}🔗 URL Generation Status${c.reset}\n`);
try {
const index = await loadIndex();
const metadata = getMetadata();
const sources = Object.values(index);
console.log(`Found ${sources.length} documentation sources:\n`);
// Group sources by status
const withUrls: any[] = [];
const withoutUrls: any[] = [];
sources.forEach((lib: any) => {
const config = getDocUrlConfig(lib.id);
if (config) {
withUrls.push({ lib, config });
} else {
withoutUrls.push(lib);
}
});
// Show sources with URL generation
if (withUrls.length > 0) {
console.log(`${c.bold}${c.green}✅ Sources with URL generation (${withUrls.length}):${c.reset}`);
withUrls.forEach(({ lib, config }) => {
console.log(` ${c.green}✅${c.reset} ${c.bold}${lib.name}${c.reset} (${lib.id})`);
console.log(` ${c.dim}📄 ${lib.docs.length} documents${c.reset}`);
console.log(` ${c.dim}🌐 ${config.baseUrl}${c.reset}`);
console.log(` ${c.dim}📋 Pattern: ${config.pathPattern}${c.reset}`);
console.log(` ${c.dim}⚙️ Anchor style: ${config.anchorStyle}${c.reset}`);
console.log();
});
}
// Show sources without URL generation
if (withoutUrls.length > 0) {
console.log(`${c.bold}${c.red}❌ Sources without URL generation (${withoutUrls.length}):${c.reset}`);
withoutUrls.forEach((lib: any) => {
console.log(` ${c.red}❌${c.reset} ${c.bold}${lib.name}${c.reset} (${lib.id})`);
console.log(` ${c.dim}📄 ${lib.docs.length} documents${c.reset}`);
console.log(` ${c.dim}💡 Needs baseUrl, pathPattern, and anchorStyle in metadata.json${c.reset}`);
console.log();
});
}
// Show URL generation handlers status
console.log(`${c.bold}${c.blue}🔧 URL Generation Handlers:${c.reset}`);
const handlers = [
{ pattern: '/cloud-sdk-*', description: 'Cloud SDK (JS/Java/AI variants)', status: 'implemented' },
{ pattern: '/sapui5', description: 'SAPUI5 topic-based URLs', status: 'implemented' },
{ pattern: '/openui5-*', description: 'OpenUI5 API & samples', status: 'implemented' },
{ pattern: '/cap', description: 'CAP docsify-style URLs', status: 'implemented' },
{ pattern: '/wdi5', description: 'wdi5 testing framework', status: 'implemented' },
{ pattern: '/ui5-tooling', description: 'UI5 Tooling (fallback)', status: 'fallback' },
{ pattern: '/ui5-webcomponents', description: 'UI5 Web Components (fallback)', status: 'fallback' },
{ pattern: '/cloud-mta-build-tool', description: 'MTA Build Tool (fallback)', status: 'fallback' }
];
handlers.forEach(handler => {
const statusIcon = handler.status === 'implemented' ? `${c.green}✅` : `${c.yellow}⚠️ `;
const statusText = handler.status === 'implemented' ? 'Specialized handler' : 'Generic fallback';
console.log(` ${statusIcon}${c.reset} ${c.bold}${handler.pattern}${c.reset}`);
console.log(` ${c.dim}${handler.description}${c.reset}`);
console.log(` ${c.dim}${statusText}${c.reset}`);
console.log();
});
// Summary
console.log(`${c.bold}${c.blue}📊 Summary:${c.reset}`);
console.log(` Sources with URL generation: ${c.green}${withUrls.length}${c.reset}/${sources.length}`);
console.log(` Specialized handlers: ${c.green}5${c.reset} (Cloud SDK, UI5, CAP, wdi5)`);
console.log(` Fallback handlers: ${c.yellow}3${c.reset} (Tooling, Web Components, MTA)`);
const coverage = Math.round((withUrls.length / sources.length) * 100);
const coverageColor = coverage > 80 ? c.green : coverage > 60 ? c.yellow : c.red;
console.log(` URL generation coverage: ${coverageColor}${coverage}%${c.reset}`);
} catch (error) {
console.error(`${c.red}Error: ${error}${c.reset}`);
}
}
showUrlStatus();
```
--------------------------------------------------------------------------------
/test-search-interactive.ts:
--------------------------------------------------------------------------------
```typescript
#!/usr/bin/env tsx
// Interactive search test with multiple keywords and analysis
import { searchLibraries } from './src/lib/localDocs.ts';
import { createInterface } from 'readline';
const rl = createInterface({
input: process.stdin,
output: process.stdout
});
// Predefined test cases with expected contexts
const testCases = [
{ query: 'wizard', expectedContext: 'UI5', description: 'UI5 Wizard control' },
{ query: 'cds entity', expectedContext: 'CAP', description: 'CAP entity definition' },
{ query: 'wdi5 testing', expectedContext: 'wdi5', description: 'wdi5 testing framework' },
{ query: 'service', expectedContext: 'CAP', description: 'Generic service (should prioritize CAP)' },
{ query: 'annotation', expectedContext: 'MIXED', description: 'Annotations (CAP/UI5)' },
{ query: 'sap.m.Button', expectedContext: 'UI5', description: 'Specific UI5 control' },
{ query: 'browser automation', expectedContext: 'wdi5', description: 'Browser testing' },
{ query: 'fiori elements', expectedContext: 'UI5', description: 'Fiori Elements' }
];
async function runSingleTest(query: string, expectedContext?: string) {
console.log(`\n${'='.repeat(60)}`);
console.log(`🔍 Testing: "${query}"${expectedContext ? ` (Expected: ${expectedContext})` : ''}`);
console.log(`${'='.repeat(60)}`);
try {
const startTime = Date.now();
const result = await searchLibraries(query);
const endTime = Date.now();
if (result.results.length > 0) {
const description = result.results[0].description;
// Extract detected context
const contextMatch = description.match(/\*\*(\w+) Context\*\*/);
const detectedContext = contextMatch ? contextMatch[1] : 'Unknown';
// Show performance and context
console.log(`⏱️ Search time: ${endTime - startTime}ms`);
console.log(`🎯 Detected context: ${detectedContext}`);
if (expectedContext) {
const isCorrect = detectedContext === expectedContext || expectedContext === 'MIXED';
console.log(`✅ Context match: ${isCorrect ? 'YES' : 'NO'} ${isCorrect ? '✅' : '❌'}`);
}
// Extract first few results for summary
const lines = description.split('\n');
const foundLine = lines.find(line => line.includes('Found'));
if (foundLine) {
console.log(`📊 ${foundLine}`);
}
// Show top result
const topResultLine = lines.find(line => line.includes('⭐️'));
if (topResultLine) {
console.log(`🏆 Top result: ${topResultLine.substring(0, 80)}...`);
}
// Show library sections found
const sections = [];
if (description.includes('🏗️ **CAP Documentation:**')) sections.push('CAP');
if (description.includes('🧪 **wdi5 Documentation:**')) sections.push('wdi5');
if (description.includes('📖 **SAPUI5 Guides:**')) sections.push('SAPUI5');
if (description.includes('🔹 **UI5 API Documentation:**')) sections.push('UI5-API');
if (description.includes('🔸 **UI5 Samples:**')) sections.push('UI5-Samples');
console.log(`📚 Libraries found: ${sections.join(', ') || 'None'}`);
} else {
console.log('❌ No results found');
if (result.error) {
console.log(`📝 Error: ${result.error}`);
}
}
} catch (error) {
console.error('❌ Search failed:', error);
}
}
async function runAllTests() {
console.log('🧪 Running all predefined test cases...\n');
for (const testCase of testCases) {
await runSingleTest(testCase.query, testCase.expectedContext);
}
console.log('\n🎉 All tests completed!');
}
async function interactiveMode() {
console.log(`
🎯 SAP Docs Interactive Search Test
Commands:
- Enter any search term to test
- 'all' - Run all predefined tests
- 'list' - Show predefined test cases
- 'quit' or 'exit' - Exit
`);
const question = () => {
rl.question('\n🔍 Enter search term (or command): ', async (input) => {
const trimmed = input.trim();
if (trimmed === 'quit' || trimmed === 'exit') {
console.log('👋 Goodbye!');
rl.close();
return;
}
if (trimmed === 'all') {
await runAllTests();
question();
return;
}
if (trimmed === 'list') {
console.log('\n📋 Predefined test cases:');
testCases.forEach((tc, i) => {
console.log(` ${i + 1}. "${tc.query}" (${tc.expectedContext}) - ${tc.description}`);
});
question();
return;
}
if (trimmed) {
await runSingleTest(trimmed);
}
question();
});
};
question();
}
// Main execution
const args = process.argv.slice(2);
if (args.length === 0) {
interactiveMode();
} else if (args[0] === 'all') {
runAllTests().then(() => process.exit(0));
} else {
const query = args.join(' ');
runSingleTest(query).then(() => process.exit(0));
}
```
--------------------------------------------------------------------------------
/src/lib/url-generation/index.ts:
--------------------------------------------------------------------------------
```typescript
/**
* Main entry point for URL generation across all documentation sources
* Dispatches to source-specific generators based on library ID
*/
import { DocUrlConfig } from '../metadata.js';
import { CloudSdkUrlGenerator } from './cloud-sdk.js';
import { SapUi5UrlGenerator } from './sapui5.js';
import { CapUrlGenerator } from './cap.js';
import { Wdi5UrlGenerator } from './wdi5.js';
import { DsagUrlGenerator } from './dsag.js';
import { AbapUrlGenerator } from './abap.js';
import { GenericUrlGenerator } from './GenericUrlGenerator.js';
import { BaseUrlGenerator } from './BaseUrlGenerator.js';
export interface UrlGenerationOptions {
libraryId: string;
relFile: string;
content: string;
config: DocUrlConfig;
}
/**
* URL Generator Registry
* Maps library IDs to their corresponding URL generator classes
*/
const URL_GENERATORS: Record<string, new (libraryId: string, config: DocUrlConfig) => BaseUrlGenerator> = {
// Cloud SDK variants
'/cloud-sdk-js': CloudSdkUrlGenerator,
'/cloud-sdk-java': CloudSdkUrlGenerator,
'/cloud-sdk-ai-js': CloudSdkUrlGenerator,
'/cloud-sdk-ai-java': CloudSdkUrlGenerator,
// UI5 variants
'/sapui5': SapUi5UrlGenerator,
'/openui5-api': SapUi5UrlGenerator,
'/openui5-samples': SapUi5UrlGenerator,
// CAP documentation
'/cap': CapUrlGenerator,
// wdi5 testing framework
'/wdi5': Wdi5UrlGenerator,
// DSAG ABAP Leitfaden with custom GitHub Pages URL pattern
'/dsag-abap-leitfaden': DsagUrlGenerator,
// ABAP Keyword Documentation with SAP help.sap.com URLs (all versions)
'/abap-docs-758': AbapUrlGenerator,
'/abap-docs-757': AbapUrlGenerator,
'/abap-docs-756': AbapUrlGenerator,
'/abap-docs-755': AbapUrlGenerator,
'/abap-docs-754': AbapUrlGenerator,
'/abap-docs-753': AbapUrlGenerator,
'/abap-docs-752': AbapUrlGenerator,
'/abap-docs-latest': AbapUrlGenerator,
// Generic sources
'/ui5-tooling': GenericUrlGenerator,
'/cloud-mta-build-tool': GenericUrlGenerator,
'/ui5-webcomponents': GenericUrlGenerator,
'/ui5-typescript': GenericUrlGenerator,
'/ui5-cc-spreadsheetimporter': GenericUrlGenerator,
'/abap-cheat-sheets': GenericUrlGenerator,
'/sap-styleguides': GenericUrlGenerator,
'/abap-fiori-showcase': GenericUrlGenerator,
'/cap-fiori-showcase': GenericUrlGenerator,
};
/**
* Create URL generator for a given library ID
*/
function createUrlGenerator(libraryId: string, config: DocUrlConfig): BaseUrlGenerator {
const GeneratorClass = URL_GENERATORS[libraryId];
if (GeneratorClass) {
return new GeneratorClass(libraryId, config);
}
// Fallback to generic generator for unknown sources
console.log(`Using generic URL generator for unknown library: ${libraryId}`);
return new GenericUrlGenerator(libraryId, config);
}
/**
* Main URL generation function
* Uses class-based generators for cleaner, more maintainable code
*
* @param libraryId - The library/source identifier (e.g., '/cloud-sdk-js')
* @param relFile - Relative file path within the source
* @param content - File content for extracting metadata
* @param config - URL configuration for this source
* @returns Generated URL or null if generation fails
*/
export function generateDocumentationUrl(
libraryId: string,
relFile: string,
content: string,
config: DocUrlConfig
): string | null {
if (!config) {
console.warn(`No URL config available for library: ${libraryId}`);
return null;
}
try {
const generator = createUrlGenerator(libraryId, config);
const url = generator.generateUrl({
libraryId,
relFile,
content,
config
});
return url;
} catch (error) {
console.warn(`Error generating URL for ${libraryId}:`, error);
return null;
}
}
// Re-export utilities and generator classes for external use
export { parseFrontmatter, detectContentSection, extractSectionFromPath, buildUrl, extractLibraryIdFromPath, extractRelativeFileFromPath, formatSearchResult } from './utils.js';
export { BaseUrlGenerator } from './BaseUrlGenerator.js';
export type { UrlGenerationContext } from './BaseUrlGenerator.js';
// Re-export generator classes
export { CloudSdkUrlGenerator } from './cloud-sdk.js';
export { SapUi5UrlGenerator } from './sapui5.js';
export { CapUrlGenerator } from './cap.js';
export { Wdi5UrlGenerator } from './wdi5.js';
export { DsagUrlGenerator } from './dsag.js';
export { GenericUrlGenerator } from './GenericUrlGenerator.js';
// Re-export convenience functions for backward compatibility
export { generateCloudSdkUrl, generateCloudSdkAiUrl, generateCloudSdkUrlForLibrary } from './cloud-sdk.js';
export { generateSapUi5Url, generateOpenUi5ApiUrl, generateOpenUi5SampleUrl, generateUi5UrlForLibrary } from './sapui5.js';
export { generateCapUrl, generateCapCdsUrl, generateCapTutorialUrl } from './cap.js';
export { generateWdi5Url, generateWdi5ConfigUrl, generateWdi5SelectorUrl } from './wdi5.js';
export { generateDsagUrl } from './dsag.js';
export { generateAbapUrl } from './abap.js';
```
--------------------------------------------------------------------------------
/docs/TEST-SEARCH.md:
--------------------------------------------------------------------------------
```markdown
# 🔍 SAP Docs Search Testing Guide
Test the enhanced context-aware search functionality using these testing tools.
## 📁 Test Files Available
### 1. **Simple Search Test** (`test-search.ts`)
Quick command-line search testing with any keyword.
```bash
# Test with default keyword
npx tsx test-search.ts
# Test specific keywords
npx tsx test-search.ts wizard
npx tsx test-search.ts "cds entity"
npx tsx test-search.ts "wdi5 testing"
npx tsx test-search.ts annotation
```
**Features:**
- ⏱️ Performance timing
- 📊 Result summary
- 🎯 Context detection display
- 📖 Top result preview
### 2. **Interactive Search Test** (`test-search-interactive.ts`)
Advanced testing with multiple modes and analysis.
```bash
# Interactive mode
npx tsx test-search-interactive.ts
# Run all predefined tests
npx tsx test-search-interactive.ts all
# Test specific query
npx tsx test-search-interactive.ts "your search term"
```
**Interactive Commands:**
- `all` - Run all predefined test cases
- `list` - Show predefined test cases
- `quit` / `exit` - Exit interactive mode
- Any keyword - Test search
**Features:**
- 🧪 Predefined test cases with expected contexts
- ✅ Context validation
- 📚 Library breakdown analysis
- 🏆 Top result highlighting
- ⏱️ Performance metrics
### 3. **HTTP API Tests** (`test-search.http`) ✅ **WORKING**
Test the HTTP server endpoints (requires VS Code REST Client or similar).
**First, start the HTTP server:**
```bash
npm run start:http
```
**Then use the `.http` file to test:**
- Server status check
- Various search queries
- Context-specific tests
- **Full search functionality** with context-aware results
**Example response for "wizard":**
```json
{
"role": "assistant",
"content": "Found 10 results for 'wizard' 🎨 **UI5 Context**:\n\n🔹 **UI5 API Documentation:**\n⭐️ **sap.f.SidePanel** (Score: 100)..."
}
```
## 🎯 Test Categories
### **UI5 Context Tests**
```bash
npx tsx test-search.ts wizard
npx tsx test-search.ts "sap.m.Button"
npx tsx test-search.ts "fiori elements"
```
Expected: 🎨 **UI5 Context** with UI5 API/Samples first
### **CAP Context Tests**
```bash
npx tsx test-search.ts "cds entity"
npx tsx test-search.ts service
npx tsx test-search.ts aspect
```
Expected: 🏗️ **CAP Context** with CAP Documentation first
### **wdi5 Context Tests**
```bash
npx tsx test-search.ts "wdi5 testing"
npx tsx test-search.ts "browser automation"
npx tsx test-search.ts "e2e test"
```
Expected: 🧪 **wdi5 Context** with wdi5 Documentation first
### **Mixed Context Tests**
```bash
npx tsx test-search.ts annotation
npx tsx test-search.ts authentication
npx tsx test-search.ts routing
```
Expected: Context varies based on strongest signal
## 📊 Understanding Results
### **Context Detection** 🎯
- **🎨 UI5 Context**: UI5 controls, Fiori, frontend development
- **🏗️ CAP Context**: CDS, entities, services, backend development
- **🧪 wdi5 Context**: Testing, automation, browser testing
- **🔀 MIXED Context**: Cross-platform or unclear context
### **Scoring System** ⭐
- **Score 100**: Perfect matches
- **Score 90+**: High relevance matches
- **Score 80+**: Good matches with context boost
- **Score 70+**: Moderate relevance
- **Score <70**: Lower relevance (often filtered out)
### **Library Prioritization** 📚
Results are ordered by relevance score, with context-aware penalties:
- **CAP queries**: OpenUI5 results get 70% penalty (unless integration-related)
- **wdi5 queries**: OpenUI5 results get 80% penalty (unless testing-related)
- **UI5 queries**: CAP/wdi5 results get 60% penalty (unless backend/testing-related)
## 🧪 Example Test Session
```bash
# Start interactive testing
npx tsx test-search-interactive.ts
🔍 Enter search term (or command): wizard
🎯 Detected context: UI5
✅ Context match: YES ✅
🏆 Top result: ⭐️ **sap.f.SidePanel** (Score: 100)...
📚 Libraries found: UI5-API, UI5-Samples
🔍 Enter search term (or command): cds entity
🎯 Detected context: CAP
✅ Context match: YES ✅
🏆 Top result: ⭐️ **Core / Built-in Types** (Score: 100)...
📚 Libraries found: CAP
🔍 Enter search term (or command): all
🧪 Running all predefined test cases...
[Runs comprehensive test suite]
```
## 🚀 Quick Start
1. **Test a simple search:**
```bash
npx tsx test-search.ts wizard
```
2. **Run the full test suite:**
```bash
npx tsx test-search-interactive.ts all
```
3. **Test HTTP API (optional):**
```bash
npm run start:http
# Then use test-search.http file
```
## 📈 Performance Benchmarks
Typical search times:
- **Simple queries**: 1-3 seconds
- **Complex queries**: 2-5 seconds
- **First search** (cold start): May take longer due to index loading
## 🔧 Troubleshooting
**No results found:**
- Check spelling
- Try broader terms
- Use predefined test cases to verify system works
**Slow performance:**
- First search loads index (normal)
- Subsequent searches should be faster
- Check available memory
**Wrong context detection:**
- Context is based on keyword analysis
- Mixed contexts are normal for generic terms
- Use more specific terms for better context detection
```
--------------------------------------------------------------------------------
/docs/FTS5-IMPLEMENTATION-COMPLETE.md:
--------------------------------------------------------------------------------
```markdown
# ✅ FTS5 Hybrid Search Implementation - Complete!
## 🎉 Successfully Implemented
I've successfully implemented the **FTS5 Hybrid Search** approach that preserves all your sophisticated search logic while providing massive performance improvements.
### 📊 **Performance Results**
- **16x faster search**: 42ms vs 700ms (based on test results)
- **7,677 documents indexed** into a **3.5MB SQLite database**
- **Graceful fallback** to full search when FTS finds no candidates
- **All sophisticated features preserved**: context-aware scoring, query expansion, fuzzy matching
### 🏗️ **What Was Built**
#### 1. **FTS5 Index Builder** (`scripts/build-fts.ts`)
- Reads your existing `data/index.json`
- Creates optimized FTS5 SQLite database at `data/docs.sqlite`
- Indexes: title, description, keywords, controlName, namespace
- Simple schema focused on fast candidate filtering
#### 2. **FTS Query Module** (`src/lib/searchDb.ts`)
- `getFTSCandidateIds()` - Fast filtering to get top candidate document IDs
- `searchFTS()` - Full FTS search for testing/debugging
- `getFTSStats()` - Database statistics for monitoring
- Handles query sanitization (quotes terms with dots like "sap.m.Button")
#### 3. **Hybrid Search Logic** (Modified `src/lib/localDocs.ts`)
- **Step 1**: Use FTS to get ~100 candidate documents per expanded query
- **Step 2**: Apply your existing sophisticated scoring ONLY to FTS candidates
- **Step 3**: If FTS fails/finds nothing, fall back to full search
- **Preserves ALL**: Query expansion, context penalties, fuzzy matching, file content integration
#### 4. **Updated Build Scripts** (`package.json`)
```bash
npm run build:index # Build regular index (unchanged)
npm run build:fts # Build FTS5 index from regular index
npm run build:all # Build both indexes in sequence
```
### 🚀 **How It Works**
#### The Hybrid Approach
```
User Query "wizard"
↓
Query Expansion: ["wizard", "sap.m.Wizard", "UI5 wizard", ...]
↓
FTS Fast Filter: 7,677 docs → 30 candidates (in ~1ms)
↓
Your Sophisticated Scoring: Applied only to 30 candidates (preserves all logic)
↓
Context Penalties & Boosts: CAP/UI5/wdi5 context awareness (unchanged)
↓
Formatted Results: Same output format as before
```
#### Why This Approach is Superior
- ✅ **16x performance improvement** without any functional regression
- ✅ **Zero risk** - Falls back to full search if FTS fails
- ✅ **All features preserved** - Context scoring, query expansion, fuzzy matching
- ✅ **Simple deployment** - Just copy the `data/docs.sqlite` file
- ✅ **Transparent operation** - Results show "(🚀 FTS-filtered from X candidates)" when active
### 🔧 **Usage Instructions**
#### Initial Setup (Run Once)
```bash
# Build both indexes
npm run build:all
```
#### Production Deployment
1. Run `npm run build:all` in your CI/CD
2. Deploy both files: `data/index.json` AND `data/docs.sqlite`
3. Your search is now 16x faster automatically!
#### Monitoring
The search results now show FTS status:
- `(🚀 FTS-filtered from X candidates)` - FTS is working
- `(🔍 Full search)` - Fell back to full search
### 🔍 **Technical Details**
#### FTS5 Schema
```sql
CREATE VIRTUAL TABLE docs USING fts5(
libraryId, -- for filtering (/cap, /sapui5, etc.)
type, -- markdown/jsdoc/sample
title, -- strong search signal
description, -- secondary search signal
keywords, -- properties, events, aggregations
controlName, -- Wizard, Button, etc.
namespace, -- sap.m, sap.f, etc.
id UNINDEXED, -- metadata only
relFile UNINDEXED,
snippetCount UNINDEXED
);
```
#### Query Processing
- Simple terms: `wizard` → `wizard*` (prefix matching)
- Dotted terms: `sap.m.Button` → `"sap.m.Button"` (phrase search)
- Multi-word: `entity service` → `entity* service*`
- Falls back gracefully on any FTS error
### 🎯 **What's Preserved**
All your sophisticated search features remain intact:
- ✅ **400+ line synonym expansion system**
- ✅ **Context-aware penalties** (CAP/UI5/wdi5 scoring)
- ✅ **Fuzzy matching** with Levenshtein distance
- ✅ **File content integration** (extracts UI5 controls from user files)
- ✅ **Rich metadata scoring** (properties, events, aggregations)
- ✅ **SAP Community integration**
- ✅ **All existing result formatting**
### 🚀 **Next Steps**
1. **Test in your environment**: The system is ready to use
2. **Monitor performance**: Check logs for FTS usage indicators
3. **CI/CD Integration**: Add `npm run build:all` to your deployment pipeline
4. **Optional**: Fine-tune FTS candidate limit (currently 100 per query)
### 📈 **Expected Benefits**
- **Faster user experience**: 16x search speed improvement
- **Better scalability**: Performance stays consistent as docs grow
- **Lower server load**: Faster searches = less CPU usage
- **Easy deployment**: Single SQLite file, no additional services needed
## 🎉 **Implementation Complete!**
Your search is now **16x faster** while preserving **all sophisticated features**. The FTS5 hybrid approach gives you the best of both worlds: blazing fast performance with zero functional regression.
Enjoy your supercharged search! 🚀
```
--------------------------------------------------------------------------------
/docs/LLM-FRIENDLY-IMPROVEMENTS.md:
--------------------------------------------------------------------------------
```markdown
# LLM-Friendly MCP Tool Improvements
This document summarizes the improvements made to make the SAP Docs MCP server more LLM-friendly, based on Claude's feedback and analysis.
## 🎯 **Key Issues Addressed**
### **Original Problem**
Claude was confused about function names, using incorrect patterns like:
- ❌ `search: query "..."` (FAILED - wrong syntax)
- ❌ `SAP Docs MCP:search` (FAILED - incorrect namespace)
- ✅ Should be: `search(query="...")` or `mcp_sap-docs-remote_search(query="...")`
## 🔧 **Improvements Implemented**
### **1. Simplified Visual Formatting**
**Before:**
```
**FUNCTION NAME: Use exactly 'search' or 'mcp_sap-docs-remote_search' depending on your MCP client**
Unified search across all SAP documentation sources...
**EXAMPLE USAGE:**
```
search(query="CAP binary data LargeBinary MediaType")
```
```
**After:**
```
SEARCH SAP DOCS: search(query="search terms")
FUNCTION NAME: search
COVERS: ABAP (all versions), UI5, CAP, wdi5, OpenUI5 APIs, Cloud SDK
AUTO-DETECTS: ABAP versions from query (e.g. "LOOP 7.57", defaults to 7.58)
```
### **2. Structured Examples in JSON Schema**
**Before:** Examples mixed into description text
**After:** Clean `examples` array in JSON schema:
```javascript
{
"examples": [
"CAP binary data LargeBinary MediaType",
"UI5 button properties",
"wdi5 testing locators",
"ABAP SELECT statements 7.58",
"415 error CAP action parameter"
]
}
```
### **3. Added Workflow Patterns**
**New sections added:**
```
TYPICAL WORKFLOW:
1. search(query="your search terms")
2. fetch(id="result_id_from_step_1")
COMMON PATTERNS:
• Broad exploration: id="/cap/binary"
• Specific API: id="/openui5-api/sap/m/Button"
• Community posts: id="community-12345"
```
### **4. Improved Error Messages**
**Before:**
```
No results found for "query". Try searching for UI5 controls like 'button', 'table', 'wizard', testing topics like 'wdi5', 'testing', 'e2e', or concepts like 'routing', 'annotation', 'authentication', 'fiori elements', 'rap'. For detailed ABAP language syntax, use abap_search instead.
```
**After:**
```
No results for "query".
TRY INSTEAD:
• UI5 controls: "button", "table", "wizard"
• CAP topics: "actions", "authentication", "media", "binary"
• Testing: "wdi5", "locators", "e2e"
• ABAP: Use version numbers like "SELECT 7.58"
• Errors: Include error codes like "415 error CAP action"
```
### **5. Query Optimization Hints**
**Added to each tool:**
```
QUERY TIPS:
• Be specific: "CAP action binary parameter" not just "CAP"
• Include error codes: "415 error CAP action"
• Use technical terms: "LargeBinary MediaType XMLHttpRequest"
• For ABAP: Include version like "7.58" or "latest"
```
### **6. Header Documentation for Developers**
```javascript
/**
* IMPORTANT FOR LLMs/AI ASSISTANTS:
* =================================
* The function names in this MCP server may appear with different prefixes depending on your MCP client:
* - Simple names: search, fetch, sap_community_search, sap_help_search, sap_help_get
* - Prefixed names: mcp_sap-docs-remote_search, mcp_sap-docs-remote_fetch, etc.
*
* Try the simple names first, then the prefixed versions if they don't work.
*/
```
## 📊 **Impact on LLM Usage**
### **Function Name Clarity**
- ✅ Explicit guidance on both naming patterns
- ✅ Clear fallback strategy (try simple names first)
- ✅ Reduced confusion about MCP client variations
### **Query Construction**
- ✅ Concrete examples for each tool type
- ✅ Technical terminology guidance
- ✅ Error code inclusion strategies
- ✅ ABAP version detection hints
### **Workflow Understanding**
- ✅ Clear search → get patterns
- ✅ Common usage scenarios
- ✅ Library ID vs document ID guidance
### **Error Recovery**
- ✅ Actionable next steps instead of long descriptions
- ✅ Alternative tool suggestions
- ✅ Specific retry strategies
## 🚀 **Tools Updated**
1. **search** - Main documentation search
2. **fetch** - Retrieve specific documentation
3. **sap_community_search** - Community posts and discussions
4. **sap_help_search** - Official SAP Help Portal
5. **sap_help_get** - Get specific Help Portal pages
## 📝 **Best Practices for LLMs**
### **Search Strategy**
1. Start with `search` for technical documentation
2. Use `sap_community_search` for troubleshooting and error codes
3. Always follow up search results with `fetch` or `sap_help_get`
### **Query Construction**
- Include product names: "CAP", "UI5", "ABAP", "wdi5"
- Add technical terms: "LargeBinary", "MediaType", "XMLHttpRequest"
- Include error codes: "415", "500", "404"
- Specify ABAP versions: "7.58", "latest"
### **Common Workflows**
```
Problem-solving pattern:
1. search(query="technical problem + error code")
2. sap_community_search(query="same problem for community solutions")
3. fetch(id="most_relevant_result")
```
## ✅ **Validation**
The improvements address the specific issues Claude encountered:
- ✅ Function naming confusion resolved
- ✅ Parameter format clarity improved
- ✅ Search strategy guidance provided
- ✅ Error messages made actionable
- ✅ Examples based on real usage patterns
---
*These improvements make the SAP Docs MCP server significantly more accessible to LLMs like Claude, reducing confusion and improving successful tool call rates.*
```
--------------------------------------------------------------------------------
/src/lib/url-generation/BaseUrlGenerator.ts:
--------------------------------------------------------------------------------
```typescript
/**
* Abstract base class for URL generation across documentation sources
* Provides common functionality and standardized interface for all URL generators
*/
import { parseFrontmatter, extractSectionFromPath, buildUrl, detectContentSection, FrontmatterData } from './utils.js';
import { DocUrlConfig } from '../metadata.js';
export interface UrlGenerationContext {
relFile: string;
content: string;
config: DocUrlConfig;
libraryId: string;
}
export interface UrlGenerationResult {
url: string | null;
anchor?: string;
section?: string;
frontmatter?: FrontmatterData;
}
/**
* Abstract base class for all URL generators
* Provides common functionality while allowing source-specific customization
*/
export abstract class BaseUrlGenerator {
protected readonly libraryId: string;
protected readonly config: DocUrlConfig;
constructor(libraryId: string, config: DocUrlConfig) {
this.libraryId = libraryId;
this.config = config;
}
/**
* Main entry point for URL generation
* Orchestrates the generation process using template method pattern
*/
public generateUrl(context: UrlGenerationContext): string | null {
try {
const frontmatter = this.parseFrontmatter(context.content);
const section = this.extractSection(context.relFile);
const anchor = this.generateAnchor(context.content);
// Try source-specific generation first
let url = this.generateSourceSpecificUrl({
...context,
frontmatter,
section,
anchor
});
// Fallback to generic generation if needed
if (!url) {
url = this.generateFallbackUrl({
...context,
frontmatter,
section,
anchor
});
}
return url;
} catch (error) {
console.warn(`Error generating URL for ${this.libraryId}:`, error);
return null;
}
}
/**
* Source-specific URL generation logic
* Must be implemented by each concrete generator
*/
protected abstract generateSourceSpecificUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null;
/**
* Generic fallback URL generation
* Uses filename and config pattern as last resort
*/
protected generateFallbackUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
// Extract just the filename without directory path to avoid duplication with pathPattern
const fileName = context.relFile
.replace(/\.mdx?$/, '')
.replace(/\.html?$/, '')
.replace(/.*\//, ''); // Remove directory path, keep only filename
let urlPath = this.config.pathPattern.replace('{file}', fileName);
// Add anchor if available
if (context.anchor) {
const separator = this.getSeparator();
urlPath += separator + context.anchor;
}
return this.config.baseUrl + urlPath;
}
/**
* Parse frontmatter from content
* Can be overridden for source-specific parsing needs
*/
protected parseFrontmatter(content: string): FrontmatterData {
return parseFrontmatter(content);
}
/**
* Extract section from file path
* Can be overridden for source-specific section logic
*/
protected extractSection(relFile: string): string {
return extractSectionFromPath(relFile);
}
/**
* Generate anchor from content
* Can be overridden for source-specific anchor logic
*/
protected generateAnchor(content: string): string | null {
return detectContentSection(content, this.config.anchorStyle);
}
/**
* Get URL separator based on anchor style
*/
protected getSeparator(): string {
return this.config.anchorStyle === 'docsify' ? '?id=' : '#';
}
/**
* Build clean URL with proper path joining
*/
protected buildUrl(baseUrl: string, ...pathSegments: string[]): string {
return buildUrl(baseUrl, ...pathSegments);
}
/**
* Get the identifier from frontmatter (id or slug)
* Common pattern used by many sources
*/
protected getIdentifierFromFrontmatter(frontmatter: FrontmatterData): string | null {
return frontmatter.id || frontmatter.slug || null;
}
/**
* Check if file is in specific directory
*/
protected isInDirectory(relFile: string, directory: string): boolean {
return relFile.includes(`${directory}/`);
}
/**
* Extract filename without extension
*/
protected getCleanFileName(relFile: string): string {
return relFile
.replace(/\.mdx?$/, '')
.replace(/\.html?$/, '')
.replace(/.*\//, ''); // Get last part after slash
}
/**
* Build URL with section and identifier
* Common pattern for many documentation sites
*/
protected buildSectionUrl(section: string, identifier: string, anchor?: string | null): string {
let url = this.buildUrl(this.config.baseUrl, section, identifier);
if (anchor) {
const separator = this.getSeparator();
url += separator + anchor;
}
return url;
}
/**
* Build docsify-style URL with # fragment
*/
protected buildDocsifyUrl(path: string): string {
const cleanPath = path.startsWith('/') ? path.slice(1) : path;
return `${this.config.baseUrl}/#/${cleanPath}`;
}
}
```
--------------------------------------------------------------------------------
/src/lib/truncate.ts:
--------------------------------------------------------------------------------
```typescript
// Intelligent content truncation utility
// Preserves structure and readability while limiting content size
import { CONFIG } from "./config.js";
export interface TruncationResult {
content: string;
wasTruncated: boolean;
originalLength: number;
truncatedLength: number;
}
/**
* Intelligently truncate content to a maximum length while preserving:
* - Beginning (intro/overview)
* - End (conclusions/examples)
* - Code block integrity
* - Markdown structure
*
* @param content - The content to truncate
* @param maxLength - Maximum length (defaults to CONFIG.MAX_CONTENT_LENGTH)
* @returns TruncationResult with truncated content and metadata
*/
export function truncateContent(
content: string,
maxLength: number = CONFIG.MAX_CONTENT_LENGTH
): TruncationResult {
const originalLength = content.length;
// No truncation needed
if (originalLength <= maxLength) {
return {
content,
wasTruncated: false,
originalLength,
truncatedLength: originalLength
};
}
// Calculate how much content to preserve from start and end
// Keep 60% from the start (intro/main content) and 20% from end (conclusions)
// Reserve 20% for truncation notice and buffer
const startLength = Math.floor(maxLength * 0.6);
const endLength = Math.floor(maxLength * 0.2);
const noticeLength = maxLength - startLength - endLength;
// Extract start portion
let startPortion = content.substring(0, startLength);
// Try to break at a natural boundary (paragraph, heading, or code block)
const naturalBreakPatterns = [
/\n\n/g, // Paragraph breaks
/\n#{1,6}\s/g, // Markdown headings
/\n```\n/g, // Code block ends
/\n---\n/g, // Horizontal rules
/\.\s+/g // Sentence ends
];
for (const pattern of naturalBreakPatterns) {
const matches = Array.from(startPortion.matchAll(pattern));
if (matches.length > 0) {
const lastMatch = matches[matches.length - 1];
if (lastMatch.index && lastMatch.index > startLength * 0.8) {
startPortion = startPortion.substring(0, lastMatch.index + lastMatch[0].length);
break;
}
}
}
// Extract end portion
let endPortion = content.substring(content.length - endLength);
// Try to break at natural boundary from the beginning of end portion
for (const pattern of naturalBreakPatterns) {
const match = endPortion.match(pattern);
if (match && match.index !== undefined && match.index < endLength * 0.2) {
endPortion = endPortion.substring(match.index + match[0].length);
break;
}
}
// Create truncation notice
const omittedChars = originalLength - (startPortion.length + endPortion.length);
const omittedPercent = Math.round((omittedChars / originalLength) * 100);
const truncationNotice = `
---
⚠️ **Content Truncated**
The full content was ${originalLength.toLocaleString()} characters (approximately ${Math.round(originalLength / 4)} tokens).
For readability and performance, ${omittedChars.toLocaleString()} characters (${omittedPercent}%) have been omitted from the middle section.
The beginning and end of the document are preserved above and below this notice.
---
`;
// Combine portions
const truncatedContent = startPortion + truncationNotice + endPortion;
return {
content: truncatedContent,
wasTruncated: true,
originalLength,
truncatedLength: truncatedContent.length
};
}
/**
* Truncate content with a simple notice at the end
* Used when preserving both beginning and end doesn't make sense
*
* @param content - The content to truncate
* @param maxLength - Maximum length (defaults to CONFIG.MAX_CONTENT_LENGTH)
* @returns TruncationResult with truncated content and metadata
*/
export function truncateContentSimple(
content: string,
maxLength: number = CONFIG.MAX_CONTENT_LENGTH
): TruncationResult {
const originalLength = content.length;
// No truncation needed
if (originalLength <= maxLength) {
return {
content,
wasTruncated: false,
originalLength,
truncatedLength: originalLength
};
}
// Reserve space for truncation notice
const noticeLength = 300;
const contentLength = maxLength - noticeLength;
// Extract content
let truncatedContent = content.substring(0, contentLength);
// Try to break at natural boundary
const lastParagraph = truncatedContent.lastIndexOf('\n\n');
const lastSentence = truncatedContent.lastIndexOf('. ');
if (lastParagraph > contentLength * 0.9) {
truncatedContent = truncatedContent.substring(0, lastParagraph);
} else if (lastSentence > contentLength * 0.9) {
truncatedContent = truncatedContent.substring(0, lastSentence + 1);
}
// Add truncation notice
const omittedChars = originalLength - truncatedContent.length;
const omittedPercent = Math.round((omittedChars / originalLength) * 100);
truncatedContent += `
---
⚠️ **Content Truncated**
The full content was ${originalLength.toLocaleString()} characters (approximately ${Math.round(originalLength / 4)} tokens).
${omittedChars.toLocaleString()} characters (${omittedPercent}%) have been omitted for readability.
---
`;
return {
content: truncatedContent,
wasTruncated: true,
originalLength,
truncatedLength: truncatedContent.length
};
}
```
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/missing-documentation.yml:
--------------------------------------------------------------------------------
```yaml
name: Missing Documentation Search Result
description: Report when expected SAP documentation cannot be found through MCP server searches
title: "[MISSING DOC]: "
assignees: []
body:
- type: markdown
attributes:
value: |
Thanks for reporting a missing documentation issue! This helps us improve our search coverage and indexing.
Please provide as much detail as possible to help us locate and index the missing content.
- type: textarea
id: problem-description
attributes:
label: Problem Description
description: Describe what you were trying to find and why the current search results are insufficient
placeholder: "I was looking for information about... but the search results didn't include..."
validations:
required: true
- type: textarea
id: llm-search-query
attributes:
label: Search Query Used in LLM
description: What did you ask the LLM? Include the exact prompt or question you used
placeholder: "How do I configure authentication in SAP CAP applications?"
render: text
validations:
required: true
- type: checkboxes
id: mcp-tool-used
attributes:
label: MCP Tools Called
description: Which MCP tools were used for the search? (Select all that apply)
options:
- label: "sap_docs_search"
- label: "sap_community_search"
- label: "sap_help_search"
- label: "sap_note_search"
- label: "Not sure/Unknown"
- type: textarea
id: search-parameters
attributes:
label: Search Parameters
description: What parameters were passed to the MCP server? (query, filters, etc.)
placeholder: |
Query: "authentication CAP"
Library: "/cap"
render: text
validations:
required: false
- type: textarea
id: mcp-response
attributes:
label: MCP Server Response
description: What results did the MCP server return? Include relevant excerpts or "No results found"
placeholder: "The server returned 3 results about basic authentication but none covered the specific JWT configuration I needed"
render: text
validations:
required: false
- type: checkboxes
id: expected-source
attributes:
label: Expected Documentation Sources
description: Which SAP documentation sources should contain this information? (Select all that apply)
options:
- label: "SAP CAP Documentation"
- label: "SAPUI5 Documentation"
- label: "OpenUI5 API Reference"
- label: "SAP Community (Blog/Discussion)"
- label: "SAP Help Portal"
- label: "SAP Notes/KBA"
- label: "wdi5 Testing Documentation"
- label: "UI5 Tooling Documentation"
- label: "Cloud MTA Build Tool Documentation"
- label: "UI5 Web Components Documentation"
- label: "SAP Cloud SDK Documentation"
- label: "SAP Cloud SDK AI Documentation"
- label: "Not sure"
- label: "Other (please specify in Expected Document)"
validations:
required: true
- type: textarea
id: expected-document
attributes:
label: Expected Document/Page
description: Provide details about the specific document, page, or section you expected to find
placeholder: |
- Document Title: "Authentication and Authorization in CAP"
- URL (if known): https://cap.cloud.sap/docs/guides/security/
- Section: JWT Configuration
- Last seen: 2024-01-15
validations:
required: true
- type: checkboxes
id: sap-product-area
attributes:
label: SAP Product/Technology Areas
description: Which SAP technology areas does this relate to? (Select all that apply)
options:
- label: "SAP CAP (Cloud Application Programming)"
- label: "SAPUI5/OpenUI5"
- label: "SAP Fiori"
- label: "SAP BTP (Business Technology Platform)"
- label: "SAP S/4HANA"
- label: "SAP Analytics Cloud"
- label: "SAP Integration Suite"
- label: "SAP Mobile Development"
- label: "SAP Testing (wdi5, etc.)"
- label: "SAP Build"
- label: "Cross-platform/General"
- label: "Other"
validations:
required: true
- type: textarea
id: alternative-searches
attributes:
label: Alternative Search Terms Tried
description: What other search terms or variations did you try?
placeholder: |
- "JWT authentication CAP"
- "token based auth SAP"
- "security configuration"
- "OAuth2 CAP"
render: text
validations:
required: false
- type: textarea
id: document-reference
attributes:
label: Document Reference/URL
description: If you have a direct link to the document that should be indexed, please provide it
placeholder: "https://help.sap.com/docs/..."
validations:
required: false
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Any other information that might help us locate and index the missing content
placeholder: |
- When did you last see this documentation?
- Is this a new feature that might not be indexed yet?
- Are there related documents that were found correctly?
validations:
required: false
```
--------------------------------------------------------------------------------
/src/lib/url-generation/sapui5.ts:
--------------------------------------------------------------------------------
```typescript
/**
* URL generation for SAPUI5 documentation sources
* Handles SAPUI5 guides, API docs, and samples
*/
import { BaseUrlGenerator, UrlGenerationContext } from './BaseUrlGenerator.js';
import { FrontmatterData } from './utils.js';
import { DocUrlConfig } from '../metadata.js';
export interface SapUi5UrlOptions {
relFile: string;
content: string;
config: DocUrlConfig;
libraryId: string;
}
/**
* SAPUI5 URL Generator
* Handles SAPUI5 guides, OpenUI5 API docs, and samples with different URL patterns
*/
export class SapUi5UrlGenerator extends BaseUrlGenerator {
protected generateSourceSpecificUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
switch (this.libraryId) {
case '/sapui5':
return this.generateSapUi5Url(context);
case '/openui5-api':
return this.generateOpenUi5ApiUrl(context);
case '/openui5-samples':
return this.generateOpenUi5SampleUrl(context);
default:
return this.generateSapUi5Url(context);
}
}
/**
* Generate URL for SAPUI5 documentation
* SAPUI5 uses topic-based URLs with # fragments
*/
private generateSapUi5Url(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
// SAPUI5 docs often have topic IDs in frontmatter
const topicId = context.frontmatter.id || context.frontmatter.topic;
if (topicId) {
return `${this.config.baseUrl}/#/topic/${topicId}`;
}
// SAPUI5 docs also use HTML comments with loio pattern: <!-- loio{id} -->
const loioMatch = context.content?.match(/<!--\s*loio([a-f0-9]+)\s*-->/);
if (loioMatch) {
return `${this.config.baseUrl}/#/topic/${loioMatch[1]}`;
}
// Extract topic ID from filename if following SAPUI5 conventions (UUID pattern)
const topicIdMatch = context.relFile.match(/([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})/i);
if (topicIdMatch) {
return `${this.config.baseUrl}/#/topic/${topicIdMatch[1]}`;
}
return null; // Let fallback handle it
}
/**
* Generate URL for OpenUI5 API documentation
* API docs use control/namespace-based URLs
*/
private generateOpenUi5ApiUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
// Extract control name from file path (e.g., src/sap/m/Button.js -> sap.m.Button)
const pathMatch = context.relFile.match(/src\/(sap\/[^\/]+\/[^\/]+)\.js$/);
if (pathMatch) {
const controlPath = pathMatch[1].replace(/\//g, '.');
return `${this.config.baseUrl}/#/api/${controlPath}`;
}
// Alternative pattern matching
const controlMatch = context.relFile.match(/\/([^\/]+)\.js$/);
if (controlMatch) {
const controlName = controlMatch[1];
// Check if it's a full namespace path
if (controlName.includes('.')) {
return `${this.config.baseUrl}/#/api/${controlName}`;
}
// Try to extract namespace from content
const namespaceMatch = context.content.match(/sap\.([a-z]+\.[A-Za-z0-9_]+)/);
if (namespaceMatch) {
return `${this.config.baseUrl}/#/api/${namespaceMatch[0]}`;
}
// Fallback to control name only
return `${this.config.baseUrl}/#/api/${controlName}`;
}
return null; // Let fallback handle it
}
/**
* Generate URL for OpenUI5 samples
* Samples use sample-specific paths without # prefix
*/
private generateOpenUi5SampleUrl(context: UrlGenerationContext & {
frontmatter: FrontmatterData;
section: string;
anchor: string | null;
}): string | null {
// Extract sample ID from path patterns like:
// /src/sap.m/test/sap/m/demokit/sample/ButtonWithBadge/Component.js
const sampleMatch = context.relFile.match(/sample\/([^\/]+)\/([^\/]+)$/);
if (sampleMatch) {
const [, sampleName, fileName] = sampleMatch;
// For samples, we construct the sample entity URL without # prefix
return `${this.config.baseUrl}/entity/sap.m.Button/sample/sap.m.sample.${sampleName}`;
}
// Alternative pattern for samples
const buttonSampleMatch = context.relFile.match(/\/([^\/]+)\/test\/sap\/m\/demokit\/sample\/([^\/]+)\//);
if (buttonSampleMatch) {
const [, controlLibrary, sampleName] = buttonSampleMatch;
return `${this.config.baseUrl}/entity/sap.${controlLibrary}.Button/sample/sap.${controlLibrary}.sample.${sampleName}`;
}
return null; // Let fallback handle it
}
}
// Convenience functions for backward compatibility
/**
* Generate URL for SAPUI5 documentation using the class-based approach
*/
export function generateSapUi5Url(options: SapUi5UrlOptions): string | null {
const generator = new SapUi5UrlGenerator(options.libraryId, options.config);
return generator.generateUrl(options);
}
/**
* Generate URL for OpenUI5 API documentation
*/
export function generateOpenUi5ApiUrl(options: SapUi5UrlOptions): string | null {
const generator = new SapUi5UrlGenerator('/openui5-api', options.config);
return generator.generateUrl(options);
}
/**
* Generate URL for OpenUI5 samples
*/
export function generateOpenUi5SampleUrl(options: SapUi5UrlOptions): string | null {
const generator = new SapUi5UrlGenerator('/openui5-samples', options.config);
return generator.generateUrl(options);
}
/**
* Main dispatcher for UI5-related URL generation
*/
export function generateUi5UrlForLibrary(options: SapUi5UrlOptions): string | null {
const generator = new SapUi5UrlGenerator(options.libraryId, options.config);
return generator.generateUrl(options);
}
```
--------------------------------------------------------------------------------
/src/lib/metadata.ts:
--------------------------------------------------------------------------------
```typescript
// Metadata and configuration management
import fs from "fs";
import path from "path";
import { CONFIG } from "./config.js";
export type SourceMeta = {
id: string;
type: string;
lang?: string;
boost?: number;
tags?: string[];
description?: string;
libraryId?: string;
sourcePath?: string;
baseUrl?: string;
pathPattern?: string;
anchorStyle?: 'docsify' | 'github' | 'custom';
};
export type DocUrlConfig = {
baseUrl: string;
pathPattern: string;
anchorStyle: 'docsify' | 'github' | 'custom';
};
export type Metadata = {
version: number;
updated_at: string;
description?: string;
sources: SourceMeta[];
acronyms?: Record<string, string[]>;
synonyms?: Array<{ from: string; to: string[] }>;
contextBoosts?: Record<string, Record<string, number>>;
libraryMappings?: Record<string, string>;
contextEmojis?: Record<string, string>;
};
let META: Metadata | null = null;
let BOOSTS: Record<string, number> = {};
let SYNONYM_MAP: Record<string, string[]> = {};
export function loadMetadata(metaPath?: string): Metadata {
if (META) return META;
const finalPath = metaPath || path.resolve(process.cwd(), CONFIG.METADATA_PATH);
try {
const raw = fs.readFileSync(finalPath, "utf8");
META = JSON.parse(raw) as Metadata;
// Build source boosts map
BOOSTS = Object.fromEntries(
(META.sources || []).map(s => [s.id, s.boost || 0])
);
// Build synonym map (including acronyms)
const syn: Record<string, string[]> = {};
for (const [k, arr] of Object.entries(META.acronyms || {})) {
syn[k.toLowerCase()] = arr;
}
for (const s of META.synonyms || []) {
syn[s.from.toLowerCase()] = s.to;
}
SYNONYM_MAP = syn;
console.log(`✅ Metadata loaded: ${META.sources.length} sources, ${Object.keys(SYNONYM_MAP).length} synonyms`);
return META;
} catch (error) {
console.warn(`⚠️ Could not load metadata from ${finalPath}, using defaults:`, error);
// Fallback to minimal defaults
META = {
version: 1,
updated_at: new Date().toISOString(),
sources: [],
synonyms: [],
acronyms: {}
};
BOOSTS = {};
SYNONYM_MAP = {};
return META;
}
}
export function getSourceBoosts(): Record<string, number> {
if (!META) loadMetadata();
return BOOSTS;
}
export function expandQueryTerms(q: string): string[] {
if (!META) loadMetadata();
const terms = new Set<string>();
const low = q.toLowerCase();
terms.add(q);
// Apply synonyms and acronyms
for (const [from, toList] of Object.entries(SYNONYM_MAP)) {
if (low.includes(from)) {
for (const t of toList) {
terms.add(q.replace(new RegExp(from, "ig"), t));
}
}
}
return Array.from(terms);
}
export function getMetadata(): Metadata {
if (!META) loadMetadata();
return META!;
}
// Get documentation URL configuration for a library
export function getDocUrlConfig(libraryId: string): DocUrlConfig | null {
if (!META) loadMetadata();
if (!META) return null;
const source = META.sources.find(s => s.libraryId === libraryId);
if (!source || !source.baseUrl || !source.pathPattern || !source.anchorStyle) {
return null;
}
return {
baseUrl: source.baseUrl,
pathPattern: source.pathPattern,
anchorStyle: source.anchorStyle
};
}
// Get all documentation URL configurations
export function getAllDocUrlConfigs(): Record<string, DocUrlConfig> {
if (!META) loadMetadata();
if (!META) return {};
const configs: Record<string, DocUrlConfig> = {};
for (const source of META.sources) {
if (source.libraryId && source.baseUrl && source.pathPattern && source.anchorStyle) {
configs[source.libraryId] = {
baseUrl: source.baseUrl,
pathPattern: source.pathPattern,
anchorStyle: source.anchorStyle
};
}
}
return configs;
}
// Get source path for a library
export function getSourcePath(libraryId: string): string | null {
if (!META) loadMetadata();
if (!META) return null;
const source = META.sources.find(s => s.libraryId === libraryId);
return source?.sourcePath || null;
}
// Get all source paths
export function getAllSourcePaths(): Record<string, string> {
if (!META) loadMetadata();
if (!META) return {};
const paths: Record<string, string> = {};
for (const source of META.sources) {
if (source.libraryId && source.sourcePath) {
paths[source.libraryId] = source.sourcePath;
}
}
return paths;
}
// Get context boosts for a specific context
export function getContextBoosts(context: string): Record<string, number> {
if (!META) loadMetadata();
if (!META) return {};
return META.contextBoosts?.[context] || {};
}
// Get all context boosts
export function getAllContextBoosts(): Record<string, Record<string, number>> {
if (!META) loadMetadata();
if (!META) return {};
return META.contextBoosts || {};
}
// Get library mapping for source ID
export function getLibraryMapping(sourceId: string): string | null {
if (!META) loadMetadata();
if (!META) return null;
return META.libraryMappings?.[sourceId] || null;
}
// Get all library mappings
export function getAllLibraryMappings(): Record<string, string> {
if (!META) loadMetadata();
if (!META) return {};
return META.libraryMappings || {};
}
// Get context emoji
export function getContextEmoji(context: string): string {
if (!META) loadMetadata();
if (!META) return '🔍';
return META.contextEmojis?.[context] || '🔍';
}
// Get all context emojis
export function getAllContextEmojis(): Record<string, string> {
if (!META) loadMetadata();
if (!META) return {};
return META.contextEmojis || {};
}
// Get source by library ID
export function getSourceByLibraryId(libraryId: string): SourceMeta | null {
if (!META) loadMetadata();
if (!META) return null;
return META.sources.find(s => s.libraryId === libraryId) || null;
}
// Get source by ID
export function getSourceById(id: string): SourceMeta | null {
if (!META) loadMetadata();
if (!META) return null;
return META.sources.find(s => s.id === id) || null;
}
```
--------------------------------------------------------------------------------
/docs/ABAP-STANDARD-INTEGRATION.md:
--------------------------------------------------------------------------------
```markdown
# ABAP Documentation - Standard System Integration
## ✅ **Integration Complete**
ABAP documentation is now integrated as a **standard source** in the MCP system, just like UI5, CAP, and other sources. No special tools needed!
## **What Was Added**
### **1. Standard Metadata Configuration**
```json
// src/metadata.json
{
"id": "abap-docs",
"type": "documentation",
"boost": 0.95,
"tags": ["abap", "keyword-documentation", "language-reference"],
"libraryId": "/abap-docs",
"sourcePath": "abap-docs/docs/7.58/md",
"baseUrl": "https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US"
}
```
### **2. Standard Index Configuration**
```typescript
// scripts/build-index.ts
{
repoName: "abap-docs",
absDir: join("sources", "abap-docs", "docs", "7.58", "md"),
id: "/abap-docs",
name: "ABAP Keyword Documentation",
filePattern: "*.md", // Individual files, not bundles!
type: "markdown"
}
```
### **3. Custom URL Generator**
```typescript
// src/lib/url-generation/abap.ts
export class AbapUrlGenerator extends BaseUrlGenerator {
generateUrl(context): string {
// Converts: abeninline_declarations.md
// To: https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/abeninline_declarations.htm
}
}
```
### **4. Git Submodule**
```bash
# .gitmodules (already exists)
[submodule "sources/abap-docs"]
path = sources/abap-docs
url = https://github.com/marianfoo/abap-docs.git
branch = main
```
## **How It Works**
### **🔍 Search Integration**
Uses the **standard `search`** tool - no special ABAP tools needed!
```javascript
// Query examples that will find ABAP docs:
"SELECT statements in ABAP" → Finds individual SELECT documentation files
"internal table operations" → Finds table-related ABAP files
"exception handling" → Finds TRY/CATCH documentation
"ABAP class definition" → Finds OOP documentation
```
### **📄 File Structure**
```
sources/abap-docs/docs/7.58/md/
├── abeninline_declarations.md (3KB) ← Perfect for LLMs!
├── abenselect.md (5KB) ← Individual statement docs
├── abenloop.md (4KB) ← Focused content
├── abenclass.md (8KB) ← OOP documentation
└── ... 6,000+ more individual files
```
### **🔗 URL Generation**
- `abeninline_declarations.md` → `https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/abeninline_declarations.htm`
- Works across all ABAP versions (7.52-7.58, latest)
- Direct links to official SAP documentation
## **Setup Instructions**
### **1. Initialize Submodule**
```bash
cd /Users/marianzeis/DEV/sap-docs-mcp
git submodule update --init --recursive sources/abap-docs
```
### **2. Optimize ABAP Source** (Recommended)
```bash
cd sources/abap-docs
node scripts/generate.js --version 7.58 --standard-system
```
This will:
- ✅ Fix all JavaScript links → proper SAP URLs
- ✅ Add source attribution to each file
- ✅ Optimize content structure for LLM consumption
- ✅ Create clean individual .md files (no complex bundles)
### **3. Build Index**
```bash
cd /Users/marianzeis/DEV/sap-docs-mcp
npm run build:index
```
### **4. Build FTS Database**
```bash
npm run build:fts
```
### **5. Test Integration**
```bash
npm test
curl -X POST http://localhost:3000/search \
-H "Content-Type: application/json" \
-d '{"query": "ABAP inline declarations"}'
```
## **Expected Results**
### **Standard Search Query**
```json
{
"tool": "search",
"query": "ABAP inline declarations"
}
```
### **Expected Response**
```
Found 5 results for 'ABAP inline declarations':
⚡ **Inline Declarations (ABAP 7.58)**
Data declarations directly in ABAP statements for cleaner code...
🔗 https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/abeninline_declarations.htm
📋 3KB | individual | beginner
⚡ **DATA - Inline Declaration (ABAP 7.58)**
Creating data objects inline using DATA() operator...
🔗 https://help.sap.com/doc/abapdocu_758_index_htm/7.58/en-US/abendata_inline.htm
📋 2KB | individual | intermediate
```
## **Key Benefits**
### ✅ **Standard Integration**
- **No special tools** - uses existing `search`
- **Same interface** as UI5, CAP, wdi5 sources
- **Consistent behavior** with other documentation
### ✅ **Perfect LLM Experience**
- **6,000+ individual files** (1-10KB each)
- **Direct SAP documentation URLs** for attribution
- **Clean markdown** optimized for context windows
### ✅ **High Search Quality**
- **BM25 FTS5 search** - same quality as other sources
- **Context-aware boosting** - ABAP queries get ABAP results
- **Proper scoring** integrated with general search
### ✅ **Easy Maintenance**
- **Standard build process** - same as other sources
- **No complex bundling** - simple file-based approach
- **Version support** - easy to add 7.57, 7.56, etc.
## **Multi-Version Support** (Future)
To add more ABAP versions:
```typescript
// Add to build-index.ts
{
repoName: "abap-docs",
absDir: join("sources", "abap-docs", "docs", "7.57", "md"),
id: "/abap-docs-757",
name: "ABAP Keyword Documentation 7.57"
},
{
repoName: "abap-docs",
absDir: join("sources", "abap-docs", "docs", "latest", "md"),
id: "/abap-docs-latest",
name: "ABAP Keyword Documentation (Latest)"
}
```
## **Performance Characteristics**
- **Index Size**: ~6,000 documents (vs 42,901 with specialized system)
- **Search Speed**: ~50ms (standard FTS5 performance)
- **File Sizes**: 1-10KB each (perfect for LLM consumption)
- **Memory Usage**: Standard - no special caching needed
## **Migration from Specialized Tools**
### **Old Approach (Specialized)**
```javascript
// Required separate tools
abap_search: "inline declarations"
abap_get: "abap-7.58-individual-7.58-abeninline_declarations"
```
### **New Approach (Standard)**
```javascript
// Uses standard tool like everything else
search: "ABAP inline declarations"
fetch: "/abap-docs/abeninline_declarations.md"
```
**Result: Same quality, simpler interface, standard integration!** 🚀
---
## **✅ Integration Status: COMPLETE**
ABAP documentation is now fully integrated as a standard source:
- ✅ **Metadata configured**
- ✅ **Build index updated**
- ✅ **URL generator created**
- ✅ **Submodule exists**
- ✅ **Tests added**
**Ready for production use with the standard MCP search system!**
```
--------------------------------------------------------------------------------
/docs/ARCHITECTURE.md:
--------------------------------------------------------------------------------
```markdown
# 🏗️ SAP Docs MCP Architecture
## System Overview
```mermaid
graph TD
A[User Query] --> B[MCP Server]
B --> C[Search Pipeline]
C --> D[FTS5 SQLite]
C --> E[Metadata APIs]
D --> F[BM25 Scoring]
E --> F
F --> G[Context Awareness]
G --> H[Result Formatting]
H --> I[Formatted Response]
J[Documentation Sources] --> K[Build Index]
K --> L[dist/data/index.json]
K --> M[dist/data/docs.sqlite]
L --> C
M --> D
N[src/metadata.json] --> E
```
## Core Components
### 🔍 **Search Pipeline**
1. **Query Processing**: Parse and expand user queries with synonyms/acronyms
2. **FTS5 Search**: Fast SQLite full-text search using BM25 algorithm
3. **Metadata Integration**: Apply source boosts, context awareness, and mappings
4. **Result Formatting**: Structure results with scores, context indicators, and source attribution
### 📊 **Data Flow**
```
Documentation Sources → Build Scripts → Search Artifacts → Runtime Search
(12 sources) → (2 files) → (index.json + → (BM25 +
docs.sqlite) metadata)
```
### 🎯 **Metadata-Driven Configuration**
- **Single Source**: `src/metadata.json` contains all source configurations
- **Type-Safe APIs**: 12 functions in `src/lib/metadata.ts` for configuration access
- **Runtime Loading**: Metadata loaded once at startup with graceful fallbacks
- **Zero Code Changes**: Add new sources by updating metadata.json only
## Server Architecture
### 🖥️ **Three Server Modes**
1. **Stdio MCP** (`src/server.ts`): Main server for Claude/LLM integration
2. **HTTP Server** (`src/http-server.ts`): Development and status monitoring (port 3001)
3. **Streamable HTTP** (`src/streamable-http-server.ts`): Production HTTP MCP (port 3122)
### 🔧 **MCP Tools (5 total)**
- `search`: Search across all documentation
- `fetch`: Retrieve specific documents
- `sap_community_search`: SAP Community integration
- `sap_help_search`: SAP Help Portal search
- `sap_help_get`: SAP Help content retrieval
## Performance Characteristics
### ⚡ **Search Performance**
- **Sub-second**: FTS5 provides fast full-text search
- **Scalable**: Performance consistent as documentation grows
- **Efficient**: BM25-only approach eliminates ML model overhead
- **Reliable**: No external dependencies for core search
### 📈 **Resource Usage**
- **Memory**: ~50-100MB for index and metadata
- **Disk**: ~3.5MB SQLite database + ~2MB JSON index
- **CPU**: Minimal - BM25 scoring is computationally light
- **Network**: Only for community/help portal integrations
## Documentation Sources (12 total)
### 📚 **Primary Sources**
- **SAPUI5**: Frontend framework documentation
- **CAP**: Cloud Application Programming model
- **OpenUI5 API**: Control API documentation
- **wdi5**: Testing framework documentation
### 🔧 **Supporting Sources**
- **Cloud SDK (JS/Java)**: SAP Cloud SDK documentation
- **Cloud SDK AI (JS/Java)**: AI-enhanced SDK documentation
- **UI5 Tooling**: Build and development tools
- **UI5 Web Components**: Modern web components
- **Cloud MTA Build Tool**: Multi-target application builder
## Context Awareness
### 🎯 **Context Detection**
- **UI5 Context** 🎨: Controls, Fiori, frontend development
- **CAP Context** 🏗️: CDS, entities, services, backend
- **wdi5 Context** 🧪: Testing, automation, browser testing
- **Mixed Context** 🔀: Cross-platform or unclear intent
### 📊 **Intelligent Scoring**
- **Source Boosts**: Context-specific score adjustments
- **Library Mappings**: Resolve source IDs to canonical names
- **Query Expansion**: Synonyms and acronyms for better recall
- **Penalty System**: Reduce off-context results
## Build Process
### 🔨 **Enhanced Build Pipeline**
```bash
npm run build:tsc # TypeScript compilation → dist/src/
npm run build:index # Sources → dist/data/index.json
npm run build:fts # Index → dist/data/docs.sqlite
npm run build # Complete pipeline (tsc + index + fts)
```
### 📦 **Submodule Management**
```bash
npm run setup # Complete setup with enhanced submodule handling
npm run setup:submodules # Submodule sync and update only
```
The enhanced setup script provides:
- **Shallow Clones**: `--depth 1` with `--filter=blob:none` for minimal size
- **Single Branch**: Only fetch the target branch (main/master)
- **Repository Compaction**: Aggressive GC and storage optimization
- **Fallback Handling**: Auto-retry with master if branch fails
- **Skip Nested**: `SKIP_NESTED_SUBMODULES=1` for deployment speed
### 📦 **Deployment Artifacts**
- `dist/data/index.json`: Structured documentation index
- `dist/data/docs.sqlite`: FTS5 search database
- `dist/src/`: Compiled TypeScript server code
- `src/metadata.json`: Runtime configuration
## Production Deployment
### 🚀 **PM2 Configuration**
- **3 Processes**: Proxy (18080), HTTP (3001), Streamable (3122)
- **Health Monitoring**: Multiple endpoints for status checks
- **Auto-restart**: Configurable restart policies
- **Logging**: Structured JSON logging in production
### 🔄 **CI/CD Pipeline**
1. **GitHub Actions**: Triggered on main branch push
2. **SSH Deployment**: Connect to production server
3. **Build Process**: Run complete build pipeline
4. **PM2 Restart**: Restart all processes with new code
5. **Health Validation**: Verify all endpoints responding
## Key Design Principles
### 🎯 **Simplicity First**
- **BM25-Only**: No complex ML models or external dependencies
- **SQLite**: Single-file database for easy deployment
- **Metadata-Driven**: Configuration without code changes
### 🔒 **Reliability**
- **Graceful Fallbacks**: Handle missing data and errors elegantly
- **Type Safety**: Comprehensive TypeScript interfaces
- **Testing**: Smoke tests and integration validation
### 📈 **Performance**
- **Fast Search**: Sub-second response times
- **Efficient Indexing**: Optimized FTS5 schema
- **Minimal Resources**: Low memory and CPU usage
### 🔧 **Maintainability**
- **Single Source of Truth**: Centralized configuration
- **Clear Separation**: Distinct layers for search, metadata, and presentation
- **Comprehensive Documentation**: Architecture, APIs, and deployment guides
```