This is page 1 of 2. Use http://codebase.md/tolik-unicornrider/mcp_scraper?lines=true&page={x} to view the full context.
# Directory Structure
```
├── .gitignore
├── jest.config.js
├── mcp_docs
│ ├── mcp-llms-full.txt
│ └── mcp-typescript-sdk.txt
├── package-lock.json
├── package.json
├── README.md
├── src
│ ├── cli.ts
│ ├── data_processing.test.ts
│ ├── data_processing.ts
│ └── index.ts
├── test.html
└── tsconfig.json
```
# Files
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
```
1 | node_modules
2 | build
3 |
```
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
```markdown
1 | # Website Scraper
2 |
3 | A command-line tool and MCP server for scraping websites and converting HTML to Markdown.
4 |
5 | ## Features
6 |
7 | - Extracts meaningful content from web pages using Mozilla's [Readability](https://github.com/mozilla/readability) library (the same engine used in Firefox's Reader View)
8 | - Converts clean HTML to high-quality Markdown with TurndownService
9 | - Securely handles HTML by removing potentially harmful script tags
10 | - Works as both a command-line tool and an MCP server
11 | - Supports direct conversion of local HTML files to Markdown
12 |
13 | ## Installation
14 |
15 | ```bash
16 | # Install dependencies
17 | npm install
18 |
19 | # Build the project
20 | npm run build
21 |
22 | # Optionally, install globally
23 | npm install -g .
24 | ```
25 |
26 | ## Usage
27 |
28 | ### CLI Mode
29 |
30 | ```bash
31 | # Print output to console
32 | scrape https://example.com
33 |
34 | # Save output to a file
35 | scrape https://example.com output.md
36 |
37 | # Convert a local HTML file to Markdown
38 | scrape --html-file input.html
39 |
40 | # Convert a local HTML file and save output to a file
41 | scrape --html-file input.html output.md
42 |
43 | # Show help
44 | scrape --help
45 |
46 | # Or run via npm script
47 | npm run start:cli -- https://example.com
48 | ```
49 |
50 | ### MCP Server Mode
51 |
52 | This tool can be used as a Model Context Protocol (MCP) server:
53 |
54 | ```bash
55 | # Start in MCP server mode
56 | npm start
57 | ```
58 |
59 | ## Code Structure
60 |
61 | - `src/index.ts` - Core functionality and MCP server implementation
62 | - `src/cli.ts` - Command-line interface implementation
63 | - `src/data_processing.ts` - HTML to Markdown conversion functionality
64 |
65 | ## API
66 |
67 | The tool exports the following functions:
68 |
69 | ```typescript
70 | // Scrape a website and convert to Markdown
71 | import { scrapeToMarkdown } from './build/index.js';
72 |
73 | // Convert HTML string to Markdown directly
74 | import { htmlToMarkdown } from './build/data_processing.js';
75 |
76 | async function example() {
77 | // Web scraping
78 | const markdown = await scrapeToMarkdown('https://example.com');
79 | console.log(markdown);
80 |
81 | // Direct HTML conversion
82 | const html = '<h1>Hello World</h1><p>This is <strong>bold</strong> text.</p>';
83 | const md = htmlToMarkdown(html);
84 | console.log(md);
85 | }
86 | ```
87 |
88 | ## License
89 |
90 | ISC
```
--------------------------------------------------------------------------------
/jest.config.js:
--------------------------------------------------------------------------------
```javascript
1 | export default {
2 | preset: 'ts-jest',
3 | testEnvironment: 'node',
4 | extensionsToTreatAsEsm: ['.ts'],
5 | moduleNameMapper: {
6 | '^(\\.{1,2}/.*)\\.js$': '$1',
7 | },
8 | transform: {
9 | '^.+\\.tsx?$': [
10 | 'ts-jest',
11 | {
12 | useESM: true,
13 | },
14 | ],
15 | },
16 | };
```
--------------------------------------------------------------------------------
/tsconfig.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "compilerOptions": {
3 | "target": "ES2022",
4 | "module": "Node16",
5 | "moduleResolution": "Node16",
6 | "outDir": "./build",
7 | "rootDir": "./src",
8 | "strict": true,
9 | "esModuleInterop": true,
10 | "skipLibCheck": true,
11 | "forceConsistentCasingInFileNames": true
12 | },
13 | "include": ["src/**/*"],
14 | "exclude": ["node_modules"]
15 | }
16 |
```
--------------------------------------------------------------------------------
/src/data_processing.ts:
--------------------------------------------------------------------------------
```typescript
1 | import TurndownService from "turndown";
2 |
3 | /**
4 | * Converts raw HTML to Markdown using TurndownService
5 | * Removes script tags for security and cleaner output
6 | * @param html Raw HTML string to convert
7 | * @returns Markdown string
8 | */
9 | export function htmlToMarkdown(html: string): string {
10 | // Remove script tags and their content before conversion
11 | const cleanHtml = html.replace(/<script\b[^<]*(?:(?!<\/script>)<[^<]*)*<\/script>/gi, '');
12 |
13 | const turndownService = new TurndownService({
14 | codeBlockStyle: 'fenced',
15 | emDelimiter: '_'
16 | });
17 |
18 | return turndownService.turndown(cleanHtml);
19 | }
20 |
21 |
22 |
```
--------------------------------------------------------------------------------
/test.html:
--------------------------------------------------------------------------------
```html
1 | <!DOCTYPE html>
2 | <html>
3 | <head>
4 | <title>Test HTML Document</title>
5 | <script>
6 | // This script should be removed in markdown
7 | console.log("This should not appear in the output");
8 | </script>
9 | <style>
10 | body { font-family: Arial, sans-serif; }
11 | </style>
12 | </head>
13 | <body>
14 | <h1>Test HTML Document</h1>
15 | <p>This is a <strong>test</strong> document with some <em>emphasized</em> text.</p>
16 |
17 | <h2>Features List</h2>
18 | <ul>
19 | <li>Headings</li>
20 | <li>Paragraphs</li>
21 | <li>Formatting (<strong>bold</strong>, <em>italic</em>)</li>
22 | <li>Lists</li>
23 | <li><a href="https://example.com">Links</a></li>
24 | <li>Code blocks</li>
25 | </ul>
26 |
27 | <h2>Code Example</h2>
28 | <pre><code>function test() {
29 | console.log("Hello, world!");
30 | return true;
31 | }</code></pre>
32 |
33 | <script>
34 | // This inline script should also be removed
35 | document.getElementById("test").innerHTML = "Modified content";
36 | </script>
37 | </body>
38 | </html>
```
--------------------------------------------------------------------------------
/package.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "name": "website-scraper",
3 | "version": "1.0.0",
4 | "description": "Command-line tool and MCP server for scraping websites and converting HTML to Markdown",
5 | "main": "build/index.js",
6 | "keywords": [],
7 | "author": "",
8 | "license": "ISC",
9 | "type": "module",
10 | "bin": {
11 | "scrape": "./build/cli.js"
12 | },
13 | "scripts": {
14 | "build": "tsc && node -e \"require('fs').chmodSync('build/cli.js', '755')\"",
15 | "start": "npm run build && node build/index.js",
16 | "start:cli": "npm run build && node build/cli.js",
17 | "test": "node --experimental-vm-modules node_modules/jest/bin/jest.js",
18 | "test:watch": "node --experimental-vm-modules node_modules/jest/bin/jest.js --watch"
19 | },
20 | "files": [
21 | "build"
22 | ],
23 | "dependencies": {
24 | "@modelcontextprotocol/sdk": "^1.6.1",
25 | "@mozilla/readability": "^0.6.0",
26 | "jsdom": "^26.0.0",
27 | "node-html-markdown": "^1.3.0",
28 | "turndown": "^7.2.0",
29 | "zod": "^3.24.2"
30 | },
31 | "devDependencies": {
32 | "@types/jest": "^29.5.14",
33 | "@types/jsdom": "^21.1.7",
34 | "@types/node": "^22.13.9",
35 | "@types/turndown": "^5.0.5",
36 | "jest": "^29.7.0",
37 | "ts-jest": "^29.2.6",
38 | "typescript": "^5.8.2"
39 | }
40 | }
41 |
```
--------------------------------------------------------------------------------
/src/cli.ts:
--------------------------------------------------------------------------------
```typescript
1 | #!/usr/bin/env node
2 |
3 | import { scrapeToMarkdown } from './index.js';
4 | import { htmlToMarkdown } from './data_processing.js';
5 | import fs from 'fs/promises';
6 | import path from 'path';
7 |
8 | /**
9 | * CLI handler for website scraping and HTML conversion
10 | */
11 | async function runCli() {
12 | const args = process.argv.slice(2);
13 |
14 | // Basic help
15 | if (args.includes('--help') || args.includes('-h') || args.length === 0) {
16 | console.log(`
17 | Usage: scrape <url> [output_file]
18 | or: scrape --html-file <html_file> [output_file]
19 |
20 | Options:
21 | --help, -h Show this help message
22 | --html-file Convert a local HTML file to Markdown instead of scraping a URL
23 |
24 | Arguments:
25 | url/html_file URL to scrape or path to HTML file (required)
26 | output_file Output file path (optional, if not provided output is printed to stdout)
27 | `);
28 | process.exit(0);
29 | }
30 |
31 | // Check if we're converting a local HTML file
32 | const htmlFileMode = args.includes('--html-file');
33 | let targetInput: string;
34 | let outputFile: string | undefined;
35 |
36 | if (htmlFileMode) {
37 | const fileArgIndex = args.indexOf('--html-file') + 1;
38 | if (fileArgIndex >= args.length) {
39 | console.error('Error: No HTML file specified after --html-file');
40 | process.exit(1);
41 | }
42 | targetInput = args[fileArgIndex];
43 | outputFile = args[fileArgIndex + 1];
44 | } else {
45 | targetInput = args[0];
46 | outputFile = args[1];
47 | }
48 |
49 | try {
50 | let markdown: string;
51 |
52 | if (htmlFileMode) {
53 | // Direct HTML file to Markdown conversion
54 | try {
55 | const htmlContent = await fs.readFile(targetInput, 'utf-8');
56 | markdown = htmlToMarkdown(htmlContent);
57 | } catch (err: any) {
58 | throw new Error(`Error reading HTML file: ${err.message}`);
59 | }
60 | } else {
61 | // Web scraping mode
62 | markdown = await scrapeToMarkdown(targetInput);
63 | }
64 |
65 | if (outputFile) {
66 | await fs.writeFile(outputFile, markdown);
67 | console.error(`Markdown saved to: ${outputFile}`);
68 | } else {
69 | console.log(markdown);
70 | }
71 | } catch (error: any) {
72 | console.error(error.message);
73 | process.exit(1);
74 | }
75 | }
76 |
77 | // Run the CLI
78 | runCli().catch((error) => {
79 | console.error("Fatal error:", error);
80 | process.exit(1);
81 | });
```
--------------------------------------------------------------------------------
/src/index.ts:
--------------------------------------------------------------------------------
```typescript
1 | #!/usr/bin/env node
2 |
3 | import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
4 | import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
5 | import { z } from "zod";
6 | import { Readability } from '@mozilla/readability';
7 | import { JSDOM } from 'jsdom';
8 | import { htmlToMarkdown } from './data_processing.js';
9 |
10 | /**
11 | * Scrapes a website and converts the HTML content to markdown
12 | * @param url The URL to scrape
13 | * @returns The markdown content as a string
14 | */
15 | export async function scrapeToMarkdown(url: string): Promise<string> {
16 | try {
17 | // Fetch the HTML content from the provided URL with proper headers
18 | const response = await fetch(url, {
19 | headers: {
20 | 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36',
21 | 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
22 | 'Accept-Language': 'en-US,en;q=0.5',
23 | }
24 | });
25 |
26 | if (!response.ok) {
27 | throw new Error(`Failed to fetch URL: ${response.status}`);
28 | }
29 |
30 | // Get content type to check encoding
31 | const contentType = response.headers.get('content-type') || '';
32 | const htmlContent = await response.text();
33 |
34 | // Parse the HTML using JSDOM with the URL to resolve relative links
35 | const dom = new JSDOM(htmlContent, {
36 | url,
37 | pretendToBeVisual: true, // This helps with some interactive content
38 | });
39 |
40 | // Extract the main content using Readability
41 | const reader = new Readability(dom.window.document);
42 | const article = reader.parse();
43 |
44 | if (!article || !article.content) {
45 | throw new Error("Failed to parse article content");
46 | }
47 |
48 | // Convert the cleaned article HTML to Markdown using htmlToMarkdown
49 | let markdown = htmlToMarkdown(article.content);
50 |
51 | // Simple post-processing to improve code blocks with language hints
52 | markdown = markdown.replace(/```\n(class|function|import|const|let|var|if|for|while)/g, '```javascript\n$1');
53 | markdown = markdown.replace(/```\n(def|class|import|from|with|if|for|while)(\s+)/g, '```python\n$1$2');
54 |
55 | return markdown;
56 | } catch (error: any) {
57 | throw new Error(`Scraping error: ${error.message}`);
58 | }
59 | }
60 |
61 |
62 |
63 |
64 | // Create an MCP server instance
65 | const server = new McpServer({
66 | name: "ScrapeServer",
67 | version: "1.0.0"
68 | });
69 |
70 | // Register the "scrape-to-markdown" tool
71 | server.tool(
72 | "scrape-to-markdown",
73 | { url: z.string().url() },
74 | async ({ url }) => {
75 | try {
76 | const markdown = await scrapeToMarkdown(url);
77 |
78 | // Return the markdown as the tool result
79 | return {
80 | content: [{ type: "text", text: markdown }]
81 | };
82 | } catch (error: any) {
83 | // Handle errors gracefully
84 | return {
85 | content: [{ type: "text", text: `Error: ${error.message}` }],
86 | isError: true
87 | };
88 | }
89 | }
90 | );
91 |
92 | // Start the MCP server using stdio transport
93 | // const transport = new StdioServerTransport();
94 | // await server.connect(transport);
95 |
96 | async function main() {
97 | // Create a stdio transport for communication
98 | // This allows the server to communicate with clients via standard input/output
99 | const transport = new StdioServerTransport();
100 |
101 | // Connect the server to the transport
102 | // This starts listening for incoming messages and enables communication
103 | await server.connect(transport);
104 |
105 | // Log a message to indicate the server is running
106 | // Note: Using console.error instead of console.log because stdout is used for MCP communication
107 | console.error("Weather MCP Server running on stdio");
108 | }
109 |
110 | // Call the main function and handle any fatal errors
111 | main().catch((error) => {
112 | console.error("Fatal error in main():", error);
113 | process.exit(1); // Exit with error code 1 if there's a fatal error
114 | });
115 |
```
--------------------------------------------------------------------------------
/src/data_processing.test.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { htmlToMarkdown } from './data_processing.js';
2 |
3 | describe('htmlToMarkdown', () => {
4 | test('converts basic HTML to markdown', () => {
5 | const html = '<p>Hello world</p>';
6 | expect(htmlToMarkdown(html)).toBe('Hello world');
7 | });
8 |
9 | test('converts headings', () => {
10 | const html = '<h1>Title</h1><h2>Subtitle</h2>';
11 | expect(htmlToMarkdown(html)).toBe('Title\n=====\n\nSubtitle\n--------');
12 | });
13 |
14 | test('converts emphasis using underscores', () => {
15 | const html = '<em>emphasized text</em>';
16 | expect(htmlToMarkdown(html)).toBe('_emphasized text_');
17 | });
18 |
19 | test('converts strong', () => {
20 | const html = '<strong>bold text</strong>';
21 | expect(htmlToMarkdown(html)).toBe('**bold text**');
22 | });
23 |
24 | test('converts code blocks with fencing', () => {
25 | const html = '<pre><code>const x = 1;</code></pre>';
26 | expect(htmlToMarkdown(html)).toBe('```\nconst x = 1;\n```');
27 | });
28 |
29 | test('converts links', () => {
30 | const html = '<a href="https://example.com">Example</a>';
31 | expect(htmlToMarkdown(html)).toBe('[Example](https://example.com)');
32 | });
33 |
34 | test('converts lists', () => {
35 | const html = '<ul><li>Item 1</li><li>Item 2</li></ul>';
36 | expect(htmlToMarkdown(html)).toBe('* Item 1\n* Item 2');
37 | });
38 |
39 | test('converts nested HTML structures', () => {
40 | const html = '<div><h1>Title</h1><p>Paragraph with <strong>bold</strong> and <em>emphasis</em>.</p></div>';
41 | expect(htmlToMarkdown(html)).toBe('Title\n=====\n\nParagraph with **bold** and _emphasis_.');
42 | });
43 |
44 | test('handles empty input', () => {
45 | expect(htmlToMarkdown('')).toBe('');
46 | });
47 |
48 | test('handles input with no HTML tags', () => {
49 | const text = 'Plain text without any HTML';
50 | expect(htmlToMarkdown(text)).toBe('Plain text without any HTML');
51 | });
52 |
53 | test('handles complex HTML page with script tags', () => {
54 | const html = `
55 | <!DOCTYPE html>
56 | <html>
57 | <head>
58 | <title>Test Page</title>
59 | <script>
60 | console.log("This script should be removed");
61 | function test() {
62 | return "Hello World";
63 | }
64 | </script>
65 | <style>
66 | body { font-family: Arial, sans-serif; }
67 | </style>
68 | </head>
69 | <body>
70 | <header>
71 | <h1>Page Header</h1>
72 | <nav>
73 | <ul>
74 | <li><a href="#section1">Section 1</a></li>
75 | <li><a href="#section2">Section 2</a></li>
76 | </ul>
77 | </nav>
78 | </header>
79 | <main>
80 | <section id="section1">
81 | <h2>Section 1 Title</h2>
82 | <p>This is <em>emphasized</em> and <strong>strong</strong> text.</p>
83 | <script>document.write("This should not appear in markdown");</script>
84 | <pre><code>const codeExample = "This should be formatted as code";</code></pre>
85 | </section>
86 | <section id="section2">
87 | <h2>Section 2 Title</h2>
88 | <p>Another paragraph with a <a href="https://example.com">link</a>.</p>
89 | <table>
90 | <tr>
91 | <th>Header 1</th>
92 | <th>Header 2</th>
93 | </tr>
94 | <tr>
95 | <td>Cell 1</td>
96 | <td>Cell 2</td>
97 | </tr>
98 | </table>
99 | </section>
100 | </main>
101 | <footer>
102 | <p>Page Footer</p>
103 | <script>
104 | // This script should also be removed
105 | const footer = document.querySelector('footer');
106 | </script>
107 | </footer>
108 | </body>
109 | </html>
110 | `;
111 |
112 | const expectedMarkdown = `Test Page body { font-family: Arial, sans-serif; }
113 |
114 | Page Header
115 | ===========
116 |
117 | * [Section 1](#section1)
118 | * [Section 2](#section2)
119 |
120 | Section 1 Title
121 | ---------------
122 |
123 | This is _emphasized_ and **strong** text.
124 |
125 | \`\`\`
126 | const codeExample = "This should be formatted as code";
127 | \`\`\`
128 |
129 | Section 2 Title
130 | ---------------
131 |
132 | Another paragraph with a [link](https://example.com).
133 |
134 | Header 1
135 |
136 | Header 2
137 |
138 | Cell 1
139 |
140 | Cell 2
141 |
142 | Page Footer`.trim();
143 |
144 | const output = htmlToMarkdown(html).trim();
145 | expect(output).toBe(expectedMarkdown);
146 |
147 | });
148 | });
```
--------------------------------------------------------------------------------
/mcp_docs/mcp-typescript-sdk.txt:
--------------------------------------------------------------------------------
```
1 | # MCP TypeScript SDK  
2 |
3 | ## Table of Contents
4 | - [Overview](#overview)
5 | - [Installation](#installation)
6 | - [Quickstart](#quickstart)
7 | - [What is MCP?](#what-is-mcp)
8 | - [Core Concepts](#core-concepts)
9 | - [Server](#server)
10 | - [Resources](#resources)
11 | - [Tools](#tools)
12 | - [Prompts](#prompts)
13 | - [Running Your Server](#running-your-server)
14 | - [stdio](#stdio)
15 | - [HTTP with SSE](#http-with-sse)
16 | - [Testing and Debugging](#testing-and-debugging)
17 | - [Examples](#examples)
18 | - [Echo Server](#echo-server)
19 | - [SQLite Explorer](#sqlite-explorer)
20 | - [Advanced Usage](#advanced-usage)
21 | - [Low-Level Server](#low-level-server)
22 | - [Writing MCP Clients](#writing-mcp-clients)
23 | - [Server Capabilities](#server-capabilities)
24 |
25 | ## Overview
26 |
27 | The Model Context Protocol allows applications to provide context for LLMs in a standardized way, separating the concerns of providing context from the actual LLM interaction. This TypeScript SDK implements the full MCP specification, making it easy to:
28 |
29 | - Build MCP clients that can connect to any MCP server
30 | - Create MCP servers that expose resources, prompts and tools
31 | - Use standard transports like stdio and SSE
32 | - Handle all MCP protocol messages and lifecycle events
33 |
34 | ## Installation
35 |
36 | ```bash
37 | npm install @modelcontextprotocol/sdk
38 | ```
39 |
40 | ## Quick Start
41 |
42 | Let's create a simple MCP server that exposes a calculator tool and some data:
43 |
44 | ```typescript
45 | import { McpServer, ResourceTemplate } from "@modelcontextprotocol/sdk/server/mcp.js";
46 | import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
47 | import { z } from "zod";
48 |
49 | // Create an MCP server
50 | const server = new McpServer({
51 | name: "Demo",
52 | version: "1.0.0"
53 | });
54 |
55 | // Add an addition tool
56 | server.tool("add",
57 | { a: z.number(), b: z.number() },
58 | async ({ a, b }) => ({
59 | content: [{ type: "text", text: String(a + b) }]
60 | })
61 | );
62 |
63 | // Add a dynamic greeting resource
64 | server.resource(
65 | "greeting",
66 | new ResourceTemplate("greeting://{name}", { list: undefined }),
67 | async (uri, { name }) => ({
68 | contents: [{
69 | uri: uri.href,
70 | text: `Hello, ${name}!`
71 | }]
72 | })
73 | );
74 |
75 | // Start receiving messages on stdin and sending messages on stdout
76 | const transport = new StdioServerTransport();
77 | await server.connect(transport);
78 | ```
79 |
80 | ## What is MCP?
81 |
82 | The [Model Context Protocol (MCP)](https://modelcontextprotocol.io) lets you build servers that expose data and functionality to LLM applications in a secure, standardized way. Think of it like a web API, but specifically designed for LLM interactions. MCP servers can:
83 |
84 | - Expose data through **Resources** (think of these sort of like GET endpoints; they are used to load information into the LLM's context)
85 | - Provide functionality through **Tools** (sort of like POST endpoints; they are used to execute code or otherwise produce a side effect)
86 | - Define interaction patterns through **Prompts** (reusable templates for LLM interactions)
87 | - And more!
88 |
89 | ## Core Concepts
90 |
91 | ### Server
92 |
93 | The McpServer is your core interface to the MCP protocol. It handles connection management, protocol compliance, and message routing:
94 |
95 | ```typescript
96 | const server = new McpServer({
97 | name: "My App",
98 | version: "1.0.0"
99 | });
100 | ```
101 |
102 | ### Resources
103 |
104 | Resources are how you expose data to LLMs. They're similar to GET endpoints in a REST API - they provide data but shouldn't perform significant computation or have side effects:
105 |
106 | ```typescript
107 | // Static resource
108 | server.resource(
109 | "config",
110 | "config://app",
111 | async (uri) => ({
112 | contents: [{
113 | uri: uri.href,
114 | text: "App configuration here"
115 | }]
116 | })
117 | );
118 |
119 | // Dynamic resource with parameters
120 | server.resource(
121 | "user-profile",
122 | new ResourceTemplate("users://{userId}/profile", { list: undefined }),
123 | async (uri, { userId }) => ({
124 | contents: [{
125 | uri: uri.href,
126 | text: `Profile data for user ${userId}`
127 | }]
128 | })
129 | );
130 | ```
131 |
132 | ### Tools
133 |
134 | Tools let LLMs take actions through your server. Unlike resources, tools are expected to perform computation and have side effects:
135 |
136 | ```typescript
137 | // Simple tool with parameters
138 | server.tool(
139 | "calculate-bmi",
140 | {
141 | weightKg: z.number(),
142 | heightM: z.number()
143 | },
144 | async ({ weightKg, heightM }) => ({
145 | content: [{
146 | type: "text",
147 | text: String(weightKg / (heightM * heightM))
148 | }]
149 | })
150 | );
151 |
152 | // Async tool with external API call
153 | server.tool(
154 | "fetch-weather",
155 | { city: z.string() },
156 | async ({ city }) => {
157 | const response = await fetch(`https://api.weather.com/${city}`);
158 | const data = await response.text();
159 | return {
160 | content: [{ type: "text", text: data }]
161 | };
162 | }
163 | );
164 | ```
165 |
166 | ### Prompts
167 |
168 | Prompts are reusable templates that help LLMs interact with your server effectively:
169 |
170 | ```typescript
171 | server.prompt(
172 | "review-code",
173 | { code: z.string() },
174 | ({ code }) => ({
175 | messages: [{
176 | role: "user",
177 | content: {
178 | type: "text",
179 | text: `Please review this code:\n\n${code}`
180 | }
181 | }]
182 | })
183 | );
184 | ```
185 |
186 | ## Running Your Server
187 |
188 | MCP servers in TypeScript need to be connected to a transport to communicate with clients. How you start the server depends on the choice of transport:
189 |
190 | ### stdio
191 |
192 | For command-line tools and direct integrations:
193 |
194 | ```typescript
195 | import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
196 | import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
197 |
198 | const server = new McpServer({
199 | name: "example-server",
200 | version: "1.0.0"
201 | });
202 |
203 | // ... set up server resources, tools, and prompts ...
204 |
205 | const transport = new StdioServerTransport();
206 | await server.connect(transport);
207 | ```
208 |
209 | ### HTTP with SSE
210 |
211 | For remote servers, start a web server with a Server-Sent Events (SSE) endpoint, and a separate endpoint for the client to send its messages to:
212 |
213 | ```typescript
214 | import express from "express";
215 | import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
216 | import { SSEServerTransport } from "@modelcontextprotocol/sdk/server/sse.js";
217 |
218 | const server = new McpServer({
219 | name: "example-server",
220 | version: "1.0.0"
221 | });
222 |
223 | // ... set up server resources, tools, and prompts ...
224 |
225 | const app = express();
226 |
227 | app.get("/sse", async (req, res) => {
228 | const transport = new SSEServerTransport("/messages", res);
229 | await server.connect(transport);
230 | });
231 |
232 | app.post("/messages", async (req, res) => {
233 | // Note: to support multiple simultaneous connections, these messages will
234 | // need to be routed to a specific matching transport. (This logic isn't
235 | // implemented here, for simplicity.)
236 | await transport.handlePostMessage(req, res);
237 | });
238 |
239 | app.listen(3001);
240 | ```
241 |
242 | ### Testing and Debugging
243 |
244 | To test your server, you can use the [MCP Inspector](https://github.com/modelcontextprotocol/inspector). See its README for more information.
245 |
246 | ## Examples
247 |
248 | ### Echo Server
249 |
250 | A simple server demonstrating resources, tools, and prompts:
251 |
252 | ```typescript
253 | import { McpServer, ResourceTemplate } from "@modelcontextprotocol/sdk/server/mcp.js";
254 | import { z } from "zod";
255 |
256 | const server = new McpServer({
257 | name: "Echo",
258 | version: "1.0.0"
259 | });
260 |
261 | server.resource(
262 | "echo",
263 | new ResourceTemplate("echo://{message}", { list: undefined }),
264 | async (uri, { message }) => ({
265 | contents: [{
266 | uri: uri.href,
267 | text: `Resource echo: ${message}`
268 | }]
269 | })
270 | );
271 |
272 | server.tool(
273 | "echo",
274 | { message: z.string() },
275 | async ({ message }) => ({
276 | content: [{ type: "text", text: `Tool echo: ${message}` }]
277 | })
278 | );
279 |
280 | server.prompt(
281 | "echo",
282 | { message: z.string() },
283 | ({ message }) => ({
284 | messages: [{
285 | role: "user",
286 | content: {
287 | type: "text",
288 | text: `Please process this message: ${message}`
289 | }
290 | }]
291 | })
292 | );
293 | ```
294 |
295 | ### SQLite Explorer
296 |
297 | A more complex example showing database integration:
298 |
299 | ```typescript
300 | import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
301 | import sqlite3 from "sqlite3";
302 | import { promisify } from "util";
303 | import { z } from "zod";
304 |
305 | const server = new McpServer({
306 | name: "SQLite Explorer",
307 | version: "1.0.0"
308 | });
309 |
310 | // Helper to create DB connection
311 | const getDb = () => {
312 | const db = new sqlite3.Database("database.db");
313 | return {
314 | all: promisify<string, any[]>(db.all.bind(db)),
315 | close: promisify(db.close.bind(db))
316 | };
317 | };
318 |
319 | server.resource(
320 | "schema",
321 | "schema://main",
322 | async (uri) => {
323 | const db = getDb();
324 | try {
325 | const tables = await db.all(
326 | "SELECT sql FROM sqlite_master WHERE type='table'"
327 | );
328 | return {
329 | contents: [{
330 | uri: uri.href,
331 | text: tables.map((t: {sql: string}) => t.sql).join("\n")
332 | }]
333 | };
334 | } finally {
335 | await db.close();
336 | }
337 | }
338 | );
339 |
340 | server.tool(
341 | "query",
342 | { sql: z.string() },
343 | async ({ sql }) => {
344 | const db = getDb();
345 | try {
346 | const results = await db.all(sql);
347 | return {
348 | content: [{
349 | type: "text",
350 | text: JSON.stringify(results, null, 2)
351 | }]
352 | };
353 | } catch (err: unknown) {
354 | const error = err as Error;
355 | return {
356 | content: [{
357 | type: "text",
358 | text: `Error: ${error.message}`
359 | }],
360 | isError: true
361 | };
362 | } finally {
363 | await db.close();
364 | }
365 | }
366 | );
367 | ```
368 |
369 | ## Advanced Usage
370 |
371 | ### Low-Level Server
372 |
373 | For more control, you can use the low-level Server class directly:
374 |
375 | ```typescript
376 | import { Server } from "@modelcontextprotocol/sdk/server/index.js";
377 | import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
378 | import {
379 | ListPromptsRequestSchema,
380 | GetPromptRequestSchema
381 | } from "@modelcontextprotocol/sdk/types.js";
382 |
383 | const server = new Server(
384 | {
385 | name: "example-server",
386 | version: "1.0.0"
387 | },
388 | {
389 | capabilities: {
390 | prompts: {}
391 | }
392 | }
393 | );
394 |
395 | server.setRequestHandler(ListPromptsRequestSchema, async () => {
396 | return {
397 | prompts: [{
398 | name: "example-prompt",
399 | description: "An example prompt template",
400 | arguments: [{
401 | name: "arg1",
402 | description: "Example argument",
403 | required: true
404 | }]
405 | }]
406 | };
407 | });
408 |
409 | server.setRequestHandler(GetPromptRequestSchema, async (request) => {
410 | if (request.params.name !== "example-prompt") {
411 | throw new Error("Unknown prompt");
412 | }
413 | return {
414 | description: "Example prompt",
415 | messages: [{
416 | role: "user",
417 | content: {
418 | type: "text",
419 | text: "Example prompt text"
420 | }
421 | }]
422 | };
423 | });
424 |
425 | const transport = new StdioServerTransport();
426 | await server.connect(transport);
427 | ```
428 |
429 | ### Writing MCP Clients
430 |
431 | The SDK provides a high-level client interface:
432 |
433 | ```typescript
434 | import { Client } from "@modelcontextprotocol/sdk/client/index.js";
435 | import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
436 |
437 | const transport = new StdioClientTransport({
438 | command: "node",
439 | args: ["server.js"]
440 | });
441 |
442 | const client = new Client(
443 | {
444 | name: "example-client",
445 | version: "1.0.0"
446 | },
447 | {
448 | capabilities: {
449 | prompts: {},
450 | resources: {},
451 | tools: {}
452 | }
453 | }
454 | );
455 |
456 | await client.connect(transport);
457 |
458 | // List prompts
459 | const prompts = await client.listPrompts();
460 |
461 | // Get a prompt
462 | const prompt = await client.getPrompt("example-prompt", {
463 | arg1: "value"
464 | });
465 |
466 | // List resources
467 | const resources = await client.listResources();
468 |
469 | // Read a resource
470 | const resource = await client.readResource("file:///example.txt");
471 |
472 | // Call a tool
473 | const result = await client.callTool({
474 | name: "example-tool",
475 | arguments: {
476 | arg1: "value"
477 | }
478 | });
479 | ```
480 |
481 | ## Documentation
482 |
483 | - [Model Context Protocol documentation](https://modelcontextprotocol.io)
484 | - [MCP Specification](https://spec.modelcontextprotocol.io)
485 | - [Example Servers](https://github.com/modelcontextprotocol/servers)
486 |
487 | ## Contributing
488 |
489 | Issues and pull requests are welcome on GitHub at https://github.com/modelcontextprotocol/typescript-sdk.
490 |
491 | ## License
492 |
493 | This project is licensed under the MIT License—see the [LICENSE](LICENSE) file for details.
494 |
```