This is page 1 of 6. Use http://codebase.md/hithereiamaliff/mcp-datagovmy?lines=true&page={x} to view the full context.
# Directory Structure
```
├── .eslintrc.json
├── .github
│ └── workflows
│ └── deploy-vps.yml
├── .gitignore
├── .npmignore
├── .prettierrc
├── .smithery
│ └── index.cjs
├── deploy
│ ├── DEPLOYMENT.md
│ └── nginx-mcp.conf
├── docker-compose.yml
├── Dockerfile
├── index.js
├── LICENSE
├── malaysia_open_data_mcp_plan.md
├── mcp-server.js
├── package-lock.json
├── package.json
├── PROMPT.md
├── README.md
├── response.txt
├── scripts
│ ├── build.js
│ ├── catalogue-index.d.ts
│ ├── catalogue-index.js
│ ├── catalogue-index.ts
│ ├── dashboards-index.d.ts
│ ├── dashboards-index.js
│ ├── deploy.js
│ ├── extract-dataset-ids.js
│ ├── extracted-datasets.js
│ ├── index-catalogue-files.cjs
│ ├── index-dashboards.cjs
│ └── update-tool-names.ts
├── smithery.yaml
├── src
│ ├── api
│ │ ├── catalogue.js
│ │ ├── client.js
│ │ ├── dosm.js
│ │ ├── transport.js
│ │ └── weather.js
│ ├── catalogue.tools.ts
│ ├── dashboards.tools.ts
│ ├── datacatalogue.tools.ts
│ ├── dosm.tools.ts
│ ├── firebase-analytics.ts
│ ├── flood.tools.ts
│ ├── gtfs.tools.ts
│ ├── http-server.ts
│ ├── index.cjs
│ ├── index.js
│ ├── index.ts
│ ├── parquet.tools.ts
│ ├── tools
│ │ ├── catalogue.js
│ │ ├── dosm.js
│ │ ├── test.js
│ │ ├── transport.js
│ │ └── weather.js
│ ├── transport.tools.ts
│ ├── types.d.ts
│ ├── unified-search.tools.ts
│ ├── utils
│ │ ├── query-builder.js
│ │ └── tool-naming.ts
│ └── weather.tools.ts
├── TOOLS.md
└── tsconfig.json
```
# Files
--------------------------------------------------------------------------------
/.prettierrc:
--------------------------------------------------------------------------------
```
1 | {
2 | "semi": true,
3 | "singleQuote": true,
4 | "trailingComma": "es5",
5 | "printWidth": 100,
6 | "tabWidth": 2
7 | }
8 |
```
--------------------------------------------------------------------------------
/.eslintrc.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "root": true,
3 | "parser": "@typescript-eslint/parser",
4 | "plugins": ["@typescript-eslint", "prettier"],
5 | "extends": [
6 | "eslint:recommended",
7 | "plugin:@typescript-eslint/recommended",
8 | "prettier"
9 | ],
10 | "rules": {
11 | "prettier/prettier": "error",
12 | "no-console": "off"
13 | },
14 | "env": {
15 | "node": true,
16 | "es6": true
17 | }
18 | }
19 |
```
--------------------------------------------------------------------------------
/.npmignore:
--------------------------------------------------------------------------------
```
1 | # Source files
2 | src/
3 |
4 | # Development files
5 | .vscode/
6 | .github/
7 | .env
8 | .env.example
9 | .eslintrc.json
10 | .prettierrc
11 | tsconfig.json
12 | smithery.yaml
13 | Dockerfile
14 |
15 | # Build artifacts
16 | node_modules/
17 | coverage/
18 | .nyc_output/
19 |
20 | # Logs
21 | *.log
22 | npm-debug.log*
23 | yarn-debug.log*
24 | yarn-error.log*
25 |
26 | # Editor directories and files
27 | .idea/
28 | *.swp
29 | *.swo
30 |
31 | # Test files
32 | __tests__/
33 | test/
34 | tests/
35 |
```
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
```
1 | # Dependencies
2 | node_modules/
3 | npm-debug.log
4 | yarn-error.log
5 | yarn-debug.log
6 |
7 | # Environment variables
8 | .env
9 | .env.local
10 | .env.development.local
11 | .env.test.local
12 | .env.production.local
13 | .env.example
14 |
15 | # Build output
16 | dist/
17 | build/
18 | .smithery/
19 |
20 | # Test files
21 | test/
22 |
23 | # TypeScript
24 | *.tsbuildinfo
25 |
26 | # Editor directories and files
27 | .idea/
28 | .vscode/
29 | *.suo
30 | *.ntvs*
31 | *.njsproj
32 | *.sln
33 | *.sw?
34 |
35 | # OS generated files
36 | .DS_Store
37 | .DS_Store?
38 | ._*
39 | .Spotlight-V100
40 | .Trashes
41 | ehthumbs.db
42 | Thumbs.db
43 |
```
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
```markdown
1 | # Malaysia Open Data MCP
2 |
3 | **MCP Endpoint:** `https://mcp.techmavie.digital/datagovmy/mcp`
4 |
5 | **Analytics Dashboard:** [`https://mcp.techmavie.digital/datagovmy/analytics/dashboard`](https://mcp.techmavie.digital/datagovmy/analytics/dashboard)
6 |
7 | MCP (Model Context Protocol) server for Malaysia's Open Data APIs, providing easy access to government datasets and collections.
8 |
9 | Do note that this is **NOT** an official MCP server by the Government of Malaysia or anyone from Malaysia's Open Data/Jabatan Digital Negara/Ministry of Digital team.
10 |
11 | ## Features
12 |
13 | - **Enhanced Unified Search** with flexible tokenization and synonym expansion
14 | - Intelligent query handling with term normalization
15 | - Support for plurals and common prefixes (e.g., "e" in "epayment")
16 | - Smart prioritization for different data types
17 | - **Parquet File Support** using pure JavaScript
18 | - Parse Parquet files directly in the browser or Node.js
19 | - Support for BROTLI compression
20 | - Intelligent date field handling for empty date objects
21 | - Increased row limits (up to 500 rows) for comprehensive data retrieval
22 | - Fallback to metadata estimation when parsing fails
23 | - Automatic dashboard URL mapping for visualization
24 | - **Hybrid Data Access Architecture**
25 | - Pre-generated static indexes for efficient searching
26 | - Dynamic API calls for detailed metadata
27 | - **Multi-Provider Geocoding**
28 | - Support for Google Maps, GrabMaps, and Nominatim (OpenStreetMap)
29 | - Intelligent service selection based on location and available API keys
30 | - GrabMaps optimization for locations in Malaysia
31 | - Automatic fallback between providers
32 | - **Comprehensive Data Sources**
33 | - Malaysia's Data Catalogue with rich metadata
34 | - Interactive Dashboards for data visualization
35 | - Department of Statistics Malaysia (DOSM) data
36 | - Weather forecast and warnings
37 | - Public transport and GTFS data
38 | - **Multi-Provider Malaysian Geocoding**
39 | - Optimized for Malaysian addresses and locations
40 | - Three-tier geocoding system: GrabMaps, Google Maps, and Nominatim
41 | - Prioritizes local knowledge with GrabMaps for better Malaysian coverage
42 | - Automatic fallback to Nominatim when no API keys are provided
43 |
44 | ## Architecture
45 |
46 | This MCP server implements a hybrid approach for efficient data access:
47 |
48 | - **Pre-generated Static Indexes** for listing and searching datasets and dashboards
49 | - **Dynamic API Calls** only when specific dataset or dashboard details are requested
50 |
51 | This approach provides several benefits:
52 | - Faster search and listing operations
53 | - Reduced API calls to external services
54 | - Consistent data access patterns
55 | - Up-to-date detailed information when needed
56 |
57 | ## Documentation
58 |
59 | - **[TOOLS.md](./TOOLS.md)** - Detailed information about available tools and best practices
60 | - **[PROMPT.md](./PROMPT.md)** - AI integration guidelines and usage patterns
61 |
62 | ## AI Integration
63 |
64 | When integrating this MCP server with AI models:
65 |
66 | 1. **Use the unified search tool first** - Always start with `search_all` for any data queries
67 | 2. **Follow the correct URL patterns** - Use `https://data.gov.my/...` and `https://open.dosm.gov.my/...`
68 | 3. **Leverage Parquet file tools** - Use `parse_parquet_file` to access data directly or `get_parquet_info` for metadata
69 | 4. **Use the hybrid approach** - Static indexes for listing/searching, API calls for details
70 | 5. **Consider dashboard visualization** - For complex data, use the dashboard links provided by `find_dashboard_for_parquet`
71 | 6. **Leverage the multi-provider Malaysian geocoding** - For Malaysian location queries, the system automatically selects the best provider (GrabMaps, Google Maps, or Nominatim) with fallback to Nominatim when no API keys are configured
72 |
73 | Refer to [PROMPT.md](./PROMPT.md) for comprehensive AI integration guidelines.
74 |
75 | ## Installation
76 |
77 | ```bash
78 | npm install
79 | ```
80 |
81 | ## Quick Start (Hosted Server)
82 |
83 | The easiest way to use this MCP server is via the hosted endpoint. **No installation required!**
84 |
85 | **Server URL:**
86 | ```
87 | https://mcp.techmavie.digital/datagovmy/mcp
88 | ```
89 |
90 | #### Using Your Own API Keys
91 |
92 | You can provide your own API keys via URL query parameters:
93 |
94 | ```
95 | https://mcp.techmavie.digital/datagovmy/mcp?googleMapsApiKey=YOUR_KEY
96 | ```
97 |
98 | Or via headers:
99 | - `X-Google-Maps-Api-Key: YOUR_KEY`
100 | - `X-GrabMaps-Api-Key: YOUR_KEY`
101 | - `X-AWS-Access-Key-Id: YOUR_KEY`
102 | - `X-AWS-Secret-Access-Key: YOUR_KEY`
103 | - `X-AWS-Region: ap-southeast-5`
104 |
105 | **Supported Query Parameters:**
106 |
107 | | Parameter | Description |
108 | |-----------|-------------|
109 | | `googleMapsApiKey` | Google Maps API key for geocoding |
110 | | `grabMapsApiKey` | GrabMaps API key for Southeast Asia geocoding |
111 | | `awsAccessKeyId` | AWS Access Key ID for AWS Location Service |
112 | | `awsSecretAccessKey` | AWS Secret Access Key |
113 | | `awsRegion` | AWS Region (default: ap-southeast-5) |
114 |
115 | > **⚠️ Important: GrabMaps Requirements**
116 | >
117 | > To use GrabMaps geocoding, you need **ALL FOUR** parameters:
118 | > - `grabMapsApiKey`
119 | > - `awsAccessKeyId`
120 | > - `awsSecretAccessKey`
121 | > - `awsRegion`
122 | >
123 | > GrabMaps uses AWS Location Service under the hood, so AWS credentials are required alongside the GrabMaps API key.
124 |
125 | ### Client Configuration
126 |
127 | For Claude Desktop / Cursor / Windsurf, add to your MCP configuration:
128 |
129 | ```json
130 | {
131 | "mcpServers": {
132 | "malaysia-opendata": {
133 | "transport": "streamable-http",
134 | "url": "https://mcp.techmavie.digital/datagovmy/mcp"
135 | }
136 | }
137 | }
138 | ```
139 |
140 | With your own API key:
141 | ```json
142 | {
143 | "mcpServers": {
144 | "malaysia-opendata": {
145 | "transport": "streamable-http",
146 | "url": "https://mcp.techmavie.digital/datagovmy/mcp?googleMapsApiKey=YOUR_KEY"
147 | }
148 | }
149 | }
150 | ```
151 |
152 | ## Self-Hosted (VPS)
153 |
154 | If you prefer to run your own instance, see [deploy/DEPLOYMENT.md](deploy/DEPLOYMENT.md) for detailed VPS deployment instructions with Docker and Nginx.
155 |
156 | ## Analytics Dashboard
157 |
158 | The hosted server includes a built-in analytics dashboard:
159 |
160 | **Dashboard URL:** [`https://mcp.techmavie.digital/datagovmy/analytics/dashboard`](https://mcp.techmavie.digital/datagovmy/analytics/dashboard)
161 |
162 | ### Analytics Endpoints
163 |
164 | | Endpoint | Description |
165 | |----------|-------------|
166 | | `/analytics` | Full analytics summary (JSON) |
167 | | `/analytics/tools` | Detailed tool usage stats (JSON) |
168 | | `/analytics/dashboard` | Visual dashboard with charts (HTML) |
169 |
170 | The dashboard tracks:
171 | - Total requests and tool calls
172 | - Tool usage distribution
173 | - Hourly request trends (last 24 hours)
174 | - Requests by endpoint
175 | - Top clients by user agent
176 | - Recent tool calls feed
177 |
178 | Auto-refreshes every 30 seconds.
179 |
180 | ## Available Tools
181 |
182 | ### Data Catalogue
183 |
184 | - `list_datasets`: Lists available datasets in the Data Catalogue
185 | - `get_dataset`: Gets data from a specific dataset in the Data Catalogue
186 | - `search_datasets`: Searches for datasets in the Data Catalogue
187 |
188 | ### Department of Statistics Malaysia (DOSM)
189 |
190 | - `list_dosm_datasets`: Lists available datasets from DOSM
191 | - `get_dosm_dataset`: Gets data from a specific DOSM dataset
192 |
193 | ### Parquet File Handling
194 |
195 | - `parse_parquet_file`: Parse and display data from a Parquet file URL
196 | - Supports up to 500 rows for comprehensive data analysis
197 | - Automatically handles empty date objects with appropriate formatting
198 | - Processes BigInt values for proper JSON serialization
199 | - `get_parquet_info`: Get metadata and structure information about a Parquet file
200 | - `find_dashboard_for_parquet`: Find the corresponding dashboard URL for a Parquet file
201 |
202 | ### Weather
203 |
204 | - `get_weather_forecast`: Gets weather forecast for Malaysia
205 | - `get_weather_warnings`: Gets current weather warnings for Malaysia
206 | - `get_earthquake_warnings`: Gets earthquake warnings for Malaysia
207 |
208 | ### Transport
209 |
210 | - `list_transport_agencies`: Lists available transport agencies with GTFS data
211 | - `get_transport_data`: Gets GTFS data for a specific transport agency
212 |
213 | ### GTFS Parsing
214 |
215 | - `parse_gtfs_static`: Parses GTFS Static data (ZIP files with CSV data) for a specific transport provider
216 | - `parse_gtfs_realtime`: Parses GTFS Realtime data (Protocol Buffer format) for vehicle positions
217 | - `get_transit_routes`: Extracts route information from GTFS data
218 | - `get_transit_stops`: Extracts stop information from GTFS data, optionally filtered by route
219 |
220 | ### Test
221 |
222 | - `hello`: A simple test tool to verify that the MCP server is working correctly
223 |
224 | ## Data-Catalogue Information Retrieval
225 |
226 | The MCP server provides robust handling for data-catalogue information retrieval:
227 |
228 | ### Date Handling in Parquet Files
229 |
230 | - **Empty Date Objects**: The system automatically detects and handles empty date objects in parquet files
231 | - **Dataset-Specific Handling**: Special handling for known datasets like `employment_sector` with annual data from 2001-2022
232 | - **Pattern Recognition**: Detects date patterns in existing data to maintain consistent formatting
233 | - **Increased Row Limits**: Supports up to 500 rows (increased from 100) for more comprehensive data analysis
234 |
235 | ### BigInt Processing
236 |
237 | - **Automatic Serialization**: BigInt values are automatically converted to strings for proper JSON serialization
238 | - **Type Preservation**: Original types are preserved in the schema information
239 |
240 | ### Schema Detection
241 |
242 | - **Automatic Type Inference**: Detects column types including special handling for date fields
243 | - **Consistent Representation**: Ensures date fields are consistently represented as strings
244 |
245 | ## Usage Examples
246 |
247 | ### Get Weather Forecast
248 |
249 | ```javascript
250 | const result = await tools.get_weather_forecast({
251 | location: "Kuala Lumpur",
252 | days: 3
253 | });
254 | ```
255 |
256 | ### Search Datasets
257 |
258 | ```javascript
259 | const result = await tools.search_datasets({
260 | query: "population",
261 | limit: 5
262 | });
263 | ```
264 |
265 | ### Parse GTFS Data
266 |
267 | ```javascript
268 | // Parse GTFS Static data
269 | const staticData = await tools.parse_gtfs_static({
270 | provider: "ktmb"
271 | });
272 |
273 | // Get real-time vehicle positions
274 | const realtimeData = await tools.parse_gtfs_realtime({
275 | provider: "prasarana",
276 | category: "rapid-rail-kl"
277 | });
278 |
279 | // Get transit routes
280 | const routes = await tools.get_transit_routes({
281 | provider: "mybas-johor"
282 | });
283 |
284 | // Get stops for a specific route
285 | const stops = await tools.get_transit_stops({
286 | provider: "prasarana",
287 | category: "rapid-rail-kl",
288 | route_id: "LRT-KJ"
289 | });
290 | ```
291 |
292 | ## API Rate Limits
293 |
294 | Please be aware of rate limits for the underlying APIs. Excessive requests may be throttled.
295 |
296 | ## Project Structure
297 |
298 | - `src/index.ts`: Main MCP server implementation and tool registration
299 | - `src/http-server.ts`: Streamable HTTP server for VPS deployment
300 | - `src/datacatalogue.tools.ts`: Data Catalogue API tools
301 | - `src/dashboards.tools.ts`: Dashboard access and search tools
302 | - `src/dosm.tools.ts`: Department of Statistics Malaysia tools
303 | - `src/unified-search.tools.ts`: Enhanced unified search with tokenization and synonym expansion
304 | - `src/parquet.tools.ts`: Parquet file parsing and metadata tools
305 | - `src/weather.tools.ts`: Weather forecast and warnings tools
306 | - `src/transport.tools.ts`: Transport and GTFS data tools
307 | - `src/gtfs.tools.ts`: GTFS parsing and analysis tools
308 | - `src/flood.tools.ts`: Flood warning and monitoring tools
309 | - `Dockerfile`: Docker configuration for VPS deployment
310 | - `docker-compose.yml`: Docker Compose configuration
311 | - `deploy/`: Deployment files (nginx config, deployment guide)
312 | - `package.json`: Project dependencies and scripts
313 | - `tsconfig.json`: TypeScript configuration
314 |
315 | ## Local Development
316 |
317 | ```bash
318 | # Install dependencies
319 | npm install
320 |
321 | # Run HTTP server in development mode
322 | npm run dev:http
323 |
324 | # Or build and run production version
325 | npm run build
326 | npm run start:http
327 |
328 | # Test health endpoint
329 | curl http://localhost:8080/health
330 |
331 | # Test MCP endpoint
332 | curl -X POST http://localhost:8080/mcp \
333 | -H "Content-Type: application/json" \
334 | -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
335 | ```
336 |
337 | ## Troubleshooting
338 |
339 | ### Container Issues
340 |
341 | ```bash
342 | # Check container status
343 | docker compose ps
344 |
345 | # View logs
346 | docker compose logs -f
347 |
348 | # Restart container
349 | docker compose restart
350 | ```
351 |
352 | ### Test MCP Connection
353 |
354 | ```bash
355 | # List tools
356 | curl -X POST https://mcp.techmavie.digital/datagovmy/mcp \
357 | -H "Content-Type: application/json" \
358 | -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
359 |
360 | # Call hello tool
361 | curl -X POST https://mcp.techmavie.digital/datagovmy/mcp \
362 | -H "Content-Type: application/json" \
363 | -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"my_hello","arguments":{}}}'
364 | ```
365 |
366 | ## Configuration
367 |
368 | ### Environment Variables
369 |
370 | This project supports the following configuration options:
371 |
372 | **Geocoding Credentials (Optional. Only for GTFS Transit Features Usage)**:
373 |
374 | The following credentials are **only needed if you plan to use the GTFS transit tools** that require geocoding services. Other features like data catalogue access, weather forecasts, and DOSM data do not require these credentials.
375 |
376 | - **googleMapsApiKey**: Optional. If provided, the system will use Google Maps API for geocoding location names to coordinates.
377 | - **grabMapsApiKey**: Optional. Required for GrabMaps geocoding, which is optimized for locations in Malaysia.
378 | - **awsAccessKeyId**: Required for GrabMaps integration. AWS access key for GrabMaps API authentication.
379 | - **awsSecretAccessKey**: Required for GrabMaps integration. AWS secret key for GrabMaps API authentication.
380 | - **awsRegion**: Required for GrabMaps integration. AWS region for GrabMaps API (e.g. 'ap-southeast-5' for Malaysia region or ap-southeast-1 for Singapore region).
381 |
382 | If neither Google Maps nor GrabMaps API keys are provided, the GTFS transit tools will automatically fall back to using Nominatim (OpenStreetMap) API for geocoding, which is free and doesn't require credentials.
383 |
384 | You can set these configuration options in two ways:
385 |
386 | 1. **Via URL query parameters** when connecting to the hosted server (see Quick Start section)
387 | 2. **As environment variables** for local development or self-hosted deployment
388 |
389 | #### Setting up environment variables
390 |
391 | Create a `.env` file in the root directory:
392 |
393 | ```env
394 | GOOGLE_MAPS_API_KEY=your_google_api_key_here
395 | GRABMAPS_API_KEY=your_grab_api_key_here
396 | AWS_ACCESS_KEY_ID=your_aws_access_key_for_grabmaps
397 | AWS_SECRET_ACCESS_KEY=your_aws_secret_key_for_grabmaps
398 | AWS_REGION=ap-southeast-5
399 | ```
400 |
401 | The variables will be automatically loaded when you run the server.
402 |
403 | **Note:** For Malaysian locations, GrabMaps provides the most accurate geocoding results, followed by Google Maps. If you don't provide either API key, the system will automatically use Nominatim API instead, which is free but may have less accurate results for some locations in Malaysia.
404 |
405 | **Important:** These geocoding credentials are only required for the following GTFS transit tools:
406 | - `get_transit_routes` - When converting location names to coordinates
407 | - `get_transit_stops` - When converting location names to coordinates
408 | - `parse_gtfs_static` - When geocoding is needed for stop locations
409 |
410 | **Note about GTFS Realtime Tools:** The `parse_gtfs_realtime` tool is currently in development and has limited availability. Real-time data access through this MCP is experimental and may not be available for all providers or routes. For up-to-date train and bus schedules, bus locations, and arrivals in real-time, please use official transit apps like Google Maps, MyRapid PULSE, Moovit, or Lugo.
411 |
412 | All other tools like data catalogue access, dashboard search, weather forecasts, and DOSM data do not require any geocoding credentials.
413 |
414 | ## License
415 |
416 | MIT - See [LICENSE](./LICENSE) file for details.
417 |
418 | ## Acknowledgments
419 |
420 | - [Malaysia Open Data Portal](https://data.gov.my/)
421 | - [Department of Statistics Malaysia](https://open.dosm.gov.my/)
422 | - [Malaysian Meteorological Department](https://www.met.gov.my/)
423 | - [Google Maps Platform](https://developers.google.com/maps) for geocoding
424 | - [GrabMaps](https://grabmaps.grab.com/solutions/service-apis) for geocoding
425 | - [Nominatim](https://nominatim.org/) for geocoding
426 | - [Model Context Protocol](https://modelcontextprotocol.io/) for the MCP framework
427 |
```
--------------------------------------------------------------------------------
/response.txt:
--------------------------------------------------------------------------------
```
1 |
```
--------------------------------------------------------------------------------
/smithery.yaml:
--------------------------------------------------------------------------------
```yaml
1 | runtime: typescript
2 |
```
--------------------------------------------------------------------------------
/scripts/dashboards-index.d.ts:
--------------------------------------------------------------------------------
```typescript
1 | export declare const DASHBOARDS_INDEX: any[];
2 |
```
--------------------------------------------------------------------------------
/src/types.d.ts:
--------------------------------------------------------------------------------
```typescript
1 | declare module './datacatalogue.tools';
2 | declare module './dosm.tools';
3 | declare module './weather.tools';
4 | declare module './transport.tools';
5 |
```
--------------------------------------------------------------------------------
/src/tools/test.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Test Tool
3 | *
4 | * A simple test tool to verify that the MCP server is working correctly.
5 | */
6 |
7 | /**
8 | * Returns a simple hello message
9 | * @returns {Promise<Object>} - Hello message
10 | */
11 | async function hello() {
12 | return {
13 | message: 'Hello from Malaysia Open Data MCP!',
14 | timestamp: new Date().toISOString()
15 | };
16 | }
17 |
18 | module.exports = {
19 | hello
20 | };
21 |
```
--------------------------------------------------------------------------------
/src/utils/tool-naming.ts:
--------------------------------------------------------------------------------
```typescript
1 | /**
2 | * Helper functions for consistent tool naming
3 | */
4 |
5 | /**
6 | * Adds the "datagovmy_" prefix to a tool name
7 | * @param toolName The original tool name
8 | * @returns The prefixed tool name
9 | */
10 | export function prefixToolName(toolName: string): string {
11 | // Don't add prefix if it already exists
12 | if (toolName.startsWith('datagovmy_')) {
13 | return toolName;
14 | }
15 | return `datagovmy_${toolName}`;
16 | }
17 |
```
--------------------------------------------------------------------------------
/tsconfig.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "compilerOptions": {
3 | "target": "ES2022",
4 | "module": "NodeNext",
5 | "moduleResolution": "NodeNext",
6 | "esModuleInterop": true,
7 | "strict": true,
8 | "skipLibCheck": true,
9 | "forceConsistentCasingInFileNames": true,
10 | "outDir": "dist",
11 | "declaration": true,
12 | "resolveJsonModule": true,
13 | "allowSyntheticDefaultImports": true,
14 | "baseUrl": "."
15 | },
16 | "include": [
17 | "src/**/*",
18 | "scripts/**/*.d.ts"
19 | ],
20 | "exclude": [
21 | "node_modules",
22 | "dist"
23 | ]
24 | }
25 |
```
--------------------------------------------------------------------------------
/scripts/deploy.js:
--------------------------------------------------------------------------------
```javascript
1 | #!/usr/bin/env node
2 |
3 | const { execSync } = require('child_process');
4 | const path = require('path');
5 |
6 | console.log('🚀 Deploying Malaysia Open Data MCP to Smithery...');
7 |
8 | try {
9 | // Build the project first
10 | console.log('📦 Building the project...');
11 | execSync('cmd.exe /c npm run build', { stdio: 'inherit', cwd: path.join(__dirname, '..') });
12 |
13 | // Deploy to Smithery
14 | console.log('🚀 Deploying to Smithery...');
15 | execSync('cmd.exe /c npx @smithery/cli deploy', { stdio: 'inherit', cwd: path.join(__dirname, '..') });
16 |
17 | console.log('✅ Deployment completed successfully!');
18 | } catch (error) {
19 | console.error('❌ Deployment failed:', error.message);
20 | process.exit(1);
21 | }
22 |
```
--------------------------------------------------------------------------------
/Dockerfile:
--------------------------------------------------------------------------------
```dockerfile
1 | # Malaysia Open Data MCP Server - Streamable HTTP
2 | # For self-hosting on VPS with nginx reverse proxy
3 |
4 | FROM node:22-slim
5 |
6 | # Set the working directory
7 | WORKDIR /app
8 |
9 | # Copy package files
10 | COPY package*.json ./
11 |
12 | # Install ALL dependencies (including devDependencies for build)
13 | RUN npm ci
14 |
15 | # Copy source code and configuration
16 | COPY tsconfig.json ./
17 | COPY src ./src
18 | COPY scripts ./scripts
19 |
20 | # Build TypeScript code
21 | RUN npm run build
22 |
23 | # Copy scripts to dist (needed for runtime imports)
24 | RUN cp -r scripts dist/
25 |
26 | # Remove devDependencies after build
27 | RUN npm prune --production
28 |
29 | # Expose port for HTTP server
30 | EXPOSE 8080
31 |
32 | # Environment variables (can be overridden at runtime)
33 | ENV PORT=8080
34 | ENV HOST=0.0.0.0
35 |
36 | # Health check
37 | HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
38 | CMD wget --no-verbose --tries=1 --spider http://localhost:8080/health || exit 1
39 |
40 | # Start the HTTP server
41 | CMD ["node", "dist/src/http-server.js"]
42 |
```
--------------------------------------------------------------------------------
/scripts/catalogue-index.d.ts:
--------------------------------------------------------------------------------
```typescript
1 | // Generated by scripts/index-catalogue-files.js on 2025-07-27T03:58:23.566Z
2 |
3 | declare module '../scripts/catalogue-index.js' {
4 | export interface SiteCategory {
5 | site: string;
6 | category_en: string;
7 | category_ms: string;
8 | category_sort: number;
9 | subcategory_en: string;
10 | subcategory_ms: string;
11 | subcategory_sort: number;
12 | }
13 |
14 | export interface DatasetMetadata {
15 | id: string;
16 | title_en: string;
17 | title_ms: string;
18 | description_en: string;
19 | description_ms: string;
20 | frequency: string;
21 | geography: string[];
22 | demography: string[];
23 | dataset_begin: number | null;
24 | dataset_end: number | null;
25 | data_source: string[];
26 | data_as_of: string;
27 | last_updated: string;
28 | next_update: string;
29 | link_parquet: string;
30 | link_csv: string;
31 | link_preview: string;
32 | site_category: SiteCategory[];
33 | }
34 |
35 | export interface CatalogueFilters {
36 | categories: string[];
37 | geographies: string[];
38 | frequencies: string[];
39 | demographies: string[];
40 | dataSources: string[];
41 | }
42 |
43 | export const CATALOGUE_INDEX: DatasetMetadata[];
44 | export const CATALOGUE_FILTERS: CatalogueFilters;
45 | }
```
--------------------------------------------------------------------------------
/docker-compose.yml:
--------------------------------------------------------------------------------
```yaml
1 | # Malaysia Open Data MCP Server - Docker Compose
2 | # For self-hosting on VPS at mcp.techmavie.digital/datagovmy
3 |
4 | services:
5 | mcp-datagovmy:
6 | build: .
7 | container_name: mcp-datagovmy
8 | restart: unless-stopped
9 | ports:
10 | - "8083:8080"
11 | environment:
12 | - PORT=8080
13 | - HOST=0.0.0.0
14 | - NODE_ENV=production
15 | - ANALYTICS_DIR=/app/data
16 | # Optional API keys - uncomment and set if needed
17 | # - GOOGLE_MAPS_API_KEY=${GOOGLE_MAPS_API_KEY}
18 | # - GRABMAPS_API_KEY=${GRABMAPS_API_KEY}
19 | # - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
20 | # - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
21 | # - AWS_REGION=${AWS_REGION}
22 | env_file:
23 | - .env
24 | volumes:
25 | - analytics-data:/app/data
26 | - ./.credentials:/app/.credentials:ro
27 | healthcheck:
28 | test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:8080/health"]
29 | interval: 30s
30 | timeout: 10s
31 | retries: 3
32 | start_period: 10s
33 | networks:
34 | - mcp-network
35 | logging:
36 | driver: "json-file"
37 | options:
38 | max-size: "10m"
39 | max-file: "3"
40 |
41 | networks:
42 | mcp-network:
43 | driver: bridge
44 |
45 | volumes:
46 | analytics-data:
47 | driver: local
```
--------------------------------------------------------------------------------
/src/api/dosm.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * OpenDOSM API Client
3 | *
4 | * Handles communication with the OpenDOSM API endpoint
5 | */
6 |
7 | const { createClient } = require('./client');
8 |
9 | /**
10 | * Creates an OpenDOSM API client
11 | *
12 | * @param {Object} options - Client configuration options
13 | * @returns {Object} - OpenDOSM API client
14 | */
15 | function createDosmClient(options = {}) {
16 | const client = createClient(options);
17 | const ENDPOINT = '/opendosm';
18 |
19 | return {
20 | /**
21 | * Gets data from a specific OpenDOSM dataset
22 | *
23 | * @param {string} datasetId - Dataset ID
24 | * @param {Object} params - Additional query parameters
25 | * @returns {Promise<Object>} - Dataset data
26 | */
27 | async getDataset(datasetId, params = {}) {
28 | return client.request(ENDPOINT, {
29 | id: datasetId,
30 | ...params
31 | });
32 | },
33 |
34 | /**
35 | * Lists available OpenDOSM datasets
36 | *
37 | * @param {Object} params - List parameters
38 | * @returns {Promise<Object>} - List of datasets
39 | */
40 | async listDatasets(params = {}) {
41 | // For listing datasets, we'll need to use the meta parameter
42 | // to get information about available datasets
43 | return client.request(ENDPOINT, {
44 | meta: true,
45 | ...params
46 | });
47 | }
48 | };
49 | }
50 |
51 | module.exports = {
52 | createDosmClient
53 | };
54 |
```
--------------------------------------------------------------------------------
/scripts/build.js:
--------------------------------------------------------------------------------
```javascript
1 | #!/usr/bin/env node
2 |
3 | const { execSync } = require('child_process');
4 | const fs = require('fs');
5 | const path = require('path');
6 |
7 | // Ensure scripts directory exists
8 | const scriptsDir = path.join(__dirname);
9 | if (!fs.existsSync(scriptsDir)) {
10 | fs.mkdirSync(scriptsDir, { recursive: true });
11 | }
12 |
13 | console.log('🚀 Building Malaysia Open Data MCP...');
14 |
15 | try {
16 | // Clean previous build
17 | console.log('📦 Cleaning previous build...');
18 | if (fs.existsSync(path.join(__dirname, '..', 'dist'))) {
19 | if (process.platform === 'win32') {
20 | execSync('cmd.exe /c rmdir /s /q dist', { stdio: 'inherit', cwd: path.join(__dirname, '..') });
21 | } else {
22 | execSync('rm -rf dist', { stdio: 'inherit', cwd: path.join(__dirname, '..') });
23 | }
24 | }
25 |
26 | // Run TypeScript compiler
27 | console.log('📦 Compiling TypeScript...');
28 | execSync('npx tsc', { stdio: 'inherit', cwd: path.join(__dirname, '..') });
29 |
30 | // Copy smithery.yaml to dist
31 | console.log('📦 Copying configuration files...');
32 | fs.copyFileSync(
33 | path.join(__dirname, '..', 'smithery.yaml'),
34 | path.join(__dirname, '..', 'dist', 'smithery.yaml')
35 | );
36 |
37 | console.log('✅ Build completed successfully!');
38 | console.log('\nTo start the development server:');
39 | console.log(' npx @smithery/cli dev');
40 | console.log('\nTo deploy to Smithery:');
41 | console.log(' npx @smithery/cli deploy');
42 | } catch (error) {
43 | console.error('❌ Build failed:', error.message);
44 | process.exit(1);
45 | }
46 |
```
--------------------------------------------------------------------------------
/src/api/weather.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Weather API Client
3 | *
4 | * Handles communication with the Weather API endpoints
5 | */
6 |
7 | const { createClient } = require('./client');
8 |
9 | /**
10 | * Creates a Weather API client
11 | *
12 | * @param {Object} options - Client configuration options
13 | * @returns {Object} - Weather API client
14 | */
15 | function createWeatherClient(options = {}) {
16 | const client = createClient(options);
17 |
18 | // Weather API endpoints
19 | const FORECAST_ENDPOINT = '/weather/forecast';
20 | const WARNING_ENDPOINT = '/weather/warning';
21 | const EARTHQUAKE_WARNING_ENDPOINT = '/weather/warning/earthquake';
22 |
23 | return {
24 | /**
25 | * Gets 7-day general forecast data
26 | *
27 | * @param {Object} params - Query parameters
28 | * @returns {Promise<Object>} - Forecast data
29 | */
30 | async getForecast(params = {}) {
31 | return client.request(FORECAST_ENDPOINT, params);
32 | },
33 |
34 | /**
35 | * Gets weather warning data
36 | *
37 | * @param {Object} params - Query parameters
38 | * @returns {Promise<Object>} - Warning data
39 | */
40 | async getWarnings(params = {}) {
41 | return client.request(WARNING_ENDPOINT, params);
42 | },
43 |
44 | /**
45 | * Gets earthquake warning data
46 | *
47 | * @param {Object} params - Query parameters
48 | * @returns {Promise<Object>} - Earthquake warning data
49 | */
50 | async getEarthquakeWarnings(params = {}) {
51 | return client.request(EARTHQUAKE_WARNING_ENDPOINT, params);
52 | }
53 | };
54 | }
55 |
56 | module.exports = {
57 | createWeatherClient
58 | };
59 |
```
--------------------------------------------------------------------------------
/src/index.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Malaysia Open Data MCP - Main Entry Point
3 | *
4 | * This file serves as the main entry point for the Malaysia Open Data MCP server.
5 | * It exports all the tools that will be available through the MCP.
6 | */
7 |
8 | // Import tools
9 | const catalogueTools = require('./tools/catalogue');
10 | const dosmTools = require('./tools/dosm');
11 | const weatherTools = require('./tools/weather');
12 | const transportTools = require('./tools/transport');
13 | const testTools = require('./tools/test');
14 |
15 | // Define the server function that Smithery expects
16 | function server({ sessionId, config }) {
17 | // Define all the tools
18 | const tools = {
19 | // Data Catalogue Tools
20 | list_datasets: catalogueTools.listDatasets,
21 | get_dataset: catalogueTools.getDataset,
22 | search_datasets: catalogueTools.searchDatasets,
23 |
24 | // OpenDOSM Tools
25 | list_dosm_datasets: dosmTools.listDatasets,
26 | get_dosm_dataset: dosmTools.getDataset,
27 |
28 | // Weather Tools
29 | get_weather_forecast: weatherTools.getForecast,
30 | get_weather_warnings: weatherTools.getWarnings,
31 | get_earthquake_warnings: weatherTools.getEarthquakeWarnings,
32 |
33 | // Transport Tools
34 | list_transport_agencies: transportTools.listAgencies,
35 | get_transport_data: transportTools.getData,
36 |
37 | // Test Tools
38 | hello: testTools.hello,
39 | };
40 |
41 | // Return an object with a connect method that returns the tools
42 | return {
43 | connect: () => tools
44 | };
45 | }
46 |
47 | // Export the server function for CommonJS
48 | module.exports = server;
49 | module.exports.default = server;
50 |
```
--------------------------------------------------------------------------------
/.github/workflows/deploy-vps.yml:
--------------------------------------------------------------------------------
```yaml
1 | name: Deploy to VPS
2 |
3 | on:
4 | push:
5 | branches:
6 | - main
7 |
8 | jobs:
9 | deploy:
10 | runs-on: ubuntu-latest
11 |
12 | steps:
13 | - name: Deploy to VPS via SSH
14 | uses: appleboy/[email protected]
15 | with:
16 | host: ${{ secrets.VPS_HOST }}
17 | username: ${{ secrets.VPS_USERNAME }}
18 | key: ${{ secrets.VPS_SSH_KEY }}
19 | port: ${{ secrets.VPS_PORT }}
20 | script: |
21 | cd /opt/mcp-servers/datagovmy
22 | git pull origin main
23 |
24 | # Create credentials directory if it doesn't exist
25 | mkdir -p .credentials
26 |
27 | # Copy shared Firebase credentials from parent directory
28 | if [ ! -f .credentials/firebase-service-account.json ]; then
29 | cp ../.credentials/firebase-service-account.json .credentials/
30 | echo "✅ Firebase credentials copied from parent directory"
31 | else
32 | echo "✅ Firebase credentials already exist"
33 | fi
34 |
35 | # Rebuild and restart with no cache
36 | docker compose down
37 | docker compose build --no-cache
38 | docker compose up -d
39 |
40 | # Wait for container to start
41 | sleep 5
42 |
43 | # Check if container is running
44 | docker compose ps
45 |
46 | # Show recent logs
47 | docker compose logs --tail 30
48 |
49 | echo "🚀 Deployment completed at $(date)"
```
--------------------------------------------------------------------------------
/src/index.cjs:
--------------------------------------------------------------------------------
```
1 | /**
2 | * Malaysia Open Data MCP - CommonJS Entry Point
3 | *
4 | * This file serves as the CommonJS entry point for the Malaysia Open Data MCP server.
5 | * It exports all the tools that will be available through the MCP.
6 | */
7 |
8 | // Import tools
9 | const catalogueTools = require('./tools/catalogue');
10 | const dosmTools = require('./tools/dosm');
11 | const weatherTools = require('./tools/weather');
12 | const transportTools = require('./tools/transport');
13 | const testTools = require('./tools/test');
14 |
15 | // Define the server function that Smithery expects
16 | function server({ sessionId, config }) {
17 | // Define all the tools
18 | const tools = {
19 | // Data Catalogue Tools
20 | list_datasets: catalogueTools.listDatasets,
21 | get_dataset: catalogueTools.getDataset,
22 | search_datasets: catalogueTools.searchDatasets,
23 |
24 | // OpenDOSM Tools
25 | list_dosm_datasets: dosmTools.listDatasets,
26 | get_dosm_dataset: dosmTools.getDataset,
27 |
28 | // Weather Tools
29 | get_weather_forecast: weatherTools.getForecast,
30 | get_weather_warnings: weatherTools.getWarnings,
31 | get_earthquake_warnings: weatherTools.getEarthquakeWarnings,
32 |
33 | // Transport Tools
34 | list_transport_agencies: transportTools.listAgencies,
35 | get_transport_data: transportTools.getData,
36 |
37 | // Test Tools
38 | hello: testTools.hello,
39 | };
40 |
41 | // Return an object with a connect method that returns the tools
42 | return {
43 | connect: () => tools
44 | };
45 | }
46 |
47 | // Export the server function
48 | module.exports = server;
49 | module.exports.default = server;
50 |
```
--------------------------------------------------------------------------------
/PROMPT.md:
--------------------------------------------------------------------------------
```markdown
1 | # Malaysia Open Data MCP Server - AI Prompt Guide
2 |
3 | When using the Malaysia Open Data MCP server, follow these guidelines to ensure optimal results:
4 |
5 | ## Primary Search Tool
6 |
7 | **ALWAYS use the `search_all` tool first** when searching for any data, statistics, or visualizations related to Malaysia's open data. This tool provides unified search across both datasets and dashboards, with intelligent fallback to ensure comprehensive results.
8 |
9 | Example:
10 | ```
11 | search_all
12 | {
13 | "query": "e-payment statistics"
14 | }
15 | ```
16 |
17 | Only use specific dataset or dashboard search tools if you need to explicitly limit your search to one type of content.
18 |
19 | ## Data Access Pattern
20 |
21 | 1. Start with `search_all` to find relevant resources
22 | 2. For detailed dataset information, use `get_dataset_details` with the dataset ID
23 | 3. For dashboard information, use `get_dashboard_by_name` with the dashboard name
24 | 4. For dashboard charts, use `get_dashboard_charts` with the dashboard name
25 |
26 | ## URL References
27 |
28 | When referring to resources in responses:
29 | - Use `https://data.gov.my/...` for general data portal resources
30 | - Use `https://open.dosm.gov.my/...` for OpenDOSM resources
31 |
32 | ## Data Format Limitations
33 |
34 | - Dashboard data is visualized on the web interface. Raw data files (e.g., parquet) cannot be directly accessed through this API.
35 | - Dataset metadata is available through this API. For downloading the actual data files, users should visit the dataset page on the data portal.
36 |
37 | ## Hybrid Architecture
38 |
39 | This MCP server uses a hybrid approach:
40 | - Pre-generated static indexes for efficient listing and searching
41 | - Dynamic API calls only when specific dataset or dashboard details are requested
42 |
43 | This ensures fast responses while maintaining up-to-date information.
44 |
```
--------------------------------------------------------------------------------
/src/api/catalogue.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Data Catalogue API Client
3 | *
4 | * Handles communication with the Data Catalogue API endpoint
5 | */
6 |
7 | const { createClient } = require('./client');
8 |
9 | /**
10 | * Creates a Data Catalogue API client
11 | *
12 | * @param {Object} options - Client configuration options
13 | * @returns {Object} - Data Catalogue API client
14 | */
15 | function createCatalogueClient(options = {}) {
16 | const client = createClient(options);
17 | const ENDPOINT = '/data-catalogue';
18 |
19 | return {
20 | /**
21 | * Gets data from a specific dataset
22 | *
23 | * @param {string} datasetId - Dataset ID
24 | * @param {Object} params - Additional query parameters
25 | * @returns {Promise<Object>} - Dataset data
26 | */
27 | async getDataset(datasetId, params = {}) {
28 | return client.request(ENDPOINT, {
29 | id: datasetId,
30 | ...params
31 | });
32 | },
33 |
34 | /**
35 | * Searches across datasets
36 | *
37 | * @param {Object} params - Search parameters
38 | * @returns {Promise<Object>} - Search results
39 | */
40 | async searchDatasets(params = {}) {
41 | return client.request(ENDPOINT, {
42 | meta: true,
43 | ...params
44 | });
45 | },
46 |
47 | /**
48 | * Lists available datasets
49 | *
50 | * @param {Object} params - List parameters
51 | * @returns {Promise<Object>} - List of datasets
52 | */
53 | async listDatasets(params = {}) {
54 | // For listing datasets, we'll need to use the meta parameter
55 | // to get information about available datasets
56 | return client.request(ENDPOINT, {
57 | meta: true,
58 | ...params
59 | });
60 | }
61 | };
62 | }
63 |
64 | // Export both as a named export and as the default export for maximum compatibility
65 | module.exports = createCatalogueClient;
66 | module.exports.createCatalogueClient = createCatalogueClient;
67 | module.exports.default = createCatalogueClient;
68 |
```
--------------------------------------------------------------------------------
/src/tools/dosm.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * OpenDOSM MCP Tools
3 | *
4 | * Tools for accessing and searching the OpenDOSM data
5 | */
6 |
7 | const { createDosmClient } = require('../api/dosm');
8 |
9 | // Create client instance with default configuration
10 | const dosmClient = createDosmClient();
11 |
12 | /**
13 | * Lists available datasets in the OpenDOSM data catalogue
14 | *
15 | * @param {Object} params - Optional parameters
16 | * @param {number} params.limit - Maximum number of datasets to return
17 | * @returns {Promise<Object>} - List of datasets
18 | */
19 | async function listDatasets(params = {}) {
20 | try {
21 | const result = await dosmClient.listDatasets(params);
22 |
23 | return {
24 | success: true,
25 | message: 'Successfully retrieved OpenDOSM datasets',
26 | data: result
27 | };
28 | } catch (error) {
29 | return {
30 | success: false,
31 | message: `Failed to list OpenDOSM datasets: ${error.message}`,
32 | error: error.message
33 | };
34 | }
35 | }
36 |
37 | /**
38 | * Gets data from a specific OpenDOSM dataset
39 | *
40 | * @param {Object} params - Parameters
41 | * @param {string} params.id - Dataset ID
42 | * @param {Object} params.filter - Optional filter parameters
43 | * @param {Object} params.sort - Optional sort parameters
44 | * @param {number} params.limit - Optional limit parameter
45 | * @returns {Promise<Object>} - Dataset data
46 | */
47 | async function getDataset(params = {}) {
48 | try {
49 | if (!params.id) {
50 | throw new Error('Dataset ID is required');
51 | }
52 |
53 | const { id, ...queryParams } = params;
54 | const result = await dosmClient.getDataset(id, queryParams);
55 |
56 | return {
57 | success: true,
58 | message: `Successfully retrieved OpenDOSM dataset: ${id}`,
59 | data: result
60 | };
61 | } catch (error) {
62 | return {
63 | success: false,
64 | message: `Failed to get OpenDOSM dataset: ${error.message}`,
65 | error: error.message
66 | };
67 | }
68 | }
69 |
70 | module.exports = {
71 | listDatasets,
72 | getDataset
73 | };
74 |
```
--------------------------------------------------------------------------------
/scripts/index-dashboards.cjs:
--------------------------------------------------------------------------------
```
1 | /**
2 | * Script to generate a dashboard index file from individual dashboard JSON files
3 | */
4 | const fs = require('fs');
5 | const path = require('path');
6 |
7 | // Path to dashboards directory
8 | const dashboardsDir = path.join(process.cwd(), 'dashboards');
9 |
10 | // Path to output file
11 | const outputFile = path.join(process.cwd(), 'scripts', 'dashboards-index.js');
12 |
13 | // Read all dashboard files
14 | try {
15 | console.log('Reading dashboard files from:', dashboardsDir);
16 | const files = fs.readdirSync(dashboardsDir).filter(file => file.endsWith('.json'));
17 | console.log(`Found ${files.length} dashboard files`);
18 |
19 | // Read and parse each file
20 | const dashboards = files.map(file => {
21 | try {
22 | const content = fs.readFileSync(path.join(dashboardsDir, file), 'utf8');
23 | return JSON.parse(content);
24 | } catch (error) {
25 | console.error(`Error reading dashboard file ${file}:`, error);
26 | return null;
27 | }
28 | }).filter(Boolean);
29 |
30 | // Generate the output file content
31 | const timestamp = new Date().toISOString();
32 | const outputContent = `// Generated from local dashboard files
33 | // Timestamp: ${timestamp}
34 | // Total dashboards: ${dashboards.length}
35 |
36 | export const DASHBOARDS_INDEX = ${JSON.stringify(dashboards, null, 2)};
37 | `;
38 |
39 | // Write the output file
40 | fs.writeFileSync(outputFile, outputContent, 'utf8');
41 | console.log(`Dashboard index generated successfully: ${outputFile}`);
42 |
43 | // Create TypeScript declaration file
44 | const declarationFile = path.join(process.cwd(), 'scripts', 'dashboards-index.d.ts');
45 | const declarationContent = `export declare const DASHBOARDS_INDEX: any[];
46 | `;
47 | fs.writeFileSync(declarationFile, declarationContent, 'utf8');
48 | console.log(`TypeScript declaration file generated: ${declarationFile}`);
49 |
50 | } catch (error) {
51 | console.error('Error generating dashboard index:', error);
52 | process.exit(1);
53 | }
54 |
```
--------------------------------------------------------------------------------
/src/tools/transport.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Transport MCP Tools
3 | *
4 | * Tools for accessing public transportation data from the Malaysia Open Data API
5 | */
6 |
7 | const { createTransportClient } = require('../api/transport');
8 |
9 | // Create client instance with default configuration
10 | const transportClient = createTransportClient();
11 |
12 | /**
13 | * Lists available transport agencies
14 | *
15 | * @returns {Promise<Object>} - List of available agencies
16 | */
17 | async function listAgencies() {
18 | try {
19 | const result = await transportClient.listAgencies();
20 |
21 | return {
22 | success: true,
23 | message: 'Successfully retrieved transport agencies',
24 | data: result
25 | };
26 | } catch (error) {
27 | return {
28 | success: false,
29 | message: `Failed to list transport agencies: ${error.message}`,
30 | error: error.message
31 | };
32 | }
33 | }
34 |
35 | /**
36 | * Gets GTFS data for a specific agency
37 | *
38 | * @param {Object} params - Parameters
39 | * @param {string} params.agencyId - Agency ID (e.g., 'mybas-jb', 'ktmb', 'prasarana')
40 | * @param {string} params.dataType - Data type ('static' or 'realtime')
41 | * @param {Object} params.filter - Optional filter parameters
42 | * @param {number} params.limit - Optional limit parameter
43 | * @returns {Promise<Object>} - GTFS data
44 | */
45 | async function getData(params = {}) {
46 | try {
47 | if (!params.agencyId) {
48 | throw new Error('Agency ID is required');
49 | }
50 |
51 | const { agencyId, dataType = 'static', filter, limit, ...otherParams } = params;
52 |
53 | const queryParams = {
54 | ...otherParams,
55 | ...(filter && { filter }),
56 | ...(limit && { limit })
57 | };
58 |
59 | const result = await transportClient.getData({
60 | agencyId,
61 | dataType,
62 | queryParams
63 | });
64 |
65 | return {
66 | success: true,
67 | message: `Successfully retrieved ${dataType} GTFS data for agency: ${agencyId}`,
68 | data: result
69 | };
70 | } catch (error) {
71 | return {
72 | success: false,
73 | message: `Failed to get transport data: ${error.message}`,
74 | error: error.message
75 | };
76 | }
77 | }
78 |
79 | module.exports = {
80 | listAgencies,
81 | getData
82 | };
83 |
```
--------------------------------------------------------------------------------
/deploy/nginx-mcp.conf:
--------------------------------------------------------------------------------
```
1 | # Nginx location block for Malaysia Open Data MCP Server
2 | # Add this to your existing mcp.techmavie.digital server block
3 | #
4 | # Endpoint: https://mcp.techmavie.digital/datagovmy/mcp
5 |
6 | # ============================================================================
7 | # Malaysia Open Data MCP Server
8 | # Endpoint: https://mcp.techmavie.digital/datagovmy/mcp
9 | # ============================================================================
10 | location /datagovmy/ {
11 | # Proxy to Malaysia Open Data MCP container (port 8083)
12 | proxy_pass http://127.0.0.1:8083/;
13 |
14 | # Required headers for Streamable HTTP transport
15 | proxy_http_version 1.1;
16 | proxy_set_header Host $host;
17 | proxy_set_header X-Real-IP $remote_addr;
18 | proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
19 | proxy_set_header X-Forwarded-Proto $scheme;
20 |
21 | # Important for streaming responses
22 | proxy_buffering off;
23 | proxy_cache off;
24 | proxy_read_timeout 3600s;
25 | proxy_send_timeout 3600s;
26 |
27 | # For potential WebSocket upgrade (future compatibility)
28 | proxy_set_header Upgrade $http_upgrade;
29 | proxy_set_header Connection "upgrade";
30 |
31 | # MCP Session ID header
32 | proxy_set_header Mcp-Session-Id $http_mcp_session_id;
33 | proxy_pass_header Mcp-Session-Id;
34 |
35 | # CORS headers (if not handled by the app)
36 | add_header Access-Control-Allow-Origin "*" always;
37 | add_header Access-Control-Allow-Methods "GET, POST, DELETE, OPTIONS" always;
38 | add_header Access-Control-Allow-Headers "Content-Type, Accept, Authorization, Mcp-Session-Id" always;
39 | add_header Access-Control-Expose-Headers "Mcp-Session-Id" always;
40 |
41 | # Handle preflight requests
42 | if ($request_method = 'OPTIONS') {
43 | add_header Access-Control-Allow-Origin "*";
44 | add_header Access-Control-Allow-Methods "GET, POST, DELETE, OPTIONS";
45 | add_header Access-Control-Allow-Headers "Content-Type, Accept, Authorization, Mcp-Session-Id";
46 | add_header Access-Control-Max-Age 86400;
47 | add_header Content-Length 0;
48 | add_header Content-Type text/plain;
49 | return 204;
50 | }
51 | }
52 |
```
--------------------------------------------------------------------------------
/package.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "name": "mcp-datagovmy",
3 | "version": "1.0.0",
4 | "description": "Malaysia Open Data MCP Server for Smithery",
5 | "main": "./src/index.ts",
6 | "module": "./src/index.ts",
7 | "type": "module",
8 | "files": [
9 | "dist/**/*",
10 | "README.md",
11 | "LICENSE",
12 | "TOOLS.md",
13 | "PROMPT.md"
14 | ],
15 | "scripts": {
16 | "start": "node index.js",
17 | "start:http": "node dist/src/http-server.js",
18 | "build": "tsc",
19 | "prepublishOnly": "npm run build",
20 | "dev": "npx @smithery/[email protected] dev",
21 | "dev:http": "tsx src/http-server.ts",
22 | "deploy": "npm run build && npx @smithery/[email protected] deploy",
23 | "test": "echo \"Error: no test specified\" && exit 1",
24 | "lint": "eslint src/**/*.ts",
25 | "lint:fix": "eslint src/**/*.ts --fix",
26 | "format": "prettier --write src/**/*.ts",
27 | "format:check": "prettier --check src/**/*.ts",
28 | "typecheck": "tsc --noEmit"
29 | },
30 | "keywords": [
31 | "mcp",
32 | "smithery",
33 | "malaysia",
34 | "open-data",
35 | "data-catalogue",
36 | "dosm",
37 | "weather",
38 | "transport"
39 | ],
40 | "author": "hithereiamaliff",
41 | "license": "MIT",
42 | "repository": {
43 | "type": "git",
44 | "url": "https://github.com/hithereiamaliff/mcp-datagovmy.git"
45 | },
46 | "dependencies": {
47 | "@aws-sdk/client-location": "^3.848.0",
48 | "@modelcontextprotocol/sdk": "^1.12.1",
49 | "axios": "^1.11.0",
50 | "cors": "^2.8.5",
51 | "csv-parser": "^3.0.0",
52 | "dotenv": "^17.2.1",
53 | "express": "^4.21.0",
54 | "firebase-admin": "^13.6.0",
55 | "gtfs-realtime-bindings": "^1.1.1",
56 | "hyparquet": "^1.17.1",
57 | "hyparquet-compressors": "^1.1.1",
58 | "jszip": "^3.10.1",
59 | "p-limit": "^5.0.0",
60 | "zod": "^3.25.57"
61 | },
62 | "devDependencies": {
63 | "@smithery/cli": "^1.2.16",
64 | "@types/cors": "^2.8.17",
65 | "@types/express": "^4.17.21",
66 | "@types/node": "^22.15.31",
67 | "@typescript-eslint/eslint-plugin": "^8.34.0",
68 | "@typescript-eslint/parser": "^8.34.0",
69 | "eslint": "^9.28.0",
70 | "eslint-config-prettier": "^10.1.5",
71 | "eslint-plugin-prettier": "^5.4.1",
72 | "prettier": "^3.5.3",
73 | "tsx": "^4.7.0",
74 | "typescript": "^5.8.3"
75 | },
76 | "engines": {
77 | "node": ">=18.0.0"
78 | }
79 | }
80 |
```
--------------------------------------------------------------------------------
/mcp-server.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Malaysia Open Data MCP - Universal Server Entry Point
3 | *
4 | * This file is a self-contained, simplified entry point designed for robust deployment on Smithery.
5 | */
6 |
7 | // All tool logic is included here to avoid pathing issues.
8 | const catalogueTools = {
9 | listDatasets: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
10 | getDataset: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
11 | searchDatasets: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
12 | };
13 |
14 | const dosmTools = {
15 | listDatasets: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
16 | getDataset: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
17 | };
18 |
19 | const weatherTools = {
20 | getForecast: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
21 | getWarnings: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
22 | getEarthquakeWarnings: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
23 | };
24 |
25 | const transportTools = {
26 | listAgencies: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
27 | getData: async (params) => ({ message: 'Tool not fully implemented in this version', ...params }),
28 | };
29 |
30 | const testTools = {
31 | hello: async () => ({ message: 'Hello from the simplified MCP server!' }),
32 | };
33 |
34 | /**
35 | * Main server function that Smithery expects.
36 | */
37 | function server({ sessionId, config }) {
38 | const tools = {
39 | // Data Catalogue Tools
40 | list_datasets: catalogueTools.listDatasets,
41 | get_dataset: catalogueTools.getDataset,
42 | search_datasets: catalogueTools.searchDatasets,
43 |
44 | // OpenDOSM Tools
45 | list_dosm_datasets: dosmTools.listDatasets,
46 | get_dosm_dataset: dosmTools.getDataset,
47 |
48 | // Weather Tools
49 | get_weather_forecast: weatherTools.getForecast,
50 | get_weather_warnings: weatherTools.getWarnings,
51 | get_earthquake_warnings: weatherTools.getEarthquakeWarnings,
52 |
53 | // Transport Tools
54 | list_transport_agencies: transportTools.listAgencies,
55 | get_transport_data: transportTools.getData,
56 |
57 | // Test Tools
58 | hello: testTools.hello,
59 | };
60 |
61 | return {
62 | connect: () => tools,
63 | };
64 | }
65 |
66 | // Export the server function for CommonJS compatibility.
67 | module.exports = server;
68 | module.exports.default = server;
69 |
```
--------------------------------------------------------------------------------
/src/tools/catalogue.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Data Catalogue MCP Tools
3 | *
4 | * Tools for accessing and searching the Malaysia Open Data Catalogue
5 | */
6 |
7 | // Import the createCatalogueClient function directly
8 | const createCatalogueClient = require('../api/catalogue');
9 |
10 | // Create client instance with default configuration
11 | const catalogueClient = createCatalogueClient();
12 |
13 | /**
14 | * Lists available datasets in the Data Catalogue
15 | *
16 | * @param {Object} params - Optional parameters
17 | * @param {number} params.limit - Maximum number of datasets to return
18 | * @returns {Promise<Object>} - List of datasets
19 | */
20 | async function listDatasets(params = {}) {
21 | try {
22 | const result = await catalogueClient.listDatasets(params);
23 |
24 | return {
25 | success: true,
26 | message: 'Successfully retrieved datasets',
27 | data: result
28 | };
29 | } catch (error) {
30 | return {
31 | success: false,
32 | message: `Failed to list datasets: ${error.message}`,
33 | error: error.message
34 | };
35 | }
36 | }
37 |
38 | /**
39 | * Gets data from a specific dataset
40 | *
41 | * @param {Object} params - Parameters
42 | * @param {string} params.id - Dataset ID
43 | * @param {Object} params.filter - Optional filter parameters
44 | * @param {Object} params.sort - Optional sort parameters
45 | * @param {number} params.limit - Optional limit parameter
46 | * @returns {Promise<Object>} - Dataset data
47 | */
48 | async function getDataset(params = {}) {
49 | try {
50 | if (!params.id) {
51 | throw new Error('Dataset ID is required');
52 | }
53 |
54 | const { id, ...queryParams } = params;
55 | const result = await catalogueClient.getDataset(id, queryParams);
56 |
57 | return {
58 | success: true,
59 | message: `Successfully retrieved dataset: ${id}`,
60 | data: result
61 | };
62 | } catch (error) {
63 | return {
64 | success: false,
65 | message: `Failed to get dataset: ${error.message}`,
66 | error: error.message
67 | };
68 | }
69 | }
70 |
71 | /**
72 | * Searches across datasets in the Data Catalogue
73 | *
74 | * @param {Object} params - Search parameters
75 | * @param {string} params.query - Search query
76 | * @param {number} params.limit - Maximum number of results to return
77 | * @returns {Promise<Object>} - Search results
78 | */
79 | async function searchDatasets(params = {}) {
80 | try {
81 | // For searching across datasets, we'll use the contains parameter
82 | // to search for the query in dataset metadata
83 | const result = await catalogueClient.searchDatasets({
84 | meta: true,
85 | ...(params.query && { contains: params.query }),
86 | ...(params.limit && { limit: params.limit })
87 | });
88 |
89 | return {
90 | success: true,
91 | message: 'Successfully searched datasets',
92 | data: result
93 | };
94 | } catch (error) {
95 | return {
96 | success: false,
97 | message: `Failed to search datasets: ${error.message}`,
98 | error: error.message
99 | };
100 | }
101 | }
102 |
103 | module.exports = {
104 | listDatasets,
105 | getDataset,
106 | searchDatasets
107 | };
108 |
```
--------------------------------------------------------------------------------
/src/dosm.tools.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
2 | import { z } from 'zod';
3 | import axios from 'axios';
4 | import { prefixToolName } from './utils/tool-naming.js';
5 |
6 | // API Base URL for Malaysia Open Data API
7 | const API_BASE_URL = 'https://api.data.gov.my';
8 |
9 | // OpenDOSM endpoint - correct endpoint for Malaysia Open Data API
10 | const OPENDOSM_ENDPOINT = '/opendosm';
11 |
12 | export function registerDosmTools(server: McpServer) {
13 | // List DOSM datasets
14 | server.tool(
15 | prefixToolName('list_dosm_datasets'),
16 | 'Lists available datasets from the Department of Statistics Malaysia',
17 | {
18 | dataset_id: z.string().optional().describe('Optional specific dataset ID to list (e.g., "cpi_core", "cpi_strata")'),
19 | limit: z.number().min(1).optional().describe('Maximum number of datasets to return'),
20 | offset: z.number().min(0).optional().describe('Number of datasets to skip'),
21 | },
22 | async ({ dataset_id, limit = 10, offset = 0 }) => {
23 | try {
24 | // Use the correct endpoint structure
25 | const url = `${API_BASE_URL}${OPENDOSM_ENDPOINT}`;
26 | // If dataset_id is provided, get specific dataset, otherwise list available datasets
27 | const params: Record<string, any> = { limit, meta: true };
28 |
29 | if (dataset_id) {
30 | params.id = dataset_id;
31 | }
32 |
33 | const response = await axios.get(url, { params });
34 | const data = response.data;
35 |
36 | return {
37 | content: [
38 | {
39 | type: 'text',
40 | text: JSON.stringify(data, null, 2),
41 | },
42 | ],
43 | };
44 | } catch (error) {
45 | return {
46 | content: [
47 | {
48 | type: 'text',
49 | text: `Error fetching DOSM datasets: ${error instanceof Error ? error.message : 'Unknown error'}`,
50 | },
51 | ],
52 | };
53 | }
54 | }
55 | );
56 |
57 | // Get DOSM dataset
58 | server.tool(
59 | prefixToolName('get_dosm_dataset'),
60 | 'Gets data from a specific DOSM dataset',
61 | {
62 | id: z.string().describe('ID of the dataset to retrieve (e.g., "cpi_core", "cpi_strata")'),
63 | limit: z.number().min(1).optional().describe('Maximum number of records to return'),
64 | offset: z.number().min(0).optional().describe('Number of records to skip'),
65 | },
66 | async ({ id, limit = 10, offset = 0 }) => {
67 | try {
68 | // Use the correct endpoint structure with dataset ID as query parameter
69 | const url = `${API_BASE_URL}${OPENDOSM_ENDPOINT}`;
70 | const params = { id, limit, offset };
71 |
72 | const response = await axios.get(url, { params });
73 | const data = response.data;
74 |
75 | return {
76 | content: [
77 | {
78 | type: 'text',
79 | text: JSON.stringify(data, null, 2),
80 | },
81 | ],
82 | };
83 | } catch (error) {
84 | return {
85 | content: [
86 | {
87 | type: 'text',
88 | text: `Error fetching DOSM dataset: ${error instanceof Error ? error.message : 'Unknown error'}`,
89 | },
90 | ],
91 | };
92 | }
93 | }
94 | );
95 | }
96 |
```
--------------------------------------------------------------------------------
/src/api/transport.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Transport API Client
3 | *
4 | * Handles communication with the GTFS Static and GTFS Realtime API endpoints
5 | */
6 |
7 | const { createClient } = require('./client');
8 |
9 | /**
10 | * Creates a Transport API client
11 | *
12 | * @param {Object} options - Client configuration options
13 | * @returns {Object} - Transport API client
14 | */
15 | function createTransportClient(options = {}) {
16 | const client = createClient(options);
17 |
18 | // Transport API endpoints
19 | const GTFS_STATIC_ENDPOINT = '/gtfs-static';
20 | const GTFS_REALTIME_ENDPOINT = '/gtfs-realtime';
21 |
22 | // Available transport agencies
23 | const AGENCIES = {
24 | MYBAS_JB: 'mybas-jb',
25 | KTMB: 'ktmb',
26 | PRASARANA: 'prasarana'
27 | };
28 |
29 | return {
30 | /**
31 | * Lists available transport agencies
32 | *
33 | * @returns {Promise<Object>} - List of available agencies
34 | */
35 | async listAgencies() {
36 | return {
37 | agencies: [
38 | {
39 | id: AGENCIES.MYBAS_JB,
40 | name: 'myBAS Johor Bahru',
41 | description: 'Bus service operator in Johor Bahru',
42 | website: 'https://www.causewaylink.com.my/mybas/en/'
43 | },
44 | {
45 | id: AGENCIES.KTMB,
46 | name: 'KTMB (Keretapi Tanah Melayu Berhad)',
47 | description: 'Railway operator providing train services across Malaysia',
48 | website: 'https://www.ktmb.com.my/'
49 | },
50 | {
51 | id: AGENCIES.PRASARANA,
52 | name: 'Prasarana',
53 | description: 'Public transport operator responsible for LRT, MRT, monorail, and bus services',
54 | website: 'https://myrapid.com.my/'
55 | }
56 | ]
57 | };
58 | },
59 |
60 | /**
61 | * Gets GTFS Static data for a specific agency
62 | *
63 | * @param {string} agencyId - Agency ID
64 | * @param {Object} params - Query parameters
65 | * @returns {Promise<Object>} - GTFS Static data
66 | */
67 | async getStaticData(agencyId, params = {}) {
68 | return client.request(`${GTFS_STATIC_ENDPOINT}/${agencyId}`, params);
69 | },
70 |
71 | /**
72 | * Gets GTFS Realtime data for a specific agency
73 | *
74 | * @param {string} agencyId - Agency ID
75 | * @param {Object} params - Query parameters
76 | * @returns {Promise<Object>} - GTFS Realtime data
77 | */
78 | async getRealtimeData(agencyId, params = {}) {
79 | return client.request(`${GTFS_REALTIME_ENDPOINT}/${agencyId}`, params);
80 | },
81 |
82 | /**
83 | * Gets GTFS data (static or realtime) for a specific agency
84 | *
85 | * @param {Object} params - Parameters
86 | * @param {string} params.agencyId - Agency ID
87 | * @param {string} params.dataType - Data type ('static' or 'realtime')
88 | * @param {Object} params.queryParams - Additional query parameters
89 | * @returns {Promise<Object>} - GTFS data
90 | */
91 | async getData({ agencyId, dataType = 'static', queryParams = {} }) {
92 | if (!agencyId) {
93 | throw new Error('Agency ID is required');
94 | }
95 |
96 | if (dataType === 'static') {
97 | return this.getStaticData(agencyId, queryParams);
98 | } else if (dataType === 'realtime') {
99 | return this.getRealtimeData(agencyId, queryParams);
100 | } else {
101 | throw new Error('Invalid data type. Must be "static" or "realtime"');
102 | }
103 | }
104 | };
105 | }
106 |
107 | module.exports = {
108 | createTransportClient
109 | };
110 |
```
--------------------------------------------------------------------------------
/scripts/update-tool-names.ts:
--------------------------------------------------------------------------------
```typescript
1 | /**
2 | * Script to update all tool names with the datagovmy_ prefix
3 | *
4 | * This script will:
5 | * 1. Find all server.tool() calls in the src directory
6 | * 2. Update them to use prefixToolName() helper
7 | * 3. Create a report of all changes made
8 | */
9 |
10 | import * as fs from 'fs';
11 | import * as path from 'path';
12 |
13 | // Define the directory to search
14 | const srcDir = path.join(process.cwd(), 'src');
15 |
16 | // Regular expression to find server.tool calls
17 | const toolRegex = /server\.tool\(\s*['"]([^'"]+)['"]/g;
18 |
19 | // Function to process a file
20 | function processFile(filePath: string): { file: string, changes: { original: string, updated: string }[] } {
21 | // Read the file content
22 | let content = fs.readFileSync(filePath, 'utf8');
23 | const originalContent = content;
24 |
25 | // Find all tool registrations
26 | const changes: { original: string, updated: string }[] = [];
27 | let match;
28 |
29 | // Add the import if it doesn't exist
30 | if (!content.includes("import { prefixToolName }")) {
31 | // Find the last import statement
32 | const lastImportIndex = content.lastIndexOf('import');
33 | if (lastImportIndex !== -1) {
34 | const endOfImport = content.indexOf(';', lastImportIndex) + 1;
35 | const beforeImports = content.substring(0, endOfImport);
36 | const afterImports = content.substring(endOfImport);
37 | content = beforeImports + "\nimport { prefixToolName } from './utils/tool-naming.js';" + afterImports;
38 | }
39 | }
40 |
41 | // Replace all tool registrations
42 | while ((match = toolRegex.exec(originalContent)) !== null) {
43 | const toolName = match[1];
44 | const original = `server.tool(\n '${toolName}'`;
45 | const updated = `server.tool(\n prefixToolName('${toolName}')`;
46 |
47 | // Only update if the tool name doesn't already have the prefix
48 | if (!toolName.startsWith('datagovmy_')) {
49 | content = content.replace(
50 | `server.tool(\n '${toolName}'`,
51 | `server.tool(\n prefixToolName('${toolName}')`
52 | );
53 | content = content.replace(
54 | `server.tool('${toolName}'`,
55 | `server.tool(prefixToolName('${toolName}')`
56 | );
57 | changes.push({ original: toolName, updated: `datagovmy_${toolName}` });
58 | }
59 | }
60 |
61 | // Write the updated content back to the file
62 | if (originalContent !== content) {
63 | fs.writeFileSync(filePath, content, 'utf8');
64 | }
65 |
66 | return { file: filePath, changes };
67 | }
68 |
69 | // Function to recursively process all files in a directory
70 | function processDirectory(dir: string): { file: string, changes: { original: string, updated: string }[] }[] {
71 | const results: { file: string, changes: { original: string, updated: string }[] }[] = [];
72 | const files = fs.readdirSync(dir);
73 |
74 | for (const file of files) {
75 | const filePath = path.join(dir, file);
76 | const stats = fs.statSync(filePath);
77 |
78 | if (stats.isDirectory()) {
79 | results.push(...processDirectory(filePath));
80 | } else if (stats.isFile() && (file.endsWith('.ts') || file.endsWith('.js'))) {
81 | const result = processFile(filePath);
82 | if (result.changes.length > 0) {
83 | results.push(result);
84 | }
85 | }
86 | }
87 |
88 | return results;
89 | }
90 |
91 | // Main function
92 | function main() {
93 | console.log('Updating tool names with datagovmy_ prefix...');
94 |
95 | // Process all files
96 | const results = processDirectory(srcDir);
97 |
98 | // Print the results
99 | console.log('\nChanges made:');
100 | let totalChanges = 0;
101 |
102 | for (const result of results) {
103 | console.log(`\nFile: ${path.relative(process.cwd(), result.file)}`);
104 | for (const change of result.changes) {
105 | console.log(` ${change.original} -> ${change.updated}`);
106 | totalChanges++;
107 | }
108 | }
109 |
110 | console.log(`\nTotal changes: ${totalChanges}`);
111 | }
112 |
113 | main();
114 |
```
--------------------------------------------------------------------------------
/index.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Malaysia Open Data MCP - Simplified Server
3 | *
4 | * A completely standalone MCP server implementation designed for direct deployment to Smithery.
5 | * This file contains everything needed to run the server without external dependencies.
6 | */
7 |
8 | // Define our tools
9 | const tools = {
10 | // Simple test tool
11 | hello: async () => {
12 | return {
13 | message: "Hello from Malaysia Open Data MCP!",
14 | timestamp: new Date().toISOString()
15 | };
16 | },
17 |
18 | // Data Catalogue Tools
19 | list_datasets: async ({ limit = 10, offset = 0 }) => {
20 | return {
21 | message: "This is a placeholder for the list_datasets tool",
22 | params: { limit, offset },
23 | datasets: [
24 | { id: "dataset-1", name: "Economic Indicators" },
25 | { id: "dataset-2", name: "Population Statistics" },
26 | { id: "dataset-3", name: "Education Metrics" }
27 | ]
28 | };
29 | },
30 |
31 | get_dataset: async ({ id, limit = 10, offset = 0, filter = "" }) => {
32 | return {
33 | message: `This is a placeholder for the get_dataset tool with ID: ${id}`,
34 | params: { id, limit, offset, filter },
35 | data: [
36 | { year: 2023, value: 100 },
37 | { year: 2024, value: 120 },
38 | { year: 2025, value: 150 }
39 | ]
40 | };
41 | },
42 |
43 | // Add more tool implementations as needed...
44 | };
45 |
46 | /**
47 | * Main MCP server function
48 | */
49 | function server({ sessionId }) {
50 | console.log(`Starting MCP server session: ${sessionId}`);
51 |
52 | return {
53 | connect: () => tools
54 | };
55 | }
56 |
57 | // Export for Smithery compatibility
58 | module.exports = server;
59 | module.exports.default = server;
60 |
61 | // If this file is run directly, start an HTTP server
62 | if (require.main === module) {
63 | const http = require('http');
64 | const PORT = process.env.PORT || 8182;
65 |
66 | const httpServer = http.createServer((req, res) => {
67 | // Enable CORS
68 | res.setHeader('Access-Control-Allow-Origin', '*');
69 | res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS');
70 | res.setHeader('Access-Control-Allow-Headers', 'Content-Type');
71 |
72 | if (req.method === 'OPTIONS') {
73 | res.writeHead(204);
74 | res.end();
75 | return;
76 | }
77 |
78 | // Root endpoint
79 | if (req.url === '/' && req.method === 'GET') {
80 | res.writeHead(200, { 'Content-Type': 'application/json' });
81 | res.end(JSON.stringify({
82 | name: 'datagovmy-mcp-hithereiamaliff',
83 | displayName: 'Malaysia Open Data MCP',
84 | description: 'MCP server for accessing Malaysia\'s Open Data APIs',
85 | version: '1.0.0',
86 | tools: Object.keys(tools)
87 | }));
88 | return;
89 | }
90 |
91 | // Handle tool invocation
92 | if (req.url.startsWith('/invoke/') && req.method === 'POST') {
93 | const toolName = req.url.split('/')[2];
94 |
95 | if (!tools[toolName]) {
96 | res.writeHead(404, { 'Content-Type': 'application/json' });
97 | res.end(JSON.stringify({ error: `Tool '${toolName}' not found` }));
98 | return;
99 | }
100 |
101 | let body = '';
102 | req.on('data', chunk => {
103 | body += chunk.toString();
104 | });
105 |
106 | req.on('end', async () => {
107 | try {
108 | const params = body ? JSON.parse(body) : {};
109 | const result = await tools[toolName](params);
110 |
111 | res.writeHead(200, { 'Content-Type': 'application/json' });
112 | res.end(JSON.stringify({ result }));
113 | } catch (error) {
114 | console.error(`Error processing tool ${toolName}:`, error);
115 | res.writeHead(500, { 'Content-Type': 'application/json' });
116 | res.end(JSON.stringify({ error: error.message || 'Internal server error' }));
117 | }
118 | });
119 | return;
120 | }
121 |
122 | // Not found
123 | res.writeHead(404, { 'Content-Type': 'application/json' });
124 | res.end(JSON.stringify({ error: 'Not found' }));
125 | });
126 |
127 | httpServer.listen(PORT, () => {
128 | console.log(`Malaysia Open Data MCP server running on port ${PORT}`);
129 | });
130 | }
131 |
```
--------------------------------------------------------------------------------
/src/api/client.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Base API Client for Malaysia Open Data API
3 | *
4 | * Handles communication with the Malaysia Open Data API, including:
5 | * - Authentication
6 | * - Rate limiting
7 | * - Request/response handling
8 | */
9 |
10 | const axios = require('axios');
11 | const pLimit = require('p-limit');
12 | const { buildQueryParams } = require('../utils/query-builder');
13 |
14 | // Base URL for all API requests
15 | const BASE_URL = 'https://api.data.gov.my';
16 |
17 | // Rate limiting configuration
18 | // Default: 5 requests per minute (300 requests per hour)
19 | const DEFAULT_RATE_LIMIT = 5;
20 | const DEFAULT_INTERVAL_MS = 12000; // 12 seconds between requests
21 |
22 | /**
23 | * Creates a rate-limited API client for the Malaysia Open Data API
24 | *
25 | * @param {Object} options - Client configuration options
26 | * @param {string} options.apiToken - Optional API token for authentication
27 | * @param {number} options.rateLimit - Maximum number of requests per minute
28 | * @param {number} options.intervalMs - Minimum interval between requests in milliseconds
29 | * @returns {Object} - API client instance
30 | */
31 | function createClient(options = {}) {
32 | const {
33 | apiToken,
34 | rateLimit = DEFAULT_RATE_LIMIT,
35 | intervalMs = DEFAULT_INTERVAL_MS
36 | } = options;
37 |
38 | // Create rate limiter
39 | const limit = pLimit(1); // Only 1 concurrent request
40 | const queue = [];
41 | let lastRequestTime = 0;
42 |
43 | // Create axios instance with default configuration
44 | const axiosInstance = axios.create({
45 | baseURL: BASE_URL,
46 | headers: {
47 | 'Accept': 'application/json',
48 | 'Content-Type': 'application/json',
49 | ...(apiToken && { 'Authorization': `Token ${apiToken}` })
50 | }
51 | });
52 |
53 | /**
54 | * Makes a rate-limited request to the API
55 | *
56 | * @param {string} endpoint - API endpoint (without base URL)
57 | * @param {Object} params - Query parameters
58 | * @returns {Promise<Object>} - API response data
59 | */
60 | async function request(endpoint, params = {}) {
61 | return limit(async () => {
62 | // Enforce minimum interval between requests
63 | const now = Date.now();
64 | const timeSinceLastRequest = now - lastRequestTime;
65 |
66 | if (timeSinceLastRequest < intervalMs) {
67 | await new Promise(resolve => setTimeout(resolve, intervalMs - timeSinceLastRequest));
68 | }
69 |
70 | try {
71 | // Build query parameters
72 | const queryParams = buildQueryParams(params);
73 |
74 | // Make request
75 | const response = await axiosInstance.get(endpoint, { params: queryParams });
76 |
77 | // Update last request time
78 | lastRequestTime = Date.now();
79 |
80 | return response.data;
81 | } catch (error) {
82 | if (error.response) {
83 | // The request was made and the server responded with a status code
84 | // that falls out of the range of 2xx
85 | const { status, data } = error.response;
86 |
87 | if (status === 429) {
88 | // Too Many Requests - retry after a delay
89 | console.warn('Rate limit exceeded. Retrying after delay...');
90 | await new Promise(resolve => setTimeout(resolve, intervalMs * 2));
91 | return request(endpoint, params);
92 | }
93 |
94 | throw new Error(`API Error (${status}): ${JSON.stringify(data)}`);
95 | } else if (error.request) {
96 | // The request was made but no response was received
97 | throw new Error('No response received from API');
98 | } else {
99 | // Something happened in setting up the request
100 | throw new Error(`Request Error: ${error.message}`);
101 | }
102 | }
103 | });
104 | }
105 |
106 | return {
107 | request,
108 |
109 | /**
110 | * Gets the current API client configuration
111 | *
112 | * @returns {Object} - Current configuration
113 | */
114 | getConfig() {
115 | return {
116 | baseURL: BASE_URL,
117 | hasToken: !!apiToken,
118 | rateLimit,
119 | intervalMs
120 | };
121 | }
122 | };
123 | }
124 |
125 | module.exports = {
126 | createClient
127 | };
128 |
```
--------------------------------------------------------------------------------
/scripts/extract-dataset-ids.js:
--------------------------------------------------------------------------------
```javascript
1 | import fs from 'fs';
2 | import path from 'path';
3 | import https from 'https';
4 | import { fileURLToPath } from 'url';
5 |
6 | // Get current directory in ES modules
7 | const __filename = fileURLToPath(import.meta.url);
8 | const __dirname = path.dirname(__filename);
9 |
10 | // GitHub API URL for the data-catalogue directory
11 | const apiUrl = 'https://api.github.com/repos/data-gov-my/datagovmy-meta/contents/data-catalogue';
12 |
13 | // Function to fetch data from GitHub API
14 | function fetchFromGitHub(url) {
15 | return new Promise((resolve, reject) => {
16 | const options = {
17 | headers: {
18 | 'User-Agent': 'Node.js GitHub Dataset Extractor'
19 | }
20 | };
21 |
22 | https.get(url, options, (res) => {
23 | let data = '';
24 |
25 | res.on('data', (chunk) => {
26 | data += chunk;
27 | });
28 |
29 | res.on('end', () => {
30 | try {
31 | const jsonData = JSON.parse(data);
32 | resolve(jsonData);
33 | } catch (error) {
34 | reject(error);
35 | }
36 | });
37 | }).on('error', (error) => {
38 | reject(error);
39 | });
40 | });
41 | }
42 |
43 | // Function to extract dataset details from a single file
44 | async function extractDatasetDetails(fileInfo) {
45 | // Extract dataset ID from filename (remove .json extension)
46 | const datasetId = path.basename(fileInfo.name, '.json');
47 |
48 | // Fetch the file content to get the title
49 | const contentUrl = fileInfo.download_url;
50 |
51 | try {
52 | // Fetch the raw content
53 | const rawContent = await new Promise((resolve, reject) => {
54 | https.get(contentUrl, (res) => {
55 | let data = '';
56 |
57 | res.on('data', (chunk) => {
58 | data += chunk;
59 | });
60 |
61 | res.on('end', () => {
62 | resolve(data);
63 | });
64 | }).on('error', (error) => {
65 | reject(error);
66 | });
67 | });
68 |
69 | // Parse the JSON content
70 | const content = JSON.parse(rawContent);
71 |
72 | // Extract the English title
73 | const description = content.title_en || datasetId;
74 |
75 | return { id: datasetId, description };
76 | } catch (error) {
77 | console.error(`Error fetching details for ${datasetId}:`, error.message);
78 | // Return basic info if we can't get the title
79 | return { id: datasetId, description: datasetId };
80 | }
81 | }
82 |
83 | // Main function to extract all dataset IDs
84 | async function extractAllDatasetIds() {
85 | try {
86 | // Fetch the list of files in the data-catalogue directory
87 | const files = await fetchFromGitHub(apiUrl);
88 |
89 | // Filter for JSON files only
90 | const jsonFiles = files.filter(file => file.name.endsWith('.json'));
91 |
92 | console.log(`Found ${jsonFiles.length} JSON files in the data-catalogue directory`);
93 |
94 | // Extract dataset details from each file
95 | const datasets = [];
96 | for (const file of jsonFiles) {
97 | const dataset = await extractDatasetDetails(file);
98 | datasets.push(dataset);
99 | console.log(`Processed: ${dataset.id} - ${dataset.description}`);
100 | }
101 |
102 | // Sort datasets alphabetically by ID
103 | datasets.sort((a, b) => a.id.localeCompare(b.id));
104 |
105 | // Format the datasets as JavaScript code
106 | const formattedDatasets = datasets.map(dataset =>
107 | ` { id: '${dataset.id}', description: '${dataset.description.replace(/'/g, "\\'")}' }`
108 | ).join(',\n');
109 |
110 | // Write to a file
111 | const outputContent = `// Generated from GitHub repository: data-gov-my/datagovmy-meta
112 | // Timestamp: ${new Date().toISOString()}
113 | // Total datasets: ${datasets.length}
114 |
115 | const EXTRACTED_DATASETS = [
116 | ${formattedDatasets}
117 | ];
118 |
119 | export default EXTRACTED_DATASETS;
120 | `;
121 |
122 | const outputPath = path.join(__dirname, 'extracted-datasets.js');
123 | fs.writeFileSync(outputPath, outputContent);
124 | console.log(`Successfully extracted ${datasets.length} dataset IDs to extracted-datasets.js`);
125 |
126 | return datasets;
127 | } catch (error) {
128 | console.error('Error extracting dataset IDs:', error);
129 | throw error;
130 | }
131 | }
132 |
133 | // Run the extraction
134 | extractAllDatasetIds().catch(console.error);
135 |
```
--------------------------------------------------------------------------------
/src/utils/query-builder.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Query Parameter Builder for Malaysia Open Data API
3 | *
4 | * Utility functions to build query parameters for API requests
5 | * based on the Malaysia Open Data API query syntax.
6 | */
7 |
8 | /**
9 | * Builds query parameters for API requests
10 | *
11 | * @param {Object} params - Parameters to build query from
12 | * @returns {Object} - Formatted query parameters
13 | */
14 | function buildQueryParams(params = {}) {
15 | const queryParams = {};
16 |
17 | // Handle dataset ID
18 | if (params.id) {
19 | queryParams.id = params.id;
20 | }
21 |
22 | // Handle filter parameters
23 | if (params.filter) {
24 | queryParams.filter = formatFilterParam(params.filter);
25 | }
26 |
27 | if (params.ifilter) {
28 | queryParams.ifilter = formatFilterParam(params.ifilter);
29 | }
30 |
31 | if (params.contains) {
32 | queryParams.contains = formatFilterParam(params.contains);
33 | }
34 |
35 | if (params.icontains) {
36 | queryParams.icontains = formatFilterParam(params.icontains);
37 | }
38 |
39 | // Handle range parameter
40 | if (params.range) {
41 | if (typeof params.range === 'object') {
42 | const { column, begin, end } = params.range;
43 | queryParams.range = `${column}[${begin ?? ''}:${end ?? ''}]`;
44 | } else {
45 | queryParams.range = params.range;
46 | }
47 | }
48 |
49 | // Handle sort parameter
50 | if (params.sort) {
51 | if (Array.isArray(params.sort)) {
52 | queryParams.sort = params.sort.join(',');
53 | } else {
54 | queryParams.sort = params.sort;
55 | }
56 | }
57 |
58 | // Handle date parameters
59 | if (params.date_start) {
60 | queryParams.date_start = formatDateParam(params.date_start);
61 | }
62 |
63 | if (params.date_end) {
64 | queryParams.date_end = formatDateParam(params.date_end);
65 | }
66 |
67 | // Handle timestamp parameters
68 | if (params.timestamp_start) {
69 | queryParams.timestamp_start = formatTimestampParam(params.timestamp_start);
70 | }
71 |
72 | if (params.timestamp_end) {
73 | queryParams.timestamp_end = formatTimestampParam(params.timestamp_end);
74 | }
75 |
76 | // Handle limit parameter
77 | if (params.limit !== undefined) {
78 | queryParams.limit = params.limit;
79 | }
80 |
81 | // Handle include/exclude parameters
82 | if (params.include) {
83 | if (Array.isArray(params.include)) {
84 | queryParams.include = params.include.join(',');
85 | } else {
86 | queryParams.include = params.include;
87 | }
88 | }
89 |
90 | if (params.exclude) {
91 | if (Array.isArray(params.exclude)) {
92 | queryParams.exclude = params.exclude.join(',');
93 | } else {
94 | queryParams.exclude = params.exclude;
95 | }
96 | }
97 |
98 | // Handle meta parameter
99 | if (params.meta !== undefined) {
100 | queryParams.meta = params.meta.toString();
101 | }
102 |
103 | return queryParams;
104 | }
105 |
106 | /**
107 | * Formats filter parameters (filter, ifilter, contains, icontains)
108 | *
109 | * @param {Object|string} filter - Filter configuration
110 | * @returns {string} - Formatted filter parameter
111 | */
112 | function formatFilterParam(filter) {
113 | if (typeof filter === 'string') {
114 | return filter;
115 | }
116 |
117 | if (typeof filter === 'object') {
118 | return Object.entries(filter)
119 | .map(([column, value]) => `${value}@${column}`)
120 | .join(',');
121 | }
122 |
123 | return filter;
124 | }
125 |
126 | /**
127 | * Formats date parameters (date_start, date_end)
128 | *
129 | * @param {Object|string} dateParam - Date parameter configuration
130 | * @returns {string} - Formatted date parameter
131 | */
132 | function formatDateParam(dateParam) {
133 | if (typeof dateParam === 'string') {
134 | return dateParam;
135 | }
136 |
137 | if (typeof dateParam === 'object') {
138 | const { date, column } = dateParam;
139 | return `${date}@${column}`;
140 | }
141 |
142 | return dateParam;
143 | }
144 |
145 | /**
146 | * Formats timestamp parameters (timestamp_start, timestamp_end)
147 | *
148 | * @param {Object|string} timestampParam - Timestamp parameter configuration
149 | * @returns {string} - Formatted timestamp parameter
150 | */
151 | function formatTimestampParam(timestampParam) {
152 | if (typeof timestampParam === 'string') {
153 | return timestampParam;
154 | }
155 |
156 | if (typeof timestampParam === 'object') {
157 | const { timestamp, column } = timestampParam;
158 | return `${timestamp}@${column}`;
159 | }
160 |
161 | return timestampParam;
162 | }
163 |
164 | module.exports = {
165 | buildQueryParams
166 | };
167 |
```
--------------------------------------------------------------------------------
/src/flood.tools.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
2 | import { z } from 'zod';
3 | import axios from 'axios';
4 | import { prefixToolName } from './utils/tool-naming.js';
5 |
6 | // Base URL for Malaysia Open Data API
7 | const API_BASE_URL = 'https://api.data.gov.my';
8 | const FLOOD_WARNING_ENDPOINT = '/flood-warning';
9 |
10 | /**
11 | * Register flood warning tools with the MCP server
12 | * @param server MCP server instance
13 | */
14 | export function registerFloodTools(server: McpServer) {
15 | server.tool(
16 | prefixToolName('get_flood_warnings'),
17 | 'Gets current flood warnings for Malaysia',
18 | {
19 | state: z.string().optional().describe('State name to filter warnings (e.g., "Selangor", "Johor")'),
20 | district: z.string().optional().describe('District name to filter warnings'),
21 | severity: z.string().optional().describe('Severity level to filter (e.g., "warning", "alert", "danger")'),
22 | },
23 | async ({ state, district, severity }) => {
24 | try {
25 | // Make a real API call to the Malaysia Open Data API
26 | const url = `${API_BASE_URL}${FLOOD_WARNING_ENDPOINT}`;
27 | const params: Record<string, any> = { meta: true };
28 |
29 | // Only add parameters if they are provided
30 | if (state) params.filter = `${state}@state`;
31 | if (district) params.filter = `${district}@district`;
32 | if (severity) params.filter = `${severity}@severity`;
33 |
34 | const response = await axios.get(url, { params });
35 |
36 | // Return the actual API response
37 | return {
38 | content: [
39 | {
40 | type: 'text',
41 | text: JSON.stringify({
42 | message: 'Flood warnings retrieved successfully',
43 | params: { state, district, severity },
44 | endpoint: `${API_BASE_URL}${FLOOD_WARNING_ENDPOINT}`,
45 | warnings: response.data,
46 | timestamp: new Date().toISOString()
47 | }, null, 2),
48 | },
49 | ],
50 | };
51 | } catch (error) {
52 | console.error('Error fetching flood warnings:', error);
53 |
54 | // If the API is unavailable, fall back to mock data for demonstration
55 | if (axios.isAxiosError(error) && (error.code === 'ECONNREFUSED' || error.response?.status === 404)) {
56 | console.warn('API unavailable, using mock data');
57 | return {
58 | content: [
59 | {
60 | type: 'text',
61 | text: JSON.stringify({
62 | message: 'API unavailable, using mock data',
63 | params: { state, district, severity },
64 | endpoint: `${API_BASE_URL}${FLOOD_WARNING_ENDPOINT}`,
65 | warnings: [
66 | {
67 | id: 'mock-flood-1',
68 | state: 'Selangor',
69 | district: 'Klang',
70 | location: 'Taman Sri Muda',
71 | severity: 'warning',
72 | water_level: '3.5m',
73 | timestamp: new Date().toISOString()
74 | },
75 | {
76 | id: 'mock-flood-2',
77 | state: 'Johor',
78 | district: 'Kluang',
79 | location: 'Kampung Contoh',
80 | severity: 'danger',
81 | water_level: '4.2m',
82 | timestamp: new Date().toISOString()
83 | }
84 | ],
85 | note: 'This is mock data as the real API is currently unavailable'
86 | }, null, 2),
87 | },
88 | ],
89 | };
90 | }
91 |
92 | // Return error information
93 | return {
94 | content: [
95 | {
96 | type: 'text',
97 | text: JSON.stringify({
98 | error: 'Failed to fetch flood warnings',
99 | message: error instanceof Error ? error.message : 'Unknown error',
100 | status: axios.isAxiosError(error) ? error.response?.status : undefined,
101 | timestamp: new Date().toISOString()
102 | }, null, 2),
103 | },
104 | ],
105 | };
106 | }
107 | }
108 | );
109 | }
110 |
```
--------------------------------------------------------------------------------
/src/tools/weather.js:
--------------------------------------------------------------------------------
```javascript
1 | /**
2 | * Weather MCP Tools
3 | *
4 | * Tools for accessing weather forecasts and warnings from the Malaysia Open Data API
5 | */
6 |
7 | const { createWeatherClient } = require('../api/weather');
8 |
9 | // Create client instance with default configuration
10 | const weatherClient = createWeatherClient();
11 |
12 | /**
13 | * Gets 7-day general weather forecast data
14 | *
15 | * @param {Object} params - Parameters
16 | * @param {string} params.location - Optional location filter
17 | * @param {string} params.locationCategory - Optional location category filter (St, Rc, Ds, Tn, Dv)
18 | * @param {string} params.date - Optional date filter (YYYY-MM-DD)
19 | * @param {number} params.limit - Optional limit parameter
20 | * @returns {Promise<Object>} - Forecast data
21 | */
22 | async function getForecast(params = {}) {
23 | try {
24 | const queryParams = {};
25 |
26 | // Handle location filter
27 | if (params.location) {
28 | queryParams.contains = `${params.location}@location__location_name`;
29 | }
30 |
31 | // Handle location category filter
32 | if (params.locationCategory) {
33 | queryParams.contains = `${params.locationCategory}@location__location_id`;
34 | }
35 |
36 | // Handle date filter
37 | if (params.date) {
38 | queryParams.filter = `${params.date}@date`;
39 | }
40 |
41 | // Handle limit
42 | if (params.limit) {
43 | queryParams.limit = params.limit;
44 | }
45 |
46 | const result = await weatherClient.getForecast(queryParams);
47 |
48 | return {
49 | success: true,
50 | message: 'Successfully retrieved weather forecast data',
51 | data: result
52 | };
53 | } catch (error) {
54 | return {
55 | success: false,
56 | message: `Failed to get weather forecast: ${error.message}`,
57 | error: error.message
58 | };
59 | }
60 | }
61 |
62 | /**
63 | * Gets weather warning data
64 | *
65 | * @param {Object} params - Parameters
66 | * @param {string} params.district - Optional district filter
67 | * @param {string} params.state - Optional state filter
68 | * @param {string} params.warningType - Optional warning type filter
69 | * @param {number} params.limit - Optional limit parameter
70 | * @returns {Promise<Object>} - Warning data
71 | */
72 | async function getWarnings(params = {}) {
73 | try {
74 | const queryParams = {};
75 |
76 | // Handle district filter
77 | if (params.district) {
78 | queryParams.filter = `${params.district}@district`;
79 | }
80 |
81 | // Handle state filter
82 | if (params.state) {
83 | queryParams.filter = `${params.state}@state`;
84 | }
85 |
86 | // Handle warning type filter
87 | if (params.warningType) {
88 | queryParams.filter = `${params.warningType}@warning_type`;
89 | }
90 |
91 | // Handle limit
92 | if (params.limit) {
93 | queryParams.limit = params.limit;
94 | }
95 |
96 | const result = await weatherClient.getWarnings(queryParams);
97 |
98 | return {
99 | success: true,
100 | message: 'Successfully retrieved weather warning data',
101 | data: result
102 | };
103 | } catch (error) {
104 | return {
105 | success: false,
106 | message: `Failed to get weather warnings: ${error.message}`,
107 | error: error.message
108 | };
109 | }
110 | }
111 |
112 | /**
113 | * Gets earthquake warning data
114 | *
115 | * @param {Object} params - Parameters
116 | * @param {number} params.magnitude - Optional minimum magnitude filter
117 | * @param {string} params.region - Optional region filter
118 | * @param {number} params.limit - Optional limit parameter
119 | * @returns {Promise<Object>} - Earthquake warning data
120 | */
121 | async function getEarthquakeWarnings(params = {}) {
122 | try {
123 | const queryParams = {};
124 |
125 | // Handle magnitude filter
126 | if (params.magnitude) {
127 | queryParams.range = `magnitude[${params.magnitude}:]`;
128 | }
129 |
130 | // Handle region filter
131 | if (params.region) {
132 | queryParams.contains = `${params.region}@region`;
133 | }
134 |
135 | // Handle limit
136 | if (params.limit) {
137 | queryParams.limit = params.limit;
138 | }
139 |
140 | const result = await weatherClient.getEarthquakeWarnings(queryParams);
141 |
142 | return {
143 | success: true,
144 | message: 'Successfully retrieved earthquake warning data',
145 | data: result
146 | };
147 | } catch (error) {
148 | return {
149 | success: false,
150 | message: `Failed to get earthquake warnings: ${error.message}`,
151 | error: error.message
152 | };
153 | }
154 | }
155 |
156 | module.exports = {
157 | getForecast,
158 | getWarnings,
159 | getEarthquakeWarnings
160 | };
161 |
```
--------------------------------------------------------------------------------
/src/weather.tools.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
2 | import { z } from 'zod';
3 | import axios from 'axios';
4 | import { prefixToolName } from './utils/tool-naming.js';
5 |
6 | // API Base URL for Malaysia Open Data API
7 | const API_BASE_URL = 'https://api.data.gov.my';
8 | // Weather API endpoints - using realtime API endpoints
9 | const WEATHER_FORECAST_ENDPOINT = '/weather/forecast';
10 | const WEATHER_WARNING_ENDPOINT = '/weather/warning';
11 | const EARTHQUAKE_WARNING_ENDPOINT = '/weather/warning/earthquake';
12 |
13 | export function registerWeatherTools(server: McpServer) {
14 | // Get weather forecast
15 | server.tool(
16 | prefixToolName('get_weather_forecast'),
17 | 'Gets weather forecast for Malaysia',
18 | {
19 | location: z.string().describe('Location name (e.g., "Kuala Lumpur", "Penang")'),
20 | days: z.number().min(1).max(7).optional().describe('Number of days to forecast (1-7)'),
21 | },
22 | async ({ location, days = 3 }) => {
23 | try {
24 | const url = `${API_BASE_URL}${WEATHER_FORECAST_ENDPOINT}`;
25 | const params: Record<string, any> = { limit: 100 };
26 |
27 | if (location) {
28 | params.contains = `${location}@location__location_name`;
29 | }
30 |
31 | if (days) {
32 | params.limit = days;
33 | }
34 |
35 | const response = await axios.get(url, { params });
36 | const data = response.data;
37 |
38 | return {
39 | content: [
40 | {
41 | type: 'text',
42 | text: JSON.stringify(data, null, 2),
43 | },
44 | ],
45 | };
46 | } catch (error) {
47 | return {
48 | content: [
49 | {
50 | type: 'text',
51 | text: `Error fetching weather forecast: ${error instanceof Error ? error.message : 'Unknown error'}`,
52 | },
53 | ],
54 | };
55 | }
56 | }
57 | );
58 |
59 | // Get weather warnings
60 | server.tool(
61 | prefixToolName('get_weather_warnings'),
62 | 'Gets current weather warnings for Malaysia',
63 | {
64 | type: z.string().optional().describe('Type of warning (e.g., "rain", "flood", "all")'),
65 | location: z.string().optional().describe('Location name to filter warnings'),
66 | },
67 | async ({ type = 'all', location }) => {
68 | try {
69 | const url = `${API_BASE_URL}${WEATHER_WARNING_ENDPOINT}`;
70 | const params: Record<string, any> = { limit: 100 };
71 |
72 | if (type && type !== 'all') {
73 | params.contains = `${type}@warning_issue__title_en`;
74 | }
75 |
76 | if (location) {
77 | params.contains = `${location}@text_en`;
78 | }
79 |
80 | const response = await axios.get(url, { params });
81 | const data = response.data;
82 |
83 | return {
84 | content: [
85 | {
86 | type: 'text',
87 | text: JSON.stringify(data, null, 2),
88 | },
89 | ],
90 | };
91 | } catch (error) {
92 | return {
93 | content: [
94 | {
95 | type: 'text',
96 | text: `Error fetching weather warnings: ${error instanceof Error ? error.message : 'Unknown error'}`,
97 | },
98 | ],
99 | };
100 | }
101 | }
102 | );
103 |
104 | // Get earthquake warnings
105 | server.tool(
106 | prefixToolName('get_earthquake_warnings'),
107 | 'Gets earthquake warnings for Malaysia',
108 | {
109 | days: z.number().min(1).max(30).optional().describe('Number of days to look back (1-30)'),
110 | magnitude: z.number().min(0).optional().describe('Minimum magnitude to include'),
111 | },
112 | async ({ days = 7, magnitude = 4.0 }) => {
113 | try {
114 | const url = `${API_BASE_URL}${EARTHQUAKE_WARNING_ENDPOINT}`;
115 | const params: Record<string, any> = { limit: 100, meta: true };
116 |
117 | if (days) {
118 | // Convert days to timestamp for filtering
119 | const pastDate = new Date();
120 | pastDate.setDate(pastDate.getDate() - days);
121 | params.timestamp_start = pastDate.toISOString().split('T')[0] + ' 00:00:00@utcdatetime';
122 | }
123 |
124 | if (magnitude) {
125 | params.number_min = `${magnitude}@magdefault`;
126 | }
127 |
128 | const response = await axios.get(url, { params });
129 | const data = response.data;
130 |
131 | return {
132 | content: [
133 | {
134 | type: 'text',
135 | text: JSON.stringify(data, null, 2),
136 | },
137 | ],
138 | };
139 | } catch (error) {
140 | return {
141 | content: [
142 | {
143 | type: 'text',
144 | text: `Error fetching earthquake warnings: ${error instanceof Error ? error.message : 'Unknown error'}`,
145 | },
146 | ],
147 | };
148 | }
149 | }
150 | );
151 | }
152 |
```
--------------------------------------------------------------------------------
/scripts/index-catalogue-files.cjs:
--------------------------------------------------------------------------------
```
1 | const fs = require('fs');
2 | const path = require('path');
3 |
4 | // Data catalogue directory
5 | const cataloguePath = path.join(__dirname, '..', 'data-catalogue');
6 |
7 | // Output file paths
8 | const outputJSPath = path.join(__dirname, 'catalogue-index.js');
9 | const outputDTSPath = path.join(__dirname, 'catalogue-index.d.ts');
10 |
11 | /**
12 | * Indexes all JSON files in the data catalogue directory
13 | * and generates a single JS module with the data and a corresponding d.ts file.
14 | */
15 | function indexCatalogueFiles() {
16 | try {
17 | const files = fs.readdirSync(cataloguePath).filter(file => file.endsWith('.json'));
18 | console.log(`Found ${files.length} JSON files in data-catalogue.`);
19 |
20 | const catalogueIndex = [];
21 | const categories = new Set();
22 | const geographies = new Set();
23 | const frequencies = new Set();
24 | const demographies = new Set();
25 | const dataSources = new Set();
26 |
27 | for (const file of files) {
28 | try {
29 | const metadata = JSON.parse(fs.readFileSync(path.join(cataloguePath, file), 'utf8'));
30 |
31 | const dataset = {
32 | id: metadata.id || file.replace('.json', ''),
33 | title_en: metadata.title_en || '',
34 | title_ms: metadata.title_ms || '',
35 | description_en: metadata.description_en || '',
36 | description_ms: metadata.description_ms || '',
37 | frequency: metadata.frequency || '',
38 | geography: metadata.geography || [],
39 | demography: metadata.demography || [],
40 | dataset_begin: metadata.dataset_begin || null,
41 | dataset_end: metadata.dataset_end || null,
42 | data_source: metadata.data_source || [],
43 | data_as_of: metadata.data_as_of || '',
44 | last_updated: metadata.last_updated || '',
45 | next_update: metadata.next_update || '',
46 | link_parquet: metadata.link_parquet || '',
47 | link_csv: metadata.link_csv || '',
48 | link_preview: metadata.link_preview || '',
49 | site_category: metadata.site_category || []
50 | };
51 |
52 | catalogueIndex.push(dataset);
53 |
54 | // Collect filter values
55 | if (dataset.frequency) frequencies.add(dataset.frequency);
56 | dataset.geography.forEach(g => geographies.add(g));
57 | dataset.demography.forEach(d => demographies.add(d));
58 | dataset.data_source.forEach(s => dataSources.add(s));
59 | dataset.site_category.forEach(sc => {
60 | if (sc.category_en) categories.add(sc.category_en);
61 | if (sc.subcategory_en) categories.add(sc.subcategory_en);
62 | });
63 |
64 | } catch (e) {
65 | console.error(`Skipping invalid JSON file: ${file}`, e);
66 | }
67 | }
68 |
69 | catalogueIndex.sort((a, b) => a.id.localeCompare(b.id));
70 |
71 | const filters = {
72 | categories: Array.from(categories).sort(),
73 | geographies: Array.from(geographies).sort(),
74 | frequencies: Array.from(frequencies).sort(),
75 | demographies: Array.from(demographies).sort(),
76 | dataSources: Array.from(dataSources).sort()
77 | };
78 |
79 | const jsContent = `// Generated from local data catalogue files\n// Timestamp: ${new Date().toISOString()}\n// Total datasets: ${catalogueIndex.length}\n\nexport const CATALOGUE_INDEX = ${JSON.stringify(catalogueIndex, null, 2)};\n\nexport const CATALOGUE_FILTERS = ${JSON.stringify(filters, null, 2)};`;
80 |
81 | const dtsContent = `// Generated by scripts/index-catalogue-files.js on ${new Date().toISOString()}\n\ndeclare module '../scripts/catalogue-index.js' {\n export interface SiteCategory {\n site: string;\n category_en: string;\n category_ms: string;\n category_sort: number;\n subcategory_en: string;\n subcategory_ms: string;\n subcategory_sort: number;\n }\n\n export interface DatasetMetadata {\n id: string;\n title_en: string;\n title_ms: string;\n description_en: string;\n description_ms: string;\n frequency: string;\n geography: string[];\n demography: string[];\n dataset_begin: number | null;\n dataset_end: number | null;\n data_source: string[];\n data_as_of: string;\n last_updated: string;\n next_update: string;\n link_parquet: string;\n link_csv: string;\n link_preview: string;\n site_category: SiteCategory[];\n }\n\n export interface CatalogueFilters {\n categories: string[];\n geographies: string[];\n frequencies: string[];\n demographies: string[];\n dataSources: string[];\n }\n\n export const CATALOGUE_INDEX: DatasetMetadata[];\n export const CATALOGUE_FILTERS: CatalogueFilters;\n}`;
82 |
83 | fs.writeFileSync(outputJSPath, jsContent);
84 | fs.writeFileSync(outputDTSPath, dtsContent);
85 | console.log(`Successfully indexed ${catalogueIndex.length} datasets to catalogue-index.js and catalogue-index.d.ts`);
86 |
87 | } catch (error) {
88 | console.error('Error indexing catalogue files:', error);
89 | throw error;
90 | }
91 | }
92 |
93 | // Run the indexing
94 | indexCatalogueFiles();
95 |
```
--------------------------------------------------------------------------------
/src/firebase-analytics.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { initializeApp, cert, ServiceAccount } from 'firebase-admin/app';
2 | import { getDatabase } from 'firebase-admin/database';
3 | import fs from 'fs';
4 | import path from 'path';
5 |
6 | const SERVER_NAME = 'mcp-datagovmy';
7 |
8 | interface ToolCall {
9 | tool: string;
10 | timestamp: string;
11 | clientIp: string;
12 | userAgent: string;
13 | }
14 |
15 | interface Analytics {
16 | serverStartTime: string;
17 | totalRequests: number;
18 | totalToolCalls: number;
19 | requestsByMethod: Record<string, number>;
20 | requestsByEndpoint: Record<string, number>;
21 | toolCalls: Record<string, number>;
22 | recentToolCalls: ToolCall[];
23 | clientsByIp: Record<string, number>;
24 | clientsByUserAgent: Record<string, number>;
25 | hourlyRequests: Record<string, number>;
26 | }
27 |
28 | let firebaseInitialized = false;
29 | let database: ReturnType<typeof getDatabase> | null = null;
30 |
31 | function initializeFirebase() {
32 | if (firebaseInitialized) return;
33 |
34 | try {
35 | const credentialsPath = path.join(process.cwd(), '.credentials', 'firebase-service-account.json');
36 |
37 | if (!fs.existsSync(credentialsPath)) {
38 | console.warn(`⚠️ Firebase credentials not found at ${credentialsPath}`);
39 | console.warn(' Analytics will only be saved locally');
40 | return;
41 | }
42 |
43 | const serviceAccount = JSON.parse(fs.readFileSync(credentialsPath, 'utf-8')) as ServiceAccount;
44 |
45 | initializeApp({
46 | credential: cert(serviceAccount),
47 | databaseURL: 'https://mcp-analytics-49b45-default-rtdb.asia-southeast1.firebasedatabase.app'
48 | });
49 |
50 | database = getDatabase();
51 | firebaseInitialized = true;
52 | console.log('✅ Firebase initialized successfully');
53 | } catch (error) {
54 | console.error('❌ Failed to initialize Firebase:', error);
55 | }
56 | }
57 |
58 | /**
59 | * Sanitize keys for Firebase - replace invalid characters with safe alternatives
60 | * Firebase keys cannot contain: . # $ / [ ]
61 | */
62 | function sanitizeKey(key: string): string {
63 | return key
64 | .replace(/\./g, '_dot_')
65 | .replace(/#/g, '_hash_')
66 | .replace(/\$/g, '_dollar_')
67 | .replace(/\//g, '_slash_')
68 | .replace(/\[/g, '_lbracket_')
69 | .replace(/\]/g, '_rbracket_');
70 | }
71 |
72 | /**
73 | * Sanitize an object's keys recursively for Firebase compatibility
74 | */
75 | function sanitizeObject(obj: any): any {
76 | if (obj === null || obj === undefined) {
77 | return obj;
78 | }
79 |
80 | if (Array.isArray(obj)) {
81 | return obj.map(item => sanitizeObject(item));
82 | }
83 |
84 | if (typeof obj === 'object') {
85 | const sanitized: any = {};
86 | for (const [key, value] of Object.entries(obj)) {
87 | const sanitizedKey = sanitizeKey(key);
88 | sanitized[sanitizedKey] = sanitizeObject(value);
89 | }
90 | return sanitized;
91 | }
92 |
93 | return obj;
94 | }
95 |
96 | /**
97 | * Desanitize keys when loading from Firebase
98 | */
99 | function desanitizeKey(key: string): string {
100 | return key
101 | .replace(/_dot_/g, '.')
102 | .replace(/_hash_/g, '#')
103 | .replace(/_dollar_/g, '$')
104 | .replace(/_slash_/g, '/')
105 | .replace(/_lbracket_/g, '[')
106 | .replace(/_rbracket_/g, ']');
107 | }
108 |
109 | /**
110 | * Desanitize an object's keys recursively
111 | */
112 | function desanitizeObject(obj: any): any {
113 | if (obj === null || obj === undefined) {
114 | return obj;
115 | }
116 |
117 | if (Array.isArray(obj)) {
118 | return obj.map(item => desanitizeObject(item));
119 | }
120 |
121 | if (typeof obj === 'object') {
122 | const desanitized: any = {};
123 | for (const [key, value] of Object.entries(obj)) {
124 | const desanitizedKey = desanitizeKey(key);
125 | desanitized[desanitizedKey] = desanitizeObject(value);
126 | }
127 | return desanitized;
128 | }
129 |
130 | return obj;
131 | }
132 |
133 | export async function saveAnalyticsToFirebase(analytics: Analytics): Promise<void> {
134 | if (!firebaseInitialized) {
135 | initializeFirebase();
136 | }
137 |
138 | if (!database) {
139 | console.log('📝 Firebase not available, skipping cloud save');
140 | return;
141 | }
142 |
143 | try {
144 | const ref = database.ref(`mcp-analytics/${SERVER_NAME}`);
145 |
146 | // Sanitize the analytics object before saving
147 | const sanitizedAnalytics = sanitizeObject(analytics);
148 |
149 | await ref.set(sanitizedAnalytics);
150 | console.log(`📊 Analytics saved to Firebase: ${SERVER_NAME}`);
151 | } catch (error) {
152 | console.error('Failed to save to Firebase:', error);
153 | }
154 | }
155 |
156 | export async function loadAnalyticsFromFirebase(): Promise<Analytics | null> {
157 | if (!firebaseInitialized) {
158 | initializeFirebase();
159 | }
160 |
161 | if (!database) {
162 | console.log('Firebase not available for loading');
163 | return null;
164 | }
165 |
166 | try {
167 | const ref = database.ref(`mcp-analytics/${SERVER_NAME}`);
168 | const snapshot = await ref.get();
169 |
170 | if (snapshot.exists()) {
171 | const sanitizedData = snapshot.val();
172 |
173 | // Desanitize the data when loading
174 | const data = desanitizeObject(sanitizedData) as Analytics;
175 |
176 | console.log(`📊 Loaded analytics from Firebase: ${SERVER_NAME}`);
177 | console.log(` Total requests: ${data.totalRequests.toLocaleString()}, Tool calls: ${data.totalToolCalls}`);
178 | return data;
179 | }
180 |
181 | console.log('No existing analytics in Firebase');
182 | return null;
183 | } catch (error) {
184 | console.error('Failed to load from Firebase:', error);
185 | return null;
186 | }
187 | }
```
--------------------------------------------------------------------------------
/src/index.ts:
--------------------------------------------------------------------------------
```typescript
1 | /**
2 | * Malaysia Open Data MCP Server
3 | * Provides tools to access Malaysia's open data APIs
4 | *
5 | * Load environment variables from .env file
6 | */
7 | import dotenv from 'dotenv';
8 |
9 | // Initialize dotenv to load environment variables from .env file
10 | dotenv.config();
11 |
12 | /**
13 | * =====================================================================
14 | * IMPORTANT GUIDANCE FOR AI MODELS USING THIS MCP SERVER:
15 | * =====================================================================
16 | * 1. ALWAYS use the 'search_all' tool FIRST for any data queries
17 | * This tool searches both datasets and dashboards simultaneously
18 | * with intelligent fallback to ensure comprehensive results.
19 | *
20 | * 2. Only use specific dataset or dashboard search tools if you need to
21 | * explicitly limit your search to one type of content.
22 | *
23 | * 3. Use correct URL patterns:
24 | * - https://data.gov.my/... for general data portal resources
25 | * - https://open.dosm.gov.my/... for OpenDOSM resources
26 | *
27 | * 4. Parquet files can now be accessed and parsed using the 'parse_parquet_file'
28 | * and 'get_parquet_info' tools. These tools use the hyparquet JavaScript library
29 | * and are fully compatible with the Smithery deployment environment.
30 | * =====================================================================
31 | */
32 |
33 | import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
34 | import { z } from 'zod';
35 |
36 | // Import external tools
37 | import { registerFloodTools } from './flood.tools.js';
38 | import { registerWeatherTools } from './weather.tools.js';
39 | import { registerTransportTools } from './transport.tools.js';
40 | import { registerDataCatalogueTools } from './datacatalogue.tools.js';
41 | import { registerDosmTools } from './dosm.tools.js';
42 | import { registerDashboardTools } from './dashboards.tools.js';
43 | import { registerUnifiedSearchTools } from './unified-search.tools.js';
44 | import { registerParquetTools } from './parquet.tools.js';
45 | import { registerGtfsTools } from './gtfs.tools.js';
46 | import { prefixToolName } from './utils/tool-naming.js';
47 |
48 | // Type definition for tool registration functions
49 | type ToolRegistrationFn = (server: McpServer) => void;
50 |
51 | // Define the config schema
52 | export const configSchema = z.object({
53 | // Optional Google Maps API key for geocoding
54 | googleMapsApiKey: z.string()
55 | .optional()
56 | .describe('Google Maps API key for improved location detection. If not provided, will use OpenStreetMap Nominatim API as fallback.'),
57 |
58 | // Optional GrabMaps API key for Southeast Asia geocoding
59 | grabMapsApiKey: z.string()
60 | .optional()
61 | .describe('GrabMaps API key for improved geocoding in Southeast Asia.'),
62 |
63 | // Optional AWS credentials for GrabMaps integration via AWS Location Service
64 | awsRegion: z.string()
65 | .optional()
66 | .describe('AWS Region where your Place Index is created. Default: ap-southeast-5 (Malaysia)'),
67 |
68 | awsAccessKeyId: z.string()
69 | .optional()
70 | .describe('AWS Access Key ID with permissions to access AWS Location Service.'),
71 |
72 | awsSecretAccessKey: z.string()
73 | .optional()
74 | .describe('AWS Secret Access Key with permissions to access AWS Location Service.'),
75 | });
76 |
77 | /**
78 | * Creates a stateless MCP server for Malaysia Open Data API
79 | */
80 | export default function createStatelessServer({
81 | config: _config,
82 | }: {
83 | config: z.infer<typeof configSchema>;
84 | }) {
85 | const server = new McpServer({
86 | name: 'Malaysia Open Data MCP Server',
87 | version: '1.0.0',
88 | });
89 |
90 | // Extract config values
91 | const { googleMapsApiKey, grabMapsApiKey, awsAccessKeyId, awsSecretAccessKey, awsRegion } = _config;
92 |
93 | // Set API keys in process.env if provided in config
94 | if (googleMapsApiKey) {
95 | process.env.GOOGLE_MAPS_API_KEY = googleMapsApiKey;
96 | console.log('Using Google Maps API key from configuration');
97 | }
98 |
99 | // Set GrabMaps API key
100 | if (grabMapsApiKey) {
101 | process.env.GRABMAPS_API_KEY = grabMapsApiKey;
102 | console.log('Using GrabMaps API key from configuration');
103 | }
104 |
105 | // Set AWS credentials for GrabMaps integration via AWS Location Service
106 | if (awsAccessKeyId) {
107 | process.env.AWS_ACCESS_KEY_ID = awsAccessKeyId;
108 | console.log('Using AWS Access Key ID from configuration');
109 | }
110 |
111 | if (awsSecretAccessKey) {
112 | process.env.AWS_SECRET_ACCESS_KEY = awsSecretAccessKey;
113 | console.log('Using AWS Secret Access Key from configuration');
114 | }
115 |
116 | if (awsRegion) {
117 | process.env.AWS_REGION = awsRegion;
118 | console.log(`Using AWS Region: ${awsRegion} from configuration`);
119 | }
120 |
121 | // Register all tool sets
122 | const toolSets: ToolRegistrationFn[] = [
123 | registerDataCatalogueTools,
124 | registerDosmTools,
125 | registerWeatherTools,
126 | registerDashboardTools,
127 | registerUnifiedSearchTools,
128 | registerParquetTools,
129 | registerGtfsTools,
130 | registerTransportTools,
131 | registerFloodTools,
132 | ];
133 |
134 | // Register all tools
135 | toolSets.forEach((toolSet) => toolSet(server));
136 |
137 | // Register a simple hello tool for testing
138 | server.tool(
139 | prefixToolName('hello'),
140 | 'A simple test tool to verify that the MCP server is working correctly',
141 | {},
142 | async () => {
143 | return {
144 | content: [
145 | {
146 | type: 'text',
147 | text: JSON.stringify({
148 | message: 'Hello from Malaysia Open Data MCP!',
149 | timestamp: new Date().toISOString(),
150 | }, null, 2),
151 | },
152 | ],
153 | };
154 | }
155 | );
156 |
157 | return server.server;
158 | }
159 |
160 | // If this file is run directly, log a message
161 | console.log('Malaysia Open Data MCP module loaded');
162 |
163 |
```
--------------------------------------------------------------------------------
/deploy/DEPLOYMENT.md:
--------------------------------------------------------------------------------
```markdown
1 | # VPS Deployment Guide for Malaysia Open Data MCP
2 |
3 | This guide explains how to deploy the Malaysia Open Data MCP server on your VPS at `mcp.techmavie.digital/datagovmy`.
4 |
5 | ## Prerequisites
6 |
7 | - VPS with Ubuntu/Debian
8 | - Docker and Docker Compose installed
9 | - Nginx installed
10 | - Domain `mcp.techmavie.digital` pointing to your VPS IP
11 | - SSL certificate (via Certbot/Let's Encrypt)
12 |
13 | ## Architecture
14 |
15 | ```
16 | Client (Claude, Cursor, etc.)
17 | ↓ HTTPS
18 | https://mcp.techmavie.digital/datagovmy/mcp
19 | ↓
20 | Nginx (SSL termination + reverse proxy)
21 | ↓ HTTP
22 | Docker Container (port 8082 → 8080)
23 | ↓
24 | Malaysia Open Data APIs (data.gov.my, OpenDOSM, etc.)
25 | ```
26 |
27 | ## Deployment Steps
28 |
29 | ### 1. SSH into your VPS
30 |
31 | ```bash
32 | ssh root@your-vps-ip
33 | ```
34 |
35 | ### 2. Create directory for the MCP server
36 |
37 | ```bash
38 | mkdir -p /opt/mcp-servers/datagovmy
39 | cd /opt/mcp-servers/datagovmy
40 | ```
41 |
42 | ### 3. Clone the repository
43 |
44 | ```bash
45 | git clone https://github.com/hithereiamaliff/mcp-datagovmy.git .
46 | ```
47 |
48 | ### 4. Create environment file (optional)
49 |
50 | ```bash
51 | cp .env.example .env
52 | nano .env
53 | ```
54 |
55 | Add optional API keys if needed:
56 | ```env
57 | GOOGLE_MAPS_API_KEY=your_api_key_here
58 | GRABMAPS_API_KEY=your_api_key_here
59 | AWS_ACCESS_KEY_ID=your_aws_key
60 | AWS_SECRET_ACCESS_KEY=your_aws_secret
61 | AWS_REGION=ap-southeast-5
62 | ```
63 |
64 | ### 5. Build and start the Docker container
65 |
66 | ```bash
67 | docker compose up -d --build
68 | ```
69 |
70 | ### 6. Verify the container is running
71 |
72 | ```bash
73 | docker compose ps
74 | docker compose logs -f
75 | ```
76 |
77 | ### 7. Test the health endpoint
78 |
79 | ```bash
80 | curl http://localhost:8082/health
81 | ```
82 |
83 | ### 8. Configure Nginx
84 |
85 | Add the location block from `deploy/nginx-mcp.conf` to your existing nginx config for `mcp.techmavie.digital`:
86 |
87 | ```bash
88 | # Edit your existing nginx config
89 | sudo nano /etc/nginx/sites-available/mcp.techmavie.digital
90 |
91 | # Add the location block from deploy/nginx-mcp.conf inside the server block
92 | # Make sure it's at the same level as other location blocks (not nested)
93 |
94 | # Test nginx config
95 | sudo nginx -t
96 |
97 | # Reload nginx
98 | sudo systemctl reload nginx
99 | ```
100 |
101 | ### 9. Test the MCP endpoint
102 |
103 | ```bash
104 | # Test health endpoint through nginx
105 | curl https://mcp.techmavie.digital/datagovmy/health
106 |
107 | # Test MCP endpoint
108 | curl -X POST https://mcp.techmavie.digital/datagovmy/mcp \
109 | -H "Content-Type: application/json" \
110 | -H "Accept: application/json" \
111 | -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
112 | ```
113 |
114 | ## Client Configuration
115 |
116 | ### For Claude Desktop / Cursor / Windsurf
117 |
118 | Add to your MCP configuration:
119 |
120 | ```json
121 | {
122 | "mcpServers": {
123 | "malaysia-opendata": {
124 | "transport": "streamable-http",
125 | "url": "https://mcp.techmavie.digital/datagovmy/mcp"
126 | }
127 | }
128 | }
129 | ```
130 |
131 | ### Using Your Own API Keys
132 |
133 | You can provide your own API keys via URL query parameters:
134 |
135 | ```
136 | https://mcp.techmavie.digital/datagovmy/mcp?googleMapsApiKey=YOUR_KEY
137 | ```
138 |
139 | Or via headers:
140 | - `X-Google-Maps-Api-Key: YOUR_KEY`
141 | - `X-GrabMaps-Api-Key: YOUR_KEY`
142 | - `X-AWS-Access-Key-Id: YOUR_KEY`
143 | - `X-AWS-Secret-Access-Key: YOUR_KEY`
144 | - `X-AWS-Region: ap-southeast-5`
145 |
146 | **Supported Query Parameters:**
147 |
148 | | Parameter | Description |
149 | |-----------|-------------|
150 | | `googleMapsApiKey` | Google Maps API key for geocoding |
151 | | `grabMapsApiKey` | GrabMaps API key for Southeast Asia geocoding |
152 | | `awsAccessKeyId` | AWS Access Key ID for AWS Location Service |
153 | | `awsSecretAccessKey` | AWS Secret Access Key |
154 | | `awsRegion` | AWS Region (default: ap-southeast-5) |
155 |
156 | User-provided keys take priority over server defaults.
157 |
158 | > **⚠️ Important: GrabMaps Requirements**
159 | >
160 | > To use GrabMaps geocoding, you need **ALL FOUR** of these parameters:
161 | > - `grabMapsApiKey`
162 | > - `awsAccessKeyId`
163 | > - `awsSecretAccessKey`
164 | > - `awsRegion`
165 | >
166 | > GrabMaps uses AWS Location Service under the hood, so AWS credentials are required alongside the GrabMaps API key. Without any one of these, GrabMaps will not work.
167 |
168 | ### For MCP Inspector
169 |
170 | ```bash
171 | npx @modelcontextprotocol/inspector
172 | # Select "Streamable HTTP"
173 | # Enter URL: https://mcp.techmavie.digital/datagovmy/mcp
174 | ```
175 |
176 | ## Analytics Dashboard
177 |
178 | The MCP server includes a built-in analytics dashboard that tracks:
179 | - **Total requests and tool calls**
180 | - **Tool usage distribution** (doughnut chart)
181 | - **Hourly request trends** (last 24 hours)
182 | - **Requests by endpoint** (bar chart)
183 | - **Top clients by user agent**
184 | - **Recent tool calls feed**
185 |
186 | ### Analytics Endpoints
187 |
188 | | Endpoint | Description |
189 | |----------|-------------|
190 | | `/analytics` | Full analytics summary (JSON) |
191 | | `/analytics/tools` | Detailed tool usage stats (JSON) |
192 | | `/analytics/dashboard` | Visual dashboard with charts (HTML) |
193 |
194 | **Dashboard URL:** `https://mcp.techmavie.digital/datagovmy/analytics/dashboard`
195 |
196 | The dashboard auto-refreshes every 30 seconds.
197 |
198 | ## Management Commands
199 |
200 | ### View logs
201 |
202 | ```bash
203 | cd /opt/mcp-servers/datagovmy
204 | docker compose logs -f
205 | ```
206 |
207 | ### Restart the server
208 |
209 | ```bash
210 | docker compose restart
211 | ```
212 |
213 | ### Update to latest version
214 |
215 | ```bash
216 | git pull origin main
217 | docker compose up -d --build
218 | ```
219 |
220 | ### Stop the server
221 |
222 | ```bash
223 | docker compose down
224 | ```
225 |
226 | ## GitHub Actions Auto-Deploy
227 |
228 | The repository includes a GitHub Actions workflow (`.github/workflows/deploy-vps.yml`) that automatically deploys to your VPS when you push to the `main` branch.
229 |
230 | ### Required GitHub Secrets
231 |
232 | Set these in your repository settings (Settings → Secrets and variables → Actions):
233 |
234 | | Secret | Description |
235 | |--------|-------------|
236 | | `VPS_HOST` | Your VPS IP address |
237 | | `VPS_USERNAME` | SSH username (e.g., root) |
238 | | `VPS_SSH_KEY` | Your private SSH key |
239 | | `VPS_PORT` | SSH port (usually 22) |
240 |
241 | ## Environment Variables
242 |
243 | | Variable | Default | Description |
244 | |----------|---------|-------------|
245 | | `PORT` | 8080 | HTTP server port (internal) |
246 | | `HOST` | 0.0.0.0 | Bind address |
247 | | `GOOGLE_MAPS_API_KEY` | (optional) | For enhanced geocoding |
248 | | `GRABMAPS_API_KEY` | (optional) | For Southeast Asia geocoding |
249 | | `AWS_ACCESS_KEY_ID` | (optional) | For AWS Location Service |
250 | | `AWS_SECRET_ACCESS_KEY` | (optional) | For AWS Location Service |
251 | | `AWS_REGION` | ap-southeast-5 | AWS region for Location Service |
252 |
253 | ## Troubleshooting
254 |
255 | ### Container not starting
256 |
257 | ```bash
258 | docker compose logs mcp-datagovmy
259 | ```
260 |
261 | ### Nginx 502 Bad Gateway
262 |
263 | - Check if container is running: `docker compose ps`
264 | - Check container logs: `docker compose logs`
265 | - Verify port binding: `docker port mcp-datagovmy`
266 |
267 | ### Test MCP connection
268 |
269 | ```bash
270 | # List tools
271 | curl -X POST https://mcp.techmavie.digital/datagovmy/mcp \
272 | -H "Content-Type: application/json" \
273 | -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
274 |
275 | # Call hello tool
276 | curl -X POST https://mcp.techmavie.digital/datagovmy/mcp \
277 | -H "Content-Type: application/json" \
278 | -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"my_hello","arguments":{}}}'
279 | ```
280 |
281 | ## Security Notes
282 |
283 | - The MCP server runs behind nginx with SSL
284 | - CORS is configured to allow all origins (required for MCP clients)
285 | - No authentication is required (public open data)
286 | - Rate limiting can be added at nginx level if needed
287 |
288 | ## Available Tools
289 |
290 | This MCP server provides tools for:
291 |
292 | - **Data Catalogue** - Search and access datasets from data.gov.my
293 | - **OpenDOSM** - Department of Statistics Malaysia data
294 | - **Weather** - Forecasts and warnings from MET Malaysia
295 | - **Transport** - GTFS data for public transit
296 | - **Flood** - Flood warning information
297 | - **Parquet** - Parse parquet data files
298 | - **Unified Search** - Search across all data sources
299 |
```
--------------------------------------------------------------------------------
/src/transport.tools.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
2 | import { z } from 'zod';
3 | import axios from 'axios';
4 | import { prefixToolName } from './utils/tool-naming.js';
5 |
6 | // API Base URL for Malaysia Open Data API
7 | const API_BASE_URL = 'https://api.data.gov.my';
8 |
9 | // GTFS endpoints - correct endpoints for Malaysia Open Data API
10 | const DATA_CATALOGUE_ENDPOINT = '/data-catalogue';
11 | const GTFS_STATIC_ENDPOINT = '/gtfs-static';
12 | const GTFS_REALTIME_ENDPOINT = '/gtfs-realtime';
13 |
14 | export function registerTransportTools(server: McpServer) {
15 | // List transport agencies
16 | server.tool(
17 | prefixToolName('list_transport_agencies'),
18 | 'Lists available transport agencies with GTFS data',
19 | {
20 | limit: z.number().min(1).optional().describe('Maximum number of agencies to return'),
21 | offset: z.number().min(0).optional().describe('Number of agencies to skip'),
22 | },
23 | async ({ limit = 10, offset = 0 }) => {
24 | try {
25 | // Using data catalogue to list GTFS datasets
26 | const url = `${API_BASE_URL}${DATA_CATALOGUE_ENDPOINT}`;
27 | const params: Record<string, any> = {
28 | limit,
29 | meta: true,
30 | contains: 'gtfs' // Search for GTFS datasets
31 | };
32 |
33 | const response = await axios.get(url, { params });
34 | const data = response.data;
35 |
36 | return {
37 | content: [
38 | {
39 | type: 'text',
40 | text: JSON.stringify(data, null, 2),
41 | },
42 | ],
43 | };
44 | } catch (error) {
45 | return {
46 | content: [
47 | {
48 | type: 'text',
49 | text: `Error fetching transport agencies: ${error instanceof Error ? error.message : 'Unknown error'}`,
50 | },
51 | ],
52 | };
53 | }
54 | }
55 | );
56 |
57 | // Get transport data
58 | server.tool(
59 | prefixToolName('get_transport_data'),
60 | 'Gets GTFS data for a specific transport agency',
61 | {
62 | dataset_id: z.string().describe('ID of the GTFS dataset (e.g., "gtfs_rapidkl", "gtfs_prasarana")'),
63 | limit: z.number().min(1).optional().describe('Maximum number of records to return'),
64 | offset: z.number().min(0).optional().describe('Number of records to skip'),
65 | },
66 | async ({ dataset_id, limit = 10, offset = 0 }) => {
67 | try {
68 | const url = `${API_BASE_URL}${DATA_CATALOGUE_ENDPOINT}`;
69 | const params = { id: dataset_id, limit, offset };
70 |
71 | const response = await axios.get(url, { params });
72 | const data = response.data;
73 |
74 | return {
75 | content: [
76 | {
77 | type: 'text',
78 | text: JSON.stringify(data, null, 2),
79 | },
80 | ],
81 | };
82 | } catch (error) {
83 | return {
84 | content: [
85 | {
86 | type: 'text',
87 | text: `Error fetching transport data: ${error instanceof Error ? error.message : 'Unknown error'}`,
88 | },
89 | ],
90 | };
91 | }
92 | }
93 | );
94 |
95 | // GTFS Static API
96 | server.tool(
97 | prefixToolName('get_gtfs_static'),
98 | 'Gets GTFS static data for a specific transport provider',
99 | {
100 | provider: z.string().describe('Provider name (e.g., "rapidkl", "ktmb", "prasarana")'),
101 | category: z.string().optional().describe('Category for Prasarana data (required only for prasarana provider)'),
102 | limit: z.number().min(1).optional().describe('Maximum number of records to return'),
103 | offset: z.number().min(0).optional().describe('Number of records to skip'),
104 | },
105 | async ({ provider, category, limit = 10, offset = 0 }) => {
106 | try {
107 | // Use the GTFS static endpoint with provider as path parameter
108 | const url = `${API_BASE_URL}${GTFS_STATIC_ENDPOINT}/${provider}`;
109 | const params: Record<string, any> = { meta: true };
110 |
111 | if (provider === 'prasarana' && !category) {
112 | return {
113 | content: [
114 | {
115 | type: 'text',
116 | text: JSON.stringify({
117 | error: 'Category parameter is required for prasarana provider',
118 | }, null, 2),
119 | },
120 | ],
121 | };
122 | }
123 |
124 | if (category) {
125 | params.category = category;
126 | }
127 |
128 | // In a real implementation, this would download a ZIP file
129 | // For now, return the URL that would be used
130 | return {
131 | content: [
132 | {
133 | type: 'text',
134 | text: JSON.stringify({
135 | message: `GTFS static data URL for provider: ${provider}${category ? `, category: ${category}` : ''}`,
136 | url,
137 | params,
138 | note: 'This endpoint returns a ZIP file in the actual implementation'
139 | }, null, 2),
140 | },
141 | ],
142 | };
143 | } catch (error) {
144 | return {
145 | content: [
146 | {
147 | type: 'text',
148 | text: JSON.stringify({
149 | error: 'Failed to get GTFS static data',
150 | message: error instanceof Error ? error.message : 'Unknown error',
151 | }, null, 2),
152 | },
153 | ],
154 | };
155 | }
156 | }
157 | );
158 |
159 | // GTFS Realtime API
160 | server.tool(
161 | prefixToolName('get_gtfs_realtime_vehicle_position'),
162 | 'Gets GTFS realtime vehicle position data for a specific transport provider',
163 | {
164 | provider: z.string().describe('Provider name (e.g., "rapidkl", "ktmb", "prasarana")'),
165 | category: z.string().optional().describe('Category for Prasarana data (required only for prasarana provider)'),
166 | limit: z.number().min(1).optional().describe('Maximum number of records to return'),
167 | offset: z.number().min(0).optional().describe('Number of records to skip'),
168 | },
169 | async ({ provider, category, limit = 10, offset = 0 }) => {
170 | try {
171 | // Use the GTFS realtime endpoint with provider as path parameter
172 | const url = `${API_BASE_URL}${GTFS_REALTIME_ENDPOINT}/${provider}`;
173 | const params: Record<string, any> = { meta: true };
174 |
175 | if (provider === 'prasarana' && !category) {
176 | return {
177 | content: [
178 | {
179 | type: 'text',
180 | text: JSON.stringify({
181 | error: 'Category parameter is required for prasarana provider',
182 | }, null, 2),
183 | },
184 | ],
185 | };
186 | }
187 |
188 | if (category) {
189 | params.category = category;
190 | }
191 |
192 | // In a real implementation, this would return a Protocol Buffer file
193 | // For now, return the URL that would be used
194 | return {
195 | content: [
196 | {
197 | type: 'text',
198 | text: JSON.stringify({
199 | message: `GTFS realtime vehicle position data URL for provider: ${provider}${category ? `, category: ${category}` : ''}`,
200 | url,
201 | params,
202 | note: 'This endpoint returns a Protocol Buffer file in the actual implementation'
203 | }, null, 2),
204 | },
205 | ],
206 | };
207 | } catch (error) {
208 | return {
209 | content: [
210 | {
211 | type: 'text',
212 | text: JSON.stringify({
213 | error: 'Failed to get GTFS realtime data',
214 | message: error instanceof Error ? error.message : 'Unknown error',
215 | }, null, 2),
216 | },
217 | ],
218 | };
219 | }
220 | }
221 | );
222 | }
223 |
```
--------------------------------------------------------------------------------
/malaysia_open_data_mcp_plan.md:
--------------------------------------------------------------------------------
```markdown
1 | # Malaysia Open Data MCP Development Plan
2 |
3 | ## Table of Contents
4 | - [Understanding Malaysia's Open Data API](#understanding-malaysias-open-data-api)
5 | - [Core Components](#core-components)
6 | - [Available APIs](#available-apis)
7 | - [Query Parameters](#query-parameters)
8 | - [Response Format](#response-format)
9 | - [Reference: Singapore's Gahmen MCP](#reference-singapores-gahmen-mcp)
10 | - [Overview](#overview)
11 | - [Features](#features)
12 | - [Available Tools](#available-tools)
13 | - [Malaysia Open Data MCP Development Plan](#malaysia-open-data-mcp-development-plan-1)
14 | - [Proposed MCP Structure and Tools](#proposed-mcp-structure-and-tools)
15 | - [Implementation Approach](#implementation-approach)
16 | - [Technical Considerations](#technical-considerations)
17 | - [Example Implementation Structure](#example-implementation-structure)
18 | - [Next Steps](#next-steps)
19 |
20 | ## Understanding Malaysia's Open Data API
21 |
22 | ### Core Components
23 |
24 | 1. **Base Structure**:
25 | - Malaysia's Open Data API is a RESTful API built using the Django framework
26 | - Designed to provide transparent data access to all citizens
27 | - Goals include transparent data access, ease of use, and diverse data sets
28 |
29 | 2. **Base URL**:
30 | - All API requests use `https://api.data.gov.my` as the base URL
31 | - Specific endpoints are appended to this base URL
32 |
33 | 3. **Authentication**:
34 | - The API can be used with or without an API token
35 | - Tokens provide higher rate limits and are available upon request
36 | - To request a token, email [email protected] with:
37 | - Your name
38 | - Your email address
39 | - Reason for requesting an increased rate limit
40 | - Authentication header format: `Authorization: Token <YOUR_TOKEN_HERE>`
41 |
42 | 4. **Rate Limits**:
43 | - Different rate limits apply based on whether you're using an API token or not
44 | - If limits are exceeded, a 429 Too Many Requests error is returned
45 |
46 | ### Available APIs
47 |
48 | 1. **Static APIs**:
49 | - **Data Catalogue API**:
50 | - Endpoint: `https://api.data.gov.my/data-catalogue`
51 | - Required parameter: `id` (dataset identifier)
52 | - Example: `https://api.data.gov.my/data-catalogue?id=fuelprice`
53 | - Provides access to various datasets in the data catalogue
54 | - Dataset IDs can be found on the [Data Catalogue page](https://data.gov.my/data-catalogue)
55 |
56 | - **OpenDOSM API**:
57 | - Endpoint: `https://api.data.gov.my/opendosm`
58 | - Required parameter: `id` (dataset identifier)
59 | - Example: `https://api.data.gov.my/opendosm?id=cpi_core`
60 | - Provides access to Department of Statistics Malaysia data
61 | - Dataset IDs can be found on the [OpenDOSM Data Catalogue page](https://open.dosm.gov.my/data-catalogue)
62 |
63 | 2. **Realtime APIs**:
64 | - **Weather API**:
65 | - Endpoints:
66 | - 7-day forecast: `https://api.data.gov.my/weather/forecast`
67 | - Weather warnings: `https://api.data.gov.my/weather/warning`
68 | - Earthquake warnings: `https://api.data.gov.my/weather/warning/earthquake`
69 | - Data source: Malaysian Meteorological Department (MET Malaysia)
70 | - Update frequency:
71 | - 7-day forecast: Updated daily
72 | - Warning data: Updated when required
73 |
74 | - **Transport API (GTFS Static)**:
75 | - Endpoint: `https://api.data.gov.my/gtfs-static/<agency>`
76 | - Provides standardized public transportation schedules and geographic information
77 | - Available agencies:
78 | - myBAS Johor Bahru: Bus service in Johor Bahru
79 | - KTMB: Railway operator across Malaysia
80 | - Prasarana: Operator of LRT, MRT, monorail, and bus services
81 | - Update frequency:
82 | - myBAS Johor Bahru: As required
83 | - Prasarana: As required
84 | - KTMB: Daily at 00:01:00
85 |
86 | - **Transport API (GTFS Realtime)**:
87 | - Provides real-time updates to public transportation data
88 |
89 | ### Query Parameters
90 |
91 | The API supports various filtering options:
92 |
93 | 1. **Row-level filtering**:
94 | - `filter`: Case-sensitive exact string match
95 | - Format: `?filter=<value>@<column>` or `?filter=<value_1>@<column_1>,<value_2>@<column_2>,...`
96 |
97 | - `ifilter`: Case-insensitive exact string match
98 | - Format: `?ifilter=<value>@<column>`
99 |
100 | - `contains`: Case-sensitive partial string match
101 | - Format: `?contains=<value>@<column>`
102 |
103 | - `icontains`: Case-insensitive partial string match
104 | - Format: `?icontains=<value>@<column>`
105 |
106 | - `range`: Filter by numerical range
107 | - Format: `?range=<column>[<begin>:<end>]`
108 |
109 | - `date_start`/`date_end`: Filter by date range
110 | - Format: `?date_start=<YYYY-MM-DD>@<date_column>` and `?date_end=<YYYY-MM-DD>@<date_column>`
111 |
112 | - `timestamp_start`/`timestamp_end`: Filter by timestamp range
113 | - Format: `?timestamp_start=<YYYY-MM-DD HH:MM:SS>@<timestamp_column>` and `?timestamp_end=<YYYY-MM-DD HH:MM:SS>@<timestamp_column>`
114 |
115 | 2. **Result manipulation**:
116 | - `sort`: Sort results by specified columns
117 | - Format: `?sort=<column>` or `?sort=<column1>,<column2>,...`
118 | - Prefix column with `-` for descending order (e.g., `-column`)
119 |
120 | - `limit`: Limit number of records returned
121 | - Format: `?limit=<value>`
122 |
123 | 3. **Column-level filtering**:
124 | - `include`: Specify which columns to include
125 | - Format: `?include=<column1,column2,...>`
126 |
127 | - `exclude`: Specify which columns to exclude
128 | - Format: `?exclude=<column1,column2,...>`
129 | - Note: When both are provided, `include` takes precedence
130 |
131 | ### Response Format
132 |
133 | 1. **Successful Responses**:
134 | - Status code: 200 OK
135 | - Default format: List of records
136 | - With `meta=true` parameter:
137 | ```json
138 | {
139 | "meta": {...},
140 | "data": [...]
141 | }
142 | ```
143 | - `meta`: Basic information about the requested resource
144 | - `data`: Collection of requested records
145 |
146 | 2. **Error Responses**:
147 | - Format:
148 | ```json
149 | {
150 | "status": <int>,
151 | "errors": [...]
152 | }
153 | ```
154 | - `status`: Response code corresponding to the error
155 | - `errors`: Error messages or descriptions
156 |
157 | ## Reference: Singapore's Gahmen MCP
158 |
159 | ### Overview
160 |
161 | The Gahmen MCP provides a Model Context Protocol server for Singapore's data.gov.sg APIs, making government datasets easily accessible through AI systems.
162 |
163 | ### Features
164 |
165 | - Access to data.gov.sg collections and datasets
166 | - Search functionality within datasets using CKAN datastore API
167 | - Dataset download with filtering support
168 | - Built-in rate limiting (5 requests per minute, 12-second minimum interval)
169 | - No authentication required (data.gov.sg APIs are public)
170 |
171 | ### Available Tools
172 |
173 | 1. **Collections**:
174 | - `list_collections`: List all collections on data.gov.sg
175 | - `get_collection`: Get metadata for a specific collection
176 |
177 | 2. **Datasets**:
178 | - `list_datasets`: List all datasets on data.gov.sg
179 | - `get_dataset_metadata`: Get metadata for a specific dataset
180 | - `search_dataset`: Search for data within a dataset using CKAN datastore
181 | - `initiate_download`: Start downloading a dataset with optional filtering
182 | - `poll_download`: Check download status and get download URL
183 |
184 | 3. **Usage Examples**:
185 | ```javascript
186 | // Search population data
187 | search_dataset({
188 | resource_id: "d_8b84c4ee58e3cfc0ece0d773c8ca6abc",
189 | q: { "year": "2023" },
190 | limit: 10
191 | })
192 |
193 | // Get collection 522 with all dataset metadata
194 | get_collection({
195 | collectionId: "522",
196 | withDatasetMetadata: true
197 | })
198 | ```
199 |
200 | ## Malaysia Open Data MCP Development Plan
201 |
202 | Based on both the Malaysia Open Data API and the Gahmen MCP reference, here's our plan for developing an MCP for Malaysia's Open Data:
203 |
204 | ### Proposed MCP Structure and Tools
205 |
206 | 1. **Data Catalogue Tools**:
207 | - `list_datasets`: List available datasets in the Data Catalogue
208 | - `get_dataset`: Get data from a specific dataset with filtering options
209 | - `search_datasets`: Search across datasets by keywords
210 |
211 | 2. **OpenDOSM Tools**:
212 | - `list_dosm_datasets`: List available DOSM datasets
213 | - `get_dosm_dataset`: Get data from a specific DOSM dataset with filtering
214 |
215 | 3. **Weather Tools**:
216 | - `get_weather_forecast`: Get 7-day weather forecast with location filtering
217 | - `get_weather_warnings`: Get current weather warnings
218 | - `get_earthquake_warnings`: Get earthquake warnings
219 |
220 | 4. **Transport Tools**:
221 | - `list_transport_agencies`: List available transport agencies
222 | - `get_transport_data`: Get GTFS data for a specific agency
223 |
224 | 5. **General Tools**:
225 | - `search_all`: Search across all available datasets
226 |
227 | ### Implementation Approach
228 |
229 | 1. **Setup MCP Server**:
230 | - Use Smithery CLI for development and building
231 | - Structure the project with clear separation of concerns
232 |
233 | 2. **API Integration**:
234 | - Create wrapper functions for Malaysia Open Data API endpoints
235 | - Implement proper error handling and rate limiting
236 | - Support authentication for higher rate limits
237 |
238 | 3. **Query Parameter Handling**:
239 | - Create helper functions to build query parameters
240 | - Support all filtering options provided by the API
241 |
242 | 4. **Response Processing**:
243 | - Parse and format API responses for MCP consumption
244 | - Handle pagination and large result sets
245 |
246 | 5. **Documentation**:
247 | - Provide clear documentation for each tool
248 | - Include usage examples
249 |
250 | ### Technical Considerations
251 |
252 | 1. **Rate Limiting**:
253 | - Implement rate limiting to respect API quotas
254 | - Consider different limits for authenticated vs. unauthenticated requests
255 |
256 | 2. **Caching**:
257 | - Implement caching for frequently accessed data
258 | - Respect data update frequencies
259 |
260 | 3. **Error Handling**:
261 | - Provide meaningful error messages
262 | - Implement retries for transient failures
263 |
264 | 4. **Authentication**:
265 | - Support API token authentication
266 | - Store tokens securely
267 |
268 | ### Example Implementation Structure
269 |
270 | ```
271 | /
272 | ├── src/
273 | │ ├── index.js # Main entry point
274 | │ ├── tools/ # MCP tools implementation
275 | │ │ ├── catalogue.js # Data Catalogue tools
276 | │ │ ├── dosm.js # OpenDOSM tools
277 | │ │ ├── weather.js # Weather tools
278 | │ │ └── transport.js # Transport tools
279 | │ ├── api/ # API client implementations
280 | │ │ ├── client.js # Base API client
281 | │ │ ├── catalogue.js # Data Catalogue API
282 | │ │ ├── dosm.js # OpenDOSM API
283 | │ │ ├── weather.js # Weather API
284 | │ │ └── transport.js # Transport API
285 | │ └── utils/ # Utility functions
286 | │ ├── rate-limiter.js # Rate limiting
287 | │ ├── query-builder.js # Query parameter builder
288 | │ └── response-parser.js # Response parsing
289 | ├── package.json
290 | └── README.md
291 | ```
292 |
293 | ### Next Steps
294 |
295 | 1. Set up the development environment with Smithery CLI
296 | 2. Create the base API client with rate limiting
297 | 3. Implement the Data Catalogue tools as a starting point
298 | 4. Expand to other APIs (OpenDOSM, Weather, Transport)
299 | 5. Test thoroughly with various query parameters
300 | 6. Document the MCP and provide usage examples
301 |
```
--------------------------------------------------------------------------------
/src/unified-search.tools.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
2 | import { z } from 'zod';
3 |
4 | // Import search functions and types from both modules
5 | import { searchDatasets, getAllDatasets, DatasetMetadata } from './datacatalogue.tools.js';
6 | import { searchDashboards, getAllDashboards, DashboardMetadata } from './dashboards.tools.js';
7 | import { prefixToolName } from './utils/tool-naming.js';
8 |
9 | // Define result interfaces
10 | interface SearchResult {
11 | type: 'dataset' | 'dashboard';
12 | id: string;
13 | title: string;
14 | description?: string;
15 | url?: string;
16 | score: number;
17 | }
18 |
19 | /**
20 | * Unified search across both datasets and dashboards
21 | * @param query Search query
22 | * @param prioritizeType Optional type to prioritize in results ('dataset' or 'dashboard')
23 | * @returns Combined search results from both sources
24 | */
25 | // Helper function to tokenize a query into individual terms
26 | function tokenizeQuery(query: string): string[] {
27 | // Remove special characters and split by spaces
28 | return query.toLowerCase()
29 | .replace(/[^a-z0-9\s]/g, ' ')
30 | .split(/\s+/)
31 | .filter(term => term.length > 0);
32 | }
33 |
34 | // Helper function to normalize terms by removing common prefixes and handling variations
35 | function normalizeTerm(term: string): string[] {
36 | // Remove hyphens and normalize spacing
37 | let normalized = term.replace(/-/g, '').trim();
38 |
39 | // Handle common prefixes/variations
40 | if (normalized.startsWith('e') && normalized.length > 1) {
41 | // e.g., 'epayment' -> also try 'payment'
42 | return [normalized, normalized.substring(1)];
43 | }
44 |
45 | return [normalized];
46 | }
47 |
48 | // A small set of common synonyms for frequently used terms
49 | const COMMON_SYNONYMS: Record<string, string[]> = {
50 | 'payment': ['payment', 'pay', 'transaction'],
51 | 'electronic': ['electronic', 'digital', 'online', 'cashless'],
52 | 'statistics': ['statistics', 'stats', 'data', 'figures', 'numbers'],
53 | 'dashboard': ['dashboard', 'visualization', 'chart', 'graph'],
54 | 'dataset': ['dataset', 'data set', 'database', 'data'],
55 | };
56 |
57 | // Helper function to expand search terms for better matching
58 | function expandSearchTerms(term: string): string[] {
59 | const normalizedTerm = term.toLowerCase().trim();
60 |
61 | // Start with the original term
62 | let expanded = [normalizedTerm];
63 |
64 | // Add normalized variations
65 | expanded = expanded.concat(normalizeTerm(normalizedTerm));
66 |
67 | // Check for common synonyms
68 | for (const [key, synonyms] of Object.entries(COMMON_SYNONYMS)) {
69 | if (normalizedTerm === key || synonyms.includes(normalizedTerm)) {
70 | expanded = expanded.concat(synonyms);
71 | break;
72 | }
73 | }
74 |
75 | // Basic stemming for plurals
76 | if (normalizedTerm.endsWith('s')) {
77 | expanded.push(normalizedTerm.slice(0, -1)); // Remove trailing 's'
78 | } else {
79 | expanded.push(normalizedTerm + 's'); // Add trailing 's'
80 | }
81 |
82 | // Remove duplicates and return
83 | return [...new Set(expanded)];
84 | }
85 |
86 | function unifiedSearch(query: string, prioritizeType?: 'dataset' | 'dashboard'): SearchResult[] {
87 | // Tokenize the query into individual terms
88 | const queryTerms = tokenizeQuery(query);
89 |
90 | // Expand each term with better matching
91 | const expandedTerms = queryTerms.flatMap(term => expandSearchTerms(term));
92 |
93 | // Search in datasets with improved scoring
94 | const datasetResults = searchDatasets(query).map((dataset: DatasetMetadata) => {
95 | const title = dataset.title_en.toLowerCase();
96 | const id = dataset.id.toLowerCase();
97 | const description = dataset.description_en.toLowerCase();
98 |
99 | // Calculate score based on term matches
100 | let score = 0;
101 |
102 | // Check for exact query match (highest priority)
103 | if (title.includes(query.toLowerCase())) score += 10;
104 | if (description.includes(query.toLowerCase())) score += 5;
105 |
106 | // Check for individual term matches
107 | expandedTerms.forEach(term => {
108 | if (title.includes(term)) score += 3;
109 | if (id.includes(term)) score += 2;
110 | if (description.includes(term)) score += 1;
111 | });
112 |
113 | return {
114 | type: 'dataset' as const,
115 | id: dataset.id,
116 | title: dataset.title_en,
117 | description: dataset.description_en,
118 | url: `https://data.gov.my/data-catalogue/${dataset.id}`,
119 | score
120 | };
121 | });
122 |
123 | // Search in dashboards with improved scoring
124 | const dashboardResults = searchDashboards(query).map((dashboard: DashboardMetadata) => {
125 | const name = dashboard.dashboard_name.toLowerCase();
126 | const route = (dashboard.route || '').toLowerCase();
127 |
128 | // Calculate score based on term matches
129 | let score = 0;
130 |
131 | // Check for exact query match (highest priority)
132 | if (name.includes(query.toLowerCase())) score += 10;
133 | if (route.includes(query.toLowerCase())) score += 5;
134 |
135 | // Check for individual term matches
136 | expandedTerms.forEach(term => {
137 | if (name.includes(term)) score += 3;
138 | if (route.includes(term)) score += 2;
139 | });
140 |
141 | return {
142 | type: 'dashboard' as const,
143 | id: dashboard.dashboard_name,
144 | title: dashboard.dashboard_name.replace(/_/g, ' ').replace(/\b\w/g, (l: string) => l.toUpperCase()),
145 | description: dashboard.route || '',
146 | url: dashboard.route ?
147 | (dashboard.sites?.includes('opendosm') ? `https://open.dosm.gov.my${dashboard.route}` : `https://data.gov.my${dashboard.route}`)
148 | : `/dashboard/${dashboard.dashboard_name}`,
149 | score
150 | };
151 | });
152 |
153 | // Combine results
154 | let combinedResults = [...datasetResults, ...dashboardResults];
155 |
156 | // If a type is prioritized, boost its score
157 | if (prioritizeType) {
158 | combinedResults = combinedResults.map(result => {
159 | if (result.type === prioritizeType) {
160 | return { ...result, score: result.score + 5 };
161 | }
162 | return result;
163 | });
164 | }
165 |
166 | // Sort by score (descending)
167 | return combinedResults.sort((a, b) => b.score - a.score);
168 | }
169 |
170 | /**
171 | * Check if a query might be referring to a dashboard based on keywords
172 | * @param query Search query
173 | * @returns True if the query likely refers to a dashboard
174 | */
175 | function isDashboardQuery(query: string): boolean {
176 | const dashboardKeywords = ['dashboard', 'chart', 'graph', 'visualization', 'visualisation', 'stats', 'statistics'];
177 | const lowerQuery = query.toLowerCase();
178 | return dashboardKeywords.some(keyword => lowerQuery.includes(keyword));
179 | }
180 |
181 | /**
182 | * Check if a query might be referring to a dataset based on keywords
183 | * @param query Search query
184 | * @returns True if the query likely refers to a dataset
185 | */
186 | function isDatasetQuery(query: string): boolean {
187 | const datasetKeywords = ['dataset', 'data', 'catalogue', 'catalog', 'file', 'download', 'csv', 'excel', 'raw'];
188 | const lowerQuery = query.toLowerCase();
189 | return datasetKeywords.some(keyword => lowerQuery.includes(keyword));
190 | }
191 |
192 | /**
193 | * Performs an intelligent search that automatically falls back to searching both datasets and dashboards
194 | * if the primary search returns no results
195 | * @param query Search query
196 | * @param prioritizeType Optional type to prioritize in results ('dataset' or 'dashboard')
197 | * @param limit Maximum number of results to return
198 | * @returns Search results with fallback if needed
199 | */
200 | function intelligentSearch(query: string, prioritizeType?: 'dataset' | 'dashboard', limit: number = 10): {
201 | results: SearchResult[];
202 | usedFallback: boolean;
203 | fallbackType?: 'dataset' | 'dashboard';
204 | originalType?: 'dataset' | 'dashboard';
205 | } {
206 | // First try with the prioritized type
207 | const initialResults = unifiedSearch(query, prioritizeType);
208 |
209 | // If we have enough results, return them
210 | if (initialResults.length >= 3 || !prioritizeType) {
211 | return {
212 | results: initialResults.slice(0, limit),
213 | usedFallback: false,
214 | originalType: prioritizeType
215 | };
216 | }
217 |
218 | // If we have few results, try the opposite type as fallback
219 | const fallbackType = prioritizeType === 'dataset' ? 'dashboard' : 'dataset';
220 | const fallbackResults = unifiedSearch(query, fallbackType);
221 |
222 | // If fallback has results, return combined results with fallback first
223 | if (fallbackResults.length > 0) {
224 | const combinedResults = [...fallbackResults, ...initialResults]
225 | .sort((a, b) => b.score - a.score)
226 | .slice(0, limit);
227 |
228 | return {
229 | results: combinedResults,
230 | usedFallback: true,
231 | fallbackType,
232 | originalType: prioritizeType
233 | };
234 | }
235 |
236 | // If neither search yielded good results, return the initial results
237 | return {
238 | results: initialResults.slice(0, limit),
239 | usedFallback: false,
240 | originalType: prioritizeType
241 | };
242 | }
243 |
244 | export function registerUnifiedSearchTools(server: McpServer) {
245 | // Unified search across datasets and dashboards
246 | server.tool(
247 | prefixToolName('search_all'),
248 | '⭐⭐⭐ PRIMARY SEARCH TOOL: Always use this first for any data or visualization queries. Searches across both datasets and dashboards with intelligent fallback. ⭐⭐⭐',
249 | {
250 | query: z.string().describe('Search query to match against all content'),
251 | limit: z.number().min(1).max(20).optional().describe('Number of results to return (1-20)'),
252 | prioritize: z.enum(['dataset', 'dashboard']).optional().describe('Type of content to prioritize in results'),
253 | },
254 | async ({ query, limit = 10, prioritize }) => {
255 | try {
256 | // Determine if query suggests a specific content type
257 | let prioritizeType = prioritize;
258 | if (!prioritizeType) {
259 | if (isDashboardQuery(query)) {
260 | prioritizeType = 'dashboard';
261 | } else if (isDatasetQuery(query)) {
262 | prioritizeType = 'dataset';
263 | }
264 |
265 | // Special case for domain-specific queries
266 | const lowerQuery = query.toLowerCase();
267 |
268 | // Payment-related terms are more likely to be found in dashboards
269 | if (lowerQuery.includes('payment') ||
270 | lowerQuery.includes('pay') ||
271 | lowerQuery.includes('transaction') ||
272 | lowerQuery.includes('electronic') ||
273 | lowerQuery.includes('digital')) {
274 | prioritizeType = 'dashboard';
275 | }
276 |
277 | // Statistics-related terms are more likely to be found in dashboards
278 | if (lowerQuery.includes('statistics') ||
279 | lowerQuery.includes('stats') ||
280 | lowerQuery.includes('chart') ||
281 | lowerQuery.includes('graph')) {
282 | prioritizeType = 'dashboard';
283 | }
284 | }
285 |
286 | // Get intelligent search results with automatic fallback
287 | const {
288 | results: searchResults,
289 | usedFallback,
290 | fallbackType,
291 | originalType
292 | } = intelligentSearch(query, prioritizeType as 'dataset' | 'dashboard' | undefined, limit);
293 |
294 | // Group results by type for better presentation
295 | const groupedResults = {
296 | datasets: searchResults.filter(r => r.type === 'dataset'),
297 | dashboards: searchResults.filter(r => r.type === 'dashboard')
298 | };
299 |
300 | return {
301 | content: [
302 | {
303 | type: 'text',
304 | text: JSON.stringify({
305 | message: 'Unified search results',
306 | query,
307 | total_matches: searchResults.length,
308 | showing: searchResults.length,
309 | prioritized_type: originalType || 'none',
310 | used_fallback: usedFallback,
311 | fallback_type: fallbackType,
312 | search_note: usedFallback ?
313 | `Limited results found in ${originalType} search, automatically included relevant ${fallbackType} results` :
314 | undefined,
315 | results: searchResults,
316 | grouped_results: groupedResults,
317 | data_access_notes: {
318 | dashboards: 'Dashboard data is visualized on the web interface. Raw data files (e.g., parquet) cannot be directly accessed through this API.',
319 | datasets: 'Dataset metadata is available through this API. For downloading the actual data files, please visit the dataset page on the data portal.',
320 | },
321 | timestamp: new Date().toISOString()
322 | }, null, 2),
323 | },
324 | ],
325 | };
326 | } catch (error) {
327 | return {
328 | content: [
329 | {
330 | type: 'text',
331 | text: JSON.stringify({
332 | error: 'Failed to perform unified search',
333 | message: error instanceof Error ? error.message : String(error),
334 | timestamp: new Date().toISOString()
335 | }, null, 2),
336 | },
337 | ],
338 | };
339 | }
340 | }
341 | );
342 | }
343 |
```