# Directory Structure
```
├── .github
│ └── npm-publish.yml
├── .gitignore
├── components.json
├── eslint.config.js
├── index.html
├── LICENSE
├── package-lock.json
├── package.json
├── public
│ └── vite.svg
├── README.md
├── src
│ ├── App
│ │ ├── context.tsx
│ │ ├── execute.tsx
│ │ └── index.tsx
│ ├── assets
│ │ └── react.svg
│ ├── components
│ │ └── ui
│ │ ├── button.tsx
│ │ ├── card.tsx
│ │ ├── click-to-edit.tsx
│ │ ├── dropdown-menu.tsx
│ │ ├── scroll-area.tsx
│ │ ├── tabs.tsx
│ │ └── textarea.tsx
│ ├── hooks
│ │ └── use-global-stage.ts
│ ├── index.css
│ ├── lib
│ │ └── utils.ts
│ ├── main.tsx
│ ├── mcp
│ │ ├── eval.ts
│ │ ├── handle-browser-event.ts
│ │ ├── index.ts
│ │ ├── logger.ts
│ │ ├── recording
│ │ │ ├── events.ts
│ │ │ ├── index.ts
│ │ │ ├── init-recording.ts
│ │ │ ├── selector-engine.ts
│ │ │ ├── snowflake.ts
│ │ │ └── utils.ts
│ │ ├── state.ts
│ │ └── toolbox.ts
│ ├── server.ts
│ ├── types.ts
│ ├── vite-env.d.ts
│ └── web-server.ts
├── tsconfig.app.json
├── tsconfig.json
├── tsconfig.lib.json
├── tsconfig.node.json
├── tsup.config.ts
└── vite.config.ts
```
# Files
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
```
1 | node_modules
2 | build
3 | .npmrc
4 | dist
```
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
```markdown
1 | # How to Use playwright-mcp?
2 |
3 | [](https://www.npmjs.com/package/playwright-mcp) [](https://ashish-bansal.github.io/playwright-mcp/)
4 |
5 | ## Introduction
6 |
7 | Ever tried using Cursor or Claude to write Playwright tests? Yeah, it's kind of a mess.
8 |
9 | Why? Because your AI assistant has no clue what's on your web page. It's like trying to describe a painting to someone who's blindfolded. The result?
10 |
11 | - **Flaky tests** → The AI is guessing selectors, and it gets them wrong.
12 | - **Broken scripts** → You spend more time fixing tests than writing them.
13 | - **Debugging nightmares** → The AI can't "see" what's happening, so you end up doing all the heavy lifting.
14 |
15 | Wouldn't it be nice if your AI could actually understand your web page instead of just making educated guesses?
16 |
17 | ### Enter playwright-mcp !
18 |
19 | `playwright-mcp` gives your AI assistant superpowers by making the browser fully visible to it. Instead of working in the dark, your AI assistant now has real-time access to the page DOM, elements, and interactions.
20 |
21 | ### How playwright-mcp Works (AKA How We Fix This Mess)
22 |
23 | Once installed, playwright-mcp spins up a Playwright-controlled browser and gives your AI assistant full access to it. This unlocks:
24 |
25 | 1. **Recording interactions** → Click, type, scroll—let playwright-mcp turn your actions into fully functional Playwright test cases.
26 | 2. **Taking screenshots** → Capture elements or full pages so your AI gets actual visual context (no more guessing).
27 | 3. **Extracting the DOM** → Grab the entire HTML structure so the AI can generate rock-solid selectors.
28 | 4. **Executing JavaScript** → Run custom JS inside the browser for debugging, automation, or just for fun.
29 |
30 | ### Why You'll Love playwright-mcp
31 |
32 | 🚀 **AI-generated tests that actually work** → No more flaky selectors, broken tests, or guesswork.
33 |
34 | ⏳ **Massive time savings** → Write and debug Playwright tests 5x faster.
35 |
36 | 🛠️ **Fewer headaches** → Your AI assistant gets live page context, so it can generate real test cases.
37 |
38 | 🔌 **Works out of the box** → Plug it into Cursor, Claude, WindSurf, or whatever you use—it just works.
39 |
40 | #### **Give Your AI the Context It Deserves**
41 |
42 | Tired of fixing AI-generated tests? Stop debugging AI's bad guesses—start writing flawless Playwright tests. Use the guide below to install playwright-mcp and let your AI assistant actually help you for once.
43 |
44 | ---
45 |
46 | ### Installation: Get Up and Running in No Time
47 |
48 | ### Prerequisites (a.k.a. What You Need Before You Start)
49 |
50 | Before you install `playwright-mcp`, make sure you have:
51 |
52 | ✅ Node.js installed on your system (because, well… it's an npm package)
53 |
54 | ✅ Playwright and its Chromium browser installed
55 |
56 | ✅ An IDE that supports MCP, like Cursor
57 |
58 | ✅ A brain that's ready to automate Playwright tests like a pro
59 |
60 | ### Setting Up playwright-mcp (It's Easy, I Promise)
61 |
62 | #### Connect It to Your IDE (Cursor Setup)
63 |
64 | If you're using Cursor, follow these steps to connect `playwright-mcp` like a boss:
65 |
66 | 1. Open Cursor IDE
67 | 2. Navigate to Settings (⚙️)
68 | 3. Select Cursor Settings → Go to the MCP tab
69 | 4. Click "Add new MCP server"
70 | 5. Fill in the following details:
71 |
72 | 
73 |
74 |
75 | - Name → `playwright-mcp`
76 | - Command → `npx -y playwright-mcp`
77 | 6. Click "Add", and boom—you're connected! 🚀
78 |
79 | Note: If clicking on "Add new MCP server", opens a file(.cursor/mcp.json), Paste the following code and save:
80 |
81 | ```jsx
82 | {
83 | "mcpServers": {
84 | "playwright-mcp": {
85 | "command": "npx",
86 | "args": [
87 | "-y",
88 | "playwright-mcp"
89 | ]
90 | }
91 | }
92 | }
93 | ```
94 |
95 | Now Cursor actually understands your web pages. No more random test suggestions based on zero context! Head to the [Claude tutorial](https://ashish-bansal.github.io/playwright-mcp/tutorials/claude-desktop-tutorial) or [Cursor tutorial](https://ashish-bansal.github.io/playwright-mcp/tutorials/cursor-tutorial) to understand it in details.
96 |
97 | ---
98 |
99 | ### **Connect It to Claude desktop**
100 |
101 | Wait… Does It Work with Other AI Assistants? Yes! While `playwright-mcp` is a match made in heaven for IDEs, you can use it with Claude desktop to write tests as well.
102 |
103 | 1. Install `playwright-mcp` (The Easy Part)
104 | 1. First things first, fire up your terminal and run:
105 | 2. `npm install -g playwright-mcp`
106 | 2. Hook It Up to Claude Desktop
107 | 1. Find the Configuration File
108 | 2. On windows
109 | 1. `%APPDATA%\Claude\claude_desktop_config.json`
110 | 3. On macOS:
111 | 1. `~/Library/Application Support/Claude/claude_desktop_config.json`
112 | 4. Update the config file
113 |
114 | ```jsx
115 | {
116 | "mcpServers": {
117 | "playwright": {
118 | "command": "npx",
119 | "args": ["-y", "playwright-mcp"]
120 | }
121 | }
122 | }
123 | ```
124 |
125 | 3. Restart Claude Desktop (Because It's a New Day)
126 | 1. Close and reopen Claude Desktop to apply the changes.
127 | 4. Verify That It's Working
128 | 1. Once everything is set up, let's test if Claude can actually talk to Playwright now.
129 | 2. Open Claude and ask: "List all available MCP tools."
130 | 3. If `playwright-mcp` is installed correctly, it should list tools like:
131 | 1. `get-context`
132 | 2. `get-full-dom`
133 | 3. `get-screenshot`
134 | 4. `execute-code`
135 | 5. `init-browser`
136 | 6. `validate-selectors`
137 | 4. Ask Claude to init browser and a browser should open up after your approval!
138 |
139 | Now that the Calude has access to the web pages. You can ask it write highly contextual tests! Head to the [Claude tutorial](https://ashish-bansal.github.io/playwright-mcp/tutorials/claude-desktop-tutorial) or [Cursor tutorial](https://ashish-bansal.github.io/playwright-mcp/tutorials/cursor-tutorial) to understand it in details.
140 |
141 |
142 | [📖 **View Documentation**](https://ashish-bansal.github.io/playwright-mcp/)
143 |
```
--------------------------------------------------------------------------------
/src/vite-env.d.ts:
--------------------------------------------------------------------------------
```typescript
1 | /// <reference types="vite/client" />
2 |
```
--------------------------------------------------------------------------------
/src/mcp/recording/index.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { initRecording } from "./init-recording.js";
2 |
3 | export { initRecording };
```
--------------------------------------------------------------------------------
/src/lib/utils.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { clsx, type ClassValue } from "clsx"
2 | import { twMerge } from "tailwind-merge"
3 |
4 | export function cn(...inputs: ClassValue[]) {
5 | return twMerge(clsx(inputs))
6 | }
7 |
```
--------------------------------------------------------------------------------
/src/mcp/recording/snowflake.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { Snowflake } from "@skorotkiewicz/snowflake-id";
2 |
3 | const snowflake = new Snowflake(42 * 10);
4 |
5 | export const getSnowflakeId = async () => {
6 | return await snowflake.generate();
7 | }
8 |
```
--------------------------------------------------------------------------------
/tsconfig.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "files": [],
3 | "references": [
4 | { "path": "./tsconfig.app.json" },
5 | { "path": "./tsconfig.node.json" }
6 | ],
7 | "compilerOptions": {
8 | "baseUrl": ".",
9 | "paths": {
10 | "@/*": ["./src/*"]
11 | }
12 | }
13 | }
14 |
```
--------------------------------------------------------------------------------
/src/main.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import { StrictMode } from 'react'
2 | import { createRoot } from 'react-dom/client'
3 | import './index.css'
4 | import App from './App'
5 |
6 | createRoot(document.getElementById('root')!).render(
7 | <StrictMode>
8 | <App />
9 | </StrictMode>,
10 | )
11 |
```
--------------------------------------------------------------------------------
/tsup.config.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { defineConfig } from 'tsup'
2 |
3 | export default defineConfig({
4 | env: {
5 | NODE_ENV: process.env.NODE_ENV || 'development',
6 | },
7 | entry: ['src/server.ts', 'src/mcp/interceptor.ts', 'src/mcp/toolbox.ts'],
8 | tsconfig: 'tsconfig.lib.json',
9 | splitting: true,
10 | format: ['cjs', 'esm'],
11 | clean: true,
12 | })
13 |
```
--------------------------------------------------------------------------------
/index.html:
--------------------------------------------------------------------------------
```html
1 | <!doctype html>
2 | <html lang="en">
3 | <head>
4 | <meta charset="UTF-8" />
5 | <link rel="icon" type="image/svg+xml" href="/vite.svg" />
6 | <meta name="viewport" content="width=device-width, initial-scale=1.0" />
7 | <title>Vite + React + TS</title>
8 | </head>
9 | <body>
10 | <div id="root"></div>
11 | <script>window.triggerSyncToReact()</script>
12 | <script type="module" src="/src/main.tsx"></script>
13 | </body>
14 | </html>
15 |
```
--------------------------------------------------------------------------------
/components.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "$schema": "https://ui.shadcn.com/schema.json",
3 | "style": "new-york",
4 | "rsc": false,
5 | "tsx": true,
6 | "tailwind": {
7 | "config": "",
8 | "css": "src/index.css",
9 | "baseColor": "neutral",
10 | "cssVariables": true,
11 | "prefix": ""
12 | },
13 | "aliases": {
14 | "components": "@/components",
15 | "utils": "@/lib/utils",
16 | "ui": "@/components/ui",
17 | "lib": "@/lib",
18 | "hooks": "@/hooks"
19 | },
20 | "iconLibrary": "lucide"
21 | }
```
--------------------------------------------------------------------------------
/vite.config.ts:
--------------------------------------------------------------------------------
```typescript
1 | // vite.ui.config.js
2 | import path from "path"
3 | import tailwindcss from "@tailwindcss/vite"
4 | import react from "@vitejs/plugin-react"
5 | import { defineConfig } from "vite"
6 |
7 |
8 | export default defineConfig({
9 | plugins: [react(), tailwindcss()],
10 | server: {
11 | port: 5174
12 | },
13 | build: {
14 | outDir: 'dist/ui',
15 | // Generate assets that will be embedded in your Node.js package
16 | emptyOutDir: true
17 | },
18 | resolve: {
19 | alias: {
20 | "@": path.resolve(__dirname, "./src"),
21 | },
22 | },
23 | })
```
--------------------------------------------------------------------------------
/tsconfig.node.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "compilerOptions": {
3 | "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
4 | "target": "ES2022",
5 | "lib": ["ES2023"],
6 | "module": "ESNext",
7 | "skipLibCheck": true,
8 |
9 | /* Bundler mode */
10 | "moduleResolution": "bundler",
11 | "allowImportingTsExtensions": true,
12 | "isolatedModules": true,
13 | "moduleDetection": "force",
14 | "noEmit": true,
15 |
16 | /* Linting */
17 | "strict": true,
18 | "noUnusedLocals": true,
19 | "noUnusedParameters": true,
20 | "noFallthroughCasesInSwitch": true,
21 | "noUncheckedSideEffectImports": true
22 | },
23 | "include": ["vite.config.ts"]
24 | }
25 |
```
--------------------------------------------------------------------------------
/tsconfig.lib.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "compilerOptions": {
3 | "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
4 | "target": "ES2022",
5 | "lib": ["ES2023", "DOM", "DOM.Iterable"],
6 | "module": "ESNext",
7 | "skipLibCheck": true,
8 |
9 | /* Bundler mode */
10 | "moduleResolution": "bundler",
11 | "allowImportingTsExtensions": true,
12 | "isolatedModules": true,
13 | "moduleDetection": "force",
14 | "noEmit": true,
15 |
16 | /* Linting */
17 | "strict": true,
18 | "noUnusedLocals": true,
19 | "noUnusedParameters": true,
20 | "noFallthroughCasesInSwitch": true,
21 | "noUncheckedSideEffectImports": true
22 | },
23 | "include": ["vite.config.ts"]
24 | }
25 |
```
--------------------------------------------------------------------------------
/src/hooks/use-global-stage.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { useCallback, useEffect, useState } from "react";
2 |
3 | export const useGlobalState = () => {
4 | const [state, setState] = useState(window.globalState);
5 |
6 | useEffect(() => {
7 | // Subscribe to changes from other components/Node
8 | const subscriptionId = window.stateSubscribers.length;
9 | window.stateSubscribers.push(setState);
10 |
11 | return () => {
12 | window.stateSubscribers = window.stateSubscribers.filter(
13 | (_, index) => index !== subscriptionId
14 | );
15 | };
16 | }, []);
17 |
18 | // Create an update function that syncs with Node
19 | const updateState = useCallback((update: any) => {
20 | window.updateGlobalState(update);
21 | }, []);
22 |
23 | return [state, updateState];
24 | }
25 |
```
--------------------------------------------------------------------------------
/.github/npm-publish.yml:
--------------------------------------------------------------------------------
```yaml
1 | name: Publish to npm
2 |
3 | on:
4 | release:
5 | types: [created]
6 | workflow_dispatch:
7 |
8 | jobs:
9 | build-and-publish:
10 | runs-on: ubuntu-latest
11 | permissions:
12 | contents: read
13 | packages: write
14 | steps:
15 | - name: Checkout code
16 | uses: actions/checkout@v3
17 |
18 | - name: Setup Node.js
19 | uses: actions/setup-node@v3
20 | with:
21 | node-version: '18.x'
22 | registry-url: 'https://registry.npmjs.org'
23 |
24 | - name: Install dependencies
25 | run: npm ci
26 |
27 | - name: Build
28 | run: npm run build
29 |
30 | - name: Set up npm authentication
31 | run: |
32 | echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}" > .npmrc
33 |
34 | - name: Publish to npm
35 | run: npm publish
36 |
```
--------------------------------------------------------------------------------
/eslint.config.js:
--------------------------------------------------------------------------------
```javascript
1 | import js from '@eslint/js'
2 | import globals from 'globals'
3 | import reactHooks from 'eslint-plugin-react-hooks'
4 | import reactRefresh from 'eslint-plugin-react-refresh'
5 | import tseslint from 'typescript-eslint'
6 |
7 | export default tseslint.config(
8 | { ignores: ['dist'] },
9 | {
10 | extends: [js.configs.recommended, ...tseslint.configs.recommended],
11 | files: ['**/*.{ts,tsx}'],
12 | languageOptions: {
13 | ecmaVersion: 2020,
14 | globals: globals.browser,
15 | },
16 | plugins: {
17 | 'react-hooks': reactHooks,
18 | 'react-refresh': reactRefresh,
19 | },
20 | rules: {
21 | ...reactHooks.configs.recommended.rules,
22 | 'react-refresh/only-export-components': [
23 | 'warn',
24 | { allowConstantExport: true },
25 | ],
26 | },
27 | },
28 | )
29 |
```
--------------------------------------------------------------------------------
/tsconfig.app.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "compilerOptions": {
3 | "tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
4 | "target": "ES2020",
5 | "useDefineForClassFields": true,
6 | "lib": ["ES2020", "DOM", "DOM.Iterable"],
7 | "module": "ESNext",
8 | "skipLibCheck": true,
9 |
10 | /* Bundler mode */
11 | "moduleResolution": "bundler",
12 | "allowImportingTsExtensions": true,
13 | "isolatedModules": true,
14 | "moduleDetection": "force",
15 | "noEmit": true,
16 | "jsx": "react-jsx",
17 |
18 | /* Linting */
19 | "strict": true,
20 | "noUnusedLocals": true,
21 | "noUnusedParameters": true,
22 | "noFallthroughCasesInSwitch": true,
23 | "noUncheckedSideEffectImports": true,
24 |
25 | "baseUrl": ".",
26 | "paths": {
27 | "@/*": ["./src/*"]
28 | }
29 |
30 | },
31 | "include": ["src"]
32 | }
33 |
```
--------------------------------------------------------------------------------
/src/components/ui/textarea.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import * as React from "react"
2 |
3 | import { cn } from "@/lib/utils"
4 |
5 | function Textarea({ className, ...props }: React.ComponentProps<"textarea">) {
6 | return (
7 | <textarea
8 | data-slot="textarea"
9 | className={cn(
10 | "border-input placeholder:text-muted-foreground focus-visible:border-ring focus-visible:ring-ring/50 aria-invalid:ring-destructive/20 dark:aria-invalid:ring-destructive/40 aria-invalid:border-destructive dark:bg-input/30 flex field-sizing-content min-h-16 w-full rounded-md border bg-transparent px-3 py-2 text-base shadow-xs transition-[color,box-shadow] outline-none focus-visible:ring-[3px] disabled:cursor-not-allowed disabled:opacity-50 md:text-sm",
11 | className
12 | )}
13 | {...props}
14 | />
15 | )
16 | }
17 |
18 | export { Textarea }
19 |
```
--------------------------------------------------------------------------------
/src/server.ts:
--------------------------------------------------------------------------------
```typescript
1 | #!/usr/bin/env node
2 |
3 | import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
4 | import { server } from "./mcp/index";
5 | import { webServer, isPortInUse } from "./web-server";
6 |
7 | async function main() {
8 | const transport = new StdioServerTransport();
9 | await server.connect(transport);
10 | console.error("MCP Server started");
11 |
12 | if (process.env.NODE_ENV !== 'development') {
13 | const portInUse = await isPortInUse(5174);
14 | if (!portInUse) {
15 | webServer.listen(5174, () => {
16 | console.error("Web server started");
17 | });
18 | } else {
19 | console.error("Port 5174 is in use, skipping web server");
20 | }
21 | }
22 | }
23 |
24 | main().catch((error) => {
25 | console.error("Fatal error in main", error);
26 | process.exit(1);
27 | });
28 |
```
--------------------------------------------------------------------------------
/src/mcp/logger.ts:
--------------------------------------------------------------------------------
```typescript
1 | type LogLevel = 'debug' | 'info' | 'warn' | 'error'
2 |
3 | class Logger {
4 | private level: LogLevel
5 |
6 | constructor(level: LogLevel = 'info') {
7 | this.level = level
8 | }
9 |
10 | private shouldLog(messageLevel: LogLevel): boolean {
11 | const levels: LogLevel[] = ['debug', 'info', 'warn', 'error']
12 | return levels.indexOf(messageLevel) >= levels.indexOf(this.level)
13 | }
14 |
15 | private log(level: LogLevel, ...args: any[]): void {
16 | if (this.shouldLog(level)) {
17 | console[level](...args)
18 | }
19 | }
20 |
21 | debug(...args: any[]): void {
22 | this.log('debug', ...args)
23 | }
24 |
25 | info(...args: any[]): void {
26 | this.log('info', ...args)
27 | }
28 |
29 | warn(...args: any[]): void {
30 | this.log('warn', ...args)
31 | }
32 |
33 | error(...args: any[]): void {
34 | this.log('error', ...args)
35 | }
36 | }
37 |
38 | const logger = new Logger()
39 |
40 | export { logger, Logger }
41 |
```
--------------------------------------------------------------------------------
/src/types.ts:
--------------------------------------------------------------------------------
```typescript
1 | declare global {
2 | interface Message {
3 | type: 'DOM' | 'Image' | 'Text' | 'Interaction';
4 | content: string;
5 | windowUrl: string;
6 | }
7 |
8 | interface Window {
9 | // for the user page
10 | mcpStartPicking: (pickingType: 'DOM' | 'Image') => void;
11 | mcpStopPicking: () => void;
12 | onElementPicked: (message: Message) => void;
13 | // for the iframe
14 | triggerMcpStartPicking: (pickingType: 'DOM' | 'Image') => void;
15 | triggerMcpStopPicking: () => void;
16 | // for the react page
17 | globalState: any;
18 | stateSubscribers: ((state: any) => void)[];
19 | notifyStateSubscribers: () => void;
20 | updateGlobalState: (state: any) => void;
21 | triggerSyncToReact: () => void;
22 | // for recording
23 | recordDOM: (dom: string, elementUUID: string) => Promise<void>;
24 | recordInput: (dom: string, elementUUID: string, value: string) => Promise<void>;
25 | recordKeyPress: (dom: string, keys: string[]) => Promise<void>;
26 | }
27 | }
28 |
29 |
30 |
```
--------------------------------------------------------------------------------
/src/mcp/recording/events.ts:
--------------------------------------------------------------------------------
```typescript
1 | export enum BrowserEventType {
2 | Click = 'click',
3 | Input = 'input',
4 | KeyPress = 'key-press',
5 | OpenPage = 'open-page',
6 | }
7 |
8 | export interface BaseBrowserEvent {
9 | eventId: string
10 | dom: string
11 | windowUrl: string
12 | }
13 |
14 | export interface ClickBrowserEvent extends BaseBrowserEvent {
15 | type: BrowserEventType.Click
16 | elementUUID: string
17 | selectors: string[]
18 | elementName?: string
19 | elementType?: string
20 | }
21 |
22 | export interface InputBrowserEvent extends BaseBrowserEvent {
23 | type: BrowserEventType.Input
24 | elementUUID: string
25 | typedText: string
26 | selectors: string[]
27 | elementName?: string
28 | elementType?: string
29 | }
30 |
31 | export interface KeyPressBrowserEvent extends BaseBrowserEvent {
32 | type: BrowserEventType.KeyPress
33 | keys: string[]
34 | }
35 |
36 | export interface OpenPageBrowserEvent extends BaseBrowserEvent {
37 | type: BrowserEventType.OpenPage
38 | title: string
39 | }
40 |
41 | export type BrowserEvent =
42 | | ClickBrowserEvent
43 | | InputBrowserEvent
44 | | KeyPressBrowserEvent
45 | | OpenPageBrowserEvent
46 |
```
--------------------------------------------------------------------------------
/src/App/index.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import React from 'react';
2 | import { Tabs, TabsList, TabsTrigger, TabsContent } from '@/components/ui/tabs';
3 | import Context from './context';
4 | import Execute from './execute';
5 |
6 | const App: React.FC = () => {
7 | return (
8 | <div className="fixed top-0 right-0 w-full h-screen bg-gray-100 border-l border-zinc-200 z-[999999] flex flex-col overflow-hidden">
9 | <div className="p-4 bg-white border-b border-zinc-200 flex items-center justify-center">
10 | <h3 className="m-0 text-base font-medium text-gray-900">
11 | Playwright MCP
12 | </h3>
13 | </div>
14 |
15 | <Tabs defaultValue="context" className="flex-1 flex flex-col">
16 | <div className="p-4 bg-white border-b border-zinc-200">
17 | <TabsList>
18 | <TabsTrigger value="context">Context</TabsTrigger>
19 | <TabsTrigger value="execute">Execute</TabsTrigger>
20 | </TabsList>
21 | </div>
22 |
23 | <TabsContent value="context" className="flex-1">
24 | <Context />
25 | </TabsContent>
26 |
27 | <TabsContent value="execute" className="flex-1">
28 | <Execute />
29 | </TabsContent>
30 | </Tabs>
31 | </div>
32 | );
33 | };
34 |
35 | export default App;
36 |
```
--------------------------------------------------------------------------------
/public/vite.svg:
--------------------------------------------------------------------------------
```
1 | <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="31.88" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 257"><defs><linearGradient id="IconifyId1813088fe1fbc01fb466" x1="-.828%" x2="57.636%" y1="7.652%" y2="78.411%"><stop offset="0%" stop-color="#41D1FF"></stop><stop offset="100%" stop-color="#BD34FE"></stop></linearGradient><linearGradient id="IconifyId1813088fe1fbc01fb467" x1="43.376%" x2="50.316%" y1="2.242%" y2="89.03%"><stop offset="0%" stop-color="#FFEA83"></stop><stop offset="8.333%" stop-color="#FFDD35"></stop><stop offset="100%" stop-color="#FFA800"></stop></linearGradient></defs><path fill="url(#IconifyId1813088fe1fbc01fb466)" d="M255.153 37.938L134.897 252.976c-2.483 4.44-8.862 4.466-11.382.048L.875 37.958c-2.746-4.814 1.371-10.646 6.827-9.67l120.385 21.517a6.537 6.537 0 0 0 2.322-.004l117.867-21.483c5.438-.991 9.574 4.796 6.877 9.62Z"></path><path fill="url(#IconifyId1813088fe1fbc01fb467)" d="M185.432.063L96.44 17.501a3.268 3.268 0 0 0-2.634 3.014l-5.474 92.456a3.268 3.268 0 0 0 3.997 3.378l24.777-5.718c2.318-.535 4.413 1.507 3.936 3.838l-7.361 36.047c-.495 2.426 1.782 4.5 4.151 3.78l15.304-4.649c2.372-.72 4.652 1.36 4.15 3.788l-11.698 56.621c-.732 3.542 3.979 5.473 5.943 2.437l1.313-2.028l72.516-144.72c1.215-2.423-.88-5.186-3.54-4.672l-25.505 4.922c-2.396.462-4.435-1.77-3.759-4.114l16.646-57.705c.677-2.35-1.37-4.583-3.769-4.113Z"></path></svg>
```
--------------------------------------------------------------------------------
/src/components/ui/scroll-area.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import * as React from "react"
2 | import * as ScrollAreaPrimitive from "@radix-ui/react-scroll-area"
3 |
4 | import { cn } from "@/lib/utils"
5 |
6 | function ScrollArea({
7 | className,
8 | children,
9 | ...props
10 | }: React.ComponentProps<typeof ScrollAreaPrimitive.Root>) {
11 | return (
12 | <ScrollAreaPrimitive.Root
13 | data-slot="scroll-area"
14 | className={cn("relative", className)}
15 | {...props}
16 | >
17 | <ScrollAreaPrimitive.Viewport
18 | data-slot="scroll-area-viewport"
19 | className="ring-ring/10 dark:ring-ring/20 dark:outline-ring/40 outline-ring/50 size-full rounded-[inherit] transition-[color,box-shadow] focus-visible:ring-4 focus-visible:outline-1"
20 | >
21 | {children}
22 | </ScrollAreaPrimitive.Viewport>
23 | <ScrollBar />
24 | <ScrollAreaPrimitive.Corner />
25 | </ScrollAreaPrimitive.Root>
26 | )
27 | }
28 |
29 | function ScrollBar({
30 | className,
31 | orientation = "vertical",
32 | ...props
33 | }: React.ComponentProps<typeof ScrollAreaPrimitive.ScrollAreaScrollbar>) {
34 | return (
35 | <ScrollAreaPrimitive.ScrollAreaScrollbar
36 | data-slot="scroll-area-scrollbar"
37 | orientation={orientation}
38 | className={cn(
39 | "flex touch-none p-px transition-colors select-none",
40 | orientation === "vertical" &&
41 | "h-full w-2.5 border-l border-l-transparent",
42 | orientation === "horizontal" &&
43 | "h-2.5 flex-col border-t border-t-transparent",
44 | className
45 | )}
46 | {...props}
47 | >
48 | <ScrollAreaPrimitive.ScrollAreaThumb
49 | data-slot="scroll-area-thumb"
50 | className="bg-border relative flex-1 rounded-full"
51 | />
52 | </ScrollAreaPrimitive.ScrollAreaScrollbar>
53 | )
54 | }
55 |
56 | export { ScrollArea, ScrollBar }
57 |
```
--------------------------------------------------------------------------------
/src/mcp/state.ts:
--------------------------------------------------------------------------------
```typescript
1 | import type { Page } from "playwright";
2 | type PickingType = 'DOM' | 'Image';
3 |
4 | let globalState = {
5 | messages: [] as Message[],
6 | pickingType: null as PickingType | null,
7 | recordingInteractions: false as boolean,
8 | code: `async function run(page) {
9 | let title = await page.title();
10 | return title
11 | }` as string,
12 | }
13 |
14 | async function initState(page: Page) {
15 | // function to notify Node.js from React
16 | await page.exposeFunction('updateGlobalState', (state: any) => {
17 | updateState(page, state);
18 | });
19 |
20 | await page.exposeFunction('triggerSyncToReact', () => {
21 | updateState(page, getState());
22 | });
23 |
24 | await page.addInitScript((state) => {
25 | if (window.globalState) {
26 | return
27 | }
28 |
29 | window.globalState = state;
30 | window.stateSubscribers = [];
31 |
32 | // function to notify other components
33 | window.notifyStateSubscribers = () => {
34 | window.stateSubscribers.forEach(cb => cb(window.globalState));
35 | };
36 | }, globalState);
37 | }
38 |
39 | async function syncToReact(page: Page, state: typeof globalState) {
40 | const allFrames = await page.frames();
41 | const toolboxFrame = allFrames.find(f => f.name() === 'toolbox-frame');
42 | if (!toolboxFrame) {
43 | console.error('Toolbox frame not found');
44 | return;
45 | }
46 |
47 | try {
48 | await toolboxFrame.evaluate((state) => {
49 | window.globalState = state;
50 | window.notifyStateSubscribers();
51 | }, state);
52 | } catch (error) {
53 | console.debug('Error syncing to React:', error);
54 | }
55 | }
56 |
57 | const getState = () => {
58 | return structuredClone(globalState);
59 | }
60 |
61 | const updateState = (page: Page, state: typeof globalState) => {
62 | globalState = structuredClone(state);
63 | syncToReact(page, state);
64 | }
65 |
66 | export { initState, getState, updateState, type Message };
67 |
```
--------------------------------------------------------------------------------
/src/components/ui/click-to-edit.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import type * as React from 'react'
2 |
3 | import { cn } from '@/lib/utils'
4 | import { useEffect, useRef } from 'react'
5 | import ContentEditable from 'react-contenteditable'
6 | import useStateRef from 'react-usestateref'
7 |
8 | const ClickToEdit = ({
9 | text,
10 | textClassName,
11 | className,
12 | placeholder,
13 | onSave,
14 | ...props
15 | }: {
16 | text: string
17 | textClassName?: string
18 | className?: string
19 | placeholder?: string
20 | onSave: (text: string) => void
21 | } & React.HTMLAttributes<HTMLDivElement>) => {
22 | const [value, setValue, valueRef] = useStateRef(text)
23 | const contentEditable = useRef<HTMLElement>(null)
24 |
25 | const handleChange = (e: { target: { value: string } }) => {
26 | setValue(contentEditable.current?.textContent || '')
27 | }
28 |
29 | const handleBlur = () => {
30 | const currentValue = valueRef.current
31 | if (currentValue && currentValue !== text) {
32 | onSave(currentValue)
33 | }
34 | }
35 |
36 | const handleKeyDown = (e: React.KeyboardEvent) => {
37 | if (e.key === 'Enter') {
38 | e.preventDefault()
39 | contentEditable.current?.blur()
40 | }
41 | }
42 |
43 | useEffect(() => {
44 | setValue(text)
45 | }, [text])
46 |
47 | return (
48 | <div className={cn('flex items-center gap-2 w-auto', className)}>
49 | <ContentEditable
50 | innerRef={contentEditable}
51 | html={value}
52 | onChange={handleChange}
53 | className={cn(
54 | 'flex-1 flex items-center gap-1 group outline-none',
55 | placeholder &&
56 | !value &&
57 | 'before:content-[attr(data-placeholder)] before:text-muted-foreground',
58 | textClassName,
59 | )}
60 | onBlur={handleBlur}
61 | onKeyDown={handleKeyDown}
62 | data-placeholder={placeholder}
63 | {...props}
64 | />
65 | </div>
66 | )
67 | }
68 |
69 | ClickToEdit.displayName = 'ClickToEdit'
70 |
71 | export { ClickToEdit }
72 |
```
--------------------------------------------------------------------------------
/src/mcp/handle-browser-event.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { updateState } from "./state";
2 | import { getState } from "./state";
3 | import { preprocessBrowserEvent } from "./recording/utils";
4 | import { Page } from "playwright";
5 | import _ from "lodash";
6 |
7 | export const handleBrowserEvent = (page: Page) => {
8 | const eventQueue: any[] = [];
9 |
10 | const processEvents = _.debounce(() => {
11 | if (eventQueue.length === 0) {
12 | return;
13 | }
14 |
15 | // Skip events for same element and type
16 | while (eventQueue.length > 1) {
17 | const currentEvent = eventQueue[0];
18 | const nextEvent = eventQueue[1];
19 | if (currentEvent.type === nextEvent.type && currentEvent.elementUUID === nextEvent.elementUUID) {
20 | eventQueue.shift();
21 | } else {
22 | break;
23 | }
24 | }
25 |
26 | const event = eventQueue.shift();
27 | const state = getState();
28 |
29 | preprocessBrowserEvent(event);
30 |
31 | if (state.messages.length > 0) {
32 | const lastMessage = state.messages[state.messages.length - 1];
33 | if (lastMessage.type === 'Interaction') {
34 | const lastInteraction = JSON.parse(lastMessage.content);
35 | if (lastInteraction.type === "input" && lastInteraction.elementUUID === event.elementUUID) {
36 | lastInteraction.typedText = event.typedText;
37 | state.messages[state.messages.length - 1] = {
38 | type: 'Interaction',
39 | content: JSON.stringify(lastInteraction),
40 | windowUrl: event.windowUrl,
41 | };
42 | updateState(page, state);
43 | return;
44 | }
45 | }
46 | }
47 |
48 | state.messages.push({
49 | type: 'Interaction',
50 | content: JSON.stringify(event),
51 | windowUrl: event.windowUrl,
52 | });
53 | updateState(page, state);
54 | }, 100, { maxWait: 500 });
55 |
56 | return (event: any) => {
57 | const state = getState();
58 | if (!state.recordingInteractions || state.pickingType) {
59 | return;
60 | }
61 |
62 | eventQueue.push(event);
63 | processEvents();
64 | }
65 | }
66 |
```
--------------------------------------------------------------------------------
/src/components/ui/tabs.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import * as React from "react"
2 | import * as TabsPrimitive from "@radix-ui/react-tabs"
3 |
4 | import { cn } from "@/lib/utils"
5 |
6 | function Tabs({
7 | className,
8 | ...props
9 | }: React.ComponentProps<typeof TabsPrimitive.Root>) {
10 | return (
11 | <TabsPrimitive.Root
12 | data-slot="tabs"
13 | className={cn("flex flex-col gap-2", className)}
14 | {...props}
15 | />
16 | )
17 | }
18 |
19 | function TabsList({
20 | className,
21 | ...props
22 | }: React.ComponentProps<typeof TabsPrimitive.List>) {
23 | return (
24 | <TabsPrimitive.List
25 | data-slot="tabs-list"
26 | className={cn(
27 | "bg-muted text-muted-foreground inline-flex h-9 w-fit items-center justify-center rounded-lg p-[3px]",
28 | className
29 | )}
30 | {...props}
31 | />
32 | )
33 | }
34 |
35 | function TabsTrigger({
36 | className,
37 | ...props
38 | }: React.ComponentProps<typeof TabsPrimitive.Trigger>) {
39 | return (
40 | <TabsPrimitive.Trigger
41 | data-slot="tabs-trigger"
42 | className={cn(
43 | "data-[state=active]:bg-background data-[state=active]:text-foreground focus-visible:border-ring focus-visible:ring-ring/50 focus-visible:outline-ring dark:data-[state=active]:border-input dark:data-[state=active]:bg-input/50 inline-flex h-[calc(100%-1px)] flex-1 items-center justify-center gap-1.5 rounded-md border border-transparent px-2 py-1 text-sm font-medium whitespace-nowrap transition-[color,box-shadow] focus-visible:ring-[3px] focus-visible:outline-1 disabled:pointer-events-none disabled:opacity-50 data-[state=active]:shadow-sm [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
44 | className
45 | )}
46 | {...props}
47 | />
48 | )
49 | }
50 |
51 | function TabsContent({
52 | className,
53 | ...props
54 | }: React.ComponentProps<typeof TabsPrimitive.Content>) {
55 | return (
56 | <TabsPrimitive.Content
57 | data-slot="tabs-content"
58 | className={cn("flex-1 outline-none", className)}
59 | {...props}
60 | />
61 | )
62 | }
63 |
64 | export { Tabs, TabsList, TabsTrigger, TabsContent }
65 |
```
--------------------------------------------------------------------------------
/src/components/ui/card.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import * as React from "react"
2 |
3 | import { cn } from "@/lib/utils"
4 |
5 | function Card({ className, ...props }: React.ComponentProps<"div">) {
6 | return (
7 | <div
8 | data-slot="card"
9 | className={cn(
10 | "bg-card text-card-foreground flex flex-col gap-6 rounded-xl border py-6 shadow-sm",
11 | className
12 | )}
13 | {...props}
14 | />
15 | )
16 | }
17 |
18 | function CardHeader({ className, ...props }: React.ComponentProps<"div">) {
19 | return (
20 | <div
21 | data-slot="card-header"
22 | className={cn(
23 | "@container/card-header grid auto-rows-min grid-rows-[auto_auto] items-start gap-1.5 px-6 has-[data-slot=card-action]:grid-cols-[1fr_auto] [.border-b]:pb-6",
24 | className
25 | )}
26 | {...props}
27 | />
28 | )
29 | }
30 |
31 | function CardTitle({ className, ...props }: React.ComponentProps<"div">) {
32 | return (
33 | <div
34 | data-slot="card-title"
35 | className={cn("leading-none font-semibold", className)}
36 | {...props}
37 | />
38 | )
39 | }
40 |
41 | function CardDescription({ className, ...props }: React.ComponentProps<"div">) {
42 | return (
43 | <div
44 | data-slot="card-description"
45 | className={cn("text-muted-foreground text-sm", className)}
46 | {...props}
47 | />
48 | )
49 | }
50 |
51 | function CardAction({ className, ...props }: React.ComponentProps<"div">) {
52 | return (
53 | <div
54 | data-slot="card-action"
55 | className={cn(
56 | "col-start-2 row-span-2 row-start-1 self-start justify-self-end",
57 | className
58 | )}
59 | {...props}
60 | />
61 | )
62 | }
63 |
64 | function CardContent({ className, ...props }: React.ComponentProps<"div">) {
65 | return (
66 | <div
67 | data-slot="card-content"
68 | className={cn("px-6", className)}
69 | {...props}
70 | />
71 | )
72 | }
73 |
74 | function CardFooter({ className, ...props }: React.ComponentProps<"div">) {
75 | return (
76 | <div
77 | data-slot="card-footer"
78 | className={cn("flex items-center px-6 [.border-t]:pt-6", className)}
79 | {...props}
80 | />
81 | )
82 | }
83 |
84 | export {
85 | Card,
86 | CardHeader,
87 | CardFooter,
88 | CardTitle,
89 | CardAction,
90 | CardDescription,
91 | CardContent,
92 | }
93 |
```
--------------------------------------------------------------------------------
/src/components/ui/button.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import * as React from "react"
2 | import { Slot } from "@radix-ui/react-slot"
3 | import { cva, type VariantProps } from "class-variance-authority"
4 |
5 | import { cn } from "@/lib/utils"
6 |
7 | const buttonVariants = cva(
8 | "inline-flex items-center justify-center gap-2 whitespace-nowrap rounded-md text-sm font-medium transition-all disabled:pointer-events-none disabled:opacity-50 [&_svg]:pointer-events-none [&_svg:not([class*='size-'])]:size-4 shrink-0 [&_svg]:shrink-0 outline-none focus-visible:border-ring focus-visible:ring-ring/50 focus-visible:ring-[3px] aria-invalid:ring-destructive/20 dark:aria-invalid:ring-destructive/40 aria-invalid:border-destructive",
9 | {
10 | variants: {
11 | variant: {
12 | default:
13 | "bg-primary text-primary-foreground shadow-xs hover:bg-primary/90",
14 | destructive:
15 | "bg-destructive text-white shadow-xs hover:bg-destructive/90 focus-visible:ring-destructive/20 dark:focus-visible:ring-destructive/40 dark:bg-destructive/60",
16 | outline:
17 | "border bg-background shadow-xs hover:bg-accent hover:text-accent-foreground dark:bg-input/30 dark:border-input dark:hover:bg-input/50",
18 | secondary:
19 | "bg-secondary text-secondary-foreground shadow-xs hover:bg-secondary/80",
20 | ghost:
21 | "hover:bg-accent hover:text-accent-foreground dark:hover:bg-accent/50",
22 | link: "text-primary underline-offset-4 hover:underline",
23 | },
24 | size: {
25 | default: "h-9 px-4 py-2 has-[>svg]:px-3",
26 | sm: "h-8 rounded-md gap-1.5 px-3 has-[>svg]:px-2.5",
27 | lg: "h-10 rounded-md px-6 has-[>svg]:px-4",
28 | icon: "size-9",
29 | },
30 | },
31 | defaultVariants: {
32 | variant: "default",
33 | size: "default",
34 | },
35 | }
36 | )
37 |
38 | function Button({
39 | className,
40 | variant,
41 | size,
42 | asChild = false,
43 | ...props
44 | }: React.ComponentProps<"button"> &
45 | VariantProps<typeof buttonVariants> & {
46 | asChild?: boolean
47 | }) {
48 | const Comp = asChild ? Slot : "button"
49 |
50 | return (
51 | <Comp
52 | data-slot="button"
53 | className={cn(buttonVariants({ variant, size, className }))}
54 | {...props}
55 | />
56 | )
57 | }
58 |
59 | export { Button, buttonVariants }
60 |
```
--------------------------------------------------------------------------------
/package.json:
--------------------------------------------------------------------------------
```json
1 | {
2 | "name": "playwright-mcp",
3 | "version": "0.0.11",
4 | "description": "Playwright integration for ModelContext",
5 | "type": "module",
6 | "main": "dist/server.js",
7 | "bin": {
8 | "playwright-mcp": "dist/server.js"
9 | },
10 | "scripts": {
11 | "dev": "concurrently \"npm run dev:ui\" \"npm run dev:lib\"",
12 | "dev:ui": "vite --config vite.config.js",
13 | "dev:lib": "tsup --watch",
14 | "build": "NODE_ENV=production tsup && vite build",
15 | "lint": "eslint .",
16 | "preview": "vite preview"
17 | },
18 | "dependencies": {
19 | "@modelcontextprotocol/sdk": "^1.7.0",
20 | "@radix-ui/react-dropdown-menu": "^2.1.6",
21 | "@radix-ui/react-scroll-area": "^1.2.3",
22 | "@radix-ui/react-slot": "^1.1.2",
23 | "@radix-ui/react-tabs": "^1.1.3",
24 | "@skorotkiewicz/snowflake-id": "^1.0.1",
25 | "@tailwindcss/vite": "^4.0.14",
26 | "class-variance-authority": "^0.7.1",
27 | "clsx": "^2.1.1",
28 | "happy-dom": "^17.4.4",
29 | "lodash": "^4.17.21",
30 | "lucide-react": "^0.482.0",
31 | "playwright": "^1.51.0",
32 | "prismjs": "^1.30.0",
33 | "react": "^19.0.0",
34 | "react-contenteditable": "^3.3.7",
35 | "react-dom": "^19.0.0",
36 | "react-simple-code-editor": "^0.14.1",
37 | "react-usestateref": "^1.0.9",
38 | "tailwind-merge": "^3.0.2",
39 | "tailwindcss": "^4.0.14",
40 | "tailwindcss-animate": "^1.0.7",
41 | "zod": "^3.24.2"
42 | },
43 | "devDependencies": {
44 | "@eslint/js": "^9.21.0",
45 | "@types/jsdom": "^21.1.7",
46 | "@types/lodash": "^4.17.16",
47 | "@types/node": "^22.13.10",
48 | "@types/prismjs": "^1.26.5",
49 | "@types/react": "^19.0.10",
50 | "@types/react-dom": "^19.0.4",
51 | "@vitejs/plugin-react": "^4.3.4",
52 | "concurrently": "^9.1.2",
53 | "eslint": "^9.21.0",
54 | "eslint-plugin-react-hooks": "^5.1.0",
55 | "eslint-plugin-react-refresh": "^0.4.19",
56 | "globals": "^15.15.0",
57 | "tsup": "^8.4.0",
58 | "typescript": "~5.7.2",
59 | "typescript-eslint": "^8.24.1",
60 | "vite": "^6.2.0"
61 | },
62 | "keywords": [
63 | "playwright",
64 | "mcp",
65 | "modelcontextprotocol",
66 | "test automation",
67 | "playwright-mcp"
68 | ],
69 | "repository": {
70 | "type": "git",
71 | "url": "git+https://github.com/Ashish-Bansal/playwright-mcp.git"
72 | },
73 | "author": "Ashish Bansal",
74 | "publishConfig": {
75 | "access": "public"
76 | }
77 | }
78 |
```
--------------------------------------------------------------------------------
/src/mcp/eval.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { Page } from 'playwright';
2 | import vm from 'vm';
3 |
4 |
5 | // Not super secure, but it's ok for now
6 | export const secureEvalAsync = async (page: Page, code: string, context = {}) => {
7 | // Set default options
8 | const timeout = 20000;
9 | const filename = 'eval.js';
10 |
11 | let logs: string[] = [];
12 | let errors: string[] = [];
13 |
14 | // Code should already be a function declaration
15 | // Just need to execute it with page argument
16 | const wrappedCode = `
17 | ${code}
18 | run(page);
19 | `;
20 |
21 | // Create restricted sandbox with provided context
22 | const sandbox = {
23 | // Core async essentials
24 | Promise,
25 | setTimeout,
26 | clearTimeout,
27 | setImmediate,
28 | clearImmediate,
29 |
30 | // Pass page object to sandbox
31 | page,
32 |
33 | // Capture all console methods
34 | console: {
35 | log: (...args: any[]) => {
36 | const msg = args.map(arg => String(arg)).join(' ');
37 | logs.push(`[log] ${msg}`);
38 | },
39 | error: (...args: any[]) => {
40 | const msg = args.map(arg => String(arg)).join(' ');
41 | errors.push(`[error] ${msg}`);
42 | },
43 | warn: (...args: any[]) => {
44 | const msg = args.map(arg => String(arg)).join(' ');
45 | logs.push(`[warn] ${msg}`);
46 | },
47 | info: (...args: any[]) => {
48 | const msg = args.map(arg => String(arg)).join(' ');
49 | logs.push(`[info] ${msg}`);
50 | },
51 | debug: (...args: any[]) => {
52 | const msg = args.map(arg => String(arg)).join(' ');
53 | logs.push(`[debug] ${msg}`);
54 | },
55 | trace: (...args: any[]) => {
56 | const msg = args.map(arg => String(arg)).join(' ');
57 | logs.push(`[trace] ${msg}`);
58 | }
59 | },
60 |
61 | // User-provided context
62 | ...context,
63 |
64 | // Explicitly block access to sensitive globals
65 | process: undefined,
66 | global: undefined,
67 | require: undefined,
68 | __dirname: undefined,
69 | __filename: undefined,
70 | Buffer: undefined
71 | };
72 |
73 | try {
74 | // Create context and script
75 | const vmContext = vm.createContext(sandbox);
76 | const script = new vm.Script(wrappedCode, { filename });
77 |
78 | // Execute and await result
79 | const result = script.runInContext(vmContext);
80 | const awaitedResult = await result;
81 |
82 | return {
83 | result: awaitedResult,
84 | logs,
85 | errors
86 | };
87 |
88 | } catch (error: any) {
89 | return {
90 | error: true,
91 | message: error.message,
92 | stack: error.stack,
93 | logs,
94 | errors
95 | };
96 | }
97 | }
98 |
```
--------------------------------------------------------------------------------
/src/mcp/recording/utils.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { BrowserEvent, BrowserEventType } from "./events.js";
2 | import { getSelectors } from "./selector-engine.js";
3 | import { Window } from "happy-dom";
4 |
5 | const parseDom = (html: string) => {
6 | const window = new Window({
7 | settings: {
8 | disableJavaScriptEvaluation: true
9 | }
10 | });
11 | window.document.write(html);
12 | return window.document as unknown as Document;
13 | }
14 |
15 | export const preprocessBrowserEvent = (event: BrowserEvent) => {
16 | if (
17 | event.type === BrowserEventType.Click ||
18 | event.type === BrowserEventType.Input
19 | ) {
20 | const dom = parseDom(event.dom)
21 | event.selectors = getSelectors(dom, event.elementUUID);
22 |
23 | const element = dom.querySelector(`[uuid="${event.elementUUID}"]`)
24 | event.elementName = element ? getElementName(element) : "unknown"
25 | event.elementType = element ? getElementType(element) : "unknown"
26 | // for efficiency, we don't need to preserve it for now
27 | event.dom = ''
28 | }
29 | }
30 |
31 | const extractText = (element: Element): string => {
32 | if (element.childNodes.length === 0) {
33 | return element.textContent?.trim() || ''
34 | }
35 |
36 | const texts = Array.from(element.childNodes).map((node) =>
37 | extractText(node as unknown as Element),
38 | )
39 | return texts
40 | .filter((text) => text.trim().length > 0)
41 | .map((text) => text.trim())
42 | .join('\n')
43 | }
44 |
45 | const extractTextsFromSiblings = (element: Element): string[] => {
46 | const siblings = Array.from(element.parentElement?.childNodes || [])
47 | return siblings
48 | .map((sibling) => extractText(sibling as unknown as Element))
49 | .map((text) => text.trim())
50 | .filter((text) => text.length > 0)
51 | }
52 |
53 | const getElementName = (element: Element) => {
54 | let text = ''
55 | const priorityAttrs = ['aria-label', 'title', 'placeholder', 'name', 'alt']
56 | for (const attr of priorityAttrs) {
57 | if (!text) {
58 | text = element?.getAttribute(attr) || ''
59 | }
60 | }
61 | if (!text) {
62 | text = extractText(element)
63 | }
64 | if (!text) {
65 | text = extractTextsFromSiblings(element).join('\n')
66 | }
67 | if (!text) {
68 | text = "unknown"
69 | }
70 | return text
71 | }
72 |
73 | const getElementType = (element: Element) => {
74 | const tagName = element?.tagName.toLowerCase()
75 | let elementType: 'button' | 'link' | 'input' | 'textarea' | 'element' =
76 | 'element'
77 | if (tagName === 'a') {
78 | elementType = 'link'
79 | } else if (tagName === 'button') {
80 | elementType = 'button'
81 | } else if (tagName === 'textarea') {
82 | elementType = 'textarea'
83 | } else if (tagName === 'input') {
84 | elementType = 'input'
85 | }
86 | return elementType
87 | }
88 |
```
--------------------------------------------------------------------------------
/src/App/execute.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import React, { useState } from 'react';
2 | import { Play } from 'lucide-react';
3 | import Editor from 'react-simple-code-editor';
4 | import { highlight, languages } from 'prismjs';
5 | import 'prismjs/components/prism-clike';
6 | import 'prismjs/components/prism-javascript';
7 | import 'prismjs/themes/prism.css';
8 | import { Button } from '@/components/ui/button';
9 | import { useGlobalState } from '@/hooks/use-global-stage';
10 |
11 | const Execute: React.FC = () => {
12 | const [state, updateState] = useGlobalState();
13 | const [result, setResult] = useState<string>('');
14 | const [error, setError] = useState<string>('');
15 | const [logs, setLogs] = useState<string[]>([]);
16 |
17 | const executeCode = async () => {
18 | try {
19 | const response = await (window as any).executeCode(state.code);
20 | if (response.error) {
21 | setError(response.message);
22 | setResult('');
23 | } else {
24 | setResult(JSON.stringify(response.result));
25 | setLogs(response.logs);
26 | setError('');
27 | }
28 | } catch (err) {
29 | setError(err instanceof Error ? err.message : 'An error occurred');
30 | setResult('');
31 | setLogs([]);
32 | }
33 | };
34 |
35 | return (
36 | <div className="flex-1 flex flex-col">
37 | <div className="p-4 bg-white border-b border-zinc-200">
38 | <Editor
39 | value={state.code}
40 | onValueChange={code => updateState({ ...state, code })}
41 | highlight={code => highlight(code, languages.js, 'javascript')}
42 | padding={10}
43 | className="font-mono mb-4 min-h-[200px] border rounded-md"
44 | style={{
45 | fontFamily: '"Fira code", "Fira Mono", monospace',
46 | fontSize: 14,
47 | }}
48 | />
49 | <Button
50 | onClick={executeCode}
51 | variant="outline"
52 | className="gap-2"
53 | >
54 | <Play size={20} />
55 | <span>Execute</span>
56 | </Button>
57 | </div>
58 | <div className="flex-1 p-4 space-y-4">
59 | {error && (
60 | <div className="bg-red-50 border border-red-200 rounded-md p-4">
61 | <div className="font-medium text-red-800 mb-1">Error</div>
62 | <div className="text-red-600 font-mono text-sm whitespace-pre-wrap">
63 | {error}
64 | </div>
65 | </div>
66 | )}
67 | {result && (
68 | <div className="bg-green-50 border border-green-200 rounded-md p-4">
69 | <div className="font-medium text-green-800 mb-1">Result</div>
70 | <div className="text-green-700 font-mono text-sm whitespace-pre-wrap">
71 | {result}
72 | </div>
73 | </div>
74 | )}
75 | {logs.length > 0 && (
76 | <div className="bg-gray-50 border border-gray-200 rounded-md p-4">
77 | <div className="font-medium text-gray-800 mb-2">Logs</div>
78 | <div className="space-y-1">
79 | {logs.map((log, i) => (
80 | <div key={i} className="font-mono text-sm text-gray-600 flex items-start">
81 | <span className="text-gray-400 mr-2">{`>`}</span>
82 | <span className="whitespace-pre-wrap">{log}</span>
83 | </div>
84 | ))}
85 | </div>
86 | </div>
87 | )}
88 | </div>
89 | </div>
90 | );
91 | };
92 |
93 | export default Execute;
94 |
```
--------------------------------------------------------------------------------
/src/web-server.ts:
--------------------------------------------------------------------------------
```typescript
1 | import http from 'http';
2 | import fs from 'fs';
3 | import path from 'path';
4 | import url, { fileURLToPath } from 'url';
5 | import { dirname } from 'path';
6 | import net from 'net';
7 |
8 | // Get the current file's path
9 | const __filename = fileURLToPath(import.meta.url);
10 | // Get the current directory
11 | const __dirname = dirname(__filename);
12 |
13 | // Define the directory from which to serve files
14 | const SERVE_DIR = path.join(__dirname, 'ui'); // Change 'public' to your desired directory name
15 |
16 | // Helper function to check if port is in use
17 | async function isPortInUse(port: number): Promise<boolean> {
18 | return new Promise((resolve) => {
19 | const tester = net.createServer()
20 | .once('error', () => resolve(true))
21 | .once('listening', () => {
22 | tester.once('close', () => resolve(false));
23 | tester.close();
24 | })
25 | .listen(port);
26 | });
27 | }
28 |
29 | // Create HTTP server
30 | const server = http.createServer((req, res) => {
31 | // Parse URL to get the file path
32 | const parsedUrl = url.parse(req.url || '');
33 | const pathname = parsedUrl.pathname || '/';
34 |
35 | // Resolve to absolute path within SERVE_DIR only
36 | let filePath = path.join(SERVE_DIR, pathname);
37 |
38 | // Security check: ensure the file path is within SERVE_DIR
39 | if (!filePath.startsWith(SERVE_DIR)) {
40 | res.writeHead(403, { 'Content-Type': 'text/plain' });
41 | res.end('403 Forbidden: Access denied');
42 | return;
43 | }
44 |
45 | // If path ends with '/', serve index.html from that directory
46 | if (pathname.endsWith('/')) {
47 | filePath = path.join(filePath, 'index.html');
48 | }
49 |
50 | // Check if file exists
51 | fs.stat(filePath, (err, stats) => {
52 | if (err) {
53 | // File not found
54 | res.writeHead(404, { 'Content-Type': 'text/plain' });
55 | res.end('404 Not Found');
56 | return;
57 | }
58 |
59 | // If it's a directory, attempt to serve index.html
60 | if (stats.isDirectory()) {
61 | filePath = path.join(filePath, 'index.html');
62 | fs.stat(filePath, (err, stats) => {
63 | if (err) {
64 | res.writeHead(404, { 'Content-Type': 'text/plain' });
65 | res.end('404 Not Found');
66 | return;
67 | }
68 | serveFile(filePath, res);
69 | });
70 | } else {
71 | // It's a file, serve it
72 | serveFile(filePath, res);
73 | }
74 | });
75 | });
76 |
77 | // Helper function to serve a file
78 | function serveFile(filePath: string, res: http.ServerResponse): void {
79 | // Get file extension to set correct content type
80 | const ext = path.extname(filePath);
81 | let contentType = 'text/plain';
82 |
83 | switch (ext) {
84 | case '.html':
85 | contentType = 'text/html';
86 | break;
87 | case '.css':
88 | contentType = 'text/css';
89 | break;
90 | case '.js':
91 | contentType = 'application/javascript';
92 | break;
93 | case '.json':
94 | contentType = 'application/json';
95 | break;
96 | case '.png':
97 | contentType = 'image/png';
98 | break;
99 | case '.jpg':
100 | case '.jpeg':
101 | contentType = 'image/jpeg';
102 | break;
103 | case '.gif':
104 | contentType = 'image/gif';
105 | break;
106 | case '.svg':
107 | contentType = 'image/svg+xml';
108 | break;
109 | case '.pdf':
110 | contentType = 'application/pdf';
111 | break;
112 | }
113 |
114 | // Read and serve the file
115 | fs.readFile(filePath, (err, data) => {
116 | if (err) {
117 | res.writeHead(500, { 'Content-Type': 'text/plain' });
118 | res.end('Internal Server Error');
119 | return;
120 | }
121 |
122 | res.writeHead(200, { 'Content-Type': contentType });
123 | res.end(data);
124 | });
125 | }
126 |
127 | export { server as webServer, isPortInUse };
128 |
```
--------------------------------------------------------------------------------
/src/assets/react.svg:
--------------------------------------------------------------------------------
```
1 | <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>
```
--------------------------------------------------------------------------------
/src/index.css:
--------------------------------------------------------------------------------
```css
1 | @import url("https://fonts.googleapis.com/css2?family=Plus+Jakarta+Sans:ital,wght@0,200..800;1,200..800&display=swap");
2 |
3 | @import "tailwindcss";
4 |
5 | @plugin "tailwindcss-animate";
6 |
7 | @custom-variant dark (&:is(.dark *));
8 |
9 |
10 | body {
11 | font-family: "Plus Jakarta Sans", sans-serif;
12 | }
13 |
14 | :root {
15 | --radius: 0.625rem;
16 | --background: oklch(1 0 0);
17 | --foreground: oklch(0.145 0 0);
18 | --card: oklch(1 0 0);
19 | --card-foreground: oklch(0.145 0 0);
20 | --popover: oklch(1 0 0);
21 | --popover-foreground: oklch(0.145 0 0);
22 | --primary: oklch(0.205 0 0);
23 | --primary-foreground: oklch(0.985 0 0);
24 | --secondary: oklch(0.97 0 0);
25 | --secondary-foreground: oklch(0.205 0 0);
26 | --muted: oklch(0.97 0 0);
27 | --muted-foreground: oklch(0.556 0 0);
28 | --accent: oklch(0.97 0 0);
29 | --accent-foreground: oklch(0.205 0 0);
30 | --destructive: oklch(0.577 0.245 27.325);
31 | --border: oklch(0.922 0 0);
32 | --input: oklch(0.922 0 0);
33 | --ring: oklch(0.708 0 0);
34 | --chart-1: oklch(0.646 0.222 41.116);
35 | --chart-2: oklch(0.6 0.118 184.704);
36 | --chart-3: oklch(0.398 0.07 227.392);
37 | --chart-4: oklch(0.828 0.189 84.429);
38 | --chart-5: oklch(0.769 0.188 70.08);
39 | --sidebar: oklch(0.985 0 0);
40 | --sidebar-foreground: oklch(0.145 0 0);
41 | --sidebar-primary: oklch(0.205 0 0);
42 | --sidebar-primary-foreground: oklch(0.985 0 0);
43 | --sidebar-accent: oklch(0.97 0 0);
44 | --sidebar-accent-foreground: oklch(0.205 0 0);
45 | --sidebar-border: oklch(0.922 0 0);
46 | --sidebar-ring: oklch(0.708 0 0);
47 | }
48 |
49 | .dark {
50 | --background: oklch(0.145 0 0);
51 | --foreground: oklch(0.985 0 0);
52 | --card: oklch(0.205 0 0);
53 | --card-foreground: oklch(0.985 0 0);
54 | --popover: oklch(0.205 0 0);
55 | --popover-foreground: oklch(0.985 0 0);
56 | --primary: oklch(0.922 0 0);
57 | --primary-foreground: oklch(0.205 0 0);
58 | --secondary: oklch(0.269 0 0);
59 | --secondary-foreground: oklch(0.985 0 0);
60 | --muted: oklch(0.269 0 0);
61 | --muted-foreground: oklch(0.708 0 0);
62 | --accent: oklch(0.269 0 0);
63 | --accent-foreground: oklch(0.985 0 0);
64 | --destructive: oklch(0.704 0.191 22.216);
65 | --border: oklch(1 0 0 / 10%);
66 | --input: oklch(1 0 0 / 15%);
67 | --ring: oklch(0.556 0 0);
68 | --chart-1: oklch(0.488 0.243 264.376);
69 | --chart-2: oklch(0.696 0.17 162.48);
70 | --chart-3: oklch(0.769 0.188 70.08);
71 | --chart-4: oklch(0.627 0.265 303.9);
72 | --chart-5: oklch(0.645 0.246 16.439);
73 | --sidebar: oklch(0.205 0 0);
74 | --sidebar-foreground: oklch(0.985 0 0);
75 | --sidebar-primary: oklch(0.488 0.243 264.376);
76 | --sidebar-primary-foreground: oklch(0.985 0 0);
77 | --sidebar-accent: oklch(0.269 0 0);
78 | --sidebar-accent-foreground: oklch(0.985 0 0);
79 | --sidebar-border: oklch(1 0 0 / 10%);
80 | --sidebar-ring: oklch(0.556 0 0);
81 | }
82 |
83 | @theme inline {
84 | --radius-sm: calc(var(--radius) - 4px);
85 | --radius-md: calc(var(--radius) - 2px);
86 | --radius-lg: var(--radius);
87 | --radius-xl: calc(var(--radius) + 4px);
88 | --color-background: var(--background);
89 | --color-foreground: var(--foreground);
90 | --color-card: var(--card);
91 | --color-card-foreground: var(--card-foreground);
92 | --color-popover: var(--popover);
93 | --color-popover-foreground: var(--popover-foreground);
94 | --color-primary: var(--primary);
95 | --color-primary-foreground: var(--primary-foreground);
96 | --color-secondary: var(--secondary);
97 | --color-secondary-foreground: var(--secondary-foreground);
98 | --color-muted: var(--muted);
99 | --color-muted-foreground: var(--muted-foreground);
100 | --color-accent: var(--accent);
101 | --color-accent-foreground: var(--accent-foreground);
102 | --color-destructive: var(--destructive);
103 | --color-border: var(--border);
104 | --color-input: var(--input);
105 | --color-ring: var(--ring);
106 | --color-chart-1: var(--chart-1);
107 | --color-chart-2: var(--chart-2);
108 | --color-chart-3: var(--chart-3);
109 | --color-chart-4: var(--chart-4);
110 | --color-chart-5: var(--chart-5);
111 | --color-sidebar: var(--sidebar);
112 | --color-sidebar-foreground: var(--sidebar-foreground);
113 | --color-sidebar-primary: var(--sidebar-primary);
114 | --color-sidebar-primary-foreground: var(--sidebar-primary-foreground);
115 | --color-sidebar-accent: var(--sidebar-accent);
116 | --color-sidebar-accent-foreground: var(--sidebar-accent-foreground);
117 | --color-sidebar-border: var(--sidebar-border);
118 | --color-sidebar-ring: var(--sidebar-ring);
119 | }
120 |
121 | @layer base {
122 | * {
123 | @apply border-border outline-ring/50;
124 | }
125 | body {
126 | @apply bg-background text-foreground;
127 | }
128 | }
129 |
```
--------------------------------------------------------------------------------
/src/mcp/index.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
2 | import { z } from "zod";
3 | import { chromium, BrowserContext, Browser, Page } from "playwright";
4 | import { injectToolbox } from "./toolbox.js";
5 | import { secureEvalAsync } from "./eval.js";
6 | import { initState, getState, updateState, type Message } from "./state.js";
7 | import { initRecording } from "./recording";
8 | import { handleBrowserEvent } from "./handle-browser-event.js";
9 |
10 | let browser: Browser;
11 | let context: BrowserContext;
12 | let page: Page;
13 |
14 |
15 | const server = new McpServer({
16 | name: "playwright",
17 | version: "1.0.0",
18 | });
19 |
20 | server.prompt(
21 | "server-flow",
22 | "Get prompt on how to use this MCP server",
23 | () => {
24 | return {
25 | messages: [
26 | {
27 | role: "user",
28 | content: {
29 | type: "text",
30 | text: `# DON'T ASSUME ANYTHING. Whatever you write in code, it must be found in the context. Otherwise leave comments.
31 |
32 | ## Goal
33 | Help me write playwright code with following functionalities:
34 | - [[add semi-high level functionality you want here]]
35 | - [[more]]
36 | - [[more]]
37 | - [[more]]
38 |
39 | ## Reference
40 | - Use @x, @y files if you want to take reference on how I write POM code
41 |
42 | ## Steps
43 | - First fetch the context from 'get-context' tool, until it returns no elements remaining
44 | - Based on context and user functionality, write code in POM format, encapsulating high level functionality into reusable functions
45 | - Try executing code using 'execute-code' tool. You could be on any page, so make sure to navigate to the correct page
46 | - Write spec file using those reusable functions, covering multiple scenarios
47 | `
48 | }
49 | }
50 | ]
51 | };
52 | }
53 | );
54 |
55 |
56 | server.tool(
57 | 'init-browser',
58 | 'Initialize a browser with a URL',
59 | {
60 | url: z.string().url().describe('The URL to navigate to')
61 | },
62 | async ({ url }) => {
63 | if (context) {
64 | await context.close();
65 | }
66 | if (browser) {
67 | await browser.close();
68 | }
69 |
70 | browser = await chromium.launch({
71 | headless: false,
72 | });
73 | context = await browser.newContext({
74 | viewport: null,
75 | userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36',
76 | bypassCSP: true,
77 | });
78 | page = await context.newPage();
79 |
80 | await page.exposeFunction('triggerMcpStartPicking', (pickingType: 'DOM' | 'Image') => {
81 | page.evaluate((pickingType: 'DOM' | 'Image') => {
82 | window.mcpStartPicking(pickingType);
83 | }, pickingType);
84 | });
85 |
86 | await page.exposeFunction('triggerMcpStopPicking', () => {
87 | page.evaluate(() => {
88 | window.mcpStopPicking();
89 | });
90 | });
91 |
92 | await page.exposeFunction('onElementPicked', (message: Message) => {
93 | const state = getState();
94 | state.messages.push(message);
95 | state.pickingType = null;
96 | updateState(page, state);
97 | });
98 |
99 | await page.exposeFunction('takeScreenshot', async (selector: string) => {
100 | try {
101 | const screenshot = await page.locator(selector).screenshot({
102 | timeout: 5000
103 | });
104 | return screenshot.toString('base64');
105 | } catch (error) {
106 | console.error('Error taking screenshot', error);
107 | return null;
108 | }
109 | });
110 |
111 | await page.exposeFunction('executeCode', async (code: string) => {
112 | const result = await secureEvalAsync(page, code);
113 | return result;
114 | });
115 |
116 | await initState(page);
117 | await initRecording(page, handleBrowserEvent(page));
118 |
119 | await page.addInitScript(injectToolbox);
120 | await page.goto(url);
121 |
122 | return {
123 | content: [
124 | {
125 | type: "text",
126 | text: `Browser has been initialized and navigated to ${url}`,
127 | },
128 | ],
129 | };
130 | }
131 | )
132 |
133 | server.tool(
134 | "get-full-dom",
135 | "Get the full DOM of the current page. (Deprecated, use get-context instead)",
136 | {},
137 | async () => {
138 | const html = await page.content();
139 | return {
140 | content: [
141 | {
142 | type: "text",
143 | text: html,
144 | },
145 | ],
146 | };
147 | }
148 | );
149 |
150 | server.tool(
151 | 'get-screenshot',
152 | 'Get a screenshot of the current page',
153 | {},
154 | async () => {
155 | const screenshot = await page.screenshot({
156 | type: "png",
157 | });
158 | return {
159 | content: [
160 | {
161 | type: "image",
162 | data: screenshot.toString('base64'),
163 | mimeType: "image/png",
164 | },
165 | ],
166 | };
167 | }
168 | )
169 |
170 | server.tool(
171 | 'execute-code',
172 | 'Execute custom Playwright JS code against the current page',
173 | {
174 | code: z.string().describe(`The Playwright code to execute. Must be an async function declaration that takes a page parameter.
175 |
176 | Example:
177 | async function run(page) {
178 | console.log(await page.title());
179 | return await page.title();
180 | }
181 |
182 | Returns an object with:
183 | - result: The return value from your function
184 | - logs: Array of console logs from execution
185 | - errors: Array of any errors encountered
186 |
187 | Example response:
188 | {"result": "Google", "logs": ["[log] Google"], "errors": []}`)
189 | },
190 | async ({ code }) => {
191 | const result = await secureEvalAsync(page, code);
192 | return {
193 | content: [
194 | {
195 | type: "text",
196 | text: JSON.stringify(result, null, 2) // Pretty print the JSON
197 | }
198 | ]
199 | };
200 | }
201 | )
202 |
203 | server.tool(
204 | "get-context",
205 | "Get the website context which would be used to write the testcase",
206 | {},
207 | async () => {
208 | const state = getState();
209 |
210 | if (state.messages.length === 0) {
211 | return {
212 | content: [
213 | {
214 | type: "text",
215 | text: `No messages available`
216 | }
217 | ]
218 | };
219 | }
220 |
221 | const content: any = [];
222 |
223 | let totalLength = 0;
224 | let messagesProcessed = 0;
225 |
226 | while (messagesProcessed < state.messages.length && totalLength < 20000) {
227 | const message = state.messages[messagesProcessed];
228 | let currentContent = message.content
229 | if (message.type === 'DOM') {
230 | currentContent = `DOM: ${message.content}`;
231 | } else if (message.type === 'Text') {
232 | currentContent = `Text: ${message.content}`;
233 | } else if (message.type === 'Interaction') {
234 | const interaction = JSON.parse(message.content);
235 | delete interaction.eventId;
236 | delete interaction.dom;
237 | delete interaction.elementUUID;
238 | if (interaction.selectors) {
239 | interaction.selectors = interaction.selectors.slice(0, 10);
240 | }
241 |
242 | currentContent = JSON.stringify(interaction);
243 | } else if (message.type === 'Image') {
244 | currentContent = message.content;
245 | }
246 |
247 | totalLength += currentContent.length;
248 |
249 | const item: any = {}
250 | const isImage = message.type === 'Image';
251 | if (isImage) {
252 | item.type = "image";
253 | item.data = message.content;
254 | item.mimeType = "image/png";
255 | } else {
256 | item.type = "text";
257 | item.text = currentContent;
258 | }
259 | content.push(item);
260 | messagesProcessed++;
261 | }
262 |
263 | // Remove processed messages
264 | state.messages.splice(0, messagesProcessed);
265 | updateState(page, state);
266 |
267 | const remainingCount = state.messages.length;
268 | if (remainingCount > 0) {
269 | content.push({
270 | type: "text",
271 | text: `Remaining ${remainingCount} messages, please fetch those in next requests.\n`
272 | });
273 | }
274 |
275 | return {
276 | content
277 | };
278 | }
279 | );
280 |
281 | export { server }
282 |
```
--------------------------------------------------------------------------------
/src/components/ui/dropdown-menu.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import * as React from "react"
2 | import * as DropdownMenuPrimitive from "@radix-ui/react-dropdown-menu"
3 | import { CheckIcon, ChevronRightIcon, CircleIcon } from "lucide-react"
4 |
5 | import { cn } from "@/lib/utils"
6 |
7 | function DropdownMenu({
8 | ...props
9 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Root>) {
10 | return <DropdownMenuPrimitive.Root data-slot="dropdown-menu" {...props} />
11 | }
12 |
13 | function DropdownMenuPortal({
14 | ...props
15 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Portal>) {
16 | return (
17 | <DropdownMenuPrimitive.Portal data-slot="dropdown-menu-portal" {...props} />
18 | )
19 | }
20 |
21 | function DropdownMenuTrigger({
22 | ...props
23 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Trigger>) {
24 | return (
25 | <DropdownMenuPrimitive.Trigger
26 | data-slot="dropdown-menu-trigger"
27 | {...props}
28 | />
29 | )
30 | }
31 |
32 | function DropdownMenuContent({
33 | className,
34 | sideOffset = 4,
35 | ...props
36 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Content>) {
37 | return (
38 | <DropdownMenuPrimitive.Portal>
39 | <DropdownMenuPrimitive.Content
40 | data-slot="dropdown-menu-content"
41 | sideOffset={sideOffset}
42 | className={cn(
43 | "bg-popover text-popover-foreground data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 z-50 max-h-(--radix-dropdown-menu-content-available-height) min-w-[8rem] origin-(--radix-dropdown-menu-content-transform-origin) overflow-x-hidden overflow-y-auto rounded-md border p-1 shadow-md",
44 | className
45 | )}
46 | {...props}
47 | />
48 | </DropdownMenuPrimitive.Portal>
49 | )
50 | }
51 |
52 | function DropdownMenuGroup({
53 | ...props
54 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Group>) {
55 | return (
56 | <DropdownMenuPrimitive.Group data-slot="dropdown-menu-group" {...props} />
57 | )
58 | }
59 |
60 | function DropdownMenuItem({
61 | className,
62 | inset,
63 | variant = "default",
64 | ...props
65 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Item> & {
66 | inset?: boolean
67 | variant?: "default" | "destructive"
68 | }) {
69 | return (
70 | <DropdownMenuPrimitive.Item
71 | data-slot="dropdown-menu-item"
72 | data-inset={inset}
73 | data-variant={variant}
74 | className={cn(
75 | "focus:bg-accent focus:text-accent-foreground data-[variant=destructive]:text-destructive data-[variant=destructive]:focus:bg-destructive/10 dark:data-[variant=destructive]:focus:bg-destructive/20 data-[variant=destructive]:focus:text-destructive data-[variant=destructive]:*:[svg]:!text-destructive [&_svg:not([class*='text-'])]:text-muted-foreground relative flex cursor-default items-center gap-2 rounded-sm px-2 py-1.5 text-sm outline-hidden select-none data-[disabled]:pointer-events-none data-[disabled]:opacity-50 data-[inset]:pl-8 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
76 | className
77 | )}
78 | {...props}
79 | />
80 | )
81 | }
82 |
83 | function DropdownMenuCheckboxItem({
84 | className,
85 | children,
86 | checked,
87 | ...props
88 | }: React.ComponentProps<typeof DropdownMenuPrimitive.CheckboxItem>) {
89 | return (
90 | <DropdownMenuPrimitive.CheckboxItem
91 | data-slot="dropdown-menu-checkbox-item"
92 | className={cn(
93 | "focus:bg-accent focus:text-accent-foreground relative flex cursor-default items-center gap-2 rounded-sm py-1.5 pr-2 pl-8 text-sm outline-hidden select-none data-[disabled]:pointer-events-none data-[disabled]:opacity-50 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
94 | className
95 | )}
96 | checked={checked}
97 | {...props}
98 | >
99 | <span className="pointer-events-none absolute left-2 flex size-3.5 items-center justify-center">
100 | <DropdownMenuPrimitive.ItemIndicator>
101 | <CheckIcon className="size-4" />
102 | </DropdownMenuPrimitive.ItemIndicator>
103 | </span>
104 | {children}
105 | </DropdownMenuPrimitive.CheckboxItem>
106 | )
107 | }
108 |
109 | function DropdownMenuRadioGroup({
110 | ...props
111 | }: React.ComponentProps<typeof DropdownMenuPrimitive.RadioGroup>) {
112 | return (
113 | <DropdownMenuPrimitive.RadioGroup
114 | data-slot="dropdown-menu-radio-group"
115 | {...props}
116 | />
117 | )
118 | }
119 |
120 | function DropdownMenuRadioItem({
121 | className,
122 | children,
123 | ...props
124 | }: React.ComponentProps<typeof DropdownMenuPrimitive.RadioItem>) {
125 | return (
126 | <DropdownMenuPrimitive.RadioItem
127 | data-slot="dropdown-menu-radio-item"
128 | className={cn(
129 | "focus:bg-accent focus:text-accent-foreground relative flex cursor-default items-center gap-2 rounded-sm py-1.5 pr-2 pl-8 text-sm outline-hidden select-none data-[disabled]:pointer-events-none data-[disabled]:opacity-50 [&_svg]:pointer-events-none [&_svg]:shrink-0 [&_svg:not([class*='size-'])]:size-4",
130 | className
131 | )}
132 | {...props}
133 | >
134 | <span className="pointer-events-none absolute left-2 flex size-3.5 items-center justify-center">
135 | <DropdownMenuPrimitive.ItemIndicator>
136 | <CircleIcon className="size-2 fill-current" />
137 | </DropdownMenuPrimitive.ItemIndicator>
138 | </span>
139 | {children}
140 | </DropdownMenuPrimitive.RadioItem>
141 | )
142 | }
143 |
144 | function DropdownMenuLabel({
145 | className,
146 | inset,
147 | ...props
148 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Label> & {
149 | inset?: boolean
150 | }) {
151 | return (
152 | <DropdownMenuPrimitive.Label
153 | data-slot="dropdown-menu-label"
154 | data-inset={inset}
155 | className={cn(
156 | "px-2 py-1.5 text-sm font-medium data-[inset]:pl-8",
157 | className
158 | )}
159 | {...props}
160 | />
161 | )
162 | }
163 |
164 | function DropdownMenuSeparator({
165 | className,
166 | ...props
167 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Separator>) {
168 | return (
169 | <DropdownMenuPrimitive.Separator
170 | data-slot="dropdown-menu-separator"
171 | className={cn("bg-border -mx-1 my-1 h-px", className)}
172 | {...props}
173 | />
174 | )
175 | }
176 |
177 | function DropdownMenuShortcut({
178 | className,
179 | ...props
180 | }: React.ComponentProps<"span">) {
181 | return (
182 | <span
183 | data-slot="dropdown-menu-shortcut"
184 | className={cn(
185 | "text-muted-foreground ml-auto text-xs tracking-widest",
186 | className
187 | )}
188 | {...props}
189 | />
190 | )
191 | }
192 |
193 | function DropdownMenuSub({
194 | ...props
195 | }: React.ComponentProps<typeof DropdownMenuPrimitive.Sub>) {
196 | return <DropdownMenuPrimitive.Sub data-slot="dropdown-menu-sub" {...props} />
197 | }
198 |
199 | function DropdownMenuSubTrigger({
200 | className,
201 | inset,
202 | children,
203 | ...props
204 | }: React.ComponentProps<typeof DropdownMenuPrimitive.SubTrigger> & {
205 | inset?: boolean
206 | }) {
207 | return (
208 | <DropdownMenuPrimitive.SubTrigger
209 | data-slot="dropdown-menu-sub-trigger"
210 | data-inset={inset}
211 | className={cn(
212 | "focus:bg-accent focus:text-accent-foreground data-[state=open]:bg-accent data-[state=open]:text-accent-foreground flex cursor-default items-center rounded-sm px-2 py-1.5 text-sm outline-hidden select-none data-[inset]:pl-8",
213 | className
214 | )}
215 | {...props}
216 | >
217 | {children}
218 | <ChevronRightIcon className="ml-auto size-4" />
219 | </DropdownMenuPrimitive.SubTrigger>
220 | )
221 | }
222 |
223 | function DropdownMenuSubContent({
224 | className,
225 | ...props
226 | }: React.ComponentProps<typeof DropdownMenuPrimitive.SubContent>) {
227 | return (
228 | <DropdownMenuPrimitive.SubContent
229 | data-slot="dropdown-menu-sub-content"
230 | className={cn(
231 | "bg-popover text-popover-foreground data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 z-50 min-w-[8rem] origin-(--radix-dropdown-menu-content-transform-origin) overflow-hidden rounded-md border p-1 shadow-lg",
232 | className
233 | )}
234 | {...props}
235 | />
236 | )
237 | }
238 |
239 | export {
240 | DropdownMenu,
241 | DropdownMenuPortal,
242 | DropdownMenuTrigger,
243 | DropdownMenuContent,
244 | DropdownMenuGroup,
245 | DropdownMenuLabel,
246 | DropdownMenuItem,
247 | DropdownMenuCheckboxItem,
248 | DropdownMenuRadioGroup,
249 | DropdownMenuRadioItem,
250 | DropdownMenuSeparator,
251 | DropdownMenuShortcut,
252 | DropdownMenuSub,
253 | DropdownMenuSubTrigger,
254 | DropdownMenuSubContent,
255 | }
256 |
```
--------------------------------------------------------------------------------
/src/mcp/recording/selector-engine.ts:
--------------------------------------------------------------------------------
```typescript
1 | import { logger } from '../logger'
2 |
3 | const ATTR_PRIORITIES: Record<string, number> = {
4 | id: 1,
5 | 'data-testid': 2,
6 | 'data-test-id': 2,
7 | 'data-pw': 2,
8 | 'data-cy': 2,
9 | 'data-id': 2,
10 | 'data-name': 3,
11 | name: 3,
12 | 'aria-label': 3,
13 | title: 3,
14 | placeholder: 4,
15 | href: 4,
16 | alt: 4,
17 | 'data-index': 5,
18 | 'data-role': 5,
19 | role: 5,
20 | }
21 |
22 | const IMPORTANT_ATTRS = Object.keys(ATTR_PRIORITIES)
23 |
24 | const _escapeSpecialCharacters = (str: string): string => {
25 | // Only escape double quotes for CSS selectors
26 | return str.replace(/"/g, '\\"')
27 | }
28 |
29 | const getNodeSimpleSelectors = (element: Element): string[] => {
30 | const selectors: string[] = []
31 | const tag = element.tagName.toLowerCase()
32 |
33 | const attrSelectors = IMPORTANT_ATTRS.map((attr) => {
34 | const value = element.getAttribute(attr)
35 | if (!value) return null
36 | return {
37 | priority: ATTR_PRIORITIES[attr] || 999,
38 | selector:
39 | attr === 'id'
40 | ? `#${_escapeSpecialCharacters(value)}`
41 | : `${tag}[${attr}="${_escapeSpecialCharacters(value)}"]`,
42 | }
43 | }).filter((item) => item !== null)
44 |
45 | const otherSelectors = []
46 |
47 | // Locate by class
48 | const classList = element.classList
49 | if (classList.length > 0) {
50 | otherSelectors.push({
51 | priority: 100,
52 | selector: `${tag}.${Array.from(classList).join('.')}`,
53 | })
54 | }
55 |
56 | const availableSelectors = [...attrSelectors, ...otherSelectors]
57 | availableSelectors.sort((a, b) => a!.priority - b!.priority)
58 |
59 | // Take top 5 selectors based on priority
60 | const topSelectors = availableSelectors.slice(0, 5)
61 | topSelectors.push({
62 | priority: 999,
63 | selector: tag,
64 | })
65 |
66 | // Add selectors in priority order
67 | for (const item of topSelectors) {
68 | selectors.push(item!.selector)
69 | }
70 |
71 | return selectors
72 | }
73 |
74 | const _getSiblingRelationshipSelectors = (dom: Document, element: Element): string[] => {
75 | const selectors: string[] = []
76 | const parent = element.parentElement
77 | if (!parent || parent.tagName === 'BODY') {
78 | return selectors
79 | }
80 |
81 | const siblings = Array.from(parent.children)
82 | const elementIndex = siblings.indexOf(element)
83 | const tagName = element.tagName.toLowerCase()
84 |
85 | const selectorPrefixes: string[] = []
86 | for (let i = 0; i < siblings.length; i++) {
87 | if (i === elementIndex) continue
88 |
89 | const sibling = siblings[i]
90 | const siblingSimpleSelectors = getNodeSimpleSelectors(sibling)
91 | siblingSimpleSelectors.forEach((siblingSelector) => {
92 | selectorPrefixes.push(`${siblingSelector} ~ `)
93 | })
94 | }
95 |
96 | const selectorSuffixes = [tagName, ...getNodeSimpleSelectors(element)]
97 | selectorSuffixes.forEach((selectorSuffix) => {
98 | selectorPrefixes.forEach((selectorPrefix) => {
99 | selectors.push(`${selectorPrefix}${selectorSuffix}`)
100 | })
101 | })
102 |
103 | return selectors
104 | }
105 |
106 | const _getChildRelationshipSelectors = (dom: Document, element: Element) => {
107 | // BFS to get all children and their depth upto level 3
108 | const children = []
109 | const currentQueue = Array.from(element.children).map((child) => ({
110 | child,
111 | depth: 0,
112 | }))
113 | while (currentQueue.length > 0) {
114 | const item = currentQueue.shift()
115 | if (!item) continue
116 |
117 | const { child, depth } = item
118 | if (depth > 3) {
119 | continue
120 | }
121 |
122 | children.push({ child, depth })
123 | currentQueue.push(
124 | ...Array.from(child.children).map((child) => ({
125 | child,
126 | depth: depth + 1,
127 | })),
128 | )
129 | }
130 |
131 | const selectorSuffixes: string[] = []
132 | children.forEach(({ child, depth }) => {
133 | const childSelectors = getNodeSimpleSelectors(child)
134 | const childIndex = Array.from(element.children).indexOf(child) + 1
135 |
136 | childSelectors.forEach((childSelector) => {
137 | if (depth === 0) {
138 | // For now, disable `>` immediate child selector, since that doesn't work properly.
139 | // In happy-dom, it's not supported - https://github.com/capricorn86/happy-dom/issues/1642
140 | // In jsdom, it's giving DOM exception
141 | // Example - Failed to validate selector
142 | // div:has(> [data-testid="adult_count"])
143 | // DOMException {}
144 | // message = 'div.`makeFlex >[data-testid="adult_count"]' is not a valid selector
145 | // code = 12
146 | // Selector for parent element, using :has() to indicate parent contains this specific child
147 | selectorSuffixes.push(`:has(${childSelector})`)
148 | // Also add nth-child variant for more specificity if needed
149 | selectorSuffixes.push(`:has(${childSelector}:nth-child(${childIndex}))`)
150 | } else {
151 | // Depth != 0, means it's a descendant, not direct child.
152 | selectorSuffixes.push(`:has(${childSelector})`)
153 | }
154 | })
155 | })
156 |
157 | const selectorPrefixes = [
158 | element.tagName.toLowerCase(),
159 | ...getNodeSimpleSelectors(element),
160 | ]
161 |
162 | const selectors: string[] = []
163 | selectorPrefixes.forEach((selectorPrefix) => {
164 | selectorSuffixes.forEach((selectorSuffix) => {
165 | selectors.push(`${selectorPrefix}${selectorSuffix}`)
166 | })
167 | })
168 | return selectors
169 | }
170 |
171 | const getMatchCount = (dom: Document, selector: string): number => {
172 | try {
173 | return dom.querySelectorAll(selector).length
174 | } catch {
175 | return Number.POSITIVE_INFINITY // Invalid selector
176 | }
177 | }
178 |
179 | const _getParentPathSelectors = (dom: Document, element: Element): string[] => {
180 | // Build path from target to root
181 | const path: Element[] = []
182 | let current: Element | null = element
183 | while (current && current.tagName !== 'HTML') {
184 | path.push(current)
185 | current = current.parentElement
186 | }
187 |
188 | logger.debug(
189 | 'Path',
190 | path.map((node) => node.tagName),
191 | )
192 |
193 | // Pre-compute selectors for each node
194 | const nodeSelectors: {
195 | node: Element
196 | selectors: string[]
197 | }[] = path.map((node) => ({
198 | node,
199 | selectors: getNodeSimpleSelectors(node),
200 | }))
201 | if (!nodeSelectors.length) {
202 | return []
203 | }
204 |
205 | const result: string[] = []
206 | const targetNode = nodeSelectors[0].node
207 | const targetSelectors = nodeSelectors[0].selectors
208 | const targetSelectorsWithNthChild = targetSelectors.map((selector) => {
209 | const index =
210 | targetNode.parentElement
211 | ? Array.from(targetNode.parentElement.children).indexOf(targetNode) + 1
212 | : 1
213 | return `${selector}:nth-child(${index})`
214 | })
215 | const allTargetSelectors = [
216 | ...targetSelectors,
217 | ...targetSelectorsWithNthChild,
218 | ]
219 | logger.debug('Target Selectors', allTargetSelectors)
220 |
221 | for (const targetSelector of allTargetSelectors) {
222 | const matches = getMatchCount(dom, targetSelector)
223 |
224 | // Skip invalid selectors
225 | if (matches === 0) continue
226 |
227 | // If unique, add to results
228 | if (matches === 1) {
229 | result.push(targetSelector)
230 | }
231 |
232 | // Try combinations with ancestors
233 | let currentSelector = targetSelector
234 | let currentMatches = matches
235 | let lastAddedNode = targetNode
236 |
237 | for (let i = 1; i < nodeSelectors.length; i++) {
238 | const ancestor = nodeSelectors[i].node
239 | const ancestorSelectors = nodeSelectors[i].selectors
240 | let bestSelector: string | null = null
241 | let bestMatches = currentMatches
242 |
243 | for (const ancestorSelector of ancestorSelectors) {
244 | const descendantOperator =
245 | Array.from(ancestor.children).indexOf(lastAddedNode) !== -1
246 | ? ' > '
247 | : ' '
248 | const possibleCombinedSelectors = [
249 | `${ancestorSelector} ${descendantOperator} ${currentSelector}`,
250 | ]
251 | if (ancestor.tagName != 'BODY' && ancestor.parentElement) {
252 | const elementIndex =
253 | Array.from(ancestor.parentElement.children).indexOf(ancestor) + 1
254 | possibleCombinedSelectors.push(
255 | `${ancestorSelector}:nth-child(${elementIndex}) ${descendantOperator} ${currentSelector}`,
256 | )
257 | }
258 |
259 | logger.debug('Possible Combined Selectors', possibleCombinedSelectors)
260 |
261 | for (const combinedSelector of possibleCombinedSelectors) {
262 | const newMatches = getMatchCount(dom, combinedSelector)
263 |
264 | // Skip invalid combinations
265 | if (newMatches === 0) continue
266 | else if (newMatches === 1) {
267 | // If unique, add to results immediately
268 | result.push(combinedSelector)
269 | bestSelector = null // Skip updating current selector
270 | } else if (newMatches < bestMatches) {
271 | // Update best if it reduces matches
272 | bestSelector = combinedSelector
273 | bestMatches = newMatches
274 | }
275 | }
276 | }
277 |
278 | // Update current if we found a better (but not unique) selector
279 | if (bestSelector && bestMatches < currentMatches) {
280 | currentSelector = bestSelector
281 | currentMatches = bestMatches
282 | lastAddedNode = ancestor
283 | }
284 | }
285 | }
286 |
287 | return result
288 | }
289 |
290 |
291 | const validateSelector = (document: Document, element: Element, selector: string) => {
292 | try {
293 | const selectedElements = document.querySelectorAll(selector)
294 | return selectedElements.length === 1 && selectedElements[0] === element
295 | } catch (e) {
296 | return false
297 | }
298 | }
299 |
300 | const getSelectors = (document: Document, elementUUID: string): string[] => {
301 | const element = document.querySelector(`[uuid="${elementUUID}"]`)
302 | if (!element) {
303 | throw new Error(`Element with UUID ${elementUUID} not found`)
304 | }
305 |
306 | const validSelectors: string[] = []
307 | const selectorGenerators = [
308 | () => _getParentPathSelectors(document, element),
309 | () => _getChildRelationshipSelectors(document, element),
310 | () => _getSiblingRelationshipSelectors(document, element)
311 | ]
312 |
313 | for (const generator of selectorGenerators) {
314 | const selectors = generator()
315 | for (const selector of selectors) {
316 | if (validateSelector(document, element, selector)) {
317 | validSelectors.push(selector)
318 | if (validSelectors.length >= 10) {
319 | return validSelectors
320 | }
321 | }
322 | }
323 | }
324 |
325 | return validSelectors
326 | }
327 |
328 | export { getSelectors }
329 |
```
--------------------------------------------------------------------------------
/src/mcp/recording/init-recording.ts:
--------------------------------------------------------------------------------
```typescript
1 | import {
2 | type BaseBrowserEvent,
3 | BrowserEventType,
4 | type ClickBrowserEvent,
5 | type InputBrowserEvent,
6 | type KeyPressBrowserEvent,
7 | type OpenPageBrowserEvent,
8 | } from './events'
9 | import type { Page } from 'playwright'
10 | import { getSnowflakeId } from './snowflake'
11 |
12 | export const initRecording = async (
13 | page: Page,
14 | onBrowserEvent: (event: BaseBrowserEvent) => void,
15 | ) => {
16 | page.addInitScript(() => {
17 | if (window.self !== window.top) {
18 | return;
19 | }
20 |
21 | function getDom(): string {
22 | const snapshot = document.documentElement.cloneNode(true) as HTMLElement
23 |
24 | // Handle all elements that need state preservation
25 | const originalElements = document.querySelectorAll<HTMLElement>('*')
26 | const clonedElements = snapshot.querySelectorAll<HTMLElement>('*')
27 |
28 | // Restore scroll positions in the clone
29 | originalElements.forEach((originalElement, index) => {
30 | const clonedElement = clonedElements[index]
31 | if (!clonedElement) return
32 |
33 | // Preserve scroll positions as data attributes
34 | if (originalElement.scrollLeft || originalElement.scrollTop) {
35 | if (originalElement.scrollLeft) {
36 | clonedElement.setAttribute(
37 | 'qaby-data-scroll-left',
38 | originalElement.scrollLeft.toString(),
39 | )
40 | }
41 | if (originalElement.scrollTop) {
42 | clonedElement.setAttribute(
43 | 'qaby-data-scroll-top',
44 | originalElement.scrollTop.toString(),
45 | )
46 | }
47 | }
48 |
49 | // Handle form elements
50 | if (
51 | originalElement instanceof HTMLInputElement ||
52 | originalElement instanceof HTMLTextAreaElement ||
53 | originalElement instanceof HTMLSelectElement ||
54 | originalElement.hasAttribute('contenteditable')
55 | ) {
56 | preserveElementState(originalElement, clonedElement)
57 | }
58 | })
59 |
60 | return snapshot.outerHTML
61 | }
62 |
63 | function preserveElementState(
64 | original: HTMLElement,
65 | cloned: HTMLElement,
66 | ): void {
67 | // Handle contenteditable elements
68 | if (original.hasAttribute('contenteditable')) {
69 | // Use innerHTML instead of textContent to preserve formatting
70 | // Escape HTML content before storing as attribute
71 | const escapedHTML = original.innerHTML
72 | .replace(/&/g, '&')
73 | .replace(/"/g, '"')
74 | .replace(/'/g, ''')
75 | .replace(/</g, '<')
76 | .replace(/>/g, '>')
77 | cloned.setAttribute('qaby-data-contenteditable', escapedHTML)
78 | }
79 |
80 | // Handle form elements
81 | if (original instanceof HTMLInputElement) {
82 | preserveInputState(original, cloned as HTMLInputElement)
83 | } else if (original instanceof HTMLTextAreaElement) {
84 | preserveTextAreaState(original, cloned as HTMLTextAreaElement)
85 | } else if (original instanceof HTMLSelectElement) {
86 | preserveSelectState(original, cloned as HTMLSelectElement)
87 | }
88 | }
89 |
90 | function preserveInputState(
91 | original: HTMLInputElement,
92 | cloned: HTMLInputElement,
93 | ): void {
94 | switch (original.type) {
95 | case 'checkbox':
96 | case 'radio':
97 | if (original.checked) {
98 | cloned.setAttribute('checked', '')
99 | } else {
100 | cloned.removeAttribute('checked')
101 | }
102 | if (original.indeterminate) {
103 | cloned.setAttribute('qaby-data-indeterminate', 'true')
104 | }
105 | break
106 | case 'range':
107 | cloned.setAttribute('value', original.value)
108 | break
109 | case 'date':
110 | case 'datetime-local':
111 | case 'month':
112 | case 'time':
113 | case 'week':
114 | if (original.valueAsDate) {
115 | cloned.setAttribute(
116 | 'qaby-data-value-as-date',
117 | original.valueAsDate.toISOString(),
118 | )
119 | cloned.setAttribute('value', original.value)
120 | }
121 | break
122 | default:
123 | // For text, email, password, etc.
124 | cloned.setAttribute('value', original.value)
125 | }
126 | }
127 |
128 | function preserveTextAreaState(
129 | original: HTMLTextAreaElement,
130 | cloned: HTMLTextAreaElement,
131 | ): void {
132 | cloned.innerHTML = original.value
133 | }
134 |
135 | function preserveSelectState(
136 | original: HTMLSelectElement,
137 | cloned: HTMLSelectElement,
138 | ): void {
139 | // First remove any existing selected attributes
140 | cloned.querySelectorAll('option').forEach((option) => {
141 | option.removeAttribute('selected')
142 | })
143 |
144 | if (original.multiple) {
145 | // For multi-select, preserve selected state of each option
146 | Array.from(original.selectedOptions).forEach((option) => {
147 | // Find the corresponding option in cloned select by index
148 | const optionIndex = Array.from(original.options).indexOf(option)
149 | // Use querySelector instead of options property
150 | const clonedOption = cloned.querySelector(
151 | `option:nth-child(${optionIndex + 1})`,
152 | )
153 | if (clonedOption) {
154 | clonedOption.setAttribute('selected', '')
155 | }
156 | })
157 | } else if (original.selectedIndex >= 0) {
158 | const clonedOption = cloned.querySelector(
159 | `option:nth-child(${original.selectedIndex + 1})`,
160 | )
161 | if (clonedOption) {
162 | clonedOption.setAttribute('selected', '')
163 | }
164 | }
165 | }
166 |
167 | function generateUUID(): string {
168 | return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, (c) => {
169 | const r = (Math.random() * 16) | 0
170 | const v = c === 'x' ? r : (r & 0x3) | 0x8
171 | return v.toString(16)
172 | })
173 | }
174 |
175 | function addAttributesToNode(node: Node): void {
176 | if (node.nodeType === window.Node.ELEMENT_NODE) {
177 | const element = node as unknown as Element
178 | if (!element.getAttribute('uuid')) {
179 | element.setAttribute('uuid', generateUUID())
180 | }
181 | for (const child of node.childNodes) {
182 | addAttributesToNode(child)
183 | }
184 | }
185 | }
186 |
187 | function removeAttributesFromNode(node: Node): void {
188 | if (node.nodeType === window.Node.ELEMENT_NODE) {
189 | const element = node as unknown as Element
190 | element.removeAttribute('uuid')
191 | }
192 | }
193 |
194 | // Event handlers
195 | const recordedEvents = new WeakMap<MouseEvent, boolean>();
196 |
197 | function handleClick(e: MouseEvent): void {
198 | // Check if event was already recorded
199 | if (recordedEvents.get(e)) {
200 | return;
201 | }
202 |
203 | const target = e.target as Element;
204 | if (!target) return;
205 | if (target.getAttribute('data-skip-recording')) return;
206 |
207 | e.stopPropagation();
208 | recordedEvents.set(e, true);
209 |
210 | addAttributesToNode(document.documentElement);
211 | const elementUUID = target.getAttribute('uuid');
212 | const dom = getDom();
213 |
214 | window.recordDOM(dom, elementUUID as string).then(() => {
215 | // Re-dispatch the event after recording
216 | target.dispatchEvent(e);
217 | });
218 | removeAttributesFromNode(document.documentElement);
219 | }
220 |
221 | function handleKeyDown(event: KeyboardEvent): void {
222 | const dom = getDom()
223 |
224 | if (['Enter', 'Escape'].includes(event.key)) {
225 | window.recordKeyPress(dom, [event.key])
226 | return
227 | }
228 |
229 | if (document.activeElement?.tagName.toLowerCase() === 'input') {
230 | if (event.key === 'Tab') {
231 | window.recordKeyPress(dom, [event.key])
232 | }
233 | return
234 | }
235 |
236 | if (document.activeElement?.tagName.toLowerCase() === 'textarea') {
237 | return
238 | }
239 |
240 | const keys: string[] = []
241 | if (event.ctrlKey) keys.push('Control')
242 | if (event.shiftKey) keys.push('Shift')
243 | if (event.altKey) keys.push('Alt')
244 | if (event.metaKey) keys.push('Meta')
245 |
246 | if (!['Control', 'Shift', 'Alt', 'Meta'].includes(event.key)) {
247 | keys.push(event.key)
248 | }
249 |
250 | if (keys.includes('Meta') && keys.includes('Tab')) {
251 | return
252 | }
253 |
254 | if (keys.length === 1 && keys[0] === 'Meta') {
255 | return
256 | }
257 |
258 | if (keys.length > 0) {
259 | window.recordKeyPress(dom, keys)
260 | }
261 | }
262 |
263 | function handleKeyUp(event: KeyboardEvent): void {
264 | if (['Enter', 'Escape'].includes(event.key)) {
265 | return;
266 | }
267 |
268 | if (
269 | document.activeElement?.tagName.toLowerCase() !== 'input' &&
270 | document.activeElement?.tagName.toLowerCase() !== 'textarea'
271 | ) {
272 | return
273 | }
274 |
275 | const dom = getDom()
276 | window.recordInput(
277 | dom,
278 | document.activeElement.getAttribute('uuid') as string,
279 | (document.activeElement as HTMLInputElement).value,
280 | )
281 | }
282 |
283 | window.addEventListener('click', handleClick, { capture: true })
284 | window.addEventListener('keydown', handleKeyDown, { capture: true })
285 | window.addEventListener('keyup', handleKeyUp, { capture: true })
286 |
287 | console.log('Recording initialized for window:', window.location.href)
288 | })
289 |
290 | let buttonClicked = false
291 | await page.exposeFunction(
292 | 'recordDOM',
293 | async (dom: string, elementUUID: string) => {
294 | buttonClicked = true
295 | const event: ClickBrowserEvent = {
296 | eventId: await getSnowflakeId(),
297 | type: BrowserEventType.Click,
298 | dom,
299 | elementUUID,
300 | selectors: [`[uuid="${elementUUID}"]`],
301 | windowUrl: page.url(),
302 | }
303 | onBrowserEvent(event)
304 | },
305 | )
306 |
307 | await page.exposeFunction(
308 | 'recordInput',
309 | async (dom: string, elementUUID: string, value: string) => {
310 | const event: InputBrowserEvent = {
311 | eventId: await getSnowflakeId(),
312 | type: BrowserEventType.Input,
313 | dom,
314 | elementUUID,
315 | typedText: value,
316 | selectors: [`[uuid="${elementUUID}"]`],
317 | windowUrl: page.url(),
318 | }
319 | onBrowserEvent(event)
320 | },
321 | )
322 |
323 | await page.exposeFunction(
324 | 'recordKeyPress',
325 | async (dom: string, keys: string[]) => {
326 | const event: KeyPressBrowserEvent = {
327 | eventId: await getSnowflakeId(),
328 | type: BrowserEventType.KeyPress,
329 | keys,
330 | dom,
331 | windowUrl: page.url(),
332 | }
333 | onBrowserEvent(event)
334 | },
335 | )
336 |
337 | page.on('load', async () => {
338 | if (!buttonClicked) {
339 | const event: OpenPageBrowserEvent = {
340 | eventId: await getSnowflakeId(),
341 | type: BrowserEventType.OpenPage,
342 | windowUrl: page.url(),
343 | // TODO: Fix navigation handling
344 | // title: await page.title(),
345 | title: '',
346 | // TODO: Fix dom content here
347 | dom: '',
348 | }
349 | onBrowserEvent(event)
350 | }
351 |
352 | buttonClicked = false
353 | })
354 | }
355 |
```
--------------------------------------------------------------------------------
/src/mcp/toolbox.ts:
--------------------------------------------------------------------------------
```typescript
1 | interface PickingState {
2 | activePickingType: 'DOM' | 'Image' | null;
3 | mouseMoveHandler: ((e: MouseEvent) => void) | null;
4 | clickHandler: ((e: MouseEvent) => void) | null;
5 | }
6 |
7 | export const injectToolbox = () => {
8 | window.addEventListener('DOMContentLoaded', function() {
9 | const inIframe = window.self !== window.top;
10 | if (inIframe) {
11 | return;
12 | }
13 |
14 | // Create sidebar if it doesn't exist
15 | if (document.querySelector('#mcp-sidebar')) {
16 | return;
17 | }
18 |
19 | const pickingState: PickingState = {
20 | activePickingType: null,
21 | mouseMoveHandler: null,
22 | clickHandler: null
23 | };
24 |
25 | const toggleSidebar = (expanded: boolean) => {
26 | const sidebar = document.querySelector('#mcp-sidebar') as HTMLElement;
27 | const toggleButton = document.querySelector('#mcp-sidebar-toggle-button') as HTMLElement;
28 | if (sidebar && toggleButton) {
29 | const width = parseInt(sidebar.style.width);
30 | sidebar.style.transform = expanded ? 'translateX(0)' : `translateX(${width}px)`;
31 | toggleButton.style.right = expanded ? `${width}px` : '0';
32 | toggleButton.textContent = expanded ? '⟩' : '⟨';
33 | localStorage.setItem('mcp-sidebar-expanded', expanded.toString());
34 | }
35 | };
36 |
37 | const mcpStopPicking = () => {
38 | // Stop picking
39 | if (pickingState.mouseMoveHandler) {
40 | document.removeEventListener('mousemove', pickingState.mouseMoveHandler);
41 | }
42 | if (pickingState.clickHandler) {
43 | document.removeEventListener('click', pickingState.clickHandler, true);
44 | }
45 | // Remove preview overlay if it exists
46 | const previewOverlay = document.querySelector('#mcp-highlight-overlay-preview');
47 | if (previewOverlay) {
48 | previewOverlay.remove();
49 | }
50 | pickingState.activePickingType = null;
51 |
52 | // Restore sidebar state
53 | toggleSidebar(true);
54 | };
55 |
56 | const mcpStartPicking = (pickingType: 'DOM' | 'Image') => {
57 | pickingState.activePickingType = pickingType;
58 |
59 | // Collapse sidebar when picking starts
60 | toggleSidebar(false);
61 |
62 | pickingState.mouseMoveHandler = (e: MouseEvent) => {
63 | const element = document.elementFromPoint(e.clientX, e.clientY) as HTMLElement;
64 | const sidebar = document.querySelector('#mcp-sidebar');
65 | const expandButton = document.querySelector('#mcp-sidebar-toggle-button');
66 | if (!element ||
67 | (sidebar && sidebar.contains(element)) ||
68 | (expandButton && expandButton.contains(element)) ||
69 | element.closest('[id^="mcp-highlight-overlay"]')) return;
70 |
71 | // Create or update highlight overlay
72 | let overlay: HTMLElement | null = document.querySelector('#mcp-highlight-overlay-preview');
73 | if (!overlay) {
74 | overlay = document.createElement('div');
75 | overlay.id = 'mcp-highlight-overlay-preview';
76 | overlay.style.cssText = `
77 | position: fixed;
78 | border: 1px dashed #4CAF50;
79 | background: rgba(76, 175, 80, 0.1);
80 | pointer-events: none;
81 | z-index: 999998;
82 | transition: all 0.2s ease;
83 | `;
84 | document.body.appendChild(overlay);
85 | }
86 |
87 | const rect = element.getBoundingClientRect();
88 | overlay.style.top = rect.top + 'px';
89 | overlay.style.left = rect.left + 'px';
90 | overlay.style.width = rect.width + 'px';
91 | overlay.style.height = rect.height + 'px';
92 | };
93 |
94 | pickingState.clickHandler = async (event: MouseEvent) => {
95 | const element = document.elementFromPoint(event.clientX, event.clientY) as HTMLElement;
96 | const sidebar = document.querySelector('#mcp-sidebar');
97 | const expandButton = document.querySelector('#mcp-sidebar-toggle-button');
98 | if (!element ||
99 | (sidebar && sidebar.contains(element)) ||
100 | (expandButton && expandButton.contains(element)) ||
101 | element.closest('[id^="mcp-highlight-overlay"]')) return;
102 |
103 | event.stopPropagation();
104 | event.preventDefault();
105 |
106 | let message: Message;
107 | if (pickingState.activePickingType === 'DOM') {
108 | const html = element.outerHTML;
109 | message = {
110 | type: 'DOM',
111 | content: html,
112 | windowUrl: window.location.href
113 | };
114 | } else {
115 | const previewOverlay = document.querySelector('#mcp-highlight-overlay-preview') as HTMLElement;
116 | if (previewOverlay) {
117 | previewOverlay.style.display = 'none';
118 | }
119 | const screenshotId = `screenshot-${Math.random().toString(36).substring(2)}`;
120 | element.setAttribute('data-screenshot-id', screenshotId);
121 | const screenshot = await (window as any).takeScreenshot(`[data-screenshot-id="${screenshotId}"]`);
122 | element.removeAttribute('data-screenshot-id');
123 | if (previewOverlay) {
124 | previewOverlay.style.display = 'block';
125 | }
126 | message = {
127 | type: 'Image',
128 | content: screenshot,
129 | windowUrl: window.location.href
130 | };
131 | }
132 |
133 | mcpStopPicking();
134 | (window as any).onElementPicked(message);
135 | };
136 |
137 | document.addEventListener('mousemove', pickingState.mouseMoveHandler);
138 | document.addEventListener('click', pickingState.clickHandler, true);
139 | };
140 |
141 | // Expose picking functions to window
142 | window.mcpStartPicking = mcpStartPicking;
143 | window.mcpStopPicking = mcpStopPicking;
144 |
145 | const getSidebarWidth = () => {
146 | const defaultWidth = localStorage.getItem('mcp-sidebar-width') || '500';
147 | return parseInt(defaultWidth);
148 | }
149 |
150 | const createSidebar = () => {
151 | const sidebar = document.createElement('div');
152 | sidebar.id = 'mcp-sidebar';
153 | const defaultWidth = getSidebarWidth();
154 | sidebar.style.cssText = `
155 | position: fixed;
156 | top: 0;
157 | right: 0;
158 | width: ${defaultWidth}px;
159 | height: 100vh;
160 | background: #f5f5f5;
161 | border-left: 1px solid rgb(228, 228, 231);
162 | z-index: 999999;
163 | display: flex;
164 | flex-direction: column;
165 | overflow: hidden;
166 | transition: transform 0.3s ease;
167 | `;
168 |
169 | const iframe = document.createElement('iframe');
170 | iframe.name = 'toolbox-frame';
171 | iframe.src = 'http://localhost:5174/';
172 | iframe.style.cssText = `
173 | width: 100%;
174 | height: 100%;
175 | border: none;
176 | `;
177 |
178 | // Add resize handle
179 | const resizeHandle = document.createElement('div');
180 | resizeHandle.id = 'mcp-resize-handle';
181 | resizeHandle.style.cssText = `
182 | position: absolute;
183 | left: 0;
184 | top: 0;
185 | width: 4px;
186 | height: 100%;
187 | cursor: ew-resize;
188 | background: transparent;
189 | `;
190 |
191 | let isResizing = false;
192 | let lastX = 0;
193 | const originalProperties: Record<string, string> = {}
194 |
195 | // Function to start resize
196 | const startResize = (e: MouseEvent) => {
197 | isResizing = true;
198 | lastX = e.clientX;
199 |
200 | // Add an overlay over the iframe while resizing to prevent mouse events going to the iframe
201 | const overlay = document.createElement('div');
202 | overlay.className = 'resize-overlay';
203 | overlay.style.cssText = 'position:absolute;top:0;left:0;right:0;bottom:0;z-index:1000;';
204 | sidebar.appendChild(overlay);
205 |
206 | // Add resize event listeners to document
207 | document.addEventListener('mousemove', resize);
208 | document.addEventListener('mouseup', stopResize);
209 |
210 | // Disable text selection during resize
211 | originalProperties.bodyUserSelect = document.body.style.userSelect;
212 | document.body.style.userSelect = 'none';
213 | const toggleButton = document.querySelector('#mcp-sidebar-toggle-button') as HTMLElement;
214 | if (toggleButton) {
215 | originalProperties.toggleButtonTransition = toggleButton.style.transition;
216 | toggleButton.style.transition = '';
217 | }
218 | };
219 |
220 | // Function to handle resize
221 | const resize = (e: MouseEvent) => {
222 | if (!isResizing) return;
223 |
224 | const deltaX = lastX - e.clientX;
225 | const newWidth = Math.min(
226 | Math.max(400, sidebar.offsetWidth + deltaX),
227 | window.innerWidth * 0.8
228 | );
229 |
230 | sidebar.style.width = `${newWidth}px`;
231 | const toggleButton = document.querySelector('#mcp-sidebar-toggle-button') as HTMLElement;
232 | if (toggleButton) {
233 | toggleButton.style.right = `${newWidth}px`;
234 | }
235 | localStorage.setItem('mcp-sidebar-width', newWidth.toString());
236 | lastX = e.clientX;
237 | };
238 |
239 | // Function to stop resize
240 | const stopResize = () => {
241 | if (!isResizing) return;
242 | isResizing = false;
243 |
244 | // Remove the overlay
245 | const overlay = sidebar.querySelector('.resize-overlay');
246 | if (overlay) sidebar.removeChild(overlay);
247 |
248 | // Remove event listeners
249 | document.removeEventListener('mousemove', resize);
250 | document.removeEventListener('mouseup', stopResize);
251 |
252 | // Re-enable text selection
253 | document.body.style.userSelect = originalProperties.bodyUserSelect;
254 | const toggleButton = document.querySelector('#mcp-sidebar-toggle-button') as HTMLElement;
255 | if (toggleButton) {
256 | toggleButton.style.transition = originalProperties.toggleButtonTransition;
257 | }
258 | };
259 |
260 | resizeHandle.addEventListener('mousedown', startResize);
261 |
262 | sidebar.appendChild(resizeHandle);
263 | sidebar.appendChild(iframe);
264 | document.body.appendChild(sidebar);
265 | }
266 |
267 | const createSidebarToggleButton = () => {
268 | const toggleButton = document.createElement('button');
269 | toggleButton.id = 'mcp-sidebar-toggle-button';
270 | toggleButton.textContent = '⟩';
271 | toggleButton.setAttribute('data-skip-recording', 'true');
272 | const sidebarWidth = getSidebarWidth();
273 | toggleButton.style.cssText = `
274 | position: fixed;
275 | right: ${sidebarWidth}px;
276 | top: 50%;
277 | transform: translateY(-50%);
278 | background: #f5f5f5;
279 | border: 1px solid rgb(228, 228, 231);
280 | border-right: none;
281 | border-radius: 4px 0 0 4px;
282 | font-size: 20px;
283 | cursor: pointer;
284 | padding: 8px;
285 | color: rgb(17, 24, 39);
286 | z-index: 999999;
287 | transition: right 0.3s ease;
288 | `;
289 | document.body.appendChild(toggleButton);
290 |
291 | let isExpanded = localStorage.getItem('mcp-sidebar-expanded') !== 'false';
292 | if (!isExpanded) {
293 | toggleSidebar(false);
294 | }
295 |
296 | toggleButton.addEventListener('click', () => {
297 | isExpanded = !isExpanded;
298 | toggleSidebar(isExpanded);
299 | });
300 | }
301 |
302 | createSidebar();
303 | createSidebarToggleButton();
304 | });
305 | }
306 |
```
--------------------------------------------------------------------------------
/src/App/context.tsx:
--------------------------------------------------------------------------------
```typescript
1 | import React, { useEffect, useRef } from 'react';
2 | import { Maximize, StopCircle, Image, CircleXIcon, GlobeIcon, KeyboardIcon, MousePointerClickIcon, TextCursorInputIcon, CodeIcon, PlusIcon } from 'lucide-react';
3 | import { useGlobalState } from '@/hooks/use-global-stage';
4 | import { Button } from '@/components/ui/button';
5 | import { ScrollArea } from "@/components/ui/scroll-area";
6 | import { Card, CardContent } from '@/components/ui/card';
7 | import { ClickToEdit } from '@/components/ui/click-to-edit';
8 | import { BrowserEvent, BrowserEventType } from '@/mcp/recording/events';
9 | import { DropdownMenu, DropdownMenuContent, DropdownMenuItem, DropdownMenuTrigger } from '@/components/ui/dropdown-menu';
10 |
11 | interface MessageProps {
12 | message: Message;
13 | onDelete: (content: string) => void;
14 | }
15 |
16 | const truncate = (text: string, maxLength = 25) => {
17 | return text.length > maxLength ? text.slice(0, maxLength) + '...' : text
18 | }
19 |
20 | const MessageCard: React.FC<{
21 | icon: React.ReactNode,
22 | title: React.ReactNode,
23 | content?: React.ReactNode,
24 | onDelete: () => void
25 | }> = ({ icon, title, content, onDelete }) => {
26 | return (
27 | <Card className="group py-4 rounded-sm">
28 | <CardContent className="px-4 flex gap-2 flex-col">
29 | <div className="flex gap-2">
30 | <div className="flex flex-1 gap-2">
31 | <div className="w-5 h-5 flex items-center justify-center">
32 | {icon}
33 | </div>
34 | <div className="text-sm font-medium text-gray-800">{title}</div>
35 | </div>
36 | <div className="">
37 | <CircleXIcon className="w-4 h-4 transition-opacity duration-200 opacity-0 group-hover:opacity-100 hover:text-destructive cursor-pointer" onClick={onDelete} />
38 | </div>
39 | </div>
40 | {content && (
41 | <div className="mt-2">
42 | {content}
43 | </div>
44 | )}
45 | </CardContent>
46 | </Card>
47 | );
48 | };
49 |
50 | const renderInteraction = (message: Message, deleteMessage: () => void) => {
51 | const rawInteraction = JSON.parse(message.content);
52 | const interaction = rawInteraction as BrowserEvent;
53 |
54 | const getIcon = (type: BrowserEventType) => {
55 | switch (type) {
56 | case BrowserEventType.Click:
57 | return <MousePointerClickIcon />;
58 | case BrowserEventType.Input:
59 | return <TextCursorInputIcon />;
60 | case BrowserEventType.KeyPress:
61 | return <KeyboardIcon />;
62 | case BrowserEventType.OpenPage:
63 | return <GlobeIcon />;
64 | default:
65 | return <GlobeIcon />;
66 | }
67 | };
68 |
69 | const getText = (interaction: BrowserEvent) => {
70 | switch (interaction.type) {
71 | case BrowserEventType.Click:
72 | return <>Click on <span className="font-bold text-gray-600 ">"{truncate(interaction.elementName || '')}"</span> {truncate(interaction.elementType || '')}</>;
73 | case BrowserEventType.Input:
74 | return <>Type <span className="font-bold text-gray-600 ">"{truncate(interaction.typedText || '')}"</span> in <span className="font-bold text-gray-600">{truncate(interaction.elementName || '')}</span></>;
75 | case BrowserEventType.KeyPress:
76 | return <>Press <span className="font-bold text-gray-600 ">{interaction.keys.join(' + ')}</span> key{interaction.keys.length > 1 ? 's' : ''}</>;
77 | case BrowserEventType.OpenPage:
78 | return <>Navigate to <span className="font-bold text-gray-600 ">{truncate(interaction.windowUrl || '')}</span></>;
79 | default:
80 | return <>Unknown interaction</>;
81 | }
82 | };
83 |
84 | const selector = 'selectors' in interaction ? interaction.selectors?.[0] : undefined;
85 |
86 | return (
87 | <MessageCard
88 | icon={getIcon(interaction.type)}
89 | title={getText(interaction)}
90 | content={selector && (
91 | <div className="flex flex-col gap-2">
92 | <ClickToEdit className="text-xs text-muted-foreground bg-gray-100 rounded-sm p-1" placeholder="CSS selector e.g. [data-testid='button']" text={selector} onSave={() => { }} />
93 | </div>
94 | )}
95 | onDelete={deleteMessage}
96 | />
97 | );
98 | };
99 |
100 | const renderImage = (message: Message, deleteMessage: () => void) => {
101 | return (
102 | <MessageCard
103 | icon={<Image className="text-gray-600" />}
104 | title="Screenshot captured"
105 | content={
106 | <img
107 | src={`data:image/png;base64,${message.content}`}
108 | className="rounded w-full"
109 | alt="Screenshot"
110 | />
111 | }
112 | onDelete={deleteMessage}
113 | />
114 | );
115 | };
116 |
117 | const renderDom = (message: Message, deleteMessage: () => void) => {
118 | const chars = message.content.length;
119 |
120 | return (
121 | <MessageCard
122 | icon={<CodeIcon className="text-gray-600" />}
123 | title="DOM Element captured"
124 | content={
125 | <div className="bg-gray-50 p-3 rounded">
126 | <div className="font-mono text-xs overflow-x-auto break-all">
127 | {message.content.length > 300 ? message.content.slice(0, 297) + '...' : message.content}
128 | </div>
129 | <div className="text-xs text-muted-foreground mt-2">
130 | {chars} characters
131 | </div>
132 | </div>
133 | }
134 | onDelete={deleteMessage}
135 | />
136 | );
137 | };
138 |
139 | const MessageComponent: React.FC<MessageProps> = ({ message, onDelete }) => {
140 | const deleteMessage = () => onDelete(message.content);
141 |
142 | if (message.type === 'Interaction') {
143 | return renderInteraction(message, deleteMessage);
144 | }
145 |
146 | if (message.type === 'Image') {
147 | return renderImage(message, deleteMessage);
148 | }
149 |
150 | if (message.type === 'DOM') {
151 | return renderDom(message, deleteMessage);
152 | }
153 |
154 | return null;
155 | };
156 |
157 | const Context: React.FC = () => {
158 | const [state, updateState] = useGlobalState();
159 | const messagesContainerRef = useRef<HTMLDivElement>(null);
160 | const prevMessagesLength = useRef(state.messages.length);
161 | const isFirstRender = useRef(true);
162 |
163 | const scrollToBottom = () => {
164 | if (messagesContainerRef.current) {
165 | const scrollArea = messagesContainerRef.current.querySelector('[data-radix-scroll-area-viewport]');
166 | if (scrollArea) {
167 | scrollArea.scrollTo({
168 | top: scrollArea.scrollHeight,
169 | behavior: 'smooth'
170 | });
171 | }
172 | }
173 | };
174 |
175 | useEffect(() => {
176 | if (state.messages.length > prevMessagesLength.current) {
177 | setTimeout(() => {
178 | scrollToBottom();
179 | }, 200);
180 | }
181 | prevMessagesLength.current = state.messages.length;
182 | }, [state.messages.length]);
183 |
184 | useEffect(() => {
185 | if (isFirstRender.current) {
186 | setTimeout(() => {
187 | scrollToBottom();
188 | }, 500);
189 | isFirstRender.current = false;
190 | }
191 | }, []);
192 |
193 | const handleDelete = (content: string) => {
194 | updateState({
195 | ...state,
196 | messages: state.messages.filter((m: Message) => m.content !== content)
197 | });
198 | };
199 |
200 | const stopPicking = () => {
201 | updateState({
202 | ...state,
203 | pickingType: null
204 | });
205 | window.triggerMcpStopPicking();
206 | };
207 |
208 | const startPicking = (type: 'DOM' | 'Image') => {
209 | updateState({
210 | ...state,
211 | pickingType: type
212 | });
213 | window.triggerMcpStartPicking(type);
214 | };
215 |
216 | const toggleRecordingInteractions = () => {
217 | updateState({
218 | ...state,
219 | recordingInteractions: !state.recordingInteractions
220 | });
221 | };
222 |
223 | const messageGroups: Message[][] = []
224 | state.messages.forEach((message: Message) => {
225 | const url = message.windowUrl
226 | const lastMessageGroup = messageGroups.length > 0 ? messageGroups[messageGroups.length - 1] : null
227 | if (!lastMessageGroup || lastMessageGroup[0].windowUrl !== url) {
228 | messageGroups.push([message])
229 | } else {
230 | lastMessageGroup.push(message)
231 | }
232 | });
233 |
234 | const recordingInteractions = state.recordingInteractions;
235 |
236 | return (
237 | <div className="flex-1 flex flex-col h-full bg-white">
238 | <div className="p-4 flex gap-2">
239 | <Button
240 | onClick={toggleRecordingInteractions}
241 | className="w-40"
242 | >
243 | <div className="flex items-center gap-2">
244 | {recordingInteractions ? (
245 | <div className="w-3 h-3 bg-red-500" />
246 | ) : (
247 | <div className="w-4 h-4 rounded-full bg-red-500" />
248 | )}
249 | {recordingInteractions ? 'Stop Recording' : 'Start Recording'}
250 | </div>
251 | </Button>
252 | </div>
253 | <ScrollArea ref={messagesContainerRef} className="flex-1 max-h-[calc(100vh-194px)] overflow-y-auto">
254 | {(messageGroups.length > 0 || recordingInteractions) ? (
255 | <div className="flex flex-col gap-8 p-4">
256 | {messageGroups.map((messageGroup: Message[], index: number) => (
257 | <div key={index} className="flex flex-col gap-4">
258 | <div className="text-sm font-medium text-gray-800 px-1">On page <span className="font-bold text-gray-600 break-all">{truncate(messageGroup[0].windowUrl || '', 120)}</span></div>
259 | <div className="flex flex-col gap-2">
260 | {messageGroup.map((message: Message, index: number) => (
261 | <MessageComponent
262 | key={index}
263 | message={message}
264 | onDelete={handleDelete}
265 | />
266 | ))}
267 | </div>
268 | </div>
269 | ))}
270 | {recordingInteractions && (
271 | <div className="flex items-center gap-1 mb-8">
272 | <div className="mr-2">
273 | <div className="w-5 h-5 rounded-full border border-dotted border-gray-300 flex items-center justify-center">
274 | <div className="w-2 h-2 rounded-full bg-red-500 animate-pulse"></div>
275 | </div>
276 | </div>
277 | <div className="text-gray-700">
278 | Recording interaction with browser
279 | </div>
280 | <div className="ml-auto relative">
281 | <DropdownMenu>
282 | <DropdownMenuTrigger asChild>
283 | <Button variant="outline" size="icon">
284 | <PlusIcon className="h-4 w-4" />
285 | </Button>
286 | </DropdownMenuTrigger>
287 | <DropdownMenuContent align="end" className="z-[999999]">
288 | <DropdownMenuItem onSelect={() => startPicking('DOM')}>
289 | <Maximize className="mr-2 h-4 w-4" />
290 | <span>Select DOM Element</span>
291 | </DropdownMenuItem>
292 | <DropdownMenuItem onSelect={() => startPicking('Image')}>
293 | <Image className="mr-2 h-4 w-4" />
294 | <span>Take Screenshot</span>
295 | </DropdownMenuItem>
296 | </DropdownMenuContent>
297 | </DropdownMenu>
298 | </div>
299 | </div>
300 | )}
301 | </div>
302 | ) : (
303 | <div className="flex flex-col gap-2 p-4">
304 | <div className="text-sm text-muted-foreground">
305 | No interactions recorded yet!
306 | <br />
307 | <br />
308 | Click 'Start Recording' to record interactions.
309 | <br />
310 | <br />
311 | Once you are done with it, go to your MCP server (like Claude, Cursor),
312 | ask it to pull context using `get-context` tool and give it instructions
313 | on what kind of testcase to write .
314 | </div>
315 | </div>
316 | )}
317 | </ScrollArea>
318 | </div>
319 | );
320 | };
321 |
322 | export default Context;
323 |
```