This is page 11 of 28. Use http://codebase.md/trycua/cua?lines=true&page={x} to view the full context.
# Directory Structure
```
├── .cursorignore
├── .dockerignore
├── .editorconfig
├── .gitattributes
├── .github
│ ├── FUNDING.yml
│ ├── scripts
│ │ ├── get_pyproject_version.py
│ │ └── tests
│ │ ├── __init__.py
│ │ ├── README.md
│ │ └── test_get_pyproject_version.py
│ └── workflows
│ ├── bump-version.yml
│ ├── ci-lume.yml
│ ├── docker-publish-cua-linux.yml
│ ├── docker-publish-cua-windows.yml
│ ├── docker-publish-kasm.yml
│ ├── docker-publish-xfce.yml
│ ├── docker-reusable-publish.yml
│ ├── link-check.yml
│ ├── lint.yml
│ ├── npm-publish-cli.yml
│ ├── npm-publish-computer.yml
│ ├── npm-publish-core.yml
│ ├── publish-lume.yml
│ ├── pypi-publish-agent.yml
│ ├── pypi-publish-computer-server.yml
│ ├── pypi-publish-computer.yml
│ ├── pypi-publish-core.yml
│ ├── pypi-publish-mcp-server.yml
│ ├── pypi-publish-som.yml
│ ├── pypi-reusable-publish.yml
│ ├── python-tests.yml
│ ├── test-cua-models.yml
│ └── test-validation-script.yml
├── .gitignore
├── .pre-commit-config.yaml
├── .prettierignore
├── .prettierrc.yaml
├── .vscode
│ ├── docs.code-workspace
│ ├── extensions.json
│ ├── launch.json
│ ├── libs-ts.code-workspace
│ ├── lume.code-workspace
│ ├── lumier.code-workspace
│ ├── py.code-workspace
│ └── settings.json
├── blog
│ ├── app-use.md
│ ├── assets
│ │ ├── composite-agents.png
│ │ ├── docker-ubuntu-support.png
│ │ ├── hack-booth.png
│ │ ├── hack-closing-ceremony.jpg
│ │ ├── hack-cua-ollama-hud.jpeg
│ │ ├── hack-leaderboard.png
│ │ ├── hack-the-north.png
│ │ ├── hack-winners.jpeg
│ │ ├── hack-workshop.jpeg
│ │ ├── hud-agent-evals.png
│ │ └── trajectory-viewer.jpeg
│ ├── bringing-computer-use-to-the-web.md
│ ├── build-your-own-operator-on-macos-1.md
│ ├── build-your-own-operator-on-macos-2.md
│ ├── cloud-windows-ga-macos-preview.md
│ ├── composite-agents.md
│ ├── computer-use-agents-for-growth-hacking.md
│ ├── cua-hackathon.md
│ ├── cua-playground-preview.md
│ ├── cua-vlm-router.md
│ ├── hack-the-north.md
│ ├── hud-agent-evals.md
│ ├── human-in-the-loop.md
│ ├── introducing-cua-cli.md
│ ├── introducing-cua-cloud-containers.md
│ ├── lume-to-containerization.md
│ ├── neurips-2025-cua-papers.md
│ ├── sandboxed-python-execution.md
│ ├── training-computer-use-models-trajectories-1.md
│ ├── trajectory-viewer.md
│ ├── ubuntu-docker-support.md
│ └── windows-sandbox.md
├── CONTRIBUTING.md
├── Development.md
├── Dockerfile
├── docs
│ ├── .env.example
│ ├── .gitignore
│ ├── content
│ │ └── docs
│ │ ├── agent-sdk
│ │ │ ├── agent-loops.mdx
│ │ │ ├── benchmarks
│ │ │ │ ├── index.mdx
│ │ │ │ ├── interactive.mdx
│ │ │ │ ├── introduction.mdx
│ │ │ │ ├── meta.json
│ │ │ │ ├── osworld-verified.mdx
│ │ │ │ ├── screenspot-pro.mdx
│ │ │ │ └── screenspot-v2.mdx
│ │ │ ├── callbacks
│ │ │ │ ├── agent-lifecycle.mdx
│ │ │ │ ├── cost-saving.mdx
│ │ │ │ ├── index.mdx
│ │ │ │ ├── logging.mdx
│ │ │ │ ├── meta.json
│ │ │ │ ├── pii-anonymization.mdx
│ │ │ │ └── trajectories.mdx
│ │ │ ├── chat-history.mdx
│ │ │ ├── custom-tools.mdx
│ │ │ ├── customizing-computeragent.mdx
│ │ │ ├── integrations
│ │ │ │ ├── hud.mdx
│ │ │ │ ├── meta.json
│ │ │ │ └── observability.mdx
│ │ │ ├── mcp-server
│ │ │ │ ├── client-integrations.mdx
│ │ │ │ ├── configuration.mdx
│ │ │ │ ├── index.mdx
│ │ │ │ ├── installation.mdx
│ │ │ │ ├── llm-integrations.mdx
│ │ │ │ ├── meta.json
│ │ │ │ ├── tools.mdx
│ │ │ │ └── usage.mdx
│ │ │ ├── message-format.mdx
│ │ │ ├── meta.json
│ │ │ ├── migration-guide.mdx
│ │ │ ├── prompt-caching.mdx
│ │ │ ├── supported-agents
│ │ │ │ ├── composed-agents.mdx
│ │ │ │ ├── computer-use-agents.mdx
│ │ │ │ ├── grounding-models.mdx
│ │ │ │ ├── human-in-the-loop.mdx
│ │ │ │ └── meta.json
│ │ │ ├── supported-model-providers
│ │ │ │ ├── cua-vlm-router.mdx
│ │ │ │ ├── index.mdx
│ │ │ │ └── local-models.mdx
│ │ │ ├── telemetry.mdx
│ │ │ └── usage-tracking.mdx
│ │ ├── cli-playbook
│ │ │ ├── commands.mdx
│ │ │ ├── index.mdx
│ │ │ └── meta.json
│ │ ├── computer-sdk
│ │ │ ├── cloud-vm-management.mdx
│ │ │ ├── commands.mdx
│ │ │ ├── computer-server
│ │ │ │ ├── Commands.mdx
│ │ │ │ ├── index.mdx
│ │ │ │ ├── meta.json
│ │ │ │ ├── REST-API.mdx
│ │ │ │ └── WebSocket-API.mdx
│ │ │ ├── computer-ui.mdx
│ │ │ ├── computers.mdx
│ │ │ ├── custom-computer-handlers.mdx
│ │ │ ├── meta.json
│ │ │ ├── sandboxed-python.mdx
│ │ │ └── tracing-api.mdx
│ │ ├── example-usecases
│ │ │ ├── form-filling.mdx
│ │ │ ├── gemini-complex-ui-navigation.mdx
│ │ │ ├── meta.json
│ │ │ ├── post-event-contact-export.mdx
│ │ │ └── windows-app-behind-vpn.mdx
│ │ ├── get-started
│ │ │ ├── meta.json
│ │ │ └── quickstart.mdx
│ │ ├── index.mdx
│ │ ├── macos-vm-cli-playbook
│ │ │ ├── lume
│ │ │ │ ├── cli-reference.mdx
│ │ │ │ ├── faq.md
│ │ │ │ ├── http-api.mdx
│ │ │ │ ├── index.mdx
│ │ │ │ ├── installation.mdx
│ │ │ │ ├── meta.json
│ │ │ │ └── prebuilt-images.mdx
│ │ │ ├── lumier
│ │ │ │ ├── building-lumier.mdx
│ │ │ │ ├── docker-compose.mdx
│ │ │ │ ├── docker.mdx
│ │ │ │ ├── index.mdx
│ │ │ │ ├── installation.mdx
│ │ │ │ └── meta.json
│ │ │ └── meta.json
│ │ └── meta.json
│ ├── next.config.mjs
│ ├── package-lock.json
│ ├── package.json
│ ├── pnpm-lock.yaml
│ ├── postcss.config.mjs
│ ├── public
│ │ └── img
│ │ ├── agent_gradio_ui.png
│ │ ├── agent.png
│ │ ├── bg-dark.jpg
│ │ ├── bg-light.jpg
│ │ ├── cli.png
│ │ ├── computer.png
│ │ ├── grounding-with-gemini3.gif
│ │ ├── hero.png
│ │ ├── laminar_trace_example.png
│ │ ├── som_box_threshold.png
│ │ └── som_iou_threshold.png
│ ├── README.md
│ ├── source.config.ts
│ ├── src
│ │ ├── app
│ │ │ ├── (home)
│ │ │ │ ├── [[...slug]]
│ │ │ │ │ └── page.tsx
│ │ │ │ └── layout.tsx
│ │ │ ├── api
│ │ │ │ ├── posthog
│ │ │ │ │ └── [...path]
│ │ │ │ │ └── route.ts
│ │ │ │ └── search
│ │ │ │ └── route.ts
│ │ │ ├── favicon.ico
│ │ │ ├── global.css
│ │ │ ├── layout.config.tsx
│ │ │ ├── layout.tsx
│ │ │ ├── llms.mdx
│ │ │ │ └── [[...slug]]
│ │ │ │ └── route.ts
│ │ │ ├── llms.txt
│ │ │ │ └── route.ts
│ │ │ ├── robots.ts
│ │ │ └── sitemap.ts
│ │ ├── assets
│ │ │ ├── discord-black.svg
│ │ │ ├── discord-white.svg
│ │ │ ├── logo-black.svg
│ │ │ └── logo-white.svg
│ │ ├── components
│ │ │ ├── analytics-tracker.tsx
│ │ │ ├── cookie-consent.tsx
│ │ │ ├── doc-actions-menu.tsx
│ │ │ ├── editable-code-block.tsx
│ │ │ ├── footer.tsx
│ │ │ ├── hero.tsx
│ │ │ ├── iou.tsx
│ │ │ ├── mermaid.tsx
│ │ │ └── page-feedback.tsx
│ │ ├── lib
│ │ │ ├── llms.ts
│ │ │ └── source.ts
│ │ ├── mdx-components.tsx
│ │ └── providers
│ │ └── posthog-provider.tsx
│ └── tsconfig.json
├── examples
│ ├── agent_examples.py
│ ├── agent_ui_examples.py
│ ├── browser_tool_example.py
│ ├── cloud_api_examples.py
│ ├── computer_examples_windows.py
│ ├── computer_examples.py
│ ├── computer_ui_examples.py
│ ├── computer-example-ts
│ │ ├── .env.example
│ │ ├── .gitignore
│ │ ├── package-lock.json
│ │ ├── package.json
│ │ ├── pnpm-lock.yaml
│ │ ├── README.md
│ │ ├── src
│ │ │ ├── helpers.ts
│ │ │ └── index.ts
│ │ └── tsconfig.json
│ ├── docker_examples.py
│ ├── evals
│ │ ├── hud_eval_examples.py
│ │ └── wikipedia_most_linked.txt
│ ├── pylume_examples.py
│ ├── sandboxed_functions_examples.py
│ ├── som_examples.py
│ ├── tracing_examples.py
│ ├── utils.py
│ └── winsandbox_example.py
├── img
│ ├── agent_gradio_ui.png
│ ├── agent.png
│ ├── cli.png
│ ├── computer.png
│ ├── logo_black.png
│ └── logo_white.png
├── libs
│ ├── kasm
│ │ ├── Dockerfile
│ │ ├── LICENSE
│ │ ├── README.md
│ │ └── src
│ │ └── ubuntu
│ │ └── install
│ │ └── firefox
│ │ ├── custom_startup.sh
│ │ ├── firefox.desktop
│ │ └── install_firefox.sh
│ ├── lume
│ │ ├── .cursorignore
│ │ ├── CONTRIBUTING.md
│ │ ├── Development.md
│ │ ├── img
│ │ │ └── cli.png
│ │ ├── Package.resolved
│ │ ├── Package.swift
│ │ ├── README.md
│ │ ├── resources
│ │ │ └── lume.entitlements
│ │ ├── scripts
│ │ │ ├── build
│ │ │ │ ├── build-debug.sh
│ │ │ │ ├── build-release-notarized.sh
│ │ │ │ └── build-release.sh
│ │ │ └── install.sh
│ │ ├── src
│ │ │ ├── Commands
│ │ │ │ ├── Clone.swift
│ │ │ │ ├── Config.swift
│ │ │ │ ├── Create.swift
│ │ │ │ ├── Delete.swift
│ │ │ │ ├── Get.swift
│ │ │ │ ├── Images.swift
│ │ │ │ ├── IPSW.swift
│ │ │ │ ├── List.swift
│ │ │ │ ├── Logs.swift
│ │ │ │ ├── Options
│ │ │ │ │ └── FormatOption.swift
│ │ │ │ ├── Prune.swift
│ │ │ │ ├── Pull.swift
│ │ │ │ ├── Push.swift
│ │ │ │ ├── Run.swift
│ │ │ │ ├── Serve.swift
│ │ │ │ ├── Set.swift
│ │ │ │ └── Stop.swift
│ │ │ ├── ContainerRegistry
│ │ │ │ ├── ImageContainerRegistry.swift
│ │ │ │ ├── ImageList.swift
│ │ │ │ └── ImagesPrinter.swift
│ │ │ ├── Errors
│ │ │ │ └── Errors.swift
│ │ │ ├── FileSystem
│ │ │ │ ├── Home.swift
│ │ │ │ ├── Settings.swift
│ │ │ │ ├── VMConfig.swift
│ │ │ │ ├── VMDirectory.swift
│ │ │ │ └── VMLocation.swift
│ │ │ ├── LumeController.swift
│ │ │ ├── Main.swift
│ │ │ ├── Server
│ │ │ │ ├── Handlers.swift
│ │ │ │ ├── HTTP.swift
│ │ │ │ ├── Requests.swift
│ │ │ │ ├── Responses.swift
│ │ │ │ └── Server.swift
│ │ │ ├── Utils
│ │ │ │ ├── CommandRegistry.swift
│ │ │ │ ├── CommandUtils.swift
│ │ │ │ ├── Logger.swift
│ │ │ │ ├── NetworkUtils.swift
│ │ │ │ ├── Path.swift
│ │ │ │ ├── ProcessRunner.swift
│ │ │ │ ├── ProgressLogger.swift
│ │ │ │ ├── String.swift
│ │ │ │ └── Utils.swift
│ │ │ ├── Virtualization
│ │ │ │ ├── DarwinImageLoader.swift
│ │ │ │ ├── DHCPLeaseParser.swift
│ │ │ │ ├── ImageLoaderFactory.swift
│ │ │ │ └── VMVirtualizationService.swift
│ │ │ ├── VM
│ │ │ │ ├── DarwinVM.swift
│ │ │ │ ├── LinuxVM.swift
│ │ │ │ ├── VM.swift
│ │ │ │ ├── VMDetails.swift
│ │ │ │ ├── VMDetailsPrinter.swift
│ │ │ │ ├── VMDisplayResolution.swift
│ │ │ │ └── VMFactory.swift
│ │ │ └── VNC
│ │ │ ├── PassphraseGenerator.swift
│ │ │ └── VNCService.swift
│ │ └── tests
│ │ ├── Mocks
│ │ │ ├── MockVM.swift
│ │ │ ├── MockVMVirtualizationService.swift
│ │ │ └── MockVNCService.swift
│ │ ├── VM
│ │ │ └── VMDetailsPrinterTests.swift
│ │ ├── VMTests.swift
│ │ ├── VMVirtualizationServiceTests.swift
│ │ └── VNCServiceTests.swift
│ ├── lumier
│ │ ├── .dockerignore
│ │ ├── Dockerfile
│ │ ├── README.md
│ │ └── src
│ │ ├── bin
│ │ │ └── entry.sh
│ │ ├── config
│ │ │ └── constants.sh
│ │ ├── hooks
│ │ │ └── on-logon.sh
│ │ └── lib
│ │ ├── utils.sh
│ │ └── vm.sh
│ ├── python
│ │ ├── agent
│ │ │ ├── .bumpversion.cfg
│ │ │ ├── agent
│ │ │ │ ├── __init__.py
│ │ │ │ ├── __main__.py
│ │ │ │ ├── adapters
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── cua_adapter.py
│ │ │ │ │ ├── huggingfacelocal_adapter.py
│ │ │ │ │ ├── human_adapter.py
│ │ │ │ │ ├── mlxvlm_adapter.py
│ │ │ │ │ └── models
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── generic.py
│ │ │ │ │ ├── internvl.py
│ │ │ │ │ ├── opencua.py
│ │ │ │ │ └── qwen2_5_vl.py
│ │ │ │ ├── agent.py
│ │ │ │ ├── callbacks
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── base.py
│ │ │ │ │ ├── budget_manager.py
│ │ │ │ │ ├── image_retention.py
│ │ │ │ │ ├── logging.py
│ │ │ │ │ ├── operator_validator.py
│ │ │ │ │ ├── pii_anonymization.py
│ │ │ │ │ ├── prompt_instructions.py
│ │ │ │ │ ├── telemetry.py
│ │ │ │ │ └── trajectory_saver.py
│ │ │ │ ├── cli.py
│ │ │ │ ├── computers
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── base.py
│ │ │ │ │ ├── cua.py
│ │ │ │ │ └── custom.py
│ │ │ │ ├── decorators.py
│ │ │ │ ├── human_tool
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── __main__.py
│ │ │ │ │ ├── server.py
│ │ │ │ │ └── ui.py
│ │ │ │ ├── integrations
│ │ │ │ │ └── hud
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── agent.py
│ │ │ │ │ └── proxy.py
│ │ │ │ ├── loops
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── anthropic.py
│ │ │ │ │ ├── base.py
│ │ │ │ │ ├── composed_grounded.py
│ │ │ │ │ ├── gelato.py
│ │ │ │ │ ├── gemini.py
│ │ │ │ │ ├── generic_vlm.py
│ │ │ │ │ ├── glm45v.py
│ │ │ │ │ ├── gta1.py
│ │ │ │ │ ├── holo.py
│ │ │ │ │ ├── internvl.py
│ │ │ │ │ ├── model_types.csv
│ │ │ │ │ ├── moondream3.py
│ │ │ │ │ ├── omniparser.py
│ │ │ │ │ ├── openai.py
│ │ │ │ │ ├── opencua.py
│ │ │ │ │ ├── uiins.py
│ │ │ │ │ ├── uitars.py
│ │ │ │ │ └── uitars2.py
│ │ │ │ ├── proxy
│ │ │ │ │ ├── examples.py
│ │ │ │ │ └── handlers.py
│ │ │ │ ├── responses.py
│ │ │ │ ├── tools
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ └── browser_tool.py
│ │ │ │ ├── types.py
│ │ │ │ └── ui
│ │ │ │ ├── __init__.py
│ │ │ │ ├── __main__.py
│ │ │ │ └── gradio
│ │ │ │ ├── __init__.py
│ │ │ │ ├── app.py
│ │ │ │ └── ui_components.py
│ │ │ ├── benchmarks
│ │ │ │ ├── .gitignore
│ │ │ │ ├── contrib.md
│ │ │ │ ├── interactive.py
│ │ │ │ ├── models
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── base.py
│ │ │ │ │ └── gta1.py
│ │ │ │ ├── README.md
│ │ │ │ ├── ss-pro.py
│ │ │ │ ├── ss-v2.py
│ │ │ │ └── utils.py
│ │ │ ├── example.py
│ │ │ ├── pyproject.toml
│ │ │ ├── README.md
│ │ │ └── tests
│ │ │ ├── conftest.py
│ │ │ └── test_computer_agent.py
│ │ ├── bench-ui
│ │ │ ├── bench_ui
│ │ │ │ ├── __init__.py
│ │ │ │ ├── api.py
│ │ │ │ └── child.py
│ │ │ ├── examples
│ │ │ │ ├── folder_example.py
│ │ │ │ ├── gui
│ │ │ │ │ ├── index.html
│ │ │ │ │ ├── logo.svg
│ │ │ │ │ └── styles.css
│ │ │ │ ├── output_overlay.png
│ │ │ │ └── simple_example.py
│ │ │ ├── pyproject.toml
│ │ │ ├── README.md
│ │ │ └── tests
│ │ │ └── test_port_detection.py
│ │ ├── computer
│ │ │ ├── .bumpversion.cfg
│ │ │ ├── computer
│ │ │ │ ├── __init__.py
│ │ │ │ ├── computer.py
│ │ │ │ ├── diorama_computer.py
│ │ │ │ ├── helpers.py
│ │ │ │ ├── interface
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── base.py
│ │ │ │ │ ├── factory.py
│ │ │ │ │ ├── generic.py
│ │ │ │ │ ├── linux.py
│ │ │ │ │ ├── macos.py
│ │ │ │ │ ├── models.py
│ │ │ │ │ └── windows.py
│ │ │ │ ├── logger.py
│ │ │ │ ├── models.py
│ │ │ │ ├── providers
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── base.py
│ │ │ │ │ ├── cloud
│ │ │ │ │ │ ├── __init__.py
│ │ │ │ │ │ └── provider.py
│ │ │ │ │ ├── docker
│ │ │ │ │ │ ├── __init__.py
│ │ │ │ │ │ └── provider.py
│ │ │ │ │ ├── factory.py
│ │ │ │ │ ├── lume
│ │ │ │ │ │ ├── __init__.py
│ │ │ │ │ │ └── provider.py
│ │ │ │ │ ├── lume_api.py
│ │ │ │ │ ├── lumier
│ │ │ │ │ │ ├── __init__.py
│ │ │ │ │ │ └── provider.py
│ │ │ │ │ ├── types.py
│ │ │ │ │ └── winsandbox
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── provider.py
│ │ │ │ │ └── setup_script.ps1
│ │ │ │ ├── tracing_wrapper.py
│ │ │ │ ├── tracing.py
│ │ │ │ ├── ui
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── __main__.py
│ │ │ │ │ └── gradio
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ └── app.py
│ │ │ │ └── utils.py
│ │ │ ├── poetry.toml
│ │ │ ├── pyproject.toml
│ │ │ ├── README.md
│ │ │ └── tests
│ │ │ ├── conftest.py
│ │ │ └── test_computer.py
│ │ ├── computer-server
│ │ │ ├── .bumpversion.cfg
│ │ │ ├── computer_server
│ │ │ │ ├── __init__.py
│ │ │ │ ├── __main__.py
│ │ │ │ ├── browser.py
│ │ │ │ ├── cli.py
│ │ │ │ ├── diorama
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ ├── base.py
│ │ │ │ │ ├── diorama_computer.py
│ │ │ │ │ ├── diorama.py
│ │ │ │ │ ├── draw.py
│ │ │ │ │ ├── macos.py
│ │ │ │ │ └── safezone.py
│ │ │ │ ├── handlers
│ │ │ │ │ ├── base.py
│ │ │ │ │ ├── factory.py
│ │ │ │ │ ├── generic.py
│ │ │ │ │ ├── linux.py
│ │ │ │ │ ├── macos.py
│ │ │ │ │ └── windows.py
│ │ │ │ ├── main.py
│ │ │ │ ├── server.py
│ │ │ │ ├── utils
│ │ │ │ │ ├── __init__.py
│ │ │ │ │ └── wallpaper.py
│ │ │ │ └── watchdog.py
│ │ │ ├── examples
│ │ │ │ ├── __init__.py
│ │ │ │ └── usage_example.py
│ │ │ ├── pyproject.toml
│ │ │ ├── README.md
│ │ │ ├── run_server.py
│ │ │ ├── test_connection.py
│ │ │ └── tests
│ │ │ ├── conftest.py
│ │ │ └── test_server.py
│ │ ├── core
│ │ │ ├── .bumpversion.cfg
│ │ │ ├── core
│ │ │ │ ├── __init__.py
│ │ │ │ └── telemetry
│ │ │ │ ├── __init__.py
│ │ │ │ └── posthog.py
│ │ │ ├── poetry.toml
│ │ │ ├── pyproject.toml
│ │ │ ├── README.md
│ │ │ └── tests
│ │ │ ├── conftest.py
│ │ │ └── test_telemetry.py
│ │ ├── mcp-server
│ │ │ ├── .bumpversion.cfg
│ │ │ ├── build-extension.py
│ │ │ ├── CONCURRENT_SESSIONS.md
│ │ │ ├── desktop-extension
│ │ │ │ ├── cua-extension.mcpb
│ │ │ │ ├── desktop_extension.png
│ │ │ │ ├── manifest.json
│ │ │ │ ├── README.md
│ │ │ │ ├── requirements.txt
│ │ │ │ ├── run_server.sh
│ │ │ │ └── setup.py
│ │ │ ├── mcp_server
│ │ │ │ ├── __init__.py
│ │ │ │ ├── __main__.py
│ │ │ │ ├── server.py
│ │ │ │ └── session_manager.py
│ │ │ ├── pdm.lock
│ │ │ ├── pyproject.toml
│ │ │ ├── QUICK_TEST_COMMANDS.sh
│ │ │ ├── quick_test_local_option.py
│ │ │ ├── README.md
│ │ │ ├── scripts
│ │ │ │ ├── install_mcp_server.sh
│ │ │ │ └── start_mcp_server.sh
│ │ │ ├── test_mcp_server_local_option.py
│ │ │ └── tests
│ │ │ ├── conftest.py
│ │ │ └── test_mcp_server.py
│ │ ├── pylume
│ │ │ └── tests
│ │ │ ├── conftest.py
│ │ │ └── test_pylume.py
│ │ └── som
│ │ ├── .bumpversion.cfg
│ │ ├── LICENSE
│ │ ├── poetry.toml
│ │ ├── pyproject.toml
│ │ ├── README.md
│ │ ├── som
│ │ │ ├── __init__.py
│ │ │ ├── detect.py
│ │ │ ├── detection.py
│ │ │ ├── models.py
│ │ │ ├── ocr.py
│ │ │ ├── util
│ │ │ │ └── utils.py
│ │ │ └── visualization.py
│ │ └── tests
│ │ ├── conftest.py
│ │ └── test_omniparser.py
│ ├── qemu-docker
│ │ ├── linux
│ │ │ ├── Dockerfile
│ │ │ ├── README.md
│ │ │ └── src
│ │ │ ├── entry.sh
│ │ │ └── vm
│ │ │ ├── image
│ │ │ │ └── README.md
│ │ │ └── setup
│ │ │ ├── install.sh
│ │ │ ├── setup-cua-server.sh
│ │ │ └── setup.sh
│ │ ├── README.md
│ │ └── windows
│ │ ├── Dockerfile
│ │ ├── README.md
│ │ └── src
│ │ ├── entry.sh
│ │ └── vm
│ │ ├── image
│ │ │ └── README.md
│ │ └── setup
│ │ ├── install.bat
│ │ ├── on-logon.ps1
│ │ ├── setup-cua-server.ps1
│ │ ├── setup-utils.psm1
│ │ └── setup.ps1
│ ├── typescript
│ │ ├── .gitignore
│ │ ├── .nvmrc
│ │ ├── agent
│ │ │ ├── examples
│ │ │ │ ├── playground-example.html
│ │ │ │ └── README.md
│ │ │ ├── package.json
│ │ │ ├── README.md
│ │ │ ├── src
│ │ │ │ ├── client.ts
│ │ │ │ ├── index.ts
│ │ │ │ └── types.ts
│ │ │ ├── tests
│ │ │ │ └── client.test.ts
│ │ │ ├── tsconfig.json
│ │ │ ├── tsdown.config.ts
│ │ │ └── vitest.config.ts
│ │ ├── computer
│ │ │ ├── .editorconfig
│ │ │ ├── .gitattributes
│ │ │ ├── .gitignore
│ │ │ ├── LICENSE
│ │ │ ├── package.json
│ │ │ ├── README.md
│ │ │ ├── src
│ │ │ │ ├── computer
│ │ │ │ │ ├── index.ts
│ │ │ │ │ ├── providers
│ │ │ │ │ │ ├── base.ts
│ │ │ │ │ │ ├── cloud.ts
│ │ │ │ │ │ └── index.ts
│ │ │ │ │ └── types.ts
│ │ │ │ ├── index.ts
│ │ │ │ ├── interface
│ │ │ │ │ ├── base.ts
│ │ │ │ │ ├── factory.ts
│ │ │ │ │ ├── index.ts
│ │ │ │ │ ├── linux.ts
│ │ │ │ │ ├── macos.ts
│ │ │ │ │ └── windows.ts
│ │ │ │ └── types.ts
│ │ │ ├── tests
│ │ │ │ ├── computer
│ │ │ │ │ └── cloud.test.ts
│ │ │ │ ├── interface
│ │ │ │ │ ├── factory.test.ts
│ │ │ │ │ ├── index.test.ts
│ │ │ │ │ ├── linux.test.ts
│ │ │ │ │ ├── macos.test.ts
│ │ │ │ │ └── windows.test.ts
│ │ │ │ └── setup.ts
│ │ │ ├── tsconfig.json
│ │ │ ├── tsdown.config.ts
│ │ │ └── vitest.config.ts
│ │ ├── core
│ │ │ ├── .editorconfig
│ │ │ ├── .gitattributes
│ │ │ ├── .gitignore
│ │ │ ├── LICENSE
│ │ │ ├── package.json
│ │ │ ├── README.md
│ │ │ ├── src
│ │ │ │ ├── index.ts
│ │ │ │ └── telemetry
│ │ │ │ ├── clients
│ │ │ │ │ ├── index.ts
│ │ │ │ │ └── posthog.ts
│ │ │ │ └── index.ts
│ │ │ ├── tests
│ │ │ │ └── telemetry.test.ts
│ │ │ ├── tsconfig.json
│ │ │ ├── tsdown.config.ts
│ │ │ └── vitest.config.ts
│ │ ├── cua-cli
│ │ │ ├── .gitignore
│ │ │ ├── .prettierrc
│ │ │ ├── bun.lock
│ │ │ ├── CLAUDE.md
│ │ │ ├── index.ts
│ │ │ ├── package.json
│ │ │ ├── README.md
│ │ │ ├── src
│ │ │ │ ├── auth.ts
│ │ │ │ ├── cli.ts
│ │ │ │ ├── commands
│ │ │ │ │ ├── auth.ts
│ │ │ │ │ └── sandbox.ts
│ │ │ │ ├── config.ts
│ │ │ │ ├── http.ts
│ │ │ │ ├── storage.ts
│ │ │ │ └── util.ts
│ │ │ └── tsconfig.json
│ │ ├── package.json
│ │ ├── pnpm-lock.yaml
│ │ ├── pnpm-workspace.yaml
│ │ └── README.md
│ └── xfce
│ ├── .dockerignore
│ ├── .gitignore
│ ├── Development.md
│ ├── Dockerfile
│ ├── Dockerfile.dev
│ ├── README.md
│ └── src
│ ├── scripts
│ │ ├── resize-display.sh
│ │ ├── start-computer-server.sh
│ │ ├── start-novnc.sh
│ │ ├── start-vnc.sh
│ │ └── xstartup.sh
│ ├── supervisor
│ │ └── supervisord.conf
│ └── xfce-config
│ ├── helpers.rc
│ ├── xfce4-power-manager.xml
│ └── xfce4-session.xml
├── LICENSE.md
├── Makefile
├── notebooks
│ ├── agent_nb.ipynb
│ ├── blog
│ │ ├── build-your-own-operator-on-macos-1.ipynb
│ │ └── build-your-own-operator-on-macos-2.ipynb
│ ├── composite_agents_docker_nb.ipynb
│ ├── computer_nb.ipynb
│ ├── computer_server_nb.ipynb
│ ├── customizing_computeragent.ipynb
│ ├── eval_osworld.ipynb
│ ├── ollama_nb.ipynb
│ ├── README.md
│ ├── sota_hackathon_cloud.ipynb
│ └── sota_hackathon.ipynb
├── package-lock.json
├── package.json
├── pnpm-lock.yaml
├── pyproject.toml
├── pyrightconfig.json
├── README.md
├── scripts
│ ├── install-cli.ps1
│ ├── install-cli.sh
│ ├── playground-docker.sh
│ ├── playground.sh
│ ├── run-docker-dev.sh
│ └── typescript-typecheck.js
├── TESTING.md
├── tests
│ ├── agent_loop_testing
│ │ ├── agent_test.py
│ │ └── README.md
│ ├── pytest.ini
│ ├── shell_cmd.py
│ ├── test_files.py
│ ├── test_mcp_server_session_management.py
│ ├── test_mcp_server_streaming.py
│ ├── test_shell_bash.py
│ ├── test_telemetry.py
│ ├── test_tracing.py
│ ├── test_venv.py
│ └── test_watchdog.py
└── uv.lock
```
# Files
--------------------------------------------------------------------------------
/libs/lume/src/Errors/Errors.swift:
--------------------------------------------------------------------------------
```swift
1 | import Foundation
2 |
3 | enum HomeError: Error, LocalizedError {
4 | case directoryCreationFailed(path: String)
5 | case directoryAccessDenied(path: String)
6 | case invalidHomeDirectory
7 | case directoryAlreadyExists(path: String)
8 | case homeNotFound
9 | case defaultStorageNotDefined
10 | case storageLocationNotFound(String)
11 | case storageLocationNotADirectory(String)
12 | case storageLocationNotWritable(String)
13 | case invalidStorageLocation(String)
14 | case cannotCreateDirectory(String)
15 | case cannotGetVMsDirectory
16 | case vmDirectoryNotFound(String)
17 |
18 | var errorDescription: String? {
19 | switch self {
20 | case .directoryCreationFailed(let path):
21 | return "Failed to create directory at path: \(path)"
22 | case .directoryAccessDenied(let path):
23 | return "Access denied to directory at path: \(path)"
24 | case .invalidHomeDirectory:
25 | return "Invalid home directory configuration"
26 | case .directoryAlreadyExists(let path):
27 | return "Directory already exists at path: \(path)"
28 | case .homeNotFound:
29 | return "Home directory not found."
30 | case .defaultStorageNotDefined:
31 | return "Default storage location is not defined."
32 | case .storageLocationNotFound(let path):
33 | return "Storage location not found: \(path)"
34 | case .storageLocationNotADirectory(let path):
35 | return "Storage location is not a directory: \(path)"
36 | case .storageLocationNotWritable(let path):
37 | return "Storage location is not writable: \(path)"
38 | case .invalidStorageLocation(let path):
39 | return "Invalid storage location specified: \(path)"
40 | case .cannotCreateDirectory(let path):
41 | return "Cannot create directory: \(path)"
42 | case .cannotGetVMsDirectory:
43 | return "Cannot determine the VMs directory."
44 | case .vmDirectoryNotFound(let path):
45 | return "VM directory not found: \(path)"
46 | }
47 | }
48 | }
49 |
50 | enum PullError: Error, LocalizedError {
51 | case invalidImageFormat
52 | case tokenFetchFailed
53 | case manifestFetchFailed
54 | case layerDownloadFailed(String)
55 | case missingPart(Int)
56 | case decompressionFailed(String)
57 | case reassemblyFailed(String)
58 | case fileCreationFailed(String)
59 | case reassemblySetupFailed(path: String, underlyingError: Error)
60 | case missingUncompressedSizeAnnotation
61 | case invalidMediaType
62 |
63 | var errorDescription: String? {
64 | switch self {
65 | case .invalidImageFormat:
66 | return "Invalid image format. Expected format: name:tag"
67 | case .tokenFetchFailed:
68 | return "Failed to fetch authentication token from registry."
69 | case .manifestFetchFailed:
70 | return "Failed to fetch image manifest from registry."
71 | case .layerDownloadFailed(let digest):
72 | return "Failed to download layer: \(digest)"
73 | case .missingPart(let partNum):
74 | return "Missing required part number \(partNum) for reassembly."
75 | case .decompressionFailed(let file):
76 | return "Failed to decompress file: \(file)"
77 | case .reassemblyFailed(let reason):
78 | return "Disk image reassembly failed: \(reason)."
79 | case .fileCreationFailed(let path):
80 | return "Failed to create the necessary file at path: \(path)"
81 | case .reassemblySetupFailed(let path, let underlyingError):
82 | return "Failed to set up for reassembly at path: \(path). Underlying error: \(underlyingError.localizedDescription)"
83 | case .missingUncompressedSizeAnnotation:
84 | return "Could not find the required uncompressed disk size annotation in the image config.json."
85 | case .invalidMediaType:
86 | return "Invalid media type"
87 | }
88 | }
89 | }
90 |
91 | enum VMConfigError: CustomNSError, LocalizedError {
92 | case invalidDisplayResolution(String)
93 | case invalidMachineIdentifier
94 | case emptyMachineIdentifier
95 | case emptyHardwareModel
96 | case invalidHardwareModel
97 | case invalidDiskSize
98 | case malformedSizeInput(String)
99 |
100 | var errorDescription: String? {
101 | switch self {
102 | case .invalidDisplayResolution(let resolution):
103 | return "Invalid display resolution: \(resolution)"
104 | case .emptyMachineIdentifier:
105 | return "Empty machine identifier"
106 | case .invalidMachineIdentifier:
107 | return "Invalid machine identifier"
108 | case .emptyHardwareModel:
109 | return "Empty hardware model"
110 | case .invalidHardwareModel:
111 | return "Invalid hardware model: the host does not support the hardware model"
112 | case .invalidDiskSize:
113 | return "Invalid disk size"
114 | case .malformedSizeInput(let input):
115 | return "Malformed size input: \(input)"
116 | }
117 | }
118 |
119 | static var errorDomain: String { "VMConfigError" }
120 |
121 | var errorCode: Int {
122 | switch self {
123 | case .invalidDisplayResolution: return 1
124 | case .emptyMachineIdentifier: return 2
125 | case .invalidMachineIdentifier: return 3
126 | case .emptyHardwareModel: return 4
127 | case .invalidHardwareModel: return 5
128 | case .invalidDiskSize: return 6
129 | case .malformedSizeInput: return 7
130 | }
131 | }
132 | }
133 |
134 | enum VMDirectoryError: Error, LocalizedError {
135 | case configNotFound
136 | case invalidConfigData
137 | case diskOperationFailed(String)
138 | case fileCreationFailed(String)
139 | case sessionNotFound
140 | case invalidSessionData
141 |
142 | var errorDescription: String {
143 | switch self {
144 | case .configNotFound:
145 | return "VM configuration file not found"
146 | case .invalidConfigData:
147 | return "Invalid VM configuration data"
148 | case .diskOperationFailed(let reason):
149 | return "Disk operation failed: \(reason)"
150 | case .fileCreationFailed(let path):
151 | return "Failed to create file at path: \(path)"
152 | case .sessionNotFound:
153 | return "VNC session file not found"
154 | case .invalidSessionData:
155 | return "Invalid VNC session data"
156 | }
157 | }
158 | }
159 |
160 | enum VMError: Error, LocalizedError {
161 | case alreadyExists(String)
162 | case notFound(String)
163 | case notInitialized(String)
164 | case notRunning(String)
165 | case alreadyRunning(String)
166 | case installNotStarted(String)
167 | case stopTimeout(String)
168 | case resizeTooSmall(current: UInt64, requested: UInt64)
169 | case vncNotConfigured
170 | case vncPortBindingFailed(requested: Int, actual: Int)
171 | case internalError(String)
172 | case unsupportedOS(String)
173 | case invalidDisplayResolution(String)
174 | var errorDescription: String? {
175 | switch self {
176 | case .alreadyExists(let name):
177 | return "Virtual machine already exists with name: \(name)"
178 | case .notFound(let name):
179 | return "Virtual machine not found: \(name)"
180 | case .notInitialized(let name):
181 | return "Virtual machine not initialized: \(name)"
182 | case .notRunning(let name):
183 | return "Virtual machine not running: \(name)"
184 | case .alreadyRunning(let name):
185 | return "Virtual machine already running: \(name)"
186 | case .installNotStarted(let name):
187 | return "Virtual machine install not started: \(name)"
188 | case .stopTimeout(let name):
189 | return "Timeout while stopping virtual machine: \(name)"
190 | case .resizeTooSmall(let current, let requested):
191 | return "Cannot resize disk to \(requested) bytes, current size is \(current) bytes"
192 | case .vncNotConfigured:
193 | return "VNC is not configured for this virtual machine"
194 | case .vncPortBindingFailed(let requested, let actual):
195 | if actual == -1 {
196 | return "Could not bind to VNC port \(requested) (port already in use). Try a different port or use port 0 for auto-assign."
197 | }
198 | return "Could not bind to VNC port \(requested) (port already in use). System assigned port \(actual) instead. Try a different port or use port 0 for auto-assign."
199 | case .internalError(let message):
200 | return "Internal error: \(message)"
201 | case .unsupportedOS(let os):
202 | return "Unsupported operating system: \(os)"
203 | case .invalidDisplayResolution(let resolution):
204 | return "Invalid display resolution: \(resolution)"
205 | }
206 | }
207 | }
208 |
209 | enum ResticError: Error {
210 | case snapshotFailed(String)
211 | case restoreFailed(String)
212 | case genericError(String)
213 | }
214 |
215 | enum VmrunError: Error, LocalizedError {
216 | case commandNotFound
217 | case operationFailed(command: String, output: String?)
218 |
219 | var errorDescription: String? {
220 | switch self {
221 | case .commandNotFound:
222 | return "vmrun command not found. Ensure VMware Fusion is installed and in the system PATH."
223 | case .operationFailed(let command, let output):
224 | return "vmrun command '\(command)' failed. Output: \(output ?? "No output")"
225 | }
226 | }
227 | }
```
--------------------------------------------------------------------------------
/blog/introducing-cua-cloud-containers.md:
--------------------------------------------------------------------------------
```markdown
1 | # Introducing Cua Cloud Sandbox: Computer-Use Agents in the Cloud
2 |
3 | _Published on May 28, 2025 by Francesco Bonacci_
4 |
5 | Welcome to the next chapter in our Computer-Use Agent journey! In [Part 1](./build-your-own-operator-on-macos-1), we showed you how to build your own Operator on macOS. In [Part 2](./build-your-own-operator-on-macos-2), we explored the cua-agent framework. Today, we're excited to introduce **Cua Cloud Sandbox** – the easiest way to deploy Computer-Use Agents at scale.
6 |
7 | <div align="center">
8 | <video src="https://github.com/user-attachments/assets/63a2addf-649f-4468-971d-58d38dd43ee6" width="600" controls></video>
9 | </div>
10 |
11 | ## What is Cua Cloud?
12 |
13 | Think of Cua Cloud as **Docker for Computer-Use Agents**. Instead of managing VMs, installing dependencies, and configuring environments, you can launch pre-configured Cloud Sandbox instances with a single command. Each sandbox comes with a **full desktop environment** accessible via browser (via noVNC), all CUA-related dependencies pre-configured (with a PyAutoGUI-compatible server), and **pay-per-use pricing** that scales with your needs.
14 |
15 | ## Why Cua Cloud Sandbox?
16 |
17 | Four months ago, we launched [**Lume**](https://github.com/trycua/cua/tree/main/libs/lume) and [**Cua**](https://github.com/trycua/cua) with the goal to bring sandboxed VMs and Computer-Use Agents on Apple Silicon. The developer's community response was incredible 🎉
18 |
19 | Going from prototype to production revealed a problem though: **local macOS VMs don't scale**, neither are they easily portable.
20 |
21 | Our Discord community, YC peers, and early pilot customers kept hitting the same issues. Storage constraints meant **20-40GB per VM** filled laptops fast. Different hardware architectures (Apple Silicon ARM vs Intel x86) prevented portability of local workflows. Every new user lost a day to setup and configuration.
22 |
23 | **Cua Cloud** eliminates these constraints while preserving everything developers are familiar with about our Computer and Agent SDK.
24 |
25 | ### What We Built
26 |
27 | Over the past month, we've been iterating over Cua Cloud with partners and beta users to address these challenges. You use the exact same `Computer` and `ComputerAgent` classes you already know, but with **zero local setup** or storage requirements. VNC access comes with **built-in encryption**, you pay only for compute time (not idle resources), and can bring your own API keys for any LLM provider.
28 |
29 | The result? **Instant deployment** in seconds instead of hours, with no infrastructure to manage. Scale elastically from **1 to 100 agents** in parallel, with consistent behavior across all deployments. Share agent trajectories with your team for better collaboration and debugging.
30 |
31 | ## Getting Started
32 |
33 | ### Step 1: Get Your API Key
34 |
35 | Sign up at [**cua.ai**](https://cua.ai) to get your API key.
36 |
37 | ```bash
38 | # Set your API key in environment variables
39 | export CUA_API_KEY=your_api_key_here
40 | export CUA_CONTAINER_NAME=my-agent-container
41 | ```
42 |
43 | ### Step 2: Launch Your First Sandbox
44 |
45 | ```python
46 | import asyncio
47 | from computer import Computer, VMProviderType
48 | from agent import ComputerAgent
49 |
50 | async def run_cloud_agent():
51 | # Create a remote Linux computer with Cua Cloud
52 | computer = Computer(
53 | os_type="linux",
54 | api_key=os.getenv("CUA_API_KEY"),
55 | name=os.getenv("CUA_CONTAINER_NAME"),
56 | provider_type=VMProviderType.CLOUD,
57 | )
58 |
59 | # Create an agent with your preferred loop
60 | agent = ComputerAgent(
61 | model="openai/gpt-4o",
62 | save_trajectory=True,
63 | verbosity=logging.INFO,
64 | tools=[computer]
65 | )
66 |
67 | # Run a task
68 | async for result in agent.run("Open Chrome and search for AI news"):
69 | print(f"Response: {result.get('text')}")
70 |
71 | # Run the agent
72 | asyncio.run(run_cloud_agent())
73 | ```
74 |
75 | ### Available Tiers
76 |
77 | We're launching with **three compute tiers** to match your workload needs:
78 |
79 | - **Small** (1 vCPU, 4GB RAM) - Perfect for simple automation tasks and testing
80 | - **Medium** (2 vCPU, 8GB RAM) - Ideal for most production workloads
81 | - **Large** (8 vCPU, 32GB RAM) - Built for complex, resource-intensive operations
82 |
83 | Each tier includes a **full Linux with Xfce desktop environment** with pre-configured browser, **secure VNC access** with SSL, persistent storage during your session, and automatic cleanup on termination for sandboxes.
84 |
85 | ## How some customers are using Cua Cloud today
86 |
87 | ### Example 1: Automated GitHub Workflow
88 |
89 | Let's automate a complete GitHub workflow:
90 |
91 | ```python
92 | import asyncio
93 | import os
94 | from computer import Computer, VMProviderType
95 | from agent import ComputerAgent
96 |
97 | async def github_automation():
98 | """Automate GitHub repository management tasks."""
99 | computer = Computer(
100 | os_type="linux",
101 | api_key=os.getenv("CUA_API_KEY"),
102 | name="github-automation",
103 | provider_type=VMProviderType.CLOUD,
104 | )
105 |
106 | agent = ComputerAgent(
107 | model="openai/gpt-4o",
108 | save_trajectory=True,
109 | verbosity=logging.INFO,
110 | tools=[computer]
111 | )
112 |
113 | tasks = [
114 | "Look for a repository named trycua/cua on GitHub.",
115 | "Check the open issues, open the most recent one and read it.",
116 | "Clone the repository if it doesn't exist yet.",
117 | "Create a new branch for the issue.",
118 | "Make necessary changes to resolve the issue.",
119 | "Commit the changes with a descriptive message.",
120 | "Create a pull request."
121 | ]
122 |
123 | for i, task in enumerate(tasks):
124 | print(f"\nExecuting task {i+1}/{len(tasks)}: {task}")
125 | async for result in agent.run(task):
126 | print(f"Response: {result.get('text')}")
127 |
128 | # Check if any tools were used
129 | tools = result.get('tools')
130 | if tools:
131 | print(f"Tools used: {tools}")
132 |
133 | print(f"Task {i+1} completed")
134 |
135 | # Run the automation
136 | asyncio.run(github_automation())
137 | ```
138 |
139 | ### Example 2: Parallel Web Scraping
140 |
141 | Run multiple agents in parallel to scrape different websites:
142 |
143 | ```python
144 | import asyncio
145 | from computer import Computer, VMProviderType
146 | from agent import ComputerAgent
147 |
148 | async def scrape_website(site_name, url):
149 | """Scrape a website using a cloud agent."""
150 | computer = Computer(
151 | os_type="linux",
152 | api_key=os.getenv("CUA_API_KEY"),
153 | name=f"scraper-{site_name}",
154 | provider_type=VMProviderType.CLOUD,
155 | )
156 |
157 | agent = ComputerAgent(
158 | model="openai/gpt-4o",
159 | save_trajectory=True,
160 | tools=[computer]
161 | )
162 |
163 | results = []
164 | tasks = [
165 | f"Navigate to {url}",
166 | "Extract the main headlines or article titles",
167 | "Take a screenshot of the page",
168 | "Save the extracted data to a file"
169 | ]
170 |
171 | for task in tasks:
172 | async for result in agent.run(task):
173 | results.append({
174 | 'site': site_name,
175 | 'task': task,
176 | 'response': result.get('text')
177 | })
178 |
179 | return results
180 |
181 | async def parallel_scraping():
182 | """Scrape multiple websites in parallel."""
183 | sites = [
184 | ("ArXiv", "https://arxiv.org"),
185 | ("HackerNews", "https://news.ycombinator.com"),
186 | ("TechCrunch", "https://techcrunch.com")
187 | ]
188 |
189 | # Run all scraping tasks in parallel
190 | tasks = [scrape_website(name, url) for name, url in sites]
191 | results = await asyncio.gather(*tasks)
192 |
193 | # Process results
194 | for site_results in results:
195 | print(f"\nResults from {site_results[0]['site']}:")
196 | for result in site_results:
197 | print(f" - {result['task']}: {result['response'][:100]}...")
198 |
199 | # Run parallel scraping
200 | asyncio.run(parallel_scraping())
201 | ```
202 |
203 | ## Cost Optimization Tips
204 |
205 | To optimize your costs, use appropriate sandbox sizes for your workload and implement timeouts to prevent runaway tasks. Batch related operations together to minimize sandbox spin-up time, and always remember to terminate sandboxes when your work is complete.
206 |
207 | ## Security Considerations
208 |
209 | Cua Cloud runs all sandboxes in isolated environments with encrypted VNC connections. Your API keys are never exposed in trajectories.
210 |
211 | ## What's Next for Cua Cloud
212 |
213 | We're just getting started! Here's what's coming in the next few months:
214 |
215 | ### Elastic Autoscaled Sandbox Pools
216 |
217 | Soon you'll be able to create elastic sandbox pools that automatically scale based on demand. Define minimum and maximum sandbox counts, and let Cua Cloud handle the rest. Perfect for batch processing, scheduled automations, and handling traffic spikes without manual intervention.
218 |
219 | ### Windows and macOS Cloud Support
220 |
221 | While we're launching with Linux sandboxes, Windows and macOS cloud machines are coming soon. Run Windows-specific automations, test cross-platform workflows, or leverage macOS-exclusive applications – all in the cloud with the same simple API.
222 |
223 | Stay tuned for updates and join our [**Discord**](https://discord.gg/cua-ai) to vote on which features you'd like to see first!
224 |
225 | ## Get Started Today
226 |
227 | Ready to deploy your Computer-Use Agents in the cloud?
228 |
229 | Visit [**cua.ai**](https://cua.ai) to sign up and get your API key. Join our [**Discord community**](https://discord.gg/cua-ai) for support and explore more examples on [**GitHub**](https://github.com/trycua/cua).
230 |
231 | Happy RPA 2.0! 🚀
232 |
```
--------------------------------------------------------------------------------
/blog/app-use.md:
--------------------------------------------------------------------------------
```markdown
1 | # App-Use: Control Individual Applications with Cua Agents
2 |
3 | _Published on May 31, 2025 by The Cua Team_
4 |
5 | Today, we are excited to introduce a new experimental feature landing in the [Cua GitHub repository](https://github.com/trycua/cua): **App-Use**. App-Use allows you to create lightweight virtual desktops that limit agent access to specific applications, improving precision of your agent's trajectory. Perfect for parallel workflows, and focused task execution.
6 |
7 | > **Note:** App-Use is currently experimental. To use it, you need to enable it by passing `experiments=["app-use"]` feature flag when creating your Computer instance.
8 |
9 | Check out an example of a Cua Agent automating Cua's team Taco Bell order through the iPhone Mirroring app:
10 |
11 | <div align="center">
12 | <video src="https://github.com/user-attachments/assets/6362572e-f784-4006-aa6e-bce10991fab9" width="600" controls></video>
13 | </div>
14 |
15 | ## What is App-Use?
16 |
17 | App-Use lets you create virtual desktop sessions scoped to specific applications. Instead of giving an agent access to your entire screen, you can say "only work with Safari and Notes" or "just control the iPhone Mirroring app."
18 |
19 | ```python
20 | # Create a macOS VM with App Use experimental feature enabled
21 | computer = Computer(experiments=["app-use"])
22 |
23 | # Create a desktop limited to specific apps
24 | desktop = computer.create_desktop_from_apps(["Safari", "Notes"])
25 |
26 | # Your agent can now only see and interact with these apps
27 | agent = ComputerAgent(
28 | model="anthropic/claude-sonnet-4-5-20250929",
29 | tools=[desktop]
30 | )
31 | ```
32 |
33 | ## Key Benefits
34 |
35 | ### 1. Lightweight and Fast
36 |
37 | App-Use creates visual filters, not new processes. Your apps continue running normally - we just control what the agent can see and click on. The virtual desktops are composited views that require no additional compute resources beyond the existing window manager operations.
38 |
39 | ### 2. Run Multiple Agents in Parallel
40 |
41 | Deploy a team of specialized agents, each focused on their own apps:
42 |
43 | ```python
44 | # Create a Computer with App Use enabled
45 | computer = Computer(experiments=["app-use"])
46 |
47 | # Research agent focuses on browser
48 | research_desktop = computer.create_desktop_from_apps(["Safari"])
49 | research_agent = ComputerAgent(tools=[research_desktop], ...)
50 |
51 | # Writing agent focuses on documents
52 | writing_desktop = computer.create_desktop_from_apps(["Pages", "Notes"])
53 | writing_agent = ComputerAgent(tools=[writing_desktop], ...)
54 |
55 | async def run_agent(agent, task):
56 | async for result in agent.run(task):
57 | print(result.get('text', ''))
58 |
59 | # Run both simultaneously
60 | await asyncio.gather(
61 | run_agent(research_agent, "Research AI trends for 2025"),
62 | run_agent(writing_agent, "Draft blog post outline")
63 | )
64 | ```
65 |
66 | ## How To: Getting Started with App-Use
67 |
68 | ### Requirements
69 |
70 | To get started with App-Use, you'll need:
71 |
72 | - Python 3.11+
73 | - macOS Sequoia (15.0) or later
74 |
75 | ### Getting Started
76 |
77 | ```bash
78 | # Install packages and launch UI
79 | pip install -U "cua-computer[all]" "cua-agent[all]"
80 | python -m agent.ui.gradio.app
81 | ```
82 |
83 | ```python
84 | import asyncio
85 | from computer import Computer
86 | from agent import ComputerAgent
87 |
88 | async def main():
89 | computer = Computer()
90 | await computer.run()
91 |
92 | # Create app-specific desktop sessions
93 | desktop = computer.create_desktop_from_apps(["Notes"])
94 |
95 | # Initialize an agent
96 | agent = ComputerAgent(
97 | model="anthropic/claude-sonnet-4-5-20250929",
98 | tools=[desktop]
99 | )
100 |
101 | # Take a screenshot (returns bytes by default)
102 | screenshot = await desktop.interface.screenshot()
103 | with open("app_screenshot.png", "wb") as f:
104 | f.write(screenshot)
105 |
106 | # Run an agent task
107 | async for result in agent.run("Create a new note titled 'Meeting Notes' and add today's agenda items"):
108 | print(f"Agent: {result.get('text', '')}")
109 |
110 | if __name__ == "__main__":
111 | asyncio.run(main())
112 | ```
113 |
114 | ## Use Case: Automating Your iPhone with Cua
115 |
116 | ### ⚠️ Important Warning
117 |
118 | Computer-use agents are powerful tools that can interact with your devices. This guide involves using your own macOS and iPhone instead of a VM. **Proceed at your own risk.** Always:
119 |
120 | - Review agent actions before running
121 | - Start with non-critical tasks
122 | - Monitor agent behavior closely
123 |
124 | Remember with Cua it is still advised to use a VM for a better level of isolation for your agents.
125 |
126 | ### Setting Up iPhone Automation
127 |
128 | ### Step 1: Start the cua-computer-server
129 |
130 | First, you'll need to start the cua-computer-server locally to enable access to iPhone Mirroring via the Computer interface:
131 |
132 | ```bash
133 | # Install the server
134 | pip install cua-computer-server
135 |
136 | # Start the server
137 | python -m computer_server
138 | ```
139 |
140 | ### Step 2: Connect iPhone Mirroring
141 |
142 | Then, you'll need to open the "iPhone Mirroring" app on your Mac and connect it to your iPhone.
143 |
144 | ### Step 3: Create an iPhone Automation Session
145 |
146 | Finally, you can create an iPhone automation session:
147 |
148 | ```python
149 | import asyncio
150 | from computer import Computer
151 | from cua_agent import Agent
152 |
153 | async def automate_iphone():
154 | # Connect to your local computer server
155 | my_mac = Computer(use_host_computer_server=True, os_type="macos", experiments=["app-use"])
156 | await my_mac.run()
157 |
158 | # Create a desktop focused on iPhone Mirroring
159 | my_iphone = my_mac.create_desktop_from_apps(["iPhone Mirroring"])
160 |
161 | # Initialize an agent for iPhone automation
162 | agent = ComputerAgent(
163 | model="anthropic/claude-sonnet-4-5-20250929",
164 | tools=[my_iphone]
165 | )
166 |
167 | # Example: Send a message
168 | async for result in agent.run("Open Messages and send 'Hello from Cua!' to John"):
169 | print(f"Agent: {result.get('text', '')}")
170 |
171 | # Example: Set a reminder
172 | async for result in agent.run("Create a reminder to call mom at 5 PM today"):
173 | print(f"Agent: {result.get('text', '')}")
174 |
175 | if __name__ == "__main__":
176 | asyncio.run(automate_iphone())
177 | ```
178 |
179 | ### iPhone Automation Use Cases
180 |
181 | With Cua's iPhone automation, you can:
182 |
183 | - **Automate messaging**: Send texts, respond to messages, manage conversations
184 | - **Control apps**: Navigate any iPhone app using natural language
185 | - **Manage settings**: Adjust iPhone settings programmatically
186 | - **Extract data**: Read information from apps that don't have APIs
187 | - **Test iOS apps**: Automate testing workflows for iPhone applications
188 |
189 | ## Important Notes
190 |
191 | - **Visual isolation only**: Apps share the same files, OS resources, and user session
192 | - **Dynamic resolution**: Desktops automatically scale to fit app windows and menu bars
193 | - **macOS only**: Currently requires macOS due to compositing engine dependencies
194 | - **Not a security boundary**: This is for agent focus, not security isolation
195 |
196 | ## When to Use What: App-Use vs Multiple Cua Containers
197 |
198 | ### Use App-Use within the same macOS Cua Container:
199 |
200 | - ✅ You need lightweight, fast agent focusing (macOS only)
201 | - ✅ You want to run multiple agents on one desktop
202 | - ✅ You're automating personal devices like iPhones
203 | - ✅ Window layout isolation is sufficient
204 | - ✅ You want low computational overhead
205 |
206 | ### Use Multiple Cua Containers:
207 |
208 | - ✅ You need maximum isolation between agents
209 | - ✅ You require cross-platform support (Mac/Linux/Windows)
210 | - ✅ You need guaranteed resource allocation
211 | - ✅ Security and complete isolation are critical
212 | - ⚠️ Note: Most computationally expensive option
213 |
214 | ## Pro Tips
215 |
216 | 1. **Start Small**: Test with one app before creating complex multi-app desktops
217 | 2. **Screenshot First**: Take a screenshot to verify your desktop shows the right apps
218 | 3. **Name Your Apps Correctly**: Use exact app names as they appear in the system
219 | 4. **Consider Performance**: While lightweight, too many parallel agents can still impact system performance
220 | 5. **Plan Your Workflows**: Design agent tasks to minimize app switching for best results
221 |
222 | ### How It Works
223 |
224 | When you create a desktop session with `create_desktop_from_apps()`, App Use:
225 |
226 | - Filters the visual output to show only specified application windows
227 | - Routes input events only to those applications
228 | - Maintains window layout isolation between different sessions
229 | - Shares the underlying file system and OS resources
230 | - **Dynamically adjusts resolution** to fit the window layout and menu bar items
231 |
232 | The resolution of these virtual desktops is dynamic, automatically scaling to accommodate the applications' window sizes and menu bar requirements. This ensures that agents always have a clear view of the entire interface they need to interact with, regardless of the specific app combination.
233 |
234 | Currently, App Use is limited to macOS only due to its reliance on Quartz, Apple's powerful compositing engine, for creating these virtual desktops. Quartz provides the low-level window management and rendering capabilities that make it possible to composite multiple application windows into isolated visual environments.
235 |
236 | ## Conclusion
237 |
238 | App Use brings a new dimension to computer automation - lightweight, focused, and parallel. Whether you're building a personal iPhone assistant or orchestrating a team of specialized agents, App Use provides the perfect balance of functionality and efficiency.
239 |
240 | Ready to try it? Update to the latest Cua version and start focusing your agents today!
241 |
242 | ```bash
243 | pip install -U "cua-computer[all]" "cua-agent[all]"
244 | ```
245 |
246 | Happy automating! 🎯🤖
247 |
```
--------------------------------------------------------------------------------
/libs/kasm/src/ubuntu/install/firefox/install_firefox.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/usr/bin/env bash
2 | set -xe
3 |
4 | # Add icon
5 | if [ -f /dockerstartup/install/ubuntu/install/firefox/firefox.desktop ]; then
6 | mv /dockerstartup/install/ubuntu/install/firefox/firefox.desktop $HOME/Desktop/
7 | fi
8 |
9 | ARCH=$(arch | sed 's/aarch64/arm64/g' | sed 's/x86_64/amd64/g')
10 |
11 | set_desktop_icon() {
12 | sed -i -e 's!Icon=.\+!Icon=/usr/share/icons/hicolor/48x48/apps/firefox.png!' "$HOME/Desktop/firefox.desktop"
13 | }
14 |
15 | echo "Install Firefox"
16 | if [[ "${DISTRO}" == @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|fedora39|fedora40) ]]; then
17 | dnf install -y firefox p11-kit
18 | elif [ "${DISTRO}" == "opensuse" ]; then
19 | zypper install -yn p11-kit-tools MozillaFirefox
20 | elif grep -q Jammy /etc/os-release || grep -q Noble /etc/os-release; then
21 | if [ ! -f '/etc/apt/preferences.d/mozilla-firefox' ]; then
22 | add-apt-repository -y ppa:mozillateam/ppa
23 | echo '
24 | Package: *
25 | Pin: release o=LP-PPA-mozillateam
26 | Pin-Priority: 1001
27 | ' > /etc/apt/preferences.d/mozilla-firefox
28 | fi
29 | apt-get install -y firefox p11-kit-modules
30 | elif grep -q "ID=kali" /etc/os-release; then
31 | apt-get update
32 | apt-get install -y firefox-esr p11-kit-modules
33 | rm -f $HOME/Desktop/firefox.desktop
34 | cp \
35 | /usr/share/applications/firefox-esr.desktop \
36 | $HOME/Desktop/
37 | chmod +x $HOME/Desktop/firefox-esr.desktop
38 | elif grep -q "ID=debian" /etc/os-release || grep -q "ID=parrot" /etc/os-release; then
39 | if [ "${ARCH}" == "amd64" ]; then
40 | install -d -m 0755 /etc/apt/keyrings
41 | wget -q https://packages.mozilla.org/apt/repo-signing-key.gpg -O- > /etc/apt/keyrings/packages.mozilla.org.asc
42 | echo "deb [signed-by=/etc/apt/keyrings/packages.mozilla.org.asc] https://packages.mozilla.org/apt mozilla main" > /etc/apt/sources.list.d/mozilla.list
43 | echo '
44 | Package: *
45 | Pin: origin packages.mozilla.org
46 | Pin-Priority: 1000
47 | ' > /etc/apt/preferences.d/mozilla
48 | apt-get update
49 | apt-get install -y firefox p11-kit-modules
50 | else
51 | apt-get update
52 | apt-get install -y firefox-esr p11-kit-modules
53 | rm -f $HOME/Desktop/firefox.desktop
54 | cp \
55 | /usr/share/applications/firefox-esr.desktop \
56 | $HOME/Desktop/
57 | chmod +x $HOME/Desktop/firefox-esr.desktop
58 | fi
59 | else
60 | apt-mark unhold firefox || :
61 | apt-get remove firefox
62 | apt-get update
63 | apt-get install -y firefox p11-kit-modules
64 | fi
65 |
66 | # Add Langpacks
67 | FIREFOX_VERSION=$(curl -sI https://download.mozilla.org/?product=firefox-latest | awk -F '(releases/|/win32)' '/Location/ {print $2}')
68 | RELEASE_URL="https://releases.mozilla.org/pub/firefox/releases/${FIREFOX_VERSION}/win64/xpi/"
69 | LANGS=$(curl -Ls ${RELEASE_URL} | awk -F '(xpi">|</a>)' '/href.*xpi/ {print $2}' | tr '\n' ' ')
70 | EXTENSION_DIR=/usr/lib/firefox-addons/distribution/extensions/
71 | mkdir -p ${EXTENSION_DIR}
72 | for LANG in ${LANGS}; do
73 | LANGCODE=$(echo ${LANG} | sed 's/\.xpi//g')
74 | echo "Downloading ${LANG} Language pack"
75 | curl -o \
76 | ${EXTENSION_DIR}langpack-${LANGCODE}@firefox.mozilla.org.xpi -Ls \
77 | ${RELEASE_URL}${LANG}
78 | done
79 |
80 | # Cleanup and install flash if supported
81 | if [[ "${DISTRO}" == @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|fedora39|fedora40) ]]; then
82 | if [ -z ${SKIP_CLEAN+x} ]; then
83 | dnf clean all
84 | fi
85 | elif [ "${DISTRO}" == "opensuse" ]; then
86 | if [ -z ${SKIP_CLEAN+x} ]; then
87 | zypper clean --all
88 | fi
89 | else
90 | if [ "$ARCH" == "arm64" ] && [ "$(lsb_release -cs)" == "focal" ] ; then
91 | echo "Firefox flash player not supported on arm64 Ubuntu Focal Skipping"
92 | elif grep -q "ID=debian" /etc/os-release || grep -q "ID=kali" /etc/os-release || grep -q "ID=parrot" /etc/os-release; then
93 | echo "Firefox flash player not supported on Debian"
94 | elif grep -q Focal /etc/os-release; then
95 | # Plugin to support running flash videos for sites like vimeo
96 | apt-get update
97 | apt-get install -y browser-plugin-freshplayer-pepperflash
98 | apt-mark hold firefox
99 | if [ -z ${SKIP_CLEAN+x} ]; then
100 | apt-get autoclean
101 | rm -rf \
102 | /var/lib/apt/lists/* \
103 | /var/tmp/*
104 | fi
105 | fi
106 | fi
107 |
108 | if [[ "${DISTRO}" != @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|opensuse|fedora39|fedora40) ]]; then
109 | # Update firefox to utilize the system certificate store instead of the one that ships with firefox
110 | if grep -q "ID=debian" /etc/os-release || grep -q "ID=kali" /etc/os-release || grep -q "ID=parrot" /etc/os-release && [ "${ARCH}" == "arm64" ]; then
111 | rm -f /usr/lib/firefox-esr/libnssckbi.so
112 | ln /usr/lib/$(arch)-linux-gnu/pkcs11/p11-kit-trust.so /usr/lib/firefox-esr/libnssckbi.so
113 | elif grep -q "ID=kali" /etc/os-release && [ "${ARCH}" == "amd64" ]; then
114 | rm -f /usr/lib/firefox-esr/libnssckbi.so
115 | ln /usr/lib/$(arch)-linux-gnu/pkcs11/p11-kit-trust.so /usr/lib/firefox-esr/libnssckbi.so
116 | else
117 | rm -f /usr/lib/firefox/libnssckbi.so
118 | ln /usr/lib/$(arch)-linux-gnu/pkcs11/p11-kit-trust.so /usr/lib/firefox/libnssckbi.so
119 | fi
120 | fi
121 |
122 | if [[ "${DISTRO}" == @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|fedora39|fedora40) ]]; then
123 | if [[ "${DISTRO}" == @(fedora39|fedora40) ]]; then
124 | preferences_file=/usr/lib64/firefox/browser/defaults/preferences/firefox-redhat-default-prefs.js
125 | else
126 | preferences_file=/usr/lib64/firefox/browser/defaults/preferences/all-redhat.js
127 | fi
128 | sed -i -e '/homepage/d' "$preferences_file"
129 | elif [ "${DISTRO}" == "opensuse" ]; then
130 | preferences_file=/usr/lib64/firefox/browser/defaults/preferences/firefox.js
131 | elif grep -q "ID=kali" /etc/os-release; then
132 | preferences_file=/usr/lib/firefox-esr/defaults/pref/firefox.js
133 | elif grep -q "ID=debian" /etc/os-release || grep -q "ID=parrot" /etc/os-release; then
134 | if [ "${ARCH}" == "amd64" ]; then
135 | preferences_file=/usr/lib/firefox/defaults/pref/firefox.js
136 | else
137 | preferences_file=/usr/lib/firefox-esr/defaults/pref/firefox.js
138 | fi
139 | else
140 | preferences_file=/usr/lib/firefox/browser/defaults/preferences/firefox.js
141 | fi
142 |
143 | # Disabling default first run URL for Debian based images
144 | if [[ "${DISTRO}" != @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|opensuse|fedora39|fedora40) ]]; then
145 | cat >"$preferences_file" <<EOF
146 | pref("datareporting.policy.firstRunURL", "");
147 | pref("datareporting.policy.dataSubmissionEnabled", false);
148 | pref("datareporting.healthreport.service.enabled", false);
149 | pref("datareporting.healthreport.uploadEnabled", false);
150 | pref("trailhead.firstrun.branches", "nofirstrun-empty");
151 | pref("browser.aboutwelcome.enabled", false);
152 | EOF
153 | fi
154 |
155 | if [[ "${DISTRO}" == @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|opensuse|fedora39|fedora40) ]]; then
156 | # Creating a default profile
157 | chown -R root:root $HOME
158 | firefox -headless -CreateProfile "kasm $HOME/.mozilla/firefox/kasm"
159 | # Generate a certdb to be detected on squid start
160 | HOME=/root firefox --headless &
161 | mkdir -p /root/.mozilla
162 | CERTDB=$(find /root/.mozilla* -name "cert9.db")
163 | while [ -z "${CERTDB}" ] ; do
164 | sleep 1
165 | echo "waiting for certdb"
166 | CERTDB=$(find /root/.mozilla* -name "cert9.db")
167 | done
168 | sleep 2
169 | kill $(pgrep firefox)
170 | CERTDIR=$(dirname ${CERTDB})
171 | mv ${CERTDB} $HOME/.mozilla/firefox/kasm/
172 | rm -Rf /root/.mozilla
173 | else
174 | # Creating Default Profile
175 | chown -R 0:0 $HOME
176 | firefox -headless -CreateProfile "kasm $HOME/.mozilla/firefox/kasm"
177 | fi
178 |
179 | # Silence Firefox security nag "Some of Firefox's features may offer less protection on your current operating system".
180 | echo 'user_pref("security.sandbox.warn_unprivileged_namespaces", false);' > $HOME/.mozilla/firefox/kasm/user.js
181 | chown 1000:1000 $HOME/.mozilla/firefox/kasm/user.js
182 |
183 | if [[ "${DISTRO}" == @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|opensuse|fedora39|fedora40) ]]; then
184 | set_desktop_icon
185 | fi
186 |
187 | # Starting with version 67, Firefox creates a unique profile mapping per installation which is hash generated
188 | # based off the installation path. Because that path will be static for our deployments we can assume the hash
189 | # and thus assign our profile to the default for the installation
190 | if grep -q "ID=kali" /etc/os-release; then
191 | cat >>$HOME/.mozilla/firefox/profiles.ini <<EOL
192 | [Install3B6073811A6ABF12]
193 | Default=kasm
194 | Locked=1
195 | EOL
196 | elif grep -q "ID=debian" /etc/os-release || grep -q "ID=parrot" /etc/os-release; then
197 | if [ "${ARCH}" != "amd64" ]; then
198 | cat >>$HOME/.mozilla/firefox/profiles.ini <<EOL
199 | [Install3B6073811A6ABF12]
200 | Default=kasm
201 | Locked=1
202 | EOL
203 | else
204 | cat >>$HOME/.mozilla/firefox/profiles.ini <<EOL
205 | [Install4F96D1932A9F858E]
206 | Default=kasm
207 | Locked=1
208 | EOL
209 | fi
210 | elif [[ "${DISTRO}" != @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|opensuse|fedora39|fedora40) ]]; then
211 | cat >>$HOME/.mozilla/firefox/profiles.ini <<EOL
212 | [Install4F96D1932A9F858E]
213 | Default=kasm
214 | Locked=1
215 | EOL
216 | elif [[ "${DISTRO}" == @(oracle8|rockylinux9|rockylinux8|oracle9|rhel9|almalinux9|almalinux8|opensuse|fedora39|fedora40) ]]; then
217 | cat >>$HOME/.mozilla/firefox/profiles.ini <<EOL
218 | [Install11457493C5A56847]
219 | Default=kasm
220 | Locked=1
221 | EOL
222 | fi
223 |
224 | # Desktop Icon FIxes
225 | if [[ "${DISTRO}" == @(rockylinux9|oracle9|rhel9|almalinux9|fedora39|fedora40) ]]; then
226 | sed -i 's#Icon=/usr/lib/firefox#Icon=/usr/lib64/firefox#g' $HOME/Desktop/firefox.desktop
227 | fi
228 |
229 | # Cleanup for app layer
230 | chown -R 1000:0 $HOME
231 | find /usr/share/ -name "icon-theme.cache" -exec rm -f {} \;
232 | if [ -f $HOME/Desktop/firefox.desktop ]; then
233 | chmod +x $HOME/Desktop/firefox.desktop
234 | fi
235 | chown -R 1000:1000 $HOME/.mozilla
236 |
237 |
```
--------------------------------------------------------------------------------
/libs/python/agent/agent/proxy/handlers.py:
--------------------------------------------------------------------------------
```python
1 | """
2 | Request handlers for the proxy endpoints.
3 | """
4 |
5 | import asyncio
6 | import json
7 | import logging
8 | import os
9 | from contextlib import contextmanager
10 | from typing import Any, Dict, List, Optional, Union
11 |
12 | from computer import Computer
13 |
14 | from ..agent import ComputerAgent
15 |
16 | logger = logging.getLogger(__name__)
17 |
18 |
19 | class ResponsesHandler:
20 | """Handler for /responses endpoint that processes agent requests."""
21 |
22 | def __init__(self):
23 | self.computer = None
24 | self.agent = None
25 | # Simple in-memory caches
26 | self._computer_cache: Dict[str, Any] = {}
27 | self._agent_cache: Dict[str, Any] = {}
28 |
29 | async def setup_computer_agent(
30 | self,
31 | model: str,
32 | agent_kwargs: Optional[Dict[str, Any]] = None,
33 | computer_kwargs: Optional[Dict[str, Any]] = None,
34 | ):
35 | """Set up (and cache) computer and agent instances.
36 |
37 | Caching keys:
38 | - Computer cache key: computer_kwargs
39 | - Agent cache key: {"model": model, **agent_kwargs}
40 | """
41 | agent_kwargs = agent_kwargs or {}
42 | computer_kwargs = computer_kwargs or {}
43 |
44 | def _stable_key(obj: Dict[str, Any]) -> str:
45 | try:
46 | return json.dumps(obj, sort_keys=True, separators=(",", ":"))
47 | except Exception:
48 | # Fallback: stringify non-serializable values
49 | safe_obj = {}
50 | for k, v in obj.items():
51 | try:
52 | json.dumps(v)
53 | safe_obj[k] = v
54 | except Exception:
55 | safe_obj[k] = str(v)
56 | return json.dumps(safe_obj, sort_keys=True, separators=(",", ":"))
57 |
58 | # Determine if custom tools are supplied; if so, skip computer setup entirely
59 | has_custom_tools = bool(agent_kwargs.get("tools"))
60 |
61 | computer = None
62 | if not has_custom_tools:
63 | # ---------- Computer setup (with cache) ----------
64 | comp_key = _stable_key(computer_kwargs)
65 |
66 | computer = self._computer_cache.get(comp_key)
67 | if computer is None:
68 | # Default computer configuration
69 | default_c_config = {
70 | "os_type": "linux",
71 | "provider_type": "cloud",
72 | "name": os.getenv("CUA_CONTAINER_NAME"),
73 | "api_key": os.getenv("CUA_API_KEY"),
74 | }
75 | default_c_config.update(computer_kwargs)
76 | computer = Computer(**default_c_config)
77 | await computer.__aenter__()
78 | self._computer_cache[comp_key] = computer
79 | logger.info(
80 | f"Computer created and cached with key={comp_key} config={default_c_config}"
81 | )
82 | else:
83 | logger.info(f"Reusing cached computer for key={comp_key}")
84 |
85 | # Bind current computer reference (None if custom tools supplied)
86 | self.computer = computer
87 |
88 | # ---------- Agent setup (with cache) ----------
89 | # Build agent cache key from {model} + agent_kwargs (excluding tools unless explicitly passed)
90 | agent_kwargs_for_key = dict(agent_kwargs)
91 | agent_key_payload = {"model": model, **agent_kwargs_for_key}
92 | agent_key = _stable_key(agent_key_payload)
93 |
94 | agent = self._agent_cache.get(agent_key)
95 | if agent is None:
96 | # Default agent configuration
97 | default_a_config: Dict[str, Any] = {"model": model}
98 | if not has_custom_tools:
99 | default_a_config["tools"] = [computer]
100 | # Apply user overrides, but keep tools unless user explicitly sets
101 | if agent_kwargs:
102 | if not has_custom_tools:
103 | agent_kwargs.setdefault("tools", [computer])
104 | default_a_config.update(agent_kwargs)
105 | # JSON-derived kwargs may have loose types; ignore static arg typing here
106 | agent = ComputerAgent(**default_a_config) # type: ignore[arg-type]
107 | self._agent_cache[agent_key] = agent
108 | logger.info(f"Agent created and cached with key={agent_key} model={model}")
109 | else:
110 | # Ensure cached agent uses the current computer tool (in case object differs)
111 | # Only update if tools not explicitly provided in agent_kwargs
112 | if not has_custom_tools:
113 | try:
114 | agent.tools = [computer]
115 | except Exception:
116 | pass
117 | logger.info(f"Reusing cached agent for key={agent_key}")
118 |
119 | # Bind current agent reference
120 | self.agent = agent
121 |
122 | async def process_request(self, request_data: Dict[str, Any]) -> Dict[str, Any]:
123 | """
124 | Process a /responses request and return the result.
125 |
126 | Args:
127 | request_data: Dictionary containing model, input, and optional kwargs
128 |
129 | Returns:
130 | Dictionary with the agent's response
131 | """
132 | try:
133 | # Extract request parameters
134 | model = request_data.get("model")
135 | input_data = request_data.get("input")
136 | agent_kwargs = request_data.get("agent_kwargs", {})
137 | computer_kwargs = request_data.get("computer_kwargs", {})
138 | env_overrides = request_data.get("env", {}) or {}
139 |
140 | if not model:
141 | raise ValueError("Model is required")
142 | if not input_data:
143 | raise ValueError("Input is required")
144 |
145 | # Apply env overrides for the duration of this request
146 | with self._env_overrides(env_overrides):
147 | # Set up (and possibly reuse) computer and agent via caches
148 | await self.setup_computer_agent(model, agent_kwargs, computer_kwargs)
149 |
150 | # Defensive: ensure agent is initialized for type checkers
151 | agent = self.agent
152 | if agent is None:
153 | raise RuntimeError("Agent failed to initialize")
154 |
155 | # Convert input to messages format
156 | messages = self._convert_input_to_messages(input_data)
157 |
158 | # Run agent and get first result
159 | async for result in agent.run(messages):
160 | # Return the first result and break
161 | return {"success": True, "result": result, "model": model}
162 |
163 | # If no results were yielded
164 | return {"success": False, "error": "No results from agent", "model": model}
165 |
166 | except Exception as e:
167 | logger.error(f"Error processing request: {e}")
168 | return {
169 | "success": False,
170 | "error": str(e),
171 | "model": request_data.get("model", "unknown"),
172 | }
173 |
174 | def _convert_input_to_messages(
175 | self, input_data: Union[str, List[Dict[str, Any]]]
176 | ) -> List[Dict[str, Any]]:
177 | """Convert input data to messages format."""
178 | if isinstance(input_data, str):
179 | # Simple string input
180 | return [{"role": "user", "content": input_data}]
181 | elif isinstance(input_data, list):
182 | # Already in messages format
183 | messages = []
184 | for msg in input_data:
185 | # Convert content array format if needed
186 | if isinstance(msg.get("content"), list):
187 | content_parts = []
188 | for part in msg["content"]:
189 | if part.get("type") == "input_text":
190 | content_parts.append({"type": "text", "text": part["text"]})
191 | elif part.get("type") == "input_image":
192 | content_parts.append(
193 | {"type": "image_url", "image_url": {"url": part["image_url"]}}
194 | )
195 | else:
196 | content_parts.append(part)
197 | messages.append({"role": msg["role"], "content": content_parts})
198 | else:
199 | messages.append(msg)
200 | return messages
201 | else:
202 | raise ValueError("Input must be string or list of messages")
203 |
204 | async def cleanup(self):
205 | """Clean up resources."""
206 | if self.computer:
207 | try:
208 | await self.computer.__aexit__(None, None, None)
209 | except Exception as e:
210 | logger.error(f"Error cleaning up computer: {e}")
211 | finally:
212 | self.computer = None
213 | self.agent = None
214 |
215 | @staticmethod
216 | @contextmanager
217 | def _env_overrides(env: Dict[str, str]):
218 | """Temporarily apply environment variable overrides for the current process.
219 | Restores previous values after the context exits.
220 |
221 | Args:
222 | env: Mapping of env var names to override for this request.
223 | """
224 | if not env:
225 | # No-op context
226 | yield
227 | return
228 |
229 | original: Dict[str, Optional[str]] = {}
230 | try:
231 | for k, v in env.items():
232 | original[k] = os.environ.get(k)
233 | os.environ[k] = str(v)
234 | yield
235 | finally:
236 | for k, old in original.items():
237 | if old is None:
238 | # Was not set before
239 | os.environ.pop(k, None)
240 | else:
241 | os.environ[k] = old
242 |
```
--------------------------------------------------------------------------------
/libs/python/computer-server/test_connection.py:
--------------------------------------------------------------------------------
```python
1 | #!/usr/bin/env python
2 | """
3 | Connection test script for Computer Server.
4 |
5 | This script tests both WebSocket (/ws) and REST (/cmd) connections to the Computer Server
6 | and keeps it alive, allowing you to verify the server is running correctly.
7 | """
8 |
9 | import argparse
10 | import asyncio
11 | import json
12 | import os
13 | import sys
14 |
15 | import aiohttp
16 | import dotenv
17 | import websockets
18 |
19 | dotenv.load_dotenv()
20 |
21 |
22 | async def test_websocket_connection(
23 | host="localhost", port=8000, keep_alive=False, container_name=None, api_key=None
24 | ):
25 | """Test WebSocket connection to the Computer Server."""
26 | if container_name:
27 | # Container mode: use WSS with container domain and port 8443
28 | uri = f"wss://{container_name}.containers.cloud.trycua.com:8443/ws"
29 | print(f"Connecting to container {container_name} at {uri}...")
30 | else:
31 | # Local mode: use WS with specified host and port
32 | uri = f"ws://{host}:{port}/ws"
33 | print(f"Connecting to local server at {uri}...")
34 |
35 | try:
36 | async with websockets.connect(uri) as websocket:
37 | print("WebSocket connection established!")
38 |
39 | # If container connection, send authentication first
40 | if container_name:
41 | if not api_key:
42 | print("Error: API key required for container connections")
43 | return False
44 |
45 | print("Sending authentication...")
46 | auth_message = {
47 | "command": "authenticate",
48 | "params": {"api_key": api_key, "container_name": container_name},
49 | }
50 | await websocket.send(json.dumps(auth_message))
51 | auth_response = await websocket.recv()
52 | print(f"Authentication response: {auth_response}")
53 |
54 | # Check if authentication was successful
55 | auth_data = json.loads(auth_response)
56 | if not auth_data.get("success", False):
57 | print("Authentication failed!")
58 | return False
59 | print("Authentication successful!")
60 |
61 | # Send a test command to get version
62 | await websocket.send(json.dumps({"command": "version", "params": {}}))
63 | response = await websocket.recv()
64 | print(f"Version response: {response}")
65 |
66 | # Send a test command to get screen size
67 | await websocket.send(json.dumps({"command": "get_screen_size", "params": {}}))
68 | response = await websocket.recv()
69 | print(f"Screen size response: {response}")
70 |
71 | if keep_alive:
72 | print("\nKeeping WebSocket connection alive. Press Ctrl+C to exit...")
73 | while True:
74 | # Send a command every 5 seconds to keep the connection alive
75 | await asyncio.sleep(5)
76 | await websocket.send(
77 | json.dumps({"command": "get_cursor_position", "params": {}})
78 | )
79 | response = await websocket.recv()
80 | print(f"Cursor position: {response}")
81 | except websockets.exceptions.ConnectionClosed as e:
82 | print(f"WebSocket connection closed: {e}")
83 | return False
84 | except ConnectionRefusedError:
85 | print(f"Connection refused. Is the server running at {host}:{port}?")
86 | return False
87 | except Exception as e:
88 | print(f"WebSocket error: {e}")
89 | return False
90 |
91 | return True
92 |
93 |
94 | async def test_rest_connection(
95 | host="localhost", port=8000, keep_alive=False, container_name=None, api_key=None
96 | ):
97 | """Test REST connection to the Computer Server."""
98 | if container_name:
99 | # Container mode: use HTTPS with container domain and port 8443
100 | base_url = f"https://{container_name}.containers.cloud.trycua.com:8443"
101 | print(f"Connecting to container {container_name} at {base_url}...")
102 | else:
103 | # Local mode: use HTTP with specified host and port
104 | base_url = f"http://{host}:{port}"
105 | print(f"Connecting to local server at {base_url}...")
106 |
107 | try:
108 | async with aiohttp.ClientSession() as session:
109 | print("REST connection established!")
110 |
111 | # Prepare headers for container authentication
112 | headers = {}
113 | if container_name:
114 | if not api_key:
115 | print("Error: API key required for container connections")
116 | return False
117 | headers["X-Container-Name"] = container_name
118 | headers["X-API-Key"] = api_key
119 | print("Using container authentication headers")
120 |
121 | # Test screenshot endpoint
122 | async with session.post(
123 | f"{base_url}/cmd", json={"command": "screenshot", "params": {}}, headers=headers
124 | ) as response:
125 | if response.status == 200:
126 | text = await response.text()
127 | print(f"Screenshot response: {text}")
128 | else:
129 | print(f"Screenshot request failed with status: {response.status}")
130 | print(await response.text())
131 | return False
132 |
133 | # Test screen size endpoint
134 | async with session.post(
135 | f"{base_url}/cmd",
136 | json={"command": "get_screen_size", "params": {}},
137 | headers=headers,
138 | ) as response:
139 | if response.status == 200:
140 | text = await response.text()
141 | print(f"Screen size response: {text}")
142 | else:
143 | print(f"Screen size request failed with status: {response.status}")
144 | print(await response.text())
145 | return False
146 |
147 | if keep_alive:
148 | print("\nKeeping REST connection alive. Press Ctrl+C to exit...")
149 | while True:
150 | # Send a command every 5 seconds to keep testing
151 | await asyncio.sleep(5)
152 | async with session.post(
153 | f"{base_url}/cmd",
154 | json={"command": "get_cursor_position", "params": {}},
155 | headers=headers,
156 | ) as response:
157 | if response.status == 200:
158 | text = await response.text()
159 | print(f"Cursor position: {text}")
160 | else:
161 | print(f"Cursor position request failed with status: {response.status}")
162 | print(await response.text())
163 | return False
164 |
165 | except aiohttp.ClientError as e:
166 | print(f"REST connection error: {e}")
167 | return False
168 | except Exception as e:
169 | print(f"REST error: {e}")
170 | return False
171 |
172 | return True
173 |
174 |
175 | async def test_connection(
176 | host="localhost", port=8000, keep_alive=False, container_name=None, use_rest=False, api_key=None
177 | ):
178 | """Test connection to the Computer Server using WebSocket or REST."""
179 | if use_rest:
180 | return await test_rest_connection(host, port, keep_alive, container_name, api_key)
181 | else:
182 | return await test_websocket_connection(host, port, keep_alive, container_name, api_key)
183 |
184 |
185 | def parse_args():
186 | parser = argparse.ArgumentParser(description="Test connection to Computer Server")
187 | parser.add_argument("--host", default="localhost", help="Host address (default: localhost)")
188 | parser.add_argument("-p", "--port", type=int, default=8000, help="Port number (default: 8000)")
189 | parser.add_argument(
190 | "-c",
191 | "--container-name",
192 | help="Container name for cloud connection (uses WSS/HTTPS and port 8443)",
193 | )
194 | parser.add_argument(
195 | "--api-key", help="API key for container authentication (can also use CUA_API_KEY env var)"
196 | )
197 | parser.add_argument("--keep-alive", action="store_true", help="Keep connection alive")
198 | parser.add_argument(
199 | "--rest", action="store_true", help="Use REST endpoint (/cmd) instead of WebSocket (/ws)"
200 | )
201 | return parser.parse_args()
202 |
203 |
204 | async def main():
205 | args = parse_args()
206 |
207 | # Convert hyphenated argument to underscore for function parameter
208 | container_name = getattr(args, "container_name", None)
209 |
210 | # Get API key from argument or environment variable
211 | api_key = getattr(args, "api_key", None) or os.environ.get("CUA_API_KEY")
212 |
213 | # Check if container name is provided but API key is missing
214 | if container_name and not api_key:
215 | print("Warning: Container name provided but no API key found.")
216 | print("Please provide --api-key argument or set CUA_API_KEY environment variable.")
217 | return 1
218 |
219 | print(f"Testing {'REST' if args.rest else 'WebSocket'} connection...")
220 | if container_name:
221 | print(f"Container: {container_name}")
222 | print(
223 | f"API Key: {'***' + api_key[-4:] if api_key and len(api_key) > 4 else 'Not provided'}"
224 | )
225 |
226 | success = await test_connection(
227 | host=args.host,
228 | port=args.port,
229 | keep_alive=args.keep_alive,
230 | container_name=container_name,
231 | use_rest=args.rest,
232 | api_key=api_key,
233 | )
234 | return 0 if success else 1
235 |
236 |
237 | if __name__ == "__main__":
238 | try:
239 | sys.exit(asyncio.run(main()))
240 | except KeyboardInterrupt:
241 | print("\nExiting...")
242 | sys.exit(0)
243 |
```
--------------------------------------------------------------------------------
/libs/python/core/tests/test_telemetry.py:
--------------------------------------------------------------------------------
```python
1 | """Unit tests for core telemetry functionality.
2 |
3 | This file tests ONLY telemetry logic, following SRP.
4 | All external dependencies (PostHog, file system) are mocked.
5 | """
6 |
7 | import os
8 | from pathlib import Path
9 | from unittest.mock import MagicMock, Mock, mock_open, patch
10 |
11 | import pytest
12 |
13 |
14 | class TestTelemetryEnabled:
15 | """Test telemetry enable/disable logic (SRP: Only tests enable/disable)."""
16 |
17 | def test_telemetry_enabled_by_default(self, monkeypatch):
18 | """Test that telemetry is enabled by default."""
19 | # Remove any environment variables that might affect the test
20 | monkeypatch.delenv("CUA_TELEMETRY", raising=False)
21 | monkeypatch.delenv("CUA_TELEMETRY_ENABLED", raising=False)
22 |
23 | from core.telemetry import is_telemetry_enabled
24 |
25 | assert is_telemetry_enabled() is True
26 |
27 | def test_telemetry_disabled_with_flag(self, monkeypatch):
28 | """Test that telemetry can be disabled with CUA_TELEMETRY_ENABLED=false."""
29 | monkeypatch.setenv("CUA_TELEMETRY_ENABLED", "false")
30 |
31 | from core.telemetry import is_telemetry_enabled
32 |
33 | assert is_telemetry_enabled() is False
34 |
35 | @pytest.mark.parametrize("value", ["0", "false", "no", "off"])
36 | def test_telemetry_disabled_with_various_values(self, monkeypatch, value):
37 | """Test that telemetry respects various disable values."""
38 | monkeypatch.setenv("CUA_TELEMETRY_ENABLED", value)
39 |
40 | from core.telemetry import is_telemetry_enabled
41 |
42 | assert is_telemetry_enabled() is False
43 |
44 | @pytest.mark.parametrize("value", ["1", "true", "yes", "on"])
45 | def test_telemetry_enabled_with_various_values(self, monkeypatch, value):
46 | """Test that telemetry respects various enable values."""
47 | monkeypatch.setenv("CUA_TELEMETRY_ENABLED", value)
48 |
49 | from core.telemetry import is_telemetry_enabled
50 |
51 | assert is_telemetry_enabled() is True
52 |
53 |
54 | class TestPostHogTelemetryClient:
55 | """Test PostHogTelemetryClient class (SRP: Only tests client logic)."""
56 |
57 | @patch("core.telemetry.posthog.posthog")
58 | @patch("core.telemetry.posthog.Path")
59 | def test_client_initialization(self, mock_path, mock_posthog, disable_telemetry):
60 | """Test that client initializes correctly."""
61 | from core.telemetry.posthog import PostHogTelemetryClient
62 |
63 | # Mock the storage directory
64 | mock_storage_dir = MagicMock()
65 | mock_storage_dir.exists.return_value = False
66 | mock_path.return_value.parent.parent = MagicMock()
67 | mock_path.return_value.parent.parent.__truediv__.return_value = mock_storage_dir
68 |
69 | # Reset singleton
70 | PostHogTelemetryClient.destroy_client()
71 |
72 | client = PostHogTelemetryClient()
73 |
74 | assert client is not None
75 | assert hasattr(client, "installation_id")
76 | assert hasattr(client, "initialized")
77 | assert hasattr(client, "queued_events")
78 |
79 | @patch("core.telemetry.posthog.posthog")
80 | @patch("core.telemetry.posthog.Path")
81 | def test_installation_id_generation(self, mock_path, mock_posthog, disable_telemetry):
82 | """Test that installation ID is generated if not exists."""
83 | from core.telemetry.posthog import PostHogTelemetryClient
84 |
85 | # Mock file system
86 | mock_id_file = MagicMock()
87 | mock_id_file.exists.return_value = False
88 | mock_storage_dir = MagicMock()
89 | mock_storage_dir.__truediv__.return_value = mock_id_file
90 |
91 | mock_core_dir = MagicMock()
92 | mock_core_dir.__truediv__.return_value = mock_storage_dir
93 | mock_path.return_value.parent.parent = mock_core_dir
94 |
95 | # Reset singleton
96 | PostHogTelemetryClient.destroy_client()
97 |
98 | client = PostHogTelemetryClient()
99 |
100 | # Should have generated a new UUID
101 | assert client.installation_id is not None
102 | assert len(client.installation_id) == 36 # UUID format
103 |
104 | @patch("core.telemetry.posthog.posthog")
105 | @patch("core.telemetry.posthog.Path")
106 | def test_installation_id_persistence(self, mock_path, mock_posthog, disable_telemetry):
107 | """Test that installation ID is read from file if exists."""
108 | from core.telemetry.posthog import PostHogTelemetryClient
109 |
110 | existing_id = "test-installation-id-123"
111 |
112 | # Mock file system
113 | mock_id_file = MagicMock()
114 | mock_id_file.exists.return_value = True
115 | mock_id_file.read_text.return_value = existing_id
116 |
117 | mock_storage_dir = MagicMock()
118 | mock_storage_dir.__truediv__.return_value = mock_id_file
119 |
120 | mock_core_dir = MagicMock()
121 | mock_core_dir.__truediv__.return_value = mock_storage_dir
122 | mock_path.return_value.parent.parent = mock_core_dir
123 |
124 | # Reset singleton
125 | PostHogTelemetryClient.destroy_client()
126 |
127 | client = PostHogTelemetryClient()
128 |
129 | assert client.installation_id == existing_id
130 |
131 | @patch("core.telemetry.posthog.posthog")
132 | @patch("core.telemetry.posthog.Path")
133 | def test_record_event_when_disabled(self, mock_path, mock_posthog, monkeypatch):
134 | """Test that events are not recorded when telemetry is disabled."""
135 | from core.telemetry.posthog import PostHogTelemetryClient
136 |
137 | # Disable telemetry explicitly using the correct environment variable
138 | monkeypatch.setenv("CUA_TELEMETRY_ENABLED", "false")
139 |
140 | # Mock file system
141 | mock_storage_dir = MagicMock()
142 | mock_storage_dir.exists.return_value = False
143 | mock_path.return_value.parent.parent = MagicMock()
144 | mock_path.return_value.parent.parent.__truediv__.return_value = mock_storage_dir
145 |
146 | # Reset singleton
147 | PostHogTelemetryClient.destroy_client()
148 |
149 | client = PostHogTelemetryClient()
150 | client.record_event("test_event", {"key": "value"})
151 |
152 | # PostHog capture should not be called at all when telemetry is disabled
153 | mock_posthog.capture.assert_not_called()
154 |
155 | @patch("core.telemetry.posthog.posthog")
156 | @patch("core.telemetry.posthog.Path")
157 | def test_record_event_when_enabled(self, mock_path, mock_posthog, monkeypatch):
158 | """Test that events are recorded when telemetry is enabled."""
159 | from core.telemetry.posthog import PostHogTelemetryClient
160 |
161 | # Enable telemetry
162 | monkeypatch.setenv("CUA_TELEMETRY_ENABLED", "true")
163 |
164 | # Mock file system
165 | mock_storage_dir = MagicMock()
166 | mock_storage_dir.exists.return_value = False
167 | mock_path.return_value.parent.parent = MagicMock()
168 | mock_path.return_value.parent.parent.__truediv__.return_value = mock_storage_dir
169 |
170 | # Reset singleton
171 | PostHogTelemetryClient.destroy_client()
172 |
173 | client = PostHogTelemetryClient()
174 | client.initialized = True # Pretend it's initialized
175 |
176 | event_name = "test_event"
177 | event_props = {"key": "value"}
178 | client.record_event(event_name, event_props)
179 |
180 | # PostHog capture should be called
181 | assert mock_posthog.capture.call_count >= 1
182 |
183 | @patch("core.telemetry.posthog.posthog")
184 | @patch("core.telemetry.posthog.Path")
185 | def test_singleton_pattern(self, mock_path, mock_posthog, disable_telemetry):
186 | """Test that get_client returns the same instance."""
187 | from core.telemetry.posthog import PostHogTelemetryClient
188 |
189 | # Mock file system
190 | mock_storage_dir = MagicMock()
191 | mock_storage_dir.exists.return_value = False
192 | mock_path.return_value.parent.parent = MagicMock()
193 | mock_path.return_value.parent.parent.__truediv__.return_value = mock_storage_dir
194 |
195 | # Reset singleton
196 | PostHogTelemetryClient.destroy_client()
197 |
198 | client1 = PostHogTelemetryClient.get_client()
199 | client2 = PostHogTelemetryClient.get_client()
200 |
201 | assert client1 is client2
202 |
203 |
204 | class TestRecordEvent:
205 | """Test the public record_event function (SRP: Only tests public API)."""
206 |
207 | @patch("core.telemetry.posthog.PostHogTelemetryClient")
208 | def test_record_event_calls_client(self, mock_client_class, disable_telemetry):
209 | """Test that record_event delegates to the client."""
210 | from core.telemetry import record_event
211 |
212 | mock_client_instance = Mock()
213 | mock_client_class.get_client.return_value = mock_client_instance
214 |
215 | event_name = "test_event"
216 | event_props = {"key": "value"}
217 |
218 | record_event(event_name, event_props)
219 |
220 | mock_client_instance.record_event.assert_called_once_with(event_name, event_props)
221 |
222 | @patch("core.telemetry.posthog.PostHogTelemetryClient")
223 | def test_record_event_without_properties(self, mock_client_class, disable_telemetry):
224 | """Test that record_event works without properties."""
225 | from core.telemetry import record_event
226 |
227 | mock_client_instance = Mock()
228 | mock_client_class.get_client.return_value = mock_client_instance
229 |
230 | event_name = "test_event"
231 |
232 | record_event(event_name)
233 |
234 | mock_client_instance.record_event.assert_called_once_with(event_name, {})
235 |
236 |
237 | class TestDestroyTelemetryClient:
238 | """Test client destruction (SRP: Only tests cleanup)."""
239 |
240 | @patch("core.telemetry.posthog.PostHogTelemetryClient")
241 | def test_destroy_client_calls_class_method(self, mock_client_class):
242 | """Test that destroy_telemetry_client delegates correctly."""
243 | from core.telemetry import destroy_telemetry_client
244 |
245 | destroy_telemetry_client()
246 |
247 | mock_client_class.destroy_client.assert_called_once()
248 |
```
--------------------------------------------------------------------------------
/tests/test_mcp_server_streaming.py:
--------------------------------------------------------------------------------
```python
1 | import asyncio
2 | import importlib.util
3 | import sys
4 | import types
5 | from pathlib import Path
6 |
7 | import pytest
8 |
9 |
10 | def _install_stub_module(
11 | name: str, module: types.ModuleType, registry: dict[str, types.ModuleType | None]
12 | ) -> None:
13 | registry[name] = sys.modules.get(name)
14 | sys.modules[name] = module
15 |
16 |
17 | @pytest.fixture
18 | def server_module():
19 | stubbed_modules: dict[str, types.ModuleType | None] = {}
20 |
21 | # Stub MCP Context primitives
22 | mcp_module = types.ModuleType("mcp")
23 | mcp_module.__path__ = [] # mark as package
24 |
25 | mcp_server_module = types.ModuleType("mcp.server")
26 | mcp_server_module.__path__ = []
27 |
28 | fastmcp_module = types.ModuleType("mcp.server.fastmcp")
29 |
30 | class _StubContext:
31 | async def yield_message(self, *args, **kwargs):
32 | return None
33 |
34 | async def yield_tool_call(self, *args, **kwargs):
35 | return None
36 |
37 | async def yield_tool_output(self, *args, **kwargs):
38 | return None
39 |
40 | def report_progress(self, *_args, **_kwargs):
41 | return None
42 |
43 | def info(self, *_args, **_kwargs):
44 | return None
45 |
46 | def error(self, *_args, **_kwargs):
47 | return None
48 |
49 | class _StubImage:
50 | def __init__(self, format: str, data: bytes):
51 | self.format = format
52 | self.data = data
53 |
54 | class _StubFastMCP:
55 | def __init__(self, name: str):
56 | self.name = name
57 | self._tools: dict[str, types.FunctionType] = {}
58 |
59 | def tool(self, *args, **kwargs):
60 | def decorator(func):
61 | self._tools[func.__name__] = func
62 | return func
63 |
64 | return decorator
65 |
66 | def run(self):
67 | return None
68 |
69 | fastmcp_module.Context = _StubContext
70 | fastmcp_module.FastMCP = _StubFastMCP
71 | fastmcp_module.Image = _StubImage
72 |
73 | _install_stub_module("mcp", mcp_module, stubbed_modules)
74 | _install_stub_module("mcp.server", mcp_server_module, stubbed_modules)
75 | _install_stub_module("mcp.server.fastmcp", fastmcp_module, stubbed_modules)
76 |
77 | # Stub Computer module to avoid heavy dependencies
78 | computer_module = types.ModuleType("computer")
79 |
80 | class _StubInterface:
81 | async def screenshot(self) -> bytes: # pragma: no cover - default stub
82 | return b""
83 |
84 | class _StubComputer:
85 | def __init__(self, *args, **kwargs):
86 | self.interface = _StubInterface()
87 |
88 | async def run(self): # pragma: no cover - default stub
89 | return None
90 |
91 | class _StubVMProviderType:
92 | CLOUD = "cloud"
93 | LOCAL = "local"
94 |
95 | computer_module.Computer = _StubComputer
96 | computer_module.VMProviderType = _StubVMProviderType
97 |
98 | _install_stub_module("computer", computer_module, stubbed_modules)
99 |
100 | # Stub agent module so server can import ComputerAgent
101 | agent_module = types.ModuleType("agent")
102 |
103 | class _StubComputerAgent:
104 | def __init__(self, *args, **kwargs):
105 | pass
106 |
107 | async def run(self, *_args, **_kwargs): # pragma: no cover - default stub
108 | if False: # pragma: no cover
109 | yield {}
110 | return
111 |
112 | agent_module.ComputerAgent = _StubComputerAgent
113 |
114 | _install_stub_module("agent", agent_module, stubbed_modules)
115 |
116 | module_name = "mcp_server_server_under_test"
117 | module_path = Path("libs/python/mcp-server/mcp_server/server.py").resolve()
118 | spec = importlib.util.spec_from_file_location(module_name, module_path)
119 | server_module = importlib.util.module_from_spec(spec)
120 | assert spec and spec.loader
121 | spec.loader.exec_module(server_module)
122 |
123 | server_instance = getattr(server_module, "server", None)
124 | if server_instance is not None and hasattr(server_instance, "_tools"):
125 | for name, func in server_instance._tools.items():
126 | setattr(server_module, name, func)
127 |
128 | try:
129 | yield server_module
130 | finally:
131 | sys.modules.pop(module_name, None)
132 | for name, original in stubbed_modules.items():
133 | if original is None:
134 | sys.modules.pop(name, None)
135 | else:
136 | sys.modules[name] = original
137 |
138 |
139 | class FakeContext:
140 | def __init__(self) -> None:
141 | self.events: list[tuple] = []
142 | self.progress_updates: list[float] = []
143 |
144 | def info(self, message: str) -> None:
145 | self.events.append(("info", message))
146 |
147 | def error(self, message: str) -> None:
148 | self.events.append(("error", message))
149 |
150 | def report_progress(self, value: float) -> None:
151 | self.progress_updates.append(value)
152 |
153 | async def yield_message(self, *, role: str, content):
154 | timestamp = asyncio.get_running_loop().time()
155 | self.events.append(("message", role, content, timestamp))
156 |
157 | async def yield_tool_call(self, *, name: str | None, call_id: str, input):
158 | timestamp = asyncio.get_running_loop().time()
159 | self.events.append(("tool_call", name, call_id, input, timestamp))
160 |
161 | async def yield_tool_output(self, *, call_id: str, output, is_error: bool = False):
162 | timestamp = asyncio.get_running_loop().time()
163 | self.events.append(("tool_output", call_id, output, is_error, timestamp))
164 |
165 |
166 | def test_run_cua_task_streams_partial_results(server_module):
167 | async def _run_test():
168 | class FakeAgent:
169 | script = []
170 |
171 | def __init__(self, *args, **kwargs):
172 | pass
173 |
174 | async def run(self, messages): # type: ignore[override]
175 | for factory, delay in type(self).script:
176 | yield factory(messages)
177 | if delay:
178 | await asyncio.sleep(delay)
179 |
180 | FakeAgent.script = [
181 | (
182 | lambda _messages: {
183 | "output": [
184 | {
185 | "type": "message",
186 | "role": "assistant",
187 | "content": [{"type": "output_text", "text": "First chunk"}],
188 | }
189 | ]
190 | },
191 | 0.0,
192 | ),
193 | (
194 | lambda _messages: {
195 | "output": [
196 | {
197 | "type": "tool_use",
198 | "id": "call_1",
199 | "name": "computer",
200 | "input": {"action": "click"},
201 | },
202 | {
203 | "type": "computer_call_output",
204 | "call_id": "call_1",
205 | "output": [{"type": "text", "text": "Tool completed"}],
206 | },
207 | ]
208 | },
209 | 0.05,
210 | ),
211 | ]
212 |
213 | class FakeInterface:
214 | def __init__(self) -> None:
215 | self.calls = 0
216 |
217 | async def screenshot(self) -> bytes:
218 | self.calls += 1
219 | return b"final-image"
220 |
221 | fake_interface = FakeInterface()
222 | server_module.global_computer = types.SimpleNamespace(interface=fake_interface)
223 | server_module.ComputerAgent = FakeAgent # type: ignore[assignment]
224 |
225 | ctx = FakeContext()
226 | task = asyncio.create_task(server_module.run_cua_task(ctx, "open settings"))
227 |
228 | await asyncio.sleep(0.01)
229 | assert not task.done(), "Task should still be running to simulate long operation"
230 | message_events = [event for event in ctx.events if event[0] == "message"]
231 | assert message_events, "Expected message event before task completion"
232 |
233 | text_result, image = await task
234 |
235 | assert "First chunk" in text_result
236 | assert "Tool completed" in text_result
237 | assert image.data == b"final-image"
238 | assert fake_interface.calls == 1
239 |
240 | tool_call_events = [event for event in ctx.events if event[0] == "tool_call"]
241 | tool_output_events = [event for event in ctx.events if event[0] == "tool_output"]
242 | assert tool_call_events and tool_output_events
243 | assert tool_call_events[0][2] == "call_1"
244 | assert tool_output_events[0][1] == "call_1"
245 |
246 | asyncio.run(_run_test())
247 |
248 |
249 | def test_run_multi_cua_tasks_reports_progress(server_module, monkeypatch):
250 | async def _run_test():
251 | class FakeAgent:
252 | script = []
253 |
254 | def __init__(self, *args, **kwargs):
255 | pass
256 |
257 | async def run(self, messages): # type: ignore[override]
258 | for factory, delay in type(self).script:
259 | yield factory(messages)
260 | if delay:
261 | await asyncio.sleep(delay)
262 |
263 | FakeAgent.script = [
264 | (
265 | lambda messages: {
266 | "output": [
267 | {
268 | "type": "message",
269 | "role": "assistant",
270 | "content": [
271 | {
272 | "type": "output_text",
273 | "text": f"Result for {messages[0].get('content')}",
274 | }
275 | ],
276 | }
277 | ]
278 | },
279 | 0.0,
280 | )
281 | ]
282 |
283 | server_module.ComputerAgent = FakeAgent # type: ignore[assignment]
284 |
285 | class FakeInterface:
286 | async def screenshot(self) -> bytes:
287 | return b"progress-image"
288 |
289 | server_module.global_computer = types.SimpleNamespace(interface=FakeInterface())
290 |
291 | ctx = FakeContext()
292 |
293 | results = await server_module.run_multi_cua_tasks(ctx, ["a", "b", "c"])
294 |
295 | assert len(results) == 3
296 | assert results[0][0] == "Result for a"
297 | assert ctx.progress_updates[0] == pytest.approx(0.0)
298 | assert ctx.progress_updates[-1] == pytest.approx(1.0)
299 | assert len(ctx.progress_updates) == 6
300 |
301 | asyncio.run(_run_test())
302 |
```
--------------------------------------------------------------------------------
/.github/workflows/publish-lume.yml:
--------------------------------------------------------------------------------
```yaml
1 | name: Publish Notarized Lume
2 |
3 | on:
4 | push:
5 | tags:
6 | - "lume-v*"
7 | workflow_dispatch:
8 | inputs:
9 | version:
10 | description: "Version to notarize (without v prefix)"
11 | required: true
12 | default: "0.1.0"
13 | workflow_call:
14 | inputs:
15 | version:
16 | description: "Version to notarize"
17 | required: true
18 | type: string
19 | secrets:
20 | APPLICATION_CERT_BASE64:
21 | required: true
22 | INSTALLER_CERT_BASE64:
23 | required: true
24 | CERT_PASSWORD:
25 | required: true
26 | APPLE_ID:
27 | required: true
28 | TEAM_ID:
29 | required: true
30 | APP_SPECIFIC_PASSWORD:
31 | required: true
32 | DEVELOPER_NAME:
33 | required: true
34 |
35 | permissions:
36 | contents: write
37 |
38 | env:
39 | APPLICATION_CERT_BASE64: ${{ secrets.APPLICATION_CERT_BASE64 }}
40 | INSTALLER_CERT_BASE64: ${{ secrets.INSTALLER_CERT_BASE64 }}
41 | CERT_PASSWORD: ${{ secrets.CERT_PASSWORD }}
42 | APPLE_ID: ${{ secrets.APPLE_ID }}
43 | TEAM_ID: ${{ secrets.TEAM_ID }}
44 | APP_SPECIFIC_PASSWORD: ${{ secrets.APP_SPECIFIC_PASSWORD }}
45 | DEVELOPER_NAME: ${{ secrets.DEVELOPER_NAME }}
46 |
47 | jobs:
48 | notarize:
49 | runs-on: macos-15
50 | outputs:
51 | sha256_checksums: ${{ steps.generate_checksums.outputs.checksums }}
52 | version: ${{ steps.set_version.outputs.version }}
53 | steps:
54 | - uses: actions/checkout@v4
55 |
56 | - name: Select Xcode 16
57 | run: |
58 | sudo xcode-select -s /Applications/Xcode_16.app
59 | xcodebuild -version
60 |
61 | - name: Install dependencies
62 | run: |
63 | brew install cpio
64 |
65 | - name: Create .release directory
66 | run: mkdir -p .release
67 |
68 | - name: Set version
69 | id: set_version
70 | run: |
71 | # Determine version from tag or input
72 | if [[ "$GITHUB_REF" == refs/tags/lume-v* ]]; then
73 | VERSION="${GITHUB_REF#refs/tags/lume-v}"
74 | echo "Using version from tag: $VERSION"
75 | elif [[ -n "${{ inputs.version }}" ]]; then
76 | VERSION="${{ inputs.version }}"
77 | echo "Using version from input: $VERSION"
78 | elif [[ -n "${{ inputs.version }}" ]]; then
79 | VERSION="${{ inputs.version }}"
80 | echo "Using version from workflow_call input: $VERSION"
81 | else
82 | echo "Error: No version found in tag or input"
83 | exit 1
84 | fi
85 |
86 | # Update version in Main.swift
87 | echo "Updating version in Main.swift to $VERSION"
88 | sed -i '' "s/static let current: String = \".*\"/static let current: String = \"$VERSION\"/" libs/lume/src/Main.swift
89 |
90 | # Set output for later steps
91 | echo "version=$VERSION" >> $GITHUB_OUTPUT
92 |
93 | - name: Import Certificates
94 | env:
95 | APPLICATION_CERT_BASE64: ${{ secrets.APPLICATION_CERT_BASE64 }}
96 | INSTALLER_CERT_BASE64: ${{ secrets.INSTALLER_CERT_BASE64 }}
97 | CERT_PASSWORD: ${{ secrets.CERT_PASSWORD }}
98 | KEYCHAIN_PASSWORD: "temp_password"
99 | run: |
100 | # Create a temporary keychain
101 | security create-keychain -p "$KEYCHAIN_PASSWORD" build.keychain
102 | security default-keychain -s build.keychain
103 | security unlock-keychain -p "$KEYCHAIN_PASSWORD" build.keychain
104 | security set-keychain-settings -t 3600 -l build.keychain
105 |
106 | # Import certificates
107 | echo $APPLICATION_CERT_BASE64 | base64 --decode > application.p12
108 | echo $INSTALLER_CERT_BASE64 | base64 --decode > installer.p12
109 |
110 | # Import certificates silently (minimize output)
111 | security import application.p12 -k build.keychain -P "$CERT_PASSWORD" -T /usr/bin/codesign -T /usr/bin/pkgbuild > /dev/null 2>&1
112 | security import installer.p12 -k build.keychain -P "$CERT_PASSWORD" -T /usr/bin/codesign -T /usr/bin/pkgbuild > /dev/null 2>&1
113 |
114 | # Allow codesign to access the certificates (minimal output)
115 | security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$KEYCHAIN_PASSWORD" build.keychain > /dev/null 2>&1
116 |
117 | # Verify certificates were imported
118 | echo "Verifying signing identities..."
119 | CERT_COUNT=$(security find-identity -v -p codesigning build.keychain | grep -c "Developer ID Application" || echo "0")
120 | INSTALLER_COUNT=$(security find-identity -v build.keychain | grep -c "Developer ID Installer" || echo "0")
121 |
122 | if [ "$CERT_COUNT" -eq 0 ]; then
123 | echo "Error: No Developer ID Application certificate found"
124 | security find-identity -v -p codesigning build.keychain
125 | exit 1
126 | fi
127 |
128 | if [ "$INSTALLER_COUNT" -eq 0 ]; then
129 | echo "Error: No Developer ID Installer certificate found"
130 | security find-identity -v build.keychain
131 | exit 1
132 | fi
133 |
134 | echo "Found $CERT_COUNT Developer ID Application certificate(s) and $INSTALLER_COUNT Developer ID Installer certificate(s)"
135 | echo "All required certificates verified successfully"
136 |
137 | # Clean up certificate files
138 | rm application.p12 installer.p12
139 |
140 | - name: Build and Notarize
141 | id: build_notarize
142 | env:
143 | APPLE_ID: ${{ secrets.APPLE_ID }}
144 | TEAM_ID: ${{ secrets.TEAM_ID }}
145 | APP_SPECIFIC_PASSWORD: ${{ secrets.APP_SPECIFIC_PASSWORD }}
146 | # These will now reference the imported certificates
147 | CERT_APPLICATION_NAME: "Developer ID Application: ${{ secrets.DEVELOPER_NAME }} (${{ secrets.TEAM_ID }})"
148 | CERT_INSTALLER_NAME: "Developer ID Installer: ${{ secrets.DEVELOPER_NAME }} (${{ secrets.TEAM_ID }})"
149 | VERSION: ${{ steps.set_version.outputs.version }}
150 | working-directory: ./libs/lume
151 | run: |
152 | # Minimal debug information
153 | echo "Starting build process..."
154 | echo "Swift version: $(swift --version | head -n 1)"
155 | echo "Building version: $VERSION"
156 |
157 | # Ensure .release directory exists
158 | mkdir -p .release
159 | chmod 755 .release
160 |
161 | # Build the project first (redirect verbose output)
162 | echo "Building project..."
163 | swift build --configuration release > build.log 2>&1
164 | echo "Build completed."
165 |
166 | # Run the notarization script with LOG_LEVEL env var
167 | chmod +x scripts/build/build-release-notarized.sh
168 | cd scripts/build
169 | LOG_LEVEL=minimal ./build-release-notarized.sh
170 |
171 | # Return to the lume directory
172 | cd ../..
173 |
174 | # Debug: List what files were actually created
175 | echo "Files in .release directory:"
176 | find .release -type f -name "*.tar.gz" -o -name "*.pkg.tar.gz"
177 |
178 | # Get architecture for output filename
179 | ARCH=$(uname -m)
180 | OS_IDENTIFIER="darwin-${ARCH}"
181 |
182 | # Output paths for later use
183 | echo "tarball_path=.release/lume-${VERSION}-${OS_IDENTIFIER}.tar.gz" >> $GITHUB_OUTPUT
184 | echo "pkg_path=.release/lume-${VERSION}-${OS_IDENTIFIER}.pkg.tar.gz" >> $GITHUB_OUTPUT
185 |
186 | - name: Generate SHA256 Checksums
187 | id: generate_checksums
188 | working-directory: ./libs/lume/.release
189 | run: |
190 | # Use existing checksums file if it exists, otherwise generate one
191 | if [ -f "checksums.txt" ]; then
192 | echo "Using existing checksums file"
193 | cat checksums.txt
194 | else
195 | echo "## SHA256 Checksums" > checksums.txt
196 | echo '```' >> checksums.txt
197 | shasum -a 256 lume-*.tar.gz >> checksums.txt
198 | echo '```' >> checksums.txt
199 | fi
200 |
201 | checksums=$(cat checksums.txt)
202 | echo "checksums<<EOF" >> $GITHUB_OUTPUT
203 | echo "$checksums" >> $GITHUB_OUTPUT
204 | echo "EOF" >> $GITHUB_OUTPUT
205 |
206 | # Debug: Show all files in the release directory
207 | echo "All files in release directory:"
208 | ls -la
209 |
210 | - name: Create Standard Version Releases
211 | working-directory: ./libs/lume/.release
212 | run: |
213 | VERSION=${{ steps.set_version.outputs.version }}
214 | ARCH=$(uname -m)
215 | OS_IDENTIFIER="darwin-${ARCH}"
216 |
217 | # Create OS-tagged symlinks
218 | ln -sf "lume-${VERSION}-${OS_IDENTIFIER}.tar.gz" "lume-darwin.tar.gz"
219 | ln -sf "lume-${VERSION}-${OS_IDENTIFIER}.pkg.tar.gz" "lume-darwin.pkg.tar.gz"
220 |
221 | # Create simple symlinks
222 | ln -sf "lume-${VERSION}-${OS_IDENTIFIER}.tar.gz" "lume.tar.gz"
223 | ln -sf "lume-${VERSION}-${OS_IDENTIFIER}.pkg.tar.gz" "lume.pkg.tar.gz"
224 |
225 | # List all files (including symlinks)
226 | echo "Files with symlinks in release directory:"
227 | ls -la
228 |
229 | - name: Upload Notarized Package (Tarball)
230 | uses: actions/upload-artifact@v4
231 | with:
232 | name: lume-notarized-tarball
233 | path: ./libs/lume/${{ steps.build_notarize.outputs.tarball_path }}
234 | if-no-files-found: error
235 |
236 | - name: Upload Notarized Package (Installer)
237 | uses: actions/upload-artifact@v4
238 | with:
239 | name: lume-notarized-installer
240 | path: ./libs/lume/${{ steps.build_notarize.outputs.pkg_path }}
241 | if-no-files-found: error
242 |
243 | - name: Create Release
244 | if: startsWith(github.ref, 'refs/tags/lume-v')
245 | uses: softprops/action-gh-release@v1
246 | with:
247 | files: |
248 | ./libs/lume/${{ steps.build_notarize.outputs.tarball_path }}
249 | ./libs/lume/${{ steps.build_notarize.outputs.pkg_path }}
250 | ./libs/lume/.release/lume-darwin.tar.gz
251 | ./libs/lume/.release/lume-darwin.pkg.tar.gz
252 | ./libs/lume/.release/lume.tar.gz
253 | ./libs/lume/.release/lume.pkg.tar.gz
254 | body: |
255 | ${{ steps.generate_checksums.outputs.checksums }}
256 |
257 | ### Installation with script
258 |
259 | /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"
260 | ```
261 | generate_release_notes: true
262 | make_latest: true
263 |
```
--------------------------------------------------------------------------------
/scripts/playground-docker.sh:
--------------------------------------------------------------------------------
```bash
1 | #!/bin/bash
2 |
3 | set -e
4 |
5 | # Colors for output
6 | GREEN='\033[0;32m'
7 | BLUE='\033[0;34m'
8 | RED='\033[0;31m'
9 | YELLOW='\033[1;33m'
10 | NC='\033[0m' # No Color
11 |
12 | # Print with color
13 | print_info() {
14 | echo -e "${BLUE}==> $1${NC}"
15 | }
16 |
17 | print_success() {
18 | echo -e "${GREEN}==> $1${NC}"
19 | }
20 |
21 | print_error() {
22 | echo -e "${RED}==> $1${NC}"
23 | }
24 |
25 | print_warning() {
26 | echo -e "${YELLOW}==> $1${NC}"
27 | }
28 |
29 | echo "🚀 Launching Cua Computer-Use Agent UI..."
30 |
31 | # Check if Docker is installed
32 | if ! command -v docker &> /dev/null; then
33 | print_error "Docker is not installed!"
34 | echo ""
35 | echo "To use Cua with Docker containers, you need to install Docker first:"
36 | echo ""
37 | echo "📦 Install Docker:"
38 | echo " • macOS: Download Docker Desktop from https://docker.com/products/docker-desktop"
39 | echo " • Windows: Download Docker Desktop from https://docker.com/products/docker-desktop"
40 | echo " • Linux: Follow instructions at https://docs.docker.com/engine/install/"
41 | echo ""
42 | echo "After installing Docker, run this script again."
43 | exit 1
44 | fi
45 |
46 | # Check if Docker daemon is running
47 | if ! docker info &> /dev/null; then
48 | print_error "Docker is installed but not running!"
49 | echo ""
50 | echo "Please start Docker Desktop and try again."
51 | exit 1
52 | fi
53 |
54 | print_success "Docker is installed and running!"
55 |
56 | # Save the original working directory
57 | ORIGINAL_DIR="$(pwd)"
58 |
59 | DEMO_DIR="$HOME/.cua"
60 | mkdir -p "$DEMO_DIR"
61 |
62 |
63 | # Check if we're already in the Cua repository
64 | # Look for the specific trycua identifier in pyproject.toml
65 | if [[ -f "pyproject.toml" ]] && grep -q "[email protected]" "pyproject.toml"; then
66 | print_success "Already in Cua repository - using current directory"
67 | REPO_DIR="$ORIGINAL_DIR"
68 | USE_EXISTING_REPO=true
69 | else
70 | # Directories used by the script when not in repo
71 | REPO_DIR="$DEMO_DIR/cua"
72 | USE_EXISTING_REPO=false
73 | fi
74 |
75 | # Function to clean up on exit
76 | cleanup() {
77 | cd "$ORIGINAL_DIR" 2>/dev/null || true
78 | }
79 | trap cleanup EXIT
80 |
81 | echo ""
82 | echo "Choose your Cua setup:"
83 | echo "1) ☁️ Cua Cloud Sandbox (works on any system)"
84 | echo "2) 🖥️ Local macOS VMs (requires Apple Silicon Mac + macOS 15+)"
85 | echo "3) 🖥️ Local Windows VMs (requires Windows 10 / 11)"
86 | echo ""
87 | read -p "Enter your choice (1, 2, or 3): " CHOICE
88 |
89 | if [[ "$CHOICE" == "1" ]]; then
90 | # Cua Cloud Sandbox setup
91 | echo ""
92 | print_info "Setting up Cua Cloud Sandbox..."
93 | echo ""
94 |
95 | # Check if existing .env.local already has CUA_API_KEY
96 | REPO_ENV_FILE="$REPO_DIR/.env.local"
97 | CURRENT_ENV_FILE="$ORIGINAL_DIR/.env.local"
98 |
99 | CUA_API_KEY=""
100 |
101 | # First check current directory
102 | if [[ -f "$CURRENT_ENV_FILE" ]] && grep -q "CUA_API_KEY=" "$CURRENT_ENV_FILE"; then
103 | EXISTING_CUA_KEY=$(grep "CUA_API_KEY=" "$CURRENT_ENV_FILE" | cut -d'=' -f2- | tr -d '"' | tr -d "'" | xargs)
104 | if [[ -n "$EXISTING_CUA_KEY" && "$EXISTING_CUA_KEY" != "your_cua_api_key_here" && "$EXISTING_CUA_KEY" != "" ]]; then
105 | CUA_API_KEY="$EXISTING_CUA_KEY"
106 | fi
107 | fi
108 |
109 | # Then check repo directory if not found in current dir
110 | if [[ -z "$CUA_API_KEY" ]] && [[ -f "$REPO_ENV_FILE" ]] && grep -q "CUA_API_KEY=" "$REPO_ENV_FILE"; then
111 | EXISTING_CUA_KEY=$(grep "CUA_API_KEY=" "$REPO_ENV_FILE" | cut -d'=' -f2- | tr -d '"' | tr -d "'" | xargs)
112 | if [[ -n "$EXISTING_CUA_KEY" && "$EXISTING_CUA_KEY" != "your_cua_api_key_here" && "$EXISTING_CUA_KEY" != "" ]]; then
113 | CUA_API_KEY="$EXISTING_CUA_KEY"
114 | fi
115 | fi
116 |
117 | # If no valid API key found, prompt for one
118 | if [[ -z "$CUA_API_KEY" ]]; then
119 | echo "To use Cua Cloud Sandbox, you need to:"
120 | echo "1. Sign up at https://cua.ai"
121 | echo "2. Create a Cloud Sandbox"
122 | echo "3. Generate an Api Key"
123 | echo ""
124 | read -p "Enter your Cua Api Key: " CUA_API_KEY
125 |
126 | if [[ -z "$CUA_API_KEY" ]]; then
127 | print_error "Cua Api Key is required for Cloud Sandbox."
128 | exit 1
129 | fi
130 | else
131 | print_success "Found existing CUA API key"
132 | fi
133 |
134 | USE_CLOUD=true
135 | COMPUTER_TYPE="cloud"
136 |
137 | elif [[ "$CHOICE" == "2" ]]; then
138 | # Local macOS VM setup
139 | echo ""
140 | print_info "Setting up local macOS VMs..."
141 |
142 | # Check for Apple Silicon Mac
143 | if [[ $(uname -s) != "Darwin" || $(uname -m) != "arm64" ]]; then
144 | print_error "Local macOS VMs require an Apple Silicon Mac (M1/M2/M3/M4)."
145 | echo "💡 Consider using Cua Cloud Sandbox instead (option 1)."
146 | exit 1
147 | fi
148 |
149 | # Check for macOS 15 (Sequoia) or newer
150 | OSVERSION=$(sw_vers -productVersion)
151 | if [[ $(echo "$OSVERSION 15.0" | tr " " "\n" | sort -V | head -n 1) != "15.0" ]]; then
152 | print_error "Local macOS VMs require macOS 15 (Sequoia) or newer. You have $OSVERSION."
153 | echo "💡 Consider using Cua Cloud Sandbox instead (option 1)."
154 | exit 1
155 | fi
156 |
157 | USE_CLOUD=false
158 | COMPUTER_TYPE="macos"
159 |
160 | elif [[ "$CHOICE" == "3" ]]; then
161 | # Local Windows VM setup
162 | echo ""
163 | print_info "Setting up local Windows VMs..."
164 |
165 | # Check if we're on Windows
166 | if [[ $(uname -s) != MINGW* && $(uname -s) != CYGWIN* && $(uname -s) != MSYS* ]]; then
167 | print_error "Local Windows VMs require Windows 10 or 11."
168 | echo "💡 Consider using Cua Cloud Sandbox instead (option 1)."
169 | echo ""
170 | echo "🔗 If you are using WSL, refer to the blog post to get started: https://cua.ai/blog/windows-sandbox"
171 | exit 1
172 | fi
173 |
174 | USE_CLOUD=false
175 | COMPUTER_TYPE="windows"
176 |
177 | else
178 | print_error "Invalid choice. Please run the script again and choose 1, 2, or 3."
179 | exit 1
180 | fi
181 |
182 | print_success "All checks passed! 🎉"
183 |
184 | # Create demo directory and handle repository
185 | if [[ "$USE_EXISTING_REPO" == "true" ]]; then
186 | print_info "Using existing repository in current directory"
187 | cd "$REPO_DIR"
188 | else
189 | # Clone or update the repository
190 | if [[ ! -d "$REPO_DIR" ]]; then
191 | print_info "Cloning Cua repository..."
192 | cd "$DEMO_DIR"
193 | git clone https://github.com/trycua/cua.git
194 | else
195 | print_info "Updating Cua repository..."
196 | cd "$REPO_DIR"
197 | git pull origin main
198 | fi
199 |
200 | cd "$REPO_DIR"
201 | fi
202 |
203 | # Create .env.local file with API keys
204 | ENV_FILE="$REPO_DIR/.env.local"
205 | if [[ ! -f "$ENV_FILE" ]]; then
206 | cat > "$ENV_FILE" << EOF
207 | # Uncomment and add your API keys here
208 | # OPENAI_API_KEY=your_openai_api_key_here
209 | # ANTHROPIC_API_KEY=your_anthropic_api_key_here
210 | CUA_API_KEY=your_cua_api_key_here
211 | EOF
212 | print_success "Created .env.local file with API key placeholders"
213 | else
214 | print_success "Found existing .env.local file - keeping your current settings"
215 | fi
216 |
217 | if [[ "$USE_CLOUD" == "true" ]]; then
218 | # Add CUA API key to .env.local if not already present
219 | if ! grep -q "CUA_API_KEY" "$ENV_FILE"; then
220 | echo "CUA_API_KEY=$CUA_API_KEY" >> "$ENV_FILE"
221 | print_success "Added CUA_API_KEY to .env.local"
222 | elif grep -q "CUA_API_KEY=your_cua_api_key_here" "$ENV_FILE"; then
223 | # Update placeholder with actual key
224 | sed -i.bak "s/CUA_API_KEY=your_cua_api_key_here/CUA_API_KEY=$CUA_API_KEY/" "$ENV_FILE"
225 | print_success "Updated CUA_API_KEY in .env.local"
226 | fi
227 | fi
228 |
229 | # Build the Docker image if it doesn't exist
230 | print_info "Checking Docker image..."
231 | if ! docker image inspect cua-dev-image &> /dev/null; then
232 | print_info "Building Docker image (this may take a while)..."
233 | ./scripts/run-docker-dev.sh build
234 | else
235 | print_success "Docker image already exists"
236 | fi
237 |
238 | # Install Lume if needed for local VMs
239 | if [[ "$USE_CLOUD" == "false" && "$COMPUTER_TYPE" == "macos" ]]; then
240 | if ! command -v lume &> /dev/null; then
241 | print_info "Installing Lume CLI..."
242 | curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh | bash
243 |
244 | # Add lume to PATH for this session if it's not already there
245 | if ! command -v lume &> /dev/null; then
246 | export PATH="$PATH:$HOME/.local/bin"
247 | fi
248 | fi
249 |
250 | # Pull the macOS CUA image if not already present
251 | if ! lume ls | grep -q "macos-sequoia-cua"; then
252 | # Check available disk space
253 | IMAGE_SIZE_GB=30
254 | AVAILABLE_SPACE_KB=$(df -k $HOME | tail -1 | awk '{print $4}')
255 | AVAILABLE_SPACE_GB=$(($AVAILABLE_SPACE_KB / 1024 / 1024))
256 |
257 | echo "📊 The macOS CUA image will use approximately ${IMAGE_SIZE_GB}GB of disk space."
258 | echo " You currently have ${AVAILABLE_SPACE_GB}GB available on your system."
259 |
260 | # Prompt for confirmation
261 | read -p " Continue? [y]/n: " CONTINUE
262 | CONTINUE=${CONTINUE:-y}
263 |
264 | if [[ $CONTINUE =~ ^[Yy]$ ]]; then
265 | print_info "Pulling macOS CUA image (this may take a while)..."
266 |
267 | # Use caffeinate on macOS to prevent system sleep during the pull
268 | if command -v caffeinate &> /dev/null; then
269 | print_info "Using caffeinate to prevent system sleep during download..."
270 | caffeinate -i lume pull macos-sequoia-cua:latest
271 | else
272 | lume pull macos-sequoia-cua:latest
273 | fi
274 | else
275 | print_error "Installation cancelled."
276 | exit 1
277 | fi
278 | fi
279 |
280 | # Check if the VM is running
281 | print_info "Checking if the macOS CUA VM is running..."
282 | VM_RUNNING=$(lume ls | grep "macos-sequoia-cua" | grep "running" || echo "")
283 |
284 | if [ -z "$VM_RUNNING" ]; then
285 | print_info "Starting the macOS CUA VM in the background..."
286 | lume run macos-sequoia-cua:latest &
287 | # Wait a moment for the VM to initialize
288 | sleep 5
289 | print_success "VM started successfully."
290 | else
291 | print_success "macOS CUA VM is already running."
292 | fi
293 | fi
294 |
295 | # Create a convenience script to run the demo
296 | cat > "$DEMO_DIR/start_ui.sh" << EOF
297 | #!/bin/bash
298 | cd "$REPO_DIR"
299 | ./scripts/run-docker-dev.sh run agent_ui_examples.py
300 | EOF
301 | chmod +x "$DEMO_DIR/start_ui.sh"
302 |
303 | print_success "Setup complete!"
304 |
305 | if [[ "$USE_CLOUD" == "true" ]]; then
306 | echo "☁️ Cua Cloud Sandbox setup complete!"
307 | else
308 | echo "🖥️ Cua Local VM setup complete!"
309 | fi
310 |
311 | echo "📝 Edit $ENV_FILE to update your API keys"
312 | echo "🖥️ Start the playground by running: $DEMO_DIR/start_ui.sh"
313 |
314 | # Start the demo automatically
315 | echo
316 | print_info "Starting the Cua Computer-Use Agent UI..."
317 | echo ""
318 |
319 | print_success "Cua Computer-Use Agent UI is now running at http://localhost:7860/"
320 | echo
321 | echo "🌐 Open your browser and go to: http://localhost:7860/"
322 | echo
323 | "$DEMO_DIR/start_ui.sh"
324 |
```
--------------------------------------------------------------------------------
/libs/python/computer-server/computer_server/handlers/base.py:
--------------------------------------------------------------------------------
```python
1 | from abc import ABC, abstractmethod
2 | from typing import Any, Dict, List, Optional, Tuple
3 |
4 |
5 | class BaseAccessibilityHandler(ABC):
6 | """Abstract base class for OS-specific accessibility handlers."""
7 |
8 | @abstractmethod
9 | async def get_accessibility_tree(self) -> Dict[str, Any]:
10 | """Get the accessibility tree of the current window."""
11 | pass
12 |
13 | @abstractmethod
14 | async def find_element(
15 | self, role: Optional[str] = None, title: Optional[str] = None, value: Optional[str] = None
16 | ) -> Dict[str, Any]:
17 | """Find an element in the accessibility tree by criteria."""
18 | pass
19 |
20 |
21 | class BaseFileHandler(ABC):
22 | """Abstract base class for OS-specific file handlers."""
23 |
24 | @abstractmethod
25 | async def file_exists(self, path: str) -> Dict[str, Any]:
26 | """Check if a file exists at the specified path."""
27 | pass
28 |
29 | @abstractmethod
30 | async def directory_exists(self, path: str) -> Dict[str, Any]:
31 | """Check if a directory exists at the specified path."""
32 | pass
33 |
34 | @abstractmethod
35 | async def list_dir(self, path: str) -> Dict[str, Any]:
36 | """List the contents of a directory."""
37 | pass
38 |
39 | @abstractmethod
40 | async def read_text(self, path: str) -> Dict[str, Any]:
41 | """Read the text contents of a file."""
42 | pass
43 |
44 | @abstractmethod
45 | async def write_text(self, path: str, content: str) -> Dict[str, Any]:
46 | """Write text content to a file."""
47 | pass
48 |
49 | @abstractmethod
50 | async def write_bytes(self, path: str, content_b64: str) -> Dict[str, Any]:
51 | """Write binary content to a file. Sent over the websocket as a base64 string."""
52 | pass
53 |
54 | @abstractmethod
55 | async def delete_file(self, path: str) -> Dict[str, Any]:
56 | """Delete a file."""
57 | pass
58 |
59 | @abstractmethod
60 | async def create_dir(self, path: str) -> Dict[str, Any]:
61 | """Create a directory."""
62 | pass
63 |
64 | @abstractmethod
65 | async def delete_dir(self, path: str) -> Dict[str, Any]:
66 | """Delete a directory."""
67 | pass
68 |
69 | @abstractmethod
70 | async def read_bytes(
71 | self, path: str, offset: int = 0, length: Optional[int] = None
72 | ) -> Dict[str, Any]:
73 | """Read the binary contents of a file. Sent over the websocket as a base64 string.
74 |
75 | Args:
76 | path: Path to the file
77 | offset: Byte offset to start reading from (default: 0)
78 | length: Number of bytes to read (default: None for entire file)
79 | """
80 | pass
81 |
82 | @abstractmethod
83 | async def get_file_size(self, path: str) -> Dict[str, Any]:
84 | """Get the size of a file in bytes."""
85 | pass
86 |
87 |
88 | class BaseDesktopHandler(ABC):
89 | """Abstract base class for OS-specific desktop handlers.
90 |
91 | Categories:
92 | - Wallpaper Actions: Methods for wallpaper operations
93 | - Desktop shortcut actions: Methods for managing desktop shortcuts
94 | """
95 |
96 | # Wallpaper Actions
97 | @abstractmethod
98 | async def get_desktop_environment(self) -> Dict[str, Any]:
99 | """Get the current desktop environment name."""
100 | pass
101 |
102 | @abstractmethod
103 | async def set_wallpaper(self, path: str) -> Dict[str, Any]:
104 | """Set the desktop wallpaper to the file at path."""
105 | pass
106 |
107 |
108 | class BaseWindowHandler(ABC):
109 | """Abstract class for OS-specific window management handlers.
110 |
111 | Categories:
112 | - Window Management: Methods for application/window control
113 | """
114 |
115 | # Window Management
116 | @abstractmethod
117 | async def open(self, target: str) -> Dict[str, Any]:
118 | """Open a file or URL with the default application."""
119 | pass
120 |
121 | @abstractmethod
122 | async def launch(self, app: str, args: Optional[List[str]] = None) -> Dict[str, Any]:
123 | """Launch an application with optional arguments."""
124 | pass
125 |
126 | @abstractmethod
127 | async def get_current_window_id(self) -> Dict[str, Any]:
128 | """Get the currently active window ID."""
129 | pass
130 |
131 | @abstractmethod
132 | async def get_application_windows(self, app: str) -> Dict[str, Any]:
133 | """Get windows belonging to an application (by name or bundle)."""
134 | pass
135 |
136 | @abstractmethod
137 | async def get_window_name(self, window_id: str) -> Dict[str, Any]:
138 | """Get the title/name of a window by ID."""
139 | pass
140 |
141 | @abstractmethod
142 | async def get_window_size(self, window_id: str | int) -> Dict[str, Any]:
143 | """Get the size of a window by ID as {width, height}."""
144 | pass
145 |
146 | @abstractmethod
147 | async def activate_window(self, window_id: str | int) -> Dict[str, Any]:
148 | """Bring a window to the foreground by ID."""
149 | pass
150 |
151 | @abstractmethod
152 | async def close_window(self, window_id: str | int) -> Dict[str, Any]:
153 | """Close a window by ID."""
154 | pass
155 |
156 | @abstractmethod
157 | async def get_window_position(self, window_id: str | int) -> Dict[str, Any]:
158 | """Get the top-left position of a window as {x, y}."""
159 | pass
160 |
161 | @abstractmethod
162 | async def set_window_size(
163 | self, window_id: str | int, width: int, height: int
164 | ) -> Dict[str, Any]:
165 | """Set the size of a window by ID."""
166 | pass
167 |
168 | @abstractmethod
169 | async def set_window_position(self, window_id: str | int, x: int, y: int) -> Dict[str, Any]:
170 | """Set the position of a window by ID."""
171 | pass
172 |
173 | @abstractmethod
174 | async def maximize_window(self, window_id: str | int) -> Dict[str, Any]:
175 | """Maximize a window by ID."""
176 | pass
177 |
178 | @abstractmethod
179 | async def minimize_window(self, window_id: str | int) -> Dict[str, Any]:
180 | """Minimize a window by ID."""
181 | pass
182 |
183 |
184 | class BaseAutomationHandler(ABC):
185 | """Abstract base class for OS-specific automation handlers.
186 |
187 | Categories:
188 | - Mouse Actions: Methods for mouse control
189 | - Keyboard Actions: Methods for keyboard input
190 | - Scrolling Actions: Methods for scrolling
191 | - Screen Actions: Methods for screen interaction
192 | - Clipboard Actions: Methods for clipboard operations
193 | """
194 |
195 | # Mouse Actions
196 | @abstractmethod
197 | async def mouse_down(
198 | self, x: Optional[int] = None, y: Optional[int] = None, button: str = "left"
199 | ) -> Dict[str, Any]:
200 | """Perform a mouse down at the current or specified position."""
201 | pass
202 |
203 | @abstractmethod
204 | async def mouse_up(
205 | self, x: Optional[int] = None, y: Optional[int] = None, button: str = "left"
206 | ) -> Dict[str, Any]:
207 | """Perform a mouse up at the current or specified position."""
208 | pass
209 |
210 | @abstractmethod
211 | async def left_click(self, x: Optional[int] = None, y: Optional[int] = None) -> Dict[str, Any]:
212 | """Perform a left click at the current or specified position."""
213 | pass
214 |
215 | @abstractmethod
216 | async def right_click(self, x: Optional[int] = None, y: Optional[int] = None) -> Dict[str, Any]:
217 | """Perform a right click at the current or specified position."""
218 | pass
219 |
220 | @abstractmethod
221 | async def double_click(
222 | self, x: Optional[int] = None, y: Optional[int] = None
223 | ) -> Dict[str, Any]:
224 | """Perform a double click at the current or specified position."""
225 | pass
226 |
227 | @abstractmethod
228 | async def move_cursor(self, x: int, y: int) -> Dict[str, Any]:
229 | """Move the cursor to the specified position."""
230 | pass
231 |
232 | @abstractmethod
233 | async def drag_to(
234 | self, x: int, y: int, button: str = "left", duration: float = 0.5
235 | ) -> Dict[str, Any]:
236 | """Drag the cursor from current position to specified coordinates.
237 |
238 | Args:
239 | x: The x coordinate to drag to
240 | y: The y coordinate to drag to
241 | button: The mouse button to use ('left', 'middle', 'right')
242 | duration: How long the drag should take in seconds
243 | """
244 | pass
245 |
246 | @abstractmethod
247 | async def drag(
248 | self, path: List[Tuple[int, int]], button: str = "left", duration: float = 0.5
249 | ) -> Dict[str, Any]:
250 | """Drag the cursor from current position to specified coordinates.
251 |
252 | Args:
253 | path: A list of tuples of x and y coordinates to drag to
254 | button: The mouse button to use ('left', 'middle', 'right')
255 | duration: How long the drag should take in seconds
256 | """
257 | pass
258 |
259 | # Keyboard Actions
260 | @abstractmethod
261 | async def key_down(self, key: str) -> Dict[str, Any]:
262 | """Press and hold the specified key."""
263 | pass
264 |
265 | @abstractmethod
266 | async def key_up(self, key: str) -> Dict[str, Any]:
267 | """Release the specified key."""
268 | pass
269 |
270 | @abstractmethod
271 | async def type_text(self, text: str) -> Dict[str, Any]:
272 | """Type the specified text."""
273 | pass
274 |
275 | @abstractmethod
276 | async def press_key(self, key: str) -> Dict[str, Any]:
277 | """Press the specified key."""
278 | pass
279 |
280 | @abstractmethod
281 | async def hotkey(self, keys: List[str]) -> Dict[str, Any]:
282 | """Press a combination of keys together."""
283 | pass
284 |
285 | # Scrolling Actions
286 | @abstractmethod
287 | async def scroll(self, x: int, y: int) -> Dict[str, Any]:
288 | """Scroll the specified amount."""
289 | pass
290 |
291 | @abstractmethod
292 | async def scroll_down(self, clicks: int = 1) -> Dict[str, Any]:
293 | """Scroll down by the specified number of clicks."""
294 | pass
295 |
296 | @abstractmethod
297 | async def scroll_up(self, clicks: int = 1) -> Dict[str, Any]:
298 | """Scroll up by the specified number of clicks."""
299 | pass
300 |
301 | # Screen Actions
302 | @abstractmethod
303 | async def screenshot(self) -> Dict[str, Any]:
304 | """Take a screenshot and return base64 encoded image data."""
305 | pass
306 |
307 | @abstractmethod
308 | async def get_screen_size(self) -> Dict[str, Any]:
309 | """Get the screen size of the VM."""
310 | pass
311 |
312 | @abstractmethod
313 | async def get_cursor_position(self) -> Dict[str, Any]:
314 | """Get the current cursor position."""
315 | pass
316 |
317 | # Clipboard Actions
318 | @abstractmethod
319 | async def copy_to_clipboard(self) -> Dict[str, Any]:
320 | """Get the current clipboard content."""
321 | pass
322 |
323 | @abstractmethod
324 | async def set_clipboard(self, text: str) -> Dict[str, Any]:
325 | """Set the clipboard content."""
326 | pass
327 |
328 | @abstractmethod
329 | async def run_command(self, command: str) -> Dict[str, Any]:
330 | """Run a command and return the output."""
331 | pass
332 |
```
--------------------------------------------------------------------------------
/docs/content/docs/computer-sdk/tracing-api.mdx:
--------------------------------------------------------------------------------
```markdown
1 | ---
2 | title: Tracing
3 | description: Record computer interactions for debugging, training, and analysis
4 | ---
5 |
6 | # Tracing
7 |
8 | The Computer tracing API provides a powerful way to record computer interactions for debugging, training, analysis, and compliance purposes. Inspired by Playwright's tracing functionality, it offers flexible recording options and standardized output formats.
9 |
10 | ## Overview
11 |
12 | The tracing API allows you to:
13 |
14 | - Record screenshots at key moments
15 | - Log all API calls and their results
16 | - Capture accessibility tree snapshots
17 | - Add custom metadata
18 | - Export recordings in standardized formats
19 | - Support for both automated and human-in-the-loop workflows
20 |
21 | ## Basic Usage
22 |
23 | ### Starting and Stopping Traces
24 |
25 | ```python
26 | from computer import Computer
27 |
28 | computer = Computer(os_type="macos")
29 | await computer.run()
30 |
31 | # Start tracing with default options
32 | await computer.tracing.start()
33 |
34 | # Perform some operations
35 | await computer.interface.left_click(100, 200)
36 | await computer.interface.type_text("Hello, World!")
37 | await computer.interface.press_key("enter")
38 |
39 | # Stop tracing and save
40 | trace_path = await computer.tracing.stop()
41 | print(f"Trace saved to: {trace_path}")
42 | ```
43 |
44 | ### Custom Configuration
45 |
46 | ```python
47 | # Start tracing with custom configuration
48 | await computer.tracing.start({
49 | 'video': False, # Record video frames
50 | 'screenshots': True, # Record screenshots (default: True)
51 | 'api_calls': True, # Record API calls (default: True)
52 | 'accessibility_tree': True, # Record accessibility snapshots
53 | 'metadata': True, # Allow custom metadata (default: True)
54 | 'name': 'my_custom_trace', # Custom trace name
55 | 'path': './my_traces' # Custom output directory
56 | })
57 |
58 | # Add custom metadata during tracing
59 | await computer.tracing.add_metadata('user_id', 'user123')
60 | await computer.tracing.add_metadata('test_case', 'login_flow')
61 |
62 | # Stop with custom options
63 | trace_path = await computer.tracing.stop({
64 | 'path': './exports/trace.zip',
65 | 'format': 'zip' # 'zip' or 'dir'
66 | })
67 | ```
68 |
69 | ## Configuration Options
70 |
71 | ### Start Options
72 |
73 | | Option | Type | Default | Description |
74 | | -------------------- | ---- | -------------- | ------------------------------------- |
75 | | `video` | bool | `False` | Record video frames (future feature) |
76 | | `screenshots` | bool | `True` | Capture screenshots after key actions |
77 | | `api_calls` | bool | `True` | Log all interface method calls |
78 | | `accessibility_tree` | bool | `False` | Record accessibility tree snapshots |
79 | | `metadata` | bool | `True` | Enable custom metadata recording |
80 | | `name` | str | auto-generated | Custom name for the trace |
81 | | `path` | str | auto-generated | Custom directory for trace files |
82 |
83 | ### Stop Options
84 |
85 | | Option | Type | Default | Description |
86 | | -------- | ---- | -------------- | ---------------------------------- |
87 | | `path` | str | auto-generated | Custom output path for final trace |
88 | | `format` | str | `'zip'` | Output format: `'zip'` or `'dir'` |
89 |
90 | ## Use Cases
91 |
92 | ### Custom Agent Development
93 |
94 | ```python
95 | from computer import Computer
96 |
97 | async def test_custom_agent():
98 | computer = Computer(os_type="linux")
99 | await computer.run()
100 |
101 | # Start tracing for this test session
102 | await computer.tracing.start({
103 | 'name': 'custom_agent_test',
104 | 'screenshots': True,
105 | 'accessibility_tree': True
106 | })
107 |
108 | # Your custom agent logic here
109 | screenshot = await computer.interface.screenshot()
110 | await computer.interface.left_click(500, 300)
111 | await computer.interface.type_text("test input")
112 |
113 | # Add context about what the agent is doing
114 | await computer.tracing.add_metadata('action', 'filling_form')
115 | await computer.tracing.add_metadata('confidence', 0.95)
116 |
117 | # Save the trace
118 | trace_path = await computer.tracing.stop()
119 | return trace_path
120 | ```
121 |
122 | ### Training Data Collection
123 |
124 | ```python
125 | async def collect_training_data():
126 | computer = Computer(os_type="macos")
127 | await computer.run()
128 |
129 | tasks = [
130 | "open_browser_and_search",
131 | "create_document",
132 | "send_email"
133 | ]
134 |
135 | for task in tasks:
136 | # Start a new trace for each task
137 | await computer.tracing.start({
138 | 'name': f'training_{task}',
139 | 'screenshots': True,
140 | 'accessibility_tree': True,
141 | 'metadata': True
142 | })
143 |
144 | # Add task metadata
145 | await computer.tracing.add_metadata('task_type', task)
146 | await computer.tracing.add_metadata('difficulty', 'beginner')
147 |
148 | # Perform the task (automated or human-guided)
149 | await perform_task(computer, task)
150 |
151 | # Save this training example
152 | await computer.tracing.stop({
153 | 'path': f'./training_data/{task}.zip'
154 | })
155 | ```
156 |
157 | ### Human-in-the-Loop Recording
158 |
159 | ```python
160 | async def record_human_demonstration():
161 | computer = Computer(os_type="windows")
162 | await computer.run()
163 |
164 | # Start recording human demonstration
165 | await computer.tracing.start({
166 | 'name': 'human_demo_excel_workflow',
167 | 'screenshots': True,
168 | 'api_calls': True, # Will capture any programmatic actions
169 | 'metadata': True
170 | })
171 |
172 | print("Trace recording started. Perform your demonstration...")
173 | print("The system will record all computer interactions.")
174 |
175 | # Add metadata about the demonstration
176 | await computer.tracing.add_metadata('demonstrator', 'expert_user')
177 | await computer.tracing.add_metadata('workflow', 'excel_data_analysis')
178 |
179 | # Human performs actions manually or through other tools
180 | # Tracing will still capture any programmatic interactions
181 |
182 | input("Press Enter when demonstration is complete...")
183 |
184 | # Stop and save the demonstration
185 | trace_path = await computer.tracing.stop()
186 | print(f"Human demonstration saved to: {trace_path}")
187 | ```
188 |
189 | ### RPA Debugging
190 |
191 | ```python
192 | async def debug_rpa_workflow():
193 | computer = Computer(os_type="linux")
194 | await computer.run()
195 |
196 | # Start tracing with full debugging info
197 | await computer.tracing.start({
198 | 'name': 'rpa_debug_session',
199 | 'screenshots': True,
200 | 'accessibility_tree': True,
201 | 'api_calls': True
202 | })
203 |
204 | try:
205 | # Your RPA workflow
206 | await rpa_login_sequence(computer)
207 | await rpa_data_entry(computer)
208 | await rpa_generate_report(computer)
209 |
210 | await computer.tracing.add_metadata('status', 'success')
211 |
212 | except Exception as e:
213 | # Record the error in the trace
214 | await computer.tracing.add_metadata('error', str(e))
215 | await computer.tracing.add_metadata('status', 'failed')
216 | raise
217 | finally:
218 | # Always save the debug trace
219 | trace_path = await computer.tracing.stop()
220 | print(f"Debug trace saved to: {trace_path}")
221 | ```
222 |
223 | ## Output Format
224 |
225 | ### Directory Structure
226 |
227 | When using `format='dir'`, traces are saved with this structure:
228 |
229 | ```
230 | trace_20240922_143052_abc123/
231 | ├── trace_metadata.json # Overall trace information
232 | ├── event_000001_trace_start.json
233 | ├── event_000002_api_call.json
234 | ├── event_000003_api_call.json
235 | ├── 000001_initial_screenshot.png
236 | ├── 000002_after_left_click.png
237 | ├── 000003_after_type_text.png
238 | └── event_000004_trace_end.json
239 | ```
240 |
241 | ### Metadata Format
242 |
243 | The `trace_metadata.json` contains:
244 |
245 | ```json
246 | {
247 | "trace_id": "trace_20240922_143052_abc123",
248 | "config": {
249 | "screenshots": true,
250 | "api_calls": true,
251 | "accessibility_tree": false,
252 | "metadata": true
253 | },
254 | "start_time": 1695392252.123,
255 | "end_time": 1695392267.456,
256 | "duration": 15.333,
257 | "total_events": 12,
258 | "screenshot_count": 5,
259 | "events": [...] // All events in chronological order
260 | }
261 | ```
262 |
263 | ### Event Format
264 |
265 | Individual events follow this structure:
266 |
267 | ```json
268 | {
269 | "type": "api_call",
270 | "timestamp": 1695392255.789,
271 | "relative_time": 3.666,
272 | "data": {
273 | "method": "left_click",
274 | "args": { "x": 100, "y": 200, "delay": null },
275 | "result": null,
276 | "error": null,
277 | "screenshot": "000002_after_left_click.png",
278 | "success": true
279 | }
280 | }
281 | ```
282 |
283 | ## Integration with ComputerAgent
284 |
285 | The tracing API works seamlessly with existing ComputerAgent workflows:
286 |
287 | ```python
288 | from agent import ComputerAgent
289 | from computer import Computer
290 |
291 | # Create computer and start tracing
292 | computer = Computer(os_type="macos")
293 | await computer.run()
294 |
295 | await computer.tracing.start({
296 | 'name': 'agent_with_tracing',
297 | 'screenshots': True,
298 | 'metadata': True
299 | })
300 |
301 | # Create agent using the same computer
302 | agent = ComputerAgent(
303 | model="openai/computer-use-preview",
304 | tools=[computer]
305 | )
306 |
307 | # Agent operations will be automatically traced
308 | async for _ in agent.run("open cua.ai and navigate to docs"):
309 | pass
310 |
311 | # Save the combined trace
312 | trace_path = await computer.tracing.stop()
313 | ```
314 |
315 | ## Privacy Considerations
316 |
317 | The tracing API is designed with privacy in mind:
318 |
319 | - Clipboard content is not recorded (only content length)
320 | - Screenshots can be disabled
321 | - Sensitive text input can be filtered
322 | - Custom metadata allows you to control what information is recorded
323 |
324 | ## Comparison with ComputerAgent Trajectories
325 |
326 | | Feature | ComputerAgent Trajectories | Computer.tracing |
327 | | ---------------------- | -------------------------- | -------------------- |
328 | | **Scope** | ComputerAgent only | Any Computer usage |
329 | | **Flexibility** | Fixed format | Configurable options |
330 | | **Custom Agents** | Not supported | Fully supported |
331 | | **Human-in-the-loop** | Limited | Full support |
332 | | **Real-time Control** | No | Start/stop anytime |
333 | | **Output Format** | Agent-specific | Standardized |
334 | | **Accessibility Data** | No | Optional |
335 |
336 | ## Best Practices
337 |
338 | 1. **Start tracing early**: Begin recording before your main workflow to capture the complete session
339 | 2. **Use meaningful names**: Provide descriptive trace names for easier organization
340 | 3. **Add contextual metadata**: Include information about what you're testing or demonstrating
341 | 4. **Handle errors gracefully**: Always stop tracing in a finally block
342 | 5. **Choose appropriate options**: Only record what you need to minimize overhead
343 | 6. **Organize output**: Use custom paths to organize traces by project or use case
344 |
345 | The Computer tracing API provides a powerful foundation for recording, analyzing, and improving computer automation workflows across all use cases.
346 |
```
--------------------------------------------------------------------------------
/blog/hack-the-north.md:
--------------------------------------------------------------------------------
```markdown
1 | # What happens when hackathon judging is a public benchmark (Hack the North edition)
2 |
3 | _Written by Francesco Bonacci — Reviewed by Parth Patel (HUD W25) — Sept 25, 2025_
4 |
5 | ## Prologue
6 |
7 | Hack the North ran Sept 12–14 at the University of Waterloo. Official count this year: **1,778 hackers**, and a [Guinness World Record for the most people building interlocking plastic brick sculptures simultaneously](https://uwaterloo.ca/news/eweal-making-hackathons-fun-again-breaking-guinness-world-record).
8 |
9 | Our team arrived from Europe and the US one day before the hackathon, after a summer scattered post–YC X25, waiting for our O-1 visas. **HUD**’s founders Parth and Jay flew in from SF to help us run evaluations, and Michael and Parth from **Ollama** joined as co-sponsors.
10 |
11 | Our plan was ambitious: run the **first state-of-the-art Computer-Use Agents track**, score it on a public benchmark, and give the top performer a guaranteed YC interview. (Interview ≠ offer. YC didn’t judge.)
12 |
13 | The rest, as they say, was a 36h story worth telling—and a playbook worth sharing for anyone thinking about running or sponsoring this type of hackathon track.
14 |
15 | 
16 |
17 | ## The sign-up problem we had to invent
18 |
19 | We joined as a sponsor at the last minute, thanks to a push from our friend @Michael Chiang at Ollama—Waterloo alum, naturally. It’s kind of an open secret that UWaterloo turns out some of the sharpest hackers around (_no pun intended, HackMIT_). It was a bit of a scramble, but also great timing—our Agent framework had just finished a major refactor, with support for **100+ VLM configurations** now live. Naturally, we wanted to stress-test it at scale—and see whether teams could come up with SOTA-level setups. _This wasn’t a blank-slate, build-whatever-you-want kind of track._
20 |
21 | From day one, though, we knew we’d have to fight for sign-ups. This was a niche track, and a guaranteed YC interview alone wouldn’t be enough to pull people in.
22 |
23 | Unfortunately, Hack the North (HTN) didn’t offer an interest form to help us estimate demand, which made capacity planning tricky—especially with early-stage infra. Stress-testing takes foresight, and multimodal language model usage is still costly (~1.5× to 3–4× the price of comparable text-only models).
24 |
25 | On top of that, we were discouraged from external promotion on [lu.ma](http://lu.ma). So we spun up our own sign-up page at **cua.ai/hackathon** and built ad-hoc Discord channels to share track details. We emphasized—repeatedly—that only students already accepted to Hack the North should register.
26 |
27 | _(Moral: the “measure-zero effect”—no matter how many times you say it, some people won’t see it. Plenty of invalid sign-ups still slipped through.)_
28 |
29 | Even so, having your own form is absolutely worth it: it gives you an **early funnel**, surfaces demand signals ahead of time, and—crucially—**lets you require platform sign-up before kickoff**. In our case, Hack the North didn’t provide Devpost access until the very end, so our form was the only way to build a working roster.
30 |
31 | Only a small trickle of sign-ups came through by the time the event kicked off—too few to plan around, but clearly the right kind of crowd. Several were already familiar with computer-use agents; one was even interning at Shopify, working on this space.
32 |
33 | ## At the Sponsor Booth
34 |
35 | Day 0 on campus made the difference. We arrived a couple of hours early to collect swag shipments (around 1,200 stickers of our new **Cua-la** mascot, plus t-shirts and hats—always plan ~1.5× the estimated number of hackers!). After walking the sponsor floor and explaining the track at our booth, ~40 hackers signed up.
36 |
37 | **Moral:** sponsor booths are still the most effective way to recruit for a track.
38 |
39 | **Suggestions to maximize booth time (for HTN this is only ~24 of the total 36 hours):**
40 |
41 | - **Be unmistakable.** Run a mini-challenge and a visible giveaway. We offered 5 × $200 Anthropic credits as a lightning raffle and constantly advertised in HTN Slack. Shout-out to our neighbors at **Mintlify**, who dressed their teammate as a mint plant - memorable and effective.
42 | - **Create multiple touchpoints.** Hand out flyers and QR codes, and ask nearby booths to cross-refer. Big thanks to the YC team for flyer space and student connections - and to Michael (Ollama) for pointing visitors our way.
43 | - **Never leave the booth empty.** Keep someone at the booth at all times and rotate shifts. With four founding engineers on-site, coverage was easy. Even after hacking kicked off, the booth stayed a point of reference - and even then multiple participants DM’d us asking where to meet up.
44 | - **Students are organic DevRel.** Our runner-up, Adam, hung out with us at the booth, pulling more people in. Peer-to-peer energy creates the network effect you need!
45 |
46 | 
47 |
48 | _(Our Founding Engineer, Morgan, hangs out with students at the stand, while Adam (runner-up) hacks on the side.)_
49 |
50 | ## 02:30 a.m. is still prime time at a hackathon
51 |
52 | Hack the North gives sponsors a 30-minute API Workshop during the early hours of the event—a perfect moment to shift from talking to building.
53 |
54 | Our slot landed at **2:30 a.m.** (_perks of the cheapest sponsor tier_). Thirty students showed up, energy surprisingly high. James, our new Founding DevRel Engineer, led the session and nailed it.
55 |
56 | **Our track rules were simple:**
57 |
58 | 1. Build a Computer-Use Agent with the [Cua framework](https://github.com/trycua/cua)
59 | 2. Benchmark the agent on [HUD](https://www.hud.so)
60 | 3. Use [OSWorld-Tiny](https://huggingface.co/datasets/ddupont/OSWorld-Tiny-Public): a 14-task distillation of the full benchmark (~360 tasks, >1h)
61 |
62 | **Suggestions:**
63 |
64 | - **Leave something tangible.** We provided a Jupyter Notebook teams could run immediately.
65 | - **Narrow scope, strong starts.** The more focused the challenge, the more **robust starting points** you should provide.
66 | - **Want the details?** [Here’s the notebook we left participants](https://github.com/trycua/cua/blob/main/notebooks/sota_hackathon.ipynb).
67 |
68 | 
69 |
70 | _(Our CUA Workshop at 2:30 AM.)_
71 |
72 | ## Making it possible to focus on the work
73 |
74 | If you’re an OSS framework, it’s tempting to have hackers self-host on laptops. **Don’t.** You’ll spend the workshop debugging setups instead of reviewing ideas.
75 |
76 | **Lesson learned:** within hours, we shifted to **cloud-only Sandboxes**. Payoff: consistent environments, faster starts, far less tech support.
77 |
78 | We provided:
79 |
80 | - **Credits:** $200 Cua Cloud + $200 HUD per team (manual top-ups for visible progress)
81 | - **LLMs/VLMs:** Anthropic assigned $50 per participant—tight for VLM iteration—so we added capped access under our org
82 | - **Pre-kickoff provisioning:** Platform sign-up auto-created projects, keys, and sandboxes
83 |
84 | **Takeaway:** every minute not spent on setup is a minute gained for iterating.
85 |
86 | ## 12 Hours in the Hackathon
87 |
88 | **After the workshop buzz.** Morning interest was high, but Docker setup + requiring focus on a single track thinned the crowd. Most sponsor prizes are broad (“use our product and you qualify”), letting students stack tracks. Ours required commitment. Upside: those who stayed shipped sharper, higher-quality submissions.
89 |
90 | **The bell curve of submissions.** Most entries used _claude-sonnet-4-20250514_—proof that docs and public leaderboards ([OSWorld](https://os-world.github.io/#benchmark)) guide choices. Results clustered around the safe pick, with fewer pushing boundaries.
91 |
92 | **Who went beyond the baseline.** A few tried multi-agent/tool graphs. One standout—[**cuala**](https://github.com/YeIIcw/cuala)—was a clean reference: deterministic actions, verifiable state changes, callbacks for saving images and trajectories.
93 |
94 | **Bottom line:** Early excitement is easy; keeping teams engaged requires reducing friction and offering multiple entry points.
95 |
96 | ### What broke (and why)
97 |
98 | We skipped a full end-to-end **Cua × HUD** dry-run. It showed.
99 |
100 | - Hackers ran out of inference credits. Desktop tasks are token-heavy. A full OSWorld run (200 max steps) for _computer-use-preview_ (OpenAI Operator API) can cost >$600. Serious attempts: ~400k tokens × 14 tasks.
101 | - Python version/build mismatches surfaced, requiring debug time across both OSS repos.
102 | - Our Cua framework lacked a **Response Agent** to complete evaluation loops. Some runs stalled until patched.
103 |
104 | ## Scoring and Results
105 |
106 | ### Participation & Outcomes
107 |
108 | - ~**30** hackers gave the track a serious try; **5** crossed the finish line
109 | - All submissions were **solo**, mostly undergrads
110 | - Judging: OSWorld-Tiny on HUD, with Cua + HUD reruns to verify scores
111 | - Final leaderboard: [HUD Leaderboard](https://www.hud.so/leaderboards/ddupont/OSWorld-Tiny-Public)
112 |
113 | 
114 |
115 | _(Leaderboard on HUD)_
116 |
117 | ### Winners
118 |
119 | **🥇 Winner — Ram**
120 |
121 | - Devpost: https://devpost.com/software/sota-computer-use-agent-challenge
122 | - Code: https://github.com/Ram-Raghav-S/cua/tree/ram
123 | - Score: 68.3%
124 |
125 | **🥈 Runner-up — Aryan**
126 |
127 | - Devpost: https://devpost.com/software/loopdeloop-computer-use-agent-sota-attempt
128 | - Code: https://github.com/Tumph/cua
129 | - Score: 55.9%
130 |
131 | **🥉 Special Mention — Adam**
132 |
133 | - Devpost: https://devpost.com/software/cuala
134 | - Code: https://github.com/YeIIcw/cuala
135 | - Score: 42.1%
136 |
137 | 
138 |
139 | _(Our finalists before the award ceremony)_
140 |
141 | ## What We’d Keep
142 |
143 | - **Sponsor Hack the North again**
144 | - **Keep a visible, staffed booth**
145 | - **Publish a compact FAQ**
146 | - **Simple, transparent scoring**
147 |
148 | ## What We’d Change
149 |
150 | - **Run a full Cua × HUD dry-run under load**
151 | - **Offer multiple on-ramps (evals, creative, RL)**
152 | - **Keep a private eval set for judging**
153 | - **Default to cloud sandboxes**
154 | - **Handle ops earlier (swag, signage, QR codes)**
155 | - **Reward generalization, not lucky runs**
156 |
157 | ## Closing Thoughts
158 |
159 | Our first outing as sponsors wasn’t perfect, but it gave us a working playbook: **provision cloud early, keep scoring simple, always dry-run infra, and make the booth unforgettable**.
160 |
161 | If more hackathon tracks leaned on **public benchmarks**, weekends like this would produce fewer demos-for-show and more measurable progress.
162 |
163 | **P.S.** Huge thanks to the Ollama and HUD teams for co-sponsoring the track, and to our YC Partner Diana for offering a **guaranteed YC interview** as first prize.
164 |
165 | Whether you’re a hacker who wants to participate, or a company looking to sponsor, let’s talk — we’re especially excited to support benchmark-first hackathon tracks in the Bay Area this year.
166 |
167 | 
168 |
169 | _(HTN Closing Ceremony — Cua Track Winner Announcement)_
170 |
```