-
-
Notifications
You must be signed in to change notification settings - Fork 648
Add QEMU Windows 11 template with CUA computer-server support #551
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
synacktraa
wants to merge
11
commits into
trycua:main
Choose a base branch
from
synacktraa:feat/qemu-windows-container
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 7 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
ece1738
feat: add QEMU Windows 11 template
synacktraa 277597e
feat(qemu): add CUA server setup script with virtual environment isol…
synacktraa 193b837
feat(qemu): add Caddy reverse proxy setup script with hidden execution
synacktraa b92e179
fix(qemu): update setup orchestration to use scheduled tasks
synacktraa 2b0da42
fix(qemu): add setup detection and boot marker file creation
synacktraa 8265403
chore(qemu): set "dev" as default DEPLOY_MODE
synacktraa cea9289
docs(qemu): update README doc
synacktraa c694c6a
refactor(qemu): centralize logging with shared Write-Log function
synacktraa c59aa52
chore(qemu): remove unnecessary comments
synacktraa 8150d02
refactor: restructure as qemu-docker with OS-specific folders
synacktraa dbc00ba
refactor: simplify setup using trycua/windows-local
synacktraa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # CUA Windows Container Dockerfile | ||
| # | ||
| # Uses windowsarena/windows-local:latest instead of windowsarena/winarena-base:latest | ||
| # to avoid unnecessary bloat from ML dependencies: | ||
| # - Removes Python 3.9 and ML client dependencies (easyocr, onnxruntime, etc.) | ||
| # - Removes CUDA 11.8 libraries | ||
| # - Removes ML model weights (GroundingDINO, OmniParser) | ||
| # This results in a significantly smaller image focused purely on QEMU/Windows with CUA computer-server. | ||
|
|
||
| # Define build argument for deployment mode (default is dev, can also be azure) | ||
| ARG DEPLOY_MODE="dev" | ||
|
|
||
| # Conditional copy of files based on build argument | ||
| FROM windowsarena/windows-local:latest AS build_dev | ||
| ONBUILD COPY src/vm/setup/. /shared/ | ||
| ONBUILD COPY src/vm/unattend-files/dev_win11x64-enterprise-eval.xml /run/assets/win11x64-enterprise-eval.xml | ||
| ONBUILD ENV FOLDER_NAME=shared | ||
|
|
||
| FROM windowsarena/windows-local:latest AS build_azure | ||
| ONBUILD COPY src/vm/setup/. /oem/ | ||
| ONBUILD COPY src/vm/unattend-files/azure_win11x64-enterprise-eval.xml /run/assets/win11x64-enterprise-eval.xml | ||
| ONBUILD ENV FOLDER_NAME=oem | ||
|
|
||
| FROM build_${DEPLOY_MODE} | ||
|
|
||
| ARG DEPLOY_MODE="dev" | ||
| ENV DEPLOY_MODE=${DEPLOY_MODE} | ||
|
|
||
| RUN echo "FOLDER_NAME: ${FOLDER_NAME}" | ||
| RUN echo "DEPLOY_MODE: ${DEPLOY_MODE}" | ||
|
|
||
| # If azure, replace windows data folder with oem folder | ||
| RUN if [ "${DEPLOY_MODE}" = "azure" ]; then \ | ||
| WINDOWS_DATA_FOLDER='\\\\host.lan\\Data'; \ | ||
| WINDOWS_OEM_FOLDER='C:\\oem'; \ | ||
| OEM_FOLDER='oem'; \ | ||
| sed -i "s|${WINDOWS_DATA_FOLDER}|${WINDOWS_OEM_FOLDER}|g" "/${OEM_FOLDER}/on-logon.ps1"; \ | ||
| sed -i "s|${WINDOWS_DATA_FOLDER}|${WINDOWS_OEM_FOLDER}|g" "/${OEM_FOLDER}/setup.ps1"; \ | ||
| fi | ||
|
|
||
| # Install fuse and socat (for port forwarding) | ||
| RUN apt-get update && apt-get install -y fuse socat | ||
|
|
||
| # Copy entry script | ||
| COPY src/entry.sh /entry.sh | ||
| RUN chmod +x /entry.sh | ||
|
|
||
| ENV YRES="900" | ||
| ENV XRES="1440" | ||
| ENV RAM_SIZE="8G" | ||
| ENV CPU_CORES="8" | ||
| ENV VERSION="win11x64-enterprise-eval" | ||
| ENV DISK_SIZE="30G" | ||
|
|
||
| # Enable QEMU's JSON-based QEMU Machine Protocol (QMP) | ||
| ENV ARGUMENTS="-qmp tcp:0.0.0.0:7200,server,nowait" | ||
|
|
||
| # Expose ports | ||
| EXPOSE 5000 8006 | ||
|
|
||
| ENTRYPOINT ["/entry.sh"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,195 @@ | ||
| # CUA Windows Container | ||
|
|
||
| Containerized Windows 11 virtual desktop for Computer-Using Agents (CUA). Utilizes QEMU/KVM with Windows 11 and computer-server pre-installed for remote computer control. | ||
|
|
||
| ## Features | ||
|
|
||
| - Windows 11 Enterprise running in QEMU/KVM | ||
| - Pre-installed CUA computer-server for remote computer control | ||
| - Caddy reverse proxy (port 9222 → 1337) for browser automation | ||
| - noVNC access for visual desktop interaction | ||
| - Automated setup via unattended installation | ||
| - Support for both dev (shared folder) and azure (OEM folder) deployment modes | ||
| - Python 3.12 with isolated virtual environment for CUA computer-server | ||
| - Services run hidden in background via Windows scheduled tasks | ||
| - Essential tools pre-installed (Chrome, LibreOffice, VLC, GIMP, VSCode, Thunderbird) | ||
|
|
||
| ## Quick Start | ||
|
|
||
| ### 1. Download and Prepare setup.iso | ||
|
|
||
| **Download Windows 11 Evaluation ISO:** | ||
|
|
||
| 1. Visit [Microsoft Evaluation Center](https://info.microsoft.com/ww-landing-windows-11-enterprise.html) | ||
| 2. Accept the Terms of Service | ||
| 3. Download **Windows 11 Enterprise Evaluation (90-day trial, English, United States)** ISO file [~6GB] | ||
| 4. After downloading, rename the file to `setup.iso` | ||
| 5. Copy it to the directory `src/vm/image/` | ||
|
|
||
| This ISO is used for automated Windows installation on first run. | ||
|
|
||
| ### 2. Build the Image | ||
|
|
||
| ```bash | ||
| # Build for dev mode (uses shared folder) | ||
| docker build --build-arg DEPLOY_MODE=dev -t cua-windows:dev . | ||
|
|
||
| # Or build for azure mode (uses OEM folder) | ||
| docker build --build-arg DEPLOY_MODE=azure -t cua-windows:azure . | ||
| ``` | ||
|
|
||
| ### 3. First Run - Create Golden Image | ||
|
|
||
| On first run, the container will install Windows from scratch and create a golden image. This takes 15-30 minutes. | ||
|
|
||
| ```bash | ||
| # Create storage directory | ||
| mkdir -p ./storage | ||
|
|
||
| # Run with setup.iso to create golden image | ||
| docker run -it --rm \ | ||
| --device=/dev/kvm \ | ||
| --platform linux/amd64 \ | ||
| --name cua-windows \ | ||
| --mount type=bind,source=$(pwd)/src/vm/image/setup.iso,target=/custom.iso \ | ||
| --cap-add NET_ADMIN \ | ||
| -v $(pwd)/storage:/storage \ | ||
| -p 8006:8006 \ | ||
| -p 5000:5000 \ | ||
| -e RAM_SIZE=8G \ | ||
| -e CPU_CORES=4 \ | ||
| -e DISK_SIZE=20G \ | ||
| cua-windows:dev | ||
| ``` | ||
|
|
||
| **What happens during first run:** | ||
|
|
||
| 1. Windows 11 installs automatically using unattended configuration | ||
| 2. Setup scripts install Python 3.12 (via Chocolatey), essential tools, and CUA computer-server in isolated venv | ||
| 3. Windows scheduled tasks created for CUA server and Caddy proxy (run hidden in background) | ||
| 4. Golden image is saved to `/storage` directory | ||
| 5. Container exits after setup completes | ||
|
|
||
| ### 4. Subsequent Runs - Use Golden Image | ||
|
|
||
| After the golden image is created, subsequent runs boot much faster (30 sec - 2 min): | ||
|
|
||
| ```bash | ||
| # Run without setup.iso - uses existing golden image | ||
| docker run -it --rm \ | ||
| --device=/dev/kvm \ | ||
| --platform linux/amd64 \ | ||
| --name cua-windows \ | ||
| --cap-add NET_ADMIN \ | ||
| -v $(pwd)/storage:/storage \ | ||
| -p 8006:8006 \ | ||
| -p 5000:5000 \ | ||
| -e RAM_SIZE=8G \ | ||
| -e CPU_CORES=4 \ | ||
| cua-windows:dev | ||
| ``` | ||
|
|
||
| **Access points:** | ||
|
|
||
| - **Computer Server API**: `http://localhost:5000` | ||
| - **noVNC Browser**: `http://localhost:8006` | ||
|
|
||
| ## Container Configuration | ||
|
|
||
| ### Ports | ||
|
|
||
| - **5000**: CUA computer-server API endpoint | ||
| - **8006**: noVNC web interface for visual desktop access | ||
|
|
||
| ### Environment Variables | ||
|
|
||
| - `RAM_SIZE`: RAM allocated to Windows VM (default: "8G", recommended: "8G" for WSL2) | ||
| - `CPU_CORES`: CPU cores allocated to VM (default: "8") | ||
| - `DISK_SIZE`: VM disk size (default: "30G", minimum: "20G") | ||
| - `YRES`: Screen height (default: "900") | ||
| - `XRES`: Screen width (default: "1440") | ||
| - `VERSION`: Windows version (default: "win11x64-enterprise-eval") | ||
| - `DEPLOY_MODE`: Deployment mode - "dev" or "azure" (set during build) | ||
|
|
||
| ### Volumes | ||
|
|
||
| - `/storage`: Persistent VM storage (golden image, disk, firmware) | ||
| - `/custom.iso`: Mount point for setup.iso (only needed for first run) | ||
|
|
||
| ### Deployment Modes | ||
|
|
||
| **Dev Mode (`DEPLOY_MODE=dev`):** | ||
|
|
||
| - Setup files mounted at `/shared` in container | ||
| - Accessible to Windows VM via `\\host.lan\Data` | ||
| - Useful for development and testing | ||
|
|
||
| **Azure Mode (`DEPLOY_MODE=azure`):** | ||
|
|
||
| - Setup files copied to `/oem` in container | ||
| - Accessible to Windows VM via `C:\oem` | ||
| - Used for production deployment on Azure ML | ||
|
|
||
| ## Architecture | ||
|
|
||
| ``` | ||
| ┌─────────────────────────────────────────────────────────┐ | ||
| │ Docker Container (Linux host) │ | ||
| │ │ | ||
| │ • Port forwarding: localhost:5000 → EMULATOR_IP:5000 │ | ||
| │ • Exposes: 5000 (API), 8006 (noVNC) │ | ||
| │ │ | ||
| │ ┌────────────────────────────────────────────────────┐ │ | ||
| │ │ QEMU VM (Windows 11) │ │ | ||
| │ │ │ │ | ||
| │ │ • CUA computer-server listens on 5000 │ │ | ||
| │ │ │ │ | ||
| │ └────────────────────────────────────────────────────┘ │ | ||
| │ │ | ||
| └─────────────────────────────────────────────────────────┘ | ||
| ``` | ||
|
|
||
| **Communication Flow:** | ||
|
|
||
| 1. External client → `localhost:5000` (host) | ||
| 2. Docker port mapping → Container's `localhost:5000` | ||
| 3. socat port forwarding → `20.20.20.21:5000` (VM) | ||
| 4. CUA computer-server in Windows VM processes request | ||
|
|
||
| ## Development | ||
|
|
||
| ### Modifying Setup Scripts | ||
|
|
||
| Setup scripts are in `src/vm/setup/`: | ||
|
|
||
| - `install.bat`: Entry point called by Windows setup | ||
| - `setup.ps1`: Main setup orchestration (installs software, configures Windows) | ||
| - `setup-cua-server.ps1`: CUA server installation with isolated venv | ||
| - `setup-caddy-proxy.ps1`: Caddy reverse proxy configuration | ||
| - `on-logon.ps1`: Runs on user logon (starts scheduled tasks) | ||
| - `setup-tools.psm1`: PowerShell helper functions | ||
|
|
||
| After modifying, rebuild the image: | ||
|
|
||
| ```bash | ||
| docker build --build-arg DEPLOY_MODE=dev -t cua-windows:dev . | ||
| ``` | ||
|
|
||
| ### Customizing Windows Configuration | ||
|
|
||
| Unattended installation files are in `src/vm/unattend-files/`: | ||
|
|
||
| - `dev_win11x64-enterprise-eval.xml`: For dev mode | ||
| - `azure_win11x64-enterprise-eval.xml`: For Azure mode | ||
|
|
||
| These XML files control: | ||
|
|
||
| - User account creation | ||
| - Auto-logon settings | ||
| - First-run commands | ||
| - Timezone and locale | ||
|
|
||
| ## Credits | ||
|
|
||
| - Built on [Dockur Windows](https://github.com/dockur/windows) base image | ||
| - Inspired by [Windows Agent Arena](https://github.com/microsoft/WindowsAgentArena) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Fix for Azure ML Job not using the correct root path | ||
| cd / | ||
|
|
||
| # Create windows.boot file if it doesn't exist (required for proper boot) | ||
| if [ -d "/storage" -a ! -f "/storage/windows.boot" ]; then | ||
| echo "Creating windows.boot file in /storage..." | ||
| touch /storage/windows.boot | ||
| fi | ||
|
|
||
| # Start the VM in the background | ||
| echo "Starting Windows VM..." | ||
| /usr/bin/tini -s /run/entry.sh & | ||
| echo "Live stream accessible at localhost:8006" | ||
|
|
||
| echo "Waiting for Windows to boot and CUA computer-server to start..." | ||
| while true; do | ||
| # Send a GET request to check if server is ready | ||
| response=$(curl --write-out '%{http_code}' --silent --output /dev/null 20.20.20.21:5000/status) | ||
|
|
||
| # If the response code is 200 (HTTP OK), break the loop | ||
| if [ "${response:-0}" -eq 200 ]; then | ||
| break | ||
| fi | ||
|
|
||
| echo "Waiting for CUA computer-server to be ready. This might take a while..." | ||
|
|
||
| # Wait for a while before the next attempt | ||
| sleep 5 | ||
| done | ||
|
|
||
| echo "VM is up and running, and the CUA Computer Server is ready!" | ||
|
|
||
| # Set up port forwarding from localhost:5000 to VM emulator IP | ||
| echo "Setting up port forwarding: localhost:5000 -> 20.20.20.21:5000" | ||
| socat TCP-LISTEN:5000,fork,reuseaddr TCP:20.20.20.21:5000 & | ||
|
|
||
| echo "Computer server accessible at localhost:5000" | ||
|
|
||
| # Detect initial setup by presence of /custom.iso (setup ISO mount) | ||
| if [ ! -f "/custom.iso" ]; then | ||
| # Keep container alive | ||
| echo "Container running. Press Ctrl+C to stop." | ||
| tail -f /dev/null | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| > Add your Win11E setup.iso to this folder | ||
|
|
||
| **Download Windows 11 Evaluation ISO:** | ||
|
|
||
| 1. Visit [Microsoft Evaluation Center](https://info.microsoft.com/ww-landing-windows-11-enterprise.html) | ||
| 2. Accept the Terms of Service | ||
| 3. Download **Windows 11 Enterprise Evaluation (90-day trial, English, United States)** ISO file [~6GB] | ||
| 4. After downloading, rename the file to `setup.iso` | ||
| 5. Copy it to the current directory. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| @echo off | ||
|
|
||
| SET ScriptFolder=\\host.lan\Data | ||
| SET LogFile=%ScriptFolder%\ps_script_log.txt | ||
|
|
||
| echo Running PowerShell script... > %LogFile% | ||
|
|
||
| :: Check for PowerShell availability | ||
| where powershell >> %LogFile% 2>&1 | ||
| if %ERRORLEVEL% neq 0 ( | ||
| echo PowerShell is not available! >> %LogFile% | ||
| echo PowerShell is not available! | ||
| exit /b 1 | ||
| ) | ||
|
|
||
| :: Add a 30-second delay | ||
| echo Waiting for 30 seconds before continuing... >> %LogFile% | ||
| timeout /t 30 /nobreak >> %LogFile% 2>&1 | ||
|
|
||
| :: Run PowerShell script with ExecutionPolicy Bypass and log errors | ||
| echo Running setup.ps1... >> %LogFile% | ||
|
|
||
| powershell -ExecutionPolicy Bypass -File "%ScriptFolder%\setup.ps1" >> %LogFile% 2>&1 | ||
|
|
||
| if %ERRORLEVEL% neq 0 ( | ||
| echo An error occurred. See %LogFile% for details. | ||
| ) else ( | ||
| echo PowerShell script has completed successfully. | ||
| ) | ||
|
|
||
| echo PowerShell script has completed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # Start the Caddy Reverse Proxy scheduled task (runs in background, hidden) | ||
| Write-Host "Starting Caddy Reverse Proxy task..." | ||
| Start-ScheduledTask -TaskName "Caddy-Reverse-Proxy" | ||
|
|
||
| # Start the CUA Computer Server scheduled task (runs in background, hidden) | ||
| Write-Host "Starting CUA Computer Server task..." | ||
| Start-ScheduledTask -TaskName "CUA-Computer-Server" |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.