Add Ray integration for SGE/Grid Engine clusters

[Copilot]

Enables Ray distributed computing framework deployment on SGE infrastructure, similar to existing SLURM integrations.

Implementation

Go cluster manager (examples/ray/ray_cluster_manager.go)

• Manages Ray head/worker lifecycle via qsub/qstat APIs
• Extracts hostnames from queue@hostname format for worker discovery
• Signal-based graceful shutdown

Shell scripts for rapid deployment

• start_ray_cluster.sh: Launches head + N workers as SGE jobs
• stop_ray_cluster.sh: Cleanup via qdel
• POSIX-compliant regex (sed) for cross-platform job ID extraction

Python example (example_ray_job.py)

• Monte Carlo Pi estimation (vectorized numpy)
• Distributed matrix ops and Ray actors
• Auto-connect to SGE-managed cluster

Documentation (docs/ray-integration.md, examples/ray/README.md)

• Manual vs. automated cluster launch patterns
• Resource configuration (memory, slots, queues)
• Network requirements and troubleshooting

Usage

# Quick start
cd examples/ray
./start_ray_cluster.sh 10  # 10 workers

# From Python
python example_ray_job.py --address <head_host>:6379

# Or programmatic Go control
go run ray_cluster_manager.go

// Go API usage
manager, _ := NewRayClusterManager()
manager.StartHead(ctx, 6379, "16G")
manager.AddWorkers(ctx, 5, headAddr, 6379, "8G")

Key API patterns discovered:

• qstat.ViewJobsOfUser([]) returns current user's jobs
• qsub.SimpleLRequest(map[string]string{"mem_free": "16G"}) for resource specs
• JobInfo uses JobID (int), not JobNumber

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• hpc-gridware.com
  • Triggering command: /usr/bin/wget wget -O /opt/ocs/ocs-9.0.8-lx-amd64.tar.gz REDACTED --pr�� la/IdenTrust_Public_Sector_Root_CA_1.crt ot_CA.crt t_ECC_Certification_Authority.crt r_manager.go (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

I want to integreate SGE (like SLURM) into Ray. where to do

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.