How to Run RL Swarm on Vast.ai

Looking to join the decentralized AI revolution? RL Swarm lets you train reinforcement learning models collaboratively with others, leveraging the power of a global peer-to-peer network. With Vast.ai, you can rent powerful GPUs at low cost and get your node running in minutes.

What is RL Swarm?

RL Swarm is an open-source, permissionless system for collaborative reinforcement learning. You can run it on anything from a home laptop to a top-tier cloud GPU. Connect your node to the Gensyn Testnet to receive an on-chain identity and track your progress.

Currently, the swarm is training models to solve reasoning tasks using the reasoning-gym dataset. Default models include:

• Gensyn/Qwen2.5-0.5B-Instruct
• Qwen/Qwen3-0.6B
• nvidia/AceInstruct-1.5B
• dnotitia/Smoothie-Qwen3-1.7B
• Gensyn/Qwen2.5-1.5B-Instruct

This system is powered by the GenRL library, enabling fully composable, decentralized RL with multi-agent, multi-stage environments.

Requirements

• CPU: arm64 or x86 with at least 32GB RAM (training may crash if you run other apps simultaneously)
• GPU (recommended):
  • NVIDIA RTX 3090, 4090, 5090, A100, H100
• OS: Ubuntu 22.04 (recommended)
• Python: >= 3.10
• Docker: Required for easy setup

Tip: Vast.ai lets you filter for machines with the right specs.

Step 1: Sign Up for Vast.ai

Sign up using our affiliate link to support GSWARM at no extra cost.

Step 2: Rent a GPU Instance

1. Click Create on Vast.ai and filter for a supported GPU (e.g., RTX 3090, A100).
2. Choose Ubuntu 22.04 as your OS image.
3. Set your SSH key and launch the instance.

Step 3: Prepare Your Instance

SSH into your instance. If Docker isn’t installed, run:

sudo apt update && sudo apt install -y docker.io docker-compose
sudo systemctl enable --now docker

Step 4: Clone RL Swarm

git clone https://github.com/gensyn-ai/rl-swarm
cd rl-swarm

Step 5: Start the Swarm

• For CPU-only (e.g., Mac, CPU VM):

  docker-compose run --rm --build -Pit swarm-cpu
  

• For GPU (recommended):

  docker-compose run --rm --build -Pit swarm-gpu
  

  If you get an error with docker-compose, try docker compose (no hyphen).

Step 6: Login & Identity

• A browser window will open for login (or visit http://localhost:3000/ on your VM).
• Log in with your preferred method (email, Google, etc.).
• This creates your on-chain identity and a swarm.pem file. Keep this file safe!
• If you want to run multiple nodes, use the same email for each.

Step 7: Training & Monitoring

• Your node will begin training and register on-chain.
• Track your progress on the RL Swarm Dashboard.
• Logs are available in the /logs directory inside the repo.

Troubleshooting & Tips

• OOM/Memory errors: Increase Docker memory allocation in Docker settings.
• Login issues: Delete swarm.pem and re-login if you change accounts.
• VM/Port forwarding: Use ssh -L 3000:localhost:3000 ... to access the login screen from your local browser.
• Multiple GPUs: Install RL Swarm separately for each GPU and expose different ports.
• Custom models: Advanced users can specify a custom model repo/name when prompted.
• Windows: Use WSL2 and Ubuntu for best results.

Note: This software is experimental. If you encounter issues, check the issues page or join the Gensyn Discord for help.

---

Ready to swarm? Rent a GPU on Vast.ai and join the decentralized AI movement today!