Overview
DI-star: A large-scale game AI distributed training platform specially developed for the StarCraft II. We’ve already trained grand-master AI！This project contains:
Usage
Testing software on Windows | 对战软件下载
Please star us (click 
button in the top-right of this page) to help DI-star agents to grow up faster :)
Installation
Environment requirement:
1.Install StarCraftII
Note: There is no retail version on Linux, please follow the instruction here
2.Install distar:
git clone https://github.com/opendilab/DI-star.git
cd DI-star
pip install -e .
3.Install pytorch:
Pytorch Version 1.7.1 and CUDA is recommended, Follow instructions from pytorch official site
Note: GPU is neccessary for decent performance in realtime agent test, you can also use pytorch without cuda, but no performance guaranteed due to inference latency on cpu.
Make sure you set SC2 at lowest picture quality before testing.
Play with pretrained agent
1. Download StarCraftII version 4.10.0
Double click the file data/replays/replay_4.10.0.SC2Replay, StarCraftII version 4.10.0 will be automatically downloaded.
Note: We trained our models with versions from 4.8.2 to 4.9.3. Patch 5.0.9 has came out in March 15, 2022, Some changes have huge impact on performance, so we fix our version at 4.10.0 in evaluation.
2. Download models:
python -m distar.bin.download_model --name rl_model
Note: Specify rl_model or sl_model after --name to download reinforcement learning model or supervised model.
Model list:
sl_model: training with human replays, skill is equal to diamond players.rl_model: used as default, training with reinforcement learning, skill is equal to master or grandmaster.Abathur: one of reinforcement learning models, likes playing mutalisk.Brakk: one of reinforcement learning models, likes lingbane rush.Dehaka: one of reinforcement learning models, likes playing roach ravager.Zagara: one of reinforcement learning models, likes roach rush.
3. Agent test
With the given model, we provide multiple tests with our agent.
Play against Agent
python -m distar.bin.play
It runs 2 StarCraftII instances. First one is controlled by our RL agent. Human player can play on the second one with full screen like normal game.
Note:
- GPU and CUDA is required on default, add
--cpu if you don’t have these.
- Download RL model first or specify other models (like supervised model) with argument
--model1 <model_name>
- In race cases, 2 StarCraftII instances may lose connection and agent won’t issue any action. Please restart when this happens.
Agent vs Agent
python -m distar.bin.play --game_type agent_vs_agent
It runs 2 StarCraftII instances both controlled by our RL Agent, specify other model path with argument --model1 <model_name> --model2 <model_name>
Agent vs Bot
python -m distar.bin.play --game_type agent_vs_bot
RL agent plays against built-in elite bot.
Building your own agent with our framework
It is necessary to build different agents within one code base and still be able to make them play against each other.
We implement this by making actor and environment as common components and putting everything related to the agent into one directory.
The agent called default under distar/agent is an example of this. Every script under default uses relative import, which
makes them portable to anywhere as a whole part.
If you want to create a new agent with/without our default agent, follow instructions here
If you want to train a new agent with our framework, follow instructions below and here is a guidance with more details of the whole training pipeline.
Supervised Learning
StarCraftII client is required for replay decoding, follow instructions above.
python -m distar.bin.sl_train --data <path>
path could be either a directory with replays or a file includes a replay path at each line.
Optionally, separating replay decoding and model training could be more efficient, run the three scripts in different terminals:
python -m distar.bin.sl_train --type coordinator
python -m distar.bin.sl_train --type learner --remote
python -m distar.bin.sl_train --type replay_actor --data <path>
For distributed training:
python -m distar.bin.sl_train --init_method <init_method> --rank <rank> --world_size <world_size>
or
python -m distar.bin.sl_train --type coordinator
python -m distar.bin.sl_train --type learner --remote --init_method <init_method> --rank <rank> --world_size <world_size>
python -m distar.bin.sl_train --type replay_actor --data <path>
Here is an example of training on a machine with 4 GPUs in remote mode:
# Run the following scripts in different terminals (windows).
python -m distar.bin.sl_train --type coordinator
# Assume 4 GPUs are on the same machine.
# If your GPUs are on different machines, you need to configure the init_mehod's IP for each machine.
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 0 --world_size 4
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 1 --world_size 4
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 2 --world_size 4
python -m distar.bin.sl_train --type learner --remote --init_method tcp://127.0.0.1 --rank 3 --world_size 4
python -m distar.bin.sl_train --type replay_actor --data <path>
Reinforcement Learning
Reinforcement learning will use supervised model as initial model, please download it first, StarCraftII client is also required.
1. Training against bots in StarCraftII:
python -m disatr.bin.rl_train
2. Training with self-play
python -m disatr.bin.rl_train --task selfplay
Four components are used for RL training, just like SL training, they can be executed through different process:
python -m distar.bin.rl_train --type league --task selfplay
python -m distar.bin.rl_train --type coordinator
python -m distar.bin.rl_train --type learner
python -m distar.bin.rl_train --type actor
Distributed training is also supported like SL training.
Chat group
Slack: link
Discord server: link
Citation
@misc{distar,
title={DI-star: An Open-sourse Reinforcement Learning Framework for StarCraftII},
author={DI-star Contributors},
publisher = {GitHub},
howpublished = {\url{https://github.com/opendilab/DI-star}},
year={2021},
}
License
DI-star released under the Apache 2.0 license.
评论