StarCraft: Brood War . How much it means to me. And for many of you. So much so that I wondered whether to give a link to the wiki.
One day, Halt knocked on me in a personal and offered to learn Rust . Like any normal people, we decided to start with hello world writing a dynamic library under Windows, which could be loaded into the address space of the game StarCraft and control units.
The article will describe the process of finding solutions, using technologies, techniques that will allow you to learn new things in the Rust language and its ecosystem or be inspired to implement the bot in your favorite language, be it C, C ++, ruby, python, etc
This article should be read without fail under the anthem of South Korea:
Bwapi
This game is already 20 years old. And it is still popular , the championships collect entire halls of people in the United States, even in 2017 , where the battle of grandmasters Jaedong vs Bisu took place. In addition to live players, soulless cars participate in the battles! And this is possible thanks to BWAPI . More useful links .
For more than ten years around the game there is a community of developers of bots. Enthusiasts write bots and participate in various championships. Many of them study AI and machine learning. BWAPI is used by universities to train their students. There is even a twitch channel that broadcasts games.
So, a few years ago, a team of fans rebuilt Starcraft's inner space and wrote on the C ++ API, which allows you to write bots, enter the game process and dominate the pitiful little people.
As often happens before to build a house, you need to mine, forge tools ... write a bot, you need to implement an API. What can Rust offer for its part?
FFI
Interacting with other languages from Rust is quite simple. For this there is FFI . Let me provide a brief excerpt from the documentation .
Suppose we have a snappy library that has a snappy-ch header file from which we will copy function declarations.
Create a project using cargo .
$ cargo new --bin snappy Created binary (application) `snappy` project $ cd snappy snappy$ tree . ├── Cargo.toml └── src └── main.rs 1 directory, 2 files
Cargo has created a standard file structure for the project.
In Cargo.toml we specify the dependency on libc :
[dependencies] libc = "0.2"
src/main.rs file will look like this:
extern crate libc;
We collect and run:
snappy$ cargo build ... snappy$ cargo run Finished dev [unoptimized + debuginfo] target(s) in 0.02s Running `target/debug/snappy` max compressed length of a 100 byte buffer: 148
You can only call the cargo run , which before the launch causes the cargo build . Or build a project and call the binary directly:
snappy$ ./target/debug/snappy max compressed length of a 100 byte buffer: 148
The code will be compiled provided that the snappy library is installed (for Ubuntu, you need to install the libsnappy-dev package).
snappy$ ldd target/debug/snappy ... libsnappy.so.1 => /usr/lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007f8de07cf000)
As you can see, our binary is linked to the libsnappy shared library. And calling snappy_max_compressed_length is a function call from this library.
rust-bindgen
It would be nice if we could automatically generate FFI. Fortunately, in the arsenal of rastomanov there is such a utility called rust-bindgen . She is able to generate FFI banding to C (and some C ++) libraries.
Installation:
$ cargo install bindgen
What does rust-bindgen look like? We take the C / C ++ header files, set the bindgen utility on them, and at the output we get the generated Rust code with the definitions of sish structures and functions. Here is what the FFI generation for snappy looks like:
$ bindgen /usr/include/snappy-ch | grep -C 1 snappy_max_compressed_length extern "C" { pub fn snappy_max_compressed_length(source_length: usize) -> usize; }
It turned out that bindgen passes in front of the BWAPI headers, generating tons of non-usable code sheets (due to virtual member functions, std :: string in a public API, etc.). The fact is that BWAPI is written in C ++. C ++ is generally difficult to use even from C ++ projects. Once a compiled library is better to link with the same linker (same versions), it’s better to parse the header files with the same compiler (same versions). Because there are many factors that can affect the outcome. For example, mangling , which the GNU GCC still cannot implement without errors . These factors are so significant that they could not be overcome even in gtest , and the documentation indicated that it would be better for you to build gtest as part of the project with the same compiler and the same linker.
BWAPI-C
C is the lingua franca of programming. If rust-bindgen works well for the C language, why not implement BWAPI for C, and then use its API? A good idea!
Yes, a good idea, until you looked into the BWAPI guts and did not see the number of classes and methods that you need to implement = (Especially all of these structures in memory, assemblers, memory patching and other horrors for which we do not have time. It is necessary to maximize the use of an existing solution.
But we must somehow deal with mangling, C ++ code, inheritance, and virtual member functions.
In C ++ there are two most powerful tools that we use to solve our problem, these are opaque pointers and extern "C" .
extern "C" {} allows the C ++ code to be disguised as C. This allows you to generate pure function names without mangling.
Opaque pointers give us the opportunity to erase the type and create a pointer to "some type", whose implementation we do not provide. Since this is only a declaration of some type, and not its implementation, it is impossible to use this type by value, it can be used only by the pointer.
Suppose we have this C ++ code:
namespace cpp { struct Foo { int bar; virtual int get_bar() { return this->bar; } }; }
We can turn this into a C header:
extern "C" { typedef struct Foo_ Foo;
And the C ++ part that will be the link between the C header and the C ++ implementation:
int Foo_get_bar(Foo* self) {
Not all class methods had to be processed in this way. In BWAPI, there are classes that you can implement yourself on using the field values of these structures, for example, typedef struct Position { int x; int y; } Position; typedef struct Position { int x; int y; } Position; and methods like Position::get_distance .
There were those over which we had to try in a special way. For example, an AIModule must be a pointer to a C ++ class with a specific set of virtual member functions. However, here is the heading and implementation .
So, after several months of hard work, 554 methods and a dozen classes, the BWAPI-C cross-platform library was born, which allows you to write bots in C. A side product was the possibility of cross-compiling and the ability to implement the API in any other language that supports FFI and the cdecl calling convention.
If you are writing a library, please write an API on C.
The most important feature of BWAPI-C is the widest possibility of integration with other languages. Python , Ruby , Rust , PHP , Java and many many others know how to work with C, so you can also write a bot on them if you work a little bit with a file and implement your wrappers.
We write bot on C
This part describes the general principles of the device modules Starcraft.
There are 2 types of bots: module and client. We will consider an example of writing a module.
The module is a downloadable library, the general principle of loading can be found here . The module must export 2 functions: newAIModule and gameInit .
With gameInit everything is simple, this function is called to pass a pointer to the current game. This pointer is very important, because in the wilds of BWAPI there is a global static variable that is used in some parts of the code. We describe gameInit :
DLLEXPORT void gameInit(void* game) { BWAPIC_setGame(game); }
newAIModule bit more complicated. It should return a pointer to a C ++ class that has a virtual method table named onXXXXX , which are called on certain game events. We define the structure of the module:
typedef struct ExampleAIModule { const AIModule_vtable* vtable_; const char* name; } ExampleAIModule;
The first field must be a pointer to the method table (magic, everything). So, the newAIModule function:
DLLEXPORT void* newAIModule() { ExampleAIModule* const module = (ExampleAIModule*) malloc( sizeof(ExampleAIModule) ); module->name = "ExampleAIModule"; module->vtable_ = &module_vtable; return createAIModuleWrapper( (AIModule*) module ); }
createAIModuleWrapper is another magic that turns the C pointer into a pointer to a C ++ class with virtual by methods member functions.
module_vtable is a static variable on the method table, the values of the methods are filled with pointers to global functions:
static AIModule_vtable module_vtable = { onStart, onEnd, onFrame, onSendText, onReceiveText, onPlayerLeft, onNukeDetect, onUnitDiscover, onUnitEvade, onUnitShow, onUnitHide, onUnitCreate, onUnitDestroy, onUnitMorph, onUnitRenegade, onSaveGame, onUnitComplete }; void onEnd(AIModule* module, bool isWinner) { } void onFrame(AIModule* module) {} void onSendText(AIModule* module, const char* text) {} void onReceiveText(AIModule* module, Player* player, const char* text) {} void onPlayerLeft(AIModule* module, Player* player) {} void onNukeDetect(AIModule* module, Position target) {} void onUnitDiscover(AIModule* module, Unit* unit) {} void onUnitEvade(AIModule* module, Unit* unit) {} void onUnitShow(AIModule* module, Unit* unit) {} void onUnitHide(AIModule* module, Unit* unit) {} void onUnitCreate(AIModule* module, Unit* unit) {} void onUnitDestroy(AIModule* module, Unit* unit) {} void onUnitMorph(AIModule* module, Unit* unit) {} void onUnitRenegade(AIModule* module, Unit* unit) {} void onSaveGame(AIModule* module, const char* gameName) {} void onUnitComplete(AIModule* module, Unit* unit) {}
By the name of the functions and their signatures, it is clear under what conditions and with what arguments they are called. For example, I made all functions empty, except
void onStart(AIModule* module) { ExampleAIModule* self = (ExampleAIModule*) module; Game* game = BWAPIC_getGame(); Game_sendText(game, "Hello from bwapi-c!"); Game_sendText(game, "My name is %s", self->name); }
This function is called when the game starts. A pointer to the current module is passed as an argument. BWAPIC_getGame returns a global pointer to the game, which we set by calling BWAPIC_setGame . So, we will show an example of cross-compilation and module operation:
bwapi-c/example$ tree . ├── BWAPIC.dll └── Dll.c 0 directories, 2 files bwapi-c/example$ i686-w64-mingw32-gcc -mabi=ms -shared -o Dll.dll Dll.c -I../include -L. -lBWAPIC bwapi-c/example$ cp Dll.dll ~/Starcraft/bwapi-data/ bwapi-c/example$ cd ~/Starcraft/bwapi-data/ Starcraft$ wine bwheadless.exe -e StarCraft.exe -l bwapi-data/BWAPI.dll --headful ... ... ...
We poke buttons and start the game. More information about the launch can be found on the BWAPI website and in BWAPI-C .
The result of the module:
A slightly more complex example of a module that shows how to work with iterators, unit management, mineral search, statistics output can be found in bwapi-c / example / Dll.c.
bwapi-sys
In the Rasta ecosystem, it is customary to call packages that are linked to native libraries. Any foo-sys package deals with two important functions:
- Linked to the libfoo native library
- Provides declarations to functions from the libfoo library. But only ads, high-level abstractions in * -sys crates are not provided.
In order for the * -sys package to be able to link successfully, a native library search and / or library build from source is built into it.
In order for the * -sys package to provide ads, you must either write them with your hands, or generate them using bindgen. Again bindgen. Attempt number two =)
Generating buyindings using bwapi-c becomes obscenely simple:
bindgen BWAPI.h -o lib.rs \ --opaque-type ".+_" \ --blacklist-type "std.*|__.+|.+_$|Game_v(Send|Print|Draw).*|va_list|.+_t$" \ --no-layout-tests \ --no-derive-debug \ --raw-line "#![allow(improper_ctypes, non_snake_case)]" \ -- -I../submodules/bwapi-c/include sed -i -r -- 's/.+\s+(.+)_;/pub struct \1;/' lib.rs
Where BWAPI.h is a file with inclusions of all sishnyh headers from BWAPI-C.
For example, for the already known functions, bindgen generated the following declarations:
extern "C" {
There are 2 strategies: storing the generated code in the repository and generating the code on the fly when building. Both approaches have their advantages and disadvantages .
Greetings to bwapi-sys , one more small step towards our goal.
Remember, I was talking about cross-platform? Nlinker joined the project and implemented a tricky strategy. If the target target is Windows, then download the already compiled BWAPIC from github. And for the rest of the target collect BWAPI-C from the source for OpenBW (talk later).
bwapi-rs
Now that we have bandings, we can describe high-level abstractions. We have 2 types to work with: pure values and opaque pointers.
With pure values, everything is easier. Take for example colors. We need to achieve convenient use from the Rust code so that colors can be used in a convenient and natural way:
game.draw_line(CoordinateType::Screen, (10, 20), (30, 40), Color::Red); ^^^
Therefore, for convenient use, it will be necessary to define the idiomatic listing for the Rust language with constants from C ++ and define the conversion methods to bwapi_sys :: Color using the type std :: convert :: From :
Although for convenience, you can use the enum-primitive-derive crate .
With opaque pointers no more difficult. To do this, use the Newtype pattern:
pub struct Player(*mut sys::Player);
That is, the Player is a kind of structure with a private field — a raw, opaque pointer from C. And this is how to describe the Player :: color method:
impl Player {
Now we can write our first bot on Rust!
We write a bot on Rust
As a proof of concept, a bot will look like one well-known country: its entire functionality will consist in hiring workers and collecting minerals.
Let's start with the required functions gameInit and newAIModule :
#[no_mangle] pub unsafe extern "C" fn gameInit(game: *mut void) { bwapi_sys::BWAPIC_setGame(game as *mut bwapi_sys::Game); } #[no_mangle] pub unsafe extern "C" fn newAIModule() -> *mut void { let module = ExampleAIModule { name: String::from("ExampleAIModule") }; let result = wrap_handler(Box::new(module)); result }
#[no_mangle] performs the same function as extern "C" in C ++. Inside the wrap_handler , all magic happens with the substitution of the virtual function table and disguise as a C ++ class.
The description of the module structure is even simpler and more beautiful than in C:
struct ExampleAIModule { name: String, }
Add a couple of methods for drawing statistics and distributing orders:
impl ExampleAIModule { fn draw_stat(&mut self) { let game = Game::get(); let message = format!("Frame {}", game.frame_count()); game.draw_text(CoordinateType::Screen, (10, 10), &message); } fn give_orders(&mut self) { let player = Game::get().self_player(); for unit in player.units() { match unit.get_type() { UnitType::Terran_SCV | UnitType::Zerg_Drone | UnitType::Protoss_Probe => { if !unit.is_idle() { continue; } if unit.is_carrying_gas() || unit.is_carrying_minerals() { unit.return_cargo(false); continue; } if let Some(mineral) = Game::get() .minerals() .min_by_key(|m| unit.distance_to(m)) {
In order for the ExampleAIModule type to become a real module, you need to teach it to respond to the onXXXX events, for which you need to implement the EventHandler type, which is analogous to the virtual table AIModule_vtable from C:
impl EventHandler for ExampleAIModule { fn on_start(&mut self) { Game::get().send_text(&format!("Hello from Rust! My name is {}", self.name)); } fn on_end(&mut self, _is_winner: bool) {} fn on_frame(&mut self) { self.draw_stat(); self.give_orders(); } fn on_send_text(&mut self, _text: &str) {} fn on_receive_text(&mut self, _player: &mut Player, _text: &str) {} fn on_player_left(&mut self, _player: &mut Player) {} fn on_nuke_detect(&mut self, _target: Position) {} fn on_unit_discover(&mut self, _unit: &mut Unit) {} fn on_unit_evade(&mut self, _unit: &mut Unit) {} fn on_unit_show(&mut self, _unit: &mut Unit) {} fn on_unit_hide(&mut self, _unit: &mut Unit) {} fn on_unit_create(&mut self, _unit: &mut Unit) {} fn on_unit_destroy(&mut self, _unit: &mut Unit) {} fn on_unit_morph(&mut self, _unit: &mut Unit) {} fn on_unit_renegade(&mut self, _unit: &mut Unit) {} fn on_save_game(&mut self, _game_name: &str) {} fn on_unit_complete(&mut self, _unit: &mut Unit) {} }
Building and running a module is as simple as for C:
bwapi-rs$ cargo build --example dll --target=i686-pc-windows-gnu bwapi-rs$ cp ./target/i686-pc-windows-gnu/debug/examples/dll.dll ~/Starcraft/bwapi-data/Dll.dll bwapi-rs$ cd ~/Starcraft/bwapi-data/ Starcraft$ wine bwheadless.exe -e StarCraft.exe -l bwapi-data/BWAPI.dll --headful ... ... ...
And video work:
A bit about cross compilation
In short, in Rust it is beautiful! In two clicks you can put a lot of toolchains for different platforms. Specifically, i686-pc-windows-gnu toolchain is set by the command:
rustup target add i686-pc-windows-gnu
You can also specify a cofig for cargo in the root of the .cargo/config project:
[target.i686-pc-windows-gnu] linker = "i686-w64-mingw32-gcc" ar = "i686-w64-mingw32-ar" runner = "wine"
And this is all you need to do to compile a Rust project from Linux under Windows.
Openbw
These guys went even further. They decided to write an open-source version of the game SC: BW! And they are doing well. One of their goals was to implement HD images, but SC: Remastered them were ahead = (At the moment, you can use their API to write bots (yes, also in C ++). But the most mind-blowing feature is the ability to view replays directly in the browser .
Conclusion
When implementing, an unsolved problem remains: we do not control the uniqueness of the links, and the simultaneous existence of &mut and & when changing the object will lead to undefined behavior. Trouble Halt tried to implement idiomatic bandings, but his fuse was slightly extinguished. Also, to solve this problem, you will have to qualitatively rewind the C ++ API and correctly type const qualifiers.
I really enjoyed working on this project, I watched replays and deeply immersed in the atmosphere. This game left a legacy of 않을 지지 않을 인. No game can not be 비교할 수 없다 in popularity with SC: BW, and its impact on 대한민국 정치 에게 was unthinkable. Korean progamers 아마도 are just as popular as Korean dorams, broadcast on prime time. 또한, 한국 에서 프로 게이머 라면 군대 의 특별한 육군 에 입대 입대 할 수 있다.
Long live StarCraft!
Links