Skip to content

Commit bdac910

Browse files
authored
docs: describe new architecture and event (#915)
This reflects the changes made in #882 and #914. I have also adjusted the claim that Hyperion supports 170k+ players to 10k+ players. We do not currently have any benchmarks supporting the 170k+ player claim before or after the Bevy PR. I believe this number was extrapolated from the 5k player ms/tick, where (1.42 ms/tick)/(5k players) = (50 ms/tick)/(176k players [extrapolated]), but we do not have a real benchmark running 170k+ players.
1 parent 3a7226f commit bdac910

File tree

8 files changed

+46
-141
lines changed

8 files changed

+46
-141
lines changed

CONTRIBUTING.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,6 @@ Essential tools to enhance your development workflow:
1818
- Debug and analyze Minecraft protocol packets
1919
- Essential for protocol-related development
2020

21-
### Debugging
22-
- **[flecs.dev/explorer](https://flecs.dev/explorer)**
23-
- View entities in the Entity Component System (ECS)
24-
2521
### Profiling
2622
- **[Tracy Profiler](https://github.com/wolfpld/tracy)**
2723
- Advanced performance profiling

README.md

Lines changed: 5 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@
55
[![Issues](https://img.shields.io/github/issues/andrewgazelka/hyperion)](https://github.com/andrewgazelka/hyperion/issues)
66
[![Last Commit](https://img.shields.io/github/last-commit/andrewgazelka/hyperion)](https://github.com/andrewgazelka/hyperion/commits)
77

8-
Hyperion is a **Minecraft game engine** that can have 170,000+ players in one world. Our pilot event hopes to break the PvP Guinness World
8+
Hyperion is a **Minecraft game engine** that can have 10,000+ players in one world. Our pilot event hopes to break the PvP Guinness World
99
Record of ([8825 by
1010
EVE Online](https://www.guinnessworldrecords.com/world-records/105603-largest-videogame-pvp-battle)). The
11-
architecture is ECS-driven using [Flecs Rust](https://github.com/Indra-db/Flecs-Rust).
11+
architecture is ECS-driven using [Bevy](https://bevy.org/).
1212

1313
> [!NOTE]
1414
> You can join the test server in 1.20.1 at `hyperion-test.duckdns.org`
@@ -63,13 +63,6 @@ https://github.com/user-attachments/assets/64a4a8c7-f375-4821-a1c7-0efc69c1ae0b
6363
- Commit hash `faac9117` run with `just release`
6464
- Bot Launch Command: `just bots {number}`
6565

66-
**Note on Performance:**
67-
The system's computational costs are primarily fixed due to thread synchronization overhead. Each game tick contains
68-
several $O(1)$ synchronization points, meaning these operations maintain constant time complexity regardless of player
69-
count. This architecture explains why performance remains relatively stable even as player count increases
70-
significantly - the thread synchronization overhead dominates the performance profile rather than player-specific
71-
computations.
72-
7366
The bulk of player-specific processing occurs in our proxy layer, which handles tasks like regional multicasting and can
7467
be horizontally scaled to maintain performance as player count grows.
7568

@@ -83,7 +76,7 @@ be horizontally scaled to maintain performance as player count grows.
8376
flowchart TB
8477
subgraph GameServer["Game Server (↕️ Scaled)"]
8578
direction TB
86-
subgraph FlecsMT["Flecs Multi-threaded ECS"]
79+
subgraph BevyMT["Bevy Multi-threaded ECS"]
8780
direction LR
8881
IngressSys["Ingress System"] --> |"1 Game Tick (50ms)"| CoreSys["Core Systems (Game Engine)"] --> GameSys["Game Systems (Event Logic)"] --> EgressSys["Egress System"]
8982
end
@@ -138,7 +131,7 @@ flowchart TB
138131
classDef async fill:#e7e7e7,stroke:#333,stroke-width:2px
139132
140133
class GameServer server
141-
class FlecsMT ecs
134+
class BevyMT ecs
142135
class IngressSys,CoreSys,GameSys,EgressSys system
143136
class Proxy1,Proxy2,ProxyN proxy
144137
class Velocity1,Velocity2,VelocityN auth
@@ -203,7 +196,7 @@ docker compose up --build
203196

204197
**Language:** Rust
205198
**Goal:** Game engine for massive events
206-
**Structure:** flecs ECS
199+
**Structure:** Bevy ECS
207200

208201
**Platform Details:**
209202
- Version: Minecraft 1.20.1

docs/architecture/game-server.md

Lines changed: 26 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,16 @@ flowchart TD
77
subgraph Tokio_Tasks[Tokio Tasks]
88
IT[I/O Tokio ingress task]
99
ET[I/O Tokio egress task]
10-
HM[HashMap<PlayerId, BytesMut>]
10+
HM[HashMap<PlayerId, packet_channel::Sender>]
1111
IT --> HM
1212
end
1313
1414
subgraph Processing_Systems[Processing Loop - 50ms]
1515
IS[Ingress System]
16-
RS[Run Events System]
16+
RS[Run packet handling systems]
1717
ES[Egress System]
1818
19-
IS -->|" Decode & create borrowed events"|RS
19+
IS -->|" Decode & create Bevy events for each packet"|RS
2020
RS -->|State changes| ES
2121
ES -->|" local BytesMut "| ET
2222
ES -->|" Next tick (50ms) "|IS
@@ -62,83 +62,37 @@ pub struct Unicast<'a> {
6262

6363
## Ingress
6464

65-
### Tokio Async Task
66-
The tokio async ingress task creates a data structure
67-
defined [here](https://github.com/andrewgazelka/hyperion/blob/0c1a0386548d71485c442cf5e9c9ebb2ed58142e/crates/hyperion/src/net/proxy.rs#L16-L23).
65+
### Packet Channel
6866

69-
```rust
70-
#[derive(Default)]
71-
pub struct ReceiveStateInner {
72-
/// All players who have recently connected to the server.
73-
pub player_connect: Vec<u64>,
74-
/// All players who have recently disconnected from the server.
75-
pub player_disconnect: Vec<u64>,
76-
/// A map of stream ids to the corresponding [`BytesMut`] buffers. This represents data from the client to the server.
77-
pub packets: HashMap<u64, BytesMut>,
78-
}
79-
```
80-
81-
### Decoding System
82-
83-
Then, when it is time to run the ingress system, we lock the mutex for `ReceiveStateInner` and process the data,
84-
decoding all the packets until we get
85-
86-
```rust
87-
#[derive(Copy, Clone)]
88-
pub struct BorrowedPacketFrame<'a> {
89-
pub id: i32,
90-
pub body: &'a [u8],
91-
}
92-
```
67+
The packet channel is a linked list of `Fragment`. Each fragment contains:
68+
- an incremental fragment id
69+
- a `Box<[u8]>` with zero or more contiguous packets with a `u32` length prefix before each packet
70+
- a read cursor, where `0..read_cursor` is ready to read and contains whole packets
71+
- an `ArcSwapOption<Fragment>` pointing to the next fragment if there is one
9372

94-
where the `'a` lifetime is the duration of the entire tick (the `BytesMut` are deallocated at the end of the tick).
73+
As packets from the client are processed in the proxy thread, the server decodes the `VarInt` length prefix to
74+
determine the packet size. If there is enough space remaining in the current fragment, the packet bytes are copied to
75+
the current fragment. Otherwise, a new fragment will be allocated and appended to the linked list.
9576

96-
### Event Generation
77+
The decoding system, running in separate threads, iterates through the linked list and reads packets up to the read
78+
cursor. These packets are decoded, sent through Bevy events, and then processed by other systems which read those
79+
events.
9780

98-
For each entity's packet, we have a switch statement over what we should do for each
99-
packet [here](https://github.com/andrewgazelka/hyperion/blob/bb30c0680ef3822aa9a30d84e289c8db39e38150/crates/hyperion/src/simulation/handlers.rs#L609-L641).
81+
When a packet contains a reference to the original packet bytes, such as a chat message packet storing a reference to
82+
the message, we use `bytes::Bytes` and the variants made in `valence_bytes` (e.g. `valence_bytes::Utf8Bytes` which
83+
wraps a `bytes::Bytes` while guaranteeing that it is valid UTF-8) to allow the packet event to live for `'static` while
84+
only performing one copy[^3] throughout the entire ingress system.
10085

101-
Note: all packet-switch logic is done in parallel (based on the number of threads) and which entity is partitioned
102-
on that thread.
103-
104-
```rust
105-
pub fn packet_switch(
106-
raw: BorrowedPacketFrame<'_>,
107-
query: &mut PacketSwitchQuery<'_>,
108-
) -> anyhow::Result<()> {
109-
let packet_id = raw.id;
110-
let data = raw.body;
111-
112-
// ideally we wouldn't have to do this. The lifetime is the same as the entire tick.
113-
// as the data is bump-allocated and reset occurs at the end of the tick
114-
let data: &'static [u8] = unsafe { core::mem::transmute(data) };
115-
116-
match packet_id {
117-
play::ChatMessageC2s::ID => chat_message(data, query)?,
118-
play::ClickSlotC2s::ID => click_slot(data, query)?,
119-
play::ClientCommandC2s::ID => client_command(data, query)?,
120-
play::CommandExecutionC2s::ID => chat_command(data, query)?,
121-
play::CreativeInventoryActionC2s::ID => creative_inventory_action(data, query)?,
122-
play::CustomPayloadC2s::ID => custom_payload(data, query)?,
123-
play::FullC2s::ID => full(query, data)?,
124-
play::HandSwingC2s::ID => hand_swing(data, query)?,
125-
play::LookAndOnGroundC2s::ID => look_and_on_ground(data, query)?,
126-
play::PlayerActionC2s::ID => player_action(data, query)?,
127-
play::PlayerInteractBlockC2s::ID => player_interact_block(data, query)?,
128-
play::PlayerInteractEntityC2s::ID => player_interact_entity(data, query)?,
129-
play::PlayerInteractItemC2s::ID => player_interact_item(data, query)?,
130-
play::PositionAndOnGroundC2s::ID => position_and_on_ground(query, data)?,
131-
play::RequestCommandCompletionsC2s::ID => request_command_completions(data, query)?,
132-
play::UpdateSelectedSlotC2s::ID => update_selected_slot(data, query)?,
133-
_ => trace!("unknown packet id: 0x{:02X}", packet_id),
134-
}
135-
136-
Ok(())
137-
}
138-
```
86+
This custom channel is more performant than a traditional channel of `Box<[u8]>` because it allows reusing the same
87+
allocation for several packets, and most packets are very small.
13988

14089
[^1]: the actual number is assigned at compile-time for maximum performance and is usually equal to the number of cores
14190
on the machine.
14291

14392
[^2]: `bytes::BytesMut` re-uses the underlying buffer and tracks allocations and deallocations so allocations each tick
14493
are not needed.
94+
95+
[^3]: The copy is done when copying from the proxy's `PlayerPackets` packet to the packet channel. Theoretically this
96+
could be made zero-copy by reading from the TCP stream directly into the packet channel, but this would be less
97+
performant because it would increase the number of read syscalls by requiring one read syscall per player, and
98+
syscalls are more expensive than small memory copies.

docs/architecture/introduction.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,14 @@ approach differs from traditional Minecraft servers that modify vanilla implemen
1919

2020
### Entity Component System (ECS)
2121

22-
Hyperion utilizes [Flecs](https://github.com/SanderMertens/flecs), an Entity Component System, as its core architecture:
22+
Hyperion utilizes [Bevy ECS](https://bevy.org/), an Entity Component System, as its core architecture:
2323

2424
- Entities are organized in a table-like structure
2525
- Rows represent individual entities
2626
- Columns represent components (e.g., health, position)
2727
- Systems process entities through efficient iterations
2828
- Components can be dynamically added (e.g., LastAttacked component for combat)
29-
- For a more accurate representation of an ECS, see the [ECS FAQ](https://github.com/SanderMertens/ecs-faq)
29+
- For a more accurate representation of an ECS, see the [Unofficial Bevy Cheat Book](https://bevy-cheatbook.github.io/programming/ecs-intro.html)
3030

3131
### Performance Optimization
3232

@@ -90,7 +90,7 @@ vanilla mechanics can be completely customized to create unique game modes and e
9090

9191
Developers interested in using Hyperion should familiarize themselves with:
9292

93-
- Entity Component Systems (particularly Flecs)
93+
- Entity Component Systems (particularly Bevy)
9494
- Minecraft networking protocols
9595
- Parallel processing concepts
96-
- Plugin development principles
96+
- Plugin development principles

docs/bedwars/introduction.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# 10,000 Player PvP Event
2+
3+
Our pilot event is a 10,000 player PvP battle to break the Guinness World Record for the largest PvP battle ever.
4+
5+
We're partnering with [TheMisterEpic](https://www.youtube.com/channel/UCJiFgnnYpwlnadzTzhMnX_Q) to run an initial
6+
proof-of-concept event with around 2k players. Following its success, we'll host the full-scale 10,000 player PvP battle
7+
alongside numerous YouTubers and streamers.
8+
9+
Our event will be a massive Bed Wars game consisting of teams with hundreds of players each.

docs/index.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,21 +5,19 @@ layout: home
55
hero:
66
name: "Hyperion"
77
text: "The most advanced Minecraft game engine built in Rust"
8-
tagline: 170,000 players in one world at 20 TPS
8+
tagline: 10,000 players in one world at 20 TPS
99
actions:
1010
- theme: brand
1111
text: Architecture
1212
link: /architecture/introduction
1313
- theme: alt
1414
text: 10,000 Player PvP
15-
link: /tag/introduction
15+
link: /bedwars/introduction
1616

1717
features:
1818
- title: Run massive events with confidence
1919
details: Built in Rust, you can be highly confident your event will not crash from memory leaks or SEGFAULTS.
2020
- title: Vertical and horizontal scalability
2121
details: In our testing I/O is the main bottleneck in massive events. As such, we made it so I/O logic can be offloaded horizontally. The actual core game server is scaled vertically.
22-
- title: Easy debugging profiling
23-
details: All tasks are easily viewable in a tracing UI. All entities are viewable and modifiable from Flecs Explorer.
2422
---
2523

docs/tag/classes.png

-411 KB
Binary file not shown.

docs/tag/introduction.md

Lines changed: 0 additions & 45 deletions
This file was deleted.

0 commit comments

Comments
 (0)