|
| 1 | +# Game Server |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +```mermaid |
| 6 | +flowchart TD |
| 7 | +subgraph Tokio_Tasks[Tokio Tasks] |
| 8 | +IT[I/O Tokio ingress task] |
| 9 | +ET[I/O Tokio egress task] |
| 10 | +HM[HashMap<PlayerId, BytesMut>] |
| 11 | +IT --> HM |
| 12 | +end |
| 13 | +
|
| 14 | +subgraph Processing_Systems[Processing Loop - 50ms] |
| 15 | +IS[Ingress System] |
| 16 | +RS[Run Events System] |
| 17 | +ES[Egress System] |
| 18 | +
|
| 19 | +IS -->|" Decode & create borrowed events"|RS |
| 20 | +RS -->|State changes| ES |
| 21 | +ES -->|" local BytesMut "| ET |
| 22 | +ES -->|" Next tick (50ms) "|IS |
| 23 | +end |
| 24 | +
|
| 25 | +HM -->|Pull raw data|IS |
| 26 | +``` |
| 27 | + |
| 28 | +## Egress |
| 29 | + |
| 30 | +### Buffers |
| 31 | + |
| 32 | +The systems are running in multiple threads with entities partitioned across them. |
| 33 | +Suppose we have 8 threads[^1]. |
| 34 | + |
| 35 | +This means that the egress system will send thread-local 8 [ |
| 36 | +`bytes::BytesMut`](https://docs.rs/bytes/latest/bytes/struct.BytesMut.html)[^2] |
| 37 | +to the tokio egress task at the end of every tick. |
| 38 | + |
| 39 | +The contents in the buffers are [rkyv-encoded](https://github.com/rkyv/rkyv) packets |
| 40 | +specified [here](https://github.com/andrewgazelka/hyperion/blob/main/crates/hyperion-proto/src/server_to_proxy.rs): |
| 41 | + |
| 42 | +```rust |
| 43 | +#[derive(Archive, Deserialize, Serialize, Clone, PartialEq)] |
| 44 | +pub struct BroadcastLocal<'a> { |
| 45 | + pub center: ChunkPosition, |
| 46 | + pub exclude: u64, |
| 47 | + pub order: u32, |
| 48 | + |
| 49 | + #[rkyv(with = InlineAsBox)] |
| 50 | + pub data: &'a [u8], |
| 51 | +} |
| 52 | + |
| 53 | +#[derive(Archive, Deserialize, Serialize, Clone, PartialEq)] |
| 54 | +pub struct Unicast<'a> { |
| 55 | + pub stream: u64, |
| 56 | + pub order: u32, |
| 57 | + |
| 58 | + #[rkyv(with = InlineAsBox)] |
| 59 | + pub data: &'a [u8], |
| 60 | +} |
| 61 | +``` |
| 62 | + |
| 63 | +### Ordering |
| 64 | + |
| 65 | +Many of the rkyv-encoded packets have a specific `order` field. The order is calculated as |
| 66 | + |
| 67 | +```rust |
| 68 | +system_id << 16 | order_id |
| 69 | +``` |
| 70 | + |
| 71 | +where `system_id` is the strictly increasing ID of the system that is sending the packet and `order_id` is a |
| 72 | +thread-local |
| 73 | +counter that is incremented on each packet write. |
| 74 | + |
| 75 | +This allows the proxy to reorder the 8 thread-local buffers into one buffer that has the same logical ordering as the |
| 76 | +order of the systems and the order of the packets within each system. |
| 77 | + |
| 78 | + |
| 79 | +## Ingress |
| 80 | + |
| 81 | +### Tokio Async Task |
| 82 | +The tokio async ingress task creates a data structure |
| 83 | +defined [here](https://github.com/andrewgazelka/hyperion/blob/0c1a0386548d71485c442cf5e9c9ebb2ed58142e/crates/hyperion/src/net/proxy.rs#L16-L23). |
| 84 | + |
| 85 | +```rust |
| 86 | +#[derive(Default)] |
| 87 | +pub struct ReceiveStateInner { |
| 88 | + /// All players who have recently connected to the server. |
| 89 | + pub player_connect: Vec<u64>, |
| 90 | + /// All players who have recently disconnected from the server. |
| 91 | + pub player_disconnect: Vec<u64>, |
| 92 | + /// A map of stream ids to the corresponding [`BytesMut`] buffers. This represents data from the client to the server. |
| 93 | + pub packets: HashMap<u64, BytesMut>, |
| 94 | +} |
| 95 | +``` |
| 96 | + |
| 97 | +### Decoding System |
| 98 | + |
| 99 | +Then, when it is time to run the ingress system, we lock the mutex for `ReceiveStateInner` and process the data, |
| 100 | +decoding all the packets until we get |
| 101 | + |
| 102 | +```rust |
| 103 | +#[derive(Copy, Clone)] |
| 104 | +pub struct BorrowedPacketFrame<'a> { |
| 105 | + pub id: i32, |
| 106 | + pub body: &'a [u8], |
| 107 | +} |
| 108 | +``` |
| 109 | + |
| 110 | +where the `'a` lifetime is the duration of the entire tick (the `BytesMut` are deallocated at the end of the tick). |
| 111 | + |
| 112 | +### Event Generation |
| 113 | + |
| 114 | +For each entity's packet, we have a switch statement over what we should do for each |
| 115 | +packet [here](https://github.com/andrewgazelka/hyperion/blob/bb30c0680ef3822aa9a30d84e289c8db39e38150/crates/hyperion/src/simulation/handlers.rs#L609-L641). |
| 116 | + |
| 117 | +Note: all packet-switch logic is done in parallel (based on the number of threads) and which entity is partitioned |
| 118 | +on that thread. |
| 119 | + |
| 120 | +```rust |
| 121 | +pub fn packet_switch( |
| 122 | + raw: BorrowedPacketFrame<'_>, |
| 123 | + query: &mut PacketSwitchQuery<'_>, |
| 124 | +) -> anyhow::Result<()> { |
| 125 | + let packet_id = raw.id; |
| 126 | + let data = raw.body; |
| 127 | + |
| 128 | + // ideally we wouldn't have to do this. The lifetime is the same as the entire tick. |
| 129 | + // as the data is bump-allocated and reset occurs at the end of the tick |
| 130 | + let data: &'static [u8] = unsafe { core::mem::transmute(data) }; |
| 131 | + |
| 132 | + match packet_id { |
| 133 | + play::ChatMessageC2s::ID => chat_message(data, query)?, |
| 134 | + play::ClickSlotC2s::ID => click_slot(data, query)?, |
| 135 | + play::ClientCommandC2s::ID => client_command(data, query)?, |
| 136 | + play::CommandExecutionC2s::ID => chat_command(data, query)?, |
| 137 | + play::CreativeInventoryActionC2s::ID => creative_inventory_action(data, query)?, |
| 138 | + play::CustomPayloadC2s::ID => custom_payload(data, query)?, |
| 139 | + play::FullC2s::ID => full(query, data)?, |
| 140 | + play::HandSwingC2s::ID => hand_swing(data, query)?, |
| 141 | + play::LookAndOnGroundC2s::ID => look_and_on_ground(data, query)?, |
| 142 | + play::PlayerActionC2s::ID => player_action(data, query)?, |
| 143 | + play::PlayerInteractBlockC2s::ID => player_interact_block(data, query)?, |
| 144 | + play::PlayerInteractEntityC2s::ID => player_interact_entity(data, query)?, |
| 145 | + play::PlayerInteractItemC2s::ID => player_interact_item(data, query)?, |
| 146 | + play::PositionAndOnGroundC2s::ID => position_and_on_ground(query, data)?, |
| 147 | + play::RequestCommandCompletionsC2s::ID => request_command_completions(data, query)?, |
| 148 | + play::UpdateSelectedSlotC2s::ID => update_selected_slot(data, query)?, |
| 149 | + _ => trace!("unknown packet id: 0x{:02X}", packet_id), |
| 150 | + } |
| 151 | + |
| 152 | + Ok(()) |
| 153 | +} |
| 154 | +``` |
| 155 | + |
| 156 | +## Ingress System |
| 157 | + |
| 158 | + |
| 159 | +[^1]: the actual number is assigned at compile-time for maximum performance and is usually equal to the number of cores |
| 160 | +on the machine. |
| 161 | + |
| 162 | +[^2]: `bytes::BytesMut` re-uses the underlying buffer and tracks allocations and deallocations so allocations each tick |
| 163 | +are not needed. |
0 commit comments