4AM Games Blog

Unity FPSSample - Takeaways from the “Deep dive into networking” talk

2019-01-27T09:23:45-05:00

Intro
Overview
New stuff
Classic
Open questions

Intro

I recently watched Peter Andreasen’s talk about the Unity FPSSample (Deep dive into networking for Unity’s FPS Sample game).

FPSSample is a reference implementation of a classic Quake-like multiplayer FPS game. The source code is available at https://github.com/Unity-Technologies/FPSSample.

It was super valuable to learn about networking techniques I had never even heard about. Also I appreciate the many pieces of wisdom sprinkled throughout the session!

This post has a few purposes:

To highlight the techniques I haven’t seen discussed anywhere else
To summarize the talk
To note some of my own remaining questions - for later investigation

Overview

FPSSample uses the classic Quake model - where clients send only their inputs and the server sends regular snapshots of the game state.

The server updates the game state and acts on all players’ inputs simultaneously at a fixed rate - this approach was chosen for its simplicity relative to the one where the server manages a separate game state for each player.

New stuff

Here’s stuff I learned and that I haven’t seen discussed elsewhere.

Asymmetric tick rates

Asymmetric update rates between client and server isn’t news but Peter discusses this topic in depth.

So yeah, a client can have a tick rate that is different from the server’s, and a render rate that is different from its tick rate.

A few paragraphs below expand on that.

Delta encoding using frame prediction

Glenn Fiedler has a few articles that explain delta encoding for snapshot compression.

The idea is to express all values that we need to replicate as a difference from the ones in the last snapshot that the client acknowledged. It reduces the magnitude of values involved so that smaller data types can be used (and eliminates values that didn’t change).

FPSSample develops this concept one step further. Instead of using the last acknowledged snapshot as the baseline for encoding, it extrapolates a snapshot using the past three snapshots and uses that as the baseline. Thanks to shared code on the server and client, the client is also able to construct this same baseline on its end.

This is a very powerful optimization. Consider just position and velocity replication for example: assuming velocity is unchanged, changes in position don’t need to be sent anymore!

Data stream structure

In data serialization the structure of the data is omitted. Only the data itself gets sent.

So for example a naive serialized structure for a position update sent by the server might look like:

Player {
    ID: 1
    position: (x, y, z)
}
Player {
    ID: 2,
    position: (x, y, z)
}

But the client already knows about the existence of Players 1 and 2 from a previously received snapshot where they were created. So a serialized structure like the following is sufficient:

(x, y, z)
(x, y, z)

The client was able to re-construct the structure of the data using its knowledge of the game state. Look at all the bandwidth that was saved!

RTT measurement

I’m not 100% confident but here’s my understanding of the sample’s RTT measurement:

Host sends packet, saves local time
Peer receives packet, saves local time
Peer processes packet, sends a response packet which includes the elapsed time between reception and response (processing time)
Host receives response, uses saved time to approximate RTT + substracts peer’s reported processing time

The important detail is that the peer transmits the processing time to the host.

58/62 FPS trick

Players often run games at multiples of 30 FPS (30, 60, 120 are common).

If we take for example a client and a server both running at 60 FPS, we will observe flip-flopping behavior where some server tick will receive no player input, and the next tick will receive two (if the client and server are ticking in sync).

Server sleep time

The team implemented a tick rate mechanism that gives ample time for the host machine to sleep (to save on compute costs).

It uses a low-resolution timer and nudges the next frame’s timing based on this frame’s performance.

Though I wonder how the irregularity in tick rate affects the smoothness on the clients. Theoretically it should be practically insignificant thanks to the interpolation buffers.

Clock drift

As of the time of the talk:

The team found that Unity’s framerate limiter accumulated drift over long periods of time.

This may have been addressed by the framerate nudging system described in the previous paragraph, or fixed some other way.

Fractional ticks on client

I don’t understand this technique yet.

The overall idea is that when the client has a tick rate which is different from the server’s tick rate, it still tries to match it to the server’s rate for smoothness by “catching up the small leftover”.

Classic

Here’s topics that have been well described on the net by popular docs from Glenn Fiedler, Gabriel Gambetta, and Valve to name just a few.

Reliable UDP implementation

Peter compares UDP and TCP, describes the dangers of UDP, and describes the sample’s reliable UDP implementation (for example: sequence IDs and ack bitfields).

Past input redundancy

Old but gold! The sample sends the past 3 input commands alongside the current one to minimize the impact of a dropped packet.

Lag compensation for hitscan

When the server checks for hit registration it rewinds the world around the player by [RTT + player’s interpolation time]. The client regularly updates the server about its interpolation time, that’s how the server knows.

Client-side interpolation

Used for framerate-independant smooth rendering and to de-jitter incoming snapshots.

Delta encoding compression

There was a summary of the delta encoding technique (against the last snapshot that was acknowledged by the client). Delta encoding through frame prediction was used instead in the end.

Open questions

Here’s stuff that wasn’t mentioned or discussed at length but that I’m curious about.

Server reconciliation

Peter mentions that if the server doesn’t receive a player’s input command(s) for a given tick, then that player is assumed to have done nothing during that period. This is only one of the many possibilities for de-sync to happen between client and server.

It’s also mentioned that a full rollback is performed on the client at the beginning of every frame.

I’m curious if/how desyncs are smoothed out.

Client-side interpolation time

I’m curious how the client determines what its interpolation time should be, and how exactly it’s shared with the server.

Multiple client ticks for a single server tick

So given that client and server can have different tick rates, the client might send multiple inputs for a single server tick.

Are the client inputs “merged” together and executed in one shot? Or does the server step through them sequentially? Or something else entirely?

Fractional ticks

How does this work?

Sonic Battle (GBA) Renderer Series - The Escher Sandwich

2019-01-24T11:55:28-05:00

The Sandwich Theory
The Escher Sandwich
Draw priority overview
The problem
Circuit behavior - in emulation
Conclusion

Welcome to the fourteenth article of the Sonic Battle (GBA) Renderer series.

The Sandwich Theory

I figured that using painter’s algorithm the draw priority would be:

First floor tilemap
First floor drop shadows
First floor walls and objects, depth sorted
Second floor tilemap
Second floor drop shadows
Second floor objects, depth sorted

Also illustrated in the frame breakdown in article #5.

The core idea is that the two tilemaps sandwich everything else that belongs on the first floor.

The Escher Sandwich

I confirmed the theory with mGBA’s debug tools.

(Quick aside - Drop shadows are treated no different from usual objects, I was wrong about that one. They have no “reserved” draw order.)

And so I went on my merry way until I noticed this situation:

At play here are two walls (outlined), the top tilemap, and the drop shadow (outlined):

All walls should be sandwiched between the tilemaps. But here the left one is drawn on top of the drop shadow which is drawn on top of the tilemap. How?

I checked the draw priorities again and they’re not the issue.

Time to hit the books…

Draw priority overview

Draw priority is done in hardware as depicted below (OBJ Processing, BG Processing, and Draw Priority Evaluation circuits). Behavior tied to hardware usually has some distinctive limitations… so we can already expect some gotchas.

From the AGB Programming Manual, Version 1.1

The manual describes its usage as follows:

(OBJ is equivalent to “hardware sprite”. OBJ number is the location of the sprite in memory, from 0 to 127. BG is equivalent to “tilemap”.)

From the AGB Programming Manual, Version 1.1

The problem

So in summary two things affect a sprite’s draw priority:

Draw priority (the lower priority is drawn first)
In-memory location for tie-breaking (the lower in-memory location is drawn first)
Sprites should be grouped in-memory by draw priority (if there’s a separating tilemap) (and the groups themselves ordered by draw priority)

Number 3 is the stern cautionary paragraph near the bottom.

So let’s observe the problematic case again:

Here’s the in-memory sprite layout that I can see with mGBA’s inspection tools:

BG 2 & 3 are just markers for reference, they're not contained in the sprite layout

This layout doesn’t respect cautionary paragraph #3, and that trips up the hardware and causes the Escher sandwich. It doesn’t seem to be intentional as the draw priorities are still set correctly. Even the in-memory locations are correctly set for tie-breaking the character and the drop shadow, it’s only the grouping which is wrong.

Here’s what a safe layout would like for the same scene:

Circuit behavior - in emulation

Ok fair enough. But why?

Disclaimer: I won’t touch on the actual circuits, I’ll only rely on mGBA’s source code. I made sure to verify that the Escher sandwich occurs identically in the mGBA, VisualBoyAdvance-M, and no$gba emulators, as well as in Nintendo’s very own GBA emulator on the 2DS. So I’m fairly certain that whatever logic mGBA is emulating behaves accurately with respect to hardware.

Render loop

Here’s an abstract summary of the render loop (only the parts which are relevant to the situation):

For each of the screen’s horizontal lines (called scanlines, there’s 160 of them):
- For every OBJ in memory (from location 0 to 127):
  - Pre-process current OBJ (only if its bounds intersect this scanline)
- For every priority level (from 0 to 3):
  - If any OBJs were pre-processed for this priority level
    - Post-process OBJs for this priority level
  - Render tilemaps for this priority level

OBJ pre-processing step

mGBA keeps a buffer of pixels for the current scanline. This buffer is reserved for OBJs only. A pixel in this buffer contains the RGB color as well as some rendering flags (they include the OBJ’s priority level).

This step renders the OBJ to that buffer. For every pixel it checks the priorities against the existing pixel (if any) before writing, using it like a depth buffer or Z buffer on top of a color buffer. Once this step executes for all OBJs mGBA has essentially finished rendering OBJs and prioritizing them amongst themselves (for the current scanline).

HOWEVER transparent pixels are handled in a very counter-intuitive manner. They don’t modify the current pixel’s color at all but still “promote” the current pixel (if it has been written to at least once before) to the transparent pixel’s priority level. Here it is in the source code: in a function that renders a single pixel to the scanline buffer.

If this transparent pixel priority level promotion didn’t exist then it would have been just a classic depth buffer without any OBJ memory location shenanigans.

It must be either a hardware particularity or a conscious decision on Nintendo’s part to support alpha masking for example.

Don’t worry if this is a bit unclear so far, I added visualizations a bit further down which help a lot.

OBJ post-processing step

This step iterates through the OBJ scanline buffer and copies the pixels belonging to the specified priority level into a general scanline buffer. Tilemaps also write to the general scanline buffer in subsequent steps.

The OBJs’ in-memory locations don’t matter at all in this step as the ordering information is only being obtained from the OBJ scanline buffer, from the earlier pre-processing step.

OBJ pre-processing visualization

I improvised some logging for mGBA to visually “step through” the construction of the OBJ scanline buffer.

I’ll use our familiar scene as an example:

Let’s inspect scanline 91:

Here’s the memory layout of the OBJs we care about:

(The cyan pixels represent transparent pixels)

Visualization continued

Here’s the construction of the OBJ scanline buffer step by step:

The thin white lines separate the steps. The thick white line separates the color buffer from the priority level buffer (in grayscale: lighter gray = lower OBJ priority = drawn on top of darker gray pixels).

Here are the steps:

The left wall OBJ is drawn with priority 3.
Transparent pixels from the character OBJ are drawn with priority 2. They don’t modify the color buffer BUT they promote the overlapping pixels that the left wall drew to priority 2 (woops).
Opaque and transparent pixels from the drop shadow OBJ are drawn with priority 2. The shadow gets clipped on the left side by the wall pixels that got promoted in the previous step (oh no!).
The right wall OBJ is drawn with priority 3. It gets clipped by the right side of the drop shadow OBJ which is the intuitive behavior - no complaints.

When it comes to compositing the OBJs with the tilemaps in a later process, we find that a portion of the left wall now renders on top of the wrong tilemap due to its level order promotion (oh no!). This is what the GBA programming manual warns us about specifically.

Conclusion

The game’s code doesn’t conform to the platform’s specs which can cause draw priority issues.

The bug could have remained in the final release because:

The developers most likely had bigger fish to fry. This bug has a very limited impact all things considered.
The specific manual used by the developers maybe didn’t contain the cautionary paragraph. It wasn’t present in the first few editions of the English manual for example.

This is the last article in the series (excluding the appendices) but it’s not over yet! I haven’t finished writing all the previous articles so I’ll be publishing them gradually.

You can subscribe to the mailing list to be notified when a new article appears.

Sonic Battle (GBA) Renderer Series - Walls

2019-01-23T05:15:38-05:00

Coordinate system
Walls

Welcome to the seventh article of the Sonic Battle (GBA) Renderer series.

Coordinate system

Quick reminder (just in case): here’s a link to the coordinate system.

Walls

Directions

All walls are axis-aligned. (Except for the 45-degree walls but they will be discussed in a later article.)

It means that walls only face 4 directions which I call:

Y_PLUS
X_MINUS
Y_MINUS
X_PLUS

Visibility

Our walls represent boxes so it’s safe to assume that if a given side is visible, then the opposite is invisible. (Ex: X_MINUS and X_PLUS visibility is exclusive)

This fact can be used for an optimization: once a side is determined to be visible, the processing of the opposite one can be skipped. Note that there’s a special case where two opposite sides can be both not visible, it happens when the dot product for both is 0.

The usual algorithm for determining if a wall is facing away from a view is to check if the dot product of its normal against (the opposite of) the camera’s direction is smaller than 0.

In short: visible = dot(wallNormal, -cam.directionIn2D) > 0.

Our wall directions are the +/-X axis and +/-Y axis.

Then the dot product for the X_PLUS direction is dot(vec(0, 1), -cam.directionIn2D). It simplifies to -cam.directionIn2D.y. The dot product for X_MINUS is cam.directionIn2D.y.

Similarly, for Y_PLUS it’s -cam.directionIn2D.x and for Y_MINUS it’s cam.directionIn2D.x.

This translates to:

WallVisible[WallDirection::Y_PLUS] = camDirXY.x < 0.0f;
WallVisible[WallDirection::Y_MINUS] = camDirXY.x > 0.0f;
WallVisible[WallDirection::X_PLUS] = camDirXY.y > 0.0f;
WallVisible[WallDirection::X_MINUS] = camDirXY.y < 0.0f;

Another way to look at it is to visualize the camera angle from the top, where the relation between wall directions, camera direction, and XY axes is clear:

Re-using affine matrices (optimization)

Once the visible wall directions are determined, the corresponding affine matrices are computed.

We re-use these affine matrices for every wall of the same facing direction because an individual wall’s position doesn’t affect its projection (only wall normal and camera direction matter - as will be described below).

So all visible X_PLUS walls share the same X_PLUS affine matrix for example.

This is a significant optimization: each frame we only compute 1 or 2 affine matrices instead of 1 per wall.

Affine matrix

My initial approach was very awkward, I’m glad that I replaced it by a much simpler one recently.

In a nutshell we need to find the matrix that will transform the wall texture’s local coordinate space to the coordinate space of the texture as it would appear on screen.

Here’s an illustration of the full process:

We can pre-compute the world-space texture X axis (wall tangents). The texture Y axis points straight down (-Z in world space). So we create a matrix which uses those two as basis vectors.

Next we transform that matrix into camera space (inCamSpace = mat3(camTransformInverse) * inWorldSpace).

Then we perform the orthographic projection (by simply discarding any depth-axis information). Finally we flip the vertical axis to account for screen/canvas coordinates.

I’m fairly confident that this method would support camera roll and arbitrary wall orientations. I’m tempted to revisit the approach I used earlier for tilemaps so that they could support these things too. Not that it matters for Sonic Battle of course, since it doesn’t use those bells and whistles.

Note: on the real hardware we wouldn’t be done just yet. There’s a few gotchas with affine matrices.

View-space position

The view space positions of individual walls are computed the same as for tilemaps: camera.transformMatrixInverse * worldSpacePosition.

Note - Tilemap and wall base scale

All dimensions of the game world (save for the characters) have been doubled! Tilemaps and most wall sprites are exactly x2 larger than their source bitmaps. That really tripped me up until I figured it out with mGBA’s inspection tools.

This wall is rendered ~64 pixels wide at native screen resolution

But its bitmap is only 32 pixels wide (the "Double Size" flag is unrelated by the way)

The wall behind it though has the same width as its bitmap (it's only scaled in height) - so there's some flexibility here

This could have been done for performance reasons, to use ~4 times fewer hardware sprites to cover the same area (which reduces load on the entire renderer).

Or for artistic reasons, to reduce the visual noise.

Note - Wall height scale

For gameplay reasons (to prevent the action from vertically overflowing the screen) all walls are additionally scaled in height to make them shorter. It also looks better that way.

Just the basic x2 scale - that's really tall

x2 scale and additional height scale (I eyeballed the value, it's ~1.4)

The next article will discuss the depth sorting of the walls.

You can subscribe to the mailing list to be notified when a new article appears.

Sonic Battle (GBA) Renderer Series - Drop Shadows

2019-01-20T05:55:28-05:00

General concept
Investigation
Overlap detection
Edge case
Draw order
Draw order - reality

Welcome to the twelvth article of the Sonic Battle (GBA) Renderer series.

General concept

Drop shadows are sprites that are drawn under the feet of Sonic Battle characters. They follow the X and Y center positions of their characters and snap to the terrain height on the Z axis. They serve as a visual clue of the characters’ positions.

The shadows are scaled uniformly in width and height according to the character’s distance to the ground under their feet. It’s done through frame animation so no affine transformations are used at all (besides the usual translation). The base image is already vertically squished to approximate the effect of the camera’s pitch.

Investigation

The mGBA emulator’s sprite inspection tool confirms the absence of affine transformations:

It also shows that there are 3 frames for the scaling animation:

I mistakenly implemented the scaling using an affine matrix, and let me tell you - the aliasing is terrible due to the GBA’s small screen resolution. Frame animation was possibly used here to avoid that specific issue, if not for performance reasons.

Overlap detection

The physics engine of Sonic Battle is out of the scope of this series. But something minimal needed to be implemented for the drop shadow vertical positioning. I would later use the same algorithm for preventing character-wall intersection.

I was interested in detecting circle overlap against an axis-aligned rectangle (AABB). The circle would represent a character’s contour on the XY plane, and the AABB would be a section of the high ground (or top floor). I found StackOverflow user Jack Ding’s solution (https://gamedev.stackexchange.com/a/120897) particularly intuitive and elegant so I chose to implement it.

It’s not representitive of the true game’s physics engine, most likely.

Edge case

It’s possible for characters to be right above rising/falling edges of the terrain. A realistic shadow would need to be split into parts and projected on a number of surfaces. That’s a detail I can live without, and so could the game’s developers: in the edge case the top terrain wins the tie and the shadow appears to levitate.

Draw order

My implementation assumes that because shadows are projected on terrain they should be the very first to be drawn after the said terrain appears on screen. Nothing in the game world is situated between the terrain and shadows so they don’t need to be sorted either (but they do need to be grouped by floor).

Callback to article 5, The big picture.

Draw order - reality

The draw order is a tad more complicated than I had thought initially.

What! How? Our established draw order here is 1st floor tilemap -> walls -> 2nd floor tilemap -> 2nd floor shadow -> 2nd floor character. But in reality there’s a wall sprite that is drawn after the 2nd floor shadow? Unacceptable I say.

I did some digging. Turns out that my draw order theory was largely correct - it corresponds to the original developers’ intentions. However they ran afoul of a tricky hardware constraint. The consequences are whacky draw priorities like the one I captured above.

I wrote another post that goes into a lot of detail about the issue: The Escher Sandwich.

The next article will discuss the culling of all the walls and objects contained in a scene.

You can subscribe to the mailing list to be notified when a new article appears.

Sonic Battle (GBA) Renderer Series - Inverted Walls

2018-11-25T05:55:28-05:00

Intro
Inverted normals
Conclusion

Welcome to the ninth article of the Sonic Battle (GBA) Renderer series.

Intro

The Holy Summit map in Sonic Battle is unlike the others. Instead of containing box-like platforms it has two inset holes in the ground (for lack of better terms).

A typical map

The outlier

When I was studying the renderer it gave me a bit of a scare because it broke my assumption that all maps were roughly the same - that there’s a floor, then walls for small boxes, then another floor to cover the top of the boxes. I feared that it would require custom logic to support.

Inverted normals

Previously we had been looking at boxes from the outside where their wall normals pointed outwards. Now we’re looking “inside” the boxes. So the trick is to invert the normals by placing the walls in the opposite of the usual directions. It doesn’t affect performance either!

Normal normals

Inverted normals

Conclusion

On the surface it looked like Holy Summit was going to require lots of custom logic. But it ended up just being a creative usage of the renderer by the original developers. Hats off to them!

The next article will discuss the 45-degree angle walls which are used in octogonal boxes.

You can subscribe to the mailing list to be notified when a new article appears.

Sonic Battle (GBA) Renderer Series - Camera and tilemaps

2018-11-14T09:20:53-05:00

Camera
Tilemap

Welcome to the sixth article of the Sonic Battle (GBA) Renderer series.

Camera

Background info

A general camera system is described in Joey de Vries’ learnopengl.com series:

Coordinate Systems
Camera

Specific

For reference, here’s the coordinate system used in this series.

Our camera has very little “primary” state. The state that the user can set is:

Position in 3D space
Heading (rotation around the Z axis)
Pitch (rotation around the X axis)

It also contains “secondary” state which is derived once per frame from the primary state:

Facing direction in 3D
Facing direction in 2D (on the XY plane)
Transformation matrix and its inverse (the inverse is the conventional view matrix)

Transformation matrix

Create translation matrix using position
Apply heading rotation
Apply pitch rotation

In conventional matrix notation this would be: translationMatrix * headingMatrix * pitchMatrix.

The GLM math library takes care of the specifics. A few details:

Base axis rotation (like rotation around the Z or X axis) is a special case which can be computed cheaply. GLM’s euler angle functions are used for that.
The inverse can be computed from a mat3, then casted back to mat4 and multiplied with the opposite translation matrix (mat3 inverse is a ton faster than the mat4 inverse)

Facing direction in 2D

vec(cos(heading), sin(heading))

(Using the trigonometric circle.)

Facing direction in 3D

The default direction is vec(0, 1, 0, 0)
The 3D direction is transformMatrix * defaultDirection

Which simplifies to just taking the second column of the transformation matrix.

Note - canvas space

I’ll ommit the subject of canvas/screen space (see this note for details). For the sake of simplicity the series will treat view/camera space as the final space for rendering.

Tilemap

To render a tilemap we need to compute its affine matrix and its view-space position.

We’re dealing with two tilemaps (bottom and top). They can both use the same affine matrix because only their positions differ.

View-space position

First, to get the view-space (or camera-space) position: camera.transformMatrixInverse * worldSpacePosition

Affine matrix

Our affine matrix is a 2D (2x2) matrix. To compute it we:

Create a scale matrix from vec(1, cos(camera.pitch))
Apply rotation to it using the opposite of the camera heading

In conventional matrix notation this would be: pitchScaleMatrix * oppositeHeadingRotationMatrix.

Scale logic

The orthographic projection used by the renderer can be represented by vector projection. In the case of unit vectors, by a dot product more specifically.

The tilemap’s normal vector needs to be projected on the opposite of the camera’s direction.

The algebraic dot product formula for 3D vectors is v1.x * v2.x + v1.y * v2.y + v1.z * v2.z.

We deal only with the YZ plane here, so let’s ignore x: v1.y * v2.y + v1.z * v2.z.

The tilemap normal is vec(0, 0, 1). Let’s plug it into our 2D formula (v1 would be the tilemap normal, and v2 the opposite of the camera direction): 0 * v2.y + 1 * v2.z which simplifies to v2.z. This is cos(camera.pitch - pi) (the minus pi accounts for the opposite direction).

We’re not quite done though. Because we want to look towards +Z by default instead of -Z, we need to use the opposite direction again. So we get cos(camera.pitch - pi - pi) which simplifies to cos(camera.pitch).

Scale logic sketch

Camera pitch = 0 degrees

Camera pitch = 45 degrees

Camera pitch = 90 degrees

Visualization

Your browser doesn't support HTML5 video. Here is a link to the video instead.

Scale

Your browser doesn't support HTML5 video. Here is a link to the video instead.

Rotation

Your browser doesn't support HTML5 video. Here is a link to the video instead.

Scale and rotation

The next article deals with walls (“how to transform the 2D surfaces to make them to look 3D”).

Sonic Battle (GBA) Renderer Series - Appendix: Notes

2018-11-07T06:35:27-05:00

Intro
Notes

Welcome to the second appendix of the Sonic Battle (GBA) Renderer series.

Intro

This page is a collection of notes and implementation details. Links to specific locations in this doc are scattered across the series.

Notes

Coordinate system

The series uses the OpenGL coordinate system:

-Z: forward
X: right
Y: up

For rotation:

Right-handed
Heading (Z axis) is applied first
Pitch (X axis) is applied next
Roll (Y axis) is never used

The default camera view direction is straight down, towards the terrain:

The camera will usually be tilted for a lateral view:

Draw

On the GBA the term draw doesn’t make much sense. What we do instead is write tilemap/sprite properties into specific memory locations and the hardware takes care of the rest. That’s a recurring pattern on the GBA by the way (the memory-mapped I/O principle)! Because there’s no operating system to speak of, practically everything on the GBA is controlled by writing to memory addresses called registers. From configuring video modes and input handling callbacks to controlling the audio output.

Long story short, draw makes sense for the demo implementation because it uses modern graphics APIs through the Oryol framework.

Canvas/screen space

Top - view-space, bottom - canvas space

Our renderer calculates the view-space position of the items we want to display. That’s not enough however. The positions need to be converted to canvas coordinates for the GBA.

Gabriel Gambetta explains it in his Computer Graphics from scratch series.

Affine matrix intricacies

There’s a few gotchas described in Tonc.

One that stands out is that the GBA actually expects the inverse of the affine matrix.

Another detail is that it takes fixed-point numbers in the 8.8 format (8 bits for integer value, 8 bits for fractional value). The GBA deals natively in 32 bits, so 16 bit data types are less precise and even underperform! Tonc recommends that the standard 32-bit signed representation always be used until the final conversion is necessary.

Sonic Battle (GBA) Renderer Series - The big picture

2018-11-03T06:35:27-04:00

Steps
Frame breakdown

Welcome to the fifth article of the Sonic Battle (GBA) Renderer series.

These are the steps that the demo renderer executes every frame. The rest of the articles in the series will cover them in detail.

Steps

Main loop (general)

Poll inputs
Apply camera movement
Update camera matrices
Compute tilemap affine matrix (shared by both tilemaps)
Compute the affine matrix for each axis-aligned wall direction
Compute the affine matrix for each 45-degree angle wall direction
Compute view-space positions for both tilemaps

Main loop (first floor)

Draw the first floor tilemap
Update drop shadows + find their view-space positions + group them by floor (first or second)
Draw the first floor drop shadows
Groups walls and objects by floor + find their view-space positions + depth-sort each floor’s group separately
Draw walls and first floor objects

Main loop (second floor)

Draw the second floor tilemap
Draw the second floor drop shadows
Draw the second floor objects

Note - draw

The term draw is technically inaccurate in the GBA’s context but good enough for my purposes.

Frame breakdown

Background color

Bottom tilemap (first floor)

Drop shadows on first floor

Depth-sorted walls on first floor

Note that contrary to what the image suggests, every chunk of the wall is treated as a separate object (they're hardware sprites). They're lumped together here only to save time and space.

Depth-sorted sprite on first floor

Depth-sorted walls on fist floor

Depth-sorted sprite on first floor

Done with the first floor

Top tilemap (second floor)

Drop shadows on second floor

Sprite on second floor

There are never any walls on the second floor. The frame is complete.

The next article finally dives into the implementation! It starts with a discussion of the camera and the tilemap rendering.

Sonic Battle (GBA) Renderer Series - Capabilities and limitations

2018-10-06T08:10:57-04:00

Coordinate system
Terminology
Capabilities
Limitations

Welcome to the fourth article of the Sonic Battle (GBA) Renderer series.

Coordinate system

I’ll be using the OpenGL coordinate system:

-Z: forward
X: right
Y: up
Right-handed

In this system the default camera view direction is straight down, towards the terrain:

The camera will usually be tilted for a lateral view:

Terminology

Objects represent characters, projectiles, and visual effect particles.

The term is used to distinguish them from walls, since walls and objects are both GBA hardware-accelerated sprites (discussed in article 2) under the hood.

Capabilities

Basic elements

As seen so far, Sonic Battle can display a floor and a “ceiling”/”second floor” which are aligned to the XY plane.

It can also display walls which are aligned to either the ZX or the ZY plane.

These elements are often combined to display “box” structures:

Maps are full of these boxes:

Inverted walls

There’s a map with inverted walls. It’s like looking at the inside of a box:

As will be discussed in a future article, this variety comes at no performance cost.

45-degree walls

There’s also walls which are twisted 45 degrees around the Z axis:

Walls and objects culling

Walls and objects which are offscreen are removed from the display list to save resources.

Backface wall culling

Only “closed” volumes are being rendered, so walls which are facing away from the camera aren’t rendered because they would be invisible anyway:

The dotted outlines show the hidden walls

Drop shadows

When characters jump, fly, fall, or otherwise move along the Z axis it can become hard to determine their position.

The game uses a round shadow under each characters’ feet to solve the issue.

The shadow’s position indicates the character’s position on the XY plane.

The shadow’s size hints at his position on the Z axis relative to the object under his feet (not necessarily the “floor”/terrain).

Depth sorting

The hardware-accelerated rendering of the GBA allows to set the draw order of tilemaps and sprites, but it can’t be more granular.

In other words the renderer is limited to the painter’s algorithm, where the draw items are sorted by distance to camera and drawn in “far to close” order. This works well overall but has some edge cases which are discussed below.

Limitations

Camera

The camera rotation is evaluated in the following order:

Z axis rotation: Heading
X axis rotation: Pitch

Note that Y axis rotation (Roll) is never used in Sonic Battle.

Also, the camera pitch is always between 0 and 90 degrees.

In the context of the game it makes no sense to break these rules. They also enable many assumptions and performance optimizations.

Projection

The projection is orthographic. I think that perspective projection would have been possible with some more work. But it wouldn’t have brought any gameplay value, and it would have hurt the clarity of the visuals.

Object-wall clipping issue

The renderer seems to depth-sort by horizontal middle, vertical bottom points. We can observe the side effects it causes, for example:

Indicated are the sorting reference points for the character, the facing wall, and the side wall:

The character’s sorting reference point is in front of the others, so he’s drawn last:

The character’s point is between the side wall’s and the facing wall’s. He’s drawn on top of the side wall but is clipped by the facing wall (in the area near his hand):

The character’s point is behind the two others. He’s clipped by both walls:

Wall-wall clipping issue

There’s a specific map with unusually positioned walls which don’t depth-sort correctly:

Illustrated wall bounds and sorting reference points

Object-ceiling clipping issue

The “ceiling” faces of the “boxes” are all part of a tilemap and can’t be depth-sorted individually.

"First floor" tilemap

"Ceiling"/"second floor" tilemap

Our draw order is roughly this:

First floor tilemap
Walls and objects
Second floor tilemap

But what happens when objects move up the Z axis, and eventually reach the second floor?

The draw order becomes as follows to accommodate it:

First floor tilemap
Walls and objects which are on the first floor
Second floor tilemap
Objects which are on the second floor

A sorting issue happens when a object is in between the first and second floors:

Jumping up from first to second floor

Jumping down from second to first floor

It’s not obvious why the issue also appears when jumping down. It will be discussed in a later article, fear not.

Wall offset issue

The walls and tilemaps don’t exactly match positions. This creates cracks at the seams of the “closed” volumes they’re supposed to represent:

Cracks at the left seams of the boxes

I don’t know if it’s caused by fixed point precision issues or by an “off by one” bug.

Wall jitter issue

I think this is caused by fixed-point math precision issues.

The further the player is from the center of the map (the bigger the magnitude of the numbers involved), the worse the jitter.

This GIF was recorded at the edge of the one of the widest maps where it’s definitely worse than usual:

Bottom line

These rendering quirks might seem like a big deal, but they’re honestly not.

I’ve played through the game twice and I never noticed even one of them. I only discovered the issues after careful inspection once I noticed them in my own implementation.

Also note that all the images and gifs that I included were upscaled by a factor of x2, so everything is less distinguishable at native resolution, and even less on the console’s small LCD screen.

The next article is a high-level summary of my implementation.

Sonic Battle (GBA) Renderer Series - Palette shifting

2018-09-25T12:42:43-04:00

Colors
Color palettes
Putting it all together
Applications

Welcome to the third article of the Sonic Battle (GBA) Renderer series.

This articles takes a slight detour to discuss palette shifting, a visual technique that Sonic Battle uses to great effect. It doesn’t have anything to do with 3D rendering but it’s so neat that I included it anyway. It’s also an example of turning a technical limitation into something great, which gamedev is famous for.

Colors

Colors are represented by 16 bits on the GBA.

5 bits (containing values 0-31) for red, 5 for green, 5 for blue, and 1 unused leftover: xRRRRRGGGGGBBBBB.

Color palettes

The GBA’s video mode 2 provides two palettes (each containing 256 colors) at our disposal, one for tilemaps and one for sprites.

Here’s two sample 256-color palettes:

Background/tilemap palette on the left, sprite/object palette on the right

Putting it all together

Just like tilemaps are built out of tiles contained in a tileset, tiles are built out of colors contained in a palette.

This way each pixel in a tile only needs to store the index of its color (8 bits, values 0-255) instead of the full color value (16 bits).

Additionally, a tile can be configured to use one of the 256-color palette’s 16 sub-palettes (one of the 16 rows). This limits the asset to the 16 colors contained in its sub-palette but it only requires 4 (values 0-15) instead of 8 bits per pixel.

Palettes also make it easy and fast to change asset colors: tile data can be left untouched because only the entries in the palettes need to be modified.

Applications

Waves

At first I thought this was done through flip-book animation by cycling through tiles.

An example of the flip-book technique would be cycling through tiles 0-7 of the 4th row (the one with the downwards arrow):

Here’s what that looks like:

(Tileset by Leonard Pabin)

But in Sonic Battle this effect is achieved instead by cycling the water and sand colors. You can see them shifting in the tilemap palette:

This palette shifting technique doesn’t need individual animation frames (which entail a copy per frame of every unique tile involved in the animation). Palette shifting eliminates these memory and production costs which helps achieve much smoother animations.

Pulsating lights and circular motion

Same concept, different effects.

Shading

Walls can be either lit or occluded. This only affects their colors, so it saves space to use the same tiles for both versions but with two different palettes.

With lit walls palette (sub-palette 12):

With occluded walls palette (sub-palette 13):

Character color variations

Players can choose the colors for their character:

Default character palette:

Custom character palette:

The same technique is also used for enemy color variations. For instance hostile versions of friendly robot characters have grayscale colors.

That’s a wrap for palette shifting. It’s by no means unique to Sonic Battle though.

Mark Ferrari had perfected the technique in the 90s. Check out his 8-bit vistas where water waves, waterfalls, rain, snow, and more are simulated through palette shifting: Old School Color Cycling with HTML5.

The next article kickstarts the 3D renderer analysis by first discussing its capabilities and limitations.

4AM Games Blog

Unity FPSSample - Takeaways from the “Deep dive into networking” talk

Table of Contents

Intro

Overview

New stuff

Asymmetric tick rates

Delta encoding using frame prediction

Data stream structure

RTT measurement

58/62 FPS trick

Server sleep time

Clock drift

Fractional ticks on client

Classic

Reliable UDP implementation

Past input redundancy

Lag compensation for hitscan

Client-side interpolation

Delta encoding compression

Open questions

Server reconciliation

Client-side interpolation time

Multiple client ticks for a single server tick

Fractional ticks

Sonic Battle (GBA) Renderer Series - The Escher Sandwich

Table of Contents

The Sandwich Theory

The Escher Sandwich

Draw priority overview

The problem

Circuit behavior - in emulation

Render loop

OBJ pre-processing step

OBJ post-processing step

OBJ pre-processing visualization

Visualization continued

Conclusion

Sonic Battle (GBA) Renderer Series - Walls

Table of Contents

Coordinate system

Walls

Directions

Visibility

Re-using affine matrices (optimization)

Affine matrix

View-space position

Note - Tilemap and wall base scale

Note - Wall height scale

Sonic Battle (GBA) Renderer Series - Drop Shadows

Table of Contents

General concept

Investigation

Overlap detection

Edge case

Draw order

Draw order - reality

Sonic Battle (GBA) Renderer Series - Inverted Walls

Table of Contents

Intro

Inverted normals

Conclusion

Sonic Battle (GBA) Renderer Series - Camera and tilemaps

Table of Contents

Camera

Background info

Specific

Transformation matrix

Facing direction in 2D

Facing direction in 3D

Note - canvas space

Tilemap

View-space position

Affine matrix

Scale logic

Scale logic sketch

Visualization

Sonic Battle (GBA) Renderer Series - Appendix: Notes

Table of Contents

Intro