Posted by JibbSmart on 23 May 2019 14:54

Whether you call it "motion controls", "motion aiming", "gyro aiming", or something else, the gyroscope is the mouse of the controller. It offers directness and exactness of control that's just impossible with tiny thumbsticks. While a controller's gyro and accelerometer are often used in games as a tool for figuring out real world orientation — rotating 3D objects, interacting with a 3D world in VR or AR, or detecting player gestures — this is not a guide for those kinds of motion controls. You can learn more about those elsewhere on this site. But the gyro is severely underutilised as a mouse.

Having an easy-to-use mouse built in to your controller is a really big deal. But few games make use of it, and those that do often make simple missteps. The goal of this guide is to establish good conventions that make it easy for developers to give gyro controls a go, improve the standard of gyro controls players expect from games, and thus make console gaming better for new gamers and established gamers alike.

None of the games in that video support gyro controls (or at least not as they're shown in the video). However, JoyShockMapper allows us to try out really good gyro controls by converting controller inputs to keyboard and mouse inputs. If you like, you can use it to follow along in parts 1 and 2 in just about any game you like (assuming it uses a mouse).

This post is Part 1 of 3. Here we'll cover principles of good gyro controls as well as what it takes to implement them well in a game. It's very simple, and a lone programmer could try these out in an already-working game in very little time, whether it's a 2D or 3D game.

Part 2 will look at what we can do with the right stick in 3D games now that precise aiming is handled by the gyro. Specifically, we'll look at flick stick, which, like just about everything in part 1, you can try out with JoyShockMapper.

Part 3 will look at game-defining gyro. If we forget about the limitations of mouse-aim games and go beyond what's possible by just faking mouse inputs, what can we do? This will be beyond what JoyShockMapper can do.

But let's get into Part 1 — better practices for using the gyro as a mouse, whether it's turning a 3D camera or controlling a 2D cursor. Look out for these boxes that highlight key conclusions reached by the paragraphs that precede them:

One principle for good gyro controls

We have one principle for good gyro controls:

Simple, right? In the same way as a regular mouse (and as opposed to a thumbstick), the gyro translates a real movement to a proportionate in-game movement, allowing the player to move a cursor, aimer, or camera as if directly with their own hand. This is actually really cool, because:

1. Many games rely on the mouse, or are both easier to learn and have greater room for mastery with a mouse than with a thumbstick.
2. Decades of the mouse being used in games have established helpful conventions that may translate well to gyro controls.

This principle informs our values, and how we go about achieving them:

• Simplicity — Being easy to understand (for players) and easy to implement (for developers) can go hand in hand.
• Transferability — A simple and straightforward foundation embraced in multiple games (just like standard mouse controls) allows players to bring skills learned in one game to others.
• Responsiveness — For such direct controls as controlling a cursor or camera with a mouse, responsiveness mustn't be compromised.

The short version

All these points come from the "Do" (green), "Don't" (red), and "Caution" (yellow) boxes strewn through the rest of the article, each one summarising a conclusion reached in its section. Here we go:

• At its core, your gyro controls convert a calibrated rotational velocity to a linearly proportional in-game rotation (3D games - yaw velocity for yaw, pitch velocity for pitch, under most circumstances) or cursor movement (2D games - yaw velocity for x axis, pitch velocity for y axis, under most circumstances)
• The player should be able to calibrate the controller at any time.
  • If you or your platform can auto-calibrate, that's great, if and only if the player can disable auto-calibration if they desire. Auto-calibration will get things wrong at crucial times.
• For 3D camera-turning games, when you're trying to figure out what the numbers on your sensitivity scale should mean, there's only one true scale: the natural sensitivity scale. You're translating real world rotational velocities to in-game rotational velocities, so let your "1" be "1".
• Make sure your scale goes high enough! Many players are most comfortable playing with a sensitivity of 4, or even as high as 8! But most games don't let players set their sensitivity that high. 10 is a nice round number that should cover the vast majority of players.
• For 2D cursor-controlling games, there's no obvious natural scale. But if you can pick something sensible, or something another popular game uses, please do! Make it easy for players to move from one game to another!
• The game should offer more options to help the player with the trade-off between range and precision as they choose their sensitivity, and they should all be optional and, ideally, configurable:
  • Acceleration lets the player have a higher sensitivity for fast movements than for slow movements. Mouse acceleration settings are not transparent to the player, and don't translate well between different games or different mice, so check out my alternative below.
  • Smoothing can stabilise the cursor/aimer, but should never be applied to big movements. A tiered smoothing solution can steady the cursor when the player's trying to hold the controller still without harming responsiveness and precision at all with bigger movements.
  • Minimum velocity threshold is the worst. Don't do it at all. Especially in 3D shooters — targets can be any distance away and be moving at any speed in screen-space. No minimum velocity threshold can help the player in some circumstances without harming the player in others. But a "tightening" threshold can help keep the cursor still when you want it to with far fewer side-effects.
• Lifting the mouse — the gyro equivalent is an input that can disable the gyro, whether it's a dedicated button, a button that does different things depending on whether you tap it or hold it, or, for 3D games, while the right-stick is being used. Of course, for games where the player spends most of the time not needing the gyro and would only need it when a weapon or other aiming-device is drawn (like just about every third-person action blockbuster on the PS4), the gyro can simply be activated when going into aim mode, and doesn't need another input to disable it.

Many of these have gotchas, complications, or simpler ways they can be expressed to the player, and I discuss these over the rest of this article. If you have any objections or questions, please read the corresponding section below, and if that doesn't make things clear as day, please reach out to me! I should either be able to help you or learn from you.

Implementation

Now, I know PlayStation and Switch controllers are capable of doing all of these well — especially the DualShock 4. I wrote JoyShockMapper and JoyShockLibrary, so I know what comes out of those controllers. I play PC games almost exclusively using JoyShockMapper to convert controller (including gyro) inputs to keyboard and mouse.

But I do not know what Nintendo and Sony make available to developers on their platforms. Automatic recalibration, for example, appears to be ubiquitous in Switch games with gyro controls. Is that because Nintendo recommends it, because the Switch can do it automatically, or because the Switch does do it automatically, whether the game developer wants it or not? I don't know.

But between Sony, Nintendo, and you, everything here is definitely doable with the hardware your players already have. And if you're supporting these controllers on PC, there's certainly nothing stopping you from doing everything described here.

So let's get right to implementing simple gyro controls. We'll have code snippets, JoyShockMapper settings, and video examples using JoyShockMapper to add great gyro controls to games that don't have them. If you want to follow along with JoyShockMapper and your favourite game, load up a configuration, and start by putting in your in-game sensitivity so JoyShockMapper's correctly calibrated:

IN_GAME_SENS = <Your in-game sensitivity>

Then, before we begin, let's disable everything that might affect gyro controls:

GYRO_SENS = 0
GYRO_CUTOFF_SPEED = 0
GYRO_CUTOFF_RECOVERY = 0
GYRO_SMOOTH_THRESHOLD = 0

The Core

So, mouse aim is pretty standard. We start with something basic like this:

ProcessMouseInput(Vec2 deltaMouse, float mouseSensitivity)
{
   Camera.Yaw += deltaMouse.X * mouseSensitivity;
   Camera.Pitch += deltaMouse.Y * mouseSensitivity;
}

Simple, right? For games with a cursor, replace Camera.Yaw and Camera.Pitch with Cursor.X and Cursor.Y. On PC, of course, 2D games will often just directly use the mouse position unless they're using raw mouse input. Some games will have (optionally) a good mouse acceleration, and we'll get to that later. But anyway, here's our simple gyro aim:

ProcessGyroInput(Vec2 calibratedGyro,
   float deltaTime, float gyroSensitivity)
{
   Camera.Yaw += calibratedGyro.Yaw *
      gyroSensitivity * deltaTime;
   Camera.Pitch += calibratedGyro.Pitch *
      gyroSensitivity * deltaTime;
}

This is pretty simple, too. There are some differences, though:

• Gyros need to be calibrated.
• Gyros give velocity, not displacement. So we use deltaTime (the time since the last input sample) to convert to a displacement. Still, mouse and gyro both map a displacement to a displacement, and a velocity to a velocity.
• For 3D camera control games, we're converting a controller Yaw velocity to a change in camera Yaw and a controller Pitch velocity to a change in camera Pitch. This is an obvious and natural mapping from real world movement to in-game movement.

The gyro will actually give velocities around different axes — x, y, and z — rather than absolute euler angles — yaw, pitch, and roll. But I refer to yaw and pitch because that makes it more clear, I think, which movements of the gyro should change which in-game angles. Fortnite's default gyro aiming settings on Switch, for example, had turning left and right depend on the controller's roll rather than yaw. Your in-game yaw was controlled by your controller's roll. It's almost as absurd as it sounds, except that when played in handheld mode, the JoyCons are facing up, so the controller's local roll corresponds better to the player's perception of yaw. Now, thankfully the player is able to change that setting — good settings go a long way — but the default probablly should've been an "auto" option that uses the controller's roll when attached to the Switch and the controller's yaw when detached.

Back to the code. You may have expected something more complicated. I know I did when I was first writing JoyShockLibrary and JoyShockMapper. Some games will do something more complicated — they'll do something called "sensor fusion", combining a gravity vector from the accelerometer with the rotations from the gyro to know what's up and down. But I suggest not doing it for two reasons:

1. Sensor fusion can be complicated. Big commercial games that are otherwise extremely well made can apply it poorly. Super Mario Odyssey, for example, uses it for aiming a tank. But when playing with the JoyCons attached to the console, they're mostly pointing up. Pitching them further up to aim up in game makes the JoyCons upside down, and the game responds by aiming down instead. It's awful. The problem isn't intrinsic to sensor fusion — World of Goo does a much better job with it — but complicated solutions are prone to problems. Just keep it simple. Your mouse doesn't care if the user is upside down, lying on their side, or holding the mouse the wrong way, so why should the gyro?
2. Sensor fusion is a compromise between two sensors that disagree with each other. If the player is moving for a lot of time and the game doesn't get a chance to get its bearings, this causes increasing errors in the game's interpretation of the player's input.

Update, 2021: there are some benefits to a sensor fusion solution if you really do get it right. Earlier this year I introduced "player space gyro" and a detailed guide on implementing it. It offers more freedom of movement for players without introducing any error over a gyro-only solution. It doesn't generalise quite as well as "local" gyro controls, so perhaps don't go down that road if you're only trying to hit the basics.

Now, while games with standard mouse controls sometimes do additional work (mouse acceleration, smoothing, etc), it's always optional. Always. If the player can't reduce things to essentially the "core" above, these are bad mouse controls. Even if the changes have clear benefits, players have the reasonable expectation that they'll be able to have the cursor or aimer respond in linear proportion to their mouse input, without delay. This inspires some of the values of good gyro controls — simplicity and transferability. These controls are easy for players to understand, easy for developers to implement, and understandably form the basis for practically all games that benefit from a mouse-controlled cursor or aimer.

This should be equally true for gyro controls. As we continue, we're going to look at doing more than just our "core" above, but everything we add will be configurable and optional. These things will include acceleration, smoothing, and we'll look at speed thresholds (which are the devil's work, but some games have them, so let's look at them). If any game that uses the gyro to control a cursor or aimer has these features and they can't be disabled or configured to the point that they're truly negligible, that game has bad gyro controls. Even if they had my preferred acceleration settings and filters and did everything "right" from my subjective point of view (which they don't), they're still bad gyro controls if those settings and filters can't be disabled.

But to fully understand our core, we need to understand the variables involved. Our inputs to the ProcessGyroInput function are:

• calibratedGyro — the calibrated gyro input from the controller, usually in degrees per second
• deltaTime — the amount of time (usually in seconds) since the last input sample
• gyroSensitivity — a user-determined multiplier so players can increase their comfortable range in the game (by increasing sensitivity) or increase precision (by decreasing sensitivity)

deltaTime is something most developers will already be familiar with — it's just the time since the last frame or input sample, and it's useful for integrating velocities. Let's talk about calibration and sensitivity.

Calibration

At present, gyros in modern controllers sometimes require calibration. They're very good at giving relative rotational velocities, but over time can have the wrong idea of where "zero" is. Calibration is just finding where the gyro thinks zero is so we can correct it.

Measuring the gyro output when the controller is stationary is how we get that zero. But since we have no reliable way of telling if the controller is still, we need to prompt the player to put the controller down or hold it still so the game can get the zero value. If the player's holding it, their hands are shaky; even if the controller is perfectly still, the gyro's output will be slightly noisy. Either way, getting the zero value at an instant is probably not going to be good enough — better to get the average output over a short window of time:

// current velocity, uncalibrated
Vec3 Velocity;
 
// for calibration
int NumOffsetSamples;
Vec3 AccumulatedOffset;
bool Calibrating;
//...
 
// settings
float Sensitivity;
//...
 
Vec3 GetCalibrationOffset() {
   if (NumOffsetSamples == 0) {
      return Vec3.Zero;
   }
   return AccumulatedOffset / NumOffsetSamples;
}
 
ResetCalibration() {
   NumOffsetSamples = 0;
   AccumulatedOffset = Vec3.Zero;
}
 
ProcessInput(float deltaTime) {
   if (Calibrating) {
      NumOffsetSamples++;
      AccumulatedOffset += Velocity;
   }
 
   // we're using gyro as a mouse, only need 2D input
   Vec2 gyroVelocity = Velocity.XY;
   Vec2 gyroCalibration = GetCalibrationOffset().XY;
   Vec2 calibratedGyro = gyroVelocity - gyroCalibration;
 
   ProcessGyroInput(calibratedGyro,
      deltaTime, Sensitivity);
}

Your API might make things even simpler. It might have functions to start calibrating (JoyShockLibrary has JslStartContinuousCalibration), stop calibrating (JslPauseContinuousCalibration), reset or manually set calibration (JslResetContinuousCalibration and JslSetCalibrationOffset, respectively).

Sensitivity

When playing 3D games where the mouse turns the camera, the mouse sensitivity scale can seem arbitrary. A sensitivity of 1.5 in one game might give you the same results as a sensitivity of 5 in another. This is understandable, as there's no obvious mapping from a 2D mouse input to an in-game rotation. This is exacerbated by different mice having different resolutions and polling rates, so the same settings in the same game won't necessarily behave the same with different mice.

An advantage specific to gyro controls in 3D games is that there really is one good scale for sensitivity, because a real world rotation is being mapped to an in-game rotation. For any 3D camera-controlling game with gyro controls, a sensitivity of 1 should mean that if you turn the controller 37.5° to the left, your camera or aimer should turn 37.5° to the left. If you want steadier aim for distant or small targets, you could reduce it — a sensitivity of 0.5 means in-game turns are only half your real world turns, making it easier to precisely fine tune your aim. If you want a wider range of motion without having to turn your controller uncomfortably far, you could increase it — a sensitivity of 2 means in-game turns are double your real world turns. Let's call this the "natural sensitivity scale".

The natural sensitivity scale makes gyro controls more simple for players, because they can easily understand what the gyro sensitivity means. It also helps the transferability of gyro controls between different games, assuming multiple games actually share this scale. This makes it easier for players to pick up gyro controls in one game if they've already learned to use them in another.

If your character rotation is given in degrees, our core snippet already honours the natural sensitivity scale. The calibrated input is in degrees per second, the time since last sample in seconds, and then it's multiplied by the player's sensitivity setting giving a change in degrees.

Here's what it looks like in action in Overwatch using JoyShockMapper with a gyro sensitivity of 1:

GYRO_SENS = 1

Overwatch doesn't natively have gyro controls (yet!), but perhaps that'll change in future? We still have our usual stick controls that we can use at the same time, but aren't relying on a stick for the finer aiming.

And here's what it looks like with a gyro sensitivity of 2:

GYRO_SENS = 2

We have much better range of movement here, but at the cost of precision. Of course, with practice, you can make up some precision. I'm not used to playing at 2 for the precise stuff, and it shows in the video above.

For 2D games the gyro is still a great mouse. But there's no super obvious "natural" sensitivity scale from an orientation in 3D to a 2D plane, so I'm not going to put anything in a green box for 2D sensitivity. But here's what I've tried to establish as convention in JoyShockMapper: a sensitivity of 1 means 1 complete revolution of the controller left-to-right will move the cursor one screen-width left-to-right. That's a long way to move the controller, and the player will never play with such a low sensitivity. A more sensible setting is something like 8 (1/8 of a revolution to screen-width, or 45°) or 16 (1/16 of a revolution to screen-width, or 22.5°).

I know the mapping isn't super obvious, and it takes a little work to convert a sensitivity to something the player understands. But it works. It's still a simple multiplier on the player's input (doubling the sensitivity doubles the speed you get from a given input), and at this scale you could likely give the player the precision they need with the sensitivity constrained to integers — although I'd suggest erring on the side of finer control. If you've got better ideas for a 2D cursor sensitivity scale for gyro, please let me know!

There's a lot to like about a higher sensitivity, whether a 2D or 3D game. We now have a much wider range of movement without having to hold the controller uncomfortably. Just like with a regular mouse, higher sensitivity comes at the cost of precision — the faster the cursor or aimer moves in response to our movements, the harder it is to make smaller movements.

Small movements are difficult enough to do precisely. A regular mouse and a gyro controller each have unique complications. A regular mouse has some friction with the mousepad or surface it's on, making small movements less consistent, but at least it's easy to keep the mouse still. A gyro, on the other hand, is essentially frictionless, as it doesn't rest on a mousepad. But with nothing to rest on, it's more difficult to keep the controller perfectly still. And while players can improve their mouse experience by getting a larger mousepad so they can keep mouse sensitivity relatively low, we can't do that with the gyro — its range is essentially fixed to what's comfortable for the player.

So what can we do to give the player as much range of movement as possible while still allowing them all the precision they need for small on-screen buttons and distant targets?

Improving range and precision

We're going to look at three tools commonly used for keeping small movements precise while allowing the player to choose a higher sensitivity, in descending order of importance:

1. Acceleration
2. Smoothing
3. Minimum velocity

They all have weaknesses either in their common implementation or fundamentally. The third option is terrible. Don't do it. But in that section we'll look at why it's bad, and we'll look at something better that's inspired by it, which we'll call a "tightening threshold".

Acceleration

The challenge in choosing a good sensitivity comes from the fact that decreasing sensitivity will improve your precision but make it harder to make fast turns, while increasing sensitivity will increase the range you can turn comfortably at the cost of precision for small or slow-moving targets. In 2D, low sensitivity makes it a chore to move the cursor from one side of the screen to the other, but high sensitivity makes it difficult to hit small buttons.

But imagine you could choose one sensitivity for when you're moving the controller slowly, another sensitivity for when you move it quickly, and have the game smoothly interpolate between the two sensitivities according to how fast you're moving the controller between "slowly" and "quickly". That's what acceleration does.

The name can conjure up awful ideas — a velocity or sensitivity that changes over time, as we usually think of acceleration. But, borrowing from "mouse acceleration", it really just means that your effective sensitivity changes according to your velocity that instant. Mouse acceleration itself has an undeservedly poor reputation, but when done right, it's consistent (even under varying framerates) and easy to learn.

But it's almost always presented in ways that are difficult for the player to interpret. In some games, you'll choose a mouse acceleration of "none", "low", "medium", or "high", whatever that means. In others, it's a single number — a gradient for the sensitivity when graphing input speed on the X axis and desired sensitivity on the Y axis. When the mouse isn't moving it'll be whatever their mouse sensitivity setting is. As input speed increases, the gradient is used to determine the sensitivity to apply to that input. This is still not very transparent, as the input speed isn't something intuitive to users, especially as mice vary in DPI and poll rates, and even if this speed is well understood by the player, figuring out a gradient such that the player has certain effective sensitivities when moving the mouse at certain speeds is difficult.

So now we come back to gyro. Conventions are yet to be established. But there's no such thing as DPI. Gyros report velocity already (rather than displacement), so lend themselves to a consistent and easy to understand acceleration system. When acceleration is enabled, the player is given four options:

1. Slow sensitivity. This is the sensitivity the player wants when they move the controller slowly, and of course we're using the natural sensitivity scale, which is easy for players to understand and hopefully means these settings can be shared by other games that use the same scale. This is MIN_GYRO_SENS in JoyShockMapper if you want to try it yourself.
2. Fast sensitivity. This is the sensitivity the player wants when they move the controller fast. This is MAX_GYRO_SENS in JoyShockMapper.
3. Slow threshold. This is the real-world controller speed that is considered "slow" for the purposes of calculating the actual sensitivity. Degrees per second seems to be the standard unit used by the gyro manufacturers for both Switch and PlayStation devices, and they're relatively easy for players to understand, so I urge you to use this scale. This is MIN_GYRO_THRESHOLD in JoyShockMapper.
4. Fast threshold. This is the real-world controller speed that is considered "fast" for the purposes of calculating the actual sensitivity. This is MAX_GYRO_THRESHOLD in JoyShockMapper.

Here's what that can look like:

ProcessGyroAcceleration(Vec2 calibratedGyro,
   float deltaTime, float sensitivitySlow, float sensitivityFast,
   float slowThreshold, float fastThreshold)
{
   // how fast is the gyro moving?
   float speed = Sqrt(calibratedGyro.Yaw * calibratedGyro.Yaw +
      calibratedGyro.Pitch * calibratedGyro.Pitch);
 
   // where do we stand between the slow threshold and the fast threshold?
   float slowFastFactor = (speed - slowThreshold) /
      (fastThreshold - slowThreshold);
   slowFastFactor = Clamp(slowFastFactor, 0.0, 1.0);
   // linearly interpolate
   float newSensitivity = sensitivitySlow * (1.0 - slowFastFactor) +
      sensitivityFast * (slowFastFactor);
 
   // now apply this sensitivity the way we originally did
   Camera.Yaw += calibratedGyro.Yaw * newSensitivity * deltaTime;
   Camera.Pitch += calibratedGyro.Pitch * newSensitivity * deltaTime;
}

We get the magnitude of the input. We find out whether it puts us on the "slow" or "fast" side of things, or where in-between. And then we interpolate between the player's two chosen sensitivities accordingly, before applying that sensitivity just as we did at the beginning.

For 2D cursor games the code is almost identical. Swap your "Yaw" for an "X", "Pitch" for a "Y", and "Camera" for a "Cursor".

This is what it looks like with sensitivities 1 and 2, thresholds 0 and 75 degrees-per-second:

MIN_GYRO_SENS = 1
MAX_GYRO_SENS = 2
MIN_GYRO_THRESHOLD = 0
MAX_GYRO_THRESHOLD = 75

I've found these settings to work well across a variety of games. The acceleration isn't steep, and I've found new players to pick it up quickly. However, depending on the pace of the game, different settings might be better — Quake, for example, moves much more quickly than Overwatch. A lot of mouse-controlled games will have their acceleration uncapped, and it's still possible to do that here: a simple checkbox to say "uncapped" or something, and the sensitivity line can project through the max sensitivity and threshold. This is one thing JoyShockMapper doesn't do yet, and may come later. But I really like the sensitivity cap so that I can make big flicks consistently without having to be too precise about how quickly I make the move.

Of course, we want some good acceleration settings by default for new players, but also want to keep the settings simple unless the player enables the advanced settings. While I stand by the transparency and usefulness of the acceleration settings I've described, they're not simple for a player who doesn't care about that kind of thing.

Perhaps you could let the player choose between "simple" and "advanced" for their gyro sensitivity settings. "Simple" exposes a single sensitivity slider and lets the player choose between an acceleration of "none", "low", "standard", and "high", which set your slow sensitivity to the player's chosen sensitivity and the fast sensitivity to an appropriate higher sensitivity. Your thresholds would be values you've chosen, and by having "standard" chosen by default, you can choose good default acceleration settings for the player without forcing them to mess with advanced settings if they want to change something.

Choosing "advanced", of course, exposes both sensitivities and both thresholds, which players who take their settings seriously will appreciate being able to fine-tune.

Smoothing

Sometimes gyro controls feel sloppy. They feel unresponsive and imprecise. It appears to me there are two main sources of this sloppiness:

1. Input latency (which we'll touch on later)
2. Smoothing

Smoothing here refers to using a rolling average of the user's input in order to stabilise the cursor or aimer, averaging out any shakiness or noise.

Here are two filters for player input. One doesn't really do anything. The other applies some simple smoothing:

Vec2 GetDirectInput(Vec2 input) {
   return input;
}
 
// smoothing buffer
Vec2[] InputBuffer;
int CurrentInputIndex;
 
Vec2 GetSmoothedInput(Vec2 input) {
   CurrentInputIndex = (CurrentInputIndex + 1) % InputBuffer.Length;
   InputBuffer[CurrentInputIndex] = input;
 
   Vec2 average = Vec2.Zero;
   foreach (Vec2 sample in InputBuffer) {
      average += sample;
   }
   average /= InputBuffer.Length;
 
   return average;
}

You can feel when a game does this excessively when you quickly move the controller back and forth, and the cursor or camera feels like it's making smaller, softer movements in response. It's a similar feeling to input lag, but it's not quite the same.

It feels bad. By spreading the consequences of a single input over multiple frames, the game takes longer to completely consume a given input, making it feel less responsive. By having a single simulation frame consider multiple frames of input, it necessarily dilutes the current frame's input. The game should be as responsive and quick as the player. It should be as agile as the player. It should be as shaky and imprecise as the player.

It feels bad with a mouse, and it feels equally bad with gyro controls. While smoothing is a useful way to stabilise input when it's small, it should never be applied to anything above very small inputs.

A game that has a smoothing filter — a rolling average — won't necessarily weigh each sample the same like this one does. It still shouldn't be done.

But with small inputs, this sloppiness is far less noticeable. It's also only with small inputs that smoothing may add any value — the player may be intending to keep the cursor or aimer still or nearly still, and some smoothing helps the player achieve that.

So here's what we do. We pick two thresholds. When the magnitude of the input is at or below the first threshold, we use the smoothing function in full. When it's at or above the second threshold, we have no smoothing at all. If it's in-between, apply some of the input through the smoothing filter and some of it directly, accordingly. Note that even when the input is large enough that we're not applying any smoothing at all, we still push zeroes into the smoothing function and use what comes out of it. I'll go into why the tiered smoothing function looks like this in another post, but here are the key reasons we do it like this:

1. No smoothing at all is applied to inputs greater than a small threshold, and
2. No matter what that threshold is, the smoothed and raw inputs are combined in such a way that the final displacement is as if no input was smoothed at all.

Vec2 GetTieredSmoothedInput(Vec2 input,
   float threshold1, float threshold2) {
 
   float inputMagnitude = Sqrt(input.Yaw * input.Yaw +
      input.Pitch * input.Pitch);
 
   float directWeight = (inputMagnitude - threshold1) /
      (threshold2 - threshold1);
   directWeight = Clamp(directWeight, 0.0, 1.0);
 
   return GetDirectInput(input * directWeight) +
      GetSmoothedInput(input * (1.0 - directWeight));
}

Now, we don't want to over-complicate the game's settings. We have three settings here for one very subtle effect: The first threshold, the second threshold, and the length of the input buffer (which determines how big the rolling average window is). So let's reduce this to one setting: the higher threshold.

For the player, this is the gyro speed (in degrees per second) at which smoothing is no longer applied. A suitable lower threshold can be calculated by the game from the higher threshold, and the length of the input buffer for smoothing can be chosen by the developer. This still leaves the player a lot of control over how much of an effect smoothing will have just by setting the higher threshold.

In JoyShockMapper, the higher threshold is set through the GYRO_SMOOTH_THRESHOLD setting, and it'll automatically choose a lower threshold of half of the GYRO_SMOOTH_THRESHOLD. I've found this to work really well. In JoyShockMapper the user also has control over the number of samples in the smoothing window. Setting GYRO_SMOOTH_TIME sets the smoothing window size in seconds, and then the required number of samples is calculated to match that time. For a real game, it might be best for the developer to choose a smoothing window big enough for most scenarios and just expose the higher smoothing threshold to the player in the game settings. However, I won't try and stop you from exposing all three variables to the player if you want to give the player as much power to customise their controls as possible.

I tend to not use it in 3D games because acceleration is enough for me to keep things steady, but JoyShockMapper's sample 2D mouse configuration has a smooth threshold of 5 degrees-per-second.

Minimum velocity

A common method for eliminating small wiggles and wanderings of the cursor is to eliminate any movement below a velocity threshold. At first blush this seems a very effective way of stabilising the cursor. Just like with a real mouse, which is kept steady by resting on the mousepad beneath it, the cursor stays still when the player isn't intentionally moving the controller.

But here's the deal. A minimum threshold will also keep the cursor still when the player is intentionally moving the controller. I've been extremely frustrated playing a game on Switch, trying to track a distant target, and with my sensitivity settings, the target's on-screen velocity was below the game's minimum velocity threshold. It was literally impossible to move the aimer at the same rate as my on-screen target. This was infuriating.

So I added a minimum velocity threshold to JoyShockMapper so I could experiment with it, cranking up the sensitivity so that the cursor was actually moving around when I held the controller "still". I picked a very low threshold at which all cursor movement stopped. I found that even tracking targets moving a little above the minimum velocity threshold, the inconsistency of my movements regularly dipped below the threshold, stopping all my movement until my input velocity bobbed above the threshold again. Even if this wasn't a problem, being unable to move the cursor below a certain velocity should be a deal-breaker if there's any possibility the player may reasonably desire to move the cursor at that rate.

So I tried a soft transition with two thresholds. Below the lower threshold, input is ignored. Above the second threshold, input is consumed as normal. But between the two thresholds, input is scaled from 0% at the lower threshold to 100% at the higher threshold. This makes it technically possible for the player to move the cursor at any velocity. In practice, however, it proved a very small scale over which to accelerate the input, and it was still virtually impossible to comfortably move the cursor as slow as I'd like to. I could mitigate this by increasing the second threshold, but I disliked having a wide range over which input scaling was less intuitive than normal.

The conclusion I came to as I explored the possibilities more and more is that a minimum velocity threshold is awful. Just terrible. Here's the simple version. Either:

• the threshold makes small or slow intentional inputs very difficult, or
• if you've fine-tuned it enough to entirely avoid that problem, it's because the threshold is so low that it might as well not be there.

So don't do it. Really, don't do it. And if you are really sure you want to do it, please make it optional, so I can disable it when playing your game.

But let's have another look at that two-thresholds solution JoyShockMapper has. If we set the first velocity threshold to zero — no cutoff — the second threshold is still useful for steadying the cursor when the controller is held relatively still. When the first threshold is zero or truly negligible, the second threshold can be kept very small and still be easy to use over that tiny range. It's effectively just another layer of input acceleration (as described earlier), with a zero multiplier at zero input, scaling up to a one multiplier at the threshold.

You can try it out in JoyShockMapper by setting GYRO_CUTOFF_RECOVERY to the velocity at which all input should be restored (in real world degrees per second), while leaving GYRO_CUTOFF_SPEED at its default value of 0. Like smoothing, I tend to not use it in 3D games because acceleration is enough for me, but JoyShockMapper's sample 2D mouse configuration has a GYRO_CUTOFF_RECOVERY of 5 degrees-per-second.

It's no longer a cutoff, but a threshold below which most inputs are squeezed towards zero, so let's call it "tightening". It's really simple:

GetTightenedInput(Vec2 input, float threshold) {
   float inputMagnitude = Sqrt(input.Yaw * input.Yaw +
      input.Pitch * input.Pitch);
   if (inputMagnitude < threshold) {
      float inputScale = inputMagnitude / threshold;
      return input * inputScale;
   }
 
   return input;
}

Lifting the mouse

A mouse translates real-world motion to a related motion in-game. This is really useful. But it has limitations. In the real world, I'm looking at a screen in a fixed position in front of me, moving a mouse across a mousepad that's probably less than 30cm from one side to the other. In the game I'm playing, I need to be able to look behind me at a moment's notice or move across an enormous scrolling map. Even if all I'm doing is moving a cursor across a static, screen-sized space, I may find myself needing to move the mouse further than there is room on the mousepad. Thankfully, I can lift up the mouse, it stops telling the game or application how it's moving, and I can put it in a more comfortable position where I have room to move it some more.

Imagine if the mouse was stuck to the mousepad. It could move across the mousepad just as easily, but there was no way to lift it off to reposition. Can you imagine how frustrating that would be? If I run out of space on the mousepad to the right there's very little that can be done to get more room to move. Perhaps moving the mouse left until it hits a virtual boundary and moving the physical mouse even further could get me more room on the right. If I have mouse acceleration, maybe strategically moving the mouse fast and slow until my cursor and physical mouse are both where I want them.

As a player, I'm going to crank up the sensitivity so that I have more in-game range of movement before hitting my real-world limits. Yes, it'll cost more precision than I'd like, but that range of movement is really important if I can't lift the mouse off the mousepad.

Of course, game developers would offer a better solution. If the mouse can't be temporarily ignored in hardware, do it in software. If the mouse has an extra button, make it so that the game ignores mouse movements while that button is pressed. Most people only have 2- or 3-button mice, but we can work around that with relatively little difficulty — in how many games or applications do you right-click and drag at the same time? By having gyro disabled when the right mouse button is pressed and the regular right-click action only occur when the mouse isn't dragged while clicking (or only when the right mouse button is tapped), developers can provide players a pretty good solution.

Now let's bring it back to gyro.

While the gyro doesn't need a physical surface to function, it's still limited to the comfortable turning range of the player. If there's no way to disable the gyro so the controller can be returned to a comfortable position, players will frequently encounter uncomfortable situations where they can't really turn any further even though they need to. In 3D games, the right thumbstick is controlling the camera, too, but taking over with the stick isn't good enough — the controller is still in an uncomfortable position, and moving it back to a comfortable one will undo much of the in-game turning that put the player in that position in the first place.

Frequently, players will crank up their sensitivity to something ridiculous so they're less likely to hit their physical limits during an encounter. But this, of course, costs the player in precision and amplifies noise and shakiness.

So I recommend three solutions, all of which you can try out using JoyShockMapper. While the first is best if you're able to do it, providing the third option as well would be ideal. They all boil down to letting the player choose a button or input to disable the gyro:

1. A dedicated button would be best, and let the player choose which button. A d-pad choice might seem easiest to add to your game without occupying more important buttons, but is also almost useless as the player would have a hard time moving at the same time. I'd usually recommend a face button or shoulder/trigger. In JoyShockMapper, the user can choose their button and decide whether it's a "gyro off" or "gyro on" button.
2. A shared button. Like the right-click example for mouse, if you have a button that is used infrequently and can work as a toggle (in 3D games, tap to change between crouching and standing would be an example), having the gyro disabled while that button is pressed can work really well. I frequently do this with JoyShockMapper when playing games that really don't have any buttons to spare. Make sure to disable gyro the moment the button is pressed, even if it might end up being just a tap — a brief interruption while tapping the button isn't a big deal, but a delay while holding the button is.
3. For 3D camera-turning games, have the option to disable gyro while the right stick is being used. While it's ideal to be able to use them both simultaneously, this makes the right stick far more useful for controlling the camera while repositioning the gyro.

All of these are immediately useful, but can take a little practice to become second nature. If you remember what it was like using a mouse for the first time, you know that's okay. And as more games add these options, a player experienced with them in one game can easily pick them up in another.

In this example, I have the Circle button as my gyro-off button. This is normally used for crouch in Overwatch, but toggling it by tapping the button can do the trick:

I've avoided using the stick to turn the camera here to focus on disabling the gyro. However, the right stick still works as you'd expect, and can be used in combo with gyro aiming. In Part 2 we'll look at an alternative use of the right stick that has some unique advantages, but for now we can take advantage of the fact that experienced players, at least, are already familiar with stick aiming.

Aim Assist

This one's really easy:

Latency

Like with a regular mouse, the feel of gyro controls is more sensitive to latency than button and key inputs. That's not to say they have more lag, but to say that lag is more noticeable with them. These aren't specific to gyro controls, but here are some helpful resources on input latency/lag in games:

• John Carmack wrote about some strategies for reducing input lag with VR in mind, but these apply to non-VR games, too.
• Akimitsu Hogge's GDC 2019 session Controller to Display Latency in 'Call of Duty' goes into great detail on throttling to reduce input lag (a practical and thorough presentation on Activision's implementation of what appears to me to be what Carmack is describing in the other article under "Late frame scheduling").

Of course, changes like these late in a project are far more difficult and far more likely to have unintended consequences than when a project is just starting. Non-developers reading this: I would generally not expect such changes to be patched into a game unless the team were making a concerted effort to deal with what was deemed a big latency issue.

Accessibility

One of the challenges of game design is choosing default settings. Sometimes there's a tension between controls that are easier for new players to learn and controls that will benefit experienced players the most in high level play. Usually, we'll favour the new player, because every experienced player starts off as an inexperienced player. Also, more experienced players will be more invested in the game and more happy to fine-tune their settings.

For games that really depend on a mouse, like real-time strategy games or dota-likes, gyro being enabled by default is a no-brainer.

But what about for genres already well-established on consoles that still benefit from gyro controls, like shooters and other action games? In my experience showing these controls to others, there's no tension here. Gyro controls are both easier for new players, and have more room for mastery. That is to say, whether you're brand new to this kind of game or a pro looking to eke the most out of your controls, the gyro is your best bet.

The only thing that makes this tricky is that thumbstick aiming has become second nature to those who've put in enough time with it. Mouse and gyro still require some learning, and the discomfort of playing with something new when you could play with what you already know can put some players off sticking to mouse or gyro aiming — perhaps even if they already play better with the mouse or gyro.

And that's okay! As long as they're enjoying the game! And here's the thing — some players will legitimately continue to be more comfortable thumbstick controls no matter how much effort they put into learning mouse or gyro controls. People come to gaming with a variety of different disadvantages of their own. Injuries and disabilities can prevent players from being able to enjoy certain controls regardless of how typical players would fare.

That's all to say: don't abandon thumbstick aiming entirely. Allow players the option. Shooters and the like on PC still tend to support controllers, even though the vast majority of players are better off with a mouse, and this is good! If and when gyro controls become the standard for playing these kinds of games on console, players should still have the option to play these games with just sticks.

Summary

If we bring it all together, it looks something like this.

// current velocity, uncalibrated
Vec3 Velocity;
 
// for calibration
int NumOffsetSamples;
Vec3 AccumulatedOffset;
bool Calibrating;
//...
 
// settings
bool GyroEnabled;
float Sensitivity;
float SmoothingThreshold;
float TighteningThreshold;
bool EnableAcceleration;
// only show the following when acceleration is enabled
float SlowSensitivity;
float FastSensitivity;
float SlowThreshold;
float FastThreshold;
//...
 
// Here's our main input processing function
ProcessInput(float deltaTime) {
   if (Calibrating) {
      NumOffsetSamples++;
      AccumulatedOffset += Velocity;
   }
 
   // the player can disable the gyro in the settings or
   // disable the gyro while using a chosen button or stick
   if (!GyroEnabled || GyroOffInputActive()) {
      return;
   }
 
   // we're using gyro as a mouse, only need 2D input
   Vec2 gyroVelocity = Velocity.XY;
   Vec2 gyroCalibration = GetCalibrationOffset().XY;
   Vec2 calibratedGyro = gyroVelocity - gyroCalibration;
 
   // smoothing
   if (SmoothingThreshold > 0) {
      calibratedGyro = GetTieredSmoothedInput(calibratedGyro,
         SmoothingThreshold / 2, SmoothingThreshold);
   }
 
   // tightening
   if (TighteningThreshold > 0) {
      calibratedGyro = GetTightenedInput(calibratedGyro,
         TighteningThreshold);
   }
 
   // acceleration
   if (EnableAcceleration) {
      ProcessGyroAcceleration(calibratedGyro, deltaTime,
         SlowSensitivity, FastSensitivity,
         SlowThreshold, FastThreshold);
   } else {
      ProcessGyroInput(calibratedGyro,
         deltaTime, Sensitivity);
   }
}

Our settings are simple, although acceleration has a lot of variables. As discussed in its section, this can be hidden behind an "advanced" or "custom" setting to avoid cluttering the settings or overwhelming the player. We do acceleration or regular gyro last — we treat smoothing and tightening as filters on the input, while the acceleration or core function converts that input into its in-game result.

These things work equally well with 2D and 3D games, except that 3D games are benefited by the natural sensitivity scale. The gyro is a mouse, and games should treat it as such. For years and years we've been using tiny joysticks for things that a mouse is much better at, and for at least as long as the PS4 has been around (since 2013!) we've essentially had a frictionless mouse built-in to the controller. Let's use it.

My video examples above were all from 3D camera-control games, but all these settings work equally well in 2D games. Here I'm playing Osu! with gyro controls using these settings from the Games Database:

I haven't had a lot of time with it, but Osu! is basically unplayable with thumbsticks. It's free, too, so give it a go!

Coming soon

We've looked at gyro controls simple enough to patch into an already published game on the PS4 or Switch. They can make games that are traditionally only on PC or touch devices viable on these platforms, too. Next, in parts 2 and 3, we'll look at taking things to the next level in 3D camera-controlling games.

Now, in Part 2 let's look at flick stick. Flick stick is a feature of JoyShockMapper specifically for 3D games that gives players an edge over using a traditional mouse by allowing players to turn to any direction quickly and easily.

In Part 3 we'll go beyond using the gyro as just a mouse. Why lock the aimer to the centre of the camera when we have a gyro and a right stick? This goes beyond what JoyShockMapper can do, since it can't be faked with just mouse inputs. I reckon these will be game-defining gyro controls.

Keep an eye out for part 3 when I finish writing it.