Posted by JibbSmart on 22 May 2019 15:36

There are a variety of reasons one might smooth out player input. With JoyShockMapper, flick-stick and gyro are both inputs that may benefit from some smoothing:

• Flick-stick, because the DualShock 4's analog stick input is too low-resolution for players to be able to fine-tune the angle they're facing. While flick-stick isn't widely used in games yet, this is equally true of top-down twin-stick shooters, where the coarseness of the stick input is amplified when trying to aim at targets far away from the player.
• Gyro, because players have shaky hands. The gyro signal is a little noisy, but the noise is almost nothing compared to the shakiness of the player's hands. If you deal with player hand shakiness, you've dealt with the noise as well.

For both of these inputs, we value directness and responsiveness. The player is able to indicate their intent in an instant, and the game should respond accordingly. Games will frequently smooth out inputs like these. That means rather than using the last received input directly, they'll use the average of the last, say, 16 received inputs. This necessarily reduces responsiveness. While the game still begins responding immediately to player input, it hasn't finished dealing with that input until a number of samples after it has finished (16, in this example). This makes the game feel sloppy and imprecise.

If we do no smoothing and take reasonable measures to minimise input lag, the game will feel much more responsive. However, for a number of reasons, small adjustments by the player can be overcome by unwanted artifacts (whether it's a real, unintended input like player hand shakiness, or a hardware limitation like noticeable input aliasing due to low resolution stick input). Smoothing is a simple way to reduce the effects of these artifacts.

So naturally, we want small inputs to be smoothed while bigger inputs are not smoothed.

Let's start with our unsmoothed code:

GetDirectInput(Vec2 input) {
   return input;
}

Simple, right? Our naive smoothing code could look something like this:

Vec2 inputBuffer[16];
int currentInputIndex;
 
GetSmoothedInput(Vec2 input) {
    currentInputIndex = (currentInputIndex + 1) % 16;
    inputBuffer[currentInputIndex] = input;
 
    Vec2 average = Vec2.Zero;
    for (i in 0..16) {
        average += inputBuffer[i];
    }
    average /= 16;
 
    return average;
}

A simple threshold

But we want smoothing to apply only to small inputs. Let's introduce a threshold and see if we encounter any problems:

GetThresholdSmoothedInput(Vec2 input, float threshold) {
    // this will be length(input) for vectors
    float inputMagnitude = Abs(input);
 
    if (inputMagnitude < threshold) {
        return GetSmoothedInput(input);
    } else {
        return GetDirectInput(input) +
            GetSmoothedInput(Vec2.Zero);
    }
}

Notice that we still call GetSmoothedInput when we no longer want smoothing. This is because there may be input that hasn't been completely consumed (sampled 16 times and then discarded). If we don't do this and move from input below the threshold to input greater than or equal to the threshold, some of the below-threshold inputs sit in the smoothing inputBuffer until input falls below the threshold again. We still actually want to finish processing a player's input within 16 samples, but the current frame's input is already accounted for by GetDirectInput, so we just pass 0 to the smoothing function.

Now, as you might've guessed this is (slightly) too simple a solution. We need to consider what happens when input is near the threshold.

Consider, for example, an input that oscillates between just above and just below the threshold. Let's say that this isn't an unlikely input, either — if input doesn't vary relatively quickly, smoothing isn't going to do much, and if we expect the input to sometimes be much larger than the threshold and sometimes smaller, we might also expect it to sometimes be approximately at the threshold. So, the input stream is about 50% of the time just above the threshold and 50% of the time just below. What happens?

The inputs below the threshold averaged with the zeroes when input is above the threshold give us an output of about half the threshold value. But when the input is above the threshold, we get the full input plus the smoothed part, which will be about 1.5 times the threshold value. So, when we're close to the threshold value, small noise can explode into much bigger noise. This is no good!

Soft tiered smoothing

Another solution I explored was that when the input is greater than or equal to the threshold, the threshold is subtracted from the input and put in the smoothing function. The leftover input is handled by GetDirectInput and consumed in full immediately.

This is pretty good. We get smoothing below the threshold, more responsive input above the threshold, and no surprises around the threshold. But while the smoothed component becomes almost negligible with large inputs, it still adds a touch of unnecessary softness to big, intentional movements. So my preferred solution gradually contributes less to the smoothing component as we move further above the threshold, to the point where, at a second threshold, all input is consumed by GetDirectInput.

In code, this is even simpler than it sounds. We map the magnitude of the input to a directWeight, where if the input magnitude is greater than or equal to the larger threshold, directWeight is 1.0. If it's less than or equal to the smaller threshold, directWeight is 0.0. In between we move linearly from 0.0 to 1.0. Then directWeight is used to split the direct input from the smoothed input:

GetSoftTieredSmoothedInput(Vec2 input, float threshold1, float threshold2) {
    // this will be length(input) for vectors
    float inputMagnitude = Abs(input);
 
    float directWeight = (inputMagnitude - threshold1) / (threshold2 - threshold1);
    directWeight = clamp(directWeight, 0, 1);
 
    return GetDirectInput(input * directWeight) +
        GetSmoothedInput(input * (1.0 - directWeight));
}

You may have noticed that as threshold2 approaches threshold1, we effectively have our first GetTieredSmoothedInput (after we've accounted for the divide by zero). So, some discernment is required when choosing thresholds.

I've been happy with having threshold2 be automatically set to double threshold1. If threshold1 is large enough to be effectively smoothing the kinds of input we're expecting, threshold2 should provide a big enough buffer to smooth out any noisy inputs that warrant smoothing.

In practice, I'm usually more interested in threshold2 than threshold1. I want to be setting the threshold at which there is no more smoothing, so I usually actually only directly set threshold2, and have threshold 1 automatically set to half of that. This is how the smoothing in JoyShockMapper works (which the user can set using GYRO_SMOOTH_THRESHOLD).

What about uniform matching?

A common method for smoothing the transition between using two different functions depending on the input is the uniform method. It's similar in concept to the method above, but instead of splitting the input between two functions, we give the same input to both functions and interpolate between the outputs.

Why not do that here? It's slightly simpler and works nicely regardless of what functions we're transitioning between. However, when smoothing a movement, we want the same input to get us to the same destination, regardless of smoothing settings. The simple rolling average smoothing ultimately has the same integral (displacement) as the raw input, but the smoothed input takes a little longer to get there. While the smoothing function might not be done with a given input, the uniform method may begin ignoring the smoothing function's output when the current input is big enough. This means we lose some displacement. This means less consistency and precision for the player.

Another problem is that the smoothing buffer will be loaded with big values when the input is large. We won't see the effects of that until input stops or goes below the top threshold. Once that happens, the smoothed out big input will continue to come out of the smoothing function until the smoothing window has passed.

My soft-tiered method doesn't have either problem. Even when the current sample's input is large enough that it's all being consumed immediately, leftover smoothed input still gets added on top until it's all consumed, and nothing else is added to the smoothing buffer. Apart from rounding errors, we'll get the same final displacement regardless of our smoothing settings.

Multiple tiers

Of course, this is just a specific kind of multi-tiered smoothing. Our example has one tier with a buffer size of 16 (that is, it averages out 16 samples) and another tier with a buffer size of 1.

If we really wanted to, there's nothing stopping us from having more than two tiers, each with their own starting threshold. Each frame, find the two tiers that the current input should contribute to, divide it between the two tiers appropriately, and add zero input to all the other tiers.

For my purposes, though, where smoothing is only useful for a very small range of input (if that — in my own configurations I often have no smoothing on gyro input at all), the specialised two-tiered smoothing above does the trick just fine.

Acknowledging the detrimental effects of smoothing and having it only apply when its pros outweigh its cons goes a long way to improving the user experience. For both gyro input and flick stick, JoyShockMapper users are able to enjoy smooth and yet responsive interactions in games.