Performance: Faster or Fast?

TL;DR

When describing the performance of systems, always prefer terms which are unambigious.

• Use "as fast as" speed ratios and "as long as" time ratios.
• Always use "faster" or "slower" when referring to speed, never time.
• When referring to time in percentage delta terms, explicitly state "time increased/decreased by X%".
• Time ratio is time_new / time_baseline and speed ratio is time_baseline / time_new.

Performance Language & Ambiguity

As part of my work on the Buck2 build system, I deal with performance analyses on a daily basis. It's extremely important to use precise and consistent language when describing the performance of systems, so that there's no ambiguity in its intepretation.

For example, a statement like "X is 50% faster" is ambiguous, as it can be interpreted in two ways: the speed was increased by 50% or the time was decreased by 50% (those are not equivalent). As another example, a statement like "X is 150% faster" is also amgiguous because it can be interpreted as either 50% or 150% faster.

"Faster" vs "As Fast As"

The terms faster/slower refer to the delta of the measurement against a baseline. Faster indicates a positive delta (i.e., 20% faster means +20% or +0.2x), while slower indicates a negative delta (i.e., 20% slower means -20% or -0.2x). For example, 20% faster means the new value is equal to 120% of the baseline.

"As fast as" refers to the ratio of the measurement against a baseline. For example, "1.2x as fast as" means that it's "0.2x faster" (i.e., 20%). 1.2x can also be expressed as 120% in percentage terms. "As fast as" is also used when a measurement is lower as well, e.g., "0.8x as fast as" (i.e., "80% as fast as" or "20% slower").

Speed vs Time

Time and speed are inversely proportional, so when time decreases, the speed increases. Crucially, a percentage time delta does not translate to the same percentage speed delta. For example, it's incorrect to say that a 50% time reduction is the same as being 50% faster.

For example, if the time decreases by 50% (i.e., halved), that means the speed is twice as fast - i.e., "2x/200% as fast as" (or equivalently "100% faster"").

Calculating Time

Assume you have measured the baseline time (time_baseline) and the new time (time_new).

• The time_ratio is defined as time_new / time_baseline
• The time_ratio_delta is defined as time_ratio - 1.0.

For example, with a time_baseline of 100 seconds and time_new of 80 seconds, the time ratio as 0.8x (80/100), thus time ratio delta is -0.2x (0.8 - 1.0). We could present this as "0.8x as long as", "80% as long as" or "time decreased by 20%".

Calculating Speed

Assume you have measured the baseline time (time_baseline) and the new time (time_new).

• The speed_ratio is defined as time_baseline / time_new.
• The speed_ratio_delta is defined as speed_ratio - 1.0.

For example, with a time_baseline of 100 seconds and time_new of 80 seconds, the speed ratio is 1.25x (100/80), thus speed ratio delta is 0.25x (1.25 - 1.0). We could present this as "1.25x as fast as", "125% as fast as" or "25% faster".

Note that a 20% decrease in time (i.e., 20s out of 100s) led to a 25% increase in speed. This becomes clear when we show the derivation of speed_ratio which is the inverse of the time_ratio.

speed_baseline = distance / time_baseline
speed_new = distance / time_new

speed_ratio = speed_new / speed_baseline

speed_new / speed_baseline = (distance / time_new) / (distance / time_baseline)
                           = (distance / time_new) * (time_baseline / distance)
                           = time_baseline / time_new

← Back to Writings