Latency

About processing latency

Latency is the time it takes for the signal to enter the processor and exit through an output. Basically, it's the time it takes the device to perform analog-to-digital-to-analog (ADC -> DAC) conversion plus additional processing overhead.

Note that this is different from DAW latency, where the time is measured between playing your thing and hearing the sound through the system.

Fractal Audio's internal processing latency is very. It depends on the hardware, preset contents, and oversampling mode (Axe-Fx III only). The Axe-Fx III has the lowest latency. The latency of the FM3 and FM9 is the same.

Important: connecting other hardware can increase the overall latency. The perceived latency by the end-user is the sum of all components in the chain. Possible components include a digital wireless guitar system, which is typically around 2-4 ms, wireless headphones, digital mixers, DAWs, DSP-controlled monitors, additional digital effects devices and so on.

Using an I/O loop on the unit increases latency when using both the input and output ports.

Adding blocks to a preset doesn't add latency unless it's an Amp or Drive block, or a Cab block which links to an IR with leading silence at the beginning. Some other effects may increase latency as well, such as through-zero flanging, time-based and pitch shifting effects which run in series and are 100% wet, look ahead compression, and feedback send/return loops.

Fractal Audio explains latency in Latency of amp modelers.

FRACTAL AUDIO QUOTES

---

[1] Most blocks do not add latency. The only blocks that add latency are Drive and Amp. The reason being that they oversample and oversampling adds latency.

[2] The theoretical latency of the FM3/9 is:
In -> Out - 96 samples = 2ms
In -> Amp -> Out -160 samples = 3.33ms
In -> Amp -> Cab -> Out - 160 samples = 3.33ms (Cab block doesn't add any latency unless IR has leading silence)
in -> Drive -> Amp -> Cab -> Out - 192 samples = 4ms
None of the other blocks add latency.

[3] (Aex-Fx III) Empty preset is 1.3ms.
Just Amp is 2.3ms.
Amp + Cab is 2.3ms.
Drive + Amp + Cab is 2.9ms.

[4] Most blocks do not add latency. The only blocks that add latency are Drive and Amp. The reason being that they oversample and oversampling adds latency.

[5] (FM9) With just an amp the latency will be 3.3ms. Adding a cab may or may not increase latency, depends on whether the IR has any leading silence. Adding a drive will add 0.67ms.

So with Drive->Amp->Cab the best you can achieve is 4ms. Any leading silence in the IR will add to this figure.

We could add a "low-latency" mode to the FM products like in the Axe-Fx. This is what Line6 uses (and perhaps other companies). I'm not a fan of this approach because it causes phase distortion which affects the high-frequency harmonics when doing nonlinear processing.

As I've said before it's all a delicate balancing act. Engineering is about trade-offs. If you want the lowest latency possible then your frame size needs to be very small but then you'll either need prohibitively costly amounts of processing power or crappy algorithms that don't use much CPU.

You can also trade-off interpolation/decimation kernels. You can use a smaller kernel length but then your transition band and stop band performance suffers.

Or you can do what Line6 does and use minimum-phase kernels which distort the phase response.

I personally prefer linear-phase kernels with at least 100 dB of stop-band rejection.

If we were to use minimum-phase kernels the latency in this case would drop to about 2.7ms (if my math is right).

[...]

Less stop-band rejection means more aliasing artifacts.

Wider transition band means more rolloff in the high end.

You want your transition band as narrow as possible but the narrower the band the longer the kernel or the less stop-band rejection.

Here's a good paper on the subject: https://www.vyssotski.ch/BasicsOfInstrumentation/SpikeSorting/Design_of_FIR_Filters.pdf

[6] FWIW here are the results of the various modelers I have:
(...)
Axe-Fx III (Min Latency): 1.6 ms
Axe-Fx III (Best Quality): 2.2 ms
FM3: 3.3 ms
(...)
Latency testing was performed using a pulse into the DUT. Preset in DUT consisted of all possible locations in a single row populated and all but the amp block bypassed, i.e., In->Drive->Eq->Amp->Cab->Delay->Reverb->etc.->Out with all but Amp bypassed.

[7] Yes, another drive block will add another 32 samples of latency if it's in series.

If you are running two amps chances are they are in parallel so there won't be any added latency.

The only effects that add latency are the amp and drive blocks because they oversample. The amp blocks also add some additional latency beyond the latency due to oversampling because the processing is performed on a dedicated core.

[8] The latency for an empty preset in the Axe-Fx III would be 1.2 ms. Of that, less than 0.5 ms is converter latency. When Oversampling Mode is set to Best Quality adding an amp block adds another 1.0 ms. If you set Oversampling Mode to Min Latency the added latency drops to less than 0.4 ms. Cab blocks do not add any latency.

The higher the sample rate the more processing power required. Anything over 48 kHz starts to become wasteful, especially when processing IRs. If you were to double the sample rate (96 kHz) you would quadruple the processing power required for IRs since you would need twice as many samples in your IR and processed at twice the speed. You could always downsample, process the IR at 48 kHz, and then upsample but then your latency is no better (and probably worse) than processing at 48 kHz (unless you use min-phase resampling which destroys phase information).

Everything is a compromise in real-time DSP. I've been doing this since DSPs first came on the market. Latency isn't everything. It's one factor among many. The good designer weighs everything and makes tradeoffs to achieve a balance. If you overemphasize latency then you waste processing power or incur aliasing or distort the phase response, etc., etc. Anything under 5ms is considered undetectable. We strive for less than 3ms in our products so that you can add devices in the loops and not exceed that 5ms threshold.

[9] Most modeling products have latencies in the range of 1-2 ms which is roughly equivalent to 1-2 feet away from the speaker. Our products have less latency than most, if not all, competing products as that was a design goal from day one. The equation is ~1 ft per millisecond of latency.

[10] The A/D/A latency of the Axe-Fx III is 1.0ms. The A/D/A latency of the FM3 and FM9 is 1.33ms. Those figures I posted were for a preset including amp, cabinet etc. blocks. Amp blocks *usually* add latency due to the need to oversample. The Helix uses minimum-phase oversampling so its amp block doesn't add latency but that causes phase distortion. The Axe-Fx III Min Latency setting also uses minimum-phase oversampling.

A basic amp + cab preset in the FM3 should be 3.17ms. In the Axe-Fx III it should be 2.2ms. Cab blocks do NOT add latency. However the IR itself may have some leading silence or the peak in the time response might be delayed which would add latency.

[11] The FM3 and FM9 are/should be the same. (...) Both products use a dedicated core for amp processing and both products use the same frame and interpolation kernel sizes.

(FM9) [12] The latency is now around 2ms.

[13] The theoretical latency of the FM3/9 is:
In -> Out - 96 samples = 2ms
In -> Amp -> Out -160 samples = 3.33ms
In -> Amp -> Cab -> Out - 160 samples = 3.33ms (Cab block doesn't add any latency unless IR has leading silence)
in -> Drive -> Amp -> Cab -> Out - 192 samples = 4ms
None of the other blocks add latency.

[14] (Axe-Fx III) Latency with a blank preset is less than 2ms. Adding an amp block increases latency to about 2ms.

[15] FM3: a little over 2 ms. Axe-Fx III is even lower at around 1.33ms.

[16] (Axe-Fx III) Empty preset is 1.3ms. Just Amp is 2.3ms. Amp + Cab is 2.3ms. Drive + Amp + Cab is 2.9ms.

[17] (Axe-Fx III) The converters are the limiting factor. It's already way faster than anything else.

[18] The data is processed in blocks even in an empty preset. That's how you do this sort of thing. You buffer up N samples and then pass that buffer to the processing routine.

The latency for an empty preset in the Axe-Fx III would be 1.2 ms. Of that, less than 0.5 ms is converter latency. When Oversampling Mode is set to Best Quality adding an amp block adds another 1.0 ms. If you set Oversampling Mode to Min Latency the added latency drops to less than 0.4 ms. Cab blocks do not add any latency.

The higher the sample rate the more processing power required. Anything over 48 kHz starts to become wasteful, especially when processing IRs. If you were to double the sample rate (96 kHz) you would quadruple the processing power required for IRs since you would need twice as many samples in your IR and processed at twice the speed. You could always downsample, process the IR at 48 kHz, and then upsample but then your latency is no better (and probably worse) than processing at 48 kHz (unless you use min-phase resampling which destroys phase information).

Everything is a compromise in real-time DSP. I've been doing this since DSPs first came on the market. Latency isn't everything. It's one factor among many. The good designer weighs everything and makes tradeoffs to achieve a balance. If you overemphasize latency then you waste processing power or incur aliasing or distort the phase response, etc., etc. Anything under 5ms is considered undetectable. We strive for less than 3ms in our products so that you can add devices in the loops and not exceed that 5ms threshold.

The Oversampling Mode parameter, for the Axe-Fx III only, lets you choose between Best Quality and Minimum Latency. This applies to Amp blocks and Drive blocks only (non-linear blocks). Best Quality is equal to previous firmware, and is linear-phase oversampling, Minimum Latency is minimum-phase oversampling.

FRACTAL AUDIO QUOTES

---

[19] It selects between two different interpolation/decimation kernels. Best Quality yields the least aliasing and best phase response at the expense of latency. This is the kernel used in prior firmwares. Min. Latency trades aliasing and phase response for reduced latency. The Amp and Drive block have about 0.5ms less latency each when using Min. Latency. You can achieve total latency of around 1.5ms or less when using Min. Latency.

[20] FWIW here are the results of the various modelers I have:
Axe-Fx III (Min Latency): 1.6 ms
Axe-Fx III (Best Quality): 2.2 ms

Latency testing was performed using a pulse into the DUT. Preset in DUT consisted of all possible locations in a single row populated and all but the amp block bypassed, i.e., In->Drive->Eq->Amp->Cab->Delay->Reverb->etc.->Out with all but Amp bypassed.

[21] The Axe-Fx III has two oversampling modes: Best Quality and Min Latency. One is not "simpler" than the other. The CPU usage is the same for either mode. Min. Latency uses a minimum-phase kernel which minimizes the latency of the block at the expense of phase distortion. The FM3 and FM9 use Best Quality mode all the time. There simply isn't enough CPU available to run two amp blocks on the FM3.

[22] There is no "Quality" setting. There is an Oversampling Mode which selects between a linear phase and a minimum phase kernel. CPU usage is the same in either case.

[23] With just an amp the latency will be 3.3ms. Adding a cab may or may not increase latency, depends on whether the IR has any leading silence. Adding a drive will add 0.67ms.

So with Drive->Amp->Cab the best you can achieve is 4ms. Any leading silence in the IR will add to this figure.

We could add a "low-latency" mode to the FM products like in the Axe-Fx. This is what Line6 uses (and perhaps other companies). I'm not a fan of this approach because it causes phase distortion which affects the high-frequency harmonics when doing nonlinear processing.

As I've said before it's all a delicate balancing act. Engineering is about trade-offs. If you want the lowest latency possible then your frame size needs to be very small but then you'll either need prohibitively costly amounts of processing power or crappy algorithms that don't use much CPU.

You can also trade-off interpolation/decimation kernels. You can use a smaller kernel length but then your transition band and stop band performance suffers.

Or you can do what Line6 does and use minimum-phase kernels which distort the phase response.

I personally prefer linear-phase kernels with at least 100 dB of stop-band rejection.

If we were to use minimum-phase kernels the latency in this case would drop to about 2.7ms (if my math is right).

[...]

Less stop-band rejection means more aliasing artifacts.

Wider transition band means more rolloff in the high end.

You want your transition band as narrow as possible but the narrower the band the longer the kernel or the less stop-band rejection.

Here's a good paper on the subject: https://www.vyssotski.ch/BasicsOfInstrumentation/SpikeSorting/Design_of_FIR_Filters.pdf

[24] Yes, another drive block will add another 32 samples of latency if it's in series.

If you are running two amps chances are they are in parallel so there won't be any added latency.

The only effects that add latency are the amp and drive blocks because they oversample. The amp blocks also add some additional latency beyond the latency due to oversampling because the processing is performed on a dedicated core.

[25] Depends on the oversampling filter kernel. A linear phase filter will have latency (more correctly: group delay). A minimum phase filter will have almost no latency.

FRACTAL AUDIO QUOTES

---

[26] The latency is fixed and adding or removing effects does not change it.

[27] Most blocks do not add latency. The only blocks that add latency are Drive and Amp. The reason being that they oversample and oversampling adds latency.

[28] (Drive blocks) Depends on the oversampling filter kernel. A linear phase filter will have latency (more correctly: group delay). A minimum phase filter will have almost no latency.

FRACTAL AUDIO QUOTES

---

[29] The latency for an empty preset in the Axe-Fx III would be 1.2 ms. Of that, less than 0.5 ms is converter latency. When Oversampling Mode is set to Best Quality adding an amp block adds another 1.0 ms. If you set Oversampling Mode to Min Latency the added latency drops to less than 0.4 ms. Cab blocks do not add any latency.

The higher the sample rate the more processing power required. Anything over 48 kHz starts to become wasteful, especially when processing IRs. If you were to double the sample rate (96 kHz) you would quadruple the processing power required for IRs since you would need twice as many samples in your IR and processed at twice the speed. You could always downsample, process the IR at 48 kHz, and then upsample but then your latency is no better (and probably worse) than processing at 48 kHz (unless you use min-phase resampling which destroys phase information).

Everything is a compromise in real-time DSP. I've been doing this since DSPs first came on the market. Latency isn't everything. It's one factor among many. The good designer weighs everything and makes tradeoffs to achieve a balance. If you overemphasize latency then you waste processing power or incur aliasing or distort the phase response, etc., etc. Anything under 5ms is considered undetectable. We strive for less than 3ms in our products so that you can add devices in the loops and not exceed that 5ms threshold.

[30] Most modeling products have latencies in the range of 1-2 ms which is roughly equivalent to 1-2 feet away from the speaker. Our products have less latency than most, if not all, competing products as that was a design goal from day one. The equation is ~1 ft per millisecond of latency.

[31] The A/D/A latency of the Axe-Fx III is 1.0ms. The A/D/A latency of the FM3 and FM9 is 1.33ms. Those figures I posted were for a preset including amp, cabinet etc. blocks. Amp blocks *usually* add latency due to the need to oversample. The Helix uses minimum-phase oversampling so its amp block doesn't add latency but that causes phase distortion. The Axe-Fx III Min Latency setting also uses minimum-phase oversampling.

A basic amp + cab preset in the FM3 should be 3.17ms. In the Axe-Fx III it should be 2.2ms. Cab blocks do NOT add latency. However the IR itself may have some leading silence or the peak in the time response might be delayed which would add latency.

[32] The FM3 and FM9 are the same. (...) Both products use a dedicated core for amp processing and both products use the same frame and interpolation kernel sizes.

[33] The theoretical latency of the FM3/9 is:
In -> Out - 96 samples = 2ms
In -> Amp -> Out -160 samples = 3.33ms
In -> Amp -> Cab -> Out - 160 samples = 3.33ms (Cab block doesn't add any latency unless IR has leading silence)
in -> Drive -> Amp -> Cab -> Out - 192 samples = 4ms
None of the other blocks add latency.

Axe-Fx III:

[34] Latency with a blank preset is less than 2ms. Adding an amp block increases latency to about 2ms.

[35] Axe-Fx III is even lower at around 1.33ms.

[36] Empty preset is 1.3ms. Just Amp is 2.3ms. Amp + Cab is 2.3ms. Drive + Amp + Cab is 2.9ms.

[37] The converters are the limiting factor. It's already way faster than anything else.

The data is processed in blocks even in an empty preset. That's how you do this sort of thing. You buffer up N samples and then pass that buffer to the processing routine.

[38] The latency for an empty preset in the Axe-Fx III would be 1.2 ms. Of that, less than 0.5 ms is converter latency. When Oversampling Mode is set to Best Quality adding an amp block adds another 1.0 ms. If you set Oversampling Mode to Min Latency the added latency drops to less than 0.4 ms. Cab blocks do not add any latency.

The Oversampling Mode parameter, for the Axe-Fx III only, lets you choose between Best Quality and Minimum Latency. This applies to Amp blocks and Drive blocks only (non-linear blocks). Best Quality is equal to previous firmware, and is linear-phase oversampling, Minimum Latency is minimum-phase oversampling.

[39] It selects between two different interpolation/decimation kernels. Best Quality yields the least aliasing and best phase response at the expense of latency. This is the kernel used in prior firmwares. Min. Latency trades aliasing and phase response for reduced latency. The Amp and Drive block have about 0.5ms less latency each when using Min. Latency. You can achieve total latency of around 1.5ms or less when using Min. Latency.

[40] FWIW here are the results of the various modelers I have:
Axe-Fx III (Min Latency): 1.6 ms
Axe-Fx III (Best Quality): 2.2 ms

Latency testing was performed using a pulse into the DUT. Preset in DUT consisted of all possible locations in a single row populated and all but the amp block bypassed, i.e., In->Drive->Eq->Amp->Cab->Delay->Reverb->etc.->Out with all but Amp bypassed.

[41]

The Axe-Fx III has two oversampling modes: Best Quality and Min Latency. One is not "simpler" than the other. The CPU usage is the same for either mode. Min. Latency uses a minimum-phase kernel which minimizes the latency of the block at the expense of phase distortion. The FM3 and FM9 use Best Quality mode all the time. There simply isn't enough CPU available to run two amp blocks on the FM3.

[42] There is no "Quality" setting. There is an Oversampling Mode which selects between a linear phase and a minimum phase kernel. CPU usage is the same in either case.

FM9:

[43] The latency is now around 2ms.

[44] Most blocks do not add latency. The only blocks that add latency are Drive and Amp. The reason being that they oversample and oversampling adds latency.

[45] With just an amp the latency will be 3.3ms. Adding a cab may or may not increase latency, depends on whether the IR has any leading silence. Adding a drive will add 0.67ms.

So with Drive->Amp->Cab the best you can achieve is 4ms. Any leading silence in the IR will add to this figure.

We could add a "low-latency" mode to the FM products like in the Axe-Fx. This is what Line6 uses (and perhaps other companies). I'm not a fan of this approach because it causes phase distortion which affects the high-frequency harmonics when doing nonlinear processing.

As I've said before it's all a delicate balancing act. Engineering is about trade-offs. If you want the lowest latency possible then your frame size needs to be very small but then you'll either need prohibitively costly amounts of processing power or crappy algorithms that don't use much CPU.

You can also trade-off interpolation/decimation kernels. You can use a smaller kernel length but then your transition band and stop band performance suffers.

Or you can do what Line6 does and use minimum-phase kernels which distort the phase response.

I personally prefer linear-phase kernels with at least 100 dB of stop-band rejection.

If we were to use minimum-phase kernels the latency in this case would drop to about 2.7ms (if my math is right).

[...]

Less stop-band rejection means more aliasing artifacts.

Wider transition band means more rolloff in the high end.

You want your transition band as narrow as possible but the narrower the band the longer the kernel or the less stop-band rejection.

Here's a good paper on the subject: https://www.vyssotski.ch/BasicsOfInstrumentation/SpikeSorting/Design_of_FIR_Filters.pdf

FM3:

[46] FM3: 3.3 ms. Latency testing was performed using a pulse into the DUT. Preset in DUT consisted of all possible locations in a single row populated and all but the amp block bypassed, i.e., In->Drive->Eq->Amp->Cab->Delay->Reverb->etc.->Out with all but Amp bypassed.

[47] Yes, another drive block will add another 32 samples of latency if it's in series.

If you are running two amps chances are they are in parallel so there won't be any added latency.

The only effects that add latency are the amp and drive blocks because they oversample. The amp blocks also add some additional latency beyond the latency due to oversampling because the processing is performed on a dedicated core.

[48] It depends on the flanger type. If it's a thru-zero flanger it necessarily adds latency because you have to delay wet and dry for that to work. Otherwise it does not.