EasyEffects Spatial Audio: Matrix Upmixer + HRTF Virtualizer

[Kevinscki]

Feature Request: Matrix Upmixer + Virtual Surround (HRIR) Plugins

Summary

Two complementary effects that together let users get convincing surround from stereo sources, and proper headphone spatialization from 5.1/7.1 sources, entirely within EasyEffects, without a separate virtual-device manager.

Motivation

• Most music/stereo content has no surround information, but users with 5.1/7.1 speaker setups want it used.
• Users with headphones watching 5.1/7.1 movies currently need external tools (e.g. the de.berny23.virtual_surround_manager Flatpak, which creates a virtual sink for this) plus manual filter-chain setup.
• Both are common enough that an integrated, per-preset solution is valuable.

Plugin 1: Matrix Upmixer (2.0 → 5.1 / 7.1)

Goal: Derive surround channels from stereo, in the spirit of Dolby Pro Logic II, with better steering and fewer artifacts.

Suggested approach:

• Compute mid (L+R) and side (L−R) signals.
• Center = correlated content (mid), optionally high-passed/low-passed so dialog stays clean.
• Surrounds = decorrelated side content, delayed (~10-20 ms) and low-passed (~7 kHz) to avoid obvious localization, as PLII does.
• Improvement over classic PLII: use frequency-domain (STFT) processing with per-band correlation/panning analysis. Classic PLII is broadband and prone to "pumping" and steering errors; a per-band approach localizes sounds more accurately.
• LFE: low-pass (~120 Hz) of the mono sum, with adjustable crossover.
• 7.1: split surrounds into side and rear pairs using progressively more decorrelation/delay.

Parameters:

• Output layout (5.1 / 7.1)
• Center level, surround level, LFE level, LFE crossover
• Surround delay
• Steering aggressiveness / "dimension" (front-back balance)
• Optional "music" vs "movie" mode

Plugin 2: Virtual Surround (HRIR Binaural Renderer)

Goal: Take 5.1/7.1 input and render it to 2-channel binaural output using HRIRs, so headphone users hear a spatial soundstage.

Behavior:

• Loads an HRIR set (SOFA and/or HeSuVi-style WAV).
• Convolves each input channel with the left/right-ear HRIRs for its virtual speaker position, then sums to L/R.
• If the input is already 5.1/7.1, use it directly. If it's stereo, the Matrix Upmixer runs first (this is where the two plugins chain nicely).
• Pass-through when the source is 2.0 and the user doesn't want upmixing.

Parameters (keep small, as requested):

• HRIR file selector
• Speaker angle (front L/R, surround, rear)
• Stereo width
• Room/reverb amount (if the HRIR set supports it)
• Master gain / LFE level

Virtual device integration

The Virtual Surround Manager Flatpak works by creating a virtual multichannel sink so applications (like a video player) see a 5.1/7.1 output device. EasyEffects already uses PipeWire and already presents virtual sinks, so the natural framing is:

• Expose a 5.1/7.1 virtual output in EasyEffects so apps can send surround content to it.
• Apply the Virtual Surround effect on that stream and output to the real headphone sink.

Open question to check early: whether EasyEffects' current pipeline supports more than 2 channels end-to-end, since this determines how much work the virtual sink part is.

Implementation notes

• The upmixer is straightforward DSP (a new LV2/native plugin or a GStreamer/PipeWire-filter element depending on the current architecture).
• The HRIR renderer is essentially partitioned convolution. EasyEffects already has a Convolver plugin, so a first version could reuse that engine with multichannel routing.
• Existing prior art to study: HeSuVi (Windows), PipeWire's filter-chain surround examples, and the sofalizer filter in FFmpeg.

Suggested phasing

1. Virtual Surround plugin using existing convolver code.
2. Multichannel virtual sink support.
3. Matrix Upmixer plugin.
4. Add 7.1 and per-band steering improvements.

[wwmm]

whether EasyEffects' current pipeline supports more than 2 channels end-to-end, since this determines how much work the virtual sink part is.

No. It doesn't. And some of the plugins we use do not even make sense outside of a stereo stream. So the application would have to block their usage in a surround pipeline.

[Kevinscki]

Ooh I see.., Thanks