Audio Effect Chain Estimation and Dry Signal Recovery from Multi-Effect-Applied Musical Signals

Osamu Take1, Kento Watanabe2, Takayuki Nakatsuka2, Tian Cheng2, Tomoyasu Nakano2, Masataka Goto2, Shinnosuke Takamichi1,3, Hiroshi Saruwatari1,
1University of Tokyo
2National Institute of Advanced Industrial Science and Technology (AIST)
3Keio University
Accepted to DAFx24

In this supplementary material for our DAFx24 submission, we present some sound examples to help you understand our introduced task, "audio effect chain (AFX chain) estimation and dry signal recovery" and our proposed method to solve it.

Contents:

  • Results for four wet signal examples
    • Results of the estimated AFX chain estimation and the recovered dry signal
    • Bypassed signals in the intermediate process of our method (multi-AFX chain only)
    • Reproduced wet signals
  • Application example for sound design

AFX Chain Estimation and Dry Signal Recovery Task

In the common sound-design process, sound engineers design a (target) AFX chain and apply each AFX to the source (dry) signal with the designed order, from left to right as depicted in the upper figure. The lower figure illustrates our task of AFX chain estimation and dry signal recovery, taking the multi-AFX-applied (wet) signal as the input to estimate the applied AFX chain and recover the dry signal.

Our proposed method

Our proposed method, illustrated in the left figure, applies the ``inverse process'' of AFX-chain processing to the existing musical (wet) signal. It repeatedly applies the single DNN model (SunAFXiNet, detailed in the right figure) to construct the estimated AFX chain from right to left.

Result 1: Target AFX Chain = (Delay)

AFX Chain Estimation & Dry Signal Recovery

Dry signal AFX chain Wet signal (used as input)
Ground truth
Recovered dry signal Estimated AFX chain
Output of our method
(SunAFXiNet=Proposed)

Wet Signal Reproduction (by Using the Estimated AFX Chain)

Dry signal AFX chain Wet signal (used as input)
Ground truth
Recovered dry signal Estimated AFX chain Reproduced wet signal
Reproduce from
the recovered dry signal
Ground-truth dry signal Reproduced wet signal
Reproduce from
the ground-truth dry signal

Result 2: Target AFX Chain = (Reverb)

AFX Chain Estimation & Dry Signal Recovery

Dry signal AFX chain Wet signal (used as input)
Ground truth
Recovered dry signal Estimated AFX chain
Output of our method
(SunAFXiNet=Proposed)

Wet Signal Reproduction (by Using the Estimated AFX Chain)

Dry signal AFX chain Wet signal (used as input)
Ground truth
Recovered dry signal Estimated AFX chain Reproduced wet signal
Reproduce from
the recovered dry signal
Ground-truth dry signal Reproduced wet signal
Reproduce from
the ground-truth dry signal

Result 3: Target AFX Chain = (Reverb, Delay, Chorus)

AFX Chain Estimation & Dry Signal Recovery

Dry signal AFX chain Wet signal (used as input)
Ground truth
Recovered dry signal Estimated AFX chain
Output of our method
(SunAFXiNet=Proposed)

Bypassed Signals in the Intermediate Inversion Process

Final dry signal
(After 3 SunAFXiNet iterations)		 After 2 iterations
(Chorus, Delay bypassed)		 After 1 iteration
(Chorus bypassed)		 Wet signal (used as input)

Wet Signal Reproduction (by Using the Estimated AFX Chain)

Dry signal AFX chain Wet signal (used as input)
Ground truth
Recovered dry signal Estimated AFX chain Reproduced wet signal
Reproduce from
the recovered dry signal
Ground-truth dry signal Reproduced wet signal
Reproduce from
the ground-truth dry signal

Result 4: Target AFX Chain = (Chorus, Distortion, Reverb)

AFX Chain Estimation & Dry Signal Recovery

Dry signal AFX chain Wet signal
(Input)
Ground truth
Recovered dry signal Estimated AFX chain
Output of our method
(SunAFXiNet=Proposed)

Bypassed Signals in the Intermediate Inversion Process

Final dry signal
(After 2 SunAFXiNet iterations)		 After 1 iteration
(Reverb bypassed)		 Wet signal (used as input)

Wet Signal Reproduction (by Using the Estimated AFX Chain)

Dry signal AFX chain Wet signal (used as input)
Ground truth
Recovered dry signal Estimated AFX chain Reproduced wet signal
Reproduce from
the recovered dry signal
Ground-truth dry signal Reproduced wet signal
Reproduce from
the ground-truth dry signal

Application Example for Sound Design

Engineers can extract the AFX chain and the unprocessed dry signal from a wet signal, manipulate the estimated AFX chain, and then reapply the modified AFX chain to the recovered dry signal. This process results in a musical signal with altered sonic characteristics.

Sound Examples

Input wet signal Recovered dry signal Reproduced wet signal Re-designed signal