AddBackgroundNoise: RMS normalization before or after repating the noise?

[raraz15]

audiomentations/audiomentations/augmentations/add_background_noise.py

Line 139 in f3a2321

noise_rms = calculate_rms(noise_sound)

In AddBackgroundNoise, if the noise file is shorter than the music, it is repeated until equal length. However, the RMS energy of the noise signal is computed on the original file. I think that computing the RMS energy on the fully processed signal is more appropriate. What do you think about this?

Solution would be moving lines 165-171 after the line 138.

[iver56]

So what you are proposing is to measure the RMS of the repeated noise (which you refer to as the fully processed signal, right?) instead of the original noise. If it gets repeated 1.5x, the first half of the audio would be taken into account twice, and the last half would be taken into account once. That would result in a slightly different noise RMS, depending on the nature of the noise (the loudness over time). This again leads to a slightly different noise gain. It could be slightly higher or slightly lower (depending on the noise). My hunch here is that in the grand scheme of things you are going to want a range of different noise gain values anyway, so this variability is not necessarily a bad thing in the context of data augmentation. If I calculate the RMS of the repeated/sliced noise, it's going to be a bit slower (because it'll process a longer slice of audio), or I can keep it fast if I break up the sound in chunks, which again increases code complexity.

One alternative solution I think would be even better here is to change how the repeating of the noise works. Look at this part of the code:

        if len(noise_sound) > num_samples:
            noise_sound = noise_sound[0:num_samples]

If the repeated noise is longer than the input audio (in your case the music), this code always chops off a part of the end of the noise. However, if we instead use a random offset when slicing it, we get more variation due to the random shift. I think this is a good thing, because variation in data augmentation is generally desired. And the other nice thing with this random offset is that the calculated noise RMS (calculated before the repeating/slicing) will then match the actual noise RMS of the repeated/sliced noise on average, assuming you do it many times, like in a model training loop.

[raraz15]

print(audio.shape)

audio_rms = np.sqrt(np.mean(np.square(audio), axis=-1))
print(audio_rms)

audio = np.concatenate((audio, audio), axis=1)
print(audio.shape)
audio_rms2 = np.sqrt(np.mean(np.square(audio), axis=-1))
print(audio_rms2)

audio_rms3 = np.sqrt(np.mean(np.square(audio[:, :190000]), axis=-1))
print(audio_rms3)

audio_rms4 = np.sqrt(np.mean(np.square(audio[:, :220000]), axis=-1))
print(audio_rms4)

prints

(1, 160000)
[0.06772368]
(1, 320000)
[0.06772368]
[0.0675229]
[0.0671485]

I agree that output variability is desirable for data augmentation, but precision can be needed too. Also, the numeric difference is very small for this music file that I used in the code above. I wanted to raise this issue for awareness. I think that in my copy of the code, I will calculate the RMS just before mixing, as array operations shouldn't slow down the computation too much(?).

[iver56]

In your case, there is a ~0.074 dB difference between 0.0671485 and 0.06772368. For comparison, a 3 dB difference is barely perceptible by human ears.

Yes, copying the code and adapting it to your exact needs is legit. One of the strengths of open source. Thanks for your suggestion, and happy computing 🤓