I also noticed a lot of noise on the speaker, and I can tell that a large part of it comes from the SPI signal going to the display, from the PWM on the display backlight, from the signal on the UART from uploading, and pretty much from every single thing the module does. The reason for this is probably that the audio lines are not designed and shielded properly, and all sort of electrical activity in the module affects the audio signal.
I noticed that moving the speaker leads under the PCB affects the intensity of the noise in different situations (probably depends on which part it is closer to).
This is exceptionally poor design, I'm afraid.