I know this is a year old, but I've found the issue and it might be helpful for others with this issue.
If you're using the ESP8266Audio libraries with M5Core2 v0.1.4 or v0.1.5, and you're running "M5.begin()", you need to comment out "Spk.begin();" in "M5Core2.cpp" in your libraries. ie:
// Spk.begin();
And add this to your code:
M5.Axp.SetSpkEnable(true);
I don't really understand why this is causing the issue but I noticed v0.1.3 (which works fine) had the speaker initialisation function commented out.
A year late in my reply, but better late than never!