@Koiru I have not found a solution but I have found a workaround.
The culprit creating the constant reboots are the touchscreen, if I call the begin function without enabling the touchscreen I can get the unit to work instead of doing constant reboots.
To initialize the unit without touchscreen, use the following arguments:
M5.begin(false, true, true, true, true);