OK it works. So for future reference this is what I did:
Turn on the unit (mine is an M5StickC with UIFlow 1.6.6 firmware)
press the M5 button quickly after that to enter setup mode (my screen shows <code>)
side button to get the Setup menu
M5 again to select it
Switch Mode
M5 again
side button to select APP
side button to select '..' at top
M5 to go back to previous menu
side button to select Reboot
M5 to reboot
From now on whenever you turn it on the Stick will run boot.py and then main.py in the /flash directory.
If when it's so running you attach the device to a computer via the USB cable, you can interrupt whatever it is doing by starting Thonny and hitting ctrl-F2 (Stop/Restart Backend), which will bring up the MicroPython REPL prompt. Cool.