So, after many experiments I'm sorry to report that the RS-485 Atom module has very bad Rx characteristics.

The issue of getting corrupted data was related to problems in the serial support of the arduino-esp32 project. After moving to git version I still had several problems:

missing bytes zero byte at end of rx block last byte of block was sometimes corrupted

I've then tried replacing the RS-485 Hat with another RS485-to-RS422 adapter and since then it works flawlessly. Something in the electrical design of the RS-485 hat must be messing up the Rx data.