MQTT subscribes on UiFlow

Buhi

@robalstona Ok i understand enough from MQTT how it works. They are published to my Broker every second.
But strange is that this code below with one subscribe topic works.
First both variables get "0" after the 3500ms wait solar gets update with 0.0 (now its night here so no solar power)

Because i have two solaredge inverters i have two different values in the same format.
So if i try to get the second one also with mqtt subscribe to a different topic (2nd inverter) the hole screen is black. Any hints?

world101

@robalstona said in MQTT subscribes on UiFlow:

@buhi It is possible that the problem is how the data is sent to the mqtt server and how often they are sent. In my case, the sensor data is sent every 25 seconds. And if I run this program it will only show me --- until the server receives new data. and then the channel is subscribed, the data are saved to the variable and displayed on the lcd display. So if you have data updated, for example, every hour, nothing may be displayed for an hour. You can combine something with the "retain" flag when sending data to the server, then you will be able to receive the last data received from the feed. But this already needs to be changed on the device / software side that this data sends.

https://mntolia.com/mqtt-retained-messages-explained-example

I opened a request for UIFlow to add support for the retain flag. You can track it here.

https://github.com/m5stack/UIFlow-Code/issues/1

world101

@buhi said in MQTT subscribes on UiFlow:

@robalstona
So if i try to get the second one also with mqtt subscribe to a different topic (2nd inverter) the hole screen is black. Any hints?

Are there any errors on HomeAssistant? I use Adafruit IO and when I get a black screen due to an issue configuring something in UIFlow , typically I can see some errors on the broker’s web console.

It could be your feed configuration or the sensor data (stored as float or int instead of strings) or something like that.

skysurfer

I have the exact same problem. 1 topic works fine, but more does not work.

world101

Two topics is working for me with Adafruit IO and this example flow. I'm using uiFlow V1.4.4 and firmware V1.4.3 on the grey M5Stack core (since firmware V1.4.4 currently has some issues).

0_1579271908570_Screen Shot 2020-01-17 at 9.36.39 AM.png

I'm not using HomeAssistant and do not have any real sensors that I'm pulling data from. However, the two topics is working for me when I simulate sensor data with button A and B presses, or when I add the data manually via the AIO feed console.

I'll try to set up HomeAssistant on a RPi and see if anything changes.

skysurfer

@world101 Your example does not read the topics, it's being set when you push the buttons....

world101

@skysurfer said in MQTT subscribes on UiFlow:

@world101 Your example does not read the topics, it's being set when you push the buttons....

You are correct. I'm having the same issues as you.

Buhi

Is there really no one that can help us? We are already 3 with the same problem.
I don't think that we did the same wrong.
Is there a other way to get the values from my mqtt broker or homeassistant? Processing by IFTT or other web service and read the values a different way onto the M5?

robalstona

@buhi there is one more option: you can read / write data from the channel using REST API queries: tab ADVANCED -> HTTP-> HTTP REQUEST. Most MQTT brokers allow this option.

When i send http request:
https://io.adafruit.com/api/v2/robalstona/feeds/feed01

i receive something like that in json format:

{"username":"robalstona","owner":{"id":300855,"username":"robalstona"},"id":1207296,"name":"feed01","description":null,"license":null,"history":true,"enabled":true,"visibility":"public","unit_type":null,"unit_symbol":null,"last_value":"20.7","created_at":"2019-10-27T19:40:47Z","updated_at":"2020-01-23T12:01:53Z","key":"feed01","writable":false,"group":{"id":198443,"key":"default","name":"Default","user_id":300855},"groups":[{"id":198443,"key":"default","name":"Default","user_id":300855}]}

in my case the "last_value" field with assigned value "20.7" is what I need.

You can use JSON->loads json and ADVANCED -> HTTP -> Get Data to store this data in variable as list.

only then do you need to know the structure of this list to get to the field with interesting data values

world101

Okay, so I was really challenged and determined by this thread to get the setup working. I installed Home Assistant and set up the Mosquitto MQTT add-on. Long story short, I was able to get multiple topics subscribed and working on 3 different devices. Two M5Stacks and one iPhone app called MQTTool. All three devices are publishing and receiving data to/from MQTTT.

Here are my uiFlows:
0_1579798423204_Screen Shot 2020-01-23 at 11.51.34 AM.png
0_1579798433528_Screen Shot 2020-01-23 at 10.32.17 AM.png

M5Stack #1 has an ENV unit providing Temperature, Pressure, and Humidity to MQTT, is reading the status of the LIGHT unit from M5Stack #2, and is reading the status of the iPhone app providing fake sensor/thermostat data. M5Stack #2 has a LIGHT unit providing the digital status of the light sensor (on/off) to MQTT, is reading the T/P/H values provided by M5Stack #1, and is reading the status of the iPhone app providing fake sensor/thermostat data.

Troubleshooting the black screen: REPL is your friend for errors. That is ultimately how I figured out what was not working. On a Mac, I opened Terminal and typed this to get to the REPL:
screen -port /dev/tty.SLAB_USBtoUART -b 115200

As I was building the setup, trying a slightly modified flow on each device, I forgot to connect the unit on the other M5stack. If you have a unit added in uiFlow UI (left side panel) and the unit blocks added to your flow, but the unit is not physically connected to the M5stack, this causes a black screen when attempting to connect to MQTT. No on screen error at all, but the REPL said something like "env0 not found" when I rebooted the M5stack and looked at the startup logs. It would just sit there and hang.

Another issue is that the Unit number might be changing on you when you add/remove it from your device or program (i.e. env0 changes to env1 or light0 changes to light1), especially when you don't refresh the entire uiFlow webpage before connecting each new device.

Anyway, I hope this helps someone. If you have other scenarios you want me to test, let me know.

Buhi

@world101 Thank you so far for your hard work. I really have to try again.

world101

I have some important discoveries... If you have another client device (i.e. MQTTlens, MQTTool, etc.) that sets the retain flag = True on the publish message, that will cause an error on the m5stack MQTT client when you subscribe to multiple topics. Here is the Traceback error.

Internal FS (FatFS): Mounted on partition 'internalfs' [size: 2031616; Flash address: 0x210000]
----------------
Filesystem size: 1978368 B
           Used: 585728 B
           Free: 1392640 B
----------------
I (497) [TFTSPI]: attached display device, speed=8000000
I (497) [TFTSPI]: bus uses native pins: false
[ M5 ] node id:d8a01d698f30, api key:32532B1C
I (4783) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (4784) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (4935) phy: phy_version: 4100, 2a5dd04, Jan 23 2019, 21:00:07, 0, 0
I (4942) modsocket: Initializing
Connect Wi-Fi: SSID:xxxxxxxx PASSWD:xxxxxxxx network...
......
Connected. Network config: ('10.0.0.5', '255.255.255.0', '10.0.0.1', '10.0.0.1')
Warning: Comparison between bytes and str
Warning: Comparison between bytes and str
Warning: Comparison between bytes and str
Traceback (most recent call last):
  File "main.py", line 48, in <module>
  File "flowlib/m5mqtt.py", line 100, in subscribe
  File "flowlib/simple.py", line 193, in subscribe
  File "flowlib/simple.py", line 243, in wait_msg
  File "flowlib/m5mqtt.py", line 52, in _on_data
KeyError: b'sensor/light'
MicroPython v1.11-319-g54c3f2613-dirty on 2020-01-15; ESP32 module with ESP32
Type "help()" for more information.
>>>

So, subscribing to multiple topics seems to be okay in general, but setting the retain flag = True on more than one of those topics seems to cause the MQTT client to throw an error. Also, with lots of debugging, I found that you can only have one topic that has the retain flag = True and it has to be the last one in your micropython code (I think because the code is processed from the bottom up???).

This code works. Topics aa, bb, cc, and dd have messages with retain=True. Topics aaa, bbb, ccc, and ddd have messages with retain=False. Topic dd is at the bottom.

from m5stack import *
from m5ui import *
from uiflow import *
from m5mqtt import M5mqtt

setScreenColor(0x000000)


m5mqtt = M5mqtt('m5stickC', '10.0.0.60', 1883, 'mqtt', 'password', 300)

label0 = M5TextBox(2, 17, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label1 = M5TextBox(2, 57, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label2 = M5TextBox(2, 96, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label3 = M5TextBox(2, 132, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)

a = None
b = None
c = None
d = None


def fun_aaa_(topic_data):
  global a, b, c, d
  a = topic_data
  label0.setText(str(a))
  pass
m5mqtt.subscribe(str('aaa'), fun_aaa_)

def fun_bbb_(topic_data):
  global a, b, c, d
  b = topic_data
  label1.setText(str(b))
  pass
m5mqtt.subscribe(str('bbb'), fun_bbb_)

def fun_ccc_(topic_data):
  global a, b, c, d
  c = topic_data
  label2.setText(str(c))
  pass
m5mqtt.subscribe(str('ccc'), fun_ccc_)

def fun_dd_(topic_data):
  global a, b, c, d
  d = topic_data
  label3.setText(str(d))
  pass
m5mqtt.subscribe(str('dd'), fun_dd_)


m5mqtt.start()
while True:
  wait(1)
  wait_ms(2)

This does not work since topic cc (retain=True) is not at the bottom.

from m5stack import *
from m5ui import *
from uiflow import *
from m5mqtt import M5mqtt

setScreenColor(0x000000)


m5mqtt = M5mqtt('m5stickC', '10.0.0.60', 1883, 'mqtt', 'password', 300)

label0 = M5TextBox(2, 17, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label1 = M5TextBox(2, 57, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label2 = M5TextBox(2, 96, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label3 = M5TextBox(2, 132, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)

a = None
b = None
c = None
d = None


def fun_aaa_(topic_data):
  global a, b, c, d
  a = topic_data
  label0.setText(str(a))
  pass
m5mqtt.subscribe(str('aaa'), fun_aaa_)

def fun_bbb_(topic_data):
  global a, b, c, d
  b = topic_data
  label1.setText(str(b))
  pass
m5mqtt.subscribe(str('bbb'), fun_bbb_)

def fun_cc_(topic_data):
  global a, b, c, d
  c = topic_data
  label2.setText(str(c))
  pass
m5mqtt.subscribe(str('cc'), fun_cc_)

def fun_ddd_(topic_data):
  global a, b, c, d
  d = topic_data
  label3.setText(str(d))
  pass
m5mqtt.subscribe(str('ddd'), fun_ddd_)


m5mqtt.start()
while True:
  wait(1)
  wait_ms(2)

However, if you move the cc topic to the bottom it works again.

from m5stack import *
from m5ui import *
from uiflow import *
from m5mqtt import M5mqtt

setScreenColor(0x000000)


m5mqtt = M5mqtt('m5stickC', '10.0.0.60', 1883, 'mqtt', 'password', 300)

label0 = M5TextBox(2, 17, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label1 = M5TextBox(2, 57, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label2 = M5TextBox(2, 96, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)
label3 = M5TextBox(2, 132, "Text", lcd.FONT_Default,0xFFFFFF, rotate=0)

a = None
b = None
c = None
d = None


def fun_aaa_(topic_data):
  global a, b, c, d
  a = topic_data
  label0.setText(str(a))
  pass
m5mqtt.subscribe(str('aaa'), fun_aaa_)

def fun_bbb_(topic_data):
  global a, b, c, d
  b = topic_data
  label1.setText(str(b))
  pass
m5mqtt.subscribe(str('bbb'), fun_bbb_)

def fun_ddd_(topic_data):
  global a, b, c, d
  d = topic_data
  label3.setText(str(d))
  pass
m5mqtt.subscribe(str('ddd'), fun_ddd_)

def fun_cc_(topic_data):
  global a, b, c, d
  c = topic_data
  label2.setText(str(c))
  pass
m5mqtt.subscribe(str('cc'), fun_cc_)

m5mqtt.start()
while True:
  wait(1)
  wait_ms(2)

Hope this helps. Also, I hope @m5stack will update the m5mqtt client to support the Retain flag, QoS, and the Last Will and Testament (LWT) messages soon to solve some of these limitations I have uncovered.

Buhi

Ok that could be they key. I will try that out.
So many thanks!

skysurfer

@world101 Makes no difference here...

world101

@skysurfer

I can’t help much without knowing your setup. What are you seeing on the REPL or serial monitor? What devices and firmware version are you running? Post your code and I’ll see if I can figure out your issue.

cortado

@world101 Thanx! After making new topics and setting them to QoS=0, and retain=0, it finally works with many MQTT subscriptions on both M5Stack and M5Stick-C! (1.4.4 and 1.4.5)

(PS! Just changing the old topics that had retain back to no retain, did not work, had to make new ones.)