Retrieving data over HTTPS (M5 Paper, but I think this is a generic question)

Hamnet

I'm sure this is basic but it's driving me crazy. I can retrieve data over wifi from an "http:" source with no problem. But I can't figure out how to access a URL over https.

All the examples I can find online - for Arduino, ESp32, or M5 - seem to use http: URLs in their demos. I've found references to WifiClientSecure, but not a usable example. Evidently simply substituting it for WifiClient is not sufficient!

The canvas.drawJpgUrl function works with an "https:" URL, so clearly there is no fundamental difficulty.

Googling led me to this excellent library "Fetch": but alas the examples don't compile, a problem others have noted.

Probably part of the problem is the number of different libraries with similar or even identical names, and different versions of the same library.

Please can someone recommend the current best (or at least good!) library to use, and point me at a working example?

Thanks in advance!

Hamnet

(I should add - as a temporary workaround I can access an https: resource by calling client.setInsecure(); but I'm looking for a proper solution.)

felmue

Hello @Hamnet

the 'Fetch' library doesn't compile for me either. But I found this example from in the official Espressif github repository which compiles and runs fine.

Thanks
Felix

ajb2k3

@felmue from what I have found, there seams to be an issue with https:// based access in Micropython and Arduino. Without knowing more I would guess it is to do with Certificates and security keys.

Hamnet

@felmue Thanks Felix, that got me over the hump! I think the issue was trying to convert my existing example directly.

Hamnet

This post is deleted!

konacurrents

I have the WiFiClientSecure working to call a https at my cloud site running node-red.

Follow these steps:

https://randomnerdtutorials.com/esp32-https-requests/

What I cannot figure out is how to keep this connection open as it takes about 3 seconds to make a call. My MQTT is much faster.

casascius

I had the same problem and was able to resolve it. I'm using a made-up CA certificate, which in turn was used to sign a server key for my MQTT server. I found all of these steps to be necessary to get WiFiClientSecure to talk to it without needing to call setInsecure().

Provide that CA certificate to the WiFiClientSecure class by calling setCACert()
Access the remote host through its DNS name, not an IP address. (It looks like ESP32 can find it either through traditional a DNS server, or by multicast mDNS such as yourhostname.local if it's on the same network)
The DNS name must be listed as a "DNS" entry in Subject Alternative Name section of the certificate on the remote host. (I was able to use ChatGPT4 to walk me through creating the openssl "conf" file needed to generate a Certificate Signing Request and then ultimately the Certificate that met this criterion).

konacurrents

@casascius Interesting. In my case I'm using real certs from my cloud and it works. I don't know how a DNS name would work outside of localhost names.

My main issue stated above was that I want a connection to stay open, and I wasn't able to figure that out.

thanks.