WBNA Notecard connectivity lost after host reboots. How to recover?

Hardware
1

New Blues user here who is amazed that I could get things working in 3 days! But need help with this question:

I have a custom host board using ESP32-S3, serial comms with the WBNA Notecard, that posts hourly reports to NoteHub. My ESP32 is programmed to soft-reboot itself (free up heap, etc) once daily at midnight, but there is no interruption of power supplies to the Notecard. Today, I found there were no reports and the Notecard LED was blinking red briefly every few seconds. I understand that indicates no cellular connectivity. A cold boot (unplug USB power) fixed it immediately.

I could have the ESP32 pull down the Notecard NRST line briefly after the ESP32 boots up. Is this a good practice, and might it solve this problem?

Thank you in advance. Great product!

2

Hi @mark_pdx and welcome to the Blues community!

We actually advice against using the Notecard’s RST. Which Notecarrier are you using and how is it being powered? Also are you using RX/TX or I2C?

Thanks,
Rob

3

Hi @RobLauer and thanks for responding.

I’m not using a Notecarrier, it’s a custom PCB I designed using an ESP32-S3. Communications with the Notecard via Rx/Tx @9600, and works great except when the ESP32 does a reboot.

Power to the Notecard and ESP is from a USB 5V,2A charger and a 3.3V LDO linear regulator.

I suspect the problem is with my code. This fragment is part of setup() which runs once when the ESP32 boots up:

	// Cellular mode: prepare Notecard serial comms, connect to cell, get time sync
	if (radioMode == "cell"){
		Serial.println("Preparing to connect to Notecard");
		myLEDDriver.display(BLUE);         // Display BLUE while waiting for cell conn's
		Serial1.setRxBufferSize(256);		
		Serial1.begin(9600, SERIAL_8N1, PIN_RX_FROM_NOTECARD, PIN_TX_TO_NOTECARD);
		notecard.begin(Serial1, 9600);
		// notecard.setDebugOutputStream(Serial); // Direct notecard debug to USB if needed
		connectCell();    // Request connection
		// Check hub status, rechecking as needed
		Serial.print("Checking hub status...");
		while (!isCellConnected) {
			J *req = notecard.newRequest("hub.status");
			J *rsp = notecard.requestAndResponse(req);
			if (rsp != NULL){
				isCellConnected = JGetBool(rsp, "connected");
				notecard.deleteResponse(rsp);
				if (isCellConnected) {
					Serial.println("connected!");
					break;
				}	
			}
			politeDelay(5000);
			Serial.print(".");
		}

		// Get time, rechecking until we get it
		myLEDDriver.display(AQUA);
		Serial.print("Getting time...");
		while (!isTimeSynced) {
			J *req = notecard.newRequest("card.time");
			J *rsp = notecard.requestAndResponse(req);
			if (rsp != NULL){
				if (JIsPresent (rsp, "time")) {
					// Set the time zone and fetch the unix time
					configTzTime(time_zone, ntpServer1, ntpServer2);
					time_t unixT = (time_t)JGetNumber(rsp, "time");
					notecard.deleteResponse(rsp);
					struct timeval tv;
					tv.tv_sec = unixT;  // epoch time (seconds)
					tv.tv_usec = 0;     // microseconds
					settimeofday(&tv,0);
					isTimeSynced = true;
					printLocalTime();
					break;
				}	
			}
			politeDelay(5000);
			Serial.print(".");
		}
		// Get the wireless signal parameters
		J *req = notecard.newRequest("card.wireless");
		J *rsp = notecard.requestAndResponse(req);
		if (rsp != NULL) {
			Serial.println("Wireless status:");
			J *netObject = JGetObject(rsp, "net");  
			Serial.println(JGetNumber(netObject, "rssi"));
			notecard.deleteResponse(rsp);
		}
		statusColor = GREEN;
		myLEDDriver.display(OFF);
	}

And here is the connectCell routine that is called by that setup code:

void connectCell(){
	// Request cellular connection, retrying up to 5 times 
	J *req = notecard.newRequest("hub.set");
	JAddStringToObject(req, "product", PRODUCT_UID);  // UID Found in pal_pins.h
	JAddStringToObject(req, "mode", "minimum");       // Connect only on sync request
	JAddStringToObject(req, "sn", deviceName.c_str());        // DeviceName is used as serial number
	Serial.println("Sending hub.set request");
	notecard.sendRequestWithRetry(req, 5); // 5 seconds
	Serial.println("Sending hub.sync");
	req = notecard.newRequest("hub.sync");
	notecard.sendRequestWithRetry(req, 5);
	return;
}

After the warm reboot, I find the last message is “Checking hub status…”.
Maybe when the ESP boots, I should be checking the Card first to make sure it’s communicating before issuing a Hub request?
Thanks so much for looking at this!

1 Like
4

Hi @mark_pdx,

I don’t see anything immediately wrong with your code. I would be curious to know what is returned in the status field of the hub.status call while this is happening though.

Rob

5

Hi @RobLauer, thanks for looking at the code. I turned on Notecard debugging to capture the serial responses.

First, here is the log from a cold boot – when the host and the Notecard are both initially powered up from USB:

Sending hub.set request
[INFO] {“req”:”hub.set”,”product”:”com.markleavitt.mark:villagepal”,”mode”:”minimum”,”sn”:”wllo_pal_0001”,”body”:{“agent”:”note-arduino”,”compiler”:”gcc 8.4.0”,”req_interface”:”serial”,”cpu_name”:”esp32”},”crc”:”0001:956B70C5”}
[INFO] {}

Sending hub.sync
[INFO] {“req”:”hub.sync”,”crc”:”0002:10BAC79A”}
[INFO] {}

Checking hub status…[INFO] {“req”:”hub.status”,”crc”:”0003:5874FEF1”}
[INFO] {“status”:”4s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0004:5874FEF1”}
[INFO] {“status”:”9s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0005:5874FEF1”}
[INFO] {“status”:”1s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0006:5874FEF1”}
[INFO] {“status”:”6s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0007:5874FEF1”}
[INFO] {“status”:”11s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0008:5874FEF1”}
[INFO] {“status”:”17s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0009:5874FEF1”}
[INFO] {“status”:”22s waiting for data service {wait-data} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”000A:5874FEF1”}
[INFO] {“status”:”connected (session open) {connected}”}

.[INFO] {“req”:”hub.status”,”crc”:”000B:5874FEF1”}
[INFO] {“connected”:true,”status”:”connected (session open) {connected}”}

connected!
Getting time…[INFO] {“req”:”card.time”,”crc”:”000C:1110F247”}
[INFO] {“minutes”:-480,”lat”:45.457362499999993,”lon”:-122.558984375,”area”:”Happy Valley OR”,”country”:”US”,”zone”:”PST,America/Los_Angeles”,”time”:1704300288}

Wednesday, January 03 2024 08:44:48
[INFO] {“req”:”card.wireless”,”crc”:”000D:3CE09784”}
[INFO] {“status”:”{network-up}”,”mode”:”auto”,”count”:1,”net”:{“iccid”:”89011704278852086760”,”imsi”:”310170885208676”,”imei”:”868050045716657”,”modem”:”EG91NAXDGAR07A01M1G_20.004.20.004”,”band”:”LTE BAND 2”,”rat”:”lte”,”internal”:true,”rssir”:-83,”rssi”:-83,”rsrp”:-115,”sinr”:3,”rsrq”:-20,”bars”:1,”mcc”:310,”mnc”:410,”lac”:36876,”cid”:84664072}}

Wireless status:
-83.00
Initializing radar sensor
Radar sensor serial port initialized
Status buffer initialized
Next update: Wednesday, January 03 2024 09:00:00
Activity detected this cycle

Now, here is the log from a “warm boot”, meaning the ESP has been rebooted but the Notecard has stayed power up:

Sending hub.set request
[INFO] {“req”:”hub.set”,”product”:”com.markleavitt.mark:villagepal”,”mode”:”minimum”,”sn”:”wllo_pal_0001”,”body”:{“agent”:”note-arduino”,”compiler”:”gcc 8.4.0”,”req_interface”:”serial”,”cpu_name”:”esp32”},”crc”:”0001:956B70C5”}
[INFO] {}

Sending hub.sync
[INFO] {“req”:”hub.sync”,”crc”:”0002:10BAC79A”}
[INFO] {}

Checking hub status…[INFO] {“req”:”hub.status”,”crc”:”0003:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:44:48Z) {disconnected} {idle}”}

.[INFO] {“req”:”hub.status”,”crc”:”0004:5874FEF1”}
[INFO] {“status”:”5s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0005:5874FEF1”}
[INFO] {“status”:”10s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0006:5874FEF1”}
[INFO] {“status”:”3s starting communications {wait-module} {connecting}”}

.[INFO] {“req”:”hub.status”,”crc”:”0007:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:49:27Z) {disconnected} {idle}”}

.[INFO] {“req”:”hub.status”,”crc”:”0008:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:49:27Z) {disconnected} {idle}”}

.[INFO] {“req”:”hub.status”,”crc”:”0009:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:49:27Z) {disconnected} {idle}”}

.[INFO] {“req”:”hub.status”,”crc”:”000A:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:49:27Z) {disconnected} {idle}”}

.[INFO] {“req”:”hub.status”,”crc”:”000B:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:49:27Z) {disconnected} {idle}”}

.[INFO] {“req”:”hub.status”,”crc”:”000C:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:49:27Z) {disconnected} {idle}”}

.[INFO] {“req”:”hub.status”,”crc”:”000D:5874FEF1”}
[INFO] {“status”:”idle (since 2024-01-03T16:49:27Z) {disconnected} {idle}”}

So it looks like it starts connected, but then drops to a “disconnected idle” state. And then it loops there forever, until I pull the plug. Ideas?

6

Hi @mark_pdx,

This is interesting. Can you check which version of the Notecard firmware you’re on (with a card.version request).

On a “warm boot”, I’m wondering since you are in minimum mode, if you need to give the Notecard a reason to sync with Notehub. Can you try adding a note.add request with sync:true in place of the hub.sync call, just to see if that’s enough to force a connection to Notehub?

Rob

7

Hi @RobLauer,

Using the Notehub facility, I find I’m on firmware V5.1.1.16026, but I see there is a newer development version 5.3.1, as well as a LTS version 4.4.2.4015704. Are either of these likely to fix the problem? I’m hesitant to do an OTA firmware update until I solve the warm-boot connectivity problem.
Mark

8

Hello @RobLauer,
Well, I updated the firmware to 5.3.1 and set the mode to “continuous” instead of “minimum”. Now it works fine after a reboot of the ESP32. So that brings me to a question:

I had chosen the “minimum” mode because I thought that would reduce the data consumption – not to save power, since the device is plugged in all the time. That may have been a wrong assumption.

I’m only uploading two ~150 byte packets, once an hour. Will “continuous” cause much higher data consumption then “minimum” mode? If not, then my problem is solved.

9

Hi @mark_pdx,

The usage of continuous vs periodic or minimum modes is primarily meant to save on power, not data usage. We have a guide that estimates data usage here. When sending small payloads of data, using periodic/minimum mode can actually use MORE data over time, since establishing a connection can use up ~28KB each time a connection is made. Compare that to letting a Notecard idle in continuous mode, which I measured at 18.6KB over 10 hours!

I’m still concerned that an explicit hub.sync request in minimum mode didn’t actually initiate a connection to Notehub. I’m going to look into that more deeply today.

Rob

10

FYI - I was unable to reproduce this using minimum mode on a device running firmware v5.3.1. So it could’ve been a bug we fixed that I wasn’t aware of, or there is something else with your firmware or host that we are missing.