I2C Connection Issues with ESP32 MCU and Notecard WiFi

Hello,

I’m working on my first project using Notecard, and I’m having some trouble with the I2C connection between the Notecard and my microcontroller. I’m using an ESP32-WROOM and the Notecarrier XS with the Notecard WiFi (v2.1) and a Starnote. The MCU and Notecarrier are powered separately through USB power, and the two are connected through the QWIIC port on the Notecarrier, as shown below.

I’ve tested the Notecard on it’s own by following the setup guide, and it has no issues, but the problem arises when I try to send messages from the MCU. Or rather, when the MCU tries to receive messages from the Notecard. The MCU is capable of sending a message to the Notecard, and that message is sent on to Notehub, but the MCU starts throwing errors in the serial log about not being able to receive from the Notecard:

[ERROR] serial-over-i2c|rx: unexpected protocol byte count {io}{i2c}
[WARN] i2c: reattempting to read Notecard response
[ERROR] serial-over-i2c|rx: unexpected protocol byte count {io}{i2c}
[WARN] i2c: reattempting to read Notecard response

These errors will repeat over and over, and sometimes the MCU will even resend the message to the Notecard. I have searched this forum for similar issues and have tried the solution from a similar problem of adding 10k pullup resistors to the SDA and SCL lines, but the result did not change. It may be worth noting that I have in rare cases seen the MCU send a few of these errors when first powering on, before sending a request and receiving the response just fine, but this is rare.

Also, for reference, the code I am using is shown below. If it’s important, my firmware and arduino libraries are up to date.

#include <Notecard.h>

#define usbSerial Serial

Notecard notecard;

void setup() {
  delay(2500);
  usbSerial.begin(115200);
  
  notecard.begin();
  notecard.setDebugOutputStream(usbSerial);

  delay(2500);

  J *req = notecard.newRequest("hub.sync");
  if (req != NULL) {
    notecard.sendRequest(req);
  }
}

void loop() {

}

I removed all of the sensors and other parts of the project and simplified the code as much as possible so I could isolate this issue. As I mentioned above, I’m able to send data to Notehub, so I’ve been able to upload sensor data despite this issue, but I can only do so once since the MCU will get stuck looping on these errors.

Any help would be appreciated.

eddy_w,

Just wondering on the I2C lines you have pull-ups?

Warm Regards,
Rob Oudendijk

I had already tested with and without pullups before, but I just tried with them again just to make sure. I think in doing so I actually ended up figuring out the problem. In my testing just now, I found that the MCU could read the response just fine. However, I was able to replicate the error by unplugging / plugging back in the MCU and Notecard too many times. When I left them both off for enough time, they communicated fine again. Based on this, I think that my error was actually a combination of problems I had seen on this forum already:

  1. Pullup resistors: Since the Note-ESP and my ESP32 both have no pullup resistors on the I2C lines, I needed to add my own.
  2. Cold-boot: As I’ve seen elsewhere on the forum, the Notecard isn’t meant to be turned off and back on again so quickly and so many times in a row, so doing so caused some communication errors.
  3. Not enough delay between MCU startup, notecard.begin(), and notecard.sendRequest(): The errors were much more likely when I took out the delay at startup. Not 100% sure about the delay between begin() and sendRequest(), but in my previous testing when I was first seeing the error I had no delay here.

Anyways, thanks Rob for asking about the resistors, testing with them again helped me to figure the issue out (I think). I’ll come back and ask if I end up having any more issues.

Thank you,

eddy_w

Hey friends,

I ended up running into the error again. I went through a whole bunch of testing, and I think I have the issue singled out. I am able to send and receive messages just fine, with the exception of templated Notefiles. The response message that seems to give errors is the following:

{
 "template": true,
 "total": 2
}

I have tested sending non-templated notefiles, and the response comes in just fine. I have made sure that having two total notefiles (which happens with one templated file) isn’t the problem by sending two non-template files. I also ended up filling the file storage on the notecard, so I began receiving this error:

error adding note: can't exceed 80% of filesystem (currently 100%) {full}{file-storage-full}

Which tells me that the error doesn’t have to do with the length of the message. I also received this error when using templated Notefiles. I had been using sync:true, full:true, and a templated file with delete:true, and I have tested with and without these as well, they do not change the outcome.

Like before, the messages are getting to the Notecard and to Notehub just fine, but the response to the MCU is having errors. When the response fails to be read, the MCU will loop on the error, and send the packet again multiple times. I tried to use sendRequestWithRetry() to have a timeout and avoid sending multiple packets, but the timeout never seems to happen, and the MCU keeps looping and sending the packet over and over again. Part of this problem is that I am using Starnote for this project, so I would like to avoid sending extra packets to keep the cost down. Using Starnote also means that I need to use templated Notefiles, so in some way or another I will need to solve this issue. I have also been trying to set up serial to the aux UART port on the Notecarrier, but I haven’t had any success.

Again, any help would be appreciated.

Hi @eddy_w,

That filesystem error you see tells us that the Notecard’s flash is almost completely full. The only way this should happen is if you’re storing untemplated Notes without syncing with Notehub. As soon as Notecard has a successful sync with Notehub, Notes on the device will get deleted. If you’re saying that you ARE seeing the Notes in Notehub, I’m pretty confused why this would be happening!

Just to clarify, you’re on the latest version of Notecard firmware? Maybe try resetting the Notecard with {"req":"card.restore","delete":true} to see if you can replicate the issue?

Rob

Yeah sorry, I forgot to mention that I fixed the filesystem error by resetting the Notecard. That is confusing though, because I’m using templated Notes with sync:true, so every time I send one of them, the card syncs and the Note ends up in Notehub. Thinking back on it, I’m guessing that the filesystem error happened from me unplugging everything (Notecard and all) every time I tried making changes and it didn’t work. I’m guessing that some data ended up left in the flash memory, and over time I just kept filling up file space by unplugging before it properly synced over and over again. For the future, I’ll try to leave the Notecard plugged in after I unplug the MCU to make sure everything is synced.

I’m not sure if that error has to do with the main one I’m seeing, since, like you said, it has to do with unsynced files, and all of mine are getting synced. And yes, I still saw the I2C communication error after resetting the filesystem, and my card is on the newest firmware.

Thanks,

eddy_w

Hi @eddy_w,

It may be worth trying a more thorough reset of the Notecard with {"req":"card.restore", "delete":true, "connected":true} which will wipe the filesystem but also clear out any existing Notefile Templates after the Notecard’s first sync with Notehub. If you’re truly using templated Notes you’d have to be saving so much data to the Notecard to hit the filesystem limit (I mean it’s possible yes, but unlikely!).

Also, the Notecard is designed to re-attempt a sync after a power cycle, so Notes shouldn’t get “trapped” on the Notecard after a reboot either.

Regarding your original I2C problem, after looking at your image more closely, the issue might be the fact that you have separate USB power supplies for the host and Notecarrier. I know QWIIC includes a ground, but if the two boards are on different USB ports, the ground reference can shift enough to corrupt I2C bits during long clock stretches. I would either power both from the same USB source, or run a dedicated GND wire directly between the ESP32 board and the Notecarrier (not just through the QWIIC cable).

You could also try dropping the I2C clock to see if that helps. Add this after notecard.begin():

Wire.setClock(100000);

Ok, I did some tests, and I have some odd results. Below is a table of the tests I ran and their outcomes. I ran the same test code every time, which attempts to send a total of 4 requests. The first is a simple sync. This is a small request and response, so it should tell us if the connection works at all. The second is a somewhat big non-templated Notefile, which should tell us if sending a bunch of data to the Notecard is a problem. Next is a card.wireless request, which gives a long response back to the MCU, telling us if the problem is receiving too much data. Last is the templated Notefile, which is the request that has been giving me trouble this whole time. As for the tests, the first was my existing circuit, connecting to the Notecard through only the QWIIC port. I tried also running a direct ground wire between the Notecard and MCU, but this was actually unsuccessful, oddly enough. Powering the Notecard through the VUSB pin (and using a direct ground) worked differently depending on how I powered the pullup resistors, but even the better of the two was iffy. I couldn’t get the clock rate change to work either, it seemed to bug out right away (I tried this on the QWIIC only circuit).

| Test | Sync | Non-template | Wireless | Template
| QWIIC Only | Y | Y | Y | N
| Direct Ground | N | N | N | N
| Same Power, QWIIC Power for Pullups | Y | N | N | N
| Same Power, USB Power for Pullups | Y | Y | N* | N
| Changed I2C Clock | N | N | N | N

*This did work when I added a LiPo to the Notecard. This was the only change from adding a LiPo.

Also, with the filesystem, after doing the card.restore you recommended, I ended up finding a new oddity. I forgot to do hub.set: mode:minimum, which was the mode I had been using, and until I did, the connection became very poor, if it worked at all. Even while using the original QWIIC only circuit, I was getting the I2C error on sync requests. This did go away after setting the mode to minimum, but it’s still very strange to me that the default mode would cause issues.

As an extra test, I left the MCU running for about 15 minutes sending non-template packets every 30 seconds (on the original circuit), and I saw no communication errors the whole time. This test made it seem like the I2C connection works just fine, but clearly something is unusual here. Between the strange filesystem error, errors when the mode is not set to minimum, and of course the problem with templated Notefiles, I’m thoroughly confused.

Again, I’d appreciate any help.

Thanks,

eddy_w

In case anyone wants to see, here’s what a typical output looks like for one of these tests:

[INFO] {"req":"hub.sync","crc":"0000:10BAC79A"}
[INFO] {}
[INFO] {"req":"note.add","file":"data.qo","full":true,"sync":true,"body":{"temp":73.19999694824219,"charge":92.40000152587891,"flow":true,"alert":false,"humidity":50,"power":50,"text":"a really long string of text to send a lot of data"},"crc":"0001:67409E2F"}
[INFO] {"total":1}
[INFO] {"req":"card.wireless","crc":"0002:3CE09784"}
[INFO] {"status":"{modem-off}","mode":"auto","count":3,"net":{"imei":"--:--:--:--:--:--","modem":"v5.2.1","ssid":"<ssid>","bssid":"--:--:--:--:--:--","rat":"wifi-2.4","rssi":-68,"bars":3,"cid":9,"updated":1776294236}}
[INFO] {"req":"note.add","file":"fast.qo","full":true,"sync":true,"body":{"temp":73.19999694824219,"charge":92.40000152587891,"flow":true,"alert":false},"crc":"0003:083FCDFE"}
[ERROR] i2c|rx: no response to read request {io}{i2c}
[WARN] i2c: reattempting to read Notecard response
[ERROR] serial-over-i2c|rx: unexpected protocol byte count {io}{i2c}
[WARN] i2c: reattempting to read Notecard response
[ERROR] serial-over-i2c|rx: unexpected protocol byte count {io}{i2c}
[WARN] i2c: reattempting to read Notecard response
[ERROR] serial-over-i2c|rx: unexpected protocol byte count {io}{i2c}
[WARN] i2c: reattempting to read Notecard response
[ERROR] serial-over-i2c|rx: unexpected protocol byte count {io}{i2c}
[ERROR] failed to query Notecard
[WARN] retrying... transaction failure

Also for reference, here are the packets I’m sending:

void notecard_sync() {
  J *req = notecard.newRequest("hub.sync");
  if (req != NULL) {
    notecard.sendRequest(req);
  }
}

void notecard_network() {
  J *req = notecard.newRequest("card.wireless");
  if (req != NULL) {
    notecard.sendRequest(req);
  }
}

void notecard_send_data() {
  J *req = notecard.newRequest("note.add");
  if (req != NULL) {
    JAddStringToObject(req, "file", "data.qo");
    JAddBoolToObject(req, "full", true);
    JAddBoolToObject(req, "sync", true);
    J *body = JAddObjectToObject(req, "body");
    if (body != NULL) {
      JAddNumberToObject(body, "temp", temp);
      JAddNumberToObject(body, "charge", charge);
      JAddBoolToObject(body, "flow", flow);
      JAddBoolToObject(body, "alert", alertStatus);
      JAddNumberToObject(body, "humidity", 50.0);
      JAddNumberToObject(body, "power", 50.0);
      JAddStringToObject(body, "text", "a really long string of text to send a lot of data");
    }
    notecard.sendRequestWithRetry(req, 1);
  }
}

void notecard_send_fast() {
  J *req = notecard.newRequest("note.add");
  if (req != NULL) {
    JAddStringToObject(req, "file", "fast.qo");
    JAddBoolToObject(req, "full", true);
    JAddBoolToObject(req, "sync", true);
    J *body = JAddObjectToObject(req, "body");
    if (body != NULL) {
      JAddNumberToObject(body, "temp", temp);
      JAddNumberToObject(body, "charge", charge);
      JAddBoolToObject(body, "flow", flow);
      JAddBoolToObject(body, "alert", alertStatus);
    }
    notecard.sendRequestWithRetry(req, 1);
  }
}

Hi @eddy_w,

Thanks for the thorough testing results, the table and log output are super helpful.

I’m starting to wonder if these issues are specific to the Notecard WiFi that you are using (it’s also ESP32-based) and there are all sorts of I2C issues we work around in firmware with the ESP32.

First question: Do you have another non-WiFi Notecard you can try?

You could also try adding a small delay between requests (especially between the non-templated and templated note.add calls). Even 500ms can help the previous sync activity settle down before the next I2C transaction.

Thanks,
Rob