I am getting inconsistent i2c error messages. My setup will work for hours (anywhere between 2-15 hours) and the errors start randomly every time, between multiple devices (4, to be exact) with the same exact firmware and hardware setup.
My sensors operate well and produce accurate readings with all notecard stuff removed from the code only (kept in hardware setup) and without initializing the Notecard.
Example of my error messages:
10:06:01.570 -> [INFO] {"req":"card.time","crc":"0002:1110F247"}
10:06:02.661 -> [ERROR] serial-over-i2c: no response to read request {io}
10:06:02.661 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:03.783 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:03.783 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:04.875 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:04.875 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:05.981 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:05.981 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:05.981 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:05.981 -> [ERROR] error occured during receive
10:06:06.202 -> [ERROR] i2c: received NACK on transmit of data {io}
10:06:06.202 -> [ERROR] error encountered during I2C transmit hook execution
10:06:07.956 -> [WARN] retrying I/O error detected by host
10:06:08.503 -> [INFO] {"req":"card.time","crc":"0002:1110F247"}
10:06:09.502 -> [ERROR] serial-over-i2c: no response to read request {io}
10:06:09.502 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:10.522 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:10.522 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:11.605 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:11.605 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:12.693 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:12.693 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:12.693 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:12.693 -> [ERROR] failed to query Notecard
10:06:14.354 -> [ERROR] serial-over-i2c: no response to read request {io}
10:06:14.354 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:15.431 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:15.431 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:16.537 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:16.537 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:17.628 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
10:06:17.628 -> [WARN] serial-over-i2c: reattempting to read Notecard response
10:06:17.628 -> [ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
More info:
I have one device using serial communication ( STEVAL-MKI208V1K), and 3 devices using i2c. Two of these i2c devices are ADS1015 from Adafruit and one MLX90614 from DFRobot. I cannot remove the pull-up resistors from these.
I am using the Notecard Cell+WiFi and Notecarrier F V1.3, and the Swan mcu. I tried the same setup on the Notecarrier v1.0 and Notecard Cellular and same issue occurs.
I have tried old versions of my code and old versions of Notecard.h (currently using 1.5.4)
Slowing the data rate down of ADS1015 devices
I tried switching Notecard communications to Serial and if I do that I just get “notecard not responding”
I am not a well-seasoned programmer so please bear with me. Since these errors are inconsistent and started randomly, it is difficult to provide accurately what may have led to this. I am happy to provide any more information needed to help me work through this.
Hi,
If I understand your setup correctly than you have four devices on I2C bus including notecard, right?
And your setup works well for around 2 hours.
Something like this happened with me with Zephyr LVGL library and my display stops working after 10 mins or so. I realized that it was due to stack overflow. I am still looking for way to overcome this situation as Zephyr tasks can’t be deleted. Or to be honest, I do not know how.
Are you sure stack overflow is not in your case?
What happens if you reset your devices? does it start working again? if so, than stack overflow is likely.
I had some problems with the I2C communication as well when using the Blues Notecard library. I wrote my own library that supports only I2C, no Serial communication and is using static memory allocation instead of dynamic. Blues-Minimal-I2C. That solved my problems.
I am still learning about pullup and termination resistors as a result of having these problems. Could you recommend the best way to determine this value?
By observation, my first I2C device (MLX90614-DCC) has a pullup resistance of 4.7kohm on the SDA/SCL lines and my next 2 devices (both ADS1015) have 10kohm pullup resistors.
I know you asked for termination resistance, but I am more unfamiliar with this concept.
On an I2C bus should be only one pullup and not multiple.
10kOhm + 10kOhm + 4.7kOhm in parallel results in a very low resistance pullup.
Remove the 10kOhm resistors and use only the 4.7kOhm as a single pullup.
Thank you, unfortunately it is not ideal to remove any of these as they are a part of the purchased boards I am using and I do not want to damage them in any way. Maybe I can investigate new hardware without any pull-up resistors.
I would not worry about multiple resistors. They are connected in parallel so your 10K+10K+4.7K will give you about 2.4K. The rule of thumb is anything between 1K and 10K but as a rule of thumb this might or might not work in your situation. There are many factors that you should take into account.
Usually, you have a speed vs. power tradeoff. Weak pullups (i.e. high resistor values) will leak less current. If your pullups are strong, you have faster transitions, if they are too strong, your signal might not reach low, causing problems.
And from what you posted in the beginning, this really looks more like a memory issue than an I2C-bus issue. If the I2C bus works for hours and then fails, it is hard to believe that suddenly the pullups are a problem. Unless you have thermal problems.
Thank you. Yes, it is odd because for some setups/devices, it will work for hours and then start having the errors, but with other devices they happen immediately… I would also think that it is some sort of memory issue.
This does not pertain to the i2c errors specifically, but I feel it may be relevant:
After switching the notecard to use serial communication, I seem to have much greater success, but notefiles with my sensor data have stopped uploading after 12+ hours on rare occasions. Because of this, it is very difficult to determine what the error is. Once I plug my device into my computer via usb to observe the serial monitor/debug output after sensor data notes have stopped uploading, it does not show anything (I’ve waited for them to show up for 10 minutes… perhaps I should wait longer to really see - my sensors are to sample every 2 minutes). I leave the devices plugged in at my desk, so unless the notes stop uploading during work hours (which has not happened yet) I cannot plug it in to see the errors at the time it actually happens.
I do not think it is a connectivity issue since inbound syncs still happen and system/session notefiles still show up in my events.
Interesting, I have the same issue using the BME280 (Grove - Barometer Sensor(BME280) | Seeed Studio Wiki) connected to a Notecarrier-F v1.0. Basically a call to “card.temp” produces random I2C errors. I see a number of retries when performing a “trace” log and sometimes it produces a value after many retries and sometimes not. I was hoping maybe a Notecarrier v1.3 might fix this but it appears not if you are having the same issue with the V1.3.
I have also tried the Adafruit BME280 breakout board with the same results. There is nothing else on the I2C bus its just Notecard and BME280.
To be clear I am simply calling card.temp and getting this error so I think even trying @bernd library wont help me.
Using Serial communication between notecard and host mcu is not an option for me.
Would be nice of the guys at Blues (@zfields ) could chime in ?
Thanks,
Serge
Extract from the output
[INFO] {"req":"card.temp","crc":"000F:CE0D7EDD"}
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[WARN] serial-over-i2c: reattempting to read Notecard response
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[WARN] serial-over-i2c: reattempting to read Notecard response
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[WARN] serial-over-i2c: reattempting to read Notecard response
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[WARN] serial-over-i2c: reattempting to read Notecard response
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] failed to query Notecard
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[ERROR] i2c: unknown error on TwoWire::endTransmission() {io}
[ERROR] error encountered during I2C transmit hook execution
[WARN] retrying I/O error detected by host
I’ve been watching this thread, but I wanted to wait to respond until I could attempt to reproduce the issue.
I’m traveling, so I don’t have access to my lab.
Off the top of my head…
I think @tsuzenski probably has too many pull-ups on the I2C bus. I suspect your pull-up value is right at the boundary of reliability. This would explain the better behavior observed after switching over to USART (the Notecard has pull-ups too). Another simple test would be to put the Notecard back in place and take an alternate peripheral off the I2C bus. If it will run for a couple days, then you can almost be certain the level of pull-up is too strong.
I believe @ssozonoff has a different issue. However, I will absolutely need access to my lab to reproduce his findings and trace the signals.
Thanks, have a good trip, looking forward to hear what you find.
As a aside do you guys publish change logs with regards to hardware revisions, for example between the Notecarrier-F V1.0 and V1.3 ?
@ssozonoff I finally got a second to revisit this, and I’m curious about your non-working setup.
Were you using the Notecarrier-F v1.3 and a BME280 with no host MCU? That would explain the problem. The Feather 3V3 (F_3V3), provides the power (Vcc) to the Qwiic connectors. Without the Feather it cannot work.
The Notecarrier-F v1.0 had a slightly different power configuration, which you helped us identify and straighten up for the Notecarrier-F v1.3.
@zfields - Hi Zak – could you please elaborate on your statement about “The Notecarrier-F v1.0 had a slightly different power configuration, which you helped us identify and straighten up for the Notecarrier-F v1.3.”?
I had the very same question (for a different issue) as @ssozonoff above – re: “…do you guys publish change logs with regards to hardware revisions, for example between the Notecarrier-F V1.0 and V1.3 ?”
I know that I could find the schematics on GitHub and compare the two versions, but for the benefit of one who doesn’t have a background in electronics, could you possibly explain some of the rationale and implications of the major changes between versions and the pros and cons of those changes and improvements?
Including comments like that in the repo might be very helpful in troubleshooting when issues do arise. Apologies if such comments already exist in the revision history - please point me to them if so.
