We have a test prototype with a North American CAT-1 Notecard out in the field. It could be a fluke, but after syncing notes consistently every 5 minutes, but got our last note at 23:58 UTC on Sunday night. Can you think of anything special about UTC midnight, particular the midnight of Sunday-to-Monday November 12-to-13, 2023? Is it possible Blues changed something in the response format, did some sort of automated Notecard update, or some other black (blue?) magic? This is just too much of a coincidence!
1 Like
Hi @daterdots,
There were no Notehub outages during that time - is there any chance you were monitoring voltage on the device and it simply ran low? Is the device back online now?
Rob
We were monitoring voltage. Battery was fine. We just got the hardware back and were able to determine there was some sort of Rust panic. Now we’re trying to figure out what caused it.
We have continued testing, and this happened again on the Sunday to Monday transition from December 24->25 at UTC midnight. It’s possible this is our firmware, but I really doubt it. Our firmware doesn’t know what midnight is, let alone what day of the week it is.
Hi @daterdots - While this isn’t likely the culprit, I’m curious if you know what version of firmware is running on the Notecard? Upgrading to the latest is not a bad idea.
4.4.1.4015700 is what’s on the Notecard. That was the latest as of May, but I see there’s 4.4.2 as of a few weeks ago. I can upgrade, but I doubt that’s it!
Oh wow! We figured this out. It’s a bug in the STM32 RTC. For 1 second on the Sunday-to-Monday transition, the RTC returns 7 for the day of the week. This is an invalid day. Days should be 0-6.
583120.449035 ERROR panicked at src/main.rs:152:45: called `Result::unwrap()` on an `Err` value: InvalidDateTime(InvalidDayOfWeek(7))
It turns out it is the output from the RTC that is wrong. It returns day of week as 7 for some reason.
And this happened exactly on unix timestamp 1705881600
1 Like
A little more color:
We are setting the internal STM32 RTC from the notecard time so it is synchronized to UTC time. The RTC has separate registers for year, month, day and day of the week. For some reason the day of week register overflows on Sunday to day 7 for just a second. The assertion that it should only have values of 0-6 fails and results in an error.
We are not using the rtc.now() function that accesses the day of week resgister and has the assertion in our production firmware. So this is not a problem for our production firmware.
This problem still should be investigated and submitted up stream to embassy for a potential fix.
2 Likes
It turns out this was an embassy issue. A classic off by one. It is now fixed in embassy-stm32! 
1 Like