Received notes getting corrupted

Firmware
1

Hi All,

I am moving my Arduino-like project to use Zephyr and C. It is mostly working but something wrong with the deserialization functions.

The notecard message is as expected:

[INFO] {“req”:“note.get”,“file”:“command.qi”,“delete”:true,“crc”:“002A:32E87BAA”}
[INFO] {“time”:1736184879,“body”:{“cmd_id”:“3D88EDBE-505C-45AE-8969-1FFF3FC21F15”,“cmd_type”:“f”,“dev_id”:“scan”}}

I try to deserialize the message the way I always have:

J *body = JGetObject(rsp, “body”);
cmdtype = JGetString(body, “cmd_type”);
devid = JGetString(body, “dev_id”);
cmdid = JGetString(body, “cmd_id”);

I end up with:

cmdid: €UË?ðñÉ?-505C-45AE-8969-1FFF3FC21F15
cmdtype: ÐñÉ?ÐñÉ?h
devid: scan

The cmd_type field, which should be a single character, is always replaced by 8 bytes. They are always similar, e.g., 208-241-201-63-208-241-201-63, 4 bytes repeated.

The first 8 bytes of cmd_id are sometimes overwritten.

The dev_id field is never overwritten regardless of the order the fields are deserialized.

I tried adding 10 leading dots to the fields:

[INFO] {“time”:1736184524,“body”:{“cmd_id”:“…B626C449-E1D3-4693-B41D-166B67FA16E9”,“cmd_type”:“…f”,“dev_id”:“…scan”}}

The result was consistent with the original:

cmd_id: øñÉ?øñÉ?..B626C449-E1D3-4693-B41D-166B67FA16E9
cmd_type: ØñÉ?ØñÉ?..f
dev_id: …scan

I can only conclude that there is a problem with the note-c Zephyr library. Any clues would be much appreciated.

Thanks in advance,

Rob

2

Hi @rlkeith,

Would it be possible for you to share the code that is causing these issues? It would be good to understand what the context of the deserialization is. Also I assume you are using I2C for communication between the Notecard and your Host?

Thanks,
Alex

3

Hi @abucknall,

Thanks for the reply. There is not much at the moment. I have sent notes to the Notehub via the Notecard without any trouble. I am using an ESP32C3 device with Zephyr and set it up exactly (as far as I can see) according to the Blinky + Notecard example in the documentation.

The code I am using to access the Notecard does a bunch of other stuff. It includes the necessary Include bits:

#include “note_c_hooks.h”
#include <note.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdarg.h>
#include <stdio.h>
#include <zephyr/kernel.h>
#include <zephyr/drivers/gpio.h>

There is a function to initialise the Notecard:

void initNotecard()
{
NoteSetUserAgent((char *)“note-zephyr”);
NoteSetFnDefault(malloc, free, platform_delay, platform_millis);
NoteSetFnDebugOutput(note_log_print);
NoteSetFnI2C(NOTE_I2C_ADDR_DEFAULT, NOTE_I2C_MAX_DEFAULT, note_i2c_reset, note_i2c_transmit, note_i2c_receive);
}

The code to get the note is the same as I used in PlatformIO for the Swan:

int getNextCommand()
{
char * devid;
char * cmdid;
char * cmdtype;
char cmdChar;
bool cmdValid = false;
int retint = 0;
J *req = NoteNewRequest(“note.get”);
JAddStringToObject(req, “file”, “command.qi”);
JAddBoolToObject(req, “delete”, true);
J *rsp = NoteRequestResponse(req);
if(rsp)
{
if(NoteResponseError(rsp))
{
printk(“No notes available\n”);
}
else
{
J *body = JGetObject(rsp, “body”);
cmdtype = JGetString(body, “cmd_type”);
devid = JGetString(body, “dev_id”);
cmdid = JGetString(body, “cmd_id”);
cmdValid = true;
}
}
if(cmdValid)
{
printk(“cmd_id: %s\n”, cmdid);
printk(“dev_id: %s\n”, devid);
printk(“cmd_type: %s\n”, cmdtype);
printk(“rubbish characters are: %d-%d-%d-%d-%d-%d-%d-%d\n”, cmdtype[0], cmdtype[1], cmdtype[2], cmdtype[3], cmdtype[4], cmdtype[5], cmdtype[6], cmdtype[7]);
}
}

Is there a way I can get the note Body as a text string. That would be useful to see which stage the corruption is happening.

Regards,

Rob

4

@rlkeith to clarify, the code you are using in PlatformIO with the Swan, is that using Arduino + note-arduino or is that also using Zephyr + note-c?

Thanks,
Alex

5

Hi @abucknall ,

The old project does not use Zephyr. It is similar to Arduino. That always “just worked.”

Regards,

Rob

6

@rlkeith I’ve replicated (with the hardware I have available) your inbound Notefile example using a Swan MCU and a Notecarrier F. I’m seeing this print out from your getNextCommand() function.

*** Booting Zephyr OS build v4.0.0-2802-g722c95846188 ***
[INFO] main(): Initializing...
i2c: Device is ready.
[INFO] {"req":"hub.set","product":"com.blues.abucknall:test","mode":"continuous","sn":"zephyr-blink","body":{"agent":"note-zephyr","compiler":"gcc 12.2.0","req_interface":"i2c"},"crc":"0000:D1A57069"}
[INFO] {}
[INFO] {"req":"hub.sync","crc":"0001:10BAC79A"}
[INFO] {}
[INFO] {"req":"note.get","file":"command.qi","delete":true,"crc":"0002:32E87BAA"}
[INFO] {"time":1736256218,"body":{"cmd_id":"3D88EDBE-505C-45AE-8969-1FFF3FC21F15","cmd_type":"f","dev_id":"scan"}}
cmd_id : 3D88EDBE-505C-45AE-8969-1FFF3FC21F15
dev_id : scan
cmd_type : f
rubbish characters are :  102 -  0 -  108 -  101 -  114 -  34 -  58 -  34

This looks to be what I would expect, so I’m wondering if we could try/check a few things:

  1. Can you try this main.c with a Swan MCU and a Notecarrier F (If you have one)
  2. Can you give me some more context regarding how your hardware is connected? You mention you are using an ESP32C3, how is that attached to the Notecard?
  3. Which version of Zephyr RTOS are you using?
7

Hi @abucknall That is what I would expect.

I am using Zephyr 3.7.8, a WiFi Notecard and a Notecarrier B, connected to a Seeed Xiao ESP32C3 using I2C. The board only has one available serial port and I am using that for monitoring. I want to use it for DFU later.

I don’t have a Swan handy. The one I do have is buried in storage and would take me days to get. I am not very mobile. I also have a bunch of other boards in the same place.

Is there any way to convert the body to a “string” so I can see if the body is uncorrupted?. I could then parse it to get out of this problem and keep working.

Regards,

Rob

8

Hi @rlkeith,

I’ve just caught up with the email thread discussing this in parallel with @Youssif. I’ve ordered a Seeed Xiao ESP32C3 to test this myself, which should be with me early next week.

Can I ask if you’ve been able to test the main.c I shared earlier, with the ESP32C3, is the result the same? I see from the email thread that you’ve had some issues working with the note-c helpers that are included in the note-zephyr repo.

I’m actually in the middle of overhauling note-zephyr to improve usability; my colleague is reviewing it and we should be able to merge it shortly. This will turn note-zephyr into a Zephyr module making considerably easier to use, along with improved examples.

I’ll keep you up to date with my progress as hardware arrives, please let me know if you have any updates!

Thanks,
Alex

9

Hi @abucknall

I found a Swan but I couldn’t find the programmer. I was about to try to set up the Particle Xenon, which has the nRF52840, and for which I do have the programmer, and see what results that gives.

One gotcha I have is that the Xiao ESP32C3 I have (from PiHut) has hardware version 0.4. The current HW version is much higher, and I have ordered some more things from Seeed.

I am still not proficient with switching boards with Zephyr. If you have a project you are using I could test the ESP32C3 with that.

Regards,

Rob

10

Hi again @abucknall

I have now got the hang of building with Zephyr for different boards. The Particle Xenon (nRF52840) works perfectly.

I have some Espressif dev boards. I will see if any of those work with Zephyr and give them a try.

Regards,

Rob

11

Hi yet again @abucknall

I tried it with another ESP32 board. It worked fine. Then I tried it with the original Xiao ESP32C3 board and its worked fine as well.

The project I am using to test this is just the original Blinky with the Blues libraries added in and my function to get message.

The Blues note-c libraries are in the clear. It looks like one of the Bluetooth functions I am using is stepping on few bytes of RAM.

Regards,

Rob

12

That would be a plausible explanation for the buffer corruption!

Thanks for letting me know, would you like me to update you when we’ve merged the changes to the Zephyr module? Hopefully it will make your development process a little easier!

Thanks,
Alex

13

Hi @abucknall

It’s odd but that problem has entirely gone away now that I have moved back to the original project. It may have been the order in which I called things. I will keep an eye out for further evidence.

I have a new problem with note-c for C/Zephyr. I’ll post it as a new thread.

Regards,

Rob

14

Hi @abucknall

The problem is back exactly as before. It went away for a while when I left the board unplugged for a few hours.

I have a work-around, which is to precede all strings with a fixed number of characters. That will do for development but is not ideal long term.

Regards,

Rob

15

Hi @abucknall

Fixed it, I think.

If you declare a string and then assign JGetString to it like this:

char *rstr;
rstr = JGetString(body,“somestring”);

It might get corrupted in strange and unpredictable ways.

If you do this:

char *rstr = JGetString(body,“somestring”);

It seems to work.

Rob

16

Hi @rlkeith ,

That sounds odd; I should have hardware arriving today to investigate it. If you can share your main.c code, I’ll try to replicate this on my side and attempt to narrow down what this might be related to.

Alex

17

Hi @abucknall

My fix seems to work faultlessly, so it is no longer a problem for me. It might bite someone else on the posterior in future. This problem also does not happen all the time, so it is hard to replicate easily.

It re-occurred again after the MCU got into a boot loop and needed to be manually reset and put into boot mode, after which I loaded the firmware again, so it an edge case IMO.

My bigger problem is the inability to create notes with long strings or binary loads. See Problem adding large strings to note requests with C/Zephyr

Regards,

Rob