Note-arduino and ESP32 serial with custom pins

We recently had to switch our project from using I2C to using serial for communicating with the Notecard.

Naturally, I anticipated this would be pretty straightforward, but unfortunately that was far from the experience I got.

The ESP32 family offers flexible pin assignments to peripherals, such as the UARTs, allowing peripherals to be used with almost any choice of IO pins. Our choice was guided by the available pins, whether they can be controlled by the uLP processor and other hardware constraints.

note-arduino tries to own the Serial instance, calling Serial.begin()/Serial.end() and then Serial.begin(). This happens during the call to Notecard.begin() where the serial connection is reset.

This obviously works fine when the default pins are used, but not when custom pins are used.

HardwareSerial on ESP32 has a setPins() method which allows the developer to override the default pins. Unfortunately, this configuration is cleared when HardwareSerial.end() is called, resulting in the default pins being used rather than the pins intended.

The workaround is to create a subclass of NoteSerial that handles the serial port, with knowledge of how to correctly set it up.

class NoteSerialConfig : public NoteSerial {
    unsigned rxPin;
    unsigned txPin;
    HardwareSerial& serial;
public:
    NoteSerialConfig(HardwareSerial& hw) : serial(hw), rxPin(-1), txPin(-1) {}

    void setPins(unsigned rxPin, unsigned txPin) {
        this->rxPin = rxPin;
        this->txPin = txPin;
    }

    bool begin() {
        serial.begin(9600, SERIAL_8N1, rxPin, txPin);
        return true;
    }

        /**************************************************************************/
    /*!
        @brief  Determines if the Notecard Serial port has data available.
        @return The number of bytes available to read.
    */
    /**************************************************************************/
    virtual size_t available(void) {
        return serial.available();
    }

    /**************************************************************************/
    /*!
        @brief  Read a byte from the Notecard Serial port.
        @return A single character byte.
    */
    /**************************************************************************/
    virtual char receive(void) {
        return serial.read();
    }

    /**************************************************************************/
    /*!
        @brief  Resets the serial port.
        @return `true` if the Serial port is available.
    */
    /**************************************************************************/
    virtual bool reset(void) {
        serial.end();
        return begin();
    }

    /**************************************************************************/
    /*!
        @brief  Writes a message to the Notecard Serial port.
        @param    buffer
                  The bytes to write.
        @param    size
                  The number of bytes to write.
        @param    flush
                  Use `true` to flush to Serial.
        @return The number of bytes transmitted.
    */
    /**************************************************************************/
    virtual size_t transmit(uint8_t * buffer, size_t size, bool flush) {
        size_t result = serial.write(buffer, size);
        if (flush) {
            serial.flush();
        }
        return result;
    }
};

Unfortunately this is mostly boilerplate code, since NoteSerial_Arduino is a final class and can’t be extended. If it weren’t final, only the begin() method would need to be overridden.

You can use a global instance of this if Notecard.end() is not called, or allocate one dynamically, when Notecard.end() is used, which attempts to deallocate the instance.

// globals
HardwareSerial notecard_serial(1);   
NoteSerialConfig noteSerialConfig(notecard_serial);
Notecard notecard;

During setup(), we can then set up the pins for the serial connection, and pass it to Notecard:

    noteSerialConfig.setPins(PIN_NOTECARD_RX, PIN_NOTECARD_TX);
    notecard.begin(&noteSerialConfig);

Ideally note-arduino would be a little more friendly with non-default serial settings, but for anyone who’s struggling with this, I hope this workaround helps.

Hi @devElert ,

The problem comes from the ESP32’s not standard Serial::begin() implementation.

note-arduino passes a callback to note-c that relies on the Serial::begin() and Serial::end() functions to properly reset the Serial peripheral.

A better description and solution are provided in the following post:

Please have a look, and see if it will suit your needs.

Best,
Zak

Hi again @devElert,

We were workshopping a workaround, and we decided it would be best for you to temporarily eliminate the Reset callback in your application by using the following strategy:

extern serialResetFn hookSerialReset;
hookSerialReset = NULL;

Then as an official solution in the next release of note-arduino (coming soon), we are going to update note-c and note-arduino to have “getter” functions for the callbacks.

This would allow you to do something similar to the following…

serialTransmitFn transmit;
serialAvailableFn available;
serialReceiveFn receive;

NoteGetFnSerial(NULL, &transmit, &available, &receive);
NoteSetFnSerial(NULL, transmit, available, receive);

You could pass in NULL as the reset callback (shown above), or implement a custom reset callback that manages your custom pin selection.

Cheers,
Zak

2 Likes

To follow up, note-arduino has now been updated to support the syntax shown above.

note-arduino version 1.6.4 will be available from the Arduino Library Manager shortly.

1 Like

I tried using this in with the 1.6.6 release, but unfortunately wasn’t able to get it to work.

   HardwareSerial notecard_serial(1);


setup() { 
...
    // set up the GPIOs to use for UART1.  These are remembered until end() is called.
   notecard_serial.setPins(boardPins.PIN_NOTECARD_TX_P, boardPins.PIN_NOTECARD_RX_P);
    

    serialTransmitFn transmit;
    serialAvailableFn available;
    serialReceiveFn receive;
    // get the original serial functions, and clear the reset function so that the configuration of the UART isn't overitten.
    NoteGetFnSerial(NULL, &transmit, &available, &receive);
    NoteSetFnSerial(NULL, transmit, available, receive);
    notecard.begin(notecard_serial);

During notecard.begin() the serial hooks are reset to their default values, including the reset function.

void Notecard::begin(NoteSerial * noteSerial_)
{
    noteSerial = noteSerial_;
    platformInit(noteSerial);
    if (noteSerial) {
        NoteSetFnSerial(noteSerialReset, noteSerialTransmit,   // <--- here
                        noteSerialAvailable, noteSerialReceive);

        // Set the default debug serial throttling
        J *req = NoteNewRequest("card.aux.serial");

Btw, I’m not in a hurry to get a fix for this - I’m happy to continue using a subclass of NoteSerial (code in the first post) that handles begin() and end() as needed.

Ah, yes. I see the problem.

To create a proper solution, it will take both changes to note-c and note-arduino.

I will get to work on that and report back.

Thanks for bringing this up!
Zak

Any Update on this? I am facing the same issue and im wondering if i should wait for a change in the library or attempt one of the fixes people did

1 Like

Hello @JakobStruebing ,

Welcome to the Blues forum! I’m so sorry I missed your question.

Yes and no, we did update note-c and I attempted to update note-arduino, but I missed an important piece. I will push an update out shortly.

In the interim, you should be able to call Notecard::begin() before you update the callbacks, and that would ensure your preference is saved. In the next release, the order will no longer matter.

Happy hacking,

Zak

Hello all,

I’ve released a new version of note-arduino, and now Notecard::begin() should no longer reset any callbacks you’ve supplied directly.

Be sure to let me know if this doesn’t resolve your issue.

~Zak

1 Like

I understand what you say, but that is some rather dense language. Could you please share an example that’s been tested on the ESP family of chips?

Hello @devElert ,

I finally slowed down and looked at the example you provided above.

setup() { 
...
    // set up the GPIOs to use for UART1.  These are remembered until end() is called.
   notecard_serial.setPins(boardPins.PIN_NOTECARD_TX_P, boardPins.PIN_NOTECARD_RX_P);
    

    serialTransmitFn transmit;
    serialAvailableFn available;
    serialReceiveFn receive;
    // get the original serial functions, and clear the reset function so that the configuration of the UART isn't overitten.
    NoteGetFnSerial(NULL, &transmit, &available, &receive);
    NoteSetFnSerial(NULL, transmit, available, receive);
    notecard.begin(notecard_serial);

I completely overlooked that the current behavior of Notecard::begin() will set all NULL values to the default serial callbacks. So in your case, you would want to call Notecard::begin() before your calls to NoteGetFnSerial() and NoteSetFnSerial().

    notecard.begin(notecard_serial);
    NoteGetFnSerial(NULL, &transmit, &available, &receive);
    NoteSetFnSerial(NULL, transmit, available, receive);

This would allow the default values to be set, then you would immediately unset them. I expect this will solve your problem.

A better solution yet is to initialize the callbacks to an invalid pointer, instead of NULL. Then I will be able to differentiate unset pointers from a pointer deliberately set to NULL, as in your case.