Payload missing after re-routing note

Notehub.io
1

This is the situation:

  1. I receive a note from my device that contains a base64 encoded payload

  2. The note is immediately routed to my web app automatically

  3. My web app receives the payload part of the note

  4. I check Notehub UI. I don’t see the payload part of the Note in the Notehub UI.

  5. I reroute the note.

  6. The re-routed note arrives at my app, but it is missing the payload part of the Note.

Now I:

  1. Turn off automatic routing

  2. Wait for a fresh note to appear in Notehub from my device.

  3. I see the payload in the Notehub UI

  4. I manually route the note

  5. I see the payload in my web app

  6. I still see the payload in Notehub

  7. I re-route the note manually for a second time

  8. I still see the payload in both Notehub and my app

So, something about automatic routing is erasing the base64 payload from my notes on Notehub. I only noticed this because I had to erase the last few days of data from my database. When I went to re-route all those Notes, none of the payloads came through.

2

Hey @daterdots,

What you’re seeing is intentional Notehub behavior, though I agree it’s confusing (and I actually learned about it for the first time from your post).

In short, Notehub deletes any payload over 256 bytes after it has been successfully routed through an “automatic” route. This was a design decision we made in the early days of Notehub because payloads can be quite large.

I’m going to add this behavior to our docs, and also bring it up for discussion internally. For now though, unfortunately your payloads are indeed gone :frowning:

Appreciate you taking the time to write this up so thoroughly, and sorry for the confusion.

TJ

3

Oh jeez. That’s a real shame. So that data is toast? I’ve always thought of the Notehub as a temporary cache of raw data in case of situations like this where I need to recompute some recent chunk of ingress.

4

Unfortunately, yes :frowning: I just checked with our cloud team and those payloads are not included in our backups either.

I’ve always thought of the Notehub as a temporary cache of raw data in case of situations like this where I need to recompute some recent chunk of ingress.

Same, and that’s how we want you to think of Notehub. Unfortunately this is an exception to that because of how we’re specifically handling payloads. I’m starting a discussion about this internally.

TJ