Hi guys
After more than a year, Starnote for Skylo continues to frustrate with erratic and unreliable communication. I keep thinking this product is only one FW update away from working “properly”, but even now, with all my devices running the latest firmware (LTS 10.1.1.17497 or 11.1.1.17494), at least a third or more devices in the field seem to hang and not communicate after a while, despite high-end antennas and plenty of battery remaining. Rebooting seems to get them going again, but only for a short time.
The most recent device to come back to the bench had not communicated for about a week. It remained powered when I connected it to the web CLI. It seemed responsive to most commands, but completely unresponsive to a hub.sync command. See attached log. It would receive a Note request from my MCU and acknowledge it, but would not act on the subsequent hub.sync command. After no response for an hour, I issued a manual hub.sync and still no response. After another hour, I finally rebooted the device and it connected almost immediately and uploaded ~50 Notes.
What is going on with Starnote and are we ever going to get it to communicate reliably? Right now, it still seems a half-baked product to me.
I’m really sorry to hear that you’re continuing to encounter problems with Starnote. I really appreciate you reporting the issues to us so we can make the product better. Your feedback here on the forums has truly been invaluable. Thanks for sticking with us.
One thing I’m noticing looking at the attached trace log is a lack of ntn.status messages coming from the Starnote prior to the reboot. This makes me wonder if the Starnote has gotten into a bad state and isn’t even communicating with the Notecard. If you are able to get another one of your devices into this state, could you keep it in that state (don’t reboot), and try sending {"req":"ntn.status"} to the Notecard via the in-browser terminal? The response, or lack thereof, will help us narrow down the source of the problem.
Additionally, if you are able to capture a long-running trace log so that we can see when exactly the Starnote stops communicating with the Notecard, that would be helpful, too.
Hi Haydon,
Thanks for chiming in - much appreciated! Attached is a continuation of my previous log. After a successful sync at “time”:1768439806, there is another Note added and sync request at “time”:1768446027. This one is also not executed. For around 19 hrs afterwards, there is no communication captured in the in-browser terminal whatsoever. I then did a {“req”:“ntn.status”} and the Notecard responded with {“status”:“{ntn-idle}”}. It was unresponsive to a few other commands before I decided to do {“req”: “card.restart”}. From there it restarted and sync’d.
In addition, about 10 days ago our diagnostic logging identified a timeout issue in our satellite packet processing pipeline that was impacting your application. This timeout was due to the lengthy throttle you have set in some of your routes. Specifically, in some routes you are sending a maximum of one event every 15 seconds. Obviously, this shouldn’t impact your incoming events but it appears it may have caused the Starnote to enter into a stuck state. Your work with Hayden should let us identify that firmware issue.
In any case, we are rolling out a fix for the packet pipeline timeout issue next week. I believe this will cure the majority if not all of the problems you are seeing.
We appreciate your feedback. Starnote is incredibly important to us and we want to make sure it works as reliably as any of our Notecards. Thank you for your patience and your help.
You sound quite confident you found the likely culprit. I certainly hope you are right. I’ve been experimenting for months trying to find out why some devices go dark for no obvious reason. The 15s route limitation is of course a Thingspeak limitation for their free tier. But I’m no longer on the free tier, so I have reduced the route rate to 2s. Hopefully this will do it. I will keep logging.
Hi @rberkelm. I do think there is a lingering firmware issue at play here. I am actively working to reproduce the failure mode you’re seeing. I will keep you posted.
Which request interface is the host MCU using to talk to the Notecard? (I’m assuming I2C, but just checking)
What Notecarrier are you using, or are you using a custom one?
The Starnote hasn’t been heard from since the host went to sleep. Can you verify the Starnote VIO is connected to the Notecard VIO and that the Starnote VMODEM is connected to the Notecard VIO — meaning that the Starnote power is in no way “switched”?
VIO pins of Notecard and Starnote are all connected to 3V3; VMODEM pins of Notecard and Starnote are all connected to Lipo battery; VBUS pin of Notecard is connected to USB connector, but Starnote VBUS pin is floating.
For the last 2 days, there have been no hick-ups. All comms and Notes delivered as expected. I’m still logging.