Questions/suggestions about the Notehub firmware API and UI

[devElert]

It’s great that Notehub now supports notes on firmware. I recently updated our release and deployment workflow to add release notes from CHANGELOG.md to github releases, and include them when deploying to notehub.

But after re-releasing, the order of the firmware in the list was changed (using the default ordering.) Does the Firmware API allow a build date/release date to be provided? That would allow idempotent releases. (Notehub could still track created date and modified date.). The binaries do include the ::firmwre::info block used with IAP DFU updates.

Is it possible to patch an existing release, say by amending the notes, or metadata, without affecting the other fields and uploaded firmware?

Does the firmware API support DELETE? I don’t see it in the OpenAPI schema from notehub-js. Having that, with a mandatory confirmation query parameter might help prevent accidental deletes. I’ve had the deploy fail after uploading firmware, and wanted to delete the firmware as part of the workflow (especially if the failure was the wrong md5 returned.)

In this case, I just manually deleted the firmware in the UI. It would be helpful if the firmware delete UI included the details for the firmware that would be deleted (the same fields as shown in the firmware list.) It can be easy to mis-click, and then there’s no clue in the popup dialog which firmware was selected and will be deleted.

Does a particular firmware have a unique ID? I don’t see that in the API response. That would be useful for the DELETE and PATCH methods.

Thanks for reading!

[tjvantoll]

Hey @devElert,

Thanks again for your feedback and reaching about all this. I’m going to answer your questions in a weird order.

Is it possible to patch an existing release, say by amending the notes, or metadata, without affecting the other fields and uploaded firmware?

Does the firmware API support DELETE?

You can only update and delete existing binaries through the UI.

But we really should allow both through the API.

I’ll request this internally.

Does a particular firmware have a unique ID?

We treat the filename as the unique ID.

And actually we’re publishing a new API request this week that’ll let you download binaries by doing a GET on /firmware/host/{filename}. We’ll likely use that same path to support updates and deletes.

But after re-releasing, the order of the firmware in the list was changed (using the default ordering.) Does the Firmware API allow a build date/release date to be provided? That would allow idempotent releases. (Notehub could still track created date and modified date.). The binaries do include the ::firmwre::info block used with IAP DFU updates.

Ok, coming around to this one. The firmware API directly only accepts a version. Your two options today are to toss a timestamp at the end of that version (e.g. 1.0.0.1770821900474), or to provide a built timestamp as part of your firmware::info block as shown here.

I think using a timestamp in your version will work for you because it’ll ensure you get a consistent sorting order, but if I’m misunderstanding let me know.

TJ

[devElert]

Thanks, that’s illuminating, and good to hear about the upcoming API to retrieve firmware binaries. We’re currently doing that via github releases from a webapp in gh-pages, but that has CORS issues which requires jumping through hoops to get around. (But we jumped and got around!)

Ah of course, I didn’t think about the filename as the unique ID. Using the firmware filename is definitely workable since it’s also in the response of the firmware upload PUT request. (I know it has a defined format, but it’s kind of opaque and created by Notehub - not the same as the original filenames given, for understandable reasons, like uniqueness. If I may say, though, I’ve always found them a bit hard on the eyes. )

We do have a built timestamp in the firmware::info, and these are visible in the metadata when expanding a firmware row in Notehub. But I’m not seeing that it has any influence on default ordering (and there’s no built column in the table to select that ordering nor in the API, which was updated about a month ago with varoius sort fields, but not built.)

I appreciate you looking for workable solutions, and suggesting using the version with an appended date. Is that handled specially by Notehub? If I’ve understood you correctly, and the version is sorted lexicographically or by increasing semver, then I don’t quite understand how the date would make any difference: 1.1.0.<date> will always come before 1.2.0.<date> no matter what the dates are when sorting on version.

I don’t need an immediate fix for this, was just wanting to share my experience in case there’s opportunity to address these. Having PATCH support would resolve the ordering, it’s only because I couldn’t do that and had to re-release that the order by creation date no longer gives the correct ordering.

And finally, one more feature request - kind of a long shot this one - firmware notes with formatting!! Markdown or html as well as plain text. We have a webapp that can be used to update firmware, and the firmware note includes a link to this app, prepopulated with the version to flash. It would be a sweet UX if users could just click the link rather than copy/paste it from the notes. Also, the notes have various sections (much like Notecard release notes) and bullet points. Some formatting there might help, although that’s a secondary concern. We could always just add a link to the github release, which has formatted release notes.

Thanks for indulging me, anything from this that you can put to the team as a feature request is much appreciated!

[devElert]

Seems I’m on a roll of making requests, so, how about grouping of firmware? Our project has 2 different firmware types at present, and that will grow - for example, production firmware, provisioning firmware, hardware QA firmware, embedded test suites and demo firmware. At present, these are mixed up when displayed as a monolithic list.

Being able to group on one or more fields from the metadata json would help organize a long list of firmware, especially as the versions are not in alignment across them, since they have separate development and release cycles. This might be useful for notecard also, to help separate developer releases from LTS releases. For host firmware, the grouping and their sort order could be part of the project settings.

[devElert]

One thing I noticed is that firmware notes are probably not intended for a full change log due to the way they are displayed, both in the firmware overview list, and in the popup that appears when choosing to update a device’s firmware.

The firmware overview list displays ok, with each one being expandable (although the notes aren’t visible in one screen because of their length and being confined in width to one column of the table.)

When choosing firmware for a device, the Apply Host/Notecard Firmware popup is quite narrow, so all the text is squeezed into a relatively narrow space. Also it seems that the line breaks are removed, although it’s still functional - just means a lot more scrolling to move between the versions.

I have a few suggestions that might help show what’s changed for a firmware version without creating these huge blocks of text.

• Use the first paragraph of the notes as a summary. This could be displayed in the Apply Host/Notecard Firmware popup dialog instead of the full notes, making better use of the limited space. Similarly, the firmware overview table could display the first sentence of the notes in a column when collapsed. Expanding the row shows the full notes as it does now, while also using the column that the collapsed view was using to display the summary, allowing more width for the notes in the table.

• Allow a hyperlink or other info to be added to an “external info” field. The hyperlink could be used to link to external resources. The firmware overview and the “Apply firmware” popup could show this link. This would also be helpful to be able to review the release notes for host and Notecard updates before choosing to update the firmware.

[tjvantoll]

Hey @devElert ,

I appreciate you looking for workable solutions, and suggesting using the version with an appended date. Is that handled specially by Notehub? If I’ve understood you correctly, and the version is sorted lexicographically or by increasing semver, then I don’t quite understand how the date would make any difference: 1.1.0.<date> will always come before 1.2.0.<date> no matter what the dates are when sorting on version.

We do sort the results lexicographically. I was suggesting using 1.2.0.<timestamp>, which does work if you use the Notehub API’s /firmware?firmwareType=host&sortBy=version request. But this does feel kind of hacky since the last digit of the version is intended to be a build number, and we only support 32-bit integers (so using a timestamp might not even work right).

The easier solution is probably just to let you sort by built.

And finally, one more feature request - kind of a long shot this one - firmware notes with formatting!! Markdown or html as well as plain text. We have a webapp that can be used to update firmware, and the firmware note includes a link to this app, prepopulated with the version to flash. It would be a sweet UX if users could just click the link rather than copy/paste it from the notes. Also, the notes have various sections (much like Notecard release notes) and bullet points. Some formatting there might help, although that’s a secondary concern. We could always just add a link to the github release, which has formatted release notes.

I did a little playing with this today. Since notes accepts a string you can provide markdown or html and Notehub will accept it, even if it doesn’t display it elegantly. But if the GET /firmware API returned notes (it doesn’t today, and it should) external apps could render the Markdown, which I think might be a good-enough solution.

I’ve looped the Notehub team into this discussion so they see the full feedback you’re left in here. I’m going to specifically request sortBy=built and notes being returned from /firmware since those seem like small things we should definitely do.

TJ

[devElert]

Thanks for doing some research, and for looping in the Notehub rockstars. After getting the notes working, I realized they were too big for inline display, particularly when selecting firmware for a device. An additional link to release notes would also suffice if that were possible. Then the firmware notes can be focused on any deployment instructions rather than a change log (say prerequisites - you must update to this version before using this release.)

The included link could direct to a page with more expanded context (full release notes, more detailed deployment instructions, etc..) which would be in whatever format, so the firmware notes don’t necessary need to render the formatting. To be concrete, for our use-case, the link would be to the corresponding GitHub release.

If PATCH ends up being supported for editing firmware notes, could the same be done for the JSON metadata? That would be useful for including machine-readable details, such as prerequisites.

Thanks again for listening and your time on this, appreciated!