I would have to think this is possible.
I am developing a control panel application on a rpi4, touch screen and notecard.
The resulting python files are modular and relatively small. Is there some kind of guidelines for doing some kind of REST GET or some other method of transferring small files through the notecard into a folder on the pi?
I could technically send them 1 line at a time I suppose, breaking on the /n and appending to a file, but that seems messy?
Is there any best practice way of accomplishing this in a secure fashion?
1 Like
Hey @vstadmin great question. You can do this using the Notecard web.get api documented here. The Notecard must be in continuous mode, and you’ll need to break the file into 4-8k chunks (depending on the strength of your connection) and re-assemble on the Pi-side.
The endpoint you retrieve the files from also has to return JSON as the Notecard doesn’t support other content types. My suggestion would be to serve the files in a JSON body with each chunk as a base-64 encoded string, then reassemble on the Pi once you have all the chunks you need.
Hope that helps.
The file size limit is larger than 4k with the over-the-air device firmware update (DFU) mechanism. I’ll see if I can track down a safe limit but I know we have examples greater than 42KB.
The Host DFU size limit is at least 700KB FYI.
I did see these settings and limitation, but I thought they were specific for DFU transfer via the Blues Cloud of binary files to supported devices.
How would I use base64 to transfer small python text files from a web server to the Raspberry PI connected to the Blues Modem?
Hi @vstadmin that’s correct, those limitations are specific to host DFU, which uses the Notecard to store a binary before delivering it to the host. And while it is possible in theory to use the DFU mechanism to obtain Python files, as discussed in this thread it’s not something that we have tested, so I cannot guarantee it will work.
If you’re looking to pull down .py files from a remote source to get into the filesystem of your device, the web.get approach is your best bet.
I just want be able to use the Blues modem and do remote updates in the field. I’m just puzzling through the options.
It would be the same with using any micropython on circuit python device with a Blues modem, I would think? They all have to be updated from time to time in the field.
1 Like
Yes it would, although I think you have a bit more flexibility on the Pi because you have direct filesystem access and can easily reassemble files from a remote location and update the appropriate file location directly. In Micro/CircuitPython, this process would require creating a custom bootloader in a host-native language to handle updating the Python files.
Hope that makes sense. If it would be helpful, I am happy to create a detailed breakdown of how I think this could work and/or a sample app to illustrate it.
That would be immensely helpful. I think it would probably benefit a lot of people as well as we attempt to buy other hardware and get hit with 58 week lead times and have to use Raspberry Pi’s.
I really do appreciate it!
Happy to help @vstadmin. I will start putting together something in the next few days and hopefully can have something to share next week.
Any chance that you have come up with a process for uploading python code through the blues modem?
Hey @vstadmin I am working on it, but don’t have anything to share yet. Will try to have something ready by the end of this week. Fingers crossed.
I’m going into a project meeting in about 15 minutes. Any ideas yet?
Ideas and progress! I’ll follow-up with details and a sample later today.
In case you are curious, here is the process the sample I am building will follow:
- A
.py file is uploaded to an S3 bucket.
- The bucket triggers a Lambda function which uses the Notehub API to send an inbound Note to the Notecard.
- The Notecard detects the inbound Note and fires the
ATTN pin.
- The Host (Raspberry Pi) detects the ATTN pin is HIGH and performs a
web.get request against an endpoint that is running another Lambda function.
- The function retrieves the file from S3 and performs a base64 encoding on the contents. That Base64 string is returned as a payload to the Notecard. The function breaks files larger than 4k into chunks and can be pinged multiple times to retrieve the entire file if over 4k.
- If multiple calls are needed to retrieve the entire file,
web.get is called until the file has been downloaded.
- Take the base64 string for all chunks, decode and save to disk.
- Restart the main python program file to reload the changed source file.
I’ll be providing examples for both updating a utility file, and the main program file.
S3 and Lambda are just example in this case and can be swapped in for whichever provider you prefer.
Thanks for the reply. I’m in the meeting now. I can share that we have several “ideas”
Hey @vstadmin here’s a working sample for you to reference. It follows the process above and the full details are here: note-samples/python-file-downloader at main · blues/note-samples · GitHub.
Hope it’s useful!
Thank you! I will enjoy digging through this!
I really appreciate this. I will be looking at it over the next couple of days
@bsatrom - I have been looking at this part of our project (finally) and have had another person looking at it for about a week. I’m an intermediate level programmer who has made considerable progress in the last month on our main monitoring program to monitor and report pressure from a vapor recovery system for the gasoline distribution industry.
That being said, we do not have a lot of experience with AWS or their version of “lambda”. I understood “lambda” to be an anonymous function that I have only touched a couple of times in programming tutorials. Your python guide for AWS seems to be extremely comprehensive and would be well suited for our needs, but we just cannot figure it out.
There seems to be something missing in the glue that connects all the functions together between the web hooks and the functions and we cannot really determine which function goes where and how they all inter-relate.
Is there a simplier way perhaps? one were we just drop a file in any accessible location and the send a note to card to begin the download process? We have our own linux server currently exposed on port 80 and responding to web hooks from particle and is being used to receive and collect the data from units already. I could easily add a python script there to received a command from a blues webhook.
Any ideas?
looking at the notifier code, you have:
def lambda_handler(event, context):
project = os.environ['PRODUCT_UID']
device = os.environ['DEVICE_UID']
I do not understand how a specific notecard actually gets notified that there is a new file waiting.
How does the notifier get the project and device within the scope of the AWS lambda function?
