When was this $0.0006 charge for routing implememented? I did not see that when we started using blues and I don’t remember seeing it recently.
I have a system that will be sending data every 10 sec to our servers that will require 172,800 “events” per month which comes to over $100 / device per month!
Totally unworkable. I would swear that charge was not there recently.
May 4, 2021:
June 3, 2021:
When I started this project in March, I was led to believe via the documentation that there was not siginficant charge for events. Now suddently this new pricing sheet tells me it will cost me OVER $100 / device per month! I might as well quit right now.
This is EXACTLY what Particle did to us that caused us to find Blues as an alternative.
Hi @vstadmin,
Unless your solution truly requires low latency event streaming (every 10 seconds as you said), you can dramatically lower the number of routed events by either:
{"all_data":[
{"temp":23, "time":1000},
{"temp":24, "time":1001},
{"temp":25, "time":1002},
{"temp":26, "time":1003},
]}
Let me know if you have any follow up questions!
Thanks,
Rob
I am making some changes to accommodate accumulating data on a different timer than sending the data to notehub.io.
Here is what I had before:
def fifteen_sec_updates():
'''
These are the things to do once every 15 Seconds
Collect Data
'''
global cont
to_modem = {key:cont[key] for key in ['pressure','runcycles','faults','mode','temp']}
if DEBUGGING:
logging.debug(f'Modem - Sending: {to_modem}')
### Output for blues modem
req = {"req" :"note.add"}
req["file"] = "vstdata.qo"
req["sync"] = True
req["body"] = to_modem
If I change it to this:
def fifteen_sec_updates():
'''
These are the things to do once every 15 Seconds
Collect Data
'''
global cont
to_modem = {key:cont[key] for key in ['pressure','runcycles','faults','mode','temp']}
if DEBUGGING:
logging.debug(f'Modem - Sending: {to_modem}')
### Output for blues modem
req = {"req" :"note.add"}
req["file"] = "vstdata.qo"
req["sync"] = False
req["body"] = to_modem
and then add:
def ten_minute_updates():
'''
These are the things to do once every 10 minutes
Blues has changes the rules on talking to their HUB, so I have changed
these routines to hit their hub once every 10 minutes instead of every 15 seconds
but i'm accumulating the 15 second data
'''
global cont
req["sync"] = True
try:
rsp = card.Transaction(req)
except:
logging.error('No response from modem. Will keep trying...')
will this accomplish accumulating data every 15 seconds to then be later sent every 10 minutes? Of course I have a main timing loop that fires these functions off at the appropriate time.
unfortunately, I set this to be:
req = {"req": "hub.set"}
req["product"] = productUID
req["mode"] = "periodic"
req["sync"] = True
req["outbound"] = 1
and it is accumulating every 15 seconds and syncing once every minute, but its registering a separate event for each record which defeats the purpose.
More ideas?
I really don’t know how to accumulate more data inside one note. What are the limitations on number of bytes / blocks, etc.
I think I see what you’re getting at.
Essentially you want req["body"] to be set to an array of dicts.
It is the note.add that counts as the event, not the sync. (Admittedly a little confusing)
My Python is a little weak, so pardon me if there’s some syntax errors in my snippet here:
to_modem= []
MaxNumDataPointsBeforePushToNotecard = 40 # 4x per minute for 10 minutes
global cont
data_to_store = {key:cont[key] for key in ['pressure','runcycles','faults','mode','temp']}
to_modem.append(data_to_store);
if(to_modem.len() >= MaxNumDataPointsBeforePushToNotecard ):
if DEBUGGING:
logging.debug(f'Modem - Sending: {to_modem}')
### Output for blues modem
req = {"req" :"note.add"}
req["file"] = "vstdata.qo"
req["sync"] = False
req["body"] = {"data":to_modem}
to_modem= []
Thank you, I will look at this in detail.
What is the maximum amount of data and overhead that can be sent in a note?
Is 250 bytes the limitation on the the body? So if my data was 25 bytes, I could only store 10 records before I ran out of space? This is going to be very difficult.
The “payload” field byte limit is 250
The “body” doesn’t have that limit
The total size for the note is limited to ~ 4KB
I strongly recommend using note.template to describe the structure of your Notes prior to adding Notes to the Notefile.
That is very good to know. How can you check to see the entire note size before you send it.
In Python you can always check the size of an object with sys.getsizeof.
I can get the size of the block I create, but can I get the size of the overall JSON that notehub will receive?
Checking the size of the JSON object you are sending is a good start, but you’re right it doesn’t tell 100% of the story.
The only way to perfectly measure the total amount of data sent to Notehub is to use the card.usage APIs, documented here: Low Bandwidth Design - Blues Developers
Good morning.
I am attempting to use note templates.
i have the following defined:
modem_template = {"seq":11, "pressure":12.1, "runcycles":12, "faults":12, "mode": 11, "temp": 12.1 }
req = {"req" : "note.template"}
req["file"] = "vstdata.qo"
req["body"] = {"data" : modem_template}
rsp = card.Transaction(req)
print(rsp)
then in my loop, I am attempting to concatenate 4 records with:
data_to_store = {key:cont[key] for key in ['seq','pressure','runcycles','faults','mode','temp']}
modem.append(data_to_store)
if cont['seq'] == 3:
req = {"req" :"note.add"}
req["file"] = "vstdata.qo"
req["sync"] = True
req["body"] = {"data" : modem}
rsp = card.Transaction(req)
print(rsp)
pprint(modem)
modem = []
cont['seq'] = 0
else:
cont['seq'] = cont['seq'] + 1
I get the correct data:
[{'faults': 196,
'mode': 0,
'pressure': -0.14,
'runcycles': 20108,
'seq': 1,
'temp': 0.0},
{'faults': 196,
'mode': 0,
'pressure': -0.14,
'runcycles': 20108,
'seq': 2,
'temp': 0.0},
{'faults': 196,
'mode': 0,
'pressure': -0.14,
'runcycles': 20108,
'seq': 3,
'temp': 0.0}]
but when I print(rsp), I get:
{'err': 'error adding note: object expected because of template'}
I also tried just setting my template to:
req["body"] = modem_template
and got:
{'err': 'error adding note: body fields not found in template: data'}
as my error.
I feel i’m close, but 'm just spinning my wheels at this point.
Incidentally, my code DOES work without the template. I would love to get it working with the template if that will save on data or bandwidth.
Editing this response to refer folks to @gwolff’s response below.
Note templates actually aren’t appropriate for arrays of data (since the structure of the array is really unknown…you could send 1 element or 100). I would stick with the original method of adding notes without using a template.
Ok, I was just following earlier concepts and would really like to remove any redundancy. I guess I can shrivel by fieldnames down to a single character.
I disagree on avoiding the templates in this case.
My concern has to do with the nuance of how templated Notefiles are handled in the sync process versus untemplated Notefile Notes. Untemplated Notes use more RAM in the sync process, as they must be serialized before the sync. For large Notes this could be a problem.
Templates permit JSON arrays, but they must be fixed length. However, they are compressed and sent as a binary blob, and then serialized back to JSON on the server, saving Notecard resources.
I will follow up with a template example.