[OpenTRV-dev] Wire formats from low-power sensor leaf nodes to more powerful upstream (concentrator) node

[Damon Hart-Davis]

On 6 Nov 2014, at 09:23, Yodit Stanton <EMAIL ADDRESS HIDDEN> wrote:

> Sounds great, will block out a few hours towards the end of next week, does that suit you timewise?  Yours is probably the most challenging use case so might as well start with that :) 

Ah, but mine os also the least settled in some ways at this stage, anyhow, yes, end of next week is so far looking good!


> 
> Currently an example schema.org looks like this 
> <div itemscope itemtype ="http://schema.org/Movie
> ">
>  <h1 itemprop="name"&g;Avatar</h1>
>  <div 
> itemprop="director" itemscope itemtype="http://schema.org/Person"
>> 
>  Director: <span itemprop="name">James Cameron</span> (born <span itemprop="birthDate">August 16, 1954)</span>
>  </div>
>  <span itemprop="genre">Science fiction</span>
>  <a href="../movies/avatar-theatrical-trailer.html" itemprop="trailer">Trailer</a>
> </div>
> 
> 
> 
> on the other hand the Qubit guys have gone for tagging 
> 
> digitalData = {
> pageInstanceID: "ProductDetailPageNikonCamera-Staging",
> page:{
> pageInfo:{
> pageID: "Nikon Camera",
> destinationURL:
> "http://mysite.com/products/NikonCamera.html"},
> category:{
> primaryCategory: "Cameras",
> subCategory1: "Nikon",
> pageType: "ProductDetail"},
> attributes:{
> Seasonal: "Christmas"}
> },
> product:[{
> productInfo:{
> productName: "Nikon SLR Camera", 
> sku: "sku12345",
> manufacturer: "Nikon"},
> category:{
> primaryCategory: "Cameras"},
> attributes:{
> productType: "Special Offer"}
> }]
> }; 
> 
> Neither are appropriate for us due to their bulkiness but we can learn from them.  I don’t have any fully formed ideas and want to experiment with a few things and see what happens.

Right, but something that is becoming clear to me is that leaf nodes can be very terse and allow more powerful concentrators (eg small Linux boxes) to expand the detail, and at that point size may not be an issue.


> 
> How shall we collaborate?  Do you want to send me what you have currently and what you would like it to look like? 

Let’s work through an example from your area and one from mine, and fold in some auth/encryption issues.

> 
> We should maybe schedule an open hangout next wed or thur evening with anyone that is interested and trash out some ideas?  I know the qubit guys so happy to invite them to understand their experiences (or maybe thats another hangout session) 

Everyone could at the very least join the OpenTRV IRC chat on Thursday evening if that works!

Thanks

Damon

> 
> Regards,
> 
> 
> 
> 
> 
> On 6 November 2014 09:02, Damon Hart-Davis <EMAIL ADDRESS HIDDEN> wrote:
> Hi,
> 
> Well, I am still pottering about with this stuff at the leaf-to-node level right now, so maybe we can work through some real examples from my side too?  Would be good to have more eyes and brains on it.
> 
> My main on-the-wire constraints:
> 
>  * For various reasons I need my entire frame to sit in << 60 characters, and have bandwidth and duty-cycle limits
> 
>  * I will need to selectively authenticate or encrypt some data
> 
> But also note that I do NOT need to transmit all fields in each TX.
> 
> Rgds
> 
> Damon
> 
> 
> On 6 Nov 2014, at 08:34, Yodit Stanton <EMAIL ADDRESS HIDDEN> wrote:
> 
>> Just getting back to thinking about this.
>> 
>> How about the equivalent of Schema.org for IoT?  So your json or whatever payload can just point to a pre defined items list.  I am going to do a thought experiment with this in the next couple of weeks with a few sensors and will show you what I mean.
>> 
>> Regards,
>> 
>> 
>> 
>> Yodit
>> 
>> On 19 October 2014 16:04, Damon Hart-Davis <EMAIL ADDRESS HIDDEN> wrote:
>> To be clear; from the concentrator onwards some more standard protocols/formats would be preferable; I’m really only trying to minimise the pain of the first hop from a low-power sensor.
>> 
>> Eg the concentrator could act as an MQTT proxy (what sort of heresy and ignorance am I indulging in here).
>> 
>> Rgds
>> 
>> Damon
>> 
>> PS. The more I think about it the more I like the unique node ID plus a simple list/map of (name, scalar value) pairs with selectively signed or encrypted parts for the TEXT form.  The interpretation of a data frame depends on the node and concentrator IDs and timestamp.  But maybe I can be talked into JSON at least!
>> 
>> On 19 Oct 2014, at 15:41, Damon Hart-Davis <EMAIL ADDRESS HIDDEN> wrote:
>> 
>>> Hi,
>>> 
>>> Thinking out loud here, so not necessarily making sense…
>>> 
>>> I’m trying to do ensure that when OpenTRV and/or any sensor platform based on it is sending data points from leaf/sensor/actuator nodes to a more powerful concentrator node there are a couple of possible mechanisms with the following features:
>>> 
>>> 1) TEXT: The ability to instantly add arbitrary new sensor types’ outputs to the data stream for quick development and without any predefined schema, and without blowing up an existing unupgraded receiver, and to be able to see what is going up the wire preferably (eg easy debugging and storage).  Maybe a node ID plus an ASCII (ASCII-7 printable only) list of name/value or name/value/units.  Syntax might be JSON or something even more terse.  Generating and sending extra bytes has a significant code-space and energy (eg CPU and radio) cost, but the ability to instantly add new values to a data stream in a robust way may make that worthwhile.  Note that maximum frame sizes are likely to be very constrained by node memory, radio duty-cycle regs, etc.  Note that this form plus may with some decoration (eg with concentrator ID and timestamp) may be directly suitable for long-term storage and sending over the Internet.  Note that printable ASCII may eliminate the need to bit-stuff or whiten on some carriers.
>>> 
>>> 2) BINARY: The ability to instead/also use a very compact binary wire form, eg once a particular set of sensors has been settled on for a particular deployment and available engineering skills can handle this.  Logically equivalent to TEXT form but possibly hand-crafted for size/cost and robustness.  Should ideally be convertible to/from TEXT FORM freely at various places in the data path, with a tight version control of the schema expected.
>>> 
>>> 3) The possibility in some cases of ‘signing’ some or all of the data points at the leaf or at the concentrator.
>>> 
>>> 4) The reverse path to a leaf node (if present) should always support the BINARY option but may also support a limited subset of the TEXT form.
>>> 
>>> Thoughts?
>>> 
>>> If JSON were to be used, then surely some stylistic choices should be made, eg flat list/map with/without units?
>>> 
>>> Damon
>> 
>> 
>> 
>> 
>> --
>> CEO & Founder | opensensors.io
>> The Operating System for the Internet of Things
>> @opensensorsIO
>> 
>> @yoditstanton
>> 
>> 
>> 
>> 
>> Open Sensors Ltd  is a registered company in England & Wales
>> 3rd Floor, 65 Clifton Street
>> London
>> EC2A 4JE
>> 
>> No 08773413
>> Vat: 188665055
> 
> 
> 
> 
> -- 
> CEO & Founder | opensensors.io
> The Operating System for the Internet of Things
> @opensensorsIO
> 
> @yoditstanton
> 
> 
> 
> 
> Open Sensors Ltd  is a registered company in England & Wales
> 3rd Floor, 65 Clifton Street
> London
> EC2A 4JE
> 
> No 08773413
> Vat: 188665055