#339 HisWrite Op modification

Shawn Jacobson Thu 19 Nov 2015

I would like to propose a modification to the HisWrite Op to allow for multiple recs to have their histories written at the same time. To provide backward compatibility, I propose that we allow for an id to not be present in the metadata of the grid. If it is not present, then the grid itself should contain a 3rd column of "id". This provides backward compatibility, but also allows a system to consolidate multiple hisWrite calls into a single transaction. I have written up the proposed change below...

HisWrite

The hisWrite op is used to post new time-series data to historized point(s). The point(s) must already be configured on the server and assigned unique identifier(s).

Request (single point): a grid metadata must define id Ref of point to write to. The rows define new timestamp/value samples to write with following columns:

  • ts: DateTime timestamp of sample in point's timezone
  • val value of each timestamp sample

Request (multi point): The rows define the id Ref and new timestamp/value samples to write with following columns:

  • id: Ref of the point being written to
  • ts: DateTime timestamp of sample in point's timezone
  • val value of each timestamp sample

Response: empty grid

Clients should attempt to avoid writing duplicate data. But servers must gracefully handle clients posting out-of-order or duplicate history data. The timestamp and value kind of the posted data must match the entity's configured timezone and kind. Numeric data posted must either be unitless or must match the entity's configured unit. Timezone, value kind or unit conversion is explicitly disallowed.

Examples:

Here is an example which posts some new history data to a single point:

// request
ver:"2.0" id:@hisId
ts,val
2012-04-21T08:30:00-04:00 New_York,72.2
2012-04-21T08:45:00-04:00 New_York,76.3

Here is an example which posts some new history data to multiple points:

// request
ver:"2.0"
id,ts,val
@hisId1,2012-04-21T08:30:00-04:00 New_York,72.2
@hisId2,2012-04-21T08:45:00-04:00 New_York,76.3

Brian Frank Fri 20 Nov 2015

I don't think that is the right design - if you have lots of data, especially if its aligned by timestamp then it results in a huge amount of duplication. Rather I would propose that batch writes specify the id in their column meta, like we do when you read out SkySpark data:

ver:"2.0"
ts,v0 id:@a,v1 id:@b
2012-04-21T08:30:00-04:00 New_York,72.2,76.3
2012-04-21T08:45:00-04:00 New_York,N,76.3

If a given history doesn't have a value for that timestamp, then it would just be null (as shown with N zinc encoding above)

Shawn Jacobson Mon 23 Nov 2015

That makes sense, thank you. Can we get it added to the specification?

Mark Oellermann Wed 25 Nov 2015

+1 for this - we have a need to support multi-point hisWrite, and also multi-point hisReads. Can we look at updating both at the same time and aligning the calling conventions?

Alex Afflick Mon 7 Dec 2015

+1 it would be great to get added to the specification. I will open a new thread for a proposed approach to multi-point hisReads as per Mark's suggestion.

For completeness:

HisWrite

The hisWrite op is used to post new time-series data to historized point(s). The point(s) must already be configured on the server and assigned unique identifier(s).

Request (multi point): a grid with with a row for every new timestamp/value(s) to write with the following columns:

  • ts: DateTime timestamp of sample in point's timezone
  • v0: value of the first timestamp sample
  • v1: ...

For each point, a value column is named v0, v1 etc, with a Ref identifier of the writable point defined in the column metadata. A null cell is used to denote a given point without a value for that timestamp.

Response: empty grid.

Example of a multi point hisWrite request:

// request
ver:"2.0"
ts,v0 id:@a,v1 id:@b
2012-04-21T08:30:00-04:00 New_York,72.2,76.3
2012-04-21T08:45:00-04:00 New_York,N,76.3

Login or Signup to reply.