Based on some performance issues with nhaystack and a bit of ambiguity with the watchPoll op, I'd like to propose the following clarifications and enhancement to the watchSub op:
The watchSub request grid meta can contain a tag named tags which is a comma separated list of tag names. If specified, then only these tags are returned by subsequent watchPoll ops (the id tag is always implied to be included). If the tags meta is omitted then watchPoll should return the entire list of tags for an entity when any change occurs.
Only one subscribed tag list is maintained per watch. If additional watchSub calls to an existing watch specify a tags list, then it overwrites any current list. If additional watchSubs omit this value, then the last value specified is assumed.
Mike JarmyTue 27 May 2014
I like the idea of the tags meta-data on the watchSub request -- that will certainly allow for slimming down the amount of information that is sent back with each watchPoll response.
I'd like to suggest that if the tags meta is omitted, then watchPoll should default to only return id, curVal, and curStatus, rather than returning every single tag. This would make the default poll behaviour on the server be to only return the information that is usually actually needed, rather than a very large amount of information that will most likely be discarded once the client gets the response.
One could perhaps argue that specifying id, curVal, and curStatus as the default set of tags in a watchPoll response is a breaking change to the existing spec, but the existing spec doesn't actually specify which tags should be returned :-). Is anyone in the user community using any other tags on the client side from the response to a watchPoll?
If for some reason you really wanted every single tag for each COV, then we could specify an allTags meta on watchSub, which would be mutually exclusive with the tags meta.
Tightening up the spec in this way will help solve a performance problem with polling in the current nhaystack release.
Mike JarmyTue 27 May 2014
One other thing we should consider is specifying that the tags meta specifies the minimum set of tags that should be returned. Implementations could be free to return other tags as well.
This would allow nhaystack to return the various AX status tags, like axOverridden, etc, etc
Richard McElhinneyTue 3 Jun 2014
+1 for this too.
As we move forward I think it will be important to have a tight specification that is clear to everyone. It may be ok to have some grey areas when we're dealing with just Niagara as a reference implementation, but moving forward we would like to see more people adopt this so continuing to improve these definitions is important.
Brian FrankWed 4 Jun 2014
Ok thinking on this some more, this a more refined proposal:
watchSub can specify grid.meta tags
tags is either * or a comma separated list of tag names
if tags is ommitted, then the server defines its own tags list for polling which must include curVal and curStatus
the server returns a entity's tags for each poll if any changes have been made to the entity regardless of the specific tags being polled (poll changes are on an per entity basis, not a per tag basis)
if multiple watchSubs are issued against the same watchId, then the tags of the last request wins
With this design servers are free to design a reasonable default to keep performance fast in case client doesn't specify an explicit tag list.
Mike JarmyFri 13 Jun 2014
The build of NHaystack that J2 just released, 1.2.1, doesn't quite address what we discussed in this thread. What it does for now is always just return id, curVal, and curStatus, no matter what you ask it to do :-).
A future release will address this issue more fully.
Brian Frank Tue 27 May 2014
Based on some performance issues with nhaystack and a bit of ambiguity with the watchPoll op, I'd like to propose the following clarifications and enhancement to the watchSub op:
The watchSub request grid meta can contain a tag named
tags
which is a comma separated list of tag names. If specified, then only these tags are returned by subsequent watchPoll ops (theid
tag is always implied to be included). If thetags
meta is omitted then watchPoll should return the entire list of tags for an entity when any change occurs.Only one subscribed tag list is maintained per watch. If additional watchSub calls to an existing watch specify a
tags
list, then it overwrites any current list. If additional watchSubs omit this value, then the last value specified is assumed.Mike Jarmy Tue 27 May 2014
I like the idea of the
tags
meta-data on the watchSub request -- that will certainly allow for slimming down the amount of information that is sent back with each watchPoll response.I'd like to suggest that if the
tags
meta is omitted, then watchPoll should default to only returnid
,curVal
, andcurStatus
, rather than returning every single tag. This would make the default poll behaviour on the server be to only return the information that is usually actually needed, rather than a very large amount of information that will most likely be discarded once the client gets the response.One could perhaps argue that specifying
id
,curVal
, andcurStatus
as the default set of tags in a watchPoll response is a breaking change to the existing spec, but the existing spec doesn't actually specify which tags should be returned :-). Is anyone in the user community using any other tags on the client side from the response to a watchPoll?If for some reason you really wanted every single tag for each COV, then we could specify an
allTags
meta on watchSub, which would be mutually exclusive with thetags
meta.Tightening up the spec in this way will help solve a performance problem with polling in the current nhaystack release.
Mike Jarmy Tue 27 May 2014
One other thing we should consider is specifying that the
tags
meta specifies the minimum set of tags that should be returned. Implementations could be free to return other tags as well.This would allow nhaystack to return the various AX status tags, like axOverridden, etc, etc
Richard McElhinney Tue 3 Jun 2014
+1 for this too.
As we move forward I think it will be important to have a tight specification that is clear to everyone. It may be ok to have some grey areas when we're dealing with just Niagara as a reference implementation, but moving forward we would like to see more people adopt this so continuing to improve these definitions is important.
Brian Frank Wed 4 Jun 2014
Ok thinking on this some more, this a more refined proposal:
tags
*
or a comma separated list of tag namescurVal
andcurStatus
With this design servers are free to design a reasonable default to keep performance fast in case client doesn't specify an explicit tag list.
Mike Jarmy Fri 13 Jun 2014
The build of NHaystack that J2 just released, 1.2.1, doesn't quite address what we discussed in this thread. What it does for now is always just return
id
,curVal
, andcurStatus
, no matter what you ask it to do :-).A future release will address this issue more fully.