Is there a tag to indicate that a point's history is pulse sum? After a a brief look i didn't see anything, but i'm sure it's there somewhere.

Do you mean like this: http://project-haystack.org/tag/hisTotalized

Or something else?

I was thinking more a pulse count point where it is read and then zeroed out, so that each sample is the number of pulses since the last. Not sure if this is common or not.

But regarding the hisTotalized, should this be associated with a maximum value? My understanding is that running-sum points typically reset at some value, and it is useful to know when that is.

Interesting thought on the hisTotalized and the maximum value.

If I understand this right, it will just look at the diff between 2 values. So if it reached it max value and rolled over and started from zero again, there would potentially be one data point (one difference) that was huge and way out of line (and negative)

Maybe it could just check to see if a new value is lower than the previous value and then know that it has rolled over?

Maybe it could just check to see if a new value is lower than the previous value and then know that it has rolled over?

Yes, that is what one would do if you didn't know the rollover value. But, say the rollover is at 1000 (exclusive), the last value read was 998, and the new value is 5. Knowing that the max is 1000 means you can calculate the delta exactly, i.e. 7. Otherwise you have to guess. I suppose it's really a matter of how accurate your data needs to be. But since these max values are usually known it would seem to be a simple thing to add, say, hisTotalizedMax or some such.

You're right. I've mostly dealt with accumulated values that are designed to only roll over after many years, so it would be an extreme rarity.

But if rollovers are a common occurrence, this could really start to become painful.

Since Haystack is a pretty high level HTTP and text protocol, seems like most servers should be to abstract this away so that clients don't have to deal with it. From what I have seen typically they keep a rollover counter and can do the math when they return the history data. Putting the burden on servers versus the client would be my vote, since I expect their will be many more client implementations. But if we thought that wasn't feasible, then might make sense to create a hisTotalizedRollover tag (or something like that).

I support Brian's comment that it should not be the users problem to convert Pulses to energy for meters. The meter should provide the calculations for energy consumed and the rate of consumption. Time stamps for consumption rates are also significant. Reading pulses is raw data and requires site specific factors to convert to consumed energy.

Sorry, i'm confused. In a perfect server where it is not the user's problem to convert pulses, we would not have any need for a hisTotalized tag, no? If we have this tag, it would also be necessary (although as Jason said, hopefully rarely) to have a hisTotalizedRollover tag in order to compute accurately. I'm not sure if you are proposing to do away with hisTotalized.

Regardless, i'd suggest that we shouldn't. We shouldn't presume that Haystack will always be used as a front to a perfect server. It's conceivable to use Haystack in a service oriented architecture where it acts only as a configuration reference system, without any real-time data. In such a case it is necessary to give client systems all of the information they need to properly handle the data.

I don't really have a strong opinion on the topic. If we have a couple votes for creating hisTotalizedRollover tag, I'm on board with that. Or we can just let it lie for now, and I'm okay with that too.

I think this a really interesting discussion as it highlights the differences between the perfect model we are trying to create and what actually happens in the real world.

I have experience with working on a 40 site metering project that was exclusively done with pulse meters. Water, electricity and gas were monitored across Australia. Our company used our own local technicians to do the installation and setup of this system and even with our best efforts the variation in how the pulse meter data was collected was huge.

Ranging from some deployments where the pulse count rollover was handled perfectly in the local controllers to deployments where no handling was implemented and it was expected the server collecting the data would "sort it out", to everything in between.

I guess my point is that Haystack shouldn't assume a perfect world and there are some scenarios where it would be nice if there was flexibility.

I wish I could dictate to everyone working on all our sites how they did things, but in reality it's just not going to happen. To know that a historised point has a certain rollover value and that I should also count how many times it has rolled over is valuable data.

Interesting. How would you store how many times a point has rolled over? If you are storing the point counts, can you not infer? (Assuming there are not breaks in the data so long that you may have missed a rollover, but then if this is the case there are probably bigger problems that need to be solved.)