Is there any community effort to build and auto-tagging tool? Meaning, a software tool that applies tags with little or no human involvement.
I know of at least two commercial analytics vendors who are or have built such a tool, but its dedicated to their product. I know of several University or DOE funded research efforts around this problem/task as well.
Such a tool would reduce the cost of deploying analytics as tagging is one of the most time consuming tasks in the deployment process.
Let me know if anyone else is interested in coordinating an effort to create such a tool.
John PetzeThu 5 Oct 2017
A few thoughts that may be helpful:
Wanted to make you aware of the Project-Builder Plus tool which has been donated to the community as open source. It includes many tools to streamline and automate tagging and accepts a range of data types as input. You can find details here: https://project-haystack.org/forum/topic/467
If you are working with Niagara systems the nHaystack tool does part of the work automatically by interpreting a range of factors from Niagara points.
We see a number of our SkyFoundry partners building tools for auto tagging to meet there project needs. Some might be of general use and some may be interested in sharing. Asking the community for comments here is a good step.
Aaron SanchezThu 5 Oct 2017
Terry,
Regarding the auto-tagging tool you've seen in two different vendors, is the auto-tagging done via the names of the points or how does the tool know which tags to apply to each point?
Christian TremblayThu 5 Oct 2017
I've added this feature to nHaystack in the latest versions. Based on the name of the point, the module will look at a CSV file and add the tags corresponding to a point name.
This format allow easy sharing of tags based on names.
The module also allow to modify the file to be used so one can switch between files if necessary.
Really useful when using always same names in our programs. Or for manufacturing using always the same names in their products.
If the tool works based off the names of the points. We, at J2 Innovations, have created a tool for this to work with FIN|Stack. It also references a CSV file and can be edited by each SI to fit their point nomenclature.
Terry HerrTue 10 Oct 2017
There are a number of tools that can apply tag values from an internal library, via pattern extraction from regular expression text search of the point name.
Better tools will take advantage of more than than the point name. Most bacnet drivers / discover tools can also extract - point type, units, description, present value,etc from the controller. And these can be used to help decipher / understand the point and add tags.
The best tools will also use historical time series data, machine learning methods, and an ever expanding dictionary as it learns from more and more data sets. The ideal is a tool that can completely and fully tag a data set without human involvement.
Machine learning methods like - multi layer feature extraction, support vector machine, logistic regression, latent semantic indexing..
There are a number of researchers from DOE labs, UofC Berkely and Davis, that have done some fundamental research in the development of such a tool. We have one tool that was developed by United Technology Research Center and DOE funded that is a reasonable start, but needs completing. The tool is written in python.
Aaron SanchezThu 12 Oct 2017
Terry,
Everything you're describing is possible with FIN|Stack, we've just never created the tool ourselves. However, our Fin Framework, allows any user to build virtually anything they want on top of what's already built within FIN|Stack. Obviously you'll have to know axon to do some of these things but we've made it easier for people to launch tools for their functions with the Form Builder 2.0 documentation page.
Christian TremblayThu 12 Oct 2017
As we can see, there are a lot of solution allowing to tag automatically points. As a community though, would it be viable that some kind of "sharing" be possible for basic auto-tagging ?
I'm wearing my integrator hat here. If I have some kind of CSV file (that can be used or modified to be used by any tool) with tags that I would just need to review, what a time saver.
I'm pretty sure everyone is doing its own list actually.
For this I work with variable names has those are typically standard for manufactured products...
This would not prevent the integrator to "validate and verify" that the tagging has been done correctly, but this certainly be valuable information.
A versionning system like github would be very cool to handle those files and anyone could participate doing pull requests.
Aaron SanchezThu 12 Oct 2017
That's a great idea in theory. If we had a csv file for each vendor's point naming structure so that anyone using a system where auto-tagging was possible, they can pull the appopriate csv file. Also, if there were new point names there weren't on the list, anyone can add to each respective csv file and push so that it is up to date. I'm a fan of this idea.
Terry HerrFri 13 Oct 2017
Christian & Aaron,
What you describing is a good start - an every growing look-up library/dictionary of standard point names from OEM equipment vendors, and BAS vendors with the appropriate tags added. In our experience OEM vendors do have point name standards, but only Johnson Controls has anything approximating a standard. Most other BAS products allowed the tech-on-the-bucket to create his own point names. The latter case is where you need more sophisticated algorithms using more than just the point name to decipher.
Project Haystack is great example of the community working together for collective benefit. The tagging standard is great, now we need to reduce the time and cost to apply them to a data set. I have seen an example of one tool and heard of another that can ingest (auto-discover) a BAS, apply tags, and then the appropriate rules/algorithms automatically apply. I am sure this will always require some human oversight /vetting, but it would be minimal.
Gabe FierroSat 14 Oct 2017
Hi all:
I'm a PhD student at UC Berkeley working on building metadata, analytics and controls. Just wanted to jump in and provide some pointers to some of the academic work in this area:
The BuildSys conference is a growing community of computer scientists, building scientists, mechanical/civil engineers, etc interested in these kinds of topics, so you can search past programs to find other related work.
The systems I linked above aren't "production ready", but they do offer promising avenues of being able to automatically extract tags from existing metadata and timeseries data. Terry, you mentioned that JCI has something like a standard for these point names. Is this written down/published anywhere that I could take a look? In the buildings I've worked with, there's been very little standardization around point names beyond common idiomatic abbreviations such as VAV and AHU. Point names usually betray more about the BMS network than they do about the actual physical relationships (such as which chiller connects to this AHU)
Christian TremblayMon 16 Oct 2017
@Terry, I don't work a lot with Lonworks anymore but I remember their variables names being the same for standard applications like VFD or VAV...
Terry HerrMon 16 Oct 2017
Gabe,
those are good articles. I found them a while back. That conference looks interesting. I am not aware that Johnson published anything regarding their standard. We have lots of BAS data sets, and many are Johnson. The standard I am referring to is typically only the last acronym in the longer point name which as you note also includes network controllers acronyms.
Christian, LON was much better than BACnet at defining and requiring standard snvt's (point) naming https://www.lonmark.org/technical_resources/resource_files/snvt.pdf
I believe BACnet is adopting equipment profiles and standard naming, which will make this problem less important some years down the road. In the meantime we are stuck with thousands of buildings with non standard point names.
Terry Herr Thu 5 Oct 2017
Is there any community effort to build and auto-tagging tool? Meaning, a software tool that applies tags with little or no human involvement.
I know of at least two commercial analytics vendors who are or have built such a tool, but its dedicated to their product. I know of several University or DOE funded research efforts around this problem/task as well.
Such a tool would reduce the cost of deploying analytics as tagging is one of the most time consuming tasks in the deployment process.
Let me know if anyone else is interested in coordinating an effort to create such a tool.
John Petze Thu 5 Oct 2017
A few thoughts that may be helpful:
Aaron Sanchez Thu 5 Oct 2017
Terry,
Regarding the auto-tagging tool you've seen in two different vendors, is the auto-tagging done via the names of the points or how does the tool know which tags to apply to each point?
Christian Tremblay Thu 5 Oct 2017
I've added this feature to nHaystack in the latest versions. Based on the name of the point, the module will look at a CSV file and add the tags corresponding to a point name.
This format allow easy sharing of tags based on names.
The module also allow to modify the file to be used so one can switch between files if necessary.
Really useful when using always same names in our programs. Or for manufacturing using always the same names in their products.
https://bitbucket.org/richiemac_77/nhaystack/src/b070b8e2ee7c8ea122878456d467c82e83697872/README.md?at=ax&fileviewer=file-view-default
Aaron Sanchez Thu 5 Oct 2017
If the tool works based off the names of the points. We, at J2 Innovations, have created a tool for this to work with FIN|Stack. It also references a CSV file and can be edited by each SI to fit their point nomenclature.
Terry Herr Tue 10 Oct 2017
There are a number of tools that can apply tag values from an internal library, via pattern extraction from regular expression text search of the point name.
Better tools will take advantage of more than than the point name. Most bacnet drivers / discover tools can also extract - point type, units, description, present value,etc from the controller. And these can be used to help decipher / understand the point and add tags.
The best tools will also use historical time series data, machine learning methods, and an ever expanding dictionary as it learns from more and more data sets. The ideal is a tool that can completely and fully tag a data set without human involvement.
Machine learning methods like - multi layer feature extraction, support vector machine, logistic regression, latent semantic indexing..
There are a number of researchers from DOE labs, UofC Berkely and Davis, that have done some fundamental research in the development of such a tool. We have one tool that was developed by United Technology Research Center and DOE funded that is a reasonable start, but needs completing. The tool is written in python.
Aaron Sanchez Thu 12 Oct 2017
Terry,
Everything you're describing is possible with FIN|Stack, we've just never created the tool ourselves. However, our Fin Framework, allows any user to build virtually anything they want on top of what's already built within FIN|Stack. Obviously you'll have to know axon to do some of these things but we've made it easier for people to launch tools for their functions with the Form Builder 2.0 documentation page.
Christian Tremblay Thu 12 Oct 2017
As we can see, there are a lot of solution allowing to tag automatically points. As a community though, would it be viable that some kind of "sharing" be possible for basic auto-tagging ?
I'm wearing my integrator hat here. If I have some kind of CSV file (that can be used or modified to be used by any tool) with tags that I would just need to review, what a time saver.
I'm pretty sure everyone is doing its own list actually.
I would see something like :
johnson_control_standard_variables.csv (DA-T : temp, sensor, air, discharge; etc...) ABB_drive_tags.csv CompanyX_HeatPump.csv
For this I work with variable names has those are typically standard for manufactured products...
This would not prevent the integrator to "validate and verify" that the tagging has been done correctly, but this certainly be valuable information.
A versionning system like github would be very cool to handle those files and anyone could participate doing pull requests.
Aaron Sanchez Thu 12 Oct 2017
That's a great idea in theory. If we had a csv file for each vendor's point naming structure so that anyone using a system where auto-tagging was possible, they can pull the appopriate csv file. Also, if there were new point names there weren't on the list, anyone can add to each respective csv file and push so that it is up to date. I'm a fan of this idea.
Terry Herr Fri 13 Oct 2017
Christian & Aaron,
What you describing is a good start - an every growing look-up library/dictionary of standard point names from OEM equipment vendors, and BAS vendors with the appropriate tags added. In our experience OEM vendors do have point name standards, but only Johnson Controls has anything approximating a standard. Most other BAS products allowed the tech-on-the-bucket to create his own point names. The latter case is where you need more sophisticated algorithms using more than just the point name to decipher.
Project Haystack is great example of the community working together for collective benefit. The tagging standard is great, now we need to reduce the time and cost to apply them to a data set. I have seen an example of one tool and heard of another that can ingest (auto-discover) a BAS, apply tags, and then the appropriate rules/algorithms automatically apply. I am sure this will always require some human oversight /vetting, but it would be minimal.
Gabe Fierro Sat 14 Oct 2017
Hi all:
I'm a PhD student at UC Berkeley working on building metadata, analytics and controls. Just wanted to jump in and provide some pointers to some of the academic work in this area:
Deconstructing human-readable tags
Identifying sensor streams
The BuildSys conference is a growing community of computer scientists, building scientists, mechanical/civil engineers, etc interested in these kinds of topics, so you can search past programs to find other related work.
The systems I linked above aren't "production ready", but they do offer promising avenues of being able to automatically extract tags from existing metadata and timeseries data. Terry, you mentioned that JCI has something like a standard for these point names. Is this written down/published anywhere that I could take a look? In the buildings I've worked with, there's been very little standardization around point names beyond common idiomatic abbreviations such as VAV and AHU. Point names usually betray more about the BMS network than they do about the actual physical relationships (such as which chiller connects to this AHU)
Christian Tremblay Mon 16 Oct 2017
@Terry, I don't work a lot with Lonworks anymore but I remember their variables names being the same for standard applications like VFD or VAV...
Terry Herr Mon 16 Oct 2017
Gabe,
those are good articles. I found them a while back. That conference looks interesting. I am not aware that Johnson published anything regarding their standard. We have lots of BAS data sets, and many are Johnson. The standard I am referring to is typically only the last acronym in the longer point name which as you note also includes network controllers acronyms.
Christian, LON was much better than BACnet at defining and requiring standard snvt's (point) naming https://www.lonmark.org/technical_resources/resource_files/snvt.pdf
I believe BACnet is adopting equipment profiles and standard naming, which will make this problem less important some years down the road. In the meantime we are stuck with thousands of buildings with non standard point names.