Does anyone have short examples of Haystack 4 they can share? I'm interested in:

• Defining custom tags under a new namespace
• Describing equipment & a few points using a combination of standard & custom tags
• Showing history of said points

The API docs are helpful, but don't provide quite enough detail for me to put it all together.

Best regards,

Josiah Johnston

Have you pulled the repo or browsed the source code for the standard tags on BitBucket? You can use this link.

Defining custom tags under a new namespace

Technically this would be a new library. I would start by copying phIct maybe you can see how all the files get put together (a library is just a zip file of text files).

Showing history of said points

This is not done with definitions, but rather an runtime which trends the points and responds to the hisRead operation over HTTP.

Brian, that repository contains 3.9.7 definitions, not 4.x.

Brian, that repository contains 3.9.7 definitions, not 4.x.

We are using the 3.9.xxx version series to indicate all the 4.0 prototypes during public review. The final release will then be 4.0.0

Thanks for this! It provides more detail than I had uncovered previously, and I hadn't realized 3.9.7 was effectively a release candidate for 4.0 (4.0.0-RC.7 in semantic versioning). This example includes several library definitions, but lacks documents that use the libraries to describe 1) equipment and 2) timeseries.

The gap for me is linking new definitions to haystack documents that describe equipment or timeseries. Many years ago when I worked on XML file standards, I could define new things within a namespace identified by a URI, and use those elements in documents by referencing the URI. A nice thing about the URI was we had the option of making it resolvable for people to access the formal definitions. This example snippet includes elements defined by two separate namespaces:

<custom:RootElement xmlns:OME="http://www.openmicroscopy.org/Schemas/OME/2016-06"
         xmlns:custom="http://www.example.org/SampleThirdPartySchema/2013-01">
    <OME:Laser .../>
    <custom:MyCustomTag .../>
</custom:RootElement>

I'm looking for some analog to that in Project Haystack. But maybe the answer is it hasn't been defined yet? In https://project-haystack.dev/doc/docHaystack/Namespaces, I see

Individual Haystack projects may pick and choose which libs are used to define their application specific data. We call this def namespace the project namespace. Project namespaces will often be a mix of standardized libs, vendor specific libs, and project specific libs. It is outside the scope of this specification to define how libs are included or enabled for project namespaces. However all project namespaces must have a mechanism to formally specify/export which libs are in scope. This is best accomplished through the libs HTTP API operation (TODO).

If I'm reading that correctly, then I should simply use new custom tags in my haystack documents for now without linking to any libraries. Separately, I write libraries to define the new tags, which serves as documentation initially and eventually can help with validation. Eventually, there will be a spec for saying which libraries each document/project is drawing from. Is that a reasonable interpretation?

Just to be clear - we are using the term "namespace" in Haystack similar to how you would describe a programming language compilation unit. Its the symbols currently in scope for a given lexical unit. For example in Java the source file's namespace is based on the imported packages.

We use the term "lib" to describe a group of symbols packaged together. This would be more akin to what an XML namespace is, like a module of symbol definitions.

The two go together - for example the symbols available within a library are based on that library's dependencies:

def: ^lib:phIoT
version: "3.9.7"
baseUri: `https://project-haystack.org/def/phIoT/`
wikipedia: `https://en.wikipedia.org/wiki/Internet_of_things`
depends: [^lib:ph, ^lib:phScience]
doc: "Project Haystack definitions for Internet of Things"

All the symbols from the ph and phScience are available for use inside the phIoT library's definitions via the depends tag.

In terms of instance data, Haystack is primarily designed to work over HTTP for runtime data (current and historical data). So there is no such thing as a single "document" that fully describes the building, points, and all the live and historical data. The operational data is changing all the time. You can get a document for the static data (sites, equip, and points). But live and historical data are typically queried on the fly via HTTP. Those tech docs have not been ported to the Haystack 4 site, but you can browse them on this site here.

Specifically if you want to get historical data from a Haystack system you would query it via the hisRead operation

Thanks for the clarification. I think we're on the same page on the following points:

Library modules (libs) store a set of definitions, and can import from other libs. A given namespace will import definitions from a set of libs. Importing is well-defined for libs, but is not defined for projects or documents. There is no support for fully-qualified names at this time (ex: phIoT.ahu), only support for unqualified names.

Individual haystack documents are generated on the fly from HTTP requests, typically from a large data store (aka project). These haystack documents can store static information, historical time series, or current values.

My questions

• Does anyone have examples of custom libs and Haystack documents that utilize a combination of ph & custom libs?

Ideally the example documents would include static descriptions and a short timeseries of historical data.

• Can haystack documents declare which libs they use?

I think the answer is no, but I was hoping for confirmation. If I'm correct, then I can write a lib with custom terms, and simply use those terms in my documents or project.

• Will haystack consider supporting fully-qualified names in the future?

The current strategy of making every term in every custom library global unique will not be scalable if Haystack is widely adopted for IoT. Different vendors will want re-use common words, and most buildings have a mix of products from vendors. Namespaces make it easy to differentiate acme.status vs globex.status, and if acme resolves to a URI like acme.com/haystackDef/2025-01, companies with similar names won't have conflicts.

I think the answer is no, but I was hoping for confirmation.

No one has really talked about that. But probably it would just be to add the includes tag in your grid meta like this:

includes: [^lib:ph, ^lib:phIoT, ...]

Or maybe we should coin another tag for that, but seems consistent to re-use the includes tag in that case.

Different vendors will want re-use common words, and most buildings have a mix of products from vendors.

That has been extensively discussed. As a general rule we've found that terms rarely conflict and that you solve the problem with groups of tags which can now be defined as conjuncts. Or if you have custom tags you should always prefix them because the whole point is the identifier needs to be unique.

What we have talked about is using the include mechanism to handle conflicts - much like you would do in C# doing a using alias. In the end its more about how to map identifiers to defs and there is a lot of flexibility how that might work (with a trade-off of complexity). So we are sticking the simplest solution as possible for now

That includes tag looks great. I'll plan to use that strategy for now, and hope that it is eventually formalized into the standard.

Re: fully-resolved names

Using a simple solution until it breaks down is usually a reasonable approach. The problem of conflicting names will only emerge with distributed definitions of custom tags that eventually need to get stored in a single database. I'm hearing those cases have been rare for you so far, and manual-workarounds to rename tags has been an adequate approach. If those issues arise more frequently in the future, I trust that this topic will re-emerge for discussion.

If/when that happens, I think the standard approach to namespaces that allow for both flat namespace and fully-qualified names (import java.util.*; vs import java.util;) could be a very good solution that would permit simplicity most of the time, and all programmers would be very familiar with. But that can be a conversation for another day.