#714 Haystack 4.0 Review - 3.9.5

Brian Frank Thu 13 Jun 2019

We upgraded the project-haystack.dev with a new version.

See previous version notes:

  • 3.9.4 topic 699
  • 3.9.3 topic 694
  • 3.9.2 topic 687


There are some new ideas here about how to define equip sub-parts and points. I have not had time to fully write-up the docs, but here is the "short version"...

We introduce a new concept called "protos". These are essentially just a dict as a predefined collection of tags. They are auto-generated by the documentation as a separate type of documentation page.

For example, we want to specify that a chiller has specific type of leaving pipe:

def: ^chiller
is: ^equip
  chilled water leaving pipe equip

That gets turned into the following proto page. You can see that protos page just lists the reflected tags under the implements section. There is also a section for children which includes chilled water leaving temp point which is all auto-generated. But by how?

The definition of pipe looks like this:

def: ^pipe
conveys: ^fluid
pipeSection: ^pipeSectionType
childrenFlatten: [^fluid, ^pipeSectionType]
  temp point

We aren't creating definitions for every combination of pipes with fluid x section. Rather we just say that if you find those tags, then flatten them into the children via childrenFlatten tag. That is how we know given a chilled water leaving pipe to flatten the fluid type (chilled-water) and section (leaving) into the children to get something like chilled water leaving temp point.

Essentially this is a way to build tools and documentation from possible combinations without manually generating them. And instead of trying to make them all first class defs, we just make them normal dicts as collection of tags. The last design tried out making all these things synthetic defs - but it was really confusing for them be like defs, but not really defs. In this case there is a clear distinction: defs are first class named tag definitions. Protos are just collections of tags that we reflect.

As a bit of design history, to solve this problem we have tried all of the following:

  • manually writing them by hand as first class defs
  • using generics like parameterized types in programming languages
  • generating synthetic defs
  • generating prototypes as just tags sets (this design)

Of all the designed tried out, this design feels like the best and the most "Haystack-like".


There is some upcoming work on data centers that will get into all sorts of computer and networking gear. I stubbed out a new pcIct module to be collection for microprocessor based devices, software, and networking terms. ICT for Information Communication Technology is usually considered a broader term than IT. I don't love the names ICT and IoT - but I think it would be nice to split the physical world stuff (site, equip, points) from the hardware/software/networking world. If you have suggestions please make them!

Currently this lib is home to the old device, network, and protocol tags. I flushed out protocols using a taxonomy tree.

Ontology Changes

We reworked fan/pump to be subtypes of motor. And likewise reworked valve/damper to be subtypes of actuator based on discussions from Haystack Connect.

We got rid of building as subtype of both site and space to make things a little closer to how it works today. While its a cool idea, it might be too radical based on how Haystack 3.0 is used today.

We will need to rework weather slightly because we cannot use weatherRef to mean both "contained by" for points and "associated with" for sites. I'm thinking we keep weatherRef for sites, and rename the tag used on points as weatherStationRef.


Version 3.9.5 (12 Jun 2019)

  • Refactor site/weatherStation to be closer to current design
  • Remove building tag
  • Add phIct library for hardware, software, and networking
  • Redesign children using dict prototype instances
  • Rework motor/actuator conjuncts for fan, pump, damper, and valve
  • Add docs on the doc sections
  • Add usage doc section

Si Chen Fri 12 Jul 2019

Hello Brian,

I have some questions about the protos feature:

  1. If protos is a pre-defined collection of tags, then what does it mean to say that it "implements" a set of tags? For example, https://project-haystack.dev/doc/proto/chilled-water-leaving-pipe-equip says that it implements chilled, chilled-water, equip, leaving, pipe, water. Does that mean that anything defined as a "chilled water leaving pipe equip" should automatically have chilled, chilled-water, equip, leaving, pipe, water?
  2. If that's the case, then does that mean a user should be able to navigate from index to equip to chiller to chilled water leaving pipe equip?
  3. In that case, then should the user still be able to navigate from chilled water leaving pipe equip to chilled water leaving equip to the equip proto, which then navigates to the equip and point protos? Note that the equip proto is its own child, so it keeps going.

Is the protos for grouping tags for user interfaces?


Brian Frank Sun 14 Jul 2019

A proto is just an "example" of a dict with a set of tags - think of it as a "template" you can copy to create an entity record. So to say a proto implements a def is just following the standard reflection rules described here.

If that's the case, then does that mean a user should be able to navigate from index to equip to chiller to chilled water leaving pipe equip?

I am not sure what you mean be "navigate". But the way we have setup the docs currently, you have two paths of documentation navigation:

  • subtype/supertype tree
  • children prototypes a given entity type might contain

So from docs you can "navigate" equip to its subtypes including a chiller. And on a chiller you can "navigate" to its children including a given pipe. But note those are two different documentation sections (subtypes and children respectively).

Si Chen Mon 15 Jul 2019

Hi Brian

What I mean by "navigate" is that if you go to


and click on "chilled water leaving equip" which is one of its children, then you get to


and there are two children there, "equip" and "point".

So what is the meaning from these two children?

Brian Frank Mon 15 Jul 2019

So what is the meaning from these two children?

I put those in there because technically a given equip can have any type of child equip or point. Its really just a question of if you consider the world open ended or restricted by default. So from a generic perspective, an space can contain other spaces, equips, or points. And an equip contains other equip and points

Si Chen Mon 15 Jul 2019

I see, thanks for clarifying that.

So if we want to implement the protos, where is the definition of a proto like https://project-haystack.dev/doc/proto/chilled-water-leaving-equip?

I cannot find a definition for it. I can find it defined as children for ^chiller and ^chilled-water-plant

Do we need to define these protos ourselves then?

Brian Frank Mon 15 Jul 2019

I cannot find a definition for it. I can find it defined as children for ^chiller and ^chilled-water-plant

That is correct protos are not first class definitions. They are just collection of tags.

They are declared in plain text within the children tag of various entity definitions. Look at the source code and then compare to what the docs are generating

Si Chen Mon 15 Jul 2019

Sorry, Brian, but I'm not seeing where they are defined.

For example, I see that in the Zinc file, ^chiller has children run state point\nenable state point\nchilled water leaving pipe equip\nchilled water entering pipe equip\ncondenser water leaving pipe equip\ncondenser water entering pipe equip

But where is the definition of what tags are part of the "chilled water leaving pipe equip" proto?

Brian Frank Mon 15 Jul 2019

If you don't understand my write-up in this post, its best you wait until we have time to fully document the feature

Login or Signup to reply.