#938 Sub-proposal: Filter Inference Operator

Brian Frank Mon 19 Jul

This is a sub-proposal of the broader 940 Haystack 4 proposal.

The current docHaystack::Relationships chapter includes a strawman proposal for how to query relationships. In this sub-proposal we tweak the syntax and tie up a few loose ends.

First lets examine the common use cases:

The overwhelming use case of relationship queries is looking up specific points for an equipment. For example a visualization or analytic on an AHU might lookup the fan enable cmd like this:

discharge and fan and enable and cmd and equipRef==@ahu

I would venture to guess that this pattern is used for 90% of all Haystack queries. The problem is that it doesn't handle varying levels of detail in the model. Haystack 4 allows an instance model to be any of the following:

ahu / enable-point
ahu / discharge-duct / enable-point
ahu / discharge-fan-equip / enable-point
ahu / discharge-duct / discharge-fan-equip / enable-point

In the first case we have a simple instance model where the AHU models the fan enable command point as a direct child. In the last case we have a very detailed model where the duct and fan are modeled as first class entities. And we have valid models between the two. We require a filter syntax so that applications can correctly resolve the point on any four of these instance models.

I propose a new operator to the filter syntax with the following patterns:

name? ^symbol
name? @ref
name? ^symbol @ref

For most the part the behavior is as described in the strawman proposal. However the syntax is slightly modified so that the name and symbol aren't combined like a conjunct. With that syntax we can query for child points as follows:

// query from above using inference operator
discharge and fan and enable and cmd and containedBy? @ahu

// same thing but little more explicit that we expect ahu
// be an equip and not some other entity type like a space
discharge and fan and enable and cmd and containedBy? ^equip @ahu

// query all the points contained recursively by the equip
point and containedBy? ^equip @ahu

// query all entities which contain the given point
contains? @point

Note that per the existing proposal a filter engine would take into account if a relationship has transitive and/or reciprocalOf defined.

Containment is by far the most important and most queried relationship. Next most important is flows including the flow of electricity, air, and fluids with thermal energy. We are capturing these relationships using a suite of tags such as elecRef, airRef, hotWaterRef, chilledWaterRef, etc.

Per the separate sub-proposal 937, we propose to rename the flow relationships as inputsFrom and outputsTo. That allows queries such as:

// query all equip which inputs hot water from the given plant 
// (all downstream equip)
equip and inputsFrom? ^hot-water @hwp

// query the AHU which outputs air to the given VAV
ahu and outputsTo? ^air @vav

Like containment these relationships are transitive so can traverse any number of levels of elecRef, airRef, etc.

The same behavior of the inference operator will also allow use of is? to query supertypes:

// query anything that subtypes from airTerminalUnit
?is ^airTerminalUnit

Gareth David Johnson Thu 12 Aug

The strawman conjunct style syntax was confusing. I like the changes proposed here.

This enhancement to filters is badly needed so people can make effective use of Haystack's onotology.

annie mroz Wed 18 Aug

I really like the proposed improvements to the containment and querying syntax. This will be super helpful for us and our analytics.

Also, the ability to query subtypes using the ?is filter is great.

A question though about the inputsFrom/outputsTo...

How does that work if you've modelled references for each component? If you've modelled each component in the system with an upstream reference, would all equipment be returned by the query?

For a really simple example, say you have:

@boiler, hotWaterRef: @ahuHWvalve
@pump, hotWaterRef: @boiler
@ahuHWvalve, hotWaterRef: @pump

What would equip and inputsFrom? ^hot-water @boiler return in this case? Just @pump or @pump and @ahuHWvalve?

Brian Frank Wed 18 Aug

What would equip and inputsFrom? ^hot-water @boiler return in this case? Just @pump or @pump and @ahuHWvalve?

It would return all upstream matches, so it would @pump and @ahuHWvalve.

Although the model is that hotWaterRef points to your source. So an actual model would be flipped (the AHU valve would point to the pump, which in turns points to the boiler)

Steve Eynon Sat 18 Sep

Hi Brian, I can certainly see all the hard work and thought process gone into these proposals, and in general they do help to simplify the common case.

However, looking at this sub-proposal from a training perspective, I fear that introducing a new style of filter syntax will have an adverse impact on the ease of adoption of Project Haystack.

It is far easier for us humans to adopt technology if it re-uses existing syntax, and I have concerns that this sub-proposal could be a little too obtuse for general human consumption.

My concerns are two fold:

1. The ? symbol is misleading

In programming languages the ? is often used to denote something optional or something to do with nulls, be it a regular expression a? b, a null safe call a?.b(), or an elvis a ?: b.

So when I first look at name? ^symbol I'm very confused as to what it means.

2. Sequentially listing terms shuns existing operator syntax

name? ^symbol @ref such as containedBy? ^equip @ahu is not easy to understand. I would like to see the use of operators to give context to the terms in the expression.

As such I would like to propose a slightly different syntax that I feel is more intuitive to us humans.

A revised sub-proposal

Use ~ to denote inferred tags.

~ in English means approx or about, so it works here by meaning not directly in the Dict.

inputs   // does the dict have a tag named "inputs"
~inputs  // does the dict implement a def with a tag name "inputs"

Use == for equals.

The equals operator is ubiquitous, so lets use it.

// all points contained by '@ahu'
point and ~containedBy == @ahu

Use is operator to denote hierarchy.

Definitions already use the is tag, so lets re-use the same terminology.

// anything that subtypes from airTerminalUnit (instance use)
is ^airTerminalUnit

// anything contained by an airTerminalUnit (definition use)
~containedBy is ^airTerminalUnit

// all ahu equips
equip and is ^ahu

In general I propose the syntax be enhanced to the following.

old                  enhanced
---                  --------
name?                ~name
name? ^symbol        ~name is ^symbol
name? @ref           ~name == @ref
name? ^symbol @ref   ~name is ^symbol and ~name == @ref

The last case is a little more verbose, but also very understandable.

// query the AHU which outputs air to the given VAV
ahu and ~outputsTo is ^air and ~outputsTo == @vav

I look forward to hearing your thoughts.

Login or Signup to reply.