The JSON section of the Haystack docs define each row of an HGrid as an object containing fieldname/value pairs. The haystack-java code builds each row of an HGrid as an array of values only, without field names.
The latter seems more compact, as the field names are available from the column metadata.
Question: Which is correct? Do they both work in practice?
References:
In the Haystack documentation, Section 9.3 describes a mapping from Grid to JSON in which "the rows field is a list of JSON objects" Consistent with this, the example in 9.3 shows each row as a JSON object containing name / value pairs, possibly equivalent to an HDict. https://project-haystack.org/doc/Json
On the other hand, the Haystack-Java code for the HGridBuilder (https://bitbucket.org/brianfrank/haystack-java) makes each row an array of HVal instances, without field names. Look at, for example, around lines 39 or 143.
Brian FrankFri 30 Mar 2018
The abstract data model for Haystack is two basic collection types: List and Dict (hash map/associative array). Grid is then essentially a list of dicts (each row is a Dict). Plus Grid adds a Dict for the grid level meta and column meta.
Implementation wise, the Java toolkit stores rows as a array of objects and shares the grid level column name hash map (because its much more efficient). But you will notice that HRow subclasses HDict.
The JSON encoding was choose to be convenient, not efficient. Zinc is what allows the column names to be encoded once and then deduced by row cell order.
Chris OwensFri 30 Mar 2018
Would there be any downside to a Java implementation that stored the grid data as HVal[][] (an array of rows, each of which is an array of HVal)?
This would eliminate the HRow class (which would indirectly solve the problem that the grid field of an HRow is not always set in HGridBuilder), at the cost of making it a two step operation to look up a row element by name -- first you find the HCol by name, and then use its index into the HVal[] array representing the row....
In any case, I want to confirm that I understand correctly that the JSON encoding of a Grid should create, for each row, a JSON object of name/value pairs.
Brian FrankFri 30 Mar 2018
>Would there be any downside to a Java implementation that stored the grid data as HVal[][] (an array of rows, each of which is an array of HVal)?
When working with Haystack data, you almost always want a convenient way to pass rows around as Dict. So that design would be more awkward and really wouldn't save you that much more RAM
In any case, I want to confirm that I understand correctly that the JSON encoding of a Grid should create, for each row, a JSON object of name/value pairs.
Correct.
Chris OwensFri 30 Mar 2018
Why does the iterator for an HRow skip nulls? i.e., lines 71 and thereafter in HRow.java. It looks like the only place it is used is in a test.
Brian FrankWed 4 Apr 2018
Why does the iterator for an HRow skip nulls? i.e., lines 71 and thereafter in HRow.java.
Because grids tend to be sparse and that way you only iterate the values in that actual row (not the sparse empty cells)
Chris Owens Thu 29 Mar 2018
The JSON section of the Haystack docs define each row of an HGrid as an object containing fieldname/value pairs. The haystack-java code builds each row of an HGrid as an array of values only, without field names.
The latter seems more compact, as the field names are available from the column metadata.
Question: Which is correct? Do they both work in practice?
References:
In the Haystack documentation, Section 9.3 describes a mapping from Grid to JSON in which "the rows field is a list of JSON objects" Consistent with this, the example in 9.3 shows each row as a JSON object containing name / value pairs, possibly equivalent to an HDict. https://project-haystack.org/doc/Json
On the other hand, the Haystack-Java code for the HGridBuilder (https://bitbucket.org/brianfrank/haystack-java) makes each row an array of HVal instances, without field names. Look at, for example, around lines 39 or 143.
Brian Frank Fri 30 Mar 2018
The abstract data model for Haystack is two basic collection types: List and Dict (hash map/associative array). Grid is then essentially a list of dicts (each row is a Dict). Plus Grid adds a Dict for the grid level meta and column meta.
Implementation wise, the Java toolkit stores rows as a array of objects and shares the grid level column name hash map (because its much more efficient). But you will notice that HRow subclasses HDict.
The JSON encoding was choose to be convenient, not efficient. Zinc is what allows the column names to be encoded once and then deduced by row cell order.
Chris Owens Fri 30 Mar 2018
Would there be any downside to a Java implementation that stored the grid data as HVal[][] (an array of rows, each of which is an array of HVal)?
This would eliminate the HRow class (which would indirectly solve the problem that the grid field of an HRow is not always set in HGridBuilder), at the cost of making it a two step operation to look up a row element by name -- first you find the HCol by name, and then use its index into the HVal[] array representing the row....
In any case, I want to confirm that I understand correctly that the JSON encoding of a Grid should create, for each row, a JSON object of name/value pairs.
Brian Frank Fri 30 Mar 2018
>Would there be any downside to a Java implementation that stored the grid data as HVal[][] (an array of rows, each of which is an array of HVal)?
When working with Haystack data, you almost always want a convenient way to pass rows around as Dict. So that design would be more awkward and really wouldn't save you that much more RAM
Correct.
Chris Owens Fri 30 Mar 2018
Why does the iterator for an HRow skip nulls? i.e., lines 71 and thereafter in HRow.java. It looks like the only place it is used is in a test.
Brian Frank Wed 4 Apr 2018
Because grids tend to be sparse and that way you only iterate the values in that actual row (not the sparse empty cells)