Hi,

i am using pyhasystack in python and trying to read histories of all zone sensors in a single data frame (calling skyspark via python). this used to work before, but since a month or so, when trying to run command as per documentation, i get the java null pointer exception.

Error message- HaystackError: sys::NullErr: java.lang.NullPointerException

Can you please help?

Vipul.

Can you provide more information about what your request looks like and also you should have a full stack trace in the response grid meta errTrace tag

Request:
op = session.his_read_frame(znt, rng='today')
op.wait()
b = op.result
b

Error Trace:

---------------------------------------------------------------------------
HaystackError                             Traceback (most recent call last)
<ipython-input-11-368d15e6746f> in <module>()
      1 op = session.his_read_frame(znt, rng='today')
      2 op.wait()
----> 3 b = op.result
      4 b

7 frames
/usr/local/lib/python3.6/dist-packages/pyhaystack/util/state.py in result(self)
     98 
     99         if self.is_failed:
--> 100             self._result.reraise()
    101 
    102         if not self._result_copy:

/usr/local/lib/python3.6/dist-packages/pyhaystack/util/asyncexc.py in reraise(self)
     27 
     28     def reraise(self):
---> 29         reraise(*self._exc_info)

/usr/local/lib/python3.6/dist-packages/six.py in reraise(tp, value, tb)
    701             if value.__traceback__ is not tb:
    702                 raise value.with_traceback(tb)
--> 703             raise value
    704         finally:
    705             value = None

/usr/local/lib/python3.6/dist-packages/pyhaystack/client/ops/his.py in _on_single_read(self, operation, col, **kwargs)
    330         self._log.debug("Response back for column %s", col)
    331         try:
--> 332             grid = operation.result
    333 
    334             if self._tz is None:

/usr/local/lib/python3.6/dist-packages/pyhaystack/util/state.py in result(self)
     98 
     99         if self.is_failed:
--> 100             self._result.reraise()
    101 
    102         if not self._result_copy:

/usr/local/lib/python3.6/dist-packages/pyhaystack/util/asyncexc.py in reraise(self)
     27 
     28     def reraise(self):
---> 29         reraise(*self._exc_info)

/usr/local/lib/python3.6/dist-packages/six.py in reraise(tp, value, tb)
    701             if value.__traceback__ is not tb:
    702                 raise value.with_traceback(tb)
--> 703             raise value
    704         finally:
    705             value = None

/usr/local/lib/python3.6/dist-packages/pyhaystack/client/ops/grid.py in _check_err(grid)
    335                         raise HaystackError(
    336                             grid.metadata.get("dis", "Unknown Error"),
--> 337                             grid.metadata.get("traceback", None),
    338                         )
    339                     return grid

HaystackError: sys::NullErr: java.lang.NullPointerException

also, just a side note-this query used to work perfectly fine a few months back and was not giving any error. Only recently i have begin to get this error. Have tried different skyspark projects, changed time interval in the query; but everytime i am not able to resolve this error.

That is the local trace, not the remote trace. The remote trace will be in the response grid meta errTrace tag

Hi Brian, Sorry for my ignorance, but i am not really sure how do i get the response grid meta errTrace Tag..do i extract this by querying using python? can you please guide me?

Hi Vipul, I'm not familiar with the python library, but in Axon you would use grid.meta()->errTrace, perhaps that helps you get in the right direction.

HI,

I hope below is the response grid for meta error, with the error description (HaystackError: sys::NullErr: java.lang.NullPointerException)

/usr/local/lib/python3.6/dist-packages/pyhaystack/client/ops/grid.py in _check_err(grid)

335                         raise HaystackError(
336                             grid.metadata.get("dis", "Unknown Error"),

--> 337 grid.metadata.get("traceback", None),

338                         )
339                     return grid

HaystackError: sys::NullErr: java.lang.NullPointerException

Is there any other error trace we are looking for?

Vipul.

Hi Vipul, I'm missing the errTrace tag in your above code snippets, but let's try this a different way:

Are you trying to handle/log the exception in your code, are are you trying to debug the exception? If you are (currently) only interested in debugging the exception, perhaps you could try to run the query directly in SkySpark and see what the results are then?

If the SkySpark result is indeed a grid with the errTrace metadata, you can post that exact content here. Also perhaps debugging/logging the full result might give you some insight where the error is hiding in your response object.

Yes, i am trying to debug the error in my python code. When i run the same/equivalent query in skyspark (read the history value of all zone temp sensors for today); i do not get any error message, the code runs successfully

Below Code executed successfully in SkySpark:

readAll(zone and temp and sensor).hisRead(today)

Hi Vipul, I think it is clear then there is an issue with the library itself. Did you try contacting the maintainer via https://github.com/ChristianTremblay/pyhaystack/issues ?

Hi, pyhaystack contributor here,

It's worth noting that his_read_frame is a high-level abstraction that translates the native Grid format used by Haystack into a form more easily used in computations Python-side, notably for numpy integration.

Furthermore, there's also state machine logic that will either execute a multi-point hisRead operation (on servers that support it) or will issue multiple single-point hisRead operations for each point, and stitch the result sets together to form a "frame".

If you're after a low-level hisRead, without the extra stuff, look no further than the his_read method. https://pyhaystack.readthedocs.io/en/latest/pyhaystack.client.html#pyhaystack.client.session.HaystackSession.his_read

Stripping of the server-side stack trace is actually a design decision here. Emitting server-side stack traces to the client is somewhat frowned upon as it's "leaking" implementation details.

Leaking a NullPointerException immediately makes me think unhappy paths have not received good test coverage: not a good look.

Anyway, I digress…

You'll note that when generating the exception, that it passes that traceback object as an argument to the HaystackException:

https://github.com/ChristianTremblay/pyhaystack/blob/master/pyhaystack/client/ops/grid.py#L337

If you catch this exception, you'll find that trace-back is preserved as the traceback attribute on the exception:

https://github.com/ChristianTremblay/pyhaystack/blob/3c3508212c0f5a4e5f254fed4d169da91f03242d/pyhaystack/exception.py#L8

from pyhaystack.exception import HaystackException

try:
    op = session.his_read_frame(znt, rng='today')
    op.wait()
    b = op.result
except HaystackException as e:
    print (e.traceback)

That might get you what you're after.

The other thing you might want to look at is enabling logging in the client:

https://github.com/ChristianTremblay/pyhaystack/issues/89#issuecomment-657866097 has some example set-up code for doing that.