Hi, I try to read from folio using pyhaystack with something like this:

hstack_session._get_grid(uri, None, expect_format='json')

Using wireshark I see that I get the right answer, however the json result includes fields with lists, something like this:

{ "meta": {"ver":"3.0"}, "cols":[ {"name":"id"}, {"name":"comentariosList"}, {"name":"comentarios"}, {"name":"mod"} ], "rows":[ {"id":"r:p:cbreholding:r:249cae59-62838cea p:cbreholding:r:249cae59-62838cea", "comentariosList":[{"fecha":"d:2019-06-19", "tipo":"Estado", "usuario":"WhyB Analytics", "comentario":"s:Cambio de Estado: Nueva"}, {"fecha":"d:2019-06-19", "tipo":"Estado", "usuario":"WhyB Analytics", "comentario":"s:Cambio de Estado: Reportada"}, {"fecha":"d:2019-06-24", "tipo":"Estado", "usuario":"CBRE FREE", "comentario":"s:Cambio de Estado: Descartada"}], "comentarios":"m:", "mod":"t:2019-06-24T11:43:35.083Z UTC"} ] }

It seems pyhaystack has trouble parsing that and I get an <GetGridOperation failed> result.

I figured I might as well just parse the json result myself but expect_format='json' and raw_response=True do not seem to work together. Any advice? Thanks.

What version of hszinc are you running? Recent versions _should_ support lists, as we use lists quite extensively in WideSky for role-based access control.

The GetGridOperation has a result property you can read: if the operation succeeded, you'd get the result data, otherwise it will raise the exception. What do you get as the exception if you try to read this property?

Ahh, just tried parsing the grid and I think I see the problem, it's easiest seen if we re-format that grid a bit better:

{
    "meta": {
        "ver": "3.0"
    },
    "cols": [
        {
            "name": "id"
        },
        {
            "name": "comentariosList"
        },
        {
            "name": "comentarios"
        },
        {
            "name": "mod"
        }
    ],
    "rows": [
        {
            "id": "r:p:cbreholding:r:249cae59-62838cea p:cbreholding:r:249cae59-62838cea",
            "comentariosList": [
                {
                    "fecha": "d:2019-06-19",
                    "tipo": "Estado",
                    "usuario": "WhyB Analytics",
                    "comentario": "s:Cambio de Estado: Nueva"
                },
                {
                    "fecha": "d:2019-06-19",
                    "tipo": "Estado",
                    "usuario": "WhyB Analytics",
                    "comentario": "s:Cambio de Estado: Reportada"
                },
                {
                    "fecha": "d:2019-06-24",
                    "tipo": "Estado",
                    "usuario": "CBRE FREE",
                    "comentario": "s:Cambio de Estado: Descartada"
                }
            ],
            "comentarios": "m:",
            "mod": "t:2019-06-24T11:43:35.083Z UTC"
        }
    ]
}

The problem is you've got a list of dicts, and dicts are not yet supported in hszinc. https://github.com/vrtsystems/hszinc/issues/8 has more work to be done.

Hi, Sorry for not formatting my json message and thanks for your reply. I am using version 1.2.3. I will take a careful look at that issue in github. In the meantime I am working around the problem by requesting my data in zinc format without parsing (raw). I then made my own parser with regular expressions for the data I need.

Actually, json format would probably be easier to deal with, you could use Python's json module to give you the raw data quite trivially:

import json
import hszinc

# Let's assume we just did a query and got back the raw JSON grid:
grid_str = '''{ "meta": {"ver":"3.0"}, "cols":[ {"name":"id"}, {"name":"comentariosList"}, {"name":"comentarios"}, {"name":"mod"} ], "rows":[ {"id":"r:p:cbreholding:r:249cae59-62838cea p:cbreholding:r:249cae59-62838cea", "comentariosList":[{"fecha":"d:2019-06-19", "tipo":"Estado", "usuario":"WhyB Analytics", "comentario":"s:Cambio de Estado: Nueva"}, {"fecha":"d:2019-06-19", "tipo":"Estado", "usuario":"WhyB Analytics", "comentario":"s:Cambio de Estado: Reportada"}, {"fecha":"d:2019-06-24", "tipo":"Estado", "usuario":"CBRE FREE", "comentario":"s:Cambio de Estado: Descartada"}], "comentarios":"m:", "mod":"t:2019-06-24T11:43:35.083Z UTC"} ] }'''

# Parse as plain JSON
grid_json = json.loads(grid_str)

# Extract the 'comentariosList' values:
for row in grid_json['rows']:
    for element in row['comentariosList']:
        # Convert the dict values to native types
        innerDict = dict([
            (key, hszinc.parse_scalar(value, mode=hszinc.MODE_JSON))
            for key, value
            in element.keys()
        ])

        # do something with innerDict

Since the dict here doesn't have any embedded surprises, that would give you the data you're after in a more convenient format. hszinc can still parse the scalars within that dict. Just not the dict itself.

Hi,

It is true that it should be easier to get json directly from the server, but the reason why I don't do that is because the request does not work properly when using expect_format='json' and raw_response=True together. At least If I do it like this (maybe there is a better way?):

request = 'readAll(incidencia)'                                                                                                                                 
result = session._get_grid(f'eval?expr={quote(request)}',                                                                                                       
                           None,                                                                                                                                
                           expect_format='json',                                                                                                                
                           raw_response=True).result

This is what it sends as obtained by wireshark:

GET /api/cbreholding/eval?expr=readAll%28incidencia%29 HTTP/1.1
Host: 192.168.48.24:9080
User-Agent: python-requests/2.21.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Authorization: BEARER authToken=sXtUkHhuwVUF0BIZ7cR6xH7tXYRNp7riWfqWSDAmbvGg-2a8
Cookie: skyarc-auth-9080=sXtUkHhuwVUF0BIZ7cR6xH7tXYRNp7riWfqWSDAmbvGg-2a8

So basically the Accept header is wrong, it should be:

Accept: application/json

And of course the server returns zinc. If I do not use the raw_response argument then the session object does get json but it tries to parse it and it crashes.

Would you mind opening an issue here : https://github.com/ChristianTremblay/pyhaystack/issues

We’ll keep track of the details and try to fix it ASAP.

Thanks

sure: https://github.com/ChristianTremblay/pyhaystack/issues/82

thanks!