Hi,

I am new to using ProjectHaystack.client .net NuGet Package. I have a running server with haystackVersion 3.0. Iam trying to use the ProjectHaystack.Client v1.4.1 nuget package to connect to my server and get some response for testing. But currently having a hard time with the auth.

First I tried using Restsharp to hit the server and were able to get successful response with the below code

var client = new RestClient("http://myServer/haystack/about");

var request = new RestRequest(Method.GET);

client.Authenticator = new HttpBasicAuthenticator("myUser", "myPass");

IRestResponse restResponse = client.Execute(request);

IRestResponse response = restResponse;

Console.WriteLine($"Result:\n{response.Content}");

When I try to do something similar like above by using the HClient or HAsyncClient method by using the below code I keep getting HTTP ERROR: 403 "Authentication failed."

var uri = "http://myServer/haystack/";

HClient hsClient = HClient.Open(uri, "myUser", "myPass");

Dictionary<string, string> searchParams = new Dictionary<string, string>();

string result = hsClient.GetString("about", searchParams, "application/json", "application/json");

Console.WriteLine($"Result:\n{result}");

My stack trace:

at System.Net.HttpWebRequest.GetResponse()
at ProjectHaystack.Auth.AuthClientContext.Get(HttpWebRequest c)
at ProjectHaystack.Auth.AuthClientContext.GetAuth(AuthMsg msg)
at ProjectHaystack.Auth.AuthClientContext.SendHello()
at ProjectHaystack.Auth.AuthClientContext.Open()

ex.InnerException.Message: The remote server returned an error: (403) Authentication failed..

ex.Message: authenticate failed

In postman I am able to get successful output using Basic Authentication with myUser and myPass for the same uri "http://myServer/haystack/about" without any problem.

Iam not sure what Iam doing wrong and why I get 403 "Authentication failed" error when using basic HClient or HAsyncClient method. Iam also not sure if it could be combatability issue between the .net client NuGet Package and the server v3.0. Please any suggestion or advice will greatly help. Thanks a lot.

Hi Indumathi,

I'm sorry to hear you are having trouble with the library. As far as I can tell you are doing everything right, however there seems to be an issue with the way the server responds to the initial "hello" request (with which the library detects the authentication scheme) as it responds with a 403 instead of a 401.

Without the actual call and response from the server I cannot quite say what is causing it, but if you can get the original queries back-and-forth it might be easier to figure out why you are getting a 403.

There is however a new client (not documented yet) that I advice using as it uses HttpClient instead of HttpWebRequest, which is much better for managing your connections, but it also allows you to provide a specific authenticator so that you can explicitly tell the library how to authenticate. This might already solve your issue.

Sample code:

var client = new HttpAsyncClient(new BasicAuthenticator(<username>, <password>, new Uri("http://myServer/haystack"));
client.OpenAsync();
...

Thanks a lot for your response Chris,

I tried using the method suggested and used the below code in my program:

var client = new HttpAsyncClient(new BasicAuthenticator("myUser", "myPass"), new Uri("http://myServer/haystack/"));

await client.OpenAsync();

Dictionary<string, string> searchParams = new Dictionary<string, string>();

var result = await client.GetStringAsync("about", searchParams, "application/json");

Console.WriteLine($"Result:\n{result}");

But I got the error : "The given header was not found." With Stack Trace as:

at System.Net.Http.Headers.HttpHeaders.GetValues(HeaderDescriptor descriptor)

at System.Net.Http.Headers.HttpHeaders.GetValues(String name)
at ProjectHaystack.Auth.BasicAuthenticator.<SendHello>d__6.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
at ProjectHaystack.Auth.BasicAuthenticator.<Authenticate>d__5.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at ProjectHaystack.Client.HttpAsyncClient.<OpenAsync>d__10.MoveNext()
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Haystack.Controllers.HomeController.<ReadData>d__6.MoveNext() in C:\Users\..\..\..\Haystack\Haystack\Controllers\HomeController.cs:line 72

To understand the error more in detail, I tried the suggested code along with dowloaded project Haystack code and tried to debug it.

Here in the method SendHello(HttpClient client, Uri authUrl), I found that the response received for the line using (var response = await client.SendAsync(message)) has thrown the error as follows: response.ReasonPhrase: Authentication failed. for Request Message: {Method: GET, RequestUri: http://myServer/user/auth, Version: 1.1, Content: <null>, Headers:{ Authorization: HELLO username=UserToken User-Agent: HaystackC# Request-Id: |Id.1.}} Status code: System.Net.HttpStatusCode.Forbidden IsSuccessStatusCode: False

{StatusCode: 403, ReasonPhrase: Authentication failed., Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:{ Set-Cookie: JSESSIONID=bb8f;Path=/;HttpOnly Cache-Control: no-store, must-revalidate, no-cache Content-Type: text/html; charset=iso-8859-1 Content-Length: 267}}

Then when it moves to the below line the value for (_wwwAuthenticateHeader) is empty {} that finally throws the error.

var auth = response.Headers.GetValues(_wwwAuthenticateHeader).First();

The final error thrown is:

InvalidOperationException: The given header was not found. System.Net.Http.Headers.HttpHeaders.GetValues(HeaderDescriptor descriptor) System.Net.Http.Headers.HttpHeaders.GetValues(string name) ProjectHaystack.Auth.BasicAuthenticator.SendHello(HttpClient client, Uri authUrl) in BasicAuthenticator.cs +

var auth = response.Headers.GetValues(_wwwAuthenticateHeader).First();

ProjectHaystack.Auth.BasicAuthenticator.Authenticate(HttpClient client, Uri authUrl) in BasicAuthenticator.cs +

await SendHello(client, authUrl).ConfigureAwait(false);

ProjectHaystack.Client.HttpAsyncClient.OpenAsync() in HttpAsyncClient.cs +

await _authenticator.Authenticate(_client, new Uri(Uri, "/user/auth"));

BasClientImp.Controllers.HomeController.Index() in HomeController.cs +

await client.OpenAsync();

Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor+TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, object controller, object[] arguments) System.Threading.Tasks.ValueTask<TResult>.get_Result() System.Runtime.CompilerServices.ValueTaskAwaiter<TResult>.GetResult() Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask<IActionResult> actionResultValueTask) Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted) Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted) Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted) Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted) Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context) Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(ref State next, ref Scope scope, ref object state, ref bool isCompleted) Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, object state, bool isCompleted) Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope) Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger) Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context) Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Looks like I am still receiving the same 403 Authentication failed error.

Kindly provide guidance to resolve this authentication issue. Please let me know if you need further information regarding the same.

Also the Haystack server is being maintained by a different team and I am only given permission to access the server's api to get the data. I don't have direct access to the Haystack server itself. Thanks

Hi Indumathi,

As the initial Hello handshake message (not containing any credentials) returns a 403 tells me the server probably doesn't implement the Haystack Auth mechanism properly as it should respond with a 401 and a list of allowed authentication methods. As your own code seems to work fine, I believe they have chosen to use a simple basic authentication scheme without the actual haystack authentication.

In that case it is best you create your own IAuthenticator implementation and inject it instead. Here that probably means just setting the authentication header on the HttpClient default values, something like below code. You may need to tweak it a bit, but this way you should get full control of the authentication mechanism, while still being able to use the rest of the library as intended.

public class VeryBasicAuthenticator : IAuthenticator
{
    private readonly string _username;
    private readonly string _password;

    public VeryBasicAuthenticator(string username, string password)
    {
        _username = username;
        _password = password;
    }

    public async Task Authenticate(HttpClient client, Uri authUrl)
    {
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
            Convert.ToBase64String(Encoding.UTF8.GetBytes(_username + ":" + _password)).Trim('='));
        try
        {
            using (var response = await client.GetAsync(authUrl))
            {
                if ((int)response.StatusCode != 200)
                {
                    throw new AuthException("Basic auth failed: " + response.StatusCode + " " + (await response.Content.ReadAsStringAsync()));
                }
            }
        }
        catch (Exception e)
        {
            throw new AuthException("basic authentication failed", e);
        }
    }
}

Thanks a lot Chris for your quick response.

As suggested I created VeryBasicAuthenticator method and tried to use it. This time I got different Error 404 Not Found

{StatusCode: 404, ReasonPhrase: Not Found, Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:{ Set-Cookie: JSESSIONID=Id;Path=/;HttpOnly Set-Cookie: JSESSIONID=Id;Path=/;HttpOnly Set-Cookie: JSESSIONID=Id;Path=/;HttpOnly X-Frame-Options: sameorigin Cache-Control: no-store, must-revalidate, no-cache Content-Type: text/html; charset=iso-8859-1 Content-Length: 241}}

When I checked the Request message as given below:

{Method: GET, RequestUri: http://myServer/user/auth, Version: 1.1, Content: <null>, Headers:{ User-Agent: HaystackC# Authorization: Basic userNameToken Request-Id: |Id.1.}}

The Request Uri is set as: http://myServer/user/auth instead of http://myServer/haystack/user/auth

So the "/haystack" segment was missing in the uri, I tried to add "/haystack" in the Uri path as below. The Uri parameter in the method is set as "http://myServer/haystack/" which itself contains "/haystack/" in it, but for some reason "/haystack/" somehow gets removed:

public virtual async Task OpenAsync() { await _authenticator.Authenticate(_client, new Uri(Uri, "/haystack/user/auth")); }

After making the above change to the OpenAsync code, Iam able to get 200 Success for the Authentication as below:

{StatusCode: 200, ReasonPhrase: OK, Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:{ Set-Cookie: JSESSIONID=Id;Path=/;HttpOnly Set-Cookie: JSESSIONID=Id;Path=/;HttpOnly Set-Cookie: JSESSIONID=Id;Path=/;HttpOnly X-Frame-Options: sameorigin Cache-Control: no-store, must-revalidate, no-cache, max-age=0 Pragma: no-cache Access-Control-Allow-Origin: * Transfer-Encoding: chunked Expires: Mon, 17 Aug 2020 03:41:05 GMT Last-Modified: Mon, 17 Aug 2020 03:41:05 GMT Content-Type: text/zinc; charset=utf-8}}

Thanks a lot for helping me to figure out the Authentication issue Chris.

Hi Indumathi,

Glad to see you got it to work!

You showed me that there is a flaw in the logic of the HttpAsyncClient: It expects that there is an authorization page on the <server-root>/uesr/auth url, but it can be something else. In the next release of the lib I'll add a way to explicitly set the value. This won't fix your issue though, but it might help people with different Haystack implementations. I'll also add an implementation for the VeryBasicAuthenticator (with a more descriptive name). Those changes together should make it possible for you to set up the connection more easily in future projects.

I have just added the promised changes: There is now a new package (1.5.0) available, which contains the new NonHaystackBasicAuthenticator class which is a copy of the VeryBasicAuthenticator code I sent you before. If you catch any more issues, please create a PR with the fix, or send me a diff of your fix. Just make sure to also specifically set the AuthUri after creating the client.

Also I have updated the documentation (finally) to describe the usage of the library with more code samples. Please check it out here

Thanks a lot Chris, for quickly adding the new NonHaystackBasicAuthenticator class to the package. It will be super helpful for my development. Really appreciate your support.

Hi Chris,

I downloaded the latest ProjectHaystack.Client v1.5.0 and tried to use the new NonHaystackBasicAuthenticator class as given in the code example.

using ProjectHaystack.Auth;

var auth = new NonHaystackBasicAuthenticator(user, pass);

var client = new HttpAsyncClient(auth, uri);

But Iam getting error:"The type or namespace name NonHaystackBasicAuthenticator could not be found(Are you missing a using directive or an assembly reference?)"

And the fixes is asking me to Generate the NonHaystackBasicAuthenticator class in the new file.

This is not the case with the already existing Authenticator's like "BasicAuthenticator" and "ScramAuthenticator". They are getting recognized correctly as ProjectHaystack.Auth methods.

Also checked if the new class is available in the list of options under ProjectHaystack.Auth but it is not listed there.

Thought of informing you about it.

Sorry, it appears I made a mistake when building the NuGet package (we still need to automate that process). I have now built it correctly (and double-checked it) and released it as 1.5.1 (1.5.0 is now de-listed).

Thanks a lot Chris, I downloaded the new v1.5.1 and now I am able to access the new class NonHaystackBasicAuthenticator as expected.

Just thought of letting you know that, I had mentioned it earlier that even though I send my url as "http://myServer/haystack/" in client code like below

var client = new HttpAsyncClient(auth, new Uri("http://myServer/haystack/"));

The "/haystack" segment gets removed while getting set in the AuthUri in HttpAsyncClient constructor at the below line even though Uri parameter is:"http://myServer/haystack/"

AuthUri = new Uri(Uri, "/user/auth");

The AuthUri is always getting set to http://myServer/user/auth instead of http://myServer/haystack/user/auth

This is throws the Error 404 Not Found Problem accessing /user/auth.

Once I updated my code like below to include segment "/haystack/" in the AuthUrl if it does not exist, then I get status code 200 ok for authentication:

public async Task Authenticate(HttpClient client, Uri authUrl) {

client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
   Convert.ToBase64String(Encoding.UTF8.GetBytes(_username + ":" + _password)).Trim('='));

string newAuthUrl = authUrl.ToString();
if (!newAuthUrl.Contains("haystack"))
{
    int index = newAuthUrl.IndexOf('/', 10);
    newAuthUrl = authUrl.ToString().Insert(index, "/haystack");//19
}
try
{
    using (var response = await client.GetAsync(newAuthUrl))
    {
        if ((int)response.StatusCode != 200)
        {
            throw new AuthException("Basic auth failed: " + response.StatusCode + " " + (await response.Content.ReadAsStringAsync()));
        }
        return;
    }
}
catch (Exception e)
{
    throw new AuthException("basic authentication failed", e);
}

}

If it is possible to update this code in the Nuget pakage then I can use it directly in my project. Thanks Chris

Hi Indumathi,

As the /user/auth is a default value I have made it possible to override it in the client. Just set the AuthUri property to the correct value for your server, like so:

client.AuthUri = new Uri("http://myServer/haystack/user/auth");

Thanks a lot for your support Chris, everything works now in the ProjectHaystack.Client v1.5.1 for my server.