Open Closed

SignalR 404 on Azure Multiple App Servers #9493


1
-bitman created

Hello ANZ Team,

We are facing a problem in our azure production environment. We get that 404 error code with signalR and we have multiple server instances running. We are getting thousands of 404 errors from it and it is causing the servers to bug down during peak operations and we cannot afford a single minute of downtime. I can also confirm that our settings from azure websockets is ON. We are on Angular + .NET Core v8.2

Please help. This is critical. Thank you!


7 Answer(s)
  • 0
    -bitman created

    I am also attaching here the initial error I get when refreshing my app.

  • 0
    -bitman created

    I believe the issue is explained here.

    https://docs.microsoft.com/en-us/aspnet/core/signalr/scale?view=aspnetcore-3.1

    We don't want to turn on ARR Affinity because it will affect load balancing on the server farm. On the other hand, we cannot continue having thousands of 404 errors from SignalR.

    Please advise.

  • 0
    -bitman created

    Hello ANZ Team,

    Apologies for the late reply. We found out the root cause of the SignalR issue in our Azure app services. It was the ARR Affinity settings turned OFF. Once we turned it back ON, all 404 exceptions went away immediately. As per documentation about ARR Affinity...

    Improve performance of your stateless app by turning Affinity Cookie off, stateful apps should keep this setting on for compatibility.

    That being the case, I saw from the forum post that the API is stateless...

    https://support.aspnetzero.com/QA/Questions/6075/Is-AspNetZero-API-stateless-for-angular-SPA-apps

    I would like to ask if this is the case then why is the API throwing 404s when ARR Affinity is turned ON in azure? Am I missing something?

    In addition to this, I found this post related to the issue.

    https://stackoverflow.com/questions/56795602/signalr-core-is-arr-affinity-needed-when-web-sockets-are-enabled-and-the-azure

    We are at Angular+Core 8.2 version of AspNetZero, if that helps.

    Bottom line, we would want to avail of the load balancing feature of Azure by turning OFF ARR Affinity and at the same time still be able to capitalize on the SignalR feature that is already integrated in your solution.

    Any advise would be appreciated.

    Thank you!

  • 1
    rickfrankel created

    Hi BitMan,

    The issue here is what SignalR backplane are you using? When there are multiple webservers they all need to share a SignalR backplan to be able to communicate correctly with each other.

    The way I solved this was switching to an Azure SignalR service which takes care of all the signalr infrastructure for you.

    Simply setup an Azure SignalR service and then update the following in your startup.cs of the web.host project.

            services
                .AddSignalR(options => { options.EnableDetailedErrors = true; })
                .AddAzureSignalR(options => options.Endpoints = new ServiceEndpoint[]
                {
                    new ServiceEndpoint(_appConfiguration["SignalR:PrimaryConnectionString"], EndpointType.Primary, "Name1"),
                    new ServiceEndpoint(_appConfiguration["SignalR:SecondaryConnectionString"], EndpointType.Secondary, "Name2"),
                });
    

    You will have to setup configuration in your appsettings.json.

    If you don't want to do that then you'll need to try and put in the Redis or DB based backplanes but websocks still would likely want ARR type connections which is why MS recommend moving away from trying to run SignalR infrastructure yourself.

    Cheers

  • 0
    -bitman created

    Hello Rickfrankel,

    Thank you for your help. I will try your suggestion and hopefully will resolve my issues.

    I appreciate it.

    Thank you!

  • 0
    bobingham created

    @rickfrankel - thanks, I've been looking for that code and that little piece of advice for ages.

  • 0
    ismcagdas created
    Support Team

    Hi @rickfrankel

    Thank you for this valuable information.