Redis Backplane - AddStackExchangeRedis #6425 | Support Center

cmthomps created 7 years ago

Have you implemented AspNetZero using the the StackExchange Redis Backplane? Trying to tackle SignlaR behind a load balancer using Set up a Redis backplane for ASP.NET Core SignalR scale-out. I've added the StackExchangeRedis NuGet package and called AddStackExchangeRedis after the AddSignalR method. Everything seems to work ok, but SignalR only works for users that are connected to the same server. If user A is on server 1 and user B is on server 2, it doesn't work. If A & B are both on Server 1, it works. This is the code we've implemented in the Startup ConfigureServices.

var signalrBuilder = services.AddSignalR(o =>
{
    o.EnableDetailedErrors = true;
});

signalrBuilder.AddStackExchangeRedis(_appConfiguration["SignalR:ConnectionString"], options => {
    options.Configuration.ChannelPrefix = "zerochannel";
});

I can see all the servers "subscribe" in the redis logs. But for some reason, when I message goes out of one server the other servers are not picking it up. This is an example of what I see in Redis. I'm not entirely sure the meaning/importance of the names on the SUBSCRIBE lines.

1549404094.848664 [0 172.17.0.12:37121] "AUTH" "XXXXXXX"
1549404094.848693 [0 172.17.0.12:37121] "ECHO" "\x89\xcf\xf7\xf4>\xaf C\x9cz\x88\xd8\xf1\x98\xb6\x05"
1549404094.848704 [0 172.17.0.12:37121] "SUBSCRIBE" "zerochannel__Booksleeve_MasterChanged"
1549404094.864930 [0 172.17.0.12:36336] "AUTH" "5NSAjVIyvGmfpBot"
1549404094.865012 [0 172.17.0.12:36336] "INFO"
1549404094.865086 [0 172.17.0.12:36336] "ECHO" "\x89\xcf\xf7\xf4>\xaf C\x9cz\x88\xd8\xf1\x98\xb6\x05"
1549404094.866966 [0 172.17.0.12:36336] "PING"
1549404094.866995 [0 172.17.0.12:36336] "GET" "__Booksleeve_TieBreak"
1549404094.867013 [0 172.17.0.12:36336] "PING"
1549404094.867017 [0 172.17.0.12:36336] "GET" "__Booksleeve_TieBreak"
1549404094.867023 [0 172.17.0.12:36336] "INFO" "replication"
1549404095.289384 [0 172.17.0.12:37121] "SUBSCRIBE" "zerochannelSMCDemo.Web.Chat.SignalR.ChatHub:all"
1549404095.333222 [0 172.17.0.12:37121] "SUBSCRIBE" "zerochannelSMCDemo.Web.Chat.SignalR.ChatHub:internal:groups"
1549404095.366710 [0 172.17.0.12:37121] "SUBSCRIBE" "zerochannelSMCDemo.Web.Chat.SignalR.ChatHub:internal:ack:smczero-deployment-858c6845f9-sn8p4_056b26b232fe4cc99fdee32792553850"

19 Answer(s)

0
cmthomps created 7 years ago
@ismcagdas - I got notifications working in a load balanced environment. What I found was that there maybe an issue with SignalRRealTimeNotifier in the Abp.AspNetCore.SignalR. In that code, there is a line that says:

var signalRClient = _hubContext.Clients.Client(onlineClient.ConnectionId);

If I change that line of code to:

var signalRClient = _hubContext.Clients.All;

I'll be completely honest, I don't really know why it works. I was going through Microsoft's example on the redis backplane and found this difference. When I changed it, it started working (at least in my initial tests).

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

aaron created 7 years ago
Support Team

Does onlineClient.ConnectionId have a valid value?

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)

cmthomps created 7 years ago

It looks like the ConnectionId is valid. I starting logging the UserId and the ConnectionId. This what that looks like:

UserID: 1 ConnectionId: wRX8xHl-XqgDsKZ8zGunNA

When I use var signalRClient = _hubContext.Clients.Client(onlineClient.ConnectionId);, I do not see any activity on the redis server. When I use Clients.All, I start to see activity on the Redis server that looks like:

<br>

1549554528.516258 [0 172.17.0.9:45157] "INFO" "replication"
1549554536.048210 [0 172.17.0.1:61383] "PUBLISH" "smczeroSmc.Abp.AspNetCore.SignalR.Hubs.AbpCommonHub:all" "\x92\x90\x81\xa4json\xc5\x02E{\"type\":1,\"target\":\"getNotification\",\"arguments\":[{\"tenantId\":null,\"userId\":1,\"state\":0,\"notification\":{\"tenantId\":null,\"notificationName\":\"App.TestNotification\",\"data\":{\"message\":{\"sourceName\":\"SMCDemo\",\"name\":\"TestNotification\"},\"type\":\"Abp.Notifications.LocalizableMessageNotificationData\",\"properties\":{\"message\":{\"sourceName\":\"SMCDemo\",\"name\":\"TestNotification\"}}},\"entityType\":null,\"entityTypeName\":null,\"entityId\":null,\"severity\":2,\"creationTime\":\"2019-02-07T10:48:52.744841-05:00\",\"id\":\"4defe4c2-997f-4cc2-9368-fda4a8ec947d\"},\"id\":\"2aeb9a04-d11a-4118-8ef8-5e592b0b06ca\"}]}\x1e"
1549554538.425415 [0 127.0.0.1:54504] "AUTH" "5NSAjVIyvGmfpBot"

I don't really know what the implications are to using Clients.All. Do you?

Thanks, Craig

0

aaron created 7 years ago
Support Team

Using Clients.All means sending it to all clients.

According to aspnet/SignalR#2460, sending to a specific client should work.

You can create a new issue referencing that here: https://github.com/aspnet/AspNetCore/issues/new

Related: aspnet/AspNetCore#5358

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0
cmthomps created 7 years ago
Here's what seems to have fixed it for me. I don't really understand why, but it seems to work nonetheless.

In the SendNotificationsAsync method, I changed the line that says:

signalRClient.SendAsync("getNotification", userNotification);

to

_hubContext.Clients.Client(onlineClient.ConnectionId).SendAsync("getNotification", userNotification);

Does that make any sense to you? if I use the signalRClient object, it doesn't seem to send the notifications properly.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

aaron created 7 years ago
Support Team

No, it's the same.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

cmthomps created 7 years ago

Not sure. The first one does not work with Redis the second one does. I guess others will find this if they need it.

I'd like to vote for AspNetZero to enhance the product to support scale out using load balancing. Given the popularity of kubernetes, having this capability will be extremely valuable. I've got it running on kubernetes now, but finding that notifications and chat do not work. We got notifications to work using the hack above and I'm trying to get chat working with Redis using the code developed by another user. Haven't had any luck yet with Chat. All credit to personball on github.

RedisOnlineClientManager.cs

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

aaron created 7 years ago
Support Team

Try reverting the change.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

cmthomps created 7 years ago

Thanks @aaron. You're right.... That leads me to my next question. As part of this, I've created my own module that represents the ABP SignalRRealTimeNotifier called SmcSignalRRealTimeNotifier. In there, I've defined a my own AbpCommonHub called SmcAbpCommonHub. The code in the module is exactly the same as what is in ABP (Abp.AspNetCore.SignalR/AspNetCore/SignalR/).

When I use the AbpCommonHub in the Mvc startup.cs, the redis notifications do not work. When I override the SignalRRealTimeNotifier dependency with my own SmcSignalRRealTimeNotifier and use the SmcAbpCommonHub, the redis notifications start working. Can you think of a reason why that is?

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0
aaron created 7 years ago
Support Team
How are you:

subscribing on the client side, and

using SignalRRealTimeNotifier on the server side

before and after switching to SmcSignalRRealTimeNotifier?

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)

cmthomps created 7 years ago

I made no changes to the client side of things. In the Web.Mvc project, I changed a couple of things:

In the Startup.cs ConfigureServices method, I changed

            var signalrBuilder = services.AddSignalR();

                var signalrBuilder = services.AddSignalR();
                signalrBuilder.AddStackExchangeRedis(_appConfiguration["SignalR:ConnectionString"], options => {
                        options.Configuration.ChannelPrefix = "smczero";
                        });
                }

In the Configure method, I changed

            app.UseSignalR(routes =>
            {
                routes.MapHub<AbpCommonHub>("/signalr");
                routes.MapHub<ChatHub>("/signalr-chat");
            });

            app.UseSignalR(routes =>
            {
                routes.MapHub<SmcAbpCommonHub>("/signalr");
                routes.MapHub<ChatHub>("/signalr-chat");
            });

The last thing I did was register the register the SmcSignalRRealTimeNotifier as IRealTimeNotifier in my Web.Core project which is where the new module code lives.

0

aaron created 7 years ago
Support Team

You didn't answer any of the 2 questions, which were about usage and not configuration.

If you're saying usage didn't change and the 2 replaced classes are subclasses that don't override or add any method, then they are pointless changes similar to assigning a variable as before.

Try reverting those changes.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

cmthomps created 7 years ago

Here's a question for you. Have you tried to reproduce the issue I'm having on your machine with Redis and the StackExchangeRedis nuget package? When I run a straight, out of the box download of AspNetZero with no changes other than adding the StackExchangeRedis nuget package and adding the call to AddStackExchangeRedis in Startup.cs, the notifications do not work. Don't know what else to tell you.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

aaron created 7 years ago
Support Team

No, because it is not a problem with ASP<span></span>.NET Zero or ABP.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

ismcagdas created 7 years ago
Support Team

Hi @cmthomps

Sorry for our late reply. Did you find any solution for this ? I just wanted to check before setting up such an environment to test this problem.

Thanks,

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

cmthomps created 7 years ago

Thanks @ismcagdas. I have a private github repository that I just shared with you. It contains an AspNetZero demo project that uses Redis for both Chat and Notifications. I've done some minimal testing. But it appears to work on Kubernetes running multiple pods behind a load balancer. Not sure if it's something that can be incorporated into the base product. For us, having the ability to run behind a load balancer is very important.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

ismcagdas created 7 years ago
Support Team

I got the invitation and I will try to check it on weekend.

Thanks,

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

ismcagdas created 7 years ago
Support Team

Hi @cmthomps

Have you figure this problem out ? It seems like the team didn't have time to check this.

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)
0

cmthomps created 7 years ago

Thanks @ismcagdas - Honestly, I have a workaround that seems to be working for us. The last time I checked, notifications do not work when using the StackExchange Redis Backplane but I have tried the v6.9.

Craig

Markdown is supported

Copy & paste or drag & drop images (max 30 MB per image)

Have an answer to this question? Log in and write your answer.