Issues with SignalR when scaling pods in Azure using Docker and Kubernetes #11699

11 Answer(s)

• 

  0

  ismcagdas created 8 months ago

  Support Team

  Hi,

  Have you implemented this on your project https://docs.aspnetzero.com/en/aspnet-core-angular/latest/Clustered-Environment#scaling-signalr ? If not, it might be the cause of the problem.

  Save Cancel
• 

  0

  mpineiro created 8 months ago

  Hi ismcagdas,

  Yes, we have configured the sticky sessions and the redis backplane, before we configured it we had some errors in the console about the signalR connection, but those have been resolved.

  A few days ago we added a test button on the frontend to test the notifications and it works fine, the problem we have now is when the worker sends the request to the backend, so we think it might be something related to the sessions or cache.

  Save Cancel
• 

  0

  ismcagdas created 8 months ago

  Support Team

  Hi,

  Is it possible to share the request details of worker sending request to server ? I mean the JSON data. If possible, it would be nice to test this on your app.

  Save Cancel
• 

  0

  mpineiro created 7 months ago

  Hi,

  Is it possible to share the request details of worker sending request to server ? I mean the JSON data.
  If possible, it would be nice to test this on your app.

  Hello ismcagdas,

  I show you the method involved in this workflow

  SendNotificationAsync (method in charge of sending the notification to the backend) - parameter CancellationToken is not used in this case

  public async Task SendNotificationAsync(WorkerNotificationData notification, CancellationToken cancellationToken = default)
          {
              if (notification is null)
              {
                  throw new ArgumentNullException(nameof(notification));
              }
  
              var baseUrl = BaseUri.AbsoluteUri;
              var url = new Uri(new Uri(baseUrl + (baseUrl.EndsWith("/") ? "" : "/")), "api/internals/notifications/v1/SendWorkerNotifications").ToString();
              
              var content = notification.SerializeToJson();
              var request = new HttpRequestMessage(HttpMethod.Post, new Uri(url))
              {
                  Content = new StringContent(
                      content: content,
                      encoding: Encoding.UTF8,
                      mediaType: "application/json")
              };
              request.Headers.Add(UserAgentHeaderName, _userAgent);
  
              using var client = new HttpClient();
              using var response = await client.SendAsync(request, cancellationToken);
          }
  

  SendWorkerNotifications endpoint in the backend

  [UnitOfWork(IsDisabled = true)]
  [HttpPost("SendWorkerNotifications")]
  public async Task<IActionResult> SendWorkerNotifications(WorkerNotificationData notification)
  {
      const string Target = TargetPrefix + "SendWorkerNotifications";
      
      if (notification.UserIds == null || notification.UserIds.Length == 0)
      {
          throw new ValidationException("Missing UserIds.");
      }
      if (string.IsNullOrWhiteSpace(notification.Message))
      {
          throw new ValidationException("Missing Message.");
      }
  
      UserIdentifier[] users = new UserIdentifier[notification.UserIds.Length];
      for (int i = 0; i < notification.UserIds.Length; i++)
      {
          users[i] = new UserIdentifier(notification.TenantId > 0 ? notification.TenantId : null, notification.UserIds[i]);
      }
  
      var data = new MessageNotificationData(notification.Message);
      if (notification.DownloadUrl.IsPresent())
      {
          data["downloadUrl"] = notification.DownloadUrl;
      }
  
      using (var uow = UnitOfWorkManager.Begin())
      using (var uowt = UnitOfWorkManager.Current.SetTenantId(notification.TenantId))
      {
          await _notificationPublisher.PublishAsync(
              notification.NotificationName,
              data: data,
              userIds: users,
              severity: notification.Severity);
  
          await uow.CompleteAsync();
      }
  
      return new OkResult();
  }
  

  WorkerNotificationData model

  public class WorkerNotificationData
  {
  	/// <summary>
  	/// Nombre de la notificación.
  	/// </summary>
  	public string NotificationName { get; set; }
  
  	/// <summary>
  	/// Mensaje a enviar.
  	/// </summary>
  	public string Message { get; set; }
  
  	/// <summary>
  	/// Suscriptor.
  	/// </summary>
  	public int TenantId { get; set; }
  
  	/// <summary>
  	/// Usuarios a los que que se envía la noificación.
  	/// </summary>
  	public long[] UserIds { get; set; } = Array.Empty<long>();
  
  	/// <summary>
  	/// Url opcional a incluir en la notificación.
  	/// </summary>
  	public string DownloadUrl { get; set; }
  
  	/// <summary>
  	/// Notification severity.
  	/// </summary>
  	public NotificationSeverity Severity { get; set; } = NotificationSeverity.Success;
  }
  

  Here are the headers and content of an example request:

  {
    "Content-Length": "336",
    "Content-Type": "application/json; charset=utf-8",
    "Host": "localhost:44301",
    "User-Agent": "Worldsys.Compliance.Notifications.InternalNotificationClient/10.3.0.0"
  }
  
  
  {
    "notificationName": "Notification.BackgroundJob",
    "message": "La exportación está lista haz click aquí para descargar.",
    "tenantId": 184,
    "userIds": [
      551
    ],
    "downloadUrl": "/File/DownloadBinaryFile?id=1cddf33f-bfd9-1ceb-fe92-3a0d635c03e8&contentType=text/csv&fileName=Batch_2023-03-22T19_40_42.csv",
    "severity": "Success"
  }
  

  I have removed all the try catch statements to shorten the code in this answer.

  Save Cancel
• 

  1

  ismcagdas created 8 months ago

  Support Team

  Hi,

  This all seems fine. Could you create a class by copying the code of https://github.com/aspnetboilerplate/aspnetboilerplate/blob/dev/src/Abp.AspNetCore.SignalR/AspNetCore/SignalR/Notifications/SignalRRealTimeNotifier.cs and then replace SignalRRealTimeNotifier with your CustomSignalRRealTimeNotifier and then add logs to understand if line below is called;

  await signalRClient.SendAsync("getNotification", userNotification);

  In order to replace SignalRRealTimeNotifier, you can do it as shown below;

  Configuration.ReplaceService<IRealTimeNotifier, CustomSignalRRealTimeNotifier>(DependencyLifeStyle.Transient);
  

  Save Cancel

  1
• 

  0

  mpineiro created 8 months ago

  Hi,

  This all seems fine. Could you create a class by copying the code of https://github.com/aspnetboilerplate/aspnetboilerplate/blob/dev/src/Abp.AspNetCore.SignalR/AspNetCore/SignalR/Notifications/SignalRRealTimeNotifier.cs and then replace SignalRRealTimeNotifier with your CustomSignalRRealTimeNotifier and then add logs to understand if line below is called;

  await signalRClient.SendAsync("getNotification", userNotification);

  In order to replace SignalRRealTimeNotifier, you can do it as shown below;

  Configuration.ReplaceService<IRealTimeNotifier, CustomSignalRRealTimeNotifier>(DependencyLifeStyle.Transient); 
  

  Hello ismcagdas,

  I did the changes you said, and i could see that when im triggering a notification from the front to backend, the onlineClients in the CustomSignalRRealTimeNotifier.cs file have values, meanwhile if i trigger the notification from worker to backend, this onlineClients is empty.

  I tried applying this changes https://github.com/aspnetboilerplate/aspnetboilerplate/commit/b58d50ad5796da2cf1bf060fe791d33652700ba9 on the project, but i couldnt understand how to do it without modifying the abp files.

  Thanks in advance

  Save Cancel
• 

  0

  ismcagdas created 8 months ago

  Support Team

  Hi,

  If you can't manually apply changes on related PR to your app, maybe you can upgrade to ABP 7.4. It sohuld automatically use RedisOnlineClientStore. There are not much breaking changes between those versions as far as I remember.

  Save Cancel
• 

  0

  mpineiro created 8 months ago

  Hello ismcagdas,

  We have tried to upgrade the whole platform version, from net 5 to net 7, including angular and abp, but it didn't end up working properly, so we decided to move on with other tasks, and resume the upgrade when we have the code more separated.

  Would there be a way to apply the changes to these files in another way, for example using something like ReplaceServices or something like that?

  Save Cancel
• 

  0

  ismcagdas created 7 months ago

  Support Team

  Hi,

  Yes, actually if you copy RedisOnlineClientStore from https://github.com/aspnetboilerplate/aspnetboilerplate/blob/dev/src/Abp.RedisCache/Runtime/Caching/Redis/RealTime/RedisOnlineClientStore.cs to your project and the replace the default implementation using the code below, it should work;

  Configuration.ReplaceService<IOnlineClientStore,RedisOnlineClientStore>();
  

  Save Cancel

  1
• 

  0

  mpineiro created 7 months ago

  Hello ismcagdas,

  It seems to be working fine after adding the replaceService line and the RedisOnlineClientStore file.

  Thanks!

  Save Cancel
• 

  0

  ismcagdas created 7 months ago

  Support Team

  Great to hear that :)

  Save Cancel