Base solution for your next web application
Open Closed

InternalServerError due to Redis Timeout #10745


User avatar
0
Ricavir created
  • Abp 6.6.1
  • Angular
  • .NET 5

Hi,

I'm getting some logged errors due to Redis cache timeout. The UI shows a message "InternalServerError". Here is the log :

WARN 2021-12-08 08:21:43,756 [102 ] Abp.MultiTenancy.TenantResolver - StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), inst: 0, qu: 19, qs: 0, aw: False, bw: Inactive, rs: ReadAsync, ws: Idle, in: 0, serverEndpoint: redis.cache.windows.net:6380, mc: 1/1/0, mgr: 10 of 10 available, clientName: , IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=17,Free=32750,Min=1,Max=32767), v: 2.2.62.27853 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)
at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 2851
at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/RedisBase.cs:line 54
at StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) in /_/src/StackExchange.Redis/RedisDatabase.cs:line 2409
at Abp.Runtime.Caching.Redis.AbpRedisCache.TryGetValue(String key, Object& value)
at Abp.Runtime.Caching.AbpCacheBase`2.Get(TKey key, Func`2 factory)
at Abp.Runtime.Caching.TypedCacheWrapper`2.Get(TKey key, Func`2 factory)
at Abp.MultiTenancy.TenantCache`2.GetOrNull(String tenancyName)
at Abp.MultiTenancy.TenantStore.Find(String tenancyName)
at Abp.AspNetCore.MultiTenancy.DomainTenantResolveContributor.ResolveTenantId()
at Abp.MultiTenancy.TenantResolver.GetTenantIdFromContributors()
StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), inst: 0, qu: 19, qs: 0, aw: False, bw: Inactive, rs: ReadAsync, ws: Idle, in: 0, serverEndpoint: redis.cache.windows.net:6380, mc: 1/1/0, mgr: 10 of 10 available, clientName: , IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=17,Free=32750,Min=1,Max=32767), v: 2.2.62.27853 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)
at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 2851
at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in /_/src/StackExchange.Redis/RedisBase.cs:line 54
at StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) in /_/src/StackExchange.Redis/RedisDatabase.cs:line 2409
at Abp.Runtime.Caching.Redis.AbpRedisCache.TryGetValue(String key, Object& value)
at Abp.Runtime.Caching.AbpCacheBase`2.Get(TKey key, Func`2 factory)
at Abp.Runtime.Caching.TypedCacheWrapper`2.Get(TKey key, Func`2 factory)
at Abp.MultiTenancy.TenantCache`2.GetOrNull(String tenancyName)
at Abp.MultiTenancy.TenantStore.Find(String tenancyName)
at Abp.AspNetCore.MultiTenancy.DomainTenantResolveContributor.ResolveTenantId()
at Abp.MultiTenancy.TenantResolver.GetTenantIdFromContributors()

Do you think that this can be linked with multiple requests to Redis due to GetAll requests ? Is "Per Request Redis Cache" implementation will solve that ?


5 Answer(s)
  • User Avatar
    0
    ismcagdas created
    Support Team

    Hi @ricavir

    Probalby it is because of high amount of requests to Redis. We assume that Per Request Redis Cache will solve this problem. By the way, is it possible to clear your redis cache and try using PerRequestRedisCache again ?

    Thanks,

  • User Avatar
    0
    Ricavir created

    Hi @ismcagdas,

    I just tried PerRequestRedisCache with a fresh new redis server. I'm getting same exception as explained here : Unable to cast object of type 'Abp.MultiTenancy.TenantCacheItem

    Here is the status of the redis database before running PerRequestRedisCache and after exceptions occurs :

  • User Avatar
    0
    ismcagdas created
    Support Team

    Hi @ricavir

    ABP 6.6.2 is released, could you try with that ?

  • User Avatar
    0
    Ricavir created

    Hi @ismcagdas,

    I'm currently trying on my dev machine. Everything works great ! I notice a huge performance improvement between PerRequestRedisCache activated and deactivated. I will deploy it on Azure today and give a feedback if something goes wrong. Thanks for your 5* support @ismcagdas :)

  • User Avatar
    0
    ismcagdas created
    Support Team

    Thank you for trying it so fast :)