Activities of "Web2workNL"
thanks for the hint. solution implemented
product version : Abp.AspNetCore 6.6.1 product type: Angular framework type: .net core
I woudl like to add caching for AbpUsers table. please advice a proper method to do this
product version : Abp.AspNetCore 6.6.1 product type: Angular framework type: .net core
our applicaiton That is build on ABP framework makes useless requests to database and this implies unneded costs for our database that is HOsted in Cloud.
for last month we ahve over 100 millions requests made to tables [AbpUserOrganizationUnits] [AbpOrganizationUnitRoles] that we do not use and have no data there.
i need a to find a way to disable this requests as it Brings aditional cost to MSSQL server hosting and has no value for our product
please advice how this can be acheived
Thanks in advance Regards
Thanks for the suggestion! I was unaware that ABP had something similar out of the box 👍.
I have a few questions about the implementation of the PerRequestRedisCache though
- Does this cache stuff to memory like the MemoryCache implementation does, or does it cache for the specific current connection so THIS user doesn't have to go to redis if they need the same value again
- We have a scale out solution with multiple instances, so we've set the expiry on the memory cache really low so it expires quickly and is updated from Redis frequently. Is it possible to set the expiry of the memorycache with PerRequestRedisCache?
- What is your product version? ABP Nuget packages 6.6.1
- What is your product type (Angular or MVC)? ANGULAR
- What is product framework type (.net framework or .net core)? .NET 5
Yesterday we rolled out a new version of our application where we updated from using version 4.12 of the ABP packages to 6.6.1
We found an unexpected issue with our caching.
We implemented our own hybrid solution, where we try to get values from the memory cache which has a small expiry and if it's not there, we get it from Redis and put in the Memory cache for a next time.
Previously we had a small amount of redis cache hits and most of the work was done by memory cache.
We made some alterations to make it possible to upgrade from 4.12 to 6.6.1, but unfortunately the colleague that made these changes for the caching has since left the company.
Can you help us understand what might have changed in ABP that could cause this, or how our solution might be responsible for this. If you need any further info, please let me know!
`namespace xxx.xxx.Web.Caching { using System; using System.Collections.Generic; using System.Threading.Tasks; using Abp.Runtime.Caching; using Abp.Runtime.Caching.Memory; using Abp.Runtime.Caching.Redis; using Abp.Timing;
/// <summary>
/// Create a cache existing of two caching layers, the first being a memory cache, the second being a Redis cache.
/// The memory cache expires quickly, so a change on another AppService instance is -through the use of the Redis cache- propagated quickly to the memory cache of this instance
/// Create, update and delete actions are always applied to the memory cache AND the Redis cache
/// Getting an item from cache will try to get it from the memory cache. If this doesn't succeed, it will try to get it from the Redis cache.
/// </summary>
public class MemoryAndRedisCache : CacheBase
{
private readonly AbpMemoryCache _memoryCache;
private readonly AbpRedisCache _redisCache;
/// <summary>
/// Constructor.
/// </summary>
public MemoryAndRedisCache(string name, AbpMemoryCacheManager memoryCacheManager, AbpRedisCacheManager redisCacheManager) : base(name)
{
// Set expiry to 1 minute for the memory cache, so a change on another AppService instance is propagated quickly to the memory cache of this instance
_memoryCache = (AbpMemoryCache)memoryCacheManager.GetCache(name + "_memory");
_memoryCache.DefaultAbsoluteExpireTime = Clock.Now.AddMinutes(1);
// Expiry of the Redis cache is done through normal configuration in ABP
_redisCache = (AbpRedisCache)redisCacheManager.GetCache(name);
}
public override bool TryGetValue(string key, out object value)
{
// Try to get the value from memory
var memorySuccess = _memoryCache.TryGetValue(key, out value);
if (memorySuccess)
{
var expiryInfo = _memoryCache.DefaultAbsoluteExpireTime != null ? _memoryCache.DefaultAbsoluteExpireTime.ToString() : _memoryCache.DefaultSlidingExpireTime.ToString();
Logger.Debug($"Get from {this.Name}: {key} [memory] [{expiryInfo}]");
return true;
}
// Getting from memory failed, try to get it from Redis
var redisSuccess = _redisCache.TryGetValue(key, out value);
if (redisSuccess)
{
var expiryInfo = _redisCache.DefaultAbsoluteExpireTime != null ? _redisCache.DefaultAbsoluteExpireTime.ToString() : _redisCache.DefaultSlidingExpireTime.ToString();
Logger.Debug($"Get from {this.Name}: {key} [redis] [{expiryInfo}]");
// We got the value from Redis, now store it in memory cache for repeated use
_memoryCache.Set(key, value, null, null);
return true;
}
return false;
}
public override object GetOrDefault(string key)
{
// Try to get the value from memory
var value = _memoryCache.GetOrDefault(key);
if (value != null)
{
var expiryInfo = _memoryCache.DefaultAbsoluteExpireTime != null ? _memoryCache.DefaultAbsoluteExpireTime.ToString() : _memoryCache.DefaultSlidingExpireTime.ToString();
Logger.Debug($"Get from {this.Name}: {key} [memory] [{expiryInfo}]");
return value;
}
// Getting from memory failed, try to get it from Redis
value = _redisCache.GetOrDefault(key);
if (value != null)
{
var expiryInfo = _redisCache.DefaultAbsoluteExpireTime != null ? _redisCache.DefaultAbsoluteExpireTime.ToString() : _redisCache.DefaultSlidingExpireTime.ToString();
Logger.Debug($"Get from {this.Name}: {key} [redis] [{expiryInfo}]");
// We got the value from Redis, now store it in memory cache for repeated use
_memoryCache.Set(key, value, null, null);
}
else
{
Logger.Debug($"Get from {this.Name}: {key} [source]");
// The caching logic of ABP will call the function to get the data from the source - no need to do anything
}
return value;
}
public override object[] GetOrDefault(string[] keys)
{
var values = _memoryCache.GetOrDefault(keys);
for (var t = 0; t < keys.Length; t++)
{
if (values[t] == null)
{
values[t] = _redisCache.GetOrDefault(keys[t]);
}
}
return values;
}
public override async Task<object> GetOrDefaultAsync(string key)
{
// Try to get the value from memory
var value = await _memoryCache.GetOrDefaultAsync(key);
if (value != null)
{
var expiryInfo = _memoryCache.DefaultAbsoluteExpireTime != null ? _memoryCache.DefaultAbsoluteExpireTime.ToString() : _memoryCache.DefaultSlidingExpireTime.ToString();
Logger.Debug($"Get from {this.Name}: {key} [memory] [{expiryInfo}]");
return value;
}
// Getting from memory failed, try to get it from Redis
value = await _redisCache.GetOrDefaultAsync(key);
if (value != null)
{
var expiryInfo = _redisCache.DefaultAbsoluteExpireTime != null ? _redisCache.DefaultAbsoluteExpireTime.ToString() : _redisCache.DefaultSlidingExpireTime.ToString();
Logger.Debug($"Get from {this.Name}: {key} [redis] [{expiryInfo}]");
// We got the value from Redis, now store it in memory cache for repeated use
await _memoryCache.SetAsync(key, value, null, null);
}
else
{
Logger.Debug($"Get from {this.Name}: {key} [source]");
// The caching logic of ABP will call the function to get the data from the source - no need to do anything
}
return value;
}
public override async Task<object[]> GetOrDefaultAsync(string[] keys)
{
var values = await _memoryCache.GetOrDefaultAsync(keys);
for (var t = 0; t < keys.Length; t++)
{
if (values[t] == null)
{
values[t] = await _redisCache.GetOrDefaultAsync(keys[t]);
}
}
return values;
}
public override void Set(string key, object value, TimeSpan? slidingExpireTime = null, DateTimeOffset? absoluteExpireTime = null)
{
_memoryCache.Set(key, value, null, null);
_redisCache.Set(key, value, null, null);
}
public override async Task SetAsync(string key, object value, TimeSpan? slidingExpireTime = null, DateTimeOffset? absoluteExpireTime = null)
{
await _memoryCache.SetAsync(key, value, null, null);
await _redisCache.SetAsync(key, value, null, null);
}
public override void Set(KeyValuePair<string, object>[] pairs, TimeSpan? slidingExpireTime = null, DateTimeOffset? absoluteExpireTime = null)
{
_memoryCache.Set(pairs, null, null);
_redisCache.Set(pairs, null, null);
}
public override async Task SetAsync(KeyValuePair<string, object>[] pairs, TimeSpan? slidingExpireTime = null, DateTimeOffset? absoluteExpireTime = null)
{
await _memoryCache.SetAsync(pairs, null);
await _redisCache.SetAsync(pairs, null, null);
}
public override void Remove(string key)
{
_memoryCache.Remove(key);
_redisCache.Remove(key);
}
public override async Task RemoveAsync(string key)
{
await _memoryCache.RemoveAsync(key);
await _redisCache.RemoveAsync(key);
}
public override void Remove(string[] keys)
{
_memoryCache.Remove(keys);
_redisCache.Remove(keys);
}
public override async Task RemoveAsync(string[] keys)
{
await _memoryCache.RemoveAsync(keys);
await _redisCache.RemoveAsync(keys);
}
public override void Clear()
{
_memoryCache.Clear();
_redisCache.Clear();
}
public override void Dispose()
{
_memoryCache.Dispose();
_redisCache.Dispose();
}
}
} `
Yes, the PermissionManager is created through dependency injection. So is the _session btw private readonly IAbpSession _session;
The background job is not a timed job but is handled like the notifications are (through AbpBackgoundJobs)
await _backgroundJobManager.EnqueueAsync<SyncWithMooJob, SyncWithMooJobArgs>(new SyncWithMooJobArgs(AbpSession.GetTenantId()));
And
public class SyncWithMooJob : BackgroundJob<SyncWithMooJobArgs>, ITransientDependency
{
private readonly IMooManager _mooSyncManager;
public SyncWithMooJob(IMooManager mooSyncManager)
{
_mooSyncManager = mooSyncManager;
}
public override void Execute(SyncWithMooJobArgs args)
{
AsyncHelper.RunSync(() => _mooSyncManager.AutomaticSyncWithMoo(args.TenantId));
}
}
}
If I strip the method down to just
public async Task AutomaticSyncWithMoo(int tenantId)
{
try
{
using (CurrentUnitOfWork.SetTenantId(tenantId))
{
using (_session.Use(tenantId, null))
{
var permission = _permissionManager.GetPermissionOrNull(AppPermissions.Pages_Administration_TeacherGroups);
}
}
}
catch (Exception ex)
{
Logger.Error(ex.Message + ex.StackTrace);
}
}
I still get a NULL result. Just for clarity, when I add _permissionManager.GetPermissionOrNull(AppPermissions.Pages_Administration_TeacherGroups); to for instance an AppService there are no issues
Hai,
My project has a situation where I want to get a Permission from the PermissionManager inside a backgroundjob. The permission is defined as follows:
administration.CreateChildPermission(AppPermissions.Pages_Administration_TeacherGroups, L("GroupsForTeachers"), multiTenancySides: MultiTenancySides.Tenant);
inside the AppAuthorizationProvider. When the following code is run
using (CurrentUnitOfWork.SetTenantId(tenantId))
{
using (_session.Use(tenantId, null))
{
var permission = _permissionManager.GetPermissionOrNull(AppPermissions.Pages_Administration_TeacherGroups);
}
}
the permission is always NULL. What am I missing?
We are currently using ABP 4.12.0
We managed to reproduce the behaviour. Please see the GitHub issue.
Hi JakeE,
We would love to hear from you wether you managed to find a solution to the problem other than expiring the caches after 1 second. We are experiencing the same issues (https://support.aspnetzero.com/QA/Questions/9015)
We know about the possibility to clear the cache by hand, but have been unable to reproduce the corrupted cache after doing this.
It doesn't seem to happen everytime, which is why we assume there is a specific scenario somewhere that leads to the tenant filter not being applied in AbpFeatureValueStore.GetTenantFeatureCacheItem(Async)
We believe that when this scenario presents itself just as the cache has been expired, these two situations together lead to the cache being corrupted. Me and my team have been going through our code to find this scenario, but have not had any luck.
We would be very grateful if you could brainstorm with us on what could possibility lead to this.