We are running into caching issues with our application. The problem seems equal to the problem mentioned by another customer https://support.aspnetzero.com/QA/Questions/8897.
AbpZeroTenantFeatures returnzs the wrong entries for a given tenant (Mismatched from what is in the database). Clearning the cache updates a tenants features to the proper list again. Then it will be fine for a period of time and then start returning incorrect features for a tenant until we clear the cache again.
We were running ABP 4.8.1 and have already upgraded to 4.12.0 but this did not solve the problem. Also, we experience this problem with in memory caching and Redis caching.
When we check the cache contents of AbpTenantFeatures for tenant 1 this normally contains the following:
{
"EditionId": 1,
"FeatureValues": {
"App.NarrowcastingFeature": "true",
"App.NarrowcastingMaxScreenCount": "10"
}
}
This corresponds to the TenantFeatureSettings stored in the database for this tenant:
At the moment the cache is “corrupted” for tenant 1 it contains te following:
{
"EditionId": 1,
"FeatureValues": {
"App.AbsenceReporter": "false",
"App.Calendar": "false",
"App.MessagingFeature": "false",
"App.NarrowcastingFeature": "true",
"App.NarrowcastingMaxFeedCount": "2",
"App.NarrowcastingMaxScreenCount": "3",
"App.PrivacyFeature": "false",
}
}
We checked AbpFeatureValueStore.GetTenantFeatureCacheItemAsyncto see if something goes wrong there. The code that gets the features from the database looks ok:
using (var uow = _unitOfWorkManager.Begin())
{
using (_unitOfWorkManager.Current.SetTenantId(tenantId))
{
var featureSettings = await _tenantFeatureRepository.GetAllListAsync();
foreach (var featureSetting in featureSettings)
{
newCacheItem.FeatureValues[featureSetting.Name] = featureSetting.Value;
}
await uow.CompleteAsync();
}
}
However, it seems that the tenant filter is not applied at the moment the problem occurs. We think this is the case because when all TenantFeatureSettings are returned from the database, the last value returned from the table for that setting will be set in the FeatureValues dictionary.
A small LinqPad script that does the same:
// Get tenant features for all tenants
var features = AbpFeatures.Where(f => f.Discriminator == "TenantFeatureSetting").OrderBy(f => f.Id).ToList();
// populate dictionary
var d = new Dictionary<string,string>();
foreach (var feature in features)
{
d[feature.Name] = feature.Value;
}
// show results
d.Dump();
Result:
The result is exactly the same as the items found in the "corrupted" cache.
Now the big question is: why would the tenant filter not be applied? Is there any circumstance that _unitOfWorkManager.Current.SetTenantId(tenantId) does not work? E.g. when we disable the tenant filter? Or when we set the tenant Id ourselves earlier?
7 Answer(s)
-
0
Hi @web2worknl
How do you get feature values at first place ? Do you use
GetFeatureValueOrNullAsyncand see this behavior ? If you can also share in which part of the application you are seeing this behavior, it might help us to reproduce the problem.Thanks,
-
0
We use the default FeatureChecker implementation to check if features are enabled for tenants. Next to this we use the RequiresFeature Attribute on almost all our AppServices. We don't know how this Attribute is implemented behind the scenes.
We don't know where this behavior comes from so it is hard to answer your second question. We only know the end result as we pointed out in the original post. We did suspect using DisableFilter for MayHaveTenant/MustHaveTenant or when using SetTenantId on the CurrentUnitOfWork but were unable to reproduce it using these scenarios.
We also have the assumption that the data gets corrupted only when the cache has expired and data is re-added through either GetTenantFeatureCacheItemAsync or GetTenantFeatureCacheItem.
In summary, we know the end result but not what leads to this.
-
0
Hi @web2worknl,
If you think this happens when the cache expires, can you go to Administration > Maintenance page and clear cache to reproduce this on your app ?
Thanks,
-
0
We know about the possibility to clear the cache by hand, but have been unable to reproduce the corrupted cache after doing this.
It doesn't seem to happen everytime, which is why we assume there is a specific scenario somewhere that leads to the tenant filter not being applied in
AbpFeatureValueStore.GetTenantFeatureCacheItem(Async)We believe that when this scenario presents itself just as the cache has been expired, these two situations together lead to the cache being corrupted. Me and my team have been going through our code to find this scenario, but have not had any luck.
We would be very grateful if you could brainstorm with us on what could possibility lead to this.
-
0
Hi,
I have created an issue about this, you can follow the related issue https://github.com/aspnetboilerplate/aspnetboilerplate/issues/5588.
If you can find a way to reproduce this in the mean time, it would be really helpful.
-
0
We managed to reproduce the behaviour. Please see the GitHub issue.
-
0
Hi @web2worknl
We have applied a fix. You can try ABP's next version when it is released.
Thanks,
