Hi Team,
We planned to develop a high traffic application so we purchased ASPNetZero V11 (LTS) framework. Since we are going to develop a heavy traffic product I want to put the system/framework into Load Testing with api/TokenAuth/Authenticate API endpoint using JMeter. While putting load test we got lot of Response code: 500 Internal Server Error. I checked with AbpAuditLogs about internal exception it’s caused due to “Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool.” We need to know why we are having these issues on Load Test for only 500 User Threads on an interval of 10 sec.
Load Test Tread Group Number of Threads (users): 500 Ramp-up period (seconds): 10 Loop Count: 1
Summary Report
With this I’m attaching JMeter jmx file and AbpAuditLogs export file. We need a quick solution for this issue because we are planning to start with our actual development soon. Note: We are facing the same issue with the Old ASPNetZero V7.0 framework and with this V7.0 we build a project that actually running in production.
Please find the attached JMeter jmx and AbpAuditLogs files: JMeter AbpAuditLogs
Thank you.
9 Answer(s)
-
0
Hi @demo.solaru,
I have also been using JMeter to troubleshoot performance-related issues. In my application, I have disabled the AbpAuditLog as it isn't a requirement for me. Instead I have my own explicit Audit Logging, because we need more business-level action tracking that the raw logs that the AbpAuditLog provide.
Can you proceed with your application without the AbpAuditLog?
The other performance issuein the /Token/Authenticate endpoint that I've encountered has been the JwtToken Validation
await _userManager.AddTokenValidityKeyAsync( user, tokenValidityKey, DateTime.UtcNow.Add(expiration.Value)You do have to store the Token Validity Key in order for the token validation to work correctly. You can cache the token in your cache (Redis).
Something I had explored was moving the JwtToken store to another datastore, like Azure Tables (I deploy to Azure).As a performance test, I'm curious if you disable the AbpAuditLog, and then if you comment out the call to
_userManager.AddTokenValidityKeyAsyncin your TokenAuthController, what your JMeter performance looks like.I hope that helps, -Brian
-
0
Hi @demo.solaru
Is your project a new/not modified project or did you update your project from an old verison to v11 ?
-
0
@ismcagdas
We are using new v11 without any modification.
-
0
Hi,
Could you try v11.1 and see if it makes any difference ?
-
0
@ismcagdas
Yes, We tried on v11.1. Please see the below summary report:
Please find the below attachment as well
-
0
Hi,
Thanks, we will try this and inform you. By the way, do you use Redis in your app ?
Thanks,
-
0
Hi @demo.solaru,
I am curious if you tried either of my recommendations in my response. What do you see if you disable Auditing?
In your .Core project, in the PreInitialize() method of your <Project>CoreModule
Configuration.Auditing.IsEnabled = auditLogEnabled; Configuration.Auditing.Selectors.Clear();I'm mainly curious just to see what the observed behavior is, if that could help the ASPNZ team. I am also curious, if you are running these on your local environment, where all of your resources are.
For the build you are running, is this running the code using the Visual Studio debugger with IIS Express? Or have you built and deployed the code somewhere else?
In a production-like environment, you wouldn't have the overhead of running Visual Studio and JMeter locally on the same machine that is also running your webserver and your database server . Also if you are seeing SQL timeout errors, is your database local as well? Or is the database somewhere else?
My thought is that performance testing can be a bit of a black hole (I know from experience over the last ~2-3 months). You want to establish a reliable baseline, but running everything locally is potentially a skewed test, because everything is running locally on your machine, including programs that wouldn't be involved in a production environment (like Visual Studio and JMeter). The combination of all of those programs all running on 1 machine could yield resource contention which could impact performance.
Lastly - are you running Azure Application Insights? If you are, and you have an Azure subscription, if you could report your telemetry metrics to an Azure Application Insights hosted dashboard that could give you some more insights into exactly what these requests are doing. Azure Application Insights has been a huge help for me for troubleshooting performance-related issues.
Trying to offer ideas. I hope my input helps, -Brian
-
0
Hi @demo.solaru
I created an issue about it. You will investigate it deeply and inform you. You can follow the progress on https://github.com/aspnetzero/aspnet-zero-core/issues/4360
-
0
Hi @sedulen
Thank you for your detailed scenarios. We will try to check this according to your scenarios. You can check the progress on https://github.com/aspnetzero/aspnet-zero-core/issues/4360
