Hello,
We are experiencing serious problems with connection leak/ not closed transactions. (System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.) The problem is occuring on production environment under higher load. There are lots of connections open and not closed after the requests and application becomes frozen. After out investigation we discovered that there is probably a problem with AsyncHelper.RunSync which is used in many places in the system and aspnetboilerplate libaries. I would like to share our solution and node application that runs reuests in parallel. With that you can reproduce the problem on your machines.
Product version: ASP.NET CORE & Angular, .NET 5.0, v10.2.0
In attached solution you can find:
- TestController
- newman-parallel-run - node application that runs postman collections in parallel
- sql-scripts - ms sql scripts that presents connection and transactions status
How to run this tests?
- First run aspnetzero API as ussual (local on https://localhost:44301/)
- Add TestController.cs with actions WorkingGood1, WorkingGood2, WorkingBad
- Open newman-parallel-run in Visual Studio Code
- Run
npm init -yand install the 3 dependencies:npm i async newman path - There are 4 collections that we can use for testing purpose in postman directory
- Look at index.js file and set PARALLEL_RUN_COUNT for testing it is set to 100
- Uncomment collection that you want to run - WorkingGood1.postman_collection.json is uncommented by default
- Run npm start and wait for results
We can see that collections for WorkingGood1 and WorkingGood2 works properly.
When we run WorkingBad endpoint collection the transactions and connections are not closing and application is frozen. We can check that by running scripts in Management Studio from sql-scripts directory.
Please check attached solution and attached presentation video. Video Solution
3 Answer(s)
-
0
Hi @HeadChannel,
Thank you for your detailed explanation. Could you also share your ABP NuGet package version ? Could you try disabling transaction of unitOfWork as a workaround and see if it makes any difference ?
[HttpPost] [UnitOfWork(isTransactional: false)] public void WorkingBad() { // working bad as AsyncHelper is used AsyncHelper.RunSync(() => _userRepository.FirstOrDefaultAsync(1)); } -
0
@HeadChannel & @ismcagdas,
While this ticket is ~8 months old, was there ever any follow-up on this finding? Was a root cause or solution ever determined?
I'm seeing a similar connection pool issue in production under heavy load, and I'm wondering if this is the same issue that was previously encountered.
Thanks, -Brian
-
0
Hi @sedulen
We have made some enhancements in the AspNet Boilerplate Framework and decreased the usage of AsyncHelper class. You can upgrade your solution to latest version of ABP and see if it helps you.
