Base solution for your next web application
Open Closed

Background process slow down everything #5530


User avatar
0
grinay created

Hi guys, I'm developing server monitoring. I implement my background worker following this instruction <a class="postlink" href="https://aspnetboilerplate.com/Pages/Documents/Background-Jobs-And-Workers#create-a-background-worker">https://aspnetboilerplate.com/Pages/Doc ... und-worker</a>

So the code looks like this

[UnitOfWork(IsDisabled = true)]
        protected override void DoWork()
        {
            List<Server> servers;

            DateTime checkTime = DateTime.UtcNow;

            using (var unitOfWork = _unitOfWorkManager.Begin(TransactionScopeOption.RequiresNew))
            {
                using (CurrentUnitOfWork.DisableFilter(AbpDataFilters.MayHaveTenant))
                {
                    //Gets all active servers
                    servers = _serverRepository.GetAll().Where(srv => srv.Active).AsNoTracking().ToList();
                }
            }

            


            ParallelOptions parallelOptions = new ParallelOptions()
            {
                MaxDegreeOfParallelism = 100 * Environment.ProcessorCount
            };

            
          //  servers.AsParallel().Asy
                
            Console.WriteLine("Check started");
            Parallel.ForEach(servers, parallelOptions, async (server) =>
            {
                ServerCheckResult checkResult = new ServerCheckResult();

               
                switch (server.Type)
                {
                    case ServerType.Ping:
                        throw new Exception("Not implemented yet");
                        break;
                    case ServerType.Service:
                        checkResult = await ServerChecker.ServiceAync(server);
                        break;
                    case ServerType.Webpage:
                        throw new Exception("Not implemented yet");
                }


//There will be the code that going to save current server state. 
            });
            
            Console.WriteLine("Check ended");
        }

ServerChecker.ServiceAync - this is async function that tries to open tcp connect to host , in case of fail or timeout it return error, latency, and result of operation .

So this code performing every 30 seconds, Without problem. But when this worker starts request to API become super slow. 10-30 seconds. Everything asynchronous here, and there is no database overload, there only one request to db to extract servers. So my question is how to make this background job don't affect on other requests . Because my service become unresponsive when background process starts. I've known there is special interface in .net core special for background services called IHostedService, but as i understand ASP.NET Zero(Boilerplate) don't use it. So please help how to separate background process from main request to make site workable.


3 Answer(s)
  • User Avatar
    0
    ryancyq created
    Support Team

    You can consider using Hangfire instead f the default background job.

    See <a class="postlink" href="https://aspnetboilerplate.com/Pages/Documents/Hangfire-Integration">https://aspnetboilerplate.com/Pages/Doc ... ntegration</a> <a class="postlink" href="https://www.hangfire.io/overview.html">https://www.hangfire.io/overview.html</a>

  • User Avatar
    0
    grinay created

    It didn't help. Anyway I solved the problem. The main problem was using Task.Wait() method. The code is checking availability of servers by TcpClient library. This library doesn't have any way to setup timeout for connect. Therefore I solved it with TcpClient.Connect(args).Wait(timeout) method. But problem that wait method is blocking thread. Such a way when I tried to scan 1000+ servers, everything became slow, because of 1000+ threads were in blocking state. I reworked my scan method and made it really asynchronous like this.

    add extension method for Task

    private static async Task WithTimeout(this Task task, TimeSpan timeout)
            {
                Task winner = await Task.WhenAny(task, Task.Delay(timeout));
                if (winner != task)
                {
                    throw new TimeoutException();
                }
    
                await task;
            }
    

    Then use tcp client with this method

    await tcpClient.ConnectAsync(serverIpAddr, server.Port ?? 1).WithTimeout(TimeSpan.FromSeconds(connectTimeout));
    

    Now code became really fast. it scans 1000 server, without any delay, and don't slow down the whole system . Hope this help someone else.

  • User Avatar
    0
    alper created
    Support Team

    thanx for the feedback