Workers and jobs management in isolated app #12567

3 Answer(s)

• 

  0

  oguzhanagir created one year ago

  Support Team

  Hi @Ricavir

  That's a great instinct for improving your architecture, and you are absolutely on the right track by wanting to separate the user facing app from the background workers.

  However, your proposed solution of putting them in the same App Service Plan will not solve your performance problem.

  An App Service Plan is the underlying hardware. Think of it as the physical or virtual server with a specific amount of CPU and RAM.

  If you put two App Services (your user app and your worker app) into the same plan, they are competing for the exact same CPU and RAM. When your huge worker job kicks off, it will consume the plan's resources, starving your user facing app and causing the exact slowdown you're trying to prevent. The auto scale rule will eventually add more instances, but the damage to the user experience will already be done.

  Two Plans + A Queue

  The architecture you're looking for is a classic "Web-Queue-Worker" pattern. This approach truly decouples your web app from your background jobs.

  Here’s what it looks like:

  1. App Service Plan 1 (e.g., "Web Plan"):

     • Hosts: Your main ASP.NET Zero user facing web application.
     • Scaling: Scales based on user facing metrics (like HTTP requests, CPU, or RAM).

  2. App Service Plan 2 (e.g., "Worker Plan"):

     • Hosts: A separate worker application (this is your "second app"). A great choice for this is an Azure WebJob or a .NET BackgroundService running as "Always On".
     • Scaling: Scales independently based on the size of the job queue.

  3. An Azure Queue (Storage Queue or Service Bus):

     • This is the "glue" that connects them. It's a message buffer.

  This flow prevents the web app from ever being impacted by the worker:

  • A user performs an action in your web app (App 1).
  • Instead of processing the "huge job" immediately, the web app creates a message describing the job and adds it to the Azure Queue. This step is extremely fast.
  • The user's web request finishes, and they get a Job started message. Their experience is snappy and responsive.
  • Separately, your worker app (App 2, in its own plan) is constantly monitoring the queue. It sees the new message, picks it up, and starts processing the huge job.
  • If 1,000 users do this, 1,000 messages pile up in the queue. The Worker Plan sees the long queue and scales out to burn through the jobs, all without ever affecting the Web Plan's performance.

  Since you're using ASP.NET Zero, this is even easier to implement.

  • By default, ASP.NET Zero's IBackgroundJobManager runs in process, which is the cause of your problem.
  • You need to replace this with an out of process provider. ASP.NET Zero has built in support for:
    • Hangfire: A very popular choice. You would configure your main app to enqueue jobs to Hangfire and run the Hangfire Server Dashboard as your second app.
    • Azure Service Bus: You can configure ASP.NET Zero to use Azure Service Bus directly.
  • Your second app would then be an Azure WebJob that is configured to pull messages from that queue bus and execute the jobs.

  So, your idea is correct in spirit! You just need to ensure the two apps are in two separate App Service Plans and are connected by a message queue.

  Markdown is supported

  Copy & paste or drag & drop images (max 30 MB per image)

  Save Cancel
• 

  0

  Ricavir created one year ago

  Thks @oguzhanagir, Having two app services is definitely the best choice. I'm already using Hangfire ; therefore, I don't see the point of using an Azure Service Bus here. Actually, the apps will share the same Hangfire connection string. One app will launch jobs and the other one will execute them ; this behavior doesn't need an additional service bus. Why Azure Service Bus will be needed in this case ?

  Markdown is supported

  Copy & paste or drag & drop images (max 30 MB per image)

  Save Cancel
• 

  0

  oguzhanagir created one year ago

  Support Team

  Hi @Ricavir

  My apologies for the lack of clarity in my previous message. I mentioned Hangfire and Azure Service Bus as two alternative ways to achieve the Queue part of the pattern, not as services to be used together.

  Since you are already using Hangfire, you do not need an Azure Service Bus.

  Hangfire itself provides the complete queuing mechanism. It uses its backing database (your shared SQL connection string) to persist, schedule, and manage the job queue. This fulfills the Queue role in the "Web-Queue-Worker" pattern.

  • App 1 (in your Web Plan): The user-facing app. It only enqueues jobs to the Hangfire database. This is a fast operation.
  • App 2 (in your Worker Plan): The background worker app (e.g., as an "Always On" WebJob). It runs the Hangfire Server, pulls jobs from that same database, and executes them.

  This setup gives you the resource isolation you're looking for. Your user facing app stays responsive, and the work is handled and scaled completely independently by the worker plan.

  Markdown is supported

  Copy & paste or drag & drop images (max 30 MB per image)

  Save Cancel