Base solution for your next web application
Open Closed

BackgroundJobManager inserting duplicate key into AbpBackgroundJobs #9482


User avatar
0
Siyeza created

Hi,

ABP Version: 5.6 AspNetZero Angular 8.6 .NET Core

I'm using the default BackgroundJobManager, provided by ABP, to enqueue and process background jobs.

I'm enqueuing a background job to send email notifications. However, every second or third attempt to enqueue the job throws the following error:

 error occurred while updating the entries. See the inner exception for details.    at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.Execute(IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
   at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IList`1 entries)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList`1 entriesToSave)
   at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(DbContext _, Boolean acceptAllChangesOnSuccess)
   at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
   at Abp.EntityFrameworkCore.AbpDbContext.SaveChanges()
   at Abp.Zero.EntityFrameworkCore.AbpZeroCommonDbContext`3.SaveChanges()
   at Abp.EntityFrameworkCore.Uow.EfCoreUnitOfWork.SaveChangesInDbContext(DbContext dbContext)
   at Abp.EntityFrameworkCore.Uow.EfCoreUnitOfWork.SaveChanges()
   at Abp.EntityFrameworkCore.Uow.EfCoreUnitOfWork.CompleteUow()
   at Abp.Domain.Uow.UnitOfWorkBase.Complete()
   at RMS.Workflow.WorkflowAppService.SubmitAsync(ItemActionRequestDto request) in C:\Development\IDS\RMS\Main\src\RMS.Workflow\Workflow\WorkflowAppService.cs:line 278
   at Abp.Authorization.AuthorizationInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Domain.Uow.UnitOfWorkInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Domain.Uow.UnitOfWorkInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Auditing.AuditingInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Auditing.AuditingInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Runtime.Validation.Interception.ValidationInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at RMS.Items.ItemAppService.SubmitAsync(ItemActionRequestDto request) in C:\Development\IDS\RMS\Main\src\RMS.Application\Items\ItemAppService.cs:line 188
   at Abp.Authorization.AuthorizationInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Domain.Uow.UnitOfWorkInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Domain.Uow.UnitOfWorkInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Auditing.AuditingInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at Abp.Runtime.Validation.Interception.ValidationInterceptor.InternalInterceptAsynchronous(IInvocation invocation)
   at lambda_method(Closure , Object )
   at Microsoft.Extensions.Internal.ObjectMethodExecutorAwaitable.Awaiter.GetResult()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.&lt;InvokeNextActionFilterAsync&gt;g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.&lt;InvokeInnerFilterAsync&gt;g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.&lt;InvokeNextExceptionFilterAsync&gt;g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
23505: duplicate key value violates unique constraint "PK_AbpBackgroundJobs"    at Npgsql.NpgsqlConnector.&lt;&gt;c__DisplayClass160_0.&lt;&lt;DoReadMessage&gt;g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlConnector.&lt;&gt;c__DisplayClass160_0.&lt;&lt;DoReadMessage&gt;g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming)
   at Npgsql.NpgsqlDataReader.NextResult()
   at Npgsql.NpgsqlCommand.ExecuteReaderAsync(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior)
   at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
   at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.Execute(IRelationalConnection ]connection) Severity: ERROR;InvariantSeverity: ERROR;SqlState: 23505;MessageText: duplicate key value violates unique constraint "PK_AbpBackgroundJobs";Detail: Key ("Id")=(6) already exists.;SchemaName: public;TableName: AbpBackgroundJobs;ConstraintName: PK_AbpBackgroundJobs;File: d:\pginstaller_12.auto\postgres.windows-x64\src\backend\access\nbtree\nbtinsert.c;Line: 570;Routine: _bt_check_unique"

As you can see, it looks the BackgroundJobManager is trying to re-insert a job entry with the same ID.

Below is how I'm enqueueing the job:

/// &lt;summary&gt;
        /// Sends the collection of templated notifications
        /// &lt;/summary&gt;
        /// &lt;param name=&quot;notifications&quot;&gt;&lt;/param&gt;
        /// &lt;returns&gt;&lt;/returns&gt;
        public async Task SendAsync(IEnumerable&lt;TemplatedNotification&gt; notifications)
        {
            foreach(var notification in notifications)
            {
                // Enqueue the job
                await _backgroundJobManager.EnqueueAsync&lt;SendTemplatedNotificationJob, TemplatedNotification&gt;(notification);
            }
        }

Below is the background job definition:

public class SendTemplatedNotificationJob : BackgroundJob&lt;TemplatedNotification&gt;, ITransientDependency
    {
        #region Constructors

        public SendTemplatedNotificationJob(
            IEmailSender emailSender,
            IRepository&lt;NotificationTemplate&gt; notificationTemplateRepository,
            UserManager userManager            
            )
        {
            Guard.ArgumentNotNull(emailSender, nameof(emailSender));
            Guard.ArgumentNotNull(notificationTemplateRepository, nameof(notificationTemplateRepository));
            Guard.ArgumentNotNull(userManager, nameof(userManager));

            _emailSender = emailSender;
            _notificationTemplateRepository = notificationTemplateRepository;
            _userManager = userManager;
            AbpSession = NullAbpSession.Instance;
        }

        #endregion

        #region Properties

        public IAbpSession AbpSession { get; set; }

        #endregion

        #region Fields


        private readonly IEmailSender _emailSender;
        private readonly IRepository&lt;NotificationTemplate&gt; _notificationTemplateRepository;
        private readonly UserManager _userManager;        

        #endregion

        #region Methods

        /// &lt;summary&gt;
        /// Sends the specified notification
        /// &lt;/summary&gt;
        /// &lt;param name=&quot;notification&quot;&gt;&lt;/param&gt;
        [UnitOfWork]
        public override void Execute(TemplatedNotification notification)
        {
            // TODO: Cater for other notification channels, besides email

            if(notification != null)
            {
                // Switch to correct tenant
                using (CurrentUnitOfWork.SetTenantId(notification.TenantId))
                {
                    // Set the session
                    using (AbpSession.Use(notification.TenantId, null))
                    {
                        // Get the template
                        var template = _notificationTemplateRepository.FirstOrDefault(t => t.Code == notification.TemplateCode);
                        if (template == null)
                        {
                            // Try to get the non-tenant template
                            using (CurrentUnitOfWork.DisableFilter(AbpDataFilters.MayHaveTenant))
                            {
                                template = _notificationTemplateRepository.FirstOrDefault(t => t.Code == notification.TemplateCode);
                            }
                        }

                        if (template != null)
                        {
                            // First build the list of recipients
                            HashSet&lt;string&gt; recipients = new HashSet&lt;string&gt;();

                            // Get recipients by user identifier
                            foreach (var userIndentifier in notification.RecipientIdentifiers)
                            {
                                var user = _userManager.GetUserOrNull(userIndentifier);
                                if (!string.IsNullOrWhiteSpace(user?.EmailAddress))
                                {
                                    recipients.Add(user.EmailAddress);
                                }
                            }

                            // TODO: Get recipients by role
                            
                            if (recipients.Any())
                            {
                                // Build template body
                                StringBuilder body = new StringBuilder(template.Body);
                                foreach (var key in notification.TemplateKeyValues.Keys)
                                {
                                    body.Replace($"&lt;%{key}%&gt;", notification.TemplateKeyValues[key] ?? string.Empty);
                                }

                                // Construct mail message
                                MailMessage mailMessage = new MailMessage();
                                recipients.ForEach(r => mailMessage.To.Add(r));
                                mailMessage.Subject = template.Subject;
                                mailMessage.Body = body.ToString();
                                mailMessage.IsBodyHtml = true;

                                // Send the email
                                _emailSender.Send(mailMessage);                                
                            }

                        }
                    }                    
                }
            }
        }

        #endregion
    }

Thanks


5 Answer(s)
  • User Avatar
    0
    ismcagdas created
    Support Team

    Hi @Siyeza

    Do you run single instance for your app ?

  • User Avatar
    0
    Siyeza created

    Hi,

    Yes, we're hosting the application in a single docker container. I also get this error when running the application locally on my development machine.

    We're also using the database per tenant architecure, but I don't think this is the issue because as far as I can tell the background jobs are always created in the host DB.

    Thanks

  • User Avatar
    0
    Siyeza created

    Hi,

    Apologies, I think I've found the issue. The method I'm calling to enqueue the job is an async method, and I was calling it in n non-async method. I thought Visual Studio would produce a compiler warning for this but it doesn't.

    Calling BackgroundJobManager.EnqueueAsync in a non-async method can produce unexpected results, because BackgroundJobManager is a singleton, which means the IBackgroundJobStore can end up participating in multiple, unrelated UnitOfWorks. This can result in multiple in UnitOfWorks trying to insert the same job, depending on thread timing.

    Apologies, again.

  • User Avatar
    0
    musa.demir created

    Thanks for the warning @Siyeza.

    Is that solves your problem?

     AsyncHelper.RunSync(() => /*your code*/);
    
  • User Avatar
    0
    Siyeza created

    Hi,

    Yes it does, thank you.