Hangfire Data Processing - InsertOrUpdate Abp.Domain.Uow.AbpDbConcurrencyException #10843

AuroraBMS created 2 years ago

Prerequisites

Please answer the following questions before submitting an issue. YOU MAY DELETE THE PREREQUISITES SECTION.

What is your product version?
- V11.0.1
What is your product type (Angular or MVC)?
- Angular
What is product framework type (.net framework or .net core)?
- Core

Good day ANZ team,

We are busy updating to V11.0.1 and attempting to implement Hangfire jobs for bulk data syncing. Previously we used to handle update & insert seperately which technically still works as individual processes but the actual parts that we want to start using is:InsertOrUpdateAndGetId We have tried a few routes of processing it but in the end even on a blank table that only needs to insert data it still fails.

This is the error we are getting when any inser/update function is processed without results existing already:

Abp.Domain.Uow.AbpDbConcurrencyException (DESKTOP-J0MPMEA:18476) The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.

Abp.Domain.Uow.AbpDbConcurrencyException: The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions. ---> Microsoft.EntityFrameworkCore.DbUpdateConcurrencyException: The database operation was expected to affect 1 row(s), but actually affected 0 row(s); data may have been modified or deleted since entities were loaded. See http://go.microsoft.com/fwlink/?LinkId=527962 for information on understanding and handling optimistic concurrency exceptions.

The data does not get modified or removed as the table is still empty and the list of information is already fixed and is not altered during the add process.

We have tried process the data as below(most basic data model):

var id =_businessPartnerMasterData.InsertOrUpdateAndGetId(new BusinessPartnerMasterData() { Id = Convert.ToString(reader.GetValue("CardCode")), CardCode = Convert.ToString(reader.GetValue("CardCode")), CardName = Convert.ToString(reader.GetValue("CardName")), IsActive = Convert.ToString(reader.GetValue("validFor")) == "Y" ? true : false });

the same structure wrapped in a unit of work etc.

Could you please assist with how to correctly InsertOrUpdateAndGetId data with a hangfire process?

Please let me know if more info is required, but in general the above is what I get in error form once the process is "completed/failed".

7 Answer(s)

0

ismcagdas created 2 years ago
Support Team

Hi @AuroraBMS

Is it possible to share the entire code for the background job ?

AuroraBMS created 2 years ago

Hi @ismcagdas,

Please see below:

using Abp.BackgroundJobs; using Abp.Dependency; using Abp.Domain.Repositories; using Abp.Domain.Uow; using Abp.EntityFrameworkCore; using Abp.Runtime.Session; using BRM.Configuration; using BRM.MultiTenancy; using BRM.MultiTenancy.Dto; using BRM.Web.Startup.Hangfire.Helpers; using Hangfire; using Microsoft.AspNetCore.Hosting; using Microsoft.Extensions.Configuration; using Sap.Data.Hana; using System.Collections.Generic; using System.Data; using System.Linq.Dynamic.Core; using System.Linq; using BRM.SystemSetup; using BRM.InventoryManagement; using System; using System.Transactions;

namespace BRM.Web.Startup.Hangfire.DataSyncs.Inbound { public class StockMasterDataInboundSync : BackgroundJob<int>, ITransientDependency { private readonly IRepository<Tenant, int> _tenantAppService; private readonly IRepository<QueryManager, long> _queryManager; private readonly IRepository<StockMasterData, string> _stockMasterData; private DataConnections _dataConnections; private IDbContextProvider<AbpDbContext> _Context; public IAbpSession _abpSession; private QueryHelpers _qH; private HanaConnection hanaConnection;

    private readonly IConfigurationRoot _appConfiguration;
    private readonly IWebHostEnvironment _hostingEnvironment;

    public StockMasterDataInboundSync(
        IWebHostEnvironment env,
        IDbContextProvider&lt;AbpDbContext&gt; Context,
        IAbpSession abpSession,
        IRepository&lt;Tenant, int&gt; tenantAppService,
        DataConnections dataConnections,
        IRepository&lt;QueryManager, long&gt; queryManager,
        IRepository&lt;StockMasterData, string&gt; stockMasterData)
    {
        _hostingEnvironment = env;
        _appConfiguration = env.GetAppConfiguration();
        _abpSession = abpSession;
        _Context = Context;
        _tenantAppService = tenantAppService;
        _dataConnections = dataConnections;
        _queryManager = queryManager;
        _stockMasterData = stockMasterData;
        hanaConnection = _dataConnections.GetHanaConnectionInstance
         (
             _appConfiguration.GetValue&lt;string&gt;("SAPHanaODBCConnectionDetail:Server"),
             _appConfiguration.GetValue&lt;string&gt;("SAPHanaODBCConnectionDetail:UserName"),
             _appConfiguration.GetValue&lt;string&gt;("SAPHanaODBCConnectionDetail:Password")
         );

    }



    [DisableConcurrentExecution(timeoutInSeconds: 10 * 60)]
    [AutomaticRetry(Attempts = 0)]
    [UnitOfWork]
    public override void Execute(int number)
    {
        
        //updated this from the standard (UnitOfWorkManager.Begin()) due to a post on here
            using (var unitOfWork = UnitOfWorkManager.Begin(TransactionScopeOption.RequiresNew))
            {
                var queryString = _queryManager.Single(x => x.QueryName == "GetDaily_OITM").QueryString;

                hanaConnection.Open();
                HanaDataReader reader = null;
                HanaCommand data = new HanaCommand(queryString, hanaConnection);
                reader = data.ExecuteReader();

                if (reader != null)
                {
                    while (reader.Read())
                    {
                     
                     
                            //_stockMasterData.InsertOrUpdateAndGetId(new StockMasterData()
                            _stockMasterData.InsertOrUpdate(new StockMasterData()
                            {
                                Id = Convert.ToString(reader.GetValue("ItemCode")),
                                ItemCode = Convert.ToString(reader.GetValue("ItemCode")),
                                ItemName = Convert.ToString(reader.GetValue("ItemName")),
                                IsActive = Convert.ToString(reader.GetValue("validFor")) == "Y" ? true : false
                            });
                       

                    }

                    reader.Close();
                }
                //this area has been modified several times with varying completion/save segments.
                UnitOfWorkManager.Current.SaveChanges();
                unitOfWork.Complete();
                //UnitOfWorkManager.Current.SaveChangesAsync().Wait();
                //unitOfWork.CompleteAsync().Wait();
            }
       
    }


}

}

0

ismcagdas created 2 years ago
Support Team

Hi,

I think using both [UnitOfWork] attribute and UnitOfWorkManager.Begin(TransactionScopeOption.RequiresNew) causes this problem. Could you remove UnitOfWorkManager.Begin(TransactionScopeOption.RequiresNew) and try again ?

Thanks,

AuroraBMS created 2 years ago

Hi Ismcagdas,

I've tried the above and it still results in the same issue.

Updated code:

public async Task DailySync()
        {
            try
            {
                //using (var unitOfWork = UnitOfWorkManager.Begin(TransactionScopeOption.RequiresNew))
                //{
                    var queryString = _queryManager.Single(x => x.QueryName == "GetDaily_OCRD").QueryString;

                    hanaConnection.Open();
                    HanaDataReader reader = null;
                    HanaCommand data = new HanaCommand(queryString, hanaConnection);
                    reader = data.ExecuteReader();
                    List<BusinessPartnerMasterData> existingDataList = new List<BusinessPartnerMasterData>();
                    existingDataList = _businessPartnerMasterData.GetAll().ToList();


                    if (reader != null)
                    {
                        while (reader.Read())
                        {

                            bool isNew = false;
                            BusinessPartnerMasterData existingData = existingDataList.Where(x => x.Id == Convert.ToString(reader.GetValue("CardCode"))).FirstOrDefault();
                            if (existingData == null)
                            {
                                existingData = new BusinessPartnerMasterData();
                                isNew = true;
                            }

                            existingData.Id = Convert.ToString(reader.GetValue("CardCode"));
                            existingData.BusinessPartnerTypeId = Convert.ToString(reader.GetValue("CardType"));
                            existingData.CardCode = Convert.ToString(reader.GetValue("CardCode"));
                            existingData.CardName = Convert.ToString(reader.GetValue("CardName"));
                            existingData.IsActive = Convert.ToString(reader.GetValue("validFor")) == "Y" ? true : false;

                        await _businessPartnerMasterData.InsertOrUpdateAndGetIdAsync(existingData);

                        //    await _businessPartnerMasterData.InsertAsync(existingData);
                        //if (isNew)
                        //    await _businessPartnerMasterData.InsertAsync(existingData);
                        //else
                        //    await _businessPartnerMasterData.UpdateAsync(existingData);
                    }


                }

                //    await UnitOfWorkManager.Current.SaveChangesAsync();
                //    await unitOfWork.CompleteAsync();
                //}
            }
            catch (System.Exception ex)
            {

                //TODO: Notification logic (TBC)
            }
        }

0

ismcagdas created 2 years ago
Support Team

Hi @AuroraBMS

Are you able to reproduce this problem on your development environment ? If so, is it possible to share your project with us ? You can send your project to [email protected] if that's OK with you.
0

AuroraBMS created 2 years ago

100% will send a sample project with the issue to [email protected]

gterdem created 2 years ago

Support Team

Can you try creating different uow scopes for updating each entity when updating multiple entities like:

public async Task DailySync()
    {
        try
        {
            var queryString = _queryManager.Single(x => x.QueryName == "GetDaily_OCRD").QueryString;

            hanaConnection.Open();
            HanaDataReader reader = null;
            HanaCommand data = new HanaCommand(queryString, hanaConnection);
            reader = data.ExecuteReader();
            List<BusinessPartnerMasterData> existingDataList = new List<BusinessPartnerMasterData>();
            existingDataList = _businessPartnerMasterData.GetAll().ToList();


            if (reader != null)
            {
                while (reader.Read())
                {
                    using (var unitOfWork = UnitOfWorkManager.Begin(TransactionScopeOption.RequiresNew)) // Moved scope to here
                    {
                        bool isNew = false;
                        BusinessPartnerMasterData existingData = existingDataList
                            .Where(x => x.Id == Convert.ToString(reader.GetValue("CardCode"))).FirstOrDefault();
                        if (existingData == null)
                        {
                            existingData = new BusinessPartnerMasterData();
                            isNew = true;
                        }

                        existingData.Id = Convert.ToString(reader.GetValue("CardCode"));
                        existingData.BusinessPartnerTypeId = Convert.ToString(reader.GetValue("CardType"));
                        existingData.CardCode = Convert.ToString(reader.GetValue("CardCode"));
                        existingData.CardName = Convert.ToString(reader.GetValue("CardName"));
                        existingData.IsActive =
                            Convert.ToString(reader.GetValue("validFor")) == "Y" ? true : false;

                        await _businessPartnerMasterData.InsertOrUpdateAndGetIdAsync(existingData);
                        await unitOfWork.CompleteAsync(); // Complete it
                    }
                }
            }
        }
        catch (System.Exception ex)
        {
            //TODO: Notification logic (TBC)
        }
    }

Have an answer to this question? Log in and write your answer.