Aaron, Forever in your debt. I will take a look at this later, at least now I can see it can be done. Now it's a matter of finding out how to do it! Thanks for your help again, it really is much appreciated. Cheers, Bob
That's a bit like telling me to replace the wimwam on the universal transmogrifier. Can you point me to the github code where the save command is?
Hi Aaron, Someone mentioned table bloat after running Entity History for a while on Azure. It would be nice to cut into the code somewhere and serialise the EntityChangeSet along with the related EntityChanges and EntitypropertyChanges and then simply throw them into a document database. After all, the data are immutable and, as such, are perfect for this functionality. Of course it would also mean cutting into code to deserialise but I don't think that would be problem. I will come back to this later, where should I start? Cheers, Bob
Hi Aaron, I think you overestimate my programming skills - I wouldn't have a clue how to do that! I go to an initial trial later this week for a fortnight or so so I would like to have some idea of what to do because the audit trail is a major USP, what with GDPR and all that. Is your source code embedded in a DLL or can I follow it through by placing a break point in Zero somewhere?
Hi Aaron, Any thoughts or pointers on this?
I am sure I have seen a post here or on the github site about implementing a Mongodb driver for entity history functionality. I have searched but can not find anything. My Azure database is growing too fast with the insertions into the three entity history tables. I would like to cut into the code and push all entity history entries into Mongodb. Does anyone have any pointers on how to do this?
Thanks for staying with me Aaron, here's the full method which is a work in progress.
[UnitOfWork]
private DataProviderUpdateTotals UpdateEntities(List<DataProviderEntityDto> entities, int tenantId)
{
var totals = new DataProviderUpdateTotals(0, 0, 0);
if (entities != null)
{
using (_abpSession.Use(tenantId, null))
{
foreach (var entity in entities)
{
var dataProviderEntityAttributes = JsonConvert.DeserializeObject<Dictionary<string, string>>(entity.ExtensionData);
var dataProviderId = dataProviderEntityAttributes
.Where(m => m.Key == "DataProviderId")
.Select(m => m.Value)
.FirstOrDefault();
//error exceptions due to duplicatedResidentId's in ResiData
var ncEntityDataProviderIdXRef = new NcEntityDataProviderIdXRef();
var ncEntity = new NcEntity.NcEntity();
try
{
ncEntityDataProviderIdXRef = _ncEntityDataProviderIdXRefRepository.FirstOrDefault(m => m.DataProviderId == Convert.ToInt32(dataProviderId));
if (ncEntityDataProviderIdXRef == null)
{
//add new entity...
var organizationUnitId = _organizationUnitAttributeRepository.FirstOrDefault(m => m.Value == entity.OrganisationalUnitXRef).OrganizationUnitId;
CreateAndSaveNcEntity(entity, tenantId, organizationUnitId).GetAwaiter().GetResult();
totals.EntitiesAdded++;
break;
}
}
catch (Exception ex)
{
Logger.Error("Error in ncEntityDataProviderIdXRef during DataProviderAPIProvider.UpdateEntities, values: TenantId; " + tenantId.ToString() + " DataProviderId;" + dataProviderId.ToString(), ex);
break;
}
try
{
ncEntity = _ncEntityRepository.Get(ncEntityDataProviderIdXRef.NcEntityId);
}
catch (Exception ex)
{
Logger.Error("Duplicate in ncEntity during DataProviderAPIProvider.UpdateEntities, values: TenantId; " + tenantId.ToString() + " DataProviderId;" + dataProviderId.ToString(), ex);
break;
}
var systemOrganizationUnit = _organizationUnitAttributeRepository.FirstOrDefault(m => m.OrganizationUnitId == ncEntity.OrganizationUnitId).Value;
if (ncEntity.DisplayName != entity.DisplayName || ncEntity.ExtensionData != entity.ExtensionData || entity.OrganisationalUnitXRef != systemOrganizationUnit)
{
ncEntity.DisplayName = entity.DisplayName;
ncEntity.ExtensionData = entity.ExtensionData;
totals.EntitiesUpdated++;
if (entity.OrganisationalUnitXRef != systemOrganizationUnit)
{
var newEntityOrganizationUnit = _organizationUnitAttributeRepository.FirstOrDefault(m => m.Value == entity.OrganisationalUnitXRef).OrganizationUnitId;
if (newEntityOrganizationUnit != 0)
{
ncEntity.OrganizationUnitId = newEntityOrganizationUnit;
}
else //TODO
{
//if home doesn't exist create it
//if location doesn't exist create it
//add the entity to the new location
//ncEntity.OrganizationUnitId = newEntityOrganizationUnit;
}
totals.EntitiesMoved++;
}
_ncEntityRepository.Update(ncEntity);
}
}
}
}
return totals;
}
Hi Aaron, Thanks for the reply. entity is returned from my data provider as a list and then and serialized as the following Dto:
public class DataProviderEntityDto : EntityDto<Guid>
{
public string DisplayName { get; set; }
public string ExtensionData { get; set; }
}
NcEntity is a domain entity defined as below:
[Table("NcEntity")]
public class NcEntity : FullAuditedEntity<Guid>, IMustHaveTenant, IMayHaveOrganizationUnit, IExtendableObject
{
public int TenantId { get; set; }
public long? OrganizationUnitId { get; set; }
[Required]
[StringLength(DbConsts.MaxNameLength)]
public virtual string DisplayName { get; set; }
public virtual string ExtensionData { get; set; }
}
My DataProviderAPIService reads the third party database and returns a list of entities. If either value (DisplayName or ExtensionData) in the entity is different to the relevant value in the NcEntity domain object the system updates NcEntity. NcEntity is a fully audited entity and I'm not sure why the history is not captured. I am not sure what you mean by >If ncEntity was untracked, then isModified = !(OriginalValue.Equals(CurrentValue)) would be false. . Thanks for your help so far.
dotnet-core, angular, 5.4.1 I implement Entity History in ProjectnameEntityFrameworkCoreModule (I am tracking history for all fully audited entities):
Configuration.EntityHistory.IsEnabled = true; //Enable this to write change logs for the entities below:
Configuration.EntityHistory.Selectors.Add("NuagecareEntities", typeof(OrganizationUnit), typeof(Role), typeof(User), typeof(Tenant));
Configuration.EntityHistory.Selectors.Add(
new NamedTypeSelector(
"Abp.FullAuditedEntities",
type => typeof(IFullAudited).IsAssignableFrom(type)
));
In my system a tenant is able to implement a data provider and the system will use Hangfire to invoke a background job to get data changes made in the data provider database. My entity (NcEntity) is made up of two properties, DisplayName and ExtensionData. The latter is a collection of properties in Json format implemented with the IExtendableObject interface. My DataProviderAPIService class receives data from the data provider, deserialises it and carries out an update, insert or soft-delete, the code below shows the update method, code simplified:
if (ncEntity.DisplayName != entity.DisplayName || ncEntity.ExtensionData != entity.ExtensionData)
{
ncEntity.DisplayName = entity.DisplayName;
ncEntity.ExtensionData = entity.ExtensionData;
}
_ncEntityRepository.Update(ncEntity);
This works aok in terms of updating data but it does not invoke entity history, I have no entry in the entity history tables. If I use the user interface and edit an entity by, for example, changing the DisplayName property the change is captured in entity history. I have read the documentation at [https://aspnetboilerplate.com/Pages/Documents/Entity-History]) but can find no pointers as to where i am going wrong. The question is:
How do I implement entity history in a ServiceAPI?
@alper - brilliant, thanks a lot. Little snippets like this should really be placed in a blog - "How do I...". Your information was detailed and allowed me to do exactly what I wanted to do but I really do think it's time to implement a blog where all contributors can add articles on implementing little pieces of functionality like this.