When I go to Application Insights for my API (dotnet core2.2 on framework 4.6.1 - Zero 6.8.1) I see outliers in my operations chart. When I drill down I see that they are GET / signalr requests:
Here's another image of top 3 operations, the third is my own implementation of signalr which is currently on test in the host with one or two connected clients. The ten failures are inline with a auto-scale out operation today.
My log is empty of errors. However, when I go to my tenant home page I notice that the chat functionality is gone.
.
There are no errors in my console and signalr connects ok:
Where do I start looking?
For some reason the standard implementation of Application Insights in the Zero project does not complete the installation. What was missing was this line in Web.Host Startup.cs:
public IServiceProvider ConfigureServices(IServiceCollection services)
{
//BI 20191115 0.14.1
// The following line enables Application Insights telemetry collection.
services.AddApplicationInsightsTelemetry();
Hope this helps someone else.
Does anyone have any guidance for implementing Application Insights into Zero?
I have read the notes against this issue in the forum for Application Insights. At one stage @walkerscott states:
we have a solution to this. We save our App Insights Instrumentation Key to AbpSettings and this then gets loaded at startup. We also do something similar for Google Analytics.
We'd be willing to contribute this to AspNetZero repo via PR if the team is interested?
I have followed instructions for implementing Application Insights here: Application Insights for ASP.NET Core applications but I am told by my Azure support guys that we're not getting metrics. In the same forum article mentioned above @mitch states:
A vote from me too. It's been a real pain to try to get it to integrate.
Can anyone shed any light on integration of Application Insights with the Zero framework. For me this is a bit of a show stopper because I want to implement an event hub in Azure to feed my logs through to Datadog or Sumo Logic. Any help gratefully appreciated.
I know this issue is closed but I also believe there is some trick to getting Application Insights to work in the Zero framework. @walkerscott is it possible for you to share your thoughts/code on this?
Hi @ismcagdas, Interestingly enough the value is set in my appsettings.json but not in my appsettings.production.json. Is your recommendation that I should set the value of AllowAnonymousSignalRConnection in my production file to true?
Hi @ismacagdas, I had already found "Writing is not allowed after writer was completed", with SignalR 1.1" which seems to suggest it's a benign message and possibly updated in a later version of SignalR, I have no idea which version of SignalR I am using because I only have Abp.AspNetCore.SignalR which is on version 4.4.0. If you're saying that SignalR should already be authorised how is it possible for it to continue working after expiration?
Hi @ismcagdas, I am currently working with datadog to try find the root cause, I am up to my ears in logs! I have UK Fast (the UK's largest Azure consultancy) also trying to track the problem. Too early to say whether it's my code or Azure, I'm sure it's not Zero. As for
Note the three FATAL messages which we have been told to ignore because "Zero continues to operate normally so just ignore these messages". Hardly professional but hey-ho there we go.
See Microsoft.AspNetCore.AzureAppServices.HostingStartup problem. Comments welcome.
@ismcagdas, I have some errors in my logs with regard to SignalR for the first time and I suspect it is from the above implementation.
INFO 2019-10-05 19:07:58,348 [74 ] ore.Mvc.Internal.ControllerActionInvoker - Executed action method Nuagecare.Tenants.Dashboard.TenantDashboardAppService.GetFormSubmissionActivityByStaffForChart (Nuagecare.Application), returned result Microsoft.AspNetCore.Mvc.ObjectResult in 1344.7239ms.
INFO 2019-10-05 19:08:17,600 [80 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 20286.381ms 101
DEBUG 2019-10-05 19:08:21,756 [109 ] Abp.AspNetCore.SignalR.Hubs.AbpCommonHub - A client is disconnected: 5o4znj9JHWHyVaLsqW5VpQ
DEBUG 2019-10-05 19:08:22,771 [39 ] Nuagecare.Web.Chat.SignalR.ChatHub - A client is disconnected: X6rtAAzycni2NZ6dSH3KSw
ERROR 2019-10-05 19:08:23,787 [109 ] .AspNetCore.SignalR.HubConnectionContext - Failed writing message. Aborting connection.
System.InvalidOperationException: Writing is not allowed after writer was completed.
at System.IO.Pipelines.Pipe.GetMemory(Int32 sizeHint)
at System.IO.Pipelines.Pipe.DefaultPipeWriter.GetSpan(Int32 sizeHint)
at System.IO.Pipelines.PipeWriter.WriteAsync(ReadOnlyMemory`1 source, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.HubConnectionContext.<TryWritePingSlowAsync>d__49.MoveNext()
ERROR 2019-10-05 19:08:23,787 [109 ] .AspNetCore.SignalR.HubConnectionContext - Failed writing message. Aborting connection.
System.InvalidOperationException: Writing is not allowed after writer was completed.
at System.IO.Pipelines.Pipe.GetMemory(Int32 sizeHint)
at System.IO.Pipelines.Pipe.DefaultPipeWriter.GetSpan(Int32 sizeHint)
at System.IO.Pipelines.PipeWriter.WriteAsync(ReadOnlyMemory`1 source, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.HubConnectionContext.<TryWritePingSlowAsync>d__49.MoveNext()
INFO 2019-10-05 19:08:26,756 [51 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 4579899.7844ms 101
INFO 2019-10-05 19:08:27,834 [51 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 4580877.9993ms 101
DEBUG 2019-10-05 19:08:29,865 [39 ] Abp.AspNetCore.SignalR.Hubs.AbpCommonHub - A client is disconnected: 5w7N5t6kRGAuhLw7we98nA
DEBUG 2019-10-05 19:08:29,865 [109 ] Nuagecare.Web.Chat.SignalR.ChatHub - A client is disconnected: Hhx2T-pD_ZqRnbFltQbOuQ
DEBUG 2019-10-05 19:08:30,881 [39 ] Abp.AspNetCore.SignalR.Hubs.AbpCommonHub - A client is disconnected: wqcCGN_gb_noqrsQQgLMeg
DEBUG 2019-10-05 19:08:30,881 [37 ] Nuagecare.Web.Chat.SignalR.ChatHub - A client is disconnected: AYzqNH3HvZbSQKykQv6v9Q
ERROR 2019-10-05 19:08:31,896 [37 ] .AspNetCore.SignalR.HubConnectionContext - Failed writing message. Aborting connection.
System.InvalidOperationException: Writing is not allowed after writer was completed.
at System.IO.Pipelines.Pipe.GetMemory(Int32 sizeHint)
at System.IO.Pipelines.Pipe.DefaultPipeWriter.GetSpan(Int32 sizeHint)
at System.IO.Pipelines.PipeWriter.WriteAsync(ReadOnlyMemory`1 source, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.HubConnectionContext.<TryWritePingSlowAsync>d__49.MoveNext()
INFO 2019-10-05 19:08:32,912 [38 ] Microsoft.AspNetCore.Server.Kestrel - Connection id "0HLQ8ED367GOK", Request id "0HLQ8ED367GOK:00000011": the connection was closed because the response was not read by the client at the specified minimum data rate.
INFO 2019-10-05 19:08:34,912 [120 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 3277540.3999ms 101
INFO 2019-10-05 19:08:34,943 [32 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 3277687.8381ms 101
INFO 2019-10-05 19:08:34,943 [112 ] Microsoft.AspNetCore.Server.Kestrel - Connection id "0HLQ8ED367GOI", Request id "0HLQ8ED367GOI:0000000F": the connection was closed because the response was not read by the client at the specified minimum data rate.
INFO 2019-10-05 19:08:35,897 [109 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 13730904.8477ms 101
INFO 2019-10-05 19:08:35,928 [32 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 13730829.5871ms 101
DEBUG 2019-10-05 19:08:35,959 [112 ] Nuagecare.Web.Chat.SignalR.ChatHub - A client is disconnected: Z-hFhgptsMNrdnm73RDuhQ
DEBUG 2019-10-05 19:08:35,959 [109 ] Abp.AspNetCore.SignalR.Hubs.AbpCommonHub - A client is disconnected: SZETRL1vsi6cqONnHh7yFg
DEBUG 2019-10-05 19:08:35,959 [37 ] Abp.AspNetCore.SignalR.Hubs.AbpCommonHub - A client is disconnected: WDBrDzCQZ69KwqbjkNKFFg
DEBUG 2019-10-05 19:08:35,959 [38 ] nalR.FormSubmissionHub.FormSubmissionHub - A client disconnected from FormSubmissionHub: ECHbXWFhvQDoA5n639DXpQ
ERROR 2019-10-05 19:08:36,975 [37 ] .AspNetCore.SignalR.HubConnectionContext - Failed writing message. Aborting connection.
System.InvalidOperationException: Writing is not allowed after writer was completed.
at System.IO.Pipelines.Pipe.GetMemory(Int32 sizeHint)
at System.IO.Pipelines.Pipe.DefaultPipeWriter.GetSpan(Int32 sizeHint)
at System.IO.Pipelines.PipeWriter.WriteAsync(ReadOnlyMemory`1 source, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.HubConnectionContext.<TryWritePingSlowAsync>d__49.MoveNext()
ERROR 2019-10-05 19:08:36,975 [37 ] .AspNetCore.SignalR.HubConnectionContext - Failed writing message. Aborting connection.
System.InvalidOperationException: Writing is not allowed after writer was completed.
at System.IO.Pipelines.Pipe.GetMemory(Int32 sizeHint)
at System.IO.Pipelines.Pipe.DefaultPipeWriter.GetSpan(Int32 sizeHint)
at System.IO.Pipelines.PipeWriter.WriteAsync(ReadOnlyMemory`1 source, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.HubConnectionContext.<TryWritePingSlowAsync>d__49.MoveNext()
DEBUG 2019-10-05 19:08:37,990 [112 ] Abp.AspNetCore.SignalR.Hubs.AbpCommonHub - A client is disconnected: rRDEBOcsg6HaKs54UWmRyg
DEBUG 2019-10-05 19:08:37,990 [38 ] Nuagecare.Web.Chat.SignalR.ChatHub - A client is disconnected: 1x-ARxK-4NfhpmaW0wecFw
ERROR 2019-10-05 19:08:39,006 [109 ] .AspNetCore.SignalR.HubConnectionContext - Failed writing message. Aborting connection.
System.InvalidOperationException: Writing is not allowed after writer was completed.
at System.IO.Pipelines.Pipe.GetMemory(Int32 sizeHint)
at System.IO.Pipelines.Pipe.DefaultPipeWriter.GetSpan(Int32 sizeHint)
at System.IO.Pipelines.PipeWriter.WriteAsync(ReadOnlyMemory`1 source, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.HubConnectionContext.<TryWritePingSlowAsync>d__49.MoveNext()
ERROR 2019-10-05 19:08:39,006 [109 ] .AspNetCore.SignalR.HubConnectionContext - Failed writing message. Aborting connection.
System.InvalidOperationException: Writing is not allowed after writer was completed.
at System.IO.Pipelines.Pipe.GetMemory(Int32 sizeHint)
at System.IO.Pipelines.Pipe.DefaultPipeWriter.GetSpan(Int32 sizeHint)
at System.IO.Pipelines.PipeWriter.WriteAsync(ReadOnlyMemory`1 source, CancellationToken cancellationToken)
at Microsoft.AspNetCore.SignalR.HubConnectionContext.<TryWritePingSlowAsync>d__49.MoveNext()
INFO 2019-10-05 19:08:40,975 [120 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 16790141.1665ms 101
INFO 2019-10-05 19:08:40,975 [51 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 429979.9867ms 101
INFO 2019-10-05 19:08:40,975 [39 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 16790482.9874ms 101
INFO 2019-10-05 19:08:40,975 [80 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 430085.5001ms 101
INFO 2019-10-05 19:08:41,037 [112 ] Microsoft.AspNetCore.Server.Kestrel - Connection id "0HLQ8ED367GOV", Request id "0HLQ8ED367GOV:00000014": the connection was closed because the response was not read by the client at the specified minimum data rate.
INFO 2019-10-05 19:08:41,428 [39 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://nuagecare.io/api/services/app/NcEntity/GetFormsForEntity?input=56907702-ae3e-4cbd-1770-08d5f63eb2e7
INFO 2019-10-05 19:08:41,428 [39 ] pNetCore.Cors.Infrastructure.CorsService - CORS policy execution successful.
INFO 2019-10-05 19:08:41,428 [51 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 17728315.9452ms 101
INFO 2019-10-05 19:08:41,443 [51 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://nuagecare.io/api/services/app/NcForm/SubmitForms
INFO 2019-10-05 19:08:41,443 [51 ] pNetCore.Cors.Infrastructure.CorsService - CORS policy execution successful.
INFO 2019-10-05 19:08:41,443 [51 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 5.4954ms 204
INFO 2019-10-05 19:08:41,443 [112 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 GET http://nuagecarewebapi.azurewebsites.net/
INFO 2019-10-05 19:08:41,443 [112 ] ft.AspNetCore.Routing.EndpointMiddleware - Executing endpoint 'Nuagecare.Web.Controllers.HomeController.Index (Nuagecare.Web.Host)'
INFO 2019-10-05 19:08:41,443 [112 ] ore.Mvc.Internal.ControllerActionInvoker - Route matched with {action = "Index", controller = "Home", area = ""}. Executing action Nuagecare.Web.Controllers.HomeController.Index (Nuagecare.Web.Host)
INFO 2019-10-05 19:08:41,459 [112 ] ore.Mvc.Internal.ControllerActionInvoker - Executing action method Nuagecare.Web.Controllers.HomeController.Index (Nuagecare.Web.Host) - Validation state: Valid
INFO 2019-10-05 19:08:41,428 [37 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://nuagecare.io/api/services/app/NcEntity/GetFormsForEntity?input=7e334e4c-db3f-4900-2853-08d6a6218745
INFO 2019-10-05 19:08:41,428 [120 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://nuagecare.io/api/services/app/NcEntity/GetFormsForEntity?input=66ae3b69-0eaf-489f-1769-08d5f63eb2e7
INFO 2019-10-05 19:08:41,428 [32 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 17728297.6264ms 101
INFO 2019-10-05 19:08:41,459 [100 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://nuagecare.io/api/services/app/NcEntity/GetFormsForEntity?input=dce6ebdb-fc90-4aa6-1771-08d5f63eb2e7
INFO 2019-10-05 19:08:41,459 [112 ] ore.Mvc.Internal.ControllerActionInvoker - Executed action method Nuagecare.Web.Controllers.HomeController.Index (Nuagecare.Web.Host), returned result Microsoft.AspNetCore.Mvc.RedirectToActionResult in 0.0518ms.
INFO 2019-10-05 19:08:41,459 [37 ] pNetCore.Cors.Infrastructure.CorsService - CORS policy execution successful.
INFO 2019-10-05 19:08:41,490 [37 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 57.5895ms 204
INFO 2019-10-05 19:08:41,475 [33 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://beverley.nuagecare.io/signalr/negotiate?enc_auth_token=wNYmO41%2F48SHNstaLVXxHCCre29BZQl1NhC6NM3R3rzpXtPQxVzH6jEzA%2FQhXFN5tu6Fk7pO53uppm1mVXMZgxbyRVz26dnepi%2FFyB6axBY%2B6gq1GL%2BuRQgoiFUCjRN2p8w6LevViwKlHyWZZJZO1DGVSjAi1m2U%2Bog9pkHw9%2FSx%2Bo570np2k82P6w%2FormNXR9PL41u7oSMKLOXt0GwuLYro99gg8VeT3cl1X4tKr7w%2FEKe17nck8u0uyzYKEm%2Fn%2BFbG0l1Vc%2B5X2BpYXoGVkFiwXXP34yArvijtwXX3VvR7JY7LekO%2B7Bp4qQpkbydhoCGUcIOBe5PPn7%2BR8WNI%2Bt2q24jimq5Bb5K9t3qhip9pjqs14ViIBbvPhxb77pmFk1nvczZXhcBXOP5RhxLxwxiBwOcxHoHkohS4i0skJrr7w9%2BYtEJXEbOc5rxLbt65Na71jBU6ZyXjYGd68S3OniWP4SKVOwkmRDSPeI%2F8Sy6cbohH%2B5Yeveb5to6u7z%2BQl35SLiK%2Frtn5JaJjqV6ioNOmUbemunGuFymOfwo73HJtIB3FEkCkHq%2FMfioPRYaJfsNdGgK3begAGQMf01613lkubrz3X%2FdQHWScIAjb3lG2xVArf4pMwPyFODhqL3JAdvm6hYNS7zbU67LJE3SoE%2FYub8kYn02Gx%2FJcHJ98HoP0lPG3%2BeUBOiZCH7GbPFV5%2Fc5qozkeDooPy21QRDTa7QetvfQrW0ecynjn%2Bu%2BYJfV82VI2ZWCznZPRRx9ddxZIJjtvLY0nKPYHOxghgzv7R05Q8ZpyMuYxHo9ceaNGrh7eBqtUiFMCaselb8Wx1xKmpHU1FuUkg%2BA%2BnKm5gb2KvOORTlzp%2F%2BG6rMjs7q1wqdmJyq%2BtUCoOtTC8Lud%2BrTPVRu7cNXeZHg%2BxfqfeyFHmI9%2B09182uqntVkMLlKfIj88imfskDQ8nTvCrRU6RTC5W1UUcHjocdQ9C7GYyFpzeVnQ8WSuIJwAUhNL2ZZjluEdXJRTMsJbmQis1HQL%2Fh6zrILORO24nkoFnryUTsvnLWTMhPZzDU8mAKR4v1AV94mx5ccEPBO8tqBbKM7pt12nsdYTOZzqEo4WW%2Brc6UMaXEbpmlN7QK%2FSIviQ8PnG%2BozdZ1kEmTOa2arLwGsqSntaG
INFO 2019-10-05 19:08:41,475 [120 ] pNetCore.Cors.Infrastructure.CorsService - CORS policy execution successful.
INFO 2019-10-05 19:08:41,490 [120 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 62.4998ms 204
INFO 2019-10-05 19:08:41,475 [100 ] pNetCore.Cors.Infrastructure.CorsService - CORS policy execution successful.
INFO 2019-10-05 19:08:41,490 [112 ] ft.AspNetCore.Mvc.RedirectToActionResult - Executing RedirectResult, redirecting to /Ui.
INFO 2019-10-05 19:08:41,428 [38 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://nuagecare.io/api/services/app/NcForm/SubmitForms
INFO 2019-10-05 19:08:41,490 [100 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 40.2434ms 204
INFO 2019-10-05 19:08:41,506 [112 ] ore.Mvc.Internal.ControllerActionInvoker - Executed action Nuagecare.Web.Controllers.HomeController.Index (Nuagecare.Web.Host) in 46.8225ms
INFO 2019-10-05 19:08:41,428 [39 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 3.3461ms 204
INFO 2019-10-05 19:08:41,475 [43 ] soft.AspNetCore.Hosting.Internal.WebHost - Request starting HTTP/1.1 OPTIONS http://nuagecare.io/api/services/app/NcForm/SubmitForms
How does SignalR work with authorisation? In other words, I have a dashboard open on the host which displays real time data using SignalR. Once the authorisation for the user expires SignalR continues, is this likely to cause the above errors? If so how do I authorise SignalR or get it to log the user out without raising these errors??
HI guys,
Last night my Azure CPU pegged high for no apparent reason at 03:32:
There is nothing to suggest any errors in any logs but it appears that Zero decided to restart for some reason, here is my log:
INFO 2019-10-04 02:31:51,228 [178 ] ore.Mvc.Internal.ControllerActionInvoker - Executed action Nuagecare.NcForm.NcFormAppService.SubmitForms (Nuagecare.Application) in 283.3617ms
INFO 2019-10-04 02:31:51,228 [178 ] ft.AspNetCore.Routing.EndpointMiddleware - Executed endpoint 'Nuagecare.NcForm.NcFormAppService.SubmitForms (Nuagecare.Application)'
INFO 2019-10-04 02:31:51,228 [178 ] soft.AspNetCore.Hosting.Internal.WebHost - Request finished in 300.4157ms 200 application/json; charset=utf-8
**DEBUG 2019-10-04 02:32:47,023 [1 ] Abp.Modules.AbpModuleManager - Loading Abp modules...**
DEBUG 2019-10-04 02:32:47,289 [1 ] Abp.Modules.AbpModuleManager - Found 24 ABP modules in total.
DEBUG 2019-10-04 02:32:47,320 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Nuagecare.Web.Startup.NuagecareWebHostModule, Nuagecare.Web.Host, Version=6.5.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,320 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Nuagecare.Web.NuagecareWebCoreModule, Nuagecare.Web.Core, Version=6.5.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,320 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Nuagecare.NuagecareApplicationModule, Nuagecare.Application, Version=6.5.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Nuagecare.NuagecareCoreModule, Nuagecare.Core, Version=6.5.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.Zero.AbpZeroCoreModule, Abp.ZeroCore, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.Zero.AbpZeroCommonModule, Abp.Zero.Common, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.AbpKernelModule, Abp, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.Zero.Ldap.AbpZeroLdapModule, Abp.Zero.Ldap, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.AutoMapper.AbpAutoMapperModule, Abp.AutoMapper, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.AspNetZeroCore.AbpAspNetZeroCoreModule, Abp.AspNetZeroCore, Version=1.2.2.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.MailKit.AbpMailKitModule, Abp.MailKit, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Nuagecare.EntityFrameworkCore.NuagecareEntityFrameworkCoreModule, Nuagecare.EntityFrameworkCore, Version=6.5.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.Zero.EntityFrameworkCore.AbpZeroCoreEntityFrameworkCoreModule, Abp.ZeroCore.EntityFrameworkCore, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,336 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.EntityFrameworkCore.AbpEntityFrameworkCoreModule, Abp.EntityFrameworkCore, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,351 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.EntityFramework.AbpEntityFrameworkCommonModule, Abp.EntityFramework.Common, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,351 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.IdentityServer4.AbpZeroCoreIdentityServerEntityFrameworkCoreModule, Abp.ZeroCore.IdentityServer4.EntityFrameworkCore, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,351 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.IdentityServer4.AbpZeroCoreIdentityServerModule, Abp.ZeroCore.IdentityServer4, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,351 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.AspNetZeroCore.Web.AbpAspNetZeroCoreWebModule, Abp.AspNetZeroCore.Web, Version=1.2.2.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,351 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.AspNetCore.AbpAspNetCoreModule, Abp.AspNetCore, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,351 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.Web.AbpWebCommonModule, Abp.Web.Common, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,382 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.AspNetCore.SignalR.AbpAspNetCoreSignalRModule, Abp.AspNetCore.SignalR, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,382 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Nuagecare.Startup.NuagecareGraphQLModule, Nuagecare.GraphQL, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,382 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.Runtime.Caching.Redis.AbpRedisCacheModule, Abp.RedisCache, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,382 [1 ] Abp.Modules.AbpModuleManager - Loaded module: Abp.Hangfire.AbpHangfireAspNetCoreModule, Abp.HangFire.AspNetCore, Version=4.4.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:47,398 [1 ] Abp.Modules.AbpModuleManager - 24 modules loaded.
DEBUG 2019-10-04 02:32:47,539 [1 ] o.Configuration.LanguageManagementConfig - Converted Abp (Abp.Localization.Dictionaries.DictionaryBasedLocalizationSource) to MultiTenantLocalizationSource
DEBUG 2019-10-04 02:32:47,539 [1 ] o.Configuration.LanguageManagementConfig - Converted AbpZero (Abp.Localization.Dictionaries.DictionaryBasedLocalizationSource) to MultiTenantLocalizationSource
DEBUG 2019-10-04 02:32:47,539 [1 ] o.Configuration.LanguageManagementConfig - Converted Nuagecare (Abp.Localization.Dictionaries.DictionaryBasedLocalizationSource) to MultiTenantLocalizationSource
DEBUG 2019-10-04 02:32:47,554 [1 ] o.Configuration.LanguageManagementConfig - Converted AbpWeb (Abp.Localization.Dictionaries.DictionaryBasedLocalizationSource) to MultiTenantLocalizationSource
DEBUG 2019-10-04 02:32:48,539 [1 ] ameworkCore.AbpEntityFrameworkCoreModule - Registering DbContext: Nuagecare.EntityFrameworkCore.NuagecareDbContext, Nuagecare.EntityFrameworkCore, Version=6.5.0.0, Culture=neutral, PublicKeyToken=null
DEBUG 2019-10-04 02:32:50,179 [1 ] Abp.Localization.LocalizationManager - Initializing 4 localization sources.
DEBUG 2019-10-04 02:32:50,539 [1 ] Abp.Localization.LocalizationManager - Initialized localization source: Abp
DEBUG 2019-10-04 02:32:50,742 [1 ] Abp.Localization.LocalizationManager - Initialized localization source: AbpZero
DEBUG 2019-10-04 02:32:50,820 [1 ] Abp.Localization.LocalizationManager - Initialized localization source: Nuagecare
DEBUG 2019-10-04 02:32:50,836 [1 ] Abp.Localization.LocalizationManager - Initialized localization source: AbpWeb
DEBUG 2019-10-04 02:32:50,851 [1 ] bp.Hangfire.HangfireBackgroundJobManager - Start background worker: Abp.Hangfire.HangfireBackgroundJobManager
DEBUG 2019-10-04 02:32:50,945 [1 ] Abp.AutoMapper.AbpAutoMapperModule - Found 6 classes define auto mapping attributes
DEBUG 2019-10-04 02:32:50,945 [1 ] Abp.AutoMapper.AbpAutoMapperModule - Nuagecare.Web.Models.TokenAuth.ExternalLoginProviderInfoModel
DEBUG 2019-10-04 02:32:50,945 [1 ] Abp.AutoMapper.AbpAutoMapperModule - Nuagecare.Friendships.Cache.FriendCacheItem
DEBUG 2019-10-04 02:32:50,961 [1 ] Abp.AutoMapper.AbpAutoMapperModule - Nuagecare.Dto.OrganizationUnitDto
DEBUG 2019-10-04 02:32:50,961 [1 ] Abp.AutoMapper.AbpAutoMapperModule - Nuagecare.Dto.RoleDto
DEBUG 2019-10-04 02:32:50,961 [1 ] Abp.AutoMapper.AbpAutoMapperModule - Nuagecare.Dto.UserDto+RoleDto
DEBUG 2019-10-04 02:32:50,961 [1 ] Abp.AutoMapper.AbpAutoMapperModule - Nuagecare.Dto.UserDto+OrganizationUnitDto
DEBUG 2019-10-04 02:33:00,507 [1 ] enancy.SubscriptionExpirationCheckWorker - Start background worker: Nuagecare.MultiTenancy.SubscriptionExpirationCheckWorker
DEBUG 2019-10-04 02:33:00,507 [1 ] cy.SubscriptionExpireEmailNotifierWorker - Start background worker: Nuagecare.MultiTenancy.SubscriptionExpireEmailNotifierWorker
INFO 2019-10-04 02:33:01,179 [1 ] fire.SqlServer.SqlServerObjectsInstaller - Start installing Hangfire SQL objects...
INFO 2019-10-04 02:33:01,289 [1 ] fire.SqlServer.SqlServerObjectsInstaller - Hangfire SQL objects installed.
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Starting Hangfire Server
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Using job storage: 'SQL Server: tcp:nuagecare-sqlserver.database.windows.net,1433@Nuagecare'
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Using the following options for SQL Server job storage:
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Queue poll interval: 00:00:15.
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Using the following options for Hangfire Server:
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Worker count: 10
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Listening queues: 'default'
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Shutdown timeout: 00:00:15
INFO 2019-10-04 02:33:01,695 [1 ] Hangfire.BackgroundJobServer - Schedule polling interval: 00:00:15
FATAL 2019-10-04 02:33:03,679 [1 ] soft.AspNetCore.Hosting.Internal.WebHost - Hosting startup assembly exception
System.InvalidOperationException: Startup assembly StartupBootstrapper failed to execute. See the inner exception for more details. ---> System.IO.FileNotFoundException: Could not load file or assembly 'StartupBootstrapper' or one of its dependencies. The system cannot find the file specified.
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
--- End of inner exception stack trace ---
FATAL 2019-10-04 02:33:03,695 [1 ] soft.AspNetCore.Hosting.Internal.WebHost - Hosting startup assembly exception
System.InvalidOperationException: Startup assembly DiagnosticServices.HostingStartup failed to execute. See the inner exception for more details. ---> System.IO.FileNotFoundException: Could not load file or assembly 'DiagnosticServices.HostingStartup' or one of its dependencies. The system cannot find the file specified.
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
--- End of inner exception stack trace ---
FATAL 2019-10-04 02:33:03,695 [1 ] soft.AspNetCore.Hosting.Internal.WebHost - Hosting startup assembly exception
System.InvalidOperationException: Startup assembly Microsoft.AspNetCore.AzureAppServices.HostingStartup failed to execute. See the inner exception for more details. ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.AspNetCore.AzureAppServices.HostingStartup' or one of its dependencies. The system cannot find the file specified.
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
at Microsoft.AspNetCore.Hosting.WebHostBuilder.BuildCommonServices(AggregateException& hostingStartupErrors)
--- End of inner exception stack trace ---
Note the three FATAL messages which we have been told to ignore because "Zero continues to operate normally so just ignore these messages". Hardly professional but hey-ho there we go. My question is why would this happen? I would attach my log file but there's no way to do it in this forum.