Test cases which are there in the downloaded project are unit tests or integration tests? And why?
I’m creating some records, test case is passed successfully, but I cannot see in the database? So if I want to run these test cases on real DB, how can I do this?
If running test cases are allowed on real db then how should I handle the cleanup code? Means if let’s say I’m running test case to insert a row for entity, but once I’m done with this test case, it should remove this records from DB. Same should happen for update and delete also.
I’m using asp.net core + angular project.
27 Answer(s)
-
0
A Unit test by definition only tests a single layer, and therefore will not interact with a DB (unless it is a unit test written in a SQL framework.)
Most of the tests that come with ASP Zero are integration tests, and they use SQL Lite to set up a mock database. The Asp Zero tests, in particular, seed data during each test. You can see examples of this in the TestDatas folder, as it is used for the OU tests and the payment tests.
It is widely considered unacceptable to write tests against a live database, for two primary reasons: testing against live data can introduce artifacts in your database if the tests crash / fail / are improperly written, and data in a live database is unpredictable, so test assumptions cannot be guaranteed to be true, which defeats the purpose of writing the tests in the first place.
-
0
Oh look someone else who wants to test with live data because it makes common sense. Just ignore strix and his semantic, most people would want to test against a DB if they are db first to verify that models were generated properly and stored procedures return back proper results... There are NUMEROUS reason to do it be he keeps saying thats not what a unit test is because he doesnt think its best practice.
-
0
Stix here is a real world example.
We have multiple clients and each database was created at various times. Over the time frame "standard" fields have changed types, clients have come, gone, come back. Because each clients DB can be "custom" but setup with a "standard" we run in to instances where a field in our new app is defined one way because the db documentation says its an 'int' now. Before we launch a client into said new platform we would like to run some standard "unit test" to verify yes if I call XXX end point/proc it will work before said client uses it and the page errors off because the fields new standard is "int" but their DB has it set to a "decimal" the mocking of fake data would say yup pass it works, but for this client it doesnt and they would have errors and it would look bad.
If you only define your DB with EF then sure I guess you can ensure everything is proper but when you dont these types of unit tests are a huge time saver for the QC/Testing team to be able to run a test project and see pretty green check boxes and they can be assured that said client can use said new feature w/out issues.
Unit testing is multi dimensional and live db testing is a valid use case in the REAL WORLD regardless if you think its a best practice or not. When you pay me then I guess I wont care ;) till then I cant rely on not having mock testing, doesnt work and is false security. Or maybe I can give them your number for support 8-) and you can tell them well the unit test was good I dont know why its not working in production now for this client.
-
0
@BBakerMMC and @strix20
So what do you suggest?
Could you please explain with some code example or sample.
-
0
I opened an issue about this months ago and put an issue on github, no one has replied with an example yet.
<a class="postlink" href="https://github.com/aspnetzero/aspnet-zero-core/issues/764">https://github.com/aspnetzero/aspnet-ze ... issues/764</a>
-
0
<cite>BBakerMMC: </cite> Stix here is a real world example.
We have multiple clients and each database was created at various times. Over the time frame "standard" fields have changed types, clients have come, gone, come back. Because each clients DB can be "custom" but setup with a "standard" we run in to instances where a field in our new app is defined one way because the db documentation says its an 'int' now. Before we launch a client into said new platform we would like to run some standard "unit test" to verify yes if I call XXX end point/proc it will work before said client uses it and the page errors off because the fields new standard is "int" but their DB has it set to a "decimal" the mocking of fake data would say yup pass it works, but for this client it doesnt and they would have errors and it would look bad.
If you only define your DB with EF then sure I guess you can ensure everything is proper but when you dont these types of unit tests are a huge time saver for the QC/Testing team to be able to run a test project and see pretty green check boxes and they can be assured that said client can use said new feature w/out issues.
Unit testing is multi dimensional and live db testing is a valid use case in the REAL WORLD regardless if you think its a best practice or not. When you pay me then I guess I wont care ;) till then I cant rely on not having mock testing, doesnt work and is false security. Or maybe I can give them your number for support 8-) and you can tell them well the unit test was good I dont know why its not working in production now for this client.
What you are describing is easily handled by User Acceptance testing, which can be easily automated with a tool such as Selenium, and can be integrated into your build / test / deployment pipeline just as easily as unit tests and integration tests can. Moreover, UAT is a critical part of QA that should not be neglected, and it's certainly not a good idea to rely on automated tests to ensure your application is production ready.
-
0
MY question is still unanswered.
-
0
I'm with @BBakerMMC. Everyone seems to be fixated with code-first and the database seems to be downgraded to something which just works. The old adage "rubbish in, rubbish out" should be applied. Testing without a database only tests "rubbish in". Testing against a database allows a DBA to review and change column ordinality (key for user reporting and queries), check column nullability, data- types and then we start to look at replication, database mirroring, recovery procedures etc etc etc. The job doesn't just finish when the input procedures work. For many companies now the key asset is data (include all telecoms companies and ISP's, content providers etc etc). By the time we start to bang badly formatted data into Hadoop and then apply map reduce functionality the consequences of requesting change can be dire. To give a simple example, restricting testing to use in-memory processes such as SQLite means we have to use the dbo schema in SQL Server/Oracle. Most professional DBA's would tear their hair out if they were told that tables have to be created in the dbo schema and prefixed to show that they belong to a particular area of functionality. That restriction alone removes a whole cart load of tools from the DBA's toolbox. I would respectfully suggest an option to use Sqlite or SQL Server/Oracle/MySQL (etc) for testing as a configuration option. Having said that the example unit tests provided by the Zero guys provide an excellent starting point for convention and procedure, well done guys!
-
0
SO again I’ll ask the sameness question.
Test cases which are there in the downloaded project are unit tests or integration tests? And why?
-
0
<cite>ManojReddy: </cite> SO again I’ll ask the sameness question.
Test cases which are there in the downloaded project are unit tests or integration tests? And why?
I already answered the question. They are primarily integration tests. This is evidenced by the fact that the use a SqlLite database and there is no mocking framework in place, such as Moq.
I'm not sure how to answer "Why?"
The purpose of Integration Testing is to expose defects in the interfaces and in the interactions between integrated components, in this case the application and domain layers of Asp Zero. It's useful for testing things like ensuring that the UoW is properly used during tenant creation, by ensuring that the process is rolled back during an error. (I'm not describing and actual test, only how integration testing is useful.)
From the tests that I've looked into thus far, I don't really see a compelling reason why they have been written as integration tests, since the bulk of them could easily be satisfied with unit tests. Someone from the Asp Zero team would have to give you guidance as to their reasoning.
-
0
<cite>BobIngham: </cite> I'm with @BBakerMMC. Everyone seems to be fixated with code-first and the database seems to be downgraded to something which just works. The old adage "rubbish in, rubbish out" should be applied. Testing without a database only tests "rubbish in". Testing against a database allows a DBA to review and change column ordinality (key for user reporting and queries), check column nullability, data- types and then we start to look at replication, database mirroring, recovery procedures etc etc etc. The job doesn't just finish when the input procedures work. For many companies now the key asset is data (include all telecoms companies and ISP's, content providers etc etc). By the time we start to bang badly formatted data into Hadoop and then apply map reduce functionality the consequences of requesting change can be dire. To give a simple example, restricting testing to use in-memory processes such as SQLite means we have to use the dbo schema in SQL Server/Oracle. Most professional DBA's would tear their hair out if they were told that tables have to be created in the dbo schema and prefixed to show that they belong to a particular area of functionality. That restriction alone removes a whole cart load of tools from the DBA's toolbox. I would respectfully suggest an option to use Sqlite or SQL Server/Oracle/MySQL (etc) for testing as a configuration option. Having said that the example unit tests provided by the Zero guys provide an excellent starting point for convention and procedure, well done guys!
What you are describing should absolutely be tested.
By the DBAs who are resposible for the design and integrity of the database. That is why there are unit test frameworks for SQL.
As for using Sql lite vs Sql server, the setup for integration tests using Sql Server isn't any different. I don't think at any point has anyone suggested there's something inherently wrong with writing integration tests against an actual server.
I have, repeatedly, pointed out the reasons why you should not write integration tests against live data.
As I posted in the github issue, this blog is very helpful and informative for how to write integration tests in EF core.
[https://www.davepaquette.com/archive/2016/11/27/integration-testing-with-entity-framework-core-and-sql-server.aspx])
-
0
@strix20 Great, just give me the option to test using a database with persisted data! That way @ManojReddy can see the results of the test and create/read/update/delete to his heart's content! Jeepers, you wouldn't honestly expect a test to connect to a production database? I recommend that developers should never even know the meaning of the word production unless the word can be used in the same sentence as "whose turn is it to get the beers in"?
-
0
<cite>BobIngham: </cite> @strix20 Great, just give me the option to test using a database with persisted data! That way @ManojReddy can see the results of the test and create/read/update/delete to his heart's content! Jeepers, you wouldn't honestly expect a test to connect to a production database? I recommend that developers should never even know the meaning of the word production unless the word can be used in same sentence as "whose turn is it to get the beers in"?
Haha.
My point is you can see the exact same effects with a proper integration test that builds a db on test initialize, runs all of your code on an actual database, and then tears it down at the end.
You're also guaranteed to get expected results, which you will never get from a persisted database.
Most of my "argument" comes from the historical context of this discussion, which primarily took place in the github issue, where it was argued that it was necessary and proper to test against persisted data ( an argument in which, every example for, has referenced customer databases, which appear to be production data.)
In any case, the difference between my approach and using a persisted database is literally which database / initial catalog you target in the connection string. In both instances, you're interacting with an actual db context that connects to a real database.
For the record, I am a customer, just like you. I respond to questions on the forums as I have time, to offer guidance and solutions as I can, because this is a community and often times the developers can't get to our questions immediately.
If you create a new test project, make the appropriate EF core DI calls to setup your db context, there is nothing stopping you from hitting live data out of the box. The issue is with test startup and teardown and ensuring that you're not mucking up a good database with your tests.
-
0
Whose turn is it to get the beers in?
-
0
<cite>BobIngham: </cite> Whose turn is it to get the beers in?
I'll take an IPA!
-
0
India Pale Ale, good choice! Brewed to keep the troops of the East India Company sober instead of drinking the local loopy juice, to keep the holds of the clippers full when returning to the Jewel of the Empire and strong enough not to go off on the way! Good man, enjoy your weekend.
-
0
Strix can you do this for me?
If you create a new test project, make the appropriate EF core DI calls to setup your db context, there is nothing stopping you from hitting live data out of the box. The issue is with test startup and teardown and ensuring that you're not mucking up a good database with your tests.Apparently Its not that simple or I missed something :) Then you can submit it to be included for those of use who want it.
-
0
<cite>BBakerMMC: </cite> Strix can you do this for me?
If you create a new test project, make the appropriate EF core DI calls to setup your db context, there is nothing stopping you from hitting live data out of the box. The issue is with test startup and teardown and ensuring that you're not mucking up a good database with your tests.Apparently Its not that simple or I missed something :) Then you can submit it to be included for those of use who want it.
I literally copied and pasted the code from the blog post, and it worked. I don't get what is so hard. The solution has repeatedly been posted.
I just duplicated it on my own. Here is the test class I used in a new project using MSTest, that connects to my local development db. I simply added a reference the the Core.EntityFramework project, and instantiated my db context class.
[TestClass] public class DemoTest { private MyDbContext _db; [TestInitialize] public void Init() { var serviceProvider = new ServiceCollection() .AddEntityFrameworkSqlServer() .BuildServiceProvider(); var builder = new DbContextOptionsBuilder<MyDbContext>(); builder.UseSqlServer("Data Source=.\\SQLEXPRESS;Initial Catalog=MyDb;Integrated Security=True;MultipleActiveResultSets=True") .UseInternalServiceProvider(serviceProvider); _db = new MyDbContext(builder.Options); } [TestMethod] public async Task PassingTest() { var t = await _db.Addresses.FirstAsync(); t.ShouldNotBeNull(); } }The test passes, which it could not if it was not connected to my database.
-
0
I'll try that later. In the meantime grow your arms or shorten your pockets so you can buy a beer. And stop taking life so seriously, it's there to be enjoyed.
-
0
<cite>BobIngham: </cite> I'll try that later. In the meantime grow your arms or shorten your pockets so you can buy a beer. And stop taking life so seriously.
I just picked up a bottle of Corryvreckan this week. I'll cheers with a finger or two :mrgreen:
-
0
Lol, this isnt the use case. It needs to run all the application services and such that are built in to aspnet zero.
<cite>strix20: </cite>
<cite>BBakerMMC: </cite> Strix can you do this for me?
If you create a new test project, make the appropriate EF core DI calls to setup your db context, there is nothing stopping you from hitting live data out of the box. The issue is with test startup and teardown and ensuring that you're not mucking up a good database with your tests.Apparently Its not that simple or I missed something :) Then you can submit it to be included for those of use who want it.
I literally copied and pasted the code from the blog post, and it worked. I don't get what is so hard. The solution has repeatedly been posted.
I just duplicated it on my own. Here is the test class I used in a new project using MSTest, that connects to my local development db. I simply added a reference the the Core.EntityFramework project, and instantiated my db context class.
[TestClass] public class DemoTest { private MyDbContext _db; [TestInitialize] public void Init() { var serviceProvider = new ServiceCollection() .AddEntityFrameworkSqlServer() .BuildServiceProvider(); var builder = new DbContextOptionsBuilder<MyDbContext>(); builder.UseSqlServer("Data Source=.\\SQLEXPRESS;Initial Catalog=MyDb;Integrated Security=True;MultipleActiveResultSets=True") .UseInternalServiceProvider(serviceProvider); _db = new MyDbContext(builder.Options); } [TestMethod] public async Task PassingTest() { var t = await _db.Addresses.FirstAsync(); t.ShouldNotBeNull(); } }The test passes, which it could not if it was not connected to my database.
-
0
public class ConnectionString_Tests : AppTestBase, IDisposable { private readonly MyContext _db; private readonly IEditionAppService _editions; public ConnectionString_Tests() { _db = Resolve<MyDbContext>(); _editions = Resolve<IEditionAppService>(); } [Fact] public void SqlConnectionStringBuilder_Test() { var csb = new SqlConnectionStringBuilder("Server=localhost; Database=Customs; Trusted_Connection=True;"); csb["Database"].ShouldBe("Customs"); } [Fact] public async Task TalksToDatabase() { LoginAsHostAdmin(); var t = await _editions.GetEditions(); t.ShouldNotBeNull(); } public void Dispose() { _db?.Dispose(); } } -
0
Also, I would note that since most of the application layer has authorization configured, then you're either going to want to test the domain layer directly, or you're going to need to disable authorization and the permission checker in your test module as follows:
var auth = IocManager.Resolve<IAuthorizationConfiguration>(); auth.IsEnabled = false; Configuration.ReplaceService<IPermissionChecker, NullPermissionChecker>();If you've built your own project to test in, by copying the existing test project, then you can also use the existing EF initialization process by removing the following from the module constructor:
abpZeroTemplateEntityFrameworkCoreModule.SkipDbContextRegistration = true;You will need to add a connection string setting to the appconfig.json in your test project. You will also need to add the following so that it picks the correct connection string from your app config:
var configuration = GetConfiguration(); Configuration.DefaultNameOrConnectionString = configuration.GetConnectionString( MyConsts.ConnectionStringName );Also, I'm not sure how compatible the tests will be with the LoginAs methods provided in AbpTestBase, since those are designed to be used with the seeded in memory database, but you can effectively log in as follows:
AbpSession.TenantId = 3; AbpSession.UserId = 4Remember that this will not actually log you in, and if you want to test actual permissions, you will have to write your own much more complex solution, and implement your own session and signin manager, since you do not have access to an HttpContext.
-
0
Now I that I have provided a solution I feel terribly unclean, so I'm going to go have a glass of scotch and play with my new 75192 Millennium Falcon UCS lego set to drink the embarrassment away :ugeek:
-
0
To clarify steps
Step 1 - Copy/Paste Existing Test project Step 2 - Add Connection String to AppSettings Step 3 - Comment Out in TestModule CTOR abpZeroTemplateEntityFrameworkCoreModule.SkipDbContextRegistration = true; Step 4 - Set ConnectionString in PreInitialize - Configuration.DefaultNameOrConnectionString = configuration["ConnectionString"]; Step 5 - Add auth disable code Step 6 - Disable Seed in TestBase Step 7 - Disable In memory and add new service collector in DI/ServiceCollectionRegistrar, also remove the inMemoryOpen and EnsureCreated.
public static void Register(IIocManager iocManager) { RegisterIdentity(iocManager); var builder = new DbContextOptionsBuilder<PlatformDbContext>(); //var inMemorySqlite = new SqliteConnection("Data Source=:memory:"); //builder.UseSqlite(inMemorySqlite); var serviceProvider = new ServiceCollection() .AddEntityFrameworkSqlServer() .BuildServiceProvider(); builder.UseSqlServer("YOURSTRING") .UseInternalServiceProvider(serviceProvider); iocManager.IocContainer.Register( Component .For<DbContextOptions<PlatformDbContext>>() .Instance(builder.Options) .LifestyleSingleton() ); // inMemorySqlite.Open(); // new PlatformDbContext(builder.Options).Database.EnsureCreated(); }public PlatformTestModule(PlatformEntityFrameworkCoreModule abpZeroTemplateEntityFrameworkCoreModule) { // abpZeroTemplateEntityFrameworkCoreModule.SkipDbContextRegistration = true; } public override void PreInitialize() { var configuration = GetConfiguration(); Configuration.DefaultNameOrConnectionString = configuration["ConnectionString"]; Configuration.UnitOfWork.Timeout = TimeSpan.FromMinutes(30); Configuration.UnitOfWork.IsTransactional = false; //Disable static mapper usage since it breaks unit tests (see https://github.com/aspnetboilerplate/aspnetboilerplate/issues/2052) Configuration.Modules.AbpAutoMapper().UseStaticMapper = false; //Use database for language management Configuration.Modules.Zero().LanguageManagement.EnableDbLocalization(); RegisterFakeService<AbpZeroDbMigrator>(); var auth = IocManager.Resolve<IAuthorizationConfiguration>(); auth.IsEnabled = false; Configuration.ReplaceService<IPermissionChecker, NullPermissionChecker>(); Configuration.ReplaceService<IEmailSender, NullEmailSender>(DependencyLifeStyle.Transient); Configuration.Modules.AspNetZero().LicenseCode = configuration["AbpZeroLicenseCode"]; }public class CustomTest : AppTestBase, IDisposable { private readonly IEditionAppService _editionAppService; public CustomTest) { LoginAsHostAdmin(); _editionAppService = Resolve<IEditionAppService>(); } [MultiTenantFact] public async Task Should_Get_Editions() { var editions = await _editionAppService.GetEditions(); editions.Items.Count.ShouldBeGreaterThan(0); } } -
0
@strix20 awesome :)
-
0
The tests that ship with the solution are mostly Integration Tests, and a few unit tests as well. Threy are integration tests because they touch multiple layers (from the AppService through to the Database).
