It is pretty easy and straight forward to create a dotnet core application and run your unit tests during a build process. However, I also have some integration tests that I run in my Dev and QA environments where I actually hit my test database. This is not something that you would run in the build process as I do not have a database on the build machine and this is not something that would be worth the trouble to install one, especially if you are using a hosted build agent.
This is a test that I would run as part of my deployment, after I have installed the application in one of my testing environments I want to know that the database is working correctly before I start running even slower tests like my automated functional graphical user interface tests. The process that you used to run the dotnet core unit tests doesn’t work quite the same. Let’s go over the normal steps that you would take in your build process to build and test, then we can cover why running integration tests during a deployment is not going to work quite the same.
These are the actual commands that are used to build and then test the dotnet core application. In both of these situations it is expecting source code that it is building and then testing. However, when you are deploying to an environment we are using the finished packages of the artifact to install the new version. Remember we build once and deploy to every environment.
dotnet test /?
Usage: dotnet test [options] <PROJECT | SOLUTION> [[--] <RunSettings arguments>...]]
<PROJECT | SOLUTION> The project or solution file to operate on. If a file is not specified, the command will search the current directory for one.
If you look at the help for dotnet test you see that it expecting either a project or a solution which are both part of source not the finished compiled code. This makes it fairly clear that the dotnet test is not going to be the way to go to run any integration tests during deployment of an environment.
If you run the help on just the dotnet command there is another command that might be the answer to all of this.
Usage: dotnet [runtime-options] [path-to-application] [arguments]
add Add a package or reference to a .NET project
build Build a .NET project
build-server Interact with servers started by a build
clean Clean build outputs of a .NET project
help Show command line help
list List project references of a .NET project
msbuild Run Microsoft Build Engine (MSBuild) commands
new Create a new .NET project or file
nuget Provides additional NuGet commands
pack Create a NuGet package
publish Publish a .NET project for deployment
remove Remove a package or reference from a .NET project
restore Restore dependencies specified in a .NET project.
run Build and run a .NET project output
sln Modify Visual Studio solution files.
store Store the specified assemblies in the runtime package store.
test Run unit test using the test runner spedified in a .NET project
tool Install or manage tools that extend the .NET experience.
vstest Run Microsoft Test Engine (VSTest) commands
Well that looks like it might have some promise. First thing we should do is just confirm what it is expecting for parameters.
dotnet vstest /?
Usage: vstest.console.exe [arguments] [Options] [[--] <RunSettings arguments>...]]
Run test from the specified files or wild card pattern. Spearate multiple test file names or pattern by spaces. Set console logger verbosity to detailed to view matched test files.
I am pretty sure that this is going to work so I include the compiled test assemblies in the artifact package of the build and use that command to run them in deployment. However, that does not seem to work just like that because I am starting to see errors related to missing dependency files. These are not files that I created but things it is expecting from the framework.
Somewhere along the way I did see some documentation mentioning something about published files. Which I was not sure what they were getting at. I run the dotnet publish command against the web site that I am deploying which makes sense as this is something that we have always done in order to get the pieces that are needed for the web site and this is usually zipped up. Turns out you do the same thing against the test project which includes all those dependencies that I was missing. My commands would look something like this:
dotnet publish <integrationTestLocation> --output <ArtifactStagingLocation>\Tests
That folder would then contain all the files I need to run these tests in a new environment. Then all I would need to do from a command line task in the release pipeline is run the following command.
dotnet vstest <integrationTestLocation>\Integration.Test.dll /logger:trx
Run integrated test run and reports to Azure DevOps just like any other full framework test has done in the past.