A couple weeks ago, I became engaged with some folks on Twitter about their specific complaints about MSTest. I received five complaints, some of which I sent off to Microsoft. I’d like to share the answers and Microsoft’s response in hopes of creating more awareness around the tool’s capabilities.

Artifact Cleanup – When tests are run, artifacts such as test results and the (optionally) shadow-copied assemblies are left behind. For a team running many, frequent tests, this can quickly eat up disk space.

In Visual Studio 2008, a new option was created to help alleviate this problem. Under Tools > Options > Test Tools > Test Execution, there is a new setting called Test Result Management which contains a numeric option to limit the number of old test results. For tests executed as part of an automated Team Build, the results should always be scorched by Team Build as they are subsequently stored in the Team Foundation database.

Extensibility: Creating custom MSTest attributes is a bit more drawn out than most would like.

Microsoft has simplified the ability to create custom test types by allowing inheritance from existing MSTest attributes. This builds on the Unit Test base and allows you to customize any part (initialize, cleanup, execution) of the test being run.

• Creating Custom Test Types in Visual Studio Team System on MSDN
• Introducing YUnit, a Microscopic Unit Test Framework (for Team Test) – on Peli’s Farm

Transactions – NUnit has an option for Transactional tests, which automatically rollback after the test is run.

This is not provided out of the box. You can either code up the transaction manually (http://www.jimbodriven.com/2007/11/on-unit-tests-seed-data-and.html), or use the extensibility points to create your own test type to handle this.

• MSTestExtensions on CodePlex

Data-driven Testing – In NUnit, there is a RowTest attribute which allows data to be specified to use in testing (http://mjeaton.net/blog/archive/2008/11/03/getting-started-with-nunits-rowtest.aspx).

Microsoft is looking into this for future releases. For now, take a look how to achieve this in the current releases. This method has the added benefit of not having to recompile tests to change test data.

• How to simulate RowTest with MSTest on Simone Chiaretta’s blog

Accessibility: It was suggested that MSTest should be available as a standalone, free product, similar to NUnit, MBUnit, xUnit, etc.

Microsoft is looking into this for future releases.

So of course my next question is, what else don’t you like about MSTest?