NCrunch is an automated concurrent testing tool for Visual Studio. It intelligently runs automated tests so that you don’t have to, while giving you amazingly useful information about your tested code, such as code coverage and performance metrics, inline in your IDE while you type. 

 NCrunch is an automated testing tool for Microsoft Visual Studio. It has an automated test runner, and gives information about the tested code, such as code coverage and performance metrics.  Article explains how to integrate Telerik JustMock with NCrunch inside Visual Studio. 

Configuring NCrunch 

JustMock supports all of the listed options from the NCrunch Configuration Wizard. This means, you will be able to execute your tests in parallel and decide whether to optimizing your test execution or memory efficiency. 

To use NCrunch along with the JustMock profiler, you will need to: 

1. Show the NCrunch configuration window for your test assembly. You could right click the test assembly from the NCrunch Tests window and select Configure selected component: 

Then, navigate to Test Settings and point the exact location of the JustMockRunner (by default: C:\Program Files (x86)\Telerik\JustMock\Libraries\JustMockRunner.exe) for “Proxy process file path“, as shown below: 

Save the solution, rebuild and enjoy the integration between both JustMock and NCrunch. 

Solution-level NCrunch configuration settings 

Default Values: Dynamic Analysis (NUnit), Static Analysis (all other frameworks) 

Dynamic Analysis 

When set to DynamicAnalysis, NCrunch will perform an additional analysis step against the test assembly after it has been built, in the form of an analysis task (visible in the processing queue). This analysis task involves creating a runtime application domain that contains the loaded test assembly. The application domain allows the analysis task to extract test information from the assembly through integration with the applicable test framework, which usually relies on runtime reflection to find tests. 

Dynamic Analysis is considered to be the most reliable and standardised approach to discovering tests within an assembly, although it has a higher overhead as runtime reflection is generally slower than static analysis, and requires an application domain to be built up-front. 

Static Analysis 

When set to StaticAnalysis, NCrunch will find tests within the test assembly by analysing the assembly in its binary form and using the CLR metadata available. This analysis happens as part of the instrumentation process NCrunch uses to manipulate the built assembly. 

Static analysis is the fastest way for NCrunch to find tests, but it relies on inferred knowledge of the workings of the applicable test framework. Because in most cases static analysis does not involve direct interrogation of the test framework for information, it is less aligned with test framework standards and may not find tests that rely on edge features specific to the test framework. 

 Installation 

If you haven’t already, first go and download an installer for the version of Visual Studio you are working with. 

There are six separate installers for six different versions of Visual Studio (2008/2010/2012/2013/2015/2017). Installing NCrunch for one version of Visual Studio should have no impact on other versions you may have running on your machine. 

Once you’ve downloaded and stepped through the MSI, NCrunch should be ready to go. Note that if you had Visual Studio open during the installation, you’ll need to restart your IDE before NCrunch will load. 

First Time Run 

For your first run with NCrunch, it’s often a good idea to try it out with a small, well tested solution that you understand well. This should give you a bit more space to get your head around how the background processing works. NCrunch also works very well with large solutions, but often requires a bit more configuration to reach the best possible experience. 

By default, NCrunch will always start as disabled for a solution before it is explicitly turned on from the top menu. 

Once you’ve opened up your solution, select Enable from the NCrunch Visual Studio menu. NCrunch should wake up and show a wizard to help you configure it for your machine, and the solution you’re working in. It’s worth stepping through this wizard and answering the questions, as this will save you from needing to configure the individual settings yourself. 

After running through the wizard, you can get a handle on what’s going on under the hood by opening up the Processing Queue from the top menu. This should open up a dockable tool window that shows the inner workings of the NCrunch engine. 

The processing queue is a useful way to get detailed information about the background processing, such as finding out which tests are being run, how the work is prioritised, etc. You could consider it your ‘what the hell is going on?’ view that you can check any time while you work. 

If all is well, you should have a nice long list of blue ticks down the side of the queue by the time the engine has finished its first run. If you have any red crosses, possibly this means you are using some kind of custom build configuration that will need some investigation. Have a look at troubleshooting project build issues for more information on how to fix situations like this. 

During its first full build and test cycle, NCrunch will keep track of the execution times of tests within your solution along with details about the build dependencies between your classes. It will then use this information to optimise future test cycles to give feedback as fast as possible. 

Analysing Test Results 

Now that you’ve cleared a full cycle of data from the engine, it’s time to make sense of it. The processing queue will show some summary details about the tests that ran in each of the tasks processed by the engine, but this isn’t an ideal way to find out what actually happened. So it’s now time to close the processing queue and instead open up the Tests window from the top menu. 

The Tests Window is a useful window to keep open while you’re developing. It provides immediate feedback on all relevant output from the engine (including failed builds and failing tests). It will also give some information about which tests/builds are being run at a particular point in time (watch out for the clock icons). 

Depending on the state of your solution and how you’ve engineered your tests, you may see some failures here. NCrunch does everything it can to run your automated tests in the same way as you usually would, but sometimes tests are programmed with certain environmental considerations that NCrunch isn’t aware of. If you have any test failures that don’t look right to you, have a look at troubleshooting test issues. 

Working With Code Coverage 

So… now we know what’s going on under the hood, and we can see the results that come from it… Now it’s time for the best bit - inline code coverage and performance metrics! 

As part of it’s build process, NCrunch will automatically instrument your source code and perform code coverage analysis while processing your tests. The analysis is language agnostic and shouldn’t interfere with your runtime code. 

When you have a look through some of your classes known to be covered by your tests, you should see a line of coloured circles (markers) stretching down the left margin. These markers indicate test code coverage for their adjacent lines of code. 

The default colouring of these markers is explained below: 

Green marker: Line is covered by passing tests 

Red marker: Line is covered, but by at least one failing test 

Black marker: No tests are covering this line (feel the fear!!!) 

Hot Spot marker: Line is covered by tests but executes slowly 

In Visual Studio 2010 and above, the markers will fade out when they are indicating coverage for tests that haven’t been run against the latest version of your source code. In Visual Studio 2008, this is shown by the markers shrinking in size. 

Note that you can also change these colours at any time. It’s also possible to change the marker shapes. 

Where exception stack traces are available for your tests, these will be projected over the markers as a series of red Xs showing where the exceptions were raised. 

At any time you can hover the mouse over a marker for tooltip information on the number of tests covering the line, exception details, etc. You can also click on a marker to access a popup menu showing all covering tests. Double-clicking a test in the popup will navigate the IDE to the first line of the test in its fixture. You can right-click on a marker for a context menu providing more options. 

The constant on-the-fly analysis of your test coverage allows NCrunch to maintain up-to-date mappings between the code coverage and your source code. These mappings are retained across multiple test cycles and always find their way back to the line of code they relate to. This means that it’s possible for the engine to kick off a long running test for 20 minutes, then still be able to make constructive use of the code coverage returned by the test when it completes – even if you made several code changes while the test was executing. 

NCrunch is an automated parallel continuous testing tool for Visual Studio .NET. It intelligently takes responsibility for running automated tests so that you don’t have to, and it gives you a huge amount of useful information about your tests (such as code coverage and performance metrics) inline in your IDE while you work.

NCrunch is a software product by Remco Software and was written by software developer Remco Mulder, who owns the company. It is a tool written specifically to allow developers to practice continuous testing in Visual Studio. NCrunch is a commercial product with a tiered pricing model and full-blown customer support. And it operates as a plugin to Visual Studio so there is no need to integrate or operate any kind of standalone application. It drops right in, comfortably with a tool with which you are already familiar.

Features :

1. Automatic ConCurrent Testing: 

 A normal cycle of test driven development makes you stop to run your tests so often that it’s just plain painful. Think about the steps you usually go through: 

• Write the test 
• Stop and run the test 
• Write the code under test 
• Stop and run tests 

• Refactor the code under test 
• Stop and run tests 
• Drink some coffee and repeat 

NCrunch takes away all the pain and leaves a warm happy feeling behind. So you end up with: 

• Write the test 

• Write the code under test 
• Refactor the code under test 

2.Code Coverage: 

NCrunch collects test coverage for your code while it runs your tests. This is shown next to your code in coloured markers showing which lines the tests touched, with marker colours indicating pass or fail status. You can also navigate to any covering tests from any line of code, making it easy to see which tests you might impact with a change. Full code coverage metrics are also available for your entire solution, allowing you to see where your code coverage is heavy and where it’s light. 

 3.Performance Metrics: 

NCrunch profiles your tests during their execution to pick up the execution time of every line of code under test. Metrics are shown inline conveniently with a tooltip, and ‘hot spots‘ are shown with special colouring on the code coverage markers. 

4.Inline Expectation Details: 

The stack traces of exceptions thrown from your tests are processed by NCrunch and projected over the code coverage markers. This makes it really easy to spot where your tests went wrong, without the information getting in your way. No matter where you are in your source code, you’ll be able to analyse problems quickly and without fuss. 

5.Intelligent test Execution:

NCrunch tracks all sorts of test related data, and it uses it in the most intelligent way possible. Tests that you have recently impacted with your code changes are prioritised for execution using sophisticated and high performance IL-based change mapping. NCrunch uses a powerful weighting system designed to give you the most important feedback as fast as possible. 

6.Distributed Processing: 

NCrunch can offload build and test work to other computers for processing. Tasks are cleanly farmed out to any number of connected machines, forming grids to execute tests. Grid servers can be shared between developers allowing teams to pool their resources. Grids can even be scaled into the cloud to maximise testing throughput. Distributed processing with NCrunch is highly effective, allowing concurrent execution of dozens or possibly even hundreds of tests at any one time. 

7.Small Memory Footprint: 

NCrunch was built with big projects in mind. It has been fully tested and designed to work well on solutions with potentially millions of lines of code and many thousands of tests. An extensive range of simple configuration options allow you to tune NCrunch’s CPU and memory consumption to best fit your specific needs. 

8.Parallel Execution: 

Where your tests support it, NCrunch can be configured to run them in parallel across separate processes. This can cut your end-to-end test times down by a huge factor over any normal synchronous test runners. Tests will never be run in parallel within the same process, so there’s no thread safety risks or problems with singletons. 

9.Optimised Build Process: 

The NCrunch build process is heavily optimised and will only build parts of your solution that have changed, making it much faster than a standard Visual Studio or command-line build. 

10.Smart Multi Processor Support: 

NCrunch splits its processing onto separate processor cores that are not used by Visual Studio, and it will only use as much CPU as you tell it to. This helps to ensure that the constant churning of tests won’t adversely impact your coding experience in Visual Studio. 

11.Easy Debugging:

NCrunch makes it really easy to debug your tests. You can break into a line of code with the debugger using a single context menu command or shortcut key. 

12.Part Continuous / Manual Mode: 

If you don’t like having all your tests run automatically, you can choose to run them manually. NCrunch lets you create very configurable filters to choose which tests you want to run automatically, letting you use it as a manual or partially continuous test runner. In constrained environments it is even possible to configure the engine to run only impacted tests, greatly reducing resource consumption. 

The Major Features Of  NCrunch:

• Code Coverage : NCrunch collects test coverage for your code while it runs your tests. This is shown next to your code in coloured markers showing which lines the tests touched, and whether the tests passed or failed. You can also navigate to any covering tests from any line of code, making it easy to see which tests you might impact with a change.
• Performance metrics : NCrunch profiles your tests during their execution to pick up the execution time of every line of code under test. Metrics are shown inline conveniently with a tooltip, and ‘hot spots’ are shown with special colouring on the code coverage markers.
• Inline Exception Details : The stack traces of exceptions thrown from your tests are processed by NCrunch and projected over the code coverage markers – making it really easy to spot where your tests went wrong, without the information getting in your way.
• Intelligent Execution of Tests : NCrunch will always run your tests in the most intelligent way possible, prioritising tests that you have recently impacted with your code changes. It uses a powerful weighting system designed to give you the most important feedback as fast as possible.
• Parallel Execution : Where your tests support it, NCrunch can be configured to run them in parallel across separate processes. This can cut your end-to-end test times down by a huge factor over any normal synchronous test runners.
• Smart Support for Multi-core/Multi-processor Systems : NCrunch will only use as much CPU as you tell it to, ensuring that the constant churning of tests won’t adversely impact your coding experience in Visual Studio.
• Easy Debugging : NCrunch makes it really easy to debug your tests, allowing you to break into a line of code with the debugger using a single context menu command or shortcut key.
• Full Synchronisation with your Codebase : NCrunch automatically picks up everything – whether in an unsaved code window or on the file system. There’s no need to manually save your work or rebuild your projects in order to run your tests. NCrunch does it all. Whenever you feel like you want more control, there’s also a manual mode.
• Support for Large Systems : NCrunch was built with big projects in mind. It has been fully tested and found to work well on solutions with hundreds of thousands of lines of code and many thousands of tests.