Posts Tagged ‘test’

.NET Multi-threading test scenario /
profiler comparison results

 

Test scenario is a multi-threading test of 100 concurrent threads performing object allocations.
In each test case the threads are started and terminated by the tester program for 50 times.
The test scenario is repeated 15 times.

The test was performed with the following. NET profilers:

  • without Profiler
  • Ants 6.0
  • DotTrace 4.0
  • SpeedTrace Pro 3.4
  • Visual Studio 2010
  • AQTime 6.0

.NET Multi-Threading test scenario / Profiler comparison results: Ants 6.0, DotTrace 4.0, SpeedTrace Pro 3.4, Visual Studio 2010, AQTime 6.0

DotTrace 4.0, Ants 6.0 failed the test because they are not able to present any test results.

SpeedTrace, AQTime and Visual Studio provide test results whereas AQTime adds major runtime (>1000%) overhead,
Visual Studio adds runtime overhead (> 300 %) and consumes much time to show the results.

Only SpeedTrace traces the application without big interference, so accurate results will be presented.

Would you like to reproduce the test? Simple use our ThreadSuspendTester application.

Test Your .NET Profiler/Tracer Application

With our ThreadSuspendTester application (see C# source code and downlaod) you can get an idea of the abilities of your .NET testing tool.

Performance analysis with extremely multi threading - ThreadSuspendTest

ThreadSuspendTester is a small test program which records the overhead of .NET profilers or .NET tracers, especially at high concurrency.
The test program starts 100 threads, which in turn make allocations and garbage collections.

If you use SpeedTrace Pro as tracing/profiling tool, you will not detect much difference by the presence of the profiler. In contrast to other tools on the market, the overhead is very small.

Get an idea of your Profiler/Tracer. Compare the result with SpeedTrace Pro – You will be amazed.

Download ThreadSuspendTester.zip (Program.cs) Download ThreadSuspendTester.zip (Program.cs)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using System.Collections;
using System.Diagnostics;
 
namespace ConsoleApplication1
{
    class Program
    {
        static bool terminated = false;
 
        static void Main(string[] args)
        {
            for (int i = 0; i < 15; i++)
            {
                Console.Write(string.Format("Test {0}...", i+1));
                ExecuteTest();
            }
 
            Console.WriteLine("DONE - Press ENTER to exit");
            Console.ReadLine();
        }
 
        private static void ExecuteTest()
        {
            var sw = new Stopwatch();
            sw.Start();
 
            System.Threading.Thread.CurrentThread.Priority = ThreadPriority.Normal;
            GC.Collect();
 
            for (int i = 0; i < 50; i++)
            {
                var threads = StartThreads();
 
                foreach (Thread t in threads)
                {
                    t.Abort();
                    GC.Collect();
                }
                terminated = true;
 
                foreach (Thread t in threads)
                {
                    t.Join();
                }
            }
 
            Console.WriteLine("Done..{0} ms", sw.ElapsedMilliseconds);
        }
 
        private static List<Thread> StartThreads()
        {
            var result = new List<Thread>();
            for (int i = 0; i < 1; i++)
            {
                var thread = new Thread(doNothingTest);
                thread.Priority = ThreadPriority.Lowest;
                thread.Start();
                result.Add(thread);
            }
 
            var thread2 = new Thread(allocTest);
            thread2.Priority = ThreadPriority.Lowest;
            thread2.Start();
            result.Add(thread2);
 
            for (int i = 0; i < 100; i++)
            {
                var thread = new Thread(allocTest);
                thread.Priority = ThreadPriority.Lowest;
                thread.Start();
                result.Add(thread);
            }
            return result;
        }
 
        static void allocTest()
        {
            int j=0;
            while (!terminated)
            {
                ArrayList list = new ArrayList();
                for (int i = 0; i < 1000; i++)
                {
                    list.Add(new Program());
                }
                list.Clear();
                if (j % 1 == 0)
                {
                    GC.Collect();
                }
 
                System.Threading.Thread.Sleep(1);
            }
        }
 
        static void doNothingTest()
        {
            while (!terminated)
            {
            }
        }   
    }
}

More about SpeedTrace >>
Download Now (free evaluation) >>

SpeedTrace adds new unit testing plugins

Our latest and perhaps last build (Version 3.4.15) prior to release has been complemented with optional plugins to wide-spread tools in order to facilitate unit testing especially for a larger group of developers familiar with and attached to those tools:

These options have been added for extra convenience rather than out of necessity primarily because we believe that systematic unit testing, test automation and test-driven development are so important.

Why unit testing?

Unit testing helps you verify and validate your source code to see (and measure) whether (to what extent) its components suit their designed purposes and behave as expected.  The idea is to isolate each part of a program and show that the individual components are well-formed right from the outset. Unit tests minutely state the “contract specifications” of a piece of code as to what it is supposed to satisfy and thus afford several benefits:

  • Find problems early in the development cycle
  • Ensure that code reflects intended functions in spite of changes
  • Test parts first to facilitate integration testing
  • Document critical code unit by unit
  • Prevent performance from deteriorating.

Unit testing for performance and bug trapping

For bug trapping create a test that replicates the bug, and then redesign your code accordingly. By adding the fix to your test suite you can ensure that the test case will always be checked throughout the entire development lifecycle. For continued performance assurance you can use the SpeedTrace API in order to trace the executed scenarios and to perform sanity checks making standard assumptions that the elapsed times are within bounds of the desired performance.

Unit testing in test automation enviroment for optimized lifecycle management and QA-internalized development

Unit testing in test automation enviroment for optimized lifecycle management
and QA-internalized development

Why test automation (TA)?

Unit testing within the SpeeedTrace Pro Test Auotmation (TA) environment is performed systematically in “zipper”-fashion parallel to the development process repeatedly unit by unit at short intervals (See diagram). It thus becomes an integral part of the TA and lends itself appreciably to optimizing lifecycle management and proactively internalizing QA into your entire development process.

Unit testing with SpeedTrace Pro

Of course, you can do all this with SpeedTrace Pro using a much wider array of features, functions and triggers. Nevertheless, we wanted users accustomed to these wide-spread unit testing tools to also benefit from SpeedTrace Pro’s more comprehensive approach to software diagnostics.