1Current 2Fixed: GITHUB-841: testName from @Test is now used and available from ITestResult#getName() and ITestResult#getTestName() (Julien Herr) 3New: GITHUB-776: Add BeforeClass/AfterClass like on ITestListener (@vguna & Julien Herr) 4Fixed: GITHUB-872: Enable end-users of TestNG to alter XmlSuite and XmlTest (Krishnan Mahadevan) 5New: GITHUB-900: Support @Listeners in annotation transformer (Julien Herr) 6New: GITHUB-898: Activate XML validation when possible (Julien Herr) 7Fixed: GITHUB-889: XmlSuite in nested directories results in FIleNotFoundException (Virender Singh) 8Fixed: GITHUB-811: Timeout is not working with parallel=tests (@michael-yxf & Julien Herr) 9Fixed: GITHUB-839: Missing encoding meta data for report file (@banbq & Julien Herr) 10Fixed: GITHUB-876: NullPointerException creating tests with parameters by a factory (@vixgeo & Julien Herr) 11New: GITHUB-886: Add some checks on factory methods (Julien Herr) 12New: GITHUB-874 / GITHUB-875 / GITHUB-882 / GITHUB-850 : Some code cleanup (Testo Nakada) 13Fixed: GITHUB-866 / GITHUB-869 : Some attributes were not cloned when XmlSuite#clone was used (Virender Singh) 14Fixed: GITHUB-842: Add TestResult#getTestName() support for @Test(testName) (Julien Herr) 15 166.9.9: 172015/10/27 18 19Fixed: GITHUB-829: Allowing suites to have duplicate names. You can now configure the same suite-file to run multiple times. (Eduardo Born) 20Fixed: GITHUB-834: nested suites not supported by 'testnames' (Tibor Digana & Julien Herr) 21 226.9.8: 232015/10/12 24 25Replace 6.9.7 that was build with Java8 by error. 26 276.9.7: 28No official release 29 30Fixed: GITHUB-798: Set suitethreadpoolsize for Maven Surefire (Jan Dundáček) 31Fixed: GITHUB-171: ISuiteListener methods called multiple times if multiple test elements (Daniel Qian & Julien Herr) 32Fixed: GITHUB-169: IInvokedMethodListener methods executed several times before/after each test method (Mario Duarte & Julien Herr) 33Fixed: GITHUB-154: MethodInterceptor will be called twice (Tim wu & Julien Herr) 34 356.9.6: 362015/07/15 37 38New: GITHUB-717: Add assertThrows and expectThrows (Ryan Schmitt) 39Fixed: GITHUB-755: Fixed reporting of retried tests (Ryan Schmitt) 40Fixed: GITHUB-773: Test should not be skipped when the exception is expected (@CandyLiuM & Julien Herr) 41 426.9.5: 432015/07/12 44 45Fixed: The ServiceLoaderTest on Windows (Mathieu Sebire) 46Fixed: GITHUB-691: Fix classloading issue when using TestNG 6.9.4 and JMockit. (Mathieu Sebire) 47Fixed: GITHUB-686: IAnnotationTransformer.transform is called for methods with testClass populated. (Łukasz Rekucki & Julien Herr) 48Fixed: GITHUB-420: Before/AfterSuite methods may not run, when use inheritance, and enabled=false (Jakub Tokaj & Julien Herr) 49Fixed: GITHUB-697: Make addFailedInvocationNumber thread-safe (Ryan Schmitt) 50Fixed: GITHUB-698: Fix exit code reporting when IRetryAnalyzer is used (Ryan Schmitt) 51Fixed: GITHUB-465: assertEquals(Collection, Collection) prints "null" when collections are different sizes (Michael Diamond) 52New: GITHUB-710: AppVeyor is used for continuous integration on Windows (Julien Herr) 53Fixed: GITHUB-599: IHookable ignored when a timeout is set (@ryanlevell & Julien Herr) 54New: GITHUB-723: Allow users to add their own suite parser (Julien Herr) 55Fixed: Allow '-testnames' option to work with '-xmlpathinjar' (@earthling) 56Fixed: GITHUB-739: TestNG skips all test classes from suite when a @BeforeClass fails (Priyanshu Shekhar & Julien Herr) 57Fixed: GITHUB-471: If @beforeMethod or @afterMethod fails then all children of the same base class will be skipped (Anton Panferov & Julien Herr) 58Fixed: GITHUB-595: testng hang at switching test cases when running test cases with high thread count (vit0rg) 59 60Eclipse: 61 62Fixed: The 57% freeze bug (Patrick Hensley and @denyska) 63 646.9.4: 652015/05/09 66 67Added: GITHUB-631: Avoid the static limitation of external DataProvider. (Julien Herr) 68Added: GITHUB-631: Allow to use Guice injection in DataProvider. (Julien Herr) 69Added: Drop support of Java6 and previous. 70Added: GITHUB-617: Allow injection of org.testng.ITestContext into the guice parent module. (Julien Herr) 71Fixed: GITHUB-606: RetryAnalyzer loops endlessly. (Krishnan Mahadevan) 72Fixed: GITHUB-618: Start TestNG from jar cause recursive run of tests from packages in Suite XML without ".*" on the end (Stas Gromov) 73Fixed: GITHUB-639: Typo on preserveOrder (tabei-k & Julien Herr) 74Fixed: GITHUB-632: Typo in doc (Pétur Ingi Egilsson & Julien Herr) 75Fixed: GITHUB-629: InvokedMethod doesn't recognize configuration method (Jan Mewes & Julien Herr) 76Fixed: GITHUB-615: XmlSuite, XmlTest: Time-out tag not preserved (jphollingworth & Julien Herr) 77New: GITHUB-638: Travis CI is used for continuous integration (Julien Herr) 78New: GITHUB-647: SonarQube is used to follow technical debt (Julien Herr) 79Added: GITHUB-616: org.testng.internal.Version will be always up-to-date (Julien Herr) 80Fixed: GITHUB-634: Review of the collections package (Julien Herr) 81Fixed: GITHUB-624: Fixed failure/error inversion in JUnitReportReporter (Jerome Jacob) 82Fixed: GITHUB-545: TestNG running JUnit tests but not reporting all results for parameterized tests (Jonathan Leitschuh & jdillet) 83Fixed: GITHUB-610: CustomizedSuites must be saved using utf-8 encoding (Juha Heljoranta) 84Fixed: GITHUB-602: NoClassDefFoundError in TestNGClassFinder.<init> (aanno) 85Fixed: GITHUB-529: Close InputStream and OuputStream after use (Andrew Gaul) 86Fixed: GITHUB-532: Create the parent directory if it's missing (Ion Savin) 87Fixed: GITHUB-541: Some OSGi manifest fixes (Evgeny Zhuravlev) 88Fixed: GITHUB-657: Fix OSGI Import-Package to make jUnit4 dependency optional (Xavier Fournet) 89Fixed: GITHUB-523: externally synchronize our use of the static SimpleDateFormat (mcosby) 90Fixed: GITHUB-477: Typo in DTD attribute comment (Kamil Szymański) 91Fixed: GITHUB-353: Typo in documentation (Jan Święcki) 92Fixed: GITHUB-656: Upgrade to JCommander 1.48 (Ryan Schmitt) 93Fixed: GITHUB-582: TestNG tests don't pass reliably on JDK8 (Ryan Schmitt) 94New: GITHUB-645: TestNG project on Google Code redirect to GitHub 95Fixed: GITHUB-310: Upgrade Guice (kronar & Julien Herr) 96Fixed: GITHUB-87: @BeforeSuite/@BeforeTest methods happens to be disabled by mistake (romlom & Julien Herr) 97Fixed: GITHUB-425: Wrong invocation order with lastTimeOnly (Rafael Winterhalter & Julien Herr) 98Fixed: GITHUB-417: Expected Exceptions Message fails to match multi-line messages (Michael Diamond) 99New: GITHUB-663: Add Guice Stage configuration for a suite (Clément Guillaume) 100 1016.8.21: 1022015/02/02 103 1046.8.15: 1052015/01/14 106 107Fixed: OutOfMemoryException while generating reports. 108Fixed: GITHUB-566: Build does not fail when successPercentage for @Test is not met 109Fixed: XmlTest#setGroupInstances was not being shown in toXml(). 110Fixed: GITHUB-376: Some results can be lost (Konstantin Savin). 111Fixed: Handle relative paths of Suite XML files properly (Nalin Makar) 112 1136.8.5: 1142013/05/13 115 116Fixed: the OutOfMemoryException in reports 117Fixed: Surefire + listeners "Can't load class" problem 118 119=========================================================================== 1206.8.1 1212013/03/30 122 123Added: Descriptions in the HTML reports 124Added: Various improvements to EmailableReporter (Abraham Lin) 125Added: Allow injection of java.lang.reflect.Constructor and org.testng.ITestNGMethod into DataProvide (Vladislav Rassokhin) 126Fixed: Assertions in the Assertions class were not failing properly. 127Fixed: GITHUB-337: ConfigurationMethod#m_instance set to Boolean.FALSE due to incorrect constructor call in clone() + auto-boxing (davidely) 128Fixed: Fix NPE for dependency methods/groups (Krishnan Mahadevan) 129Fixed: preserve-order bug (found by VladSarrokhin). 130Fixed: GITHUB-300: OutOfMemoryException from reporters when there are a lot of tests 131Fixed: GITHUB-137: Main parameters with a default value should be overridden if a main parameter is specified 132Fixed: GITHUB-107: Allow enum values without converting them to uppercase. 133Fixed: @Guice with no modules specified is now supported 134Fixed: Reporter.log() invoked from listeners were being discarded 135 136Eclipse: 137Added: Predefined listeners (Tim Wu) 138Fixed: Compare dialog 139 140=========================================================================== 1416.7 1422012/07/15 143 144Added: Big performance improvement when generating the reports (Frank Pavageau) 145Added: <dependencies> allows you to specify group dependencies in testng.xml 146Added: Blow up early if trying to include/exclude an unknown method 147Added: <parameters> can now be specified under <include> (Storm Qi) 148Added: GITHUB-243: Add Reporter Output per Test in XMLReporter (dunse) 149Fixed: Better HTML escaping of the stack traces 150Fixed: The failed assertions now use [] as delimiters instead of <> (better for the HTML reports) 151Fixed: GITHUB-237: Wrong time format in XML reporter 152Fixed: Threads were started sequentially instead of being interleaved 153Fixed: dataProvider(parallel = true) was not killing its threads properly 154Fixed: XmlSuite#toXml wasn't outputting the <groups> tag correctly 155Fixed: testng-failed.xml was not carrying over the parameters from the original testng.xml 156Fixed: BeforeClass failing in parent failed to skip methods in sub classes 157Fixed: Better error message if <suite name=""> is missing 158Fixed: GITHUB-221: Honor excludeGroups on testng tests when run in mixed mode (criccio) 159Fixed: dependsOnGroups = {regexp} wasn't working properly (Alistair Ward) 160Fixed: GITHUB-205: white-space was spelled whitespace in testng.css (carlin-scott) 161 162Eclipse: 163Fixed: Environment is not transferred when rerunning failed tests. 164Fixed: Rerunning failed tests will preserve the environment of the original launch 165 166=========================================================================== 167 1686.5.1 16904/10/2012 170 171Added: <suite allow-return-values="true"> (and in <test> as well) 172Added: data-provider attribute to testng-results.xml 173Added: Reporter display the results in the same order as test methods (Libor Zoubek) 174Added: Support for running JUnit 4 tests (Lukas Jungmann) 175Added: Ability to auto-detect JUnit tests ('-mixed' mode) (Lukas Jungmann) 176Added: Support for ResourceCollections in an Ant tasks (requires Ant >= 1.7.0) (Lukas Jungmann) 177Fixed: GITHUB-198: JUnitReportsReporter use commas in certain locales, which JUnitReports doesn't like 178Fixed: GITHUB-173: Dependent methods executed out-of-order if method names match across classes (jjedMoriAnktah) 179Fixed: ThreadLocal<ITestResult> leak (aslakknutsen) 180Fixed: In the HTML reports, only show the first 100 characters of the parameters 181Fixed: SkippedException are considered as real exception with @Test(expectedExceptions) 182 183Eclipse: 184Fixed: Java constants are properly resolved if they are used as group names (susanin) 185Fixed: @Test(groups = Foo.CONSTANT) (susanin) 186Fixed: Failed tests with allow-return-values="true" were not rerun 187Added: <suite allow-return-values="true"> (and in <test> as well) 188 189=========================================================================== 190 1916.4: 19202/15/2012 193 194Added: @DataProvider(indices) to return specific indices of a data provider 195Added: New HTML reports 196Added: configfailurepolicy=continue with DataProviders (toddq) 197Added: ITestResult#getTestContext (bpedman) 198Fixed: invocationCount > 1 + timeOut wasn't timing out properly 199Fixed: When running TestNG programmatically, child xml suites are not run (when added using setSuiteFIles()) (Gaurav Gupta) 200Fixed: GITHUB-145: Excessive test method execution (githubCast) 201Fixed: GITHUB-149: reversed arguments in failAssertEqualsNoOrder(). 202Fixed: EmailableReporter: methods are now *really* sorted chronologically. 203 204Eclipse: 205 206Added: You can now add the testng.jar sources as a library (Nick Tan) 207Added: Upgraded the plug-in to 3.4+ (Nick Tan) 208Added: dependsOnGroups now fully supported 209Fixed: @Parameters now works with both ("foo") and ({"foo"}) (davekerber) 210 211=========================================================================== 2126.3.1 21310/22/2011 214 215Added: New system property: dataproviderthreadcount (Bill Ross) 216Fixed: Configuration methods were reported incorrectly in listeners. 217Fixed: Was creating too many listeners (Jacek Pulut) 218Fixed: IAnnotationTransformer2 beforeTest/afterTest booleans were not being set 219Fixed: GITHUB-92: @BeforeTest method in a super class will be called multiple time when alwayRun = true (Bubuntux) 220Fixed: GITHUB-111: @AfterClass on base classes run once too many (lrivera) 221Fixed: GITHUB-107: Displaying 0 tests run if a listener modifies the parameters of the suite 222 223=========================================================================== 2246.3 22510/17/2011 226 227Added: "description" attribute on <include>, made available on ITestNGMethod#getDescription 228Added: RemoteTestNG waits infinitely for a connection (Aleksey Kabanov) 229Fixed: A method that's both a test and a factory would not invoke its data provider 230Fixed: @AfterClass was not called if one of the methods was not enabled (Aleksey Kabanov) 231Fixed: Groovy access bug 232Fixed: The XML parser doesn't recognize parallel="instances" 233Fixed: NPE when using inner classes 234Fixed: GITHUB-90: @AfterClass not being run when the class contains included and not included methods 235Fixed: @AfterClass not being run in some subclassing situations 236 237Eclipse: 238 239Fixed: Verbose levels specified in suites not respected 240Fixed: Variable substitution in VM arguments is not working properly (svenhoff) 241 242=========================================================================== 243 2446.2 24508/18/2011 246 247Added: xmlpathinjar to the TestNG ant task 248Added: TestNG can now invoke package protected constructors 249Added: Injectors created by the @Guice annotation are now shared at the <test> level 250Added: IConfigurationListener is now a public listener, along with a new one: IConfigurationListener2 251Added: When a method fails, only dependents of the same instance will be skipped 252Added: parallel=instances for factory instance parallel runs 253Added: @Factory(enabled) 254Fixed: JUnitReports reports now report the cumulated time @{Before,After}Method+@Test for each test method 255Fixed: JUnitReports reports have the name of the <test> instead of that of the first class 256Fixed: Using preserve-order with a factory that creates instances of a different class causes NPE 257Fixed: GITHUB-74: Bad ordering of test methods when using a @Factory constructor with dataProvider 258Fixed: Changing the test result from success to failure in a listener would still count the test as a success 259Fixed: ServiceLoader wasn't resolving correctly if no service loader classloader was specified 260Fixed: Better ordering with mixed priorities and dependencies 261Fixed: Improved detection of graph cycles in parallel runs 262Fixed: @BeforeTest was invoked multiple times if a factory is used 263Fixed: GITHUB-57: Allow usage of package protected constructor of test classes 264Fixed: Injecting both Object[] and Method in @BeforeMethod didn't always work 265Fixed: testng-results.xml now lists the results chronologically 266Fixed: @Listeners specified on a base class will only be run once per listener class (dbriones) 267Fixed: -groups and -excludegroups were no longer overriding testng.xml 268 269Eclipse: 270 271Added: Each data provider method now has a separate node entry in the TestNG view 272Fixed: Nodes in error would sometimes remain green 273Fixed: The TestNG context menu no longer appears where it shouldn't 274 275=========================================================================== 276 2776.1.1 2787/5/2011 279 280Fixed: https://github.com/cbeust/testng/issues/56 testng-results.xml was reporting the instance name instead of the method name 281Fixed: NPE when using preserve-order and factories. 282Fixed: Depending on a skipped method would not cause a method to be skipped 283 284=========================================================================== 285 2866.1 2876/30/2011 288 289Possible backward incompatible changes: 290 291- Don't mutate the value returned by XmlTest#getIncludedGroups and XmlTest#getExcludedGroups. 292Instead, use addIncludedGroup/addExcludedGroup. 293- Failing methods that have dependees will only cause skips in the same instance. Different 294test instances will not be affected 295 296Added: Support for ServiceLoader for ITestNGListener 297Added: @Factory(dataProvider / dataProviderClass) on constructors 298Added: assertNotEquals() to Assert 299Added: assertArrayEquals() to AssertJUnit 300Added: Nested classes are now automatically added for consideration for inclusion 301Added: <suite preserve-order="true"> will cause this attribute to be propagated to all <test> tags 302Added: <groups> can now be specified under a <suite> 303Added: Tycho compatibility (Aleksander Pohl) 304Added: New <test> and <suite> flag: group-by-instances 305Added: -xmlpathinjar to specify the path of testng.xml inside a test jar file 306Added: ISuite#getAllMethods, to retrieve all the methods at the start of a suite. 307Added: Output ITestResult attributes in xml report (nguillaumin) 308Fixed: Thread safety problem in MethodInvocationHelper (Baron Roberts) 309Fixed: Group dependencies were not being skipped properly. 310Fixed: Dependency failures only impact the same instance 311Fixed: Static classes could cause a StackOverFlowError 312Fixed: IConfigurationListener was not extending ITestNGListener 313Fixed: IConfigurationListener#onConfigurationFailure was never called 314Fixed: TESTNG-476: <test> tags are now run in the order found in testng.xml 315Fixed: Now showing failed/skipped error messages on the console for verbose >= 2 316Fixed: ITestResult#getEndMillis() return 0 317Fixed: TESTNG-410: Clearer error message 318Fixed: TESTNG-475: @DataProvider doesn't support varargs 319Fixed: Performance problems in EmailableReporter 320Fixed: TESTNG-472: Better output for assertNull() 321Fixed: ConcurrentModificationException when using parallel data providers. 322Fixed: TESTNG-282: Problem when including+excluding packages (addicted) 323Fixed: TESTNG-471: assertEquals(Map, Map) fails if a map is a subset of the other 324Fixed: JUnitReporter generates an <error> tag for successful expectedExceptions tests 325Fixed: ISSUE-47: Don't allow two <test>s with same name within same suite (Nalin Makar) 326Fixed: If a listener implements both ISuiteListener and IInvokedMethodListener, only one of them gets invoked 327 328Eclipse: 329 330Added: New quick fix "Add static import org.testng.AssertJUnit.assertXXX" 331Added: New workspace wide setting: excluded stack traces, to provide shorter stack traces in the view 332Added: New "Clear results" icon in the tool bar 333Added: When the search filter is modified, don't update the tree live if it is too big 334Added: Two new @Test refactorings (pull to class level, push to method level) 335Added: JUnit conversion: @Ignore 336Added: JUnit conversion: assertArrayEquals() 337Added: JUnit conversion: @RunWith(Parameterized.class) 338Added: Support for Hamcrest failed assertions in the compare dialog 339Added: JUnit conversion: suite() methods can now either be removed, commented out or left untouched 340Fixed: JUnit conversion: super.setUp()/tearDown() were being removed when extending a class other than TestCase 341Fixed: "Run as" menu not appearing for methods that take a generic parameter. 342Fixed: The tree was incorrect if the same class is used in different <test> tags 343Fixed: When creating a new Run/Debug configuration, "Launch.label" was displayed 344Fixed: TESTNG-459: TestNG menu should not always be present in context menu (Mykola Nikishov) 345Fixed: Performance problems in the plug-in 346Fixed: Workspace-wide XML template files are not being honored. 347Fixed: @BeforeClass/@AfterClass from JUnit4 are not being properly converted 348Fixed: Conversions generate @Test() instead of @Test 349 350=========================================================================== 351 3526.0 3532011/03/16 354 355Added: @Guice(moduleFactory) and IModuleFactory 356Added: @Guice(module) 357Added: timeOut for configuration methods 358Added: -randomizesuites (Nalin Makar) 359Added: IConfigurable 360Fixed: @Test(priority) was not being honored in parallel mode 361Fixed: @Test(timeOut) was causing threadPoolSize to be ignored 362Fixed: TESTNG-468: Listeners defined in suite XML file are ignored (Michael Benz) 363Fixed: TESTNG-465: Guice modules are bound individually to an injector meaning that multiple modules can't be effectively used (Danny Thomas) 364Fixed: Method selectors from suites were not properly initialized (toddq) 365Fixed: Throw an error when two data providers have the same name 366Fixed: Better handling of classes that don't have any TestNG annotations 367Fixed: XmlTest#toXml wasn't displaying the thread-count attribute 368Fixed: TESTNG-438: Regression in 5.14.1: JUnit Test Execution no longer working 369Fixed: TESTNG-436: Deep Map comparison for assertEquals() (Nikolay Metchev) 370Fixed: Skipped tests were not always counted. 371Fixed: test listeners that throw were not reporting correctly (ansgarkonermann) 372Fixed: <suite junit="true"> wasn't working. 373Fixed: In parallel "methods" mode, method interceptors that remove methods would cause a lock up 374Fixed: EmailableReporter now sorts methods chronologically 375Fixed: TESTNG-411: Throw exception on mismatch of parameter values (via DP and/or Inject) and test parameters 376Fixed: IDEA-59073: exceptions that don't match don't have stack trace printed in console (Anna Kozlova) 377Fixed: IDEA's plug-in was not honoring ITest (fixed in TestResultMessage) 378Fixed: Methods depending on a group they belong were skipped instead of throwing a cycle exception 379Fixed: TESTNG-401: ClassCastException when using a listener from Maven 380Fixed: TESTNG-186: Rename IWorkerApadter to IWorkerAdapter (Tomas Pollak) 381Fixed: TESTNG-415: Assert.assertEquals() for sets and maps fails with 'null' as arguments 382Fixed: typo -testRunFactory 383Fixed: NPE while printing results for an empty suite (Nalin Makar) 384Fixed: Invoke IInvokedMethodListener.afterInvocation after fixing results for tests expecting exceptions (Nalin Makar) 385Fixed: TESTNG-441: NPE in SuiteHTMLReporter#generateMethodsChronologically caused by a race condition (Slawomir Ginter) 386 387Eclipse: 388Added: Convert to YAML 389Added: New global preference: JVM args 390Added: Eclipse can now monitor a test-output/ directory and update the view when a new result is created 391Added: Right clicking on a class/package/project now offers a menu "TestNG/Convert to TestNG" 392Added: Excluded methods are now listed in the Summary tab 393Added: "Description" column in the excluded methods table 394Added: Dialog box when the plug-in can't contact RemoteTestNG 395Added: Double clicking on an excluded method in the Summary tab will take you to its definition 396Added: If you select a package before invoking the "New TestNG class" wizard, the source and package text boxes will be auto-filled 397Added: When an item is selected in a tab, the same item will be selected when switching tabs 398Added: A new "Summary" tab that allows the user to see a summary of the tests, sort them by time, name, etc... 399Added: It's now possible "Run/Debug As" with a right click from pretty much any element that makes sense in the tree. 400Added: JUnit conversion: correctly replaces assertNull and assertNotNull 401Added: JUnit conversion: removes super.setUp() and super.tearDown() 402Added: JUnit conversion: removes @Override 403Added: JUnit conversion: replaces @Test(timeout) with @Test(timeOut) (5.14.2.4) 404Added: JUnit conversion: replaces @Test(expected) with @Test(expectedExceptions) (5.14.2.4) 405Added: JUnit conversion: replaces fail() with AssertJUnit.fail() (5.14.2.2) 406Added: JUnit conversion: replaces Assert with AssertJUnit (5.14.2.1) 407Added: The progress bar is now orange if the suite contained skipped tests and no failures 408Added: Skipped test and suite icons are now orange (previously: blue) 409Added: New method shortcuts: "Alt+Shift+X N", "Alt+Shift+D N" (Sven Johansson) 410Added: "Create TestNG class" context menu 411Added: When generating a new class, handle overridden methods by generating mangled test method names 412Fixed: Green nodes could override red parent nodes back to green 413Fixed: Was trying to load the classes found in the XML template file 414Fixed: Stack traces of skipped tests were not showing in the Exception view 415Fixed: XML files should be run in place and not copied. 416Fixed: NPE when you select a passed test and click on the Compare Result icon (Mohamed Mansour) 417Fixed: When the run is over, the plug-in will no longer force the focus back to the Console view 418Fixed: The counter in the progress bar sometimes went over the total number of test methods (5.14.2.9) 419Fixed: org.eclipse.ui.internal.ErrorViewPart cannot be cast to org.testng.eclipse.ui.TestRunnerViewPart (5.14.2.9) 420Fixed: Workspace preferences now offer the "XML template" option as well as the project specific preferences (Asiel Brumfield) 421Fixed: TESTNG-418: Only last suite-file in testng.xml run by Eclipse plugin 422 423Documentation: 424Added: Section on Selenium (Felipe Knorr Kuhn) 425Added: Link to an article on TestNG, Mockito and Emma in the Misc section 426 427=========================================================================== 428 4295.14.7 4302011/01/27 431 432Release for IDEA 433 434=========================================================================== 435 4365.14.1 4372010/10/2 438 439Fixed: TESTNG-401: ClassCastException when using a listener from Maven 440 441=========================================================================== 442 4435.14 4442010/08/28 445 446Added: test suites can now be run in parallel with -suitethreadpoolsize 447Fixed: @Listeners now aggregate through base classes 448Fixed: ISuite was no longer serializable 449Fixed: Injection was sometimes not working properly when used with @Parameters 450Fixed: TESTNG-400: afterMethod was called after onTestFailure() 451Fixed: "excludedgroups" was not working on the ant task because of a typo 452Fixed: ant task error if <classfileset> is used with no classes (welex91) 453Fixed: TESTNG-404: threaded tests fail due to use of non-threadsafe collections (Marcus Better) 454Fixed: preserve-order was not preserving class order with dependent methods 455Fixed: RetryAnalyzer wasn't working properly with factories 456Fixed: The ant task was no longer supporting ',' for testclass 457 458Eclipse: 459 460Fixed: The plug-in wasn't running Groovy tests correctly (Andrew Eisenberg) 461Fixed: TESTNG-402 [Eclipse Plug-In] NPE occurred when I run twice a custom "Run configuration" on a group 462 463=========================================================================== 464 4655.13.1 4662010/08/05 467 468Added: -methods 469Added: -configfailurepolicy (Todd Quessenberry) 470Added: -methodselectors (Todd Quessenberry) 471Added: @NoInjection 472Added: <test preserve-order="true"> 473Added: -testnames (command line) and testnames (ant) 474Added: New ant task tag: propertyset (Todd Wells) 475Added: ITestNGListenerFactory 476Added: Passing command line properties via the ant task and doc update (Todd Wells) 477Added: Hierarchical XmlSuites (Nalin Makar) 478Added: Reporter#clear() 479Fixed: NullPointerException when a suite produces no results (Cefn Hoile) 480Fixed: Identical configuration methods were not always invoked in the correct order in superclasses (Nalin Makar) 481Fixed: @DataProvider(parallel = true) was passing incorrect parameters with injection 482Fixed: Replaced @Test(sequential) with @Test(singleThreaded) 483Fixed: If inherited configuration methods had defined deps, they could be invoked in incorrect order (Nalin Makar) 484Fixed: Initialize all Suite/Test runners at beginning to catch configuration issues right at start (Nalin Makar) 485Fixed: Issue7: Issue86 Incorrect dates reported for configuration methods 486Fixed: Issue24: OOM errors in SuiteHTMLReporter (Nalin Makar) 487Fixed: Time outs specified in XML were not honored for <suite parallel="tests"> 488Fixed: <suite> and <test> time outs were hardcoded, they now honor their time-out attribute 489Fixed: TestNG was hanging if no test methods were found 490Fixed: onTestSuccess() was called after @AfterMethod instead of after the test method (test: test.listener.ListenerTest) 491Fixed: XML test results contained skipfailedinvocationCounts instead of skipfailedinvocationcounts 492Fixed: Issue4 assertEquals for primitive arrays, Issue34 assertNull javadoc updated 493Fixed: Issue78 NPE with non-public class. Now throws TestNG exception 494Fixed: NPE with @Optional null parameters (Yves Dessertine) 495Fixed: TESTNG-387 TestNG not rerunning test method with the right data set from Data Provider (Francois Reynaud) 496Fixed: Show correct number of pass/failed numbers for tests using @DataProvider 497Fixed: Return correct method status and exception (if any) in InvokedMethodListener.afterInvocation() 498Fixed: Trivial fixes: TESTNG-241 (log message at Info), Issue2 (throw SAXException and not NPE for invalid testng xml) 499Fixed: Configuration methods couldn't depend on an abstract method (Nalin Makar) 500Fixed: TestNG#setTestClasses was not resetting m_suites 501Fixed: Exceptions thrown by IInvokedMethodListeners were not caught (Nalin Makar) 502Fixed: @Listeners now works on base classes as well 503Fixed: Test priorities were not working properly in non-parallel mode 504Fixed: @Listeners wasn't working properly with ITestListener 505 506Eclipse 507 508Fixed: TESTNG-395 New wizard was creating classes called "NewTest" 509Fixed: TESTNG-397 Class level @Test was preventing groups from showing up in the launch configuration 510 511Doc 512Updated Maven documentation (Brett Porter) 513 514=========================================================================== 515 5165.12.1 5172010/03/29 518 519Maven update 520 521=========================================================================== 5225.12 523 524Removed: Javadoc annotation support 525 526Added: @Listeners 527Added: IAttributes#getAttributeNames and IAttributes#removeAttribute 528Added: testng-results.xml now includes test duration in the <suite> tag (Cosmin Marginean) 529Added: Injection now works for data providers 530Added: TestNG#setObjectFactory(IObjectFactory) 531Added: Priorities: @Test(priority = -1) 532Added: New attribute invocation-numbers in <include> 533Added: testng-failed.xml only contains the data provider invocations that failed 534Added: IInvokedMethodListener2 to have access to ITestContext in listeners (Karthik Krishnan) 535Fixed: @Before methods run from factories were not properly interleaved 536Fixed: The TextReporter reports skipped tests as PASSED (Ankur Agrawal) 537 538Eclipse: 539 540Added: New file wizard: can now create a class with annotations, including @DataProvider 541Added: You can now select multiple XML suites to be run in the launch dialog 542Fixed: @Test(groups = <constant>) was taking name of the constant instead of its value. 543Fixed: http://jira.codehaus.org/browse/GRECLIPSE-476 NPE with Groovy Tests (Andrew Eisenberg) 544Fixed: The custom XML file is now created in the temp directory instead of inside the project 545Fixed: In the launch dialog, now display an error if trying to pick groups when no project is selected 546Fixed: Was not setting the parallel attribute correctly on the temporary XML file 547 548=========================================================================== 5495.11 5502009/12/08 551 552Added: Dependent methods can now run in their own thread 553Added: dataProviderThreadCount can be set from the command line and from ant (Adrian Grealish) 554Added: ITestAnnotation#setDataProvider 555Added: Assert#assertEquals() methods for Sets and Maps 556Fixed: The text reporter was no longer reporting stack traces for verbose >= 2 557Fixed: dataProviderClass was not respecting inheritance (like most attributes still) 558Fixed: @BeforeSuite/@AfterSuite would run multiple times when used in a @Factory 559Fixed: packages=".*" wasn't working properly (sandopolus) 560Fixed: TestResult#getName now returns the description instead of the method 561Fixed: @DataProvider and dependent methods were not skipping correctly (Francois Reynaud) 562Fixed: TESTNG-347 suite with parallel="tests" and test with parallel="classes" doesn't work correctly (Rob Allen) 563Fixed: TESTNG-67: @Configuration/@Factory methods in base class being ignored 564Fixed: Inner test classes were not excluded properly (Carsten Gubernator) 565Fixed: threadPoolSize without invocationCount was causing reporters not to be invoked 566Fixed: A @Factory throwing an exception did not cause any error 567Fixed: <classfilesetref> was not working properly in the ant task (Ed Randall) 568Fixed: @BeforeClass methods were not running in parallel (Aidan Short) 569Fixed: Test class with @ObjectFactory doesn't get instantiated via the factory 570Fixed: Allow IObjectFactory to load from non-standard classloader (for PowerMock support) 571 572Eclipse 5.11.0.19: 573Added: New "parallel" preference setting (Windows / Preferences / TestNG) 574Fixed: IIinvokedMethodListeners were not invoked 575 576=========================================================================== 5775.10 578 579Added: The output in the testng-results.xml is now sorted by the starting timestamp (Daniel Rudman) 580Added: Better display of the test name and method description in the default and Emailable report 581Added: If both -testjar and an XML file are provided on the command line, the latter will be used 582Added: @Before and @After methods can be injected with the current XmlTest 583Added: Methods that time out now display the stack trace showing where the time out occurred 584Added: ITestResult#getAttribute and ITestResult#setAttribute 585Added: @After methods can now be injected with an ITestResult 586Added: @BeforeMethod and @AfterMethod methods can now be injected an ITestResult 587Added: ISuite#getAttribute and ISuite#setAttribute to share data within a suite 588Added: @Test(expectedExceptionsMessageRegExp = ".*foo.*") 589Added: @DataProvider(parallel=true) 590Fixed: @Test(dataProvider) was not working at the class level 591Fixed: Display a better error message if the wrong exception is thrown with an expectedExceptions 592Fixed: Classes created by factories were not run in the order they were created 593Fixed: Dependent methods are now run closer to methods within their class 594Fixed: xmlFileSet in ant was not working correctly (Sean Shou) 595Fixed: Various oversights in the DTD (Will McQueen) 596Fixed: XMLUtils was not escaping XML attribute values 597Fixed: TESTNG-317: Sequence order mis-calculation: testing using suite in sequence for classes and same method names creates non-sequential order 598Fixed: Test names (classes that implement org.testng.ITest) now appear more prominently in the HTML reports 599Fixed: expectedExceptions=RuntimeException.class was not failing when no exception was throw 600Fixed: TESTNG-291: Exceptions thrown by Iterable DataProviders are not caught, no failed test reported (Roberto Tyley) 601Fixed: TESTNG-301: Need to include parameters in testNG report for test created by @Factory 602Fixed: testng-failed.xml now includes skipped tests 603Fixed: TestNG couldn't find Groovy files (Haw-Bin) 604 605Eclipse 606 607Fixed: TESTNG-313: Provide extension point to contribute test and report listeners (Erik Putrycz) 608Fixed: Quick fixes no longer introduce deprecated annotations (Greg Turnquist) 609 610=========================================================================== 6115.9 6122009/04/09 613 614Added: New ant task boolean flag: delegateCommandSystemProperties (Justin) 615Added: skipfailedinvocations under <suite> in testng-1.0.dtd (Gael Marziou / Stevo Slavic) 616Added: -testrunfactory on the command line and in the ant task (Vitalyi Pamajonkov) 617Added: TESTNG-298: parallel="classes", which allows entire classes to be run in the same thread 618Added: @BeforeMethod can now declare Object[] as a parameter, which will be filled by the parameters of the test method 619Added: IAnnotationTransformer2 620Added: @Test(invocationTimeOut), which lets you set a time out for the total time taken by invocationCount 621Added: IInvokedMethodListener 622Added: -testjar supports jar file with no testng.xml file 623Fixed: IInvokedMethodListener wasn't properly recognized from the command line (Leonardo Rafaeli) 624Fixed: TESTNG-309 Illegal default value for attribute in DTD file 625Fixed: TESTNG-192: JUnit XML output includes wrong tests (Aleksandar Borojevic) 626Fixed: Set a generated suite to default to non-parallel (Mark Derricutt) 627Fixed: -testJar command line parsing bug 628Fixed: testng-failed.xml didn't include the listeners 629Fixed: annotation transformers were not run when specified in testng.xml 630Fixed: TESTNG-192: JUnit XML output includes wrong tests (Borojevic) 631Fixed: @Parameters was not working correctly on @BeforeMethods with @DataProvider used on @Test methods 632Fixed: testng-failed.xml was sometimes incorrectly generated (Borojevic) 633Fixed: TestNG-228: Assert.assertEqualsNoOrder 634Fixed: TestNG-229: Assert.assertEquals does not behave properly when arguments are sets 635Fixed: TESTNG-36: assertEquals(Collection actual, Collection expected, String message) may have bug 636Fixed: TESTNG-296: Malformed jar URLs breaking -testJar 637Fixed: TESTNG-297: TestNG seemingly never stops running while building failed test suite (Gregg Yost) 638Fixed: TESTNG-285: @Test(sequential=true) works incorrectly for classes with inheritance 639Fixed: TESTNG-254: XMLSuite toXML() ignores listeners 640Fixed: TESTNG-276: Thread safety problem in Reporter class 641Fixed: TESTNG-277: Make Reporter.getCurrentTestResult() public 642Fixed: Potential NPE in XmlTest#getVerbose (Ryan Morgan) 643Fixed: EmailableReporter only displayed the first group for each test method 644Fixed: time-outs were not working in <test> and <suite> 645Fixed: @BeforeTest failing in a base class would not cause subsequent test methods to be skipped 646Fixed: TESTNG-195: @AfterMethod has no way of knowing if the current test failed 647Fixed: TESTNG-249: Overridden test methods were shadowing each other if specified with <include> 648Fixed: DataProviders from @Factory-created tests were all invoked from the same instance 649Fixed: enabled was not working on configuration methods 650Fixed: IIinvokedMethodListener was not correctly added in TestNG 651Fixed: NPE in XmlSuite#toXml 652Fixed: TESTNG-231: NullPointerException thrown converting a suite to XML (Mark) 653 654Doc: 655Added: 5.20: IInvokedMethodListener 656Added: -testjar 657 658=========================================================================== 6595.8 660 661Fixed: TestNG-220: Ignore class definition/loader issues when scanning classpath for implicit classes 662Fixed: TestNG-224: Fix for relative suite filenames in XML file 663Added: TestNG-213: @Optional on a method parameter to allow optional @Parameters 664Fixed: TestNG-214: SkipException and TimeBombSkipException should accept nested exceptions 665Fixed: TestNG-211: new Parser(inputStream) doesn't work 666Added: Methods that form a cycle are now shown when the cycle is detected 667Added: Support for <listeners> in testng.xml 668Added: IMethodInterceptor 669Added: @TestInstance on a data provider method parameter 670Fixed: @AfterMethod(lastTimeOnly) didn't work properly with data providers 671Added: antlib.xml to allow autodiscovery of Ant task definition 672Fixed: name attribute on <test> is required 673 674Doc: 675Added: Method Interceptor 676Added: @Optional 677Added: Doc for IMethodInterceptor (5.16) and TestNG listeners (5.18) 678Added: 5.19: Dependency injection 679 680=========================================================================== 6815.7 682 683Added: @BeforeMethod(firstTimeOnly) and @AfterMethod(lastTimeOnly) 684Added: @BeforeMethods can now take a Method and ITestContext parameters (like @DataProvider) 685Fixed: logging about abstract classes moved to level 5 686Added: if @Parameter is missing from testng.xml then it is read from the System properties 687Fixed: Don't run a @DataProvider method as a test when a class-level @Test is present 688Added: Attribute @Test#skipFailedInvocations 689Fixed: TESTNG-169 Error message: <method> is depending on nonexistent method null ("null" is uninformative) 690Fixed: -listener takes comma-separated classes 691Added: RetryAnalyzer (experimental) (Jeremie) 692 693=========================================================================== 6945.6 6952007/06/14 696 697Added: SkipException/TimeBombedSkipException for manual skipping 698Added: <tests> can now be disabled at xml level using <test enabled="false"> 699Added: Suite files that only contain other suites do not get reported 700Fixed: @BeforeClass methods would incorrectly report cyclic graphs 701Added: get/setAttribute to ITestContext 702Added: plugging in factory objects to handle the actual instantiation of tests 703Added: dataProvider to @Factory 704Added: ISuite now gives access to the current XmlSuite 705Fixed: TESTNG-139 dependsOnMethods gets confused when dependency is "protected" 706Fixed: TESTNG-141 junit attribute set to false in testng-failed.xml when it should be true 707Fixed: TESTNG-142 Exceptions in DataProvider are not reported as failed test 708Added: Improved behavior for @Before/@AfterClass when using @Factory 709(http://forums.opensymphony.com/thread.jspa?threadID=6594&messageID=122294#122294) 710Added: Support for concurrent execution for invocationCount=1 threadPoolSize>1 and @DataProvider 711(http://forums.opensymphony.com/thread.jspa?threadID=64738&tstart=0) 712Added: New TestNG specific XML report, generated by default in 'xml' subdirectory of test-output 713Added: support in strprotocol for passing the ITest.getTestName() information 714Fixed: TESTNG-152 If DataProvider is not found, the exception message should tell exactly what happened 715 716Eclipse plug-in 717 718Fixed: Bug that made group launch configurations unusable 719Fixed: The plugin doesn't create the correct launch configuration for @Factory 720Fixed: Method based launchers cannot be editted 721Fixed: Plugin hangs while executing test with dataprovider that sends \n, \r messages 722Added: display ITest.getTestName() 723 724IDEA plug-in 725 726Fixed: IDEA 7.0 compatibility 727Fixed: occasional 'illegal arguments exception' 728Fixed: TESTNG-151 Final passing test result is not properly hidden 729Added: Auto-completion for dependsOnMethods 730Added: Highlighting of invalid groups/methods in dependsOn* 731 732=========================================================================== 7335.5 7342007/01/25 735 736Fixed: @BeforeGroup methods were run twice when in a base class 737Fixed: @BeforeGroup methods were run twice with a @Test at class level 738Fixed: parallel="tests" didn't work as advertised 739Added: Support for thread-count at test level 740Added: Method selectors receive a Context and can stop the chain with setStopped() 741Fixed: XmlMethodSelector was always run first regardless of its priority 742Added: @BeforeGroups/@AfterGroups can live in classes without @Test methods 743Added: DataProvider can now take an ITestContext parameter 744Fixed: Wasn't parsing <selector-class-name> correctly 745Fixed: Annotation Transformers now work on class-level annotations 746Fixed: Some class-level @Test attributes were not always honored 747Added: Clean separation between @Test invocation events and @Configuration invocation events 748 (see also TESTNG-111) 749Added: Test instances created by @Factory now run in multiple threads in parallel mode 750Fixed: @Before/@AfterGroups invocation order 751Fixed: TESTNG-27: Parameters are not used on <test> level anymore 752Fixed: TESTNG-107 don't create an output directory if "outputDirectory" is null 753Fixed: TESTNG-127 UseDefaultListeners in Ant Task does not work 754Fixed: TESTNG-119 Running TestNG runner with invalid '-sourcedir' on JDK14 JavaDoc annotated test classes won't fail. 755Fixed: TESTNG-113 Dependent methods within the same static inner class are not found 756Fixed: TESTNG-125 TestNG failed for test classes under *.java*.* pakages 757 758Eclipse plug-in 759Fixed: issue with launch configuration 760Fixed: TESTNG-124: setting location of testng reports output 761 762=========================================================================== 7635.4 764 765Fixed: Ant task issue with paths containing spaces 766Added: for @BeforeGroups and @AfterGroups specifying the groups() attribute will auto-include the method 767 into those groups by default (previously you had to also provide the value() attribute). 768Added: the load @Tests (invocationCount + threadPoolSize) are triggered simultaneous 769Fixed: reports are correctly displaying the thread info 770Added: @DataProvider name defaults to method name 771Added: support for remote protocol to pass parameter information 772Fixed: TextReporter logs information about the parameters of the test methods 773Fixed: concurrency issue in JUnitXMLReporter 774Fixed: output of JUnitXMLReporter must be CDATA 775Fixed: XML unsupported annotations/parallel attribute values are reported 776 777Eclipse plug-in 778Fixed: groups with multi-attribute javadoc annotations 779Fixed: consistent behavior for dependsOnMethods 780Fixed: consistent behavior for tests with dependsOnGroups (a warning is emitted) 781Fixed: consistent merge of configuration arguments when an existing launch configuration exists 782=========================================================================== 7835.3 7842006/10/30 785 786Fixed: use a single instance of bsh.Interpreter 787Added: @Before/@AfterMethod can declare a java.lang.reflect.Method parameter to be informed about the @Test method 788Fixed: super classes must not be listed in testng-failures.xml 789Fixed: parallel attribute must not appear if empty or null in testng-failures.xml 790Fixed: parsing for javadoc annotations is done on request only 791Added: improved multiple suite summary page report 792Added: -target option deprecated in favor of -annotations javadoc|jdk 793Fixed: filesets in the ant task didn't work if the paths have spaces in them 794Fixed: Before/After Suite were behaving wrong in parallel execution 795Added: A generic/extensible RemoteTestNG was added to the core 796Fixed: Before/AfterGroup-s were behaving wrong when using invocationCount, dataProvider and threadPoolSize 797Fixed: improved support for running different annotation type tests in the same suite 798Fixed: testng-failed.xml was generated even if there were no failures/skipps 799Fixed: -usedefaultlisteners was wrongly passed to JVM instead of TestNG options 800Added: Attribute dataProviderClass for @Test and @testng.test 801Fixed: Forgot to account for cases where both invocationCount and DataProviders are present 802Fixed: AfterGroups were invoked out of order with invocationCount and DataProviders 803Fixed: Reporter.getOutput() returned an empty array if a timeOut was specified 804Added: testng.xml now supports <suite-files> 805Added: ant task can receive several listeners 806Fixed: TESTNG-109 Skipped tests with expected exceptions are reported as failures 807Added: ant task can now select the parallel mode for running tests 808Fixed: ant task correctly deals with empty groups and excludedgroups parameters 809Added: ant task can override default suite and test names 810Added: comand line support for setting parallel mode, suite and test names 811 812Eclipse plug-in 813Added: Support for configuring per project usedefaultlisteners 814Added: Contextual drop-down menu on failures tab of the TestNG view to enable running/debugging method failure only 815Added: Suppport for configuring per project TestNG jar usage (project provided one or plugin provided one) 816 817=========================================================================== 8185.2 819 820Added: "-usedefaultlisteners true/false" to command line and ant 821Added: EmailableReporter (from Paul Mendelson) 822Added: parallel can now be "methods" or "tests". Boolean version deprecated 823Added: TestNGAntTask now uses the @ syntax to invoke TestNG 824Added: Command line understands @ syntax 825Added: JUnitConverter uses the new syntax 826Added: -groups to JUnitConverter 827Fixed: Throw proper exception when a DataProvider declares parameters 828Added: completely revamped JUnit support (should run all kind of JUnit tests) 829Fixed: TESTNG-40 (Bug in testng-failed.xml generation) 830Fixed: TESTNG-106 (Failed "@BeforeSuite" method just skipps the last test in xml-file) 831Fixed: Success on 0 tests (http://forums.opensymphony.com/thread.jspa?threadID=41213) 832 833Eclipse plug-in 834Added: TESTNG-105 Automaticaly define TESTNG_HOME classpath variable 835 836=========================================================================== 8375.1 8382006/08/18 839 840Added: @Test(sequential = true) 841Fixed: TESTNG-102 (Incorrect ordering of @BeforeMethod calls when a dependency is specified) 842Fixed: TESTNG-101 (HTML output contains nested <P> tags and a missing <tr> tag) 843Added: support for specifying test-only classpath (http://forums.opensymphony.com/thread.jspa?messageID=78048&tstart=0) 844Fixed: TESTNG-93 (method selectors filtering @BeforeMethod) 845Fixed: TESTNG-81 (Assert.assertFalse() displays wrong expected, actual value) 846Fixed: TESTNG-59 (multiple method selectors usage results in no tests run) 847Fixed: TESTNG-56 (invocation of @Before/AfterClass methods in parallel/sequential scenarios) 848Fixed: TESTNG-40 (failures suite does not contain @Before/After Suite/Test methods) 849Fixed: TESTNG-37 (allow passing null parameter value from testng.xml) 850Fixed: TESTNG-7 (display classname when hovering method) 851 852 853Eclipse plug-in 854 855Added: run contextual test classes with parameters from suite definition files 856Added: TESTNG-100 (Show HTML reports after running tests) 857Added: TESTNG-97 (Double click top stack to raise comparison) 858Added: TESTNG-84 (plug-in UI for suite option should support absolute path) 859Added: TESTNG-20 (copy stack trace) 860 861Fixed: TESTNG-72 (display groups with non-array values) 862Fixed: TESTNG-64 (Eclipse plug-in applies added groups to all launch configurations) 863Fixed: TESTNG-28 (Cannot select groups from dependent eclipse projects) 864Fixed: TESTNG-25 (do not display fully qualified method name when running contextual test class) 865 866Improved behavior: 867 TESTNG-98 (temporary files have guaranteed fixed names) 868 TESTNG-95 (Assertion failed comparison trims trailing ">") 869 TESTNG-70 (TestNG prevents eclipse from opening an older CVS version of a java class) 870 display of test hierarchy information (TESTNG-29) 871 872=========================================================================== 873 8745.0.1 875 876Eclipse plug-in 877 878Added: Output directory for the tests 879Added: Can now specify listener classes 880 881=========================================================================== 8825.0.1 883 884Fixed: reports generated by SuiteHTMLReporter do not work with JDK1.4 885 886=========================================================================== 887 8885.0 8892009/04/01 890 891Added: Ant task: support for JVM, workingDir, timeout 892Added: Stack traces can be interactively shown in the HTML reports 893Added: Link to testng.xml in the reports 894Added: New structure for reports, suites go in their individual directory 895Added: @Test(suiteName) and @Test(testName) 896Added: The stack traces in reports do not include TestNG frames (system property testng.exception) 897 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/9f4d46ade10b0fda) 898Fixed: Exit with error when no methods are run 899 (see: http://groups.google.com/group/testng-dev/browse_thread/thread/3c26e8a5658f22ac) 900Added: List of methods in alphabetical order 901Fixed: Class-scoped annotations were not recognized when inherited 902Added: Deprecated @Configuration and introduced @BeforeSuite/Test/Class/TestMethod 903Added: Deprecated @ExpectedExceptions and moved it into @Test 904Added: expectedExceptions to @Test, deprecated @ExpectedExceptions 905Added: New annotations: @BeforeSuite, @BeforeTest, etc... 906Fixed: Was returning an exit code of 0 if a cyclic graph was detected 907Added: Interface org.testng.ITest so that tests can declare a name 908Fixed: The Text reporter was reporting the square of the actual number of methods 909Fixed: Bug reported by Eran about dependencies with an afterClass method 910Added: IHookCallBack now receives the ITestResult in its run() method 911Added: Name of suite for command line can be set with -Dtestng.suite.name=xxx 912Fixed: TestNGAntTask was hardcoding m_haltOnFSP to true 913Fixed: Passing a null parameter caused an NPE in the reports 914Added: "listener" to the ant task (and documentation) 915Added: if patch-testng-sourcedir.properties is found in the classpath 916 with a property "sourcedir" containing a ; separated list of 917 directories, this list will override -sourcedir. 918 919=========================================================================== 920 921 9224.7 923 924Added: Maven 2 plug-in 925Fixed: Message formattings in TestNG assertion utility class 926Fixed: @Factory methods were counted as @Test as well 927 http://jira.opensymphony.com/browse/TESTNG-51 928Fixed: All DataProvider parameters were shown in the HTML report 929Fixed: Bug in testng-failed.xml generation 930Fixed: <packages> bug when using a jar file to load the test classes 931Added: alwaysRun for before @Configuration methods 932 http://jira.opensymphony.com/browse/TESTNG-35 933Fixed: groupless @Configurations were not invoked if a method depends on a group 934 http://jira.opensymphony.com/browse/TESTNG-45 935Added: beforeGroups/afterGroups to @Configuration 936 937Eclipse plugin: 938 939Added: last contextual launch is available in Eclipse launcher lists 940Fixed: 3.2M5 integration (removed dependency on non-existing class) 941Fixed: testng-failures.xml generation 942 943=========================================================================== 944 9454.6 9462006/27/02 947 948Added: Documentation contains the new reports 949Added: TestNG.setUseDefaultListeners(boolean) 950Added: Descriptions now appear in TextReporter (verbose>=2) and the HTML reports 951Added: description attribute to @Test and @Configuration 952Added: combined Reporter output in the reports 953Added: methods not run in the reports 954Added: org.testng.IReporter 955Added: threadPoolSize to @Test 956Added: Reports now show relative timings (start at 0) 957Added: Reports now show different colors depending on the methods' classes 958Added: Reports now show all parameters used to invoke the test method 959Added: org.testng.Reporter 960Added: DataProviders can accept a Method as first parameter 961Fixed: Extraneous implicit inclusion of a method 962 963Eclipse plugin: 964 965Added: Run/Debug as TestNG test from the editor contextual menu 966Fixed: TESTNG-24: 'Run as testng test' does not appear of the Test annotation does not have a group 967Fixed: TESTNG-18: Eclipse plugin ignores Factory annotation 968Fixed: TESTNG-21: Show differences when double clicking assertion exceptions 969Added: UI allows setting orientation (even more space) 970 http://forums.opensymphony.com/thread.jspa?threadID=17225&messageID=33805#33805 971 972=========================================================================== 973 9744.5 9752007/07/02 976 977Core: 978 979Fixed: Methods were not implicitly included, only groups 980Fixed: Bug with failed parent @Configuration don't skip child @Configuration/@Test invocations 981Fixed: Bug with overridding @Configuration methods (both parent and child were run) 982Fixed: Bug when overriding beforeClass methods in base class (cyclic graph) 983Added: Support for JAAS (see org.testng.IHookable) 984Fixed: Problem with nested classes inside <package name="foo.*" 985Fixed: If a group is not found, mark the method as a skip instead of aborting 986Fixed: testng-failed.xml was not respecting dependencies 987Fixed: class/include method in testng.xml didn't work on default package 988Fixed: DTD only allowed one <define> 989Fixed: ArrayIndexOutOfBoundsException for jMock 990Added: dependsOnMethods can contain methods from another class 991Fixed: JUnitConverter required -restore, not any more (option is now a no-op) 992Fixed: JUnit mode wasn't invoking setName() on test classes 993Added: Regular expressions for classes in <package> 994Added: Distributed TestNG 995Fixed: Command line parameters and testng.xml are now cumulative 996Fixed: Reports now work for multiple suites 997Fixed: Was ignoring abstract classes even if they have non-abstract instances 998Fixed: If setUp() failed, methods were not skipped 999Fixed: Was not clearly indicating when beforeSuite fails 1000Added: @Configuration.inheritGroups 1001Fixed: inconsistency between testng.xml and objects regarding method selectors 1002 1003Eclipse plug-in: 1004 1005New look for the progress view. 1006 1007=========================================================================== 1008 10094.4 1010 1011Core: 1012 1013Fixed: testng-failures.xml was not excluding methods from base classes 1014Fixed: Bug in suites of suites for JUnit mode 1015 1016=========================================================================== 1017 10184.3 1019 1020Core: 1021 1022Fixed: testng-failures.xml was not excluding methods from base classes 1023Fixed: Bug in suites of suites for JUnit mode 1024Added: Excluded groups on command line and ant task 1025Fixed: When including a group, implicitly include groups depended upon 1026Fixed: When depending on several groups, wasn't skipped if one of them failed 1027Fixed: Failures weren't reported accurately in the JUnitReports report 1028Fixed: Wasn't throwing an exception if depending on a non-existing group 1029 1030=========================================================================== 1031 10324.2 1033 1034Core: 1035 1036Fixed: wasn't excluding methods in base classes 1037Added: alwaysRun for tests (soft dependencies) 1038Fixed: Class-level enabled=false were not honored 1039Fixed: Bug with multiple dataproviders on same class 1040Fixed: Bug with dataprovider defined in the parent class 1041Fixed: Bug with dataprovider defined in a subclass 1042Fixed: Bug with dataprovider defined in an abstract class 1043Fixed: testng-failures generation was excluding the methods even if a failed test depended on it 1044 1045=========================================================================== 1046 10474.1 1048 1049Core: 1050 1051Added: @DataProviders can return Iterable<Object[]> 1052Fixed: Superclass test methods were not called in the presence of a class @Test 1053Added: Reporter class to log messages in the HTML reports 1054 1055=========================================================================== 1056 10574.0 10582005/11/10 1059 1060Core: 1061 1062Fixed: suite methods now invoked only once in a hierarchy 1063Added: @DataProvider and @testng.data-provider 1064Fixed: Interleave order now respected for before/afterClass methods 1065Added: Can now invoke java -jar testng-2.6.jar <...> 1066Added: Support for BeanShell 1067Added: Method Selectors (IMethodSelector) 1068Fixed: In the absence of dependencies, @Configuration methods respect inheritance 1069Fixed: Bug in multithreaded dependencies on methods 1070Fixed: dependsOnGroups wasn't working on regular expressions 1071Fixed: Bug in <package> when directories contain spaces in their names 1072Fixed: Introduced a JDK5 dependency in the JDK1.4 build (getEnclosingClass()) 1073Fixed: Output directory in ant task was not honored if it didn't exist 1074Fixed: Problem with timeout according to 1075 http://forums.opensymphony.com/thread.jspa?threadID=6707 1076 1077Eclipse plug-in: 1078 1079Fixed: Wasn't handling linked directories correctly 1080Fixed: Bug in QuickFix implementation 1081Added: Quick Fix for JUnit conversion (Annotations and JavaDoc) 1082Fixed: Methods Run as TestNG test 1083Added: Package level Run as TestNG test 1084Fixed: Resources from the linked directories are using a wrong path when 1085 passed to command line TestNG 1086 1087IDEA plug-in: 1088 1089Added: Support for JDK 1.4 (both projects and IDEA itself) 1090Fixed: Classes that contained only configuration were ignored 1091 1092=========================================================================== 1093 10942.5 10952005/08/08 1096 1097Added: ITestListener.onTestStart(ITestResult) 1098Added: Support for <packages> 1099Added: Resource files for easier ant taskdefs 1100Fixed: @Configuration methods were not invoked with individual test methods 1101Fixed: Bug with ExpectedExceptions 1102Fixed: Didn't support nested factory classes 1103Fixed: NPE if -target is omitted with JDK 1.4 1104Fixed: @Configuration failures in a class would cause other classes to fail 1105Added: alwaysRun 1106Fixed: beforeTestClass/afterTestClass were broken for a pathological case 1107Added: @Configuration(alwaysRun) 1108Added: JUnitConverter task 1109Fixed: < and > characters in reports were not escaped 1110 1111Eclipse plug-in: 1112 1113Fixed: Class dialog wasn't showing @Factory classes 1114 1115IDEA plug-in: 1116 1117First release! 1118 1119Documentation: 1120 1121Added: Brand new look!!! 1122Added: Section on testng.xml 1123Fixed: Numbering of sections 1124 1125=========================================================================== 1126 11272.4 11282005/07/05 1129 1130Changed: New package: testng.org 1131Fixed: Bug with @ExpectedException occuring the parallel mode 1132Fixed: Bug with parameters and beforeTest 1133Added: IInstanceInfo support 1134Fixed: methods were not excluded when included by groups 1135Fixed: testng-failures.xml is now including also the beforeSuite/afterSuite methods 1136Fixed: generating the testng-failures.xml is now working as expected 1137Fixed: Factories call all the tests even if some of them fail along the way 1138Fixed: Better JUnit support (wasn't creating individual instances) 1139Fixed: dependsOnGroups didn't work across different classes 1140Added: command line (and Ant) -groups option 1141Added: @Parameters (and made parameters attribute deprecated) 1142Added: Parameters for constructors 1143Fixed: Better interleaving of before/afterTestMethods 1144Fixed: Ant task 1145Fixed: TestNGException thrown when TestNG conditions are not fulfilled 1146 1147Documentation: 1148- New assert classes 1149- New ways to launch 1150- JUnitConverter documentation 1151- new beforeSuite/afterSuite 1152 1153=========================================================================== 1154 11552.3 11562005/04/12 1157 1158Fixed: Spaces are now legal in JavaDoc comments 1159Added: documentation for @Factory 1160Fixed: factories were called multiple times 1161Added: beforeSuite and afterSuite 1162Fixed: inheritance and scope now working properly for annotations 1163Fixed: dependsOnMethods wasn't working for 1.4 1164Added: Better stack traces 1165Added: Better syntax for included/excluded methods 1166Fixed: Better verbose support 1167Fixed: Various fixes for the Eclipse plug-in 1168Added: Can specify a class name on the command line 1169Fixed: Default package bug in JUnitConverter 1170Added: Regression tests for JUnitConverter 1171Added: -quiet option to JUnitConverter 1172 1173=========================================================================== 1174 11752.2 1176 1177Fixed: Wasn't handling several testng.xml files correctly 1178Fixed: Renamed -src to -sourcedir 1179Fixed: Complains if no sourcedir is specified in 1.4 1180Added: In 1.4, don't require annotations="javadoc" 1181Fixed: If setUp fails, complain and mark test methods as skips 1182Fixed: Dependent methods weren't working for 1.4 1183 1184=========================================================================== 1185 11862.1 11872005/02/12 1188 1189Added: Parser can accept an InputStream for testng.xml 1190Fixed: expected-exceptions now fails if test passes 1191Fixed: reports now use the suite name in HTML 1192Added: invocationCount and successPercentage 1193Added: dependsOnMethods 1194Added: timeOut works in non-parallel mode 1195 1196=========================================================================== 1197 11982.0 11992004/12/06 1200 1201Added: port on JDK 1.4 1202 1203=========================================================================== 1204 12051.3 1206 1207Added: new view: classes (still experimental) 1208Added: timeout on methods 1209Added: thread-count 1210Added: TestNG is now multithread, see "parallel" in <suite> 1211 1212=========================================================================== 1213 12141.2 1215 1216Added: JUnitConverter 1217Fixed: Bug with afterClasses (test: AfterClassCalledAtTheEnd) 1218 1219=========================================================================== 1220 12211.1 1222 1223Added: new links for methods and groups in the HTML report 1224Added: <methods> 1225Added: <fileset> to <testng> 1226 1227=========================================================================== 1228 12291.0 12302004/04/28 1231http://beust.com/weblog/2004/04/28/ 1232 1233Fixed: Updated to the new DTD 1234Fixed: Suite table of contents displays failures first 1235Fixed: Bug in afterTestClass 1236Added: Validating testng.xml 1237Added: Scoped parameters 1238Added: testng.xml 1239Removed: Property quiet 1240Changed: Verbose is now an integer 1241Added: Dependent methods 1242 1243=========================================================================== 1244 12450.9 1246 1247Added: Groups of groups 1248Added: Groups for Configuration methods 1249Added: Parameters 1250 1251=========================================================================== 1252 12530.2 1254 1255Fixed: Merged TestMethod and TestClass into Test 1256Added: HTML report 1257Added: Regexps for groups 1258Fixed: Inheritance of methods 1259Fixed: ExpectedException is now called ExpectedExceptions 1260