Skip to content

Eclipse Platform UI - JUnit Migration Analysis Report #3414

@vogella

Description

@vogella

#Eclipse Platform UI - JUnit Migration Analysis Report

Executive Summary

The Eclipse Platform UI codebase contains 35+ test bundles with approximately 7,024 @test annotations across 1,008 test files. The project employs a mixed JUnit version strategy with a clear migration path from JUnit 3 (legacy) through JUnit 4 (current mainstream) toward JUnit 5 (modern). The complexity of migration varies significantly based on test architecture patterns.

1. Test Bundle Overview

Total Test Infrastructure

  • Test Bundles: 35 directories
  • Test Files with @test: 1,008 files
  • Total @test Annotations: 7,024 occurrences
  • Test MANIFEST.MF Files: 65 files

Test Bundle Categories (33 bundles)

E4 Platform Tests (8 bundles):

  • org.eclipse.e4.core.commands.tests
  • org.eclipse.e4.emf.xpath.test
  • org.eclipse.e4.ui.bindings.tests
  • org.eclipse.e4.ui.tests (comprehensive)
  • org.eclipse.e4.ui.tests.css.core
  • org.eclipse.e4.ui.tests.css.swt
  • org.eclipse.e4.ui.workbench.addons.swt.test

JFace Tests (5 bundles):

  • org.eclipse.jface.tests (primary)
  • org.eclipse.jface.tests.databinding
  • org.eclipse.jface.tests.databinding.conformance
  • org.eclipse.jface.tests.notifications
  • org.eclipse.jface.text.tests

Legacy Workbench Tests (7 bundles):

  • org.eclipse.ui.tests (primary - large)
  • org.eclipse.ui.tests.harness (base classes)
  • org.eclipse.ui.tests.rcp
  • org.eclipse.ui.tests.navigator
  • org.eclipse.ui.tests.forms
  • org.eclipse.ui.tests.views.properties.tabbed
  • org.eclipse.ui.tests.browser

Text & Editors (5 bundles):

  • org.eclipse.text.tests
  • org.eclipse.text.quicksearch.tests
  • org.eclipse.ui.editors.tests
  • org.eclipse.ui.genericeditor.tests
  • org.eclipse.ui.workbench.texteditor.tests

Other Tests (8 bundles):

  • org.eclipse.core.filebuffers.tests
  • org.eclipse.ltk.core.refactoring.tests
  • org.eclipse.ltk.ui.refactoring.tests
  • org.eclipse.search.tests
  • org.eclipse.tests.urischeme
  • org.eclipse.ui.ide.application.tests
  • org.eclipse.ui.monitoring.tests
  • org.eclipse.ui.tests.performance

2. JUnit Version Usage Analysis

Import Statistics (Current State)

JUnit 4 Dominance:

921    import org.junit.Test;                    (JUnit 4 - default)
481    import org.junit.Before;                  (JUnit 4 - setup)
322    import org.junit.After;                   (JUnit 4 - teardown)
180    import org.junit.Rule;                    (JUnit 4 - rules)
94     import org.junit.Ignore;                  (JUnit 4)
60     import org.junit.Assert;                  (JUnit 4)

JUnit 5 Adoption (Emerging):

87     import org.junit.jupiter.api.Test;       (JUnit 5)
29     import org.junit.jupiter.api.BeforeEach;
28     import org.junit.jupiter.api.AfterEach;
11     import org.junit.jupiter.api.Disabled;
4      import org.junit.jupiter.api.condition.*
2      import org.junit.jupiter.params.* (parameterized)

JUnit 3 Legacy (Minimal):

1      extends junit.framework.TestCase

Test Suite Pattern (JUnit Platform):

92     import org.junit.platform.suite.api.Suite;
92     import org.junit.platform.suite.api.SelectClasses;

Advanced Patterns

JUnit 4 Rules (Migration complexity indicator):

180    @Rule annotations
23     @ClassRule annotations
31     TestWatcher implementations
4      ExternalResource/TemporaryFolder rules

JUnit 4 Runners (Requires refactoring):

14     @RunWith annotations
12     @RunWith(Parameterized.class)
7      @FixMethodOrder / @MethodSorters
1      Custom BlockJUnit4ClassRunner

Lifecycle Annotations:

12     @BeforeClass / @AfterClass (JUnit 4 static)

3. Migration Complexity Assessment

Tier 1: Simple (Easy to Migrate)

Characteristics: Pure JUnit 4, no inheritance, no complex rules
Estimated: 60-70% of codebase

Example - TextHoverPopupTest

// File: org.eclipse.jface.text.tests
public class TextHoverPopupTest {
    @Test
    public void testSearch() {
        // Pure assertions, no setup/teardown
        assertEquals(i, result);
    }
}

Migration: Replace @Test@Test, Assert.assertEqualsAssertions.assertEquals


Tier 2: Moderate (Medium Complexity)

Characteristics: JUnit 4 with @Before/@after, possible inheritance, simple setup
Estimated: 25-30% of codebase

Example - EditorTest (from AbstratGenericEditorTest)

// File: org.eclipse.ui.genericeditor.tests/EditorTest.java
public class EditorTest extends AbstratGenericEditorTest {
    @Test
    public void testGenericEditorHasWordWrap() throws Exception {
        // Uses inherited state from parent
        this.editor.setFocus();
        // ...
    }
}

// Parent: AbstratGenericEditorTest
public class AbstratGenericEditorTest {
    @BeforeEach  // Already JUnit 5!
    public void setUp() throws Exception {
        // Initialize shared test resources
    }
    
    @AfterEach   // Already JUnit 5!
    public void tearDown() throws Exception {
        // Cleanup
    }
}

Migration: Parent already uses @BeforeEach/@AfterEach (JUnit 5)


Tier 3: Complex (Challenging - Migrate Later)

Characteristics: JUnit 4 Rules, @RunWith runners, inheritance hierarchies, parametrized tests
Estimated: 5-10% of codebase

3a. Rule-Based Tests

Example - StackRendererTest

// File: org.eclipse.e4.ui.tests
public class StackRendererTest {
    @Rule
    public WorkbenchContextRule contextRule = new WorkbenchContextRule();
    
    @Inject  // Dependency injection via rule
    private IEclipseContext context;
    
    @Before
    public void setUp() throws Exception {
        // Uses rule context
    }
}

Issue: Custom rule provides DI - requires DI migration strategy
Migration Path: Adapt rule to work as extension or use ParameterResolver


3b. Parameterized Tests

Example - EModelServicePerspectiveFindTest

// File: org.eclipse.e4.ui.tests
@RunWith(Parameterized.class)
public class EModelServicePerspectiveFindTest {
    @Parameters
    public static Object[] data() {
        return new Object[] { true, false };
    }
    
    @Parameter
    public boolean simple;
    
    @Before
    public void setUp() {
        if (simple) {
            setupSimpleApplication();
        } else {
            setupWorkbenchApplication();
        }
    }
}

Migration:

  • JUnit 5 requires @ParameterizedTest and @ValueSource or @MethodSource
  • Multiple test methods needed instead of parameter field

3c. Inheritance Hierarchies with Template Methods

Example - AbstractPairMatcherTest

// File: org.eclipse.jface.text.tests
public abstract class AbstractPairMatcherTest {
    private final boolean fCaretEitherSideOfBracket;
    
    public AbstractPairMatcherTest(boolean caretEitherSideOfBracket) {
        // Constructor-based test variation (JUnit 3 pattern)
        fCaretEitherSideOfBracket = caretEitherSideOfBracket;
    }
    
    @Test
    public void testSimpleMatchSameMatcher() throws BadLocationException {
        // Shared test logic, executed with different constructor params
    }
}

// Subclass with specific parameters
public class DefaultPairMatcherTest extends AbstractPairMatcherTest {
    public DefaultPairMatcherTest() {
        super(false);  // Constructor parameter for behavior
    }
    
    @Test
    public void testTestCaseReader1() {
        // Extends parent tests + additional
    }
}

Issue: Constructor-based parameterization is JUnit 3 pattern
Migration:

  • Extract constructor parameters as @ParameterizedTest
  • Use @MethodSource for complex scenarios
  • Consider refactoring inheritance to composition

3d. UITestCase Base Class (Hybrid Pattern)

Example - UITestCase

// File: org.eclipse.ui.tests.harness/util/UITestCase.java
public abstract class UITestCase extends TestCase {  // JUnit 3 base!
    @Rule
    public TestWatcher testWatcher = new TestWatcher() { /* ... */ };
    
    @Before
    @Override
    public final void setUp() throws Exception {
        super.setUp();  // JUnit 3 style
        closeTestWindows.before();
        doSetUp();
    }
    
    @After
    @Override
    public final void tearDown() throws Exception {
        doTearDown();
        closeTestWindows.after();
    }
    
    protected void doSetUp() throws Exception { /* override */ }
    protected void doTearDown() throws Exception { /* override */ }
}

Used By: Many workbench tests inherit from this
Challenge: Bridge between JUnit 3 and JUnit 4 patterns
Migration Strategy:

  1. Convert UITestCase to non-extending class
  2. Use composition pattern with @BeforeEach/@AfterEach
  3. Update all subclasses to use new pattern

3e. Test Suites

Example - AllTests

// File: org.eclipse.jface.tests/AllTests.java
@Suite
@SelectClasses({ AllActionTests.class, AllDialogTests.class, /* ... */ })
public class AllTests {
}

Status: Already using JUnit Platform Suite API (JUnit 5)
No Migration Needed: Just verify dependencies


4. MANIFEST.MF Dependencies

Current JUnit Dependencies (Sample)

org.eclipse.e4.ui.tests:

Require-Bundle: 
 org.junit;bundle-version="3.8.2",
 org.mockito.mockito-core;bundle-version="2.13.0",

org.eclipse.ui.tests.navigator:

Require-Bundle: org.junit,
Import-Package: 
 org.junit.jupiter.api;version="[5.14.0,6.0.0)",
 org.junit.platform.suite.api;version="[1.14.0,2.0.0)"

org.eclipse.ltk.core.refactoring.tests:

Require-Bundle: org.junit,
Import-Package: 
 org.junit.jupiter.api;version="[5.14.0,6.0.0)",
 org.junit.platform.suite.api;version="[1.14.0,2.0.0)"

Dependency Strategy

  • org.junit: JUnit 4 (required by legacy projects)
  • org.junit.jupiter.api: JUnit 5 (already imported in newer tests)
  • org.junit.platform.suite.api: Suite support (already present)

Action Required: Most bundles already have dual dependency capability


5. Key Findings by Bundle

Highest Priority for Migration

Bundle Current Issues Complexity
org.eclipse.jface.text.tests 90% JUnit 4 Inheritance hierarchies HIGH
org.eclipse.ui.tests 85% JUnit 4 UITestCase base, many rules VERY HIGH
org.eclipse.ui.tests.performance 80% JUnit 4 Parameterized runners HIGH
org.eclipse.e4.ui.tests 95% JUnit 4 WorkbenchContextRule, complex DI HIGH

Already Partially Migrated

Bundle JUnit 5 % Notes
org.eclipse.ui.genericeditor.tests 100% All @BeforeEach/@AfterEach
org.eclipse.jface.tests ~70% Mixed Suite/Test patterns
org.eclipse.ui.tests.navigator ~60% Using JUnit 5 imports

JUnit 3 References

Only Found In:

  • org.eclipse.ui.tests.harness/util/UITestCase.java (intentional bridge class)
  • Handful of legacy tests extending UITestCase

6. Test Patterns Breakdown

Pattern 1: Simple @test Methods (Migrate First)

Count: ~4,500 tests
Complexity: TRIVIAL
Example: TextHoverPopupTest.testSearch()
Strategy: Direct annotation replacement

Pattern 2: @Before/@after Setup/Teardown

Count: ~481 @Before, ~322 @After
Complexity: LOW
Example: Model setup/cleanup
Strategy: Replace @Before → @BeforeEach, @After → @AfterEach

Pattern 3: @rule Usage

Count: ~180 @Rule, ~23 @ClassRule
Complexity: MEDIUM-HIGH
Examples: 
  - TemporaryFolder (→ @TempDir)
  - TestWatcher (→ TestReporter or Extension)
  - Custom WorkbenchContextRule
Strategy: Convert to JUnit 5 extensions

Pattern 4: @RunWith(Parameterized.class)

Count: ~12 instances
Complexity: HIGH
Example: EModelServicePerspectiveFindTest
Strategy: 
  - Use @ParameterizedTest
  - Define @MethodSource or @ValueSource
  - Refactor test methods

Pattern 5: Inheritance Hierarchies (Template Method)

Count: ~15-20 test class hierarchies
Complexity: VERY HIGH
Example: AbstractPairMatcherTest → DefaultPairMatcherTest
Strategy:
  - Extract constructor parameters as @ParameterizedTest
  - Consider composition over inheritance
  - Update all subclasses

Pattern 6: Test Suites

Count: ~10 suite classes
Complexity: NONE
Status: Already JUnit 5 compatible (@Suite/@SelectClasses)

7. Recommended Migration Strategy

Phase 1: Foundation (Low Risk)

Timeline: Weeks 1-2
Target: 60-70% of codebase

  1. Update parent POM to include JUnit 5 Jupiter/Platform
  2. Migrate simple standalone test classes
  3. Replace @Test, @Before, @After annotations
  4. Update imports from org.junit.* to org.junit.jupiter.api.*
  5. Replace Assert assertions with Assertions

Bundles to Start:

  • org.eclipse.jface.text.tests (many independent tests)
  • org.eclipse.ui.editors.tests (mostly simple)
  • org.eclipse.ui.genericeditor.tests (already partially migrated)

Phase 2: Rules & Inheritance (Medium Risk)

Timeline: Weeks 3-4
Target: 20-25% of codebase

  1. Create JUnit 5 Extension equivalents for custom rules
    • WorkbenchContextRule → WorkbenchContextExtension
    • TestWatcher patterns → TestReporter
  2. Refactor test inheritance hierarchies
    • Extract constructor-based parameterization
    • Use @ParameterizedTest for variations
  3. Update UITestCase and subclasses

Bundles to Focus:

  • org.eclipse.e4.ui.tests (convert WorkbenchContextRule)
  • org.eclipse.jface.text.tests (refactor inheritance)
  • org.eclipse.ui.tests.harness (update base class)

Phase 3: Complex Scenarios (High Risk)

Timeline: Weeks 5-7
Target: 5-10% of codebase

  1. Convert @RunWith(Parameterized.class) to @ParameterizedTest
  2. Migrate performance tests with custom runners
  3. Handle conditional tests (@EnabledOnOs, etc.)
  4. Update static @BeforeClass/@afterclass to @BeforeAll/@afterall

Bundles to Address:

  • org.eclipse.ui.tests.performance
  • org.eclipse.e4.ui.tests (advanced scenarios)
  • org.eclipse.ui.tests (selective)

Phase 4: Verification & Cleanup

Timeline: Weeks 8-9
Target: Full validation

  1. Run full test suite with JUnit 5
  2. Verify all tests pass
  3. Remove JUnit 4 dependencies (if possible)
  4. Update documentation

8. Specific Migration Examples

Example 1: Simple Test → JUnit 5

BEFORE (JUnit 4):

public class TextHoverPopupTest {
    @Test
    public void testSearch() {
        int[] values = { 0, 1, 2, 3 };
        int result = search(values, 2);
        assertEquals(2, result);
    }
}

AFTER (JUnit 5):

public class TextHoverPopupTest {
    @Test
    void testSearch() {
        int[] values = { 0, 1, 2, 3 };
        int result = search(values, 2);
        assertEquals(2, result);
    }
}

Changes: Just the annotation and access modifier


Example 2: Setup/Teardown → JUnit 5

BEFORE (JUnit 4):

public class ModelServiceImplTest {
    @Rule
    public WorkbenchContextRule contextRule = new WorkbenchContextRule();
    
    @Before
    public void setUp() throws Exception {
        editor = modelService.createModelElement(MPart.class);
    }
    
    @After
    public void tearDown() {
        cleanup();
    }
}

AFTER (JUnit 5):

public class ModelServiceImplTest {
    @RegisterExtension
    static WorkbenchContextExtension contextExtension = 
        new WorkbenchContextExtension();
    
    @BeforeEach
    void setUp() throws Exception {
        editor = modelService.createModelElement(MPart.class);
    }
    
    @AfterEach
    void tearDown() {
        cleanup();
    }
}

Example 3: Inheritance Hierarchy → Parameterized

BEFORE (JUnit 4 + Template Method):

public abstract class AbstractPairMatcherTest {
    private boolean caretEitherSide;
    
    public AbstractPairMatcherTest(boolean caretEitherSide) {
        this.caretEitherSide = caretEitherSide;
    }
    
    @Test
    public void testSimpleMatch() { /* ... */ }
}

public class DefaultPairMatcherTest extends AbstractPairMatcherTest {
    public DefaultPairMatcherTest() {
        super(false);
    }
}

AFTER (JUnit 5):

public class PairMatcherTest {
    @ParameterizedTest
    @ValueSource(booleans = { true, false })
    void testSimpleMatch(boolean caretEitherSide) { 
        // Unified test with parameter
    }
}

Example 4: @RunWith(Parameterized.class) → @ParameterizedTest

BEFORE (JUnit 4):

@RunWith(Parameterized.class)
public class EModelServicePerspectiveFindTest {
    @Parameters
    public static Object[] data() {
        return new Object[] { true, false };
    }
    
    @Parameter
    public boolean simple;
    
    @Before
    public void setUp() {
        if (simple) setupSimple();
        else setupWorkbench();
    }
    
    @Test
    public void testInActivePerspective() { /* ... */ }
}

AFTER (JUnit 5):

public class EModelServicePerspectiveFindTest {
    @ParameterizedTest(name = "[{index}] simple={0}")
    @ValueSource(booleans = { true, false })
    void testInActivePerspective(boolean simple) {
        if (simple) setupSimple();
        else setupWorkbench();
        // test logic
    }
    
    // Alternative with @MethodSource for complex scenarios
    @ParameterizedTest
    @MethodSource("testData")
    void testWithData(TestData data) { /* ... */ }
    
    static Stream<TestData> testData() {
        return Stream.of(
            new TestData(true),
            new TestData(false)
        );
    }
}

9. Risk Assessment

Green Light (Low Risk)

  • Simple test replacements: 60-70%
  • Suite-based tests: Already JUnit 5 compatible
  • Basic @Before/@after migrations: 481+ instances

Risk Level: MINIMAL

Yellow Light (Medium Risk)

  • Custom rule conversions: ~180 @rule instances
  • Simple inheritance refactoring: ~10-15 classes

Risk Level: LOW-MEDIUM

Red Light (High Risk)

  • Complex inheritance hierarchies: ~5-10 classes
  • Parameterized runner conversions: ~12 instances
  • UITestCase migration: Affects 50+ test classes
  • DI-based rules: WorkbenchContextRule patterns

Risk Level: MEDIUM-HIGH

Mitigation Strategies

  1. Create extension wrapper layer for backward compatibility
  2. Run tests in parallel with old/new versions during transition
  3. Incremental conversion by bundle/category
  4. Automated static analysis for pattern detection
  5. Comprehensive test suite validation

10. Key Files for Reference

Base Classes & Utilities

  • /tests/org.eclipse.ui.tests.harness/src/org/eclipse/ui/tests/harness/util/UITestCase.java - Legacy base class
  • /tests/org.eclipse.ui.genericeditor.tests/src/org/eclipse/ui/genericeditor/tests/AbstratGenericEditorTest.java - JUnit 5 base example

Simple Migration Examples

  • /tests/org.eclipse.jface.text.tests/src/org/eclipse/jface/text/tests/TextHoverPopupTest.java

Complex Patterns

  • /tests/org.eclipse.e4.ui.tests/src/org/eclipse/e4/ui/workbench/renderers/swt/StackRendererTest.java - Rules + DI
  • /tests/org.eclipse.e4.ui.tests/src/org/eclipse/e4/ui/tests/application/EModelServicePerspectiveFindTest.java - Parameterized

Inheritance Hierarchies

  • /tests/org.eclipse.jface.text.tests/src/org/eclipse/jface/text/tests/AbstractPairMatcherTest.java
  • /tests/org.eclipse.jface.text.tests/src/org/eclipse/jface/text/tests/DefaultPairMatcherTest.java

Test Suites (Already JUnit 5)

  • /tests/org.eclipse.jface.tests/src/org/eclipse/jface/tests/AllTests.java

11. Conclusions

Current State

  • 7,024 tests already written with JUnit 4 annotations
  • 60-70% could be migrated with simple find-replace
  • 25-30% require moderate refactoring
  • 5-10% need significant architectural changes

Migration Feasibility

  • Very Feasible: 60-70% of codebase
  • Feasible with Planning: Additional 25-30%
  • Requires Architecture Review: Last 5-10%

Recommended Next Steps

  1. Select Phase 1 bundles for pilot migration
  2. Create JUnit 5 extension equivalents for common patterns
  3. Establish test execution validation framework
  4. Execute incremental migration by bundle

Success Criteria

  • All tests pass with JUnit 5 Jupiter engine
  • Performance tests maintain baseline execution time
  • Zero functional regressions in test coverage
  • Complete deprecation of JUnit 4 (final phase)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions