Feature request: test discovery

[kubajal]

It would be great if there was some API for test discovery. This would enable easier IDE integrations and improve overall user experience. The current alternative is to parse test content and extract test_that / describe / it calls ( e.g. in https://github.com/meakbiyik/vscode-r-test-adapter/blob/master/src/testthat/parser.ts#L84).

Writing parsers of test files is painful and raises issues such as meakbiyik/vscode-r-test-adapter#32, ie. edge cases when the tests are generated using paste0 or a for loop.

I propose to expose a function such as testthat::discover_tests(path) that would return or print the list of all test_that / describe / it expressions in a CSV format.

The way how it would work is that given this file with tests of the addition function:

addition <- function (x, y) { x + y }

describe("addition", {
  it("works with ints", {
    expect_equal(addition(1, 2), 3)
  },
  it("works with floats", {
    expect_equal(addition(1.2, 3.4), 4.6)
  }
})

test_that("addition works with negative numbers", {
  test_that(addition(1, -3), -2)
})

it would return something like:

selector,line_number # <-----  header
addition&works for ints,4
addition&works for floats,7
addition works with negative numbers,12

This functionality would would work in tandem with #2077 making running arbitrary tests through IDE UIs trivial to implement.

[hadley]

I chatted with the Positron team about this and they're not convinced that R is where you want to be running this code. You want test discovery code to run frequently and independently of the state of the current R interpreter. That suggests it should be implemented outside of R.