Testing & Assertions

A test is a claim about your code, written down, that a machine rechecks on every run. That is worth more than it sounds. Six months from now you will not remember that this function assumed no missing values, and neither will whoever changes it next. The test will.

The suite on this page is real, and it contains one test that fails on purpose, because a passing suite proves nothing about the suite. That failure documents an actual bug in the function under test, and it is a bug worth meeting.

Assertions come first

An assertion lives inside the function and runs on every call. No framework, no second file, no test runner.

#' CountsPerMillion.
#'
#' @param counts Input value.
#' @return Computed result.
CountsPerMillion <- function(counts) {
  stopifnot(
    "counts must be numeric" = is.numeric(counts),
    "counts must not be negative" = all(counts >= 0, na.rm = TRUE)
  )
  counts / sum(counts) * 1e6
}

CountsPerMillion(c(-5, 10))

cpm(c(-5, 10)) ->  counts must not be negative

Naming each condition turns it into the error message. Without names you get the expression back, which is less useful when someone else hits it.

The difference from a test: an assertion checks every future call, including the ones you never thought to try. A test checks one case you did think of. If you write one thing after finishing a function, write the assertion.

What is worth testing in analysis code

Not every line. Four things pay for themselves:

	what it catches
Shape	returns the wrong length, the wrong type, a matrix transposed
Known answer	a tiny input you can work out by hand, so the maths is checked once
Invariant	something true of any output, whatever the input
Edge case	empty, one element, NA, zero, negative, all-identical

The invariant is the one people miss and the one that keeps working as the code changes. For counts per million it writes itself: whatever goes in, the output sums to a million.

The edge case is where bugs concentrate, because the happy path is the part you already checked by running it once.

A real suite, with a real failure

test_cpm.R:

test_that("CountsPerMillion scales to a million", {
  expect_equal(sum(CountsPerMillion(c(10, 20, 70))), 1e6)     # the invariant
  expect_equal(sum(CountsPerMillion(c(1, 1, 1, 1))), 1e6)
})

test_that("CountsPerMillion gives the answer you can work out by hand", {
  # 25 out of a library of 100 is a quarter, so 250,000 per million.
  expect_equal(CountsPerMillion(c(25, 75)), c(250000, 750000))
})

test_that("CountsPerMillion rejects input it cannot handle", {
  expect_error(CountsPerMillion(c("10", "20")), "must be numeric")
  expect_error(CountsPerMillion(c(-1, 10)), "must not be negative")
})

test_that("CountsPerMillion handles a single count", {
  expect_equal(CountsPerMillion(c(42)), 1e6)                  # an edge case
})

test_that("CountsPerMillion survives an NA count (FAILS: documents a real bug)",
  {
  out <- CountsPerMillion(c(10, NA, 70))
  expect_false(all(is.na(out)))
})

Run it with testthat::test_file("test_cpm.R"):

cpm: ........1

══ Failed ══════════════════════════════════════════════════════════════════════
── 1. Failure ('test_cpm.R:37:3'): cpm survives an NA count (FAILS: documents a
all(is.na(out)) is not FALSE

`actual`:   TRUE
`expected`: FALSE

8 passed, 1 failed. expect_equal compares with a tolerance, which is what you want for doubles; see floating point for why == is the wrong test.

The bug the failing test found

sum() propagates NA. So one missing count turns the entire output vector into NA, not just its own entry:

cpm(c(10, NA, 70))       ->  NA NA NA
cpm_fixed(c(10, NA, 70)) ->  125000 NA 875000

The function does not error. It does not warn. It returns the right length and the right type with every value wrong. That is the shape of bug that survives review, gets into a figure, and is found six months later by someone else.

The repair is one argument, sum(counts, na.rm = TRUE), and then the missing value stays in its own slot where it belongs. The point is that nothing except a test was ever going to tell you the difference.

An unseeded test is not a test

Without a seed, two calls disagree, so any test asserting a value flakes:
  run 1: 88, 108, 102, 88, 95
  run 2: 100, 105, 105, 106, 86
  identical: FALSE

With set.seed(42) before each:
  run 1: 113, 94, 100, 88, 98
  run 2: 113, 94, 100, 88, 98
  identical: TRUE

The first two rows differ on every run of that script, which is the problem. The last two are the same on every run, on every machine, forever.

Seed inside each test, not once at the top of the file. A single seed at the top makes every test depend on the order the others ran in, so adding a test at line 10 breaks a test at line 90 and the failure looks like it came from nowhere.

Seeding is not only for tests. Any analysis with a random component needs one to be reproducible: bootstraps, permutations, k-means starts, train and test splits, UMAP layouts.

Testing figures and models

You cannot assert that a plot looks right, and you should not try. Test the numbers that go into the figure, which is where the bugs are, and look at the picture yourself.

Same for a model. Assert the shapes, assert the ranges, assert that a known-signal input is recovered. Do not assert an exact accuracy: that pins your suite to a package version and it will fail on an upgrade for reasons that have nothing to do with your code.

Quick reference

Task	Call
Guard an input	stopifnot("message" = condition)
A test block	test_that("description", { ... })
Compare with tolerance	expect_equal(a, b)
Compare exactly	expect_identical(a, b)
Expect a failure	expect_error(expr, "pattern")
Length and type	expect_length(x, n), expect_type(x, "double")
Run one file	testthat::test_file("test_cpm.R")
Deterministic randomness	set.seed(42)

Next steps

• Debugging & Reading Errors, the other half: a guard catches a bad input, a test catches a bad implementation, and debugging is what you do when neither fired.
• Documentation and Style, for the roxygen2 blocks, Google R names, and package layout that turn a tested script into a documented package.
• Wickham, R Packages, the Testing chapter, for testthat inside a package.

The runnable scripts, the suite, and the container are in the companion code repo under guides/programming/r/testing/.