Smart Test Suites with Ruby

December 15, 2025

Lessons from Ruby World Conference 2025

At Ruby World Conference 2025, Masatoshi Seki (関 将俊) and Miwa Fukaya (深谷 美和) presented a talk grounded in something rare in our industry: more than 20 years of real testing history.

Their presentation, “How to Create Today’s Recommended Tests”, introduces Ninja Testing — a testing practice that extends TDD principles to acceptance testing and focuses on sustainability at scale.

The problem they address is familiar:

As systems grow, running every test every day becomes unrealistic. Yet skipping tests increases risk.

A Simple Idea, Borrowed from Music

Before streaming services, music players faced a similar challenge: how to play everything without repeating the same tracks over and over.

The solution was smart playlists.

Ninja Testing applies this idea directly to test suites:

• Newly created or recently modified features are tested first
• Failed tests are never “rested”
• Stable tests gradually appear less often
• Over time, all tests are executed
• Daily test volume stays intentionally manageable

Each day, the system generates a focused list called “Today’s Recommended Tests.”

The Core Algorithm (Ruby)

One of the most refreshing aspects of the talk is how clearly the idea translates into Ruby code.

At the heart of the system is a simple rule: tests that recently passed can rest for a while.

def idle_period?(test)
  return false unless test.last_result

  periods = [300, 90, 28, 14, 7, 1] # days, based on importance
  rest_days = periods.fetch(test.test_rate)

  test.last_date > Time.now - rest_days * 24 * 60 * 60
end

Tests are then selected and prioritized like this:

tests
  .reject { |t| idle_period?(t) }
  .sort_by { |t|
    [
      t.last_result ? 1 : 0, # failed tests first
      t.last_date,           # older tests first
      t.name                 # stable ordering
    ]
  }
  .first(25)

No complex frameworks. No hidden magic.

Just explicit rules, written in Ruby, and easy to reason about.

What “Failure” Really Means

In Ninja Testing, a failed test is not only a broken feature.

It can also mean:

• a specification that no longer makes sense
• behavior that technically works but feels wrong
• usability regressions
• outdated assumptions hidden in old tests

This is why manual testing remains central in their approach. Automation supports the process, but humans still explore.

Proven Over Decades

The team validated this method using 20 years of test execution data inside a Ruby-based ticket system.

The results were consistent:

• Old tests are never forgotten
• Risky areas receive attention immediately
• Obsolete tests reveal themselves naturally
• Test coverage improves without increasing daily workload

This is testing designed for systems that are meant to last.

A Ruby Lesson

Ninja Testing is a reminder of something Ruby developers often value:

Simplicity scales when it is honest about constraints.

Good testing is not about running more tests. It’s about choosing the right tests today, without losing sight of tomorrow.

A quiet, thoughtful idea from Ruby World Conference 2025 — and a very Ruby way to approach long-term quality.