test_requires all the way
tl;dr Today possibly for the first time ever, CPAN toolchain ecosystem all support test requirements as separate from build requirements. I can’t be happier ever.
Little bit of history. Timeline might be not in the correct order.
Module::Install always had test_requires() with forward compatibility in mind. However META spec v1.4 didn’t support specifying test requirements separate from build requirements. So the most tools merge them as a build requirement, including Module::Install itself. This means most of the Test:: module dependencies, even though they are only used for testing, were installed even when notest options is supplied (with CPAN.pm and cpanminus).
META spec v2.0 came out, and it allows META (and MYMETA) to specify test requirements separately.
cpanm supported parsing test requirements separately, and skips installing test prereqs when –notest is in use. But at that time, most of the tools did not support emitting test requires apart from build requires.
Except Module::Build::Tiny, which just copies META.json to MYMETA.json upon installation, so if you use an authoring tool such as Dist::Zilla or Milla to emit META.json with test requirements apart, it will be skipped.
Meanwhile rjbs added TEST_REQUIRES support to MakeMaker at Paris QA hackathon 2012. 7 months later MakeMaker 6.63_03 was released with the fix, and non-dev version was just out in December.
Same fix was added to Module::Build by tokuhirom and released as 0.40004 today.
Now, what should happen is to update Makefile.PL and Build.PL to make use of these new features — I patched Dist::Zilla so that MakeMaker and ModuleBuild emitters will use test_requires when the installer has the version with the support for it.
I’m trying to fix Module::Install similarly, but as of this writing I’m not that hopeful — if someone wants to take over my work to make it merged that’d be super. But at this point I’d rather give up and stop using Module::Install.
Anyway, now when you have a module that is only used for testing, it will be (and has to be) declared and emitted correctly as a test dependency, and when you run CPAN installers, at least cpanm, with “no test” option, they won’t be installed. This is a very important milestone.