Test::Distribution checks for several important gotchas many authors tend to overlook. (Particularily those who haven't released much to CPAN yet and thus don't have much experience with creating distributions.)
These include: pod errors, inclusion of all use()d modules in the prerequisites section of the Makefile.PL or Build script, signature, the existance of important distribution files like README, Changes, etc., versions in all included modules, and that all enclosed modules load okay.
Using Test::Distribution isn't hard either. Just copy its synopsis to a t/something.t file and you're done. Most of the time, anyway. Sometimes, you'll want to configure the testing process, but unfortunately, Test::Distribution isn't too flexible about that. You can specify certain types of tests to carry out or not to carry out, but that's all.
One problem I found while using Test::Distribution was that it gets the files to test from some File::Find magic instead of the MANIFEST. I think it should use the files in MANIFEST by default and revert to testing all files in the directory only if MANIFEST doesn't exist. It should issue an optional and non-fatal warning if MANIFEST and the contents of the build directory don't match. Or one should be able to explicitly tell the module which pm and pl files to test.
For example, one of my development directories contains a script that generates some code required for a new module release. That script uses YAML.pm to get the module's current version from META.yml and includes the version number in the generated code. Trying to build the module in that directory throws a Test::Distribution error: "YAML" isn't listed as a prerequisite for building in the Makefile.PL.
Regardless of the problems I faced, the module is a step in the right direction and I will be using it for my non-trivial distributions in the future.