Module::Build seems to have been written with Module Authors in mind, with little-to-no thought given to the end users, the ones who would use the modules written with Module::Build.
One of the main perks of Perl in general and CPAN in particular is the ease with which one can install and use a module. Simply by typing
we can have the module and all of its dependencies downloaded, built, tested, and installed.
Modules that are written with Module::Build completely destroy this ability. I've read the author's reasoning for not supporting PREFIX, and it just doesn't hold up. PREFIX is the standard, it is what is used, and alternative OS's aside, it works for 90% of the cases in which its used. By not supporting PREFIX, the author makes it impossible to install a module via the cpan shell. It would even be acceptable if the author allowed makepl_arg to have both PREFIX and destdir or install_base set, and simply ignore PREFIX if it's there. But that doesn't work either, as the Build.PL script dies with an error if PREFIX is given at all.
Going through the four step process of manually building Module::Build-dependent modules is not acceptable, for the same reason that led to the creation of CPAN.pm to begin with - all of the module's dependencies and sub-dependencies have to be retrieved, built, tested, and installed independently and in the correct order.
Until CPAN.pm, Module::Build.pm or CPANPLUS.pm get together and make a coherent way to install all modules regardless of whether they use build or make, I will choose the simple alternative of not using any module which relies on Module::Build. Thus far, I haven't found any burning hole in my programming abilities caused by not using them.