Of the modules on CPAN which attempt to guess mime types by reading the file, rather than from looking at the extension, File::LibMagic seems to be the best one both in terms of speed and accuracy.
Unfortunately, the underlying library cannot properly detect encodings, and falsely claims UTF-8 to be "binary".
The solution I eventually settled on for my job was a combination of "read_binary" from File::Slurper, "valid_utf8" from Unicode::UTF8 to validate the encoding, and "info_from_string" from this module used on the text read in by "read_binary". I decided to use File::LibMagic rather than MIME::Detect because MIME::Detect is extremely slow to load and operate, and the encoding problems this has could be worked around using the above methodology.
After comparing against File::MMagic, File::MMagic::XS, File::Type, I ended up choosing File::LibMagic because it has the least problems and looks like being the most maintained (although it would be nice if the author cleans up the RT queue).
For those stuck without a C compiler, File::Type or File::Magic can be an alternative.