I like to use packages (in my case, Debian packages) because I like everything neat and nice in my system, with everything in its correct path. Plus Debian has a superb package management system that makes administration and keeping things up-to-date a breeze.
When I want to see what a package will do, I can just get the source of the package (apt-get source packagename) or download the package and run the "ar" command to unpackage it. .deb packages are simply .ar files with special headers.
For things that I can't use a package, I download the source, compile and use a program called "stow" that, by using symlinks, makes installation on correct paths and uninstalling very easy (it deletes the symlinks and you're done).
So yes, using a package system can greatly reduce your headaches. I wish Red Hat would ship the version of APT with RPM support. That way you could just do as we Debian'ers do: "apt-get install whatever" or "apt-get remove whatever" and live happily ever after.
