I don't think it's too bad to build a site from HEAD if you are careful to keep a reference copy of the code (as a vendor branch or a tarball) so you could more easily merge what's coming. One thing that will happen in the not too distant future (when I free up a little or someone else steps up to the plate) is that we will merge all the 4.6 bug fixes over
which would likely cover bugs on your own site.
That will be a lot of changes but hopefully not too many conflicts to resolve.
If you are patching bugs on your copy I would suggest you submit patches since if you do so, they will get applied to head and reduce the number of conflicts you will have to resolve yourself (this is a general suggestion to everyone,
not you in particular :) )
