1) If I don't install acs-automated-testing on a server, the tests that exist in various packages should have no effect on me. They can sit in the file system, but they should just be ignored. In other words: It should be possible to have an OACS server without acs-automated-testing installed.
2) If I do install acs-automated-testing, I think it's reasonable that that package has an option that says "should I run all tests on server startup". It belongs in acs-automated-testing, not in core, correct?
3) I myself would much rather run the automated tests every time I make a change. So it should be possible to run them without restarting the server. Also, it should be possible to run only the tests registered for a single package to save time, so I can run them more frequently.