First can you visit the LAMS administration console by just visiting the URL LAMS is running on in a browser?
If so, can you then double check the URL the popup window is trying to acccess by right clicking inside the window and choosing properties?
In addition you should consult the LAMS server logs under
${jboss-install-dir}/server/default/log
to see if there are any problems.
