this kind of error is usually a symptom of kernel not well configured, you should double check that the ipc parameters are well sized (maximum shared memory size, message queue parameter, ...). The Oracle installation on Linux should be quite similar than the one on Solaris.
