the error message says that the aolserver was running out of memory during a fork. a typical situation for a fork is, when an external program is started. historically, unix made a copy of the process that called the external progam and executes it there. modern unixes don't make a full copy, but share as much as possible.
for running out of memory, there might be many different reasons. maybe, the process has a memory limit set, maybe it is running out of address space, maybe the machine has not enough swap space defined.
The amount of memory needed for the aolserver is not so much determined by the number of pages, or the size of the database, but more by the number of packages loaded and the number of connection threads defined (at least, when you have min-threads = max-threads) and you orient yourself on the configuration hints on the people running huge sites (like us).
In general, it might help if you provide some more info about your inhstallation (such as operating system, hardware, configuration parameters, etc.). Otherwise, readers can only make vague guesses about what's happening.
