I had a bug in earlier versions of nsmail regarding grabage collecting, by default it runs evey 10 mins, so it might be it. Try to disable it by setting
ns_section ns/server/${servername}/module/nsmail
ns_param gc_interval 0
or you can download most recent nsimap 2.4 module which is renamed nsmail 2.4 with this bug fixed.