I believe it's more efficient to have it actually recorded as "not null" at the attribute level than as a check constraint. To do that you can do this:
UPDATE pg_attribute SET attnotnull = TRUE
WHERE attname = 'my_column_name'
AND attrelid = (SELECT oid FROM pg_class WHERE relname = 'my_table_name')
It's quite safe, although you'll need to separately check for any existing NULL values.
