Hi Dave,
actually, the code seems to make the right things. With a little debugging in naviserver activated, one can see that the binary string is correctly handled as a bytearray:
% set b [binary format H* "abcd1234"]
«Í4
% set h [ns_hmac string -digest SHA512 $b "This is a test"]
hmac key (len 4): ab cd 12 34
7d783715c6aedd1e9d0ee5f310da502a0540048a525d64964a9bb7e7b8c6782cacbfba8a2870cd7c9892a99da89404ca47f219dcb50c7ae5fb454704734473b8
Per default, the ouput encoding is hex, but one can specify as well base64url, base64, or binary. What result did you expect?