Fix a few tests that fail on VMS

Fix a few tests that fail on VMS

In one spot, files aren't properly closed, so the sub-process program
that's supposed to read them can't, because it's locked out.

In another spot, srctop_file() was used where srctop_dir() should be
used to properly format a directory specification.

Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/16518)