Only set the verify callback if there's one to set!