fix BIGNUM flag handling