Close the accept socket on error