Refactor the async wait fd logic