I saw something similar happening to a colleague.

After a long day, he refused to join the team for dinner and drinks. He was vegetarian, did not drink, and was a yoga practitioner. He admitted this when he opted to go back to his hotel room.

However, his decision was not perceived positively. The partner, valuing the team building aspect crucial for a strong team, felt the consultant's absence was a setback. While I empathized with my colleague's perspective and was willing to support him, he was reassigned shortly after.