It was narrated from Aishah that:
the Messenger of Allah (ﷺ) swore to keep away from his wives, because Zainab had sent back her gift and 'Aishah said: "She has disgraced you." He became angry and swore to keep away from them.
English reference: Vol. 3, Book 10, Hadith 2060
Arabic reference: Book 10, Hadith 2138