In a narration which is Marfu' (attributed to the Prophet) reported by Abu Dawud and an-Nasa'i from Abu Hurairah (RA) it has:
"Do not swear by your fathers, nor by your mothers, nor by the rivals (set up as equals with Allah); and do not swear by Allah except when you are speaking the truth."
Reference: Bulugh al-Maram 1375
In-book reference: Book 13, Hadith 2
English translation: Book 13, Hadith 1375