Malik related to me that he had heard that when Umar ibn Abd al-Aziz left Madina, he turned towards it and wept. Then he said, "OMuzahim! Do you fear that we might be among those that Madina castsoff?"
USC-MSA web (English) reference: Book 45, Hadith 9
Arabic reference: Book 45, Hadith 1609