Narrated Az-Zuhri:
that 'Urqah said: "I said to 'Aishah: 'I do not see anything wrong if someone does not go between As-Safa and Al-Marwah, nor any harm if I do not go between them.' She said: 'How horrible is what you have said O my nephew! The Messenger of Allah (ﷺ) would go between them, and the Muslims go between them. It was only that the people who assumed Ihram in the name of the false deity Mannah, which was in Al-Mushallal, would not go between As-Safa and Al-Marwah. So, Allah Blessed and Most High revealed: So it is not a sin for those who perform Hajj or go 'Umrah to the House to go between them (2:158). And if it were as you say, then it would be: "Then there is no harm on him if he does not go between them." Az-Zuhri said: "I mentioned that to Abu Bakr bin 'Abdur-Rahman bin Al-Harith bin Hisham. He was surprised at that and he said: 'Indeed this is knowledge. I had heard some men among the people of knowledge saying that those Arabs who would not go between As-Safa and Al-Marwah said, that going between these two rocks is a matter from Jahiliyyah. And others among the Ansar said: "We have only been ordered with going around the House, we were not ordered to do so with As-Safa and Al-Marwah." So Allah Most High revealed: Indeed As-Safa and Al-Marwah are of the symbols of Allah...' (2.158) Abu Bakr bin 'Abdur-Rahman said: 'So I thought that it was revealed about these people, and those people.'"
English reference: Vol. 5, Book 44, Hadith 2965
Arabic reference: Book 47, Hadith 3228