All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and messenger.
The Hadeeth to which you referred in the question does not indicate that the Prophet, sallallaahu ‘alayhi wa sallam, or ‘Abdullaah ibn ‘Umar offered prayer in the very spot where a grave or graves were found. The Hadeeth was reported by Al-Bukhari with the following wording, “… there were two or three graves by ('Inda in Arabic) that mosque, on those graves there were piles of stones. They were located to the right of the path…”
The Arabic word (‘inda), translated as “by” does not necessarily means “inside” or “next to”, so there is nothing to support the claim that the Prophet, sallallaahu ‘alayhi wa sallam, performed prayer in a place where there was a grave or next to a grave. It (the word ‘inda) can also mean “beside,” according to the Arabic dictionary, Al-Qaamoos Al-Muheet.
However, there are many authentic Ahaadeeth that are explicitly indicative of the forbiddance of taking graves as mosques. For example:
The Prophet, sallallaahu ‘alayhi wa sallam, said: “Beware of taking graves as mosques; I forbid you to do that.” [Muslim]
He, sallallaahu ‘alayhi wa sallam, also said: “Know that the most wicked people are those who take graves as places of prayer.” [Ahmad]
These explicit texts and others which indicate the prohibition of taking graves as places of (performing) prayer should not be left and replaced by such implicit texts (i.e. this Hadeeth). Verily, those who do so are like those who resort to ambiguous texts and overlook definitive and lucid ones.
Allaah Knows best.