All perfect praise be to Allah, the Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and Messenger.
If your mother no longer possesses her mental capacity, then she is excused from the religious obligations. So do not order her to pray if such is the case, because she is no longer accountable.
However, if she still possesses her mental capacity, you are obliged to order her to pray and she should pray according to her ability. She is excused from observing the conditions and pillars of prayer that she is incapable of performing. If she cannot perform the pillars of the prayer (like bowing and prostrating), she can pray by nodding. If she cannot perform ablution, she can perform Tayammum (dry ablution). If she cannot remove the impurity, she is exempted from that. If she cannot speak, she recites the Quran and Thikr (expressions of remembrance) of the prayer in her heart.
Allah knows best.