All perfect praise be to Allah, the Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallahu ʻalayhi wa sallam, is His slave and Messenger.
You should completely ignore this Waswaas (obsessive doubt), and do not repeat any of the prayers that you claim to have performed without ablution, following the view of Shaykh Al-Islam Ibn Taymiyyah who saw that whoever neglects one of the conditions of the prayer out of ignorance is not obliged to repeat the prayer.
As for the creamy-colored and brownish discharges, you should know that the basic principle is that a yellowish or brownish discharge that comes out after menstruation stops is not considered part of menstruation. As for the reddish discharge, it is considered menstruation from which you must perform Ghusl (ritual bath) as long as it came out during a time interval that makes it possible to be part of menstruation (i.e. a continuation of your period).
As for the prayers which you performed during the interval of purity between the two intervals of menstruation (before seeing the red discharge again), they are valid because you attained valid ritual purity between them. It is not required of the menstruating woman to make up for acts of worship that she performs during this interval of purity, which is the view of the Hanbali scholars.
You should hasten to perform Ghusl when you see that your menstruation has ended. You are not considered to be menstruating again except if you see blood, and if you do, then you consider it menstruation and perform Ghusl after the bleeding stops.
Lastly, we advise you to completely disregard such Waswaas; verily, getting carried away with it leads to a grave evil.
Allah knows best.