All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger.
First of all, may Allaah reward all of you for being keen on learning religious matters as this is one of the best acts of worship by which one gets closer to Allaah. Abu Hurayrah narrated that the Prophet said: “Whoever treads a path for seeking knowledge, Allaah will make his path to Paradise easy (because of his seeking Knowledge)." [Muslim] For more benefit, please refer to Fatwa 88977.
Therefore, if there is no other better place where you can learn your religion, we do not advise you to stop going to this class.
On the other hand, those sisters should try their best not to bring their children with them to such classes, and you may cooperate with them in finding a way for solving this problem, like hiring a maid to take care of the children away from the place where the lesson is held until the lesson finishes, or you may find any other possible solution.
Allaah Knows best.