All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
The scholars differed in opinion concerning the ruling on conditional divorce, and the majority of the scholars are of the view that it takes place in any case. However, Ibn Taymiyyah is of the view that it does not take place if the husband only intended to threaten his wife and that he is only obliged to expiate for breaking an oath.
The Fatwa that the Shaykh issued to you is an opinion which is adopted by some scholars, so you can act according to it. The layman is obliged to ask a scholar whom he trusts and then act according to his Fatwa.
Al-Qurtubi said in his Tafseer:
“Regarding matters of the religion, it is an obligation upon a layman who does not have knowledge of and is unable to derive rulings from their principles (i.e., Religious texts) because he is not qualified to do so; it is an obligation upon him to ask the most knowledgeable person of his time in his country. After he issues a Fatwa for him, then he should act upon his Fatwa, as Allah says (what means): {So ask the people of the knowledge if you do not know.} He must endeavor to find the person whom the majority of the people consider as the most knowledgeable person.” [End of quote]
Therefore, you must ignore these doubts and have good marital relations with your husband. Both of you must endeavor to avoid things that lead to dissension, and your husband must be careful not to make divorce the way to solve problems, as this may lead to unpleasant consequences especially that Allah blessed you with children. In most cases divorce has a bad impact on children.
If your husband is in doubt whether or not he intended repetition, then in principle he did not intend it. So, the condition was invalidated when it happened the first time, and the arguments that happen in the future do not lead to anything.
Allah knows best.