All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger.
First of all, you should know that Ibn Hajar, the one who said the last statement, is not Ibn Hajar Al-‘Asqalaani, the author of Fat'h Al-Baari, but rather Ibn Hajar Al-Haytami said it in his book Al-Fataawa Al-Hadeethiyyah.
It is incorrect to infer the permissibility of Sufi dance from the tradition about the Abyssinian dance, as you already know what Ibn Hajar quoted the majority of scholars as saying. The Kuwaiti Fiqh Encyclopedia reads: “The Hanafis, Maalikis, Hanbalis and Al-Qaffaal from the Shaafi'i School of Fiqh are all of the view that dancing is disliked, justifying that by its lowliness and foolishness, and that it undermines manhood, and it is also vain play. Al-Abiyy said: The scholars interpreted the Hadeeth of the Abyssinians dancing to mean that they were jumping with their weapons and playing with their spears; in conformity with what was reported in the narration: They played in the presence of the Messenger of Allaah with their spears. All this applies if dancing is not accompanied by a forbidden matter like drinking alcohol or exposing one’s ‘Awrah (parts of the body that must be covered in Islam), in which case, it would be forbidden by consensus.” [End of quote]
With regard to the inference for the permissibility of dancing from the report about Ja'far ibn Abi Taalib, then Ibn Al-Qayyim said in Zaad Al-Ma’aad: “As for what was reported in this story that when Ja'far looked to the Prophet he hopped on one leg as a way of glorifying the Messenger of Allaah and that the bear-like dancers provided this as evidence for the permissibility of dancing, Al-Bayhaqi said that this was narrated on the authority of Ath-Thawri from Abu Zubayr from Jaabir, but in its chain of narrators to Ath-Thawri there are narrators who are not known. I said, even if this narration is authentic, this is not evidence for the permissibility of bending and swaying and moving like women which is contrary to the guidance of the Prophet . Perhaps this might have been the habit of the Abyssinians when they glorified their great people, like the marching of the Turks and the like, and Ja’far imitated them in doing that once and then he left it for the Sunnah of Islam, so how can this be compared to jumping, bending, swaying and moving like women, we ask Allaah to guide us.”
Allaah Knows best.