All perfect praise be to Allaah, the Lord of the worlds. I testify that there is none worthy of worship except Allaah and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and Messenger.
The majority of Muslim scholars define the limits of ‘Awrah of a man to be the area that is between the navel and the knee. The navel and knee themselves are not included in the ‘Awrah. There are many proofs supporting their opinion including the following:
• It is narrated on the authority of ‘Ali that the Messenger of Allaah, sallallaahu ‘alayhi wa sallam, said: "Do not unveil your thigh, and do not look at the thigh of a living or dead human." [Abu Daawood and Ibn Maajah]
• It is narrated on the authority of Ibn ‘Abbaas that he said: "The Messenger of Allaah, sallallaahu ‘alayhi wa sallam, once passed by a man whose thigh was uncovered. He said to him, 'Cover your thigh, indeed the thigh is (a part) of ‘Awrah.'" [Ahmad and At-Tirmithi]
• It was narrated on the authority of Jarhad Al-Aslami that he said: "The Messenger of Allaah, sallallaahu ‘alayhi wa sallam, once passed by me while my thigh was uncovered, he said to me, 'Cover your thigh, for it is (a part) of the ‘Awrah.'" [Maalik, Ahmad, Abu Daawood and At-Tirmithi]
It is permissible to go to the swimming pool if there are no women and the men there are covering their ‘Awrah, including yourself.
Allaah Knows best.