Harem (Turkish, from Arabic حرم ḥaram 'forbidden place; sacrosanct, sanctum', related to حريم ḥarīm 'a sacred inviolable place; female members of the family' and حرام ḥarām, 'forbidden; sacred') refers to the sphere of women in a usually polygynous household and their quarters which is enclosed and forbidden to men. It originated in the Near East and came to the Western world via the Ottoman Empire, says Wikipedia.
The term serraglio (Italian from Persian sarāy "palace, enclosed courts") carries a similar meaning.
In modern colloquial (and humorous) English, "harem" may denote a number of female followers of a charismatic man.