The distinctive episcopal vestment, of course, is the mitre, which is said to come from the Greek word for 'turban'. My Greek teacher once commented that 'mitra' was the name given to the distinctive headdress of prostitutes, although I have not found any evidence for this assertion!
The mitre's origin is uncertain although some scholars attribute it to the Phyrgian cap, but this is variously said to have been worn by freed slaves, or athletes. Certainly, only freed men could cover their heads. Dom Gregory DIx says that mitres were developed specifically for ecclesiastical use, and he says they are first mentioned in 4th-century Africa as the headgear of the now defunct office of deaconesses(!). He notes that mitres "passed thence to Spain where a 7th-8th century mention of the mitra religiosa in the form for the installation of an abbess... is preserved in the Mozarabic Liber Ordinum".
The clergy of Rome are said to have worn the mitre from the 9th-century, but it gradually becomes restricted. By the 10th-century it was reserved by Pope Leo VIII for non-liturgical use, and in the 11th-century was given by the Bishops of Rome as a mark of distinction to certain bishops, and later abbots as well. An interesting correlation is the gift of Caps of Maintenance, which are soft velvet caps worn under the gold crown, to monarchs as a mark of papal favour. If hats were bestowed as a sign of the dignity of freedmen, one can see how the custom of giving hats as a sign of favour developed.
Whatever its origins, it was originally a soft cap, which developed peaks from around the 12th century.It is worth noting the decoration and scale of the mediæval mitres depicted below. No doubt, as they later became highly decorated, the mitres' peaks had to be stiffened, and they were enlarged to allow for even more embroidery and decoration.
Tomb of Bishop Rodrigo Díaz (d.1339) in Salamanca's Catedral Vieja. The polychrome decoration is well preserved. In addition to the squat mitre, one can see the medallion pattern on the chasuble, and its beautiful full shape and flowing lines. Beneath this he wears a decorated dalmatic, and a long thin stole. 
From Burgos Cathedral. This mitre, like the chasuble seems to be decorated with pearls. Dix mentions that the crozier is first mentioned in early 7th-century Spain and they were borne by Spanish abbots and abbesses as well as bishops as symbols of their pastoral office. From the crozier hangs the mappula, or sudarium, which is a large handkerchief used to wipe away perspiration (so it is said) where the bishop gripped the crozier. The ceremonial handkerchief, which some say became the maniple, was carried by Roman consuls and magistrates. After Constantine, bishops were given the rank of magistrate, hence it became part of episcopal insignia. 
From St Nicholas' church in Burgos. Three mites are shown here, as well as another fine mediæval chasuble, and the mappula attached to the crozier.
