Not known its foundation history, it is only known that this region has been inhabited since the Paleolithic, due to remains found in archaeological excavations in 2008, carried out next to the castle, featuring artifacts from prehistory and was heir of a Castro-Portuguese -romano with the name Castra Leuca.
After the reconquest in 1165, D. Afonso Henriques who donated to the Order of the Temple, and later confirmed by D. Sancho I, in 1198, so this region of Beira was promoted to stand and line of defense against attacks. In the thirteenth century, attends a document reporting the donation to the Templars, depicting a homestead called Vila Franca of Cardosa issued by Fernando Sanches, and in 1214, King Alfonso II donates the remainder of the estate, and so this He was fully in power of the Templars, being recognized the royal donation by Pope Innocent III, whose village fortified and dubbed Castelo Branco.
With rapid development, Castelo Branco gained importance, being held on the chapters of the Temple, but with the extinction of the Knights Templar, its property passed to the Order of Christ, who made the commendation this extinct order, allowing the residence of commanders and jurisdiction in other commendations.
Despite the ground not being so generous, agricultural areas were quite abundant, but their fertility sinned with heavy authority that the Templars exercised, not being something appealing for fixing the village, so much of the land were handed over to military orders who had not as prosperous development as neighboring land, and slaughters applied to its neighbors, despite the punishment imposed by the king, they terrorized the neighboring town of Covilhã.
In the twelfth century, the Castelo Branco village just unfolded within its walls, a reason that made Dinis, in Queen Santa Isabel campaign, notify this issue in 1285 and only in 1343, with Afonso order IV, the walls were extended. Its expansion and consequent development were recognized by King Manuel I, who granted a new charter in 1535 and the reign of King John III, Castelo Branco receives the title of Vila Remarkable, getting a considerable population growth. This growth is also due in Guarda, Castelo Branco and Covilhã, the reason that Sephardic Jews fleeing the orders of the Catholic kings of Spain, thus fixing it in the Beira Interior.
So with the development of commercial activities that came from the Jewish commune, occurs in this region an economic expansion that despite the expulsion of the Jews of Portugal, in the year 1496, continued to be a large volume of construction. In Castelo Branco saw the construction of the convent of Augustinian friars, the Capuchin and St. Michael’s Church, today the White Castle See. The end of the century, Nuno de Noronha, Bishop of the Guard sends the building of the Episcopal Palace. It can be concluded that the Jewish presence in the region of Castelo Branco was very influential for the development of the city, being visible in the sixteenth century ported in the historic area their symbolic presence.