The original castle was built by the Freisings of Škofja Loka in 1202. The castle was almost entirely destroyed by an earthquake in 1511 and Bishop Filip took on its restoration. In the castle courtyard there once stood an impressive four-cornered tower, the floorplan of which has been restored. Today the castle is home to one of the best-arranged museums in Slovenia, with archeological, historical, cultural, artistic, natural science and ethnological collections. Visitors can also see the excellently-preserved 16th century farmhouse, Škopar's House (Škoparjeva hiša) in the castle garden.