The village of Starý Rybník (literally meaning "Old Pond") had belonged to the demesne of the Rabs of Mechelsgrün since 1299. At some time in the 14th century, the Rabs built a new manor house between two ponds, after which they called themselves for the first time in 1364. Soon afterwards, the estate somehow ended in the possession of the Frankengrüner burgher family from Cheb who were succeeded by the Gumerauers and the Brambachs. The estate was held by the Lords of Wirsperg throughout the major part of the 16th century. In the following years, the estate frequently changed its owners – it was in possession of the Trautenbergs, Hartenbergs, Perglars of Perglas, and many more. In 1787, it was acquired by Johann Georg Wilhelm whose descendants were raised to noble status and began denoting themselves as "von Helmfeld". The family held the estate up to 1945.
The entrance gate to the castle with an almost perfect rectangular layout was located on the north side next to a square tower structure. A three-storey palace was located in the south. Apart from cellarage, a major part of the western wall has been preserved while the eastern and the inner curtain are hardly noticeable. Two half-cylindrical walls supported the palace from the south of which only the western section has been preserved. The castle was rebuilt in the mid 15th century. The entrance part was replaced by a new wing that swallowed up the remains of the old tower structure.
The castle had remained unchanged and had been probably inhabited until a fire in 1792 when a part of the castle became a ruin. Nonetheless, the entrance building had remained in use up to the early 20th century when it was abandoned and gradually tumbled down. Following the destruction of the castle, the Wilhelms built an Empire manor house north of it.