Damage to a huge dam can cause great loss of human life and property, but disasters and their consequences can be minimized by implementing effective dam safety monitoring strategies. However, establishing a permanent monitoring system on a huge dam is costly. Additionally, for reasons of national security, many dams and information about them may not be able to be accessed by researchers. Accordingly, continuously monitoring the structural health of a dam by measurement may be difficult. This study presents a way to continuously monitor the health of a dam using vibration signals that are measured not on the dam but close to it. The Sayano-Shushenskaya Dam in Russia is used to demonstrate the idea. Intensive ambient vibration measurements were firstly made once to determine the natural frequencies of the dam. Then the natural frequencies of the dam under varying environmental effects are obtained from the spectra of the seismic records obtained at Cheryomushki seismic station, which is located 4.4 km northeast of the dam. To account for the effects of varying environmental conditions on the natural frequencies, an autoencoder in the form of an unsupervised learning neural network, was employed. The autoencoder was trained using the natural frequencies without using any environmental factors to learn the intrinsic behavior of the dam under varying environmental conditions. The errors between input data to the trained autoencoder and the regenerated data from the autoencoder can be used to determine whether the dam is under normal conditions. A finite element model of the dam was constructed to simulate changes of natural frequencies due to cracks in the dam structure. The results demonstrate that the proposed method can feasibly monitor the structural health of the dam.