It has been said that Oyama-ji was founded in 724 by Reverend Roben. However, the exact year of foundation is still unknown.
An archive documenting a land donation in 1338 indicates that, at the latest, the temple existed since the first half of 14th century.
At about that time, the temple belonged to the Tendai Buddhist school and became prosperous as a temple of esoteric mountain Buddhism, a syncretism of Buddhism, Shintoism and Animism.
During the Sengoku period (1467 – 1590), the temple was under protection of Satomi and Masaki feudal clans.
Oyama-ji was converted to the Shingon Buddhist school in 1872. However, there are still vestiges of this ancient syncretism, evidenced by Shinto-based mikoshi festivals held by the precincts of the shrine at the temple.