According to the temple, it was founded in 815 by Kūkai, who carved a statue of Dainichi Nyorai and enshrined it in the temple he built. At the end of the Heian period, Awa Ichinomiya Shrine was moved here from Kamiyama, and Dainichi-ji became its associated temple.

During the Northern and Southern Dynasties period, Ichinomiya Castle was built near the top of the mountain just east of the shrine. The clan who built the castle deeply revered the shrine, and also served as its priests. Between 1573 and 1592, during the invasion of Shikoku by the Toyotomi, Chōsokabe Motochika destroyed the Ichinomiya clan and gave control of the castle to a subordinate. But the castle fell, and Motochika was forced to cease his resistance and submit to the Toyotomi. The castle was given to the Hachisuka family and it was finally dismantled in 1615. The foundations of the castle can still be seen on the hill. The climb up to the castle is steep, but is rewarded with a panoramic view.

In the early Edo period, Hachisuka Mitsutaka, who became the third lord of the Tokushima domain, also revered the shrine, and rebuilt both the shrine and the temple. At some point, when the shrine became a pilgrimage temple, the temple began to be called Ichinomiya Temple.

Due to the separation of Shintō and Buddhism in the early Meiji period, the eleven-faced Kannon, the principle image of the shrine said to have been carved by Gyōki, was moved to the main hall of the temple as its principal image. As of that time, the temple became independent of the shrine.

The current chief priest, Kim Myō-sung, visited Japan in 1995 as a traditional Korean dancer, and the following year he married the former chief priest, Hiroe Oguri, and had a son, Kōmyō. When his wife died in 2007, he considered returning to Korea, but he decided to train as a priest and take over until his son became an adult and succeeded him. So he became the only chief priest from a foreign country serving in the 88 pilgrimage temples.