The ruins of the Venkataramana temple is located in the outer lower fort inside the Gingee Fort complex. Parts of the temple were dismantled by different personages. In 1761 CE, when Gingee fell to French occupation, many tall graceful monolithic ornamental pillars were dismantled from this temple and taken to Pondicherry to be set around the base of the statue of Governor Dupleix. Later, in 1860 CE, a Jain official in the Madras Provincial Services, Sri Baliah, facilitated the dismantling of several stone-pieces including the great stone elephants from the Gingee Venkataramana temple, to make edifices in the Sittamur Jain temple. The successor of Muthialu Nayaka, named Venkatappa Nayaka, had permitted a Jain merchant to build the Sittamur shrine.