The Sri Mahamariamman Temple was founded by K. Thamboosamy Pillai in 1873 and was initially used as a private shrine by the Pillai family. The family threw the temple doors open to the public in the late 1920s and eventually handed the management of the temple over to a board of trustees.

This is the oldest functioning Hindu temple in Malaysia. It is also reputed to be the richest in the country. The temple was originally sited somewhere near the Kuala Lumpur railway station. It shifted to its present location along Jalan Tun H.S. Lee (next to KL's Chinatown) in 1885.

The initial attap structure was demolished in 1887 and a brick building was erected in its place. That structure was demolished to make way for the current temple building which were completed in 1968. The impressive gateway to the temple, known as the gopuram, was completed in 1972. The new temple was consecrated in 1973.