The first stage of construction work of the library started in 1933 according to Indo-Saracenic style designed by famous architect Narasimhan from Madras, India.

It began as a private collection but with the help of primarily local citizens, it became a full-fledged library soon. The Library also preserved archival material written in Palm leaf manuscripts, original copies of regionally important historic documents and newspapers that were published hundred of years ago in the Jaffna peninsula. It thus became a place of historic and symbolic importance to Sri Lankan Tamil people.

After about twenty six years of work, the first major wing of the library was opened in 1959 by then Jaffna Mayor Alfred Duraiappah. Prominent Indian librarian S.R. Ranganathan served as an advisor to ensure that the library was built to international standards.

The Jaffna library became the pride of the local people as even researchers from India and other countries began to use it for their research purposes.