Jeremiah 31:31 talks about a new covenant that is written on your heart. This covenant is not like the old covenant, which was a literal covenant. The purpose of the old covenant was to point to the new covenant, which is fulfilled by Jesus.
Jer 31:31 Behold, the days come, saith the LORD, that I will make a new covenant with the house of Israel, and with the house of Judah: Not according to the covenant that I made with their fathers in the day [that] I took them by the hand to bring them out of the land of Egypt; which my covenant they brake, although I was an husband unto them, saith the LORD: But this [shall be] the covenant that I will make with the house of Israel; After those days, saith the LORD, I will put my law in their inward parts, and write it in their hearts; and will be their God, and they shall be my people.
In order to understand the new covenant you must understand the old one.
The Forwilliam Bible Wiki has more in-depth information about the old and new covenant.