Shadow Hearts: Covenant is a direct continuation of Shadow Hearts; Its story starts a year and a half after the first one in the midst of World War I. It focuses on Karin Koenig, a lieutenant in the German Army who assists in the exorcism of a man able to turn into demons, Yuri Volte Hyuga. However, after learning of Yuri's caring nature, Karin goes on a quest with him to save him from a spell inside him and are drawn into a conflict involving demons of demonolatry.

The game begins with Karin Koenig, a lieutenant in the German Army during World War I, traveling to the town of Domremy with a Vatican Inquisitor, Nicolas "Nicolai" Conrad, to exorcise a demon there. When they arrive, the demon is revealed to be Yuri from the previous game. Nicolai casts a curse on Yuri; this severely weakens Yuri, but also reveals Nicolas as a member of Sapientes Gladio, a faction of dark sorcerers seeking world domination.

Yuri escapes with Karin and they build a small party of allies to thwart Nicolas' plan. Nicolai, a bastard son of Tsar Nicolas II, hopes to take over Russia with the help of Rasputin, the current leader of Sapientes Gladio. Yuri manages to recover his strength with the help of a former Sapientes Gladio sorcerer, Jovis, though his memory is still affected by Nicolai's curse. The group then finds and defeats Rasputin and Nicolai. Before Nicolai can be finished off, however, he is rescued by Kato, a Japanese Envoy and old friend of Yuri's, and taken to Japan.

In Japan, the party tracks Nicolai to a research lab, where he is possessed by a demon. After learning where he is, the party tracks him, but not before Kato's wife is killed. In despair, Kato attempts to remake the world using his own sorcery, forcing Yuri and his party to put him down

The game has two possible endings. In the “bad” ending, Yuri's personality and memories are erased by Nicolai's curse, and he spends the rest of his life under the care of Roger Bacon, his old friend. In the “good” ending, Yuri chooses to die so that he doesn't lose his memories and personality to the curse. He then finds himself at the beginning of the previous game. In both endings, Karin travels back to 1887 and is revealed to be Anne, Yuri's mother.