The SPD gained 25.8% against 24.1% for the CDU / CSU, according to figures from German channels ARD and Reuters.
The Greens came next with 14.6%, followed by the FDP with 11.5%.
The two parties have indicated that they are ready to discuss a three-way alliance with one of their two biggest rivals to form a 50% majority government.
The right-wing populist AfD came in fifth with 10.4%.
Armin Laschet from the CDU / CSU bloc and outgoing Finance Minister Olaf Scholz for the Social Democrats are in the running to become the next Chancellor.
The Greens, along with candidate Annalena Baerbock, are also making their first round in the Chancellery.
Germany could face months of coalition talks, which means Merkel must remain in an interim role.
Speaking after the revelation of exit polls, Mr Laschet said: “We will do our utmost to form a government under the leadership of the Union, as Germany now needs a coalition to the future that modernizes our country.
“This will probably be the first time that we have a government with three partners. “
CDU Secretary General Paul Ziemiak admitted that the party suffered “bitter losses” compared to previous elections which saw it win 32.9% of the vote four years ago.
He suggested it would be a “long election night” but hinted at the possibility of working alongside the Greens and Free Democrats.
Meanwhile, his Social Democratic counterpart, Lars Klingbeil, said his party “is back” having previously obtained just 20.5% in 2017.
The vote was closed at 5 p.m. UK time in a hotly contested election to replace the country’s 16-year-in-power leader.
Around 60.4 million people were eligible to elect the new Bundestag – the German parliament – which will then choose the new chancellor.