The 22-year-old beat Djokovic 7-5 6-4 in one hour and 37 minutes to end the Serb's 22-match unbeaten run.
14-time grand slam champion, Djokovic, won the first set 7-6.
Federer faced two break points in the opening game of the second set as Djokovic threatened to take control, but matters stayed on serve until a few minutes of uncharacteristic sloppiness from the Serb gave Federer a chance to level, which he took with a fizzing forehand victor.
The Serb threw his racket down in frustration as Federer again escaped from 15-40 down to move 5-4 ahead, but the 20-time Grand Slam champion's magic fizzled out as the final set headed to a tie-break.
Khachanov took ruthless advantage of the German's physical struggles as Zverev, who was battling shoulder trouble, needed a medical time-out in the first set at 4-1 down.
Djokovic defeated Federer 6-4, 6-4 in the Cincinnati final on 19 August to improve to 24-22 in their rivalry. "I came here to Paris with no expectations and figuring that I'd just play a match or two, thinking I'd probably lose because I was too exhausted after Basel".More news: Driver was huffing before Girl Scout hit-and-run
"Paris. I played some good tennis so I can be happy", he pointed out.
The 31-year-old, now unbeaten in 22 matches, is a four-time victor of the Paris Masters.
The Russian went on to break the Serbian early on in the second set but failed to capitalise off another three break points in the seventh game. I'm just pleased to be in the semis. He absolutely deserved to win the match today, ' he said.
"And in general, it's a breakthrough season", said Khachanov, who Moscow's Kremlin Cup last month.
Djokovic, victor of the Wimbledon and US Open championships this year, will now face Roger Federer or Kei Nishikori for a place in Sunday's final. Five months ago, it was highly improbable considering my ranking and the way I played and felt on the court.
Federer was seeking to become only the second man after Jimmy Connors to win a century of professional tour titles following his 99th win in Basel, only to fall agonizingly short.
The Russian, who had prevailed in just three of his 19 encounters against players in the top 10, became the third different victor of a 1000 series event this season, could still take part in the ATP Tour Finals in London as a second alternate. What more can I ask for? But already, an established player, top player. Djokovic heads into the match with full confidence as he won 30 of the last 31 matches played since the Wimbledon first round.
"I'm sure we'll see a lot of great games from him in the future", conceded Djokovic.