The 32-year-old former Australian Open champion was blitzed in the opening set but discovered his serving touch to claim his fifth career victory in 10 matches against the Frenchman.
"After the first set, I thought to myself, 'It can't get any worse than this, it can only get better'," said Johansson, the world number 74. "I tried to go for it a little more in the second set."
In other second-round matches, sixth seed Mardy Fish of the United States lost to Germany's Michael Berrer 7-6, 6-4 while wild card John Isner of the United States stunned eighth seed Benjamin Becker of Germany 3-6, 7-5, 7-6.
Johansson managed to break Clement once in each of the final two sets but it was enough to secure the win and set up a third-round affair against Chile's Paul Capdeville.
Clement, ranked 30th in the world, was nursing a pulled stomach muscle but refused to use it as an excuse.
KNEE PAIN
"I really didn't play too good in the first set but I thought, 'I can win this match,'" said the 29-year-old Clement. "But he just started to play better.
"He only broke me twice. It wasn't much of a difference but it was enough."
Fish is suffering from tendinitis in both knees and had his serve broken four times against Berrer.
"I didn't lose because of lack of opportunities or chances," he said. "I was up 4-2 in the second set and my knee pain made it tough for me to get up on my serve. It boils down to the fact that I couldn't hold my serve.
"It's worth looking into cutting back my play. I don't have much to defend right now and I really want to peak at the U.S. Open."
Top seed Andy Roddick had the night off and will play Czech Radek Stepanek on Thursday.
