Larsson put United ahead ten minutes after re-start when he controlled Wayne Rooney's pass on the edge of the area before burying the ball home.
The former Barcelona striker is on a three-month loan to United from Helsingborg till the Swedish league starts in March.
Czech Republic striker Milan Baros leveled the score for the visitors in the 74th minute when he took advantage of Gary Cahill's poor shot and slot home past United keeper Gabor Kiraly.
Substitute striker Ole Gunnar Solskjaer sealed the precious victory for the Red Devils in stoppage time when his low shot went beneath Kiraly's hands and into the net.
Elsewhere, Tottenham, without Egyptian duo Ahmed Hossam 'Mido' and Hossam Ghaly, were held to a goalless draw by Cardiff City.
The Spurs will face Cardiff in a replay at the White Hart Lane.