A Seamus Mallon try had helped Ulster to a 13-6 interval advantage but three second half penalties from Jeremy Staunton seemed set to end the home side's three-match losing run.
However, Larkin, who had missed three penalties early in the second half, held his nerve with the last kick of the game to earn Ulster's fourth win out of their opening five Celtic League matches.
Gary Ella is an unhappy man as his Leinster side slumped to their fourth Celtic League defeat of the season at Arms Park, to David Young's Cardiff.
Out-half Nick Robinson couldn't provide the ideal start as his fourth minute penalty veered away from the Leinster right upright before opposite number Matt Leek also was unsuccessful with his first kick at goal. Robinson hit over the game's first points soon after with a penalty before fiery blindside Baugh bustled
Lansdowne prop John Lyne was sidelined nearing the end of the first-half with a suspected broken left ankle, Leinster's first injury of four on the night, as things went from bad to worse for Ella's charges.
Kicker Leek hit the upright with his second kick in first half stoppage time before he managed to open Leinster's account after the break with a well-struck penalty. The Blues stepped it up as a 52nd minute offload from lock Jim Brownrigg sent Baugh skipping over for his and Cardiff's second try.
Robinson converted and also added the points to the Blues' third try six minutes later. Winger Walne deservedly sneaked over in the corner as superb hands from the Welsh side gave the number 14 his first try of the season.
Leinster pressed for a try but were left short changed by a rugged Blues' defence, and the loss of flanker Aidan McCullen, scrum-half Brian O'Riordan and captain Leo Cullen to injury.
Full Detaila at: leinsterrugby.ie