Fly-half David Humphreys - who had scored 52 of Ulster's 62 points in the previous four Rounds - was one of their heroes again, as he struck the match-winning penalty - his third of the night - seven minutes from time.
Full back Bryn Cunningham angled through for a 46th-minute try as the northerners won out thanks to some dogged defending and good decision-making in the final helter skelter minutes, picking off their 12th straight Cup win in Belfast.
Ulster were have been down 12-3 at half-time, as Nigel Melville's side dictated much of the first half territory, and crucially had Henry Paul in goal-kicking form as he struck three penalties in poor conditions.
England Sevens captain Simon Amor clipped over a 40th-minute drop goal to see the Cherry-and-Whites to their interval lead - but Ulster were unfortunate to lose out on a late try as wing Tommy Bowe had a seemingly legitimate score disallowed for offside, after collecting Humphreys' punt through.
Back in the hunt however, thanks to Cunningham's follow-up on another Humphreys kick and Bowe's neat collect and inside pass, Ulster almost allowed Gloucester an immediate reply as wing Marcel Garvey was unlucky not to pick up a loose ball cleanly with the Ulster line in sight.
Centre Terry Fanolua was also in the mire as he dropped the ball on the hour, with an evident Gloucester overlap out wide, but Humphreys' superb kicking - slotting two penalties in six minutes - with his first on 69 minutes, brilliantly hit from near the touchline, and second drawing the home side in front for the first time, saw Ulster victorious.
His opposite number Amor did have a late chance to steal the points, but his 20-metre drop goal crept wide, and in truth, on their second half performance, Gloucester hardly merited what would have been a fourth Pool win.
Ulster's hopes of making a first quarter final since winning the competition back in 1999 are slim - given they can only finish on 17 points. All eyes and ears will be at Kingsholm next Sunday, as Pool leaders Stade Francais (19 points) take on Melville's second-placed side (14).At the same time, McCall's charges take on bottom side Cardiff - who lost 35-16 to Stade on Friday - with nothing but a bonus point win on their minds at the Arms Park (kick-off 3.00pm).
Former Ireland Under-21 coach McCall expressed his satisfaction afterwards.
"12-3 down at half time - perhaps a couple of months ago, given our form at the time - I don't think we would have come back from that," he said."But we stayed calm in the second half and picked up points at the right time. It was tiring and we had a lot of defending to do, but to win three games out of five in a group that also includes Stade Francais, Gloucester and Cardiff is not too bad."
Heineken Cup: Pool 6:Ulster 14 Gloucester 12, RavenhillScorers: Ulster: Try: Bryn Cunningham; Pens: David Humphreys 3Gloucester: Pens: Henry Paul 3; Drop gl: Simon Amor
Replacements used: Paul Shields for R Best (46 mins), Andy Ward for Feather (60), Rowan Frost for Longwell (66), Paddy Wallace for Howe (69), Ronan McCormack for Moore (82). Not used: Neil Doak, Adam Larkin.
HT: Ulster 3 Gloucester 12; Attendance: 11,435Referee: Nigel Whitehouse (Wales)