But Mark Anscombe's men took the verdict thanks to a late penalty try and the superior goal-kicking of Niall O'Connor, who finished with 17 points.
The recently-formed Italian franchise - without a win in the RaboDirect PRO12 to date - will rue 12 points missed by the boots of their half-back pairing of Tito Tebaldi and Daniel Halangahu.
This round 4 fixture was rescheduled from September as a mark of respect for Ulster centre Nevin Spence, who tragically lost his life in a farming accident.
With several key players lost to injury and half a team's worth involved in the Ireland XV match against Fiji on Saturday, a number of Ulster's fringe players were given the opportunity to shine in at the Stadio XXV Aprile.
After a slow opening to the game from both sides, Ulster broke the deadlock after 10 minutes courtesy of a close range penalty from O'Connor, but the Zebre reaction was immediate.
Andries van Schalkwyk rose highest to collect Halangahu's restart and swiftly supplied Tebaldi, who ran over unopposed before sluggish Ulster had a chance to react.
Halangahu's conversion attempt was not quite as accurate as his restart had been, yet Zebre continued to make the running for the next 10 minutes.
They stole an Ulster lineout early in the second quarter to apply some extra pressure, before solid defence from the visitors allowed scrum half Michael Heaney to clear his lines.
A promising Ulster break then came to an end when Chris Cochrane just ran out of room on the left wing, but with Zebre penalised for coming off their feet in the 27th minute, O'Connor nudged his side back in front with the boot.
Once Tebaldi had narrowly missed a penalty attempt from halfway, Christian Gajan's side appeared to run out of steam temporarily, with O'Connor adding to his tally in the 34th minute after sloppy Zebre defending.
As half-time approached, however, the hosts' rhythm returned and only some determined defending from Roger Wilson on his own try-line prevented the concession of a second touchdown after strong ball-carrying from the Italian forwards had made the try look an inevitability.
There was just enough time though for Halangahu to knock over a penalty just seconds before the break and bring the difference down to a single point - 9-8.
A crucial missed tackle early in the second period, following Tebaldi's clever reverse pass, led to Halangahu pulling off a sharp dummy and racing clear for the corner and a terrific 44th minute try.
The out-half just failed to convert his own score, however, leaving Ulster only four points adrift, three of which O'Connor gratefully clawed back shortly afterwards with a penalty from outside the 22.
But Ulster continued to make life difficult for themselves, firstly with the concession of a 52nd minute penalty which Halangahu hit the post with.
Zebre were first to the loose ball and from a subsequent lineout, powerful South African van Schalkwyk barged past Michael Allen to grab try number three for the hosts.
Halangahu's conversion was again narrowly off target, leaving the gap at 18-12 when more accurate goal-kicking from the Italians would have closed the game out by this stage.
Ulster's response was patient and composed, as they moved the ball from left to right after strong mauling metres from the Zebre line. Allen then capitalised on a marginal gap to put experienced centre Paddy Wallace in for a crucial try on the right.
O'Connor was unlucky to see his conversion hit the woodwork, but with Italian centre Mario Pratichetti sin-binned for not rolling away in Ulster's very next attack, the out-half atoned for the miss with his fifth successful penalty.
Poor restart work again by Ulster put them right back under pressure and influential number van Schalkwyk soon drove over for his second try and Zebre's bonus point score.
He struck from a five-metre scrum awarded after Heaney had carried the ball over his own try-line. Halangahu converted to leave Ulster 25-20 behind with a quarter of an hour remaining.
The province's first attempt to salvage the match in the last 10 minutes amounted to nothing as they were left ruing an unfortunate knock on.
But the second - from a scrum just outside the Zebre five-metre line - proved more successful, as referee Neil Patterson awarded a penalty try after adjudging that the Italian front row stood up under pressure from the Ulster drive.
O'Connor added the extras from in front of the posts to edge his side two points clear, and put them within reach of their tenth straight win in all competitions.
With Zebre now frantically scampering to find points from any quarter, Ulster held their nerve and their discipline over the remaining minutes to leave Parma with a fortuitous victory and their unbeaten record intact.
|Zebre Score Card|
|Dries van Schalkwyk||2||10|
|Ulster Rugby Score Card|