Bayonne scored early after just eight minutes with a try and added a penalty on 16 minutes to lead 8-0.
Ulster out-half Niall O'Connor was on target with the boot shortly afterwards with a penalty, but Bayonne stretched their lead with two kicks on target to turn around at half-time with a 14-3 advantage.
At half-time there was a huge thunder storm which made playing conditions extremely difficult for both sides throughout the second half.
Bayonne opened their account in this period with another penalty, however Ulster ended the match strongly and scored a try on 79 minutes when number 8 Robbie Diack broke off the back of the scrum and dotted down.
The score was converted by O'Connor, reducing the arrears to 17-10.
Speaking afterwards, Ulster's Operations Director David Humphreys said that he was particularly pleased with the efforts of the younger players on show.
"However, I am really pleased with our performance and the young players in the team really fronted up.
"We played the first half of the match against Bayonne's 2nd XV and the second half of the game against their 1st XV, so it was a real test but our young team defended very well."
There were no no significant injuries or knocks for Ulster from the game and while those who played will no doubt be a little stiff and sore today, it is a rest and recovery day for them before they return home having completed a long pre-season regime.
On Monday, their attentions will turn to match preparations with the first proper pre-season friendly against Newcastle Falcons at Ravenhill in less than two weeks' time - tickets will go on general sale from Monday (August 10).
ULSTER (v Bayonne): Bryn Cunningham; Tommy Seymour, Mark McCrea, Jonny Shiels, Simon Danielli; Niall O'Connor, Paul Marshall; Justin Fitzpatrick (capt), Nigel Brady, Declan Fitzpatrick, Dan Tuohy, Neil McComb, Thomas Anderson, Willie Faloon, Robbie Diack.
Replacements: BJ Botha, Tom Court, Andy Kyriacou, Matt McCullough, Chris Napier, Cillian Willis, Jamie Smith, Clinton Schifcoske.