BAYONNE 15 ULSTER 20, Stade Jean Dauger
Scorers: Bayonne: Tries: David Roumieu 2; Con: Jacques-Louis Potgieter; Pen: Cedric Garcia
Ulster: Tries: Michael Heaney, Neil McComb; Cons: Niall O'Connor 2; Pens: Paddy Jackson, Niall O'Connor
Second row Neil Mccomb raced through for the winning score three minutes from time to ensure that Mark Anscombe's reign as Ulster head coach began on a winning note.
Bayonne also have a new coaching team and both sides used multiple replacements in the strength-sapping heat as Ulster wrapped up their brief tour to the south of France.
Ulster turned down a few kicks at the posts in the early stages in favour of going to touch but they were unable to get through the home defence.
Eventually, Paddy Jackson opted for a kick at the posts after 15 minutes and he made no mistake to give Ulster the lead from 43 metres out.
But Bayonne, still smarting after failing to score against neighbours and arch rivals Biarritz last weekend, responded with a series of good drives and they crossed after 27 minutes when hooker David Roumieu barged over, gaining some reward for several close range scrums.
They repeated the dose just before the break after Ulster tighthead prop Adam Macklin was sin-binned as Ulster conceded a succession of penalties in front of their own posts.
It was hooker Roumieu who again found the gap to cross and leave the French club 12-3 in front at the break.
Ulster got on top after the restart and reduced the deficit when scrum half Michael Heaney cut a great line to cross after 49 minutes. Another player returning to the Ulster fold, replacement Niall O'Connor, added the conversion.
O'Connor was off target with a penalty from inside halfway but then pushed Ulster in front with a 46-metre kick.
Cedric Garcia restored Bayonne's lead but then McComb burst through in the dying minutes to seal a deserved victory for last season's Heineken Cup runners-up.
BAYONNE: Scott Spedding; Joe Rokocoko, Thibault Lacroix, Manu Ahotaeiloa, Marvin O'Connor; Jacques-Louis Potgieter, Mike Phillips; Aretz Iguiniz, David Roumieu, Neemia Tialata, Dewald Senekal, Abdellatif Boutaty, Mark Chisholm, Julien Puricelli, Dwayne Haare.
Replacements used: Gregory Arganese, Pierre Lafond, Walter Desmaison, Marc Baget, Cedric Garcia, Matthieu Ugalde, Cedric Heymans, Vitolio Manukula, Robin Linde, JJ Marmouyet, Sam Gerber, Bastien Fuster, Guillaume Bernad.
ULSTER: Peter Nelson; Craig Gilroy, Jared Payne, Chris Farrell, Chris Cochrane; Paddy Jackson, Blane McIlroy; Callum Black, Niall Annett, Adam Macklin, Johann Muller, Neil McComb, Mike McComish, Sean Doyle, Robbie Diack.
Replacements used: Rob Herring, Richard Lutton, John Afoa, Lewis Stevenson, Ali Birch, Michael Heaney, Niall O'Connor, Michael Allen, Luke Marshall, Ricky Andrew.
Referee: Romain Poite (France)