The Ulstermen were unable to build on this lead though, and Dan Biggar put Ospreys ahead with six well-struckpenalties, while they added further scores through a penalty try and an effort from Richard Fussell.
Ulster got a consolation score at the end through a penalty try, with young replacement James McKinney landed the extras on his debut.
"There were a lot of young guys out there and that will have been an experience for them. There are good days and bad days when you play and for most of them so far there have been a lot more good days than bad," he said.
"This will have been a very steep learning experience. When you come to a place like the Ospreys, you've got to be squeaky clean with your discipline, organised with the ball, put it in the right areas of the pitch and make sure you put a team like the Ospreys under pressure.
"You can't allow yourself to get blown away in and around the contact areas. Those were the areas where we were guilty of underachieving."