Now the reigning champions return to Le Stadium on Saturday for their semi-final showdown against three-time Heineken Cup winners, Toulouse, knowing only a repeat performance of four years ago will be good enough to take them to the Stade de France on Saturday, May 22 and back-to-back tournament finals.
"That quarter-final against Toulouse is a great memory for sure - along with last season's final against Leicester Tigers and the semi-final with Munster, it has to be in the top three," said the 35-year-old.
"No one expected us to do anything (against Toulouse in 2006) but we had a certain belief in ourselves and we managed to pull off a truly massive performance.
"Now we have to repeat that and I don't think history will do it for us, we will require a whole new approach for what will be a totally different game.
"There are a number of players still around who played that day, so they will know we can do it but at the same time they know there is no guarantee of us doing it again.
"Basically we are a changed team with a different structure from 2006."
This will be the eighth Heineken Cup meeting between the teams with Toulouse 4-3 ahead. O'Kelly pinpointed former All Black scrum half Byron Kelleher and French internationals Thierry Dusautoir and Yannick Jauzion as the main dangermen among a star-studded line-up.
"Toulouse are obviously one of the truly great European sides, along with the likes of Munster and Leicester Tigers," he added.
"They have a tremendous history and tradition and consistently do well in the Heineken Cup, as well as being one of the most professional outfits in Europe.
"They are littered with French stars and also have some great players like Byron Kelleher and their Samoan prop Census Johnston.
"Yannick Jauzion and the likes of Thierry Dusautoir are instrumental in how they play as a team behind a very strong scrummaging front five.
"Toulouse are real class, dangerous and strong in all areas and we will have to be tactically aware of their strengths and really do our homework on them.
"It was interesting that in the quarter-finals, Stade Francais were really competitive for around 55 minutes but then Toulouse confirmed they have players who can turn a game on its head in a flash and when they are rampant like that, there is nothing much you can do but just try and hang in there.
"They are also the kind of side who realise the importance and prestige of the Heineken Cup and take it more seriously than most of the other French sides.
"Their fans always create a fantastic atmosphere at the ground but we certainly hope that a certain amount of blue will follow us this weekend and get behind the team."