The move ends David Humphreys' 22-year association with his home province. He made 162 appearances for Ulster as a player and won the European Cup, Celtic Cup and the Celtic League.
He retired from playing in 2008 and took up a role as Ulster's operations director before becoming their director of rugby four years ago.
Commenting on the announcement, Humphreys said: "This was a really difficult decision for me personally but having received a direct approach from Gloucester, I have now chosen to accept an offer to become their director of rugby. It is, I believe, the right move for me at this stage of my career.
"I would like to thank the supporters and all those involved in Ulster Rugby over more than two decades who have lent me support, encouragement and wisdom.
"There is no doubt that the province will continue to move forward. It has a fantastic stadium, the best supporters and a wealth of great talent coming through. Ulster Rugby will be a force within rugby for many years to come."
Ulster Rugby Chief Executive Shane Logan added: "I would like to wish David every success with Gloucester and to thank him for the outstanding contribution that he has made to Ulster Rugby.
"As director of rugby, David has helped to grow the game at all levels - there are more people playing rugby in the province than ever before, we have successful age-grade sides, we have a strong Academy and we now have a senior squad that can beat the best in Europe.
"We are disappointed with the decision but respect David's decision to leave Ulster. We know we already have the structures in place to ensure we have success in the long-term.
"Ulster Rugby will now begin the process of recruiting a new director of rugby and we look forward to building on the progress that the province has made in recent years."