His last game in charge saw Ireland put in one of their best ever performances against one of the world's top ranked teams as they ran England extremely close in their final Six Nations game.
O'Sullivan also guided Ireland to wins over Italy and Scotland this season but the former Munster Junior player, who was captain of Cobh Pirates for seven seasons before going on to coach the club and more recently AIB League side Highfield, will not take the team to the Women's European Championships.
The European tournament is being held in Amsterdam next month.
O'Sullivan thanked his dedicated squad of players for their commitment over the past two seasons, saying: "I thoroughly enjoyed my time working with the national squad and am very honoured to have worked with a very dedicated and talented squad of players.
"I would like to wish the national team every success in the future and have no doubt they will go from strength to strength."
Irish Women's Rugby Football Union President Susan Carty added: "On behalf of the IWRFU I would like to sincerely thank John for his hard work and commitment to women's rugby over the past two seasons.
"John has spent many hours travelling to see players all over the country in cup and league competitions to watch players and he also spent time with all of the interprovincial squads during their preparation last year.
"The time and effort he put into his head coach position was truly inspiring and he has been a pleasure to work with. We would like to wish him well in the future."
One of O'Sullivan's assistant coaches, Simon Broughton, has been appointed as Ireland's interim head coach for next month's FIRA Women's European Championships.
The process to appoint the next coach will commence once this competition has finished. O'Sullivan will coach the Munster Youths team in their interprovincial campaign next season.