From Hawera, New Zealand, the 48-year-old coached Swansea RFC to success in the Welsh League (1998), Cup (1999) and the Welsh-Scottish League (2001) before taking the Wellington Lions to three play-off finals (2003, 2004 and 2006) in the New Zealand NPC, and guiding the Natal Sharks to two Currie Cup victories (2008 and 2010).
John was named South African Coach of the Year in 2012. He will join the Irish set-up at the start of September.
Commenting on the announcement, Joe Schmidt said: "It's great to have John joining us as part of the Irish coaching team. His proven success and broad experience across a number of competitions along with his ability to provide quality direction will add real value to the team."
John Plumtree added: "I am really looking forward to becoming part of the Irish coaching set-up. Having watched a number of recent Test matches there are some really positive challenges in front of us.
"I'm conscious of the responsibility we have and will be working hard to play my part in driving the team forward."