The black-and-white film was a chilling depiction of malice at a German village ahead of World War I. It offered a rare insight into the roots of the Nazi savagery.
He was handed the award by the Cannes jury president, French star Isabelle Huppert, who picked up the Cannes best actress award for her role in Haneke's 2001 film "The Piano Teacher."
Haneke "keeps the perfect distance from his subject matter," said Huppert. "He doesn't deliver messages, he just shows things in a very subtle way," she said.
Meanwhile, "A Prophet", a powerful prison drama by French director Jacques Audiard, garnered the runner up prize.
This year's better actor and actress went to Austrian Christoph Waltz and French-English Charlotte Gainsbourg for their roles in "Inglourious Basterds" and "Antichrist" respectively.
Filipino Brillante Mendoza was awarded the best director for his grisly crime drama "Kinatay."