Rogge noted that South Africa had previously hosted a very successful All Africa Games (in 1999), a Rugby World Cup (in 1995) and international cricket championships (including the inaugural World Twenty20 in 2007).
"Now it is up to you as a nation to decide if you want to host this event."
At the same briefing, South Africa's deputy sport minister Gert Oosthuizen said the South African government had already shown interest in bidding for a future Olympics and would be in discussions with SASCOC about the 2020 and 2024 Games.
"I think it's safe to say that we have expressed interest in hosting an event of this magnitude that will leave legacies," he said.
Last week authorities in the South African port city of Durban expressed interest in staging the Olympics.
Following Beijing's successful hosting of the 2008 Olympics, London will be host in 2012, and Rio de Janeiro in 2016.