Later 10 Tibetan activists staged their own protest in the town's main street before they were detained or chased by police.
Three members of Reporters Sans Frontieres (Reporters Without Borders -- RSF) staged the first protest as the chief Chinese Olympics organiser, Liu Qi, who is Beijing's communist party chief, made a speech before the flame was lit.
One man unfurled a flag declaring "Boycott the country that tramples on human rights".
Another tried to grab the microphone from Liu and shouted "freedom! freedom!" in front of International Olympic Committee (IOC) President Rogge and other top officials were sat.
Security officers quickly dragged all three away.
Greek police had imposed heavy security, including armed police watching down on the site from nearby hills.
Greek state television cut its live broadcast away from the protesters.
China's state broadcaster also quickly changed and did not mention the demonstrators.
Actors in ancient Greek costume then carried out the traditional ceremony, lighting the torch using a parabolic mirror to focus the sun's rays.
But as dignatories dispersed, about 10 Tibetan activists, covered in red paint, marched out of a hotel in Olympia and lay down in the town's central street, shouting slogans against China's rule in Tibet.
At least two were detained by police and the rest ran away.
Rogge said: "I think it's always sad when there are protests, but they were not violent and that's the most important thing.
Speaking before the ceremony, Rogge repeated his comments reported in insidethegames yesterday that "the major political leaders don't want a boycott."
He added: "There is no momentum for a boycott."
"(George) Bush doesn't want a boycott, (Nicolas) Sarkozy doesn't want a boycott, (Gordon) Brown doesn't want a boycott."
But he acknowledged that the torch relay across 20 countries - and Mount Everest and Tibet - might be hijacked.
He said: "Of course it's a concern.
"I would hope that potential protesters will understand that public opinion would not want the torch relay painted by political protests.
"It would be counterproductive."
In his speech at the ceremony, Rogge said the Beijing Games should be an opportunity for China and the world "to learn, discover and respect each other."
The lighting of the flame at the venue of the ancient Olympics launched a relay that marks the final countdown for each Games.
The journey to Beijing is the longest ever planned, lasting 130 days and covering 137,000 kilometres (85,000 miles) worldwide.
Most of it will be on Chinese soil however.
Aside from Athens, the flame will only stop in London and Paris among European capitals.
It will only stop in San Francisco and Buenos Aires in the Americas and just Dar es Salaam in Africa.
Upon arrival in Beijing, one flame will be separated from the torch and kept in a special lantern to be taken to the peak of Mount Everest during early May and then return to Tibet.
A crackdown on anti-Chinese protests in Tibet, which exiled Tibetans say have left at least 130 dead, has overshadowed the buildup to the Games.
Various rights groups have drawn up plans aiming to galvanise opposition to China's record on Tibet, Darfur, human rights, religious freedom and other issues in the run-up to the Beijing Games.
The Falungong group is running a rival torch relay to highlight the plight of its followers in China, who it says are subject to brutal persecution.
Dream for Darfur, an organisation set up to pressure China into helping end the bloodshed in the western Sudanese region, is planning protests along the torch relay route.
Thai environmental activist Narisa Chakrabongse, chosen to carry the Olympic torch when it crosses Thailand next month, has declined in protest against Beijing's crackdown.Pictures of the protest can be seen at http://www.youtube.com/watch?v=UzT0rw47-j4&eurl=http://
Picture by Alex Broadway/SWpix.com