They said the gel, containing Aids drug tenofovir, cut infection rates among 889 women by 50% after one year of use, and by 39% after two and a half years.
If the results are confirmed it would be the first time that a microbicidal gel has been shown to be effective.
Aids 'key cause of female death'
African young 'change sex habits'
Such a gel could be a defence for women whose partners refuse to wear condoms.
New ways of curbing the spread of HIV are badly needed, particularly in sub-Saharan Africa, where nearly 60% of those infected with the virus are women.
Many women are often forced to take part in unsafe sex, and are biologically more vulnerable to HIV infection than men, making a gel they apply an attractive option.
Welcoming the results, UN agencies said they would convene an expert consultation in South Africa next month to discuss the next steps with the product.
The results of the three-year study, which was completed by the Centre for the Aids Programme of Research in South Africa (Caprisa), are being presented at an international aids conference in Vienna and were published on Monday by the US magazine Science.
The gel was found to be both safe and acceptable when used once in the 12 hours before sex and once in the 12 hours after sex by women aged 18 to 40 years.
Salim Abdool Karim, one of the two leading co-researchers, told reporters in Vienna that the 889 women involved in the trial, conducted in the coastal city of Durban and a remote rural village, had largely used the gel as directed.
They were also given condoms and advice about sexually transmitted diseases, and tested for HIV once a month.
After 30 months, 98 women became infected with HIV - 38 in the group that got tenofovir in the gel and 60 in the group that got placebos.
"We showed a 39% lower incidence of HIV in the tenofovir group," Dr Karim said.
Tenofovir, he added, lowered the risk of infection by 50% at 12 months but then the efficacy declined.
Women who used the gel more consistently were much less likely to be infected, he said.
He added that he did not know how much each dose would cost but said the applicators and gel cost "just pennies".
"Boy, have we been doing the happy dance," Dr Karim, from the University of KwaZulu-Natal in Durban, said.
'Hope for women'
"It's the first time we've ever seen any microbicide give a positive result that you could say was statistically significant," said Dr Anthony Fauci, director of the National Institute of Allergy and Infectious Diseases.
The researchers say women who used the gel also showed a significant reduction in genital herpes, a common sexually transmitted infection, which itself increases the risk of HIV infection.
The UN's HIV/Aids agency noted that nearly 20 years of research had gone into microbicides that can be controlled by a woman, independent of her partner.
"We are giving hope to women," said Mr Michel Sidibe, executive director of UNAids.
"For the first time we have seen results for a woman-initiated and controlled HIV prevention option."
A microbicide, he said, would be a "powerful option for the prevention revolution and help us break the trajectory of the Aids epidemic".
Dr Margaret Chan, director-general of the World Health Organization, welcomed Caprisa's findings.
"We look forward in seeing these results confirmed," she said.
"Once they have been shown to be safe and effective, WHO will work with countries and partners to accelerate access to these products."
Continue reading the main story
An easy-to-use microbicidal gel proven to significantly reduce the risk of HIV infection would be a very important breakthrough in the fight to control the spread of HIV/Aids.
The best way to minimise the risk of infection during sex is to use a condom - but this is not an option for many women around the world who find it difficult to insist that a man take the necessary precautions.
As a result, women have become particularly vulnerable to infection in recent years and in Sub-Saharan Africa, where the Aids pandemic is most severe, they make up nearly 60% of those who are infected with the virus.
To compound the problem, science has shown that women are biologically more at risk of infection than men.
An effective gel would finally give women the chance to do something to protect themselves from infection - to take control of their own sexual health.
However, several earlier trials have produced disappointing results, and even the results of the latest trial - impressive though they are - underline that a gel is far from a fail-safe form of protection.
The latest results also need to replicated in a much bigger trial.
But provided a gel could be made widely accessible to women in some of the world's poorest countries, where it is needed the most, it could help to transform many lives for the better.
Read all about it here
If this isn't just some wild/impossible fluke, this is probably the greatest win yet in the war on AIDS, and could point the way for greater things yet to come.