"Really ignorant," fumed Japanese tennis superstar Naomi Osaka in response to the derogatory comments made about women by Yoshiro Mori, the head of the Tokyo Olympic organizing committee.

Last week, the 83-year-old sparked a furious backlash by claiming that women "talk too much" and "get competitive" with each other in meetings, and Osaka's reaction perfectly summed up the public mood.

An online campaign demanding Mori's immediate dismissal has attracted over 140,000 signatures, and Asahi Shimbun, one of the biggest national newspapers in Japan, called for the Olympics chief's resignation in an editorial.

However, despite mounting pressure, Mori refused to step down and only acknowledged that his remarks were "inappropriate" before explaining that "I don't talk to women that much lately so I don't know."

With Mori's defiant response adding fuel to the fire and calls for his sacking increasingly multiplying, the International Olympic Committee (IOC) could not afford to stand idle.

On Tuesday, the organization posted a strongly worded statement, pointing out that Mori's comments were "absolutely inappropriate and in contradiction to the IOC's commitments and the reforms of its Olympic agenda 2020."

"Besides Mr. Mori's apology, the Tokyo 2020 organizing committee also considers his comment to be inappropriate and has reaffirmed its commitment to gender equality," the IOC added. "As the leader of the Olympic Movement, we are committed to our mission to encourage and support the promotion of women in sport at all levels and in all structures, as stated in the Olympic Charter."

The latest intervention, which stopped short of taking any further action against Mori, did little to tamp down the public fury.

According to Kyodo News, over 400 of the 80,000 people who signed up to volunteer at the Tokyo Games have resigned. "We are taking this very seriously," Olympic minister Seiko Hashimoto was quoted as saying on Tuesday when asked about the exodus of volunteers.

Japan's public broadcaster NHK revealed that two people have pulled out of the torch relay and around 4,000 people complained to organizers about Mori's behavior. Meanwhile, a poll at the weekend by Kyodo found that almost 60 percent of people thought Mori was "unfit" to lead the organizing committee, with just 6.8 percent saying he was qualified for the role.

The controversy has added to the problems facing the coronavirus-postponed Tokyo Olympics, which released its virus rulebook on Tuesday.

Under the guidelines, athletes will be tested for COVID-19 at least once every four days, and will be barred from competing if they return a confirmed positive test. Athletes' time in Japan will be "minimized to reduce the risk of infection" and those staying at the Olympic Village will be expected to "avoid unnecessary forms of physical contact."

"If you have been to the Games before, we know this experience will be different in a number of ways," the rulebook warns. "For all Games participants, there will be some conditions and constraints that will require your flexibility and understanding."

Around 80 percent of people in Japan oppose holding the Olympic Games as scheduled in July, according to a Kyodo survey in January.