Police deployed tear gas and water cannons after scuffles broke out in Brussels on Sunday as thousands of people answered a call by far-right groups to reject a UN migration pact.

Police said some 5,500 people rallied in Brussels' European quarter housing a number of EU institutions. Some held aloft banners bearing slogans including "Our people first" and "We have had enough, close the borders."

Police stepped in to disperse the crowd after projectiles and firecrackers were thrown, an AFP journalist on the scene reported. Some 90 people were detained, according to the Belga news agency.

Many marchers were supporters of the Vlaams Belang party, which opposes last week's signing of the UN accord in the Moroccan city of Marrakesh.

The agreement is designed to frame an effective international approach to migration, a deeply divisive issue across Europe and beyond.

A counter-demonstration drew around 1,000 people to another Brussels district, according to police.

Belgian authorities initially banned both rallies fearing unrest but the country's supreme administrative court, the Council of State, overturned the decision.

More than 150 states last Monday approved the pact, a Global Compact for Safe, Orderly and Regular Migration.

But NGOs have voiced doubts about its implementation both on the ground and the high seas.

In Belgium, the issue has become a political hot potato.

Belgian Prime Minister Charles Michel was left leading a minority administration after the biggest party in his coalition, the Flemish nationalist N-VA, which held four ministries, quit over the accord.

Some protesters called for Michel to resign at Sunday's rally and a number of opposition parties have called for a vote of confidence in the government.