Television footage showed the world No.1 arriving at Park Hotel in Carlton on Thursday morning after his visa was refused on grounds of insufficient evidence, ahead of the Australian Open.

The 20-time grand slam champion has launched legal action in a federal court to challenge the decision, and remains inside pending the outcome.

By the afternoon, supporters draped in Serbian flags danced outside the Swanston St hotel watched on by a sizeable media congregation.

The hotel has a chequered history.

For more than a year, the facility has housed detainees flown to Australia from Manus Island and Nauru for urgent medical care under now-defunct Medevac laws.

The detainees had previously been held at the Mantra Hotel in Preston.

More than 50 were freed on bridging visas in January last year, but about a dozen remained behind.

There was a COVID-19 outbreak at the hotel in October, with at least 22 of the 46 refugees and asylum seekers testing positive.

Two weeks ago, two fires broke out on the third and fourth floors. A person was taken to hospital with smoke inhalation and police said a staff member was assaulted.

Refugee advocates estimate about 32 men are detained there.

Protesters scaled the hotel facade about 2pm on Thursday, holding banners and signs calling for the refugees' release.

"These people were held by Australia as maritime arrivals for seeking asylum," advocate Jane Salmon said in a statement.

"They have been held nine years. Perhaps the empathy of a man of Novak's profile will help them obtain freedom."

Before it was turned into a makeshift detention centre, the Park Hotel was used as a quarantine hotel in 2020.

Ninety per cent of cases from Victoria's second wave - which led to 18,000 infections, 800 deaths and a lockdown that lasted 112 days - were linked to an outbreak at the hotel, the state's hotel quarantine inquiry was told.

It was then known has the Rydges on Swanston but has since been rebranded.

Australian Border Force has been contacted for comment.

Australian Associated Press