Some leaders invited to a private meeting on Thursday have opted to boycott it, after the prime minister told them they should be doing more on the issue.

"I've got a general rule in life that you're always better talking than having a stand-off. It would be better if the leaders turned up. I don't think you solve anything by sitting in different rooms," Mr Shorten told the Nine Network.

"I also have to say though that most of the Muslim community are very law-abiding. What we're talking about is Islamist ideology. Let's not tar the whole sub-population with the actions of the crazy few.

"But it would be better if people turned up and spoke. I always think that's the best way to do it, that's how you solve the problem, sit around the table."

The meeting follows a deadly terror attack in central Melbourne.

A group of leaders, including Grand Mufti IbrahimAbuMohamed, signed a letter on Tuesday night telling Mr Morrison they won't be attending.

"ManyintheMuslimcommunity... aredeeplyconcernedand disappointed withstatementsmadebysenior government ministersandthe prime minister intherecentpast, whichinferthatthecommunityiscollectivelyculpablefor thecriminalactionsofindividuals andshouldbedoingmoretopreventsuch actsof violence," the letter said.

"Thesestatementshaveachievednothingtoaddressunderlyingissues,but rather,havealienatedlargesegmentsoftheMuslimcommunity."

Mr Morrison tweeted in response that the meeting would be going ahead, with those who want to seriously deal with the issue "rather than look the other way".

"Some have chosen to publicly boycott this meeting. Continuing down a path of denial only lets their communities down. It makes their communities less safe and more vulnerable," he wrote.

Mr Morrison told Islamic leaders they need to do more to combat extremism after a man stabbed another man to death in Melbourne's Bourke Street on November 11.

Labor MP Jason Clare, whose seat of Blaxland has a significant Muslim population, said the leaders should talk to Mr Morrison.

"If people have decided to boycott it I think that is a mistake. Take the opportunity and tell the prime minister what you think," Mr Clare told Sky News.

"Nothing beats dialogue. If you have got a difference of opinion then tell the person face to face, but never walk away from an opportunity to talk to the prime minister of Australia about the issues that you have got."

Australian Associated Press