Australia and its ASEAN neighbours vowed to boost defence ties while stressing the importance of non-militarisation in the disputed South China Sea Sunday at a summit where the "complex" Rohingya crisis took centre stage.

Leaders from the Association of Southeast Asian Nations, at the three-day meeting in Sydney, also agreed to work more closely to tackle the growing menace of violent extremism and radicalisation.

Read more

Rohingya crisis: US to continue support to Bangladesh

But while a final communique noted a resolve to "protect the human rights of our peoples", it failed to condemn member state Myanmar's treatment of the Muslim-minority Rohingya.

Nearly 700,000 Rohingya have fled the troubled Rakhine state for Bangladesh since authorities launched a brutal crackdown six months ago that the UN has called "ethnic cleansing".

Myanmar, whose de-facto leader Aung San Suu Kyi was in Sydney, has vehemently denied the allegations.

"We discussed the situation in Rakhine state at considerable length today," Australian Prime Minister Malcolm Turnbull said at a closing press conference.

"Aung San Suu Kyi addressed the matter comprehensively, at some considerable length herself," he said.

"It's a very complex problem ... Everyone seeks to end the suffering that has been occasioned by the events, the conflict."

Singapore Prime Minister Lee Hsien Loong, said the crisis was "a concern for all ASEAN countries, and yet ASEAN is not able to intervene to force an outcome".

Security threat

Tensions in the South China Sea remain a big worry for regional leaders, as Beijing continues to build artificial islands capable of hosting military installations -- much to the chagrin of other claimants to the area.

Vietnam remains the most vocal in the dispute with the Philippines backing off under China-friendly President Rodrigo Duterte. Taiwan, Malaysia and Brunei also have claims.

Canberra and ASEAN reaffirmed "the importance of maintaining and promoting peace, stability, maritime safety and security, freedom of navigation and overflight in the region", without naming Beijing.

The leaders added they wanted to see an "early conclusion of an effective code of conduct in the South China Sea".

"We will uphold our commitment to the rules-based order and international law in the region, including the South China Sea," stressed Turnbull.

With China flexing its muscle, they also committed to enhancing "the scope and sophistication of defence cooperation", while expressing "grave concern" about escalating tensions on the Korean peninsula.

Day two of the summit on Saturday was devoted to counter-terrorism, with an agreement to work together to tackle extremism amid growing concern about the use of the "dark web", or encrypted messaging apps, by terrorists to plan attacks.

Fears have been heightened by jihadists now being forced out of Syria and Iraq with the Islamic State caliphate mostly crushed, and into other countries.

Malaysian Prime Minister Najib Razak cited the flow of the displaced Rohingya as a potential new security threat, with desperate people more susceptible to radicalisation.

Human rights issues were a key focus of protests during the summit, with thousands denouncing Aung San Suu Kyi, Cambodian strongman Hun Sen and Vietnam's Nguyen Xuan Phuc, who are accused of oppression.

With the Trans-Pacific Partnership free trade pact, without the United States, now signed, Turnbull urged leaders to get behind a "high quality" Regional Comprehensive Economic Partnership deal.

Australia, the full ASEAN bloc, as well as China and India are among countries still negotiating that deal.

Singapore's Lee said there was hope it could be finalised this year.

"This is a historic opportunity to establish the world's largest trade bloc," he said.

ASEAN groups Brunei, Cambodia, Indonesia, Laos, Malaysia, Myanmar, the Philippines, Singapore, Thailand and Vietnam, with Australia, a dialogue partner since 1974.