It comes just months after Scott Morrison, and then Foreign Minister, Julie Bishop, ruled out moving the embassy, and former PM, Malcolm Turnbull, refused to walk away from the long-held policy.
Morrison told news site The Australian on Monday that the recognition of Jerusalem had previously been discussed by top cabinet members, including former foreign minister Julie Bishop, after Trump announced the U.S. recognition of the Israeli capital last December.
He described proposals to recognize Jerusalem and move Australia's embassy there as "sensible" and "persuasive" and said they would be considered by the government.
Trump's move ruptured decades of global consensus that Jerusalem's status should be settled as part of a two-state peace deal between Israel and the Palestinians.
Israeli Prime Minister Benjamin Netanyahu tweeted his approval on Monday.
"We're committed to a two-state solution, but frankly it hasn't been going that well, not a lot of progress has been made, and you don't keep doing the same thing and expect different results", Morrison said.
Sharma, who was ambassador to Israel between 2013 and 2017, is on Saturday standing as a candidate for Morrison's ruling Liberal Party in a crucial by-election in Sydney.
But the Labor opposition's Senate leader, Penny Wong, said Mr Morrison was playing "dangerous and deceitful word games with Australian foreign policy".
"This is an irresponsible policy that compromises the future of millions of people in the Middle East for a handful of votes in Wentworth", Browning said in a statement.
Labor is concerned the approach could undermine the prospect of a two-state solution between Israel and Palestine.
"I have made this decision without any reference to the United States", he said. "It has not come up in any discussion that I have had with the president or officials".
As part of the policy re-think, the Prime Minister has also announced a possible hard line stance against Iran, and saying "without prejudice" Australia will re-examine its support for the Iran nuclear deal, abandoned by the United States. "We do so in our own national interests consistent with our own beliefs and our own values".