The Prime Minister’s Office has announced the federal cabinet will retreat the day of Donald Trump’s inauguration.