The Supreme Court justice faced scrutiny from conservatives for voting against the president in a recent ruling.