At a White House ceremony on Monday night, Kavanaugh said he was starting his new job without bitterness.
He was confirmed by the Senate in a 50-48 vote on Saturday.
The other eight justices are all in attendance for Monday's swearing-in, which is entirely ceremonial.
Kavanaugh officially took the oath in a more hurried, private procedure Saturday, but the White House version will be a chance for the Trump administration to celebrate publicly.
"What happened to the Kavanaugh family violates every notion of fairness, decency and due process", the president said in his remarks at the swearing in. "He's the one in his rallies have said things like this, I'd like to punch him in the face", said Waters.
He says that "under historic scrutiny", Kavanaugh was "proven innocent".
"If an opening comes in the previous year of President Trump's term, and the primary process has started, we'll wait until the next election", Graham said in remarks at the Atlantic Festival last week.
Any technical arguments are likely to be overshadowed by Kavanaugh's first public appearance wearing the traditional black robe donned by Supreme Court justices. McConnell said that threats of violence and murder were made against senators and their staff.
Some analysts said the court's reputation could suffer as it becomes perceived as a political, rather than a legal, institution. He emphatically denied the allegations.
At the mention of Feinstein, who is the top Democrat on the Judiciary Committee and the lawmaker whom Trump has accused of leaking to the media Christine Blasey Ford's allegation that Kavanaugh sexually assaulted her, the crowd here chanted "lock her up!" - a phrase that previously had been reserved for 2016 Democratic presidential nominee Hillary Clinton.
They point out he served for 12 years on the country's second-most important court, the U.S. Court of Appeals for the D.C. Circuit, where he penned more than 300 opinions, many of which were upheld by the Supreme Court.
And yet, it's quite plausible that installing a historically unpopular Supreme Court justice - whom most Americans regard as a sex offender - was, nonetheless, a political win for the GOP. Chuck Grassley of Iowa, the Senate Judiciary Committee chairman who led the contentious but successful fight to confirm Brett Kavanaugh to the Supreme Court.
Kavanaugh was accused of sexual misconduct in high school and college.
Alaska Sen. Murkowski was the only Republican who voted against advancing Kavanaugh's nomination to a full roll call, and she voted "present" on the final tally.
"The older the voter, the angrier they are about the treatment of both Kavanaugh and Ford, but those 40 and over are madder about how Kavanaugh was treated". Lisa Murkowski for voting against Brett Kavanaugh.