It said they planted an "explosive device" which blew up and "set the security fence on fire".
It said all the victims were males aged between 17 and 29.
Israeli military spokesman Lt. Col Jonathan Conricus tweeted that one group had "detonated a bomb on the Israel-Gaza border fence", allowing around 20 people to climb through the hole. It says its troops have used "riot dispersal means" and have fired "in accordance with standard operating procedures".
Addressing the unrest, Israeli Defense Minister Avigdor Lieberman chose to cut Qatari fuel shipments into the Gaza Strip in response to the mass protests.
Although the fuel was aimed at making the situation better in Gaza- where residents are now only receiving around four hours of electricity per day on average - the Qatar-bought delivery was protested by Palestinian President Mahmoud Abbas, as it was arranged with Hamas rather than his government.
One Israeli soldier has been killed by a Palestinian sniper during the weekly protests, and tracts of Israeli land have been scorched by incendiary kites and balloons.
The protests have regularly occurred on Fridays, with as many as tens of thousands of Palestinians gathering at multiple locations along the Gaza fence.
In recent weeks, protests have begun occurring on other days as well, ratcheting up the tension between Israel and militant factions in Gaza, led by Hamas.
A spokesperson for the Palestinian health ministry said six Palestinian men were killed as thousands of protesters approached the heavily-guarded Israeli border.
Israel asserts it is protecting its sovereignty from violent rioters.
The last few months have also seen Gaza militants fire more than 100 rockets and mortars into Israel, which has responded with dozens of air strikes.
Hamas chief Ismail Haniyeh, addressing a conference in Istanbul from Gaza, said on Friday that Hamas was seeking to reach understandings with several parties, including Qatar, Egypt and the United Nations.