Thousands of prisoners are being forced to work as porters for the armed forces - depriving them of food, and in some cases, killing them - the agency said.
And communities living along Burma's border with Thailand - where the military is fighting ethnic rebels - are being deliberately targeted by troops, the statement said.
Soldiers are accused of destroying food stocks, making arbitrary arrests and, again, in some cases killing civilians, forcing thousands to flee their villages.
All of these things, the ICRC said, have been witnessed by its staff, or documented by them in private interviews with civilians.
"Despite repeated entreaties by the ICRC, the authorities have consistently refused to enter into a serious discussion of these abuses with a view to putting a stop to them," Mr Kellenberger said.
"The continuing deadlock... has led the ICRC to take the exceptional step of making its concerns public."
Last October several ICRC offices were closed by the Burmese authorities, and ICRC delegates were forbidden to make prison visits.
The last time the ICRC spoke out this forcefully was about the genocide in Rwanda. They must think the situation in Burma is dire.