Maybe he should. Since Panetta took over the CIA he seems to reflexively be defense of the agency's past - possibly illegal -activities. At every turn he has opposed efforts by the White House and Congress to uncover what was done during the Bush administration.
We can't ignore this stuff. The past has shown us that when the CIA goes rogue it ends up hurting us all, departing from its traditional mission and it success at that, moving into the sort of immoral activity that increases distrust of America from the outside and inside.
If Director Panetta is trying to obstruct that, perhaps he should step down. And people thought a guy from outside the national security club would be more interested in letting some sunshine in.