Yes, I believe you are going to have to remove that cotter pin and remove the pivot bolt.
Reason being is that, unless you fell over and bent something, the likely reason you're having a stiff brake pedal is that the grease that was once lubricating your pedal and pivot bolt junction is now no longer in the building.
The pivot bolt has a shouldered length to it that is a wee bit wider than the brake pedal so you should be able to torque that bolt down properly and not bind the brake pedal movement.
Having to run it loose is a good indication that you need to remove the pivot bolt for an inspection.