Seat surgery?

[ripsass]

I just got a Corbin Gunfighter for my FXR and while I absolutely love it, there is one issue.
It has the option of installing a backrest into this little square shaft behind you.

Well this little shaft in the foam hurts the hell out of my back when accelerating hard or doing wheelies.

I won't ever use a backrest so I want to remove this post from inside the seat. Anyone know a good way to do it?
I already tried sticking a bar in it and prying it around a bit to see if it would break loose. No luck.

I hate to have to pull it all apart and reinstall the cover.