Presumably this is an Olds 307 and not a Chevy 307 given the list in your signature?
If so, then you may need to remove one of the bolts that allow it to rotate in the slotted bracket (or one of the brackets).
Whoever designed the power steering bracket for the Olds motors was an overachiever! When the world comes to an end, every Olds power steering pump will still be solidly in position!