Add your length below the armhole/above the waist and again below the waist to hem. There's no need to let it affect the armhole.
Length added/removed below the chest line will not affect the balance. It will only move the hem line up or down.
What I think I might try next is:
- Letting out the back yoke seam 1/4" (1/2" total added length). This will hopefully bring the collar closer to the back neck, and the shoulder seam closer to the middle of the shoulder. Of course, this assumes it will push the front forward and not the back down.
- Adding a second front wedge at the waist line. This will give more length on the CF line and keep the front closer below the waist.