If you’re considering stopping hormonal therapy because of side effects, talk to your doctor right away. Other side effects, like high cholesterol, lymphedema, and osteoporosis may take longer to subside or may continue indefinitely.