Hopefully you can get a second opinion from a different psychiatrist. I have heard stories before of people having additional conditions to adhd, which get missed because adhd symptoms can overlap with other things. If the medication is causing your life quality to be lower than without it, this sounds like it’s definitely the wrong stuff and needs a review. Taking it should be helpful, not make things worse.