God, isn't this just the core of addiction/compulsive behaviours in general? I genuinely believe it is. I had an eating disorder because I feared I enjoyed food too much, and that if I didn't restrict myself I would "lose control" and become obese.
Now that I've recovered I realise my own desires are actually very reasonable and I can trust myself.
I also worried recently whether I like marijuana "too much" and if I didn't place restrictions on myself I would "lose control and become addicted" and then started wondering "does the fact I have thoughts about weed on my off days mean that I am in fact addicted" which of course most of the thoughts are actually "oh god am I thinking about weed now what does it MEAN is it a bad sign am I at risk of becoming addicted".
Then I went to visit Australia again over the holidays and I had some weed in the house for over a week with no desire to take it. Because I had a lot of other things I preferred to do while I was there.
So I actually think my relationship to weed is pretty healthy but the fear of becoming addicted was what actually was leading to more compulsive thoughts about it.