I feel that too, that my Aquarius child's father is such a rebel but a loner. We can see each other all week long which is always great but then he'll say something to the extent "I'll be back" and it'll be that he's over his brother's by himself or he just be in my finished basement by himself reading for hours and he doesn't want to be bothered. And I notice how he talks- such a loner, not a lot of friends maybe two, but yet when he walks into a room he grabs your attention and he is very friendly and can hold a conversation. At first I was doubtful of him cuz he needed so much space but he is honest to a tee, I'm the type of cap who desires attention but it not given but I now like the fact that we do allow each other space because by being attached alot can bore a relationship
its very possible that you have venus in aquarius and he has venus in capricorn giving the two of you some reciprocity.