I think you make a smart point. Some people do suck time/effort etc and they don't give back.
It's not about keeping score, I don't mind giving time and energy and ideas to anyone, but people close to me - I want them not to suck. I want them to mirror my values.
Fortunately they are few and far between.
If you let them in and let them close, they eat away at your perspective and that's no healthy