Bismihi ta'ala
As long as you are not "clean shaven", that's fine. That is the minimum requirement from a shar'i perspective. According to A.U Seyid Sistani, according to ihtiyat, it is haram.
It is good to trim your beard, and make it look presentable, but you should not shave it. Although shaving could help it grow, but in cases, you become too hairy, and you don't want that as well.
Just try to fix up your facial hairs, so your family might leave you alone and not persist on this.
And Allah knows best