How are your wards? Are you banishing but not shutting the door behind it, as it were?
So me, being an asshole about this kind of thing, would attack it and try to kill it, until it decides it’s better to just fuck off permanently, or dies. But if you’re nicer than me, then you could try just destroying the emotional relationship connecting you that makes it want to be around you. For this I have my usual poppet method in that link there, but a binding might also work, a cord cutting, that kind of thing, after doing your regular method to banish.