Unfortunately, he's already put at the risk of being evicted. Even though he's not on the lease, he has taken up residence with you and the landlady could kick both of you out, with notice of course. The only legal way you can kick him out is to get an eviction notice yourself for him to move out.
Several years ago, an ex of mine moved over from Indiana to where I'm at now to try the job market here. After a few months, I was served notice by the landlord that we had to move. The reason, the other tenants were uncomfortable with him around. And he wasn't the type to cause trouble.