Im renting a duplex and being hared?

im renting a duplex and i take care of my nieces in the morning while their dad works and he picks them up after work. my boyfriend comes over everyday to hangout. the landlord says that if he keeps seeing my nieces and boyfriend here he will ask us to leave? its been 1 month since he told me and now he told me again what can i do? is that against the law? i have no lease. if it is against the law what can i do?