I agree with trusting, and went through the same thing early on, too. Know that pointing out to him that he's being rude won't necessarily garner you the result of having him change his behavior - on the contrary, he may start being even ruder, as he will just think you are judging him or telling him what to do.
Treat it with the weight you would if any other person - one of your kids, a co-worker, or family member - did the same thing. Since he's still coming around often, you might save the important things for in-person contact, so you can more than likely get an immediate response.