Husband told me he swiped my sisters panties?

Wow that is some crazy stuff. I have never known guys to do such a thing and would rather keep believeing that your husband is the only one that does this LOL. I would talk to him tell him you are not confortable with this and you need him to stop. And I really dont know how you can forget something like this but I would try if not leave.