In that case, it seems unlikely to truly be unethical. The knowledge they have is not their own, but that they gained from others, so you are not stealing from them. And even invading their space is questionably unethical, as it does not present a danger, a trauma, or a true invasion of their person, just their mental space.