There was that, but there was also a large undercurrent of antisemitism in the population (Europe had been rife with it for a millenia) that Hitler was blatantly tapping into as well. In Mein Kampf he was very clear who he placed the blame on for Germany's fall and people elected him knowing full well that he was going to "do something" about the Jews and other "non-Aryan" groups (The Slavs were also on the hit list in Mein Kampf). The German's knew what was happening to the Jews, most of it was written into public law and it's hard to cover up something like Kristallnacht. And people in the towns surrounding the camps, many of whom worked in the camps, they knew what was happening in those places.