To add to the possible causes/reasons that people have already replied with, I'd suggest that in countries with mandatory voting, there is a natural tension that keeps parties from straying too far from the centre (i.e. where most of the citizenry sit). In countries with optional voting regimes, those that feel disenfranchised or cynical or apathetic don't vote, while those with extremist views are probably less inclined to let such things get in the way of their vote, resulting in over-representation in the pool of actual voters (vs potential voters). This has led parties to chase the edges more so than pleasing the centre.
In Australia, we have mandatory voting, and while the media is still active and slowly moving the electoral needle, overall, attempts to introduce extremist positions have largely faltered (with a few exceptions). I don't expect this position will last forever though.
As someone that lives in Australia but comes from a similar country with optional voting (NZ), Australia's mandatory voting regime seems very much like choosing a government by chance - if voters don't know enough to vote without it being compulsory, they don't suddenly know any more by making them vote, and forcing a bunch of low information voters to go and vote anyway doesn't seem like it's going to get a good outcome. NZ's turnout is not all that much below AU without compulsory voting (admittedly, we did have John Key for what felt like forever, so it's not like we've got great form either).
While I'm moaning about the electoral system, the lack of a representative system (winner takes all in each electorate, leading to an enormous focus on battleground electorates while the safe seats are largely ignored) is also hugely problematic.
The founding principles of the USA prevent things like compulsory voting laws. It is illiberal, and contrary to the principles. I don't know what the founding principles of Australia are/ were, but I'm pretty sure it was a Commonwealth country until somewhat recently.
The unease from the industrial revolution, 2 world wars, and the Great Depression allowed much of the original American vision to be eroded for expediency. People who fail to understand history, are doomed to repeat it.
In Australia, we have mandatory voting, and while the media is still active and slowly moving the electoral needle, overall, attempts to introduce extremist positions have largely faltered (with a few exceptions). I don't expect this position will last forever though.