A recent research paper, A new measure of issue polarization using k-means clustering: US trends 1988–2024 and predictors of polarization across the world, describes results researchers got from trying to measure political polarization over time and across countries. Trying to assess political polarization has been messy and error-prone. This paper describes the use of a data collection and analysis method they call k-means clustering (KMC). KMC appears to afford a better way to assess what issues are dividing Americans so bitterly and thus threatening our democracy, rule of law and civil liberties.
In this paper, scientists tracked polarization over time and compared it across countries. Instead of relying on whether people call themselves "liberal" or "conservative," or "Democrat" or "Republican," the researchers used KMC, a computer algorithm. The algorithm looks at how people actually answer a wide range of policy questions, then sorts them into groups of like-minded people from the ground up, without any named political labels. Polarization was measured by separation (how far apart groups are in their views), cohesion (how much people within each group agree with one another), and size (groups sizes compared). That is new in analysis of political polarization. Clusters of issues were assessed in the groups, e.g., abortion, gun control, immigration, gender roles, social values, etc.
The results are interesting. First, KMC analysis found polarization increased in the USA from 1988 to 2024, driven almost completely by a period of rising separation between clusters of beliefs from 2008 to 2020. Data from across the world indicated that mass issue polarization is driven primarily by disagreement over cultural issues, but that varied according to each nation's Human Development Index (HDI), a measure of health (life expectancy at birth), education level (years of schooling) and standard of living (income). Since 1988, the US has been more highly polarized compared to high HDI countries.
Most of the increasing polarization came from the groups moving farther apart from each other on issues. not from people in each group becoming more uniform, or from one side growing much larger than the other.
Data analysis indicated that people's opinions become more "packaged" or clustered. In 1988, someone with right-leaning views on the economy didn't necessarily oppose abortion. Today, positions on different issues increasingly go together in a bundle. If you know someone's view on one topic, you can more reliably guess their views on many others. Interestingly, data analysis also indicated that some of American society was liberalizing, with the left-leaning cluster being significantly more socially liberal in 2024 than in 1988, while the right-leaning cluster was rather stable becoming only slightly more conservative.
What that data and analysis does not capture is radicalization of the right-leaning cluster. It is only slightly more conservative on the issues analyzed, immigration, taxes, gun regulations, social spending, etc. But support for radical right authoritarianism, e.g., supporting strong leaders breaking laws to set things right, or loss of respect for democratic norms and institutions, is a different political dimension than what this research paper analyzed.
In other words, many or most conservatives (about half the country) generally hold about the same policy views they held in 2008 (abortion probably being an exception), but became more authoritarian in how they believe those policies should be implemented (another link). Meanwhile, about half of the US, more or less liberalized and retained their support for democracy and the rule of law.