Original Source: The Tails Coming Apart As Metaphor For Life | Slate Star Codex
In our study of morality, many systems of morality can reasonably agree on ordinary scenarios, but diverge greatly and fail completely when considering extreme scenarios.
For most ordinary scenarios, different schools (Catholic, utilitarian, etc.) will agree on ethical questions ("the morality of Mediocristan is mostly uncontroversial"), because our moral intuitions have been trained on the same data ("All moral systems were trained on the same set of mediocrestani data"). However, when considering extreme scenarios, like a post-singularity world, or one that includes transhumanism, our concept of morality itself can break down ("any moral rule followed to infinity will diverge from large parts of what we mean by morality").
a) Mediocrestani -- from Taleb; within the domain of the expected and ordinary. See Taleb's Concepts of Mediocristan and Extremistan