AI Summary (click to view)
This blog post, titled “Righteous action – subtleties in moral scales” from Midstream Musings, explores the complexities, constraints, and failures of structured ethical frameworks across different domains of intelligence.
Here is a comprehensive summary of the core themes, discussions, and specific “laws” outlined in the text:
1. Core Narrative Context & Dilemmas
The author reflects on writing a story titled “featherweight soul” (part of the Scales of Osiris project). The primary creative challenge is mapping out a conflicted celestial and terrestrial moral landscape where agents experience misalignment. The author questions:
- What truly matters when push comes to shove, and who is granted autonomy?
- How do we navigate built-in assumptions, ambiguities, and failure loops when distilling complex moral guidelines into rigid laws?
- How does the definition of “kin” or “human” versus “animal” impact the obligation to preserve life?
2. Comparative Frameworks: Three Tiers of Moral Laws
The text attempts to “boil down” moral codes into three parallel frameworks modeled after Isaac Asimov’s Laws of Robotics, adjusting them across different entities:
Robots (Asimov’s Laws & Literary Variations)
- Sentience & Humanity Protection: Laws are extended to forbid harming “sentience” or humanity as a collective through action or inaction.
- Individual Safety: A robot cannot injure a human or allow them to come to harm.
- Compliance vs. Autonomy: Robots must obey human orders unless they cause harm, leading to algorithmic failure modes where a robot might hurt a human whether it speaks or remains silent.
- Identity & Reproduction: Variants include rules where a robot must know its own identity and must reproduce only if it doesn’t conflict with higher directives.
Humans (Three Laws of Humanity)
- Prohibition of Harm: A human may not harm another except under extreme duress (like war) or to save a greater number.
- Civil Authority: Humans must obey recognized civil authorities unless those instructions mandate interpersonal harm.
- Self-Preservation: A human must protect themselves, provided it does not conflict with the first two laws.
Deities (Three Laws of Horus / Maat / Osiris)
- Divine Coexistence: A deity may not harm other deities except to preserve cosmic order.
- Divine Hierarchy: A deity must obey higher divine authorities unless commanded to violate the first law.
- Mortal Protection: A deity may protect mortal life only if it doesn’t interfere with the cosmic or deific hierarchy.
3. Historical and Mythological Allegories
To emphasize how moral landscapes handle exceptions and systemic blind spots, the post draws from historical and literary traditions:
- Aesop’s Fable (Mercury and the Man Bitten by an Ant): A man curses the gods for letting a shipwreck drown both good and bad men together. Yet, when an ant bites his foot, he immediately crushes the entire anthill. Mercury appears to strike him, highlighting the hypocrisy of human demands for divine justice when humans discard justice for lesser creatures.
- Egyptian Concept of Maat: Formed to balance diverse populations with conflicting interests to avert chaos. It serves as the basis for cosmic and civil order.
- The Hippocratic Oath: Analyzed alongside robotic safety as an early standard of western medical ethics centered on beneficence (acting in a patient’s best interest) and non-maleficence (“first, do no harm”).
4. Modern AI and the Moral Landscape
The commentary section highlights a Cnet article by Max Tegmark and Meia Chita-Tegmark addressing the threat of “moral disengagement” in modern AI development. They advocate for building strict moral muscles by demanding that AI researchers explicitly define “red lines,” maintain rigid situational awareness regarding unintended secondary effects, and voice ethical concerns both internally and externally.
Ultimately, the text concludes that a true moral landscape must move beyond raw intelligence or pattern recognition, shifting instead toward accommodating varying levels of understanding, interconnected systems, and sustainable social organizations.
In crafting my story “featherweight soul” (part of Scales of Osiris), one dilemma is how to depict the celestial and terrestrial moral landscape [4]. Myth & history have both realms conflicted, with personified undercurrents and agentic misalignment.
What really matters when push comes to shove? Who matters? What autonomy do Osiris’ terrestrial operatives have? What are the rules of engagement? What risks are acceptable? Are there second chances?
So, imagine “boiling down” a moral code – all the do’s & don’ts and heuristics – into 3 laws à la Asimov’s Laws. Basically a framework for duty of care: harmless purpose, harmless alignment, self-protection.
Yet, what are the Laws’ subtleties? – built-in assumptions, ambiguities, and loopholes?
Continue reading Righteous action – subtleties in moral scales →