The Alchemical Nature of AI
Artificial Intelligence (AI) has become a cornerstone of modern innovation, but its current trajectory raises important philosophical and scientific questions. The way AI, particularly machine learning (ML), is practiced today draws a striking parallel to the ancient and discredited discipline of alchemy. Alchemy, a precursor to modern chemistry, was once a respected form of inquiry that sought to transmute base metals into gold and uncover a universal elixir. Despite some useful discoveries like metallurgy and glassmaking, alchemy ultimately failed due to its lack of scientific rigor and understanding.
Similarly, many contemporary AI systems operate in ways that are effective but poorly understood, relying on complex mathematical functions that achieve impressive results without offering transparency. This has led some experts to liken AI to a new form of digital alchemy.
Machine Learning: A Modern-Day Alchemy?
In 2017, Google AI researcher Ali Rahimi famously compared machine learning to alchemy, sparking debate within the scientific community. Though criticized by AI pioneer Yann LeCun, Rahimi’s analogy still resonates. Despite significant advancements, the essence of the critique remains: many AI models function effectively without offering insight into how or why they work.
These models often operate as “black boxes”—systems that take input and produce output without revealing the inner logic driving their decisions. In practice, this means AI can outperform humans in tasks like image recognition or game-playing but lacks explainability. When decisions are made, especially in high-stakes environments, the inability to understand the ‘why’ behind them becomes a critical flaw.
The Explainability Crisis
A 2024 McKinsey survey revealed that 40% of respondents identified explainability as a key risk in adopting generative AI. An earlier survey conducted by Fair Isaac Corporation in 2021 found that nearly 70% of professionals couldn’t explain how specific AI decisions were made. This widespread opacity has real-world consequences, particularly in critical fields like healthcare and law, where understanding the rationale behind a decision is paramount.
To illustrate, consider an AI model trained to predict cancer risk using historical patient data. While this model may achieve high predictive accuracy, it often fails to disclose the specific factors influencing its decisions. This lack of transparency creates a gap between the model’s effectiveness and its trustworthiness. Users seek a causal explanation—rooted in science and logic—but find only statistical correlations.
The Data Generating Process vs. Statistical Regularities
At the heart of the issue lies the difference between identifying statistical regularities and understanding causal mechanisms. AI models are designed to excel in the former—they detect patterns in data and optimize for predictive accuracy. However, they do not aim to uncover the underlying rules that govern the real-world phenomena generating that data, often referred to as the “data generating process.”
This is akin to recognizing a mountain by its shadow rather than its actual structure. While the shadow offers clues, it does not provide a complete or accurate picture of the mountain itself. Similarly, AI models often latch onto superficial indicators that correlate with outcomes, without grasping the deeper causality. This approach may work for repetitive, low-risk tasks but proves inadequate in dynamic, high-stakes environments.
Implications for High-Stakes Domains
The lack of explainability is particularly problematic in areas like medicine and law. In healthcare, diseases may have multiple causes, and treatments can vary accordingly. If an AI model cannot justify its diagnosis, clinicians cannot determine the most appropriate course of action. In legal contexts, reliance on opaque algorithms raises ethical and procedural concerns, especially when human lives and freedoms are at stake.
While some argue that AI is best suited for simple, repetitive tasks, this perspective contradicts the significant investments being made in AI research and development. If AI is to be more than just a digital assistant, it must evolve to become a tool capable of contributing to complex decision-making processes.
Toward a More Transparent AI Future
To make AI a reliable and transformative force in society, it must become more explainable. Achieving this goal requires a long-term commitment to developing models that not only perform well but also offer insights into their reasoning. This involves rethinking AI design to prioritize interpretability alongside performance.
Such advancements would make AI more effective in areas like public policy, where decisions must be justified with empirical evidence and logical reasoning. Transparent AI systems would not only improve outcomes but also enhance public trust and accountability. In this context, the substantial investments in AI would be fully justified, leading to more meaningful and sustainable applications.
In conclusion, while today’s AI may resemble alchemy in its methods, the future lies in transforming it into a true science—grounded in transparency, causality, and understanding.
This article is inspired by content from Original Source. It has been rephrased for originality. Images are credited to the original source.