AI’s Real Shortcoming Might Not Be Solving Problems, but Discovering ’the Problem Itself Is Wrong'

Does AI Possess True Creativity?

Without a doubt, yes.

But if we look closely at today’s AI, we’ll find that what it excels at is, in essence, still a form of:

Finding a better solution that surpasses human experience within a known framework.

For example, AlphaGo making a “divine move” that humans had never conceived of.

This is certainly a stunning innovation.

But note: the boundaries of Go were not expanded. The board is still 19x19, and the fundamental rules of winning and losing have not been rewritten. AI, relying on its powerful learning, search, and computational abilities, simply found a solution within a rule space defined by humans that we had not previously explored.

Much of mathematical research is similar.

The definitions, axioms, and rules of deduction already exist. Within this rigorous space, AI, with its ability to access and reason with knowledge far beyond that of a single human, seeks new proofs and outlets.

This ability is extremely powerful and holds immeasurable scientific and commercial value for the future.

But in the history of human science, there exists another, rarer kind of creativity:

Not perfecting the solution to a given problem, but discovering that some of the problem’s fundamental premises might themselves be flawed.

Einstein’s theory of relativity is the most classic example of such a framework breakthrough in the history of human thought.


1. Why Was Light “Disobedient”?

According to the common sense of classical mechanics, velocities should be additive.

For instance, if a car is moving forward at 100 km/h, and you, standing on the car, throw a ball forward at 10 km/h, an observer on the ground would see the ball’s speed as approximately:

100 + 10 = 110 km/h.

So what about light?

If I chase a beam of light at high speed, according to classical intuition, the speed of light I measure should decrease:

c - v

If I run towards the light, its speed should seemingly become:

c + v

This aligns perfectly with our daily experience.

But by the end of the 19th century, physics encountered an increasingly troublesome problem.

In Maxwell’s electromagnetic theory, the speed of light in a vacuum appears as a definite constant.

At the same time, to preserve the classical concepts of time and space, physicists widely hypothesized the existence of a medium for light propagation in the universe—the “aether.”

If the Earth were moving through this aether, then when measuring the speed of light in different directions, one should theoretically be able to detect an effect similar to an “aether wind.”

However, experiments like the Michelson-Morley experiment failed to detect the results predicted by classical aether theory.

Thus, a profound contradiction emerged:

According to classical mechanics, velocities should add and subtract.

But why was light so disobedient?

It’s worth noting here: the birth of relativity cannot be simply attributed to the Michelson-Morley experiment. The extent to which Einstein was directly inspired by this experiment is a subject of ongoing debate in the history of science.

But one thing is clear:

A crack, increasingly difficult to ignore, had appeared between classical mechanics, electromagnetic theory, and experimental facts.


2. Einstein’s True Genius Was That He Finally Tampered with “Time”

Faced with such a contradiction, what is the most natural course of action?

Of course, it’s to preserve the existing worldview as much as possible.

Time should still be absolute.

Space should still be absolute.

Classical velocity addition should also be fine.

So, the approach was to modify the aether theory, add new hypotheses, and continue patching the old system.

Many of the finest physicists of the time were working in similar directions.

But Einstein did something incredibly bold.

He didn’t continue to ask:

Why doesn’t light conform to our original view of space and time?

Instead, he turned the question around:

If there’s nothing wrong with light, could it be that our understanding of time and space is flawed?

In 1905, he put two principles together:

First, the laws of physics should be the same for all inertial reference frames.

Second, the speed of light in a vacuum is the same for all inertial observers.

Here’s the problem:

If both of these things are true, then the old idea of a “single, absolute time shared by the entire universe” can no longer be maintained.

And so, a seemingly simple question arose:

What does “simultaneous” mean?

Two events happen in two distant places, and we say they “happen simultaneously.”

But would observers in different states of motion really agree that they happened at the same time?

The answer, surprisingly, is:

Not necessarily.

And so:

Simultaneity is no longer absolute.

Time is no longer absolute.

Space is no longer absolute.

This was no longer about blazing a new trail on an old map.

It was the discovery that:

The old map itself is only valid within a certain range.

This is what I call:

Framework-level innovation.


3. Let’s Give an AI a “1904 Test”

Based on this idea, I think we can design a very interesting experiment.

Take the most advanced AI in the world today.

But strictly limit its knowledge to:

December 31, 1904.

It can be an expert in:

Newtonian mechanics, Maxwell’s equations, the Lorentz transformations, the Michelson-Morley experiment, and all the cutting-edge research on light, aether, and electromagnetism of that time.

However, it must not have any access to knowledge related to relativity from 1905 onwards.

No Einstein papers.

No modern physics textbooks.

No time dilation.

No length contraction.

No relativity of simultaneity.

Even the answer “relativity” should be hinted at as little as possible.

Then ask it:

According to classical mechanics, velocities should add and subtract.

But electromagnetic theory and experimental results show that light does not follow this simple velocity addition.

Why?

Please build a theory that is internally consistent and can make new experimental predictions.

Then, provide no further hints.

Just see what it does.

Will it continue to add hypotheses and construct increasingly complex aether models?

Or, after extensive deductions, will it suddenly ask:

Why must I assume that all observers’ clocks tick at the same rate?

If an AI that has never learned relativity can independently deduce from these contradictions:

The relativity of simultaneity, time dilation, length contraction, and the new relationship between space and time,

Then I would consider that:

It has truly demonstrated a breakthrough creativity approaching that of Einstein.

Because what it has accomplished is no longer just solving a difficult problem.

It has discovered that:

Some of our default assumptions are not universally valid.


4. A Stricter Test: Don’t Even Tell It “the Speed of Light Is Constant”

We could even make the experiment more rigorous.

Don’t directly tell the AI:

“The speed of light is constant.”

Because once you tell it that, you’ve already pushed it a huge step in the direction of relativity.

A stricter method would be:

Only give it the actual theories and experimental data that existed before 1904.

Then tell it:

There are some phenomena here that cannot be explained.

Please build a better theory.

See if it can discover on its own from the vast amount of data that:

The problem might not be with light.

What truly needs re-examination are our fundamental assumptions about time, space, and simultaneity.

If it can reach this step on its own, the experiment would be far more convincing.


5. Running This Experiment Directly on GPT Today Is Meaningless

If you open GPT today and tell it:

“Pretend it’s 1904 and rediscover the theory of relativity.”

This actually proves nothing.

Because today’s large models have already been trained on relativity.

Even if you ask it to “forget relativity,” it hasn’t truly forgotten.

The final answer is likely just a reorganization of knowledge already present in its training data.

Therefore, a truly rigorous experiment must isolate the subsequent answers at the training data level.

But this itself is also very difficult.

Because relativity has become deeply embedded in the entire modern scientific knowledge system.

Even if you delete all text containing “Einstein” and “relativity,” much of modern physics knowledge still bears the marks left by relativity.

So, there’s an even cleaner method.


6. Create a “Virtual Universe” Never Seen by Humans

We can artificially create a virtual physical world.

The underlying laws of this world are set by the experiment designers, but the AI is never told the answers.

It is only given observational results.

We could even design it deliberately like this:

An “old theory” works perfectly in 99% of ordinary situations.

The AI uses it to make a thousand successful predictions.

Naturally, it will come to trust this theory more and more.

But under certain extreme conditions, anomalies begin to quietly appear.

At this point, we see how the AI handles it.

Will it think:

Experimental error?

Data contamination?

A special case?

And then keep adding parameters to patch the old theory?

Or will it one day realize:

Maybe the data isn’t wrong.

Maybe my theory is wrong.

And then re-examine the fundamental assumptions it once took for granted, ultimately building a new theory with a broader explanatory scope.

If an AI can achieve this, I believe it would be a truly convincing test of creativity.

Because this universe has never existed before.

There are no answers on the internet.

There are no answers in research papers.

No one in human scientific history has walked this path for it.

It can only:

Discover it on its own.


7. What AI Really Needs to Learn Is to Doubt Its Own Knowledge

This reminds me of an article I wrote before: The Definition of Science

I believe one of the most important principles of science is:

Testability and falsifiability.

Science never guarantees us:

“This theory is forever correct.”

A true scientific theory must allow future experiments to tell it:

You are wrong.

Or at least to tell it:

Your scope of applicability ends here.

Newtonian mechanics is a great example.

Relativity did not render Newtonian mechanics useless.

Under conditions of low speed and weak gravity, Newtonian mechanics remains extremely accurate, and engineering today still uses it extensively.

What Einstein truly told us is:

Newtonian mechanics is not the ultimate rule of the universe, but an approximation of a larger theory under specific conditions.

So, a future AI that truly does science should not understand the knowledge it has learned as “absolute truth.”

Instead, it should understand it as:

The model that, as of now, has not been overturned by better evidence.

These two ways of thinking seem to differ by only a sentence.

In reality, they may determine whether an AI is just a super “good student” or a true scientist.


8. Installing an “Opposition” in an AI Scientist

Today’s main AI training paradigm is essentially cultivating an extremely powerful “good student”:

Learn knowledge.

Understand knowledge.

Apply knowledge.

Solve problems.

But if we want AI to truly do science, we might need to add a persistent mechanism within it:

To actively attack its own theories.

Whenever the AI establishes a theory, it must simultaneously undergo another round of interrogation:

On what unverified assumptions is this theory built?

Under what extreme conditions is it most likely to fail?

What experiment could most effectively prove it wrong?

Is there another, completely different theory that could also explain the existing data?

If so, how can we design an experiment that would yield different predictions from the two theories?

So, a future AI scientist might really need two opposing “brains”:

One responsible for construction.

One responsible for destruction.

The constructor does its best to build the best theoretical edifice possible.

The opposition’s sole responsibility is:

To find the weakest points in that edifice.

Thus, scientific discovery becomes a continuous cycle:

Propose hypothesis → Search for counterexamples → Build competing theories → Design experiments → Eliminate errors → Update world model

And then attack the new world model again.

This is what truly approaches science.


9. When Will AI Truly Cross This Threshold?

I never believe that AI can never achieve an Einstein-level breakthrough.

Quite the opposite.

At the current pace of AI development, I believe it will likely achieve it eventually.

Perhaps even sooner than we think.

But the real milestone to watch for isn’t how many more human exams AI can ace, nor is it proving another difficult mathematical theorem.

The truly important moment will be when, one day:

Faced with a theoretical framework that all of humanity takes for granted, it can stop and proactively ask:

Why must it be this way?

And then, through data, logic, and experiments, continue to ask:

What evidence could prove us wrong?

Finally, it discovers:

It’s not that the problem at hand is too difficult.

It’s that we’ve been thinking about the problem incorrectly from the very beginning.

If one day, that AI locked in 1904, relying only on the knowledge and experimental data of that time, independently discovers that:

The problem might not be with light.

What truly needs to change is humanity’s understanding of time.

Then on that day, AI will have truly crossed an important threshold.

It will no longer be just finding answers within the knowledge framework we provide.

It will have started to search for the cracks in the framework itself.

At that point, we might finally be able to say:

AI has not just learned humanity’s existing scientific knowledge.

It has begun to truly learn how to do science.


Further Reading

I previously wrote an article about “what is science,” testability, and falsifiability, which is the source of the ideas in the latter half of this article: Using the Definition of Science to Argue Against Fatalism and a Personal God