Introduction
The aspiration to build machines capable of exhibiting intelligent behavior has accompanied the history of computing since its earliest days. During the 1960s, when I began working with computers, the notion of creating an electronic brain—what we would now describe as an Autonomous Intelligent System (AIS)—appeared more as a long-term vision than an achievable engineering objective.
Over the following decades, advances in computer science, control theory, artificial intelligence, robotics, machine learning, and most recently foundation models and generative AI have transformed that vision into practical reality. As a software engineer who has spent many years developing systems that today could be classified as intelligent, I take pride in having contributed, in a modest way, to this technological evolution.
An Autonomous Intelligent System continuously:
- acquires information from its environment,
- processes that information,
- reasons about possible actions,
- modifies its internal state, and
- executes actions intended to achieve one or more objectives.
Each of these stages raises important scientific and engineering questions, including:
- the quality, reliability, and credibility of the information being processed;
- the representation of knowledge and uncertainty;
- decision making under incomplete information;
- learning and adaptation;
- explainability and transparency;
- safety and robustness;
- ethical and legal considerations.
Despite remarkable progress in artificial intelligence, two fundamental research problems remain largely unsolved:
- transferring skills and knowledge efficiently between different tasks and domains, and
- modelling common sense.
This document focuses exclusively on the second problem: the importance of incorporating common-sense reasoning into Autonomous Intelligent Systems.
Fundamental Concepts
To establish a common terminology, consider that an Autonomous Intelligent System receives information from two broad categories of sensors.
Internal information describes the current state of the system itself. It originates from internal components and their interactions and is commonly associated with feedback, self-monitoring, diagnostics, or proprioception.
External information describes the system's relationship with its environment, including other systems, physical objects, human users, and external events. This information enables perception of the surrounding world.
Both categories are acquired through sensors, whether physical (e.g., cameras, LiDAR, microphones, temperature sensors) or virtual (e.g., software logs, databases, network traffic, APIs).
Data Characteristics
Big Data is traditionally characterized by three fundamental properties (Laney, 2001):
- Volume – the quantity of data generated.
- Velocity – the speed at which data are produced and processed.
- Variety – the diversity of data formats, sources, and modalities.
Later research expanded this framework by introducing additional characteristics, among which Veracity has become particularly relevant for artificial intelligence.
Many practical challenges associated with Volume, Velocity, and Variety can be addressed using established engineering techniques such as:
- selective sensing;
- intelligent sampling;
- data compression;
- feature extraction;
- dimensionality reduction;
- filtering and denoising;
- distributed processing.
However, these techniques alone cannot guarantee that the information being processed accurately represents reality.
The Fourth V: Data Veracity
Among the extended Big Data characteristics, Veracity refers to the trustworthiness, quality, consistency, and reliability of data.
For intelligent systems, poor-quality information inevitably degrades decision quality. This observation follows directly from the well-known principle of Garbage In, Garbage Out (GIGO).
Consequently, an AIS should not only process incoming information but also continuously estimate its reliability.
Signal Processing provides a useful analogy. The Signal-to-Noise Ratio (SNR) measures how much useful information exists relative to unwanted disturbances. A high SNR generally improves the probability of accurate inference.
However, unlike classical communication systems, an intelligent agent cannot simply discard all apparent noise.
Unexpected or anomalous observations may correspond to:
- equipment failures,
- cyberattacks,
- environmental changes,
- sensor degradation,
- previously unknown situations.
Consequently, modern AI systems increasingly incorporate:
- anomaly detection,
- uncertainty estimation,
- probabilistic reasoning,
- Bayesian inference,
- confidence calibration,
- continual monitoring.
These mechanisms enable the system not only to identify expected situations but also to recognize when its own knowledge may be insufficient.
Common Sense and Natural Language
Common sense has frequently been studied through the lens of natural language understanding because human knowledge is largely expressed linguistically.
Natural language reflects centuries of social evolution, shared experiences, cultural conventions, and implicit assumptions. Consequently, much common-sense knowledge is embedded within language itself.
Large Language Models have demonstrated impressive capabilities in capturing statistical regularities of language, and benchmark datasets such as CommonsenseQA, HellaSwag, PIQA, and Winograd have shown significant improvements over previous AI systems.
Nevertheless, recent research consistently shows that language modelling alone does not constitute genuine common-sense reasoning. Current models often fail when confronted with:
- novel situations,
- causal reasoning,
- physical reasoning,
- temporal consistency,
- long-term planning,
- counterfactual reasoning.
Moreover, restricting common sense exclusively to natural language unnecessarily narrows the problem.
Many Autonomous Intelligent Systems—industrial robots, autonomous vehicles, spacecraft, manufacturing systems, and sensor networks—interact successfully without using human language.
Communication among such systems relies on structured data, control signals, and shared protocols rather than linguistic expressions. Are these "other" common sense?
Therefore, a more fundamental question emerges:
Can an intelligent system exhibit common sense independently of natural language?
In other words: Is common sense fundamentally a linguistic phenomenon, or is it a broader capability involving perception, reasoning, prediction, adaptation, and decision making under uncertainty?
In my opinion, this question remains one of the central open problems in artificial intelligence research, and it extends beyond the scope of this document.
References
Classical AI
- Russell, S., & Norvig, P. (2021). Artificial Intelligence: A Modern Approach (4th ed.) Pearson.
- Turing, A. M. (1950). Computing Machinery and Intelligence. Mind, 59(236), 433–460.
Autonomous Intelligent Systems
Big Data
- Laney, D. (2001). 3D Data Management: Controlling Data Volume, Velocity and Variety.
- Thelwall, M., et al. (2021). "Garbage in, garbage out" revisited...
Common Sense
- Davis, E., & Marcus, G. (2015). Commonsense Reasoning and Commonsense Knowledge in Artificial Intelligence.
- Gary Marcus (2022). Deep Learning Is Hitting a Wall.
- John McCarthy (1959–2007). Programs with Common Sense
- CYC Project (Douglas Lenat) Cycorp (Cyc Project)
- CommonsenseQA Dataset. CommonsenseQA Paper
- Winograd Schema Challenge. Winograd Schema Challenge
- HellaSwag Benchmark. HellaSwag Paper
- PIQA Benchmark. PIQA Paper
Foundation Models
Large Language Models
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
- OpenAI Research / Model documentation: OpenAI Documentation
