How useful are Recommendation Systems?

Typically, recommendation engines and systems enhance the user experience, because they assist us in finding information, reduce search and navigation time, and increase our satisfaction. 

However, I am still receiving recommendations about options to buy vacation packs, books, movies, music, etc.; that I had reviewed more than one year ago. Even worse, I receive friend suggestions because they are friends of someone that I know… Why? Really, most of the time I am not interested in receiving those kinds of recommendations… 

Therefore, I asked myself: How many people accept and follow these recommendations? 

Unfortunately, I don’t have access to all the required data in order to evaluate this; but according to Symeonidis and Zioupos (“Matrix and Tensor Factorization Techniques for Recommender Systems” ISBN 978-3-319-41356-3):

• Amazon – 35% of product sales come from recommendations in Amazon.com  
• Netflix – 66% of movies rented in Netflix.com are recommended  
• Google – 38% more click-throughs are generated from recommendations in Google news

In my opinion, the main reason of this behavior can be traced back to the origin of the algorithms that have been used to create recommendations: clustering, ranking, scoring, pattern matching, etc. This means “the History”, usually understood as Big Data, OLAP, OLTP, etc. But it is clear that this "historical approach" demands many resources: storage, computing power, and time. 

But: What happen if we’re looking for recommendations about the outcome of a “random” process? 

The situation becomes harder if we don’t have enough information about the process itself. Let’s put it on an easy way: We need recommendations that could not be related to the previous history of the process. 

Example: A recommendation with probability 0.7 is not a winning one in gambling. Unfortunately, the number of alternatives experiences an exponential growth in order to achieve a greater probability. It is difficult to create such recommendations using "brute force" algorithms, and the task will demand the most powerful computers. Even worse: There is not heuristics to "prune" the decision tree.

This means that the inputs to the recommendation’s process are not well suited because the data could be either too poor or too much, the representation of the recommendation’s knowledge has not been identified as it should be, the inference rules could not be useful because there is not a previous experience about the behavior of the process, and the explanation about how the recommendations were created does not allow to identify the whole reasoning process: the domain is not well formalized. 

Then, I'm asking myself: How to predict the future behavior of a system whose previous history might not be relevant to the predictive process?

I believe that a new approach is needed to the recommendation’s process. This approach should redefine how to analyze the inputs, how to build new models of knowledge’s representation, how to propose different inference mechanisms that might not be well suited and computed by Turing’s Machines, many alternatives of solution, and explanations about how these recommendations were reasoned that may not match the common sense.

As a conclusion, I think that research in Artificial Intelligence should include more than machine learning, deep learning, and neural networks; because the focus of the problem should consider not only the data -the history-, but also the mechanisms to extract real knowledge of them: models, representations, inference rules, and explanations.

This path will lead us to “skilled intelligent systems”: solutions that can be transferred from one domain to other domains, and recommendations that will be really useful.   

Comentario - Comment

Estimados lectores:

Cuando oficialmente dejé de trabajar como Profesor Titular en el año 2010, publiqué mis clases en Español para dominio público y sin fines de lucro en OneDrive < https://1drv.ms/f/s!ApPTAVJ07A-CgRQ2NbUjw6U1nDnB > .

A lo largo de varios años he visto dichos materiales referenciados y reproducidos por diversos sitios como SlideShare también sin fines de lucro. (Ejemplo en SlideShare: < https://www.slideshare.net/fuvylvp/almacenes-de-datos-olap-y-minera-de-datos?qid=2634f8ee-fb1d-4c57-aec0-a0add8ce77e6&v=&b=&from_search=1 > )

Sin embargo, hoy encontré un sitio que se declara "sin fines de lucro", pero que pide "apoyo al compartir y/o descargar" uno de mis documentos: "Aspectos Avanzados de la Tecnología de Objetos".

Personalmente considero que esta información debe ser conocida por todos.

Agradezco de antemano cualquier comentario ó sugerencia sobre acciones futuras al respecto.

Dear readers:

When I officially quit working as a Professor in 2010, I published my lectures in Spanish for public domain and non-profit use in OneDrive <https://1drv.ms/f/s!ApPTAVJ07A-CgRQ2NbUjw6U1nDnB >.

Over the years I have seen such materials referenced and reproduced by various non-profit sites such as SlideShare. (Example in SlideShare: <https://www.slideshare.net/fuvylvp/almacenes-de-datos-olap-y-minera-de-datos?qid=2634f8ee-fb1d-4c57-aec0-a0add8ce77e6&v=&b=&from_search= 1>)

However, today I found a site that is declared "non-profit", but "needs support to share and download" one of my documents: "Aspectos Avanzados de la Tecnología de Objetos".

Personally I consider that this information should be known by everyone.

I appreciate in advance any comments or suggestions on future actions in this regard.

Dr. Juan Jose Aranda Aboy

Building an intelligent Chef

Let's start by analyzing the scope of the job: Building an intelligent machine means that the system must pass Turing's Test. Therefore, we are introducing by default a "common sense" rule in our research.

An interesting problem appears if we want to create an intelligent machine that can act as a Chef.

Cooking is an open problem, and it introduces some challenges to the intelligent machine. We will consider only three:

1. Should it cook by replicating a recipe step by step? What about measures? Cooking time? Is Fuzzy Logic the tool that could help solving these problems?

2. Could it create its own recipes by using the available products only? Could it transfer some skills that it has learned before such as music composing or poetry? How? Would Evolutionary Algorithms help?

3. What results can be accepted as "tasty meals"?

We can continue writing challenges that our "Chef" should overcome, but the last one is the main problem: the expectations about the resulting meal vary for each person, and even worse: each culture redefines cooking according to its history, location, and standards.

Therefore, which one is the appropriate "output"? I'm afraid that there are many solutions, and all these "tasty meals" would pass Turing's Test, but some of them would not pass the people’s taste.

The next step should be to "model" our Chef. To do so, we will analyze the problem again by reviewing the required actions to build an intelligent machine transforming the inputs (i.e.: beef, salt, lemon, onion, garlic, margarine, etc.) in an output: the meal (Steak!).

First, the desired recipe should be selected. This can be done by surfing the Internet. Therefore, our Chef can solve this easily.

Second, it should be verified if there are all the required products in the recipe. Also, the intelligent machine must check if there is the required quantity of all those products. The cooperation of another intelligent system is needed to keep records of the existing products.

Third, preparation: the beef must be sprinkled each side with salt. Then it must be added lemon juice, garlic, and onion. So, the intelligent Chef will need some additional devices:
- Sprinkler,
- Squeezer, to extract the lemon juice
- Peel the garlic clove, and chop it
- Cut the onion 

Fortunately, we can assume that have been created a set of intelligent machines that can solve these problems.

Finally, the intelligent Chef should melt margarine in a large skillet over "medium-high heat", fry the steak on each side, and transfer to a hot serving plate. Two comments: 

A) Which one is the appropriate temperature of the skillet? What is the meaning of medium-high heat? 

B) How much time is needed to fry the steak on each side?

There are many other problems that the intelligent machine may ask itself. A few examples are:  

- What if there is "not enough" (less than the required) margarine, but there is enough vegetable oil and butter?
- What if the recipe must be cooked without garlic and onions, or less salt because of the consumer's requirements? 
- Can be used the same procedure to cook fry chicken?

This is only a preliminary exercise. Thus, I have written more questions than answers. However, I feel confident about the future, because Artificial Intelligence is still young, and there are many researchers contributing to the field, so my expectations are high.

References: 

THE WORLD'S FIRST ROBOTIC KITCHEN 

The World's First Home Robotic Chef Can Cook Over 100 Meals 

Man v machine: can computers cook, write and paint better than us? 

Beef Steak Recipes 

5 Tips for Squeezing the Most Juice from Limes & Lemons 

How to Peel a Garlic Clove 

Lend me the phone

This was the first request that my grandson made me when he came on vacation: my smartphone. By using this, he chatted, watched videos, and played different games... All in one!

After lending it to him, my memory began to remember what a telephone was like just a few years ago when I was his age.

I made a list of some applications, and therefore of things: devices, functions and media, which are incorporated in my current phone, occupying a volume of only 14.5x7x0.6 cubic centimeters:

• Phone of course, including caller ID, answering machine, call waiting, and even FAX.
• The classic agenda, that printed notebook having a calendar to remind important dates, our list of contacts, and the possibility of adding handwritten notes.
• Typewriter - Teletypes.
• Cameras, photo rolls development, video camera.  
• Clock - Stopwatch - Alarm clock.
• Calculator.
• News printed in newspapers and magazines.
• Radio.
• Phonograph / Turntable record player.
• Recorder / Player - Dictaphone 
• Television - Remote control 
• Cinema.
• Mail: Letters / Telegrams / Cables.
• Maps - Global Positioning Systems (GPS)
• Photo albums, storage of discs, tapes, cassettes, etc ...

For comparison, I tried to remember what those devices were like: dimensions, weight, and other characteristics; as well as how were added the functions that are included in my "phone".

After thinking some time, I decided to tell my grandson some of my stories working with computers, and how they evolved to allow him to play, watch videos, send instant messages, and so on in such a short time. 

However, I stumbled upon a barrier: their ability to listen the stories.

Consequently, I decided to create "apps" and videos to tell these stories, so here we go ... 
 

Coding for fun

I love computer programming, because I find that it is easy, although extremely challenging.

During my professional development, I worked as Researcher, Professor, Principal Analyst for Technical Support, and Software Architect.

I had forgotten what a wonderful thing is to write computer code just for fun, as I did at the very beginning of my life, before my PhD.

As computer programming is one of the most beautiful jobs that I have done in my whole life, I joined to the online practices at HackerRank.

I understand that most programmers develop computer software as their way of living. Anyway, I have the following concerns:

• Some colleagues are writing highly complex code, that includes unnecessary function calls, to develop the solution of a simple problem.
• Some solutions didn't validate the input data.

My question is: Why?

Programming languages have evolved enough in order to provide many tools that help improving the quality of the code. Even more: many frameworks have been built, allowing to reduce the time of development.

However, I believe that programming is still an Art..., and the programmer must still being in control of a huge amount of details.

There are four programming rules that it seems some developers have forgotten, and I believe that these should be rescued:

1. Avoid unnecessary code. Keep it as simple as possible. If the code is growing, it could be useful to divide it on manageable modules. Notice that this issue could appear even if you are writing software by using Object Oriented programming.
2. Validate the inputs.
3. Initialize all your variables. Do not trust that the compilers will do that for you.
4. Comment the code in a descriptive, and useful way. 

A last comment: Although the hardware is more powerful each day, the inappropriate use of the resources: processors, memories, etc;  will not solve the problems faster, or cheaper.

Autonomous Intelligent Systems and (Un)Common Sense - Part 1

Introduction

In the sixties of the past 20th Century, when I started thinking and working with computers, the idea of creating an "electronic brain" (in this document: an Autonomous Intelligent System - AIS) was still far away, but it was a possible dream.
As a "humble programmer", I have worked many years programming software that could be classified as "intelligent", so I feel proud of my contribution to transform this dream in a reality.
An AIS receives input data, processes this data, and generates responses modifying its behavior. These three tasks introduce some related issues:
- Veracity and credibility of the data being processed.
- Knowledge (information) that can be obtained about the current status of the AIS.
- Actions that should be done to reach the desired goal.
- Ethical aspects of the behavior of the AIS.
- Etc.
However, in the creation of AIS, there are two problems that - in my opinion-, remain hard to solve:
1. Transferring skills,
    and
2. Modelling common sense.
The objective of this document is to discuss the second of this problem only: the relevance of modelling common sense inside an autonomous intelligent system.

Concepts

First, I would like to remind some classical concepts in order to settle a "common ground".
There are two main kinds of data:
a) Internal data describing the "absolute position" (status) of the system in the environment. This data is originated by the different elements inside the system, and their interactions. Usually this kind of data is considered as "feedback".
b) External data showing the "relative position" of the system, and its relationships to other systems. This data is gathered to measure how the system interacts with the environment that surrounds it.
All this data is collected by using "sensors".

The three classical Vs

Now a comment about the three classical "Vs" that are usually related to data:
1.- Variety: There are different sources: system, subsystem, element, or process, that generate different kinds of data. This implies that there is a wide range of sensors being used to get "representations" of these different kinds of data.
2.- Velocity: The data can be received at different rates, according to the system, subsystem, element, or process that produces each particular kind of data.
3.- Volume: Depending of the system, subsystem, element, or process that produces the data, the volume that is received can be huge.
Most of the issues related to these three Vs can be solved (or minimized, at least) by using well known approaches as:
- Processing specific data, and therefore, ignoring other sources of data.
- Sampling at low rates, avoiding "real time".
- Filtering the input data in order to consider only those that are "meaningful" to the current status of the system.
- Etc.

The 4th V: Verisimilitude

Verisimilitude, considered as the veracity and credibility of the data being processed in order to reduce the "Data Uncertainty", is closely related to the common sense.
As a first approach, the problem can be simplified in this way: if the quality of the data "is not as good as expected", the response of the AIS will not be as good as expected. Then, an objective should be to collect the "good data" only, and the question becomes: What data should be taken as "good"?
The Signal to Noise Ratio (SNR) has been used widely to do so: Signal is the data that is considered "relevant" to the problem, and Noise is the data that should be "discarded".
Then, due that the SNR measures the quality of the data been received, its value should be maximized in order to get the "maximum likelihood".
In the real world, the intelligent systems have some kind of mechanism that is discarding some data "automatically". However, if some "threshold" is exceeded, low or high, the system triggers an "alarm", in order to generate the actions that produce the appropriate response(s). Therefore, an AIS should have capabilities to review the noise continuously, detecting unexpected changes: both internal (inside the system), or external (the environment), and reacting to these by providing appropriate responses that modifies its behavior to avoid or reduce "danger" and/or "damages".

Understanding Natural Languages

The common sense has been researched as related to the processing and understanding of natural languages.
Natural languages are the consequence of the communication needs that appear due to the existing relationships and rules inside a socially evolved set of human beings. Then, common sense could be thought as the expected behavior of all the members of this set, because they share a "culture", a "how-to" solve problems, and a "know how" that should be transmitted inside the members of the community.
In my opinion, reducing common sense to a natural language understanding issue is an approach that narrows the scope of the problem, but adds a high level of complexity to the solution, because if it is a communication issue between human beings mainly, the AIS must receive and send data, and explain its behavior by using natural languages. Unfortunately, under this umbrella, many AIS might be excluded.
If we examine the communication issue by using a different approach, we will notice that the AISs don't need a "natural language" in order to interchange data successfully among them, and even more, they can communicate with human beings without any word.
Then, the question should be: Can a behavior be considered as "common sense" without a representation by using a natural language? In other words, to perceive, understand, and judge things in a reasonably way: Is a natural language the unique alternative? Unfortunately, this discussion is beyond the scope of this document.
In the Part 2 of this document, I will analyze how it could be added some kind of common sense to an AIS.