My AI quest started because of my interest in understanding decision making:
- How to find a solution to a problem?
- How to know that my solution is the best?
- How can this process be systematized?
Nowadays, we live in a world of fancy words and trends that bear an uncanny resemblance to fashion. And it seems that the ’90s are back! Machine learning, SVM, deep learning, search algorithms, genetic algorithms, neural networks all have in common being tools to help find solutions to problems (if you want to jump on the new trend before anyone else just look at what was hot in research 20 years ago)…
Finding solutions to problems is at the root of all Artificial Intelligence. Searching algorithms is one of the main pillars of the book that I spent the most time consulting during my studies – Artificial Intelligence – a Modern Approach.
Artificial Intelligence: A Modern Approach
Therefore, in spite of whatever fancy new deep trend we’ve got going on at the moment, I’ve got some pieces of wisdom from the basics, the grandfathers of all, the search algorithms.
Optimization, topology, and humbleness
All problems can have a solution plane, that looks a little bit like the ground of a landscape like the one bellow. All the possible solutions for a problem can be mapped out on that plane and the better the solution the highest the ground.
The simplest example would be.
Problem: walking to the highest point in the region.
Solution plane: all walkable points in that region.
If we would have the topological map of said region, it would be relatively easy to understand where we would have to walk. A precise path and the best solution would be a map consultation away.
However, most problems are more complex and have solution planes that are yet to be uncharted. Some solution planes can’t even be completely charted with today’s technology. And most times we are happy at getting a good solution to our problems.
For most intents and purposes:
“Good is better than perfect.” (Why Good Is Better Than Perfect)
So, if you deal with data, insights and helping with decision making. Don’t forget this motto, because it’s the nature of the world that we live in.
The pitfalls of Optimization
One of my favourite search algorithms is “The Hill Climber”. The idea is very simple. You have your solution plan where the highest mountain is the best solution. In order to reach the highest mountain without any tools or prior knowledge, you’re going to start walking up, for as long as you can.
If you were lucky enough to start at the base of the highest mountain, you will reach the top using this procedure. But chances are that you’re going to end up stuck at another place higher than where you started. This place is a local maximum and if you want to reach the highest point from there your solution will become worse before it will become good again.
If you’re using most machine learning approaches to problem-solving, this Hill Climber algorithm is a good cautionary tale.
In the end it about getting a good result, managing risk and living with the consequences of our decisions.
Recap:
- All problems can be formulated as Search Problems.
- All the solutions for a problem can be visualized as living on a topological plane.
- Sometimes this solution space is so big and complex that we can’t find a systematic way to find the best answer (most of the times)
- We have to be comfortable wanting to strive for good solutions instead of perfect ones.
- The process of optimization might require making things worse in order to overcome a local maximum.