Q-Learning: Teaching an Agent by Trial and Error
Q-Learning teaches an agent the 'quality' of an action in a given state through trial and error, like training a pet with treats. It's used in robotics for navigation or in games where an AI learns optimal moves.
Why it exists
How do you teach a machine to make optimal decisions in an environment when you don't have a perfect model of that environment? Q-Learning was developed to solve this by allowing an agent to learn from direct interaction and experience, rather than from a pre-written rulebook. It answers the question: "What is the best action to take right now to get the best results in the long run?"
The mental model
Imagine you're in a dark room with many doors (states) and you can take certain actions (open door A, open door B). Behind some doors are rewards. Q-Learning is like slowly building a mental map of expected rewards. You try a door, get a result, and update your belief about that door's value. The "Q" stands for Quality—a score for how good an action is in a particular state, considering not just the immediate reward but all future rewards that follow.
How it works
The agent maintains a "Q-table," a spreadsheet with rows for every state and columns for every action. Each cell, Q(s, a), holds the expected future reward for taking action 'a' in state 's'. The agent starts with a table of zeros and explores the environment. When it takes an action and receives a reward, it updates the corresponding Q-value. This update incorporates the immediate reward and a discounted estimate of the best future reward from the new state. Over many iterations, the Q-values converge, giving the agent a policy for how to act.
When to use it
Q-Learning is ideal for problems with a discrete, manageable number of states and actions. Think of classic grid-world problems (a robot navigating a maze), simple board games (Tic-Tac-Toe), or optimizing simple processes like inventory management. It shines when the environment's rules are unknown or stochastic (unpredictable).
When not to use it
It struggles with continuous or very large state spaces. A Q-table for a self-driving car, where the state includes precise positions of all nearby objects, would be impossibly large. In these cases, Deep Q-Networks (DQNs), which approximate the Q-table with a neural network, are used instead. It can also be less sample-efficient than other methods.
One canonical example
A simple robot cleaner in a 4-room house. The state is the robot's current room. The actions are moving to an adjacent room. The goal is to reach a room with a charging station (positive reward). The robot starts randomly, builds a Q-table for each (Room, Action) pair, and after many trips, the table's values will guide it to take the shortest path to the charger from any room.
Interview question
For which type of problem is Q-Learning most effectively applied?
- a.Training an AI to play a board game with a finite number of states and actions.Correct
- b.Creating a predictive model for continuous financial market data.
- c.Designing a control system for a robot arm with precise, continuous joint movements.
- d.Developing a self-driving car's navigation system in a dynamic, real-world environment.
Why? this is the answer
Q-Learning is ideal for problems with discrete, manageable numbers of states and actions, such as simple board games. It struggles with continuous or very large state spaces, like those found in self-driving cars or systems with continuous movements.
Just read this? Test yourself on what you have been reading.
Read the original → en.wikipedia.org
- #reinforcement learning
- #machine learning
- #ai
- #algorithms
Put your scrolling time to good use
Learn one idea, try a quiz and save useful cards for revision. Tezvyn makes it easy to learn and stay current in your tech field, a few minutes at a time.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Every open role lists the topics its interview covers, so you can prepare for the real thing rather than guessing.
See open roles