Training and testing sets
n Where does the test set come from?
1. Collect a large set of examples
2. Divide into training and testing data
3. Train on training data, assess on testing
4. Repeat 1-3 for different splits of the set.
n Same distribution
   “Learning … enable[s] the system to do the task or tasks
drawn from the same population” – Herb Simon
¡ To think about: Why?