I bought a little Ace-Hanson tap kit about 50 years ago - machine thread from 4-40 to 1/4-20 with drill bits and a little "T" handle.
As the years went by, I picked up taps and dies as I needed them - onesie-twosies at the local hardware store. They're all Ace Hanson or Irwin.
Today I own a complete "set" of taps and dies from 4-40 up to 1/2-20 (and everything in between.)
I have taps and dies that have never been taken out of the packages yet.
The only problem is that I don't have a fancy-schmancy box in which to store all the pieces neatly arranged by size.
I had a full 40-piece "Columbia" (TRW) metric tap & die set that I gave to a buddy of mine about 20 years ago. If I need to re-work a metric hole I'll give him a call.
Just get what you need. Pick 'em up as you go along. Sooner or later you'll have every damn one of them.
On the other hand, if you're aspiring to be a machinist when you grow up, by all means, go big and buy the whole enchilada now. Don't go cheap though - get the good stuff.
this eBay seller bought out a warehouse of Irwin inventory and has some good prices on drills/taps/dies (and other Irwin stuff):
(I usually get 20%-25% off from him depending on how much I buy. YMMV.)