whoever told you that was significantly handicapping their abilities
tool choice is as much dependant upon case by case situation as it is material
to state that one style of tool is always better is usually false
I have found 3 flutes to be better as a general purpose end mill for aluminum work
2 flutes are good for roughing or if you need the extra chip clearance
4 flutes leave a decent finish but can chatter like crazy in corners if overengagement occurs, they also have much less chip clearance than 2 or 3 flutes (one of the reasons for the chatter in corners)
3 flutes tend to give you the best balance of both styles, more chip clearance than a 4 flute, much less corner chatter, and the additional flute compared to a 2 flute that gives your feed rates a boost
keep in mind most of my advice is stemming from cnc, but most of what I said applies accross the spectrum