That is an old saying which has guided my life for a long time (for whatever that's worth). It's not an absolute of course ("Only a Sith deals in absolutes") but part of a larger doctrine (a system of interrelated beliefs) which includes Ockham's Razor* as an equally-important element. Both are classically conservative concepts. 

*Ockham's Razor - The idea that the correct explanation for anything is generally the one which requires the least number of assumptions.