So I would say (1) if new features, regarding output, code performance, or
other information is added, or (2) if new methods for solving a given problem
are added, or (3) if some errors/bugs are fixed, then we would want a new
executable.
Usually I just make a new executable if there have been any changes, just in
case they are useful.