With these lines, the speaker gets even broader in his philosophy, declaring that everything beautiful must eventually fade away and lose its charm, either by chance or by the natural flow of time. Kind of like teen pop stars.
Now what exactly does "untrimm’d" refer to?
We might read it as what happens to "fair" or beautiful things. By that reading, things that are beautiful eventually lose their trimmings, or their decorations, and thus fade from beauty.
On the other hand, "untrimm’d" is also a term from sailing, as you "trim," or adjust, the sails to take advantage of the wind. This gives "untrimm’d" a completely opposite meaning; instead of "made ugly and plain by natural changes," it means "unchanged in the face of nature’s natural changes.