1) What special qualities of these new lamps make them able to
withstand the destruction that came to the others? Can they be
considered "better" in a way?
These lamps are not bound to the earth and are moving targets, so they are
harder for Morgoth to assail, and they have Maiar "charioteers." His one
attack on Tirion with spirits was not very successul, and he's simply scared of
Arien's greater power, perhaps because he put so much of his own native energy
into Ungoliant earlier. All in all, compared to some torch lamps sitting
at either pole or even trees in Valinor, these lights are more secure, even if
they are not as intense.
2) It seems that each creation of a light source is somewhat inferior
than its predecessors. Is this merely an expression of nostalgia and
regret of later generations looking back to a perceived "golden era"?
Or does it represent a true downward trend in creation -- moving from
more perfect to a flawed form? Is this how elves see history? Is this
how Tolkien sees history?
Yes, it is a tangible expression of the "long defeat." Tolkien is not a
meliorist, but he is not a pessimist either. These lamps are not as great
as the former lights, but they are lights that can endure for the long term and
provide more balanced light both to Aman and Middle-earth. In that
sense, the "lesser" lights reveal part of the harmony inherent in Eru's plan,
his desire to create a natural cycle that can make Arda a more stable,
productive environment. Better to have a lesser amount of light and
darkness covering all Arda than one section filled with light and the other
filled with perpetual night. I'll pretend Tolkien's revisions of the
astronomical makeup of Arda in Morgoth's Ring just don't exist here--a creative
fantasy writer shouldn't try to play Dr. Carl Sagan IMHO! :-)
3) What of the notion of progress and improvement from one generation
to the next? If the view of a slow downward spiral is accurate, what
can be expected for the future? What of hope?
A downward spiral yes, but to a kind of history that can exist for the
duration. It's all about balance and equilibrim, and the Arda of the time
of the trees was too lopsided: glory and light on one side, darkness and
Orcs on the other, with some leftover Elves and Dwarves sticking it out.
Men, in my theory, are the necessary additions to the equation because they
have short lives, but bring the right mix of change and novelty to the universe
and serve as the bridge, the "missing link" between the Valar and the Eldar,
the immortal essence of Eru, and temporally-bound world of the kelvar and
olvar. It's fitting that Men awaken with the sun and the moon, because
that creates a diurnal cycle they can use to give structure to their
lives. Men couldn't possibly exist in the vast temporal cycles of the
trees or in the cycle-less darkness of the stars. In order to prepare the
way for Men, the key to Eru's whole plan, there must be a new way to establish
a rhythm in the cosmos.
In a way, this might be how Morgoth's evil becomes a central part of Eru's
overall plan. In damaging the earlier iterations of Arda, Morgoth makes
possible a more long-lasting version of creation that, Eru willing, continues
to endure to this
day.

Cor Blok, Battle of the Hornburg (detail)