I've always thought Tolkien was actually rather anxious to get it published,
but his Stanley Unwin kept rejecting it, see Letter's 124-126 for example,
where Tolkien tried to persuade him to publish LotR and The Silmarillion
together. Of course it wasn't in a finished form yet, and when it
was rejected, Tolkien carried on working on it, modifying and adding to
it. Then again, I suspect the if LotR's had been rejected Tolkien would
have carried on working on that as well.
Since I first posted I've heard that in the Osanwe essay (I realy must pick
this up) Tolkien comes to some compromise solution, where the Valar
choose not to have children, which would I think satisfy the bit from
The Peoples of Middle-earth. I can't confirm it myself as I haven't read
it myself, but I'll have to modify my canon I think.
There was a recent discusion of Ungoliant, where I tried to explain why I think
she must ultimately be one of the Ainur. Briefly again, I think it says
somewhere, (can't remember where at the moment), that she was one of Melkor's
first followers, but took her spider form and independant ways when she first
descended into Ea, then down to Arda, setting up home in Northern
Valinor. This implies she came from the same timeless halls as Melkor,
and, taking Tolkien's views on the limitations of evil as a corruption of
something once good (eg Orcs, Melkor, Sauron etc), must therefore have
originally been a good being herself who had fallen. As Eru created the
Ainur, who it seems by definition were the inhabitants of these halls (though
of many types), I think she must therefore have been one once.
As to the many spirits inhabiting Arda, these I think would either be lesser
spirits, or orders of Maiar, who also descended to Arda, and also bred there,
for example the Eagles, or were created within Arda as new spirits in much the
same way as the spirits of the Elves, Men and Dwarves were (for example the
Ents, who considered themselves a people much like the others). Of course
though, Bombadil is unique as you say, I just don't think Goldberry is.
The other animals are enigmas
though.
