I would think that the Ring affects viewers' minds rather than controlling light, or else its wearer wouldn't cast any shadow, shaky or otherwise.  Since the process is a mental effect, the basic quanta involved are not distinct solid bodies, but basic notions of what self and invisibility entail.  Clothing, I suspect, is closely bound with a person's sense of self.  When contemplating the power of invisibility,  as in the old choice of whether one would prefer to be able to disappear or to fly, people generally presume that clothing would be involved (which caused a surmountable problem in an one X-files episode); and even many stories which discount clothing forget about details such as dust and newly ingested food (the novel "Memoirs of an Invisible Man" discussed this effect).  In achieving its effect, I guess that the Ring is working on this specific notion of self which includes clothing, minor dirt, etc., rather than calibrating for the specific physical object, which it would have to if the process involved bending light.  Swords, especially if carried in scabbards, would straddle the line, and could fall either way.  If Tolkien had written about the Samurai, it might be presumeded that they would be invisible.  

I would say that this would apply to magic in general.  Magic, and faerie, which hearken to the past, deal with traditional phenomenological categories, such as person, clothing, etc.  Science fiction, looking to the future, would insist on updated concepts about the physical world.  So a magician could simply exercise power over the weather without any problem, since weather is traditionally viewed as a specific phenomenon; while a machine in a sci-fi setting would have to influence weather through some other specified means, such as heat.

So I personally feel that while a mechanical process such as physical contact could be used to explain the Ring's invisibility, it isn't necessary to do so.  That is a demand of a different paradigm.