We are normally to assume that Elves and Men look different enough that they are easy to tell apart, but of course Aragorn's Rangers, the Grey Company, are confused for Elves by the Rohirrim, so that can't always work.

The Rohirrim do not mistake them for Elves, they are simply awed by the Rangers to the point where they look like boys besides men. It is the people of Morthond who call the Grey Company 'Elvish wights'- Gondorian citizens. It appears that while the Rohirrim are cautious towards the Elves, the Gondorians are downright scared, as Faramir explains regarding Lothlorien and his own reluctance. Ironic that Boromir, the embodiment of this attitude ends up in Lothlorien.

It would take a trained eye to spot the difference between a High Man and an Elf, though creatures like Hobbits seem to have an intuition about these things, and of course the Ents do know as well.
Everyone else just makes a sloppy conclusion.

The Rangers have been moving further upfield than usual, says Damrod and Mablung. They couldn't have been raiding a Harad march unless they were raiding the middle (thus leaving the head of the column oblivious to the skirmish) and that would have been a shock. Attack any other part of the convoy and every Haradrim man will turn to fight. This convoy curiously lacks a king or lord, though the Ranger-centric point of view would not have looked for one.

The war in Ithilien is strictly a war of attrition, since neither side has fortified it for long term occupation or manned any serious push. The Rangers do count themselves as the foraging border guards, while the Haradrim and Orcs use it as a transit point. The bottom line is that if they can kill anyone from the other side,they'd be happy- the Haradrim want to move on to Mordor, the Rangers want to make sure they don't do so in numbers; neither side will get through Ithillien with a mission perfectly fulfilled, but the commanders probably know this.

I'm wondering if the Haradrim ruler sees Mordor as a land of hope, as Denethor sees Gondor as the last hope of Men. The Rangers would be those who try to dash hopes of the 'pious' attacking like demons in the dark (or light).

There is but one task remaining in Arda-we,the Dragons of Valinor will make it happen.

Lone resident of the Republic of Gondor

PO-TA-TOE-S not PO -TAH-TOE-S!