Do I have to file yearly Federal Income Tax returns in the US if I have a single-member LLC?

Yes. It has just recently come to our attention that the IRS has now made it clear that a US Tax return must be filed 'up the chain' for any disregarded entity. This means that if you are a non-resident alien and you don't have an ITIN then your LLC cannot be a disregarded entity even if you are the sole owner.
The IRS language states that an individual owner of a disregarded LLC must file a 1040 (that's a personal tax return in the USA), and although it states that a single corporate owner should account for the company in it's tax return as a division, they also say that that corporate owner must have an EIN and file a tax return in the USA, or if it's a single member LLC, then it's owner must file a tax return, etc..
So, no matter what the circumstance a tax return must be filed in the USA, though that doesn't necessarily translate into paying tax in the USA. After all, it's the profit that is taxed, and if no profit is left in the Delaware company, then no tax has to be paid, but a tax return still must be filed. It is common for a parent company to draw off all the potential profit in a subsidiary as management fees, leaving no profit in the subsidiary, and therefore no tax owing for it.
If someone tells you that you can have a disregarded entity anywhere in the USA and not have to file a tax return in the USA they are not up to date, and could get you into trouble. Come to us and we'll help you get setup right.
With this in mind most of our clients are forming Corporations as they are a more straight forward and understood company type around the world, and leave no question as to how they should be taxed. They also have a great paper-trail: tax returns from the US government to show the activities of the business that are acceptable everywhere.