The gray area is a matter of scale and purpose. If you have a small business which serves as specific market it's a good idea to make the site as user and disability friendly as possible. If, however, your market serves segments with a likeliness to have various handicaps or disabilities you should address that with an ADA Compliance review of the site. Also consider the scale and purpose of an organization. Publicly traded large businesses need to concern themselves more than a single location family owned retailer and if someone sues your flower shop because the text was too small or didn't have "image alt tags" causing them distress... That's pretty frivolous and would likely not stand.
Thank you all for your comments.  In my world of space planning and design of commercial facilities of all types , job # 1 is to make sure you can exit a building in case of heavy smoke and fire and make sure that the space is handicapped accessible and that a disabled person can get out of a building in case of fire.  I have never heard of a handicapped accessible website, however the explanation of a website that refers to a physical space makes sense. Alt text to describe something on a website and /or possible audio enhancements also make sense.   That being said I am also very aware of real and let’s call them faux lawsuits and have concerns about opportunistic lawsuits.  They are already happening and to think that could morph into the web is frankly frightening,  
If you do get sued, if you immediately remediate your website, you may be able to get the lawsuit dismissed on mootness (there’s no longer anything in dispute, i.e. plaintiffs are arguing your website is inaccessible but you’ve already made it accessible). This definitely does not mean you should wait to fix your website but it does mean you may have an out.
The ADA is the Americans with Disabilities Act, passed almost 30 years ago in 1990. This act was originally put in motion pertaining solely to physical location. It requires establishments to provide people with disabilities easy access to various levels throughout the business. This act was set to achieve an equal experience to all people, handicapped or not.