The gray area is a matter of scale and purpose. If you have a small business which serves as specific market it's a good idea to make the site as user and disability friendly as possible. If, however, your market serves segments with a likeliness to have various handicaps or disabilities you should address that with an ADA Compliance review of the site. Also consider the scale and purpose of an organization. Publicly traded large businesses need to concern themselves more than a single location family owned retailer and if someone sues your flower shop because the text was too small or didn't have "image alt tags" causing them distress... That's pretty frivolous and would likely not stand.
This is the reason why I think web design firms have to position themselves as solution providers and not just web guys/gals. A client doesn't need a website, they need sales. The website is just part of that solution but it's not the solution and unlike web design, service like email marketing automation, SEO, and PPC requires an experienced professional.
Upon discovering issues faced by disabled government employees and the public, revisions to section 508 were made. The issues that were uncovered involved websites, documents, and software programs. As of January 2018, the updated 508 standards require all federal agencies and contractors to create web content accessible to all. The updated standards also include Website 508 compliance.
...the full and equal enjoyment of the goods, services, facilities, or accommodations of any place of "public accommodation" by any person who owns, leases, or operates a place of public accommodation. Public accommodations include most places of lodging (such as inns and hotels), recreation, transportation, education, and dining, along with stores, care providers, and places of public displays.
The ADA is the Americans with Disabilities Act, passed almost 30 years ago in 1990. This act was originally put in motion pertaining solely to physical location. It requires establishments to provide people with disabilities easy access to various levels throughout the business. This act was set to achieve an equal experience to all people, handicapped or not.