The gray area is a matter of scale and purpose. If you have a small business which serves as specific market it's a good idea to make the site as user and disability friendly as possible. If, however, your market serves segments with a likeliness to have various handicaps or disabilities you should address that with an ADA Compliance review of the site. Also consider the scale and purpose of an organization. Publicly traded large businesses need to concern themselves more than a single location family owned retailer and if someone sues your flower shop because the text was too small or didn't have "image alt tags" causing them distress... That's pretty frivolous and would likely not stand.
What do you want your website to look like? Consider websites that are similar to the one you’d like to build, ideally in the same industry or serving similar types of customers. Build a set of examples of types of pages, design aspects, and website features that you can hand off to the web designer — the person you hire should have experience creating websites with the features you want. If they don’t have the right skill set, they’re not the right pro for you.