Xyzcf, I really think the Outer Banks or VA Beach would be the best option for an in between place for people coming from the North or the South. I have seen several comment from as far north as NJ and as far south as GA. So if the beach is of interest then either of those and Wilmington, NC are good options.
Another I can can give input on lodging options for either Outer Banks (usually go a couple times a year) as well as Virginia Beach (from that area and still have ALOT of family there). I guess the next step is seeing when would be a good time for everyone. How soon or far out? Many are on fixed income so if everyone wants the beach sharing rooms, or waiting until after Labor Day (past peak season) are best. Either of those locations can still be expensive even when sharing a room depending on where you stay, especially if people want oceanfront rooms. Now I know the prices are much better at Outer Banks even if you are just across the beach road. Just depends on what everyone wants.