I'd say the sweater-and-khakis combo is just fine. I think it's more professional if a physician wears a lab coat, but it's still optional. A dress might seem overdone, but still okay. Jeans are more casual than I'd prefer, both as a patient and as an admin.

Two chickens are on opposite sides of a road. One yells, "How do you get to the other side?" and the other hollers back, "You ARE on the other side!"