We did gymnastics (I know, not dance, but reasonably similar) at age 3 and DD was pretty young for it. She spent more time wandering around, staring off into space, giggling with the other kids and in general fooling around than she did doing gymnastics. DD is 4 now and still loses focus a bit but in general is much more attentive.
Based on my experience, I would say 4 or 5. Stick to things like library storytime for socialization purposes at ages 2 and 3, its cheaper and less frustrating when they lose attention.
BTW: You might want to check with your local library and see if they offer (or know of) any dance or rhythm type activites for young kids. In my area (west of Cleveland) the local library system has a free once a week rhythm and movement storytime for little ones that my DD loved at age 2/3. They banged on drums, clapped, hopped, moved to music and got the wiggles out.