I collected a fairly large wisteria from NYC a couple of years ago that I'm working on. I guess it depends on what you think is a pain with them. They do put out growth fairly quickly - growth can extend a few inches in less than 24 hours. That being said, with wisteria, it's fairly easy to just cut it back every week or so since you're not necessarily doing precise branch work as you would with most deciduous material.
For me, the biggest pain is that it continually sends up new runners out of the pot that I'm constantly removing. I'll dig back some of the soil and cut them down low, and the next weekend have to do it again. That bothers me more than anything.
They are thirsty tree, so I have mine in a fairly heavy organic mix. All my other trees, with the exception of my bald cypress, are in 100% inorganic mix, and I wouldn't be able to keep up with watering my wisteria in that mix.
Otherwise, they are fairly resistant to pests and diseases, love full sun, and are fairly maintenance free in my opinion.