Looks like leaf spot to me.  How do you water?  Do you get the foliage wet?  Did you do any Spring clean up to this tree?  Keeping the soil surface clear of all spent leaves will help reduce this.  Spray with a light fungicide such as Daconil and try to keep the foliage dry.

I actually haven't had to water this for awhile because of the rain, but when I do I just water the soil, and consequently some lower foliage. The lower foliage actually doesn't have many infected leaves though. I pluck all the infected leaves I see daily, and throw them away. No litter is on the soil surface. One more thing, I dug this up out of a yard late this spring. It went through a pretty bad shock, and I thought I was going to lose it, but as you can see it has flushed out with lots of healthy new growth. I suspect it had this problem when I dug it up. Actually, the reason I was able to dig it up is because it was smaller than all the others and not putting out as many flowers, so my neighbor told me to dig it.
As for the leaf surfaces, when I look at the bottom I can see the spots on the top. It looks the same. The spot goes all the way through the leaf. I also thought it was leaf spot after doing a google search, but I wanted to see if any of you guys had found a really good treatment before I went out and bought one.
Thanks for the help,
Nick