I think grit would be a good thing to add.  I know in Japan they use raised beds full of Kanuma for Azaleas.  I've also heard good things about filling raised beds with bonsai soil that's been used once and still in good shape and using all pumice.  I believe if you just want to grow trunks then a larger particle size would be better, however then you get less root divide. 

You know, the dynamics of soil and in the ground are quite different. blending compost and soil, then mulching around your trees, with a fabric mulch or with an organic mulch, or both, should work beautifully. The key is to do the root work before you get the trees in the ground, getting good flat root systems with no tap root. If you don't do this up front, you are wasting your time rowing trees in the ground- and I am speaking from experience.