Same thing happend to me, never did figure it out. After about the 4th adjustment (May 2012), it seems to be staying put.
It's not the cams on the axle, I marked them the first time I adjusted the belt and they haven't moved. It seems the belt wants to find it's own 'sweet spot'. The adjustment (08 FLHX) is 5/8" (I think, don't remember, went by the shop manual) and loosens to about 3/4".
Since everything I read about the drive belt says they don't stretch, I started to get real nervous about 'if the belts not stretching, what is?' Front pulley breaking, trans main shaft bending?
So far there's been no ill effects (about 6,000 miles since last adjustment) and I've got my fingers crossed, and just decided to live with it.