Shooter fatigue would be my first guess. As you said, since the bullet doesn't contact the suppressor, I wouldn't think there's much interaction there.
QD or direct thread? If direct thread, you could argue that while you're screwing it on and getting everything ready, the hot suppressor begins to transfer heat right at the end of the barrel. Depending on the weight of the .308 barrel, I suppose there is a small chance that heat transfer is causing a shift?
20/20, but what happens if you let it cool for 15-20 minutes, then transfer it and shoot it?