Yes, this can be correct, depending on the caliber and barrel twist. But what I have found, is that your rifle doesn't like or dislike a bullet or a bullet weight. What it likes (or dislikes) is the load used with that bullet. Meaning, the powder and powder charge used with that bullet.
For example, I had a customer recently that had a 300 Win Mag that he wanted to shoot the 180 grain Nosler Accubond (AB) in his rifle. It's a great bullet and should shoot well. He tried several different offerings from various factory ammo with this bullet. Nothing shot good at all. So he wanted to get a custom load worked up from me, and he told me do not use the 180 grain AB because "my rifle does not like that bullet". So, I explain to him it's not the bullet your rifle does not like, it's the load used with that bullet. I said I have no doubt I can get it to shoot well, so let's choose this bullet. I worked up a load that was about 1/2 moa for him, and it shot great with a low extreme spread on velocity. When I showed him the target and the results, he was shocked it shot that good.
Generally speaking, as long as a bullet is stable, it can be made to shoot well in a rifle. Now I have had some rifles that were right on the edge of stability, and did not like the heavier (i.e., longer bullets) bullets. Bullet stability is based on bullet length, not the weight. So a short and fat 117 grain RN will be much more stable than say a 115 grain Berger VLD. The VLD will be a much longer bullet, and will need more twist to stabilize.
On your 25-06, it is most likely a 1:10" twist barrel. The 120 grain bullets will have plenty of stability. What your rifle doesn't like is the load used with that bullet. And, another major factor of factory ammo, is the internal quality of the consistency of the powder charges. I have seen as high of 2.1 grain spread on the powder charges inside factory ammo when pulling bullets on factory ammo. Most of the time, a "good" spread will be 1 to 1.5 grains, which is still very bad. I load my powder charges accurate to .02 grains, which is to the kernel of gun powder. When I work up a load, I am testing in .3 to .4 grain increments. So factory ammo having a powder spread of 1 to 1.5 grain, covers about 5 of my test loads, if that tells you anything. Hope this helps!