Rifles are like women. They all have similarities, but they must be treated as an individual.
^^That is mine^^
Each rifle will degrade accuracy at a different round count. Some are a few rounds, 50, 100, or 500. Know where that number is, and know how many rounds it takes to start acting normal. Fortunately I don't have to worry about hunting with a clean barrel. If it needs cleaning I will do it at the range, then promptly foul it back up. Seen more than one have a good zero, get cleaned, zero check and it has shifted. Keep shooting and the zero returns to where it wad before cleaning.
This occurs when you set the zero, using a dirty barrel. If you do that, it will only shoot to zero if it is sufficiently dirty, as you have pointed out.
Some target and recreational shooters do this to good advantage. - But by trying to carry that attitude over to the hunt, you place yourself and your hunting partners at a disadvantage.
For my hunting rifles, I set the zero with a clean barrel, cleaning and cooling the gun every three rounds while I am at the range with it.
There are a number of reasons why I do this:
* On a hunting rifle, the first round is what counts the most. Following rounds are less important by an order of magnitude, each. By the time you get to firing a third round at a game animal, they will typically be holed up in the brush, or well on the way to the next county. The chances of a third round doing any good after the first two did not are vanishingly small. Only the first and perhaps a second round need to be considered, big-game hunting is not a firefight, nor is it a shooting competition.
* If some mishap or inclement weather prompts you to clean your rifle in the hunting camp, nobody is going to want you to fire "fouling shots" in the morning and scare all of the game away for miles around. By setting your zero with a clean barrel, you know that a clean gun is your best chance for accuracy for that crucial first round. - And you can clean your barrel at any time without concern over "fouling shots", secure in the knowledge that the next time you pull the trigger, that bullet will be going precisely where you expect it to.
* Carrying a gun with a dirty bore around, humidity will react with the carbon, combustion byproducts and other crud in your barrel, causing some of the crud to swell up and alter its chemical composition. The point here is that a dirty barrel that has just been fired is not the same as a dirty barrel that has been carried around in the woods for a few days.
This not true with a barrel cleaned every evening at camp. ( Whether the gun needs it or not. )
A clean barrel will not markedly change from simply being carried around. - And as we all know, consistency is the groundwork that good accuracy is built upon. A clean barrel is a known quantity, while a dirty barrel that has been transported, then carried around for a while is not.
* Clean barrel: No carbon or copper, no oil or Hoppes#9, just cold, clean steel. The last step in cleaning is to run tight-fitting dry patches through until they come out with no stain. Doing this every evening at camp while you tell lies precludes any possibility of humidity forming rust or corrosion on the bare steel.
When I'm at camp with somebody who has set up his gun to shoot with a dirty barrel, I'm thinking, "What if this goober slips and his barrel goes into the mud, or he gets water in there from this snow? - After he cleans it, is he going to insist upon firing "fouling shots" and scare off all of the effing game? Or will he go out there not really knowing where his gun is going to shoot, and tee off the rancher by crippling or losing one of his animals?"
Neither alternative is what you could call optimal. - And both are simply, responsibly addressed by setting your hunting rifle's zero with a cold, clean barrel in the first place. - A known, easily repeatable quantity that you and your hunting pardners can count on.