I have been an avid handgun hunter for over 30 years. I would not go smaller than a 44 mag. Here is a list of hunting handguns that I own and hunt with and I will give you my recommendation at the end:
S&W 29 44mag 8 3/8 barrel with open sights
S&W 629 44mag 6 barrel with open sights
S&W 629 44mag mountain gun 4 barrel with open sights
S&W 500mag 4 barrel with open sights
S&W 460XVR 8 3/8 barrel with variable scope
Ruger Redhawk 44 mag 7 1/2 barrel with fixed power scope
Ruger Redhawk 44mag 5 1/2 barrel with open sights
Ruger Super Blackhawk 7 1/2 barrel with open sights
Freedom Arms Premier Grade 454 Casull 7 1/2 barrel with variable scope
Freedom Arms Premier Grade 454 Casull 6 barrel with open sights
TC Encore 30-06 15" barrel with variable scope
Of all my 44's I much prefer the S&W because of their superior trigger pull. The Rugers are good guns, just not as refined as a Smith. I had trigger jobs done on all the Rugers and it improved them quite a bit. The 4" guns are mainly carry guns for when I am elk hunting or big game hunting out west. I would not go shorter than a 6" barrel for strictly handgun hunting and a 7 1/2 or 8 3/8 is even better.
I have taken deer with all my 44's, and black bear and deer with the 454 and TC Encore. I much prefer hunting with a variable scope and a good solid rest or shooting stick. My 454's were my favorite all around hunting handguns until I got the 460XVR. The 460XVR is not only a good short range gun, but a good long range gun out to 150 or so yards without a problem. If I think I am going to have a shot longer than that, then I use my TC Encore 30-06 which is good out to 225 yards or so.
For a starter, I would pick any good 44 that you like with at least a 7 1/2 barrel, put a good variable scope on it and practice plenty of shooting with a shooting stick or solid rest. If you want to go to the best one right away, get the S&W 460XVR with 8 3/8 barrel with a good variable scope and you won't have to buy another hunting handgun for a long time!! Good luck and let me know if I can answer any questions for you.
I have read a few accounts of a hunting expedition by Maj Doug Wesson in 1935 to promote the new 357 Magnum cartridge.
some of the game he took were:
Antelope - 200 yards, 2 shots.
Elk - 130 yards, 1 shot.
Moose - 100 yards, 1 shot.
Grizzly Bear - 135 yards, 1 shot.
Antelope was first hit at 125 yards. It ran & stopped and was shot the second time at 200 yards, The second shot killed it!
Bull Elk was killed with one shot! (through the lungs).
Moose was hit in the chest near the base of the neck, It cut the second rib and passed through both lungs, shearing the 8th rib on the off side and stopped just under the hide. No follow-up shot was needed!
Animals were taken on a Fall hunt, near the West entrance of Yellowstone Park, in Wyoming.
The Grizzly was taken in Canada at a later date.
All game was taken using factory loads, of the day, which were a 158 grain bullet at 1515 fps. from a 8 3/4 inch (not 8-3/8 inch) barreled Smith & Wesson Large frame revolver, that produced 812 ft. lbs of muzzle energy.
this was documented in several magazines of the time and the article was reran in 2009 in American Handgunner Magazine. The only animals I am thinking of taking are whitetail and maybe a pig.
I'm very confident in the 357 to do the job, however I put my hands on a new Ruger Super Redhwawk 44 Mag yesterday and it has bitten me and I may be in love with it. So I can use a S&W 357 I already own and mount a scope on it or go with a new Redhawk. I was pleased to see that the Ruger's all come with the scope rings in the box