I set up trail cameras on public land, in an area that I always see/hear deer anytime I go to or past that spot. Last season I had as many, if not more sightings of bucks than does in that area. (I only hunted a few times in November)
It took about a week until the deer started showing back up in that area, but it has only been does and fawns so far. As a second year deer hunter I have a hard time reading the subtle signs, like last year’s rub scarring. So I just set the camera up on a heavily used trail and near a spot where I saw buck tracks crossing the creek-bed.

Should I ever get concerned if I only ever see does in the area, or trust that once the rut comes around the does will stay and draw the bucks in?

I would not be concerned at all. If they does are there, the bucks will be there in the rut. What probably has changed is the preferred food source the bucks were eating last year to this year. Last year that area had something they wanted or preferred and this year it is not there. So now they have something more preferred in another location.

I hunt a small tract in East Texas that has excellent bedding and fawning cover on it. During the off season I only see does during the day and get a few photos of bucks at night. However, once the rut kicks in, you might say the place turns into a dance hall where all the local bucks go to find female companionship.

Same experience here.

This time of year....I see mostly Does, bucks come filtering in about October, but by the 1st week of November I am covered up in bucks.

The bucks will be where the ladies are.....so do all you can to keep the ladies on your place.

The bucks will come, you may even see less does during the season once they are pressured. Stay out of the area and let the girls be happy for now.

Loook for plan B in small pockets within 500 yards of your set up.

i'm having the thing on my land in Oklahoma. Nothing but does

At least you have deer

Just remember this, Bucks like Doe's when the Rut comes......

I almost never see any bucks during the off-season on my place. It is a definite doe sink and fawn rearing ground, with an occasional yearling scrub buck still trying to hand around mama.

But around October 1st every year like clockwork, the bigger boys start snooping around. They hang out for a few days, then vanish until the first week of November... then it's on like crazy.