That’s definitely for a detent, not a set screw. This is probably a Defiant or some other cheapo knob.
OP: The detent probably isn’t in line with that little hole so to be able to press it you will likely have to rotate the knob CW or CCW in order to be able to push it in.