It makes me think of a common trick in stargazing that is to look at stars with peripheral vision. Fixating them will make them disappear.
That's because the fovea, the "high resolution" part of the eye we use when focusing on some point is entirely made of cone cells, which give you color vision, but are less sensitive, as opposed to rod cells, that make up most of the peripheral vision. In other words, peripheral vision is better in low light situations.
I discovered this growing up when lying in bed at night. I'd always feel like there was light coming under the door. Then I'd look directly at it and it was pitch black to me. I'd look to the side and the light would seemingly come back. Only years later did I discover the cause you described.
I discovered it similarly in a camping trip when I was a kid. We were walking around in near pitch darkness, and I saw out of the corner of my eye someone walk away from the group and try to hide behind a table or something. When I turned to look I couldn't see anything at all, but if I averted my eyes I could clearly see there was a person moving around crouched behind a table. I can't remember exactly, but I think I wasn't so much seeing them, but rather their movement, and mentally perceiving their silhouette.
This was used as a magical trait in Tamora Pierce's Circle of Magic books. One of the characters has the ability to see magic and beings at the edge of her vision ("around" her glasses).
That's because the fovea, the "high resolution" part of the eye we use when focusing on some point is entirely made of cone cells, which give you color vision, but are less sensitive, as opposed to rod cells, that make up most of the peripheral vision. In other words, peripheral vision is better in low light situations.