You're basically asking what the limits of human vision are. And you're in luck, there was a paper published this summer answering that question (summary here). The tl;dr of the paper is that the human eye can detect a single photon. If it happens to be one of the 10% of photons that enter your eye and actually trigger a rod cell.
“The most amazing thing is that it’s not like seeing light. It’s almost a feeling, at the threshold of imagination,” says Alipasha Vaziri, a physicist at the Rockefeller University in New York City, who led the work and tried out the experience himself.
If you want to "see" 1 photon per second, then if my math is right (assuming your pupil has a radius of 4mm), you could go up to 300 km away and still get that one photon per second entering your eye.
EDIT: People below have brought up some interesting points. 1 photon/sec isn't really "seeing" it constantly. Humans seem to have a 'frame rate' of ~100 fps, so say we want to see 1 photon per 'frame', and assume that only 10% of the photons entering our eye hit the rod and trigger this photon perception.
Then, to constantly be registering ~1 photon in your eye from the candle the answer shifts to a distance of 9 km(5.5 miles). Which is still crazy far away for a single candle.