Phosphene sensation is commonly used to measure cortical excitability during transcranial magnetic stimulation (TMS) of the occipital cortex. However, some individuals lack this perception, and the reason for it is still unknown. In this work, we used functional magnetic resonance imaging (fMRI) to detect brain activation during local TMS of the occipital cortex in twelve healthy subjects. We found that TMS modulated brain activity in areas connected to the stimulation site, even in people unable to see phosphene. However, we observed a trend for a lower bloodoxygenation-level dependent (BOLD) signal, and smaller brain-activation clusters near the stimulated site than in the interconnected brain areas, suggesting that TMS pulse is more effective downstream than at its application site. Furthermore, we noted prominent differences in brain activation/deactivation patterns between subjects who perceived phosphene and those who did not, implying a functional distinction in their neuronal networks that might explain the origin of differences in phosphene generation.