Among 42,306 participants, 2,351 participants carried rare protein-disrupting variants; 1,402 carried pathogenic variants, among which 113 carried CADASIL-causing variants. Higher total WMH were found in carriers of any rare variants (β[95%CI] = 0.064[0.029,0.110], p<0.001), EGFr-involving variants (β[95%CI] = 0.064[0.025,0.104],p=0.002), and pathogenic rare variants (β[95%CI] = 0.119[0.074,0.164], p<0.001). Carriers of pathogenic rare variants had increased stroke prevalence (OR[95%CI] = 1.419[1.046,1.881], p=0.019). After excluding CADASIL-causing variants, the effects of carrying any pathogenic variants remained significant in total WMH, deep WMH, and periventricular WMH (p=0.003, p=0.004, p=0.005 respectively), but disappeared in stroke.