The NIHSS is widely used to assess stroke severity, including language impairment. However, its picture-based language items may be culturally specific and less applicable in diverse settings. Updated images have been introduced to improve global relevance, but their clinical performance in Zambian populations has not been systematically examined.

Fifty participants were included. Agreement with clinical assessment was 54% for both image-based tasks. Weighted kappa indicated fair agreement (Cookie Theft: κ=0.456; Precarious Painter: κ=0.592). Bowker’s test revealed significant asymmetry (p<0.01 for both). Patients with hemorrhagic stroke performed worse than those with ischemic stroke, and Cookie Theft performance differed significantly by language group (p=0.0055). No significant correlations were observed between image-based scores and overall NIHSS totals.

In this Zambian cohort, image-based NIHSS language tasks demonstrated poor agreement with clinical examination, particularly among patients with hemorrhagic stroke and non-English speakers. Findings suggest that cultural and linguistic incongruities may contribute to systematic bias, potentially overestimating language impairment and stroke severity as classified by the NIHSS. These results highlight the need for culturally adapted assessment tools to ensure accurate stroke evaluation in African settings.