Eleven residents, six program directors/associate program directors, and CCCs from the three residencies were recruited. Forty interviews and CCC observations were conducted. Participants constructed meaning from evaluations by negotiating across several sources of tension. Residency programs had created criterion-referenced rating scales from ACGME competencies and milestones. However, residents and leaders typically interpreted these rating using comparisons with peers in the same level of training (i.e. a normative interpretation). Residents and leaders identified narrative comments as the most useful source of competency judgements, though comments were sometimes sparse and generic. These comments were often necessary to interpret the numerical ratings despite anchors based in milestone language. While residents were grateful for positive comments, they frequently expressed desire for critique of their performance, which was often absent. Both residents and leaders expressed some ambivalence as to whether critique should be delivered in writing or verbally.