With around 34,000 North Korean defectors having arrived in South Korea (as of June, 2021), perceptions toward them remain ambiguous and unbalanced. The dominant discourse about North Korean defectors centers on adaptation, and cultural difference is often identified as one of the most challenging obstacles. This article examines how a specific conceptualization of culture is utilized to alienate North Korean defectors, while securing the belief in a single ethnicity of all Koreans. As a result, North Korean defectors are rendered as cultural other in South Korean society. While cultural difference is often believed to be the basis of discrimination for North Korean defectors, this article argues that social prejudice and discrimination reproduce and reinforce the discourse about cultural difference of North Korean defectors.