When we class something as sexist or racist we tend to be looking at the bigger picture, whether those beliefs and attitudes are reflected in society. For example the racism against black people in America is a more important issue than single cases of harassment because it's direct oppression recurring throughout the society (more cases with the police keep happening).