(CNN) - Outbursts like Donald Sterling's recorded rant or Nevada rancher Cliven Bundy's controversial comments are not the real problem, says Attorney General Eric Holder.
"More subtle" forms of racism that "cut deeper" cause the real harm, he argued Saturday.
In a commencement speech at Morgan State University, a historically black college in Baltimore, Holder used the 60th anniversary of the Brown v. Board of Education ruling that desegregated schools to highlight inequalities that persist today.
[twitter-follow screen_name='politicalticker'][twitter-follow screen_name='ConorCNN']
Referencing the "systematic and unwarranted racial disparities" of the criminal justice system and the "moral failings" of voter identification laws, Holder argued legitimate problems usually get lost in the din over major public incidents.
Most recently, Los Angeles Clippers owner Donald Sterling caused outrage after he went on a racist rant in a leaked audio recording. Bundy, who is challenging the federal government over land fees, suggested blacks may be better off as slaves.
While it is good to see such remarks condemned swiftly and widely, Holder said, it often misses the point.
"If we focus solely on these incidents - on outlandish statements that capture national attention and spark outrage on Facebook and Twitter - we are likely to miss the more hidden, and more troubling, reality behind the headlines," he said.
"Policies that disenfranchise specific groups are more pernicious than hateful rants," he continued. "Proposals that feed uncertainty, question the desire of a people to work, and relegate particular Americans to economic despair are more malignant than intolerant public statements, no matter how many eyebrows the outbursts might raise. And a criminal justice system that treats groups of people differently - and punishes them unequally - has a much more negative impact than misguided words that we can reject out of hand."
The attorney general went on to speak to specific issues.
In particular, he addressed the higher incarceration rates and longer sentences that African-American men face. African-American men have received sentences that are nearly 20% longer than those imposed on white males convicted of similar crimes, said Holder, citing the U.S. Sentencing Commission. Black men are also more than six times as likely as white men to be incarcerated, according to the Pew Research Center.
While schools may no longer be segregated legally, some districts have moved in that direction, said Holder, also arguing that disciplinary policies often unfairly target young black men.
"In too many of our school districts, significant divisions persist and desegregation has reoccurred - including zero-tolerance school discipline practices that, while well-intentioned and aimed at promoting school safety, affect black males at a rate three times higher than their white peers," he said.
Laws may forbid racism, but "there are other policies that too easily escape such scrutiny because they have the appearance of being race-neutral. Their impacts, however, are anything but," he said.
To Holder, voter identification laws exemplify the problem best.
"In too many jurisdictions, new types of restrictions are justified as attempts to curb an epidemic of voter fraud that - in reality - has never been shown to exist," he said, and instead they "disproportionately disenfranchise African-Americans, Hispanics, other communities of color, and vulnerable populations such as the elderly."
Holder even went after Chief Justice John Roberts. In April, the U.S. Supreme Court upheld a Michigan law banning the use of racial criteria in college admissions. Last June, the court struck down a key provision of the historic Civil Rights Act.
At the time, Roberts said that the section was no longer necessary because "our country has changed" for the better. Holder disagrees.
"Chief Justice John Roberts has argued that the path to ending racial discrimination is to give less consideration to the issue of race altogether. This presupposes that racial discrimination is at a sufficiently low ebb that it doesn't need to be actively confronted," he said.
Holder concluded by calling for a national conversation about race. More dialogue, he argued, can lead to more progress.
–CNN's Evan Perez contributed to this report.