By the time Grant began his second term, Congress was already considering bills to restore officeholding rights to former Confederates who had not yet sworn allegiance to the Union. Black representatives, including Georgia congressman Jefferson Long, as well as some white lawmakers, remained opposed to such measures, but in 1872 Congress removed the penalties placed on former Confederates by the Fourteenth Amendment and permitted nearly all rebel leaders the right to vote and hold office. Two years later, for the first time since the start of the Civil War, the Democrats gained a majority in the House of Representatives and prepared to remove the remaining troops from the South.

Republican leaders also rethought their top priority with economic concerns increasingly replacing racial considerations. In 1873 a financial panic resulting from the collapse of the Northern Pacific Railroad triggered a severe economic depression lasting late into the decade. Tens of thousands of unemployed workers across the country worried more about finding jobs than they did about black civil rights. Businessmen, too, were plagued with widespread bankruptcy. When strikes erupted across the country in 1877, most notably the Great Railway Strike, in which more than half a million workers walked off the job, employers asked the U.S. government to remove troops from the South and dispatch them against strikers in the North and West.

While white Northerners sought ways to extricate themselves from Reconstruction, the Supreme Court weakened enforcement of the civil rights acts. In 1873 the Slaughterhouse cases defined the rights that African Americans were entitled to under the Fourteenth Amendment very narrowly. Reflecting the shift from moral to economic concerns, the justices interpreted the amendment as extending greater protection to corporations in conducting business than to blacks. As a result, blacks had to depend on southern state governments to protect their civil rights, the same state authorities that had deprived them of their rights in the first place. In United States v. Cruikshank (1876), the high court narrowed the Fourteenth Amendment further, ruling that it protected blacks against abuses only by state officials and agencies, not by private groups such as the Ku Klux Klan. Seven years later, the Court struck down the Civil Rights Act of 1875, which had extended “full and equal treatment” in public accommodations for persons of all races.