The majority of slaveless white Southerners were hardworking, landholding small farmers, but Northerners held a different image of this group. They believed that slavery had condemned most whites to poverty and backwardness. One antislavery advocate charged that the South harbored three classes: “the slaves on whom devolves all the regular industry, the slaveholders who reap all the fruits, and an idle and lawless rabble who live dispersed over vast plains little removed from absolute barbarism.” Critics called this third class a variety of derogatory names: hillbillies, crackers, rednecks, and poor white trash. According to critics, poor whites were not just whites who were poor. They were also supposedly ignorant, diseased, and degenerate.

Contrary to northern opinion, only about one in four nonslaveholding rural white men was landless and very poor. Some worked as tenants, renting land and struggling to make a go of it. Others survived by herding pigs and cattle. And still others worked for meager wages, ditching, mining, logging, and laying track for railroads.

Some poor white men earned reputations for mayhem and violence. One visitor claimed that a “bowie-knife was a universal, and a pistol a not at all unusual companion.” Edward Isham, an illiterate roustabout, spent about as much time fighting as he did working. When he was not engaged in ear-biting, eye-gouging free-for-alls, he gambled, drank, stole, had run-ins with the law, and in 1860 murdered a respected slaveholder, for which he was hanged.

Unlike Isham, most poor white men did not engage in ferocious behavior but worked hard and dreamed of becoming yeomen. The Lipscomb family illustrates the possibility of upward mobility. In 1845, Smith and Sally Lipscomb and their children abandoned their worn-out land in South Carolina for Benton County, Alabama. “Benton is a mountainous country but ther is a heep of good levil land to tend in it,” Smith wrote back to his brother. Alabama, Smith said, “will be better for the rising generation if not for ourselves but I think it will be the best for us all that live any length of time.”

Because they had no money to buy land, they squatted on seven unoccupied acres. With the help of neighbors, they built a 22-by-24-foot cabin, a detached kitchen, and two stables. In the first year, Smith and his sons produced several bales of cotton and enough food for the table. The women worked just as hard in the cabin, and Sally contributed to the family’s income by selling homemade shirts and socks. In time, the Lipscombs bought land and joined the Baptist church, completing their transformation to respectable yeomen.

Many poor whites succeeded in climbing the economic ladder, but in the 1850s upward mobility slowed. The cotton boom of that decade caused planters to expand their operations, driving the price of land beyond the reach of poor families. Whether they gained their own land or not, however, poor whites shared common cultural traits with yeoman farmers.