In addition (or rather, as part of) John Stuart Mill's moral critique of the Southern secessionists, he asks:
Before admitting the authority of any persons, as organs of the will of the people, to dispose of the whole political existence of a country, I ask to see whether their credentials are from the whole, or only from a part. And first, it is necessary to ask, Have the slaves been consulted? Has their will been counted as any part in the estimate of collective volition? They are a part of the population. However natural in the country itself [meaning the United States], it is rather cool in English writers who talk so glibly of the ten millions [of southerners]…., to pass over the very existence of four millions who must abhor the idea of separation.
More on this theme from Seth Barrett Tillman:
First, we do not have good evidence that even a majority of the adult white males in each rebel state supported secession at the time purported state conventions issued their ordinances of secession. The secession conventions were hardly models of transparency or one-(white)-man-one-(white)-vote equality in terms of fair representation. See, e.g., Akhil Reed Amar, America’s Constitution: A Biography 354 (2005) (explaining—in model clarity—that “state-secession votes occurred in assemblies skewed by state-law variants of the federal three-fifths clause—laws that gave plantation belts undue weight in the ultimate outcome”). I am in Ireland now, and so, I do not have easy access to much American material, but (as I remember) there is good authority for the view that a majority of the adult white males in Georgia did not support secession in 1861.
Second, if secession were/is a valid political principle, even absent concrete and substantial wrongdoing by the government from which one is seceding, then it does not stop with states seceding from the federal government. Many Southern states had counties and large regions where the white population was overwhelmingly loyal to the Union. E.g., Northern Alabama and Mississippi, western Virginia (prior to recognition of West Virginia statehood as the legitimate successor to rebel Virginia). Likewise, every rebel state (South Carolina excepted) produced organized loyalist regiments, and even South Carolina sent many white men who enlisted in Union regiments. Those loyalist counties and regions also produced active pro-Union militias in their home states. At no time did any rebel assembly or governor allow these counties and regions to remain in the Union or secede from the rebel state. The rebel position was never secession pure and simple, but secession in the context of organic, fixed, immutable states—against all. Secession was something rebels could do to others (based on a whim and fear of future wrongdoing at the hands of the newly elected Lincoln administration), but not which others could do to them (to the extent that rebels murdered Unionists, exiled them, and had their homes burned to the ground merely because loyalists continued to express political sympathy for the flag their fathers and grandfathers fought and bled for).
The problem of vindicating the citizenship of minority unionists trapped behind the lines of territorial secession, and to ensure that the territory claimed by secessionists bears some proportion to the popular sentiment of its inhabitants, is perhaps the most fraught of the issues implicated in a generalized support for self-government. It certainly applies to the North Alabamans . . . as it did to the Confederates of the Allegheny Mountain region of West Virginia, the Tories after the War for Independence, the Sudenten Germans in the 1930s, and the ethnic Russians of the Baltics in the 1990s. Ideally, such issues can be resolved by negotiation, but as these examples illustrate, by the time secession has been gotten around to, idealism of any kind is in short supply. The depressing reality seems to be that the strong do as they will, the weak endure as they must.
But to the extent we look to reasoning rather than force of arms to decide the matter, I want to inspect the claims made on behalf of the Union loyalists.
The slaves didn't vote. True enough. And freedmen in the North were allowed to vote in only four states at the time of the Civil War. (I researched this, but if this or any other of the historical assertions I make in this post are shown to be in error, I will correct them.)
The votes of poor whites were diluted. Also generally true in the North. Indeed, this was established practice in may states until the Supreme Court decision in Reynolds vs. Simms in 1964; it probably wasn't much questioned at the time of the Civil War. Can the North claim in good faith to vindicate rights they themselves do not observe?
Blacks and/or poor whites did not support secession, and whose equal franchise would have blocked it. This is certainly a plausible assertion, and is probably what is got at by the first two objections above. But is there any actual evidence for it? I can think of very little that the industrial capitalism of the 1860 North offered to either blacks or poor whites that was much superior to what they had: cradle-to-grave care, however mean, for slaves, and some measure of rough independence for poor whites. I am struck by the fact that even after emancipation, many slaves chose to remain with the life they knew, even in the service of such as Alexander Stephens.
The South only wanted "secession in the context of organic, fixed, immutable states". This was certainly the South's strongest legal (as opposed to moral or philosophical) case under the existing political arrangements, and arguments to the contrary strike me as particularly bad. The Federal union was the creature of the state governments, not the other way around, and I see very little in American history prior to the war supporting the notion that union ought be anything other than voluntary. The same cannot be said of sub-state regions. That doesn't vitiate any claims to independence these regions might make, only to say that those claims are philosophical rather than legal.
The South did not respect loyalist regional predominance. True enough in the cases cited above. But while I would recognize the right of Northern Alabamans to self-government, it is not at all clear that they were actually seeking to be a land-locked Union micro-province in the heart of the Confederacy; rather, they supported the status quo ante: a unified republic. That is certainly respectable, but it doesn't mean that they opposition they faced was merely a matter of the South failing to live up to its principles.
Which is not to say that in its drive for independence, the South lived up to its principles. This history of the run-up to secession is chuck-full of nasty, vindictive, and even murderous behavior by . . . well, most everybody, the South included. It would be very difficult to find a legitimate claimant to self-government less sympathetic than the antebellum South. Knowing where it would eventually lead, I can't help but wonder where were the Henry Clays of 1860, the voices of moderation that might have talked everyone down from the precipice of war if not secession.