I understand what you’re saying, but you’re missing context.

The Federalists were anti-slavery and its expansion. Regardless of what Lincoln said in his initial address, he wasn’t the only one who was anti-slavery.

How do we know all this? We can look at sources that provide historical context, like this for example,

https://billofrightsinstitute.org/essays/the-missouri-compromise

In 1819, Missouri wished to be admitted as a slave state because enslaved persons had already been brought to the region and were an important part of its local economy.

Northern politicians and, indeed, regular citizens had become concerned with what they considered southern dominance of the federal government, an influence that would only be enhanced with the addition of another slave state. The Three-Fifths Clause in Article I, Section Two of the Constitution provided for the counting of three-fifths of the slave population for purposes of determining representation in Congress. That rule gave southern states more congressional representatives than warranted by their white population and, hence, more electoral votes for president. As a result of the three-fifths rule, southern presidents became the norm after John Adams, and much of the federal government was staffed or run by southerners, from the judiciary to the chairs of key congressional committees.

This dominance had begun to grate on perceptive northern politicians, who used the phrase “Slave Power” to refer to southern political control of the federal government. Missouri as a slave state would simply cement that supremacy, and worse still, because it was located west of the Mississippi in the as-yet unsettled Louisiana Purchase region, its admission might mark the beginning of the creation of more slave states and thereby render perpetual the South’s control of the federal government.

That concern prompted New York’s newly sworn congressional representative James Tallmadge to introduce two amendments to the Missouri enabling resolutions. The first prohibited the further introduction of slaves into Missouri, and the second freed the children of existing slaves in Missouri when they reached the age of twenty-five years. Together these amendments would gradually end slavery in Missouri, and they passed the House by a northern majority in a sectional vote. Therefore, Missouri’s enabling act would not add to southern control of the federal government.