I think I’ve settled on the latter. Disagreement is maybe best communicated by the absence of an upvote? And downvotes work best when they signal something that is just off base, and while not reportable, is not appreciated at a broad cultural level.
Originally up- and down votes were intended to crowd source filtering and rating content in a community. So voting up for things you want to see more of and vote down spam or content that is unfit for the community. But people will tend to upvote things they agree with and downvote those they deem wrong - I also find myself doing something like that. I now try to follow these rules:
It helps that lemmy currently shows the number of up and down votes instead of just the score, it gives a bit more inhibition before downvoting stuff.