No, i mean ‘should’, as in:

There’s no reason to expect a program that calculates the probability of the next most likely word in a sentence should be able to do anything more than string together an incoherent sentence, let alone correctly answer even an arbitrary question

It’s like using a description for how covalent bonds are formed as an explanation for how it is you know when you need to take a shit.