4000 random people does mean something, you can extrapolate the data with it, it’s the whole point of surveys. Of course, the numbers won’t be very accurate, but it gives a good approximate.
You’re assuming that the set of people surveyed was truly random, but it never is—people who don’t like answering surveys are always underrepresented (obviously), and the article isn’t specific about how people were recruited for this one. There isn’t enough information to tell how much skew might have been introduced as a result. Surveys are always kind of iffy as information sources: not meaningless, but with a lot of subtle noise in the signal.
The skew is something they’re supposed to take into account, if they realize a given factor is important. Models are seldom perfect, and people can be incompetent or self-centered even when they’re not being actively malicious. I think you’re overestimating both the quality of the statistical models and the purity of the motives behind this survey.
Does it though? I’d think you’d probably want at a minimum, n = 100,000 to have any sort of representative sample. I don’t think it’s a stretch to surmise that there are at least, what…a hundred million anime fans in one capacity or another, worldwide? Anything much less than 0.1% of the actual population is susceptible to some major deviation from population-wide statistics.
The problem is that this assumes a perfectly normal distribution. There’s a possibility that anime fandom likely doesn’t follow a Gaussian distribution (dare I say it’s almost certainly not Gaussian, because anecdotally, people seem to either be neck deep into it, or are disgusted by it, with nary anything in between). If this is true, the above calculator doesn’t exactly work.
It’s been a while since I took statistics, but yes, I guess that is a binomial distribution. It does not influence the results in the way you are implying it does, though. The calculator does actually account for it (the Population Proportion input), and the sample size actually decreases the lower/higher your proportion is. My point was that a question like, “Do you watch anime weekly,” is not like a question like, “How many hours of anime do you watch in a week,” where you certainly couldn’t assume a normal distribution for the number of hours watched.
Don’t worry, survey models in competent surveys company are made by peoples competent in statistics.
The number of people needed to survey is something you can calculate.
FYI I dont know the reputation of americans survey companies, nor found a survey report, so we can’t really judge this except saying the articles should share the survey report
I mean, I would hope that a company that exists to survey people would be competent in said field. But as someone who has a diploma in maths, I’ll say that I do think four thousand people simply isn’t enough.
4000 random people does mean something, you can extrapolate the data with it, it’s the whole point of surveys. Of course, the numbers won’t be very accurate, but it gives a good approximate.
You’re assuming that the set of people surveyed was truly random, but it never is—people who don’t like answering surveys are always underrepresented (obviously), and the article isn’t specific about how people were recruited for this one. There isn’t enough information to tell how much skew might have been introduced as a result. Surveys are always kind of iffy as information sources: not meaningless, but with a lot of subtle noise in the signal.
The skew is something they take in account. They have models to take this into account.
The skew is something they’re supposed to take into account, if they realize a given factor is important. Models are seldom perfect, and people can be incompetent or self-centered even when they’re not being actively malicious. I think you’re overestimating both the quality of the statistical models and the purity of the motives behind this survey.
I already said I couldn’t judge it since they didn’t published any report.
Does it though? I’d think you’d probably want at a minimum, n = 100,000 to have any sort of representative sample. I don’t think it’s a stretch to surmise that there are at least, what…a hundred million anime fans in one capacity or another, worldwide? Anything much less than 0.1% of the actual population is susceptible to some major deviation from population-wide statistics.
You don’t need a massive sample size for surveys to give meaningful information. Play around with this sample size calculator if you want to see what the margins of error are: https://www.calculator.net/sample-size-calculator.html?type=2&cl2=95&ss2=4000&pc2=5&ps2=500000000&x=Calculate
The problem is that this assumes a perfectly normal distribution. There’s a possibility that anime fandom likely doesn’t follow a Gaussian distribution (dare I say it’s almost certainly not Gaussian, because anecdotally, people seem to either be neck deep into it, or are disgusted by it, with nary anything in between). If this is true, the above calculator doesn’t exactly work.
Normal distribution with regards to what? “Do you watch anime weekly” is a binary question. There really isn’t a distribution associated with that.
Wouldn’t that be a binomial distribution then?
It’s been a while since I took statistics, but yes, I guess that is a binomial distribution. It does not influence the results in the way you are implying it does, though. The calculator does actually account for it (the Population Proportion input), and the sample size actually decreases the lower/higher your proportion is. My point was that a question like, “Do you watch anime weekly,” is not like a question like, “How many hours of anime do you watch in a week,” where you certainly couldn’t assume a normal distribution for the number of hours watched.
Yeah I didn’t look at the exact questions. My bad.
Don’t worry, survey models in competent surveys company are made by peoples competent in statistics.
The number of people needed to survey is something you can calculate.
FYI I dont know the reputation of americans survey companies, nor found a survey report, so we can’t really judge this except saying the articles should share the survey report
I mean, I would hope that a company that exists to survey people would be competent in said field. But as someone who has a diploma in maths, I’ll say that I do think four thousand people simply isn’t enough.