Update, 7/28/29, 11:00 a.m PT: Userbenchmark has seemingly changed course from its previous statements, which have been almost entirely removed from its FAQ. That section of the FAQ is now replaced by a related section on the criticism Userbenchmark received and what will happen moving forward. The overall message isn't totally different, but Userbenchmark is changing its weighting very soon.
"Following the July 2019 cohort of new CPUs, we noticed that our CPU gaming and desktop indices were overestimating all CPUs with core counts beyond 8 so we updated the index. We estimate that our updated index, is now accurate to around 8% over the entire spectrum of 8500+ CPUs. There was an uproar from certain online communities with accusations of bias. Here are some of the actual outcomes arising from the latest update: The Ryzen 3000 effective speed ranks were impacted as follows: 3900X -2, 3800X +7, 3600X +14, 3600 +13. On the other hand the AMD Threadripper CPUs were demoted and the new top spot for gaming was taken by the Intel 9900K, up from its previous rank of 7."
Userbenchmark still stands by its changes, but does acknowledge the criticism and said "We frequently tune our effective speed indices and expect to add an octa-core component to the index in due course. Replacing our 64 core weight of 2% would be equivalent to, up to a maximum weight of 16% on an octa-core component." That will better reflect the performance gains associated with eight-core CPUs. The company also removed its strongly-worded warning about an "organized army of shills."
Original Article, 7/27/19, 10:54 a.m. PT:
While it is not a benchmark that most reviewers use, Userbenchmark has hundreds of thousands of benchmark submissions in its database that influence the purchasing decisions of nearly 10 million visitors each month (SimilarWeb), which is why there has been so much controversy among some enthusiasts over Userbenchmark's recent decision to adjust its score weighting mechanism. Now the average score is weighted much more heavily in favor of single-threaded performance than multi-core performance due to "unrealistic" scores from all CPUs over eight cores, which includes AMD Ryzen 3000 processors.
This has riled AMD fans, but the issue actually impacts the rankings for all processors, as evidenced by the test result above. Due to the $120 quad-core Intel Core i3-8100's higher boost frequency, it beats Intel's $2,000 18-core 36-thread Core i9-9980XE in the rankings. It doesn't take a degree in computer science to see the obvious disconnect here.
Previously, Userbenchmark weighed single-core performance as 40% of the score, quad-core as 50%, and multi-core as 10%. However, due to the "unrealistic" scores of many-core CPUs like Ryzen 3000, Userbenchmark changed their weighing system to 40% single-core, 58% quad-core, and 2% multi-core.
A new question and answer section on Userbenchmark's FAQ address complaints about this recent change. The question "what is the effective CPU speed index?" has three new sections, one of which is titled "AMD community" to address AMD fans. Userbenchmark states that while they "emphatically welcomed" Ryzen 3000, they felt that high core-count CPUs were scored too highly and that the scores didn't reflect the reality of software, stating:
Back in the days of the AMD FX-8350 our effective speed index was predominantly single core and at that time we were heavily lobbied with cries of "cores are only getting more and more relevant." Any professional software developer that has actually tried to write scaleable multi-threaded code will understand that the challenges are both far from trivial and highly unlikely to be overcome during the lifetime of a typical CPU. We frequently tune our effective speed indices to match the latest developments
Though Userbenchmark does not state it directly, presumably they feel that CPU performance on a single thread is becoming more and more relevant than performance over multiple cores. While single-threaded performance is still important, it's certainly not becoming more important than multi-core performance. Games are slowly becoming more and more friendly to multiple cores, and as frequency gains for CPUs slow down and eventually reverse into frequency drops, single-thread performance will be difficult to improve, arguably making multicore performance a more reliable way to speed up a processor.
Userbenchmark finishes its answer pointing out that nobody is complaining about the RX 5700 series GPUs being "overestimated" and gives a warning about an "organized army of shills who pump one brand or another and deal in hot air rather than reason." Though we can't say whether or not Userbenchmark is at fault here, their response to critics certainly won't be met with much positivity.
