Geekbench 6.7 adds Intel BOT detection to spoof out 'unrealistic' CPU scores — Benchmark runs with BOT enabled will be marked as invalid

News
By published

Panther Lake & Arrow Lake Refresh chips affected

Geekbench 6 logo
(Image credit: Primate Labs)

Intel's latest Core Ultra 200 Plus (Arrow Lake refresh) and Core Ultra 300 (Panther Lake) chips support a new feature called Binary Optimization Tool (BOT) that improves performance in some apps. We've tested it extensively and found it to be largely consistent, but Geekbench developer Primate Labs says it "paints an unrealistic picture" in terms of daily usage. The latest update, Geekbench 6.7 came out today with BOT detection built-in to ensure benchmark results with it enabled are flagged as invalid.

Geekbench results with Intel BOT enabled

(Image credit: Future)

The issue is how biased these performance uplifts can be since BOT isn't supported in every app. Geekbench argues its benchmark is designed to evaluate against a broad range of workloads that accurately represent real-world performance; any test with BOT enabled goes against that by replacing the existing varied code with specifically tuned binaries that only two desktop CPUs (currently) benefit from.

Article continues below

BOT runs a checksum on every executable to identify and compile optimized binaries for it — that's how and why the previous version was affected. It seems BOT doesn't yet have the tuned binaries for Geekbench 6.7, so you'll see no difference in performance. Even when they're ready to apply, runs with BOT enabled will be marked invalid and incomparable to other CPUs in the database.

Intel Binary Optimization Tool

(Image credit: Intel)

The update also brings some other welcome changes, such as improved SoC identification on Android that now reports the actual model of the chip along with its architecture. Similarly, Geekbench 6.7 will now display the name of RISC-V processors instead of just their ISA string. And lastly, Arm-based Linux systems should be more stable running Geekbench's multi-threaded workloads.

Google Preferred Source

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

Hassam Nasir
Hassam Nasir
Contributing Writer

Hassam Nasir is a die-hard hardware enthusiast with years of experience as a tech editor and writer, focusing on detailed CPU comparisons and general hardware news. When he’s not working, you’ll find him bending tubes for his ever-evolving custom water-loop gaming rig or benchmarking the latest CPUs and GPUs just for fun.

13 Comments Comment from the forums
  • JamesJones44
    I get the idea and is Intel "cheating" here, there is at least an argument to be made. However, if we are actively tweaking overall benchmark scores simply because CPU A supports a feature and CPU B doesn't then where do you draw the line? Do AMD CPUs that support AVX 512 now get tests specifically tweaked and tweaked numbers simple because Apple and Intel don't support 512 bit vectorization on a large number of their products?
    Reply
  • usertests
    JamesJones44 said:
    I get the idea and is Intel "cheating" here, there is at least an argument to be made. However, if we are actively tweaking overall benchmark scores simply because CPU A supports a feature and CPU B doesn't then where do you draw the line? Do AMD CPUs that support AVX 512 now get tests specifically tweaked and tweaked numbers simple because Apple and Intel don't support 512 bit vectorization on a large number of their products?
    When AMD added AVX-512 support, they weren't optimizing the performance of very many applications, but applications that benefit do exist and it's automatic. And some people have been worried about AVX-512 disproportionately dragging up combined scores, and may have made tweaks to deemphasize it, or called attention to these outliers.

    Intel iBOT is a whitelist of a small number of the hundreds of thousands of games/applications that can possibly run on your computer. Correct me if wrong, but I think it's currently 12 games and one non-game: Geekbench. Out of everything they could have picked, they picked a benchmark meant to give an objective measurement of the PC's performance (even if it isn't well-regarded. I use PassMark instead).

    In summary, Intel deserves flak. Benchmarketing is afoot. iBOT may never be applicable to even 1% of software. But if you do use some popular game/app that it speeds up, enjoy the free performance.
    Reply
  • ejolson
    usertests said:
    When AMD added AVX-512 support, they weren't optimizing the performance of very many applications, but applications that benefit do exist and it's automatic. And some people have been worried about AVX-512 disproportionately dragging up combined scores, and may have made tweaks to deemphasize it, or called attention to these outliers.
    My understanding is that if iBOT speeds an executable up by 30 percent this mostly indicates the developers did not expend the effort to properly vectorise that executable in the first place. Thus, iBOT means the work needed to vectorise each executable ahead of time can be skipped because vectorisation is done at runtime.

    I think you can go into the correct control panel and turn iBOT on for any program you want to see what happens.
    Reply
  • Gururu
    I neither play Geekbench nor read its benchmarks. How will this affect my game performance? Do games need to explicitly say iBOT certified, or will Intel driver updates indicate which games are optimized for it?
    Reply
  • hotaru251
    JamesJones44 said:
    However, if we are actively tweaking overall benchmark scores simply because CPU A supports a feature and CPU B doesn't then where do you draw the line?
    less its just the feature and more its the feature that is only on 2 cpu and also is extremely nichely used in wild.

    both compound into something that isn't the average use case which is what their goal seems to be.
    Reply
  • JamesJones44
    usertests said:
    In summary, Intel deserves flak. Benchmarketing is afoot. iBOT may never be applicable to even 1% of software. But if you do use some popular game/app that it speeds up, enjoy the free performance.
    My intention isn't to excuse Intel or argue iBOT in general. My argument was the justification from Geekbench, in which they claim this is needed due to the limited support nature of the feature. Which seems like the wrong justification because CPUs support various different enhancements product to product, vendor to vendor, generation to generation, etc.

    If Geekbench came out and said we are doing this because Intel targeted us with iBOT just to make Intel look better as justification I wouldn't have posted my comment as that would seem justified.
    Reply
  • JamesJones44
    hotaru251 said:
    less its just the feature and more its the feature that is only on 2 cpu and also is extremely nichely used in wild.

    both compound into something that isn't the average use case which is what their goal seems to be.
    Assuming Intel never uses this again on any future products, I can see that argument
    Reply
  • Gururu
    Intel lists these as using iBOT:
    Assassin's Creed Mirage*
    Borderlands 3*
    Cyberpunk 2077*
    Far Cry 6*
    Final Fantasy XIV*
    Geekbench* 6.3+ (Proof of Concept)
    Hitman 3*
    Hogwarts Legacy*
    Marvel’s Spider-Man Remastered*
    Naraka: Bladepoint*
    Remnant 2*
    Shadow of the Tomb Raider*
    Tiny Tina's Wonderlands*
    Reply
  • thestryker
    usertests said:
    When AMD added AVX-512 support, they weren't optimizing the performance of very many applications, but applications that benefit do exist and it's automatic. And some people have been worried about AVX-512 disproportionately dragging up combined scores, and may have made tweaks to deemphasize it, or called attention to these outliers.
    While this is a real problem I think so long as it's noted it's fine. It definitely throws off big benchmark runs like Phoronix does which makes it annoying when they don't split them up in the conclusion area though.

    I'm ambivalent towards whether or not Geekbench considers IBOT results legit, but it very much needed to be noted either way.
    usertests said:
    In summary, Intel deserves flak. Benchmarketing is afoot.
    They deserve flak because they did a proof of concept of a non gaming benefit on a program everyone can test with and were very clear and open about what they were doing? :rolleyes:

    Maybe if this was a feature which was on by default upon installing required support software I'd agree with you, but since it requires intent very much nope.
    ejolson said:
    I think you can go into the correct control panel and turn iBOT on for any program you want to see what happens.
    Nope it's a whitelist situation in its current implementation. There is no ability to turn it on for specific software or universally on.
    Gururu said:
    Do games need to explicitly say iBOT certified, or will Intel driver updates indicate which games are optimized for it?
    Currently the way it functions is as part of APO. Until the release of ARL refresh it was a Windows Store application that you could install on any 12th Gen+ part as long as DTT was installed (on desktop this was almost never supplied by motherboard vendors until APO came out). APO basically just has a list of games and you can enable it or not, and now IBOT is another column in APO. APO when installed defaults to on for everything and IBOT is off for everything. For the current implementation it's a case by case whitelist from Intel.
    hotaru251 said:
    less its just the feature and more its the feature that is only on 2 cpu and also is extremely nichely used in wild.
    It supports every PTL SKU that doesn't have the big Graphics Tile. Robert Hallock also said in plain english (as opposed to marketing speak) it would be supported with every forthcoming Intel CPU. I also got the impression from one of the engineers working on it that they're trying to figure out a way to tie the functionality of APO and IBOT together which may make it simpler to validate.
    Reply
  • ravewulf
    The goal of benchmarking is to measure the performance of the hardware using stock software suites, meaning the software should be identical between platforms wherever possible. Intel BOT goes against that and is not a hardware feature.

    It's basically the same as when the Intel compiler generated optimized binaries for Intel CPUs and would run worse codepaths on non-Intel processors, just applied at runtime instead of at compile time.

    https://en.wikipedia.org/wiki/Intel_C%2B%2B_Compiler#Support_for_non-Intel_processors
    Reply