Note: This is a transcript of the YouTube video and is not a 1:1 copy.
Alright guys how’s it going?
The world continues to spin and the tech world continues to dish out the drama. And that means it’s time for yet another report on everyone’s favorite anti-consumer and anti-competitive company – Intel.
2017
I can’t remember exactly what video it was, but a couple of years ago towards the end of the 2017, I said that 2018 would be a very dirtily fought year in tech. AMD had come back strong in 2017 with Ryzen and their sales had begun to pick up, but Intel still had a clear lead in single-threaded performance and in many games. A big concern for them though was the existence of Threadripper, which back in 2017 had 16 cores, and this huge increase in core count at the high-end forced Intel to change previous plans of first of all launching only 10 cores on the platform, then 12 before finally realizing that they had to launch the whole 18 just to stay ahead.
But most telling for me back in 2017 was how the server chip Naples scaled with extra cores and chiplets: very close to 100% scaling. This made me realize in my You’re beaten, Intel video that Zen was the real deal and at some point within the next couple of years Intel would likely be in a lot more trouble.
But we very quickly realized that they would not go out quietly and in the Epyc wins, Intel Prepares to Fight Dirty video, we learned of their plot to downplay Epyc as a “glued together, re-purposed desktop CPU”, while noting AMD’s dubious track-record on supply. AMD’s “supply issues” were in fact caused by Intel bribing OEMs like Dell and HP billions of dollars to not use AMD CPUs. AMD had such terrible “supply” issues that they couldn’t even give 1 million CPUs away to HP for free, because it wasn’t worth to HP losing Intel’s rebate – had they accepted those free chips.
But this multi-page hit-piece document for sure set out Intel’s stall and was a major reason why I predicted things would get ugly in 2018 and for the first part of the video I’ll recap much of what Intel got up to last year.
2018
In fact, 2018 was something of an annus horribilis for Intel. It started with a story that went on to dominate the headlines throughout the year, as Intel reeled from one security vulnerability to the next. Spectre and Meltdown were here and clearly affected Intel CPUs far more drastically than AMD, with AMD claiming invulnerability to Meltdown and providing quick fixes for Spectre.
Intel released this blog, however, and were sure to note that they were working with AMD and ARM to “resolve the issue promptly and constructively”.
But then, in March, we were treated to something altogether… absurd, with the launch of the AMDFlaws.com website and accompanying videos from the Israeli “security company” CTS Labs.
After some heavy scrutiny by Anandtech, who brought in industry veteran David Kanter, the security experts beat a hasty retreat, leaving unanswered some deep questions regarding possible attempts to manipulate the stock. I covered the whole episode in this AMDFlaws video and concluded that I wouldn’t be at all surprised had our friends at Intel been involved in it.
The thing about this is, in a year when Intel’s flaws were never out of the news, this one, which was basically a non-story from the outset – actually made the biggest headlines.
AMDFlaws.com now redirects to cts-labs.com and there is no sign of the previous dirty work except a link promising a publication “soon”, however clicking on their past publications list shows no such publication.
We didn’t have to wait all that long – just until June and Computex, when Intel’s next shenanigan was exposed but not before a bunch of tech press rather dimly fell for it. I am of course referring to the infamous 5GHz 28-core monster CPU.
We’d just caught wind of a 32 Core Threadripper incoming at Computex, but that was no surprise to any of us as we already knew from the previous year that Zen’s architecture allows for 32 cores on one server package. We can all do the maths, 32 cores will crush 18. Intel also had a 28-core server CPU to fall back on and we were amazed to see all 28 of them on display running at 5GHz. Rest in pieces Threadripper? Well, not quite. You see, Intel had “forgotten” to mention that they had overclocked the CPU. And also that they had to use a 1700W industrial chiller to keep the chip cool at 5GHz. A chiller that is banned in both the EU and North America for its use of the R22 gas.
Nevermind that this 28-core 5GHz monster was clearly marketed at gamers while Intel later admitted it would be a workstation grade CPU launching at the end of the year. The W-3175X launched with a 3.1 GHz base clock and 3.8GHz boost.
“Forgetting” a 61% overclock at Computex is one thing, but manipulating benchmarks is something else. And in October 2018 that’s exactly what we got.
Intel were just about the launch the i9 9900K and rather than going through the usual review avenues, they hired the ironically named “Principled Technologies” to benchmark their new chip instead. The resulting processor study of the i9-9900K and their competitors over 19 popular games raised quite a few eyebrows.
First up was Total War Warhammer II where the result in the Laboratory battle benchmark heavily favored the 9900K over AMD’s competing 2700X. 52% ahead for the 9900K is quite a result, but possibly not that surprising given that the benchmark is sponsored by Intel. An Intel sponsored benchmark being benchmarked in an Intel sponsored processor study… why am I not surprised?
But that was just the start of the shenanigans. In Gears of War 4 I noted that the 2700X was even further behind, while also pointing out the utterly ludicrous 13600% (!) advantage for the Intel chip in the “GPU bound” benchmark.
Another 50% win over the 2700X followed in Ashes of the Singularity and by this point I was beginning to smell a rat. A really smelly rat as it turned out, as the 9900K’s mammoth 42% victory in Assassin’s Creed: Origins had me reeling with incredulity. Even weirder was that the Threadripper chips weren’t anywhere near as far behind, when normally the 2700X would have been considered the faster gaming CPU.
Reaching the end of the study we finally understood the reason why the 2700X was performing so poorly. Principled Technologies had provided a step-by-step breakdown of the actions they took to get the results they got. I noted that the person who did these benchmarks clearly knew what they were doing in order to get a “fair” benchmark, for example removing performance bias options in the BIOS.
They also clearly knew that on their final point – “On AMD Systems, download and install the AMD Ryzen Master Utility, Launch the Utility, Select Game Mode and Apply”, would reduce the number of cores on all the AMD systems by half.
The 8 core 16 Thread 2700X was now effectively a quad core, 8 thread CPU. Very easy to win in heavily multithreaded games like Ashes and Assassin’s Creed when you hobble your main competitor by disabling half of it’s cores.
It came as zero suprise to me when I learned that Intel were also a sponsor of Principled Technologies XPRT benchmark suite as well as others like Sysmark from BapCo.
As I said in this video “Intel’s History of Contrived Benchmarks”… Principled Technologies my arse. I also noted that a certain Ryan Shrout – who we’ll hear quite a bit more about later on in this video – used both of those benchmarks in his test suite, benchmarks where Threadripper lost to an i5.
A few weeks later a certain Ryan Shrout joined Intel as their “Chief Performance Strategist”, a move which had me wondering on my Discord why exactly, performance needed a strategy.
2018 was indeed fought quite dirtily by Intel but possibly not as bad as I had imagined. However at the end of 2018 in another video I can’t recall, I noted that as Intel got ever more desperate, 2019 would make the previous year look like child’s play.
2019
Not a lot happened in the early part of the year, or more likely is Intel got away with whatever they did.
If 2018 was bad for them from a security perspective though, 2019 was proving to be even worse with each new month bringing new vulnerabilities. One of the worst came in May with the MDS vulnerability leading Intel to declare that disabling hyperthreading on chips previous to 8th gen may be warranted. Yep, your i7 7700K is probably better off – at least safer – as an i5 7600K.
But much worse than that was the way Intel dealt with the situation.
According to the Dutch publication NRC, Intel offered to pay the researchers a $40,000 dollar “reward” to allegedly get them to downplay the severity of the vulnerability, and backed their offer with an additional $80,000. The research team politely refused both offers before telling Intel to disclose the vulnerability on the 14th or the university would publish the information themselves.
The researchers were also quoted as saying:
If it were up to Intel, they would have wanted to wait another six months
Intel now has their “Bug Bounty” program which “rewards” finders of vulnerabilities with anything between $500 and $100,000 dollars. The catch? You have to keep mum about it, obviously. Given how many security vulnerabilities Intel has had outed this year, how many more do you think are going undisclosed as finders opt for the bounty instead?
A couple of months later in July and drama was incoming at the AMD Subreddit, which to be fair is never far away from drama. This time the benchmarking website UserBenchmark came under scrutiny for their curious decision to adjust the score weighting mechanism in their benchmark. Now the average score is weighted much more heavily in favor of single-threaded performance than multi-core performance due to “unrealistic” scores from all CPUs over eight cores, which includes AMD Ryzen 3000 processors.
Previously, Userbenchmark weighed single-core performance as 40% of the score, quad-core as 50%, and multi-core as 10%. However, due to the “unrealistic” scores of many-core CPUs like Ryzen 3000, Userbenchmark changed their weighing system to 40% single-core, 58% quad-core, and 2% multi-core.
Now, whether Intel’s marketing money found a way to them, I simply don’t know. However there is no disclosure of any sort on the website to suggest that happened. If I were part of Intel’s marketing strategy, I’d certainly consider a website with around 10 million visitors every month to be a good place to start. Note also that the best 3 CPUs are now all Intel CPUs.
What I do know is that choosing to change their benchmark weighting right after AMD’s new chips arrived on the scene smells a bit off regardless of their true intentions. Why would you do that?
My suspicions weren’t exactly abated when in the following month, Intel’s VP of “Tech Leadership Marketing” Jon Carvill and “Chief Performance Strategist” Ryan Shrout revealed their latest baby – “Real World Performance Testing”, showing that (just as appears to be the case over at UserBenchmark) lower threaded applications should be given more weight in reviews.
Coming under particular fire was Maxxon’s Cinebench (you know, the one that AMD really likes to use a lot in their benchmarks) for appearing in 82% of tech press reviews yet in the “real world” appearing in only 0.22% of the near 11 million tested systems. Of course, these near 11 million systems were all notebooks and 2-in-1’s, not the kind of platform where you’d expect to be doing a lot of heavily productivity.
Also note that “Intel is offering help to OEMs and press with realistic usage performance testing”. I wonder what exactly that “help” entails…
Extremetech noted “Intel has been only too happy to use Maxon’s Cinebench as a benchmark at times when its own CPU cores were dominating performance” before continuing with:
Ryan Shrout wrote on a topic that’s very near and dear to their heart — Real World Performance.
He said,
Intel had been holding these events for a few months now beginning at Computex and then at E3, and the process has reinforced our opinion on synthetic benchmarks: they provide value if you want a quick and narrow perspective on performance. We still use them internally and know many of you do as well, but the reality is they are increasingly inaccurate in assessing real-world performance for the user, regardless of the product segment in question.
Ryan then demonstrated the supposed inferiority of synthetic tests by showing 14 separate results, 10 of which are drawn from 3DMark and PCMark, both of which are generally considered to be synthetic applications.
And then they did the same again in the comparison against ARM.
It seems like “Real World Performance” is only very near and dear to Intel’s hearts when it suits them and they remain open to using synthetics – like Sysmark and XPRT benchmarks that they played a major part in developing – when that suits them.
We’re in October and Intel is “meeting competition” with their scale advantage and “Financial Horsepower”. This is a legit internal Intel slide, the source has been correct on two different topics, including Chris Hook leaving and from what I hear not actually going to NuVia, but we’ll see on that.
Oh if I could only find out where this money is going…
Around that same time in early October I got a huge info dump, mostly on Intel. One part of the information I got was “Intel’s 9000-series… their Cascade Lake AP with up to 56 glued-together cores doesn’t really exist”. They can’t be bought anywhere and never could be. This piece of information was confirmed very quickly in a leaked Asus slide showing Cascade Lake with only up to 28 cores.
This is important because this fantasy chip was almost certainly created simply for marketing purposes. From the outset I said that the TDP was just ludicrous and no OEM would even consider sticking these in a box. In fact, Patrick Kennedy over at ServeTheHome reported some time ago that no vendor had announced mainstream support for the 9200 series, but that the chip was however quite capable in certain niche HPC applications.
At the very end of his article he noted that that:
“Intel however are not positioning the Platinum 9200 series as a HPC rather than a mainstream product. The company is, however, using the Platinum 9200 series as a halo benchmark vehicle in segment comparisons outside of the narrowly defined HPC niche.”
The point of this we discovered a month ago when Intel’s “Strategy Team” yet again came under fire for publishing intentionally misleading benchmarks. Intel had apparently, in the case of GROMACS at least, used an older version which disadvantaged Zen 2. In the latest version, 2019.4 there was a small, but very important fix called: “Added AMD Zen 2 detection”, which says:
“The AMD Zen 2 architecture is now detected as different from Zen 1 and uses 256-bit wide AVX2 by default. […] This has a significant impact on performance.”
Afterwards, there was some discussion on the testing methodology and a note that the EPYC was apparently only running one thread per core instead of two. Turbo was enabled on the Epyc but disabled on the Xeon – you might think that’s a good thing for AMD but the author points out that:
“In GROMACS, transitions in and out of AVX-512 code can lead to differences in boost clocks which can impact performance.”
They also added more NUMA nodes on the AMD platform but refused the opportunity to increase the cTDP to the maximum allowed 240 Watts of the 7742. Essentially, we’re looking at a 400W CPU vs a 225W CPU. The conclusion was fairly damning.
“One can only conclude that Intel’s “Performance at Intel” blog is not a reputable attempt to present factual information. It is simply a way for Intel to publish misinformation to the market in the hope that people do not do the diligence to see what is backing the claims.”
All with a part that doesn’t really exist in the mainstream.
As it happened though, Intel retested with the updated GROMACS and found almost no difference in performance. In a followup interview with ServeTheHome, it was pointed out that the one thread per core on Epyc was a typo and in fact both CPUs had two threads per core. In other words, Intel maintain that their results are legitimate.
The problem here is, nobody has a 9282 to test and likely never will have.
The problem here is, nobody has a 9282 to test and likely never will have. So at some point, we look to Intel for those numbers and it is up to us and our readers to decide whether the optimizations Intel presented are acceptable.
It seems that Intel are doing their own benchmarks now and expecting us to just swallow them without any means of verification. These results could be legit however at best they will have been cherry picked to the extreme and there will certainly be many cases where even their 400W part gets beaten by the 225W Rome. But without the CPUs to test, we’ll never know.
Hot on the heels of that marketing mishap, came yet another, this time apparently the i3 is better than the Ryzen 7. More specifically, the i3 8145U is better than the Ryzen 7 3700U. And we’re not kidding. Intel even promises up to 65% better overall performance, note the up to and also a whole bunch of other up to’s.
Over at Medium, however, the author compared the 3700U to Intels 8 thread i5-8250U and found the AMD chip was ahead of that. How is it possible that this dual core i3 can be so much better than the quad core Ryzen 7? Simple really, you just do what you’ve been doing and cherry pick benchmarks.
This marketing presentation apparently appeared over at a Thai IT retailer in order to persuade clueless consumers into buying Intel products. I wasn’t actually able to dig up the full deck for this but these are clearly Intel slides. I was however able to dig up this deck for the Indian market, created in September.
Pretty quickly we see where this is going to go with this whole “real world” angle. We get a split between mainstream PC uses and high end desktop uses. That looks like Cinebench coming under fire again while Intel promotes XPRT and Sysmark and mobilemark again.
- Sysmark is the best proxy for real programs. Using REAL applications.
- WebXPRT from Principled Technologies represents real-world web application use cases.
- Cinebench… a BIG no-no
Something else caught my eye though in all these recent benchmarks and it was also noted by Charlie over at SemiAccurate in his article last month which centered on the Xeon vs Epyc benchmarks seen over at ServeTheHome, Intel messaging hits a new low. What interested me most though was this part where Charlie got to his next point:
“Intel is technically complying with that disclosure agreement but in name only. Recently the company started to not include the disclosures and legally mandated disclosures on the materials in question. Instead they put it in a link which may or may not still be there when you read that article in the future.
And under:
Links rather than disclosure
There is no question that this is intentional and meant to negate the point of the disclosure. There is no defense for it, it costs Intel no more to add a few lines of a footnote to the blog in question but they don’t. This skirts the intent of the FTC order because that order would make Intel’s claims looks even worse than they do now.
Now the disclosure agreement was forged between Intel and the FTC after Intel settled all of their previous anti-competitive and anti-consumer actions back in 2009.
That disclosure agreement is, and this is the actual Intel decision and order paper.
“Whenever Respondent:
- makes a claim comparing the performance of a Mainstream Microprocessor and a Compatible x86 Microprocessor, or
- makes any claim that references the performance of a Mainstream Microprocessor on any benchmark
Respondent shall Clearly and Prominently make the following disclosure:
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchase, including the performance of that product when combined with other products.“
Now, up until very recently that exact disclosure was made on any of Intel’s marketing materials. The whole point of this is that while Intel is happily showing you the benchmarks from Sysmark and XPRT – benchmarks which they basically created themselves or at best had a large hand in creating – you’re supposed to think “well sure but why not show me something that’s actually independent?”. That’s the whole point of this disclosure.
And here we can see it on the 9th gen slides for mobile – they’re making a bunch of benchmark claims and the exact disclosure is at the bottom while also including the link. Same again with 8th gen.
Looking back at the HPC slides on Intel’s Medium blog, comparing Xeon and EPYC and that disclosure is indeed now gone on both slides, replaced by the link only. That seems like a clear and flagrant abuse of the FTC’s demand. But wait, oh Intel, you sneaky bastards. The disclosure mentions mainstream microprocessors specifically, and as we saw over at ServeTheHome, the author actually mentioned that these are not really mainstream chips.
“At the same time, it is not a mainstream market architecture appropriate for the same markets that the Intel Xeon Gold, Platinum 8200, and AMD EPYC servers are targeted at.”
So, they can justify the exclusion of the disclosure here. But what about the rest? Let’s have a look through the September slide deck.
Okay, here’s one on “Mega Tasking Performance Lead”, claiming a win of 64 FPS vs 47 FPS while streaming and playing PUBG… which to be frank seems a tad unlikely but we’ll run with it.
What about the disclosure though? Nope, no disclosure there, simply a link to the same page. There’s no way the “non mainstream” argument will wash here, we’re talking about as mainstream as you can get.
So that one is cut and dry? Not so fast, over at AMD.com and their page on Intel Antritrust Rulings states that:
When the form of the promotion does not physically allow room for this language, Intel must state: “For more complete information about performance and benchmark results, visit www.intel.com/benchmarks,” and that website must contain the disclosure language.
Oh Intel, you dirty… there’s not quite enough room, is there? Why, because you’ve managed to stick a bunch of other stuff in there instead, stuff like the date of testing which would normally be in the appendix at the end? Very, very dirty… but wait, maybe I’m giving them just a little bit too much credit. Let’s be blunt: up till now everything we’ve seen of their marketing is dirty and not very smart, this is just too clever for guys like Shrout to figure out.
And a few slides later my point is proven. Obvious benchmark, more than enough room for the disclosure but instead we just have the link sentence.
And again in the comparison vs Ryzen.
What about Carvill’s and Shrouts “Real World Performance” marketing materials?
It starts off with a slide that doesn’t even have the link, maybe they think that because it’s HEDT it’s not mainstream so they’re immune to the ruling? Moving on… clearly more than enough room for the full disclosure there, there, there, there again with AIXPRT and every other slide thereafter there is clearly enough room for the real disclosure.
Note: we encourage viewers watch the video (at 28:57) to see exactly how many slides lack the disclosure.
In actual fact the text of paragraph VIII in the settlement says…
“where the form of the promotion does not reasonably allow inclusion of this language (such as in an audiovisual advertisement or on a retail tear sheet that is too small to allow inclusion of this language in a font size that would be readable), Respondent may instead Clearly and Prominently make the following disclosure: “For more complete information about performance and benchmark results, visit www.intel.com/benchmarks,”
Intel, maybe you can justify whether your CPUs are mainstream or not and maybe you can justify whether your latest marketing slides cannot reasonably include the full disclosure that you’d had up until very recently.
You’ll be justifying it to the FTC Compliance division, whom I have just reported you to for violating the 2009 antitrust Settlement. I have also offered my services in pursuing a deeper investigation into Intel’s recent behaviour.
Now to finish this one off, and I ran out of time before I was able to talk about the Python Matlab scandal where again it appears that Intel has used a much slower codepath when compiling with ICC on AMD CPUs. This was also a major part of the 2009 anti-trust settlement with Intel being forced to disclose whenever ICC was used. This whole topic is too large to be included in this video though and to be honest I haven’t looked deeply enough into it to come to a conclusion either way.
So on with the final conclusion.
First of all, let’s talk about Ryan Shrout who has clearly had a major part to play in Intel’s worsening marketing. From my “Ryan Shrout’s Flawed Perspective” video, you know that I believe Ryan ran a program of undermining AMD’s products using his website PC Perspective as the vehicle. Note that PC Perspective is under new management and, from what I can see, are legit these days. Ryan also had a rather dubious disregard for disclosure, operating “Shrout Research” where he got paid to “research” hardware before public release, then used the same hardware in his website’s reviews. Bit of a conflict of interest there. Now to be fair, after that point they included a full disclosure.
But is it really suprising that a guy who made a living out of downplaying AMD and failing to disclose is now at Intel… downplaying AMD and failing to disclose? All of these benchmark slides from Intel are essentially them promoting what is effectively their OWN benchmarks and now without the relevant disclosures.
Now, Jon Carvill who presumably was responsible for giving Shrout the job as “Chief Performance Strategist”? He’s now moved on to NuVia. Somebody will need to fill his position as VP of “Tech Leadership Marketing”. Shrout might try that for job. He certainly has all the required credentials… however….
…he lacks the credibility. Intel’s marketing has been a disaster ever since he got there. Linus, 10 million subs absolutely slammed them over their 10980XE shenanigans, Techpowerup slammed them over attempting to hide yet another vulnerability, ServeTheHome slammed them over the EPYC slides, Medium slammed them for the i3 over Ryzen 7 claims. SemiAccurate slammed them for more dubious cherry-picking with the Comet Lake launch.
Whatever Shrout thinks he can get away with, he’s clearly not. It’s being exposed by a tech press who simply do not trust him and simply do not trust Intel.
People are gonna say, “here’s Jim having yet another go at Intel.” Believe me, I would like nothing more than Intel to just clean up their acts on multiple fronts, to just behave and to just comply with the rulings against them. But they can’t, and with guys like Shrout there they never will. I do not hate Intel, in fact I know many people within the company. The vast, vast majority of Intel employees are absolutely great people. But it only takes a few bad eggs to drag the reputation of the entire company down and sadly marketing is the company front.
As Charlie over at SemiAccurate said,
“Things have gotten so bad and are so prevalent there is no way it is unsanctioned from the top. At this point SemiAccurate is calling on Intel management to force a change in this behavior and fire those responsible for this, not necessarily the people lower down on the chain of command who can be easily blamed but those higher up who set the culture and allow this to happen.
Things have long progressed past the point where it is hurting Intel’s image and standing as a company. An ethical company would simply not allow this to happen but Intel is. It is never too late to affect change and start cleaning up the rot if the corporate will is there, is it at Intel?”
I am not comfortable with demanding people lose their jobs, for the simple fact that people have families. However, I will say this for an absolute certainty:
So long as Ryan Shrout remains at Intel in this position or worse – get’s Carvill’s old job – there will never be peace between the tech press and Intel. On a personal level I will never tire of outing this kind of false marketing and I will never tire of reporting compliance failures to the relevant authorities.
In fact, I intend to ramp it up. I believe I’ve said something similar in the past but if you thought 2019 was a dirtily fought year in tech, 2020 is gonna be on a whole new level. See all this stuff here, where Intel is doing what they can to hold on to the last advantages they have? Next year, they even lose that. The only thing left for them is even dirtier marketing… or worse.
I’ll catch you later guys.
