Benchmarks demolish Apple speed boasts

[morriswindgate]

Benchmarks demolish Apple speed boasts
The Register | 03/11/2002 | Andrew Orlowski

Posted on 3/12/02 9:12 PM Pacific by Ouachita



As of this morning, thirty one entries have been submitted to the SPEC performance list for the year so far. But if you're wondering why Apple hasn't yet dispatched its latest "workstation class" G4 hardware for examination by the council, in what is the industry's most respected set of benchmark tests, C't has the answer.

The German tech bible has put the latest dual G4s through the SPEC CPU2000 processor benchmark, and the results make dismal reading for hardcore Apple loyalists. C't found that the RISC-based machines running OS X fall severely short of expectations, being bested in the floating point tests by an eighteen month old Pentium III-based machine

You might still be able to buy one of these, on eBay.

In the integer tests, which give a better indication of performance for the kind of general-purpose desktop computing most of use every day, the G4 held its own against the ancient Pentium running at 1Ghz.

But when C't ran the floating point benchmarks it found that performance was particularly wanting. The elegant PowerPC was designed with floating point as a key consideration, and with these kind of tests, it should be on home turf.

As C't wryly notes, "In theory the PowerPC FPU with its 32 registers ought to have been superior to the x86 FPU with its antiquated stack structure and eight registers only."

The G4 scored between 147 and 187 on the floating point tests, while the Pentium III scored 297. Today's Pentium 4's double that figure, and as a result, today's PCs are four times as fast as Apple's professional line in some situations.

These are specialized tests, it's true, but they're the most closely watched by the kind of scientific and visualization users Apple has been trying to woo, and 'scientists' have been hardwired into Apple's mission statement for three years now. Credibility depends on success here.

But ever since Motorola decided to focus on embedded markets for its processors in Fall 1997, Steve Jobs has had to rely on one highly dubious bake off for his Pentium-trouncing demos. This squares a Mac Photoshop against a buggy, older version of Photoshop on Windows, and one that doesn't take advantage of the Screaming Sindy v2 instructions. Can you guess which one wins?

Apple has two performance headaches right now: the processor and the OS. Apple's director of core engineering Brett Halle last month promised us that OS X performance was a paramount concern, and to be fair, his division need take no blame. The BSD he inherited has the industry's best respected IP stack, for example.

Apple can draw on a few crumbs of comfort. The SPEC benchmarks disable a second CPU if present, and don't necessarily take advantage of the Altivec and Screaming Sindy dedicated FP instructions - that's down to the compiler. C't noted that it wasn't using the latest gcc compilers on the G4, but then again, it had shunned the latest Intel compilers too, which give a 20 per cent speed hike. And C't records that G4 performance is scaling better than the clock frequency, compared to the Pentium, with a 18.6:15.5 against the Pentium's 7:8. But given that that the Pentium is currently 2.2Ghz, and the G4 1Ghz, that will take a very long time indeed. (We've already disclosed how Apple's forthcoming chips use a larger 4MB cache, which is a much better way of getting there from here).

The problem is, one former Apple engineer told us, in serializing the twenty five year old BSD layer with the fifteen year old code of the extensions NeXT began to add in the mid 1980s.

Apple's attack parrots unfailingly point out that Mac OS X is "the most modern OS" you can use. In marketing terms, perhaps that's true. In technical terms, the integration point has been described to us as so crufty that no one wants to touch it. This is very old code indeed, and explains why you see the spinning disc cursor so often in OS X. Serialization is one of the toughest problems for an engineer to solve.

The more thoughtful majority of Mac users have real-world concerns about performance and have been expressing them to Apple, and to us, since the launch of X, and the latest SPEC results show that there's no place for self-delusion.

Over the past year we've consistently pointed out that Apple can make much of low-cost SMP systems, now that 's it got an OS capable of taking advantage of it. But we've also seen performance and latencies consistently fall short of equivalents in the PC world.

Apple needs a company-wide "performance task force" every much as Microsoft needs its "security task force", which was its belated response to persistent and chronic bad security practices. Any chance of a carefully leaked memo, Steve?®

[shecky981]

::::sigh:::::::: this is utter and complete bulls**t.

ill put a dual 1 GHZ G4 against ANY 2.2Ghz P4 running photoshop ANY day of the week. "oh, so it only runs photoshop better?" NO! it runs almost any graphics based, Velcoity engine optimized program WAY faster than a P4. (photoshop runs 72% faster on the dual Ghz Mac than on the P4) and the thing you all fail to realize is that the Apple machines WORK BETTER.

flat out benchmarking comparisons are meaningless. the floating point stuff is crap. of course 2.2GHZ is faster than dual 1 GHZ. ALL THAT MATTERS is how it runs specific applications. And the specific applications that are made to run on a G4 machine SPANK pentiums all day every day.

[Xayd]

The thing that kept Apple going through its hard times were those who would buy the machine to get the OS, not get the machine just for the machine.

When your only alternative was Win98 or NT4, was there much of an alternative really? On the other hand with Win2000 out there the choice isn't as clear as it once was, IMHO.

[LawyerRon]

Hey X,
I was originally trained on a Mac back in '91 when I set up an in-house marketing department for a large mail order company. We were using system 6, then later system 7. The best you had on a PC at the time was WIN 3.1. System 7 back then was every bit as good as WIN98 is today. So, at that time, the comparison was not even fair; it would be like comparing WIN98 to WIN 3.1 today. Problem was, nobody could afford a Mac. The Mac IIFX I used was state of the art at the time and cost over $5,000. Cheap Macs were about $3500. But if you were into desktop publishing, as I was at the time, you had no choice. Software for professionals was only available for a Mac. So why did the Mac lose the PC war? There's been scores of articles written on the subject and you can take you pick. But the bottom line is this: don't ever think they lost because the IBM "clone" was a better machine. It wasn't even in the same league with a Mac.

[Sauron]

I am not going to say anything critical, (mostly cuz I saw quite a few macs...and worked with them...) But Thank God for the PCs! And Microsoft!

[LawyerRon]

Hey Sauron,
Say anything you want. The war's over and Mac lost. Sure, they still have a niche in the graphics industry and with some die-hards, but that's about it. In summary: Fine machine; way ahead of it's time; lousy marketing, dumb leadership that was outsmarted by Bill Gates.

[alexmacdonald]

Funny thing is shecky, apple compare their dual 1ghz to a 5 ghz p4, which obviously doesent even exist yet, so what were you saying about benchmarks? I use AutoCad R14, Terramodel and photoshop every day for a living and would never even consider a Mac, The performance doesnt even compare for Vector modeling.

[audiyoda]

Originally posted by LawyerRon Sure, they still have a niche in the graphics industry and with some die-hards, but that's about it.

Ummmm......and the audio industry, and the film industry, and the video post industry....all the CPU intensive stuff seems to go MAC (in the professional realm anyway). Sure, I do audio/video on the PC but I am in a BIG minority -- I started using PeeCee's back when 640k was all you needed. But before that I had already done some studio audio work. So it was a natural migration for me -- but it was slow in coming. MAC's could do audio long before a PeeCee could -- years before. Same with video. And I guarantee if you went to the top 5 movie production houses right now the closest thing to a PeeCee you'll find is a SGI or maybe a dual Zeon running on linux renderer. MAC's are the machines that do the actualy editing.

-Craig

[LawyerRon]

Yeah Craig,
Of course you're right and I agree, I should have mentioned the audio industry too. My point is though, those are "niche" markets in my view and Apple lost the mass market war for all the reasons I mentioned above:

Fine machine; way ahead of it's time; lousy marketing, dumb leadership that was outsmarted by Bill Gates.

[Sauron]

I hope I didn't offend you err LawyerRon...Now maybe something can be done now about Compaq. Someone explain why in every school, museum, everyone uses Compaqs?

[Xayd]

Compaq was for a long time a seller of high-end workstations and servers, not every Compaq was the crap-ass PC you see at best buy .

As I remember it Compaq was the most prevalent, if not the first as well, computer manufacturer to bring pre-built x86 machines to institutions and corporations for workstations and servers.

With Compaq and HP not around anymore doing what they once did in the small server market, Apple may yet find another niche with OS X and that market as well. Unix/Linux servers aren't that expensive to build, but are expensive to hire admins for. If OS X thrives as the "user-friendly Unix OS" that Apple wants it to be, Apple could very well capture alot of that above mentioned server market based on that.

[Sauron]

Amazing, the few ppl I know that have them don't really love them. But Compaq is really the oldest hardware company besides IBM isn't it?

[LawyerRon]

"I hope I didn't offend you err LawyerRon...Now maybe something can be done now about Compaq. Someone explain why in every school, museum, everyone uses Compaqs?"

Takes more than that to offend me Sauron! As to why so many schools use Comraps, the answer is easy; Money. School Districts offer big contracts when they go shopping for computers and they go with who gives them the best deal.

For a long time it was Apple. Back in the days when a cheap Apple was $3000.00, they would cut the school districts big discounts. They could afford to because they had so much profit built into each machine. The theory was, when little Susie and Johnny got big, they would naturally want an Apple too since that's what they knew how to use. Problem was, when little Susie and Johnny got big, they bought PCs because they couldn't afford an Apple.

[shecky981]

Originally posted by alexmacdonald Funny thing is shecky, apple compare their dual 1ghz to a 5 ghz p4, which obviously doesent even exist yet, so what were you saying about benchmarks? I use AutoCad R14, Terramodel and photoshop every day for a living and would never even consider a Mac, The performance doesnt even compare for Vector modeling.

no....yer wrong. CAD programs dont really exisit on a Mac platform, so there can be no discussion there. But as far as Photoshop (which is a RASTER program by the way, not a vector. You should really know this before you started your yammering) the Mac blows away PC performance. As far as the 1 GHZ vs. 5 GHZ comparison, I still stand by what i said in that its a meaningless comparison. The reason they do that is most people are ignorant and only want to hear about GHZ vs. GHZ. So a wintel machine that takes the same amout of time to do the same stuff a Dual 1GHZ Mac does would be the equivalent of a 5 GHZ P4. Big deal, so what. Ya will pardon the pun, but its trying to make an Orange to Orange comparison when one of the oranges is an Apple.

And i will say it yet again, since you all are too dense to listen:

MACS WORK BETTER. PERIOD.

Integrated hardware and software = Better machines that usually work wonderfully. Separate hardware and software vendors = really cheap computers with a lot of really crappy software, a small percentage of which works wonderfully.

[padawan]

Integrated hardware is better? Ok, now I think I better get my boots on, this is getting deep. Look at what this integration has done. Macs are outrageously expensive, and they don't have half the software the mainstream user wants. I was going to buy a Mac for my classroom, but it cost more than 3 PCs. I bought three 1.6GHz Dell's for the price of one Mac. Sometimes I think owning a Mac is some kind of yuppie status symbol. The PC versus Mac debate is as bad as the Intel versus AMD debate. One side claims the benchmarks are rigged, and neither side can ever agree on anything.

I'll stick with my 1.6GHz AMD/Win2K box for now. I play games, so unless I want to wait a couple years for Mac ports, I will enjoy my rig, which incidentally, costs 1/2 as much as a Mac. Hell, I might just build another one.

One more thing, since I am too dense to listen. PCs work just fine, and I don't have to pay thousands extra to get some stupid clear case with a speaker that looks like a button. Then again, we are too dense to listen. Seems like Apple is not exactly that brilliant either, genius. Why else would they be where they are today?

[shecky981]

OK well, lets see here:

Macs are more expensive, and you pay for what you get, and they do not have half the software the "mainstream" PCs have, WHICH IS GOOD beacuse 90% of software for the Wintel platform is complete crap.

For a serious gamer i would ABSOLOUTLY agree that a PC is better, if for no other reason than games are generally made for the platform.

And you are kidding yourself if you think that "PC's work fine" When they work, they work fine, however, it has been my experience, ever since Win 3.1, that my Windows based systens crash FAR FAR FAR more frequently than my Mac based ones do. and i mean by like a factor of 50. I am not saying Macs are perfect, because they sure as heck are NOT, i am just saying they work MUCH more consistetly well.

And, incidentally, if you are so blind as to think that something which costs half as much must be twice as good, then not only are you blind, your ignorant. i dont care if you can build 20 PCs for the cost of one G4; what does that say about the PC's?

And as far as where Apple is today? Lets see... a 5% market share of the computer industry, which, to quote Apple's propoganda is "proportionally more than the market share of BMW and Mercedes-Benz combined." Which is more money than you or I are ever gonna have, I would say they are doing just fine. They also are industry leaders in terms of looking ahead. Apple was the first with USB, Firewire, and LCD screens, all of which are or will soon be industry standards.

And you are right, the beige box is really a wonderful piece of design. How foolish of Apple to make something that is functional as well as beautiful. And, i'll bet you ANYTHING you want that i can get in, add RAM, and get back out of my PowerMac G4 faster than it takes you to wipe the drool off your chin. Part of that is due to the "stupid clear case with the speaker that looks like a button" that is also designed to be opened in less than 1 second, and lays the motherboard down horizontally and makes it incredibly easy to acces.

[LawyerRon]

Play nice boys.

[padawan]

Gee, let me open my Antec SX1030B. Takes me about 1 second to release the handle. You still have not convinced me why I should get a Mac. I'm running Windows 2000, and other than a couple of games, I have had zero issues with stability.

90% of PC software is crap? Ok, let's think about that. First, I doubt 90% of it is crap. Yes, there is a lot of crap. BUT, let's compare the number of software titles to the number of Mac titles. When any small company of script kiddies can throw together software and sell it, you are bound to have crap emerge. If Apple ever gained the same level of market control, I can assure you that there would be just as much crapware. Maybe you need more experience with PCs before you start claiming they suck. Windows 2000 does not crash 50 times more than a Mac, that I can assure you. Microsoft has come very close to actually having the 99.999% uptime they claim. Ask any IT pro, and they will tell you that Win2K is an excellent platform.

Price...whatever. You can spend your money on whatever you want. PCs are cheaper because of competition, not lack of quality. Again, it there were Apple clones, you would see some poor quality stuff. If you buy good PC parts, you will have a good PC. For less than half the price.
"And, incidentally, if you are so blind as to think that something which costs half as much must be twice as good, then not only are you blind, your ignorant. i dont care if you can build 20 PCs for the cost of one G4; what does that say about the PC's?"
I am not blind. I switched to AMD CPUs when I found their power to price ratio was better than Intel. I am not brand loyal, although I get the impression that you sir, are the blind one.

5% market share. Computers are not the same as cars, so that is a poor analogy. There is far more competition in the automotive arena than the computer market. Apple is relying on flashy looks and performance "claims" to sell their boxes. So far neither has been a smashing success. The same people that bought the new iMac are likely the same people that bought the old one.

[audiyoda]

Originally posted by LawyerRon Play nice boys.

I'll second and third that -- I'm big enough I get two votes.

-Craig

[glc]

Thanks, Ron and Craig - I'm watching carefully, keep this debate respectful, please.

[shecky981]

Padawan, you keep re-affirming what i am saying, the idea of a debate is to make a CONTRARY position, not to affirm the other side.

First of all, i am NOT trying to convince you or anyone on here to buy a Mac. I don't care what you buy. I have no vested interest in any of you buying anything. I use both platforms, and I am sick of all the crap Apple gets thrown by people who do not know any better.

The Apple case is easyer to get in and out of. end of story. there is nothing to debate here. on the Apple case, you have a motherboard that swings out, and lays horizontally and is completely accessible, much more so than a board that either sits down inside a case, or that comes out on a motherboard plate that needs to have all the cables detached before it can be worked on. I will also admit that the iMac case is virtually IMPOSSIBLE to work on (well, not impossible, but, it is one huge pain in the ass)

"Yes, there is a lot of crap." True. i am glad you agree. And yes, if Apple was as big as PC there would be a lot of crapware. But, they are not. I am not debateing this. So, once again, you proove me right in what you say. And, no, i do not need more experience with PCs since i have been using them for quite some time, and i have seen a lot of stuff come and go. And, while i will certainly agree than Win2K is probbly the best OS from MS, its is still very crash prone, i know, the servers in my office were Win2K, untill we got rid of them and all the IIS garbage that goes with them. Now we have Linux and Apache. No problems yet.

"Price...whatever" wow. ya got me there. Excellent point. PCs are cheaper beacause anyone with a screwdriver can put one together, therefore, there is a proliferation of poorly made, and poorly designed PCs out there. And, do me a favor, stop theorizing about "what if" "if there were apple clones" "if they had a bigger market share" "if i wasn't a microsoft zombie" I am not debateing WHAT IF i am debateing WHAT IS.

"I am not blind. I switched to AMD CPUs when I found their power to price ratio was better than Intel. I am not brand loyal, although I get the impression that you sir, are the blind one." OK...uhmmmmmmmmm...........how does that possibly respond to what i was saying? I'll let you get back to me on that one.

And do not think i do not realize that this is clearly a Wintel based forum. I have great respect for all the people on here who have helped me in building my PC, however, i have little use for those of yuou who jump on a bandwagon because someone told you to. So, i am doing my best to "play nice" even though I am well aware of whom you all rooting for. I figure i will keep on the forums untill the mods decide to kick me out of them for having a contrary opinion.

<---serves ball with a vicious overhead swing to the other site of the court

[LawyerRon]

shecky981
A non-Mac user will never understand what it's like to use a Mac. I used a Mac IIFX 8hrs a day for 5 years straight. It's been said the Mac interface is "more elegant". I think that a good way to describe the Mac.

[Sauron]

I love my Intel I am very rough on all my stuff and it hasn't died yet..PC is cheaper, as you might say, anyone can build one. According, the Mac you can't do that...so they are like hand created? Something like Rolls Royce? Macs, well I don't know very much about them. But like I said eariler, I know ppl that had them...and the comments weren't terribly good. So far it sounds to me like the Mac suffered hardware to create better software? The PC, well good hardware came with good software The Intel 8088, the 286, 386, 486, Pentium, Pentium Pro, Pentium II, Pentium III, and lastly the Pentium IV. Kinda impressive list *grin* Lately we have had the AMD come out which supposedly is easier to overclock and "faster"...while Intel is cooler, takes lots of hurt, easier to work with I will stay with Intel. Just like I will stay with the PC. As like I will stay with European cars...

[padawan]

How am I supposed to prove anything? When a PC user mentions anything, you bash it. You guys say the benchmarks are unfair, it is OK to cost nearly double the price, etc. etc. Why argue? You are just as biased as you say we are.

The AMD scenario was an example of how I am willing to jump ship to use a better product. Three years ago I would not have used an AMD chip, but AMD has proven themselves as a competent CPU builder. The problem with Mac people is that you guys cannot acknowledge that the PC is better at a lot of things than the Mac. I understand that the Mac can do some stuff better the the PC, but for 99% of the people it the world (or 95% according to your numbers), the PC makes more sense. Why would I use a Mac with limited software, and it is slower in the majority of the common apps that people run. If we were all graphic designers, then by golly, we would all be Mac users.

Your counter points are pretty weak, as well, so don't go blasting me. You said the Mac looks nice, is easy to open, and is just plain better. Define just better...how about some proof. Obviously 95% of the people in the world don't agree

My "what ifs" were simple examples. You can live in your dreamland all you want, but IF Macs were as popular as PCs, there would be just as many crapware titles and junk hardware. You logic reminds me of a poll I once read that said "People with a gun in the house are 75% more likely to die from that gun that people without a gun in the house." Obviously the people without the gun in the house are not going to die from their own gun, so the statistic was useless. You Mac argument is similar. Say there are 5 Macs in America. That means there are 95 PC in America. The odds are, the PC users will have more problems, just by the fact that there are so many more of them.

Be happy with your Mac, I can careless. I am happy with my PC, so that does not make me an idiot or a bad person, the way some Mac owners would have you think. I have a kid in my class that will not shut up about how much faster a dual 800MHz G4 would have been over the dually P3 1.13GHz server I bought. Of course, he can produce very little real fact to back it up, other than the traditional Photoshop scores. Since I am not teaching a Photoshop class, I can hardly agree with him.

Now, the problem with this comparison is that there is no standard software we can test with. I don't know of any software that was originally written from the ground up for both PC and Mac. So testing can easily be done to make either machine appear faster. Read this article, and you will see just how it is done, even at the professional level. This article is by the guys at TechYV, and they are not Mac-haters:
http://www.techtv.com/products/hardw...339311,00.html

Now, compare that with Apple's take on Photoshop, using a version known to have problems with the P4:
http://www.apple.com/g4/myth/

Yikes, an independent company has the P4 winning, Apple naturally has the G4 win. Who do you believe? Check out this quote from Apple: "Quite simply the fastest personal computer ever built to run graphics applications"
Graphics. What about the other 1,000,000 applications out there?

One last thing. Why does it seem like Photoshop is the defacto Mac benchmark? Read this from the VP of Adobe:

"Q: Does your excitement about Windows XP lessen your support for the Macintosh platform?
A: No; Adobe got its start on the Macintosh platform and this platform remains a substantial and important part of our business. We are committed to supporting and developing quality products for both Windows and Macintosh platforms. For more information on Mac OS X support, please visit:"

http://www.adobe.com/products/adobesupportsXP.html

Now, how about some more non-Apple biased benchmarks. Video ones even:
http://www.dv.com/magazine/2002/0102...up_3_0102.html

Read it to see how the mighty G4 fairs. Oddly, the AMD K7 kicks some ass.

More benchmarks, again, AMD making a good showing:
http://www.barefeats.com/pentium4.html

And if you really want a good laugh, go over the http://www.apple.com , log into the forums, and read about the kernel traps other problems people are having with OSX. How can this be? I thought the Mac was "much better" than the PC?!?!?!

Seriously, it is impossible to compare the two. But for Mac people to bash on PCs is as stupid as it is for motocycle owners to bash car owners. Bikes are fast, but there are times when cars are faster. And the two are so different, it is hard to compare without giving an advantage to one or the other. BUT, being a fan of the PC does not mean we are uncivilized or less computer savy.

[padawan]

One more quick note...here is another forum to look at:
http://www.macworld.com/cgi-bin/ubb/...?ubb=forum&f=1

Mac users have the same problems we do. There are just more of us ranting about it, and the Mac guys keep it quiet.

[audiyoda]

Good Gawd boys, can you just agree to disagree and stop the B$? You're both flinging it left and right -- it's like a flaming liberal democrat trying to convert a conservative straight-laced republican. It's just not going to happen.

As LawyerRon put it -- if you are not a Mac fan and have never used one to it's fullest, you will never really appreciate what it can do. And you've also got to admit that regardless of how elegant the Mac is, the vast majority of computer users -- for whatever reason -- chose to use PeeCee's. Period.

-Craig

[glc]

Are we through yet? I think it's time for padawan and shecky to take this one somewhere else..........

[padawan]

Cool. Shecky, meet me behind the CompUSA on Main Street. We'll settle this once and for all

[glc]

Scott - better not take it behind HIS CompUSA - he might have some "friends" with him..........

[padawan]

The Apple site said you could pick up the new iMac by the neck, looks like a great club to me...
iDeath by iMac.