Processors: On Multiple Cores And Multi-Threading  

vrec_dawn 40M
854 posts
7/17/2006 3:22 pm

Last Read:
7/18/2006 3:24 pm

Processors: On Multiple Cores And Multi-Threading

Okay, I know, I know. Who really wants to read what already sounds like a majorly complex techie spew? Not even most techie-journalists it would seem. I'm seeing it more and more, but once it hit Vulture Central, I had to say something about how these supposed experts aren't. So here's what I hope is a very easy to understand explanation that you may even find interesting. If you're looking to buy a computer any time between now and the end of days, this could be an important post to read.

What is dualcore? What is multicore?

As the old saying goes, two heads are better than one. This can be true, even inside of your computer. In ages past people who needed a lot more power than what was in a normal computer could get their kicks by squishing two processors inside of one box. And now, through the miracle of modern technology, as the size of processors shrinks and shrinks, we can bring this same concept to you for much cheaper. You can get two cores in one processor. In every way this is like having two processors. In some cases it can be even better, because these two cores sit and play in the same sandbox (processor), they can share their toys much more quickly than if they were in two seperate sandboxes (processors). So now, for nearly the same price, you can get theoretically twice the processing power.

Why is the processing power of dualcore only theoretically double that of the traditional single core processor?

For many reasons most software is what we call single-threaded. Imagine a railroad. There sits a train with all of its cars connected in one line. That is single-threaded software, that one linked train. Imagine each core in the processor as a track. While there is only one track, this single train can use all of it to run as fast as it needs. But if you add another track, all you get is an empty track. That one connected train cannot use all of that extra potential available. This is what happens when you run a single-threaded computer program on a dualcore processor. Your train still runs the same speed, and half of the available potential goes unused.

But then what is the point of a dualcore processor?

There are two points. The first is that yes, this one train cannot run any faster, but now you can run two trains at once without them running into each other or slowing each other down. So if you have other programs (trains) to run, you will still benefit.

And, in fact, you have a lot more than just two trains running at one time. Modern operating systems like Windows XP are running many little trains in the background. Look at all of those icons next to your clock. Each of those is a seperate train running. Most of them don't need much to run, and so won't get in the way, but every little bit helps. Giving them their own seperate track to run on lets your big important trains run without interruption.

The second point is that not all computer software is written as single-threaded. Some of it is multi-threaded.

What is multi-threaded software?

Multi-threaded software is written where each car on the train is designed to run together on one track, or each on a seperate track. Imagine each individual car having its own small engine built on to the front of the car. They can be split up to use as many rails (cores) as available all at once to get to where they need to go faster.

If software can be multi-threaded, why don't programmers always multi-thread their software?

There are many reasons why multi-threading is not always as perfect as it first seems, and I will try to give you the most common reasons. The first is the customer base. If you can imagine each car having its own seperate engine, but only one track is available, then it would become a very inefficient means of moving that one train. So while most people are still using single core processors, having just one track to run on, it makes more sense to program using the efficient single-threaded train than the inefficient multi-threaded train. It's about how to maximize the benefit for your target customer base. There are some cases where companies can expect their customers to have multiple processors or multiple cores, and so they will take the effort to make their code multi-threaded. The average desktop user however still has one single core processor, and so the time for companies to switch to multi-threading has not yet come.

The second reason is that multi-threaded software is a lot harder to write. There are so many more things that can go wrong, so programmers have to work that much harder to work out all of the possible bugs. This enormously increases the cost of developing the software, and of maintaining it. Like it or not, companies are out there to make money, not to make you happy. So they will do what is the most economical for them, not what is the most beneficial for you. It would do them no good to spend so much on making you happy that they put themselves out of business.

The third reason is that the benefits of multi-threading are only theory. Let me give you three scenes to picture:

I ) Sculling You have a boat full of ambitious people, each with their own set of oars. Though one person will keep them in sync, they can each work completely independantly on their own seperate task. So, say you have a program that renders a 3D animation. You know how many frames you need to render. You can render one frame for every processor core available like a finely tuned machine, and finish the work in record time.

II ) Olympic Relay You have a number of runners all set to go. Each has to wait for the baton from the other before it can run, but the sum of the parts being split up equally allows the distance to be run faster than if one person had to run it alone. Even if some time is wasted in handing off that baton, you still come out close to the maximum potential.

III ) Egg-limp-ick Relay Imagine a gaggle of kids in a relay where they hand off a raw egg from one to the next. These kids are not all pro atheletes. There are skinny ones, fat ones, fast ones, and clumsy ones. They each run at their own pace, and are not always skilled in handing off that egg from one to the next. Some times the egg might even drop, and someone will have to pick it up or grab a new one if it breaks. It is a comical race where working together as a team might, or might not, actually be beneficial compared to one person going it on their own.

In a perfect world, most computer programs could be broken up between I and II. We could extract all (or at least most) of the theoretical potential that is there by splitting up the work into well-managed teams. And there would always be enough people on each team to fill the number of cores available. This would be the perfect world. This is what most sales-people and techno-journalists would have you believe is possible. That we can live this dream. The hardware is (or will be) there, and surely software will split up into perfect teams like magic.

The reality is that actually the incredible vast majority of cases are more like III. Not only does it become very difficult to evenly break up the processing done in a program so that all parts are evenly matched, but often you end up with the one fat kid. He's big, he's bloated, and everyone has to wait on him. It just can't be helped. Computer threads, like people, are not all the same, no matter how much the world tries to say otherwise. In fact, often because most of the computer program lies in this one heavy algorithm that cannot be broken up any smaller it means that there are only so many effective splits that you can even make in the code. So while we have only two cores now, we can fill them. One day though, when we double that to four, or again to eight, we will find that we no longer have enough people to fill the team. We will no longer have enough cars in the train. We will have more cores than we can fill, at least usefully so.

You say that in the future we won't be able to fill all of these cores. But you also said that most programs are single-threaded, not multi-threaded. How does that impact the future?

It impacts it quite badly actually. The reason that the computer industry is moving to multiple cores now, with the dualcore you can get today for desktops, is that processors can only run so fast. We are reaching a point where the speed is being limited. With as small as processors are getting, the easy answer was to put less effort into the low-hanging fruit of speed, and more effort into the next tier of fruit, to squish two cores into one processor. And next will be four. Then even as many as eight. This is the new future that the industry is aiming toward, to increase processing potential with more cores than speed.

But as I said, single-threaded software can only run on one core. This means that on a dualcore processor the most this one application can use is 50% of the power in that processor. This becomes 25% in a quadcore. It is just a miserable little 12.5% in an octacore. Imagine a computer program being only able to use, at most, 12.5% of your processor. This is a sad future.

So then why doesn't everyone just start writing multi-threaded software?

Again, as much as the utopians would have you believe that I and II are the norm, it is actually III, the Egg-limp-ick Relay, that is more often than not the case. It is not for a lack of trying, but simply because not all algorithms are equal. As multiple cores become common in the computers we use at home we will see more software developers trying to write multi-threaded software. That said, what we can dream in theory will just never be as good as what we achieve in practice. We will try, but that does not mean we will always (or even often) succeed.

What does this mean to me?

If you are looking for a new computer, the time will come when we will really have to look more closely than ever before on how we use our computer. The old standard of speed is no longer a meaningful measure. (Some would argue it never was, but that's another topic.) We are moving to a new complexity of measurement, of the number of cores, and this requires a whole new world of thought to understand if what we are buying is what we need.

What programs do we commonly run? What programs do we really need power for? Are they single-threaded or multi-threaded? Right now, most of us will benefit from a dualcore computer, simply because of all of the background work that an operating system like Windows has to do just to give you a place to run your software. But as the techno-utopians begin to try to sell you on quadcore processors (or more), before you hand over your money in fistfulls to them and their words of a perfect idealism, it might be wise to really look into what you actually use and what can run it well. Unused and under-used cores are just wasted potential. Buying a computer is only going to get more and more complex. We can either pay through the nose to have some 'expert' decide what we need for us, or to sate our desire for what we think is the best. Or, we can start learning with steps like these and turn ourselves into more frugal, educated, and discerning customers.

The choice is yours. I'm just here to help.

druidrocker 62F

7/17/2006 4:02 pm

Thanks - I almost understood all of it. I especially loved the egg relay - that explains what the programmers have been doing at work - at least that all makes sense now. I can't wait to pass this along to my friends.
Have you ever considered becoming a teacher ? You do this really well.

vrec_dawn replies on 7/17/2006 6:53 pm:
Ja, an awful lot of egg relays out there.

And a teacher? I don't know ...

Become a member to create a blog