The Future of Concurrency and Parallelism in C++: notes from the C++ subcommittee meeting

DrBartosz · 12 days

The committee is just a group of volunteers. Anybody with better ideas can join in and propose their solution. People with experience in designing aquatic fowl are especially welcome ;-)

DrBartosz · 12 days

@Kranar: That's one way of looking at it, and I won't deny that Herb Sutter can be very vocal promoting Microsoft's agenda. On the other hand their implementation offers substantially better performance, and could be available this year. I don't support the emasculation of thread-local variables, but I sympathize with the push towards better performance.

andralex · 14 days

Okay, so Voldemord types are not exactly existential types. They are poor-man's existential types. I like the name though.

tompa_coder · 1 month

I agree. Most Haskell books and tutorials put too much stress on arithmetic examples. How many ways are there to sum a list of integers?

DrBartosz · 1 month

do
   print 0
   print 1

DrBartosz · 1 month

Simon P-J put it nicely, "You get respectable improvement in performance for quite modest investment of programmer's time on cores that you otherwise were not using anyway."

DrBartosz · 1 month

Parallelism in this context is defined as improving throughput by splitting work between multiple cores or GPUs. The important difference is that to achieve speedup you can't just create a separate thread per unit of work or you'll be overwhelmed with the overhead. Rather you attach a few threads to cores and let the runtime split your tasks between them. Because of massive parallelism, you can't really use locks in your code. They would become bottlenecks. It's possible to write parallel code in C++ using various libraries, but they are all vulnerable to data races.

DrBartosz · 1 month

You understand that allowing external IO inside a transaction would open doors to all kinds of disasters (you can't undo the launching of ballistic missiles). So your example can't be done using transactional memory. You have to fall back on MVars.

DrBartosz · 1 month

It's not about how common they are, it's about how hard they are to detect. Are you sure you've found all the race conditions?

DrBartosz · 1 month

I think you totally missed the point. It was the hardware that required a read after a write to commit the write. They just didn't realize that. But inside the do block, a read whose result is ignored is still an IO action and will be executed for side effects. That's in perfect accordance with the tenets of functional programming.

DrBartosz · 1 month

I could add to this list ownership systems and sophisticated effects systems. When can we expect the fruits of this new research in D?

DrBartosz · 1 month

You might be right, but what should we do in the next 10 or more years before this hypothetical disruptive language materializes? I'm all for disruptive new languages, but I'd like to do something in the meantime.

As for Herb Sutter's call to arms, he correctly extrapolated changes in hardware and predicted the need to embrace parallel programming, yet this need remains largely unfulfilled. The situation was tolerable as long as we were dealing with dual or even quad cores, but what about 8 or 16? I hear people saying that there aren't that many programs that can be parallelized, and that might be true. The problem is that the new hardware offers previously unheard of power that could find totally new applications, but the software technology doesn't enable it. If I told you that I can put a Cray in your laptop, would you say that you don't need it because your mail program doesn't need that much computing power? Think of all the new applications that would be possible given that much computing power. Could you write them in Java?

DrBartosz · 1 month

Again, you're talking about concurrency not parallelism. And you are obviously and expert if you've been doing it for a decade. Would you be comfortable, though, passing your code to a new hire for maintenance?

DrBartosz · 1 month

But you are citing Java, C, and C++ in the same breath. They all deal with concurrency only, and C++ embraced it only in the latest standard. Parallelism is still at the level of manufacture in those languages. What we need is an industry.

DrBartosz · 1 month

You are listing some pervasive stereotypes about Haskell. (Have you seen Manuel Chakravarty's video I linked to?) It's true that a hand-optimized parallel C program is faster than Haskell. If you have time and resources, and reliability is not an issue, C might be the best option. What Haskell can provide is a quick, cheap, and reliable way to take advantage of the cores that otherwise would be idle. While the optimizers in imperative languages concentrate on low-level optimizations, functional languages can optimize at a much higher level. And even supposing the worst case scenario: a Haskell program running slower on a GPGPU than on a single general purpose core, it would still let you use less energy -- something that's becoming more important than speed. It's a new world.

DrBartosz · 1 month

We are still at the beginning of this trend so you might not see the signs. But the free lunch is over, hardware vendors are raining cores on us, and software developers have huge problems catching up. This creates a non-equilibrium, which is bound to cause a major disruption. My thesis is that there is a major roadblock that prevents imperative languages from embracing parallelism. Functional languages offer a way to bypass this roadblock. What do you think will happen?

DrBartosz · 1 month

There is a crucial difference between concurrency and parallelism.

DrBartosz · 1 month

You're missing the point. We want the guarantee enforced at compile time that parallel code doesn't share mutable memory. What you're talking about is enforcing the discipline of not sharing memory in your parallel code. This goes against nature: Humans are supposed to be creative, computers are supposed to be reliable; not the other way around.

DrBartosz · 1 month

I don't think anybody is giving up on the core tenets of functional programming. Haskel came up with ingenious ways of incorporating side effects inside purely functional paradigm. I don't want to get into deep theoretical discussion here because it's really not necessary to program in Haskell. The bottom line is that you have something that looks like imperative programming in Haskell except that it's safe. You can mix in functional code into your "imperative" code, but not the other way around.

DrBartosz · 1 month

To my defense: The granularity problem is not specific to functional programming. It's even more pronounced in C++ with PPL and TBB. The difference is that in C++ you not only have to concentrate on how to best parallelize your program to squeeze more performance out of it, but you also have to worry about data races. The naive parallelization will result in sub-par performance in Haskell and in serious bugs and subpar performance in C++.

As for flames, this is normal on reddit, no matter how balanced the article is. The alternative is complete silence, when the topic is boring.

DrBartosz · 1 month

Hi Andrei! I know where you're coming from, so let me explain it to other readers. In the D programming language you have to declare a mutable variable "shared" if you want to share it across threads. Otherwise it will be thread-local. So the combination of pure functions and immutable and shared variables could make D a good platform for concurrent programming, as I freely admit at the end of my blog.

As for the "difficulties that functional paradigm has," a lot of this is just hearsay. Haskell doesn't advocate celibacy, it only promotes the use of monads for your own protection.

DrBartosz · 1 month

You mean it in a good sense or a bad sense? :-)

DrBartosz · 1 month

This is really a topic for a separate blog post. Some aggressive compiler optimization might manifest themselves as data races on a single processor, but the ones associated with weak processor memory models will show up only on a multicore.

DrBartosz · 1 month

Conceptually, every time you want to mutate a data structure you create a new one instead, with the mutation already in it.

Behind the scenes Haskell eliminates most of the copying by clever use of (read-only) sharing. Think of a singly linked list. When you want to add a new element to the head of the list, you create a new list with this element as head, and with a copy of the old list as tail. Except that you don't copy the old list, just point to it from the new head. Since the tail never changes, the result is as if you made a fresh copy.

Obviously, if you want to add an element to the end of the list, you have to copy it. But you almost never append to lists in Haskell programs.

DrBartosz

TROPHY CASE

you'll need to login or register to do that

create a new account

login