How do Linux Threads Compare to NT Threads? 6
BurnMage asks: "I run a small network at a company that does heavy web and graphic work, as well as digital video production. The DV stuff is absolutely wonderful, cutting edge really nice hardware that costs thousands of dollars... and is only supported on the Windows platform. My personal experience with the machines has really been a constant nightmare until recently; we have gotten things running relatively smoothly. We use the DVRex and RexFX from Canopus, beautiful stuff and great tech support... but a few small messages asking why not port to Linux had a couple people muttering that threads in Linux 'aren't properly implemented'. I have also heard the same about NT. What's the story? I want a clear answer to post on their site."
``Not properly implemented.'' (Score:4)
There are some differences in signal handling, and also notable differences in file locking. According to POSIX, a lock set by fcntl() is owned by a process, so if one thread places a lock, and another thread in the same process places a conflicting lock, the lock region should simply be extended In Linux, the second thread will block because it is de-facto a second process and the flock() code doesn't take into account that the process is actually part of a ``cluster'' of threads.
There also have been issues in the threading library as recently as glibc 2.1.1. I submitted a patch that fixed a serious crash in the case that the process creates too many threads (which could happen even if the number of threads is low, due to ulimit).
Other than some stability issues and incompatibilities with POSIX, I would not conclude that LinuxThreads is improperly implemented. The API is there and works. Things like mutexes, condition variables and thread-specific keys are nicely implemented. I particularly like the wait-free queuing algorithm used to resolve contentions for locks, based on an atomic compare-exchange sequence.
As for NT threads: there are some nasty issues that require painful workarounds. I could write volumes about the deficiencies, but here are some highlights:
1. No cancellation mechanism! In NT, if you want to shut down a thread in an orderly manner, you have to figure out what object(s) it is blocking on and disentangle it. In POSIX threads, you simply cancel the thread and when it hits the next cancellation point, it executes registered cleanup handlers and terminates. In NT, there are no cleanup handlers; a thread killed by TerminateThread just leaks all of its local resources.
2. Thread local storage is a joke! In POSIX, every thread specific key lets you register a cleanup handler. When a thread terminates, it executes cleanup handlers for all thread specific keys that have handlers. The only way I know of to do this in NT is to put your stuff into a DLL, so that you can hook the clean up to the THREAD_DETACH command passed to DllMain()---under this solution you have to write your own thread specific storage subsystem that supports destructors.
3. No condition variables! NT's events are poor substitutes for condition variables. There is no way to create an event that can wake up all threads waiting on it AND which can wake up just a single thread. With POSIX conditions, you have pthread_cond_signal() and pthread_cond_broadcast(). Furthermore, pthread_cond_wait() is a cancellation point (recall that NT has no cancellation). Events are horrible, stateful objects. Conditions are stateless, making it easier to verify code to be free of race conditions and other problems.
4. Poor exclusion mechanisms! Critical sections are the rough equivalent of the POSIX mutex. But they are fraught with problems. For one thing, each critical section has its own internal event that is used to resolve contention. This event is allocated when needed. Thus in low memory conditions, EnterCriticalSection() can throw a win32 exception due to failure to allocate the event handle! InitializeCriticalSection() returns nothing. (But there is a little known hack in NT only: if you pass a spin count with its high order bit set to InitializeCriticalSectionAndSpincount() then it will pre-allocate the event). Preallocated or not, each critical section potentially consumes non-swappable kernel memory!
By contrast, Linux mutexes do not consume any kernel resources. Nothing has to ever be allocated other than the space for the pthread_mutex_t object itself. Contention is resolved by suspending, and each thread comes with a way to suspend and resume so no semaphore-like object needs to be allocated.
Another problem with NT critical sections is that there is no way to atomically release one of these and wait on some object. POSIX has pthread_cond_wait() which releases a mutex and waits on a condition in a way that avoids losing the wakeup signalled from some other thread that subsequently acquires the mutex.
My experiences with Win32 programming under NT have led me to implement everything myself that is missing in the low level API: thread specific storage, mutexes and condition variables, cancellation, etc.
Looks like your co-workers read Russinovich (Score:5)
The basic story is that Linux, unlike most operating systems, does not have a hard distinction between a thread and a process. Instead it has the idea of a context of execution, and with the clone() call a context of execution can reproduce a copy of itself, and decide how much is copied. The last is important, clone() can be anything from spawning another thread to a traditional fork.
This threading model is somewhat different than what most operating systems provide, but it is quite sufficient to provide a full POSIX thread implementation with very good speed on a context-switch between threads or processes. (In fact Linux does a faster context switch between processes than most operating systems do switches between threads on the same hardware.)
By contrast NT has a different problem. NT has tremendous difficulty with processes. The time to create a process is abysmal. Context switches are not cheap. And once you start paging, the paging algorithm has beeen found to literally worse than straight chance!
However NT has a pretty good time on context switches between threads and (on paper) some nice specs for working with them. But NT's threading model is somewhat different from the POSIX model and anyone who is experienced with Microsoft knows that what is on paper and what really happens are not always the same...
Cheers,
Ben Tilly
Re:crap. (Score:1)
aren't answered in the FAQ and kernel documentation, I
suggest that you ask the author of the thread system,
Dr Xavier Leroy (Xavier.Leroy@inria.fr).
Threads in Linux are kernel-level only. (Score:2)
Kernel-level threads can take advantage of multiple processors, but are quite expensive in process switching. User-levels, on the other hand, cannot take advantage of SMP machines because they exist in a single process with respect to the kernel, but are able to switch thread much faster because a context switch unnecessary.
This is also the reason who Java benchmarks so poorly on Linux boxes, because the most often seen tests are based on the performance with MANY threads. Linux is really poor when trying to create hundreds of threads.
My own experiements have shown that creating 8 threads using thread_create() can take a few seconds. On Solaris, which implements BOTH user-level and kernel-level threads, creating thousands of user-level threads take no time at all.
What I really would like is that someone would write a Solaris-similiar API for both user and kernel level threads. This would make it possible for application developers to choose (or even combine) the use of the two thread models for their specific purpose. The Solaris thread API allows the programmer to combine a number of user level thread into a kernel level thread which is scheduled by the kernel.
Would anyone care to take up this challenge?
crap. (Score:1)
libc5 or glibc2.1? (Score:1)
Bye,
Antonio