Understanding Linux Kernel Inter-process Communication: Pipes, FIFO & IPC (Part 2)

IPC Semaphores

In my last article entitled "Linux Kernel Synchronization", I did talk about "Semaphores". Right? But, readers must NOT confuse "POSIX Realtime Extension Kernel Thread Semaphores" with "System V Semaphores", more specifically referred to as IPC Semaphores. They are totally different entities, and MUST NOT be confused. Moreover, I have seen people trying their level best to translate the former interface functions into the later. This is not only dangerous (for the application using it), but personally I feel it's illegal. Thus, I use a simple distinction between them to avoid confusion: I like POSIX semaphores, I hate IPC Semaphores. As simple as that. Here in this section, we talk about IPC Semaphores. IPC semaphores are counters used to provide controlled access to shared data structures for multiple processes. The semaphore value is positive if the protected resource is available, and negative or 0 if the protected resource in not currently available. A process that wants to access the resource decrements by 1 the semaphore value. It is allowed to use the resource, if and only if the old value was positive. If not, the process waits until the semaphore becomes positive. When a process releases a protected resource, it increments its semaphore value by 1, in doing so, any other process waiting for the semaphore is woken up. This is how IPC Semaphores are designed to operate, thereby locking critical sections of code. Before we move on any further, let's look at a particular term: "primitive semaphores". What are they?

In Kernel semaphores (i.e. "POSIX Realtime Extension Kernel Thread Semaphores"), a semaphore is JUST a single value. But, in case of IPC semaphores, each IPC semaphore is a set of one OR more semaphore values. This means that the same IPC resource can protect several independent shared data structures. A function named semget() exists, and the number of semaphore values in each IPC semaphore must be specified as a parameter of semget() function when the resource is being allocated. But readers must note, that this value musn't exceed SEMMSL, whose value is 32. The counters inside an IPC semaphore is called a "primitive semaphore". Let's now consider a code snippet that illustrates a System V semaphore in action.

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>

int semget ( key_t key, int nsems, int semflg );
int semop ( int semid, struct sembuf *sops, unsigned nsops);

struct sembuf
{

short sem_num; /* semaphore number: 0 = first */
short sem_op; /* semaphore operation */
short sem_flg; /* operation flags */

};

int semctl (int semid, int semnum, int cmd, union semun arg);

union semun
{

int val; /* value for SETVAL */
struct semid_ds *buf; /* buffer for IPC_STAT, IPC_SET */
unsigned short int *array; /* array for GETALL, SETALL */
struct seminfo *__buf; /* buffer for IPC_INFO */

};

IPC_STAT
IPC_SET
IPC_RMID
GETALL
SETALL
SETVAL
struct semid_ds;

When semget() is called and it acquires the semaphore set's ID, one can perform operations on this set using semop. The struct sembuf has fields that are filled in to represent the requested action on the semaphore. The first field specifies the number of the semaphore in the set you wish to operate on. The second field (sem_op) contains the operation you wish to do. If sem_op is positive, the value is added to the semaphore (ie release of resources). If sem_op is less than or equal to zero, semop(2) will block until the semaphore reaches abs(sem_op). You may also specify various options in sem_flg, including IPC_NOWAIT, which tells semop(2) NOT to block. When SEM_UNDO is specified, all actions are undone when the current process exits. Undo is guaranteed with private semaphores only. Notice that unlike the other IPC operations, the third argument to semctl is a union, not a simple struct. This is because the various options must occurr on different data. Various operations can be performed using semctl(2) IPC_STAT can be used to fill in the struct semid_ds structure. You can then set various fields (including changing ownership and premissions of ipc_perm) using IPC_SET. But the most important ctl operation is IPC_RMID. When all of your programs are done using a semaphore, you MUST semctl it with IPC_RMID (the shmid_ds structure may be NULL), or else the semaphore will remain in memory forever. You may also set and get specific values of the semaphore using GETVAL, SETVAL, GETALL, and SETALL.

Now that we have seen how a semaphore works, let's get down into the Linux Kernel, and see what exactly happens inside the Kernel. The typical steps performed by a process wishing to access one or more resources protected by an IPC semaphore are as follows:

1. The semget() wrapper function is invoked, to get the IPC semaphore identifier. If the process wants to create a new IPC semaphore, it also specifies the IPC_CREATE or IPC_PRIVATE flag and the number of "primitive semaphores" required.
2. Invokes the semop() wrapper function to test, and decrement all primitive semaphore values involved. If all tests succeed, the decrements are performed, the function terminates, and the process is allowed to access the protected resources.
3. When relinquishing the protected resources, it then invokes the semop() function again, this time, for atomically incrementing all primitive semaphores involved.
4. Optionally, it then invokes the semctl() wrapper functions, specifying in its parameter the IPC_RMID flag to remove the IPC semaphore from the system.

This is how an IPC semaphore is created, acts, and then is deleted from a system. So, let's now move on to the next exciting section, IPC Messages.