Labo ID

Laboratoire Informatique et Distribution

Athapascan-0 Athapascan-1 Athapascan-tr Pajé Givaro PEPS PSI Takakaw Ka-Tools NFSP

APACHE Project

Laboratoire Informatique et Distribution (ID)
ENSIMAG - antenne de Montbonnot
ZIRST 51, avenue Jean Kuntzmann

Athapascan-0 : Integrating Multithreading and Communications


Athapascan-0 was conceived to integrate communications and multithreading. Its design is based on a simple juxtaposition of multithreading  and communication concepts. Athapascan-0 is organised on two layers. The upper layer , called Athapascan-0 layer, is a programming interface which offers thread-to-thread communication between threads on different adress spaces, local and remote thread creation (popup model), and remote thread execution (upcall model). The bottow layer, called portability layer, integrates multithreading and communications adding thread-awareness to MPI.

Portability is one of  Athapascan-0  main design edgoals. To achieve it, Athapascan-0 is based on two well-known standards : MPI and POSIX threads. 

Programming model and main fonctionnalities

 Athapascan-0 extends the static heavyweighted process communicating model offered by MPI implementation to a lighter one,  expressed as a dynamic network of communicating threads. An Athapascan-0 program runs over a virtual machine composed  by at least one Athapascan-0 task using a SPMD programming model.  Athapascan-0 tasks correspond to processes created by a MPI application.  Like MPI, several Athapascan-0 tasks can be created on the same physical node. 

A computation is first divided on Athapascan-0 tasks (processess) and further, inside a task, divided on smaller pieces called slaves (threads). A slave can create another slave locally or remotely. Remote creation is given by service concept (popup model) and remote execution is given by urgent service concept (upcall model). 

Threads (slaves) on the same adress space communicate through a shared memory.  Synchronisation is done by classical mechanisms : mutex, semaphores and conditon variables.

Threads (slaves) on different adress spaces communicate and synchronise using point to point communication primitives based on MPI ones. Communications channels between Athapascan-0 tasks are created using a port concept. Any thread can  send or receive data on a given port. A communication end point is defined by a pair <source/destination id , port>.

1990 - The OUF library (Occam Users Frustration) was written for the Meganode (a 128 Inmos transputer-based parallel machine). OUF provided a common parallel programming interface, which allowed programs to be compiled with any of the existing compilers (Inmos, C3L, etc.). The programming model was communicating processes and there was no support for profiling or debugging.

1991 - The OUF library was ported to Sun3 and Sun4 architectures, in order to benefit from Unix debuggers. It served as the lower layer for an experimental toolkit used to write dynamic schedulers. It was written using PVM 1.0 and Sun's light  weight processes (LWP) library. LWP provided preemptive threads which emulated the transputers time-sharing and PVM provided the communication primitives and the virtual parallel machine. The remote procedure call abstraction (RPC) was chosen as a convenient means for writing distributed programs and  to implement them  on top of Sun OUF.

1993 - The APACHE project was created across the LMC-IMAG (now ID-IMAG) and LGI-IMAG laboratories. Athapascan was defined as a portable and efficient environment for parallel programming, along with profiling, analysis, deterministic reexecution and debugging. It was conceived on two layers: Athapascan-0, which implemented the parallel RPC extension and Athapascan-1, the user-level interface and the load balancing. Athapascan-0a kernel was the first version of Athapascan-0, written on top of PVM and different threads libraries.

1995 - MPI and POSIX threads were chosen as the new basic libraries of Athapascan, for performance and portability reasons.Athapascan-0 was redesigned to be fully preemptive with priorities, with remote thread creation and thread-to-thread communication..

Papers Christaller, M.  and  Briat, J. and  Rivière, M. "Athapascan-0 : concepts structurants simples pour une programmation parallèle efficace". Calculateurs Parallèles, 1995, vol. 7, no. 2,  pp. 173-196.

Briat, J. and Ginzburg, I. and Pasin, M. and Plateau, B. "Athapascan Runtime : Efficiency for Irregular Problems". Springer Verlag. Proceedings of the Europar'97 Conference. Passau, Germany, Aug. 1997, pp. 590-599.

Briat, J. and Ginzburg, I. and Pasin, M. "Athapascan-0b : un noyau exécutif parallèle".  Lettre du Calculateur Parallèle, 1998, vol. 10, no. 3, pp. 273-293.

A. Carissimi and M. Pasin "Athapascan: An Experience on Mixing  MPI Commmunications and Threads". Lecture Notes in Computer Science (LNCS 1497).  Spring Verlang.  5th European PVM/MPI User's Group Meeting.  Proceedings, Liverpool, UK, Sept. 1998, pp. 137-144.

These papers (we guess) can be very useful  as an introduction to Athapascan-0. You can find a complete list of papers references consulting our publication list.

Thesis (French) Christaller, Michel "Vers un support d'exécution portable pour applications parallèles irrégulières : Athapascan-0". Thèse de doctorat en informatique. Université Joseph Fourier, Grenoble, France, Nov. 1996. (check  here for an eletronic version.)

Ginzburg, Ilan "Athapascan-0b : Intégration efficace et portable de multiprogrammation légère et de communications".Thèse de doctorat en informatique. Institut Polytechnique de Grenoble, Grenoble, France, Sept. 1997. (ps and pdf files here.)

Rivière,  Michel "Concepts structurants pour la mise en oeuvre d'applications irrégulières : Application au support exécutif parallèle Athapascan-0". Thèse de doctorat en informatique. Université Joseph Fourier, Grenoble, France, Oct. 1997. (available only at mediathèque. Sorry!)

Carissimi, Alexandre da Silva "Athapascan-0 : Exploitation de la multiprogrammation légère sur les grappes de stations  multiprocesseurs".Thèse de doctorat en informatique. Institut Polytechnique de Grenoble, Grenoble, France, Nov. 1999. (ps and pdf files here.)

Current Supported Systems
Muliprocessor version
(runs on monoprocessors also)
We tested SMP/monoprocessor version in the following systems :
  • Intel with Linux 2.x kernel 
  • Sparc or Intel with Solaris 2.6/2.7
  • IBM SP with AIX 4.2/AIX 4.3
Using MPICH (see InstallationTips), LAM MPI (see Installation Tips) and MPI-IBM.  At thread level we used the POSIX threads library available on these systems.

We are able to take benefits from high performance networks like myrinet and SCI. For that we are based on MPI-BIP, MPI-GM for myrinet and MP-MPICH for SCI.

Monoprocessor version
We tested monoprocessor version in the following system :
  • IBM SP with AIX 3.2.5 and IBM/MPI-F 
  • IBM SP with AIX 4.2 and IBM/MPI 
  • DEC/Alpha with OSF/1 4.0 and LAM/MPI 6.1 
  • HP-9000 with HP-UX 10.20 and LAM/MPI 6.1 
  • SGI/MIPS with IRIX 6.3 and LAM/MPI 6.1 
  • SGI/MIPS with IRIX 6.4 and SGI/MPI-3.1 
  • Sparc or Intel with Solaris 2.5 and LAM/MPI 6.1 
  • Intel with Linux 2.0.25, MIT threads 1.60.6 and LAM/MPI 6.1
What version should I use? Athapascan-0 is distributed on two versions:
  • Multiprocessor
  • Monoprocessor
The multiprocessor version was conceived to exploit efficiently the existing processors on multiprocessor machines using kernel threads. If you have a multiprocessor machine this version is more suitable for you.  If you have an heterogeneous network mixing monoprocessor and
multiprocessor machines it  is better  for you to use this version.

The monoprocessor version is kept by historical reason :  it was our first product and it was broadly tested , ported  and tunned for several systems (see current supported systems).  This version uses user level threads (or kernel threads if the target systems offers only this model, e.g., AIX 4.2). This version is more suitable for monoprocessors. It works on multiprocessor machines BUT it offers an awful performance.

What  do I need to install Athapascan?? Install Athapascan-0 is very easy!!

You only need an MPI implantation and a thread package on your system.

First of all you need to choose an MPI implantation. This can be done considering your personnal preference for an MPI implantation, to take benefits from special hardware like myrinet, or use of an already installed MPI package in your system. Nevertheless you need to pay attention to some details about MPI and threads; for that, see Installation Tips.

You need also a thread package. Nowadays most  systems include one, but if you wish to install Athapascan-0 on a system which doesn't offer a pre-existing thread-package you can find some pointers for them here.

Multiprocessor version  Download here the current Athapascan-0 multiprocessor (SMP) version (3.0.0).
Monoprocessor version  Download here the current Athapascan-0 monoprocessor version (2.5.0).
Installation Tips (MPI)
MPICH  ATTENTION :  MPICH over TCP/IP uses  ch_p4  as device. P4 is an ancient
communication library developed on a not thread-safe fashion. You CAN'T use ch_p4
device with threads consequently Athapascan doesn't run on this configuration.
LAM MPI  LAM on AIX 4.2 systems

Before installing LAM on AIX 4.2 systems you need  to modify the following lines on  share/h/lam_config.h file :

#define HAVE_BSD43_FD_PASSING      1
#define HAVE_BSD44_FD_PASSING      0

#define HAVE_BSD43_FD_PASSING      0
#define HAVE_BSD44_FD_PASSING      1

LAM on Linux systems

You MUST use on LAM configure command line the following options:

  --with-cflags='-D_REENTRANT' --with-signal=SIGUNUSED

LAM on Solaris systems

You MUST use on LAM configure command line the following options:

--with-cflags='-D_REENTRANT' --with-ldflags='-lpthread -lthread -lposix4'
Brigitte Plateau (Project Manager)
Jacques Briat (Athapascan-0 leader)
Philippe Waille 
Michel Christaller (left nov. 96)
Ilan Ginzburg (left sept. 97)
Michel Rivière (left oct. 97)
Marcelo Pasin (left aug. 98)
Alexandre Carissimi (left nov. 99)

Despite these "people left", Athapascan-0 is still supported.  Questions? problems ? Contact us using

logo ID
loupeChercher logo_info.gif Pratique enveloppe Ecrire ŕ l´administrateur

dernière mise à jour : 08 janvier 2003