# 13.7. Source code

## 13.7.1. Code style

We intentionally do not have too many code conventions in the Open MPI code base.

### 13.7.1.1. All languages

- 4 space tabs. No more, no less.

- NEVER use actual tab characters; always use spaces. Both emacs and vim have secret mojo that can automatically use spaces when you hit the `<TAB>` key. This makes the code look the same in every browser, regardless of individual tab display settings.

### 13.7.1.2. C / C++

- When comparing constants for equality or inequality, always put the constant on the left. This is defensive programming: if you have a typo in the test and miss a `!` or `=`, you’ll get a compiler error. For example:

      /* Do this */
      if (NULL == foo) { ... }

      /* Because if you have a typo (i.e., = instead of ==), this will
         be a compile error rather than a subtle bug */
      if (NULL = foo) { ... }

- More defensive programming: always include blocks in curly braces `{ }`, even if they are only one line long. For example:

      /* Do this */
      if (whatever) {
          return OMPI_SUCCESS;
      }

      /* Not this */
      if (whatever)
         return OMPI_SUCCESS;

- Starting with Open MPI 1.7, Open MPI requires a C99-compliant compiler.

  - C++-style comments are now allowed (and preferred).

  - C99-style mixing declarations are allow allowable (and preferred).

- ALWAYS include `<level>_config.h` as your first #include file, where `<level>` is one of `ompi`, `oshmem`, or `opal` — the level that you’re writing in. There are very, very few cases where this is not true (E.g., some bizarre Windows scenarios). But in 99.9999% of cases, this file should be included first so that it can affect system-level #include files if necessary.

- Filenames and symbols must follow the prefix rule (see [e-mail thread](http://www.open-mpi.org/community/lists/devel/2009/07/6389.php)):

  - Filenames must be prefixed with `<framework>_<component>`.

  - Public symbols must be prefixed in components with `<project>_<framework>_<component>`, where `<project>` is one of `mca`, `ompi`, `oshmem`, or `opal`. Note that mca used to be the most common, but it has fallen out of favor compared to the other `<project>` prefixes. When in doubt about whether a symbol is public, be safe and add the prefix.

  - Non-public symbols must be declared `static` or otherwise made to not appear in the global scope.

- ALWAYS #define macros, even for logical values.

  - The GNU Way is to `#define` a macro when it is “true” and to `#undef` it when it is “false”.

  - In Open MPI, we always `#define` a logical macro to be either 0 or 1 — we never `#undef` it.

  - The reason for this is defensive programming: if you are only checking if a preprocessor macro is defined (via `#ifdef FOO` or `#if defined(FOO)`), you will get no warning when compiling if you accidentally misspell the macro name. However, if you use the logic test `#if FOO` with an undefined macro (e.g., because you misspelled it), you’ll get a compiler warning or error.

    Rationale

    Misspelled macro names can be tremendously difficult to find when they are buried in thousands of lines of code; we will take all the help from the preprocessor/compiler that we can get!

      /* GNU Way - you will get no warning from the compiler if you
         misspell "FOO"; the test will simply be false */
      #ifdef FOO
      ...
      #else
      ...
      #endif

      /* Open MPI Way - you will get a warning from the compiler if you
         misspell "FOO"; the result of the test is a different value
         than whether you spelled the macro name right or not */
      #if FOO
      ...
      #else
      ...
      #endif

### 13.7.1.3. Fortran

We do not have specific coding style guidelines for Fortran. Please read some of the existing Fortran code in the source code tree and try to use a similar style.

### 13.7.1.4. Shell scripting

Please read some of the existing shell code in the source code tree and try to use a similar style.

- Always enclose evaluated shell variables in quotes to ensure that multi-token values are handled properly.

      # This is bad
      if test $foo = bar; then

      # This is good
      if test "$foo" = "bar"; then

  - The one exception to this is that when doing an assignment to a shell variable from another shell variable, it is not necessary to use quotes on the right hand side:

        # This is harmless, but unnecessary
        foo="$bar"

        # This is actually sufficient, even for multi-token values of $bar
        foo=$bar

- Do not use the `==` operator for `test` — this is a GNU extension and can cause portability problems on BSD systems. Instead, use the single `=` operator.

      # This is bad
      if test "$foo" == "bar"; then

      # This is good
      if test "$foo" = "bar"; then

### 13.7.1.5. m4

We do not have specific coding style guidelines for m4 (the language used to create the `configure` script). Please read some of the existing m4 code in the source code tree and try to use a similar style.

## 13.7.2. Tree layout

There are a few notable top-level directories in the source tree:

- The main sub-projects:

  > - `oshmem`: Top-level OpenSHMEM code base

  > - `ompi`: The Open MPI code base

  > - `opal`: The OPAL code base

- `config`: M4 scripts supporting the top-level `configure` script `mpi.h`

- `etc`: Some miscellaneous text files

- `docs`: Source code for Open MPI documentation

- `examples`: Trivial MPI / OpenSHMEM example programs

- `3rd-party`: Included copies of required core libraries (either via Git submodules in Git clones or via binary tarballs).

  Note

  While it may be considered unusual, we include binary tarballs (instead of Git submodules) for 3rd party projects that are:

  1. Needed by Open MPI for correct operation, and

  2. Not universally included in OS distributions, and

  3. Rarely updated.

Each of the three main source directories (`oshmem`, `ompi`, and `opal`) generate at least a top-level library named `liboshmem`, `libmpi`, and `libopen-pal`, respectively. They can be built as either static or shared libraries. Executables are also produced in subdirectories of some of the trees.

The `libopen-pal` top-level library is built internally in two parts:

- `libopen-pal_core` Internal “core” portion of OPAL containing the essential source and MCA needed for tools like mpicc/mpirun to link against. The “core” library is not installed.

  > - Includes the following MCA frameworks: `backtrace`, `dl`, `installdirs`, `threads`, `timer`

  > - Includes all of the source under `opal/class` and most of `opal/util`

  > - Includes the files suffixed with `_core` in `opal/runtime`

- `libopen-pal` Includes “core” plus all of the other OPAL project sources. This is installed.

Each of the sub-project source directories have similar (but not identical) directory structures under them:

- `class`: C++-like “classes” (using the OPAL class system) specific to this project

- `include`: Top-level include files specific to this project

- `mca`: MCA frameworks and components specific to this project

- `runtime`: Startup and shutdown of this project at runtime

- `tools`: Executables specific to this project

- `util`: Random utility code

There are other top-level directories in each of the sub-projects, each having to do with specific logic and code for that project. For example, the MPI API implementations can be found under `ompi/mpi/LANGUAGE`, where `LANGUAGE` is `c`, `fortran`, or `java`.

The layout of the `mca` trees are strictly defined. They are of the form:

    PROJECT/mca/FRAMEWORK/COMPONENT

To be explicit: it is forbidden to have a directory under the `mca` trees that does not meet this template (with the exception of `base` directories, explained below). Hence, only framework and component code can be in the `mca` trees.

That is, framework and component names must be valid directory names (and C variables; more on that later). For example, the TCP BTL component is located in `opal/mca/btl/tcp/`.

The name `base` is reserved; there cannot be a framework or component named `base`. Directories named `base` are reserved for the implementation of the MCA and frameworks. Here are a few examples (as of the v5.0.x source tree):

    # Main implementation of the MCA
    opal/mca/base

    # Implementation of the btl framework
    opal/mca/btl/base

    # Implementation of the sysv framework
    oshmem/mcs/sshmem/sysv

    # Implementation of the pml framework
    ompi/mca/pml/base

Under these mandated directories, frameworks and/or components may have arbitrary directory structures, however.

## 13.7.3. Symbol Visibility

The `*_DECLSPEC` macros provide a method to annotate symbols to indicate their intended visibility when compiling dynamically shared object files (e.g., `libmpi.so`). The macros are defined on a per project basis:

- Open MPI: `OMPI_DECLSPEC`

- Open PAL: `OPAL_DECLSPEC`

- OpenSHMEM: `OSHMEM_DECLSPEC`

The macros expand to the appropriate compiler and platform flags for marking whether a symbol should be explicitly made public in the target project’s library namespace. The `*_DECLSPEC` attributes are used to declare that a symbol is to be visible outside of that library/DSO’s scope. For example, `OMPI_DECLSPEC` is used to control what symbols are visible in the `libmpi.so` scope.

Note

This is entirely related to dynamic library compilation and does not apply to static compilation.

Note

The macros were originally introduced when Open MPI supported Windows (circa Open MPI v1.0.0) and are motivated by the Windows __declspec. While support for Windows has been dropped from Open MPI, the symbol visibility macros remain.