Open Watcom on Linux

Peter Chapin

Abstract: Cross compile to and from Linux with Open Watcom


Peter Chapin is a professor of computer engineering technology at Vermont
Technical College and a computer science graduate student at the University
of Vermont. He has been involved with Open Watcom since 2004 and the
project maintainer since early 2007. He can be reached at
peter@openwatcom.org.


Introduction
-------------

Open Watcom is the open source version of the commercial Watcom
system that was well known in the 1990s. Like its predecessor, Open
Watcom provides compilers for Fortran 77, C, and C++. It also provides
a rich collection of related tools such as an assembler, a debugger,
a linker, an editor, and a simple IDE to name just a few. In addition,
Open Watcom provides good cross platform support for Windows, OS/2, DOS,
and their variations. Indeed, Open Watcom is one of the few actively
maintained C/C++ compilers that can still generate 16 bit x86 code for
DOS or embedded targets.

The Open Watcom community has been working on adding Linux to the list
of supported systems.  Right now you can create Linux executables using
Open Watcom on a Windows, OS/2, or even a DOS system. It is also possible
to create Windows, OS/2, or DOS executables on a Linux system.  However,
Open Watcom's Linux port still requires a lot of work before it will be
fully mature.  In this article I hope to describe the current state of
Open Watcom on Linux and how you can get your hands on the source code
of the system to start playing with it.

It is our hope that ultimately Open Watcom will become a viable
alternative to gcc on Linux.  Because of Open Watcom's tradition of
being as self contained as possible, it provides its own libraries and
an independent, self-consistent tool set with all the advantages (and
disadvantages) pertaining.


History
-------

Open Watcom's history is long. It dates back to a Fortran compiler created
by a group of undergraduate students at the University of Waterloo in the
1960s. In the late 1980s a C compiler was also created. The first
commercial PC version of Watcom, version 6.0, was released in 1988. A
corresponding C++ compiler was released with version 9.5 in 1993, and the
three languages have been a part of the system ever since.

The last commercial version of Watcom, version 11.0, was released in
1997/1998. A year later Sybase, the owner of the system at the time,
announced that Watcom would be moved to "end of life" status and no longer
available as a commercial product. However, in 2000 Sybase released the
Watcom system as an open source product and thus Open Watcom was born.
SciTech Software then used the Perforce source code management system to
maintain and host the Open Watcom code base until 2007; Perforce Software
has provided hosting for the project since then.

Releasing the Watcom source openly was feasible because it had relatively
few dependencies on third party components. Most of the tools and libraries
used during the build of Open Watcom are part of Open Watcom. Yet despite
this a considerable amount of work was still necessary before Open Watcom
could compile its own source code. Certain parts of Open Watcom, such as
the port to QNX, are still not functional because of missing proprietary
libraries.

At the time of this writing, the current version of Open Watcom is 1.7a. In
the years since it has been opened, progress has been made on updating the
C compiler to the C99 standard and the C++ compiler to the C++98 standard.
Each release brings Open Watcom incrementally closer to these goals as well
as fixes numerous bugs in the previous release. In addition, the tools and
libraries have been enhanced. For example, Open Watcom supports the "Safer
C" extensions to the standard C library (ISO/IEC TR 24731, "Extensions to
the C Library, Part I: Bounds-checking interfaces")[should this be a
footnote?] and provides a (partially complete), fresh implementation of the
C++ Standard Template Library.


What Works, What Doesn't
-------------------------

Currently Open Watcom is immature on Linux. The system can be both
compiled using the Linux version of Open Watcom, for example as generated
on a Windows machine, and bootstrapped from gcc. However the build
process, and especially the bootstrap process, is a bit delicate. Work
is underway to make the Open Watcom source more portable specifically to
allow reliable bootstrapping not only on Linux, but other *nixes as well.

Open Watcom uses its own C and C++ run time libraries and thus does not
interact with glibc. While this is nice in some respects, it clearly would
be useful for it to have the ability to use standard Linux shared
libraries. The object file format used by the compilers (OMF) is unusual on
Linux systems. The Open Watcom linker is able to read this format and write
ELF executables, but can not at this time create or use ELF shared objects.
This means that Open Watcom programs are somewhat isolated and are not able
to tap into important shared libraries such as those for the X Window
System. Currently on Linux the entire source code for a program must be
compiled with Open Watcom.

On Windows and OS/2 Open Watcom provides a collection of GUI based
tools. This is accomplished by using an Open Watcom specific cross
platform GUI library that has been ported to the supported graphical
systems. However, at the time of this writing, none of the GUI tools work
on Linux. To fix this, one approach would be to port the current cross
platform GUI library to a Linux graphical tool kit such as GTK. However,
it has been suggested that such a port would be difficult due to the
current Windows and OS/2 assumptions made in the GUI library. The Open
Watcom wiki discusses these matters in more detail.

Despite the limitations there is much that does work. The language required
standard libraries are provided, to the extent they are provided on any
platform, and enough Linux specific functionality is available to write
some useful programs. Essentially all of the command line tools work, and
in addition text mode ("TUI") versions of the editor and debugger work.
Figure 1 shows the editor running on FreeDOS (one of Open Watcom's
supported host platforms) and Figure 2 shows the debugger on FreeDOS
debugging a 16 bit application. These same tools also work on Linux now.


Cross Compiling to Linux
------------------------

Since Open Watcom is a cross compiler, one must distinguish between "host"
systems and "target" systems. The terminology used by the Open Watcom
community is similar to that used by other cross compilation environments:
the host is the system on which the tools are run, and the target is the
system for which programs are being generated.

=======================
EDITOR: This should be fairly common terminology to Linux users,
as this is the same terminology that a standard "configure" script uses.
So, you might want to reword this to say something to that effect.
=======================
[Reworded above]

I will talk a little about how to build Open Watcom on Linux
shortly. However, you don't actually need to build the compiler if you
just want to experiment with Open Watcom's support for Linux. Instead
you can cross compile to the Linux target from any of the supported host
systems. Proceed as follows:

1. From the Open Watcom web site download the latest installer for either
Windows or OS/2.

2. During installation you will be prompted for which hosts you want to
install. Ordinarily you only need a single host, and usually that's the
same as the system on which you are installing.  You can install multiple
hosts if you intend to dual-boot your machine, or if you intend to make
the installation visible to other operating systems in some other way,
such as over a network or in a virtual machine.

3. Later in the installation you will be prompted for which targets you
want to install. Each host can generate all possible targets so the host
and target questions are independent. For our purposes you will want to
select at least the experimental Linux target.

When the installation finishes you are ready to cross compile to Linux
from your Windows or OS/2 system. To make a rudimentary test, enter the
classic program shown in Listing 1. Compile it with the 32 bit generating
C compiler as follows

wcc386 -i=C:\WATCOM\lh -bt=linux hello.c

Listing 1. Classic Test Program

#include <stdio.h>

int main(void)
{
  printf("Hello, Linux!\n");
  return 0;
}

=======================
EDITOR: A note here that points out more explicitly that you use the
same compiler to build all targets would be good.
This is different than when you use gcc, you need a target specific
version for each target, so some extra emphasis will help this sink in
(assuming I'm understanding Open Watcom correctly).
=======================
[Added a paragraph just below the next paragraph to clarify this.]

The -bt option specifies the "build type." By default wcc386 assumes you
are targeting the same system as your host. You need to override this
default for cross compilations. You also need to explicitly specify the
path to the Linux library headers. The result of the above command is
an OMF object file named hello.obj.

Notice that all 32 bit x86 targets are handled by the same compiler. System
dependencies are largely hidden in the run time libraries against which the
object files are linked. Sixteen bit targets, if you installed support for
any, are handled by a separate compiler named simply wcc.

Next link the object file you made above against the Linux version of the
run time library as follows

wlink sys linux file hello.obj

The Open Watcom linker's command line is a collection of keywords followed
by arguments for those keywords. The command above specifies that you are
trying to build a Linux executable from the given object file. The result
of this command is a file named hello.elf. To run it, just copy this file
to your favorite Linux system and give yourself execute permission to it.

Compilation of Fortran programs is just as simple. Listing 2 shows a
program that computes the roots of a quadratic equation with coefficients
read from the standard input device. Use the Open Watcom Fortran compiler
as follows

wfc386 hello.for

Listing 2. Classic Test Program, Fortran Style

      PROGRAM Hello
      IMPLICIT NONE
      REAL A, B, C
      REAL d, r1, r2

      READ *, A, B, C
      d = B * B - 4 * A * C
      IF ( d .LT. 0 ) THEN
         PRINT *, 'Complex Roots'
      ELSE
         r1 = ( -B + SQRT( d ) )/( 2 * A)
         r2 = ( -B - SQRT( d ) )/( 2 * A)
         PRINT *, r1, r2
      ENDIF
      END

Then link the resulting object file for Linux exactly as you did for the C
program.

You can also use the Open Watcom IDE (a thin graphical wrapper around the
command line tools) to cross compile to Linux by simply selecting the
"Linux" target type when defining your project.  The IDE takes care of
all the necessary compiler and linker options. Figure 3 shows a screen
shot of the Open Watcom IDE running on eComStation, the modern version
of OS/2.
=======================
EDITOR: Maybe a screen shot of the IDE.
Try to hide all the extra Windows stuff so we don't annoy our
readers to much.  Actually, a screen shot from OS/2 would be neat
if you have OS/2 running somewhere.
=======================
[Added screen shot of the IDE under OS/2.]

Notice that because Open Watcom provides its own run time library and
tools, it is not necessary to install any GNU-like tools or libraries on
the host system. Furthermore since Open Watcom currently only uses static
linking when building Linux executables, the final program is entirely
self-contained and independent of the libraries that are present on the
Linux machine where it runs.

=======================
EDITOR: How about a "Hello, Linux" program in Fortran also?
=======================
[Added one for fun. Maybe this will get some readers interested in
Fortran :-)]

=======================
EDITOR: You say above that Open Watcom will compile DOS programs.
Will it target FreeDOS?  How about compiling a "Hello, world" program
for FreeDOS?  A screen shot of it running on FreeDOS would be great.
=======================
[Added screen shots of the editor and debugger on FreeDOS at the end of
the "What Works, What Doesn't?" section.]


Building Open Watcom for Linux
------------------------------

If your nearest Windows machine is in the dumpster out back, you might
want to build the experimental Linux host natively on your Linux box.

The first step is, of course, getting the source code itself. The Open
Watcom project uses the Perforce source code management system. Since you
might not be familiar with Perforce, I will give detailed instructions
on how to use Perforce to access the Open Watcom source repository.
Although Perforce is a commercial product, qualified open source projects
can use the server for free. In addition client programs can be downloaded
for free from the Perforce web site. Both Windows and Linux clients are
supported. Although Perforce provides a graphical client for their system,
using the command line client (P4) is often more convenient and that is
the approach I describe here.

Once you have downloaded and installed a Perforce client for your system
you will need to configure some environment variables as shown in Listing
3. These settings direct the client to the Open Watcom repository (or
"depot" in Perforce terms). Each working copy of the source code (or
"client workspace" in Perforce terms) is identified by the P4CLIENT
environment variable.  The name of your workspace is up to you, but it
must be unique on the server. Use a name in the form shown in the listing;
for example, PeterC_DEVBOX_Linux. If everyone uses this form uniqueness
is virtually guaranteed since it is highly unlikely that another person
using your name would also choose the same names and operating systems
for his or her machines. Execute the command 'p4 info' to check your
connection to the server. You should see some information about the
server and not, for example, error messages.

Listing 3. Environment variables for building Open Watcom

export P4PORT=perforce.openwatcom.com:3488
export P4USER=anonymous
export P4PASSWD=anonymous
export P4CLIENT=YOURNAME_YOURMACHINE_YOUROS
export P4EDITOR=vi


Next you need to define the mapping from files in the depot to locations
on your machine. This is called the "client mapping" and it is part of
the definition of your client workspace.  Execute the command 'p4 client'
to load the current definition (a default) into your editor of choice. It
is essential to change two fields. The first names the location of the
source tree root. The other specifies how a particular view of the source
is mapped into your client workspace.

Root: /home/pchapin/OW
View: //depot/openwatcom/... //PeterC_DEVBOX_Linux/...

When you exit your editor p4 will update the definition of your client
workspace on the server.  Finally download the head revision with the
command 'p4 sync'.

Once you have the source you need to adjust one file before attempting a
bootstrap build. In the root of the source tree, copy the file setvars.sh
to setvars and edit the file to reflect your configuration. Note
especially the setting of the OWROOT environment variable. This variable
should contain the absolute path of the root of the source tree. Next
execute the build.sh shell script to bootstrap the Linux compilers
using gcc.

The Linux bootstrap build of Open Watcom is not as well tested or
as refined as the build procedure on Windows or OS/2. However, if it
completes successfully you will have a rel2 directory beneath the source
tree root containing the native Linux version of Open Watcom. Only a
subset of the entire system is built this way; specifically only the
Linux host and target are generated.

If you want to build the entire system using the freshly created tools,
you should source setvars into a shell, which adds the Linux host just
created to the path (among other things).  Then change into the bld
directory beneath the source tree and execute the command 'builder
rel2'. Note that even this does not build the documentation. This is
because some of the documentation generation tools are DOS programs, and
executing DOS programs on Linux requires extra steps that have not been
automated. There is work on correcting this by creating Linux versions
of the necessary tools, but that work is not yet completed.

Once the system has been built you might want to run some regression
tests. The tests are driven by wmake, Open Watcom's make utility, and must
be run separately for each subproject. For example, to run the C compiler
regression tests, change to the bld/ctest/regress directory and execute
wmake. The C++ compiler regression tests are in bld/plustest/regress.


Using Open Watcom on Linux
--------------------------

Once the build is complete, you can copy the rel2 directory anywhere
on your system and set up the environment variables as shown in Listing
4. Since you are now using native compilers you can use the Open Watcom
compile and link utility to build simple programs easily. For example
issuing the following command on your Linux system builds the simple
hello program I discussed before.

wcl386 hello.c

Listing 4. Environment variables for using Open Watcom

# Top level of the installation directory.
export WATCOM=/opt/OW

# Set up other environment variables.
export EDPATH=$WATCOM/eddat
export FINCLUDE=$WATCOM/src/fortran
export INCLUDE=$WATCOM/lh
PATH=$WATCOM/binl:$PATH

You might also find the owcc utility interesting. This program is a front
end that is intended to provide a GNU-like interface to the Open Watcom
compilers. The idea is to make the Open Watcom compilers sufficiently
compatible with gcc so that one can easily switch to Open Watcom in gcc
enabled makefiles. Open Watcom provides a Microsoft nmake emulation for
similar reasons.


Performance
-----------

In the mid-1990s the Watcom compilers had a reputation of producing
excellent code for x86 processors. Is that reputation still warranted? In
this short article it isn't possible to do a comprehensive performance
evaluation of Open Watcom versus gcc, but a quick test can be suggestive.

The program in Listing 5 [This program is rather long. Would it be better
to just include it on the web site?] sorts one million element arrays of
integers and C++ std::strings. It uses both the C library qsort function
for the integers and the C++ library std::sort algorithm template for both
the integers and the strings. Open Watcom's -ot (optimize for time) option
was selected during compilation. The results on my test Linux system
(OpenSUSE 10.2 on a 2.0 GHz processor with 512 MB RAM) are shown in Table
1. Times are in seconds of CPU time for 20 runs.

Table 1. Open Watcom 1.8beta on Linux (times in seconds)

                | qsort  |  std::sort
================|========|===========
random int      | 5.50   |   2.23
sorted int      | 3.13   |   0.55
random strings  | n/a    |  44.61

In comparison the same program was compiled using g++ v4.1.2 using the -O
option for optimization. When run on the same test system, the times are as
shown in Table 2.

Table 2. g++ v4.1.2 on Linux (times in seconds)

                | qsort  |  std::sort
================|========|===========
random int      | 7.71   |   2.73
sorted int      | 3.14   |   0.54
random strings  | n/a    |  54.64

Of course a test of this nature compares not only the code generated by the
compiler, but also design decisions made in the library. However, to a
practicing programmer it is often the combination of library and compiler
performance that matters the most anyway. For example, Open Watcom's
implementation of std::string is not a copy-on-write implementation as is
used by g++. The simpler design gives benefits with short strings due to
lower overhead but shows poorer performance in some programs when strings
are very long. The test above was conducted with 50 character strings.


The Future
----------

We believe the future of Open Watcom on Linux is bright. Although
not yet usable for serious programming in the Linux environment, Open
Watcom already has many of the necessary pieces ready to support such
programming. With a bit more tender loving care, Open Watcom could become
a viable alternative to gcc. When this happens, Linux developers will have
more choices. That is, after all, what open source software is all about.

For more information about the current state of Open Watcom on Linux we
invite you to review the wiki and participate in the newsgroups. We look
forward to talking with you!

=======================
EDITOR: Have any large applications been compiled on Linux with Open Watcom?
=======================
[The only large application compiled on Linux with Open Watcom that I know
of at this time is the Open Watcom source itself.]

=======================
EDITOR: Do you have any performance comparisons available?
- Compile speed compared to GCC.
- Execution speed compared to a gcc compiled program.
=======================
[Included some benchmark results above. Compilation speed of OW is good as
well, but I am worried that the article may be getting too long to discuss
that.]

Resources
---------

Open Watcom wiki web site: http://www.openwatcom.org.
Open Watcom newsgroups: news.openwatcom.org.
Perforce web site: http://www.perforce.com.
FreeDOS: http://www.freedos.org.
eComStation: http://www.ecomstation.com.

