BENV Manual
Table of Contents

Introduction

This document describes the design and usage of the BENV software build environent.

BENV is composed of two parts:

The repository structure describes the organization of the directories in which the build system executes to build the software that is maintained there.

Repository Structure

The repository structure is illustrated below. The BENV document Repository Evolution describes the why of the BENV repository structure.

Repository Layers

Included in this illustration are the directories that contain the components that comprise BENV (a.k.a. benvc.)

Using BENV, The Short Answer

While the intent of this document is to provide a manual describing many design details of BENV, it is possible to become overwhelmed by the details and loose sight of the simplicity that BENV brings to the software development process.

It is the purpose of this section is to give the user a feel for the way in which BENV is used in practice. Remember that BENV was designed to simplify the practice of software development!

Creating a New Project

There are two fundamental ways to create of new BENV project: Building a project from scratch is by far the more tedious method, so I'll present that last. This is usually only necessary when the project involves the use of a new toolchain may involve creating new BENV toolsets. Most new BENV projects are started by copying and modifying an existing BENV project.

Cloning an Existing Project

Cloning an existing project is easily the fastest way to create a new project. The basic sequence is this:
  1. Choose an existing project that is the closest to the new project.
  2. Copy the existing project directory to a new location, giving it a new name.
  3. Modify the BENV project specification files for the new project.

The criteria for choosing an existing project is best done based on the following factors:

Of these, the most important is the similarity between the tools.b files. An identical tools.b file means that the following already exist:

If any one of these lines in the tools.b files does not match, then you will need to go to the corresponding toolset directory and either find one that already meets your needs, or create one, and then modify the new tools.b file accordingly.

If the location of your new project directory is deeper or shallower than the cloned directory, you will need to modify the relative paths in the project Makefile, adding or deleting double dots ("..") as necessary.

The most volatile BENV project specification file is libdirs.b. One approach is to clone it from another project that is most similar application-wise to the new project (perhaps the same project already cloned.) Edit the file, deleting, adding and editing entries while iteratively building the project.

The most difficult thing to do accurately during this process is to remove unncessary entries in the libdirs.b file. Even when the final application links, there is no indication that particular enries/libraries have gone unused. Having unused entries does no harm other than increase the build time.

During each iteration of the build libdirs.b Remove all shared source libraries that will not build. Source libraries that will not build is usually the result of them not applicable to your target.

However, watch the errors during this iterative process. If header files cannot be found, that is usually an indication that either the shared source code has not been put into the BENV repository tree, or there is a need to add an appropriate include path to the LIB_INC_ROOTS variable in your overrides.b file. This kind of problem usually only occurs when one of the project's tools.b entries has changed from the cloned original due to a change in toolchain or targetos. However, include files can also come up missing if any part of the target platform is different.

Creating a Project From Scratch

Building a Project

The example below shows how to build a project called myProject that lives in my BENV vendor namespace called myNamespace in a repository named myRoot.
$ cd myRoot/project/myNamespace/myProject
$ make

It doesn't get much easier than that ... right?

If you have many projects that you wish to build on a regular basis, then using the buildAll utility is really handy. I find this technique quite useful when I need to ensure that a change to shared source code or the environment has not broken any of the projects which depend upon the changed file.

Cleaning a Project

The example below shows how to clean a project called myProject that lives in my BENV vendor namespace called myNamespace in a repository named myRoot.
$ cd myRoot/project/myNamespace/myProject
$ make cleaning=y

The make cleaning=y thing is a hack that I should take the time to fix, but I lack the motivation.

Note that while this does delete all derived objects, it does not remove the derived object directory tree structure, created during the build process when the project uses any shared source code as listed in libdirs.b or libfiles.b.

Thus, if you want things really clean, then use this method.

$ cd myRoot/project/myNamespace/myProject
$ make cleaning=y
$ rm -rf src
Warning the rm -rf command is a dangerous weapon in the wrong hands (directory.)

Notice that this technique assumes that all of your shared source code is located under the BENV repository src directory. If you put it anywhere else (why would you do that?) you need to adjust this sequence accordingly.

If you have many projects that you wish to clean on a regular basis, then using the cleanAll utility is really handy. I find this technique quite useful when I need to ensure that a change to shared source code or the environment has not broken any of the projects which depend upon the changed file.

Build System

During this presentation of the BENV build system, we will use a sample project to describe how the various components interact. The organization of the sample project within the BENV repository directory structure is illustrated below.

BENV Example

The project/mnm/sample is a project directory that contains all of the project specific files. Among the files are source files, a special header file "buildCfg.h" and a set of BENV build specification files. By convention, all BENV build specification files end with a ".b" suffix.

BENV Build Specification Files

This section describes the set of BENV files that are contained within each project directory. These files are used to specify how the project is built. Each of these files has a particular purpose within the BENV build process.

Makefile

The project Makefile is the first file that is involved with the BENV build process. It is a standard makefile, and it is evaluated whenever the make command is invoked from within the project directory.

The purpose of the Makefile is to define some of the fundamental variables of the project, and then invoke the remainder of the build process.

The following is a typical Makefile

############################################################
# Project root makefile
#

ifeq ($(toolroot),)
        toolroot=../../../../tools/benvc
endif

srcroot=../../../..

ifeq ($(HOST),)
HOST:=$(shell uname)
endif

ifeq ($(HOSTMACH),)
HOSTMACH:=$(shell uname -p)
endif

ifneq ($(HOST),)
        tooldir=$(toolroot)/toolset
        include $(tooldir)/host/$(HOST)
        include $(toolroot)/maketools/master
endif

The most important and variable aspect of this file are the definition of the srcroot and toolroot variables. All project Makefiles at the same depth in the project tree of the BENV repository structure have the same definition of these two variables.

Other BENV variables defined by this file include:

Near the end of the Makefile two BENV makefile fragments are included, which invoke the build process. The first includes build HOST specific set of variable definitions. The last includes the master makefile fragment that preforms the rest of the build.

tools.b

The tools.b file specifies four the most fundamental parts of the BENV process. This is done by defining the value of four make variables.

The following is an example of the contents of a tools.b file.

hosttools=Linux
toolchain=h8300-elf.Linux
targetos=h8300h-elf-bare
linkfile=h8300-elf-bare
Note that this file is a makefile fragment that overrides the four variables.

hosttools

The hosttools variable specifies the type of host upon which this project is executed. This is done by giving the name of one of the files contained within the benvc/toolset/host directory. Each file in this directory describes a set of make variables that perform essential functions for the platform. For example, benvc/toolset/host/Linux defines the functions for a Linux build host.

toolchain

The toolchain variable specifies the compiler toolchain used to build the project. This is done by defining the value of the variable to one of the files contained within the benvc/toolset/toolchain directory. Each file in this directory defines a set of make variables that describe the essential compiler, assembler, linker and archiver tools as well as their default options.

targetos

The targetos variable specifies the target operating environment. This is done by defining the value of the targetos variable to the name of one of the files contained within the benvc/toolset/targetos directory. Each file in this directory defines a set of make variables that describe the include paths and libraries to use for a particular OS and toolcahin combination.

linkfile

The linkfile variable specifies the final link stage of the project. The final link stage typically executes the toolchain linker to combine the object files and library files into a single executable. It is also possible to specify a linkfile that produces multiple output files (such as map, ASCII hex, s-record, and others ) or a library.

libdirs.b

From a source code reuse perspective, the libdirs.b file is one of the most important and commonly manipulated specification files of the BENV build system. It contains a simple list of shared source code directories that are to be compiled for use by this project.

The following is an example of the contents of a libdirs.b file.

#
# This file describes the libraries
# required for an application
# executing under the KRUX RTOS 
# on a Hitachi/Renasas 3664 device.
#
src/oscl/platform/krux/oscl/mt/itc/mbox/platform
src/oscl/driver/hitachi/h8/libfake
src/oscl/stream/hdlc/fcs
src/oscl/error/krux
src/oscl/mt/itc/mbox
src/oscl/mt/platform/tocks/31.25ms
src/oscl/driver/hitachi/h8/platform/16MHz
src/oscl/driver/hitachi/edk3664/krux
src/oscl/krux/kernel/tiny
src/oscl/krux/thread
src/oscl/krux/mutex
src/oscl/krux/plist
src/oscl/krux/sema
src/oscl/krux/spri
src/oscl/krux/timer
src/oscl/krux/platform/h8

Note that each line of the file contains a path name that is relative to the BENV repository root directory.

Comments can be present in the file with the '#' symbol preceding each line of comment.

By default, BENV compiles all source files in each of the listed directories. Use the libfiles.b file to specify individual files rather than the entire directory.

After the files in each directory have been compiled, a static library with the name library.a is created that contains all of the compiled components in the directory. Each of these libraries is added to a list of libraries that are available to the linkfile stage of the build through the linkargs makefile variable.

All derived objects produced by this process are placed in a subdirectory of the project directory with a path name that is the same as each line of the file.

libfiles.b

Similar in some respects to the libdirs.b the libfiles.b specifies source files that are to be compiled and linked with to the final project output. However, rather than specifying entire directories to be compiled, this file contains a list of individual files to be compiled.

The following is an example of the contents of a libfiles.b file.

#
# This file describes some
# specific files that need to be built
# which live in a directory that cannot
# be built as a whole. A poorly designed
# (hopefully legacy) source code directory.
#
src/screwed/module/file.c

Note that each line of the file contains a path name that is relative to the BENV repository root directory.

Comments can be present in the file with the '#' symbol preceding each line of comment.

After the files have been compiled, a static library with the name library.a in created and placed in each directory that contains all of the compiled components of that directory. Each of these libraries is added to a list of libraries that are available to the linkfile stage of the build through the linkargs makefile variable.

All derived objects produced during this process are placed in a subdirectory of the project directory with a path name that is the same as the directory part of each line of the file.

sources.b

The sources.b file contains a list of project specific source files, contained within the project directory, that are to be compiled and linked to the final application.

The following is a typical example of the contents of a project sources.b file.

# This is a comment.
main.cpp

overrides.b

The overrides.b file is a makefile fragment that defines many of the project specific variable aspects of the BENVbuild process. Its name comes from the fact that the project can override the default values of many variables related to the build process.

The following is a typical example of the contents of a project overrides.b file.

FINAL_OUTPUT_FILENAME=main

benvFirstObjects = \


benvExtraObjects = \


LIB_INC_ROOTS=  -I$(srcroot)/src \
				-I$(srcroot)/src/oscl/platform/compiler/GCC/$(shell uname -p) \
				-I$(srcroot)/src/oscl/platform/posix \
				-I$(srcroot)/

EXTRA_CLEANABLE_OBJECTS_LIST=first

CFLAGS+=-Werror -O2

See also:

buildCfg.h

The buildCfg.h header file is automatically included during the build of every C/C++ source file. The header is also included for assembly language files that end with a '.S' suffix.

Defining global preprocessor macros in this file allows the BENV dependency managment to recognize that all files need to be re-built when a global preprocessor macro is changed. For this reason, using the buildCfg.h file is preferable to using the "-Dsymbol" style construct to pass in compile time macros.

Shared BENV Files

The BENV files that are shared by projects are contained within the tools/benvc directory of the repository structure. This section documents the purpose of each of these directories and files.

benvc/maketools

The maketools directory contains the makefile fragments that implement the core logic of the BENV build system.

benvc/maketools/master

The master file is a makefile fragment that controls the top level of the BENV build process.

This file is included by project Makefiles.

benvc/maketools/submake

The submake file is a makefile fragment that controls the building of subdirectories during the BENV build process.

This file is included by the master makefile fragment.

benvc/maketools/common

The common file is a makefile fragment that contains components shared by both the master and the submake makefile fragments.

This file is included by both the master and the submake makefile fragments.

benvc/maketools/rules

The rules file is a makefile fragment that contains the rules for building the various makefile targets supported by the BENV build system. This includes rules for converting C and C++ source files to object files and similar transformations.

This file is included by both the master and the submake makefile fragments.

benvc/toolset

This directory contains shared makefile fragments that describe the rules for building various projects. Each project specifies the set of rules it uses in its tools.b file.

This directory contains the following four directories, which further catagorize the types of fragment:

benvc/toolset/toolchain

This directory contains reusable makefile fragments that define the toolchain (e.g. compiler, assembler, archiver, and linker) that is used during the execution of the BENV build system for a particular project type.

The file chosen from this directory is specified as the value of the toolchain variable specified in a projects tools.b file.

benvc/toolset/targetos

This directory contains reusable makefile fragments that define the aspects of the target operating system that is used during the execution of the BENV build system for a particular project type.

The file chosen from this directory is specified as the value of the targetos variable specified in a projects tools.b file.

benvc/toolset/linkfile

This directory contains reusable makefile fragments that specify how the final linker stage of the project build is performed.

The makefile variables defined during the BENV build process are available to these makefile fragments. The most important are the following:

The file chosen from this directory is specified as the value of the linkfile variable specified in a projects tools.b file.

benvc/toolset/host

This directory contains reusable makefile fragments that specify utilities used by the BENV build system when executing on a particular build platform.

The file chosen from this directory is specified as the value of the host variable specified in a projects tools.b file.

benvc/bin

This directory contains scripts that may be useful to the BENV user.

This directory contains the following scripts:

BENV Makefile Variables

This section contains a description of some of the more important and commonly used BENV makefile variables.

FINAL_OUTPUT_FILENAME

FINAL_OUTPUT_FILENAME is defined in the project overrides.b file, and is the name of the final file to be produced by the project.

buildLibraries

The buildLibraries variable is defined as a list of all of the library.a files built for the project as specified in the libdirs.b and libfiles.b files.

linklibs

Defined in the master file, this variable is a list of all of the libraries to be linked into the final application. This is how it is defined:
linklibs=$(buildLibraries) $(OSLIBS) $(TOOLCHAINLIBS)

linkargs

Defined in the master file, this variable is a list of all of the libraries defined by linklibs enclosed in the GNU linker construct --start-group and --end-group. This construct eliminates ordering issues between libraries. Of course, this assumes that there are no duplicate symbols within the libraries.

This is how it is defined:

linkargs=$(linkobjs) --start-group $(linklibs) --end-group

srcroot

Defined in the project Makefile, the srcroot variable specfies the relative path to the root of the BENV repository directory from the project directory. Yes, the name is misleading.

toolroot

Defined in the project Makefile, the toolroot variable specfies the relative path to the tools/benvc directory from the project directory.

tooldir

Defined in the project Makefile, the toolroot variable specfies the relative path to the tools/benvc/toolset directory from the project directory.

HOST

Defined in the project Makefile, the HOST variable defines the type of operating system on which BENV is executing. This is typically determined by the output of the uname -s. This variable can be used by parts of the BENV build system to locate particular executables.

HOSTMACH

Defined in the project Makefile, the HOSTMACH variable defines the type of machine (micro-processor type) on which BENV is executing. This is typically determined by the output of the uname -p.

benvFirstObjects

The benvFirstObjects variable is a list of object files to be linked in-order to the beginning of the final application, and is defined as required by a project in its overrides.b file.

This is often used when building embedded applications to ensure that certain files appear first in the final binary load file.

For example:

benvFirstObjects = \
	$(srcroot)/tools/benvc/$(HOST)/h8300-elf/lib/gcc-lib/h8300-elf/3.3/h8300h/normal/crti.o

benvExtraObjects

The benvExtraObjects variable is a list of object files to be unconditionally linked into the final application, and is defined as required by a project in its overrides.b file.

Usually, BENV places all object files into a library and the object files are only included in the application if they are referenced by another object file. There are times, however, when certain files are not referenced by other object files, and need to be linked into the final project regardless. This is one function provided by this variable.

Another use for this variable is to force inclusion of an object file that is not built from source as a part of the usual BENV build process.

Example usage:

benvExtraObjects = \
    clkint.o \
    m2681int.o

LIB_INC_ROOTS

The LIB_INC_ROOTS variable is a list of include file root directories in a compiler/preprocessor specific format, and is defined as required by a project in its overrides.b file.

Example usage:

LIB_INC_ROOTS= -I$(srcroot)/src \
               -I$(srcroot)/src/oscl/platform/compiler/GCC/$(shell uname -p) \
               -I$(srcroot)/src/oscl/platform/posix \
               -I$(srcroot)/

EXTRA_CLEANABLE_OBJECTS_LIST

The EXTRA_CLEANABLE_OBJECTS_LIST variable is a list of files to be deleted when the project is "cleaned".

Normally, most files generated during the build process are deleted when make cleaning=yes is issued in the project directory. However some files that are outside of the "knowledge" of BENV (e.g. files created during the final link stage) must be explicitly specified using this variable.

Example usage:

EXTRA_CLEANABLE_OBJECTS_LIST=$(FINAL_OUTPUT_FILENAME)

CFLAGS

The CFLAGS variable specifies flags passed to the preprocessor, C , and C++ compiler during the BENV build process.

The value of CFLAGS is usually defaulted in the tools/benvc/toolset/toolchain file chosen by the project in its tools.b file.

The value of CFLAGS may be modified by each project by changing or appending to its value in the overrides.b file.

Example usage:

CFLAGS+=-Wall -O2 -g

CXXFLAGS

The CXXFLAGS variable specifies flags passed to the C++ compiler during the BENV build process.

The value of CXXFLAGS is usually defaulted in the tools/benvc/toolset/toolchain file chosen by the project in its tools.b file.

The value of CXXFLAGS may be modified by each project by changing or appending to its value in the overrides.b file.

Example usage:

CXXFLAGS+=-fno-rtti

Scripts

benvc/bin/rmd

Recursively deletes derived objects from a project tree.

benvc/bin/buildAll

Builds a list of projects. Accepts an input file which consists of a list of project directories to be built, and subsequently builds each project.

The utility is a simple bashscript that looks like this:

#!/bin/bash

#
# This scripts builds all of the projects
# listed in the input file.
#

projects=`cat $1`

for project in $projects
do
	pushd $project
		nice make > make.temp 2>&1
	popd
done

As an example of its use, assume that I have the following three projects:

I create a file name myProjects and I want to put it in the directory myRoot/project. The content of the file will look like this:

myNamespace/myProject1
myNamespace/myProject2
myNamespace/myProject3

To build all of three projects, I use the following sequence:

$ cd myRoot/project
$ buildAll myProjects

After the buildAll script has completed, there is a file named make.temp in each of the project directories that contains the result of each build.

To examine the results, I do something that looks like this:

$ find . -type f -name make.temp -exec vi {} \;

After I am finished, I use the cleanAll utility to clean it all up.

benvc/bin/cleanAll

Cleans a set of projects previously built with the buildAll utility. It accepts an input file which consists of a list of project directories to be cleaned, and subsequently cleans each project.

The utility is a simple bashscript that looks like this:

#!/bin/bash

#
# This script cleans the projects listed
# in the input file.
#

projects=`cat $1`

for project in $projects
do
	pushd $project
		nice make cleaning=yes
		rm -rf src
		rm -rf cf
		rm -f make.temp
	popd
done

As an example, assume that I have the following three projects that I wish to clean:

Before using the buildAll utility, I created a file with the name myProjects and placed it in the directory myRoot/project. The content of the file looks like this:

myNamespace/myProject1
myNamespace/myProject2
myNamespace/myProject3

To clean this same set of projects, I use the following sequence:

$ cd myRoot/project
$ cleanAll myProjects

After the cleanAll script has completed all of the derived objects in the specified projects are deleted, along witht the derived object directory structure and the make.temp file.

Forking History

BENV has evolved since circa 1995. During that time, various colleagues have aquired a taste, and modified it for their own uses.

As far as I know, there is only Shift-Right Technologies, LLC distributes a heavily modified/forked version of BENV. To their credit, their version has focused on supporting non-*NIX development platforms, whereas my focus has been focusing on the Linux development platform almost exclusively.

Because of Shift-Right Technologies focus on smaller processors (8 and 16 bit), supporting many small proprietary compilers for these beasts, their version of BENV has evolved to accommodate more of these variants.

As a result of this fork, the official name for my version of BENV is BENV Classic (yeah, I know ... very original.) This is why the directory containing the shared BENV files under the tools directory is named benvc.

From the beginning, one of my goals has been to eliminate the use of environment variables, such that a developer can simply change working directory to the project directory, type make and the right thing happens. The original BENV has no environment variable requirements other than those imposed toolchains. Enjoy.

Contents