Architecture & development information

System structure diagram

Click to view at full resolution.

The system structure diagram describes how the various modules of cmail interact with eachother as well as remote systems.

Starting to contribute

Looking for a place to start? Here are some pointers:

• If you're interested in contributing to the core cmail code, the development guide in the repository may be interesting to you, as it covers issues such as code style & conventions, licensing and tools used for development.
• There are some issues marked help wanted and good first issue on the issue tracking system
• Find the documentation somewhat lacking? Consider extending and updating the homepage
• If you're interested in learning about the protocols involved in mail processing, consider writing protocol level tests against the cmail implementations - edge cases are particularly welcome!
• Grep your clone of the source repository for occurences of FIXME or TODO and see if there's something simple to do
• Created a great connector or helper program? Consider adding it to the contrib repository
• Opening well researched issues on Github (e.g. against the homepage or main repositories) is also a great way to help

Development tooling

cmail uses a variety of tools to help maintain code quality, automate tasks and aid development.

At the most basic level, this means cmail is regularly built and checked for compile-time warnings using a variety of compilers, and operating systems, including

• gcc
• clang
• tcc

Code quality is regularly being analyzed by

• Running development test builds within valgrind to check for access violations and memory leaks at run-time (invoked via make run)
• Running cppcheck to gather a quick list of warnings from the source (via make cppcheck)
• Using clangs scan-build static analyzer to generate warnings from the source (invoked via scan-build make)
• Scanning builds with the free offer of the Coverity static analyzer solution (Coverity Scan)
• Stress-testing selected interfaces with fuzzers (eg. afl-fuzz, further information about how to invoke fuzzing are in the works)

Testing & Test Coverage

cmail features a growing suite of automated tests, written mostly in Tcl using the Expect toolkit. In order to run them, you'll need a complete installation of cmail which is NOT in production use, as the test suite will heavily modify and overwrite any existing configuration in order to exercise different parts of the code. Running the suite on a dedicated VM is probably the best way to proceed.

Coverage information can be extracted by running the test suite against a build with the gcov coverage analyzer enabled and thereafter analyzing the output with lcov. As these steps need to assume a lot about the environment they are run in, there is as of yet no completely automated way of running the test suite.

Coverage reports

You can view generated coverage reports, which will be updated at irregular intervals.

• cmail-smtpd (commit 013e2e9c78e956ff6acc3e3aee6c189e4a839c3f)
• cmail-popd (commit 013e2e9c78e956ff6acc3e3aee6c189e4a839c3f)

Generating Callgraphs

Call graphs are a visual representation of the architecture and are tremendously helpful in understanding code and control flow in projects. This makes them highly useful to newcomers for gaining a high-level understanding of how a project works, as well as to seasoned developers wanting to visualize how best to fit a particular feature into the exisiting architecture.

The cmail main makefile contains a special target named rtldumps which generates GCC rtldump files into the rtldumps/ folder. These can be used to easily create call graphs for all the cmail daemons with the help of rtl2dot and GraphViz.

For example, to create a graph of the call hierarchy of cmail-smtpd, perform the following steps:

1. In your cmail repository clone, run make rtldumps. This will create a folder named rtldumps, containing the raw data for the graph.
2. In the rtldumps folder, locate the file containing the smtpd data. It will usually be named something like smtpd.c.170r.expand.
3. Create the graph data for feeding into GraphViz with rtl2dot.py smtpd.c.170r.expand > smtpd.dot. This will create smtpd.dot.
4. Create the final image with GraphViz by running dot -Tsvg smtpd.dot > smtpd.svg. This will create an SVG image of the call graph. Note that dot is also able to create raster images in formats like PNG and JPG by supplying the appropriate -T<type> parameters.

The latter steps can also be automated by pipelining the commands like so: rtl2dot.py smtpd.c.170r.expand | dot -Tsvg > smtpd.svg

The previous steps generate a full callgraph, containing all functions called via any one possible path from main. This includes common functions such as logprintf, which are called from almost every function, eg. to print messages to the log files, as well as library functions which are not directly part of the project.

In order to create a clearer visualisation of interesting code paths, it is necessary to exclude some functions from the graph. rtl2dot.py supports exclusion of functions by their name with the --ignore argument, as well as exclusion of library calls with the --local argument. A beautified callgraph can thus be created by running rtl2dot.py smtpd.c.170r.expand --root core_loop --ignore "client_send|logprintf|common_*" --local | dot -Tsvg > smtpd.svg This also sets the root function of the graph to core_loop, which is where the main processing functions are called in the cmail daemons.