**********
Dune Cache
**********

.. TODO(diataxis) This is reference material with some explanation.

Dune implements a cache of build results that is shared across different
workspaces. Before executing a build rule, Dune looks it up in the shared
cache, and if it finds a matching entry, Dune skips the rule's execution and
restores the results in the current build directory. This can greatly speed up
builds when different workspaces share code, as well as when switching branches
or simply undoing some changes within the same workspace.


Configuration
=============

There are three ways to configure the Dune cache.  Choose the one that is more
convenient for you:

* Add ``(cache <setting>)`` to your Dune configuration file
  (``~/.config/dune/config`` by default).
* Set the environment variable ``DUNE_CACHE`` to ``<setting>``
* Run Dune with the ``--cache=<setting>`` flag.

Here, ``<setting>`` must be one of:

* ``disabled``: disables the Dune cache completely.

* ``enabled-except-user-rules``: enables the Dune cache, but excludes
  user-written rules. This setting is a conservative choice that can avoid
  breaking rules whose dependencies are not correctly specified. Currently the
  default.

* ``enabled``: enables the Dune cache unconditionally.

By default, Dune stores the cache in your ``XDG_CACHE_HOME`` directory on \*nix
systems and ``%LOCALAPPDATA%\Microsoft\Windows\Temporary Internet Files\dune`` on Windows.
You can change the default location by setting the environment variable
``DUNE_CACHE_ROOT``.


Cache Storage Mode
==================

Dune supports two modes of storing and restoring cache entries: `hardlink` and
`copy`. If your file system supports hard links, we recommend that you use the
`hardlink` mode, which is generally more efficient and reliable.

The `hardlink` Mode
-------------------

By default, Dune uses hard links when storing and restoring cache entries. This
is fast and has zero disk space overhead for files that still live in a build
directory. There are two disadvantages of this mode:

* The cache storage must be on the same partition as the build tree.
* A cache entry can be corrupted by modifying the hard link that points to it
  from the build directory. To reduce the risk of cache corruption, Dune
  systematically removes write permissions from all build results. It is worth
  noting that modifying files in the build directory is a bad practice anyway.

The `copy` Mode
---------------

If you specify ``(cache-storage-mode copy)`` in the configuration file, Dune
will copy files to and from the cache instead of using hard links. This mode is
slower and has higher disk space usage. On the positive side, it is more
portable and doesn't have the disadvantages of the `hardlink` mode (see above).

You can also set or override the storage mode via the environment variable
``DUNE_CACHE_STORAGE_MODE`` and the command line flag ``--cache-storage-mode``.

Trimming the Cache
==================

Storing all historically produced build results in the cache is infeasible, so
you'll need to occasionally trim the cache. To do that, run the ``dune cache
trim --size=BYTES`` command. This will remove the oldest used cache entries to
keep the cache overhead below the specified size. By "overhead" we mean the
cache entries whose hard link count is equal to 1, i.e., which aren't used in
any build directory. Trimming cache entries whose hard link count is greater
than 1 would not free any disk space.

Note that previous versions of Dune, cache provided a "cache daemon" that could
periodically trim the cache. The current version doesn't require an additional
daemon process, so this automated trimming functionality is no longer provided.


Reproducibility
===============

Reproducibility Check
---------------------

While the main purpose of Dune cache is to speed up build times, it can also be
used to check build reproducibility. By specifying ``(cache-check-probability
FLOAT)`` in the configuration file, or running Dune with the
``--cache-check-probability=FLOAT`` flag, you instruct Dune to re-execute
randomly chosen build rules and compare their results with those stored in the
cache. If the results differ, the rule is not reproducible, and Dune will print
out a corresponding warning.

Non-Reproducible Rules
----------------------

Some build rules are inherently not reproducible because they involve running
non-deterministic commands that, for example, depend on the current time or
download files from the Internet. To prevent Dune from caching such rules, mark
them as non-reproducible by using ``(deps (universe))``. Please see
:doc:`concepts/dependency-spec`.
