.. _sardana-explanation-environment-backends:

============================
Environment storage backends
============================

The :ref:`macro environment <macro-environment>` must survive Macro Server
restarts: variables like the active measurement group or the scan file
configuration are expected to still be there after a server upgrade or a
crash. Where and how this persistence happens is a deployment decision, so
since Sardana 3.7.0 the environment storage is designed as a set of
interchangeable *backends* behind a single manager.

A modular manager
=================

The `~sardana.macroserver.msenvmanager.EnvironmentManager` implements all the
environment logic that is independent of storage: parsing of values, the
scoping rules (a lookup cascades from the door+macro scope, through the macro
and door scopes, down to the global scope) and the notification of clients.
The actual persistence is delegated to a backend object with a minimal
get/set/unset/list interface. Three backends are provided:

* ``ShelveEnvironmentBackend`` — the default, file based storage,
* ``RedisEnvironmentBackend`` — storage in a Redis_ server,
* ``TangoDBEnvironmentBackend`` — storage in the Tango_ database.

This refactoring kept the public :term:`API` unchanged: macros and clients
use the same environment commands regardless of the configured backend.

Backend selection
=================

The backend is selected through the ``EnvironmentDb`` Tango_ device property
of the MacroServer device, whose value is interpreted as a URI or path:

* ``property`` — Tango database backend, storing the environment as
  properties of the MacroServer device itself; ``property:<name>`` stores
  them under an arbitrary object name instead,
* ``redis://...`` or ``rediss://...`` — Redis backend, with keys prefixed by
  the MacroServer name,
* anything else is treated as a filesystem path for the shelve backend
  (default:
  ``/tmp/tango-<user>/<ds_exec_name>/<ds_inst_name>/macroserver.properties``).

If the configured backend cannot be initialized at server startup, the
MacroServer falls back to a freshly created temporary shelve file (and
rewrites its ``EnvironmentDb`` property accordingly) — except for the Redis
backend, where a connection failure aborts the server startup instead of
silently switching storage.

Trade-offs
==========

Shelve (default)
----------------

The shelve backend uses the Python :mod:`shelve` module on top of a
:mod:`dbm` database (GNU dbm when available, with a fallback to the pure
Python *dumb* dbm; the choice can be forced with the
``MS_ENV_SHELVE_BACKEND`` sardana custom setting). It needs no external
service, which makes it the zero-configuration default. The flip side is
that the environment becomes a set of files on the host running the
MacroServer: it is not shared, the exact on-disk format depends on which dbm
implementation happened to be available, and the storage relies on
file-system locking semantics, which makes hosting it on network file
systems (e.g. NFS-mounted directories) fragile. Historically the
dumb dbm fallback was also a source of corrupted/empty environment files on
unclean shutdowns.


Redis
-----

The Redis backend stores each variable as a separate key (prefixed with the
MacroServer name) in an external Redis_ server. This decouples the
environment from the MacroServer host: it can be inspected and backed up
with standard Redis tooling and survives machine reinstallation. The cost is
an external service to deploy and operate, an extra Python dependency
(``redis-py``), and a hard runtime dependency: the connection is verified at
startup and the MacroServer refuses to start if the Redis server is not
reachable.

Tango database
--------------

The TangoDB backend stores the environment as free object properties in the
Tango_ database, encoding the door/macro scope in the property name. Its
appeal is that a Tango database is by definition already present in any
Sardana installation, so, like shelve, it requires no additional service —
while still being centralized and shared. On the other hand every
environment access becomes a Tango database operation and the values must be
encoded into database-friendly strings.


.. _Tango: http://www.tango-controls.org/
.. _Redis: https://redis.io/