Configuration: settings and options

fresco.core.FrescoApp.options is a dict-like Options object that can be used for your application’s configuration.

Basic usage

In the simplest case, fresco.core.FrescoApp.options can be used like a regular dict:

from fresco import FrescoApp
app = FrescoApp()

app.options["SMTP_HOST"] = "smtp://localhost"
app.options.update({"DATABASE": "sqlite://"})

Settings are made available globally via context.app.options:

from fresco import context

db = connect(context.app.options.DATABASE)

Using configuration files

Use load() to load options from configuration files:

app = FrescoApp()

# Load options from all matching files
app.options.load("conf.d/*")

# A fully loaded example
app.options.load(
    ["myapp.toml", "conf.d/*"],

    # Search for files in this directory
    dir="/path/to/config",

    # Also load any of "myapp.{hostname}.*",
    # "myapp.{FRESCO_PROFILE}.*", or "myapp.local.*", if found.
    tags=["{hostname}", "{FRESCO_PROFILE}", "local"],

    # Allow configuration files to add keys not present in the first file
    strict=False,

    # Use environment variables in preference to values loaded from files
    use_environ=True
)

The dir argument specifies the directory to be searched for configuration files (defaults to the current directory).

The default behaviour is that the first loaded file must specify all available configuration keys. A subsequent file that introduces a new key will raise an exception. Specify strict=False to turn off this check.

If use_environ=True is specified, any environment variable with the same name as a loaded option key will be used in preference to the value loaded from the file. Environment variables are subject to the same parsing and interpolation rules as described in key-value properties files’

Selecting configuration files based on environment

load() can load files selectively based on ‘tags’ embedded in the file names (tags are embedded in the filename, separated by dots, eg settings.tag1.tag2.conf).

For example with these configuration files:

settings.conf                 # 1
settings.dev.conf             # 2
settings.staging.conf         # 3
settings.staging.local.conf   # 4
settings.local.conf           # 5

This is how to load subsets of the available files:

# Load all files
options.load("settings*")

# Load base config file plus those
# tagged 'dev' or 'local' (files #1, #2 and #5)
options.load("settings*", ["dev", "local"])

In addition to tags, filenames can contain priority numbers to allow a specific loading order to be achieved. Priority numbers go after the first dot, and may be separated from a tag with a hyphen. For example:

settings.100.conf
settings.200-local.conf

The string {hostname} in a tag name will be interpolated with the current hostname (as returned by socket.gethostname), but with dots replaced by underscores (eg myserver_example_org). A common pattern is:

options.load(
    "settings*",
    [
        os.environ.get("FRESCO_PROFILE", "prod"),
        "host-{hostname}",
        "local"
    ]
)

This will load:

  • a base config file (eg settings.conf);

  • then any config files tagged with the profile specified in the FRESCO_PROFILE environment variable (eg settings.dev.conf or settings.prod.conf)

  • then any host-specific config files (eg settings.host-myserver_example_org.conf);

  • finally, any local config files (eg settings.local.conf).

Configuration file loading order

Configuration files are sorted according to three criteria, from most to least important:

  • By priority number, if present, from low to high.

  • Then by any tags present in the filename, according to the order of tags provided to load(). Filenames containing tags specified later in this list are loaded later than those specified at the start

  • Finally, in lexicographical order

To ensure that a configuration file is loaded last, and thus can override settings specified in earlier files, assign it a high priority number.

Processing configuration data

Fresco exposes functions to help load structured configuration data from strings. These may be useful when storing your configuration data as simple key-value pairs, or specifying options from environment variables.

list_from_str() splits a string into a list on a separator, stripping out intervening whitespace, for example:

from fresco.options import Options, list_from_str
options = Options(
    ADMIN_EMAIL="me@mydomain.com, you@yourdomain.com",
)

# Convert to `['me@mydomain.com', 'you@yourdomain.com']`
options["ADMIN_EMAIL"] = list_from_str(options.ADMIN_EMAIL)

dict_from_options() populates a dictionary from options named with the same prefix.

from fresco.options import Options, dict_from_options
options = Options(
    DATABASE_HOST="localhost",
    DATABASE_USER="scott",
    DATABASE_PASSWORD="tiger",
)

# Convert to {"HOST": "localhost", "USER": ...}
options["DATABASE"] = dict_from_options("DATABASE_", options)

When post-processing options in this way, it’s a good idea to set trigger_onload=False when calling load() and then call trigger_onload() once processing is complete.

Configuration file formats

The following formats are supported by load():

key-value properties files

These are simple newline separated lists of key-value pairs. The format is designed to be easy to parse, and as such does not support features such as multi-line strings or escaping special characters. For more flexibility use one of the other formats.

Example:

# This is a comment
database = postgresql://myserver/mydb

# There are two special variables available:
config_path = $__FILE__
data_dir = $__DIR__/data

# Environment variables may also be interpolated
cache_dir = $HOME/.cache/myapp-cache

# As can configuration keys that have already been loaded
sessions_dir = $data_dir/sessions

# Boolean values: 'true' and 'false' are case-insensitive here
apples_are_green = true
horses_are_green = false

# Integers
x = 1
y = 2

# Values with decimal points are parsed as `decimal.Decimal` objects
z = 11.3

# Everything else is a string
hello = world

# Strings may be quoted if required to disambiguate
# or to preserve whitespace
apikey = "123456"
message = "       HELLO $hello       "

JSON (.json)

JSON files are loaded using python’s builtin JSON module.

Toml (.toml)

Loading from Toml files is only supported if the toml package is available.

Python files (.py)

The file will be loaded and parsed as a Python module. Module-level variables are loaded into the options dict unless they start with an underscore.

The __all__ variable may be used to specify a list of keys to load.

Example:

# Use leading underscores to avoid creating
# unnecessary entries in the options dict
from os.path import join as _pathjoin
from os import environ as _env

DATABASE = f"sqlite:///{_env['HOME']}/my.db"

Loading settings from environment variables

If you also want to load your default settings from files and then update this from environment variables, use load():

app = FrescoApp()
app.options.load("myapp.conf", use_environ=True)

If you only want to load values from the environment, use update():

app = FrescoApp()
settings_keys = ['DATABASE']
app.options.update({k: os.environ[k] for k in settings_keys})

Onload hooks

Register callbacks to be run once load() has completed by using onload():

options = fresco.options.Options()

@options.onload
def do_something_with(options):
    print("Howdy!")

options.load(".env*")

Overriding options for testing

override_options() allows you to temporarily override settings during testing:

from myproj import app
from fresco.options import override_options

with override_options(app.options, DATABASE="test_db"):
    # Code inside this block will see the overridden value for
    # app.options.DATABASE
    ...

Note that this globally modifies the options object, and is not thread safe. If you are using this in a multi-threaded environment other threads will see the overridden values, leading to unpredictable results.

API reference: fresco.options

class fresco.options.Options(*a, **kw)[source]

Options dictionary. An instance of this is attached to each fresco.core.FrescoApp instance, as a central store for configuration options.

copy() → a shallow copy of D[source]
load(sources: str | Iterable[Path | str], tags: Sequence[str] = [], use_environ=False, strict=True, dir=None, trigger_onload=True) → Options[source]

Find all files matching glob pattern sources and populates the options object from those with matching filenames containing tags.

Parameters:

  • sources – one or more glob patterns separated by “;”, or a list of glob patterns

  • tags – a list of tags to look for in file names.

  • use_environ – if true, environment variables matching previously loaded keys will be loaded into the options object. This happens after all files have been processed.

  • strict – if true, the first file loaded is assumed to contain all available option keys. Any new key found in a later file will raise an error.

Files may be in python (.py), json (.json), TOML (.toml) format. Any other files will be interpreted as simple lists of `key=value pairs.

Filename format

The general format of filenames is:

<base>(.<priority number>-)(<tags>)(<suffix>)

Example filenames:

# Just <base>
.env

# <base>.<suffix>
.settings.toml

# <base>.<tags>.<suffix>
.env.dev.local..py

# <base>.<priority>-<tags>.<suffix>
.env.100-dev.py

Priority number, if specified, is used to determine loading order. Lower numbers are loaded first. Priority numbers must be positive integers.

Tags are delimited with periods, for example .env.production.py. The filename setttings.dev.local.ini would be considered to have the tags ('dev', 'local')

Where filename contain multiple tags, all tags must match for the file to be loaded.

Tag names may contain the names of environment variable surrounded by braces, for example {USER}. These will be substituted for the environment variable’s value, with any dots or path separators replaced by underscores.

The special variable {hostname} will be substituted for the current host’s name with dots replaced by underscores.

Files with the suffix “.sample” are unconditionally excluded.

Loading order

Files are loaded in the following order:

  1. On priority number, from low to high. If the priority number is not given, a priority of zero is assumed

  2. Then in tag order, based on the ordering given in the tags parameter

    For example, calling options.load(..., tags=["dev", "local"]) would cause a file named “settings.dev” to be loaded before one named “settings.local”.

  3. Finally in lexicographical order.

Environment variables, if requested, are loaded last.

Example:

opts = Options()
opts.load(".env*", ["dev", "host-{hostname}", "local"])

Would load options from files named .env, .env.json, .env.dev.py and .env.local.py, in that order.

onload(fn: Callable) → Callable[source]

Register a function to be called once load has finished populating the options object.

trigger_onload()[source]

Mark the options object as having loaded and call any registered onload callbacks

update_from_dict(d, load_all=False)[source]

Update from the given list of key-value pairs.

If load_all is True, all key-value pairs will be loaded.

Otherwise, if the special key ‘__all__’ is present, only those keys listed in __all__ will be loaded (same semantics as from … import *)

Otherwise only those NOT beginning with _ will be loaded.

update_from_file(path, load_all=False)[source]

Update the instance with any symbols found in the python source file at path.

Parameters:

  • path – The path to a python source file

  • load_all – If true private symbols will also be loaded into the options object.

update_from_object(ob, load_all=False)[source]

Update the instance with any symbols found in object ob.

Parameters:

load_all – If true private symbols will also be loaded into the options object.