AssetBuilder: Compile, Minify and Bundle CSS, JavaScript and other web assets

AssetBuilder solves the problem of managing static assets — CSS files, JavaScript bundles, images — in Python web projects.

  • AssetBuilder lets you tag together asset files:

    assetbuilder.add_path('frontend-scripts', 'static/jquery.js')
assetbuilder.add_path('frontend-scripts', 'static/intercooler.js')

assetbuilder.add_path('styles', 'static/main.css')

assetbuilder.add_path('admin-scripts', 'static/jquery.js')
assetbuilder.add_path('admin-scripts', 'static/admin.js')

  • AssetBuilder offers a WSGI endpoint for serving those files:

    MyApplication.route_wsgi('/assets', assetbuilder)

  • AssetBuilder generates links to the served files:

    <link py:for="url in assetbuilder.urls('styles', environ)" rel="stylesheet" href="$url"/>

  • AssetBuilder checks asset files are up to date before serving them and calls your build system as required (which could be Gulp, Webpack, Make, …)

AssetBuilder doesn’t: build, compile, minify, uglify or optimize JS, CSS or images. You already have a build system for that. If you want to inline files, It will concatenate – but only if you ask it to.

AssetBuilder doesn’t wrap your build tools or require integration modules. As long as you can call your build script from the CLI, so can AssetBuilder.

AssetBuilder works with Flask, Fresco, Django, Pyramid and any other Python WSGI compatible web framework.

Quickstart

Install AssetBuilder:

pip install assetbuilder

Configure an AssetBuilder instance with your asset paths:

assetbuilder = AssetBuilder('/assets',
                            'static/_build/',
                            depdirs=['static/src/'],
                            autobuild=True)

assetbuilder.add_path('js', 'jquery.min.js')
assetbuilder.add_path('js', 'mysite.min.js', deps=['js/mysite.js', 'js/includes/**/*.js'])
assetbuilder.add_path('css', 'mysite.css', deps=['css/mysite.scss', 'css/includes/**/*.scss'])

Include the assetbuild WSGI application in your configuration:

# Example configuration for Django

from django.conf.urls import url
from django_wsgi.embedded_wsgi import make_wsgi_view

urlpatterns = [url(r'^assets(/.*)$', make_wsgi_view(assetbuilder)),
               ...
               ]

# Example configuration for Fresco
app = FrescoApp()
app.route_wsgi('/assets', assetbuilder)

AssetBuilder Configuration

Asset file configuration

from assetbuilder import AssetBuilder

# [1] '/assets': the base URL assets will be served from. Used for
# generating links to your asset files.
#
# [1] 'myproj/static/build': the filesystem directory where compiled asset
# files are placed.
#
# [2] 'myproj/static/src': where AssetBuilder should scan for dependency
# files. You can add multiple sources here.
#
# [3] autobuild=True: watch dependency files and rebuild assets whenever
# they change. Always turn this off in production!
assets = AssetBuilder('/assets',                      # [1]
                      'myproj/static/.build/',        # [2]
                      depdirs=['myproj/static/src/'], # [3]
                      autobuild=True)                 # [4]

# This is the default shell command used to build asset files.
assets.set_default_build_command('make assets', cwd='myproj/static')

# Now we add paths to each asset files. Each asset file is tagged ('js',
# 'css'), so that we can refer to groups of asset files at the same time
assets.add_path('js', 'jquery/dist/jquery.min.js')
assets.add_path('js', 'site.min.js')
assets.add_path('css', 'site.min.css')

# Additional arguments specify dependencies using glob syntax.
# A prefix of ! can be used to exclude files.
# If any dependencies change, a rebuild is triggered.
assets.add_path('admin-js', 'admin.min.js', '**/*.js', '!**/exclude/**')

# Individual assets can have custom build commands specified
assets.add_path('images', 'img/logo.png', command='make images', cwd='img')

Web framework integration

Once the AssetBuilder object is configured, you need to mount it in your larger application. How you do this is framework specific – here’s an example using django-wsgi:

from django.conf.urls import url
from django_wsgi.embedded_wsgi import make_wsgi_view

urlpatterns = [url(r'^assets(/.*)$', make_wsgi_view(assetbuilder)),
               ...
              ]

And here’s an example for fresco:

app = FrescoApp()
app.route_wsgi('/assets', assetbuilder)

Linking to assets from HTML templates

Finally, you’ll want to add links to your assets. Make sure that the assetbuilder object is available to your template namespace, then add code similar to the following:

{% for url in assetbuilder.urls('js', request.environ) %}
<script src="{{ url }}"></script>
{% end %}

{% for url in assetbuilder.urls('css', request.environ) %}
<link rel="stylesheet" href="{{ url }}" />
{% end %}

Note the use of the tags we defined in the calls to add_path (above), to link to js and css asset sets separately. You can use whatever tags you want, for example to maintain separate sets of assets for different sections of your website.

Inlining assets

Use assetbuilder.inline to inline one or more asset files as a single string:

<script>{{ assetbuilder.inline('js', separator=';') }}</script>

Concatenating multiple asset files

If you want to combine multiple files into a single file, you can do this with

<script>{{ assetbuilder.url(‘admin-js’, ‘site-js’, environ=request.environ, separator=’;’) }}</script>

The generated URL, when routed through assetbuilder’s WSGI server will concatenate all the files in the bundle(s) into a single response.

While this is convenient, it has a disadvantage: you can’t serve assets like this through a regular front-end web server (eg Apache or Nginx), because the files served do not exist on the filesystem.

Rebuilding

One of the major benefits of using assetbuilder it’s ability to scan your asset’s dependencies and rebuild any stale files whenever required. This feature is intended for use during development, so is only available if when you have configured autobuild=True. For production use you must manually rebuild asset files through whatever build tool you use, and the features described in this section do not apply.

Automatic asset rebuilding

To rebuild asset files, AssetBuilder relies on you having configured configured build commands. You can do this globally (through set_default_build_command) and/or per-asset (through add_path). AssetBuilder also relies on you having configured dependencies for each asset (also in add_path). As long as these are configured and autobuild is on, assets will be checked for freshness and rebuilt:

  • on every request to the asset file itself

  • on every call to urls()

Manually triggering asset rebuilds

You can manually trigger rebuilds by pinging the update WSGI endpoint. Assuming AssetBuilder is mounted at http://localhost:8080/assets, you can trigger an update of all assets:

curl http://localhost:8080/assets/update

You can also trigger a ‘clean’ rebuild:

curl http://localhost:8080/assets/update?clean

A clean rebuild will DELETE all asset files configured through add_path, then call the configured build system to recreate them. Only use AssetBuilder to serve compiled asset files that can be completely reconstructed from sources.

Note that these endpoints are only available if autobuild is set to True.

Watching files

If you want builds to happen in the background as soon as you modify an asset file, combine this with a tool such as entr that will watch your files for you:

find myproj/static -name '*.css' -o '*.js' | \
    entr -d sh -c 'curl http://localhost:8080/assets/update'

Troubleshooting

AssetBuilder will complain if asset building fails for any reason. You can turn up the logging verbosity with standard python logging configuration.

The logger name is assetbuilder. You can set this to log at INFO or DEBUG level in your logging configuration file, or programmatically with:

logging.getLogger('assetbuilder').setLevel(logging.DEBUG)

API documentation

class assetbuilder.AssetBuilder(baseurl, directory, depdirs=None, autobuild=False, secret=None)[source]

AssetBuilder objects hold information on managed asset files, their dependencies and file modification times. They allow applications to generate URLs for static asset files, serve the asset files via WSGI and trigger rebuilds when the asset files are no longer up to date.

Parameters

  • baseurl – base url for serving assets. this must correspond to the URL the AssetBuilder WSGI application is mounted on. If only the path section is given, a fully qualified URL will be generated from the WSGI environ when generating asset urls with AssetBuilder.urlfor.

  • directory – directory to search for asset files

  • depdirs – list of directories to search for asset dependencies

add_global_dep(dep)[source]

Add a dependency that will be checked for all asset files

add_path(tag, p, deps=[], command=None, chdir=False, cwd=None, env=None, shell=True)[source]

Add a path to a managed asset file.

Parameters

  • tag – a tag for this asset, that can later be used to retrieve groups of related assets. For example if your application has a shopping cart page, then the assets required for this page might be tagged ‘cart-js’ and ‘cart-css’.

  • p – The filesystem path to the asset file

  • deps – A list of dependency file specs to be monitored for changes. Example: deps=['includes/*.js', '!includes/jquery.js']

  • command – a build command specific to this asset file.

  • chdir – if True, change directory to the asset file’s directory when building (only applies if the command argument is supplied)

  • cwd – path to a directory in which command should be run (only applies if the command argument is supplied)

  • env – mapping of environment variables to be passed to command

  • shell – If True (the default), run command through the shell. If False, execute command directly.

add_paths(tag, paths, deps=[], command=None, chdir=False, cwd=None, env=None, shell=True)[source]

Add paths to multiple asset files.

Parameters

paths – a list or other iterable of string paths

Other arguments are the same as for add_path().

baseurl

Base URL for generating asset URLs, eg ‘https://example.org/’

build_commands

Mapping of asset virtual paths to BuildCommand objects

cachebusters

Mapping of asset virtual paths to (mtime, cachebuster querystring)

cat(*tags: str, separator='\n', encoding='UTF-8') → Iterator[str][source]

Iterate over the content of all asset files with the given tags

dependencies

Mapping of asset virtual paths to dependencies

dependency_mtimes

Mapping of dependency files to last updated times

ensure_up_to_date(assetfile, lock=None)[source]

Ensures that the given assetfile is up to date.

global_deps

List of files which are considered dependencies for all assets

negative_deps

Mapping of asset virtual paths to a set of files to exclude from the dependency list

rebuild_all(clean=False)[source]

Trigger a rebuild of all asset files.

Parameters

clean – If True, unlink asset files before rebuilding (ie this will DELETE the file).

If the rebuild_all_command property is set, this will be used to build all assets in a single step. Otherwise the build command for each registered asset will be called in turn.

This method also clears the unbuildable set, causing previously unbuilt assets to be retried.

served

Mapping of asset virtual paths to asset Path

set_default_build_command(command, chdir=False, cwd=None, env=None, shell=True)[source]

Set a build command to be used for all assets that do have a build command configured.

set_rebuild_all_command(command, cwd=None, env=None, shell=True)[source]

Configure a command to rebuild all assets in one go (eg “make all”). If this is left unset, each registered asset file will be rebuilt individually.

tags

Mapping of tags to asset virtual paths

unbuildable

Set of asset virtual paths which have never successfully been built Membership of this set prevents the assetbuilder from retrying the build command (which otherwise would run on every request, potentially causing signifant server load).

update_cachebusters()[source]

Check all cachebusters and update any that are out of date

url(*tags, environ=None, separator='\n')[source]

Generate a single URL for a dynamically generated file of all assets with the given tag.

Parameters

  • *tags –

    tags, as previously configured through add_path().

  • environ – (optional) a WSGI environ dict, used to generate absolute URLs in the absence of baseurl

urls(*tags: str, environ=None)[source]

Generate URLs for asset files with the given tag. If environ is supplied and no baseurl has been configured, use this to generate absolute URLs.

Parameters

  • tag – a string tag, as previously configured through add_path().

  • environ – (optional) a WSGI environ dict, used to generate absolute URLs in the absence of baseurl

class assetbuilder.BuildCommand(command, chdir, cwd, shell, env)
property chdir

Alias for field number 1

property command

Alias for field number 0

property cwd

Alias for field number 2

property env

Alias for field number 4

property shell

Alias for field number 3

assetbuilder.serve_static_files(paths: List[pathlib.Path], separator: bytes, cachebuster_valid: bool, environ: Dict[str, Any], start_response: Callable, bufsize: int = 8192)[source]

Serve the concatenated static files located at paths.