To reduce the amount of setup required to start using Deb Constrictor for new projects, build configurations now allow for multiple parents (as of version 0.7). Now, when writing configurations, your build-config.json can inherit from a parent project config containing the project name, and a “base” config that defines the type of project it is. This is intended to be used to the same effect as Maven’s archetypes. It is a powerful way to compose projects to avoid repetition.

Here’s an updated example for deploying a Django application with separate DPKGs for the application code, virtualenv and configuration.

Python Django Application

The build-config.json file specifically for the Django app is now very simple, it uses variables and parents to define most of its configuration:

{
  "parent": ["../parent-config.json", "~/deb-constrictor/python-app.json"],
  "version": "1.0",
  "description": "Django Web Frontend",
  "deb_constrictor": {
    "environment_variables": [
      ["PYTHON_MAJOR_SUFFIX", "3"]
    ]
  }
}

Of particular note is the use of the PYTHON_MAJOR_SUFFIX environment variable. By setting this we can choose to use the Python 3, or 2 (by setting to an empty string) version of any dependencies. Also see that the parent option is now an array.

parent-config.json lives one directory up and has this content:

{
    "deb_constrictor": {
        "environment_variables": [
                ["PROJECT_NAME", "example-web-frontend"]
        ]
    }
}

The PROJECT_NAME variable is used throughout all generated configuration files.

Finally the python-app.json from my home directory. This uses the variables defined in the child configuration to populate itself and installs a bunch of common dependencies.

{
  "package": "${PROJECT_NAME}",
  "architecture": "all",
  "extra_control_fields": {
    "Section": "misc",
    "Priority": "optional",
    "Depends": [
      "bbit-postgres-bash-functions",
      "bbit-python-bash-configuration",
      "bbit-json2shell",
      "nginx",
      "uwsgi",
      "uwsgi-plugin-python${PYTHON_MAJOR_SUFFIX}",
      "postgresql",
      "${PROJECT_NAME}-config",
      "${PROJECT_NAME}-virtualenv"
    ]
  },
  "directories": [
    {
      "source": "src",
      "destination": "/srv/python/${PROJECT_NAME}",
      "uname": "www-data"
    }
  ],
  "maintainer_scripts": {
    "postinst": "scripts/after-install",
    "preinst": "scripts/before-install"
  }
}

The package name is generated automatically from the PROJECT_NAME environment variable. The dependencies are a bunch of helpful packages I’ve created for installation, servers, the uwsgi-plugin-python for the correct version of Python and so on. The config and virtualenv dependency names for the Django app are generated based on the PROJECT_NAME variable, and so is the installation path.

Virtualenv

Next is the virtualenv setup. Its build-config.json file contains this:

{
  "parent": ["../parent-config.json", "~/deb-constrictor/virtualenv-parent.json"],
  "version": "1.0",
  "deb_constrictor": {
      "environment_variables": [
        ["PYTHON_VERSION", "3.6"]
      ]
  }
}

The only setting required here is the PYTHON_VERSION, the rest is stored in the virtualenv-parent.json (again, in my home directory).

{
  "package": "${VENV_NAME}",
  "architecture": "amd64",
  "description": "The virtual environment for ${PROJECT_NAME}",
  "deb_constrictor": {
      "environment_variables": [
        ["VENV_NAME", "${PROJECT_NAME}-virtualenv"],
        ["VENV_DIR", "build/${VENV_NAME}"],
        ["VENV_BIN_DIR", "${VENV_DIR}/bin"]
      ],
      "commands": {
        "prebuild": ["~/deb-constrictor/build-venv.sh"]
      },
      "remove_after_build": true
  },
  "extra_control_fields": {
    "Depends": [
      "python${PYTHON_VERSION}",
      "libpython${PYTHON_VERSION}",
      "python${PYTHON_VERSION}-distutils"
    ]
  },
  "directories": [
    {
      "source": "build/virtualenvs/${VENV_NAME}",
      "destination": "/var/virtualenvs/${VENV_NAME}"
    }
  ],
  "links": [
    {
      "path": "/var/virtualenvs/${VENV_NAME}/lib/python${PYTHON_VERSION}/encodings",
      "target": "/usr/lib/python${PYTHON_VERSION}/encodings"
    },
    {
      "path": "/var/virtualenvs/${VENV_NAME}/lib/python${PYTHON_VERSION}/lib-dynload",
      "target": "/usr/lib/python${PYTHON_VERSION}/lib-dynload"
    }
  ]
}

The VENV_NAME, VENV_DIR and VENV_BIN_DIR are automatically generated from the PROJECT_NAME set in parent-config.json (the same as above). All these environment variable are available in the build-venv.sh script when it is run (I won’t show the contents of that here, but it basically builds the virtualenv and install the requirements from the requirements.txt file).

The installation paths and Python-related dependencies are automatically generated too. Executing the constrictor-build command for the virtualenv’s build-config.json will also execute the build-venv.sh script.

Configuration

Finally there’s building the configuration package. Again the build-config.json just for the configuration directory is quite small now:

{
  "parent": [
    "../parent-config.json",
    "~/deb-constrictor/uwsgi-nginx-config-parent-dev.json",
    "~/deb-constrictor/uwsgi-django-config-parent.json"
  ],
  "version": "1.0"
}

Once again the parent-config.json is included to set the PROJECT_NAME. Next is the uwsgi-nginx-config-parent-dev.json. I wanted to be able to build both dev and prod versions of a specific config, so starting with with this uwsgi-nginx-config-parent-dev.json:

{
    "parent": "uwsgi-nginx-config-parent.json",
    "deb_constrictor": {
      "environment_variables": [
        ["ENVIRONMENT_LEVEL", "dev"]
      ]
  }
}

This is exactly the same as uwsgi-nginx-config-parent-prod.json, except as you might expect, the ENVIRONMENT_LEVEL in that file is prod. This configuration’s parent file contains this:

{
  "package": "${PROJECT_NAME}-${ENVIRONMENT_LEVEL}-config",
  "architecture": "all",
  "description": "Configuration for ${PROJECT_NAME}",
  "extra_control_fields": {
    "Provides": ["${PROJECT_NAME}-config"]
  },
  "directories": [
    {
      "source": "src/etc",
      "destination": "/etc"
    }
  ],
  "links": [
    {
      "path": "/etc/nginx/sites-enabled/${PROJECT_NAME}",
      "target": "../sites-available/${PROJECT_NAME}"
    }
  ]
}

The package is generated from the project name and environment level, so I can choose to install either the dev or prod version of configuration to a particular server. Either way, the same package is provided so the application’s dependencies are met. The PROJECT_NAME is used to put the Nginx config in the correct place and link it.

The Nginx config file is not that important (it’s a standard one for Uwsgi) but it lives in the config project’s directory, so the path src/etc is relative to the main build-config.json that is being executed. This way the specific configuration can be kept inside the project directory.

Finally the uwsgi-django-config-parent.json file:

{
    "extra_control_fields": {
        "Depends": [
          "bbit-django-uwsgi-configuration"
        ]
    },
    "links": [
        {
            "path": "/etc/uwsgi/apps-enabled/${PROJECT_NAME}.ini",
            "target": "../apps-available/django-app.skel"
        }
    ]
}

It depends on a special configuration DPKG which installs a standard skeleton Uwsgi file in the correct location, and then simply creates a link with the right name to that configuration. Uwsgi does some magic with the %n variable to interpolate the name of the link into the config file — since the same PROJECT_NAME is being used throughout everything magically is in the right place.

New Project Setup

While it was a fair amount of work to get all these files set up initially, now it is trivial to set up new projects for deployment. I simply create a parent-config.json with the right PROJECT_NAME, and then I can pretty much just copy the build-config.json files from existing projects, set their versions back to 1.0, and that’s it. Previously without the use of variables and multiple parents I would have to define everything in multiple places – this keeps everything nice and DRY.

I’m also using a global constrictor-build-config.json file in my home directory that defines a postbuild script to upload the DPKG to my reprepro server. The config also contains my maintainer details and defines paths to ignore when gathering files (e.g. .git).

Conclusion

In conclusion, the ability to set multiple parents allows the powerful composition of build configurations. For example, uwsgi-flask-config-parent.json which can be used instead of uwsgi-django-config-parent.json for deploying Flask apps instead — everything else remains the same.

The next feature I feel like I am needing is some kind of templating for config files. For instance, the Nginx config files are basically the same each time except for different domains and path to the Uwsgi socket. It would be good to populate these from the same build-config.json files before packaging them up (similar to how it is done with Ansible). However that might be a feature for a future post. Stay tuned… maybe.