GitFS Backend Walkthrough¶

While the default location of the salt state tree is on the Salt master, in /srv/salt, the master can create a bridge to external resources for files. One of these resources is the ability for the master to directly pull files from a git repository and serve them to minions.

Note

This walkthrough assumes basic knowledge of Salt. To get up to speed, check out the walkthrough.

The gitfs backend hooks into any number of remote git repositories and caches the data from the repository on the master. This makes distributing a state tree to multiple masters seamless and automated.

Salt's file server also has a concept of environments, when using the gitfs backend, Salt translates git branches and tags into environments, making environment management very simple. Just merging a QA or staging branch up to a production branch can be all that is required to make those file changes available to Salt.

Simple Configuration¶

To use the gitfs backend only two configuration changes are required on the master. The fileserver_backend option needs to be set with a value of git:

fileserver_backend:
  - git

To configure what fileserver backends will be searched for requested files.

Now the gitfs system needs to be configured with a remote:

gitfs_remotes:
  - git://github.com/saltstack/salt-states.git

Note

The salt-states repo is not currently updated with the latest versions of the available states. Please review https://github.com/saltstack-formulas for the latest versions.

These changes require a restart of the master, then the git repo will be cached on the master and new requests for the salt:// protocol will send files found in the remote git repository via the master.

Note

The master caches the files from the git server and serves them out, minions do not connect directly to the git server meaning that only requested files are delivered to minions.

Multiple Remotes¶

The gitfs_remotes option can accept a list of git remotes, the remotes are then searched in order for the requested file. A simple scenario can illustrate this behavior.

Assuming that the gitfs_remotes option specifies three remotes:

gitfs_remotes:
  - git://github.com/example/first.git
  - git://github.com/example/second.git
  - file:///root/third

Note

This example is purposefully contrived to illustrate the behavior of the gitfs backend. This example should not be read as a recommended way to lay out files and git repos.

Note

The file:// prefix denotes a git repository in a local directory. However, it will still use the given file:// URL as a remote, rather than copying the git repo to the salt cache. This means that any refs you want accessible must exist as local refs in the specified repo.

Assume that each repository contains some files:

first.git:
    top.sls
    edit/vim.sls
    edit/vimrc
    nginx/init.sls

second.git:
    edit/dev_vimrc
    haproxy/init.sls

third:
    haproxy/haproxy.conf
    edit/dev_vimrc

The repositories will be searched for files by the master in the order in which they are defined in the configuration, Therefore the remote git://github.com/example/first.git will be searched first, if the requested file is found then it is served and no further searching is executed. This means that if the file salt://haproxy/init.sls is requested then it will be pulled from the git://github.com/example/second.git git repo. If salt://haproxy/haproxy.conf is requested then it will be pulled from the third repo.

Serving from a Subdirectory¶

The gitfs_root option gives the ability to serve files from a subdirectory within the repository. The path is defined relative to the root of the repository.

With this repository structure:

repository.git:
    somefolder
        otherfolder
            top.sls
            edit/vim.sls
            edit/vimrc
            nginx/init.sls

Configuration and files can be accessed normally with:

gitfs_root: somefolder/otherfolder

Multiple Backends¶

Sometimes it may make sense to use multiple backends. For instance, if sls files are stored in git, but larger files need to be stored directly on the master.

The logic used for multiple remotes is also used for multiple backends. If the fileserver_backend option contains multiple backends:

fileserver_backend:
  - roots
  - git

Then the roots backend (the default backend of files in /srv/salt) will be searched first for the requested file, then if it is not found on the master the git remotes will be searched.

Branches, environments and top.sls files¶

As stated above, when using the gitfs backend, branches will be mapped to environments using the branch name as identifier. There is an exception to this rule thought: the master branch is implicitly mapped to the base environment. Therefore, for a typical base, qa, dev setup, you'll have to create the following branches:

master
qa
dev

Also, top.sls files from different branches will be merged into one big file at runtime. Since this could lead to hardly manageable configurations, the recommended setup is to have the top.sls file only in your master branch, and use environment-specific branches for states definitions.

GitFS Remotes over SSH¶

In order to configure a gitfs_remotes repository over SSH transport the git+ssh URL form must be used.

gitfs_remotes:
  - git+ssh://git@github.com/example/salt-states.git

The private key used to connect to the repository must be located in ~/.ssh/id_rsa for the user running the salt-master.

Note

GitFS requires the Python module GitPython, version 0.3.0 or newer.

Why aren't my custom modules/states/etc. syncing to my Minions?¶

In versions 0.16.3 and older, when using the git fileserver backend, certain versions of GitPython may generate errors when fetching, which Salt fails to catch. While not fatal to the fetch process, these interrupt the fileserver update that takes place before custom types are synced, and thus interrupt the sync itself. Try disabling the git fileserver backend in the master config, restarting the master, and attempting the sync again.

This issue will be worked around in Salt 0.16.4 and newer.