Full Table of Contents¶

Globbing¶

The default matching that Salt utilizes is shell-style globbing around the minion id. This also works for states in the top file.

Match all minions:

salt '*' test.ping

Match all minions in the example.net domain or any of the example domains:

salt '*.example.net' test.ping
salt '*.example.*' test.ping

Match all the webN minions in the example.net domain (web1.example.net, web2.example.net … webN.example.net):

salt 'web?.example.net' test.ping

Match the web1 through web5 minions:

salt 'web[1-5]' test.ping

Match the web-x, web-y, and web-z minions:

salt 'web-[x-z]' test.ping

Regular Expressions¶

Minions can be matched using Perl-compatible regular expressions (which is globbing on steroids and a ton of caffeine).

Match both web1-prod and web1-devel minions:

salt -E 'web1-(prod|devel)' test.ping

When using regular expressions in a State's top file, you must specify the matcher as the first option. The following example executes the contents of webserver.sls on the above-mentioned minions.

base:
  'web1-(prod|devel)':
  - match: pcre
  - webserver

Lists¶

At the most basic level, you can specify a flat list of minion IDs:

salt -L 'web1,web2,web3' test.ping

Listing Grains¶

Available grains can be listed by using the 'grains.ls' module:

salt '*' grains.ls

Grains data can be listed by using the 'grains.items' module:

salt '*' grains.items

Grains in the Minion Config¶

Grains can also be statically assigned within the minion configuration file. Just add the option grains and pass options to it:

grains:
  roles:
    - webserver
    - memcache
  deployment: datacenter4
  cabinet: 13
  cab_u: 14-15

Then status data specific to your servers can be retrieved via Salt, or used inside of the State system for matching. It also makes targeting, in the case of the example above, simply based on specific data about your deployment.

Grains in /etc/salt/grains¶

If you do not want to place your custom static grains in the minion config file, you can also put them in /etc/salt/grains. They are configured in the same way as in the above example, only without a top-level grains: key:

roles:
  - webserver
  - memcache
deployment: datacenter4
cabinet: 13
cab_u: 14-15

Grains in Top file¶

With correctly setup grains on the Minion, the Top file used in Pillar or during Highstate can be made really efficient. Like for example, you could do:

'node_type:web':
    - match: grain
    - webserver

'node_type:postgres':
    - match: grain
    - database

'node_type:redis':
    - match: grain
    - redis

'node_type:lb':
    - match: grain
    - lb

For this example to work, you would need the grain node_type and the correct value to match on. This simple example is nice, but too much of the code is similar. To go one step further, we can place some Jinja template code into the Top file.

{% set self = grains['node_type'] %}

    'node_type:{{ self }}':
        - match: grain
        - {{ self }}

The Jinja code simplified the Top file, and allowed SaltStack to work its magic.

Writing Grains¶

Grains are easy to write. The grains interface is derived by executing all of the "public" functions found in the modules located in the grains package or the custom grains directory. The functions in the modules of the grains must return a Python dict, where the keys in the dict are the names of the grains and the values are the values.

Custom grains should be placed in a _grains directory located under the file_roots specified by the master config file. They will be distributed to the minions when state.highstate is run, or by executing the saltutil.sync_grains or saltutil.sync_all functions.

Before adding a grain to Salt, consider what the grain is and remember that grains need to be static data. If the data is something that is likely to change, consider using Pillar instead.

Used With Boto¶

Boto will accept a string for user data which can be used to pass our bootstrap script. If the script is saved to a file, you can read it into a string:

import boto

user_data = open('salt_bootstrap.sh')

conn = boto.connect_ec2(<AWS_ACCESS_ID>, <AWS_SECRET_KEY>)

reservation = conn.run_instances(image_id=<ami_id>,
                                 key_name=<key_name>,
                                 user_data=user_data.read())

Additional Notes¶

Sometime in the future the ppa will include and install an upstart file. In the meantime, you can use the bootstrap to build one.

It may also be useful to set the node's role during this phase. One option would be saving the node's role to a file and then using a custom Grain to select it.

Setting up Hypervisors¶

The first step to set up the hypervisors involves getting the correct software installed and setting up the hypervisor network interfaces.

libguestfs:
  pkg.installed

qemu-kvm:
  pkg.installed

libvirt:
  pkg.installed

libvirtd:
  service.running:
    - enable: True
    - watch:
      - pkg: libvirt

libvirt_keys:
  libvirt.keys

Getting Virtual Machine Images Ready¶

Salt Virt, requires that virtual machine images be provided as these are not generated on the fly. Generating these virtual machine images differs greatly based on the underlying platform.

Virtual machine images can be manually created using KVM and running through the installer, but this process is not recommended since it is very manual and prone to errors.

Virtual Machine generation applications are avilable for many platforms:

vm-builder:

http://wiki.debian.org/VMBuilder

Using Salt Virt¶

With hypervisors set up and virtual machine images ready, Salt can start issuing cloud commands.

Start by deploying

Use cron to initiate a highstate¶

If you would like the Salt Minion to regularly check in with the master you can use the venerable cron to run the salt-call command.

# PATH=/bin:/sbin:/usr/bin:/usr/sbin

00 00 * * * salt-call state.highstate

The above cron entry will run a highstate every day at midnight.

Getting Started¶

To build frozen applications, you'll need a suitable build environment for each of your platforms. You should probably set up a virtualenv in order to limit the scope of Q/A.

This process does work on Windows. Follow the directions at https://github.com/saltstack/salt-windows-install for details on installing Salt in Windows. Only the 32-bit Python and dependencies have been tested, but they have been tested on 64-bit Windows.

You will need to install esky and bbfreeze from Pypi in order to enable the bdist_esky command in setup.py.

Building and Freezing¶

Once you have your tools installed and the environment configured, you can then python setup.py bdist to get the eggs prepared. After that is done, run python setup.py bdist_esky to have Esky traverse the module tree and pack all the scripts up into a redistributable. There will be an appropriately versioned salt-VERSION.zip in dist/ if everything went smoothly.

Using the Frozen Build¶

Unpack the zip file in your desired install location. Scripts like salt-minion and salt-call will be in the root of the zip file. The associated libraries and bootstrapping will be in the directories at the same level. (Check the Esky documentation for more information)

To support updating your minions in the wild, put your builds on a web server that your minions can reach. salt.modules.saltutil.update() will trigger an update and (optionally) a restart of the minion service under the new version.

Gotchas¶

RHEL 6 / CENTOS 6¶

The lokkit command packaged with some Linux distributions makes opening iptables firewall ports very simple via the command line. Just be careful to not lock out access to the server by neglecting to open the ssh port.

lokkit example:

lokkit -p 22:tcp -p 4505:tcp -p 4506:tcp

The system-config-firewall-tui command provides a text-based interface to modifying the firewall.

system-config-firewall-tui:

system-config-firewall-tui

openSUSE¶

Salt installs firewall rules in /etc/sysconfig/SuSEfirewall2.d/services/salt. Enable with:

SuSEfirewall2 open
SuSEfirewall2 start

If you have an older package of Salt where the above configuration file is not included, the SuSEfirewall2 command makes opening iptables firewall ports very simple via the command line.

SuSEfirewall example:

SuSEfirewall2 open EXT TCP 4505
SuSEfirewall2 open EXT TCP 4506

The firewall module in YaST2 provides a text-based interface to modifying the firewall.

YaST2:

yast2 firewall

iptables¶

Different Linux distributions store their iptables rules in different places, which makes it difficult to standardize firewall documentation. Included are some of the more common locations, but your mileage may vary.

Fedora / RHEL / CentOS:

/etc/sysconfig/iptables

Arch Linux:

/etc/iptables/iptables.rules

Debian

Follow these instructions: http://wiki.debian.org/iptables

Once you've found your firewall rules, you'll need to add the two lines below to allow traffic on tcp/4505 and tcp/4506:

-A INPUT -m state --state new -m tcp -p tcp --dport 4505 -j ACCEPT
-A INPUT -m state --state new -m tcp -p tcp --dport 4506 -j ACCEPT

Ubuntu

Salt installs firewall rules in /etc/ufw/applications.d/salt.ufw. Enable with:

ufw allow salt

pf.conf¶

The BSD-family of operating systems uses packet filter (pf). The following example describes the additions to pf.conf needed to access the Salt master.

pass in on $int_if proto tcp from any to $int_if port 4505
pass in on $int_if proto tcp from any to $int_if port 4506

Once these additions have been made to the pf.conf the rules will need to be reloaded. This can be done using the pfctl command.

pfctl -vf /etc/pf.conf

Simple Configuration¶

To use the gitfs backend only two configuration changes are required on the master. The fileserver_backend option needs to be set with a value of git:

fileserver_backend:
  - git

To configure what fileserver backends will be searched for requested files.

Now the gitfs system needs to be configured with a remote:

gitfs_remotes:
  - git://github.com/saltstack/salt-states.git

These changes require a restart of the master, then the git repo will be cached on the master and new requests for the salt:// protocol will send files found in the remote git repository via the master.

Multiple Remotes¶

The gitfs_remotes option can accept a list of git remotes, the remotes are then searched in order for the requested file. A simple scenario can illustrate this behavior.

Assuming that the gitfs_remotes option specifies three remotes:

gitfs_remotes:
  - git://github.com/example/first.git
  - git://github.com/example/second.git
  - file:///root/third

Assume that each repository contains some files:

first.git:
    top.sls
    edit/vim.sls
    edit/vimrc
    nginx/init.sls

second.git:
    edit/dev_vimrc
    haproxy/init.sls

third:
    haproxy/haproxy.conf
    edit/dev_vimrc

The repositories will be searched for files by the master in the order in which they are defined in the configuration, Therefore the remote git://github.com/example/first.git will be searched first, if the requested file is found then it is served and no further searching is executed. This means that if the file salt://haproxy/init.sls is requested then it will be pulled from the git://github.com/example/second.git git repo. If salt://haproxy/haproxy.conf is requested then it will be pulled from the third repo.

Serving from a Subdirectory¶

The gitfs_root option gives the ability to serve files from a subdirectory within the repository. The path is defined relative to the root of the repository.

With this repository structure:

repository.git:
    somefolder
        otherfolder
            top.sls
            edit/vim.sls
            edit/vimrc
            nginx/init.sls

Configuration and files can be accessed normally with:

gitfs_root: somefolder/otherfolder

Multiple Backends¶

Sometimes it may make sense to use multiple backends. For instance, if sls files are stored in git, but larger files need to be stored directly on the master.

The logic used for multiple remotes is also used for multiple backends. If the fileserver_backend option contains multiple backends:

fileserver_backend:
  - roots
  - git

Then the roots backend (the default backend of files in /srv/salt) will be searched first for the requested file, then if it is not found on the master the git remotes will be searched.

Branches, environments and top.sls files¶

As stated above, when using the gitfs backend, branches will be mapped to environments using the branch name as identifier. There is an exception to this rule thought: the master branch is implicitly mapped to the base environment. Therefore, for a typical base, qa, dev setup, you'll have to create the following branches:

master
qa
dev

Also, top.sls files from different branches will be merged into one big file at runtime. Since this could lead to hardly manageable configurations, the recommended setup is to have the top.sls file only in your master branch, and use environment-specific branches for states definitions.

GitFS Remotes over SSH¶

In order to configure a gitfs_remotes repository over SSH transport the git+ssh URL form must be used.

gitfs_remotes:
  - git+ssh://git@github.com/example/salt-states.git

The private key used to connect to the repository must be located in ~/.ssh/id_rsa for the user running the salt-master.

Why aren't my custom modules/states/etc. syncing to my Minions?¶

In versions 0.16.3 and older, when using the git fileserver backend, certain versions of GitPython may generate errors when fetching, which Salt fails to catch. While not fatal to the fetch process, these interrupt the fileserver update that takes place before custom types are synced, and thus interrupt the sync itself. Try disabling the git fileserver backend in the master config, restarting the master, and attempting the sync again.

This issue will be worked around in Salt 0.16.4 and newer.

Order your minions around¶

Now that you have a master and at least one minion communicating with each other you can perform commands on the minion via the salt command. Salt calls are comprised of three main components:

salt '<target>' <function> [arguments]

salt '*' test.ping
salt '*.example.org' test.ping

salt -G 'os:Ubuntu' test.ping

salt -E 'virtmach[0-9]' test.ping

salt -L 'foo,bar,baz,quo' test.ping

salt -C 'G@os:Ubuntu and webser* or E@database.*' test.ping

salt '*' sys.doc

salt '*' test.ping

salt '*' cmd.run 'uname -a'

salt '*' cmd.exec_code python 'import sys; print sys.version'

salt '*' pip.install salt timeout=5 upgrade=True

Summary of Steps¶

1. Create a redundant master server
2. Copy primary master key to redundant master
3. Start redundant master
4. Configure minions to connect to redundant master
5. Restart minions
6. Accept keys on redundant master

Prepping a Redundant Master¶

The first task is to prepare the redundant master. There is only one requirement when preparing a redundant master, which is that masters share the same private key. When the first master was created, the master's identifying key was generated and placed in the master's pki_dir. The default location of the key is /etc/salt/pki/master/master.pem. Take this key and copy it to the same location on the redundant master. Assuming that no minions have yet been connected to the new redundant master, it is safe to delete any existing key in this location and replace it.

Once the new key is in place, the redundant master can be safely started.

Configure Minions¶

Since minions need to be master-aware, the new master needs to be added to the minion configurations. Simply update the minion configurations to list all connected masters:

master:
  - saltmaster1.example.com
  - saltmaster2.example.com

Now the minion can be safely restarted.

Now the minions will check into the original master and also check into the new redundant master. Both masters are first-class and have rights to the minions.

Sharing Files Between Masters¶

Salt does not automatically share files between multiple masters. A number of files should be shared or sharing of these files should be strongly considered.

Setting Up Pillar¶

The pillar is already running in Salt by default. The data in the minion's pillars can be seen via the following command:

salt '*' pillar.items

By default the contents of the master configuration file are loaded into pillar for all minions, this is to enable the master configuration file to be used for global configuration of minions.

The pillar is built in a similar fashion as the state tree, it is comprised of sls files and has a top file, just like the state tree. The pillar is stored in a different location on the Salt master than the state tree. The default location for the pillar is in /srv/pillar.

To start setting up the pillar, the /srv/pillar directory needs to be present:

mkdir /srv/pillar

Now a simple top file, following the same format as the top file used for states needs to be created:

/srv/pillar/top.sls:

base:
  '*':
    - data

This top file associates the data.sls file to all minions. Now the /srv/pillar/data.sls file needs to be populated:

/srv/pillar/data.sls:

info: some data

Now that the file has been saved the minions' pillars will be updated:

salt '*' pillar.items

The key info should now appear in the returned pillar data.

users:
  thatch: 1000
  shouse: 1001
  utahdave: 1002
  redbeard: 1003

base:
  '*':
    - data
    - users

{% for user, uid in pillar.get('users', {}).items() %}
{{user}}:
  user.present:
    - uid: {{uid}}
{% endfor %}

Paramaterizing States With Pillar¶

One of the most powerful abstractions in pillar is the ability to parameterize states. Instead of defining macros or functions within the state context the entire state tree can be freely parameterized relative to the minion's pillar.

This approach allows for Salt to be very flexible while staying very straightforward. It also means that simple sls formulas used in the state tree can be directly parameterized without needing to refactor the state tree.

A simple example is to set up a mapping of package names in pillar for separate Linux distributions:

/srv/pillar/pkg/init.sls:

pkgs:
  {% if grains['os_family'] == 'RedHat' %}
  apache: httpd
  vim: vim-enhanced
  {% elif grains['os_family'] == 'Debian' %}
  apache: apache2
  vim: vim
  {% elif grains['os'] == 'Arch' %}
  apache: apache
  vim: vim
  {% endif %}

The new pkg sls needs to be added to the top file:

/srv/pillar/top.sls:

base:
  '*':
    - data
    - users
    - pkg

Now the minions will auto map values based on respective operating systems inside of the pillar, so sls files can be safely parameterized:

/srv/salt/apache/init.sls:

apache:
  pkg.installed:
    - name: {{ pillar['pkgs']['apache'] }}

Or, if no pillar is available a default can be set as well:

/srv/salt/apache/init.sls:

apache:
  pkg.installed:
    - name: {{ salt['pillar.get']('pkgs:apache', 'httpd') }}

In the above example, if the pillar value pillar['pkgs']['apache'] is not set in the minion's pillar, then the default of httpd will be used.

Pillar Makes Simple States Grow Easily¶

One of the design goals of pillar is to make simple sls formulas easily grow into more flexible formulas without refactoring or complicating the states.

A simple formula:

/srv/salt/edit/vim.sls:

vim:
  pkg:
    - installed

/etc/vimrc:
  file.managed:
    - source: salt://edit/vimrc
    - mode: 644
    - user: root
    - group: root
    - require:
      - pkg: vim

Can be easily transformed into a powerful, parameterized formula:

/srv/salt/edit/vim.sls:

vim:
  pkg:
    - installed
    - name: {{ pillar['pkgs']['vim'] }}

/etc/vimrc:
  file.managed:
    - source: {{ pillar['vimrc'] }}
    - mode: 644
    - user: root
    - group: root
    - require:
      - pkg: vim

Where the vimrc source location can now be changed via pillar:

/srv/pillar/edit/vim.sls:

{% if grain['id'].startswith('dev') %}
vimrc: salt://edit/dev_vimrc
{% elif grain['id'].startswith('qa') %}
vimrc: salt://edit/qa_vimrc
{% else %}
vimrc: salt://edit/vimrc
{% endif %}

Ensuring that the right vimrc is sent out to the correct minions.

More On Pillar¶

The pillar data is generated on the Salt master and securely distributed to minions. Salt is not restricted to the pillar sls files when defining the pillar but can retrieve data from external sources. This can be useful when information about an infrastructure is stored in a separate location.

Reference information on pillar and the external pillar interface can be found in the Salt Stack documentation:

Pillar

Bootstrap Salt Minion¶

First we need to install the salt minion. The salt-bootstrap script makes this incredibly easy for any OS with a Bourne shell. You can use it like this:

wget -O - http://bootstrap.saltstack.org | sudo sh

Or see the salt-bootstrap documentation for other one liners. Additionally, if you are using Vagrant to test out salt, the salty-vagrant tool will provision the VM for you.

Create State Tree¶

Now we build an example state tree. This is where the configuration is defined. For more in depth directions, see the tutorial.

1. Create the top.sls file:

/srv/salt/top.sls:

base:
  '*':
    - webserver

2. Create our webserver state tree:

/srv/salt/webserver.sls:

apache:               # ID declaration
  pkg:                # state declaration
    - installed       # function declaration

The only thing left is to provision our minion using the highstate command. Salt-call also gives us an easy way to give us verbose output:

salt-call --local state.highstate -l debug

The --local flag tells the salt-minion to look for the state tree in the local file system. Normally the minion copies the state tree from the master and executes it from there.

That's it, good luck!

Telling Salt Call to Run Masterless¶

The salt-call command is used to run module functions locally on a minion instead of executing them from the master. Normally the salt-call command checks into the master to retrieve file server and pillar data, but when running standalone salt-call needs to be instructed to not check the master for this data. To instruct the minion to not look for a master when running salt-call the file_client configuration option needs to be set. By default the file_client is set to remote so that the minion knows that file server and pillar data are to be gathered from the master. When setting the file_client option to local the minion is configured to not gather this data from the master.

file_client: local

Now the salt-call command will not look for a master and will assume that the local system has all of the file ad pillar resources.

Running States Masterless¶

The state system can be easily run without a Salt master, with all needed files local to the minion. To do this the minion configuration file needs to be set up to know how to return file_roots information like the master. The file_roots setting defaults to /srv/salt for the base environment just like on the master:

file_roots:
  base:
    - /srv/salt

Now set up the Salt State Tree, top file, and SLS modules in the same way that they would be set up on a master. Now, with the file_client option set to local and an available state tree then calls to functions in the state module will use the information in the file_roots on the minion instead of checking in with the master.

Remember that when creating a state tree on a minion there are no syntax or path changes needed, SLS modules written to be used from a master do not need to be modified in any way to work with a minion.

This makes it easy to "script" deployments with Salt states without having to set up a master, and allows for these SLS modules to be easily moved into a Salt master as the deployment grows.

Now the declared state can now be executed with:

salt-call state.highstate

Or the salt-call command can be executed with the --local flag, this makes it unnecessary to change the configuration file:

salt-call state.highstate --local

It is All Just Data¶

Before delving into the particulars, it will help to understand that the SLS file is just a data structure under the hood. While understanding that the SLS is just a data structure isn't critical for understanding and making use of Salt States, it should help bolster knowledge of where the real power is.

SLS files are therefore, in reality, just dictionaries, lists, strings, and numbers. By using this approach Salt can be much more flexible. As one writes more state files, it becomes clearer exactly what is being written. The result is a system that is easy to understand, yet grows with the needs of the admin or developer.

The Top File¶

The example SLS files in the below sections can be assigned to hosts using a file called top.sls. This file is described in-depth here.

Default Data - YAML¶

By default Salt represents the SLS data in what is one of the simplest serialization formats available - YAML.

A typical SLS file will often look like this in YAML:

apache:
  pkg:
    - installed
  service:
    - running
    - require:
      - pkg: apache

This SLS data will ensure that the package named apache is installed, and that the apache service is running. The components can be explained in a simple way.

The first line is the ID for a set of data, and it is called the ID Declaration. This ID sets the name of the thing that needs to be manipulated.

The second and fourth lines are the start of the State Declarations, so they are using the pkg and service states respectively. The pkg state manages a software package to be installed via the system's native package manager, and the service state manages a system daemon.

The third and fifth lines are the function to run. This function defines what state the named package and service should be in. Here, the package is to be installed, and the service should be running.

Finally, on line six, is the word require. This is called a Requisite Statement, and it makes sure that the Apache service is only started after a successful installation of the apache package.

Adding Configs and Users¶

When setting up a service like an Apache web server, many more components may need to be added. The Apache configuration file will most likely be managed, and a user and group may need to be set up.

apache:
  pkg:
    - installed
  service:
    - running
    - watch:
      - pkg: apache
      - file: /etc/httpd/conf/httpd.conf
      - user: apache
  user.present:
    - uid: 87
    - gid: 87
    - home: /var/www/html
    - shell: /bin/nologin
    - require:
      - group: apache
  group.present:
    - gid: 87
    - require:
      - pkg: apache

/etc/httpd/conf/httpd.conf:
  file.managed:
    - source: salt://apache/httpd.conf
    - user: root
    - group: root
    - mode: 644

This SLS data greatly extends the first example, and includes a config file, a user, a group and new requisite statement: watch.

Adding more states is easy, since the new user and group states are under the Apache ID, the user and group will be the Apache user and group. The require statements will make sure that the user will only be made after the group, and that the group will be made only after the Apache package is installed.

Next,the require statement under service was changed to watch, and is now watching 3 states instead of just one. The watch statement does the same thing as require, making sure that the other states run before running the state with a watch, but it adds an extra component. The watch statement will run the state's watcher function for any changes to the watched states. So if the package was updated, the config file changed, or the user uid modified, then the service state's watcher will be run. The service state's watcher just restarts the service, so in this case, a change in the config file will also trigger a restart of the respective service.

Moving Beyond a Single SLS¶

When setting up Salt States in a scalable manner, more than one SLS will need to be used. The above examples were in a single SLS file, but two or more SLS files can be combined to build out a State Tree. The above example also references a file with a strange source - salt://apache/httpd.conf. That file will need to be available as well.

The SLS files are laid out in a directory structure on the Salt master; an SLS is just a file and files to download are just files.

The Apache example would be laid out in the root of the Salt file server like this:

apache/init.sls
apache/httpd.conf

So the httpd.conf is just a file in the apache directory, and is referenced directly.

But when using more than one single SLS file, more components can be added to the toolkit. Consider this SSH example:

ssh/init.sls:

openssh-client:
  pkg.installed

/etc/ssh/ssh_config:
  file.managed:
    - user: root
    - group: root
    - mode: 644
    - source: salt://ssh/ssh_config
    - require:
      - pkg: openssh-client

ssh/server.sls:

include:
  - ssh

openssh-server:
  pkg.installed

sshd:
  service.running:
    - require:
      - pkg: openssh-client
      - pkg: openssh-server
      - file: /etc/ssh/banner
      - file: /etc/ssh/sshd_config

/etc/ssh/sshd_config:
  file.managed:
    - user: root
    - group: root
    - mode: 644
    - source: salt://ssh/sshd_config
    - require:
      - pkg: openssh-server

/etc/ssh/banner:
  file:
    - managed
    - user: root
    - group: root
    - mode: 644
    - source: salt://ssh/banner
    - require:
      - pkg: openssh-server

Now our State Tree looks like this:

apache/init.sls
apache/httpd.conf
ssh/init.sls
ssh/server.sls
ssh/banner
ssh/ssh_config
ssh/sshd_config

This example now introduces the include statement. The include statement includes another SLS file so that components found in it can be required, watched or as will soon be demonstrated - extended.

The include statement allows for states to be cross linked. When an SLS has an include statement it is literally extended to include the contents of the included SLS files.

Note that some of the SLS files are called init.sls, while others are not. More info on what this means can be found in the States Tutorial.

Extending Included SLS Data¶

Sometimes SLS data needs to be extended. Perhaps the apache service needs to watch additional resources, or under certain circumstances a different file needs to be placed.

In these examples, the first will add a custom banner to ssh and the second will add more watchers to apache to include mod_python.

ssh/custom-server.sls:

include:
  - ssh.server

extend:
  /etc/ssh/banner:
    file:
      - source: salt://ssh/custom-banner

python/mod_python.sls:

include:
  - apache

extend:
  apache:
    service:
      - watch:
        - pkg: mod_python

mod_python:
  pkg.installed

The custom-server.sls file uses the extend statement to overwrite where the banner is being downloaded from, and therefore changing what file is being used to configure the banner.

In the new mod_python SLS the mod_python package is added, but more importantly the apache service was extended to also watch the mod_python package.

Understanding the Render System¶

Since SLS data is simply that (data), it does not need to be represented with YAML. Salt defaults to YAML because it is very straightforward and easy to learn and use. But the SLS files can be rendered from almost any imaginable medium, so long as a renderer module is provided.

The default rendering system is the yaml_jinja renderer. The yaml_jinja renderer will first pass the template through the Jinja2 templating system, and then through the YAML parser. The benefit here is that full programming constructs are available when creating SLS files.

Other renderers available are yaml_mako and yaml_wempy which each use the Mako or Wempy templating system respectively rather than the jinja templating system, and more notably, the pure Python or py and pydsl renderers. The py renderer allows for SLS files to be written in pure Python, allowing for the utmost level of flexibility and power when preparing SLS data; while the pydsl renderer provides a flexible, domain-specific language for authoring SLS data in Python.

apache:
  pkg.installed:
    {% if grains['os'] == 'RedHat'%}
    - name: httpd
    {% endif %}
  service.running:
    {% if grains['os'] == 'RedHat'%}
    - name: httpd
    {% endif %}
    - watch:
      - pkg: apache
      - file: /etc/httpd/conf/httpd.conf
      - user: apache
  user.present:
    - uid: 87
    - gid: 87
    - home: /var/www/html
    - shell: /bin/nologin
    - require:
      - group: apache
  group.present:
    - gid: 87
    - require:
      - pkg: apache

/etc/httpd/conf/httpd.conf:
  file.managed:
    - source: salt://apache/httpd.conf
    - user: root
    - group: root
    - mode: 644

include:
  - moosefs

{% for mnt in salt['cmd.run']('ls /dev/data/moose*').split() %}
/mnt/moose{{ mnt[-1] }}:
  mount.mounted:
    - device: {{ mnt }}
    - fstype: xfs
    - mkmnt: True
  file.directory:
    - user: mfs
    - group: mfs
    - require:
      - user: mfs
      - group: mfs
{% endfor %}

/etc/mfshdd.cfg:
  file.managed:
    - source: salt://moosefs/mfshdd.cfg
    - user: root
    - group: root
    - mode: 644
    - template: jinja
    - require:
      - pkg: mfs-chunkserver

/etc/mfschunkserver.cfg:
  file.managed:
    - source: salt://moosefs/mfschunkserver.cfg
    - user: root
    - group: root
    - mode: 644
    - template: jinja
    - require:
      - pkg: mfs-chunkserver

mfs-chunkserver:
  pkg:
    - installed
mfschunkserver:
  service:
    - running
    - require:
{% for mnt in salt['cmd.run']('ls /dev/data/moose*') %}
      - mount: /mnt/moose{{ mnt[-1] }}
      - file: /mnt/moose{{ mnt[-1] }}
{% endfor %}
      - file: /etc/mfschunkserver.cfg
      - file: /etc/mfshdd.cfg
      - file: /var/lib/mfs

#!py

def run():
    '''
    Install the django package
    '''
    return {'include': ['python'],
            'django': {'pkg': ['installed']}}

#!pydsl

include('python', delayed=True)
state('django').pkg.installed()

include:
  - python

django:
  pkg.installed

Next Reading¶

With an understanding of states, the next recommendation is to become familiar with Salt's pillar interface:

Pillar Walkthrough

Setting up the Salt State Tree¶

States are stored in text files on the master and transferred to the minions on demand via the master's File Server. The collection of state files make up the State Tree.

To start using a central state system in Salt, the Salt File Server must first be set up. Edit the master config file (file_roots) and uncomment the following lines:

file_roots:
  base:
    - /srv/salt

Restart the Salt master in order to pick up this change:

pkill salt-master
salt-master -d

Preparing the Top File¶

On the master, in the directory uncommented in the previous step, (/srv/salt by default), create a new file called top.sls and add the following:

base:
  '*':
    - webserver

The top file is separated into environments (discussed later). The default environment is base. Under the base environment a collection of minion matches is defined; for now simply specify all hosts (*).

base:
  'os:Fedora':
    - match: grain
    - webserver

Create an sls module¶

In the same directory as the top file, create an empty file named webserver.sls, containing the following:

apache:                 # ID declaration
  pkg:                  # state declaration
    - installed         # function declaration

The first line, called the ID declaration, is an arbitrary identifier. In this case it defines the name of the package to be installed. NOTE: the package name for the Apache httpd web server may differ depending on OS or distro — for example, on Fedora it is httpd but on Debian/Ubuntu it is apache2.

The second line, called the state declaration, defines which of the Salt States we are using. In this example, we are using the pkg state to ensure that a given package is installed.

The third line, called the function declaration, defines which function in the pkg state module to call.

Install the package¶

Next, let's run the state we created. Open a terminal on the master and run:

% salt '*' state.highstate

Our master is instructing all targeted minions to run state.highstate. When a minion executes a highstate call it will download the top file and attempt to match the expressions. When it does match an expression the modules listed for it will be downloaded, compiled, and executed.

Once completed, the minion will report back with a summary of all actions taken and all changes made.

salt -t 60

salt-minion -l debug &          # On the minion
salt '*' state.highstate -t 60  # On the master

Next steps¶

This tutorial focused on getting a simple Salt States configuration working. Part 2 will build on this example to cover more advanced sls syntax and will explore more of the states that ship with Salt.

Call multiple States¶

You can specify multiple state declarations under an ID declaration. For example, a quick modification to our webserver.sls to also start Apache if it is not running:

1
2
3
4
5
6
7

apache:
  pkg:
    - installed
  service:
    - running
    - require:
      - pkg: apache

Try stopping Apache before running state.highstate once again and observe the output.

Expand the SLS module¶

As you have seen, SLS modules are appended with the file extension .sls and are referenced by name starting at the root of the state tree. An SLS module can be also defined as a directory. Demonstrate that now by creating a directory named webserver and moving and renaming webserver.sls to webserver/init.sls. Your state directory should now look like this:

|- top.sls
`- webserver/
   `- init.sls

Require other states¶

We now have a working installation of Apache so let's add an HTML file to customize our website. It isn't exactly useful to have a website without a webserver so we don't want Salt to install our HTML file until Apache is installed and running. Include the following at the bottom of your webserver/init.sls file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

apache:
  pkg:
    - installed
  service:
    - running
    - require:
      - pkg: apache

/var/www/index.html:                        # ID declaration
  file:                                     # state declaration
    - managed                               # function
    - source: salt://webserver/index.html   # function arg
    - require:                              # requisite declaration
      - pkg: apache                         # requisite reference

line 9 is the ID declaration. In this example it is the location we want to install our custom HTML file. (Note: the default location that Apache serves may differ from the above on your OS or distro. /srv/www could also be a likely place to look.)

Line 10 the state declaration. This example uses the Salt file state.

Line 11 is the function declaration. The managed function will download a file from the master and install it in the location specified.

Line 12 is a function arg declaration which, in this example, passes the source argument to the managed function.

Line 13 is a requisite declaration.

Line 14 is a requisite reference which refers to a state and an ID. In this example, it is referring to the ID declaration from our example in part 1. This declaration tells Salt not to install the HTML file until Apache is installed.

Next, create the index.html file and save it in the webserver directory:

<html>
    <head><title>Salt rocks</title></head>
    <body>
        <h1>This file brought to you by Salt</h1>
    </body>
</html>

Last, call state.highstate again and the minion will fetch and execute the highstate as well as our HTML file from the master using Salt's File Server:

salt '*' state.highstate

Verify that Apache is now serving your custom HTML.

/etc/httpd/extra/httpd-vhosts.conf:
  file:
    - managed
    - source: salt://webserver/httpd-vhosts.conf

apache:
  pkg:
    - installed
  service:
    - running
    - watch:
      - file: /etc/httpd/extra/httpd-vhosts.conf
    - require:
      - pkg: apache

Next steps¶

In part 3 we will discuss how to use includes, extends and templating to make a more complete State Tree configuration.

Templating SLS modules¶

SLS modules may require programming logic or inline execution. This is accomplished with module templating. The default module templating system used is Jinja2 and may be configured by changing the renderer value in the master config.

All states are passed through a templating system when they are initially read. To make use of the templating system, simply add some templating markup. An example of an sls module with templating markup may look like this:

{% for usr in ['moe','larry','curly'] %}
{{ usr }}:
  user.present
{% endfor %}

This templated sls file once generated will look like this:

moe:
  user.present
larry:
  user.present
curly:
  user.present

Here's a more complex example:

{% for usr in 'moe','larry','curly' %}
{{ usr }}:
  group:
    - present
  user:
    - present
    - gid_from_name: True
    - require:
      - group: {{ usr }}
{% endfor %}

Using Grains in SLS modules¶

Often times a state will need to behave differently on different systems. Salt grains objects are made available in the template context. The grains can be used from within sls modules:

apache:
  pkg.installed:
    {% if grains['os'] == 'RedHat' %}
    - name: httpd
    {% elif grains['os'] == 'Ubuntu' %}
    - name: apache2
    {% endif %}

Calling Salt modules from templates¶

All of the Salt modules loaded by the minion are available within the templating system. This allows data to be gathered in real time on the target system. It also allows for shell commands to be run easily from within the sls modules.

The Salt module functions are also made available in the template context as salt:

moe:
  user:
    - present
    - gid: {{ salt['file.group_to_gid']('some_group_that_exists') }}

Note that for the above example to work, some_group_that_exists must exist before the state file is processed by the templating engine.

Below is an example that uses the network.hw_addr function to retrieve the MAC address for eth0:

salt['network.hw_addr']('eth0')

Advanced SLS module syntax¶

Lastly, we will cover some incredibly useful techniques for more complex State trees.

python-dateutil:
  pkg.installed

include:
  - python.python-libs

django:
  pkg.installed:
    - require:
      - pkg: python-dateutil

apache:
  pkg.installed

include:
  - apache.apache

extend:
  apache:
    service:
      - running
      - watch:
        - file: /etc/httpd/extra/httpd-vhosts.conf

/etc/httpd/extra/httpd-vhosts.conf:
  file.managed:
    - source: salt://apache/httpd-vhosts.conf

include:
  - apache.apache

extend:
  apache:
    service:
      - running
      - watch:
        - file: mywebsite

mywebsite:
  file.managed:
    - name: /etc/httpd/extra/httpd-vhosts.conf
    - source: salt://apache/httpd-vhosts.conf

stooges:
  user.present:
    - names:
      - moe
      - larry
      - curly

Next steps¶

In part 4 we will discuss how to use salt's file_roots to set up a workflow in which states can be "promoted" from dev, to QA, to production.

Salt fileserver path inheritance¶

Salt's fileserver allows for more than one root directory per environment, like in the below example, which uses both a local directory and a secondary location shared to the salt master via NFS:

# In the master config file (/etc/salt/master)
file_roots:
  base:
    - /srv/salt
    - /mnt/salt-nfs/base

Salt's fileserver collapses the list of root directories into a single virtual environment containing all files from each root. If the same file exists at the same relative path in more than one root, then the top-most match "wins". For example, if /srv/salt/foo.txt and /mnt/salt-nfs/base/foo.txt both exist, then salt://foo.txt will point to /srv/salt/foo.txt.

Environment configuration¶

Configure a multiple-environment setup like so:

file_roots:
  base:
    - /srv/salt/prod
  qa:
    - /srv/salt/qa
    - /srv/salt/prod
  dev:
    - /srv/salt/dev
    - /srv/salt/qa
    - /srv/salt/prod

Given the path inheritance described above, files within /srv/salt/prod would be available in all environments. Files within /srv/salt/qa would be available in both qa, and dev. Finally, the files within /srv/salt/dev would only be available within the dev environment.

Based on the order in which the roots are defined, new files/states can be placed within /srv/salt/dev, and pushed out to the dev hosts for testing.

Those files/states can then be moved to the same relative path within /srv/salt/qa, and they are now available only in the dev and qa environments, allowing them to be pushed to QA hosts and tested.

Finally, if moved to the same relative path within /srv/salt/prod, the files are now available in all three environments.

Practical Example¶

As an example, consider a simple website, installed to /var/www/foobarcom. Below is a top.sls that can be used to deploy the website:

/srv/salt/prod/top.sls:

base:
  'web*prod*':
    - webserver.foobarcom
qa:
  'web*qa*':
    - webserver.foobarcom
dev:
  'web*dev*':
    - webserver.foobarcom

Using pillar, roles can be assigned to the hosts:

/srv/pillar/top.sls:

base:
  'web*prod*':
    - webserver.prod
  'web*qa*':
    - webserver.qa
  'web*dev*':
    - webserver.dev

/srv/pillar/webserver/prod.sls:

webserver_role: prod

/srv/pillar/webserver/qa.sls:

webserver_role: qa

/srv/pillar/webserver/dev.sls:

webserver_role: dev

And finally, the SLS to deploy the website:

/srv/salt/prod/webserver/foobarcom.sls:

{% if pillar.get('webserver_role', '') %}
/var/www/foobarcom:
  file.recurse:
    - source: salt://webserver/src/foobarcom
    - env: {{ pillar['webserver_role'] }}
    - user: www
    - group: www
    - dir_mode: 755
    - file_mode: 644
{% endif %}

Given the above SLS, the source for the website should initially be placed in /srv/salt/dev/webserver/src/foobarcom.

First, let's deploy to dev. Given the configuration in the top file, this can be done using state.highstate:

salt --pillar 'webserver_role:dev' state.highstate

However, in the event that it is not desirable to apply all states configured in the top file (which could be likely in more complex setups), it is possible to apply just the states for the foobarcom website, using state.sls:

salt --pillar 'webserver_role:dev' state.sls webserver.foobarcom

Once the site has been tested in dev, then the files can be moved from /srv/salt/dev/webserver/src/foobarcom to /srv/salt/qa/webserver/src/foobarcom, and deployed using the following:

salt --pillar 'webserver_role:qa' state.sls webserver.foobarcom

Finally, once the site has been tested in qa, then the files can be moved from /srv/salt/qa/webserver/src/foobarcom to /srv/salt/prod/webserver/src/foobarcom, and deployed using the following:

salt --pillar 'webserver_role:prod' state.sls webserver.foobarcom

Thanks to Salt's fileserver inheritance, even though the files have been moved to within /srv/salt/prod, they are still available from the same salt:// URI in both the qa and dev environments.

Continue learning¶

The best way to continue learning about Salt States is to read through the reference documentation and to look through examples of existing state trees. Many pre-configured state trees can be found on Github in the saltstack-formulas collection of repositories.

If you have any questions, suggestions, or just want to chat with other people who are using Salt, we have a very active community and we'd love to hear from you.

Welcome!¶

Welcome to Salt Stack! I am excited that you are interested in Salt and starting down the path to better infrastructure management. I developed (and am continuing to develop) Salt with the goal of making the best software available to manage computers of almost any kind. I hope you enjoy working with Salt and that the software can solve your real world needs!

• Thomas S Hatch
• Salt creator and chief developer
• CTO of Salt Stack, Inc.

Getting Started¶

systemctl start salt-master

service salt-master start

/etc/init.d/salt-master start

salt-master -d

salt-master -l debug

master: saltmaster.example.com

salt-minion -d

salt-minion -l debug

salt-key -L

salt-key -A

# salt-key -p foo.domain.com
Accepted Keys:
foo.domain.com:  -----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA0JcA0IEp/yqghK5V2VLM
jbG7FWV6qtw/ubTDBnpDGQgrvSNOtd0QcJsAzAtDcHwrudQgyxTZGVJqPY7gLc7P
5b4EFWt5E1w3+KZ+XXy4YtW5oOzVN5BvsJ85g7c0TUnmjL7p3MUUXE4049Ue/zgX
jtbFJ0aa1HB8bnlQdWWOeflYRNEQL8482ZCmXXATFP1l5uJA9Pr6/ltdWtQTsXUA
bEseUGEpmq83vAkwtZIyJRG2cJh8ZRlJ6whSMg6wr7lFvStHQQzKHt9pRPml3lLK
ba2X07myAEJq/lpJNXJm5bkKV0+o8hqYQZ1ndh9HblHb2EoDBNbuIlhYft1uv8Tp
8beaEbq8ZST082sS/NjeL7W1T9JS6w2rw4GlUFuQlbqW8FSl1VDo+Alxu0VAr4GZ
gZpl2DgVoL59YDEVrlB464goly2c+eY4XkNT+JdwQ9LwMr83/yAAG6EGNpjT3pZg
Wey7WRnNTIF7H7ISwEzvik1GrhyBkn6K1RX3uAf760ZsQdhxwHmop+krgVcC0S93
xFjbBFF3+53mNv7BNPPgl0iwgA9/WuPE3aoE0A8Cm+Q6asZjf8P/h7KS67rIBEKV
zrQtgf3aZBbW38CT4fTzyWAP138yrU7VSGhPMm5KfTLywNsmXeaR5DnZl6GGNdL1
fZDM+J9FIGb/50Ee77saAlUCAwEAAQ==
-----END PUBLIC KEY-----

# cat /etc/salt/pki/minion/minion.pub
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA0JcA0IEp/yqghK5V2VLM
jbG7FWV6qtw/ubTDBnpDGQgrvSNOtd0QcJsAzAtDcHwrudQgyxTZGVJqPY7gLc7P
5b4EFWt5E1w3+KZ+XXy4YtW5oOzVN5BvsJ85g7c0TUnmjL7p3MUUXE4049Ue/zgX
jtbFJ0aa1HB8bnlQdWWOeflYRNEQL8482ZCmXXATFP1l5uJA9Pr6/ltdWtQTsXUA
bEseUGEpmq83vAkwtZIyJRG2cJh8ZRlJ6whSMg6wr7lFvStHQQzKHt9pRPml3lLK
ba2X07myAEJq/lpJNXJm5bkKV0+o8hqYQZ1ndh9HblHb2EoDBNbuIlhYft1uv8Tp
8beaEbq8ZST082sS/NjeL7W1T9JS6w2rw4GlUFuQlbqW8FSl1VDo+Alxu0VAr4GZ
gZpl2DgVoL59YDEVrlB464goly2c+eY4XkNT+JdwQ9LwMr83/yAAG6EGNpjT3pZg
Wey7WRnNTIF7H7ISwEzvik1GrhyBkn6K1RX3uAf760ZsQdhxwHmop+krgVcC0S93
xFjbBFF3+53mNv7BNPPgl0iwgA9/WuPE3aoE0A8Cm+Q6asZjf8P/h7KS67rIBEKV
zrQtgf3aZBbW38CT4fTzyWAP138yrU7VSGhPMm5KfTLywNsmXeaR5DnZl6GGNdL1
fZDM+J9FIGb/50Ee77saAlUCAwEAAQ==
-----END PUBLIC KEY-----

salt '*' test.ping

salt '*' sys.doc

salt '*' cmd.run 'ls -l /etc'

salt '*' pkg.install vim

salt '*' network.interfaces

salt '*' pkg.install vim

salt '*' test.echo 'foo: bar'

Salt States¶

Now that the basics are covered the time has come to evaluate States. Salt States, or the State System is the component of Salt made for configuration management. The State system is a fully functional configuration management system which has been designed to be exceptionally powerful while still being simple to use, fast, lightweight, deterministic and with salty levels of flexibility.

The state system is already available with a basic salt setup, no additional configuration is required, states can be set up immediately.

vim:
  pkg.installed

salt '*' state.sls vim

vim:
  pkg.installed

/etc/vimrc:
  file.managed:
    - source: salt://vimrc
    - mode: 644
    - user: root
    - group: root

nginx:
  pkg:
    - installed
  service:
    - running
    - require:
      - pkg: nginx

salt '*' state.sls nginx

vim:
  pkg.installed

/etc/vimrc:
  file.managed:
    - source: salt://edit/vimrc
    - mode: 644
    - user: root
    - group: root

So Much More!¶

This concludes the initial Salt walkthrough, but there are many more things to learn still! These documents will cover important core aspects of Salt:

• Pillar
• Job Management

A few more tutorials are also available:

• Remote Execution Tutorial
• Standalone Minion

This still is only scratching the surface, many components such as the reactor and event systems, extending Salt, modular components and more are not covered here. For an overview of all Salt features and documentation, look at the Table of Contents.