Developing Salt¶
There is a great need for contributions to salt and patches are welcome! The goal here is to make contributions clear, make sure there is a trail for where the code has come from, and most importantly, to give credit where credit is due!
There are a number of ways to contribute to salt development.
Sending a GitHub pull request¶
This is the preferred method for contributions. Simply create a GitHub fork, commit changes to the fork, and then open up a pull request.
The following is an example (from Open Comparison Contributing Docs ) of an efficient workflow for forking, cloning, branching, committing, and sending a pull request for a GitHub repository.
First, make a local clone of your GitHub fork of the salt GitHub repo and make edits and changes locally.
Then, create a new branch on your clone by entering the following commands:
git checkout -b fixed-broken-thing
Switched to a new branch 'fixed-broken-thing'
Choose a name for your branch that describes its purpose.
Now commit your changes to this new branch with the following command:
git commit -am 'description of my fixes for the broken thing'
Note
Using git commit -am
, followed by a quoted string, both stages and
commits all modified files in a single command. Depending on the nature of
your changes, you may wish to stage and commit them separately. Also, note
that if you wish to add newly-tracked files as part of your commit, they
will not be caught using git commit -am
and will need to be added using
git add
before committing.
Push your locally-committed changes back up to GitHub:
git push --set-upstream origin fixed-broken-thing
Now go look at your fork of the salt repo on the GitHub website. The new branch will now be listed under the "Source" tab where it says "Switch Branches". Select the new branch from this list, and then click the "Pull request" button.
Put in a descriptive comment, and include links to any project issues related to the pull request.
The repo managers will be notified of your pull request and it will be reviewed. If a reviewer asks for changes, just make the changes locally in the same local feature branch, push them to GitHub, then add a comment to the discussion section of the pull request.
Note
Travis-CI
To make reviewing pull requests easier for the maintainers, please enable Travis-CI on your fork. Salt is already configured, so simply follow the first 2 steps on the Travis-CI Getting Started Doc.
Keeping Salt Forks in Sync¶
Salt is advancing quickly. It is therefore critical to pull upstream changes from master into forks on a regular basis. Nothing is worse than putting in a days of hard work into a pull request only to have it rejected because it has diverged too far from master.
To pull in upstream changes:
# For ssh github
git remote add upstream git@github.com:saltstack/salt.git
git fetch upstream
# For https github
git remote add upstream https://github.com/saltstack/salt.git
git fetch upstream
To check the log to be sure that you actually want the changes, run the following before merging:
git log upstream/develop
Then to accept the changes and merge into the current branch:
git merge upstream/develop
For more info, see GitHub Fork a Repo Guide or Open Comparison Contributing Docs
Posting patches to the mailing list¶
Patches will also be accepted by email. Format patches using git format-patch and send them to the Salt users mailing list. The contributor will then get credit for the patch, and the Salt community will have an archive of the patch and a place for discussion.
Installing Salt for development¶
Clone the repository using:
git clone https://github.com/saltstack/salt
Note
tags
Just cloning the repository is enough to work with Salt and make contributions. However, fetching additional tags from git is required to have Salt report the correct version for itself. To do this, first add the git repository as an upstream source:
git remote add upstream http://github.com/saltstack/salt
Fetching tags is done with the git 'fetch' utility:
git fetch --tags upstream
Create a new virtualenv:
virtualenv /path/to/your/virtualenv
On Arch Linux, where Python 3 is the default installation of Python, use the
virtualenv2
command instead of virtualenv
.
Note
Using system Python modules in the virtualenv
To use already-installed python modules in virtualenv (instead of having pip
download and compile new ones), run virtualenv --system-site-packages
Using this method eliminates the requirement to install the salt dependencies
again, although it does assume that the listed modules are all installed in the
system PYTHONPATH at the time of virtualenv creation.
Activate the virtualenv:
source /path/to/your/virtualenv/bin/activate
Install Salt (and dependencies) into the virtualenv:
pip install M2Crypto # Don't install on Debian/Ubuntu (see below)
pip install pyzmq PyYAML pycrypto msgpack-python jinja2 psutil
pip install -e ./salt # the path to the salt git clone from above
Note
Installing M2Crypto
swig
and libssl-dev
are required to build M2Crypto. To fix
the error command 'swig' failed with exit status 1
while installing M2Crypto,
try installing it with the following command:
env SWIG_FEATURES="-cpperraswarn -includeall -D__`uname -m`__ -I/usr/include/openssl" pip install M2Crypto
Debian and Ubuntu systems have modified openssl libraries and mandate that a patched version of M2Crypto be installed. This means that M2Crypto needs to be installed via apt:
apt-get install python-m2crypto
This also means that pulling in the M2Crypto installed using apt requires using
--system-site-packages
when creating the virtualenv.
Note
Installing psutil
Python header files are required to build this module, otherwise the pip
install will fail. If your distribution separates binaries and headers into
separate packages, make sure that you have the headers installed. In most
Linux distributions which split the headers into their own package, this
can be done by installing the python-dev
or python-devel
package.
For other platforms, the package will likely be similarly named.
Note
Important note for those developing using RedHat variants
For developers using a RedHat variant, be advised that the package provider for newer Redhat-based systems (yumpkg.py) relies on RedHat's python interface for yum. The variants that use this module to provide package support include the following:
- RHEL and CentOS releases 6 and later
- Fedora Linux releases 11 and later
- Amazon Linux
Developers using one of these systems should create the salt virtualenv using the
--system-site-packages
option to ensure that the correct modules are available.
Note
Installing dependencies on OS X.
You can install needed dependencies on OS X using homebrew or macports. See OS X Installation
Running a self-contained development version¶
During development it is easiest to be able to run the Salt master and minion that are installed in the virtualenv you created above, and also to have all the configuration, log, and cache files contained in the virtualenv as well.
Copy the master and minion config files into your virtualenv:
mkdir -p /path/to/your/virtualenv/etc/salt
cp ./salt/conf/master /path/to/your/virtualenv/etc/salt/master
cp ./salt/conf/minion /path/to/your/virtualenv/etc/salt/minion
Edit the master config file:
- Uncomment and change the
user: root
value to your own user. - Uncomment and change the
root_dir: /
value to point to/path/to/your/virtualenv
. - If you are running version 0.11.1 or older, uncomment and change the
pidfile: /var/run/salt-master.pid
value to point to/path/to/your/virtualenv/salt-master.pid
. - If you are also running a non-development version of Salt you will have to
change the
publish_port
andret_port
values as well.
Edit the minion config file:
- Repeat the edits you made in the master config for the
user
androot_dir
values as well as any port changes. - If you are running version 0.11.1 or older, uncomment and change the
pidfile: /var/run/salt-minion.pid
value to point to/path/to/your/virtualenv/salt-minion.pid
. - Uncomment and change the
master: salt
value to point atlocalhost
. - Uncomment and change the
id:
value to something descriptive like "saltdev". This isn't strictly necessary but it will serve as a reminder of which Salt installation you are working with.
Note
Using salt-call with a Standalone Minion
If you plan to run salt-call with this self-contained development
environment in a masterless setup, you should invoke salt-call with
-c /path/to/your/virtualenv/etc/salt
so that salt can find the minion
config file. Without the -c
option, Salt finds its config files in
/etc/salt.
Start the master and minion, accept the minion's key, and verify your local Salt installation is working:
cd /path/to/your/virtualenv
salt-master -c ./etc/salt -d
salt-minion -c ./etc/salt -d
salt-key -c ./etc/salt -L
salt-key -c ./etc/salt -A
salt -c ./etc/salt '*' test.ping
Running the master and minion in debug mode can be helpful when developing. To
do this, add -l debug
to the calls to salt-master
and salt-minion
.
If you would like to log to the console instead of to the log file, remove the
-d
.
Once the minion starts, you may see an error like the following:
zmq.core.error.ZMQError: ipc path "/path/to/your/virtualenv/var/run/salt/minion/minion_event_7824dcbcfd7a8f6755939af70b96249f_pub.ipc" is longer than 107 characters (sizeof(sockaddr_un.sun_path)).
This means the the path to the socket the minion is using is too long. This is a system limitation, so the only workaround is to reduce the length of this path. This can be done in a couple different ways:
- Create your virtualenv in a path that is short enough.
- Edit the
sock_dir
minion config variable and reduce its length. Remember that this path is relative to the value you set inroot_dir
.
NOTE:
The socket path is limited to 107 characters on Solaris and Linux,
and 103 characters on BSD-based systems.
Note
File descriptor limits
Ensure that the system open file limit is raised to at least 2047:
# check your current limit
ulimit -n
# raise the limit. persists only until reboot
# use 'limit descriptors 2047' for c-shell
ulimit -n 2047
To set file descriptors on OSX, refer to the OS X Installation instructions.
Using easy_install to Install Salt¶
If you are installing using easy_install
, you will need to define a
USE_SETUPTOOLS environment variable, otherwise dependencies will not
be installed:
USE_SETUPTOOLS=1 easy_install salt
Running the tests¶
You will need mock
to run the tests:
pip install mock
If you are on Python < 2.7 then you will also need unittest2:
pip install unittest2
Note
In Salt 0.17, testing libraries were migrated into their own repo. To install them:
pip install git+https://github.com/saltstack/salt-testing.git#egg=SaltTesting
Failure to install SaltTesting will result in import errors similar to the following:
ImportError: No module named salttesting
Finally you use setup.py to run the tests with the following command:
./setup.py test
For greater control while running the tests, please try:
./tests/runtests.py -h
Editing and previewing the documentation¶
You need sphinx-build
command to build the docs. In Debian/Ubuntu this is
provided in the python-sphinx
package. Sphinx can also be installed
to a virtualenv using pip:
pip install Sphinx
Change to salt documentation directory, then:
cd doc; make html
- This will build the HTML docs. Run
make
without any arguments to see the available make targets, which include html, man, and text. - The docs then are built within the docs/_build/ folder. To update
the docs after making changes, run
make
again. - The docs use reStructuredText for markup. See a live demo at http://rst.ninjs.org/.
- The help information on each module or state is culled from the python code
that runs for that piece. Find them in
salt/modules/
orsalt/states/
. - To build the docs on Arch Linux, the python2-sphinx package is required. Additionally, it is necessary to tell make where to find the proper sphinx-build binary, like so:
make SPHINXBUILD=sphinx-build2 html
- To build the docs on RHEL/CentOS 6, the python-sphinx10 package must be installed from EPEL, and the following make command must be used:
make SPHINXBUILD=sphinx-1.0-build html