This fixes a fairly large tempfile leak. Vagrant uses a template
renderer to write network configuration files locally to disk. Then,
that temporarily file is uploaded to the remote host and moved into
place. Since Vagrant is such a short-lived process, GC never came along
and cleaned up those tempfiles, resulting in many temporary files being
created through regular Vagrant usage.
The Util::Tempfile class uses a block to ensure the temporary file is
deleted when the block finishes. This API required small tweaks to the
usage, but provides more safety to ensure the files are deleted.
With this change, the `raw_arguments` and `raw_ssh_args` options are:
- STILL automatically converted as an Array when they are set a String
(no behaviour change)
- rejected if they are not of Array data type otherwise
Additional Notes:
- the 'as_array' tiny helper has been removed since it was no longer
used.
- there is for now no deeper validation (i.e. verifying that the Array
elements are only *String* objects)
With cb80286a4a, the helper function
stringify_ansible_playbook_command was also applied on the
`raw_arguments` content, which is not wanted. Given that users have used
the `raw_arguments` option as a workaround to avoid the bug GH-6726,
this new change ensure that any `--extra-vars` option passed as a raw
argument won't be additonally enquoted by the ansible_local
provisioner.
This change also improves the ansible remote provisioner verbose output,
but has no impact on its behaviour, which was already correct.
Note that this refactoring introduces some code duplications that are not
very elegant (see ansible_playbook_command_for_shell_execution in
host.rb and execute_ansible_playbook_from_host in base.rb). I hope we
can find a better implementation later, but it is good enough for now
since all these parts are covered by corresponding unit tests (the
`ansible_local` stuff being tested via the verbose output of the ansible
remote provisioner).
When updating the inventory, write to a temp file and replace the
original once writing is complete, to allow for an atomic replacement
of the contents.
Ensures that ansible reading an inventory file will get either the old
or new contents, but never the truncated version of the file that
appears should you open it with 'w' mode set to replace the contents.
Solves the 'provided hosts list is empty' error, which is emitted by
ansible should it manage to be reading the inventory file just as it
was truncated, but before the new contents were flushed to disk.
Partially-Fixes: #6526
Before this minor change, the '--limit' and '--start-at-task'
ansible-playbook command line arguments were enclosed into single
quotes. Using double quotes adds a bit more flexibility, especially
about the task name referred by `start_at_task` option.
It also aligns with the handling of the '--extra-vars' parameter
(see cb80286).
Without this change, the JSON string generated from the `extra_vars`
Ruby hash is passed without enclosing quotes and is then not parseable
by the ansible-playbook command when exectuted in a usual shell context.
In this changeset, the ansible (remote) unit test coverage is improved
to cover both usage of `extra_vars` (ansible_local unit tests are still
missing).
Additional Notes:
- Double quotes are favored to single quotes in order to allow usage of
any character for the variable values. For this reason additional
escaping is appended to JSON-inner double quotes and backslashes.
- This problem was not affecting the `ansible` remote provisioner
(which is running the ansible-playbook command via the childprocess
Ruby library). But with this change, the `verbose` output will also
now be correct for a copy-paste reuse.
- After this change, all the "--extra-vars" arguments (also a var
file passed with the @-syntax or anything coming via the
`raw_arguments` option) are "blindly" and systematically enclosed
in double quoted and double-escaped.
This is not optimal and can potentially break with peculiar values
(e.g. a double quote character (") cannot be used in a json value
when using `raw_arguments`). That said, I think that the current
solution is a reasonable trade-off, since the official `extra_vars`
option should now be able to cover a great majority of use cases.
Fix#6726
Before this change, the detection of a non-existing path on the guest
machine was considered as an error and lead to interrupt the current vagrant
action. This was actually a mistake to do so, since the config checks
are performed before many other vagrant actions than `provision`.
The config.validate phase is also intended to primarily check the options
sanity, but it cannot be too strict with the guest state (which can easily
get "out of automatic control").
With this change, we still apply these checks (when possible), but only warn
about possible configuration problems. This way, the subsequent
statements will happen anyway (e.g. ansible commands will be
executed, vagrant machine will be destroyed, etc.)
At least for ansible 2.0.0.1 the command `ansible-galaxy --help` is inappropriate for testing if ansible is installed, as it yields an error:
```
vagrant@vagrant-ubuntu-trusty-64:~$ ansible-galaxy --help && echo "OK"
Usage: ansible-galaxy [delete|import|info|init|install|list|login|remove|search|setup] [--help] [options] ...
Options:
-h, --help show this help message and exit
-v, --verbose verbose mode (-vvv for more, -vvvv to enable connection
debugging)
--version show program's version number and exit
ERROR! Missing required action
```
In cd93721, I relied on a suprising combination of quotes to protect ssh
execution to strip the quoted path to the private key file.
Since any ssh command line argument can be passed via
`ANSIBLE_SSH_ARGS`, it is quite more readable and easy to rely on the
`-i` argument, which is not affected like `-o IdentityFile=...` and also
supports multiple occurences.
See also http://sourceforge.net/p/fuse/mailman/message/30498048/
Finally fix#6671
Note that I decided to not squash both commits for better
documentation and traceability.
Surprisingly (to me at least), a simple quote enclosure was not enough
to fix the problem.
Caveat: the stringified ansible-playbook command logged in verbose mode
is wrongly formatted (no quotes are escaped).
Fix#6671
Like in the (remote) `ansible` provisioner, it is preferred to pass the
directory that contains the generated inventory file. This way, advanced
inventory usages can be achieved by adding more inventory files into the
same directory.
Related to #2103 and #6500
[ci skip]
String and Symbol types are different when used as a Hash key. By
default the Vagrant machine names are set in Symbol format, but users
may write their `host_vars` entries with String keys. This is a very
simple way to ensure smooth experience, without having to coerce the
data types during the config validation (e.g. with a library like
Hashie, which is currently not in the Vagrant dependencies)
See also:
- https://bugs.ruby-lang.org/issues/5964#note-17
- https://github.com/intridea/hashie#keyconversion
I missed to rename the refactored exceptions as AnsibleCommandFailed in
the guest-based parts. The lack of unit tests for these parts hurts...
on my agenda, I swear!
See c1f3d114f5
With this change, the existing host-based Ansible provisioner is
refactored to share a maximum of code with this new guest-based Ansible
provisioner.
At this stage of development, the existing unit tests are intentionally
modified as little as possible, to keep safe the existing funtionalities.
Other issues resolved by this changeset:
- Display a warning when running from a Windows host [GH-5292]
- Do not run `ansible-playbook` in verbose mode when the `verbose` option
is set to an empty string.
The benefits of the following "breaking change" are the following:
- default behaviour naturally fits with most common usage (i.e. always
connect with Vagrant SSH settings)
- the autogenerated inventory is more consistent by providing both the
SSH username and private key.
- no longer needed to explain how to override Ansible `remote_user` parameters
Important: With the `force_remote_user` option, people still can fall
back to the former behavior (prior to Vagrant 1.8.0), which means that
Vagrant integration capabilities are still quite open and flexible.
When provisioning multiple machines in sequence (the default vagrant
behaviour), it doesn't make sense to require to provide the private ssh
key(s) via the custom ansible inventory script/file.
To align with the handling of multiple ssh keys per machine, we won't
rely any longer on `--private-key` command line argument, but only pass
the keys via `ANSIBLE_SSH_ARGS` environment variable.
Note that when vagrant generates the ansible inventory and that only one
key is associated to a VM, this step would be redundant, and therefore
won't be applied.
This change fixes the breaking change introduced by 3d62a91.
Vagrant 1.7.1 creates and injects new ssh keys for each virtual machine.
When it started ansible with the "parallel provisioning trick",
it would only send the ssh key of the targeted virtual machine.
With this change, vagrant now stores the ssh key for each virtual
machines directly in the generated ansible inventory, and thus allow
ansible parallelism.
Note that this change is not sufficient, as it would break vagrant
configuration based on a custom inventory (file or script). This issue
will be addressed in a next commit.
Signed-off-by: Luis Pabón <lpabon@redhat.com>
The Ansible Vagrant provisioner has a race where the inventory file is
updated every time the provisioner runs unless a file is provided.
Therefore if Ansible attempts to provision two nodes in parallel, you
may see the following race:
* System A writes the inventory file and calls Ansible.
* System B starts to provision and truncates the file before
creating a new one.
* Ansible on system A now attempts to read the inventory
file, which is blank. Ansible bombs out with "ERROR: provided
hosts list is empty".
To fix this, we only allow Vagrant to update the inventory file if
it needs to.
Revert 1c884fa4e5 which introduced the
following bug:
Instead of allowing to dump the `ansible-playbook` command details when
VAGRANT_LOG=debug was defined, it was then impossible to disable this
console output when VAGRANT_LOG was undefined (in such case,
``@logger.debug? systematically returns `true`)
In order to keep things simple and focused, it is preferable to drop the
bad idea to mix Ansible verbosity and Vagrant log level.
Fix#5803
After #5532 (e745436df3), it was no longer
possible to enable ansible colorized output. Even though
`ANSIBLE_NOCOLOR` has no effect *at the moment* in vagrant+ansible
integration, I agree to keep it for clarity and consistence.
The new `--no-color` behaviour (bug fix#5531) is now covered by a unit
test.
//cc @marsam, @sethvargo
This change helps to avoid troubles like reported in #5018 and #4860.
Note that for sake of configuration simplicity, no new `ansible.timeout`
option has been added. The users who want to set a different value can
rely on `ansible.raw_arguments`.
This SSH option is always set, except when Vagrant is running from an
operating system fo the Solaris-family, as this parameter is not
supported by SunSSH. Logic taken from
bed1f8335f/lib/vagrant/util/ssh.rb (L116-L121)Fix#5017
Like Vagrant's default SSH behaviors (e.g ssh or ssh-config commands),
the Ansible provisioner should by default not modify or read the user
known host file (e.g. ~/.ssh/known_hosts).
Given that `UserKnownHostsFile=/dev/null` SSH option is usually combined
with `StrictHostKeyChecking=no`, it seems quite reasonable to bind the
activation/disactivation of both options to `host_key_checking`
provisioner attribute.
For the records, a discussion held in Ansible-Development mailing list
clearly confirmed that there is no short-term plan to adapt Ansible to
offer an extra option or change the behavior of
ANSIBLE_HOST_KEY_CHECKING. For this reason, the current implementation
seems reasonable and should be stable on the long run.
Close#3900
Related References:
- https://groups.google.com/forum/#!msg/ansible-devel/iuoZs1oImNs/6xrj5oa1CmoJ
- https://github.com/ansible/ansible/issues/9442
- force `--connection=ssh` (any other modes like paramiko or smart are not
supported)
- give the highest priority to `raw_arguments` for sake of simplicity (in
usage, in code and in documentation)
- fix position of the `--limit` argument (the generated inventory could be
shadowed by `raw_arguments`, while ansible.limit was able to override
`raw_arguments`
ref #3396
When `--connection` argument is not specified, Ansible will use the
'smart' mode, which can either use `ssh` or `paramiko` transports,
depending of the version of OpenSSH available. If OpenSSH version is new
enough to support ControlPersist technology, `ssh` will be used.
See also http://docs.ansible.com/intro_configuration.html#transport.
In order to support some advanced features of Vagrant (e.g. multiple ssh
private key identities or ssh forwarding), the Ansible provisioner
already must force `ssh` connection mode.
Having to deal with the possible fallback to `paramiko` increase the
burden of special cases that Ansible provisioner must handle, without
any added value, as Vagrant is based on OpenSSH and its users are
usually using modern operating systems.
With this change, the Ansible provisioner will officially only support
`ssh`. It will still be possible to switch to another connection mode
via `raw_arguments`, but it will breach the "contract", and no
(community) support can be expected in such use case.
ref #3900, #3396
As a result of #4670 and the safe decision to not memoize
machine.ssh_info (see 89a4a29d65 and
5036d16461), it is preferable to store the
ssh_info hashes and avoid multiple function calls when generating the
ansible inventory.
Motivation:
By printing out the ansible command used behind the scene, we can ease
the support effort to very quickly identify whether a problem is due to
Vagrant provisioner or Ansible itself.
By referring the directory that contains the generated inventory file,
users can easily provide more settings with additional files stored in
the same directory.
Since the Ansible provisioner now potentially exports ANSIBLE_SSH_ARGS
variable, it is fair to allow to extend the content of this environment
variable (`ssh_args` parameters from ansible.cfg file have lower
priority)
Note that this feature requires to force `--connection=ssh`. This is not
a big deal as `paramiko` mode is deprecated and in most cases `smart`
mode enables `ssh` mode.
- The implicit default limit is always set
- ansible.limit as an empty string won't disable the default limit, but
will be passed as "--limit=" argument and ansible-playbook will return
an error (provided host list is empty)
- Support arbitrary depth of "groups of groups of ... groups"
- Skip ':vars' suffix, but allow group names with ':' (yes, Ansible
accepts this character)
- Like for groups of machines, groups of groups can result "empty", but
it is not an issue for Ansible. Recursive filter on the group tree is
a bit hard to implement, and don't brind real added value at Vagrant
level.
Except ':children' for groups of groups, it is safer to avoid generating
':suffix' blocks. At the moment Ansible only supports (but doesn't
recommend) group variables (:vars), and the Vagrant Ansible provisioner
won't support this way to define variables.
Syntax errors in `ansible.groups` definition are not well handled:
Error returned: undefined method `each' for "machine1":String (NoMethodError)
Being tolerant here doesn't hurt and may avoid people get
confused/annoyed.
env.active_machines can potentiall return 'invalid' machines:
- Ignore machines that are not declared in current Vagrantfile
- Warn when machines are missing (it usually occurs when the VM is
removed without `vagrant destroy` and some orphan metadata remains
in .vagrant/machines/...)
The Ansible provisioner will now only create a single inventory file named,
"vagrant_ansible_inventory". All defined Vagrant machines will be added to
this inventory file. Provisioning will now include a "--limit=#{machine}"
option to scope Ansible provisioning tasks to just the current machine. Setting
the Ansible provisioner's "limit" config option will override the new default
limit. Ansible provisioning scripts will now have access to all other defined
machines and what groups they reside in.
Without this change, it is not possible to pass more than one "raw"
argument, which was not the expected behavior. In addition to Array
format, String (for a single argument) is still accepted (for sake of
"backward compatibility" and ease of use).
Note: Due to low/expert usage of this option, I think that it is not
necessary to add more robust validation on this parameter (e.g. Array
of String type checking or argument syntax pattern matching). Use it at
your own risk ;-)
Seth Vargo
Gilles Cornu
Darragh Bailey
Daniel Schreiber
char16t
Rickard von Essen
Kamjar Gerami
Christian Henz
konomae
Luis Pabón
Nicholas Randon
Mario Rodas
Veres Lajos
Kalman Hazins
jjshoe
Emilien Kenler
Mitchell Hashimoto
Bryan Hunt
Mark Aaron Shirley
Matthew Johnson
egghead
jabclab