haskal
/
vagrant
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge branch 'f-acceptance' 

This brings in the new acceptance testing style. See the README.
This commit is contained in:
Mitchell Hashimoto 2013-12-05 14:33:12 -08:00
parent 6b1493ec3f 77100ad417
commit b484c8d628
9 changed files with 56 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -6,6 +6,7 @@ acceptance_config.yml
boxes/*
/Vagrantfile
/.vagrant
/vagrant-spec.config.rb
# Bundler/Rubygems
*.gem

6
Gemfile
View File

@ -1,3 +1,9 @@
source "http://rubygems.org"
gemspec
if File.exist?(File.expand_path("../../vagrant-spec", __FILE__))
gem 'vagrant-spec', path: "../vagrant-spec"
else
gem 'vagrant-spec', git: "https://github.com/mitchellh/vagrant-spec.git"
end

45
README.md
View File

@ -54,36 +54,31 @@ This will run the unit test suite, which should come back all green! Then you're
If you want to run Vagrant without having to install the gem, you may use `bundle exec`,
like so:
bundle exec bin/vagrant help
bundle exec vagrant help
### Acceptance Tests
Vagrant also comes with an acceptance test suite which runs the system
end-to-end, without mocking out any dependencies. Note that this test
suite is **extremely slow**, with the test suite taking hours on even
a decent system. A CI will be setup in due time to run these tests
automatically. However, it is still useful to know how to run these
tests since it is often useful to run a single test if you're working
on a specific feature.
Vagrant also comes with an acceptance test suite that does black-box
tests of various Vagrant components. Note that these tests are **extremely
slow** because actual VMs are spun up and down. The full test suite can
take hours. Instead, try to run focused component tests.
The acceptance tests have absolutely _zero_ dependence on the Vagrant
source. Instead, an external configuration file must be used to give
the acceptance tests some parameters (such as what Vagrant version is
running, where the Vagrant `vagrant` binary is, etc.). If you want to
run acceptance tests against source, or just want to see an example of
this file, you can generate it automatically for the source code:
To run the acceptance test suite, first copy `vagrant-spec.config.example.rb`
to `vagrant-spec.config.rb` and modify it to valid values. The places you
should fill in are clearly marked.
rake acceptance:config
Next, see the components that can be tested:
This will drop an `acceptance_config.yml` file in your working directory.
You can then run a specific acceptance test like so:
```
$ rake acceptance:components
cli
provider/virtualbox/basic
...
```
ACCEPTANCE_CONFIG=./acceptance_config.yml ruby test/acceptance/version_test.rb
Then, run one of those components:
That's it!
If you're developing an acceptance test and you're unsure why things
might be failing, you can also view log output for the acceptance tests,
which can be very verbose but are a great help in finding bugs:
ACCEPTANCE_LOGGING=debug ACCEPTANCE_CONFIG=./acceptance_config.yml ruby test/acceptance/version_test.rb
```
$ rake acceptance:run COMPONENTS="cli"
...
```

2
plugins/providers/virtualbox/synced_folder.rb
View File

@ -62,7 +62,7 @@ module VagrantPlugins
end
def cleanup(machine)
driver(machine).clear_shared_folders
driver(machine).clear_shared_folders if machine.id && machine.id != ""
end
protected

119
tasks/acceptance.rake
View File

@ -1,113 +1,22 @@
require 'digest/sha1'
require 'net/http'
require 'pathname'
require 'uri'
require 'yaml'
require 'childprocess'
require 'vagrant/util/file_checksum'
namespace :acceptance do
desc "Downloads the boxes required for running the acceptance tests."
task :boxes, :directory do |t, args|
# Create the directory where the boxes will be downloaded
box_dir = Pathname.new(args[:directory] || File.expand_path("../../boxes", __FILE__))
box_dir.mkpath
puts "Boxes will be placed in: #{box_dir}"
# Load the required boxes
boxes = YAML.load_file(File.expand_path("../../test/config/acceptance_boxes.yml", __FILE__))
boxes.each do |box|
puts "Box: #{box["name"]}"
box_file = box_dir.join("#{box["name"]}.box")
checksum = FileChecksum.new(box_file, Digest::SHA1)
# If the box exists, we need to check the checksum and determine if we need
# to redownload the file.
if box_file.exist?
print "Box exists, checking SHA1 sum... "
if checksum.checksum == box["checksum"]
print "OK\n"
next
else
print "FAIL\n"
end
end
# Download the file. Note that this has no error checking and just uses
# pure net/http. There could be a better way.
puts "Downloading: #{box["url"]}"
destination = box_file.open("wb")
uri = URI.parse(box["url"])
Net::HTTP.new(uri.host, uri.port).start do |http|
http.request_get(uri.request_uri) do |response|
total = response.content_length
progress = 0
count = 0
response.read_body do |segment|
# Really elementary progress meter
progress += segment.length
count += 1
puts "Progress: #{(progress.to_f / total.to_f) * 100}%" if count % 300 == 0
# Write out to the destination file
destination.write(segment)
end
end
end
destination.close
# Check the checksum of the new file to verify that it
# downloaded properly. This shouldn't happen, but it can!
if checksum.checksum != box["checksum"]
puts "Checksum didn't match! Download was corrupt!"
abort
end
end
desc "shows components that can be tested"
task :components do
exec("vagrant-spec components --config=vagrant-spec.config.rb")
end
desc "Generates the configuration for acceptance tests from current source."
task :config, :box_dir do |t, args|
require File.expand_path("../../lib/vagrant/version", __FILE__)
require File.expand_path('../../test/support/tempdir', __FILE__)
desc "runs acceptance tests"
task :run do
args = [
"--config=vagrant-spec.config.rb",
]
# Get the directory for the boxes
box_dir = Pathname.new(args[:box_dir] || File.expand_path("../../boxes", __FILE__))
# Generate the binstubs for the Vagrant binary
tempdir = Tempdir.new
process = ChildProcess.build("bundle", "install", "--binstubs", tempdir.path)
process.io.inherit!
process.start
process.poll_for_exit(64000)
if process.exit_code != 0
# Bundle install failed...
puts "Bundle install failed!"
abort
if ENV["COMPONENTS"]
args << "--components=\"#{ENV["COMPONENTS"]}\""
end
# Generate the actual configuration
config = {
"vagrant_path" => File.join(tempdir.path, "vagrant"),
"vagrant_version" => Vagrant::VERSION,
"env" => {
"BUNDLE_GEMFILE" => File.expand_path("../../Gemfile", __FILE__)
},
"box_directory" => box_dir.to_s
}
File.open("acceptance_config.yml", "w+") do |f|
f.write(YAML.dump(config))
end
puts <<-OUTPUT
Acceptance test configuration is now in this directory in
"acceptance_config.yml." Set your ACCEPTANCE_CONFIG environmental
variable to this file and run any of the acceptance tests now.
OUTPUT
command = "vagrant-spec test #{args.join(" ")}"
puts command
puts
exec(command)
end
end

4
tasks/test.rake
View File

@ -11,8 +11,4 @@ namespace :test do
t.libs << "test/unit_legacy"
t.pattern = "test/unit_legacy/**/*_test.rb"
end
RSpec::Core::RakeTask.new(:acceptance) do |t|
t.pattern = "test/acceptance/**/*_test.rb"
end
end

2
test/acceptance/base.rb Normal file
View File

@ -0,0 +1,2 @@
require "vagrant-spec/acceptance"
require_relative "shared/context_virtualbox"

5
test/acceptance/shared/context_virtualbox.rb Normal file
View File

@ -0,0 +1,5 @@
shared_context "provider/virtualbox" do
let(:extra_env) {{
"VBOX_USER_HOME" => "{{homedir}}",
}}
end

7
vagrant-spec.config.example.rb Normal file
View File

@ -0,0 +1,7 @@
require_relative "test/acceptance/base"
Vagrant::Spec::Acceptance.configure do |c|
c.provider "virtualbox",
box_basic: "<PATH TO MINIMAL BOX>",
contexts: ["provider/virtualbox"]
end