Rerun failed cucumber scenarios with formatting options

 

Some setup: We are a data warehouse line using Jruby/Cucumber to write acceptance tests for our ETL process. This end-to-end process involves kicking off ~15 Informatica workflows in succession. I mention this only so that anyone familiar with Informatica will understand why when I say that running a single Cucumber scenario takes approximately 3 minutes. Put 20 of these in a feature, and the run is now an hour long.

We use Jenkins as a CI server to run our cucumber tests nightly. Currently, if we have a scenario fail, we go in and fix the offending code, then have to kick off the entire job again, waiting up to an hour for feedback. What we are looking for is a way to kick off a job in jenkins using an optional parameter which will only re-run the scenarios which failed from the previous run, giving us faster red-to-green turn around.

Looking at some other questions around re-running failed scenarios, I was able to come up with the following Rake task: 

Cucumber::Rake::Task.new(:task_name) do |t|
  rerun_file = "temp/task_name_rerun.txt"

  tags = "--tags ~@wip --tags @task_name"

  html_output = "--format html --out features/build_results/task_name.html"
  junit_output = '--format junit --out features/build_results/reports'
  rerun_output = "--format rerun --out #{rerun_file}"

  outputs = [html_output, junit_output, rerun_output]

  if (ENV['Rerun'] == 'TRUE')
    t.cucumber_opts = "@#{rerun_file} #{html_output}"
  else
    t.cucumber_opts = "--format pretty #{tags} #{outputs.join(' ')}"
  end
end

When I run the first time with Rerun = FALSE, it works beautifully, and creates the task_name_rerun.txt file exactly as expected. However, when I then turn on Rerun and start it up again, it acts as if I passed it no scenarios whatsoever, giving this output: 

C:jruby-1.7.20binjruby.exe --1.9 -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) C:jruby-1.7.20bin/rake task_name
io/console not supported; tty will not be manipulated
C:/jruby-1.7.20/bin/jruby -S bundle exec cucumber @temp/task_name_rerun.txt --format html --out features/build_results/task_name.html
io/console not supported; tty will not be manipulated
io/console not supported; tty will not be manipulated

Process finished with exit code 0

When I alter the options to simply be 

t.cucumber_opts = "@#{rerun_file}"

it does find the scenario, but now it can't find the associated steps: 

C:jruby-1.7.20binjruby.exe --1.9 -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) C:jruby-1.7.20bin/rake task_name
io/console not supported; tty will not be manipulated
C:/jruby-1.7.20/bin/jruby -S bundle exec cucumber @temp/task_name_rerun.txt
io/console not supported; tty will not be manipulated
io/console not supported; tty will not be manipulated
@task_name
Feature: My Feature

  Scenario: One should Equal One   # features/scenarios/extended/path/my_test.feature:4
    Given I have a passing test    # features/scenarios/extended/path/my_test.feature:5

1 scenario (1 undefined)
1 step (1 undefined)
0m0.180s

You can implement step definitions for undefined steps with these snippets:

Given(/^I have a passing test$/) do
  pending # Write code here that turns the phrase above into concrete actions
end


Process finished with exit code 0

So, my question is somewhat twofold:

1) Is it not possible to use cucumber with a rerun file and also apply other formatting? Every other related question I found involved an immediate rerun with the Rake task, so the build report for the failing scenario would still get published to Jenkins. However, since I want to kick it off again performing only a rerun, If Jenkins cannot find a recent html report, if will mark the job as unstable, even if all scenarios went green. I would also ideally like to output the failures from the rerun again, so that if say 10 scenarios fail initially, we fix 5 of them and rerun, then the rerun.txt file will only contain the 5 most recent failures.

2) I'm not sure why the rerun suddenly can't find my step definitions. My current speculation is because my feature file is not directly in features/scenarios/. This has never given us a problem before when running based on tags, so I don't see why calling out specific features with scenario line numbers should suddenly break that. I have not yet verified if moving the file works however.

EDIT : I solved the problem with the second question. Above I used a bunch of dummy file/folder names for posting the question, in our actual test suite the nested folder name has spaces in it. Once I deleted the Spaces it was able to run the test normally.

Some extra details that might help:

  • Jruby version: 1.7.20
  • Cucumber version: 2.0.0
  • running on: Windows 7

    • It turns out the solution to this was two-fold. One problem was that some of our actual folder names had spaces in them. Apparently Cucumber doesn't care about spaces when running using tags, but fails when passed in via file (which makes perfect sense). Renaming all of the folders solved the second issue where it wouldn't run the second time.

    The solution to the First problem of not being able to rerun using additional formats and outputs was solved by upgrading to the latest version of Cucumber (2.0.2). I was not able to track it down to a specific change log, but as soon as I upgraded it started working with no other changes.

    链接地址: http://www.djcxy.com/p/38784.html

    上一篇: Ruby on Rails。 捆绑。 黄瓜。 耙子中止了！ Command failed with status (1)

    下一篇: 使用格式化选项重新运行失败的黄瓜场景