Run additional tests by using a feature flag to "cargo test"

16
votes

I have some tests that I would like to ignore when using cargo test and only run when explicitly passed a feature flag. I know this can be done by using #[ignore] and cargo test -- --ignored, but I'd like to have multiple sets of ignored tests for other reasons.

I have tried this:

#[test]
#[cfg_attr(not(feature = "online_tests"), ignore)]
fn get_github_sample() {}

This is ignored when I run cargo test as desired, but I can't get it to run.

I have tried multiple ways of running Cargo but the tests continue to be ignored:

cargo test --features "online_tests"

cargo test --all-features

I then added the feature definition into my Cargo.toml as per this page, but they continue to be ignored.

I am using workspaces in Cargo. I tried adding the feature definition in both Cargo.toml files with no difference.

1
testingrustrust-cargo
Have you tried just using #[cfg(feature = "online_tests")] on the respective tests? – Neikos
@Neikos the downside to that is that the tests won't even be compiled. If there was some compilation error, you wouldn't see it until those features were enabled. Also, you won't see that there are ignored tests in the test output, potentially reminding you to enable them. – Shepmaster
I understand that they need #[test] also and if I did that I understand that "cargo test" would run them no? And that's not what I want. – Andrew Mackenzie

1 Answers

13
votes

Without a workspace

Cargo.toml

[package]
name = "feature-tests"
version = "0.1.0"
authors = ["An Devloper <[email protected]>"]

[features]
network = []
filesystem = []

[dependencies]

src/lib.rs

#[test]
#[cfg_attr(not(feature = "network"), ignore)]
fn network() {
    panic!("Touched the network");
}

#[test]
#[cfg_attr(not(feature = "filesystem"), ignore)]
fn filesystem() {
    panic!("Touched the filesystem");
}

Output

$ cargo test

running 2 tests
test filesystem ... ignored
test network ... ignored

$ cargo test --features network

running 2 tests
test filesystem ... ignored
test network ... FAILED

$ cargo test --features filesystem

running 2 tests
test network ... ignored
test filesystem ... FAILED

(some output removed to better show effects)

With a workspace

Layout

.
├── Cargo.toml
├── feature-tests
│   ├── Cargo.toml
│   ├── src
│   │   └── lib.rs
├── src
│   └── lib.rs

feature-tests contains the files from the first section above.

Cargo.toml

[package]
name = "workspace"
version = "0.1.0"
authors = ["An Devloper <[email protected]>"]

[features]
filesystem = ["feature-tests/filesystem"]
network = ["feature-tests/network"]

[workspace]

[dependencies]
feature-tests = { path = "feature-tests" }

Output

$ cargo test --all

running 2 tests
test filesystem ... ignored
test network ... ignored

$ cargo test --all --features=network

running 2 tests
test filesystem ... ignored
test network ... FAILED

(some output removed to better show effects)

With a workspace with a virtual manifest

Virtual manifests do not support specifying features (Cargo issue #4942). You will need to run the tests from within the sub project or specify the path to the appropriate Cargo.toml

Layout

.
├── Cargo.toml
└── feature-tests
    ├── Cargo.toml
    └── src
        └── lib.rs

feature-tests contains the files from the first section above.

Cargo.toml

[workspace]
members = ["feature-tests"]

Output

$ cargo test --all --manifest-path feature-tests/Cargo.toml --features=network 

running 2 tests
test filesystem ... ignored
test network ... FAILED

$ cargo test --all --manifest-path feature-tests/Cargo.toml

running 2 tests
test filesystem ... ignored
test network ... ignored

(some output removed to better show effects)