---
title: Usage
author:
  - Brett Porter
  - Allan Ramirez
  - Stephen Connolly
date: 2011-06-27
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

# Usage

#{if}(${project.artifactId}=="maven-surefire-plugin")
Best practice is to define the version of the Surefire Plugin that you want to use in either your `pom.xml` or a parent `pom.xml`:
#{else}
To use the ${thisPlugin} Plugin, you need to add the following configuration to your `pom.xml`:
#{end}

```xml
<project>
  [...]
  <build>
#{if}(${project.artifactId}=="maven-surefire-plugin")
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>${project.groupId}</groupId>
          <artifactId>${project.artifactId}</artifactId>
          <version>${project.version}</version>
        </plugin>
      </plugins>
    </pluginManagement>
#{else}
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
#{end}
  </build>
  [...]
</project>
```

#{if}(${project.artifactId}=="maven-surefire-plugin")
The ${thisPlugin} Plugin can be invoked by calling the `test` phase of the build lifecycle.
#{else}
The ${thisPlugin} Plugin can be invoked by calling the `verify` phase of the build lifecycle.
#{end}

```text
#{if}(${project.artifactId}=="maven-surefire-plugin")
mvn test
#{else}
mvn verify
#{end}
```

Using Different Testing Providers
---------------------------------

Tests in your test source directory can be any combination of the following:

- TestNG (6.14.3 or later)
- JUnit (4.12+ or 5.x)

Since Surefire 3.6.0, all tests run via the JUnit Platform. The appropriate engine is automatically selected based on your dependencies:

- `junit:junit` (4.12+) - runs via Vintage Engine
- `junit-jupiter-engine` - runs via Jupiter Engine
- `org.testng:testng` (6.14.3+) - runs via TestNG JUnit Platform Engine

Since the test framework dependency is required to compile the test classes anyway, no additional configuration is required.

Note that any normal Surefire integration works identically no matter which test framework is in use - so you can still produce a Cobertura report and a Surefire results report on your project web site for your TestNG tests, for example.

There are additional options available if you are running TestNG tests.

See [ Using TestNG](./examples/testng.html) for more information.

#{if}(${project.artifactId}=="maven-failsafe-plugin")

Using jetty and ${project.artifactId}
-------------------------------------

You need to bind one of `jetty:start`, `jetty:run`, `jetty:run-exploded` or `jetty:run-war` to the `pre-integration-test` phase with `daemon` set to true, bind `failsafe:integration-test` to the `integration-test` phase, bind `jetty:stop` to the `post-integration-test` phase and finally bind `failsafe:verify` to the `verify` phase. Here is an example:

```xml
<project>
  [...]
  <build>
    [...]
    <plugins>
      [...]
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
        <executions>
          <execution>
            <id>integration-test</id>
            <goals>
              <goal>integration-test</goal>
            </goals>
          </execution>
          <execution>
            <id>verify</id>
            <goals>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>9.2.2.v20140723</version>
        [...]
        <configuration>
          [...]
          <scanIntervalSeconds>10</scanIntervalSeconds>
          <stopPort>8005</stopPort>
          <stopKey>STOP</stopKey>
          [...]
        </configuration>
        [...]
        <executions>
          [...]
          <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start</goal>
            </goals>
            <configuration>
              <scanIntervalSeconds>0</scanIntervalSeconds>
              <daemon>true</daemon>
            </configuration>
          </execution>
          <execution>
            <id>stop-jetty</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
          [...]
        </executions>
        [...]
      </plugin>
      [...]
    </plugins>
    [...]
  </build>
  [...]
</project>
```

You then invoke Maven with a phase of `verify` or later in order to run the integration tests. **DO NOT** directly invoke any of the phases `pre-integration-test`, `integration-test`, or `post-integration-test` as these are too long to type and they will likely leave a jetty container running.

```bash
mvn verify
```

Note: during test development, you will likely run a jetty instance in the background. to help running the integration tests, it can be handy to bind `jetty:stop` to the `pre-integration-test` phase before `jetty:run` to flush out any running jetty instance before starting the integration test jetty instance, e.g.

```xml
<project>
  [...]
  <build>
    [...]
    <plugins>
      [...]
      <plugin>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>9.2.2.v20140723</version>
        [...]
        <executions>
          [...]
          <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
              <!-- stop any previous instance to free up the port -->
              <goal>stop</goal>
              <goal>start</goal>
            </goals>
            [...]
          </execution>
          [...]
        </executions>
        [...]
      </plugin>
      [...]
    </plugins>
    [...]
  </build>
  [...]
</project>
```

Reporting integration test results
----------------------------------

The ${thisPlugin} Plugin uses the exact same format as the ${thatPlugin} Plugin, so to generate a report you just add a second Surefire Report Plugin report set using the ${thisPlugin} reports directory, e.g.

```xml
<project>
  [...]
  <reporting>
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>maven-surefire-report-plugin</artifactId>
        <version>${project.version}</version>
        <reportSets>
          <reportSet>
            <id>integration-tests</id>
            <reports>
              <report>failsafe-report-only</report>
            </reports>
          </reportSet>
        </reportSets>
      </plugin>
    </plugins>
  </reporting>
  [...]
</project>
```

Running integration tests multiple times
----------------------------------------

If you need to run your integration tests multiple times, just use multiple executions of the `integration-test` goal. You will need to specify a different summary file for each execution, and then configure the `verify` goal with the multiple summary files in order to fail the build when any one execution has failures, e.g.

```xml
<project>
  [...]
  <reporting>
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
        <executions>
          <execution>
            <id>integration-test-red-bevels</id>
            <goals>
              <goal>integration-test</goal>
            </goals>
            <configuration>
              <systemPropertyVariables>
                <bevels>red</bevels>
              </systemPropertyVariables>
              <summaryFile>target/failsafe-reports/failsafe-summary-red-bevels.xml</summaryFile>
            </configuration>
          </execution>
          <execution>
            <id>integration-test-no-bevels</id>
            <goals>
              <goal>integration-test</goal>
            </goals>
            <configuration>
              <systemPropertyVariables>
                <bevels>none</bevels>
              </systemPropertyVariables>
              <summaryFile>target/failsafe-reports/failsafe-summary-no-bevels.xml</summaryFile>
            </configuration>
          </execution>
          <execution>
            <id>verify</id>
            <goals>
              <goal>verify</goal>
            </goals>
            <configuration>
              <summaryFiles>
                <summaryFile>target/failsafe-reports/failsafe-summary-red-bevels.xml</summaryFile>
                <summaryFile>target/failsafe-reports/failsafe-summary-no-bevels.xml</summaryFile>
              </summaryFiles>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </reporting>
  [...]
</project>
```

Usage in multi-module projects
------------------------------

The recommendations for using the ${thisPlugin} Plugin listed at the top of this page are fine for 95% of use cases. When you are defining a shared definition of the ${thisPlugin} Plugin in a parent pom, it is considered best practice to define an execution id in order to allow child projects to override the configuration. Thus you might have the following in your parent `pom.xml`:

```xml
<project>
  [...]
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>${project.groupId}</groupId>
          <artifactId>${project.artifactId}</artifactId>
          <version>${project.version}</version>
          <executions>
            <execution>
              <id>integration-test</id>
              <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
  [...]
</project>
```

The child projects can then trigger usage of the ${thisPlugin} Plugin with

```xml
<project>
  [...]
  <build>
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
      </plugin>
    </plugins>
  </build>
  [...]
</project>
```

For very complex builds, it may be better to separate the executions for the `integration-test` and `verify` goals.

```xml
<project>
  [...]
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>${project.groupId}</groupId>
          <artifactId>${project.artifactId}</artifactId>
          <version>${project.version}</version>
          <executions>
            <execution>
              <id>integration-test</id>
              <goals>
                <goal>integration-test</goal>
              </goals>
            </execution>
            <execution>
              <id>verify</id>
              <goals>
                <goal>verify</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
  [...]
</project>
```
#{end}
