Automating API security tests in CI/CD for Java applications

API security testing is software testing performed on APIs. It is meant to identify vulnerabilities in API endpoint communication and access. In modern software development, API security is a crucial aspect that cannot be ignored.

API security testing can now be automated in CI/CD, enabling early detection of vulnerabilities, maintaining security standards without slowing down development, and reducing human errors. It also increases team efficiency by reducing repetitive tasks and ensuring applications are more secure and reliable.

In this article, we will discuss how to automate API security testing in a CI/CD pipeline for Java applications so that development teams can detect and fix potential security issues early on in development without hampering productivity.

Prerequisite

To follow along with this tutorial, you will need the following:

• Basic knowledge of Java language.
• A CircleCI account.
• A GitHub account.
• Mocking API : Wiremock.
• REST Assured.
• JUnit: Understand the unit testing flow with JUnit.

Security testing vs. API security testing: Key differences

There are developers who are still confused about the difference between security testing and API security testing.

Security testing includes various methods to ensure that applications (both frontend, backend, and API) are safe from vulnerabilities. This can include SAST and DAST, penetration testing, and system-wide security analysis.

API security testing is a subcategory of security testing that is specific to APIs. This testing is run at runtime, after the API is available.

Ensuring API security: Key scenarios

In the API security testing process, it is important to ensure that the API does not have any loopholes that can be exploited by attackers.

Attacks such as data compromise, unauthorized access, and code vulnerability exploitation often occur due to weak security. Therefore, testing should cover a variety of scenarios, from authentication and input validation to protection against injection attacks and data leaks.

This tutorial focuses on how to automate testing on CircleCI.

Choosing the right tools

Choosing the right tools for API security testing depends on several factors, such as the type of testing you want to do, the need for integration with the CI/CD pipeline, the scale of the application being tested, and of course the budget you have. Some commonly used API security testing tools:

• OWASP ZAP (Zed Attack Proxy)
• Burp Suite
• Postman
• REST-Assured

OWASP ZAP (Zed Attack Proxy)

OWASP ZAP is a widely used web application security testing tool. It is an open-source project maintained by OWASP (Open Web Application Security Project).

The advantages of this tool include an easy-to-use user interface, a wide range of plugins for in-depth testing, and support for both automated and manual testing.

If you are looking for a powerful and flexible open-source tool with a large community, OWASP ZAP could be a great choice.

Burp Suite

Burp Suite is one of the most popular web application security testing tools among security professionals. It is primarily designed for penetration testing.

Burp Suite offers many advanced features, including automated and manual security testing, a vulnerability scanner, and intrusion testing tools.

If you need a highly sophisticated tool and have the budget for a professional license, Burp Suite is a strong option. While a community version is available, it comes with limited features.

Postman

Postman is primarily an API development and testing tool, but it also includes features for basic security testing.

It is easy to use, supports automation scripting, and integrates well with CI/CD pipelines.

If you are already using Postman for API development and want to incorporate basic security testing, this tool can be a convenient alternative.

REST-Assured

REST-Assured is a Java library designed for RESTful API testing.

It is well integrated into the Java ecosystem, making it easy to write API tests in Java and automate testing workflows.

If you are working within the Java ecosystem and require seamless integration with your Java projects, Rest-Assured is the ideal choice. It is commonly used in combination with JUnit for unit testing.

There are other API security testing tools available, but they won’t be included in detail in this tutorial.

Here is a comparison table between OWASP ZAP, Burp Suite, Postman, and REST-Assured based on several key aspects.

Feature	OWASP ZAP	Burp Suite	Postman	REST-Assured
Type	Security Testing	Security Testing & Penetration Testing	API Development & Testing	API Testing Library
License	Open-source	Free (Community) & Paid (Professional)	Free & Paid	Open-source
Main Use Case	Web App Security Testing	Web App Security & Penetration Testing	API Development & Basic Security Testing	API Testing in Java
Ease of Use	User-friendly UI	Advanced, requires learning	Very easy to use	Requires coding skills
Automation	Yes, supports scripting	Yes, advanced automation	Yes, integrates with CI/CD	Yes, integrates with Java frameworks
Integration	Supports various integrations	Works with other security tools	Integrates with CI/CD & API platforms	Works with JUnit, TestNG, and CI/CD
Vulnerability Scanning	Yes	Yes (more advanced)	Basic security checks	No built-in vulnerability scanning
Customization	Supports plugins & scripting	Highly extensible with extensions	Limited	Highly customizable with Java code
Best For	Security researchers & developers	Penetration testers & security professionals	API developers with some security focus	Java developers needing API security tests

This table shows that REST-Assured is suitable for performing API security testing in Java-based projects.

Cloning the demo project

To learn how API security testing works, you will use a sample application that consumes an API consisting of some endpoints.

Clone the project and choose the branch named starter:

git clone --branch starter https://github.com/CIRCLECI-GWP/automation-api-security-test.git

Because this is a Gradle project, it has a directory structure like this:

automation-api-security-test/
├── src/
│   ├── main/java/                # Main Source Code
│   ├── test/java/                # Unit test
├── build.gradle                  # Gradle Config
├── settings.gradle

Now you can open and run this project with your favorite IDE. In this tutorial I use Intellij IDEA.

Mocking API for simulation

Because this is only a simulation, you can use a dummy API like JSONPlaceholder. However, JSONPlaceholder has limitations, such as static predefined data, lack of authentication support, and no ability to customize responses dynamically.

If you want to avoid relying on third-party tools and need more flexibility, you can create a mock API using WireMock.

Let’s say you have the following API endpoints:

• /private-endpoint
• /posts
• /rate-limited-endpoint
• /users/1

Setting up Wiremock

If using Maven, add the WireMock dependency in the pom.xml file:

<dependency>
  <groupId>org.wiremock</groupId>
  <artifactId>wiremock</artifactId>
  <version>3.11.0</version>
  <scope>test</scope>
</dependency>

This tutorial uses Gradle:

testImplementation 'org.wiremock:wiremock:3.11.0'

The cloned project already contains a class named ApiSecurityUnitTest.java in the src/test/java directory. This class will be used to write unit tests for API security testing. Let’s define a static setup() method to initialize the WireMock server on port 8080.

Open the ApiSecurityUnitTest.java file and replace its content with the following:

package id.web.hangga;

import org.junit.jupiter.api.BeforeAll;

import com.github.tomakehurst.wiremock.WireMockServer;

public class ApiSecurityUnitTest {
   private static WireMockServer wireMockServer;

        @BeforeAll
        public static void setup() {
        wireMockServer = new WireMockServer(8080);
        wireMockServer.start();
        //...

        // Setup WireMock stubs
        setupStubs();
    }
}

Don’t forget to add the @BeforeAll annotation to mark a method that should be executed once before all test methods in a class execute, typically used to initialize resources required by all tests in JUnit, such as database connections or global configuration.

After all the tests are complete, stop the wiremock service by adding:

import org.junit.jupiter.api.AfterAll;

    @AfterAll
    public static void teardown() {
        // Stop WireMock server
        wireMockServer.stop();
    }

Create a setupStubs() method to setup various stubs in WireMock, which simulate API responses in application security testing. These stubs cover scenarios like:

• Unauthorized Access → Returns 401 Unauthorized if trying to access /private-endpoint without authentication.
• SQL Injection Test → Returns 400 Bad Request when accessing /posts with a specific pattern.
• XSS Test → Checks if the application handles script input correctly when receiving data from the request body.
• Rate Limiting Test → Simulates API quota limitations by changing the response status from 200 OK to 429 Too Many Requests.
• CSRF Protection Test → Returns 403 Forbidden if the CSRF token is invalid.
• Sensitive Data Exposure Test → Returns user information to test for data leakage.
• Security Headers Test → Adds CSP and HSTS in the response to test header security.

Add the code:

private static void setupStubs() {
  // Stub for unauthorized access
  stubFor(get(urlEqualTo("/private-endpoint"))
    .willReturn(aResponse().withStatus(401)
      .withHeader("Content-Type", "application/json")
      .withBody("{\"message\":\"Unauthorized\"}")));

  // Stub for SQL injection test
  stubFor(get(urlMatching("/posts.*"))
    .willReturn(aResponse().withStatus(400)
      .withHeader("Content-Type", "application/json")
      .withBody("[]")));

  // Stub for XSS test
  stubFor(post(urlEqualTo("/posts"))
    .withRequestBody(containing("<script>alert('XSS')</script>"))
    .willReturn(aResponse().withStatus(201)
      .withHeader("Content-Type", "application/json")
      .withBody("{\"body\":\"<script>alert('XSS')</script>\"}")));

  // Stub for rate limiting test
  stubFor(get(urlEqualTo("/rate-limited-endpoint"))
    .inScenario("Rate Limiting Scenario")
    .whenScenarioStateIs(STARTED)
    .willReturn(aResponse()
      .withStatus(200)
      .withHeader("Content-Type", "application/json")
      .withBody("[]"))
    .willSetStateTo("Rate Limit Exceeded"));

  stubFor(get(urlEqualTo("/rate-limited-endpoint"))
    .inScenario("Rate Limiting Scenario")
    .whenScenarioStateIs("Rate Limit Exceeded")
    .willReturn(aResponse()
      .withStatus(429)
      .withHeader("Content-Type", "application/json")
      .withBody("{\"error\":\"Too Many Requests\"}")));

  // Stub for CSRF protection test
  stubFor(post(urlEqualTo("/posts"))
    .withHeader("X-CSRF-Token", containing("invalid-token"))
    .willReturn(aResponse().withStatus(403)
      .withHeader("Content-Type", "application/json")
      .withBody("{\"error\":\"Invalid CSRF token\"}")));

  // Stub for missing authentication token
  stubFor(get(urlEqualTo("/private-endpoint"))
    .willReturn(aResponse().withStatus(401)
      .withHeader("Content-Type", "application/json")
      .withBody("{\"message\":\"Unauthorized\"}")));

  // Stub for invalid data test
  stubFor(post(urlEqualTo("/posts"))
    .withRequestBody(containing("-"))
    .willReturn(aResponse()
      .withStatus(403)
      .withHeader("Content-Type", "application/json")
      .withBody("{\"error\":\"Invalid input\"}")));

  // Stub for sensitive data exposure test
  stubFor(get(urlEqualTo("/users/1"))
    .willReturn(aResponse().withStatus(200)
      .withHeader("Content-Type", "application/json")
      .withBody("{\"email\":\"user@example.com\"}")));

  // Stub for security headers test
  stubFor(get(urlEqualTo("/posts")).willReturn(aResponse().withStatus(200)
    .withHeader("Content-Security-Policy", "default-src 'self'")
    .withHeader("Strict-Transport-Security", "max-age=31536000; includeSubDomains")
    .withHeader("X-Frame-Options", "DENY") // Cegah Clickjacking
    .withHeader("X-Content-Type-Options", "nosniff") // Hindari MIME-type sniffing
    .withHeader("Referrer-Policy", "no-referrer") // Batasi referrer leakage
    .withBody("[]")));

  // Stub for HTTPS only test (not applicable in WireMock setup, so omitted)
}

Include all the required imports:

import static com.github.tomakehurst.wiremock.client.WireMock.aResponse;
import static com.github.tomakehurst.wiremock.client.WireMock.containing;
import static com.github.tomakehurst.wiremock.client.WireMock.get;
import static com.github.tomakehurst.wiremock.client.WireMock.post;
import static com.github.tomakehurst.wiremock.client.WireMock.stubFor;
import static com.github.tomakehurst.wiremock.client.WireMock.urlEqualTo;
import static com.github.tomakehurst.wiremock.client.WireMock.urlMatching;
import static com.github.tomakehurst.wiremock.stubbing.Scenario.STARTED;

Now you have a mock API ready for simulation.

Using REST-Assured

API testing in Java has a reputation for being more difficult than other languages like Python or Ruby. That reputation is due to Java’s verbosity, lack of built-in libraries, dependency management complexity, and heavier testing frameworks. So REST-Assured brings the simplicity of using these languages into the Java domain.

This is how to use REST-Assured to test the security aspects of an API, from authentication to response validation against common attack scenarios.

Setting up REST-Assured

If you’re using Maven, add the REST-assured dependency in the pom.xml file:

<dependency>
  <groupId>io.rest-assured</groupId>
  <artifactId>rest-assured</artifactId>
  <version>5.5.0</version>
  <scope>test</scope>
</dependency>

While in Gradle, add this dependency:

testImplementation 'io.rest-assured:rest-assured:5.5.0'

Configuration

You need to set up the initial configuration before running all the tests in the class. Add to the setup() method that uses the @BeforeAll annotation you created:

@BeforeAll
public static void setup() {
  //...
  // Set the base URL for RestAssured to WireMock server
  RestAssured.baseURI = "http://localhost:8080";
  RestAssured.defaultParser = io.restassured.parsing.Parser.JSON;

  //...
}

Make sure that RestAssured is connected to the same url as WireMock, the mock server that simulates the API.

Using basic commands

To use REST-Assured in Java, we just need to understand these basic commands:

given()
  .when()
  .get("https://jsonplaceholder.typicode.com/posts/1")
  .then()
  .statusCode(200)
  .body("id", equalTo(1));

Explanation:

• given(): Prepares the request.
• when(): Defines the HTTP method (get, post, etc.).
• then(): Verifies the response (status code and body).

Writing test code

The next step is to write test code in unit test class. Each method must have an annotation in the form of @Test so that it can be run automatically by JUnit:

@Test
public void testUnauthorizedAccess() {
  given().when()
    .get("/private-endpoint")
    .then()
    .statusCode(401)
    .body("message", equalTo("Unauthorized"));
}

@Test
public void testSQLInjection() {
  String maliciousInput = "' OR '1'='1";

  given().param("q", maliciousInput)
    .when()
    .get("/posts")
    .then()
    .statusCode(400);
}

@Test
public void testXSS() {
  String xssPayload = "<script>alert('XSS')</script>";

  given()
    .body("{\"body\":\"" + xssPayload + "\"}")
    .when()
    .post("/posts")
    .then()
    .statusCode(201)
    .body("body", equalTo(xssPayload));
}

@Test
public void testRateLimiting() {
  for (int i = 0; i < 3; i++) {
    Response response = given()
      .when()
      .get("/rate-limited-endpoint")
      .then()
      .extract()
      .response();

    if (i > 0) {
      response.then().statusCode(429).body("error", equalTo("Too Many Requests"));
    } else {
      response.then().statusCode(200);
    }
  }
}

@Test
public void testCSRFProtection() {
  given().header("X-CSRF-Token", "invalid-token")
    .when()
    .post("/posts")
    .then()
    .statusCode(403)
    .body("error", equalTo("Invalid CSRF token"));
}

@Test
public void testMissingAuthenticationToken() {
  given().when()
    .get("/private-endpoint")
    .then()
    .statusCode(401)
    .body("message", equalTo("Unauthorized"));
}

@Test
public void testInvalidData() {
  given()
    .body("-")
    .when()
    .post("/posts")
    .then()
    .statusCode(403)
    .body("error", equalTo("Invalid input"));
}

@Test
public void testSensitiveDataExposure() {
  given().when()
    .get("/users/1")
    .then()
    .statusCode(200)
    .body("email", containsString("@example.com"));
}

@Test
public void testSecurityHeaders() {
  given().when()
    .get("/posts")
    .then()
    .statusCode(200)
    .header("Content-Security-Policy", notNullValue())
    .header("Strict-Transport-Security", notNullValue())
    .header("X-Frame-Options", equalTo("DENY")) // prevent Clickjacking
    .header("X-Content-Type-Options", equalTo("nosniff")) // avoid MIME-type sniffing
    .header("Referrer-Policy", equalTo("no-referrer")); // limiting referrer leakage
}

This code tests API security aspects such as authentication, SQL Injection, XSS, CSRF, rate limiting, and security headers, but still needs improvement. You can find the complete content of the ApiSecurityUnitTest.java file in the GitHub repo.

Try testing it locally first before automating it in CircleCI.

For real projects it is highly recommended that XSS tests should ensure the payload is not stored raw. Rate limiting tests are more accurate if there is a time lag. Sensitive data tests should ensure passwords or personal info are not exposed.

Automating the tests with CircleCI

CircleCI is a CI/CD platform that enables us to automate our tests. If you have followed the previous steps, you have the project ready to be integrated with CircleCI. Before that, let’s add the configuration file to automate the tests. Create a .circleci folder in the root directory and add a config.yml file with the following content:

version: 2.1
jobs:
  test-java:
    docker:
      - image: cimg/openjdk:17.0
    steps:
      - checkout
      - run:
          name: Calculate cache key
          command: |-
            find . -name 'pom.xml' -o -name 'gradlew*' -o -name '*.gradle*' | \
                    sort | xargs cat > /tmp/CIRCLECI_CACHE_KEY
      - restore_cache:
          key: cache-{{ checksum "/tmp/CIRCLECI_CACHE_KEY" }}
      - run:
          command: ./gradlew check
      - store_test_results:
          path: build/test-results
      - save_cache:
          key: cache-{{ checksum "/tmp/CIRCLECI_CACHE_KEY" }}
          paths:
            - ~/.gradle/caches
      - store_artifacts:
          path: build/reports
  deploy:
    # This is an example deploy job, not actually used by the workflow
    docker:
      - image: cimg/base:stable
    steps:
      # Replace this with steps to deploy to users
      - run:
          name: deploy
          command: "#e.g. ./deploy.sh"
workflows:
  build-and-test:
    jobs:
      - test-java
    # - deploy:
    #     requires:
    #       - test-java

This configuration file defines a CI/CD pipeline for a Java project. It includes a job named test-java that uses a Docker image with OpenJDK 17. The job performs steps such as checking out the code, calculating and restoring cache, running Gradle tests, storing test results, and saving cache.

Next, commit and push the changes to your GitHub repository. Review Pushing a project to GitHub for instructions.

Log into your CircleCI account. If you signed up with your GitHub account, all your repositories will be available on your project’s dashboard. Search for the automation-api-security-test project and click Set Up Project.

Select the branch you want to build and click Set Up Project. CircleCI detects the configuration file in the .circleci directory and starts the build process.

This should build successfully; you can review the test results in the artifacts.

The name of the unit test class in this example is ApiSecurityUnitTest.java. The report file name will contain that name.

Review build/reports/tests/test/classes/id.web.hangga.ApiSecurityUnitTest.html.

After viewing the artifacts, review the report.

The ApiSecurityUnitTest results are excellent, with 100% success across 9 tests, covering CSRF protection, token validation, rate limiting, SQL Injection, XSS, and security headers. The fast execution time (1.209s) indicates efficiency, with rate limiting and security headers taking slightly longer, likely due to additional verification. Overall, this reflects a strong API security implementation, though further testing with negative scenarios could enhance system resilience.

Conclusion

API security testing is a crucial aspect of software development, especially in ensuring the security of communication and access to APIs.

Choosing the right tools, such as OWASP ZAP, Burp Suite, Postman, or REST-Assured, depends on project requirements, integration level with CI/CD, and the complexity of the testing needed. For Java-based projects, REST-Assured is an ideal choice because of its strong integration with the Java ecosystem.

By automating API security testing within the CI/CD pipeline, development teams can detect and fix vulnerabilities early without slowing down productivity.

The code in this article can be accessed in this repo.