Get Fuzzing: An Introduction to Advanced Software Testing for Java and JavaScript

Fuzz testing

There are many techniques for software testing. However, if you haven’t heard of or used fuzzing, also known as “fuzz testing”, this may be the best post you have ever read. This article will explain what fuzzing is, from its inception in the late 1980s to its evolution to be “smart” today. After that, this article will explain how to use smart fuzzers like Jazzer and Jazzer.js to fuzz Java and JavaScript code, respectively. This article is not meant to make you a fuzzing expert but to introduce you to fuzzing and a few simple test cases to try. If you really want to be an expert in fuzzing for software testing your code, then you will want to read the docs for Jazzer and Jazzer.js and, most importantly, try it yourself! Please feel free to contact me with any questions, and I will do my best to support you. 

hEmuF32IzURe P wU HTzcFvtKIVi7QSdIepz97D8yIaKZQMaG
Photo of a tube television with static, or “fuzz”. Sitting on the TV is someone wearing red, patent leather high heels.

SOURCE: Photo by Isi Parente on Unsplash

A Fuzzy History

In the late 1980s, fuzzing was invented. This rudimentary form of fuzzing was akin to brute forcing. That is, the fuzzer injected random inputs into the software under test. It’s not much different than a monkey on a keyboard, or, as Antonio Nappa and Eduardo Bláquez describe in their book, Fuzzing Against the Machine, fuzzing is like giving something to a toddler or small child; that is, eventually, through the shear chaos of play and curiosity, the child will find a way to break the thing. Old school fuzzers worked like this, but purely randomly. It’s not much different than brute forcing. 

DTSghvjhNYBlDN3LQkQQ5wI qP akMtQs6NhkWzS60DTaar5pdDXMgHP4VcsO fH7Qd8LS7PvYD97ZjJPhMSV6oI9NX2wBcqd5
Child using a tablet computer. Fuzzing is like giving a child something; given enough time, the child will find a way to break it.

SOURCE: Photo by Kelly Sikkema on Unsplash.

Fuzzers work by crashing software under test. A simple acronym that I use to remember what fuzzers do is RUI. Fuzzers use RUI (Random, Unexpected, and Invalid) inputs to test and ultimately crash software. As you can imagine, using pure randomness is not an efficient way to test software because there are infinite possibilities. Since the invention of fuzzing, fuzzers have evolved considerably. 

In 2014, a security researcher at Google, Michal Zalewski, created AFL, or “American Fuzzy Lop,” a smarter way to fuzz. Compared to other fuzzers, AFL “employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover clean, interesting test cases that trigger new internal states in the targeted binary” (emphasis mine). To simplify that, these new fuzzers instrument the code, so that they can produce coverage metrics. Additionally, the genetic algorithm in the fuzzer helps the fuzzer make the best possible guess as to what will be an acceptable input for the software under test. The fuzzer then modifies its inputs (test cases) to look for other code paths and novel crashes in order to create a better test case to crash the app. The fuzzer AFL and its actively maintained counterpart, AFL++, as well as the many variations and customizations of both, are both tools at your disposal for testing software. They are optimal if you cannot access the source code but only the binary. You can use AFL/AFL++ for security research and bug hunting. 

Feedback Based Fuzzing

Fuzzing evolution didn’t stop there. It evolved to become “smart.” This means that these fuzzers are seeded with a valid input. The fuzzer then evolves versions of this input that are similar but different. The fuzzer does this at scale, creating hundreds to as many as thousands of test cases a second—even on a store bought laptop you can achieve this scale. Furthermore, with cloud resources, you can scale the number of test cases by several orders of magnitude. The fuzzer learns from the software on the test, and modifies its test cases to be more likely to achieve a crash. Additionally, whenever these test cases crash the software, they are saved in a corpus. The fuzzer re-uses and modifies the test cases in the corpus to go deeper into the code. These type of feedback based, smart fuzzers require access to the source code. A good example is a fuzzer in Brave Software’s adblock feature. The maintainers built a fuzzer to make the adblock feature in the browser less buggy and achieved more robustness.

Fuzz Java and JavaScript with Jazzer and Jazzer.js

Fuzzing can happen in a variety of stages, unit test, integration tests, or system/module testing. The later you get into the testing phase, the more complex it can be to write fuzz tests and mock the corresponding dependencies. For this introductory article, I will explain how to convert unit tests into fuzz tests. Fuzzing at the unit level is highly effective, because fuzzing is nondeterministic and will generate thousands of test cases a second, right at the unit level. 

I will bypass the basic download and setup instructions for Jazzer and Jazzer.js, because you can read the docs

0AM07oGAEk1P5k7EHdRg3 o0Z4TWTR3zymiTuaxPM0hk30q9KYW1iXDw6t36Uyhf2E4dJ2UuPpwVWuFy1M qkuDjLIW8bhJyAaCAq6ngWl9uZ3hAIwyLZHmA01iZJsJIfQPMz6m1StuGl4Ii9YzjUKU
Photo shows a porcelain coffee cup with the logo for “Java” on it. It is filling with coffee from an espresso machine.

SOURCE: Photo by Michiel Leunens on Unsplash 

Fuzzing Java via JUnit

If you write Java code, and use JUnit, then this will give you a basic idea how to convert your JUnit unit tests into fuzz tests, and fuzz test software. 

JUnit tests use the “@Test” abbreviation. With Jazzer, you will use “@FuzzTest”. As outlined in the docs and Jazzer’s maintainers demos, you will use logic similar to @Test. Thus, a sample is: 

@FuzzTest

 void fuzzTest(FuzzedDataProvider data) {

      String input = data.consumeRemainingAsString();

      String encoded = SomeScheme.encode(input);

      assertEquals(input, SomeScheme.decode(encoded));

 }

Another alternative that you can use is the autofuzz feature. You specify the .jar file you want to fuzz in the target java library. The command you will need for autofuzz is: 

jazzer --cp=commons-compress-1.20.jar

--autofuzz = org.apache.commons.compress.archivers.sevenz.SevenZFile::new

To run this example, you can use docker and this command. 

docker run -it cifuzz/jazzer-autofuzz 

org.apache.commons:commons-compress:1.20 

org.apache.commons.compress.archivers.sevenz.SevenZFile::new 

--autofuzz_ignore=java.nio.file.InvalidPathException,java.lang.NullPointerException

These examples give you some ways to setup fuzzing, using JUnit or with autofuzz targeting a .jar file in a java library. Of course, nothing is ever that easy, so as I emphasized before, check out the docs. 

Fuzzing JavaScript with Jazzer.js

Install Jazzer.js and read the docs. Once you have setup Jazzer.js, you can create your fuzz target, like this: 

// file "FuzzTarget.js"

module.exports.fuzz = function (data /*: Buffer */) {

const fuzzerData = data.toString();

myAwesomeCode(fuzzerData);

};

You can run the fuzzer with this command: npx jazzer FuzzTarget

If you want to use the Jest test framework, follow these steps. You will need to write a fuzz test, a TypeScript example looks like this: 

// file: "Target.fuzz.ts"

import "@jazzer.js/jest-runner";

import * as target from "./target";


describe("Target", () => {

        it.fuzz("executes a method", (data: Buffer) => {

                target.fuzzMe(data);

        });

});

A fuzz target would look like this:

// file: "Target.fuzz.ts"

import "@jazzer.js/jest-runner";

import * as target from "./target";

describe("Target", () => {

it.fuzz("executes a method", (data: Buffer) => {

target.fuzzMe(data);

});

});
IjTSSUWGRSkYgSWbtvvYzGVs3DdSevlISGjBaDpkf9maY47BeNog bF7pjsnHsu3LB402B0A1rCh8s47TxBJWvfSgxozfCISD

Quick note: Jazzer and Jazzer.js are both open source. Jazzeer is also integrated into Google’s OSS-Fuzz, so you can use it there as well. Jazzer is also integrated into the semi-open source tool cifuzz which makes the setting setup even easier. 

Conclusion

Fuzzing is an advanced software testing technique that you can use for JavaScript and Java. I have explained the history and benefits of using white box fuzzing, and explained a brief setup for Jazzer and Jazzer.js for java and JavaScript, respectively. Using this approach will help you generate and test thousands of test cases every second to find bugs and security vulnerabilities in your code. The larger technology of fuzzing has many practical applications as well, that you can explore. This article should have given you an introduction to the subject at large.

David Merian

Written by

David Merian is a prolific writer with over 8 years of experience in the field of cybersecurity. From his early days in sales to his current focus on cyber topics, he shares insights on cybersecurity, business, and sales. David advocates for a “Minimum effort, maximum effect” approach, encouraging easy deployment of fuzz testing, scaling cybersecurity impact, and optimizing resources. Grateful for mentors and passionate individuals in the field, he invites collaboration to help others achieve cybersecurity efficiency.

Leave a Reply

Your email address will not be published.

Related Posts

Testing Courses at Thrive EdSchool

Advertisement (Know More)

Get Top Community News

    Top Event and other The Test Tribe updates to your Inbox.

     

    Categories

    Tags