HackerOne Researcher API - Introduction

The HackerOne researcher API can be used to retrieve ‘Program’ objects, and what it terms their ‘Structured Scope’. The structured scope contains in-scope assets, many of which will be websites.

Project Overview

The problem I was trying to solve is as follows. There is a security vulnerability, which is already in the public domain, in a popular website technology. While research is ongoing, I won’t name it, but I want to find open bug bounty programmes where this vuln is exploitable.

The first step was to extend an existing HackerOne API client. The second was to use it to retrieve a big list of in-scope assets with open bug bounty programmes. Next, I wrote a scanning tool that could find a fingerprint of the vulnerable technology I was looking for on websites. The final step, in progress at time of writing, is to manually test the resulting websites to look for the vulnerability I am hunting.

Building Bug Bounty Tools in Go

HackerOne API Client Written in Go

The first step is to enumerate a list of all websites with a bug bounty on HackerOne. Building on the excellent contributions of Liam Galvin (https://lia.mg/), who wrote an initial HackerOne researcher API client, I have built an extended HackerOne API client with support for the Structured Scope endpoint, essential for finding actual target assets.

Automating Target Retrieval on HackerOne in Go

The next step was to build another tool: the HackerOne Target Retrieval Tool (github). This tool uses the API client to enumerate a massive list of all websites that are in scope in any HackerOne bug bounty programme. Because some of these are not relevant, or relate to programmes which are closed, the tool provides a filtering mechanism. It can be used like this:

// Only show HackerOne Programmes which are open
programmeIsRelevant := func (prog h1.Programme) bool {
    return prog.SubmissionState == "open"
}

// Only show Targets (within a Programme) which are websites where a bug bounty is available
targetIsRelevant := func (target h1.Target) bool {
    return target.AssetType == "URL" && target.EligibleForBounty
}

// Now you have a bug bounty Programme filter and a Target filter
filter := h1.NewFilter(programmeIsRelevant, targetIsRelevant)

// Get all the relevant targets from the API
targetRetriever := h1.NewTargetRetriever("username", "api-token", output, filter)
targetRetriever.RetrieveTargets()

At time of writing, there are around 2,000 web applications which have an open, remunerated bug bounty programme in HackerOne.

Because this project is written in Go, I was able to use goroutines to parallelise the processing. A single thread hits the HackerOne API, and gets a list of all the Programs. A workerpool of multiple (~100) workers using the API client then request the Structured Scope for each of the Programs. There were about 450 open programmes, and 2k websites in scope and eligible for financial reward if hacked.

Fingerprinting Vulnerable Technologies

The vulnerable technology I’m hunting for leaves a fingerprint - a sign on the web application that it is being used. Another Go tool (which for the time being needs to stay private, but I’ll publish it when I can) can scan a website to detect this fingerprint. Out of 2,000 websites with active bug bounties, a very small number are using the potentially vulnerable technology.

This part of the project in particular has been a wonderful learning opportunity. A person who is an engineering manager at Google kindly volunteered to do a code review with me, and I now have a much better understanding of practical approaches to Go concurrency.

The overall architecture is a fan-out/fan-in approach, where a single goroutine reads from the input .csv file of potential targets, and another goroutine writes to the output file. In the middle, there are about 100 HTTP workers querying the potential target web applications, and using a message queue to pass HTTP status codes and HTML to the fingerprint checker component.

Manual Testing of Potentially Vulnerable Assets

Manual testing for the actual vuln is still in progress. As it is one which has been publicly known about for a while (albeit without a patch), it’s possible that some sites have already been monkey-patched without a new deployment of the underlying library. Time and hard work will tell.

Further Steps

This is only the beginning of what can be done in the area of bug bounty automation. Ideas for further work include:

  • Significantly expanding the detection of fingerprinting of different frameworks and programming languages
  • Notifications when new in-scope assets become available to be hacked
  • Automating the actual detection of vulnerabilities

The author hopes Go bug bounty hacking tools that have been published will be of use to other security researchers, and wishes you happy hacking!