• If you are new to Mozilla or Treeherder, read the A-Team Bootcamp.
• Install Git
• Clone the treeherder repo from GitHub.

If you only want to hack on the frontend, see the UI Development section below. If you want to hack on the backend or work full-stack, see the Server and Full-stack Development section.

Before pushing new code, please make sure you are following our Code Style and Accessibility Guidelines.

If you would like pre-commit linting checks you can set it up like this:

% pip install pre-commit
% pre-commit install
pre-commit installed at .git/hooks/pre-commit

From here on, linting checks will be executed every time you commit.

To get started:

• Install Node.js and pnpm (see package.json for known compatible versions, listed under engines and packageManager).
• Run pnpm install to install all dependencies.
• Run pnpm build to build necessary files.

The default development server runs the unminified UI and fetches data from the production site. You do not need to set up the Docker environment unless making backend changes.

• Start the development server by running:

  pnpm start:stage

  !!! note We recommend developing against stage to avoid accidentally affecting the live production front-end of Treeherder (e.g. classifying a job). If you need production data specifically, you can use pnpm start instead, which proxies to production.

• The server will perform an initial build and then watch for new changes. Once the server is running, you can navigate to: http://localhost:5000 to see the UI.

To get started:

• Install [Docker] (Docker Compose is included with Docker Desktop for Windows/Mac; on Linux, install the Docker Compose plugin).

• If you just wish to run the tests, you can stop now without performing the remaining steps.

By default, data will be ingested from autoland and try using a shared pulse account. However, if you want to use your own pulse account or change the repositories being ingested, you need to export those env variables in the shell first or inline with the command below. See Pulse Ingestion Configuration for more details.

• Open a shell, cd into the root of the Treeherder repository, and type:

  docker compose up --build

• Wait for the Docker images to be downloaded/built and container steps to complete.

• Visit http://localhost:5000 in your browser (NB: not port 8000).

Both Django's runserver and rspack-dev-server will automatically refresh every time there's a change in the code. Proceed to Running the ingestion tasks to get data.

If you would like to use the minified production version of the UI with the development backend:

• Run the build task:

  docker compose run frontend sh -c "corepack enable && pnpm install && pnpm build"

• Start Treeherder's backend:

  docker compose up --build

• Visit http://localhost:8000 (NB: port 8000, unlike above)

Requests to port 8000 skip rspack-dev-server, causing Django's runserver to serve the production UI from .build/ instead. In addition to being minified and using the non-debug versions of React, the assets are served with the same Content-Security-Policy header as production.

Proceed to Running the ingestion tasks to get data.

If you want to develop both the frontend and backend, but have the database pointing to an external DB host, you have a few choices. The environment variable of DATABASE_URL is what needs to be set. You can do this in a file in the root of /treeherder called .env:

DATABASE_URL=psql://user:password@hostname/treeherder

Alternatively, you can export that value in your terminal prior to executing docker compose up or just specify it on the command line as you execute:

DATABASE_URL=psql://user:password@hostname/treeherder SKIP_INGESTION=True docker compose up

!!! note If you are using a database on one of our instances (production, stage or prototype) then you should also disable data ingestion via Pulse. It will ONLY ingest to your local DB, even if DATABASE_URL is set. But it will use your system's resources unnecessarily. To skip data ingestion, set the var SKIP_INGESTION=True

The Postgres database is kept locally and is not destroyed when the Docker containers are destroyed. If you want to start from scratch type the following commands:

docker compose down
docker volume rm treeherder_postgres_data

Celery tasks include storing of pushes, tasks and parsing logs (which provides failure lines and performance data) and generating alerts (for Perfherder). You can either run all the queues or run only specific queues.

Open a new shell tab. To run all the queues type:

docker compose run -e PROJECTS_TO_INGEST=autoland backend celery -A treeherder worker --concurrency 1

!!! note If you skip the PROJECTS_TO_INGEST flag, this command will store pushes and tasks from all repositories and will take a very long time for data to load. However, you can change it to ingest from other repositories if you wish.

You can find a list of different celery queues in the the CELERY_TASK_QUEUES variable in the settings file. For instance, to only store tasks and pushes and omit log parsing:

docker compose run -e PROJECTS_TO_INGEST=autoland backend celery -A treeherder worker -Q store_pulse_tasks,store_pulse_pushes --concurrency 1

!!! note You have to include --root-url https://community-tc.services.mozilla.com in order to ingest from the Taskcluster Community instance, otherwise, it will default to the Firefox CI.

Open a terminal window and run docker compose up. All following sections assume this step.

!!! note Only the push information will be ingested. Tasks associated with the pushes will not. This mode is useful to seed pushes so they are visible on the web interface and so you can easily copy and paste changesets from the web interface into subsequent commands to ingest all tasks.

These steps should help you set up everything you need to ingest data locally while working on perfherder.

!!! note This step is optional and used if you want to ingest from pulse messages.

export PULSE_URL=amqp://USER:PASSWORD@pulse.mozilla.org:5671/?ssl=1
pnpm install

docker compose up --build
docker compose run -e PROJECTS_TO_INGEST=autoland backend celery -A treeherder worker --concurrency 1

You can use any database viewer of your choice (e.g. dbeaver-ce, mysql workbench, etc.)

Connect to the following while docker compose up --build from the previous step is running:

Serverhost: localhost Port: 5432 Database: treeherder Username: postgres Password: mozilla1234

!!! note These commands perform fetches for the data, they are run sequentially in the same window. The first command makes the second one run faster.

docker compose exec backend ./manage.py ingest push -p autoland -r 1ee42a54a431acdd6cbe43b49de0237fe67eddd9

Warning: This command can take a long time to ingest and parse everything.

docker compose exec backend ./manage.py ingest push -p autoland -r 1ee42a54a431acdd6cbe43b49de0237fe67eddd9 -a --enable-eager-celery

--enable-eager-celery triggers the log parsing which is required to capture the PERFHERDER_DATA output.

docker compose exec backend ./manage.py ingest push -p autoland --last-n-pushes 100

docker compose exec backend ./manage.py ingest task -p autoland -r 1ee42a54a431acdd6cbe43b49de0237fe67eddd9 --task-id <TASK-ID> --enable-eager-celery

docker compose exec backend ./manage.py ingest git-push -p servo-try -c 92fc94588f3b6987082923c0003012fd696b1a2d
docker compose exec -e GITHUB_TOKEN=<foo> backend ./manage.py ingest git-pushes -p android-components

!!! note You can ingest all tasks for a push. Check the help output for the script to determine the parameters needed.

!!! note If you make too many calls to the Github API you will start getting 403 messages because of the rate limit. To avoid this visit your settings and set up GITHUB_TOKEN. You don't need to grant scopes for it.

!!! note This will only ingest the commits if there's an active Github PRs project. It will only ingest the commits.

docker compose exec backend ./manage.py ingest pr --pr-url https://github.com/mozilla-mobile/android-components/pull/4821

This will work if the push associated to the task exists in the database.

# Make sure to ingest 1bd9d4f431c4c9f93388bd04a6368cb07398f646 for autoland first
docker compose exec backend ./manage.py ingest task --task-id KQ5h1BVYTBy_XT21wFpLog

Continue to Working with the Server section after looking at the Code Style doc.