Connecting LAVA Lab to the pipeline instance

Connecting a LAVA lab to the KernelCI pipeline

As we are moving towards the new KernelCI API and pipeline, we need to make sure all the existing LAVA labs are connected to the new pipeline instance. This document explains how to do this.

Token setup

The first step is to generate a token for the lab. This is done by the lab admin, and the token is used to submit jobs from pipeline to the lab and to authenticate LAVA lab callbacks to the pipeline. The token is generated in Auth Tokens section under Linaro_Django_Xmlrpc.

Requirements for the token:

Description: a string matching the regular expression [a-zA-Z0-9\-]+, for example “kernelci-new-api-callback”
Value: arbitrary, kept secret

IMPORTANT! You need to have both fields, as that’s how LAVA works:

You submit the job with the token description in job definition
LAVA lab sends the result back to the pipeline with the token value (retrieved by that token-description) in the header

Pipeline configuration

Update pipeline configuration

The first step is to add the lab configuration to pipeline configuration file.

Please add a new entry to the runtimes section of the configuration file as follows:


  lava-broonie:
    lab_type: lava
    url: 'https://lava.sirena.org.uk/'
    priority_min: 10
    priority_max: 40
    notify:
      callback:
        token: kernelci-new-api-callback
        url: https://staging.kernelci.org:9100

Where lava-broonie is the name of the lab, lab_type indicates the lab is of a lava type, url is the URL of the lab, priority_min and priority_max are the priority range allowed to jobs, assigned by lab owner, and notify is the notification configuration for the lab. The callback section contains the token description that you received from the above step and the URL of the pipeline instance LAVA callback endpoint. More details on how LAVA callback and token works can be found in the LAVA documentation.

Please submit a pull request to kernelci-pipeline repository to add the lab configurations. See the pull request for reference.

KernelCI configuration (TOML) file

The next step is to add the token to the pipeline services configuration file i.e. config/kernelci.toml file. Every lab/runtime should have a section runtime.<lab-name> in the TOML file. The lab token should be stored in a key named runtime_token inside the section. For example,

[runtime.<lab-name>]
runtime_token="<lab-token-value>"

Section name lab-name should be replaced with the actual lab name, matching the name of the lab in the pipeline configuration i.e. config/pipeline.yaml. lab-token-value should be replaced with the actual token value that you received in the Token setup step. Usually, it is a long string of random characters. For example, in our documentation we used lava-broonie as the lab name, so the section will look like this:

[runtime.lava-broonie]
runtime_token="N0tAS3creTT0k3n"

If you want to connect your Lab to the official kernelci Pipeline under https://dashboard.kernelci.org, then you need to contact the KernelCI team. Have a look at https://foundation.kernelci.org/community-contact/ and see what needs to be done here.

`docker-compose` file

We are running all the pipeline services as docker containers. You need to provide lab name to --runtimes argument to the scheduler-lava service in the docker-compose.yml file to enable the lab. For example, the following configuration adds the lava-broonie lab along with other labs:

scheduler-lava:
    <<: *scheduler
    container_name: 'kernelci-pipeline-scheduler-lava'
    command:
      - './pipeline/scheduler.py'
      - '--settings=${KCI_SETTINGS:-/home/kernelci/config/kernelci.toml}'
      - 'loop'
      - '--runtimes'
      - 'lava-collabora'
      - 'lava-collabora-staging'
      - 'lava-broonie'

Jobs and devices specific to the lab

Now you want to describe to KernelCI the boards we have in your lab. Add the board under platforms.yaml, or see if they are already present.

platforms:
  sun50i-h5-libretech-all-h3-cc:
    <<: *arm64-device
    mach: allwinner
    dtb: dtbs/allwinner/sun50i-h5-libretech-all-h3-cc.dtb

Next, create a job under jobs.yaml

jobs:
  baseline-arm64-broonie: *baseline-job

The last step is to add some jobs that you want KernelCI to submit to the lab. For this create your scheduler job in scheduler.yaml with the conditions under which the job should be run:

scheduler:
  - job: baseline-arm64-broonie
    event:
      channel: node
      name: kbuild-gcc-10-arm64
      state: available
    runtime:
      type: lava
      name: lava-broonie
    platforms:
      - sun50i-h5-libretech-all-h3-cc

Jobs usually define tasks to be run such as kernel build or a test suite running on a particular device (platform). More details about pipeline configuration can be found in the pipeline configuration documentation (TBD).

Note We have lava-callback service that will receive job results from the lab and send them to the API.