Building JS for a Drupal Contrib Module

January 16, 2024

By: Kyle Einecker

RJSF is a bit of a unique module in that there is more JavaScript than php in the module. This presents some challenges when trying to host and package the module from a Drupal.org repository. Like that, a release on d.o is a copy of a specific commit and not a build artifact controlled by the repo. Because of this RJSF ships the JS part of the module as an NPM package that is only updated when a release is tagged. This has some significant downsides like if you want to use the module you need to install both the module and the npm package. Or to use the dev version of the module you have to build the JS yourself. Or to tag a release I have to manually package a release for NPM. Recently I've been thinking about how to simplify the installation and release process for RJSF and decided to start by attempting to include the built js with the module instead of as an NPM package.

The Goal

• Include built javascript when the module is installed via composer
• Update the built javascript whenever the source changes
• Do it all automatically without any manual steps

The How

With CI/CD of course. At DrupalCon Pittsburgh it was announced that Gitlab CI was being made generally available to projects on Drupal.org. So it's as simple as creating a .gitlab-ci.yml and scripting out the command like so:

1 build_js:
2   rules:
3     - if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "1.0.x"
4   script:
5     - npm install
6     - npm run build
7     - git add .
8     - git status
9     - git commit -m "CI: JS Build"
10    - git push

This was my first attempt and I ran into a couple of issues

• The correct version of npm needed to be installed
• The CI job didn't have permission to write to the repository

To get the correct version of npm installed in the ci container I added some steps to install nvm and with the addition of a .nvmrc file to the project, I can now be sure that CI is always using the correct version of node and npm during builds. As a benefit contributors with nvm will easily be able to use the right version.

1   - curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
2   - export NVM_DIR="$HOME/.nvm" && . "$NVM_DIR/nvm.sh" --no-use
3   - eval "[ -f .nvmrc ] && nvm install || nvm install ${NODE_VERSION}"
4   - node --version
5   - npm --version
6   - ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/node" "/usr/local/bin/node"
7   - ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/npm" "/usr/local/bin/npm"
8   - nvm install
9   - nvm use

Thanks to the answer on stackoverflow I was able to figure out the correct set of commands to get nvm usable. Without some of these lines, nvm would install but trying to use nvm resulted in a command not found error.

The job not being able to push a commit to the repository was a bit more confusing. Admittedly I've spent more time in GitHub Actions than in Gitlab CI but how could a job from the project not have permission to write back to that project? Turns out this is a known issue in Gitlab that the $CI_JOB_TOKEN does not have write permissions and they are considering a change to that behavior here. Until that is resolved the only way to push to a repository from CI is to create a Project Access Token. To do that I went to https://git.drupalcode.org/project/module_name/-/settings/access_tokens and added a new token scoped to write to the repository.

With the access token created I then added it to the CI environment by configuring a new variable ACCESS_TOKEN at https://git.drupalcode.org/project/module_name/-/settings/ci_cd. Since this token has write permission I made sure both the protected and masked boxes were checked. These will prevent the token from being exposed in the CI job logs and will ensure the token is only added to jobs for protected branches. In my case, I've protected the 1.0.x branch of the project.

Then it was just a matter of checking out the correct branch, setting the remote to use the project access token, and committing the changes.

1   - git config user.email "[email protected]"
2     - git config user.name "ci-bot"
3     - git checkout $CI_COMMIT_BRANCH
4     - git remote set-url origin https://oauth2:[email protected]/project/module_name.git
5     - git add .
6     - git status

One Last Thing

With those issues sorted my job was running successfully and everything was green. To green in fact. My first job ran but then it started another job and another and another. My job is configured to run anytime there is a push to the 1.0.x branch BUT the job itself pushes to the 1.0.x branch. I created an infinite loop of CI jobs.

With a quick little addition of an option to the push command, I was able to skip triggering ci jobs when committing the built javascript.

1 git push -o ci.skip

The Result

Putting it all together the final gitlab-ci job definition looks like:

1 build_js:
2   rules:
3     - if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "1.0.x"
4   script:
5     - export NVM_DIR="$HOME/.nvm" && . "$NVM_DIR/nvm.sh" --no-use
6     - eval "[ -f .nvmrc ] && nvm install || nvm install ${NODE_VERSION}"
7     - node --version
8     - npm --version
9     - ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/node" "/usr/local/bin/node"
10    - ln -s "$NVM_DIR/versions/node/$(nvm version)/bin/npm" "/usr/local/bin/npm"
11    - nvm install
12    - nvm use
13    - npm install
14    - npm run build
15    - git config user.email "[email protected]"
16    - git config user.name "ci-bot"
17    - git checkout $CI_COMMIT_BRANCH
18    - git remote set-url origin https://oauth2:[email protected]/project/search_web_components.git
19    - git add .
20    - git commit -m "CI: JS Build for commit $CI_COMMIT_SHORT_SHA"
21    - git status

I haven't committed this experiment to the module yet as there are some other features I'd like to try adding like publishing to NPM from the d.o repo but overall I'm happy with the results and plan to take advantage of Gitlab CI wherever possible in the modules I maintain.

Comments