Deploy your project in Gitlab CI

As a continuation of my series for “Automating your Versioning”, I found valid to include a deploy stage in the series after the “release post”.

Below I am posting the links for the complete series, if you want read it all:

• Automate Your Versioning (Part 1)
• Automate Your Versioning (Part 2)
• Automate Your Versioning (Part 3)
• Release Automation with Gitlab CI
• Deploy with Gitlab CI

In this post, I will try to explain how I managed to create a deploy stage for sending new versions of the software to CompactRIO, to an IHM or possibly to any other target running any application.

The inspiration

After setting up the whole CI process, I started to thinking that I could do some more, the CI tests the application, builds the application, uploads it, releases it, so why not deploying it and completing the cycle CI/CD.

In the place I work, it is not always possible to have your targets connected to the PC, and sometimes they are far away, but in the same network. With Covid times and home-office it is more than ever a common situation among developers.

So I looked at some alternatives and I got to the Packages and SSH.

Setting up, Warnings and Disclaimers

If you are using the Gitlab, make sure you are able to run Linux Docker Images. This process will be done in three steps:

• Build a compatible deploy package;
• Upload the package to Package Registry (optional);
• Deploy to remote target on a click.

Some point to get attention to:

1. It is important that the previous steps (tests, downloading dependencies) are well done, because this will impact directly on the building/running application.
2. Second thing, there may be security issues doing this action, so if you are setting this up I assume you know the risks and you are somehow responsible for the actions your target is performing (controlling a nuclear power plant for instance).
3. Third thing, this is usually very useful in development stage, where you already have a functional version (>1.0.0), it could be used with Gitlab environments or with different jobs. If using in production, it is a completely different thought.
4. Fourth thing, again, I am not responsible for any of your actions.

Building

One important step is to build you application to suit the need of the deploy stage.

Usually, in RT Projects you just need the RT executable and click “Run as Startup” to deploy the new version of your application to target.

If you want to deploy without LabVIEW I would recommend using the Package Feature, which generates an .ipk package which contains the executable and pre/post scripts which can be interesting for Linux Applications.

If you are deploying to a windows target (e.g. TPC IHM), you will need to generate an installer at first sight.

Yes, you could do this job without the package and the installer, but they make things much easier.

Uploading

This is an optional step, as you could use the build job artifact for deploying, but if you don’t want to rely on the artifacts (because they will expire someday), you can always download the package before deploying.

In Gitlab it is possible to upload packages using the Generic Packages API, which is explained in detail here: https://docs.gitlab.com/ee/user/packages/generic_packages/

Upload:
  stage: release
  image: curlimages/curl:latest
  needs:
    - job: Package
      artifacts: true
    - job: Versioning
      artifacts: true
  before_script:
    - VERSION=$(cat .versionrc | grep -oE '\"version\"\:\s\"(.*)\"' | grep -oE '\d{1,2}\.\d{1,2}\.\d{1,2}')
    - VERSIONBUILD=$(cat .versionrc | grep -oE '\"version\"\:\s\"(.*)\"' | grep -oE '\d{1,2}\.\d{1,2}\.\d{1,2}' | awk '{print $1".0"}')
    - echo $VERSION
    - echo $VERSIONBUILD
  script:
    - 'curl --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file build/build-custom-cli-operation_${VERSIONBUILD}_windows_all.nipkg "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/build-custom-cli/${VERSION}/build-custom-cli-operation_${VERSION}_windows_all.nipkg"'

An example of uploading a package from my build custom CLI operation (https://gitlab.com/felipe_public/gitlab-tools-labview/build-cli-operation).

Deploying

With all set, now it comes the easiest part. You can use the SSH function from CompactRIO and the IHM (it is possible to install OpenSSH in Windows 7/10, you can google it around).

For this job, I used the alpine docker image which is very lightweight image and SSH Pass for the passwords (there are plenty of alternatives for authenticating remotely with SSH).

Steps:

1. Copy the file
2. Install using opkg/msiexec, delete the package and rebooting the application (if not set in the package configuration).
3. Allow failure (in manual jobs, it means optional job). You could do automatically, but I’d rather choose the time for rebooting my machine.

Here is the example job for the CompactRIO (using artifacts from build job, if you decide using the uploaded the package you could download it before):

deploy:
  stage: deploy
  image: alpine
  needs:
    - job: Build
      artifacts: true
  before_script:
    - apk add openssh-client
    - apk add sshpass
  script:
    - sshpass -p $CRIO_DEPLOY_PWD scp -o StrictHostKeyChecking=no build/*.ipk  $CRIO_USER@$CRIO_IP:/home/lvuser/pkgs
    - sshpass -p $CRIO_DEPLOY_PWD ssh -o StrictHostKeyChecking=no $CRIO_USER@$CRIO_IP 'opkg install /home/lvuser/pkgs/*.ipk; rm -r /home/lvuser/pkgs/*.ipk; reboot'
rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual  
allow_failure: true

Here is the example for the IHM (Win7 Embedded), just changing the script part, which is a bit different:

script:
  - sshpass -p $IHM_DEPLOY_PWD scp -o StrictHostKeyChecking=no build/*.zip  $IHM_USER@$IHM_IP:'C:\Users\NI'
  - sshpass -p $IHM_DEPLOY_PWD ssh -o StrictHostKeyChecking=no $IHM_USER@$IHM_IP '7z e -oc:\install -y C:\Users\NI\install.zip && msiexec.exe /I c:\install\install.msi /quiet && del /F /Q c:\install && shutdown /r /t 0'

I don’t have to remember you to correctly setting your secrets in CI/CD Variables options of your Gitlab instance. Also, make sure the users you are using to deploy have the right permissions to reboot the machine and install new software, otherwise it will fail.

Deploying (in Fact)

If everything ran accordingly you can hit the button in the Gitlab Pipeline page and deploy the code to your target. It sounds like magic, but it isn’t. It is software engineering.

Complete Pipeline

After all the previous posts, I can put an image of the complete pipeline of a CompactRIO Project, including all what I’ve learned and wrote before.

Unfortunately this is a private repository and I can’t post the link, but you can have an idea of what is going on there.

I hope you all enjoyed it, and if you have any doubts or liked the post, leave your comments below.