DevOps Tools for Continuous Delivery in Jelastic Private Cloud. Part 3: Automated Jenkins Configuration via Cloud Scripting Add-On

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

image05

Evolving the concept of the DevOps approach at the Jelastic Platform, this time we are pleased to present you with the third guide from this devoted series. The new solution, described in today’s article, gives even greater automation through not just ensuring the project lifecycle management, but also simplifying the process of Jenkins continuous integration server configuration, required for that.

This is achieved through combining two progressive technologies: Jelastic pluggable add-ons and Cloud Scripting operations – the next generation solution for application and server management within the Cloud, which is currently being intensely developed by the Jelastic team. In particular, such an integration allows to automatically create the complete list of job templates, which just need to be slightly tuned (with operations like setting a URL to the repository with the required project, instead of the default one) to complete the Jenkins server adjustment.

The true efficiency of such an implementation is maximally revealed in the case of handling multiple projects, as instead of the countless step-by-step manual configurations, required for each of the Jenkins servers to be properly set, you’ll need to create a single add-on just once. Subsequently, it can be easily applied to the required environment in a few clicks with only a few user-dependent parameters required to be adjusted. Such an approach saves an increasing amount time on each following installation.

Project-update_2

As this article is tightly connected with the previous two parts of the ”DevOps Tools for Continuous Delivery in Jelastic Private Cloud” series, we will periodically refer to them in order not to repeat describing the same steps. That’s why we recommend you get acquainted with these guides firstly before proceeding further:

General Preparations

First of all, let us remind you of  a few obligatory arrangements, required to be accomplished in order to ensure the best DevOps experience:

  • your Jelastic installation should include three individual hardware sets allocated for the development, testing and production teams separately:

image07

  • since the core of the continuous application delivery approach we are referring to is represented with the Jenkins server, you need to have an environment with the appropriate software running. As you can remember from our previous guides within the series, getting it becomes fairly easy with the embedded Jelastic Marketplace, which can be accessed either via the appropriate site page or directly through the dashboard.

image00

We’ve considered these steps in details previously, thus we are sure you won’t meet any troubles here.

Configuring Jenkins using Cloud Scripting

As it was already mentioned within this guide’s introduction, the greatest part of the work at the stage of Jenkins server’s setting up is going to be automated. To gain this, the appropriate configurations should be implemented within the code firstly, which will be subsequently executed at your application server with Jenkins handling. At Jelastic, this could be done through preparing a special JSON manifest file, which, being integrated to the platform, will represent a pluggable add-on, that can be easily run by any user directly through the dashboard.

Creating Add-On

Let’s start with composing the basis for our add-on in order to get the ability to apply it to the required Jelastic environment further:

  1. Log into your JCA panel as an administrator user and navigate to the Marketplace section using the main menu to the left.

image04

Once inside, expand the Add list at the top pane and select the Add-on option.

  1. Within the opened editor, paste the general code template of the following structure:
{
    "jpsType": "update",
    "jpsVersion": "0.3",
    "application": {
     "name": "package_name",
     "logo": "logo_url",
     "type": "java",
     "onInstall": {
         "executeShellCommands": {
             "nodeType": "tomcat7",
             "commands": [
                     custom_code
             ]
         }
     }
    }
}

The highlighted parameters should be substituted with the appropriate data – here you need to define the general information about your package and specify the operations it is intended to perform:

    • package_name – add-on name it will be titled at the users’ dashboard (e.g Jenkins Configurations)
    • logo_url – link to the image that will be used as a package icon (you can use our default one through copying and specifying this URL)
    • custom_code list of comma-separated shell commands, that will be run on add-on’s appliance

Whilst the first two settings are obvious, the last one needs to be considered in more detail, thus the next section of the guide is devoted to the specifics of the required operations’ scripting.

Composing Script

In our example, the Jenkins integration tool is installed over Tomcat 7, so all the commands used would be based on its specifications. We intend to automate the process through scripting the following operations:

  • uploading the scripts, required for setting the jobs
  • configuring Jenkins itself
  • creating and linking jobs to automatically lead an application through the lifecycle

Further, we’ll examine each of these points in more detail and prepare the core of our manifest according to this flow.

Note: Below we provide you with the example code, which needs to be modified according to the specifics of your application. In order to ensure this was done correctly and all of the actions were scripted properly, we recommend a preliminary test of your code. This could be done through creating a separate environment with the same topology your add-on is being created for (i.e. Tomcat 7 with Jenkins app deployed in our case) and running the adjusted commands directly inside your compute node’s container using SSH. If you need any assistance with this, please refer to the complete guidance on the DevOps Add-on for Jenkins Troubleshooting, which additionally includes descriptions of the expected results for each of the presented operations.

So, let’s reveal how the above suggested processes can be easily accomplished, with the help of the appropriate Cloud Scripting options. Subsequently, all of them need to be inserted instead of the custom_code parameter in the manifest example, in the same order they are described below.

  1. Here is the first part of the script we propose to create:

“mkdir /opt/tomcat/demo“,

“cd /opt/tomcat/demo”,

“echo ENV_NAME= > /opt/tomcat/demo/variables”,

“curl -fsSL file_url -o install.sh“,

“curl -fsSL file_url -o transfer.sh“,

“curl -fsSL file_url -o migrate.sh“,

“curl -fsSL file_url -o runtest.sh“,

In this piece of code you can recognize the process of uploading the scripts (they are required for setting Jenkins jobs) to the container. Previously, we’ve done this manually and considered everything in detail, thus here, we’ll just remind you of some crucial points (highlighted in the example above) that you need to pay attention to:

  • /opt/tomcat/demo – the full path to the folder where all your scripts information will be stored (we’ll use demo, but you could create and use any other desired directory – just don’t forget to specify the correct path everywhere throughout your code)
  • curl -fsSL file_url -o file_name – this command initiates file uploading inside the container; for this example we require the following scripts:
    • install.sh – automatically logs in as a specified user and creates a new environment with the topology set according to the Jelastic manifest file
    • transfer.sh passes the environment ownership to the specified user based on the environment transferring feature
    • migrate.sh – relocates the environment to the specified hardware set (hardnode group)
    • runtest.sh – in our example, it simulates the testing activities for demonstration purposes, thus we don’t provide its code. If required, create your own one according the specifics of your application and prescribe its uploading alongside the rest of the scripts

Note that before applying, each of the script templates, linked above, have to be additionally adjusted by substituting the following parameters according to your platform’s settings:

  • /path/to/scripts/ – the same path as you’ve set above (/opt/tomcat/demo in our case)
  • {cloud_domain} – your Jelastic Platform domain name
  • {JCA_dashboard_appID} – your dashboard ID, that could be seen within the Platform.DASHBOARD_APPID parameter at the JCA > About section
  • {JCA_appstore_appID} – appstore ID, listed within the same section (the Platform.APPSTORE_APPID parameter)
  • {url_to_manifest} – link to the Jelastic manifest file created according to our documentation (we’ll use the same one as before – it sets up two Tomcat application servers fronted by the NGINX load-balancer)

In addition, the 3rd string defines creation of the variables file alongside the uploaded scripts, intended to be used for further storing the automatically generated name for the environments with your app’s different versions.

  1. The second part of code will remove all the existing (i.e. that are installed automatically during the initial tool deployment) Jenkins plugins in order to prevent incompatibility issues or a mismatch of their versions with the upcoming configurations:

“rm -rf /opt/tomcat/temp/.jenkins/plugins”,

  1. After that, you need to move to the main .jenkins directory and add the necessary plugins and configuration files to the Jenkins integrator instead of the deleted ones by uploading and extracting the appropriate plugins.zip archive.

“cd /opt/tomcat/temp/.jenkins/”,

“curl -fsSL file_url -o plugins.zip“,

“unzip -o plugins.zip >/dev/null 2>&1″,

We recommend to use the package we’ve prepared for this example, which already contains all the necessary modules. However, you can complement it with other plugins if necessary or just pack your own zip archive – in this case, note that there are several obligatory tools it should comprise (namely: SCM API Plugin, GIT plugin, GIT client plugin, Environment File Plugin) and all of them should be put in the plugins folder inside the archive.

  1. The next section of code uploads and extracts another required package (jobs.zip), that will set Jenkins jobs according to the data inside.

“curl -fsSL file_url -o jobs.zip“,

“unzip -o jobs.zip >/dev/null 2>&1″,

The archive structure should be the following: one general jobs folder, which may include multiple directories, named after the appropriate comprised jobs. Each of such directories should contain the config.xml file (with all the required configs specified inside) and the builds folder with an empty legacyIds file. We’ve already prepared such an archive, which contains settings for creation of the jobs, devoted to each step of the proposed application lifecycle flow:

Create Environment > Build and Deploy > Dev Tests > Migrate to QA > QA Tests > Migrate to Production

Note that in case you are using your custom path to the scripts folder, you need to edit the Build and Deploy job in our example package, as it contains the defined location of the file inside that directory:

<filePath>/opt/tomcat/demo/variables</filePath>

Everything else may remain as is, however, you can also edit it or add your own jobs if needed.

5. Finally, define the restart of the server in order to apply all of the executed configs.

“sudo /etc/init.d/tomcat restart >/dev/null 2>&1″

Being combined together, all these code parts will automatically perform almost all the required Jenkins configurations.

Thus, now you can place them within the appropriate commands manifest section as it was suggested above – you’ll get something similar to the following:

image03

Click Save to finish and append your add-on to the list of the already available installations.

Enabling the Add-On

Finally, to make our module accessible via the dashboard, find and select it within the list, click on the Publish button above and confirm this action at the shown pop-up.

image09

Everything is prepared now, so let’s just see how quick and simple the implementation of our DevOps approach becomes now.

New DevOps Experience

Navigate to the Jelastic dashboard (for the account your Jenkins environment is running at) and perform the following to plug your add-on to the appropriate application server.

  1. Hover over the Tomcat 7 node and click the appeared Add-ons icon to display the same-named tab with the list of suitable pluggable modules.

image08

Locate your add-on (named Jenkins configuration in our case) and Install it with the appropriate button.

  1. Wait for the process to be successfully completed and access the Jenkins dashboard (i.e. Open your environment in browser).

image10

As you can see, we’ve got the list of already prepared jobs.

  1. Now you need to slightly adjust each of these actions, namely:
  • specify the link to the repository with your project for the Build and Deploy job  
  • set the correct credentials inside the Execute shell box for the rest of the jobs

In order to access a particular job’s parameters and change them, use the Configure context menu option (can be called through right-clicking) for the required record:

image02

Note: As an example for this guide, we use a Git repo with a simple HelloWorld app, intended to be deployed to our environment. It contains the pom.xml file with the Jelastic Maven plugin settings specified, which is designed to automatically deploy the already built .war file to your environment. You can use the linked file as example to adjust your own project’s deployment in the same way:

  • {dev_user}@example.com – login (email address) of the dev user’s account
  • {password} – password for the user, specified above
  • {cloud_domain} – domain name of your Jelastic Platform
  1. The last thing left to perform is to bind the launching of the first job from our cycle (i.e. Create Environment) to the commit action, that occurred at the repository. As you already know from the previous guides, this could be done through the Webhook option – here is a reminder of the workflow (with the GitHub repo settings as an example):

image01

That’s all! Now you have a fully configured and ready-to-work integration complex, that waits only for the project change to be committed in order to automatically create an environment, deploy a new application version and move it through the set of required testing and relocation steps. After that, your checked application will appear at production.

And having the Jelastic attachable add-on for Jenkins configuration already prepared, you can quickly apply this approach to any of your projects through deploying a new continuous delivery mechanism within a few clicks and with slight adjustments done.

If you’d like to test this approach live, don’t hesitate to contact us for a free demo cloud evaluation and try it yourself or start with a two week trial at one of the dozens of Jelastic installations worldwide. In addition, you could share your experience on the project lifecycle automation or request for a new tutorial you are interesting in, within the comments below.

Related Articles:

DevOps Tools for Continuous Delivery in Jelastic Private Cloud. Part 1: Workloads Distribution and Jenkins Installation

DevOps Tools for Continuous Delivery in Jelastic Private Cloud. Part 2: Configure Jenkins for App Life Сycle Automation

DevOps with Containers for Java – JavaOne Latin America 2015

Virtual Private Cloud for DevOps – Ruslan Synytsky, WHD.usa 2015

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

Leave a Reply

Subscribe to get the latest updates