Hosting Costs Reduction Tools for Software Development Companies: Scalified Use Case

By | December 1, 2017
Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

Every software company would like to get maximum result for minimum cost. Today we are sharing a real example how Scalified, outsource software development company, cut their hosting spendings with the help of resource consumption scheduler and scalability of Jelastic PaaS.

Scalified is involved in the professional development of distributed, scalable applications, as well as in mobile-/web-development, and QA/Automation. About 2 years ago Scalified opted Jelastic PaaS for software development:

Scalified MIRhosting hosting cost reduction“As for the reasons and motivation why we’ve chosen Jelastic over other PaaS providers, it was a quite interesting story: we were developing one of the most popular insurance comparison services in Switzerland – swupp.ch, so we needed some Swizz-based hosting. First of all, it must have been fast, reliable and scalable. Second major requirement we had was out-of-the-box Docker support so we could be platform independent in terms of technologies, frameworks, databases, etc. That’s how we found Flow App Engine powered by Jelastic.

The funny thing was that I knew nothing about this platform at that point, except I graduated the same university as Jelastic CEO, Ruslan Synytsky. However, it was very easy to convince our customer that Jelastic is worth using it 🙂 And I’m really happy it’s been proven by two-years-long production usage now.

Also, we’ve been using MIRhosting (one more Jelastic partner) for several years, mostly for our development processes. In case of some issues, they respond quickly and help to fix everything, providing good technical support level”, – Serhii Siryk, Founder & CEO, Scalified.

reduce cloud hosting spendeings

Below Scalified share their lifehacks how to fix money leaks (initially published on their blog):

Introduction

While software development of big and small projects at Scalified we often have situations when development environment includes multiple nodes. Starting from Continuous Integration (CI) which is crucial for an effective development team, and ending with dev/test/stage environments, every one of those nodes consumes some resources and therefore costs money.

The sad thing about this is that you can’t effectively develop qualitative software working in a team if you don’t use CI, auto deployments, static code analysis tools, etc. But do we really use all of the mentioned systems 24/7? Probably not, unless your development team is international which is rather rare case in outsource software development, you can make use of the fact that people usually tend to sleep at night and turn-off development environments at all.

It makes a real sense when using cloud-based providers like Amazon/Azure/Jelastic where you can use flexible pricing plans and pay for only consumed resources. So, in this article we’re going to show how we can automate our environments to be turned on/off according to a schedule, as well as to share how much money it can save.

Preconditions

Let’s get started with defining the preconditions. We’ll use Jelastic cloud based hosting which supports Docker containers to be published in the cloud. Every Docker container can be used as a separate node in the cloud, therefore we can easily manage each of them separately. Here’s the real environment configuration we use for one of our projects: 

Node NameCloudlets MemoryCPUPrice
CI server (Teamcity)6-81 GiB3.2 GHz10.5
CI database (Postgres)7896 MiB2.8 GHz9.2
CI agent (Teamcity Agent) * 31-172.12 GiB6.8 GHz21 (* 3)
Development Database (Docker Oracle)10-303.75 GiB12 GHz34.3
Test Cases & Requirements management tool (SquashTM)7-91.12 GiB3.6 GHz7.7
App server – DEV (Jboss)11-151.87 GiB6 GHz14.8
App server – TEST (Jboss)11-151.87 GiB6 GHz
14.8
Selenium for e2e tests (Standalone Chrome)2-91.25 GiB4 GHz12.7

As you can see, the maximum monthly cost for this configuration might be 167 €, assuming all the nodes consume max available resources 24/7, i.e. memory and CPU. Of course in the real world it won’t be the case that memory and CPU are used so hard there, however, we can take this number just to calculate how much we can save running servers for only 10h per day.

The average month has 30 days with  24 hours per day, then 30 * 24 = 720 hours that we have monthly. Therefore, our configuration costs us 167 / 720 =0.2319 per hour. Assuming we need our servers to be operating 10 hours per day (excluding weekends): 5 business days in a week * 4 weeks in a month * 10 hours in a day = 200 hours in total. It’s great optimization since now we can pay 200 * 0.2319 = 46.38€ max which is 3.6 times less than the original price. Supposing CPU and memory are used ~70-80% on average, the price will be even less expensive. Great, so let’s take a look how can we perform that.

Scheduler is Watching You

To do the main job we’ll need some blazing lite environment which is always running and doesn’t consume a lot. It will be responsible for starting/stopping nodes according to a configured schedule. We recommend using an Alpine docker image with pre-installed Cron. Alpine is fast and efficient Linux distribution, and Cron is a well-known time-based job scheduler in Unix-like systems. Our real scheduler instance on Jelastic has up to 256MiB of memory, 800 MHz CPU and costs us less than 1.1€ max monthly.

So, now when we have scheduler running and we know that Cron can execute any command in a timely manner, we can wrap commands we need to be executed in a single shell script. The script will be only responsible for turning on/off nodes, but for sure we don’t want to reinvent the cycle. That’s why we’re gonna make use of Jelastic command-line interface (Jelastic CLI) tool to the job for us.

Make use of a CLI

The Jelastic CLI is a tool, designed to simplify interaction with your Cloud Platform account by executing the appropriate commands directly from any machine. It allows handling the vast majority of available operations remotely, complementing the wide stack of capabilities, provided via dashboard and Jelastic API.

Though Jelastic CLl allows us to do things easier within the cloud, it requires setup and some preparations to be performed before.

1.First of all, we need to set up Java 1.8:

sudo apk add --update --no-cache openjdk8

2.After we have Java installed, we can install Jelastic CLI itself:

$ sudo curl –s ftp:// ftp.jelastic.com/pub/cli/jelastic-cli-installer.sh | bash

After a successful installation, the folder ~/jelastic should be created in user’s home directory.

3.Next step will be authenticating in Jelastic cloud, – it’s required for CLI to execute its commands. It can be done manually, by just executing the following:

~/jelastic/users/authentication/signin --login {email} --password {password} --platformUrl {platform_Url}

After a successful authentication, CLI will keep authentication token so that you don’t need to do this manual step frequently. However, if you don’t want to do it manually you can store login and password, e.g. in environment variables and automate this step as well.

Here’s the example of the final script you can use for starting/stopping environments, it also supports automatic authentication on every run:

#!/bin/sh                                                                                                                                                                                                                                             
JELASTIC_BIN=~/jelastic/environment/control
JELASTIC_AUTHENTICATION=~/jelastic/users/authentication                                                                                 
COMMAND_EXECUTION_TIMEOUT=45                                                                                                                                                                                                                       
login() {
  echo "[INFO][$(date -u)]: Starting Logging to the cloud ..."
  timeout ${COMMAND_EXECUTION_TIMEOUT} bash ${JELASTIC_AUTHENTICATION}/signin --login ${JELASTIC_LOGIN} 
--password ${JELASTIC_PASSWORD} --platformUrl app.mircloud.host --silent true
  if [ $? == 0 ]; then
     echo "[INFO][$(date -u)]: Successfully logged to the cloud"
    else
     echo "[INFO][$(date -u)]: Failed to log to the cloud"
    fi
}                                                                                                                                                                                                      
start() {                                                                                                                   
  login
  for jelastic_environment in ${ENVIRONMENTS}                                                                               
  do                                                                                                                        
    echo "[INFO][$(date -u)]: Starting ${jelastic_environment} environment"                                                 
    timeout ${COMMAND_EXECUTION_TIMEOUT} ${JELASTIC_BIN}/startenv --envName ${jelastic_environment} --silent true                                                                                                                                 
    if [ $? == 0 ]; then                                                                                                    
     echo "[INFO][$(date -u)]: Successfully started ${jelastic_environment} environment"                                    
    else                                                                                                                    
     echo "[INFO][$(date -u)]: Failed to start ${jelastic_environment} environment in ${COMMAND_EXECUTION_TIMEOUT} seconds" 
    fi                                                                                                                                                                                                                                           
  done                                                                                                        
}                                                                                                                                                                                                                                          
stop() {                                                                                                                    
  login                                                                                                              
  for jelastic_environment in ${ENVIRONMENTS}                                                                               
  do                                                                                                                        
    echo "[INFO][$(date -u)]: Stopping ${jelastic_environment} environment"                                                 
    timeout ${COMMAND_EXECUTION_TIMEOUT} ${JELASTIC_BIN}/stopenv --envName ${jelastic_environment} --silent true                                                     
   if [ $? == 0 ]; then                                                                                                    
     echo "[INFO][$(date -u)]: Successfully stopped ${jelastic_environment} environment"                                    
    else                                                                                                                    
     echo "[INFO][$(date -u)]: Failed to stop ${jelastic_environment} environment in ${COMMAND_EXECUTION_TIMEOUT} seconds" 
    fi                                     
  done                                                                                                                      
}                                                                                                                                                                                                                                                    
$@

Note, that this script requires some system variables, such as ${ENVIRONMENTS}, ${JELASTIC_LOGIN}, ${JELASTIC_PASSWORD} to be pre-configured before execution. For example, first one is just space-separated list of Jelastic environments needed to be processed by start/stop command, e.g app-dev squash-tm app-test oracle-database selenium-chrome teamcity. It can be easily done by using Jelastic web UI like here:

Start/Stop Scheduler

Last step to make it work is to give a permission for running the script:

$ chmod u+x script

Great, so now we can use the script to start/stop environments specified in ${ENVIRONMENTS} variable.

Just run the script providing a corresponding parameter to check it out:

$ ./script start
$ ./script stop

Configuring the Schedule

The final step left is to configure Cron. To do it we need to create a crontab file to schedule automatic execution at the desired time.

The main root crontab file is located in /var/spool/cron/crontabs directory by default. In order to view and edit it, run the following command:

$ crontab –e

Now let`s configure the time for executing our script:

TZ=UTC
00 21 * * 1-5 /path/to/script stop
00 09 * * 1-5 /path/to/script start

As you can see in this example configuration, our script will automatically shut down all servers at 21 PM and start them at 9 AM. On Saturday and Sunday, none of the commands will be executed.

What Next

This article demonstrates how this simple and effective solution helps to reduce expenses on development infrastructure and increases profits from using modern cloud services. To make it even easier to use for real software development projects, Scalified team have created a ready to use Docker image. If you use Jelastic you can grab it from Docker Store or create a fork to support cloud provider you use.

*********

Penny saved is a penny earned – Scalified team proved that it is possible to save money on software development tasks and redirect resources to the company growth.

Inspired by this successful case? Register for free trial and try out the Scheduler to organize your environments uptime and maximize resource saving.

Related Articles

Scheduler for Automatic Start and Stop of Development Environments to Save Cloud Costs

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInEmail this to someone

Subscribe to get the latest updates