CI/CD tools landscape

For any software development team is critical to delivering value as quickly as possible, safely and reliably. It is proven that speed of delivery is directly correlated with the organisation performance (see, e.g. State of DevOps report. So the delivery process influence company valuation and is critical for scaling the engineering effort withholding the desired quality of the product. How to achieve this is one of the cornerstones of DevOps and SRE. To get an idea of how the modern software delivery works in a successful company, see how delivery pipeline works in AWS.
This blog post is by no means a replacement for deep-dive specialised or best practice literature, e.g. Continous Delivery, Continous Integration but rather an evaluation of the current tooling landscape which can help in achieving project goals. No need to mention that tool alone won’t make the magic happen without correct delivery pipeline design. But in the post, we will focus solely on the tooling.

Firstly, let’s clarify the essential terms (for reference, see a good article from Atlassian)

Continuous integration (CI) is the practice of automating the integration of code changes from multiple contributors into a single software project.

Continuous Delivery (CD) is the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments— into production, or into the hands of users, safely and quickly in a sustainable way.

Continuous deployment (CD) is a strategy for software releases wherein any code commit that passes the automated testing phase is automatically released into the production environment. It is paramount to software delivery processes. 

The delivery process is critical in any software company. From my perspective and experience current state is far from being “solved”, and the number of tools appearing every year confirms that. The amount of money spend by VCs is just confirming that. The majority of tools are imperative, while the next big trend seems to be a “declarative” CI/CD tooling. Curious about what the future will bring.

Wide variety of tools available (by no means list is extensive):

Our selection and evaluation criteria base on our current and future needs:

  • cost-effective (auto-scaling workers, etc.)
  • cost of maintenance
  • speed of development/ability to contribute
  • manual approval stage
  • ability to pass certification (audit-ability, permission and roles, etc.)
  • multi-cloud support
  • support VMs + kubernetes deployments + potentially serverless
  • ability to integrate Infrastructure as Code to delivery pipeline
  • do not scratch all our development infra (keep in mind cost/benefit ratio)
  • majority of our workloads are running in GCP
  • deals with mono-repo
  • support for long term support (LTS) branches

Following tools made it into shortlist for evaluation and deep dive. See dedicated post for each of those:

Summary:
Our ideal solution would be tooling provided by our primary cloud provider, which meets our current and near feature needs and is fully managed. We partially matched that with a combination of Cloud Build and Spinnaker for GCP based on tutorial provided by GCP.
Generally, my impression from the study and evaluation of tools listed is that claim of “full CI/CD” support are neither great in CI nor CD and lay somewhere in the middle. They provide a platform a let you code the rest. Another pain point is to tackle the monorepo and provide a means to be efficient. Platforms seem to be somewhat pricy, and the amount of infra work needed is not that low to justify it when providing all necessary features. Curious about what the Harness will provide in this space.
Not promoting the combination with end up with but was clear win moving away from Concourse CI. Where missing resource management for stages was a total killer, insufficient authorisation and role management and absence of manual steps was clear do not continue this journey. For a fresh new project, a GitLab would be a brainer to start with. It provides all needed for development, but when the project grows significantly, it can become pricy, and you are motivated even by GitLab to move partially to your infrastructure. Needless to say, that setup requires some amount of work, especially proxying and create network waypoints.
If you have some experiences with tools evaluated or disagree with the points, please use the comment section to share your view and don’t forget to like and follow me on Twitter!

Processing…
Success! You're on the list.

CD with Spinnaker – evaluation

Spinnaker one of the popular continuous delivery platform originally developed in Netflix. I am evaluating a version 1.23.5 . Spinnaker is a multi-cloud continuous delivery platform supporting VM and Kubernetes based deployments (server-less under development). Extensible platform with HA setup possible. This post is supposed to be part of the bigger series with a unified structure.

Overview:
Spinnaker Architecture
Spinnaker basic concepts (Spinnaker started for VM deployments, Kubernetes concepts mapped to it in provider)
Pipeline stages
– Support for manual Judgement stage though no detailed permission model for actions (non OSS plugins exists e.g. Armory)
– Nesting pipeline supported (either fire and forget or wait for completion)
Custom stages development (Rest call, Kubernetes job or Jenkins job, …)
– Development of new stage

Authentication & Authorisation (Spinnaker security concepts):
Spinnaker Authentication
Spinnaker Authorisation with Role Based Access
– Spinnaker can be accessed through GCP Identity Aware Proxy (or other service on different cloud providers)
– Authentication G-Suite identity provider or GitHub teams. Other options exist as well, see overview here.
– Authorisation with Google Groups (only support flat structure, role = name of the group), GitHub teams , raw mapping or others
Pipelines are versioned automatically
Pipeline triggers
– Concept of providers which integrates pipelines with target platform or cloud providers, e.g. Kubernetes provider v2
– Support for complex deployment strategies
– Management CLI – Halyard (spinnaker configuration) and Spinn for pipeline management
– Deployment to Kubernetes in the form of native manifests, Helm packages transformed in Helm Bake Stage to native manifests (using native Helm support for templating)
– Terraform stage as a custom stage e.g. oss implementation
– Wide variety of notification options
– Monitoring support via Prometheus
Backup configuration to storage

Pricing:
– There is no price for Spinnaker itself only for resources consumed when deployed
– Requires VMs, Redis or CloudSql(Postgress)
– Loadbalancer
Spinnaker for GCP if you are running on GCP, where you pay for resources needed only.

Resources:
https://spinnaker.io/
https://www.slideshare.net/Pivotal/modern-devops-with-spinnaker-olga-kundzich
https://spinnaker.io/concepts/ebook/

Summary:
Tool with focus on CD with manual approval stages, security model which makes it SOC2 compliant. Good audit-ability in place (possible to integrate to GCP audit log). Scripted stages and manual approval stage is possible to specify just a group. It is done on application/ pipeline level. Tool eliminate Helm from kubernetes cluster as it works based on Kubernetes native manifest. Propagates Immutable infrastructure as those artefacts are stored for possible rollbacks.  Authorisation/Authentication seems to be a complex but variable to integrate with wide variety of the systems. Pretty active user group, offering help. Pricing is based on resources used.

CI/CD with GitLab – evaluation

GitLab one of the popular DevOps platform out there, currently. I am evaluating a version GitLab 13.7-pre- release features. This post is supposed to be part of the bigger series with a unified structure. Evaluation in the context of existing infrastructure GitHub + Prometheus + Grafana.

High level overview: 

Authentication/Authorisation:

CI/CD capabilities:

Pricing:

  • Has the concept of minutes in the plan + buying extra ($10 per 1000min)
  • Pay for the storage $60/10GB  see details
  • Based on my understanding, we need at least Premium $19/user/month.
  • GitLab pricing

I haven’t studied GitLab offering super profoundly, but for building a new project, I would consider starting with it as it provides complete SDLC support (compared to Spinnaker it is CI + CD). Acts as SDLC management on top of the cloud provider – providing an easy way how to comply with the majority of measures from certification, e.g. SOC 2, but those are the gold plan features ($99/user/month). This might be pricy, but if you use ticket management, documentation (instead of, e.g. Jira), roadmap tooling, release notes management, Terrafrom stage seems like a no-brainer!

I see the following challenges:

  • Pipeline deployment ordering as parallel pipelines run
  • Shared runners are small machines step to registered add admin infra work
  • A security model is similar to Spinnaker, additionally doesn’t allow custom groups, but I guess that you can create custom apps (users)
  • Pricing seems scary at the end runners probably run on your infra and registered to the platform, OTOH if managed to keep on shared runners, need to buy a lot of build minutes. 
  • Storage cost seems high 
  • Docker registry has 30 days expiry (probably can be extended) => you will be uploading to your GCR

I haven’t studied in deep deployment capabilities:

  • Integration with Helm – probably rendering via helm template and then deploy
  • Support for deployment strategies – requires appropriate kubernetes object manifests as everywhere
  • Registered kubernetes seems to have an agent running in them
  • Has all concepts from Spinnaker more less
  • Has starting support for Terraform in alpha

Potential pain points:

  • Having a whole pipeline in git(including deployment strategies configurations, approvals) – might pose challenges when there is no pure trunk-based development – requires a need for backporting and harder for surveillance. 

GitLab is built on top of plenty of OS projects where I can imagine that integration between your infrastructure and GL might be extensive.

The only reasonable scenario that you fully migrate to GitLab and reduce extra tooling like Assana, GitHub, Confluence, … or for new projects that might be a no-brainer. That migration can be pretty heavy, but you might get some compliance checks for that in a single workspace. 

Resources:

CI/CD with Jenkins – evaluation

Jenkins Evaluation happened January 2021 and I believe that Jenkins is still improving. This post is supposed to be part of the bigger series with a unified structure.

Overview:
Pipeline definition completely lives in GIT together with code ~> Jenkinsfile
– Support for jenkinsfile via graddle DSL
You can chain the pipelines
– Single pipeline triggered on various branches ~> Multi-branch pipelines (tutorial)
Parallel pipeline stages
– Access to build meta-data (e.g.  build-number, commit hash, …)
Jenkins as a code plugin
– Managing secrets via secrets plugin
Audit trail plugin
Try notifier
– Better UI with Blue Ocean
– Tooling – Jenkins Job Builder (Job builder tutorial)
Pull-request Jenkins pipeline
– Deployment topology – master x slave/agent
Jenkins Helm deployment – seems has autoscaling agents – based on Configuration as a code plugin
– Manual approvals – seems as not so straightforward via input option
Jenkins on Google Kubernetes Engine

Security model:
– Default has no roles – all has single view -> plugins
GitHub OAuth and here
Role base authorisation plugin –  (strategy plugin – role) – that probably doesn’t work together with gitHub OAuth, but can work with Matrix access

Resources:
Jenkins for beginners

Summary:
Jenkins – one of the most popular open source CI/CD systems. Necessary to be self-hosted. But even Kubernetes plugin seems to have agent autoscaling capabilities which should be cost effective. Seems that whole Jenkins configuration can be bootstrapped from code. 

Security model has various options not sure how all fits together e.g. gitHub OAuth + Roles and Securities but there is multiple ways e.g. control matrix. 

Has concept of pipelines and jobs. Pipelines are next generation where they live completely in code-base ~> LTS should be ok. Seems that have some basic manual approvals stages, question how that goes together with auth. Has concept of multi-branch jobs/pipelines = single definition for whole bunch of branches where definition is dynamically taken from source. 

CD capabilities are somewhat simplistic – no advanced release strategies. Like roll back, monitoring etc. That would need to be scripted probably.

Scaling Terraform(IaC) across the team

HashiCorp Terraform is a popular tool for managing your cloud infrastructure as code (IaC) in a cloud-agnostic way (same tool for various cloud platforms). Instead of unifying all capabilities for different cloud platforms, the core concepts exposed to the end-user via Terraform provider concept. Terraform offers providers for all major cloud vendors and other cloud services and technologies as well, e.g. Kubernetes.  

This blog post doesn’t aim to be an introduction into Terraform concepts (official documentation is quite ok) but instead sharing an experience with using a Terraform in a distributed team, tools that come handy and all things that make life easier. Even though HashiCorp offers Terraform Enterprise this option is used quite rarely at least on “small/er” projects so we won’t be discussing this option here. I openly admit that I have zero experience with this service so I cannot objectively compare. I will solely focus on using the open-sourced part of the project and Terraform version 0.12.x and higher. 

Terraform maintains the sate of the infrastructure that manages in the state file. Format of Terraform state file is version dependant without strict rules on version compatibility (at least I wasn’t able to find one that was reliably followed and guaranteed). Managing state file poses two main challenges:
1) manage/share state across the team
2) control/align the version used across the team

The first aspect, different teams solve differently. Some commit state file alongside the configuration to version control system which is far from ideal as there might be multiple copies of such resource across the team and requires some team coordination. On top of that, state file contains sensitive information which is impossible to mask and such doesn’t belong to the source control system. A lot better approach is using Terraform remote backend, which allows true concurrent approach. Capabilities depend on the concrete implementation used. The backend can be changed from local to remote easily as is migrated automatically. The only limitation is that merging and splitting state file is allowed only for the locally managed state. 

Managing Terraform version management is centred around providing frictionless version upgrades for different Terraform configurations that align across the team with assuring that state file won’t get upgraded accidentally. To make sure that your state file won’t get upgraded accidentally put version restriction to every configuration managed e.g.

terraform {
  required_version = "0.12.20"
}

To align the team on a uniform Terraform version for every single configuration managed use tool for Terraform version management, e.g. tfenv. Put the desired version of to .terraform-version file located in the folder together with configuration. Tfenv automatically switches to appropriate version as needed, when new version encountered you need to run tfenv install to download a new version. If you want to check version available:

$ tfenv list
* 0.12.20 (set by /Users/jakub/test/terraform/.terraform-version)
  0.12.19
  0.12.18

As the number of resources or organisation grows so does the state file. Which leads to increased time for configuration synchronisation and competing for a lock on a remote state file. To increase the throughput and allow team DevOps mode(clear ownership of the solution from end to end), you might want to divide the infrastructure and associated state files into smaller chunks with clear boundaries. To keep your configuration DRY hierarchical configuration tools like Terragrunt comes to rescue and reduce repetition. 

A growing number of users poses challenges as well as benefits which are the same as on application code written in, e.g. java or any other programming language. What is the real motivation for Infrastructure as a Code (IaC). How to setup standards and best practices on the project? Terraform offers a bunch of tools embedded. To make sure that code is properly formatted according to standards use fmt utility, which is pluggable to your CI/CD pipeline or pre-commit hooks.

terraform fmt --recursive --check --diff

For your re-usable Terraform modules it is good to make sure they are valid though it doesn’t catch all the bugs as it doesn’t checks against cloud APIs, so it doesn’t replace integration tests.

terraform validate

Getting an idea what will change, diff of your current infrastructure against proposed changes can be easily achieved via the generated plan

terraform plan

Enforcing security and standards are a lot easier on IaC as you can use tools like tflint or checkov which allows writing custom policies. We conclude the tool section with awesome Terraform tools which provide a great source if you are looking for something specific.

In this blog post, we just scratched the surface of Terraform tooling and completely skipped design and testing, which are topics for separate posts. What are your favourite tools? What did you find really handy? Leave a comment, share your tips or you can ask me on twitter.

Processing…
Success! You're on the list.

Kubernetes Helm features I would wish to know from day one

Kubernetes Helm is a package manager for Kubernetes deployments. It is one of the possible tools for deployment management on Kubernetes platform. You can imagine it as an RPM in Linux world with package management on top of it like an apt-get utility.

Helm release management, ability to install or rollback to a previous revision, is one of the strongest selling points of Helm and together with strong community support makes it an exciting option. Especially the number of prepared packages is amazing and make it extremely easy to bootstrap a tech stack on the kubernetes cluster. But this article is not supposed to be a comparison between the kubernetes tools but instead describing an experience I’ve made while working with it and finding the limitations which for some else might be quite ok but having an ability to use the tool in those scenarios might be an additional benefit.
Helm is written in GO lang with the usage of GO templates which brings some limitation to the tool. Helm works on the level of string literals, and you need to take care of quotation, indentation etc. to form a valid kubernetes deployment manifest. This is a strong design decision from the creators of the Helm, and it is good to be aware of it. Secondly, Helm merges two responsibilities: To render a template and to provide kubernetes manifest. Though you can nest templates or create a template hierarchy, the rendered result must be a Kubernetes manifest which somehow limits possible use cases for the tool. Having the ability to render any template would extend tool capabilities as quite often the kubernetes deployment descriptors contain some configuration where you would appreciate type validation or possibility to render it separately. At the time of writing this article, the Helm version 2.11 didn’t allow that. To achieve this, you can combine Helm with other tools like Jsonnet and tie those together.
While combining different tools, I found following Helm advanced features quite useful which greatly simplified and provide some structure to the resulting Helm template. Following list enumerates those which I found quite useful.

Template nesting (named templates)
Sub-templates can be structured in helper files starting with an underscore, e.g. `_configuration.tpl` as files starting with underscore doesn’t need to contain valid kubernetes manifest.

{{- define "template.to.include" -}}
The content of the template
{{- end -}}

To use the sub-template, you can use

{{ include "template.to.include" .  -}}

Where “.” passes the actual context. When there are problems with the context trick with $ will solve the issue.

To include a file all you need is specify a path

{{ $.Files.Get "application.conf" -}}

Embedding file as configuration

{{ (.Files.Glob "application.conf").AsConfig  }}

it generates key-value automatically which is great when declaring a configMap

To provide a reasonable error message and make some config values mandatory, use required function

{{ required "Error message if value not specified" .Values.component.port }}

Accessing a key from value file which contains a dot e.g. application.conf

{{- index .Values.configuration "application.conf" }}

IF statements combining multiple values

{{- if or (.Values.boolean1) (.Values.boolean2) (.Values.boolean3) }}

Wrapping reference to value into braces solves the issue.

I hope that you found tips useful and if you have any suggestions leave the comment below or you can reach me for further questions on twitter.

Processing…
Success! You're on the list.

Software Deployment – java applications as a RPM linux package

Java applications archives as jar, war and ear files are elementary distribution blocks in the java world. At the beginning managing all of these libraries and components were a bit cumbersome and error prone as the project dependencies depends on another libraries and all those transitive dependencies creates so called dependency hell. In order to ease developers of this burden Apache Maven (maven like tools) were developed. Every artefact has so called coordinates which uniquely identifies it and all dependencies are driven by those coordinates in a recursive fashion.

Maven ease the management at the stage of the artefact development but doesn’t help that much when we want to deploy the application component. Many times that’s not such a big deal if your run time environment is clustered J2EE aplication server e.g. Weblogic cluster. You hand the ear or war over to your ops team and they deploy it to the cluster via cluster management console to all nodes at once. They need to maintain an archive of deployed components in case of roll back etc. This is the simplest case (isolated component and doesn’t solve dependencies e.g. libraries provided in the cluster etc.) where management is relatively clean but relying on the process a lot. When we consider different run time environment like run the application as a server less java process (opposed to J2EE cluster) then the stuff gets a bit more complicated even for a simplest case. Your java applications are typically distributed as a jar file and you need to distribute it to every single linux server where instance of this process is running. Apart of that standard jar file doesn’t contain dependencies. One possible solution to that would be to create shaded (fat) jar file which has all dependencies embedded. I suppose that you have a repository where all builds are archived. Does it make sense to store those big archives where the major part are 3rd party libraries? This is probably not the right way to go.

Another aspect of roll out process is ability to automate it. In case of J2EE clusters like weblogic there is often scripting tool provided (WLST ~ weblogic scripting tool). The land of pure jar is again a lot worse. You can take some advantage of maven but that doesn’t solve all the problems. Majority of production environments in java world run on linux operating system so why not to try to take advantage of linux server standard distribution management like yum, apt etc. for distributing rpm linux packages. This system provides atomicity, dependency management between rpm linux packages, easy way to roll back (keeps track of versions), minimise the number of manual steps – potential of human error is reduced and involves native auditing. It is pretty easy to get an info about installation history.

To pack your java jar file application you need a tool called rpmbuild which creates a linux package from a SPEC file. SPEC file is something like pom in maven world plus it contains instruction how to install, uninstall etc. Packages containing a required plus handy tools are rpmdevtool and rpmlint. On linux OS it is simple to install it. On Windows OS you need a cygwin installed with the same tool set. In order to build your rpm work space run the following command. It is highly recommended not to run it under root account if there is no special need for it.

rpmdev-setuptree

This command creates rpmbuild folder – that is the place where all linux RPM packaging will happen. It contains sub-folders: BUILD, RPMS, SOURCES, SPECS, SRPMS. For us the important ones are RPMS – will contain final linux rpm and SPECS – this is the place we need to put our SPEC file describing installation and content of our application.
This file is the core of the linux rpm packaging. It contains all the information about version, dependencies, installation, un-installation, upgrade etc. We can create our skeleton SPEC file by running following command:

rpmdev-newspec

Majority of directives in this file are clear from its name, e.g. Name, Version, Summary, BuildArch etc. BuildRoot require special attention. It is a sort of proxy which mimics a root of system under the construction e.g. If I want to install my [application_name] (replace this place holder with actual name) to /usr/local/[application_name] location I have to create this structure under BuildRoot during the installation. Then there are important sections which corresponds to various phases of installation: %prep, %build, %install – which is the most important for as as we do not build from sources but just pack already built jar file to linux rpm package. Last very important section of this file is %files which lists all files which will be in the final linux rpm package and hence installed in the target machine. Apart from that there can be additional hooks to installation and un-installation process as %post, %preun, %postun etc. which allows you to customize a process as you need. Sample SPEC file follows:

%define _tmppath /home/virtual/rpmbuild/tmp
Name: [application_name]
Version: 1.0.2
Release: 1%{?dist}
Summary: Processor component which feed data into DB
Group: Applications/System
License: GPL
URL: https://jaksky.wordpress.com/
BuildRoot: %{_topdir}/%{name}-%{version}-%{release}-root
BuildArch: noarch
Requires: jdk >= 7
%description
Component which process incoming messages and store them to DB.

%prep

%build

%install
rm -rf $RPM_BUILD_ROOT
mkdir -p $RPM_BUILD_ROOT/usr/local
cp -r %{_tmppath}/[application_name] $RPM_BUILD_ROOT/usr/local
mkdir -p $RPM_BUILD_ROOT/usr/local/[application_name]/logs
mkdir -p $RPM_BUILD_ROOT/etc/init.d
cp -r %{_tmppath}/[application_name]/bin/[application_name] $RPM_BUILD_ROOT/etc/init.d
mkdir -p $RPM_BUILD_ROOT/var/run/[java application]

%files
%defattr(644,[application_name],[application_name])
%dir %attr(755, [application_name],[application_name])/usr/local/[application_name]
%dir %attr(755,[application_name],[application_name]) /usr/local/[application_name]/lib
/usr/local/[application_name]/lib/*
%attr(755,[application_name],[application_name]) /usr/local/[application_name]/logs
%dir %attr(755,[[application_name],[application_name]) /usr/local/[application_name]/conf
%config /usr/local/[application_name]/conf/[application_name]-config.xml
%config /usr/local/[application_name]/conf/log4j.properties
%dir %attr(755,[application_name],[application_name]) /usr/local/[application_name]/deploy
/usr/local/[application_name]/deploy/*
%doc /usr/local/[application_name]/README.txt
%dir %attr(755,[application_name],[application_name]) /usr/local/[application_name]/bin
%attr(755,[application_name],[application_name]) /usr/local/[application_name]/bin/*
%attr(755,root,root) /etc/init.d/[application_name]
%dir %attr(755,[application_name],[application_name]) /var/run/[application_name]

%changelog
* Wed Nov 13 2013 Jakub Stransky <Jakub.Stransky@jaksky.com> 1.0.2-1
- Bug Fixing wrong messages format
* Wed Nov 13 2013 Jakub Stransky <Jakub.Stransky@jaksky.com> 1.0.1-1
- Bug Fixing wrong messages format
* Mon Nov 11 2013 Jakub Stransky <Jakub.Stransky@jaksky.com> 1.0.0-1
- First relaese of [application_name]

Several things to highlight in the SPEC file example: tmppath points to the location where the installed application is prepared, that is essentially what is going to be packed to rpm package. %defattr set the standard attributes to files if special one are not specified. %config denotes configuration files which means that for the first installation those standard one are provided but in case of upgrade those file will not be overwritten as they are probably customized to this particular instance.
Now we are ready to create the linux rpm package just the last step is pending:

rpmbuild -v -bb --clean SPECS/nameOfTheSpecFile.spec

Created package can be found in RPMS subfolder. We can test the package locally by

rpm -i nameOfTheRpmPackage.rpm

To complete the smoke test lets remove the package by

rpm -e nameOfTheApplication

Creating a SPEC file should be pretty straightforward process and once you create your SPEC file for the application building of linux rpm package is one minute job. But if you want to automate it there is a maven plugin which generates a SPEC file for you. It is essentially wrapper of rpmbuild utility which means that plugin works fine on linux with tool set installed but on windows machine you need have cygwin installed and create wrapper bat file to mimic rpmbuild utility for the plugin. Detailed manual can be found for example here.

Couple things to highlight when creating a SPEC file. Prepare the linux package for all scenarios – install, remove, upgrade and configuration management right from the beginning. Test it properly. It can save you a lot of troubles and manual work in case of large installations. Creating a new version of java application is only about about replacing jar file, re-packaging rpm bundle.

In this quick walk through I tried to show that creating of linux rpm package as a unit for software deployment of the java application is not that difficult and can neaten a roll out process. I just scratch the surface of linux rpm packaging and I was far away from showing all capabilities of this approach. I will conclude this post by several links which I found really useful.

Great tutorial on RPM packaging in general
Good rpmbuld manual pages
Maven rpm plugin
Maximum RPM book

If you have some tips or wan’t share your experience please find the comment section bellow or reach me on twitter

Build Number

One of the most important thing during the SDLC (for sure apart from the other stuff) is to keep control over deployed artifacts to all environments at any given time. Lack of control leads to chaos and generates a lot of extra work to the team, degrades throughput, morale and motivation. No need to even mention that arguments among team members regarding deployed features or fixes definitely do not contribute well to the team spirit.
One of the common approaches mitigating this risk is generating a build number to every single build fully automatically. Let’s take a look at how to accomplish this in common project set up – maven project build on build server e.g. TeamCIty. Sample web application follows.
Common place where to store such kind of info is MANIFEST.MF file. All kind of archives have this file located in /META-INF/MANIFEST.MF. Various technologies like OSGi use this location for various metadata. Taking advantage of maven-war-plugin the content of MANIFEST.MF can be easily customized as follows (${xx} are maven variables):
Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: ${user.name}
Build-Jdk: ${java.version}
Specification-Title: ${project.name}
Specification-Version: ${project.version}
Specification-Vendor: ${project.organization.name}
Implementation-Title: ${project.name}
Implementation-Version: ${project.version}
Implementation-Vendor-Id: ${project.groupId}
Implementation-Vendor: ${project.organization.name}

To set up a maven project pom file is pretty easy:


org.apache.maven.plugins
maven-war-plugin

true
true

${build.number}

Where build.number variable gets supplied by build server in arbitrary format, e.g. for TeamCity build server:

Build number is visible in build queue status page as well:
To access these project build specific information simple jsp page can be created:
The controller accessing these information using Spring MVC (simplified example) can look like:
@Controller
public class ProjectInfoController {

    @RequestMapping("/info")
    public ModelAndView getProjectInfo(HttpServletRequest request, HttpServletResponse response) throws IOException {

        ModelAndView modelAndView = new ModelAndView("projectInfoView");

        ServletContext servletContext = request.getSession().getServletContext();

        Properties properties = new Properties();
        InputStream is = servletContext.getResourceAsStream("/META-INF/MANIFEST.MF");

        properties.load(is);

        modelAndView.addObject("buildBy",properties.getProperty("Built-By"));
        modelAndView.addObject("buildJdk",properties.getProperty("Build-Jdk"));
        modelAndView.addObject("specificationVersion",properties.getProperty("Specification-Version"));
        modelAndView.addObject("specificationTitle",properties.getProperty("Specification-Title"));
        modelAndView.addObject("implementationVendor",properties.getProperty("Implementation-Vendor-Id"));
        modelAndView.addObject("buildNumber",properties.getProperty("Build-Number"));

        return modelAndView;
    }
}

Accessing MANIFEST.MF in JAR file has a different approach. Motivation taken from Spring source code:

Package  package = someClass.getPackage( );
String version = package.getImplementationVersion();

JSP page or other presentation layer shouldn’t be a problem for anyone.