In this article, Mike O’Hara, publisher of The Trading Mesh – discusses the subject of DevOps with Jim Davies of MoneySuperMarket.com, Peter Evison and Ani Chakraborty of Cake Solutions and Independent Consultant Tom Stockton, looking at some of the best practices firms are using to bridge the gap between development and operations.
Cloud-based infrastructure is evolving rapidly. Compared with just a couple of years ago, there is now a much wider range of options around how software can be deployed in the cloud. While this has created a wealth of new opportunities, there are also some emerging challenges, not least of which is the general shortage of people who are skilled in deploying applications on these new platforms. In some senses, the technology available has overtaken the skills of the people on the ground, resulting in a gap between development and operations, the so-called ‘DevOps skills gap’.
The net result of this is twofold. First, application developers are not able to get their innovations to market quickly enough; they might have features and functions ready but may struggle to deploy them. And secondly, cloud providers are becoming frustrated by the fact that they have all of these new infrastructure initiatives and innovations but their clients are unable to take advantage of them.
“Moving to the cloud brings a lot of benefits, but sourcing the skills for that to happen takes us away from the more traditional skills base, so in that sense when we started it was initially quite challenging.”
Jim Davies, Infrastructure Tech Lead, DevOps/Sysadmin at MoneySuperMarket.com
It is probably fair to say that there has always been a gap – some might say a gaping chasm – between development and operations. That, in itself, is not new. What is new, is the speed at which new technology and infrastructure platforms are becoming available – particularly in the cloud – and the fact that decisions need to be made regarding deployment much earlier in the development cycle.
Jim Davies, Infrastructure Tech Lead, DevOps/Sysadmin at MoneySuperMarket.com, the leading price comparison portal, explains how things are changing within his organisation as it increasingly adopts cloud-based infrastructure.
“Moving to the cloud brings a lot of benefits, but sourcing the skills for that to happen takes us away from the more traditional skills base, so in that sense when we started it was initially quite challenging”, he says.
The approach taken by Davies and his colleagues at MoneySuperMarket was to put together small delivery teams known as ‘squads’, able to operate fairly autonomously, each containing a technology lead, an architect, a group of developers, a couple of QA/testers and sometimes a business analyst.
“Previously when anyone wanted infrastructure services, they would come to the central Infrastructure & Operations (I&O) team, where resource time from sys-admins and server engineers would be ‘rented out’ to build servers, put network changes in place and build out the applications on the servers, before handing over that service”, explains Davies.
“Although that’s the way things were set up traditionally, the goal now is to have the sys-admins & systems engineers, the DevOps function, within those small delivery squads”.
What sorts of skills are typically required in these roles?
“I’ve found that it’s mainly Java developers, classically trained software engineers, who tend to be the strongest candidates. Managing infrastructure today is not about servers, not about cables, not about contracts, it’s about managing the code base, that’s the be-all and the end-all”, say Davies.
“Understanding inheritance and the way we bootstrap services into new servers, the way we use tools like Puppet, that’s how the skills are changing”, he says.
Having smaller teams that take care of everything – including developing the features, deploying the applications and maintaining them on a cloud-based infrastructure such as AWS – rather than having dedicated teams performing a single function can make a lot of sense, according to Peter Evison, Commercial Director at enterprise software solutions provider Cake Solutions.
“We recognised that DevOps needed to be adopted and be seen as a skill within our development teams and not a process undertaken by a department or specific individuals.”
Peter Evison, Commercial Director at Cake Solutions
Evison agrees that DevOps is an area where good talent is hard to find. This prompted the firm to develop a solution delivered in-house through its Cake Academy process.
“We recognised that DevOps needed to be adopted and be seen as a skill within our development teams and not a process undertaken by a department or specific individuals”, says Evison.
“Once we understood and implemented this, the reliance was reduced and the service levels increased”.
Evison’s colleague Ani Chakraborty, Technical Director at Cake Solutions explains how Cake enables its team members to become Cross Functional Engineers.
“All our teams have DevOps skills within the team and can deploy, integrate and maintain their clients’ applications independently from the rest of the company, almost like its own start-up company”, he says.
“We’ve achieved this by developing a rock solid and cloud agnostic CICD environment that empowers our engineers to deploy directly to the client’s chosen environment without risk. We also adopted new tools that helped us to monitor and audit this, which make the whole process quicker and more robust than traditional methods. Finally, we set about training our teams on DevOps skills and cloud environments, thus giving them a fuller appreciation and understanding of the end-to-end development pipeline”.
According to Evison, this is an evolutionary process.
“I don’t believe in the word ‘perfect’ so our process will continually improve and evolve”, he says.
“I do however appreciate that gaps in skills or resource can be dealt with in many ways and the best approach is to put bleeding edge technology in the hands of a highly talented team”.
Independent Consultant Tom Stockton, a specialist in DevOps, points out that although it is effective to have cross functional teams, it isn’t effective to have every member of the team working on creating DevOps tools. His view is that DevOps specialists can create tools that can be shared with the wider team, using their feedback to enhance and modify the toolset as required.
“You do need specialists in certain subjects”, he says.
“However, it’s probably more important to put the effort into developing some standard tools and then to correctly document and distribute that knowledge around the team by having good workshops and a good interface to the tools that you’ve written”, he says.
This is particularly important as cloud-based infrastructures are constantly evolving, according to Stockton.
“Awareness of DevOps is important across the organisation. If you don’t have that awareness then it’s going to be very difficult to get your application out there because of all the traditional reasons why code doesn’t get deployed.”
Tom Stockton, Independent Consultant
“With AWS for example, you might develop a solution that works but then you may have to re-develop it because they’ve changed either the service or the API. You may be given some notice on it but you can end up re-developing a solution that you maybe only worked on three or six months ago. So you have to be ready and you have to be pretty agile. By writing good code in the first place, it makes it easier for you to maintain that code well. And you’re better coming at it from a development point of view than you are from a systems admin point of view. A developer will always write better code than a sys admin can”, says Stockton.
Awareness of DevOps is important across the organisation, according to Stockton.
“If you don’t have that awareness then it’s going to be very difficult to get your application out there because of all the traditional reasons why code doesn’t get deployed. If there’s a disconnect between development and operations then you’re losing the benefit of writing all that code”, he says.
“But the level of understanding can be different for different people. For example, I would consider myself a DevOps engineer and I’ve developed a certain set of tools but I don’t necessarily need to be the person that always runs those tools. I can take them to the developers who I consider to be better or more technically capable than I am and they can easily go and run them, be effective using them and even give me feedback on them if they need to. To me, it’s about collaboration, sharing the knowledge and using the great community and open source tools that are out there in the right way”.
Ani Chakraborty of Cake Solutions sums things up.
“The key point about DevOps is that it allows businesses to deliver value to their end customers faster, maintaining the flow of delivery”, he says.
“It enables the infrastructure, operations and development teams to communicate and collaborate better together in delivering software. In that way, it’s actually a software development methodology that helps to deliver real value, starting from development of the feature to taking that feature up to production”.
“The key point about DevOps is that it allows businesses to deliver value to their end customers faster, maintaining the flow of delivery.”
Ani Chakraborty, Technical Director at Cake Solutions
The key to bridging the skills gap is to focus on value, according to Chakraborty.
“The important thing is having teams that deliver software, tools, techniques and ways
of working that empower other teams to deliver value”, he says.
“Rather than having set tools and techniques, look at the context. Look at the organisation, its structure, its motivation of what it’s actually trying to achieve and most importantly what is the business value that needs to be delivered. Everything else fits around that”, says Chakraborty.
Is it possible to fully bridge the DevOps skills gap? Almost certainly.
Although gaps have always existed and will undoubtedly continue to exist between development and operations, it is the firms that adopt the four key elements of collaboration, communication, taking a cross-functional approach through appropriate training, and focusing on value in the software delivery chain, who are able to most effectively cross