Those are certainly trendy topics. But based on the work I do on an everyday basis helping businesses plan and execute DevOps strategies, I’m noticing a different set of salient trends in the world of DevOps. Although much is being said about how technologies like AI might impact DevOps, the biggest changes I’m seeing right now involve other types of solutions and techniques.

Here’s a look at what I view as the three most important DevOps trends at present, as well as a breakdown of how they are poised to change DevOps tools and processes.

Trend 1: Policy-based management and IaC enable drive DevOps security innovation

Security has always been a priority for most DevOps practitioners. But right now, I’m seeing DevOps teams adopting new strategies in a bid to improve the security of the applications and environments they support.

One large-scale change is greater use of cloud-based policy management as a means of enforcing security best practices in cloud environments. Teams are configuring cloud services and resources using the code-based configuration frameworks that cloud providers support, then scanning the configurations to detect risks.

This approach makes it possible to enforce cloud governance consistently, centrally and automatically. Instead of simply writing governance policies and hoping that engineers remember to follow them when they are configuring cloud resources, businesses are increasingly building automated governance guardrails via policy-based management.

In a similar vein, more and more of the DevOps teams I work with are embracing static code analysis of Infrastructure-as-Code (IaC) templates as a means of detecting risks. As with policy-based management of cloud resources, an IaC-centric approach to infrastructure provisioning makes it possible not just to automate infrastructure management, but also to identify security risks earlier in the development lifecycle.

What’s more, some teams are making use of cloud policy and IaC code scanning as a way of warning each other about security policy changes that might cause an application or service to break. They do this by inserting configuration changes into their code using “audit” mode if their configuration framework supports it, or by simply configuring IaC scanners to flag changes if an audit feature is not available. This allows engineers to detect whether a change might cause a problem for an existing deployment.

This is important because within many organizations, the security team operates separately from application teams. When the two groups lack an efficient way of communicating with each other about changes, they may end up disrupting each other’s operations – an issue I like to call the “right hand/left hand” problem. Configuration scanning provides a systematic way of ensuring that each group is on the same page when it comes time to introduce changes – and the “audit” mode approach provides a grace period that allows time to react before a change actually takes effect.

Trend 2: Doubling down on DevOps automation with GitOps

Another overarching trend that is currently reshaping DevOps is the use of GitOps to make DevOps automation more efficient and consistent.

GitOps is the use of Git (or a similar source control system) to manage automated DevOps workflows. It involves defining configurations using code, then applying them through features like GitHub actions.

When you opt for GitOps, you move DevOps automation controls from individual workstations to centralized source control repositories. The result is the ability to track and manage all automated workflows via a central hub, which increases efficiency and mitigates issues like different engineers working with different versions of the same automation frameworks on their personal desktops.

In addition, GitOps automatically generates records of what has changed and how it has changed, since every action is logged through the source control system. This isn’t exactly documentation in the traditional sense, but it does mean that GitOps comprehensively documents every change – which is beneficial because human engineers tend not to be so thorough when it comes to documenting their actions.

To be sure, GitOps is not without its challenges. Implementing GitOps effectively requires additional skills – namely, expertise with both IaC frameworks and source control systems – that not all DevOps engineers possess. I also notice a tendency on the part of some teams to set up GitOps pipelines, but rely on manual approvals instead of automated actions to trigger changes – an approach that largely undercuts the value of automating pipelines in the first place.

However, these are challenges that teams can solve through education and by fully leaning into GitOps. In addition, techniques like automated testing of GitOps configuration code can help to build teams’ confidence in automations and reduce reliance on manual approvals.

Going forward, expect to see more and more adoption of GitOps techniques among teams seeking to level-up their approach to DevOps automation. Automating individual DevOps processes like software testing and deployment won’t be enough; truly efficient organizations will turn to GitOps as a way of automating their entire DevOps workflows, from end-to-end.

Trend 3: Investing in developer experience

Making software delivery processes more predictable and efficient is merely a step toward the ultimate goal of DevOps, which is to help developers become more productive and satisfied with their jobs.

To that end, I’m noticing a great deal of interest and investment right now in the realm of developer experience. This is playing out through two interrelated types of initiatives.

One is platform engineering, which involves creating DevOps teams who specialize in certain functions – such as network management or security – and designating them to support those functions throughout the organization. This approach reduces cognitive overhead for developers by freeing them from having to handle types of work that are not their main focus. In other words, instead of forcing developers to be DevOps generalists, platform engineering lets different teams focus on doing what they know and enjoy best – leading to greater productivity and higher levels of job satisfaction.

The other major trend currently playing out in the realm of developer experience is developer self-service. This means the ability of developers to obtain the technical solutions they need on-demand, without a complicated procurement process. In most cases, organizations enable self-service by implementing Internal Development Platforms, or IDPs, which host ready-made infrastructure resources and software environments that developers can deploy on a self-service basis.

There are risks inherent in these trends. They require specialized types of skills, and when poorly implemented, platform engineering and IDP solutions can create more problems than they solve. However, when you ensure that your teams have the requisite expertise, and when you deploy a carefully planned IDP that gives developers access to the resources they actually need, you’re likely to see a significant reduction in friction within your organization, and a boost in developer productivity and happiness.

Conclusion

Admittedly, discussing DevOps trends that center on security, automation and developer experience may not be as exciting as debating whether AI will take away DevOps engineers’ jobs. But if you want to know what’s actually changing in the world of DevOps – as opposed to which conversations are most hype-worthy – these are the places to look.

Security, automation and developer experience are also among the domains of DevOps where there is a great deal of opportunity at present to innovate – and, indeed, where adopting new tools and techniques will be critical for organizations that don’t want to be left behind as DevOps evolves.

The post Security, automation and developer experience: The top DevOps trends of 2024 appeared first on SD Times.

If you think of application environments like a pizza (or in my case, a vegan pizza), IaC is just the unbaked dough, and the individual IaC files alone are simply flour, salt, yeast, water and so on. Without the other necessary components like the data, network topology, cloud services and environment services – the toppings, if you will – you don’t have a complete environment. Additionally, the need for proper governance, cost controls, and improved cross-team collaboration has become even more critical. 

While the needs of developers are application-centric, IaC is infrastructure-centric. There is a disconnect between the expectations of the development and operations teams that creates delays, security risks, and friction between those two teams. For IaC to be used effectively, securely and in a scalable manner, there are some challenges that need to be addressed.

Let’s discuss the top four challenges of IaC and how developer and DevOps teams can overcome these pain points and obstacles using Environments-as-a-Service (EaaS). 

Integrating IaC assets 

One of today’s central challenges is in generating a pipeline that provides a way to deploy infrastructure assets continuously and consistently. Many DevOps organizations are sitting on top of mountains of IaC files, and it’s a monumental task for these teams to understand, track and deploy the right infrastructure for the right use case. 

EaaS solves this problem by automating the process of discovering, identifying, and modeling infrastructure into complete, automated environments that include all the elements that the end user requires. 

Furthermore, EaaS solutions eliminate the application environment bottleneck and enable faster innovation at scale by defining elements in modular templates, otherwise known as “blueprints,” and help organizations manage the environments throughout the entire application life cycle. Existing IaC scripts can easily be imported and managed in an infrastructure stack, or users can choose to build “blueprints” from scratch. 

Distributing the right environments to the right developers

Using the wrong environment definitions in different stages of the SDLC is like using a chainsaw to slice your pizza; it won’t get the job done right and could create more problems. It’s crucial for developers to have access to properly configured environments for their use case. Developers don’t necessarily have the expertise to properly configure environments. Yet, in some cases, they’re expected to, or they attempt to do it because there aren’t enough people in their organization with the cloud infrastructure skills to do so in a timely manner. The result could be an environment that’s horribly misconfigured like putting sauce on top of your pizza (sorry, Chicago) or even worse, pineapple and ham (not sorry).

Organizations should distribute complete environments to their developers with “baked-in” components and customized policies and permissions. To accomplish this, most EaaS solutions have the ability to provide a self-service environment catalog that simplifies this process, while also dramatically reducing provisioning times. Operations teams can take advantage of role-based policies, so developers have access only to the environments that are appropriate for their use case, ensuring consistency throughout the pipeline.  Consumption of this service should be available via command line or API, so it can seamlessly integrate into your CI/CD pipeline.

Managing the environment life cycle & controlling costs 

The orchestration of environments is only one piece of the pie. It has to be served, consumed, and then, of course, you have to clean up afterward. In addition to configuring and serving up the right environments for the developers to consume, EaaS allows for seamless enforcement of policy, compliance, and governance throughout the entire environment life cycle, providing information on how infrastructure is being used. During deployment, end users can set the environments for a specified runtime, automating teardown once resources are no longer required to ensure the leanest possible consumption of cloud resources. 

We all know there’s no such thing as a free lunch, so understanding and managing cloud resource costs is a crucial element of the full environment life cycle and demonstrates the business value of a company’s infrastructure. By leveraging auto-tagging and custom-tagging capabilities, businesses can easily track how environments are deployed in a centralized way, providing complete operational transparency, and ensuring resources are being provisioned in line with an organization’s prescribed standards. Understanding the business context behind cloud resource consumption allows businesses to optimize costs and better align those expenses with specific projects, applications, or development teams.

Creating a reliable IaC infrastructure 

There are several critical steps to ensure infrastructure reliability. This includes depositing IaC code into a source control repository, versioning it, running tests against it, packaging it, and deploying it in a testing environment – all before delivering it to production in a safe, secure, and repeatable manner. 

In maintaining a consistent and repeatable application architecture, the objective is to treat IaC like any application code. You can meet the changing needs of software development by creating a continuous IaC infrastructure pipeline that is interwoven with the software development and delivery process, leveraging best practices from software delivery, and transposing them to the infrastructure delivery process.

To ensure that your infrastructure is reliable, you must consider the larger picture. IaC has become ubiquitous and has certainly advanced infrastructure provisioning, but that’s where it ends. Organizations need to start thinking about not just configuring and provisioning infrastructure but managing the entire life cycle of complete environments to realize the true value of infrastructure. Just like you wouldn’t go to a pizza parlor and order a blob of raw dough, you wouldn’t serve your developers just the infrastructure – they need the complete environment.

Using EaaS, developers are able to achieve their project objectives, support the entire stack, integrate IaC assets, and deliver comprehensive environments needed to orchestrate the infrastructure life cycle. Buon appetito!

The post Why using IaC alone is a half-baked infrastructure strategy appeared first on SD Times.

Provisioning monitoring checks by hand is slow, too slow when the checks need to keep up with an application that is evolving quickly. This stays true no matter which monitoring service or platform you are using.

Another problem is with documentation. Who documents what the monitoring setup should look like, as a whole and in its parts? What is each check’s configuration? What about the alerting logic for when things catch on fire? It’s up to you to put that all down on paper to avoid the risks that come with having everything live in somebody’s head.

Possibly the worst and often unseen issue, though, is that manual monitoring workflows do not fit in the bigger picture. They do not tie into how software is being built, really. They lie by the wayside, hopefully getting enough of our attention to prevent costly outages while not diverting it too much from our real objective: shipping incredible applications.

Click here to read the full story on ITOps Times.

The post It’s time to embrace Monitoring-as-Code appeared first on SD Times.

Due to the new Artifcatory support for Terraform, developers are enabled to manage IaC configurations with existing DevOps processes in order to manage shared binaries. 

With tools such as Terraform, DevOps teams are able to avoid drift or mismatches between applications and the configurations used to run them. 

“JFrog’s support for Terraform ensures developers can retain mechanisms to centrally manage and share their applications’ cloud infrastructure, by utilizing JFrog Artifactory as a registry for Providers and Modules and as a reliable state backend,” said Yoav Landman, co-founder and CTO of JFrog. “Artifactory also acts as a local cache for other Terraform registries. This allows developers to use a single platform without the overhead of maintaining multiple systems, all while using robust and secure DevOps solutions they already utilize across their development pipelines.”

According to JFrog, this added support makes the platform a one-stop choice for developers looking to ensure continuous delivery of software updates utilizing the binaries themselves or pairing them with the proper infrastructure configurations needed to run them.

“We’re very excited that JFrog Artifactory is adding support for Terraform modules, providers, and state files,” noted Dr. Stefan Freitag, senior service engineer at RWE Supply and Trading. “For us, this new functionality comes at the right time and brings us one step forward in our DevOps journey.”

JFrog’s support for Terraform files is available now. For more information, visit the website.

The post JFrog extends support for Terraform appeared first on SD Times.

KICS automatically parses infrastructure-as-code files of any type to detect insecure configurations that could expose applications, data and services to attack.

Users of Ansible, AWS CloudFormation, K8S or Terraform can now scan their IaC and manage IaC vulnerabilities alongside other comprehensive security scan results with GitLab’s vulnerability management capabilities.

“The fact that we now see infrastructure-as-code (IaC) integrated as part of any DevOps pipeline shows that application security must now extend far beyond application source code,” added Razi Sharir, CPO at Checkmarx. “The world runs on code, and we secure it, from source code to open source to infrastructure-as-code.”

Additional details on KICS are available here. 

The post Checkmarx KICS now integrated into GitLab 14.5 appeared first on SD Times.

IaC speeds up deployment, eliminates one-off non-standard configurations, and builds consistency and repeatability into the software provisioning process. But while this type of automation is a major boon for developers, it’s a potential source of vulnerability if there’s a flaw in the template itself. This is the flipside of agility: when simple configuration changes can leave sensitive data and private servers exposed in the cloud, the security implications of this level of automation are profound.

As such, the next step in shifting left in software development is to push security compliance to the very start of the process by ensuring that the IaC templates, the initial foundation upon which code is written, are free from known vulnerabilities. Up until now, the nature of the templates has proven to be problematic for traditional security teams, and may explain why they have continued to favour a reactive posture, because the software platforms on which the templates are written – CloudFormation, Terraform etc – are not familiar to them. However, a new generation of automated scanning tools enables pre-deployment static analysis of IaC templates, treating them just like any other piece of code and performing the relevant security and compliance tests.

Just as tools have been developed specifically for the cloud to automate and speed up all stages of the software process, so too do security solutions now exist that support the non-stop CI/CD paradigm from beginning to end. But tools won’t change the post-deployment fix culture on their own – that will only happen when developer and security teams work closer together rather than in opposition to each other. Security must accept that it’s both possible and desirable to enforce corporate policies before run-time, while developers must allow security to be involved from the ground up.

The closer we can shift “hard left” and push security consciousness to the very start of the process, the nearer we will be to establishing a threat protection model that’s based on proactive prevention rather than reactive remediation.

The post Shifting hard left: Why securing IaC is the next step in compliant software development appeared first on SD Times.

Terratag allows users of the popular IaC framework Terraform to streamline time-consuming resource tagging processes by enabling the automatic creation and maintenance of tags across an entire set of AWS, Azure, and GCP resources, according to env0. 

It also aims to provide increased visibility into cloud costs and offers a more efficient way to define and share best practices for tagging, labelling, and backtracking cloud usage. 

The lack of automation, governance, and inefficiency in the resource tagging process forces many developers to submit support tickets to provision resources while building new applications and services, according to env0.

“While collaborating with our customers that use Terraform, we quickly understood the cost and possible errors that result from manual resource tagging,” said Omry Hay, the co-founder and CTO of env0. “We decided to automate this task, and make it freely available. Our hope for open sourcing Terratag is that it will foster new learnings and additions to streamlining the tagging process, and provide ease-of-use in infrastructure-as-code environments.”

More information is available here

The post SD Times Open-Source Project of the Week: Terratag appeared first on SD Times.

“Most IaC tools are immature, modern architectures are complicated, and seemingly minor changes to infrastructure code sometimes cause severe bugs, such as wiping out a server, a database, or even an entire data center,” Brikman wrote in a development blog post.

Originally an internal utility for maintaining Gruntwork’s own Infrastructure as Code Library, which hundreds of businesses rely on for ready-made infrastructure code, Terratest is a Go library that makes it easier to write automated tests for, as listed on the project’s GitHub:

• Testing Terraform code
• Testing Packer templates
• Testing Docker images
• Executing commands on servers over SSH
• Working with AWS APIs
• Making HTTP requests
• Running shell commands
• And much more

“I won’t claim that writing these tests is actually easy—it’s takes a considerable amount of work to get them just right—but it’s worth the effort, because these tests can run after every commit and verify that the code works as expected, thereby giving you the confidence to make the code changes you need,” Brikman wrote.

A breakdown of how to use the library along with copious examples can be found in the blog post and on the GitHub repository.

Top 5 projects trending on GitHub this week:

1. Architect awesome: A backend architecture technical map
2. Gvisor: Container runtime sandbox
3. Proton–native: a React environment for cross-platform native desktop apps
4. Gh-ost: GitHub’s Online Schema Migrations for MySQL
5. ELF: a platform for game research

The post SD Times GitHub Project of the Week: Terratest appeared first on SD Times.