Configuration Management / DevOps / IT Operations
Why Deployment Automation is Becoming a Necessity
The development of corporate information systems is gradually changing the requirements for their operation.
Traditional enterprise infrastructure often consists of a relatively small number of large software complexes: ERP, CRM, document management systems, corporate data warehouses, and auxiliary services.
Such systems can be updated several times a year according to approved regulations.
However, the development of web portals, mobile applications, APIs, and microservice architecture creates a different model.
Software components become smaller, but their number increases.
Each service has its own configuration, network dependencies, software version, and interaction mechanisms with corporate data.
If all these components are configured manually, the risk of configuration errors increases. For example, one server may use a different library version, another may have outdated database connection parameters, and a third may have incorrect network permissions.
As a result, applications behave differently even when using the same codebase.
To solve this problem, companies need to develop Configuration Management — systematic management of settings for software, operating systems, and infrastructure components.
Combined with automated deployment and version control, it allows for reproducible environments, reduces manual operations, and increases change reliability.
Configuration as part of Enterprise Architecture
Configuration should be understood as more than just a single application parameter file.
In an enterprise environment, it covers multiple levels.
Operating System
These include package versions, system services, kernel parameters, user accounts, and file permissions.
For example, an application server might require a specific Java version and defined memory settings.
Network
The configuration includes ports, routes, DNS, firewall rules, and secure connection settings.
For example, the integration service should have access to the defined ERP API, but not to all other servers in the corporate network.
Application Environment
These are service addresses, logging parameters, timeouts, database connection pools, and other settings.
Data Schema
Table structures, indexes, procedures, and integrity rules should also be under change control. However, they should not be equated with ordinary configuration parameters.
Changing the structure of a production database can affect stored information and require a separate migration plan.
Secrets and Credentials
Passwords, private keys, and access tokens are a separate category of configuration information.
They require enhanced protection and should not be stored uncontrollably in regular code repositories.
Infrastructure as Code: infrastructure as a described state
One of the important approaches is Infrastructure as Code (IaC).
It involves formalizing infrastructure configuration in the form of files and software descriptions that can be stored in a version control system. For example, instead of manually installing packages on each server, an administrator creates a corresponding description.
Based on this, an automated tool installs the necessary software and ensures a defined state.
Puppet
Puppet uses declarative configuration descriptions. For enterprise environments, this allows defining which packages, services, and files should exist on the server.
For example, the system can monitor the presence of an installed web server and its configuration files.
If the actual state differs from the defined state, Puppet can perform appropriate actions to reconcile it.
Chef
Chef uses recipes and corresponding programming constructs to manage configurations.
It can be used for automated server provisioning, software installation, and maintaining standard environments.
Ansible
Ansible allows describing setup procedures in YAML format. It can use SSH to manage Linux servers without needing to install a specialized continuously running agent on each node.
For example, an administrator can centrally install a package, create a user, or update a service configuration.
However, the absence of an agent does not mean that managed servers do not require appropriate preparation and access control.
Practical Ansible Example
Consider a hypothetical corporate web service running on Ubuntu Server 16.04 LTS. To deploy it, you need to install Nginx and ensure the corresponding service is running.
In Ansible versions available in early 2017, the following playbook can be used:
---
- hosts: webservers
become: yes
tasks:
- name: Install Nginx
apt:
name: nginx
state: present
update_cache: yes
- name: Ensure Nginx is running
service:
name: nginx
state: started
enabled: yes
This is a simplified example. In an industrial environment, it is also necessary to control configuration files, network permissions, logs, and result verification procedures.
The advantage is that a single description can be applied to multiple servers.
This reduces the risk of accidental differences between nodes.
Declarative and Procedural Management
When automating, it is important to distinguish between two approaches.
The procedural approach describes a sequence of actions. For example: install package, create directory, copy file, start service.
The declarative approach describes the desired state. For example: a package should be installed, a file should have a defined content, and a service should be running.
Many tools combine both approaches.
For enterprises, it is especially important to ensure execution repeatability. If a script is run a second time, it should not unnecessarily create duplicates, change correct settings, or disrupt service operation.
This property is called idempotency.
However, not all administrative operations are inherently idempotent. For example, re-executing an SQL script for adding business records can lead to information duplication.
Therefore, configuration management and data management must have separate control mechanisms.
Kubernetes: declarative application management
Kubernetes uses a desired state model. The administrator defines the number of application instances, the container image, resource requirements, and corresponding configuration parameters.
The platform attempts to maintain this state.
For example, if three API instances are supposed to be running and one fails, the controller will try to restore the required number.
This approach can reduce manual intervention.
However, Kubernetes does not independently determine which application version meets business requirements or if it is compatible with a new database schema.
Such decisions should remain part of corporate Change Management.
ConfigMap
Kubernetes ConfigMap can be used to store non-sensitive application parameters. For example, it can contain the address of an internal API, the logging mode, or individual application service parameters.
This allows the Docker image to be used in different environments by changing only the relevant configuration.
Secret
For credentials, Kubernetes has a Secret object.
However, in Kubernetes 1.5, it should not be assumed that storing a value in a Secret automatically provides its full cryptographic protection across all cluster components.
For example, a Base64 representation is not encryption.
Therefore, it is necessary to restrict access to the Kubernetes API, protect etcd, and apply appropriate infrastructure measures.
It is particularly important to consider that advanced RBAC mechanisms and secret encryption, known from later versions, cannot be unconditionally transferred to the January 2017 architecture.
Helm 2: Managing Kubernetes Resource Sets
For complex enterprise applications, a single manifest is insufficient.
For example, a web portal may include multiple Deployments, Services, ConfigMaps, and other objects.
Helm is being developed to manage such sets.
The project was formed by merging several previous initiatives, and Helm 2.0 was released in 2016.
Its important feature is the use of the server-side component Tiller.
Helm allows describing a set of Kubernetes resources as a Chart.
For enterprises, this creates an opportunity to standardize the deployment of typical components.
However, the security of Tiller and the rules for granting it permissions must be considered.
In January 2017, Helm should not be described as if it already uses the serverless architecture of the later Helm 3.
These are different generations of the platform.
Version Control and Environment Reproducibility
One of the central principles of reliable deployment is storing configurations in a version control system.
Git can be used for this purpose.
For example, an enterprise may have separate directories with descriptions of test and production environments.
Every change undergoes review and receives a corresponding history.
Advantages
It is possible to determine who changed a parameter and when it happened.
It is also possible to compare two versions of a configuration and reproduce the previous state of certain software settings.
Limitations
Reverting a Git repository to a previous version does not automatically restore the production database.
For example, if a new application version changed table structures and created new records, reverting the old configuration file will not undo the consequences.
Therefore, database management system (DBMS) changes require separate backup and migration procedures.
Secrets also need to be controlled.
Passwords and private keys should not be stored in a regular repository accessible to all development team members.
CI/CD and Automated Deployment
Configuration Management is closely related to Continuous Integration and Continuous Delivery.
Tools such as Git, Jenkins, TeamCity, GitLab CI, and others can be used to create the appropriate process.
A typical sequence includes code retrieval, building, automated testing, package creation, and deployment.
In a containerized environment, the result of the build can be a Docker image.
From Development to Testing
- A developer makes a change to the repository.
- The system automatically runs checks.
- Upon successful compilation, an image with the corresponding version number is created.
Deployment to Test Environment
The image is deployed to a controlled infrastructure.
Integration tests, API verification, and interaction testing with relevant databases are performed.
Production Deployment
After approval and completion of established procedures, the change is moved to the production environment.
For critical enterprise systems, it is advisable to maintain approval checkpoints.
Automation does not necessarily mean fully automatic implementation of every change without the involvement of responsible persons.
Docker 1.13: New Deployment Management Capabilities
On January 19, 2017, Docker introduced Docker Engine 1.13.
The release expanded Swarm Mode functionality, in particular, allowing the use of Compose files for deploying service sets via docker stack deploy.
This is an important improvement for enterprises.
Previously, Compose was mainly used to describe multi-component environments, especially during development.
Now, defined configurations can be applied to deploy services in a Swarm cluster using the relevant capabilities of the new release.
Instance Count
In the configuration, you can define the desired number of service replicas.
Update Policy
For services, you can set gradual update rules.
Placement Constraints
You can define conditions under which a service starts on specific nodes.
However, it is necessary to check the compatibility of specific Compose file capabilities with Swarm Mode.
Not all parameters of a normal local launch have the same behavior in cluster mode.
DBMS Structure Change Management
For enterprise information systems, changes in database structure pose a particular danger.
For example, a new CRM version requires an additional column, index, or table modification.
If the change is performed manually, the test and production environments can gradually diverge.
Specialized schema migration mechanisms are used for this purpose.
Flyway
Flyway allows for the sequential execution of versioned SQL scripts.
For example:
V1__create_customers.sql
V2__add_customer_status.sql
V3__create_customer_index.sql
The system tracks which migrations have already been executed.
Liquibase
Liquibase allows describing database schema changes and controlling their application.
Specialized rollback mechanisms can be used for specific platforms. However, it is important to note that not all DDL operations equally support transactional execution.
Therefore, automation does not eliminate the need to test migration on a real volume of data.
Compatibility during gradual deployment
If a new version of the application is deployed gradually, old and new instances may run simultaneously for some time.
In such a situation, the database structure must be compatible with both versions.
For example, adding a new optional column is usually easier to reconcile with a gradual update than immediately renaming a field used by all old instances. Therefore, for complex systems, it is advisable to separate schema extensions, application logic migration, and subsequent removal of obsolete components.
Monitoring and Operational Reliability
Automated deployment does not guarantee that the system will remain operational.
It is necessary to monitor the actual state of the infrastructure.
Metrics
For containerized environments, it is important to analyze CPU, RAM, disk, and network usage. Application-specific metrics need to be monitored separately.
For example, an API might be operational but return errors when creating orders.
Prometheus
Prometheus is an open-source time-series monitoring platform that is gradually gaining traction in containerized environments.
It allows collecting metrics and using them to analyze the state of services. However, in a corporate infrastructure, it is necessary to correctly define metrics and alerting rules.
The mere collection of a large number of metrics does not automatically ensure the detection of all problems.
Centralized Logging
Elasticsearch, Logstash, and Kibana can be used for event analysis.
These components allow collecting logs from various servers and applications. However, it is necessary to establish retention and security rules.
Logs should not uncontrollably contain passwords, tokens, or sensitive customer information.
Fault Tolerance and Incident Scenario Control
Distributed applications create additional redundancy opportunities but simultaneously increase the number of dependencies.
For example, a web portal can consist of an authentication service, a product catalog, order processing, and an external ERP. Failure of a single component should not always lead to complete portal unavailability.
Timeouts
For network requests, it is necessary to define a maximum waiting time.
If the ERP does not respond, the web service should not indefinitely hold all worker threads.
Retries
Retries can help during temporary outages. However, it is necessary to control the number of retries and consider the idempotency of business operations.
For example, re-creating an order can lead to duplication.
Circuit Breaker
A Circuit Breaker allows temporarily limiting requests to a component that systematically returns errors.
For Java applications, corresponding libraries can be used, in particular Netflix Hystrix. However, the call fallback should have a clear business outcome.
For example, if the messaging service is unavailable, an order can be created with a delayed notification.
But if the system that confirms stock availability is unavailable, the company must determine whether to allow orders to be placed without such confirmation.
Practical Scenario: Automated Deployment of an Enterprise API
Let’s consider a hypothetical company that has an on-premises ERP and several external services.
The team maintains an API for retrieving the product catalog and placing orders.
Previously, new versions were installed manually on three application servers. This created a risk of configuration differences.
Stage 1. Standardization
The team defines the required software environment versions, network rules, and access order to the ERP.
Stage 2. Configuration Control
Server parameters are described using Ansible.
The application is built into a Docker image.
Stage 3. Version Control
Source code and configuration descriptions are stored in Git.
Every change undergoes verification.
Stage 4. Automated Testing
Jenkins runs unit and integration tests.
After successful execution, a new image is created.
Stage 5. Deployment
The enterprise uses Kubernetes 1.5 or Docker Swarm Mode, depending on the chosen infrastructure.
The new version is launched gradually.
Stage 6. Monitoring
Monitoring checks technical availability and actual business operation results.
Stage 7. Rollback
For application code, the ability to revert to the previous image is maintained.
For DBMS changes, a separate migration and recovery plan exists. This allows for reducing update risks without the erroneous assumption that container rollback automatically undoes all data changes.
Operational Model
Configuration automation requires appropriate responsibility allocation. The development team defines software dependencies and verifies functionality.
Infrastructure specialists are responsible for servers, networks, storage systems, and the cluster platform. DBA administrators control data structure, backups, and performance.
The system integration team ensures API compatibility and message exchange. Business process owners approve critical functionality changes.
It is especially important to avoid situations where an automated deployment process has unlimited administrative privileges across the entire corporate infrastructure.
Separate accounts and the corresponding minimum set of privileges must be established for each tool.
Intecracy Group Competencies in Deployment Automation
Managing distributed applications requires a combination of technological specializations. Data Management IG specializes in ETL, MDM, Data Quality, DWH, and enterprise data governance. These competencies are crucial for controlling information dependencies and correctly migrating changes between environments.
IQusion IT operates in the fields of system integration, electronic document management, and BPM. These areas are important for ensuring the stability of application interfaces and process logic after updates. Softline IT has expertise in server and network infrastructure, storage systems, and enterprise infrastructure projects. These are directly related to managing computing nodes and the fundamental reliability of the environment.
SL Global Service specializes in cloud integration, virtualization, and hybrid infrastructure. These areas are significant for the automated deployment of applications in various computing environments.
At the same time, technological automation must comply with internal change management procedures.
Practical Implementation Guidelines
The development of Docker, Kubernetes, and distributed applications is changing the nature of enterprise information system operations. The increasing number of components makes manual configuration increasingly inefficient.
To ensure reproducibility and reliability, enterprises need to develop Configuration Management, Infrastructure as Code, and automated deployment. Ansible, Puppet, Chef, Jenkins, Docker, and Kubernetes enable the implementation of various components of this approach.
Docker 1.13 extends cluster deployment capabilities, and Helm 2 allows organizing configuration packages for Kubernetes. However, automation should not eliminate control over critical changes.
Secret management, API compatibility, testing database schema changes, and disaster recovery procedures are of particular importance.
A robust enterprise infrastructure is defined not by the number of automated operations, but by the enterprise’s ability to reproducibly implement changes, control their consequences, and restore information systems without loss of critical data.
Author