Managing Data Integrity in Hybrid Infrastructure

Database Administration / IT Operations / Data Management

Why the Traditional Database Administrator Role is Changing

The Database Administrator (DBA) has been one of the key specialists in a corporate IT department for many years.

Their traditional tasks include installing and configuring database management systems, backups, access control, performance optimization, and disaster recovery. For enterprises using ERP, CRM, financial, and manufacturing systems, these functions are critically important.

However, the development of corporate information architecture is gradually changing the requirements for DBA work.

Databases are increasingly hosted in virtualized environments, integrated with cloud services, used by distributed applications, and become information sources for analytical platforms. At the same time, automation of administrative operations reduces the volume of some routine tasks.

For example, managed cloud services can perform some installation, update, and backup operations without direct involvement from the enterprise administrator. However, this does not mean the disappearance of the DBA profession.

Conversely, more complex information architectures require specialists who understand not only the mechanisms of a specific DBMS but also the dependencies between information systems, business requirements, and the principles of ensuring the availability of corporate data.

The role of the DBA is gradually shifting from an administrator of an individual server to a specialist who ensures the reliability, performance, security, and manageability of the corporate information platform.

Traditional DBA Functions Remain Essential

Despite the development of new technologies, the fundamental tasks of database administration remain relevant.

DBMS Installation and Configuration

The administrator is responsible for preparing the execution environment.

They define memory usage parameters, configure file structures, disk resources, transaction logs, and network connections. For Microsoft SQL Server, Oracle Database, PostgreSQL, and MySQL, the corresponding procedures have significant differences.

For example, Oracle uses its own memory and process architecture, while PostgreSQL has a different model for managing server processes and buffers.

Therefore, universal recommendations cannot always be directly transferred between platforms.

Backup and Recovery

The DBA ensures backups are created and verifies the recoverability of information. It is important that the successful completion of the backup task is not sufficient confirmation of reliability.

It is necessary to regularly test recovery on a separate server and monitor the completeness of the required transaction logs.

Coordination of procedures with business requirements for RPO and RTO is of particular importance.

Performance Optimization

The administrator analyzes SQL queries, index structures, locks, memory usage, and I/O operations.

They can assist developers in optimizing inefficient queries and determining the need for changes to the information model. However, increasing performance is not always achieved by adding server resources.

In some cases, a proper index or a change in query execution method can have a greater effect than purchasing new hardware.

Access Management

The DBA configures accounts, roles, and permissions according to corporate policy. They also oversee administrative privileges and interact with information security specialists.

It is important to separate technical implementation from the decision to grant access.

An administrator should not independently determine which financial or personal data a specific employee is allowed to view without the approval of the responsible department.

Virtualization Changes Infrastructure Management

One of the significant trends in recent years has been the spread of server virtualization. VMware vSphere, Microsoft Hyper-V, and other platforms are used for corporate databases.

Virtualization allows for more flexible allocation of computing resources, creation of test environments, and simplification of certain server migration procedures.

However, it introduces additional requirements for performance administration.

Shared Physical Resources

In a traditional environment, a DBA might work with a dedicated physical server. In a virtualized infrastructure, multiple DBMSs can share resources of a single physical node.

If other virtual machines create a significant load, the database’s performance may degrade without obvious changes within the DBMS itself.

Therefore, the DBA needs to interact with virtualization administrators and understand the allocation of CPU time, RAM, and disk operations.

Resource Over-allocation Problem

Virtualization platforms allow assigning resources to virtual machines that are shared by multiple workloads.

However, for critical databases, over-consolidation can lead to unstable response times.

Transactional systems with high disk I/O latency requirements are particularly sensitive.

Therefore, the DBA must participate in resource planning and performance testing of virtualized environments.

DBaaS: What Tasks are Transferred to the Cloud Provider

The development of Database as a Service creates a different model for operating enterprise DBMS.

For example, Amazon RDS allows an enterprise to create a managed MySQL, PostgreSQL, Oracle, or Microsoft SQL Server database. The provider handles some infrastructure functions, including environment provisioning, backup, and certain administrative operations.

This reduces the need for manual operating system and base software installation.

However, a significant portion of DBA tasks remains with the enterprise.

Query Optimization

A cloud provider cannot automatically fix flawed logic in a complex enterprise application.

If an ERP executes inefficient SQL queries, increasing computing resources will only partially compensate for the problem.

The DBA must analyze the workload, index structure, and data schema.

Information Management

DBaaS does not define which tables contain personal data, who is responsible for directories, or what business rules apply.

Such issues fall under corporate architecture and Data Governance.

Cost Control

The cloud model creates a new task – evaluating the economics of resource utilization. The DBA must understand how configuration choices, disk space, backups, and network traffic affect costs.

For example, an inefficient query can increase not only response time but also the need for more expensive resources.

Thus, performance optimization directly impacts the company’s operational expenses.

Hybrid Infrastructure Administration

In many companies, cloud technologies do not replace on-premises information systems but are used in conjunction with them. For example, the main ERP continues to operate in its own data center, while the partner portal uses a cloud database.

This creates additional integration tasks.

It is necessary to monitor network connections, synchronization, data consistency, and disaster recovery mechanisms.

Replication and Latency

If data is transferred between sites asynchronously, the DBA must monitor replica lag. For example, for a report generation system, an acceptable latency might be several minutes.

However, for a financial balance check service, such a delay may be unacceptable.

Therefore, the DBA needs to understand the business purpose of the information.

Responsibility Distribution

Several administrative teams may be involved in a hybrid environment. One is responsible for the local DBMS, another for the cloud platform, and the third for integration processes.

Without a clear division of responsibilities, incident diagnostics can be prolonged.

Therefore, it is important to document dependencies and establish joint response procedures.

Automation of Routine Administration

Administrators increasingly use scripts and specialized tools to automate repetitive operations. These include creating backups, checking service status, deploying test environments, and monitoring disk space usage.

For Windows environments, PowerShell can be used.

In Linux and UNIX-like systems, shell scripts, Python, and standard system tools are widely used.

For server configuration automation, Puppet, Chef, and Ansible can be used.

Automation must be controlled

Not every administrative operation is advisable to perform without verification. For example, automatic deletion of old backups can lead to the loss of a necessary recovery chain.

Similarly, mass changes to DBMS configuration can affect multiple production systems.

Therefore, scripts must be tested and accompanied by logging of results.

For critical operations, additional approval and change control may be required.

Configuration Reproducibility

One of the important advantages of automation is the ability to create identical environments. For example, a DBA can prepare a script for deploying a test DBMS with the necessary parameters.

This reduces reliance on manual setup and simplifies the verification of new configurations.

However, an automated environment must account for differences between test and production data.

Using real confidential information in test databases requires appropriate protection rules.

The importance of performance monitoring and analysis is growing

The traditional approach to administration often involves reacting after receiving a problem notification. However, for critical enterprise systems, it is important to detect signs of degradation before a complete failure.

DBAs must monitor not only the availability of the DBMS process but also the actual performance. For example, the database may respond to queries, but the execution time of a key business operation gradually increases.

Reasons can include table growth, changes in SQL query plans, blocking, or insufficient disk subsystem performance.

Key Monitoring Metrics

These metrics should be analyzed in the context of the specific workload. For example, high RAM usage for a DBMS is not necessarily a problem if it is effectively used for caching.

At the same time, an increase in disk operation latency can directly impact the performance of a transactional system.

The DBA’s Role in Change Management

Modernization of enterprise applications is often accompanied by changes in database structures. Developers add tables, modify indexes, and alter procedures.

If such changes are made without administrator involvement, performance and recovery issues can arise.

Therefore, the DBA must participate in the Change Management process.

Change Verification

Before implementation, it is necessary to assess the impact of the new structure on existing queries and integration processes.

For example, changing a column type can break compatibility with an ETL procedure or an analytical view.

Performance Testing

Changes must be tested with characteristic data volumes.

A query that executes quickly on a test table with a thousand records may create a significant load on a production database with millions of rows.

Rollback

For critical changes, a recovery plan must be in place. However, not every structural modification can be safely reverted by simply executing a reverse SQL script.

This is especially true for changes that involve the transformation or deletion of information. Therefore, the DBA must assess the consequences of changes before their execution.

SQL Server 2016 Future Opportunities

Throughout 2015, Microsoft introduced preview versions of the next generation of SQL Server. On May 27, the first public Community Technology Preview of SQL Server 2016 was released.

Among the announced directions are Always Encrypted, Stretch Database, improvements to In-Memory OLTP, and analytics tools.

On October 28, Microsoft released CTP 3.0, and on November 30, CTP 3.1, intended for testing and evaluating new features.

For DBAs, these directions demonstrate the further complication of the DBMS role.

Information Protection

Always Encrypted focuses on additional protection of sensitive values using client-side cryptographic processing.

This creates new requirements for key management and interaction between applications and the DBMS.

Hybrid Storage

Stretch Database demonstrates the ability to combine on-premises transactional data with cloud resources.

This approach will require new competencies in managing data placement and network interaction.

Advanced Analytics

The integration of SQL Server with analytical computing tools indicates a gradual convergence of traditional DBMS and specialized analytical tools. At the same time, it should be emphasized that as of December 2015, SQL Server 2016 remains a pre-release product, not a generally available stable version.

For enterprise systems, companies should focus on supported releases, particularly SQL Server 2014, and separately evaluate the prospective capabilities of the new platform.

PostgreSQL and the Development of Open Source DBMS

Open source database management systems also continue to develop. For PostgreSQL, an important direction is increasing performance, improving replication mechanisms, and expanding SQL functionality.

As of the end of December 2015, the stable branch PostgreSQL 9.4 is available to companies. On December 18, PostgreSQL 9.5 Release Candidate 1 was published.

The new version includes, among other things, UPSERT, Row-Level Security, and additional capabilities for working with large data sets. However, at this moment, PostgreSQL 9.5 is still undergoing final testing.

For DBAs, this means the need to evaluate not only the capabilities of new features but also the upgrade procedures and application compatibility. Migrating a production database to a new version requires testing SQL queries, drivers, extensions, and backup mechanisms.

Choosing an open-source DBMS also does not eliminate the need for qualified administration. The absence of license fees for the base software does not mean the absence of operational and support costs.

DBAs and Data Governance: a New Division of Responsibility

With the development of corporate data governance, the organizational role of the administrator is also changing. In the traditional model, the IT department often was responsible for almost all database-related issues.

However, this creates a problem: technical specialists do not always have the authority to define the correct content of business information. For example, a DBA may find duplicate counterparty records, but should not independently decide whether they belong to the same legal entity.

Similarly, an administrator can ensure the availability of a financial report, but is not responsible for the correctness of the methodology for calculating its indicators.

Role Separation

In a mature Data Governance system, responsibility is distributed among several participants. The Data Owner approves the rules for the use and quality of information.

The Data Steward monitors their practical implementation. The Data Architect defines the information architecture and data structures.

The DBA ensures the reliability, performance, security, and operation of the DBMS. This distribution does not imply strict isolation of specialists.

Instead, they should interact constantly.

For example, a decision to change a corporate directory may require simultaneous analysis of business rules, table structures, integration processes, and performance.

Practical Scenario: Modernizing DBA Operations in a Large Enterprise

Consider a hypothetical organization that uses multiple instances of Microsoft SQL Server, Oracle Database, and PostgreSQL. Some databases serve ERP and CRM, while others are used for document management and analytics.

Administrators work mostly reactively: they respond to incidents, manually check backups, and fulfill user creation requests.

The enterprise plans to improve infrastructure manageability.

First Stage: Inventory

A registry of all production and test DBMS is created. For each, the version, responsible administrator, criticality, application systems, and dependencies are determined.

This allows for the detection of unofficial or outdated databases that were not previously under centralized control.

Stage Two: Standardization

The enterprise forms typical rules for configuration, backup, monitoring, and access management. Appropriate requirements are defined for different categories of systems.

For example, a critical ERP system receives stricter backup parameters than a secondary test environment.

Stage Three: Automation

• Routine checks are gradually transferred to automatic execution scripts.
• The monitoring system controls backup results, disk usage, and replication status.
• This reduces the number of manual operations.

Stage Four: Participation in Architecture Development

DBAs are involved in new service creation projects from the design stage. They assess requirements for transactions, indexing, availability, and data placement.

This approach allows for the identification of architectural risks before the production environment is launched.

Stage Five: Performance Measurement

Availability metrics, recovery time, incident frequency, and critical operation execution time are used to evaluate performance.

It is important to assess not the number of administrative commands executed, but the actual reliability of corporate information services.

What Competencies Do DBAs Need

The evolution of corporate technologies changes the requirements for administrator professional training. Deep knowledge of a specific DBMS remains essential.

However, additional competencies are becoming increasingly important.

Understanding Application Architecture

A DBA must understand how information systems use the database, which operations are critical, and the consequences of its unavailability.

Networking and Virtualization

To diagnose performance issues, it’s necessary to understand the specifics of virtual machines, network latency, and storage systems.

Automation

PowerShell, shell scripting, or Python skills allow for reducing the amount of routine work.

Information Security

An administrator must understand the principles of least privilege, logging, encryption, and backup protection.

Change Management

Database system modernization requires planning, testing, and dependency assessment.

Business Interaction

A DBA must be able to explain technical risks in terms of business consequences. For example, it is not enough to inform management that the backup replica is 20 minutes behind.

It is necessary to explain what potential operational loss this creates in the event of a failure.

Role of Intecracy Group Competencies

The changing role of the DBA demonstrates the interdependence of data management, infrastructure, and corporate business processes. Data Management IG specializes in enterprise data, ETL, MDM, Data Quality, DWH, and analytics. These areas define the requirements for information reliability, its structure, and integration between systems.

IQusion IT competencies in system integration, electronic document management, and BPM are important for understanding how corporate applications use databases and what business consequences changes in their operation may have. Softline IT specializes in server, network infrastructure, and storage systems. Relevant competencies are necessary to ensure the performance, backup, and fault tolerance of DBMS.

The SL Global Service division covers cloud integration, virtualization, and hybrid infrastructure. It is these technologies that form additional requirements for the administration of corporate databases in a distributed environment.

Combining different competencies allows for improving the quality of operation without the need to assign all architectural and organizational tasks to a single specialist.

Practical Guidelines for Implementation

The role of the database administrator is gradually changing with the development of corporate information systems. Traditional functions of backup, recovery, access control, and performance optimization remain critically important.

However, virtualization, cloud services, and distributed integration require additional knowledge in the areas of infrastructure, automation, and corporate architecture. At the same time, the development of Data Governance allows for a clearer distinction between the technical responsibilities of a DBA and the business responsibility for the content of information.

In the coming years, administrators who can not only maintain the operability of DBMS but also participate in the design of reliable information platforms will become increasingly valuable.

The future of the DBA profession is defined by the transition from manual maintenance of individual databases to systemic ensuring the availability, performance, security, and manageability of the corporate information infrastructure.