Geographic redundancy for enterprise data

In the spring of 2024, Ukrainian enterprises once again faced large-scale threats to their energy infrastructure. Russian strikes on generation facilities and electrical substations demonstrated that even the relatively stable operation of corporate data centers does not guarantee their future availability. For large organizations, geographical distribution of information resources, managed replication, independent backups, and the ability to restore corporate data without compromising its integrity are becoming critically important.

1. April 2024: Energy Infrastructure Under Attack Again

On March 22, 2024, Russia launched a new wave of massive attacks on the Ukrainian energy system. Strikes were carried out against power plants, hydroelectric facilities, substations, and other critical infrastructure.

Particularly severe consequences were observed in the Kharkiv region, where damage to energy facilities caused widespread power outages. On April 11, another attack occurred. According to NPC “Ukrenergo”, generation facilities and substations in the Odesa, Kharkiv, Zaporizhzhia, Lviv, and Kyiv regions were damaged. Emergency power outages were introduced in the Kharkiv region.

On April 27, Russian strikes again hit the energy infrastructure. DTEK reported serious damage to equipment at four of its six thermal power plants.

These events are of direct relevance to corporate IT.

Throughout 2022–2023, many Ukrainian companies have already upgraded their backup power supply, created remote data backups, and migrated some workloads to cloud environments.

However, new attacks demonstrate that the problem cannot be considered definitively resolved. The risks concern not only the physical loss of server equipment. Prolonged or repeated power outages can lead to unstable network channel operation, loss of availability of individual sites, interruption of integration processes, and accumulation of inconsistent data.

For an enterprise using ERP, CRM, WMS, MDM, and a corporate data warehouse, a situation where different components of the information architecture operate with varying availability becomes particularly dangerous.

For example, the central ERP may remain available in the cloud, but a regional warehouse will temporarily lose connection.

Another scenario is when the primary data center loses power, while the backup continues to operate, but some transactions have not yet managed to replicate.

In such conditions, not only the physical preservation of data but also the correct restoration of their consistent state is of fundamental importance.

The main task of corporate Data Management in April 2024 is to ensure information resilience against multiple infrastructure failures, not just a single accident.

2. Why a single data center redundancy is insufficient

A traditional corporate data center can have a high level of local fault tolerance.

Servers are equipped with redundant power supplies, disk arrays use RAID, network switches are redundant, and automatic restart mechanisms are configured for virtual machines.

These solutions protect against the failure of individual components.

However, they do not eliminate the dependency on physical location.

If the site loses power, cooling, or access to backbone channels, local redundancy may prove insufficient.

It is necessary to distinguish several levels of resilience.

No single level replaces the others.

For example, geographical replication can ensure a quick switch of the database to another site, but will not protect against accidental deletion of records if the corresponding changes have already been transferred to the replica.

Therefore, the corporate architecture should combine replication and independent backup.

3. Replication and Backup - Different Mechanisms

The terms Data Replication and Backup are often used together, but they perform different functions.

Replication maintains one or more instances of data in a relatively current state.

Backup creates controlled points to which one can return after data loss or corruption.

Let’s consider a simple example.

The enterprise uses PostgreSQL for its operational database.

The primary server is located in data center A. The standby server is in data center B.

WAL streaming replication is configured between them.

If the primary server becomes unavailable, the standby can be promoted to take over operations.

However, if a user accidentally deletes a critical table, replication will also propagate the corresponding changes.

Restoration will require an independent backup or another suitable rollback mechanism.

Comparison

For critical enterprise systems, it is usually necessary to use both approaches.

4. Synchronous and Asynchronous Replication

The choice of replication mode significantly affects performance, availability, and the risk of data loss.

Synchronous Replication

In the appropriate configuration, the system confirms a transaction only after receiving the necessary confirmation from the defined synchronous replicas.

This can reduce the risk of losing confirmed transactions during a failover.

However, geographical distance and network latency increase transaction commit time.

If the link to the synchronous replica is unavailable, new operations may be blocked according to the system configuration.

Therefore, stronger consistency guarantees can reduce write availability.

Asynchronous Replication

The primary server acknowledges the transaction without waiting for it to be committed on the remote replica.

This allows for reduced latency for users.

However, there can be a lag between the primary system and the backup server.

In case of sudden loss of the primary site, some of the latest transactions may be unavailable on the backup.

Architectural Comparison

It should be noted that specific synchronous replication guarantees depend on the DBMS, acknowledgment mode, and switching policy.

For example, in PostgreSQL, the synchronous_commit setting allows defining different levels of acknowledgment waiting. The choice of mode should align with transaction preservation requirements.

5. RPO, RTO, and Recovery Consistency

The enterprise needs to define not only the acceptable downtime but also the requirements for information consistency after recovery.

RPO is the maximum acceptable data loss in terms of time.

RTO is the target time for service recovery to an agreed level of operability. However, for multi-system processes, another question arises: to what consistent state should interdependent systems recover?

Suppose the ERP is recovered to 10:15.

WMS contains operations up to 10:20.

CRM has orders created at 10:25.

If all integrations are launched immediately, duplicates, missed messages, or balance discrepancies may occur.

Therefore, the enterprise needs a controlled information reconciliation procedure.

Example Target Requirements

These are conditional target values. Actual requirements are determined by Business Impact Analysis.

Recovery should be performed considering dependencies, not by independently launching each system.

6. Geographically Distributed Architecture

For Ukrainian enterprises in spring 2024, reducing common risks of the primary and backup sites becomes important.

Geographic redundancy should consider more than just distance.

It is necessary to evaluate:

• power dependencies;
• communication channels;
• administration accessibility;
• physical security;
• emergency startup capability;
• data storage rules;
• operational costs.

Logical Diagram

• Primary Data Center
• ERP · CRM · Transactional Databases · Integrations
• Replication
• Backup
• Secondary Data Center
• DB Replicas
• Backup Applications
• Separate Networks
• Independent Storage
• Historical Backups
• Immutability
• Separate Rights
• Managed Disaster Recovery
• Replication Monitoring · Failover Decision · Data Integrity Check

The key principle is the absence of a single point of failure that could simultaneously disable all recovery mechanisms.

For example, even geographically dispersed backups can be vulnerable if they can be deleted through a single compromised administrative account.

Therefore, not only geographical but also administrative independence is required.

7. Practical Example PostgreSQL: Replication State Control

Let’s consider a hypothetical ERP system that uses PostgreSQL 16.

The primary server transmits WAL records to the standby server.

The administrator can check the status of replicas through the system view pg_stat_replication.

SELECT
    application_name,
    client_addr,
    state,
    sync_state,
    sent_lsn,
    write_lsn,
    flush_lsn,
    replay_lsn,
    write_lag,
    flush_lag,
    replay_lag
FROM pg_stat_replication;

This query allows you to assess the status of connected replicas and the characteristics of log transmission.

However, latency metrics do not always provide a sufficient answer to the question of actual RPO. For example, in the absence of new transactions, timing characteristics may not align with the expected monitoring model.

Therefore, it is advisable to additionally monitor WAL positions, replica availability, and actual recovery capability.

To assess the difference between the current WAL position and the confirmed write position on the standby node, you can use:

SELECT
    application_name,
    pg_wal_lsn_diff(
        pg_current_wal_lsn(),
        flush_lsn
    ) AS bytes_behind
FROM pg_stat_replication
WHERE flush_lsn IS NOT NULL;

The result shows a delay in WAL bytes, but it is not a direct measure of time loss or the number of business transactions. It is also important to verify that the standby server is actually capable of taking over the primary role.

The mere arrival of WAL does not guarantee the correct failover of all application components.

8. Split-brain: the danger of two primary nodes operating simultaneously

One of the most dangerous scenarios in a distributed architecture is split-brain. It occurs when, due to a violation of network connectivity, several nodes independently consider themselves primary and accept changes.

Suppose data center A temporarily loses connection with data center B. The monitoring system mistakenly determines that the primary server is unavailable and activates the standby.

But the server in data center A continues to operate and accept transactions from local users. As a result, two independent change histories arise.

For transactional ERP, this can mean duplicate document numbers, stock balance violations, or financial transaction conflicts. It is not always possible to automatically merge such data.

Therefore, failover systems must use:

• quorum or another arbitration mechanism;
• fencing to prevent the old node from writing;
• controlled primary selection rules;
• replication status check;
• documented rollback procedure.

Particular care should be taken with automatic failover between geographically remote sites.

Loss of network connectivity does not always mean the primary server has stopped working.

9. Data Warehouse: a different recovery strategy

For enterprise DWH, recovery requirements can differ significantly from transactional ERP.

DWH usually contains large volumes of historical data, transformation results, and analytical views. Some information can be recreated from operational systems.

However, non-reproducible historical records require independent protection.

Therefore, it is advisable to categorize information by recoverability.

Protection Strategy

Applying full synchronous replication to a large DWH may be economically unfeasible. In certain cases, periodic backup and the ability to reload data are sufficient.

When doing so, recovery time must be taken into account. If the corporate storage contains tens of terabytes, recovering the entire array can take significantly longer than recovering a small ERP.

Therefore, it is necessary to determine which analytical functions should be recovered first.

10. Data Lake and Tiered Storage

For large arrays of historical information, it is advisable to consider object storage and a tiered storage model. However, a cheaper storage class does not necessarily mean faster or cheaper recovery.

Archival services may have delays in obtaining information and separate charges for its retrieval.

Therefore, the backup strategy must consider not only storage costs but also the expected access time. For example, historical data required for regulatory audits can be stored in cold storage.

However, critical data for daily business resumption must be available within the agreed RTO.

Data Classes Comparison

It is especially important not to delete historical information solely based on low access frequency.

In wartime conditions, archival data may be the only available source for restoring lost operational systems.

11. ETL Recovery After Extended Interruptions

After power loss or communication failure, some data streams may accumulate significant latency. For example, five regional warehouses continued local operations but could not transmit transactions to the central repository for a day.

Upon restoring communication, simultaneously launching all integrations can overload the ERP or DWH. Therefore, a controlled catch-up processing procedure is required.

It should consider:

1. Source priority.
2. Packet sequence.
3. Transaction uniqueness.
4. Checkpoint status.
5. Idempotency.
6. Receiver throughput.
7. Completeness check.

For critical streams, idempotent loading mechanisms should be implemented.

For example, each batch can have its own identifier, and the control table can store the last confirmed state.

SQL Example of Batch Logging

CREATE TABLE etl_batches (
    batch_id BIGSERIAL PRIMARY KEY,
    source_name VARCHAR(100) NOT NULL,
    source_from TIMESTAMPTZ NOT NULL,
    source_to TIMESTAMPTZ NOT NULL,
    status VARCHAR(20) NOT NULL,
    rows_loaded BIGINT,
    completed_at TIMESTAMPTZ
);

Finding the last confirmed point:

SELECT
    source_name,
    MAX(source_to) AS last_completed
FROM etl_batches
WHERE status = 'SUCCESS'
GROUP BY source_name;

However, this approach is only reliable if the SUCCESS status is set consistently with data commit. If the log is updated separately from the actual load, discrepancies may arise after a failure.

Therefore, for critical processes, transactional batch control needs to be designed.

12. Technological Capabilities as of April 2024

As of the publication date, technologies are available to enterprises that allow for the creation of geographically distributed information systems.

Apache Kafka 3.7.0 was released on February 27, 2024. This version continues the development of KRaft, notably introducing early support for JBOD for KRaft clusters, which should not yet be considered universally ready for critical production use.

Apache Airflow 2.9.0 was released on April 8. New features include the development of dataset-based task scheduling and interface improvements.

Apache NiFi 1.25.0 is available as of January 29. The next release branch, 1.26.0, has not yet been released as of the date of this article.

However, the availability of a new version is not a reason for immediate update of the production environment. During wartime, infrastructure changes must undergo risk assessment, testing, and preparation of a rollback procedure.

13. Economics of Replication and Backup Sites

Geographic resilience requires additional resources. A backup data center may contain servers, storage, network equipment, and licensed software.

Cloud replication also incurs costs for storage, compute, and data transfer.

Therefore, it is necessary to compare several strategies.

Conditional Financial Model

All amounts are illustrative. Actual costs depend on data volumes, products, licenses, and providers. The most expensive strategy is not always the best.

If the business allows for the recovery of a specific service within 48 hours, continuous synchronous replication may be economically unjustified.

Instead, for ERP, on which continuous production depends, investments in rapid recovery can have significant economic value.

14. Practical Scenario: A Company with Two Data Centers

Let’s consider a hypothetical Ukrainian distribution company that uses ERP, CRM, WMS, DWH, and MDM.

The primary data center is located in the central part of Ukraine.

The backup site operates in a different region. Independent backups are stored in a separate environment.

Following a new wave of attacks on the energy sector, the company decides to test the readiness of its information systems for the loss of the primary site.

Stage 1. Inventory

Critical applications, databases, integrations, and network dependencies are identified.

Stage 2. Replication Check

The actual lag and the ability of backup servers to restore the transactional state are measured.

Stage 3. Independence Verification

Accounts, backups, DNS, VPN, and other dependencies are analyzed.

Stage 4. Controlled Switchover

In a test scenario, the enterprise simulates the unavailability of the primary data center.

Before activating the backup, it is confirmed that the old primary node is not continuing to accept transactions.

Stage 5. ERP Recovery

Documents, inventory balances, and critical business operations are checked.

Stage 6. Integration Recovery

CRM–ERP, ERP–WMS, and ERP–DWH flows are launched after checking control points.

Stage 7. Data Reconciliation

Operations for the switchover period are reconciled.

Stage 8. Return to the Primary Site

After its availability is restored, a controlled failback procedure is performed. In this process, the standby site that accepted transactions is considered the current authoritative source until the agreed-upon return is completed.

This is a conditional scenario for architectural testing.

15. 60-Day Roadmap

Actual timelines depend on the scale of the enterprise and infrastructure availability.

Work with physical equipment should only be performed under safe conditions.

16. Geographic Information Resilience KPIs

Metrics need to be practically verified.

Even correctly configured replication does not guarantee that all dependent applications will automatically work on the backup site.

17. Risks and Common Mistakes

Using replication instead of backup. Logical corruption can be transferred to the backup server. Insufficient network latency control. Synchronous replication can significantly degrade performance.

Automatic failover without fencing. There is a risk of split-brain. Independent recovery points for interdependent systems. ERP and WMS may end up in different logical states.

Unverified backups. Successful backup execution does not guarantee recovery. Excessive complexity. The geographical architecture should match the operational team’s capabilities.

Underestimating data transfer costs. Continuous replication of large volumes can incur significant expenses. Lack of Data Quality control. A technically functional database may contain incomplete business data.

Ignoring user backup connection. Server availability does not guarantee business process availability. Unprepared failback. Returning to the primary site may be more complex than the initial switchover.

18. Intecracy Group Competencies

Ensuring corporate data resilience requires a combination of infrastructural, integration, and information competencies. Data Management IG (dmig.com.ua) specializes in ETL, MDM, Data Quality, Data Warehouse, Data Lake, Data Governance, and corporate analytics. In geographic redundancy projects, information integrity control, ETL process recovery, reference data harmonization, and analytics reliability assurance after switchover are particularly important.

IQusion IT (iqusion.com) represents system integration, electronic document management, BPM, and business process automation. Relevant competencies can be used for end-to-end process recovery, document state management, and application system integration between sites. Softline IT (softline.kiev.ua) works in the area of server and network infrastructure, storage systems, and infrastructure projects. These competencies form the basis for designing backup data centers, network connectivity, and reliable data storage.

SL Global Service (sgs4business.com) specializes in cloud integration, virtualization, and hybrid infrastructure. These technologies allow for the consideration of geographically distributed environments, backup placement of application systems, and flexible use of computing resources.

The combination of these areas allows for a comprehensive approach to corporate resilience—from physical infrastructure to the logical consistency of business operations.

Geographic resilience must start with data architecture

The new wave of Russian attacks on energy infrastructure in the spring of 2024 demonstrated that Ukrainian enterprises must maintain readiness for prolonged and repeated disruptions to information systems. Protecting a single data center cannot guarantee the availability of the entire corporate architecture.

Therefore, it is necessary to combine local redundancy, geographic independence, protected copies, and controlled disaster recovery procedures.

The correct choice of replication mode becomes particularly important. Synchronous replication allows for increased guarantees of saving confirmed transactions but creates a dependency on latency and network availability.

Asynchronous replication provides greater flexibility between remote sites, but allows for some lag. The choice should be determined by agreed-upon RPO, RTO, and business requirements for consistency.

Moreover, replication does not replace backup. The enterprise must maintain independent recovery points that can survive not only a physical disaster but also logical corruption or compromise.

For Data Warehouse, the distinction between non-reproducible historical data and derived analytical results remains important. This allows for the definition of an economically viable recovery strategy.

For ERP and other transactional systems, protection against split-brain and consistency control after failover are particularly important. Server recovery cannot be considered complete until the enterprise confirms the correctness of critical business operations.

The main conclusion of April 2024: geographical redundancy should ensure not just server survival, but the preservation of a unified, reliable, and recoverable enterprise information state.

This requirement precisely defines the further development of corporate Data Management in wartime conditions.