When the cloud isn't enough (triple modular redundancy)

You're using a single source for your applications, and you're wrong.

Dear administrator, I'm humbled to make this appeal to you, after yet another failure by our team to ensure the availability of our application. I'm going back to my engineering classes, taking the opportunity to optimize, cut costs and harmonize.

This article is aimed at all applications that require a high level of availability or whose data is sensitive.

Triple modular redundancy, engineering curriculum

https://en.wikipedia.org/wiki/Triple_modular_redundancy

During your engineering courses, you must have studied processes for reinforcing your product: emergency stops, mistake-proofing, monitoring tools. Today, remember triple modular redundancy: a critical service must be available from at least 3 sources.

For your application, this means you need to think about:

• provide a service from 3 sources,
• duplicate the data in 3 sources,
• allow a balance from one to the other.

Standards, ISO-27001

https://en.wikipedia.org/wiki/ISO/IEC_27001

When you're working with large entities or internationally, you need to ensure your customers' your ability to follow standards such as ISO-27001, which will enable you to work with industries in particular. These standards were introduced in 2005 and are regularly revised.

ISO-27001 is based on 3 pillars:

• Confidentiality of information,
• Information integrity,
• Information availability.

To obtain this certification, having a single data source and a single application source won't be enough: you'll have to apply triple modular redundancy. You can't depend solely on the server running on your premises, nor can you depend solely on a cloud.

The benefits of Clouds and internal servers

The advantages of a cloud:

✅ faster start-up,
✅ no hardware management,
✅ outsourced hardware technical skills (fewer employees),
✅ standards already applied (e.g. redundancy),
✅ flexibility and on-demand dimensions,
✅ service close to the customer.

The advantages of an in-house server:

✅ complete, self-supervised management,
✅ fixed costs,
✅ information security assured,
✅ easy migrations,
✅ self-managed breakdowns,
✅ controlled liability,
✅ evolutions as needed,
✅ servers developed to suit the customer, with no superfluous extras.

In this tangle of benefits, it's important to understand:

1. that a cloud will help you get started and take good practices early in your application development ⭐,
2. that it's in your interest to favor an in-house application and data service (as soon as possible) ⭐⭐,
3. that you will need to switch to cloud in case of in-house problems and program a balance ⭐⭐⭐,
4. that you'll need to enable service redundancy on the cloud ⭐⭐⭐⭐.

By choosing a triple redundancy of your service (by having at least 3 internal servers and clouds), you'll even be able to determine the qualitative data in case of conflict between two services: the two servers that give the same answer are right, the server that answers differently must correct its corrupted data ⭐⭐⭐⭐⭐.

Successfully balancing

Encourage in-house use of the application: its fixed cost means you can budget correctly, while the cloud can take over in the event of overload or breakdown. Cloud costs are harder to predict. When only your in-house server is running, you save a lot of money on the cloud.

Once you've decided how to disperse your services (how many internal servers and cloud servers) and once you've defined the type of service you need (FaaS, SaaS, PaaS, IaaS, On-Premise), then you can determine the cloud closest to your needs, start development with it, try to get the same level of service internally, and finally look for another cloud with the same level of service to get triple redundancy.

Consider working with at least one cloud that can provide services as close to the customer as possible, particularly to avoid additional download costs.

Available tools

Containers and management

For simple development, I'd recommend pairs like Kubernetes and Docker, which can easily export a similar configuration between the local work environment, the internal server and all the clouds that enable this kind of IaaS.

Developing with Kubernetes ensures that all servers are the same, and reduces the risk of getting different results (such as corruption).

Software-as-a-Service

For a simple, inexpensive web application, you'll need a domain name, HTTP server, file server and database, take your pick:

• A registrar that allows balancing on its CNAME (OVH, Gandi, ...).
• AWS: lambda, S3, RDS
• Azure: Functions, Blob Storage, Database
• Google: Cloud Run, Cloud CDN, Cloud SQL
• A multitude of equivalents for in-house services.

Developing compatible SaaS is more difficult, but it does help to reduce runtime costs.

Cleaning, synchronization and security

This redundancy calls for exceptional rigor and costly management. You'll also need to ensure that the perimeters are secure from one another, and that synchronization does not propagate attacks. You'll need to ensure 4 mandatory services:

• determine the server closest to the client, to favor nearby downloads,
• always copy all data to other cloud and internal servers,
• check integrity regularly (e.g. daily at midnight UTC with hash keys) and deal with data corruption,
• produce synchronization reports.

Block-chain

In the spirit of redundancy and data integrity, block-chain proposes a systemic approach to determining a secure, decentralized source. In reality, we're not far from a block-chain.

Depending on your data format, you can use blockchain. My guess is that most applications will need services that handle data in very different formats (user, file, checkout, auth), and your CDN or PostgreSQL servers deserve synchronization services at their scale.

Conclusion

Make sure you're not dependent on a single data source. An in-house server and a cloud can work in a complementary way. You'll get a secure, robust application that's ready to work with global companies.