Base128 Algorithm: Tool for Encoding and Decoding text data

  __________                          ____ ________    ______  
  \______   \_____     ______  ____  /_   |\_____  \  /  __  \ 
   |    |  _/\__  \   /  ___/_/ __ \  |   | /  ____/  >      < 
   |    |   \ / __ \_ \___ \ \  ___/  |   |/       \ /   --   \
   |______  /(____  //____  > \___  > |___|\_______ \\______  /
          \/      \/      \/      \/               \/       \/

Encoding and decoding algorithms are critical in today's digital world.
One of the interesting and reliable solutions is the Base128 algorithm.
This article will introduce you to the principles of operation of this algorithm and present the practical application of the source code.

Base128 is a program written in Go that allows you to encode and decode files using Base128 encoding. The algorithm processes the input data by converting it to a sequence of integers and then creating text from it.

The program can be run with various flags that determine whether the data is to be encoded or decoded, whether help is displayed, or whether information about the program version is available.

If the user selects the -e flag, the program encodes the input using Base128 encoding. Text data is read from the input file or standard input, and whitespace and newlines are stripped from it. The input is then converted to a sequence of integers using the encodeBase128 function.
Finally, the sequence of numbers is formatted as a single string and displayed on standard output.

If the user selects the -d flag, the program decodes the input, which is read from the input file or from standard input. As with the encoding, whitespace and newlines are removed. The input is then split into single numbers, which are converted to a sequence of integers using the decodeBase128 function. Finally, the sequence of numbers is converted to text and displayed on standard output.

The program also supports the -h, -version, and -copyright flags, which display the appropriate information. These flags require no input and do no encoding or decoding.

In conclusion, the Base128 program allows you to encode and decode files using Base128 encoding. It can be used to transform text data into a sequence of integers and vice versa. The program is simple to use, with a command-line interface and various flags that can be used depending on your needs.

Below is a step-by-step description of this algorithm and the source code written in Go that implements the Base128 algorithm. This code will allow you to encode and decode data in this format.

1. Importing needed packages:

• "flag" - package for handling command line flags.
• "fmt" - package for output formatting.
• "io/ioutil" - a package that allows operations on files.
• "os" - a package enabling operations on the operating system.
• "strconv" - a type conversion package.
• "strings" - variables.

package main

import (
    "flag"
    "fmt"
    "io/ioutil"
    "os"
    "strconv"
    "strings"
)

2. Definition of a constant named "base" with value 128.

This constant will be used in base128 encoding and decoding.

const base = 128

3. The "encodeBase128" function takes an input string as an argument and returns slice ints.

This function initializes the "result" variable as an empty slice int. Then, in a loop, it iterates through each character in the "input" string.
On each iteration, it converts that character to an int and adds it to the "result" slice. Returns slice "result".

    func encodeBase128(input string) []int {
        result := []int{}
        for _, char := range input {
            result = append(result, int(char))
        }
        return result
    }

4. The "decodeBase128" function takes slice ints as arguments and returns a string.

This function initializes the "result" variable as an empty string.
Then, it loops through each element of slice "input".
On each iteration, it converts this item to a string and adds it to the "result" variable. Returns the string "result".

    func decodeBase128(input []int) string {
        result := ""
        for _, code := range input {
            result += string(rune(code))
        }
        return result
    }

5. The "main" function is the main function of the program.

The function initializes the "encodeFlag", "decodeFlag", "helpFlag", "versionFlag" and "copyrightFlag" flags.
Later, it calls the "flag.Parse()" function to parse the command line flags.

func main() {
    encodeFlag := flag.Bool("e", false, "Converts the input's base128 encoding into an output text file.")
    decodeFlag := flag.Bool("d", false, "Recovers the original input file by decoding the information that was previously encoded using base128.")
    helpFlag := flag.Bool("h", false, "Print instructions for calling and a list of available alternatives.")
    versionFlag := flag.Bool("version", false, "Print the program's version.")
    copyrightFlag := flag.Bool("copyright", false, "Print copyright information.")
    flag.Parse()

6. If the "helpFlag" flag is set, the program displays the instruction and available options and exits.

The text of the instruction is displayed on the standard output using the "fmt.Println()" function.
When prompted, the program uses "os.Exit(0)" to exit.

    if *helpFlag {
        fmt.Println("base128 - Encodes or decodes FILE, or standard input, to standard output or a file as base128.")
        fmt.Println("base128 [OPTION]... [FILE input] > [FILE output]")
        fmt.Println()
        fmt.Println("EXAMPLE encode: base128 -e encode.txt > decode.txt")
        fmt.Println("EXAMPLE decode: base128 -d decode.txt > encode2.txt")
        fmt.Println()
        fmt.Println("Options:")
        fmt.Println()
        fmt.Println("  -d, --decode          Recovers the original input file by decoding the information that was previously encoded using base128.")
        fmt.Println("  -e, --encode          Converts the input's base128 encoding into an output text file.")
        fmt.Println("  -h, --help            Print instructions for calling and a list of available alternatives.")
        fmt.Println("      --version         Print the program's version.")
        fmt.Println("      --copyright       Print copyright information.")
        fmt.Println()
        fmt.Println("(c) by Lukasz Wojcik 2023")
        fmt.Println("http://www.base128.com")
        os.Exit(0)
    }

7. If the "versionFlag" flag is set, the program displays the program version and exits.

The program version text is displayed on the standard output using the "fmt.Println()" function.
After displaying the version, the program uses "os.Exit(0)" to exit.

    if *versionFlag {
        fmt.Println("base128 version 1.0/2023")
        os.Exit(0)
    }

8. If the "copyrightFlag" flag is set, the program displays the copyright information and exits.

Copyright text is displayed on standard output using the "fmt.Println()" function.
After displaying the copyright notice, the program uses "os.Exit(0)" to exit.

    if *copyrightFlag {
        fmt.Println("base128 (C) 2023 by Lukasz Wojcik")
        os.Exit(0)
    }

9. If both the "encodeFlag" and "decodeFlag" flags are set, the program displays an error on standard error and exits.

The error text is displayed on standard error using the function "fmt.Fprintln(os.Stderr, …)".
When an error is displayed, the program uses "os.Exit(1)" to exit with an error code.

    if *encodeFlag && *decodeFlag {
        fmt.Fprintln(os.Stderr, "base128: cannot specify. Please -e for ENCODE or -d for DECODE or -h for HELP")
        os.Exit(1)
    }

10. If neither the "encodeFlag" nor the "decodeFlag" flag are set, the program displays an error on standard error and exits.

The error text is displayed on standard error using the function "fmt.Fprintln(os.Stderr, …)".
When an error is displayed, the program uses "os.Exit(1)" to exit with an error code.

    if !*encodeFlag && !*decodeFlag {
        fmt.Fprintln(os.Stderr, "base128: cannot specify. Please -e for ENCODE or -d for DECODE or -h for HELP")
        os.Exit(1)
    }

11. The "flag.Args()" function returns the positional arguments given on the command line.

Then, the program checks if the number of arguments is 0 or if the first argument is equal to "-".
If one of these conditions is met, the input data will be read from the standard input.
In this case, the program uses the function "ioutil.ReadAll(os.Stdin)" to read all the bytes from standard input.
Then, the received bytes are terminated with the "strings.TrimSpace()" function and assigned to the "input" variable.

    args := flag.Args()
    var input string
    if len(args) == 0 || args[0] == "-" {
        bytes, err := ioutil.ReadAll(os.Stdin)
        if err != nil {
            panic(err)
        }
        input = strings.TrimSpace(string(bytes))
    } else {
        bytes, err := ioutil.ReadFile(args[0])
        if err != nil {
            panic(err)
        }
        input = strings.TrimSpace(string(bytes))
    }

12. If the "encodeFlag" flag is set, the program removes new lines from the input, getting the assigned "input" value.

Then, the program performs base128 encoding based on the "input" by calling the "encodeBase128(input)" function and assigning the slice reference ints to the "numbers" variable.
Subsequently, the program concatenates these numbers into one string by calling the "fmt.Sprint(numbers)" function and removes the square brackets by calling the "strings.Trim()" function.
Finally, the program writes the result to STDOUT using the "fmt.Println(output)" function.

    if *encodeFlag {
        input = strings.Replace(input, "\n", "", -1)
        input = strings.Replace(input, "\r", "", -1)
        input = strings.TrimSpace(input)
        numbers := encodeBase128(input)
        output := strings.Trim(strings.Join(strings.Fields(fmt.Sprint(numbers)), " "), "[]")
        fmt.Println(output)

13. If the "decodeFlag" flag is set, the program removes newlines from the input, getting the assigned "input" value.

Then, the program splits this string into single numbers using the "strings.Split(input, "")" function and assigns the result as a slice of strings to the variable "numbersStr".
Next, the program iterates through these numbers and, for each of them, converts it to an int using the function "strconv.Atoi(numberStr)" and assigns the result to the variable "number".
Finally, the program performs base128 decoding based on "numbers" by calling the "decodeBase128(numbers)" function and assigning the received string to the "output" variable. Finally, the program writes the result to STDOUT using the "fmt.Println(output)" function.

    } else if *decodeFlag {
        input = strings.Replace(input, "\n", "", -1)
        input = strings.Replace(input, "\r", "", -1)
        input = strings.TrimSpace(input)
        numbersStr := strings.Split(input, " ")
        numbers := make([]int, len(numbersStr))
        for i, numberStr := range numbersStr {
            numberStr = strings.TrimSpace(numberStr)
            number, err := strconv.Atoi(numberStr)
            if err != nil {
                panic(err)
            }
            numbers[i] = number
        }
        output := decodeBase128(numbers)
        fmt.Println(output)
    }

At the end I present the entire base128 algorithm written in GO.

The Base128 encoding and decoding algorithm is applicable to both general use and computing. It offers effective tools for converting text data into other forms of representation, which can greatly facilitate the processing and storage of this data.

For general use, the Base128 algorithm can be used in a variety of situations. Application examples are:

Using the Base128 algorithm:

1. Secure data storage:
The Base128 algorithm can be used to encode sensitive information such as passwords or credit card numbers. Converting data into numeric form makes it difficult for unauthorized users to read it.

2. Communication over the network:
Base128 encoded data can be transmitted over the network, reducing the size of the transmitted information and speeding up the data transmission.

3. Data Compression:
The Base128 algorithm can also be used in the data compression process, reducing file size without losing quality.
In computer science, the Base128 algorithm has found wide applications in various fields.

Examples are:

1. File encoding and decoding:
When files need to be transferred or stored in text form, the Base128 algorithm can be used to encode and decode them.
This is especially useful when passing binary data over protocols that only support text data.

2. Data security:
The Base128 algorithm is also used in cryptographic algorithms to encrypt data. Converting the input data into numerical form increases their entropy and makes them more difficult to read by a potential attacker.

Summary:

The Base128 algorithm is an effective tool for encoding and decoding data, offering a wide range of applications for both general users and in the field of computer science.
The Go source code presented in this article enables practical application and experimentation with this algorithm.
It can be used to secure data, compress information, or transmit data over a network. Thanks to its simplicity of operation and flexibility, the Base128 algorithm is a tool worth using in various fields of computer science and more.

The entire base128 algorithm:

The base128 site: HERE
The GitHub repositories: THERE

Donation

If you find Base128 useful and would like to support its development or maintenance, you can do so by making a voluntary donation.

PayPal one-time donation

Thank you for reading.
If you have any questions, or tips, you can leave them here in the comments. I will answer as soon as possible.