DEV Community

Cover image for ๐ŸŽ‰One must-have tool for anyone in Data field ๐Ÿคฏโœจ
Shrijal Acharya
Shrijal Acharya

Posted on

๐ŸŽ‰One must-have tool for anyone in Data field ๐Ÿคฏโœจ

TL;DR

SQL databases are great for handling structured data and complex queries, but they fall short when performing complex calculations and optimizing performance. ๐Ÿข

I am not sure if you are ready to read this. ๐Ÿค I think most of you might be shocked by reading this.

โœจ There is a faster and probably better alternative to SQL ๐Ÿ˜ฎ

I recently came across this, and let me tell you one thing: 'It is really good!' Let me introduce you to esProc. ๐Ÿƒโ€โ™‚๏ธ๐Ÿ’จ

esProc SPL is an open-source next-gen data processing language that can surpass the limitations of SQL and boost your data processing. ๐Ÿคฉ

Buckle Up GIF


What even is SPL? ๐Ÿค”

๐Ÿ’ก SPL short for Structured Process Language is software for computing structured/semi-structured data.

I am sure many of you may not know about SPL. There is just a letter's difference between SQL and SPL, yet they vary in many ways. ๐Ÿ˜ตโ€๐Ÿ’ซ

SPL uses a grid-based ๐Ÿ“Š syntax instead of text, making it more visual and concise. As a data computing engine, SPL can achieve higher performance ๐Ÿ’จ compared to traditional SQL.

SPL is specially dedicated to low code ๐Ÿง‘โ€๐Ÿ’ป and high performance. At its core, it is software comprising three major pieces:

  • Programming Language ๐Ÿš€: SPL is a language that facilitates the process of structured data computing, and has excellent performance in handling most computing scenarios.
  • Data Computing Middleware ๐Ÿค: SPL provides independent computation ability independent of databases. It can perform various structured data computations on its own and is easily embeddable in applications. ๐Ÿ“ฑ
  • Big Data Computing Platform ๐Ÿ“ˆ: SPL has its cluster computing system for large data, providing a distributed environment for programmers to control task distribution and implement efficient algorithms. ๐Ÿง 

๐Ÿ‘‰ To learn more about SPL, follow this link.


How SPL can be an alternative to SQL? ๐Ÿง

Before talking about whether SPL can be an alternative to SQL, let's first take a look into some limitations of SQL.

  • SQL queries grow in complexity fairly quickly ๐Ÿงต

SQL is considered a pretty easy language to write. It often reads like plain English but it is not always the case. ๐Ÿ˜“

When writing nested queries, it grows in complexity and even a senior developer ๐Ÿง‘โ€๐Ÿ’ป finds it hard to make sense of it.

Here's a simple SQL query to calculate the maximum number of consecutive days that a stock price keeps rising. See how quickly the SQL statement is harder to read. ๐Ÿ˜ตโ€๐Ÿ’ซ

SELECT MAX(consecutive_day)
FROM (
    SELECT COUNT(*) as consecutive_day
    FROM (
        SELECT SUM(rise_mark) OVER (ORDER BY trade_date) AS days_no_gain
        FROM (
            SELECT trade_date,
                   CASE WHEN closing_price > LAG(closing_price) OVER (ORDER BY trade_date)
                        THEN 0 ELSE 1 END AS rise_mark
            FROM stock_price
        )
    )
    GROUP BY days_no_gain
);
Enter fullscreen mode Exit fullscreen mode
  • SQL can be super slow sometimes ๐Ÿ’ฉ

If you have ever worked in a real-world situation, you likely had to perform queries on a database containing thousands or even millions of entries.

With such a vast amount of data, SQL often performs poorly in such situations.

Let's take an example of this below SQL statement ๐Ÿ‘‡

SELECT TOP 10 COLUMN_NAME FROM TABLE_NAME ORDER BY COLUMN_NAME ASC;
Enter fullscreen mode Exit fullscreen mode

This code in itself is not complex but SQL has to perform the following operations to return the final data.

  • First, sort the entire data.
  • From the entire sorted dataset, select the top 10.

Rewriting the previous SQL query in SPL will provide a clear idea of ease when using SPL. To calculate the maximum consecutive days that a stock keeps rising ๐Ÿ‘‡

stock_price.sort(trade_date).group@i(closing_price<closing_price[-1]).max(~.len())
Enter fullscreen mode Exit fullscreen mode

Although the logic is the same, it is much easier to read/write and is a lot more intuitive. ๐Ÿ˜„


Finally, what is esProc SPL? ๐Ÿคฉ

๐Ÿ’ก esProc is a scripting language for data processing with rich library functions and powerful syntax.

esProc SPL

So now you have a basic understanding of SPL, let's discuss what esProc actually is.

Now that you're aware that SPL is an alternative to SQL, think of esProc as an even superior alternative to traditional SPL with added functionalities. ๐Ÿ˜ฒ

esProc is a pure Java written that makes it easy to add powerful data processing capabilities to your Java ๐Ÿต applications but non-Java applications can invoke esProc through RESTful APIs ๐ŸŽฏ.

It provides structured data types and functions ๐Ÿ‘ท that let you perform operations like filtering, grouping, joining ๐Ÿชข, and many other computations - similar to what you can do with SQL - but using simple Java code. โœจ esProc implements the basic class libraries entirely within itself.

๐Ÿค” What platforms can esProc run?

As it is built purely in Java, it can run smoothly in any OS equipped with JVM (Java Virtual Machine), cloud servers, or even containers. ๐Ÿ˜Ž

โœจ In short, esProc is blazing fast, independent, and provides advanced scripting capabilities for data handling and analysis over traditional SPL.


Let's Wrap Up! โœจ

By now, you have a generic understanding of what SPL is. What esProc is and how can it come in handy in situations, and whether or not you should replace SQL with SPL or esProc.

After reading this article, if you feel that esProc is a great project with huge potential, don't forget to give it a star!

๐ŸŒŸ esProc on GitHub

Please let me what you think of esProc in the comments section below. ๐Ÿ‘‡

So, that is it for this article. Thank you so much for reading! ๐ŸŽ‰๐Ÿซก

Top comments (6)

Collapse
 
rajeshj3 profile image
Rajesh Joshi

Here's an OpenSource project, helpful in Scheduling Jobs.

Get Job Execution Reminders โฐ via Webhook using WebhookPlan

View on GitHub

Collapse
 
shricodev profile image
Shrijal Acharya

Thanks for sharing, Rajesh!

Collapse
 
shradhhu_53 profile image
Shraddha Khattar

What? can we implement it in our existing database then?

Collapse
 
shricodev profile image
Shrijal Acharya

As far as I know, it definitely works ๐Ÿ˜€. For more information visit scudata.com/

Some comments may only be visible to logged-in visitors. Sign in to view all comments.