Introduction to Lecter or quick and easy investigation of web applications

You're looking, but you're not seeing.
Hannibal Lecter

Today we gonna talk about Rails based applications.

The Problem

We all know that modern web applications are complex. Their structure can be so intricate that it takes a new developer to understand what it is dealing with. Think about it: programmers spend dozens of hours learning other people's code, writing down callable classes, and simply comprehending the business logic of the project.

An experienced reader will say: document your code, and life will become easier. Maybe in the best of both worlds, everyone really does, but in the actual world, the code looks different for some reason.

Ajith disliked this part of his work the most. In general, digging through someone else's code is not fun right away. Even when the code is well structured and described. And if strange naming is used, or if there are strange strange strange clues that only the author understands, you really want to drop everything.

But giving up was impossible: Ajith had just been given a legacy project which he have to:
a) investigate
b) give tasks to two juniors
c) outline a plan of finalization for the company's needs.

So, as usual, we launch the IDE, set breakpoints, and try to understand the creator's somewhat insane logic. Ajith put a call notepad next to him, poured a large cup of coffee, and proceeded to parse...

Here's a simple example: let's say we're learning a simple application like a blog. To have something to learn, let's create it by ourselves :) To speed up development, we will use scaffolding. Let's scaffold a Post model, create a database, and run the migrations.

Then we will start to study. To understand how the application works, we must know where the request is sent. In our blog, it will be /posts.

Then we need to do the following steps:

1. Find the correct endpoint in the routes.rb file.
2. Define the controller.
3. Find the required action in the controller.
4. Analyzing the code, go from the called classes deep into the application, keeping in mind at the same time the meaning and modification of input parameters and headers.
5. Keeping the entire chain of calls in mind.

It wasn't hard, was it? Now imagine we're facing a legacy application with a flawed architecture. Plus, the project uses the illogical naming of classes and modules. Also, it doesn't follow basic development patterns. All of this leads to the fact that you can't visualize the whole chain of called classes and can't get a complete picture of the work.

Ajith was on a roll! For the third day, he had been tinkering with the application, trying to understand how it worked. He had the feeling that the project was just teasing him, leading him to a dead-end over and over again.

It is believed that the programmer works by the Pareto rule when he or she interacts with a new project: 80% of the time he or she studies other people's code and 20% writes his or her own. Our hero could not get to the desired 20%.

During my career, I have dealt with onboarding specialists and decomposing projects into tasks to be reworked hundreds of times. Such work is very time-consuming, and any experienced developer will attest to that. If you use an IDE it will help you to save time because you will immediately see the available variables and their values. I also usually recommend using breakpoints, in order to avoid confusion with too many if expressions.

Sometimes ago, being under time pressure and trying to guide new employees in the right direction, I started thinking about automating the task of application analysis. The tool I wanted to develop had to solve four important tasks:

1. Reduce the onboarding period for a new developer.
2. Increase the expertise of unknown application components.
3. Allow you to quickly read a legacy project.
4. Be used to analyze open-source libs.

The Solution

After several months of development, the Lecter project appeared, allowing you to delve into the innards of web applications (distributed under the MIT License). The name is a little funny, but I wanted to get something more intellectual than a simple Jack the Ripper :) In my opinion, it turned out just right.

To illustrate how Lecter works, let's return to the blog application example. Before we start the analysis, we need to perform some preparatory steps:

1. Set to use more than one worker for the puma: workers: 3
2. Let's add a gem to the Gemfile: gem 'lecter'
3. Run the command: bundle install
4. Add new Routes to routes.rb mount Lecter::Engine => '/lecter' if Rails.env.development?

Now we are ready to explore our application. To do that let's go to http://localhost:3000/lecter and fill out the form with the following parameters:

HTTP method: POST
Endpoint: localhost:3000/posts
Headers: content_type=text/html,accept=*/*
Body(raw): "post"=>{"title"=>"New title", "description"=>"Desciption"}

Then we press a submit button and see the result:

As you can see from the example, we will immediately see:

1. All called classes in the right order.
2. The order in which the lines are executed in each file is shown.
3. All the codes to be called will be on the same page.
4. We get class listings.
5. ...and there is even a highlighting.

That's just great! Hint: if we have a lot of files, we will see the whole chain of code being executed.

In Sanskrit, Ajit translates as "one who is not defeated. Using Lecter Ajit was able to quickly read the legacy code and find the transition that was driving him crazy.

Ajit is sure that the next time he needs to investigate someone else's the legacy project, he will use this tool again.

Shortcomings of the solution and development plans

As you can see, I love my Lector. It has already saved me dozens or hundreds of hours of work (which could be spent on reworking it, haha). But alas, my Lector is not yet perfect. For example, its code analysis is available only through API endpoints. I understand that I can evolve towards Kafka, RabbitMQ, and take jobs from Redis, gut web sockets, and webhooks, but this direction still needs to be thought over. That's why I'd be grateful if you could tell me your opinion on the project development in the comments.

I understand that Lecter is not yet flexible enough. In the future, I plan to add options to hide some structural files and/or entire folders.

In addition, I am looking for maintainers for the project to make it perfect. There are some issues which you could choose. If you like it, please give it a star on the github page.

Write me and I would be happy to discuss his work with you.