Written by Eletroswing

Introduction

Certainly, AI-based technologies are here to stay and represent our most imminent technological boom. Much of this is due to OpenAI's launch of its LLM (large language model), ChatGPT. This advancement has motivated the creation of thousands of new services in an industry that, just months before its release, was considered unfeasible. However, this progress comes with a price, one that could be costly in the long run.

Dependence on Large Models

With the emergence of OpenAI and its API, many companies have been built exclusively around these technologies. Many researchers in the field still adhere to the premise of keeping everything in a single large model, despite the daily launch of thousands of new technologies, models being trained, and datasets being created.

Challenges of Training Large Models

When faced with a task, the common approach is to train a single specific model, such as "using the OpenAI API". But why spend so many resources on something that might not be as satisfactory as expected? Consider an example: an eye movement substitution model.

Example of an Eye Movement Substitution Model

If you try to develop this by training a model from scratch, you quickly realize the complexity. Your model needs to identify, track, generate, and substitute the eye in videos. Look at how much work goes into a single objective.

Additionally, suppose your model takes 4 minutes per video. How would you scale this? Your only solution would be hardware, which is expensive and unfeasible.

Micro-Models

This brings us to the concept of micro-models.

Concept of Micro-Models

What if, instead of generalizing everything into a single model, we broke it into stages and utilized existing specialist models? This way, we create a modular and scalable pipeline by adding models and code between them. We could have a queue for identifying, another for tracking, another for generating, and another for substituting.

By preparing between the stages with code, and since the models are already specialists in their respective subjects, we can easily reduce inference time. We can also run parts in parallel if necessary. If tracking is fast but generation is not, we can simply add another consumer to the queue to handle the load.

Transformation with Micro-Models
Basically, we transform this:

Into this:

Benefits of Micro-Models

With this approach, despite seeming more complex, we achieve greater scalability and efficiency in our system. Additionally, we improve integration efficiency and speed, as we can modify only specific parts of the system instead of having to regenerate a model or perform fine-tuning, right?

Conclusion

Adopting micro-models allows for the creation of more scalable and efficient systems, making the most of existing resources and facilitating the continuous maintenance and evolution of AI-based solutions.