I made Drag and drop builder for Python UI 🤯. Here's how I built it

If you remember, couple of weeks ago, I announced that I was building a Drag and drop UI builder for Python.

Well... It's finally here.

You can check it out at PyUIBuilder

Source code: https://github.com/PaulleDemon/PyUIBuilder

What can the builder do?
In brief, it can help you quickly build UI for Python and generate UI code in multiple libraries/frameworks, including Tkinter and customtkinter. you can read more on the features section

But I don't just want to launch a project, I'd also like to share my experience with you. In this blog, I'll be going over my thought process and a high-level overview of how I built the app.

Coming up with the idea.

Contrary to popular belief, Python is often used to build quick applications, its especially popular among developers working in data science, automation, scripting tasks etc. Many internal tools and GUIs, particularly in scientific and research settings, are built with Python due to its simplicity and the availability of frameworks like Tkinter, PyQt, and others.

Now, there were a lot of Drag and drop builders for the web, but very few for Python GUIs, especially for tkinter. I saw a few, but the problem was they were very limited number of widgets or would generate code in XML format, which isn't ideal if you are developing UI in Python.

So, at first, I just wanted to build a proper drag and drop UI builder just for Tkinter.

I kept tinkering around the idea of an ideal GUI builder (no pun intended). I was inspired by Canva's UI, and came up with few features that would make my GUI an ideal one.

1. All widgets should be made like plugins.
2. Should support 3rd party UI widgets in the form of plugins.
3. Should be able to upload assets such as images, video's etc.
4. It should generate code in Python.

So, around end of July, I decided to start working on the project

Expanding on the idea

In the beginning it was called tkbuilder, indicating its a GUI builder for Tkinter UI library.

But, if you noticed I can also expand on the same idea to support multiple Python GUI frameworks and libraries, since everything is made like a plugin and that's exactly what I planned to do.

Initial version planning.

For the initial version, I didn't want to add too many features that would overwhelm users. I wanted to build it based on feedback from people using it. This way I am not wasting time on building things that people don't want.

From the very beginning I decided not to have a backend or any or a signup form. This way its much simpler for me to develop and for the users using it. I just wanted a simple frontend that people can get started with.

Choosing language JS, TS or Python

Yes, this was something that I was pondering on for quite sometime, most GUI builders for Python out there were built using Python. My first choice for Python was PySide.

The most complex GUI based app I built using PyQt/Pyside was a node-based editor few years back.

But I quickly realized the limitations of using python to build the initial version.

• Python UI libraries don't have many 3rd party widgets to help me quickly build the initial version.
• It's not easy to distribute Python app as exe files, where as using JS we can distribute it in the form of an electron app.
• Most people prefer to use web instead of downloading an executable from an unfamiliar website.

Typescript was also an option, but with Typescript I always felt it to be too verbose

These were the only things that I immediately noticed, so my first choice became using JS.

PS: I later went on to regret not starting with TS, but that will be a story for another time.

Framework or no framework.

The framework-like library I'm most comfortable with is React.js, but creating an abstraction would require using classes, which is not recommended since the introduction of hooks.

The problem of not using a framework was I'd have to build everything myself and not have access to the vast component libraries that react has to offer.

Both had trade-offs, but React classes can still be used, so it became obvious choice to me.

Bumpy start

I started by building the very base and sidebar in the beginning of August, and had to stop due to lack of funds, so I to took up a client's work, who unfortunately didn't pay up the final amount. I tried crowd funding but wasn't lucky there either.

So, in the month September with the little funds I had left, I decided to go all in on this project. On around 9th september I restarted the work.

Planning ahead ...

A lot of the time went into thinking about the base abstraction, that can be extended to scale to meet the needs.

1. Wanted to have a Canvas, that can be zoomed and panned similar to Figma.
2. A base widget from which all the other widgets can extend from.
3. A Drag and drop feature to drag and drop UI elements into canvas.

To build with React, you need to think and build it in a certain way, despite argument over whether it's a library or a framework, it always feels more like Framework than a library.

UI design

I always liked how Canva built their sidebar, I wanted to have the something similar for my drag and drop builder.

I drew up what was on my mind on a piece of paper. Not the best artist out there 🙄

My thought process regarding the canvas and widget interaction.

So, who should be in-charge of dragging, resizing, selecting. The canvas or the base widget. How will be the widgets inside the widget be handled?

Will the base widget know their children or is it going to be managed with a single data structure by the canvas itself. How will I render children inside children?

How will the drag and drop work inside the canvas and other widgets?

How are layouts going to be managed?

These were some of the questions I started asking before building the entire thing.

Though now the UI looks simpler, a lot of thought was put into building the base, so it looks much simpler for users.

HTML Canvas based approach or non-canvas approach.

Canvas based approach

Now html has a default Canvas element, that allows you to do a lot of things like draw, add image and stuff, now it looked like an ideal element to use for my program.

So, I started checkout if there was and existing implementation of a drag and drop, resizing, zoom and pan. I found FabricJs, this seemed like a fantastic library for my use case.

I tried experimenting with Fabric.Js and tried to implement the entire thing in fabric.js as you can see this implementation, but there was something about canvas that I didn't foresee.

1. I started off experimenting with hooks-based approach when building the canvas, but fabric.js dispose function was async, so it wouldn't play well with Hooks.
2. Canvas cannot have child elements such as Div or other elements which would make it a bit more harder to build layout managers
3. Debugging anything on canvas is quite hard as the inner elements of canvas don't show up in the developer tools inspect element

Non-canvas-based approach
Now after experimenting, the non-canvas approach seemed better, since I have access to default layout manager provided, plus there were many UI pre-built components available that would make this ideal choice when scaling.

I planned to simulate canvas by using two different div's one inner div and outer container div.

Now creating zoom and pan were fairly easy to implement, since CSS already had transform, scale and translate.

First, to implement this, I had to have a container which holds a canvas. Now this canvas is invisible element (without overflow hidden), this is where all the elements are dropped, and scaling and translation is applied.

For zoom in I had to increment the scale and for zoom out decrement it.

Try this simple example. (+ key to zoom and - to zoom out)

Panning worked similarly

Drag and drop

When starting out I had researched on a couple of libraries such as React-beautiful-Dnd, React Dnd-kit and React Swappy.

After researching I saw that react-beautiful-dnd was no longer maintained and started with React dnd-kit. As a started building, I found the dnd-kit's documentation quite limited for what I was building, Plus, a new release with major changes to library was coming out soon, so I decided to drop react-dnd-kit until the major release.

I rewrote the parts of where I used DND-kit with HTML's Drag and Drop API. Only limitation with the native Drag and drop API was that it's still not supported by some touch devices, which didn't matter to me because I was building for non-touch devices.

Single source of truth

when building an app like this, it can become easy to lose track of all the variables and changes. So, I can't have multiple variables keeping track of the same piece of information.

The information/state of every widget should either be held by the canvas or the widget's themselves, which then passes the information upon request.

Or maybe use state management library like redux

I chose to have all the information about the widgets managed by the Canvas component after experimenting different approaches.

The data structure looks something like this

[
  {
    id: "", // id of the widget
    widgetType: WidgetClass,  // base widget
    children: [], // children will also have the same datastructure as the parent
    parent: "", // id of the parent of the current widget
    initialData: {} // information about the widget's data that's about to be rendered eg: backgroundColor, foregroundColor etc.
  }
]

React Context managers

Now I wanted the assets uploaded in the sidebar accessible by toolbar of the widgets. But everytime I switch the sidetabs, the re-render caused the uploaded assets to disappear.

One of the biggest limitations with Redux is that you can only store serializable data. Non-serializable data such as image, video, other assets cannot be stored on redux. This would make it harder to pass common data around different component.

One way to overcome this is to use React Context. In brief, React Context provides a way to pass data through the component tree without having to pass props down manually at every level.

All I would have to do to have the data in different components was to wrap it around a React context provider.

I Created my own context providers for two things:

1. Drag and drop - Enabling dragging and dropping from sidebar + dragging and dropping withing child elements.
2. File upload - To make the uploaded files accessible on the toolbar for each widget.

Here is a simple example of how I used React context for Drag and drop.

import React, { createContext, useContext, useState } from 'react'

const DragWidgetContext = createContext()

export const useDragWidgetContext = () => useContext(DragWidgetContext)

// Provider component to wrap around parts that need drag-and-drop functionality
export const DragWidgetProvider = ({ children }) => {
    const [draggedElement, setDraggedElement] = useState(null)

    const onDragStart = (element) => {
        setDraggedElement(element)
    }

    const onDragEnd = () => {
        setDraggedElement(null)
    }

    return (
        <DragWidgetContext.Provider value={{ draggedElement, onDragStart, onDragEnd }}>
            {children}
        </DragWidgetContext.Provider>
    )
}

Yes! that's it. All I had to do now was to wrap it around the component where I needed the context, which in my case was over Canvas and sidebar.

Generating code

Since each widget behaves differently and has their own attributes, I decided that widgets must be responsible for generating their own code and a code engine will only handle variable name conflicts and putting the code together.

This way, I was easily able to expand to support many pre-built widgets as well as some 3rd party UI plugins.

Going live

I didn't have a backend or a signup form and there were a lot of companies providing free hosting for static pages. I had first decided to go with Vercel, but often times I have seen Vercel free tire go down if there were too many requests.

Thats when I found out about Cloudflares pages offering. Their free tire had almost everything unlimited. So, using cloudflare became my primary choice.

The only cons were the build times were quite slow and had lacked quite a bit of documentation.

The most annoying part of the build step was the build failure, It worked on Vercel, but not on cloudflare pages??? The logs were also not that clear. and in free tire we have only 500 builds per month, so I didn't want to waste too many on unsuccessful builds.

I tried for hours then I decided to set continuous integration to empty string

CI='' npm install

And it finally went live.

Want to see how it has progressed throughout the months?

I have been building this entire thing in public. I you are interested in seeing it progressed from a simple sidebar to a fully blown Drag n drop builder you can check the entire timeline here

#buildinpublic

Oh! don't forget to follow along for updates

Star repo ⭐️

If you liked this type of content, I'll be writing more blogs going into more depths of how I plan and build stuffs, to follow along you can subscribe to my substack newsletter :)

Substack newsletter