I love the smaller projects that engineering departments create, although I have no engineering department, I'm sure that my project will be as successful... 🔥🚒 Today I have been thinking about my insane desktop environment in JavaScript from the perspective of a UI designer and the problems it faces.
It looks like somehow I'm going to need to declaratively define layouts in html via the use of html custom elements.
<jsde-layout>
<jsde-button></jsde-button>
</jsde-layout>
I think that the DOM is just not an option for the level of webviews I am going to embed, atleast one per application nested in a main browser context, how can I hand this work off to the GPU?. The answer is probably a mix of Lit-Element and PIXI.js, where the UI is wrote in html and the well wrote applications utilize webgl.
The idea of GUI on canvas sounds absolutely terrible I assure you I know, very hard to handle and a hell of a lot to write from scratch, so instead let's coble something together. Anyway hear me out, there is no DOM in your desktop but there is accessibility, my goal first and foremost is to somehow have a canvas that is interactable and accessible. The plan then is probably retain invisible accessible elements that retain that accessibility, handing off that sweet rendering speed to the GPU to get the good native feelings, warm and fuzzy.
The canvas GUI is actually inspired by Ben Lovey who you should really check out on here, his post on QT in Wasm kind of took me around the internet on a safari.
https://dev.to/deciduously/qt-for-webassembly-5dlm
I found the following weird and wonderful projects that make you think... Oh dear god why?
In order from cringe too oh, really that's interesting 🤔.
QT renders in this way as well through wasm. Okay personality I'm sold. How can I make this nice to develop in?
I want every web developer to be able to write some CSS and HTML or close to that and feel like they did when writing a website, but I want them to have actually drawn boxes in a webgl canvas within a floating window frame (client decoration, close, min max, titlr) wrapped around it, or, as I said also support the regular experience of writing html CSS and js and getting a document. (I already have the latter working, maybe this is simply enough to drum up interest?) you should be able to use React, Vue, Angular etc in your application, anything really, the UI should extend your ability to talk to the OS and not hinder you.
So back to basics, for a canvas GUI I'm going to need a layout engine, you know like how you float and flexbox work. If CSS cannot directly apply to what is effectively a box of pixels, perhaps I could abstract the concept. I need a pointer to a thing, a class, and a styling language that everyone knows. Given that GTK uses a grid like layout which is also common to most UI frameworks, if I scrap float entirely and just support that sort of idea. Enter Facebook, now they as you know make good software, forget about the product, don't care, the software is quite interesting. Look what I found:
Boom, flexbox in canvas (I have read that its rather trivial to get working in Pixi.js.
So why PIXI? Well I'm not needing three.js and although I have tried to write glsl in the past and do all that vert shader frag shader stuff, I'm afraid I am boarding on my limits of what I can actually learn and take in at one time. PIXI looks like a great abstraction for 2D based graphics and actually it's pretty common practice to embed the GUI into a game, I'm just talking about scrapping the game.
Why lit? Okay I think Lit represents a potential future for React and Vue and probably the death of angular as it goes, Lit-html is functional and lovely and Lit Element is declarative and kind of a better angular 2+. Also the framework has basic state management, lifecycles, templating and no vdom diffing. It's bloody fast and probably the most like JavaScript I have ever felt. It's the framework of choice when I want other developers to be able to embed custom elements (I have checked this) into react or Vue so that they can have UI that looks "native".
Guys what else am I missing, am I still a madman, let me know in the comments.
Top comments (0)