Hello! In this series we're going to build a Gemini client. What is Gemini you ask? Before we look at Gemini it's helpful to understand it's beginnings.
It was designed as a souped up version of Gopher which in turn is a way to share files across the internet. These files are usually people's thoughts and ideas, not movies and pictures.
You can also start at chapter 3 as the first 2 chapters are really just set up, it should be fine to start coding first and then come back to read chapter 1 and 2.
History
Gopher came out of the University of Minnesota at roughly the same time as when the WWW (World Wide Web) came out of CERN. Both Gopher and WWW (today known as the web and in general the internet) are protocols to send and request files from servers. Both go over TCP, Gopher goes over port 70, WWW goes over port 80. Gopher is a very simple protocol with no extensibility whereas WWW (also known as HTTP) had extensibility built into which in turn is probably one factor in why WWW is so popular now. So popular that the internet and the World Wide Web are often seen as the same thing! In reality though the internet is so much more, the internet could be thought of as the road in cities and the protocols such as WWW and Gopher are just cars that use those roads. Many things can use those roads, we have e-mail, FTP, SSH, telnet and more that all go over the internet in addition to WWW.
Now the World Wide Web has taken over as the main source way people interact with the internet but there are other spaces that we can visit and take part in. Gopher is one of them, and in my opinion quite an interesting one. Today, no one can build a browser by themselves and take part on the WWW. But! Gopher is so simple a protocol that you can build your own client and interact with gopher space completely under your own power. There is something quite seductive about that.
I wrote a series before outlining how I built a Gopher client but I was muddling my way through the code so couldn't write down the steps. In this series I hope to be able to go a bit slower with less muddling and hopefully be able to show you how to build a client for one of these spaces that is so interesting to me.
Gemini is a simple protocol, designed so that the entire spec will fit in your head and a client and server implementation can be made in a weekend. This is very much in the vein of Gopher, and the Gopher spec made it clear that simplicity was king. This is true for Gemini as well.
You can find more information on the Gemini website.
https://gemini.circumlunar.space/
With that being said, I think you should go directly to the source and read Solderpunk's (the creator of the protocol) phlog (gopher blog) posts on Gemini on zaibatsu.circumlunar.space.
> telnet zaibatsu.circumlunar.space 70
Trying 167.88.113.156...
Connected to circumlunar.space.
Escape character is '^]'.
/~solderpunk/gemini
Using Gopher!
Just a quick intro because it is quite fun!
We telnet directly to the server on port 70.
We enter the selector string /~solderpunk/gemini to navigate to Solderpunk's gopherhole. Had we not known the selector string we can hit enter and we will get a list of gopher items, and the selector strings for them.
0(2019-04-07) More on gopher and crypto /~solderpunk/phlog/more-on-gopher-and-crypto.txt zaibatsu.circumlunar.space 70
0(2019-03-31) Why gopher needs crypto /~solderpunk/phlog/why-gopher-needs-crypto.txt zaibatsu.circumlunar.space 70
0(2019-03-03) Pondering what's inbetween gopher and the web /~solderpunk/phlog/pondering-whats-inbetween-gopher-and-the-web.txt zaibatsu.circumlunar.space 70
After we entered the above selector we should see listing like above. It is quite messy but that's because we are using straight telnet. A gopher client would clean this up drastically.
Now Gopher has no concept of keeping connections open, once the request and response cycle is finished, the connection is closed immediately. This means we'll need to do another telnet again with the new selector string.
So to read the first phlog post Solderpunk made, we would do the following commands.
> telnet zaibatsu.circumlunar.space 70
Trying 167.88.113.156...
Connected to circumlunar.space.
Escape character is '^]'.
/~solderpunk/phlog/pondering-whats-inbetween-gopher-and-the-web.txt
We telnet into the server, and then we pass in the selector we found in the above listing, then we would get the text file the selector references and can read the document! A very strange way to browse the internet eh (This is why I built a client....) :)
Gemini
Now this was a long way to get to the actual point. Gopher is a very simple protocol and WWW is a very complex protocol. Gemini seeks to exist in the middle which is a very interesting idea!
Now before we begin I need to mention something that I find quite funny. You would think Gemini is pronounced Gemini but it's actually pronounced Gemini. Joking! We, at least I, pronounce Gemini, Gemini ending with the 'eye' sound. NASA pronounces it Gemini, Gemini ending a knee sound. This is quite an interesting thing to me for some reason and I can help but smiling thing about it.
On with the show!
I've made some notes on the Gemini spec and I'm going outline them here. Feel free to leave comments about anything I've missed. We'll be using these notes to write our client, after all it is a protocol and a protocol is just a bunch of rules on how data is transferred and processed.
Let's get started!
The server will close the connection immediately after sending a response. There is no connection reuse.
Requests made by the client are simply the URL, either absolute or relative, followed by a CRLF (Carriage Return Line Feed, \r\n)
URLs are prefixed with gemini:// and this is also the default for if the URL doesn't have a prefix already. Other URL prefixes such as http:// and https:// can also be sent and the server may act as a translator.
Responses made by the server are a single header line, with a CRLF. This can be followed by a response body if the header line indicates success. This is the only time a response body is sent.
Statuses
- The header line will be followed by a single space, followed by a string which can be a error message or the document type of the response body, which is followed by CRLF.
20 text/plain; charset=utf-8
<RESPONSE BODY formatted as plain text using UTF-8>
40 Failed to Retrieve Page
Here we have 2 examples, the first which is successful and will also have a response body and the other is an error with a single header line.
Status codes are 2 digits, but a single digit is enough to know the error, the second digit just adds a little bit more information for logging purposes.
If the status is not 20, success, the server must not send a response body.
There are 6 status types, because we are building a basic client, we'll only worry about the standard statuses.
- Status of 10 - INPUT, the META text is a prompt that should be shown to the user and when they enter their response, the selector should get a query string attached to it.
If we requested gemini://test.gemini/hello-world and received the below response
10 What is your name?
And the user responded with Alice.
We would then make a request for gemini://test.gemini/hello-world?name=Alice
Status of 20 - SUCCESS, A response body will follow the header line, the META string will be the mime type of the document.
Status of 30 - REDIRECT, The meta string is the new selector string the client should use to follow the redirect.
Status of 40 - TEMPORARY FAILURE, Something has gone wrong
Status of 50 - PERMANENT FAILURE, 40 and 50 can be combined and we can show just a generic error message and the meta string. The meta string will be error message the server wants to show the client.
Status of 60 - CLIENT CERTIFICATE REQUIRED - This error we'll leave out for now as it functions as a way for a client to generate a session in Gemini. I'm not entire sure how this is set up yet but I'll come back and fill this in afterwards. This isn't required for basic clients so we can safely ignore this.
- Response bodies can be anything, the meta string in the header is what determines what happens. Basic clients can get away with only dealing with text/* and everything else can get saved to the disk. For now that is what we'll implement but you can make it so that if the mime type is an image, to open the data in an image viewer.
- Text response bodies can use both CRLF(\r\n) and LF(\n) as line breaks and the client should handle both of them.
Certificates
TLS is mandatory! This one is maybe my least favorite option in the entire spec. TLS is the one place we will need to rely on a dependency. With Gopher, everything was under our control as programmers, we could build the client with 0 dependencies, this however isn't true for Gemini. But I get the reason for it! (I think...)
TLS must be 1.2 or higher with 1.3 being recommended! Not sure what this means quite yet but hopefully whatever crate we end up using has sane defaults.
Clients can handle certificates in any way, including ignoring them! The recommended way is to use TOFU certificate pinning, meaning when we first connect to a server, we will assume that that certificate is the real certificate and will save that and its expiry date on our machine. From that point on, we will validate our connection to the server using that certificate. TOFU stands for Trust on First Use. If for some reason the certificates stop matching and the expiry date on the saved certificate hasn't passed yet, then a warning should be shown to the user.
Client certificates can also generated by the client and used to communicate with a server. It works the same way as TOFU except the flow is the other way. This way the client has a way of identifying itself to the server and this allows for sessions to be set up. We're going to leave this out of our basic client but I think it would be quite interesting to implement.
Document
Just as HTTP/WWW has HTML as it's default format, Gemini uses text/gemini as it's default.
text/gemini is a very barebones format and the meta string can optionally contain the encoding charset and it can optionally contain the language parameter.
text/gemini; charset=utf-8; lang=en
Links must go on their own lines and the first 3 characters of a line will dictate their type
The first line type is link, any line starting with => is a link and should be rendered as such.
=>URL This is the Link Text
=> URL This is the Link Text
=> URL This is the Link Text
The URL must be after the link signifier and can be surrounded by any whitespace. This works because the space isn't allow in URLs and so we can use whitespace to get the link and then everything after it will be the text that we want to show the user.
- There are also preformat toggles which will cause lines in between the toggles to be displayed without any styling. Luckily because we'll be writing a terminal program, almost everything will look the same anyway!
\```
This is the preformat toggle (the beginning slash is just to escape)
One thing to note is anything after the leading preformat toggle on the same line as the toggle should not be displayed to the user.
We also have #, ##, ### to mark headers but because we are doing a very basic client, we'll just show these lines as usual, had we done something a little more complex, we could add coloring or font size changes.
Lines starting with * are lists.
Lines starting with > are quotes.
Lines with nothing on them are blank lines and should be rendered as blanks, multiple blanks shouldn't be collapsed into a single a blank line.
Lines starting with anything else are regular lines!
and with that, we are done! We have the gist of the specification outlined now. The key point is that because we are doing a basic client, we can safely ignore a great deal of things, but we also have the option of adding complexity. This I think is quite valuable, everything is very much opt in rather than being forced to do things a certain way.
Alright! In the next chapter we can start coding something up.
Top comments (0)