It’s been pretty quiet on the forums until the last 24 hours, and the activity if you were watching probably looked pretty weird compared to the previous types of posts. Some major things have happened, and the project once again has taken a sharp curve from where I was at in my last post. The biggest event was that @dAxpeDDa joined the project right around the time of my last post.
Upon entering the new year, my mindset was to strip down my ideas so that I could write a game. As a lone developer, it really was daunting the vision I had. If you look at how I was approaching Kludgine, I was trying to design a great application framework and game engine, because ultimately I really wanted to build an amazing cross-platform toolkit. Yes, I want to make a game, but building those tools is what really excites me.
Yet, I realized when I started the year that if I wanted a playable game by the end of the year, I needed to scale back.
Then came along dAxpeDDa. It was truly a random event. We talked about it recently because I was telling him how I loved to tell the story of him showing up right after I open-sourced the Cosmic Verge repository. From my perspective, I pushed this commit, changed the access privileges and went to bed without posting about it anywhere.
Hours later, while I was fast asleep, dAxpeDDa joined the Discord server and posted that he might be interested in contributing. After I told him my side, he told me he didn’t discover the game project first. He had stumbled across Magrathea originally. It’s funny because I thought me publishing Cosmic Verge as open-source was the catalyst, but it was purely a coincidence in reality. As many smart people have said, correlation is not causation.
The Journey to Cosmic Verge
Regardless of how he came to join the Discord, I couldn’t be happier for it. He talked me out of my nonsense of trying to simplify everything, and instead, we’ve embarked on an exciting journey:
We want to create a well-architected “single-shard” massively multiplayer game, and we aim to do it with as must Rust code as possible.
The devil is in the details. To dAxpeDDa and I, “well-architected” includes top-notch security, privacy, reliability, and scalability. Our discussions started with how to build a highly available cluster to power Cosmic Verge. We began identifying key pieces of the architecture: QUIC for networking and OPAQUE for password security. But, we had several problems that all came down to data:
-
In the implementation at the time, Redis was used for PubSub style communication between servers. Redis is an amazing product, and I highly recommend it. However, getting it deployed in an HA setup isn’t fun inside of Kubernetes. I tried and failed several times to get a reliable setup working that could survive my attempts to break it in situations that it should have survived.
The redis dependency also increased developer friction: one more dependency developers needed to have running locally.
-
For “solar system” data, we were using Redis to store and synchronize that data. The fast nature of Redis meant that it would almost never slow down, no matter how much system data we threw at it (limited by RAM). Ideally, if we want to scale horizontally, this system data shouldn’t be stored in a single shared redis instance.
Instead, it would be nice if each set of system servers could replicate amongst themselves. We could assign solar systems to 3 servers in the cluster, and the load for the system data would be localized to those 3 servers.
-
For reliable transactional storage, we were using PostgreSQL. PostgreSQL is an absolutely incredible database, and I again highly recommend it.
But, my preferred access mechanism was sqlx. It provides compile-time, type-safe checking of your SQL queries. It’s a revolutionary way to integrate with a database in a truly type-safe manner.
That may sound like a benefit, but it’s also a curse in disguise. To run unit tests that rely on that database functionality, you must be able to compile that code, which means you need to have a PostgreSQL instance installed. Or, alternatively, you could limit yourself to SQL that works on both SQLite and PostgreSQL, but that isn’t a solution I could live with.
Ultimately, I began to see the appeal of trying to write crates in Rust that needed databases to be powered. The idea of asking someone to complicate their CI process because they brought in a crate as a dependency is rough.
Those were the debates we were having, until one night, I had a crazy realization: I built my last company off of CouchDB. I was very familiar with how CouchDB worked and had even pondered writing my own version atop PostgreSQL’s jsonb columns as a fun project sometime. In the process of trying to come up with a solution that would allow us to synchronize data between servers in a cluster, we had identified Sled as a great option to use as the storage layer.
I realized that Sled’s primary interface would allow me to implement a CouchDB-like database pretty easily. It’s that thought that really has changed our entire approach to writing Cosmic Verge.
PliantDb: The core of Cosmic Verge
PliantDb started as a vision to replace PostgreSQL and Redis with something that we could make as easy to deploy as possible. Sled was going to allow us to provide ACID-compliant storage, and by implementing the API layer to mimic how CouchDB’s internal mechanisms worked, I could provide a really powerful feature set that allowed developers to write database “views” in native Rust code.
Focusing on the developer flow first: We wanted to allow contributors to clone the Cosmic Verge repository, and as long as they had their toolchain set up properly, cargo run should be able to bring a server up without extra dependencies. For this, PliantDb’s primary layer is implemented as a local, file-based database similar to SQLite. This simplifies running code in CI environments as well.
Unlike SQLite, PliantDb also offers PubSub even when running in local mode. And, it also has a high-performance key-value store that provides many of the benefits of Redis.
We also have the networking layer done. All of this functionality works with a client-server connection, either over a custom QUIC-based protocol or over WebSockets (eventually WebRTC too).
Additionally, PliantDb has an API platform built-in, which enables writing permissions-aware APIs over the PliantDb networking protocol. This is how the Cosmic Verge game protocol will work – over a connection to the PliantDb server (and eventually cluster).
All of this is done with 89% line coverage in just two months. This can only be attributed to the power of the Rust ecosystem. I don’t want to discount the amount of work we’ve done, but it truly is building a powerful solution by combining so many crates from the Rust ecosystem: Sled, flume, quinn, serde, and many many others.
The next steps
As we continued, we kept writing our code in a modular fashion. We’ve published many crates: fabruic, actionable, PliantDb, and we were working on others. We both think that Github’s multi-project management is abysmal. As we were nearing the stage of being ready to integrate PliantDb into Cosmic Verge, the question arose: Maybe we should create a smaller example project to flush out PliantDb before embarking on the big game.
I agreed, and we both were interested in maybe porting or rewriting Khonsubase onto PliantDb.
But, when dAxpeDDa and I were chatting a couple of weeks ago, he brought up the idea of building a UI framework that could be used to build apps that worked with PliantDb easily. You would have the option of a native executable, or you could use WASM to build your app and host it over the HTTP layer we were planning to add to PliantDb.
Of course, if you remember what I said at the start of this post, this was exactly something I wanted to do. We’re in the proof of concept phase still, but we’re excited about what we’ve done so far.
Circling back to the problem of project management: In thinking of our workflows and the challenges we face when running an open-source project and a small team, we started thinking about building a hybrid forums/issue-tracker/chat-like app.
That thought inspired us to try using Discourse for our project management and communication. It probably will change quite a bit as we use it in the coming weeks, but we’re also taking notes to figure out if we think it’s worth building this still or just continuing to use other tools and get back to the game.
In the meantime, we’re just designing the UI framework the way we want a UI framework to work – regardless of if it’s Cosmic Verge or building a website, the core of how layout works is something both of us are very opinionated on.
When will there be more news on Cosmic Verge?
I still have a goal to get Cosmic Verge ported onto PliantDb as soon as we can, but we’re purposely not taking shortcuts. Since PliantDb is going to manage users, we need to implement that before updating Cosmic Verge. These projects are too big to allow ourselves shortcuts that introduce technical debt.
In the meantime, keep following along here to watch our progress. I’m looking forward to showing off Gooey once we’re at a more meaningful stage of progress.