Recent Posts

May 17th 2019

Pixmap Release: Launching Rewind

Today I'm excited to announce a new feature for Pixmap. While I had planned on mostly focusing on the native iOS port in Swift a couple friends brought up an idea that sounded too fun to implement, so here it is.

Blog diagram

Think version control for pixel art, except it's just linear. It's useful and fun to use. If you buy Pixmap Ultimate you can use rewind to download snapshots or rewind an image.

When you rewind an image it modifies the history like this:

Blog diagram

How it works is pretty simple. Luckily we chose to store every change everyone makes which made this possible. When you start a rewind the application simply paginates the event history from the backend and when you rewind an image we use the same architecture for drawing. All of the changes get added to the same queue, processed by the same servers, etc. The only real difference is that a lock is created on the image and then released once the rewind is done so that no editing can happen during the rewind.

The main challenge client side was performance.

If you drag the bar all the way to the beginning of time and start dragging it forward that is very "light" since each "tick" of the bar is simply applying changes to the scene from the last tick. However, when you drag backwards you have to re-render the whole scene from the beginning of time to where the user dragged the bar. Doing this on the UI thread resulted in lots of lag so I had to implement an "observer" thread that waits for things to render, does the processing, and pushes the result to the UI thread. I also ended up changing how concurrency worked in the renderer to reduce CPU usage, making both rewind and drawing smoother.

There were three challenges server side.

The first was that since I'm using websockets the original plan was for the client to send an event and then the server would paginate through the data and send the pages to the client. This didn't work with websockets because no matter the frame size or delay between messages the connection could get clogged resulting in high memory server side, delay in WS handshakes, and ultimately the connection closing. The solution was to implement traditional pagination over websockets where the client says "give me the next 10k after this id". At that point you don't even need websockets but I already have the infrastructure for it for other features.

The second problem was the pagination itself. We use MongoDB and it runs on a $5 VPS for Pixmap. That instance can handle tens of thousands of people drawing at once, but one rewind could destroy it with my first implementation. You see, Mongo doesn't optimize pagination queries like "find X, sort by Y, start 1000, limit 100". It will literally find all of X, sort it all, and then select the page from the result even though we are sorting in the same direction as the index etc (this is another thing about Mongo (or B-Trees I guess) I will write up later, where you can optimize for sort direction). This would kill the instance.

The solution was to still use pagination but without "start after N" - instead we "start after this _id" since Mongo's ids are naturally sortable (assuming you let the DB create the id and you don't create it application side). This drops cpu usage from around 80% during a rewind of "Mid Peninsula" to a negligible amount.

Anyway, I had fun building it and hope people enjoy it. Cheers.

May 5th 2019

How does this blog work?

For those curious, I decided against having this blog be dynamically generated at runtime (think Wordpress, Pico, etc). It really didn't need to be interactive, doesn't need any client side javascript, and really doesn't even need a database. However, writing HTML manually for each post is a pain to maintain and not worth my time.

So to get the best of both worlds - maintainability and simplicity - I do dynamically generate the blog but only at build time.

What this means that I write the posts in Markdown and check them into github in the same repo. The repository contains a NodeJS app and when a github event fires to the build server the build server runs the NodeJS app just one time which generates the site. Then it just sends the build result off to the appropriate server running nginx.

Blog diagram

May 5th 2019

What is winrickLabs?

winrickLabs is a place for executing ambitious ideas. Things that may be deemed "too hard" or "not profitable" or "too fun" end up here. This doesn't mean elegant solutions are rejected. In fact they are encouraged. But the end product should be something to be proud of and the journey should have a story worth telling.