A brief story of how Momiji all started and a peek of where it is going.

Back in 2013, I was an avid developer for Apple platforms and created a bookmark manager for macOS, and iOS (both on iPhone and iPad) called Momiji. The pendulum kept swinging back and forth for me between relying on bookmarks and relying on web search to find the sites and pages I wanted to revisit, but I had finally decided to put my full weight behind bookmarks.

Despite the skeumorphic phase its author was going through, Momiji had its modicum of success. But, as we all know, it's a mere handful of apps in the App Store which walk away with the lion's share of users. Around the same time, I had become extremely busy for this and that reason, and pursuing further development was becoming progressively harder, and unfortunately the app was shuttered.

Fast forward 13 years and a lot has changed.

It feels more and more that tech doesn't work for us users any longer, or perhaps it would be more accurate to say that for every unit of work it does for us, it does two or three units for its creators. We now live a world where our otherwise still-functioning thermostats can and are being bricked by the manufacturer at the flip of a switch, our Smart TVs are allegedly taking 2 snapshots per second of the programs we are watching, and dishwashing machines are refusing to offer their full functionality unless connected to Wi-Fi. And this is all before reckless AI even came onto the scene.

For far too long I have found myself complaining about it all, but doing little to nothing about it. Inspired by others, I felt it was high time to roll up my own sleeves. To be clear, this is too large a problem for any one person or even a team of people to solve. Nevertheless, like all problems, it's about breaking things down into smaller chunks and addressing them individually.

Two years ago, I decided the small chunk I was most passionate about was user privacy, and specifically private/anonymous web search. Thus Momiji would have its second act.

After a full year of ideation and laying out all the objectives, development began. Now, nearly one year into development, the first general release version (v0.71.0) of Momiji is ready.

Momiji is a free, open-source, privacy-focused web search engine (written in Kotlin and supporting Linux, macOS and Windows), in addition to being an already great bookmark manager. This is not its final form, but Momiji is at a stage where it can finally begin to address the problems it sets out to solve.

In future posts I will get into the technical details of Momiji (and all the subsystems that make it possible), but for now I wanted to talk about the high level stuff.

Solving Private Web Search

To solve private web search, I began with asking myself the question: What does the epitome of private web search actually look like? It is not worth investing time and energy on a '10% more private' solution. I wanted to fast forward to the very end (past the closing credits) on this one.

Today so much of our software is in the form of SaaS and, consequently, privacy usually hinges on a relationship based on trust with this or that service provider, not to mention the security measures of the service itself. SaaS, I believe, also incentivizes a value-extracting business model that often times comes to undermine this very trust.

Well, what if it didn't need to be this way? What if, in the case of web search, we didn't even need to trust the search engine? What if we knew that 100% of our searches were private because that was the only possibility, both now and into the future?

The only real way that I know to make this all a reality is to place the search engine onto the user's computer and open source the code to prove that search is occurring 100% on-device and with no internet communication.

This is the epitome of private web search and this is exactly what Momiji does.

So it obviously begs the question: How do we fit a whole search engine onto each user's computer? It sounds crazy right? Well, this is another one of these big problems that, with an open mind, can be broken down into smaller chunks. Here are the two chunks that I could identify...

1. Compression - To what degree can a distributed search index be compressed?
2. Utility - How replete must a distributed search index really be, in order to be useful?

The compression part was a challenge and also the most pressing concern. Asking users to set aside considerable room on their hard drives is something to avoid if possible. At the same time, the terms 'highly-compressed' and 'fast search' don't usually go hand in hand, but with a technique inspired by Paint by Number, Momiji is currently able to fit just under 4,400 search results per MiB, and the compression ratio actually improves, over time, the more data is added.

Also, it's impressively fast! Most common searches with one or two terms, while Momiji is otherwise idle, are serviced in under 2 seconds (tested on a 2021 edition MacBook Pro with M1 Pro CPU + 16GB of RAM).

As for the composition of the search index, the answer is not as prosaic. In the end, it boils down to the not-so-simple task of making the search index as useful to as many people as possible, while simultaneously NOT striving to capture anything and everything on the web. This is all highly subjective and it should be clear that Momiji will not appease everybody on day one, nor day two for that matter (even today's crop of traditional web search engines don't appease everybody all the time).

However, just as in the scenario where a minority of the apps are getting most of the users, a small share of web sites get the bulk of visitors. We all know which web sites these are and I won't name them here. These sites are the first to cover and then Momiji will branch from here. Since all searches in Momiji take place on users' computers, in private (it's the whole raison d'être for Momiji after all) it is not possible to know which are failing to come back with the appeasing results. To this end, a peripheral project called MomijiIndexHealth, attached to Path10's local copy of the Momiji index, will work to ensure there is a healthy enough coverage, both in terms of general quality and quantity, for an ever-growing number of common search terms.

Back to Basics

While much of the tech industry is busy haphazardly shoehorning AI into everything (even toothbrushes!), Momiji is going back... to the 90s, when web search was basic and just worked for users.

It turns out that going back to basics is also an effective way to promote privacy. There will be ZERO tracking of users, no AI, no personalized results of any kind. There'll also be no ads (I'm not against ads per se, and I do believe they can be done responsibly and with taste, but I think the web has enough ads as it is).

But fret not, basic doesn't mean featureless. From objectionable content filtering, and a powerful search syntax, to custom search result sorting, Momiji is able to give control to users that may in fact be unparalleled. This is all possible because Momiji is a locally-installed app whose only focus needs to be on each individual user.

And this is just the beginning! There is much work ahead and my sleeves will be staying rolled up for quite some time.