Introduction

Hello everybody, About 5 months ago I started building an alternative to the Searx metasearch engine called Websurfx which brings many improvements and features which lacks in Searx like speed, security, high levels of customization and lots more. Although as of now it lacks many features which will be added soon in futures release cycles but right now we have got everything stabilized and are nearing to our first release v1.0.0. So I would like to have some feedbacks on my project because they are really valuable part for this project.

In the next part I share the reason this project exists and what we have done so far, share the goal of the project and what we are planning to do in the future.

Why does it exist?

The primary purpose of the Websurfx project is to create a fast, secure, and privacy-focused metasearch engine. While there are numerous metasearch engines available, not all of them guarantee the security of their search engine, which is critical for maintaining privacy. Memory flaws, for example, can expose private or sensitive information, which is never a good thing. Also, there is the added problem of Spam, ads, and unorganic results which most engines don’t have the full-proof answer to it till now. Moreover, Rust is used to write Websurfx, which ensures memory safety and removes such issues. Many metasearch engines also lack important features like advanced picture search, which is required by many graphic designers, content providers, and others. Websurfx attempts to improve the user experience by providing these and other features, such as providing custom filtering ability and Micro-apps or Quick results (like providing a calculator, currency exchanges, etc. in the search results).

Preview

Home Page

Search Page

404 Page

What Do We Provide Right Now?

  • Ad-Free Results.
  • 12 colorschemes and a simple theme by default.
  • Ability to filter content using filter lists (coming soon).
  • Speed, Privacy, and Security.

In Future Releases

We are planning to move to leptos framework, which will help us provide more privacy by providing feature based compilation which allows the user to choose between different privacy levels. Which will look something like this:

  • Default: It will use wasm and js with csr and ssr.
  • Harderned: It will use ssr only with some js
  • Harderned-with-no-scripts: It will use ssr only with no js at all.

Goals

  • Organic and Relevant Results
  • Ad-Free and Spam-Free Results
  • Advanced Image Search (providing searches based on color, size, etc.)
  • Dorking Support (in other words advanced search query syntax like using And, not and or in search queries)
  • Privacy, Security, and Speed.
  • Support for low memory devices (like you will be able to host websurfx on low memory devices like phones, tablets, etc.).
  • Quick Results and Micro-Apps (providing quick apps like calculator, and exchange in the search results).
  • AI Integration for Answering Search Queries.
  • High Level of Customizability (providing more colorschemes and themes).

Benchmarks

Well, I will not compare my benchmark to other metasearch engines and Searx, but here is the benchmark for speed.

Number of workers/users: 16
Number of searches per worker/user: 1
Total time: 75.37s
Average time per search: 4.71s
Minimum time: 2.95s
Maximum time: 9.28s

Note: This benchmark was performed on a 1 Mbps internet connection speed.

Installation

To get started, clone the repository, edit the config file, which is located in the websurfx directory, and install the Redis server by following the instructions located here. Then run the websurfx server and Redis server using the following commands.

git clone https://github.com/neon-mmd/websurfx.git
cd websurfx
cargo build -r
redis-server --port 8082 &
./target/debug/websurfx

Once you have started the server, open your preferred web browser and navigate to http://127.0.0.1:8080/ to start using Websurfx.

Check out the docs for docker deployment and more installation instructions.

Call to Action: If you like the project then I would suggest leaving a star on the project as this helps us reach more people in the process.

“Show your love by starring the project”

Project Link:

https://github.com/neon-mmd/websurfx

  • INeedMana@piefed.zipEnglish
    2·
    3 days ago

    If you don’t mind i have a few noob questions

    1. From a very high POV, how does it work?
      I write a query and it passes it to a bunch of engines and then does some internal ranking of results?
    2. can I configure engine weights?
    3. Is some metadata stored (for example which engine response I chose) or is it rather stateless?
    4. technically speaking it would still be my IP querying those external engines, right?
      So any kind of “bubbling” they do based on IP is still in effect?
    5. The general idea is that we would have a few shared instances of those, or rather everyone should be hosting an instance for themselves?
    6. Do you foresee how the external engines might block Websurfx once they discover their ads don’t land?
    • neon-mmd@programming.devOP
      2·
      1 day ago

      Yes, sure, no problem, I will answer all you questions one by one:

      1. Yes, that’s how it works at high POV but also in a privacy oriented similar to searxng (so only your IP address is shared with the upstream engine but you can spoof that using VPN or by using the proxy feature in websurfx). Additionally we also rerank the search results after fetching search results from the upstream engines, to make it more organic (though, we admit not the best but contributions will help us alot in that area for sure. 😅 )

      2. No, currently no, but you can open a feature request issue about it at our project here.

      3. Yes, it is stored as a config, cookie and also you can export the cookie from the UI and then import it again (this can be useful if you change browser or clear cookies for some reason).

      4. Yes, I explained about it and how to get around with it in the first point. 😅

      5. I think it is better to self-host then depend on like an instance because you still have many privacy issues with VPSs (because it is still someone else’s computer), but we do provide privacy enhancing features like encypting cache results which can help in evading the VPS from spying your/users search results in the cache.

      6. Not really, as far as I know they don’t block on the basis of that their ads not being displayed but yes there is still a chance that you could flagged by some engines because they want users to use their search engines because they do want to sell the user as their product (which is the same problem, searxng does have too, so there is nothing we can do about it but yes we can improve the bot evasion system, so that even less engines do it and we would welcome contributions in that area). 😅

      Also, if you have more questions about anything feel free to ask, we would be very glad to answer them 😊 and we appreciate it 👍 too as it also helps answer and clarify other people’s doubts too.

      • INeedMana@piefed.zipEnglish
        1·
        14 hours ago

        Thank you :)

        5. What I meant by that question was rather “what’s the design vision?”
        Is the intended usage more like everyone should have their own, or rather those should be shared with more people (to lump up different topics originating from the same machine)

    • neon-mmd@programming.devOP
      0·
      2 years ago

      Thanks for pointing this out, I just improved this by upgrading the algorithm to use tokio::spawn so I think I will update this benchmarks soon.