• 0 Posts
  • 21 Comments
Joined 2 years ago
cake
Cake day: May 22nd, 2024

help-circle

  • You can have more than 2x 22tb, the SX line is full of storage servers!

    SX65 2x 1TB nvme 4x 22tb (~104€ 0%VAT) SX135 2x 2tb nvme 8x 22tb (~204€ 0%VAT) SX295 2x 8tb nvme 14x 22tb (~384 0%VAT)

    If you manually add the disks to configurations like AX-line, there is a limit, that may be bypassed if you contact hetzners support, but that’d be expensive compared to the SX line I reckon.

    Also, Hetzner’s server auction has quite affordable 4x 16tb servers, starting from 63€ 0%VAT, that’d be pretty affordable RAID1- solution, ~1€ per TB/month.

    Unlimited traffic on all of the above, and a sidenote: SX line servers have one-off setup fees, server auction has no setup fees.







  • Hi!

    I’ve been selfhosting Yacy for some years, even tho I rarely use it (I’m mostly using Kagi these days).

    But some tips:

    • Set up something like this to your browser, this sends Yacy to crawl pages that you visit https://github.com/JeremyRand/YaCyIndexerGreasemonkey .
    • Get familiar with blacklists and try to find some public ones to filter out bad sites and adult content.
    • Tinker with Ranking & heuristics -> Solr boosts to get results that fit your use case more.
    • And in general, tinker with all the settings you can find!

    And not directly Yacy related, but you can use your own Yacy through Searxng as well, even in ‘private’(non P2P) mode.






  • I’ve been toying with Perplexica over the last few weeks occasionally, it feels really restrictive.

    I’ve had to modify the internal prompts to make it generate better search terms with my SearxNG (And depending on what LLM model you use, you need to fine tune this…) and having to rebuild the container image to do this has just been annoying. Overall, I’ve had experience with self-hosted LLM web searches on Open-webui, but perplexica is a fun project to try out nevertheless.


  • This is a bit off-topic, but did you try to increase the JVM limits inside Yacy’s administration panel?

    Spoilering to hide wall of text related to this topic.

    This setting located in /Performance_p.html-page for example gives the java runtime more memory. Same page also has other settings related to ram, such as setting how much memory Yacy must leave unused for the system. (These settings exist so people who run Yacy on their personal machines can have guaranteed resources for more important stuff)

    Other things that would reduce memory usage is to limit the concurrency of the crawler for example. There’s quite a lot of tunable settings that can affect memory usage. Would recommend trying to hit up one of the Yacy forums is also good place to ask questions. The Matrix channel (and IRC) are a bit dead, but there are couple of people including myself there!

    Also, theres new docs written by the community, they might help as well! https://yacy.net/docs/ https://yacy.net/operation/performance/



  • Teclis - Includes search results from Marginalia, free to use at the moment. This search index has been in the past closed down due to abuse.

    Kagi, whose creation Teclis is, is a paid search engine (metasearch engine to be more precise) also incorporates these search results in their normal searches. I warmly recommend giving Kagi a try, it’s great, I’ve been enjoying it a lot.

    Other options I can recommend; You could always try to host your own search engine if you have list of small-web sites in mind or don’t mind spending some effort collecting such list. I personally host Yacy [github link] (and Searxng to interface with yacy and several other self-hosted indexes/search engines such as kiwix wiki’s.). Indexing and crawling your own search results surprisingly is not resource heavy at all, and can be run on your personal machine in the background.




  • Plex is a great example here. I’ve been Hetzner customer for many many years, and bought a lifetime license to Plex. Only to receive few months later a notification from Plex that I am no longer allowed to self-host Plex for myself(and only myself) at Hetzner and that they will block all access to my self-hosted Plex instance. I tried to ask for leniency or a refund, but that was wasted effort as well.

    In short, I was caught on a crossfire when for-profit company tried to please hollywood by attempting to reduce piracy, so they could get new VC funding.

    I am now a happy Jellyfin user and warmly recommend all Plex users to try it, the Jellyfin community is awesome!

    (Use your favourite search engine to look up “Hetzner Plex ban” for more details)