Ethereal Search uses servers that access the YaCy search network. Any site can be added to that network by running a free YaCy webcrawler on your computer or server.
Installation example for Ubuntu. Click here for Windows, Mac, Linux. Lines with # are terminal commands.
# sudo apt update
# sudo apt upgrade
Reboot if neccesary.
# sudo snap install yacy-search
# yacy-search
Wait a few minutes, then close terminal. Then start terminal.
# yacy-search.passwd
Enter new password for yacy admin.
To keep YaCy running:
# crontab -e
Choose nano. Then add a line of this text:
* * * * * /snap/bin/yacy-search
Press [ctrl + s] to save, [ctrl + x] to quit.
Use browser to access localhost:8090.
Click "Administration", then choose peer to peer mode.
Click here for YouTube demo.
Click "Advanced Crawler" and enter URLs of the sites you want to crawl.
A crawl depth of 1, only crawls those sites. A depth of 2 also crawls the sites those sites link to.
Click "Process Scheduler" to setup automatic webcrawling.
If a remote server is used to run YaCy, access administration with SSH:
# ssh -L 8090:localhost:8090 username@serverhostname
Optionally boost performance by editing /home/username/snap/yacy-search/310/DATA/SETTINGS/yacy.conf
Enter the ammount of RAM in mb. Example 64 gb or 64000 mb:
javastart_Xms=Xms64000m
Enter the maximum ammount of hardrive space dedicated to YaCy (must be more than RAM). Example 960 gb or 960000 mb:
javastart_Xmx=Xmx960000m