UPDATE: To everyone who suggested YUNO, thank you so much. This seems like it is about to make my journey much easier. It is basically almost exactly what I was looking for, but I was unaware that it existed.
Thank you ALL for your suggestions, actually. It’s a bit overwhelming for an almost complete noobie but I an going to look into all of the suggestions in time. I just saw that there were several mentions of YUNO so I decided to make that one of the first things I investigated.
So, about two months ago, I had a very eye opening experience. As the result of a single misconfigured security setting on my Android, I was locked out of my Google Account on my phone AND all of my PCs. I had no access whatsoever to Google, or any of the literally hundreds of services that I get through Google.
This is when I realized that I relied entirely on Google/Android because those two days were actually very difficult, being cut off from media, services, passwords, everything, from the past almost twenty years of my life, could be taken away from me in an instant. The decades of my life that were locked away in my Google Account included hundreds of thousands of pictures, almost a hundred thousand audio tracks, several hundred books, several hundred apps, thousands of videos, etc. ad infinitum. Unfortunately, very little of this material was backed up at that point. That is my fault. Also, the misconfigured security setting was my fault as well.
The amount of data, media, memories, services, etc. that would have been lost is actually endless and it would have affected my life in several ridiculously negative ways.
Luckily, in the end, I was able to get my access back and then basically immediately grabbed all of the several terabytes of information and media of mine that they had, and that I was almost locked out of. I have it all in my house now on a drive in my computer, with a backup made on another disconnected disk.
I then decided that no corporation was ever going to have such an insanely high level of influence on and control over my entire life and my media ever again. That experience was actually very scary.
I’ve been trying to get into SelfHosting, but am finding it quite daunting and difficult.
There is a LOT of stuff that I have to learn, and I am mostly unsure of where to even begin. I know basically nothing about networking.
I need to learn the very basic stuff and work my way up from there, but everything that I’ve seen on the Internet assumes that the reader already has a basic to intermediate understanding of networking and the subjects that surround it. I do not, but I am going to learn.
I just need someone to show me where to start.
Thanks in advance for any assistance!
I will probably get flogged by this answer but here it goes:
I’d throw you right into the deep end: get a spare machine (an old laptop or PC) and install proxmox on it. Play around, breaks shit, delete the container/VM and start over.
Grab stuff from the Community Helper Scripts and see new stuff, try alternatives, see what works for you and don’t be afraid of breaking stuff.
It takes a bit longer and some basic concepts might fly over your head, but the stuff you learn like this, you learn by heart.
It’s been a few years since I started tinkering with a laptop with a busted video output circuit. Now I serve NextCloud and Immich to my family, keep receipts and documents neatly organised on Paperless, have a decent arr stack and a bunch of extra goodies. All from “a PC without video? Might as well make a server” now with a proper machine with several drives on ZFS pools, health checks and redundancy.
Its a helluva rabbit hole.
I was unaware that those Community Helper Scripts existed! They should definitely be helpful at some point down the road!
Good luck!
Isn’t that how everyone does it? 😬
I didn’t start with a spare, so by the time I was semi-reliant on my self hosted stuff, a breakage was an issue. Also I started with bare Linux, then CasaOS. There was no easy rollback from snapshot/restore backup like on proxmox
Hi @MTZ , #selfhosting could be a move in the right direction for you. I started managing my own servers over 10 years ago, locally, from my home, later VPS and finally again from my home. Eventually I moved toward @yunohost - it simplifies a lot of things! I documented some my experiences at https://wasi.ovh
Start small: setup file/photo sync (@nextcloud), calendars and contacts and gradually start adding data from old backups once you feel comfortable.
Have fun and good luck :)start small … nextcloud
@lpryszcz @MTZ woohoo! good luck, and in case you go for it, we have a really nice community here to help you out: https://help.nextcloud.com/
Thanks for the suggestions! Someone else has mentioned YUNOHost and that is definitely on my radar and my mental list of things to look into!
Damn, that’s scary indeed! First of all, congratulations on your resolve to take control of your data. You have a long journey ahead of you, but don’t be discouraged, take one step at a time and don’t be afraid to ask for help.
As for where to start, I think you’ve already figured it out yourself: invest some time in learning the basics of networking. You don’t need to become an enterprise-level networking wizard, just learn the basics: learn what an IP address is, what a network mask is (sometimes also referred to as “prefix length”), what DNS is and does, how to change these settings on your home network and why you’d want to change them. Try stuff, break it, fix it, repeat. Also, if you’re not familiar with or already using it, it might be a good opportunity to pick up Linux. If you’re coming from Windows, a beginner friendly distribution like Linux Mint will do nicely. Try installing it on an old computer to see what it’s like, poke at it until you’re comfortable, then maybe make it your main operating system. Knowing Linux basics (command-line shenanigans in particular) will give you a big edge when you decide to start hosting your own services.
I’m not very familiar at all with Linux. I have used a few distros here and there where the majority of the work is done in a GUI. The CLI intimidates me, hahaha. I began exploring distros through WSL a little while back. I still haven’t “dove in” yet though.
As far as the networking terms that you used, I have at least a slight grasp on all of that and what the different network “components” do, but I do not yet know how to implement or configure most of the things that you mentioned.
This is an area where AI can be helpful. Tell the AI what Linux distro your on and what you want to do. Most of the time it will give you pretty good answer. If you don’t understand what it is telling you to do, ask it to explain the thing in detail. Most important thing though is to always verify what it tells you before you run stuff. Google search specific commands or use the “man” command to get documentation. The key thing is the AI can make you aware of CLI commands and tools more easily then trying to find what you need on your own.
Thanks! That is good advice and I will begin doing that to try to familiarize myself with the CLI.
There’s a GitHub repo called awesome self hosted. I would get the link but I’m mobile. It has a pretty exhaustive list of self hosted software with a link to their page. Might be worth loooking at.
edit: https://github.com/awesome-selfhosted/awesome-selfhosted
This seems to be an extremely valuable resource. I love the GitHub awesome lists, and I definitely starred this one as a reference point!
In addition to what another poster said about getting an off-site backup hard drive, I would recommend looking into setting up a raid array for data redundancy with your online storage. You don’t want one hard drive failure to make all of your data inaccessible.
The FUTO guide is meant for total beginners.
Thank you! I will begin to look over it tomorrow!
I am a sysadmin with over 30 years of experience managing servers and networks for businesses of all sizes as well as for myself, friends, and family.
The FUTO guide is extremely detailed, accurate, and accessible. It does not always follow best practices, and it’s not a comprehensive guide to all of the possibilities for self-hosting. It’s not trying to be. It is a guide for someone with no technical expertise (but with basic technical ability) to degoogle/deapple themselves at a reasonable level of cost and effort.
You do not have to do everything in the list, you can pick and choose the parts you’re interested in. That said, I would recommend reading through the whole article as you have time, because it does a very good job of explaining the concepts involved in building a self-hosted setup, and understanding how everything works is the biggest step toward being able to effectively troubleshoot problems when they inevitably crop up.
If you have specific questions about things that aren’t answered in the guide or via a quick web search, post them here.
Love the explanation. I’ve had a homelab for 20 years now and have never heard of FUTO. You’re explainer has made me bookmark the site now for future skimming.
This, so much. I remember when Louis told everyone about it, people (mostly Reddit) were so nitpicky over every minor detail.
I’d be curious seeing the discussion. Its healthy to have respectful competing opinions and see what rises to the top
It was a while ago, I don’t remember much of it unfortunately. Only the idiots.
Thanks for the detailed explanation! I will definitely begin studying the document tomorrow!
I didn’t know I needed this. Thanks!
deleted by creator
They use OpenVPN for some reason. Wireguard is superior in every way. In case you set up a VPN.
I’m about 90% decoupled from Google, it’s been a journey.
I’m at the difficult stage of contemplating how to decom my gmail email, and the Google account itself.
I’ll throw my hat in the ring and offer any help if you need it. Similar to others here, I suggest you start with something discrete like photos.
I’m at the difficult stage of contemplating how to decom my gmail email, and the Google account itself.
I have a couple of domains at OVH, and for the last few years I’ve been using email aliases when making accounts, all redirecting to my Gmail. The other day I had enough, created a mailbox on one of the domains, changed all aliases to this new address, and installed Thunderbird mobile. So far it’s been going well.
The webmail runs roundcube, which is usable but not great. The biggest problem I’ve had is the inability to create automatic filters.Edit: forgot about the calendar, I’ve moved that to my Nextcloud instance
I moved my calendar to Nextcloud, then radicale. My contacts too. Gmail is just a wean away.
My problem is how I’ll be able to deal with work apps like ms authentication. Even if I set up a 2nd “normal” phone for work only, I need to sign in to the play store to get the app… Its a chicken-and-egg problem.
@non_burglar @Damage Are they just TOTP authentication codes you need? If so, you there’s alternatives
Certain apps do not allow one to use freeotp et al (o365).
Right, forgot about the contacts, those are on Nextcloud as well. What’s the advantage of radicale over NC?
I still run the play store, but I get what I can from F-Droid. I’m not familiar with the MS authentication app, but I use Ente Auth, idk if that’s comparable.
What’s the advantage of radicale over NC?
Functionally, they work the same. I got kinda tired of fixing NC every other upgrade, though. It was always some “occ add missing indices” or some similar garbage. Like just solve this, already. Make that part of the upgrade.
Hmmm, luckily I haven’t had issues so far, but I’m using the AIO docker container
Unless you have experience with ethernet equipment and such it is probably better to start with some hosted service of an open-source app like Nextcloud or Immich or (slightly more advanced) a VPS somewhere. Doing it immediately from home with your own server has a steep learning curve.
Thank you for the advice! After I get a firm grasp of the concepts at hand, I will look into NextCloud and Immich.
You seem to imply you also want to selfhost some email service. But that’s sadly one of the few things that will always be better at a trusted third party email provider.
Besides that it seems the most important thing you want is pure data storage, and that kind of selfhosting is not hard. In many cases one would not even consider it as part of “selfhosting” as it can be as simple as a local NAS or external HDD.
So my question is what do you actually want to accomplish? Because I think for a lot of your concerns you don’t even need to go and host something.
I don’t want to replace email, per se. More so, the services that come along with it. Stuff like photo storage and sync, notes and reminder sync, calendar sync, and a lot of the hundreds of other small things that I have just been leaving up to Google that would cause a huge problem for me if I lost access.
I have the password situation handled with KeePassXC on my PCs and KeePassDX on my phone, but that’s about as far as I have come as of yet.
Note taking? Joplin. It’s been amazing for me and replaced Evernote after they started with their BS.
Photo storage. Been using Photo prism but I’m open to a better one.
Keepass is a great way of password management, I use keepass as well. I also use syncthing to sync my password database across all devices and then I have the server acting as the “always on” device so I have access to all passwords at all times. Works amazing because syncthing can also be setup so when a file is modified by another device, it makes a backup of the original file and moves it to a dedicated folder (with retention settings so you can have them cleaned every so often). Life is so much easier.
For photo access you can look into immich, its a little more of an advanced setup but, I have immich looking at my photos folder in syncthing on the server, and using that location as the source. This allows me to use one directory for both photo hosting and backup/sync
Do you know anywhere that I could find a guide on how to set that up correctly? I’ve used SyncThing in the very recent past and it ended up being not good because of my own ignorance. I have syncthing on my phone and my desktop, but neither of them are syncing anything to anywhere. I’m terrified that I will mess something up and possibly lose data, or end up with a gigantic mess of files that do not belong in the synced directory. That’s what happened the only time I ever did use Syncthing. For some reason, it basically transferred a gigantic directory of things that were not relevant at all from my PC to my phone, filling up all of the storage and it was a hassle to get rid of everything that was not supposed to be there. I REALLY want the KeePass databases on my phone and on my main PC to communicate and update each other though. Entering everything in twice has been frustrating.
I haven’t used a guide aside from the official getting started with syncthing page.
It should be similar to these steps though, I’ll use your desktop as the origin device.
- install syncthing on all devices you want to be syncing with
- on your desktop syncthing page, click “add remote device” and add the device ID of your phone(found on your phones syncthing app), you can also add any other device you want to have communications with (you will need to approve this action on the phone as well so be on the lookout for a notification)
- make a backup of your current keepass file just in case these steps shouldn’t cause files to change but, since the end goal is syncing two devices that you have mentioned have differences with files with the same name better safe than sorry
- create a keepass share on one of the devices (the folder path of this file should be wherever your keepass file is stored on your device. If this file is in a folder with a bunch of other files, you may want to move the file to it’s own subfolder or you will end up sharing all of the files in that path)
- under file versioning chose what type of file version control you want. I prefer staggered since it when a remote device changes the file it moves the old file to a folder, and then deletes them according to the settings
- at this point you should double check the name of your mobile devices keepass file name, if its the same as the name of the db on the desktop, you should rename it prior to continuing. Keepass should be able to detect a file conflict and rename it on it’s own but, better safe than sorry.
- share the folder with the device you want to sync it(your phone in this case)
- Your phone should get a notification that a device wants to share something with it. Approve it, be careful not to clear it because it’s a pain in the butt to get that notification back if you accidentally deny or swipe it away, the mobile app isn’t /amazing/ with it’s UI (but it has gotten better)
- once approved configure it to where you wanted the file to be on your mobile device.
- You should be done at this point. Syncthing should be automatically syncing the keepass files between the two
Some things you may want to keep into consideration. Syncthing only operates when there are two devices or more that are online. I would recommend if you are getting into self hosting a server, having the server be the middle man. If you end up going that route these steps stay more or less the same, it’s just instead of sharing with the phone, its sharing with the server, and then moving to the server syncthing page and sharing with the mobile. This makes it so both devices use the server instead of trying to connect to each other. Additionally, if you do go that route, I recommend setting your remote devices on the server’s syncthing instance to “auto approve” this makes it so when you share a folder to the server from one of your devices, it automatically approves and makes a share using the name of the folder shared in the syncthing’s data directory. (ex. if your folder was named documents and you shared it to the server, it would create a share named “documents” in where-ever you have it configured to store data). You would still need to login to the server instance in the case of sharing said files to /another/ device, but if your intent was to only create a backup of a folder to the server, then it removes a step.
Another benefit that using the server middleman approach is that if you ever have to change a device later on down the road, you are only having to add 1 remote device to the server instance, instead of having to add your new device onto every syncthing that needs access to that device.
Additionally, if you already have the built in structure but it isn’t seeming like it is working, some standard troubleshooting steps I’ve found helpful:
- if trying to share between devices, make sure that there is at least two devices that are connected as remote devices active in order to sync
- If above is true, make sure the folder ID’s are the same between both devices. that is how syncthing detects folders that should be sync’d
- If also true, make sure the devices are being seen as online in remote devices. If it isn’t showing as online, the connection is being blocked somewhere, verify you don’t have a firewall or router blocking it somewhere.
That is so awesome. Thank you so very much for taking the time out to inform me on this matter.
get your password situation squared away! every time i spin something new up i am grateful to have a pw manager to keep it all unique and maximum character limit
don’t even have to memorize the user of a lot of em
That’s the only thing that I do have taken care of! I basically immediately grabbed them out of Chrome and put them in KeePassXC on my PCs and KeePassDX for my Android.
Baby steps!
@MTZ @SidewaysHighways You may wish to disable all browser password managers, on all devices and use an alternate method of password management that suits your needs.
This is a cautionary tale on browser password managers (amongst other facepalms) that saw about millions of people’s personal details stolen - https://www.oaic.gov.au/__data/assets/pdf_file/0037/228979/Medibank-data-breach-alleged-timeline-infographic.pdf
While YUNO is a great way to get started, I strongly encourage you to understand basic concepts, like docker, and maybe try to run something outside of it for fun. While not even remotely the same thing since YUNO is just the OS and “app store”, you would be very similarly tied to that ecosystem the same way you are to Google now. Not to mean that YUNO would have any control over your stuff, but you would be dependent on them for what you can self host.
Yes, I’m definitely trying to understand the “why” of what it is that I am doing. I’m just going to be using YUNO as a starting point.
I’m 100% self taught & was in exactly the same place. I’d never used Linux before I got my first Pi. I spent a bit of time trying to familiarise myself with & made some notes regarding command line (notes I still rely on).
There’a so many ways to achieve the goal, you’ll eventually find a way that works for you. My personal preference was Docker/Docker Compose deployed via Portainer.
Even that was confusing. Until I found this excellent video on how to read Docker requirements & apply them step by step into Portainer. He explains slowly & methodically exactly what he is doing & why.
Portainer is a method of handling Docker stacks/containers via a web UI. Both Docker & Portainer are simple to install.
It’s easier to use Docker Compose files and/or .env (environment variable) files (both are even simpler to deploy through Portainer) but this video taught me what was going on & gave me confidence to have a go. What attracted me to Docker is you can easily remove stacks/containers if/when you make a mess rather than wiping the drive & start again, which is how I went about things initially.
This gave me the tools to set up Nginx Proxy Manager & I never looked back.
As you’ve realised, a robust backup solution is essential (plus off site backup for particularly important stuff) as things will inevitably go wrong along the way (I see Borg, Restic mentioned often, I went for Kopia).
I can’t recommend highly enough making detailed notes along the way, I rely on Joplin.
If you start using Docker, dont fall into the trap of using the “latest” tag. If you know the version number you’re running its far easier to re-deploy if an update is bad.
Enjoy your new time consuming, teeth gnashingly frustrating …and yet rewarding hobby 👍
Start with a nas, the rest will naturally come when you try to access your data for outside, or organize your data, or save more data types to your nas.
Your nas should be the central device and you build the rest around it.
Now, The question is, which nas? I would recommend synology, they are not too performance, a bit expensive and the company is lately doing suspicious moves, but the sw and the hw are rock solid and they are quite good for beginners from almost all angles. Extra point for how many howtos and tutorials are present in internet.
Once you are comfortable with them, you will realize the rest
Check out YUNOhost - it’s pre-configured for you and designed for beginners. Mine’s been running for about three years on a VPS with no problems and I had no previous experience with self-hosting.
Definitely keep your files backed up locally though. No server is invincible.
I will certainly look into that. I’ve never heard of YUNOhost but I’m going to give it a look soon!
Do it! It’s madness that YUNOhost is not more well-known, considering what it does.
Write things down
You will break something - and that’s good, it’s the best way to learn - but you’ll want to make a note of what you did / went wrong / how you fixed it.
Future you will still break things and be grateful that you wrote that thing down
You’ll buy something and find next year it was the wrong thing (too small, too large, too old, too new), so just get second hand stuff until you know what you need.
Cabled networks are so much better than wireless, but then you’ll need switches and cables and shelves and stuff… so using today’s wifi is fine, but know where you’re heading.
You need to store you stuff - that’ll be in a NAS
You need something to run services on - that’ll be your server
These might be the same physical metal lump (your 2nd laptop?), they might be separate… play around, break something and work out what feels right for you… and then put your data on there
… and that’ll break too.
Just be aware… if sync files between devices. That’s not a backup. (Consider you’ve deleted / corrupted something - it’s now replicated everywhere)
Having a NAS with 10 drives in a RAID6 array, is not a backup. It’s just really robust against a drive failure, but a deleted file is still a deleted file.
Take a full copy of your data off your system - then restore it somewhere else.
Did it work? If so, that’s a backup.
Thanks for the advice. I am using Trilium to create a knowledge base as I go, and I am keeping meticulous notes on my progress, successes and failures.
I’ve always struggles with practicing restoring backups. Do you have to buy an identical 2nd machine to see if everything still works w/o messing up the first one?
Not really.
I keep my data backups (docs, photos, etc) separate from the OS backups.
So, depending on what you’re using to do the backup, often they can just simulate a restore and just check the backup’s not corrupted. Not really a restore, but at least you know it’s not trash.
If you’ve backed up your data with a simple copy / sync (ie not a “backup” program), then you can restore your data somewhere else (maybe even jist a part of it) and do a compare.
But, yeah, if you’re restoring the OS, then it might be ok restoring it in a VM to check it…
I’m slowly moving towards no OS backups and using Ansible to be able to recreate the system(s) from scratch… of course I need to backup the ansible files too 😉
I would like to backup and test restore app configs not sure how to do that yet :p
Maybe OS backup is the way somehow. Maybe I need an atomic distro …
Well, it depends on the app & how it’s installed.
Most bare-metal installed apps are in
/etc/or~/.config/… maybe under/opt/or/usr/- just copying those files out on a regular basis is usually good enough.Might be a little more tricky if you’re trying to backup a database / live files, but those apps usually have some documentation about backups.
If it’s a container or flatpak, etc. then it’s probably somewhere obscure.
I wanna say thank you for making this post OP. I’ve got a spare laptop that I want to try to turn into my own cloud server but I find the endeavour similarly hard as well. I’ll be looking at the tips in the comments. Good luck OP!
Good luck to you, as well friendly stranger!
