Does lemmy have any communities dedicated to archiving/hoarding data?
Be smart and keep it all on thumb drives.
I kind of want that hackermans diy pc that runs on 18650 cells
I thought the whole point of torrenting was to decentralise distribution. I use torrents to get my distros.
In my own little bubble, I thought that’s how most people got their distro.
magnetlink to any linux package repo torrent if you don’t mind?
i don’t wanna scrape since it takes forever and burdens
@Maroon I thought torrent technology to be a godsend for package managers.
Why none of them use it?
I mean, damn.
git and the lot are a lot better at this than people realize.
Torrents are often used for installers, but for packages it tends to be more trouble than what it’s worth. Is creating a torrent for a 4k library worth it?
Turns out hosting a bunch of files is very cheap.
What happens when they just cut the underwater cables? Torrent over carrier pigeon for a linux distro would take ages
Pigeon latency is horrible, but the bandwidth is pretty great. You could probably load up an adult pigeon with at least 12TB of media.
https://en.wikipedia.org/wiki/IP_over_Avian_Carriers
Just gonna leave this here for whoever wants to read more on the methodology and potential risks.
Over a 30-mile (48 km) distance, a single pigeon may be able to carry tens of gigabytes of data in around an hour, which on an average bandwidth basis compared very favorably to early ADSL standards, even when accounting for lost drives.
Compared to what I use at home now, this sounds great
A good way to see what the future of places like the U.S are is to look at places like North Korea, where they do exactly this, move files around on flash media to avoid the state censors.
Sneakernet to the rescue. Some of you are too young to know about walking around with boxes full of disks.
A wise man once said
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
It was trading CD-R’s during my high school days… good times. Napster was just starting to take off by the time we had a CD-R trading network set up, Napster just increased the amount of CD’s that got passed around.
Tiny jump drives on pigeons is low key excellent imo
We need some more community wifi projects
Community Wisps are cool
How would one go about making an offline copy of the repos? Asking for a friend.
Hmm, 3tb to mirror all the architectures I have around is very tempting. I have an unused dual bay external, and another one that desperately needs (wants) to have its 4tb drives upgraded.
Start from here https://wiki.debian.org/DebianRepository/Setup
Arch: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors
The official repo is only about 80GB, I have an old copy from when I was running an airgapped system. Not sure about the AUR, it’s probably in the TBs range though.
AUR might not be as big as you think. It would wouldn’t work for offline tho since many AUR packages pull archives from websites during the build process.
Oh yeah, didn’t think about that. Having a bunch of PKGBUILD files isn’t very useful.
I guess you could compile all the things, if you had a lot of spare processing power. A once a year snapshot would probably be enough for anything short of a Mad Max future.
That future might not be far off considering what Trump did today. Balance of power is seriously about to shift.
You ain’t wrong
man rsync man cron xdg-open \ https://lmgtfy.app/?q=debian+rsync+mirror
Get out of my mind.
When the Arch wiki was getting DDOS’d a few weeks ago I got a local copy from the AUR that was pretty handy.
Yeah not gonna lie, i think i heard someone in a youtube video a while back talk about how the entirety of wikipedia takes up like 200 gigs or something like that, and it got me seriously considering to actually make that offline backup. Shit is scary when countries like the uk are basically blocking you from having easy access to knowledge.
UKGOV haven’t started on things like Wikipedia yet. They know kids use it for school and blinded by ideology though they are, even they can see there’d be an enormous backlash if they blocked it any time soon.
If that’s going to happen at all, I doubt it would be before the next election. That’s whether Labour get re-elected or the Tories make an unexpected comeback. You can tell how far Labour have fallen in the eyes of their party faithful when they’ve taken a Tory-drafted policy and made it their own.
Ironically, the up and coming third option fascist party, have said they’re going to repeal the Online Safety Act. They have other fish to fry if they get in, and they’ll want to keep their preferred demographic(s) happy while they do it.
I assume that eventually something like the OSA would come back to “protect the children”. They love the current US President.
None of this is hopeful. Take this as more of a rant.
Every day it seems the entire west is gonna bee a fascist hellhole in a decade
I’m certain that when UK forces DigitalID upon the nation it will be a requirement for access to every website
Yeah, it’s surprisingly small when it’s compressed if you exclude things like images and media. It’s just text, after all. But the high level of compression requires special software to actually read without uncompressing the entire archive. There are dedicated devices you can get, which pretty much only do that. Like there are literal Wikipedia readers, where you just give it an archive file and it’ll allow you to search for and read articles.
if you remove topics you are not interessed it can shrink even more
Sure, but removing knowledge kind of goes against what creating a Wikipedia backup is about…
If my experience with mashing the random article button is any indicator, you could reduce the size by 30% just by removing articles on sports players. I doubt I’ll need those
Well, i doubt i will ever need to know anything about a football player or a car
“Fellow survivors, oh my God! What are your names?”
“I’m OJ Simpson. This is my friend Aaron Hernandez. And this is his car, Christine.”
I downloaded wikipedia a month or two ago, I recommend it.
How big is Wikipedia?
If you don’t care about edit history, and only care about English, there are zim files w/ images for <150 GiB
Sorry, I’m out of the loop. Is there something particular that triggered this that I missed?
Well for starters, teachers have had to start telling students that .gov websites are no longer considered credible sources for research.
Nice!
gestures broadly
I saw that post about texas requiring app stores (specifically says mobile devices, so not the typical distro repository… yet) to have age verification. If they expand that, it would mean all linux distros, while maybe leaving the windows .exe downloads (ugh, shudder) alone. Wikipedia is probably more relevant in most folks’ minds for having a backup though.
Nothing in particular that I’m aware of, just a growing recognition that things are very much not well in the US these days.
The broad censorship of government data in the US, combined with the recent political attacks on Wikipedia caused me to download the whole English Wikipedia earlier this year. Guessing OP is similar
Not sure why they’d download Debian with all packages though
Edit: I should mention it’s less about a potential loss of Wikipedia as it is a personal source of truth on politically sensitive topics that get censored, or turned to propaganda by bots
For example the Wounded Knee Massacre. Pete Hegseth has recently been calling it the, “Battle of Wounded Knee”. I wouldn’t be surprised if the current administration went to war with Wikipedia and forced them to 1) Change articles they disagree with, and 2) Hide those changes from history
I had downloaded the full (no pictures) Wikipedia earlier this year for exactly this reason. This thread told me about kiwix, which is awesome, so I downloaded the “Wikipedia .08” using kiwix, which is the best 45,000 articles from Wikipedia with pictures and it’s 7G, very manageable, has most topics anyone would care about.
My rationale with Debian is that distros are kind of like portals to entire compendiums of free and open-source software. With the increasing attacks on vpns in particular right now, I’m concerned there are any number of programs we take for granted that we might not have access to soon.
The internet is already deeply enshittified. There is a real possibility that it will no longer be a free and open web in any capacity soon. So it’s past time to make archives, and start setting up meshnets.
You’ll need about 500gb of free space. not too much of an ask tbh
i know this because i actually do this. its more like ~300gb of space but its better to have even more just in case
It makes me really happy that people can say “500gb … not too much of an ask” these days.
Well we are talking about the greatest repository of human knowledge ever created. So we can afford to spend a little on it at least.
I also recommend downloading “Flashpoint archive” to have flash games and animations to stay entertained.
There is a 4gb version and a 2.3TB version.
Is that Flash exclusive or do they accept other games from that era?
I’m not sure, but I do think it’s just flash
There is a 4gb version and a 2.3TB version.
That’s quite the range
When I downloaded it years ago it was 1.8TB. It’s crazy how big the archive is. The smaller one is just so it’s accessible to most people.
I still have a copy of wikipedia from 2021 somewhere on my NAS.
Okay so where do I find some cheap hard drives? Europe if possible :-)
look for dvr’s they have huge hdds in them and you can find them at thrift stores for cheap
I stumbled across this sort of fascinating area of doomsday prepping a few weeks back.
A nice addition to that, don’t just make it a USB, but a raspberry pi. So you’d have a reasonably low-powered computer you could easily take with you.
Not suggesting this one as it seems a bit expensive to me, but https://www.prepperdisk.com/products/prepper-disk-premium-over-512gb-of-survival-content?view=sl-8978CA41
Just built one of these myself. I went NVME M.2 instead of SD Card to avoid data corruption. I know SD Cards are fine if you don’t write to them a lot but if you wanna update or add your own stuff, scares me. Plus NVME is just so much faster.
How would you access the info if electricity permanently goes out?
You find a generator, or solar panels, or wind mill, or water turbine, or a bicycle hooked up to a generator.
If electricity permanently goes out then we’re in a scavenger situation and it is time to start taking apart things that are no longer necessary to build the things that are.
start taking apart things that are no longer necessary to build the things that are
Hey finally a good use for all of those cars, grab the alternators out for small generators (since bicycles are the ultimate apocalypse vehicle: simple, small, easy to maintain and don’t require complex fuels)
You only need 20 watts of power. One of those dinky fold up solar panels would work. Add a USB power brick for cloudy days.
2W for a RPi Zero with data on a microSD
You’re going to need a monitor as well.
I have a PaPiRus ePaper eInk e.g. https://media.digikey.com/pdf/Data Sheets/Pi Supply PDFs/PaPiRus_ePaper_Web.pdf and even though I don’t know the watts for a refresh but I assume it’s one of the lowest solution you can use.
PS: FWIW if you don’t refresh the display can keep the information on for months, if not years.
Eink displays are pretty awesome for this sort of thing, I repuposed a kobo ereader as a household info display and it worked nicely. Those PaPiRus screens look easier to interface with, but a little small for reading wikipedia articles. They’d do in a pinch, but the eyestrain would have me looking for a bigger solution.
Pretty much what Sinthesis said; USB power brick and/or solar panels. Both at the ready and tested. Also got a big ass battery backup that will charge off solar panels.
deleted by creator
at this point why not just use a phone running postmarketos?
Last I checked (3 years ago) postmarketOS drained the pinephone battery in record time :(
You’d first have to buy a phone that can run postmarketos and these are much rarer than I wish they were. Is there even anything new that can run it? Pine64 stopped making phones and said they’ll make a new one when they can make it RISC-V.
Fairphone maybe I guess. 4 is listed as a supported device, but someone has gotten it working on 6 too.
there’s lots of devices it runs on iirc, something like the pixel 3a can be had for less than a new rpi3b+ where I live
But hardware ages and dies. Will you trust a pixel 3a for the next 10 years? I’d rather have a new device for this.
Cause if ya wanna go overboard like I did, 1TB of NVME storage, can add with SD Card if necessary. 16GB RAM. Very little learning curve for my part as I use SBCs often. Plus almost every Docker container and program I want works on RPi without any hassle.
There’s also more robust guides and community for RPi.
Just my thoughts.
I heart there is Wikipedia on ipfs. Is that a good solution for Linux packages too?
