
Originally Posted by
maborosi
Thank you for chiming in, neko_kawaii. I really appreciate that. <3
I can go ahead and start archiving the forums ASAP.
The first tool basically creates an offline mirror of the forum- it preserves the structure of the boards and all of the HTML/CSS/hyperlinks, images in the posts, etc. It would be like browsing the boards normally as best as I can tell, but it will just not be searchable thru google.
*ETA: However, I can eventually implement a search functionality if I were to host the archive on a server so that people can look through it more easily. That will just take some time to do. Not a bad project by any means though. :P
If in the future we have to start a new forum from scratch, this could be enormously helpful in rebuilding some of the knowledge base.
It would take a while to create a robust archive- probably several weeks, since fast crawls would chew through a lot of bandwidth and I don't want to be a jerk. I think the site probably totals around 50GB-75GB? (Not counting anything hosted externally like images, etc) That's my best guess just based on forum statistics, but if y'all have better access to those metrics that would really help to get a better sense of the scale of this archiving.
I have several big ol' harddrives laying around that could hold a local copy of the site, no problem. Space won't be the issue. I just want to be considerate and not eat up bandwidth like a madman while trying to get things backed up. :P
The other thing to consider is that this would not scrape anything behind a login wall. If people think it's worthwhile to archive things like blogs, or the forums that are hidden from the public, then maybe people can archive individual things manually, but I'm prioritizing trying to grab anything that is public-facing, if that makes sense.
Nightblooming- I saw you replied while I was, lol!
I think it wouldn't be a bad idea to start looking into what it might take to get a forum up. <3
--
If we can't get database access then this will probably be the next best thing to make sure we don't lose everything. I want to try my best to make sure as much of the forum is preserved if it goes down.
Bookmarks