Leaderboard

For Christmas I'm making my dad a RetroPie Raspberry Pi so he can play all of his old DOS games. I also found an archive online that has 1405 of them for free http://www.abandonia.com/en/game/all. As far as I can tell, however, they have no option to download all of them at once. You have to sift through 141 different pages and click a series of links to download all of them individually. For that reason I wrote this program to do the dirty work and download them all. ##!/usr/bin/perl -w use strict; use LWP::UserAgent; my($ua, $response, $mainContents, $thisLink, $check, $page, $thisContents, $downloadLink, $name); my @links = (); $ua = LWP::UserAgent->new( protocols_allowed => ['http', 'https'], timeout => 10, agent => "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0", #Necessary otherwise 403 forbidden. ); $page = 0; while(1){ $response = $ua->get('http://www.abandonia.com/en/game/all?page='.int($page)); if($response->is_success){ $mainContents = $response->decoded_content; while($mainContents =~ /<a href=\"\/en\/games\/(.*?)\.html\"/g){ $thisLink = $1; if(scalar(@links) == 0){ push(@links, $thisLink); }else{ $check = 1; for(my $i = 0; $i<scalar(@links);++$i){ if($thisLink eq $links[$i]){ $check = 0; last; } } if($check){ push(@links, $thisLink); } } } if($mainContents =~ /current\">(.*?)<\/strong/g){ if($page - $1 == 0){ last; } } $page++; }else{ die $response->status_line; } } for(my $i = 0; $i<scalar(@links);++$i){ $downloadLink = &getDownloadLink($links[$i]); if($downloadLink =~ /\?game=(.*?)\&/){ $name = $1; } $ua->get($downloadLink, ':content_file' => $i." - ".$name.'.zip'); } sub getDownloadLink{ $response = $ua->get('http://www.abandonia.com/en/games/'.$_[0]); if($response->is_success){ $thisContents = $response->decoded_content; if($thisContents =~ /game_downloadpicture\"><a href=\"\/en\/downloadgame\/(.*?)\">/){ $response = $ua->get('http://www.abandonia.com/en/downloadgame/'.$1); if($response->is_success){ $thisContents = $response->decoded_content; if($thisContents =~ /files\.abandonia\.com\/download\.php(.*?)\"/){ return "http://files.abandonia.com/download.php".$1; } }else{ die $response->status_line; } } }else{ die $response->status_line; } } It's worth noting that all of their direct download links change on a timer, so you couldn't actually just make a big list of links that could replace this program. This program retrieves the links as it downloads them so they'll never be expired. I also sort of made this in a strange way. I had the program go to every page on the "All Games" list and find each of the links, but after I already had written most of the program this way, I learned that all of the download links are in the source code of URLs that are all in the format "http://www.abandonia.com/en/downloadgame/ ####" so theoretically I could have just made like one simple for loop and go through a certain number of numbers, but I noticed that some of the games have a number code that is far larger than 1405 which is the number of games that the site claims to have, so many of the links of that format might not be valid, and there's no way to know what the upper bound should be. This program automatically clicks through all of the different pages on the list, so even if they add more games in the future, this program would still get all of them (barring any formatting changes that might break my regex). For those reasons I still kind of like my way of doing it. Enjoy. Edit: Just as an update, now that I've downloaded them all it looks like there's actually only 1,134 games. I'm fairly certain that this is because a few of the games on their list are not free (such as "Alien Rampage"). But 1,134 games is still pretty good imo.