Hello,
This may come across as a bit strange, but my girlfriend needs some help with the data collection for her PhD, and it seems i'm not man enough to help her.
Basically what she wants to do is collect the details of some 32,000 members of a fansite, so she can analyse the breakdown of the members.
This brings two problems.
1) The first is getting the information off the page. I thought you could save the html file and then write some code to subtract the info you wanted (she only wants age, country and gender). But i'm not sure what you would use to do this.
2) The second problem is that the members are listed 30ish people to a page, and there are 1000 pages. Is there any way you could download all 1000 pages in one go so you didn't have to manually download 1000 html files?
Any help towards this would be much appreciated!