Old f4g posts for UO guilds on europa - Page 4

**Adam** · 17th July 2009, 12:53 PM

There's a lot of duplicate entries, that much I do know
I remember when looking through the raw MySQL tables there's copies of almost each thread/post, as it was grabbed from the VB archive of the old F4G boards.
The archive basically being a copy of every thread/post.
It stands to reason then that half the data is copies of the normal stuff.

**Adam** · 17th July 2009, 01:13 PM

Did a quick search, out of 58,814 entries, 21.743 entries contain the tag [Archive]
So 21,743 are duplicates. (I believe)

MMoudry · 17th July 2009, 07:36 PM

Hmm that sounds logical. Does anyone know how much data (pages, threads, posts) were on the old f4g forums?

**Adam** · 17th July 2009, 07:40 PM

Does this sound about right?

209 forums
16186 threads
1953 members
174038 posts

MMoudry · 17th July 2009, 08:15 PM

Hmm it doesn't add up... if we assume that there were about 10 posts per page then the data would cover about 210k posts, however we must assume there was at least 1 page per thread, therefore for it to add up only 5k threads must have had more than one page. Also the table 'user' which contains member names that I parsed from the posts contains 3749 lines.... hmmm I guess we'll just have to wait and see.

MMoudry · 17th July 2009, 08:47 PM

Although on the other hand, when I try to search for some posts that I remember I find them all

that is encouraging. Below is the screen of 2005 Sosarian Morning Poste.

**Adam** · 18th July 2009, 08:55 PM

Quote:

Originally Posted by MMoudry

Hmm it doesn't add up... if we assume that there were about 10 posts per page then the data would cover about 210k posts, however we must assume there was at least 1 page per thread, therefore for it to add up only 5k threads must have had more than one page. Also the table 'user' which contains member names that I parsed from the posts contains 3749 lines.... hmmm I guess we'll just have to wait and see.

I'm not sure either
But there was a LOT of duplicate information, especially with threads and posts

The raw data I'm left with right now is...

Posts - 173,142
Threads - 16,169

No doubt it's not all of the data which was lost, but if we can get back even some of it, it's a bonus.

The guy who was kind enough to write the python script to do this, has also uploaded the scripts to the UOF server, so if need be, others can work on fine tuning it.

MMoudry · 19th July 2009, 08:53 AM

Would it be possible for me to grab that script? I'd like to have a look at what he has done and at the parsed data.

**Adam** · 19th July 2009, 09:26 AM

Aye, check your PM's MMoudry

Ok, so this is what I have so far, but I must warn you, it's not perfect, heh

VBTest - Powered by vBulletin

There are a few problems, to list a few that I know of, include..

1) Some posts are blank
2) Posts don't have a date/time of creation
3) Threads don't have a username showing in the forums
4) You can't search for threads created by a certain user.

With the majorly bad out of the way..

1) You can search users posts
2) You can search the forum itself (I believe, it works)
3) Posts seem to be relatively intact and most have usernames

I can provide the python script files used to extract the data from the raw html that was scraped from google by Murad
I can't provide the database itself, as it's a VBulletin product (I Believe it's illegal to share it? not sure)
But, what I can do is convert it into another (free) forum software like MyBB and then people can have copies of it freely
F4G admins of course, can have all the stuffs

If people want to help recover more data etc, they're welcome to help out

MMoudry · 19th July 2009, 09:48 AM

right, I am checking the script now....

Murad

17th July 2009, 12:53 PM	#31
Adam "Can Set Custom Title" Join Date: Sep 2002 Posts: 9,126 Thanks: 82 Thanked 2,068 Times in 754 Posts Member's Picture Albums	Re: Old f4g posts for UO guilds on europa There's a lot of duplicate entries, that much I do know I remember when looking through the raw MySQL tables there's copies of almost each thread/post, as it was grabbed from the VB archive of the old F4G boards. The archive basically being a copy of every thread/post. It stands to reason then that half the data is copies of the normal stuff.

17th July 2009, 01:13 PM	#32
Adam "Can Set Custom Title" Join Date: Sep 2002 Posts: 9,126 Thanks: 82 Thanked 2,068 Times in 754 Posts Member's Picture Albums	Re: Old f4g posts for UO guilds on europa Did a quick search, out of 58,814 entries, 21.743 entries contain the tag [Archive] So 21,743 are duplicates. (I believe)

17th July 2009, 07:36 PM	#33
MMoudry Join Date: Jul 2009 Posts: 21 Thanks: 0 Thanked 4 Times in 1 Post	Re: Old f4g posts for UO guilds on europa Hmm that sounds logical. Does anyone know how much data (pages, threads, posts) were on the old f4g forums?

17th July 2009, 07:40 PM	#34
Adam "Can Set Custom Title" Join Date: Sep 2002 Posts: 9,126 Thanks: 82 Thanked 2,068 Times in 754 Posts Member's Picture Albums	Re: Old f4g posts for UO guilds on europa Does this sound about right? 209 forums 16186 threads 1953 members 174038 posts

17th July 2009, 08:15 PM	#35
MMoudry Join Date: Jul 2009 Posts: 21 Thanks: 0 Thanked 4 Times in 1 Post	Re: Old f4g posts for UO guilds on europa Hmm it doesn't add up... if we assume that there were about 10 posts per page then the data would cover about 210k posts, however we must assume there was at least 1 page per thread, therefore for it to add up only 5k threads must have had more than one page. Also the table 'user' which contains member names that I parsed from the posts contains 3749 lines.... hmmm I guess we'll just have to wait and see.

17th July 2009, 08:47 PM	#36
MMoudry Join Date: Jul 2009 Posts: 21 Thanks: 0 Thanked 4 Times in 1 Post	Re: Old f4g posts for UO guilds on europa Although on the other hand, when I try to search for some posts that I remember I find them all that is encouraging. Below is the screen of 2005 Sosarian Morning Poste. Attached Thumbnails Last edited by MMoudry; 18th July 2009 at 11:22 AM.

19th July 2009, 08:53 AM	#38
MMoudry Join Date: Jul 2009 Posts: 21 Thanks: 0 Thanked 4 Times in 1 Post	Re: Old f4g posts for UO guilds on europa Would it be possible for me to grab that script? I'd like to have a look at what he has done and at the parsed data.

19th July 2009, 09:26 AM	#39
Adam "Can Set Custom Title" Join Date: Sep 2002 Posts: 9,126 Thanks: 82 Thanked 2,068 Times in 754 Posts Member's Picture Albums	Re: Old f4g posts for UO guilds on europa Aye, check your PM's MMoudry Ok, so this is what I have so far, but I must warn you, it's not perfect, heh VBTest - Powered by vBulletin There are a few problems, to list a few that I know of, include.. 1) Some posts are blank 2) Posts don't have a date/time of creation 3) Threads don't have a username showing in the forums 4) You can't search for threads created by a certain user. With the majorly bad out of the way.. 1) You can search users posts 2) You can search the forum itself (I believe, it works) 3) Posts seem to be relatively intact and most have usernames I can provide the python script files used to extract the data from the raw html that was scraped from google by Murad I can't provide the database itself, as it's a VBulletin product (I Believe it's illegal to share it? not sure) But, what I can do is convert it into another (free) forum software like MyBB and then people can have copies of it freely F4G admins of course, can have all the stuffs If people want to help recover more data etc, they're welcome to help out

19th July 2009, 09:48 AM	#40
MMoudry Join Date: Jul 2009 Posts: 21 Thanks: 0 Thanked 4 Times in 1 Post	Re: Old f4g posts for UO guilds on europa right, I am checking the script now.... Murad

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Europa Guilds	Oyst	UO General Discussion	1	15th November 2008 05:13 AM
List of RP Guilds on Europa.....anywhere?	Moz	Europa RP General	7	30th June 2008 07:47 AM
Europa Guilds	Full Circle	UO General Discussion	16	12th July 2007 10:34 AM
Guilds?	merth	UO General Discussion	8	28th November 2006 09:59 AM
Faction Guilds Can't Alliance with Non-Faction Guilds	Thradia	UO News	0	12th April 2006 10:24 AM