Rafazwonull vs Joe I/O

Hi everyone,

I have almost finished formating my work and I guess I will go and print it in the next few days. I have learnt a lot of things about cleansing data and problems you stumble upon while data mining. Before really closing this chapter here and getting on to other projects I thought I might give you some ideas on my results.

In general it works quite well to determine the high quality blogs with the features used, even with rather simple algorithms. The decision tree in J48 for recognizing the controlled collection consists only of three nodes but gives you an accuracy of 98% for the German collection. The most interesting feature seems to be relationOutLinksToSize. This is confirmed by other collections and algorithms, and is quite an intuitively important feature. Just imagine link-spam blogs with a huge amount of outgoing links but no own content! What is more, with all collections the link-based features as the number of incomings links were really good discriminating features.

Interestingly, the accuracy (this means, how many blogs they recognize correctly) is not determined by the number of features you consider, but rather by how good they work together. With only 10 attributes taken together I get about the same accuracy for most algorithms, while some others improve substantially. This shows how important it is to have reliable data. The more useless features you give your algorithms to analyze, the more unreliable the resulting models will be. So it is much better to take a reduced number, which is easier to analyze and make your results with these ones…

J48 decision tree

So, what about the other collections? Here is the offside. The models seem to be at least partly language specific although I cannot really tell to what extent. All I can tell is that the Russian and the English collection created different models. While I achieved somewhat weaker results with the Russian one (67% with J48), the English collection was really a catastrophy: J48 got only 8% correct. I wondered why that was and tried to reduce the number of features considered, but only the Russian collection improved substantially through this (78% with J48). The English one stayed on that low level.

I guess the reason for the difference in accuracy is due to a certain lack of homogenity of the data. While I got rather homogenous collections in Russian and German, there are a lot of people blogging in English (as me right now). Necessarily the amount of different types of blogs raises in this language, and the overall number of blogs available increases. What is more, the average quality does not increase, so I get a random collection which is really heterogenous and the classifier does not know how to determine a profile of high qulity blogs. However, this idea needs to be proven by other results.

So, what do you think? Anybody caring for the results of the language recognition (russian blogs work best!)?

-r-