Why Mining Internet Social Media Is Difficult

by Curt Hall

The problem with mining blogs, message boards, online forums, and other social media is that it requires the use of text-mining tools that can analyze unstructured data. Similar to structured data mining, text mining uses sophisticated algorithms, such as neural networks, case-based reasoning (CBR), probabilistic reasoning, advanced statistical methods, and other machine learning techniques, to automate data analysis and discovery in unstructured data. But a key differentiator between the two is that text mining can also makes use of natural language processing (NLP) techniques, such as lexical processing and analysis, word/phrase parsing, and other methods, to enable text mining systems to identify and highlight key concepts and relationships among words in text. All of these techniques, however, are not widely understood by most corporate IT departments. Consequently, the mining and analysis of unstructured data is not widely used by mainstream organizations.

Password Protected Cutter Consortium clients, please log in:


This document is available to Cutter Consortium Resource Center clients only. Retrieve password.
If you would like further information about how to become a client, please contact us at +1 781 648 8700 or sales@cutter.com.
Why Mining Internet Social Media Is DifficultTue Apr 08 08:41:22 CDT 2008