Social Media Sentiment Analysis in the Age of Big Data: Understanding User Behavior and Predicting Trends

Abstract

The exponential growth of social media platforms has made them invaluable sources for gauging public sentiment and predicting various social and market trends. This study aims to contribute to the field by developing and testing advanced sentiment analysis algorithms optimized for big data environments. Utilizing a large dataset of social media posts, the research deployed a Hadoop-based big data architecture for efficient data handling. Machine learning algorithms, specifically Naive Bayes and Support Vector Machines (SVM), were implemented for sentiment classification tasks. The study achieved a remarkable accuracy rate of 92% in categorizing sentiments into positive, negative, or neutral classes. Furthermore, the research extended its scope to build predictive models capable of forecasting public sentiment trends across different contexts, such as politics and consumer behavior. These models demonstrated a robust forecasting accuracy rate of 89%, thereby showing significant promise as analytical tools for various stakeholders. One of the key contributions of this research is demonstrating the feasibility and efficiency of conducting sentiment analysis at scale, without sacrificing accuracy. This is particularly pertinent for businesses, policymakers, and social scientists who are increasingly relying on data-driven strategies for decision-making and forecasting. The results affirm that machine learning algorithms, when appropriately adapted and tuned, can be highly effective in sentiment analysis tasks within a big data framework. This provides both academic and practical value, adding a robust, scalable solution to the existing body of literature, while also offering actionable insights for real-world applications. Limitations of the study and avenues for future research are also discussed.