Dear Data Science...

Introduction: Statistics, a subject we encounter from our school days, often intimidates many. Yet, its importance spans across various industries, highlighting its significance. In this blog post, I aim to simplify this subject, making it easier to grasp. Let's dive in together. Types: Let's start by discussing the two main types: Descriptive and Inferential. Well! what are they? When you say " descriptive , " you would think of it as 'detailed,' right? Yeah, that's the meaning. In statistics, it refers to considering the entire population . When you say " inferential ", you would think of it as 'getting something from something', right? In statistics, it refers to considering a small sample from the entire population. Now, let's delve into Descriptive Statistics! Stay tuned for our next blog post, where we'll explore Inferential Statistics.   Fig. Descriptive Statistics Central tendency is a way of figuring out where the midd...

Introduction: Imagine training a model to predict house prices. You want it to be spot-on, right? But just being accurate isn't enough. You need a model that's reliable, consistently nailing predictions even for houses it's never seen before. This seemingly simple task becomes a complex dance between accuracy and generalizability, where bias and variance step into the spotlight. WHY Bias and Variance Matter: Bias deals with Training error. Variance deals with difference in Test errors while using different training sets. Bias: In machine learning, bias refers to the extent of disparity between a model's predictions and the actual target variable when utilizing the training data, i.e., training error. High bias can result in underfitting, a scenario in which the algorithm fails to grasp the pertinent relationships between the available features and the target values. Alternatively, if there's minimal bias against the training data, it can lead to overfitting. Th...

Introduction: This blog simplifies the learning of discrete random variables. When we understand the underlying relationships between different types of discrete random variables, we are more likely to remember the concepts. Let's dive into the learning. Why Discrete Random Variables: Discrete random variable are a fundamental concept in probability and statistics, and they have many practical applications for modeling and analyzing real-world phenomena.  Modeling Real-World Events: Many real-world events and situations involve countable or distinct outcomes. For example, the number of defective products in a manufacturing batch, the number of customers entering a store in a given hour, or the number of times a student raises their hand in a classroom. Discrete random variables are well-suited to model these kinds of events. Interpretable Results: When you work with discrete random variables, the resulting probabilities are often more interpretable. For example, if you're stud...

Introduction Outliers are data points that deviate significantly from the rest of the data in a set. They can be caused by a variety of factors, such as data entry errors, measurement errors, or anomalies in the underlying process. Outliers can distort the results of data analysis and make it difficult to identify trends and patterns. Two  common methods for outlier detection are: z-score and IQR.  Z-score method to identify outliers The z-score is a measure of how far a data point is from the mean of the data set. A z-score of 3 or more is generally considered to be an outlier. To calculate the z-score for a data point, you can use the following formula: z = (x - mean) / standard_deviation where: x is the data point. mean is the mean of the data set. standard_deviation is the standard deviation of the data set. The below code explains how to detect and remove outlier. As you can see, the outlier 1000 has been removed from the data set. IQR method to identify outlier...

Introduction Data measurement scales are a fundamental concept in statistics. They provide a way to categorize and quantify data, and they play a key role in determining the types of statistical analyses that can be performed. Researchers gather data through surveys, often using Multiple Choice Questions. In these instances, having an understanding of Data Measurement Scales – namely, Nominal, Ordinal, Interval, and Ratio – is crucial. This knowledge informs the appropriate statistical analyses that can be conducted on each scale. In the following blog post, we will delve into an exploration of the four fundamental data measurement scales: nominal, ordinal, interval, and ratio. Why these different measurement scales are being used? Each of these scales carries distinct characteristics that influence the way we interpret and analyze data. Fig. Data Measurement Scales 1. Nominal Scale At the foundational level, the nominal scale is the simplest form of measurement. Data at this scale are...