In data science or , one of the most important tasks performed when working with large amounts of data is data cleaning, data cleaning is the process of removing typing errors, inconsistencies, and missing values from a dataset. In this article, we'll see how to clean a dataset using pandas step by step. What is Data Cleaning? Data cleaning is the process of removing errors or inconsistencies from a dataset. To clean a dataset of information in data science or machine learning we have to make sure that there are no missing values, typing errors, type inconsistencies, duplicate rows, , or values, unnecessary columns, and many things more. The offers us a powerful toolkit for data cleaning. It contains a variety of functions that help us deal with these errors and fix them. In the following example, you will see how to clean a mock dataset of information step by step with the pandas library. This dataset is a simulation of a customer list. Data cleaning tutorial step by step First, you need to import pandas in Python After that, use the following code to create the user's data set: The user's data set will display in this way Here we use the pandas function to create a mock dataset, this dataset contains 7 columns and 11 rows, the columns are, a which is the user's unique id, a column, a column, the user's , the user's number, the user's , and finally a non useful column called which we will use as an example of how to delete an unnecessary column from a dataset. As you can see in the example dataset, the data has some inconsistencies in the columns, a few unnecessary symbols in the column, some values in the column are not capitalized, and each of the values in the column have different syntax which makes it difficult to work with them. 1. Delete duplicated rows The first thing to do when cleaning a dataset is to check and delete duplicated rows. As you can see in the dataset example the rows in index and as well as in index and have the same values, to delete these duplicated rows use the following code: df users As you can see the function removes all the duplicated rows, in our example dataset, the function successfully removed the rows at index and . 2. Delete unnecessary columns After deleting the duplicated rows, we have to delete all the columns that we're not gonna need, in our case the column. To delete this column use the following code: df users The method with the parameter removes the column from our example dataset, but it must be assigned back to the variable to replace its values with those of the new dataset. 3. Correct syntax errors in columns Now that we have removed all the duplicated rows and unnecessary columns, we have to check column by column to see if any of them need to be corrected. Some of the values in the column have unnecessary symbols at the beginning or end of the name, we can correct these errors with the following code: df users The method removes empty values at the beginning and at the end of a string, if we pass a specific value as a parameter it will look for that value at the beginning or at the end and remove it. In this example, we call the method in the column and pass it as parameters the values that we want to remove in a string , then we access the dataset of the column and we assign it the new values. Now that we have cleaned the column, we have to clean the column, as you can see in the dataset example, some of the values in the column do not have the first letter capitalized, to fix this use de following code: df users The function is used to apply a function to each element in a column of a dataset, in this example, it is used to apply a function to the column that capitalize the first letter of each string with the syntax but we have to check that the current value is of type , if it's not then we have to return the same value, this can be done with the syntax . 4. Set a unique pattern for a column After cleaning the column and the column, we have to set a single pattern for the column, as shown in the example dataset there are different patterns in the values of this column and we have to set a single one for all of them. To do this use the code: df users Here we want to establish the pattern on all the values in the column, for this first, we have to delete all the values that are not a number with the function and the syntax , after this, we want to set this pattern on all the phone numbers in this column, for this, we use a function and the f string (format string) expression which is used to embed expressions inside string literals, but we do this only if the current value has a type and have ten characters, as you can see in the index of the column we have a value and in the index index of the same column we have a number with only 9 digits, none of these characters meet the condition so they are replaced by a value that will be removed later. 5. Delete rows with None values Now that we have cleaned all the rows in the dataset, we have to delete the r