Introduction to Data Science -2: Data Path Line
Hello🌺! In this article, we will discuss the process from the birth of data to its processing. We have understood that data is of great importance for both producers and user demands. So, what stages does data go through after it reaches us? Let’s look! 👀
First, we need to define the problem we have. The between we know our enemy, the better we can fight and win this battle. The problem definition phase includes the data collection part. This data may appear on one or more websites, in databases, as a service or API. This data can be in the form of audio files, videos, image files, or it could also be a JSON, XML, or Excel file. Once we define the problem, the more data we obtain, the better off we’ll be because information always equals power, and the one who can wield this power emerges victorious from this war.
After identifying the problem and collecting the data, we must explore the data. We must first determine the type of data, understand the expressions contained in data points and columns, and take action based on this understanding. First, we need to check what we have, how the column structure is, and whether there is any missing or incorrect information, so that we can prepare the data for analysis.
After exploring the data and understanding its structure, we must analyze it. What is the content of the columns in this data, which one is useful to us when sorted in ascending or descending order, and what kind of relationship exists between the columns? For example, we can consider an Excel file like the one below.
The ID column in the table in Figure 4.2 contains only the person IDs. In this data table, each person has a unique ID. Similarly, the Season column also contains specific data. For example, if I want to obtain data in ascending order from the information in this column, I must arrange the data analysis section accordingly.
Get Beyza Karakaya’s stories in your inbox
Join Medium for free to get updates from this writer.
After analyzing the data, we can model the data if it is necessary for us. For example, by looking at the table above, we can create a pie chart or a diagram of our choice to show which season is busiest in terms of the name. The data is now in our hands, and it is up to us to shape this data and put it to work according to our desires. The model creation phase will become extremely important for us in machine learning, deep learning, and artificial intelligence. The data model represents the world to be learned for us, and if the data is misrepresented, the model may fall into the error of learning incorrect patterns. For example, if an image recognition system is given many pictures of cats and very few pictures of dogs, the system may conclude that “There are only cats in the world!” it may get an impression like. A well-modeled data allows the model to learn faster and prevent situations such as underfitting or overfitting. Therefore, the modeling processes must be performed with sufficient accuracy and consistency.
After analyzing and modeling the data, we have reached the final stage. We manipulated the data, put it into action, conducted the necessary analyses, and built our own requirements based on this data. At the end of these processes, we must assess the data’s performance and subject it to measurement and evaluation stages. Because there may be interruptions, bugs, or errors in a portion of the data. Since we will be storing this data even after the completion of projects, we must periodically maintain and check these processed data.
Even if our phases are completed and the project is finished, these projects do not become mere waste. Information is always valuable to us. The information we have is stored, and perhaps later, when a problem arises regarding this information or when these data are needed for a new project, they may be required. Therefore, it will be beneficial to store data in a systematic manner.
As a result, data goes through specific stages of preparation, analysis, visualization-modeling, and maintenance from the moment it is generated. During these stages, data takes shape according to the needs of the individual or company. It is possible to obtain different results at the end of two different projects that require different outcomes with the same data.
✨ This article was about the stages that data goes through. If you want to learn more about data science, please don’t miss the rest of the series! 🚀
