Muhammad Sameer Hussain

Look into data science myths in this blog. The field of Data is an ever-growing field and often you’ll come across buzzwords surrounding it. Being a trendy field, sometimes you will come across statements about it that might be confusing or entirely a myth. Let us bust these myths, and ensure your doubts are clarified!

Myths

Now, let us dive into some of the myths:

1. Data Science is all about building machine learning and deep learning models

Although building models is a key aspect, it does not define the entirety of the role of a Data Scientist. A lot of work goes on before you proceed with building these models. There is a common saying in this field that is “Garbage in, garbage out.” Real-life data is rarely available in a clean and processed form, and a lot of effort goes into pre-processing this data to make it useful for building models. Up to 70% of the time can be consumed in this process.

This entire pipeline can be split up into multiple stages including acquiring, cleaning, and pre-processing data, visualization, analyzing, and understanding it, and only then are you able to build useful models with your data. If you are building machine learning models using the readily available libraries, your code for your model might end up being less than 10 lines! So, it is not a complex part of your pipeline.

2. Only people with a programming or mathematical background can become Data Scientists

Another myth surrounding is that only people coming from certain backgrounds can pursue a career in it, which is not the case at all! Data science is a handy tool that can help a business enhance its performance in almost every field.

For example, human resources is a field that might be distant from statistics and programming, but it has a very good implementation of data science as a use case. IBM, by collecting employee data, has built an internal AI system that can predict when an employee might quit using machine learning. A person with domain knowledge about the human resource field will be the best fit for building this model.

Regardless of your background, you can learn it online with our top-rated courses from scratch. Join one of our top-rated programs including Data Science Bootcamp and Python for Data Science and get started!

Join our Data Science Bootcamp today to start your career in the world of data. 

3. Data Analysts, Data Engineers, and Data Scientists all perform the same tasks

Data Analysts and Data Scientists roles have overlapping responsibilities. Data analysts carry out descriptive analytics, collecting current data and making informed decisions using it. For example, a data analyst might notice a drop in sales and will try to uncover the underlying cause using the collected company data. Data Scientists also make these informed business decisions. However, they involve using statistics and machine learning to predict the future!

Data Scientists use the same collection of data but use it to make predictive models that can predict future decisions and guide the company on the right actions to take before something happens. Data engineers on the other hand build and maintain data infrastructures and data systems. They’re responsible for setting up data warehouses and building databases where the collected data is stored.

4. Large data results in more accurate models

This myth might be partially wrong but partially right as well. Large data does not necessarily translate to higher accuracy of your model. More often, the performance of your model depends on how well you carry out the cleaning of your dataset and extraction of the features. After a certain point, the performance of your model will start to converge regardless of how much you increase the size of your dataset.

As per the saying “garbage in, garbage out”, if the data you have provided for the model is noisy and not properly processed, likely, the accuracy of the model will also be poor. Therefore, to enhance the accuracy of your models, you must ensure that the quality of the data you are providing is up to the mark. Only a greater quantity of relevant data will positively impact your model’s accuracy!

5. Data collection is the easiest part of data science

When learning how to build machine learning models, you would often go to open data sources and download a CSV or Excel file with a click of a button. However, data is not that readily available in the real world and you might need to go to extreme lengths to acquire it.

Once acquired, it will not be formatted and in an unstructured form and you will have to pre-process it to make it structured or meaningful. It can be a difficult, challenging, and time-consuming task to source, collect and pre-process data. However, this is an important part because you cannot build a model without any data!

Data comes from numerous sources and is usually collected over a period by using automation or manual resources. For example, for building a health profile of a patient, data about their visits will be recorded. Telemetry data from their health device such as sensors can be collected and so on. This is just the case for one user. A hospital might have thousands of patients they deal with every day. Think about all the data!

Please share with us some of the myths that you might have encountered in your data science journey.

Want to upgrade your data science skillset? checkout our Python for Data Science training. 

There are two key schools of thought on good practices for database management: data normalization and standardization. We will learn why does each matter? 

Organizations are investing heavily in technology as artificial intelligence techniques, such as machine learning, continue to gain traction across several industries.

• A Price Water Cooper Survey pointed out that 40% of business executives in 2018 make major decisions at least once every 30 days using data and this is constantly increasing
• A Gartner study states the 40% of enterprise data is either incomplete, inaccurate, or unavailable

As the speed of data entering the business increases with the Internet of Things becoming more mature, the risk of disconnected and siloed data grows if it is poorly managed within the organization. Gartner has suggested that a lack of data quality control costs average businesses up to $14 million per year.

The adage of “garbage in, garbage out” still plagues analytics and decision making and it is fundamental that businesses realize the importance of clean and normalized data before embarking on any such data-driven projects.

When most people talk about organizing data, they think it means getting rid of duplicates from their system which, although important, is only the first step in quality control and there are more advanced methods to truly optimize and streamline your data.

There are two key schools of thought on good practice: data normalization and standardization. Both have their place in data governance and/or preparation strategy.

Why data normalization?

A data normalization strategy takes database management and organizes it into specific tables and columns with the purpose of reducing duplication, avoiding data modification issues, and simplifying queries. All information is stored logically in one central location, reducing the propensity for inconsistent data (sometimes known as a “single source of truth”). In simple terms, it ensures your data looks and reads the same across all records.

In the context of machine learning and data science, it takes the values from the database and where they are numeric columns, changes them into a common scale. For example, imagine you have a table with two columns; one contains values between 0 and 1 and the other contains between 10,000 and 100,000.

The huge differences in scale might cause problems if you attempt to do any analytics or modeling. This strategy will take these two columns by creating a matching scale across all columns whilst maintaining the distribution e.g. 10,000 might become 0 and 100,000 becomes 1 with values in-between being weighted proportionality.

In real-world terms, consider a dataset of credit card information that has two variables, one for the number of credit cards and the second for income. Using these attributes, you might want to create a cluster and find similar applicants.

Both of these variables will be on completely different types of scale (income being much higher) and would therefore likely have a far greater influence on any results or analytics. Normalization removes the risk of this kind of bias.

The main benefits of this strategy in analytical terms are that it allows faster searching and sorting as it is better at creating indexes via smaller, logical tables. Also, in having more tables, there is a better use of segments to control the tangible placement of data store.

There will be fewer nulls and redundant data after modeling any necessary columns and bias/issues with anomalies are greatly reduced by removing the differences in scale.

This concept should not be confused with data standardization, and it is important that both are considered within any strategy.

What is data standardization?

Data standardization takes disparate datasets and puts them on the same scale to allow easy comparison between different types of variables. It uses the average (mean) and the standard deviation of a dataset to achieve a standardized value of a column.

For example, let’s say a store sells $520 worth of chocolate in a day. We know that on average, the store sells $420 per day and has a standard deviation of $50. To standardize the $520 we would do a calculation as follows:

520-420/50 = 100/50 = 2- our standardized value for this day is 2. If the sales were $600, we’d scale in a similar way as 600-420/50 = 180/50 = 3.6.

If all columns are done on a similar basis, we quickly have a great base for analytics that is consistent and allows us to quickly spot correlations.

In summary, data normalization processes ensure that our data is structured logically and scaled proportionally where required, generally on a scale of 0 to 1. It tends to be used where you have predefined assumptions of your model. Data standardization can be used where you are dealing with multiple variables together and need to find correlations and trends via a weighted ratio.

By ensuring you have normalized data, the likelihood of success in your machine learning and data science projects vastly improves. It is vital that organizations invest as much in ensuring the quality of their data as they do in the analytical and scientific models that are created by it. Preparation is everything in a successful data strategy and that’s what we mainly teach in our data science bootcamp courses.

Raja Iqbal, Chief Data Scientist and CEO of Data Science Dojo, held a community talk on AI for Social Good. Let’s look at some key takeaways.

This discussion took place on January 30th in Austin, Texas.  Below, you will find the event abstract and my key takeaways from the talk.I’ve also included the video at the bottom of the page.

Event abstract

“It’s not hard to see machine learning and artificial intelligence in nearly every app we use – from any website we visit, to any mobile device we carry, to any goods or services we use. Where there are commercial applications, data scientists are all over it. What we don’t typically see, however, is how AI could be used for social good to tackle real-world issues such as poverty, social and environmental sustainability, access to healthcare and basic needs, and more.

What if we pulled together a group of data scientists working on cutting-edge commercial apps and used their minds to solve some of the world’s most difficult social challenges? How much of a difference could one data scientist make let alone many?

In this discussion, Raja Iqbal, Chief Data Scientist and CEO of Data Science Dojo, will walk you through the different social applications of AI and how many real-world problems are begging to be solved by data scientists.  You will see how some organizations have made a start on tackling some of the biggest problems to date, the kinds of data and approaches they used, and the benefit these applications have had on thousands of people’s lives. You’ll learn where there’s untapped opportunity in using AI to make impactful change, sparking ideas for your next big project.”

1. We all have a social responsibility to build models that don’t hurt society or people

2. Data scientists don’t always work with commercial applications

• Criminal Justice – Can we build a model that predicts if a person will commit a crime in the future?
• Education – Machine Learning is being used to predict student churn at universities to identify potential dropouts and intervene before it happens.
• Personalized Care – Better diagnosis with personalized health care plans

3. You don’t always realize if you’re creating more harm than good.

“You always ask yourself whether you could do something, but you never asked yourself whether you should do something.”

4. We are still figuring out how to protect society from all the data being gathered by corporations.

5. There is not a better time for data analysis than today. APIs and SKs are easy to use. IT services and data storage are significantly cheaper than 20 years ago, and costs keep decreasing.

6. Laws/Ethics are still being considered for AI and data use. Individuals, researchers, and lawmakers are still trying to work out the kinks. Here are a few situations with legal and ethical dilemmas to consider:

• Granting parole using predictive models
• Detecting disease
• Military strikes
• Availability of data implying consent
• Self-driving car incidents

7. In each stage of data processing there are possible issues that arise. Everyone has inherent bias in their thinking process which effects the objectivity of data.

8. Modeler’s Hippocratic Oath

• I will remember that I didn’t make the world and it doesn’t satisfy my equations.
• Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
• I will never sacrifice reality for elegance without explaining why I have done so.
• I will not give the people who use my model false comfort about accuracy. Instead, I will make explicit its assumptions and oversights.
• I understand that my work may have an enormous impact on society and the economy, many of them beyond my comprehension.
• I will aim to show how my analysis makes life better or more efficient.

Highlights of AI for social good

Working within the Data Science industry has made me religiously follow a few data science blogs that I use to stay up to date with industry trends, learn new concepts, and understand the vernacular.

As a new member, these three things were originally hard for me to grasp until I started reading everything I could. These are the data science blogs I follow, and you should too.

Data science blogs I follow:

R-Bloggers

R-bloggers began when creator, Tal Galili, was fed-up with trying to find blogs about R. Instead of continuing his search, Tal created a site that pulls feeds from contributing blogs. R-bloggers “is a blog aggregator of content contributed by bloggers who write about R”. If your blog is all about R, you can create an RSS feed and contribute to the “R blogosphere”. This aggregator is a great place to find different blogs, especially if you’re new to the industry (like me).

Towards Data Science

Whether you enjoy data science as a hobby or a profession, you should be reading Towards Data Science (TDS). In October 2016 TDS joined Medium with the goal of “gathering good posts and distributing them to a broader audience”. Now, Towards Data Science includes 1,500 authors from around the world. TDS offers contributors an editorial team to help raise the quality of posts being submitted. While reading an article on TDS, you know you’re getting high-quality content you can trust.

KDNuggets

KDnuggets is another staple of data science blogs. The site has received so many impressive awards, I couldn’t decide which ones to list. You’ll have to settle with viewing them yourself.

It may seem messy when you first visit, but, much like original Reddit users, That’s the way I like it, and the 500,000 monthly visitors would probably agree. Posts range from courses and tutorials to news, meetings, and opinions. Like TDS, KDnuggets offers high-quality content you can trust to help you learn.

Entrepreneur

Entrepreneur is different than the three blogs above. Instead of focusing solely on anything within data science, it keeps its content specifically about how data science and big data affect entrepreneurship and small business. This blog is great for entrepreneurs and small business owners who want to absorb the concepts into their businesses. The market for using data science to make data-driven business decisions is growing and should not be overlooked.

DataFloq

One of my favorite things about DataFloq is how easy it is to navigate the site. It has a list of tags at the top of the Articles page that makes sorting through the posts very easy. It’s also easy to find events going on around the world.

The blog itself mostly focuses on big data, artificial intelligence, and new technologies. I usually find myself cruising through the AI or IoT tags.  There’s always a new article to read about one of those topics. You can also see how many views the article has received without having to click on it. I use it to gauge what the quality of the content’s like within the post. The higher the views, typically the higher the quality will be. If you’re looking for anything to do with new, emerging technologies, I suggest browsing DataFloq.

Dataconomy

I use Dataconomy almost strictly for learning about the trends within the blockchain. It isn’t updated as frequently as DataFloq, or the other above blogs, but it still gives helpful insights into what is trending within the data science industry.

Dataconomy prides itself in having a global network of contributors that don’t just look at the major tech companies. Authors are encouraged to find new and promising tech startups that will take the world by storm.

Who do you follow?

Is there a data science blog you think I have to read? Let me know! Follow the discussion link below to start a conversation. I’m always looking for new blogs to read to continue my data science education and learn new industry trends.

US-AI vs China-AI – What does the race for AI mean for data science worldwide? Why is it getting a lot of attention these days?

Although it may still be recovering from the effects of the government shutdown, data science has received a lot of positive attention from the United States Government. Two major recent milestones include the OPEN Government Data Act, which passed in January as part of the Foundations for Evidence-Based Policymaking Act, and the American AI Initiative, which was signed as an executive order on February 11th.

The future of data science and AI

The first thing to consider is why and more specifically the US administration has passed these recent measures. Although it’s not mentioned in either of the documents, any political correspondent who has been following these topics could easily explain that they are intended to stake a claim against China.

China has stated its intention to become the world leader in data science and AI by 2030. And with far more government access, data sets (a benefit of China being a surveillance state), and an estimated $15 billion in machine learning, they seem to be well on their way. In contrast, the US has only $1.1 billion budgeted annually for machine learning.

So rather than compete with the Chinese government directly, the US appears to have taken the approach of convincing the rest of the world to follow their lead, and not China’s. They especially want to direct this message to the top data science companies and researchers in the world (especially Google) to keep their interest in American projects.

So, what do these measures do?

On the surface, both the OPEN Government Data Act and the American AI Initiative strongly encourage government agencies to amp up their data science efforts. The former is somewhat self-explanatory in name, as it requires agencies to publish more machine-readable publicly available data and requires more use of this data in improved decision making. It imposes a few minimal standards for this and also establishes the position of Chief Data Officers at federal agencies. The latter is somewhat similar in that it orders government agencies to re-evaluate and designate more of their existing time and budgets towards AI use and development, also for better decision making.

Critics are quick to point out that the American AI Initiative does not allocate more resources for its intended purpose, nor does either measure directly impose incentives or penalties. This is not much of a surprise given the general trend of cuts to science funding under the Trump administration. Thus, the likelihood that government agencies will follow through with what these laws ‘require’ has been given skeptical estimations.

However, this is where it becomes important to remember the overall strategy of the current US administration. Both documents include copious amounts of values and standards that the US wants to uphold when it comes to data, machine learning, and artificial intelligence. These may be the key aspects that can hold up against China, having a government that receives a hefty share of international criticism for its use of surveillance and censorship. (Again, this has been a major sticking point for companies like Google.)

These are some of the major priorities brought forth in both measures: Make federal resources, especially data and algorithms, available to all data scientists and researchers; Prepare the workforce for technology changes like AI and optimization; Work internationally towards AI goals while maintaining American values; and finally, Create regulatory standards, to protect security and civil liberties in the use of data science.

So there you have it. Both countries are undeniably powerhouses for data science. China may have the numbers in its favor, but the US would like the world to know that they have an American spirit.

Not working for both? –  US-AI vs China-AI

In short, the phrase “a rising tide lifts all ships” seems to fit here. While the US and China compete for data science dominance at the government level, everyone else can stand atop this growing body of innovations and make their own.

The thing data scientists can get excited about in the short term is the release of a lot of new data from US federal sources or the re-release of such data in machine-readable formats. The emphasis is on the public part – meaning that anyone, not just US federal employees or even citizens, can use this data. To briefly explain for those less experienced in the realm of machine learning and AI, having as much data to work with as possible helps scientists to train and test programs for more accurate predictions.

A lot of what made the government shutdown a dark period for data scientists suggests the possibility of a golden age shortly.

R and Python remain the most popular data science programming languages. But if we compare r vs python, which of these languages is better?

As data science becomes more and more applicable across every industry sector, you might wonder which programming language is best for implementing your models and analysis. If you attend a data science Bootcamp, Meetup, or conference, chances are you’ll run into people who use one of these languages.

Since R and Python remain the most popular languages for data science, according to  IEEE Spectrum’s latest rankings, it seems reasonable to debate which one is better. Although it’s suggested to use the language you are most comfortable with and one that suits the needs of your organization, for this article, we will evaluate the two languages. We will compare R and Python in four key categories: Data Visualization, Modelling Libraries, Ease of Learning, and Community Support.

Data visualization

A significant part of data science is communication. Most of the time, you as a data scientist need to show your result to colleagues with little or no background in mathematics or statistics. So being able to illustrate your results in an impactful and intelligible manner is very important. Any language or software package for data science should have good data visualization tools.

Good data visualization involves clarity. No matter how complicated your model is, there will be a simple and unambiguous way of illustrating your results such that even a layperson would understand.

Python

Python is renowned for its extensive number of libraries. There are plenty of libraries that can be used for plotting and visualizations. The most popular libraries are matplotlib and  seaborn. The library matplotlib is adapted from MATLAB, it has similar features and styles. The library is a very powerful visualization tool with all kinds of functionality built in. It can be used to make simple plots very easily, especially as it works well with other Python data science libraries, pandas and numpy.

Although matplotlib can make a whole host of graphs and plots, what it lacks is simplicity. The most troublesome aspect is adjusting the size of the plot: if you have a lot of variables it can get hectic trying to neatly fit them all into one plot. Another big problem is creating subplots; again, adjusting them all in one figure can get complicated.

Now, seaborn builds on top of matplotlib, including more aesthetic graphs and plots. The library is surely an improvement on matplotlib’s archaic style, but it still has the same fundamental problem: creating figures can be very complicated. However, recent developments have tried to make things simpler.

R

Many libraries could be used for data visualization in R but ggplot2 is the clear winner in terms of usage and popularity? The library uses a grammar of graphics philosophy, with layers used to draw objects on plots. Layers are often interconnected to each other and can share many common features. These layers allow one to create very sophisticated plots with very few lines of code. The library allows the plotting of summary functions. Thus, ggplot2 is more elegant than matplotlib and thus I feel that in this department R has an edge.

It is, however, worth noting that Python includes a ggplot library, based on similar functionality as the original ggplot2 in R. It is for this reason that R and Python both are on par with each other in this department.

Modelling libraries

Data science requires the use of many algorithms. These sophisticated mathematical methods require robust computation. It is rarely or maybe never the case that you as a data scientist need to code the whole algorithm on your own. Since that is incredibly inefficient and sometimes very hard to do so, data scientists need languages with built-in modelling support. One of the biggest reasons why Python and R get so much traction in the data science space is because of the models you can easily build with them.

Python

As mentioned earlier Python has a very large number of libraries. So naturally, it comes as no surprise that Python has an ample amount of machine learning libraries. There is scikit-learn, XGboost, TensorFlow, Keras and PyTorch just to name a few. Python also has pandas, which allows tabular forms of data. The library pandas make it very easy to manipulate CSVs or Excel-based data.

In addition to this Python has great scientific packages like numpy. Using numpy, you can do complicated mathematical calculations like matrix operations in an instant. All of these packages combined, make Python a powerhouse suited for hardcore modelling.

R

R was developed by statisticians and scientists to perform statistical analysis way before that was such a hot topic. As one would expect from a language made by scientists, one can build a plethora of models using R. Just like Python, R has plenty of libraries — approximately 10000 of them. The mice package, rpart, party and caret are the most widely used. These packages will have your back, starting from the pre-modelling phase to the post-model/optimization phase.

Since you can use these libraries to solve almost any sort of problem; for this discussion let’s just look at what you can’t model. Python is lacking in statistical non-linear regression (beyond simple curve fitting) and mixed-effects models. Some would argue that these are not major barriers or can simply be circumvented. True! But when the competition is stiff you have to be nitpicky to decide which is better. R, on the other hand, lacks the speed that Python provides, which can be useful when you have large amounts of data (big data).

Ease of learning

It’s no secret that currently data science is one of the most in-demand jobs, if not the one most in demand. As a consequence, many people are looking to get on the data science bandwagon, and many of them have little or no programming experience. Learning a new language can be challenging, especially if it is your first. For this reason, it is appropriate to include ease of learning as a metric when comparing the two languages.

Python

Designed in 1989 with a philosophy that emphasizes code readability and a vision to make programming easy or simple, the designers of Python succeeded as the language is fairly easy to learn. Although Python takes inspiration for its syntax from C, unlike C it is uncomplicated. I recommend it as my choice of language for beginners since anyone can pick it up in relatively less time.

R

I wouldn’t say that R is a difficult language to learn. It is quite the contrary, as it is simpler than many languages like C++ or JavaScript. Like Python, much of R’s syntax is based on C, but unlike Python R was not envisioned as a language that anyone could learn and use, as it was specifically initially designed for statisticians and scientists. IDEs such as RStudio have made R significantly more accessible, but in comparison with Python, R is a relatively more difficult language to learn.

In this category Python is the clear winner. However, it must be noted that programming languages in general are not hard to learn. If a beginner wanted to learn R, it won’t be as easy in my opinion as learning Python but it won’t be an impossible task either.

Community support

Every so often as a data scientist you are required to solve problems that you haven’t encountered before. Sometimes you may have difficulty finding the relevant library or package that could help you solve your problem. To find a solution, it is not uncommon for people to search in the language’s official documentation or online community forums. Having good community support can help programmers, in general, to work more efficiently.

Both of these languages have active Stack overflow members and also an active mailing list available (where one can easily ask for solutions from experts). R has online R-documentation where you can find information about certain functions and function inputs. Most Python libraries like pandas and scikit-learn have their official online documentation that explains each library.

Both languages have a significant amount of user base, hence, they both have a very active support community. It isn’t difficult to see that both seem to be equal in this regard.

Why R?

R has been used for statistical computing for over two decades now. You can get started with writing useful code in no time. It has been used extensively by data scientists and has an insane number of packages available for a lot of data science-related tasks. I have almost always been able to find a package in R to get the task done very quickly. I have decent python skills and have written production code in python. Even with that, I find R slightly better for quickly testing out ideas, trying out different ways to visualize data and for rapid prototyping work.

Why Python?

Python has many advantages over R in certain situations. Python is a general-purpose programming language. Python has libraries like pandas, NumPy, scipy and sci-kit-learn, to name a few which can come in handy for doing data science-related work.

If you get to the point where you have to showcase your data science work, Python once would be a clear winner. Python combined with Django is an awesome web application framework, which can help you create a web service/site with both your data science and web programming done in the same language.

You may hear some speed and efficiency arguments from both camps – ignore them for now. If you get to a point when you are doing something substantial enough where the speed of your code matters to you, you will probably figure out things on your own. So don’t worry about it at this point.

You can learn Python for data science with Data Science Dojo!

R and Python – The most popular languages

Considering that you are a beginner in both data science and programming and that you have a background in Economics and Statistics, I would lean towards R. Besides being very powerful, Python is without a doubt one of the friendliest programming languages to beginners – but it is still a programming language. Your learning curve may be a bit steeper in Python as opposed to R.

You should learn Python, once you are comfortable with R, and have grasped the general concepts of data science – which will take some time. You can read “What are the key skills of a data scientist? To get an idea of the skill set you will need to become a data scientist.

Start with R, transition to Python gradually and then start using both as needed. Both are great for data science but one is better than the other in certain situations.