An Introduction To Python For Microsoft Azure Data Scientist | DP-100

Data science is hard. You’ll have to learn a handful of libraries as a beginner, even to solve the most fundamental tasks. In this blog, we go through the concept of Python For Azure Data Scientist.

The goal of a good Data Scientist is to produce a strong understanding of a problem or idea and to build useful ML models based on that understanding. Developed as a powerful and flexible language used in everything from Data scientists to cutting-edge Artificial Intelligence solutions, Python has become an essential tool for doing data science and machine learning.

Topics that we’ll cover in this blog:

1. Introduction to Python
2. Python for Azure Data Scientist
3. What is the Azure Machine Learning SDK for Python

So, let’s jump into this post right away.

What Is Python?

Python is a versatile coding language that can be used for back-end development, software development, and data science among other areas. Because this programming language is accessible, portable, and can be run on Mac, Windows, or Unix, Python has become increasingly popular in several companies and universities. In fact, a 2019 Kaggle Machine Learning and Data Science Survey report that 87% of data scientists use Python regularly.

1) Python Features:

Here are few Features of python:

• Code readability, shorter codes, ease of writing
• Simple syntax
• GUI Programming Support
• Dynamically typed language (Data type is based on the value assigned)
• Extensive support libraries (Django for web development, Pandas for data analytics, etc)
• Large scale libraries
• Free and Open Source

2) Python Application:

Python is used in many application domains. Here are few applications:

1. Web Development
2. Desktop GUI
3. Audio and Video Applications
4. Data Science, Machine Learning and AI

Python For Azure Data Scientist

Python is the most popular programming language for data scientists. Data scientists can use various tools and techniques to explore, visualize, and manipulate data. With Azure Machine Learning you get a fully configured and managed development environment in the cloud.

Libraries:

Python provides extensive functionality with powerful libraries:

• NumPy and Pandas simplify analyzing and manipulating data
• Matplotlib provides attractive data visualizations
• Scikit-learn offers simple and effective predictive data analysis
• TensorFlow and PyTorch supply machine learning and deep learning capabilities

For example- suppose a university professor collects data from their students, including the number of lectures attended, the hours spent studying, and the final grade achieved on the end-of-term exam. The professor could analyze the data to determine if there is a relationship between the amount of studying a student undertakes and the final grade they achieve. The professor might use the data to test a hypothesis that only students who study for a minimum number of hours can expect to achieve a passing grade.

1) What Is NumPy?

NumPy stands for Numerical Python. It is the basic library used for numerical and scientific computations, It is mainly used for data analysis. It provides a high-performance multidimensional array of objects and tools for working with their arrays. It is open-source you can use it freely.

Why Use NumPy?

NumPy arrays are faster and more compact than Python lists. An array consumes less memory and is convenient to use. NumPy uses much less memory to store data and it provides a mechanism of specifying the data types. The array object in NumPy is called ndarray

To install the NumPy library we use

pip install numpy

Use the following import convention:

import numpy as np

Creating a Numpy Array:

>>> import numpy as np
>>> arr = np.array([])
>>> type(arr)
numpy.ndarray

2) What Is Pandas?

It is used for data clearing, data storage, and time series. Pandas is like excel for Python – providing easy-to-use functionality for data tables.

Why Use Pandas?

Data scientists make use of Pandas in Python for its following advantages:

• Easily handles missing data.
• It uses Series for one-dimensional data structure and DataFrame for multi-dimensional data structure
• It provides an efficient way to slice the data.
• It includes a powerful time-series tool to work with real data sets and more.

To install the Pandas library we use

pip install pandas

Use the following import convention:

import pandas as pd

Creating a Series:

>>> pd.Series([1,2,3,4,5])
0    1
1    2
2    3
3    4
4    5
dtype: int64

Fun fact: The container that a Pandas data object sits on top of a NumPy array.

Note: Just as we use the np shorthand for NumPy and the pd shorthand for Pandas.

3) What Is Matplotlib?

It is the data visualization library used to analyze correction, determine outliers using scatter plots, and visualizes data distribution.

Why Use Matplotlib?

To install the Matplotlib library we use

pip install matplotlib

Single Attribute Distribution with Matplotlib:

import matplotlib.pyplot as plt
%matplotlib inline

dataset = pd.read_csv("../dataset/student_result.csv")

# shows the result distribution
# result attribute contains two types of value. 
# 1 indicates `pass` and `0` indicates `fail`
dataset.result.value_counts().plot.bar()

Output:

4) TensorFlow

It is a high-performance computation that reduces error by 50%. It is used for speed, image section, video detection, real-time series.

TensorFlow is the primary software tool of deep learning. It is an open-source artificial intelligence library, using data flow graphs to build models. TensorFlow is mainly used for: Classification, Perception, Understanding, Discovering, Prediction and Creation.

Why TensorFlow

TensorFlow is an end-to-end platform that makes it easy for you to build and deploy ML models and solve real-world problems with machine learning.

Use Cases of TensorFlow:

• Text-Based Applications
• Image Recognition
• Real-time dataset or Time Series
• Video Detection
• Voice/Sound Recognition and more…

Initialize the installation of TensorFlow −

create --name tensorflow python = 3.5
activate tensorflow

To install the TensorFlow library we use

pip install tensorflow
And,
pip install tensorflow-gpu

Q. What are Tensors?

Ans. Tensor is a generalization of vectors and matrices of potentially higher dimensions. Arrays of data with varying dimensions and ranks that are fed as input to the neural network are called tensors.

5) Scikit-Learn

A library that provides a range of Supervised and Unsupervised Learning Algorithms. This library mainly focused on model building. Built on NumPy, SciPy, and matplotlib.

The functionality that scikit-learn provides include:

1. Classification category an object belongs to.

• • Applications: Spam detection, image recognition.

2. Regression Predicting a continuous-valued attribute associated with an object.

• • Applications: Drug response, Stock prices.

3. Clustering Automatic grouping of similar objects into sets.

• • Applications: Customer segmentation, Grouping experiment outcomes

4. Model selection Comparing, validating, and choosing parameters and models.

• • Applications: Improved accuracy via parameter tuning

To install the Scikit-learn library we use

pip install -U scikit-learn
from sklearn import linear_model

6) PyTorch

PyTorch is an open-source machine learning library for Python and is completely based on Torch, which uses the power of graphics processing units, natural language processing. PyTorch is the fastest-growing Deep Learning framework and it is also used by Fast.ai in its MOOC, Deep Learning for Coders, and its library. PyTorch is also very pythonic, meaning, it feels more natural to use it if you already are a Python developer.

PyTorch has two main features:

• Tensor computation (like NumPy) with strong GPU acceleration
• Automatic differentiation for building and training neural networks

Install PyTorch by running this simple command:

pip install PyTorch torchvision -c PyTorch
import torch

What is the Azure Machine Learning SDK for Python?

Data scientists and AI developers use the Azure Machine Learning SDK for Python to build and run machine learning workflows with Azure Machine Learning. We can interact with the service in any Python environment, including Jupyter Notebooks, Visual Studio Code, or any Python IDE.

The Azure Machine Learning SDK for Python provides both stable and experimental features in the same SDK.

Main capabilities of the SDK include:

• Manage cloud resources for monitoring, logging, and organizing your machine learning experiments.
• Train models using cloud resources, including GPU-accelerated model training.
• Deploy your models as web services on Azure Container Instances (ACI) and Azure Kubernetes Service (AKS)

Key Features:

• Workspace – A machine learning workspace is a top-level resource for Azure Machine Learning. The workspace class is a foundational resource in the cloud that you use to experiment, train, and deploy machine learning models.

Source: Microsoft

Store assets you create when you use Azure Machine Learning, including:

• Environments – An Azure Machine Learning Environment allows you to create, manage, and reuse the software dependencies required for training and deployment. Environments specify the R packages, environment variables, and software settings around your training and scoring scripts for your containerized training runs and deployments. Azure ML workspace that enables reproducible, auditable and portable machine learning workflows across different compute targets.
• Experiments – Experiments, including run history with logged metrics and outputs.
• Pipelines – Pipelines that define orchestrated multi-step processes.
• Datasets – An Azure Machine Learning Dataset allows you to explore, transform, and manage your data for various scenarios such as model training and pipeline creation. When you are ready to use the data for training, you can save the Dataset to your Azure ML workspace to get versioning and reproducibility capabilities.
• Models – Cloud representations of machine learning models that help you transfer models between local development environments and the workspace object in the cloud. Models that you have trained.
• Endpoints – An endpoint is an instantiation of your model into a web service that can be hosted in the cloud.

Also check: Azure Machine Learning works: Architecture and concepts

To Install Azure ML SDK package:

install_azureml

Related/References:

• [DP-100] Microsoft Certified Azure Data Scientist Associate: Everything you must know
• Introduction to Using Python with Microsoft Azure
• Microsoft Certified Azure Data Scientist Associate | DP 100 | Step By Step Activity Guides (Hands-On Labs)

Next Task for You

To know more about AI, ML, Data Science for beginners, why you should learn, Job opportunities, and what to study Including Hands-On labs you must perform to clear [DP-100] Microsoft Azure Data Scientist Associate Certification register for our FREE CLASS.

The post An Introduction To Python For Microsoft Azure Data Scientist | DP-100 appeared first on Cloud Training Program.