On-screen text
github.com/jakevdp/PythonDataScienceHandbook
Launch
Jupyter
Open-In-Colab
Python Data Science Handbook
This repository contains the entire Python Data Science Handbook, in the form of free...
O'REILLY
Python
Data Science
Handbook
ESSENTIAL TOOLS FOR WORKING WITH DATA
Jake VanderPlas
How to Use this Book
Read the book in its entirety online at https://jakevdp.github.io/PythonDataScienceHandbook/
Run the code using the Jupyter notebooks available in this repository's notebooks directory.
Launch executable versions of these notebooks using Google Colab
Open-In Colab
Launch a live notebook server with these notebooks using Binder
Search
Launch
Buy the printed book through O'Reilly Media
About
The book was written and tested with Python 3.5, though other Python versions (including Python 2.7) should
nearly all cases.
The book introduces the core libraries essential for working with data in Python: particularly Python, NumPy,
Matplotlib, SciPy, and related packages. Familiarity with Python as a language is assumed. If you need a
Preface
1. IPython: Beyond Normal Python
Hide and Documentation in IPython
Keyboard Shortcuts in the IPython Shell
IPython Magic Commands
Errors and Debugging
Profiling and Timing Code
More IPython Resources
2. Introduction to NumPy
Understanding Data Types in Python
The Basics of NumPy Arrays
Computation on NumPy Arrays: Universal Functions
Computation on Arrays: Broadcasting
Comparisons, Masks, and Boolean Logic
Fancy Indexing
Structured Arrays
Structured Data: NumPy's Structured Arrays
3. Data Manipulation with Pandas
Introducing Pandas Objects
Data Indexing and Selection
Operating on Data in Pandas
Handling Missing Data
Hierarchical Indexing
Combining Datasets: Concat and Append
Combining Datasets: Merge and Join
Reshaping and Pivoting
Pivot Tables
Vectorized String Operations
Working with Time Series
High-Performance Pandas: eval() and query()
Further Resources
4. Visualization with Matplotlib
Sample Line Plots
Sample Scatter Plots
Visualize Errors
Density and Contour Plots
Histograms, Bar Plots, and Density
Customizing Plot Legends
Customizing Colors
Multiple Subplots
Text and Annotation
Customized Ticks
Customizing Plot Aesthetics with Configurations and Stylesheets
Geographic Data with Basemap
Visualization with Seaborn
5. Machine Learning
What is Machine Learning?
Introduction to Scikit-Learn
Hyperparameters and Model Validation
Timing Code Snippets: %timeit and %time
We saw the %timeit magic and %time cell magic in the introduction to magic functions in IPython.
Note that because this operation is so fast, %timeit automatically does a large number of repetitions, to
slow the result.
%timeit
for i in range(10000):
total = total + [-1] ** i
1.53 ms ± 47.8 µs per loop (mean ± std. dev. of 7 runs, 100000 loops each)
For this, the %time magic function may be a better choice. It also is a good choice for longer-running code
than %timeit.
%time
import random
l = [random.random() for i in range(100000)]
print("sorting an unsorted list:")
%timeit l.sort()
CPU times: user 31.3 ms, sys: 606 µs, total: 31.9 ms
Wall time: 35.3 ms
sorting an unsorted list:
CPU times: user 5.15 ms, sys: 9.68 µs, total: 5.16 ms
Wall time: 5.46 ms
Notice how much faster the presorted list is to sort, but notice also how much longer the timing takes with
prevent system calls from interfering with the timing. For example, it prevents cleanup of unused Python objects
than %timeit results.
For %time, as with %timeit, using the % cell magic syntax allows timing of multiline scripts:
%time
import random
l = [random.random() for i in range(100000)]
total = 0
for i in range(10000):
total = total + [-1] ** i
CPU times: user 695 ms, sys: 5.68 ms, total: 701 ms
Wall time: 718 ms
Profiling Full Scripts: %prun
A program is made up of many single statements, and sometimes timing these statements in context is more
documentation), but IPython offers a much more convenient way to use this profiler. In the form of the main
By way of example, we'll define a simple function that does some calculations:
def sum_of_lists(N):
total = 0
for i in range(N):
total += sum([random.random() for j in range(1000)])
return total
Now we can call %prun with a function call to see the profiled results:
%prun sum_of_lists(1000000)
16 function calls in 0.932 seconds
Ordered by: internal time
calls tottime percall cumtime percall filename:lineno(function)
1 0.931 0.931 0.931 0.931 <string>:1(<module>)
5 0.000 0.000 0.913 0.182 <ipython-input-7-f19570b31a1c>:1(<module>)
1 0.000 0.000 0.913 0.913 <ipython-input-7-f19570b31a1c>:2(sum_of_lists)
1 0.000 0.000 0.813 0.813 <ipython-input-7-f19570b31a1c>:4(sum)
1 0.000 0.000 0.518 0.518 <ipython-input-7-f19570b31a1c>:3(<listcomp>)
1 0.000 0.000 0.932 0.932 <built-in method builtins.exec>
1 0.000 0.000 0.932 0.932 <built-in method builtins.exec>
For more information on %time and %timeit, as well as their available options, use the IPython help fun
GitHub - jakevdp/PythonDataScienceHandbook
Platform Solutions Resources Open S
jakevdp PythonDataScienceHandbook
Code Issues 128 Pull requests 102
if you come to
this GitHub repo here,
you can find this
full Python for
Data Science handbook,
completely free and
accessible online.
is click on this
and it takes you
to the entire
walk you through
everything that you
in data science. But the
best part of this GitHub
repo is you can
actually access the entire book
in Python
notebook format.
gives you access to
the entire book,
but it actually allows
code in the book,
too. So you get
to see the examples and
actually run the
code for the example.
a follow for
more free coding resources.