python Numpy, scipy and matplotlib:- In this article we will introduce you to modules that Python can use to create a numerical solutions of math problems can be used. The Opportunities are comparable to environments like MATLAB or Scilab. With the help of the modules numpy and scipy presented here, for example Solve equations and optimization problems, calculate integrals, statistical Perform calculations or simulate differential equations. The results can be visualized with the matplotlib module. The calculations are carried out internally by hardware-related routines, so that efficient programs can be written with skillful programming.
This article will not give you an exhaustive explanation of the many Skills from python numpy, scipy and matplotlib are presented but you will be based on them a simple example introduced to the basic handling of the modules. In addition, at the end of the article you will find an overview of their structural Structure of scipy as well as further examples for the use of the data type numpy.ndarray.
The aim of this article is to give you an overview of whether python numpy, scipy and matplotlib are useful for your specific programs. After reading the given here Introduction you will then be able to refer to the online documentation in a targeted manner. The following table gives a brief description of the three modules.
|numpy||Provides a flexible data type for multidimensional arrays Available, with the help of which extensive invoices are efficient can be carried out.|
|scipy||Implemented a variety of math based on python numpy Operations.|
|matplotlib||Visualizes data graphically. It is the declared goal of the developers to generate high-quality graphics, for example in have scientific publications used.|
Table 1: The modules for numerical calculations and visualization
The modules are not included in the standard Python library and must be must therefore be installed separately. You can find out how this works in the following installation Section.
The latest versions of the python numpy and scipy can be found on the official Homepage at
http://www.scipy.org/Download/ or on the associated download Pages from Sourceforge:
Both the source code and precompiled packages for Windows and Mac OSX offered. If you work on Linux, there is a high probability that there are pre-built ones Packages for your distribution containing numpy and scipy. We encourage you to access these packages.
Visit http://matplotlib.sourceforge.net/ to find the latest version of matplotlib
model program of python numpy, spicpy and matplotlib:
In the following we consider a simple model program that leads to a function f (x) = x3 – 10sin (x) – 4 the derivative and an anti-derivative calculated numerically and graphically in the interval [-3, 3]. Using this model program will be typical procedures for dealing with python numpy, scipy and matplotlib are explained.
import numpy as np import scipy as sp import matplotlib as mpl import matplotlib.pyplot as plt import scipy.misc import scipy.integrate def f(x): return x**3 - 10*np.sin(x) - 4 def df(x): return sp.misc.derivative(f, x) @np.vectorize def F(x): return sp.integrate.quad(f, 0, x) X = np.linspace(-3, 3, 100) Y = f(X) Y1 = df(X) Y2 = F(X) plt.plot(X, Y, linewidth=2, label="f") plt.plot(X, Y1, linewidth=2, linestyle="dashed", label="f'") plt.plot(X, Y2, linewidth=2, linestyle="dotted", label="F") plt.legend() plt.show()
This simple example program already shows the most important features that one should pay attention to when working with python numpy, scipy and matplotlib. We will Now discuss the program piece by piece.
Explanation of model program:
import of numpy, scipy and matplotlib:
In the first four lines the modules numpy, scipy and matplotlib are imported and provided with short names.
import numpy as np import scipy as sp import matplotlib as mpl import matplotlib.pyplot as plt
This way of importing the modules has proven itself in practice and was therefore introduced as a convention. We are getting into the examples in this article assume that the modules were imported in this way and recommend You to follow the convention in your own programs as well. Following the basic import instructions, there are additional the modules scipy.misc and scipy.integrate are loaded. Since scipy is very extensive, it does not make sense that every program has all functions imported at once. Because of this, scipy is divided into several sub-modules subdivided, which must be integrated if necessary. In our example we use the function derivative from the sub-module scipy.misc for the calculation the derivation. We determine the antiderivative from the using the function quad Module scipy.integrate.
Vectorization and the data type of python numpy.ndarray:
After the import statements, three functions f, df and F are defined, which the Evaluate function f, its derivative f ‘and an antiderivative F. To work with For numpy and scipy it is important that the functions f, df and F are not just for individual Numbers, but work for whole arrays of numbers. An array is one ordered sequence of similar objects, usually numbers. Before we get on Let’s briefly discuss the interaction of arrays and functions basic handling of arrays.
To create an array, the function numpy.array can be used, which you pass a list with the desired elements of the array.
>>> a = np.array([0.5, 1, 2, 3]) >>> a array([ 0.5, 1. , 2. , 3. ])
The special thing about instances of the data type numpy.ndarray is that with them like with Numbers can be calculated. The operations are carried out element by element.
>>> a+a array([ 1., 2., 4., 6.]) >>> a*a array([ 0.25, 1. , 4. , 9. ]) >>> a**3 array([ 0.125, 1. , 8. , 27. ]) >>> 4*a array([ 2., 4., 8., 12.])
In addition to arithmetic operations, functions can also be applied to arrays by inserting each element of the array into the function and the Results are summarized as a new array.
For example, we can apply the function f from our example program to the Apply array a.
>>> f(a) array([ -8.66925539, -11.41470985, -5.09297427, 21.58879992])
Here the values f (0.5), f (1), f (2) and f (3) are calculated and stored in a new array and returned.
This principle of applying the same operation to an entire array of numbers is called vectorization. When a function processes arrays in addition to numbers can, it is referred to as vectorized.
The functions of the math module of the Python standard library such as sin,cos, tan or exp are not vectorized, which is why python numpy and scipy have their own vectorized Bring versions of these features. Use in our example we therefore use the function of the python numpy.sin instead of math.sin, which is able to use for an array to calculate the sine values of numbers.
With a knowledge of arrays and vectorization, we can move on to the next two lines understand the model program.
X = np.linspace(-3, 3, 100) Y = f(X)
First, the function numpy.linspace generates a new array X to which then the function f is applied element by element. The calculated function values are stored in the Y array. Numpy.linspace can be used to create an array with a specific number of elements which are all equally spaced. About the first two Numpy.linspace parameters become the largest and smallest element of the array set. For example, the following call creates an array with 5 elements in the interval from -1 to 1.
>>> np.linspace(-1,1,5) array([-1. , -0.5, 0. , 0.5, 1. ])
In our model program, an array is generated that contains 100 numbers from the Contains the interval from -3 to 3 and for each of these numbers the corresponding value of the Function f calculated. The following line Y1 = df (X) then determines the derivative of f at the locations that are stored in the X array. Use when defining df we assume that the function scipy.misc.derivative is vectorized and therefore also with an array as a parameter.
Vectorize non-vectorized functions using python numpy.vectorize:
If a function like above f or df is only composed of vectorized operations is, the result is automatically vectorized again. There are, however also cases in which vectorization is not guaranteed automatically. Instead of worrying about how best to vectorize a function of python numpy offers a practical tool for this. As an example, consider a function named clip_positive that takes the value should return a number if it is positive and 0 if the number is negative Has value. The following listing shows a possible implementation of this function.
>>> def clip_positive(x): if x > 0: return x else: return 0 >>> clip_positive(10) 10 >>> clip_positive(-5) 0 >>> clip_positive(2) 2
We also want to pass arrays to this function as the value for x, so a new An array is created that contains the entry itself for positive entries and the entry itself for negative Entries contains the value 0. The way the function is implemented now works however, they do not apply to arrays, since the case distinction in this form does not apply Arrays can be transferred.
The remedy is the function decorator numpy.vectorize, which creates a function like this adapts so that it accepts arrays as parameters in addition to individual numbers.
>>> @np.vectorize ... def clip_positive(x): ... if x > 0: return x ... else: return 0 >>> clip_positive(10) array(10) >>> clip_positive(-5) array(0.0) >>> b = np.array([4, -3, 0.7, -10, 8]) >>> clip_positive(b) array([ 4. , 0. , 0.7, 0. , 8. ])
As you can see, the clip_positive function now works with both Numbers as well as whole arrays as parameters. However, now there will be a Array generated if a number was passed for the parameter x. This is why no problem, since arrays of length 1 behave like numbers and therefore also in others Calculations as such can be used.
With this knowledge we can implement the function F in our Understand the model program. The following listing repeats the definition from the Program.
@np.vectorize def F(x): return sp.integrate.quad(f, 0, x)
The function scipy.integrate.quad expects the one to be integrated as the first parameter
Function. The integration limits are set with the following two parameters specified. The return value of scipy.integrate.quad is a tuple,consisting of two elements, the first entry being an approximation of the integral and the second entry represents an upper bound for the approximation error.
>>> sp.integrate.quad(f, 0, 4) (31.463563791363885, 7.668063971742951e-13)
The function F thus provides for an x the integral over f from 0 to x, that is
It is one according to the main law of differential and integral calculus Anti-derivative of f. The function decorator numpy.vectorize creates the Function for using arrays as parameter values extended. The line Y2 = F (X) of the model program determines the values of an anti-derivative of f at the locations that are stored in the X array.
Visualize data with matplotlib.pyplot:
Now that the desired data has been calculated, we would like it graphically represent. We use the matplotlib.pyplot module, which is connected to the drawing interface is based on MATLAB.
You can think of this interface as a blank sheet of paper to which Calls of functions from the module matplotlib.pyplot new graphic elements to be added. When the drawing is finished, the result is with the function matplotlib.pyplot.show will appear on the screen.
The most important function is matplotlib.pyplot.plot, with which any data visualize. In the simplest case, you pass a list to the plot function of numbers.
>>> plt.plot([1, 3, 5, 2, 1, 4, 6, 7, 3]) [<matplotlib.lines.Line2D object at 0xb813acc>] >>> plt.show()
These numbers are now interpreted as function values, whereby the indices of the list, i.e. 0, 1, 2, …, 8, are interpreted as associated values on the x-axis. Between the specified function values are linearly interpolated so that the individual Data points are connected by straight lines. Figure 2 illustrates the result.
The toolbar at the top of the window offers a number of operations, to change the graphic. In addition, the graphic can be in various graphic formats
(e.g. PNG, PS, EPS, PDF, SVG).
Usually you want the values on the x-axis that lead to the function values must be specified explicitly and not automatically determined via the indices. To do this, pass two lists or arrays to the plot function, with the first parameter the x-values and the second parameter contains the associated y-values.
For example, to draw a period of the sine function, we generate with numpy.linspace first an array containing data points in the interval [0,2], and then apply the numpy.sin function to it. Give the results we then continue to plot.
>>> X = np.linspace(0, 2*np.pi, 200) >>> Y = np.sin(X) >>> plt.plot(X, Y) [<matplotlib.lines.Line2D object at 0xb819eac>] >> plt.show()
If you want to display several functions in one graph, you can plot several times before the result is displayed using show, as in our Model program is made.
The appearance of the graphic can be adjusted using additional parameters of the plot function become. The following listing shows the commands used in our model program the graphic output is generated. We use the parameter for this linewidth to adjust the thickness of the lines, and change that via linestyle Appearance of the lines. The label parameter can be used to add a character string to each plot which is shown in the legend of the plot. Calling the function legend ensures that the legend becomes visible.
plt.plot(X, Y, linewidth=2, label="f") plt.plot(X, Y1, linewidth=2, linestyle="dashed", label="f'") plt.plot(X, Y2, linewidth=2, linestyle="dotted", label="F") plt.legend() plt.show()
The matplotlib module is able to create complex 2D and 3D visualizations produce.
NumPy and scipy modules:
This section provides information on practical work with the data type numpy.ndarray. In addition, an overview table of the sub-modules of scipy presents.
numpy.ndarray data type:
The heart of numpy and scipy is the data type numpy.ndarray, the multidimensional one Can manage arrays. We will therefore give you the basics in this article for dealing with numpy.ndarray convey. This introduction is the first Getting started and does not cover all the possibilities that numpy.ndarray offers you.
shape and data type of an array:
An array is a combination of several elements of the same type to form a unit, where the elements can be accessed via integer indices. We in the examples in this article we will refer to on and on for reasons of clarity constrain two-dimensional arrays.
Each array has a shape and a data type. The shape defines the dimensions of the array, and the data type indicates what kind of data the elements of the Can store arrays. Both the form and the data type are used when creating of the array and can no longer be changed afterwards. The following table lists is the main data types available for elements of numpy.ndarray instances.
Table 2: the most important data types of numpy.
|Name||Description of the possible values|
|numpy.bool||Boolean values (True or False), each internally defined as a Bytes are stored|
|numpy.int||whole numbers, depending on the platform, 32 or 64 bits for Storage can be used|
|numpy.float||Double precision floating point numbers, for the 64 bits Storage space can be used|
|numpy.complex||complex numbers, with both real and imaginary parts as 64-bit floating point number|
In addition to these basic types, there are a number of other types that behave in the same way like the basic data types, but occupy a different amount of memory. So For example, int8 denotes a type that can store 8-bit integers. When creating arrays, you should always make sure they are of the correct type otherwise inaccuracies may occur if the selected Data type cannot represent the range of values with which you want to work.
Creating arrays and basic properties:
In the model program of the last section, one-dimensional arrays using the functions numpy.array and numpy.linspace, it creates the following Example shows.
>>> np.array([-1, 4, -5, 7]) array([-1, 4, -5, 7]) >>> np.linspace(0, 1, 11) array([ 0. , 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1. ])
Each array has an attribute dtype that stores the data type of the array.
>>> x = np.array([-1, 4, -5, 7]) >>> x.dtype dtype('int32') >>> y = np.array([-1, 4, -5.2, 7]) >>> y.dtype dtype('float64')
As you can see, the data type chosen depends on the values of the function numpy.array were passed. Python Numpy always chooses a simple one Type that can save all transferred values. This can be an unintended limitation if an array should store other values after creation.
For example, the array x in the example above cannot store floating point numbers. Most of the functions pass a keyword parameter dtype to create arrays.
>>> x = np.array([-1,4,-5,7], dtype=np.float) >>> x array([-1., 4., -5., 7.]) >>> x.dtype dtype('float64')
The shape as the second fundamental property of an array is specified in the attribute shape saved.
>>> x.shape (4,)
The shape attribute stores a tuple that contains the number for each dimension of the array of the components. In the example, the array x therefore has one dimension with one component. To create an array with multiple dimensions, one can use the function numpy.array pass a nested structure of iterable objects. in the In the simplest case, this is a list that contains further lists.
>>> b = np.array([[1,2], [3,4], [5,6]]) >>> b array([[1, 2], [3, 4], [5, 6]]) >>> b.shape (3, 2)
In this example an array with two dimensions, i.e. a matrix, is created, which contains the elements of the transferred lists line by line. There are a number of special functions that allow arrays of a particular shape can be generated. The following table gives a brief overview of some these functions.
|array(object)||Generates an array from the data referenced by object. Often times, object is a reference to a nested list.|
|Creates an array containing num equidistant data points of contains start to stop (both inclusive).|
|empty(shape)||Creates an array with the dimensions shape. The parameter shape is a tuple that represents the number for each dimension contains the desired components. The values of the array are not initialized, which is why they can contain any numbers.|
|eye(N[, M[, k]])||Creates a two-dimensional array with N rows and M. Columns that have zeros everywhere except for the main diagonal contains. The value 1 is set on the main diagonal. The diagonal can be set to 1 via the parameter k be moved. If the M parameter is not specified, an array with N rows and N columns generated.|
|ones(shape)||Creates an array of the shape, the elements of which all have the Have value 1.|
|zeros(shape)||Creates an array of the shape, the elements of which all have the Have value 0.|
|diag(v[, k])||Creates an array that shows the values of the Sequence v and its other elements padded with 0 become.
The diagonal can be shifted via the parameter k, which should take the values of v.
As an example, we would like to create a 9 × 9 matrix with the Has values from 1 to 9 and the value 5 on the two secondary diagonals. All other Values should be assigned 3.
>>> M = 3*(np.ones((9,9)) - np.eye(9,9)) >>> M += 2*(np.eye(9,9,1) + np.eye(9,9,-1)) >>> M += np.diag(range(1,10))
To do this, we first create a matrix M with np.ones and np.eye, which is everywhere except the main diagonal has the value 3. Then we add to the two secondary diagonals add the value 2 using np.eye to get to the desired 5. Finally, we use np.diag to create a diagonal matrix with the values from 0 to 9 on the main diagonal and add this to M to get the final result receive.
>>> M array([[ 1., 5., 3., 3., 3., 3., 3., 3., 3.], [ 5., 2., 5., 3., 3., 3., 3., 3., 3.], [ 3., 5., 3., 5., 3., 3., 3., 3., 3.], [ 3., 3., 5., 4., 5., 3., 3., 3., 3.], [ 3., 3., 3., 5., 5., 5., 3., 3., 3.], [ 3., 3., 3., 3., 5., 6., 5., 3., 3.], [ 3., 3., 3., 3., 3., 5., 7., 5., 3.], [ 3., 3., 3., 3., 3., 3., 5., 8., 5.], [ 3., 3., 3., 3., 3., 3., 3., 5., 9.]])
This procedure of assembling more complex arrays with the help of the prefabricated building blocks, is typical for dealing with python numpy.ndarray.
Access to the elements of an array:
As with lists, the individual elements of an array can be accessed via an index become. Slicing also works in the same way with one-dimensional arrays for slicing lists
>>> a = np.array([-1,4,-5,7]) >>> a -5 >>> a[2:] array([-5, 7])
Since python numpy.ndarray is a mutable data type, the values of an array can also be after the generation can be changed.
>>> a = 1337 >>> a array([ -1, 4, 1337, 7]) >>> a[0:2] = [-20, -20] >>> a array([ -20, -20, 1337, 7])
If an array has several dimensions, the indices exist for individual elements from several numbers.
>>> a = np.zeros((5,5)) >>> a[0,1] = 1 >>> a[3,2] = 2 >>> a[1,3] = 3 >>> a array([[ 0., 1., 0., 0., 0.], [ 0., 0., 0., 3., 0.], [ 0., 0., 0., 0., 0.], [ 0., 0., 2., 0., 0.], [ 0., 0., 0., 0., 0.]])
Slicing also works with multidimensional arrays by adding Dimension a range is specified. In this way you can for example change individual blocks of a matrix in a targeted manner.
>>> a[0:3,0:3] = 5*np.ones((3,3)) >>> a[3:,3:] = 11 >>> a array([[ 5., 5., 5., 0., 0.], [ 5., 5., 5., 3., 0.], [ 5., 5., 5., 0., 0.], [ 0., 0., 2., 11., 11.], [ 0., 0., 0., 11., 11.]])
You can assign a number to a whole block, as in the second line of the example above. In this case, each entry of the relevant Blocks assigned the value 11. It is also possible to extract a part from an existing matrix.
>>> b = a [1: 4.1: 4] >>> b array ([[5th, 5th, 3rd], [5., 5., 0.], [0., 2., 11.]])
However, caution is advised here, because the matrices a and b share the same Data so that changes to matrix b also affect matrix a and vice versa. The copy method of the data type is used to copy part of a matrix numpy.ndarray. The following example extracts the same area from a as that Example before, where the data is copied this time.
>>> b = a [1: 4.1: 4] .copy ()
To conclude this introduction, we provide an overview of a selection of Attributes and methods of the data type numpy.ndarray. Where a is an instance of python numpy.ndarray. Some parameter lists have been simplified for clarity.
Python numpy Attributes and methods:
a.shape: Describes the shape of the array a as a tuple.
a.ndim: Indicates the number of dimensions of the array a.
a.size: Indicates the total number of elements in the array a.
a.itemsize: Specifies how many bytes each individual element of the array a is in Memory occupied.
a.dtype: Describes the data type of the elements of the array a.
a.nbytes: Specifies the memory, in bytes, that the elements of a occupy a total of.
a.real: Returns an array containing the real parts of the elements of a as contains elements.
a.imag: Returns an array that contains the imaginary parts of the elements of a contains as elements.
a.tolist (): Creates a nested list that contains the elements of a.
a.dump(file): Writes the array a to the file object so that it is subsequently can be read out again with the pickle module.
a.astype(dtype): Creates a copy of the array a. The elements of the resulting Arrays have the data type dtype.
a.copy(): Creates a copy of the array a.
a.fill(value): Sets all elements of a to the value value.
Methods for resizing:
a.reshape(shape): Returns an array of the shape shape that contains the same data like a contains. Shares the returned array and his Data with a.
a.resize(new_shape): Changes the shape from a in-place to the shape new_shape.
a.transpose(): Returns an array that contains the main diagonal contains mirrored values of a. This divides the returned Array and a same data.
a.flatten(): Copies the elements of the array a into an array with a Dimension by putting data from multiple dimensions together hangs.
a.ravel(): like a.flatten, except that no copy is made, so the returned array shares the data with a
a.argmax(): Returns the index of an element of a that is the largest Has value in a.
a.max: Returns the largest value in a.
a.argmin(): Returns the index of an element of a that is the smallest Has value in a.
a.min(): Returns the smallest value in a.
a.clip(min, max): Returns an array where the values of a are replaced by min and max were limited.
a.conj(): Creates a new array from the array a by removing all elements be complex conjugated.
a.sum (): Returns the sum of all elements of a.
a.mean(): Returns the mean of all elements in a.
a.var(): Computes the variance of all elements of a.
a.std (): Calculates the standard deviation of all elements of a.
a.prod(): Returns the product of all elements of a.
a.all(): Returns True if the truth value of all elements of a is true. If this is not the case, False is returned.
a.any(): Returns True if there is an element in a whose truth value Is true. Otherwise False is returned.
The skillful use of the python numpy.ndarray data type requires some practice. But once you have got used to the special features of vectorization, you can write very compact and efficient programs with it.
The scipy module offers a number of sub-modules for certain types of Calculations. The following table gives you an overview of which Modules are available and what functionalities they provide.
|cluster||Provides cluster analysis functions to categorize data sets.|
|constants||Provides mathematical and physical constants|
|fftpack||Provides functions for the discrete Fourier transform (implemented with the Fast Fourier Transform). They are also differential operators and folds provided.|
|integrate||Provides functions to numerically approximate integrals. Function objects as well as lists with discrete Functional values are integrated. In addition, there are functions that use ordinary differential equations numerically in one or more dimensions can be solved.|
|interpolate||Provides functions for interpolation. Thereby are under other polynomial interpolation (Lagrange and Taylor) and splines supported in one and more dimensions.|
|io||Offers convenience functions to transfer data from different formats read and write.
Data from the programs MATLAB and IDL as well as the Matrix Market file format and Arff files. Also can Data in NetCDF format as well as WAV sound files are read and to be written.
|linalg||Offers functions from linear algebra. This includes functions for solving linear equations, for Calculation of eigenvalues and eigenvectors as well as standard decompositions of matrices (e.g. LU, Cholesky, SVD).
In addition, matrix functions such as the Matrix exponential function implemented and functions for generation special matrices (for example the Hilbert matrix) are offered.
|misc||Offers general functions that are not in any of the other categories fit.
This includes, for example, functions for calculating the factorial or numerical differentiation.
|ndimage||Provides functions with which image data are processed can.
There are filter and interpolation functions as well as functions for Morphology and analysis.
|odr||Implements regression routines for orthogonal regression, with which measurement errors in the independent variables are also compensated can be.|
|optimize||Provides functions to solve extreme value problems and to zero to find. There will be problems in one and more dimensions supported with and without constraints.|
|signal||Provides functions for signal processing. These include, for example Filter and convolution functions as well as B-splines and Wavelets.|
|sparse||Provides functions for the efficient handling of sparsely populated Matrices.|
|spatial||Provides functions to answer area inquiries
A class for Delaunay triangulation is also offered.
|special||Provides a lot of special functions such as the Gamma function or orthogonal polynomials ready.|
|stats||Offers a variety of statistical functions and probability distributions.|
|weave||Enables C / C ++ code to be embedded in Python programs.|