Fast Array Operations with NumPy and Pandas

NumPy is the de facto standard for scientific computing in Python. It extends Python with a flexible multidimensional array that allows fast and concise mathematical calculations.

NumPy provides common data structures and algorithms designed to express complex mathematical operations using a concise syntax. The multidimensional array, numpy.ndarray, is internally based on C arrays. Apart from the performance benefits, this choice allows NumPy code to easily interface with the existing C and FORTRAN routines; NumPy is helpful in bridging the gap between Python and the legacy code written using those languages.

In this chapter, we will learn how to create and manipulate NumPy arrays. We will also explore the NumPy broadcasting feature used to rewrite complex mathematical expressions in an efficient and succinct manner.

Pandas is a tool that relies heavily on NumPy and provides additional data structures and algorithms targeted toward data analysis. We will introduce the main Pandas features and its usage.  We will also learn how to achieve high performance from Pandas data structures and vectorized operations. 

The topics covered in this chapter are as follows:

• Creating and manipulating NumPy arrays
• Mastering NumPy's broadcasting feature for fast and succinct vectorized operations
• Improving our particle simulator with NumPy
• Reaching optimal performance with numexpr
• Pandas fundamentals
• Database-style operations with Pandas