Programming fundamental concepts: Modules and Numpy

Programming fundamental concepts: Modules and Numpy#

This assignment is not obligatory

Introduction#

In the last tutorial we have covered the basics of Python programming, including variables, data types, and control structures. We also introduced some fundamental programming concepts such as loops and conditionals. In this tutorial we will take a closer look at functions and how to use them effectively in your code. We will not only use built-in functions but also import them from external libraries.

Throughout this tutorial, we will also focus on file handling in Python, including reading from and writing to files. This is an essential skill, since we will be working a lot with data stored in files.

Useful tools#

During coding, you will often need to look up how certain functions work. There are several useful tools to help you with this:

Python packages#

for this assignment we will be using the following packages:

  • numpy: for numerical operations

  • pandas: for data manipulation and analysis

  • os: for interacting with the operating system

These should be installed in your Python environment. If you haven’t installed them yet, you can do so using conda:

conda install numpy

\(\text{Task 4.1:}\)

Let’s start by importing the necessary libraries.

import numpy as np
import pandas as pd
import os

Numpy#

Numpy (numpy documentation: https://numpy.org/doc/stable/) is one of the most popular libraries for numerical computing in Python. It provides support for arrays, matrices, and a wide range of mathematical functions to operate on these data structures. Numpy is particularly useful for handling large datasets and performing complex mathematical operations efficiently. You will be using this library extensively throughout MUDE.

We will start by creating a simple array using Numpy after which we will focus on indexing and slicing. Indexing and slicing works similarly to Python lists, but with some additional features.

A list of useful numpy functions:

  • np.array(): Create an array

  • np.arange(): Create an array with a range of values

  • np.zeros(): Create an array filled with zeros

  • np.ones(): Create an array filled with ones

  • np.eye(): Create an identity matrix

  • np.reshape(): Reshape an array

  • np.transpose(): Transpose an array

  • np.concatenate(): Concatenate arrays

  • np.split(): Split an array

  • np.mean(): Compute the mean of an array

  • np.sum(): Compute the sum of an array

  • np.max(): Find the maximum value in an array

  • np.min(): Find the minimum value in an array

  • np.std(): Compute the standard deviation of an array

  • np.dot(): Compute the dot product of two arrays

  • np.cross(): Compute the cross product of two arrays

  • np.linalg.inv(): Compute the inverse of a matrix

  • np.linalg.det(): Compute the determinant of a matrix

  • np.random.rand(): Generate random numbers

  • np.random.randn(): Generate random numbers from a standard normal distribution

# Create a 1D array

# Create a 2D array

# Append a new row to the 2D array

\(\text{Task 4.2:}\)

We will now create a np.array with 10 rows and 3 columns. The first column is filled from 1 to 10, the second column is filled with the squares of the first column, and the third column is filled with the cubes of the first column. After we have created the array we will save it to a file.

I will show you two ways to do this:

  • Starting with an empty array and filling it in a loop.

  • creating 3 separate arrays for each column and then combining them.

# Method 1

# Method 2

# Lets check

# save the file as csv

\(\text{Task 4.3:}\)

Now we will use some of numpy’s powerful features to manipulate our data. Let’s try some matrix operations. First, we will create a 3x3 matrix and perform some basic operations on it.

# Perform basic operations

# Matrix multiplication

# Compute the inverse and determinant

os#

Now that we have our data saved in a CSV file, we can use the os module to interact with the file system. For example, we can check if the file exists, remove it, or rename it.

some useful os functions:

  • os.path.exists(path): Check if a file or directory exists

  • os.remove(path): Remove a file

  • os.rename(old, new): Rename a file or directory

  • os.listdir(path): List all files and directories in a directory

  • os.getcwd(): Get the current working directory

  • os.chdir(path): Change the current working directory

  • os.mkdir(path): Create a new directory

  • os.makedirs(path): Create a new directory and all intermediate directories

  • os.path.join(path, *paths): Join one or more path components intelligently

# List all files in the current directory

# Get the current working directory

Pandas#

Now we will use pandas to import our newly created CSV file and perform some data analysis. We could also do this with other libraries, such as numpy, but pandas is specifically designed for data manipulation and analysis.

Some useful pandas functions include:

  • pd.read_csv(): Read a CSV file into a DataFrame.

  • df.head(): Display the first few rows of a DataFrame.

  • df.describe(): Generate descriptive statistics.

\(\text{Task 4.4:}\)

Import our my_array.csv file using pandas and display the first few rows as well as the summary statistics.

# Import our my_array.csv file using pandas

# Display the first few rows

# Display the summary statistics

# Transform the data

# Create numpy array from DataFrame

By Berend Bouvy, Delft University of Technology. CC BY 4.0, more info on the Credits page of Workbook.

Contents