Introduction notes to Python deep learning: use Pandas to read batch CSV files with sequential variables in the file name

Keywords: Python Deep Learning


This article mainly shows how to import CSV files with variable names into Python, such as a series of continuously changing files, such as:

  • r1.csv
  • r2.csv
  • r3.csv
  • ...

I'll use a simple example to illustrate this concept and provide complete Python code.

Typical application scenarios

We often encounter the following scenarios:

  • Create a simple graphical user interface (GUI) with input boxes. Then enter a variable in the box (for example, a specific date)
  • The GUI will also contain a button. After clicking this button, you can read batch CSV files according to the variables you enter

In order to achieve the above goal, we need to import tkinter (for creating GUI) and pandas package (for importing CSV file into Python).

What we call a GUI should look like this:

Examples and codes

We want to import a CSV file into Python, where the file name changes every day. For example, a CSV file name might contain a date that varies from day to day.

In this example, the CSV file is stored in the following path. Please note that you need to change this path (in the Python code provided below) to the storage location of the CSV file on your own computer.

'C:\Users\Ron\Desktop\Import into Python\Sales_' + x1 + '.csv'

Some explanations for the above path:

  • The part in red is the part of the pathname that will never change
  • The highlighted part in yellow indicates the variable (in our example, it is the date)
  • Black word is the CSV file extension, and it will not change
  • Note that the + symbol is used to connect the different components of the pathname.
  • Also note that variables should not be enclosed in quotation marks (and non variable parts should be enclosed in quotation marks)

Suppose you store the following data in a CSV file. And the CSV file name contains the date 27042019 (this date is your variable), so the full CSV file name is: sales_ twenty-seven million forty-two thousand and nineteen

To import a CSV file with variable names into python, you can use the following Python code (remember to change the path). Additional comments included in the code:

import pandas as pd
from pandas import DataFrame
import tkinter as tk 
root= tk.Tk()
canvas1 = tk.Canvas(root, width = 300, height = 300) # create the canvas
entry1 = tk.Entry (root) # create the entry box
canvas1.create_window(150, 100, window=entry1)
def insert_number(): # add a function/command to be called by the button (i.e., button1 below)
    global x1 # add 'global' before the variable x1, so that you can use that variable outside of the command/function if ever needed 
    x1 = str(entry1.get()) # store the data input by the user as a variable x1 
    PATH = r'C:\Users\Ron\Desktop\Import into Python\Sales_' + x1 + '.csv' #(use "r" before the path string to address special character, such as '\'). Don't forget to put the file name at the end of the path + '.csv'
    read_sales = pd.read_csv (PATH)   #read the csv file using the 'PATH' varibale 
    df = DataFrame(read_sales,columns=['Client Name','Country','Product','Purchase Price','Date'])  # assign column names
    print (df)
button1 = tk.Button (root, text='Input date to import file (ddmmyyyy) ',command=insert_number, bg='green', fg='white') # button to call the 'insert_number' command above 
canvas1.create_window(150, 140, window=button1)

Notice that in Python code, we use DataFrame to assign column names (highlighted in yellow)

If your CSV file contains different column names and / or different numbers of columns, you need to change the yellow section to match your dataset.

Run the above code in Python

(1) Click Run

(2) Enter 27042019 in the box

(3) Finally, click the button 'Input date to import file (ddmmyyyy)‘

After running, you can see the results (provided that I already have a file named 27042019.csv in my local location)

Repeated use of this method can achieve interactive reading of files named with variables. Of course, you can also discard the GUI and use the loop to traverse all files named according to a certain law

Posted by Zero20two on Sat, 30 Oct 2021 19:13:06 -0700