Working with Plain Text Files

Files are the only way to `store` on a computer.

Your entire computer hard drive is comprised of files, is the most low-level way to store information, your computer runs on a file system or file directory control that how data is stored and retrieved.

Using a backend language you can access the majority of the computer files, and that gives you almost endless power!

Retrieving data from files

Let's say that you have the bitcoin prices from the last day in a bitcoin_prices.csv file with the following format:

<before-after width="400px" before="https://github.com/breatheco-de/content/blob/master/src/content/lesson/../../assets/images/97f74cd8-acdd-4ce9-aa26-bfd494e9b550bitcoin_price_csv.png?raw=true" after="https://github.com/breatheco-de/content/blob/master/src/content/lesson/../../assets/images/709ff7ce-f7f6-4b16-a172-521fe1787733bitcoing_prices_table.png?raw=true" />

📺 Five minute video explaining what is a CSV file

Basically, every line in the CSV file represents one price, for example:

Currency,Date,Closing Price (USD),24h Open (USD),24h High (USD),24h Low (USD)
BTC,2019-10-29,9455.7246926058,9228.0745024715,9551.7787262272,9125.7784571584

You can read the file with any backend programing language and interpret it based on the positions of the values:

import csv, json

file = open("bitcoin_prices.csv", "r") 
csv_f = csv.reader(file)

prices = []
for row in csv_f:
  prices.append({
      "ticker": row[0],
      "price": row[2]
  })

Saving data into files

Let's say you are running a script that has a variable called todos that contains a todo list:

todos = ['make the bed', 'do the laundry', 'finish homework']

That variable is being stored in the RAM memory until you decide to save it to a text file or database. The RAM memory is not reliable because your computer could lose power at any moment (turned off).

You can save that variable into a todos.csv file with the following python code:

todos = ['make the bed', 'do the laundry', 'finish homework']

todos_as_csv = ','.join(todos) # convert the list into a string
file = open('todos.csv', 'w+') # open the file for writing 'w', create if it doesn't exist
file.write(todos_as_csv) # write the content
file.close() # close the file

The code above will create or update a todos.csv with content similar to this:

make the bed, do the laundry, finish homework

Different file formats

Format	Explanation
CSV	Comma `,` separated values, one line for each row or different entity.
JSON	Very similar to Javascript syntax, made especially for developers and the most used format when transmitting information over the internet (HTTP)
Yaml or YML	The easiest format to understand, developers love it because it is fast but it's also very similar to a simple text file, it allows comments and uses indentation instead of commas or braces for delimitation
XML	Very popular in the 90's and still being used in a lot of legacy software

Converting from CSV Text to Python Object in memory

import csv
file = open("bitcoin_prices.csv", "r") 
file_content = csv.reader(file)
for row in file_content:
    print("First element: " + row[0])
    print("Second element: " + row[1])
    # etc..

Converting from JSON Text to Python Object in memory

import json
filePointer = open("bitcoin_prices.json", "r") 
data = json.load(filePointer)
prices = []
for row in data:
    print("First element: " + row["ticker"])
    print("Second element: " + row["date"])
    # etc..

Converting from Yaml Text to Python Object in memory

import yml #you have to install pip package pyyaml

filePointer = open("bitcoin_prices.yml", "r") 
data = yaml.load(filePointer)
prices = []
for row in data:
    print("First element: " + row["ticker"])
    print("Second element: " + row["date"])
    # etc..

Here is a live demonstration loading all three types of files.