Data Wrangling III: JT's Python path to YAML

It took me more than 25 years to write this

My sister and I got our first computer in the late 1990s. It was ours. We were homeschoolers and the computer was loaded with Edutainment games. I know we had both Amazon and Oregon Trail. There was also a phonics game. I learned HTML on dial up while playing Neopets.

During my Philosophy degree, I took a class on Aristotle with Margaret Cameron. She told me two things: I should go to DHSI and I should look into the UBC Archival Studies Program. I didn’t take her advice immediately. After graduating from my first degree, I worked as HR and tech support for a small 10-year startup in Victoria, BC. After paying off my loans, I went back to university.

During my Greek and Latin BA, I did go to DHSI and from that I developed my BA thesis, I argued that the territories of nomadic people described in Herodotus Book 4 could be determined by mapping ancient rivers onto modern satellite imagery (and accounting for river drift over time). At DHSI I met a librarian from UBC who had worked to digitize epigraphic texts and make them available online for everyone to access. I decided to take Cameron’s second piece of advice and looked into UBC’s Archival Studies Program.

I found myself in 2025, an aspiring digital humanist wrapping up my Dual Master of Archival Studies and Library and Information Studies. For my final course, I wanted to collaborate with a digital humanities project. I met with John Maxwell to explain my research and showed him a Python script I had written. To my surprise, he hired me as a GRA for the Aldus@SFU project. My job would be to do technical development for the projects website and help process the data that the team had collected.

I try not to reinvent the wheel, so I looked for a tool that would convert the Aldus@SFU book data into YAML and found Javed Ali’s package.

But I didn’t need it to convert both ways like it did. What I needed was to convert the data in a spreadsheet (which I exported as a CSV) into a YAML header in a markdown file, so it could be the basis of the book pages in this website.

I stripped out the unneeded part of the code and was left with this:

# Author: Jesse "JT" Thomas
# Version: 1.4
# References: Based on Javed Ali's "Bi-directional CSV-YAML Converter." 
https://github.com/javedali99/bi-directional-csv-yaml-converter. 
Last updated Oct 21, 2023.

import csv, yaml, sys
file_in = (sys.argv[1])

# Read the CSV file
with open(file_in, 'r',encoding='utf-8') as csv_file:
    reader = csv.DictReader(csv_file)
    data = list(reader)

output = file_in + ".md"
# Write to a markdown file
with open(output, 'w',encoding='utf-8') as md_file:
    yaml.dump(
    data,
    md_file,
    width=9000, # Extra width avoids linebreaks.
    canonical=False, # Use "True" for additional formatting options
    explicit_end=True,
    explicit_start=True,
    sort_keys=False,
    allow_unicode=True,
    indent=4)

print("Task Completed")

This script was the basis for a more complex one that converted a single CSV into multiple Markdown files, one per row. But this simple script is actually quite handy. I have used it to convert a spreadsheet of peer reviews into an easily readable document for the IDEAH Journal, create inventory sheets for CiTR 101.9 FM’s Music Library, and generate a single document containing all the Aldus@SFU book measurements.

The takes a file from console command, reads it using the DictReader, and then creates a list called ‘data.’

It then dumps the yaml into a new file, named with the name of the input file plus the “.md” ending.

And that’s all there is to it.