Cleaning Messy Data

In early 2017, I was hired by the Essex-Kent Mennonite Historical Association to recommend and implement a solution to upgrade their collection management system from a home-brewed Microsoft Access database to something easier to work with and maintain. I researched several possibilities appropriate for small museums and they settled on PastPerfect, a museum-specific software tool.

A screenshot of OpenRefine open with many records displayedI exported the Access database into a spreadsheet, which I then imported into OpenRefine. OpenRefine is “a free, open source, power tool for working with messy data.”  Data wrangling with OpenRefine relies on building up facets and filters. It is perfect for this kind of work.

I started by sorting almost 4000 records into PastPerfect’s four distinct catalogs (Object, Archive, Library, Photos). After that came assigning object names to all items using Nomenclature 4 (PastPerfect’s lexicon). By combining facets filtered on catalog, object name, the headings imported from MS Access, and others as needed, I was able to create a clean spreadsheet for the Conversion Specialist to import into PastPerfect.

It was no surprise to find that over the years, different museum volunteers had collected different data about the collection and that data had been entered in different ways. I had to correct and standardize any errors and omissions at the record level, and define clear procedures for future acquisitions. I continue to provide training for new volunteers to work with PastPerfect.

3D Printing

I finally had a chance to start experimenting with 3d printing in 2017.

I’ve perfected skimming Thingiverse for things to print and I’ve got the hang of slicing and generating support material, brims, and G-code.

Waiting is hard and the whole thing can get pretty frustrating when prints fail – which, unfortunately, is often.

My design skills are still pretty limited: I can scale designs up or down, and I’m getting better at predicting whether a print will succeed in Slic3r before loading the filament. I’m also pretty good at cleaning the nozzle end (when filment jams) and the print bed (when prints don’t stick).

It’s kind of exciting to see what’s being done in the museum world — I’m especially excited about the potential for things like Museum in a Box.

 

Transcribing Oral History

I learned a lot about oral history during my Masters project. At that time, I developed my own process for transcribing using Audacity and a plaintext editor; since then I’ve been hired to help on several academic oral history projects.

I’ve left Audacity behind, and switched to using a combination of ExpressScribe and GoogleDocs for this work. Some clients also request help selecting audio recording equipment, file organizing, storage and backup.

With the Breaking the Colour Barriers project, I began with transcribing and was later asked to do interview indexing as well.

Breaking the Colour Barriers acknowledges the support of the Ontario Trillium Foundation, the Chatham Sports Hall of Fame, the Chatham-Kent Black Historical Society, the Multicultural History Society of Ontario, the Department of History (University of Windsor), Leddy Library (University of Windsor), Faculty of Arts, Humanities and Social Sciences (University of Windsor), and many individuals.

Screenshot of Breaking the Colour Barrier Interviews available at http://cdigs.uwindsor.ca/BreakingColourBarrier/exhibits/show/oral-history/oral-histories

Breaking the Colour Barrier Interviews

 

Digitizing Texts

In late 2016, I was hired by the Essex-Kent Mennonite Historical Association to begin digitizing their collection to share online. After some copyright research and consultation with the Archives volunteers, we set a priority list and established a place to begin. The group has published 14 books about local Mennonite history since they incorporated in 1987 and these books could be useful to local and Mennonite historians, and the local Mennonite high school. Together, we identified this as the best place to begin.

Screenshot of EKMHA's online repository at ekmha.ca/collections

EKMHA’s online repository at ekmha.ca/collections

I worked on every stage of this project, from identifying current best practices, scanning equipment, OCR software, and repository options, to doing the scanning, OCR, metadata, publishing, and training and supervising volunteers to help.

The repository is built using self-hosted Omeka with a custom theme designed to match the organization’s website. In 2012, EKMHA hired me to convert their static html site built using MS Word to something more modern looking and easier to update. After four years, that site needed to be upgraded and updated before adding the Omeka repository. The result launched in early 2017: a combination WordPress and Omeka site that meets the organizations needs and is easy to update and maintain.

When the 14 books were complete, the project moved to digitizing high school yearbooks from United Mennonite Educational Institute (UMEI), 1947-1980, local senior biographies written by UMEI history students, and EKMHA’s newsletter. The online collection continues to grow.

The repository is available at www.ekmha.ca/collections/

 

Raspberry Pi Museum Kiosk

Raspberry Pi microcomputer with flashdrive installed

Raspberry Pi microcomputer with flashdrive

I got the idea for the kiosk from this tutorial about creating digital signage with WordPress and a Raspberry Pi, but felt like WordPress was more than I needed for the project I had in mind. Instead, this Pi runs feh, a command line image viewer and cataloguer.

Images are loaded onto a USB flashdrive, which is then inserted into the Pi. Any volunteer can provide a collection of images for the display. The slideshow script runs when the Pi boots up.  Slideshow is feh’s default mode with custom options including adjustable display time, auto-rotation, and borderless windows. I add captions to the images using an ImageMagick script which extracts the caption field from Picasa and appends it to the bottom of each photograph using -annotate. (Funny story: I happened to be using a Windows computer the day I was working on this. If you’re not used to using ImageMagick on Windows, you might not know you must preface each command with magick. I did not know this when I started, but I do now…)

So now every weekday morning, visitors to the Mennonite Heritage Centre in Leamington, Ontario are greeted by three things: a friendly guten tag from Walter Koop behind the desk, the aroma of freshly-brewed coffee prepared by cafe volunteers, and a slideshow of photos.

Recently, the front desk acquired a digital kiosk, consisting of a 23-inch computer monitor and a raspberry pi. A raspberry pi is a microcomputer, only slightly larger than a deck of cards. Each day, Walt Koop selects a collection of photos to feature on the kiosk; so far the most popular set is EKMHA’s collection of wedding photos spanning from the 1920s to the 1970s.

Whether they’re on their way to or from coffee, choir, or exercises in the auditorium, you can often find a group of people enjoying the display, and remembering their friends and families on their special days.

At the Heritage Centre, you’ll often find a mix of the old and the new. We’re glad to be able to share items from our collection with our visitors in this way. The archives contain many more treasures like these and we encourage visitors to come in and explore. We also welcome donations of photographs and other items. Email info@ekmha.ca for more information.

Museum visitors watching the kiosk slideshow

Museum visitors watching the kiosk slideshow

A less-technical version of this article appeared in EKMHA’s Spring 2017 newsletter.

Muckle Ado – the blog

Screenshot of MuckleAdo.com's home page on May 19, 2012Robbie Burns knew about having a lot to do – which is what the Boat Song is all about. To have muckle ado is to have much to do and this makes it a fitting title for a blog. Muckle Ado starts with posts from 2005 when I started blogging for undergraduate assignments, carries through to my graduation from Women’s Studies, shifts into writing about History for my Masters program. In between, all around, and since are thoughts on technology, family, Life, the Universe and Everything.

Virtual Museum Canada

cover image from Mennonite Memories of Pelee Island online exhibit
When the Essex-Kent Mennonite Historical Association received a grant to create a virtual museum exhibit about the history of Mennonites on Pelee Island, Ontario, from 1925-1950, they hired me because I offered a combination of technical expertise and a background in Pelee Island’s social history.

Volunteers collected photos from families near and far, and two local teens recorded interviews with community members as they browsed the photos.

My job was organizing and editing the masses of visual, textual, and audio data and using software from the Virtual Museum of Canada at the Canadian Heritage Information Network to create the exhibit. I worked with am currently working with a team of subject experts to review the exhibit content and the site is scheduled to launch sometime in late April 2012 and the site is now live!

Visit Mennonite Memories of Pelee Island, Ontario, 1925-1950.

ProfHacker OpenSearch Plugin

Firefox search options in a drop down menu

Photo credit: Brian Croxall

I’ve been a regular reader of ProfHacker since its launch (i.e. before it moved to the Chronicle of Higher Education Blogs). The tips and tutorials are helpful – and usually timely. I found the site so useful that I was constantly sharing links with colleagues, but I’d accumulated so many bookmarks that sometimes finding the right article was a challenge. What I really needed was an easy way to search the site.

I took this as an opportunity to create a ProfHacker.com Firefox search plugin using OpenSearch. Then, when I needed to find something on ProfHacker, I could use the browser search bar (using Ctrl+k / Cmd+k) to choose ProfHacker from the list of available search engines.

Once ProfHacker moved over to the Chronicle of Higher Ed, the ProfHacker.com site specific search plugin broke, but the tool was great while it lasted.

Hacking Wearables

In 2010, I went to the Great Lakes THATCamp and was able to attend a “Hacking Wearables” workshop with Bethany Nowviskie and Bill Turkel and I made a bracelet. It’s hand-sewn out of felt, with various crisscrossing bits of fabric and French knots. A circuit sewn with conductive thread connects the battery with a small LED:

 

a felt bracelet with a circuit sewn with conductive thread which lights a small LED bulb

Mapping the City of Windsor’s Open Data

This map uses open data from the City of Windsor Open Data Catalogue. If you’re interested in the process used to make this map I’ve included step-by-steps at the bottom of the page. Drag and zoom to explore.

Key to icons:

red dot Large red = Community Centres
purple dot Large purple = Libraries
blue dot Large blue = Arenas
green-circle Small green = Heritage Sites (listed & designated)

 

Process

This is how I made this map, but not exactly what I’d do next time. See note below.

  1. download csv files from the City of Windsor Site
  2. refine data using Excel* (see note below): columns were in different order on different spreadsheets, not all included all columns, added data type and icon type.
  3. import all spreadsheets into Google Fusion Tables
  4. modify data to recognize lat & long coordinates as Location
  5. Visualize map to view the data points on GoogleMaps.
  6. Embed map in blog.
  7. Share!

*Next time I plan to use Google Refine to clean the data. Working across multiple spreadsheets was a pain.

Total time to make the map: 30 mins.