IMT4892 - Digital Workflow

Introduction

Welcome to this course where the primary goal is to teach you tools and systems necessary to formalise and build flexible workflows. This page will be updated as the course progress and will include information to all the things you need to take this course. The formal course description can be found here.

Our expectations is that a 10 studypoint course requires a minimum of 15 hrs of work per week, and even more if your goal is to get a grade above 'C'. You will be expected to deliver a project proposal and a project report (40%) in addition to the final written exam (60%) to pass this course.

To create robust students that are well prepared for the rest of their master studies and and their continued professional lives, we will mostly use open free and flexible programs and the command-line driven UNIX system throught this cours. We will also include a brief introduction to academic writing using LaTex and BibTex in addition to a seminar on intellectual property rights and the Creative Commons movement in the context of marking material as part of a digital workflow. During the first lecture we will talk more about how you get access to a UNIX-system if you do not already have one.

This page will be updated throughout this course.

CONTENTS

The Tools

The use of the command-line UNIX may be a new experience to many of you, and may seem different to the systems you may be using today that include windows, icons, menus and pointing devices (WIMP). Command-line UNIX do however have the advantage of providing the user with a highly flexible way of stringing several programs together to easily create automated workflows - a feature that we will use in this course to learn how to work more efficiently with larger sets of data.

To follow this course you will need access to a computer running a UNIX command-line system. You can get access to such a system in several ways. We suggest that you quickly familiarize yourself with a comman-line driven UNIX system using one of the approaches below.

  1. Installing and running UNIX on your own machine will provide you with added flexibility and increased learning. If you have your own computer running a Microsoft operating system we suggest you take a look at Ubuntu and read more about how you can install it or run it off a memory stick, and the necessary software on your machine and run it beside your current operating system.
  2. If you have a machine running a linux distribution, refer to the documentation on your platform to find out how to install software.
  3. Macintosh computers running MacOS X are already running a UNIX system and will just need to install the software preferably using a software package installer, to be ready for this course. This is done by doing the following:
    1. Enable root access to your computer (Apple computer support)
    2. Download and install download and install MacPorts (a software package installer system)
    3. Install some of the UNIX software packages by issuing the following commands in the terminal window:
      • sudo port install imagemagick
      • sudo port install gnuplot
      • sudo port install ffmpeg
    4. Some of the software we need, comes as separeate stand-alone installers and will just have to be downloaded and installed manually from the developers websites. This applies to exiftool and MacTex (the latter if you have a Mac).
  4. If you do not have your own machine, you may use one at the college. In lab A211, we have prepared machines with the appropriate software and UNIX foundation, but some upgrades may be necessary. Please contact us if you do not have a machine and want borrow one.
  5. If you are a distant student, you may use your main GUC username and password to get (ssh) access to loke.hig.no (using a terminal window program like PuTTY) and from there do another ssh to the UNIX host stan.hig.no with a new username and password. Please contact us and we will provide you with this last username and password.

If you do not find yourself covered by any of these scenarios or have problems getting up and running, please contact us so we can work out a solution for you.

Literature

Books

This course is based on 3 books, and they all contribute their unique part to this course, and are at the same time the part of your 'bildung' during your master studies here at GuC.

  • Unix Power Tools, O'Reilly Media, Inc.; 3rd edition, 978-0596003302
  • XML In A Nutshell - A Desktop Quick Reference, 3rd edition, 2004, O’Reilley Media Inc., ISBN-978-0596007645
  • Suggested reading: The KML Handbook: Geographic Visualization for the Web, Josie Wernecke, 1st Edition, 2008, Addison-Wesley,ISBN-13: 978-0321525598

Books are available at our local bookstore "Mjøsbok" on the GuC campus or online at bookstores like Amazon.

Additional reading relevant for the final exam is:

Back to top of this page

Flexible learning

Wireless microphone

We will do our best to make this course flexible and our primary way of doing so in this course will be to record them and make them available after each lecture. You will be able to subscribe to the videos in your favourite RSS-reader by clicking here.

For IT problems regarding your web folder, email or other general IT resources at GUC, please contact the IT helpdesk.

Additional information about being at student at GUC can also be found on the GUC web site.

Back to top of this page

Study skills

...

Learning how to organize your day and be a productive student may be a challenge. We suggest you visit the web, a bookstore, the library or all of the above to read more about how to be a productive student, including how to avoid procrastination, how to manage speed-reading or other study skills.

Please note that we discourage the use of laptops during lectures, and would like to suggest that you take notes on paper. We believe that this approach makes for a better learning environment for us all in most situations and as a result - provide you with a better way of internalising the material that is being presented and stimulates discussions during the lecture.

We would also like you to encourage you to look at how you multitask when you study, as a growing amount of research seems to suggest that multitasking may reduce your learning outcome:

Lecture outlines will be made available for download after the lecture has taken place but will require your input and processing to value for you as a student.

If you run into problems or notice other students that may need someone to talk to for any reason, please know that we have a low threshold for talking to our students and helping out with problems. Do consider contacting the Teacher, the Student Counsellor, IMT, the Study Program Leader, IMT, the Student Chaplain or the Student Information Centre for a talk about any kind of challenges. A further source of information about social services and psychological treatment can be found at the The Foundation for Student Service in Oppland.

For IT problems regarding your web folder, email or other general IT resources at GUC, please contact the IT helpdesk.

Additional information about being at student at GUC can also be found on the GUC web site.

Back to top of this page

Teacher

...

The teacher for this course - Assistant Professor Kjell Are Refsvik can be found both in the physical world (Building A, Room A224A-B at the Gjøvik University Campus), and in the virtual world through several channels ( email, telephone:+47 61 13 52 99, mobile phone/sms/mms: +47 405 50 454, skype:refsvik, Twitter:refsvik, Google+:Kjell Are Refsvik

More information about his background can be found at his websites at his web page at GUC.

Do not hesitate to step by my office or contact me if you have something you want to talk about.

Back to top of this page

Lectures

The date, time and location of the lectures can be found in TimeEdit and subscribed to using your favourite iCalendar-savvy calendar reader.

We will try our best to make lecture recordings available through this RSS-feed.

Back to top of this page

Project

During this course, you will design, implement, test and document and a small workflow-related project. The project is individual and stretch over a period of 4 weeks (45-50 hrs of work) and require a short project proposal to be delivered, in addition to final project report/code/examples.

Doing this project will be mandatory, count for 40% of your final grade and will let us know if you:

  • have grasped the idea of formalising and creating a workflow that makes life easier
  • are starting to getting the hang of formalising an academic project
  • are getting to grips with the tools needed to write a master thesis (we strongly suggest giving LaTex/BibTex a try).

Feel free to develop an idea you already have into a working workflow. In the event that you may not have any ideas of your own, or just need inspiration, we have provided you with a list of suggestions.

1. Digital Pilgrims Route Project

The college has started collecting raw track data and images from the pilgrims route that pass through out district. Our data can be found here.

Based on the data we have collected so far, we need UNIX-based workflow tools to help us process this data and prepare it for desktop and mobile platforms as follows:

  • sort (locate and arrange data),
  • process (scale, photos:color optimization, compression),
  • describe (using different forms of metadata)
  • present (mashup with other online data services, HTML5/CSS3),
  • and upload (using secure/fast transport mechanisms)

2. The Changing Face of Gjøvik

The City of Gjøvik is 150 years old in 2012 and to celebrate we would like to make old photographs and stories available to new generations of citizens.

Mjøsmuseet has provided us with old images from Gjøvik, as well as track information for interesting cultural walks within the city of Gjøvik.

For this project we need several workflows that helps us process the images we have.

  1. We need to explore relevant open software solutions and see if we can create a UNIX-based workflow that matches old and new images taken from the same location.
  2. We need a workflow that can process geo-referenced JPEG-images and make them available on a mobile platform wrapped in a suitable environment. ARML (Augmented Reality Markup Language) may be a suitable choise, as may KML (Keyhole Markup Language).

The project have 2 smartphones students can use during this project.

3. Video cleanup project

The Gjøvik University College Library has got a number of aging VHS-tapes with educational content. The project should try to use UNIX-like systems and open software to automate the repair of these recordings by saving out individual frames from the captured video files (mpeg/mp4) and improve the quality using image processing software. The final task will be to collect all still frames and audio back into a single suitable video file format again.

4. Video composite project

A lot of people use their digital still cameras and mobile phones to shoot video clips. We would like to create a workflow that can combine a number of video files, perhaps add music, and credits at the end so that it can be used by us when concatenating videofiles.

5. Coastal Steamer

This summer, the Norwegian Broadcasting Corporation (nrk.no) produced the longest uninterrupted broadcasted show ever. The production aboard the coastal steamer "MS Nordnorge" was 134 hours long, but is now divided up into segments of video taken betweeen each port of call. It is also shared openly using a Creative Commons license so that everyone can download it all and remix or re-edit the content as long as they share their ideas and use it in a non-commercial setting.

Your challenge is to create a UNIX-based workflow that could be useful to anyone wanting to process these data. More information is available at http://nrk.no/hurtigruten/.

5. Self-selected project

You may define your own project, but it will have to use the tools, systems and methods used in the course and will have to be about developing a workflow.

The report from the project should answer the following questions:

  1. How automation solves one or more challenges in this project
  2. How users could build upon what you have done to do even more
  3. How do you choose to share what you have done into the world, and why

Earlier projects

This is a list of the workflow projects done in 2009 that could be used as inspiration, but please note the change from last year, that we are requiring you to use the technology we teach in this course when you create workflow solutions:

Back to top of this page

Week 34: Course Introduction

Learning outcome

After this lecture you will know the goal, the methods and the means of this course better. You will also know the members of your class a little better, including the lecturer. You will also know what you need to do in terms of getting started on getting access to a machine powered by UNIX, and also some hints on what to look for when searching for academic writing tools.

Suggested reading

Suggested work

  • Find and install a UNIX system on your computer as well as a software package installer. Install ffmpeg, imagemagick, exiftool, gnuplot and LaTex/BibTex.

Lecture notes

RSS and pdf.

Back to top of this page

Week 35: A Brief History of Computing

Learning outcome

After this week, you will know:

  • Be able to identify the key phases in the development and use of computers
  • Know the two main user interface paradigms
  • Know the main attributes of a UNIX system
  • Know the key differences between the command-line user interface and the graphical user interface and why we use the former as the basis for this course

Suggested reading and work

See the lecture notes and list above.

Lecture notes

Refresh the rss-feed for this course and look at the lecture notes to revisit this lecture.

...

Back to top of this page

Week 36: Devices and data formats, wrappers and compressors

Learning outcome

After this week, you will know the difference between file formats and wrapper formats, know the different kind of formats and also know the difference between a proprietary file format and an open one.

Please read your email about a guest lecture this wednesday.

Lecture notes

Refresh the rss-feed for this course and look at the lecture notes to revisit this lecture.

Suggested reading

  • UNIX: p.147-290
  • KML: p.15-107
  • XML: p.261-277

Suggested work

  • Install tools to manipulate different data formats. See section above about what UNIX tools to install.
  • Mount a device you have and download files form it using UNIX.
  • Try to convert a file from one format to another.
  • Read wiki articles embedded in the lecture

Back to top of this page

Week 37: Tools for transforming common datatypes

Learning outcome

Understand the difference between transforming and transcoding data, and be able to process common data types using UNIX-based tools.

Lecture notes

Refresh the rss-feed for this course and look at the lecture notes to revisit this lecture.

Suggested reading

  • UNIX:
  • KML:
  • XML:

Suggested work

  • Try to get hold of, and transform/transcode different data-types (images, geo, video, audio)

Back to top of this page

Week 38: Moving data, remote processing and workflow automation

Learning outcome

After this week, you should know how to access remote machines and move data to and from those remote machines.

Suggested reading

  • UNIX: Relevant pages regarding moving data. p936-963+994-1030

Suggested work

Suggested work is outlined in the lecture.

Lecture notes

Refresh the rss-feed for this course and look at the lecture notes to revisit this lecture.

Back to top of this page

Week 39: Workflow automation

Learning outcome

After this week, you should know how to bring single UNIX-commands together to form an automated workflow.

Suggested reading

  • UNIX: Relevant pages regarding scripting (p. 513-784), including material on cron jobs for background/times jobs (p. 488-498).

Suggested work

<

Suggested work will be outlined in the lecture.

Lecture notes

Refresh the rss-feed for this course and look at the lecture notes to revisit this lecture.

Back to top of this page

Week 40: Intellectual property rights (IPR), including Creative Commons

Learning outcome

Know some of the basic mechanisms for handling the rihts to Intellectual Property and specifially know the purpose, challenges and opportunities related to the Creative Commons licensing system in an educational context.

Suggested reading

Suggested work

Suggested work and additional reading will be outlined in the lecture.

Lecture notes

Refresh the rss-feed for this course and look at the lecture notes to revisit this lecture.

Back to top of this page

Week 41: Mashup fundamentals and applied mashup using KML and Google Maps

Learning outcome

After this lecture, the students should be able to identify the core elements of a KML file, know some of the possibilities included in the KML file format, be able to build a simple KML file and extend excisting ones and mashup own data with digital maps using calls to Google Maps.

Suggested reading

Suggested work

Suggested work will be outlined in the lecture.

Lecture notes

Refresh the rss-feed for this course and look at the lecture notes to revisit this lecture.

Back to top of this page

Week 42: XML Fundamentals, part 1

Guest lecturer, Professor Rune Hjelsvold.

Learning outcome

After these two lectures, the students should:

  • ...know the background, purpose and key topics related to the area of semantic web / web 2.0
  • ...be able to identify the elements and workings of a simple XML-based file format
  • ...be able to write a simple XML-document and validate the content
  • ...extend an existing XML-based file format with more semantic meaning

Suggested reading

  • UNIX:
  • KML:
  • XML: Will be announced

Suggested work

  1. XML Introduction: Complete the ZVON XML Tutorial.
  2. XML Namespaces: Complete the ZVON XML Namespace Tutorial.
  3. DOM: Download the rss feed from Kjell Are's course page. Rename the file to rss.xml. Open the file in Firefox (with Firebug add-on installed). Use Firebug to navigate the DOM structure (first and last sibling, first and last child node, attribute values, parent node, owner document, text content, etc.)
  4. RDF Introduction: Complete pages 1 through 13 in the ZVON RDF Tutorial.

Lecture notes

See below.

Back to top of this page

Week 43: XML Fundamentals, part 2

Guest lecturer, Professor Rune Hjelsvold.

Learning outcome

See lecture spesification above.

Suggested reading

  • UNIX:
  • KML:
  • XML: Will be announced

Suggested work

  1. RDFa:Read the RDFa Primer.
  2. XML Schema:Complete the w3schools' XML Schema Tutorial and read section 5.5 in the XML Schema Primer.
  3. Extending RSS:Add copyright information to all items in the rss feed from the course, assigning the CC non-commercial, share alike license.
  4. Study the RSS XML schema

Lecture notes

A complete list of lecture documents from the two last weeks are available from the list below:

* Podcasted through the podcast feed.

Back to top of this page

Week 44: Academic planning and writing

Learning outcome

After this week you will be able to start writing a project proposal and a project report using LaTex and BibTex.

Suggested reading

We suggest you visit the library to get help finding resources relevant to how do an academic project.

You may also benefit from downloading a finished master thesis and having a look at how it was put together (latex/tools).

Suggested work

  • Write a project proposal for your project (outline given in earlier lectures), preferably using LaTex. Find a LaTex installation that fits your Linux platform of choice. Mac OS X users should install MacTex before they start.

Lecture notes

Refresh the rss-feed for this course. We are sorry for the lack of a pdf-file from this lecture.

Back to top of this page

Week 45: Project, week 1 - project outline delivery

No lectures this week. Students are free to seek guidance as needed. Project proposals as pdf-files needs to be delivered as email or in Fronter within 11.59AM Friday November 11th. Please feel free to send draft version for review before you deliver your final proposal. The delivery of a project proposal will make you eligible for the final exam. The delivery a final project report together with the final exam will make you eligible for a final grade in this course.

Back to top of this page

Week 46: Project, week 2 - work

No lectures this week.

Back to top of this page

Week 47: Project, week 4 - project delivery/presentation

Final delivery deadline of the report (pdf) are 0830 at Wednesday November 23., 2011. and we would like you to deliver your reports in the Fronter LMS. Please use the following filenaming convention before you upload:

  • imt4892_2010_yoursurname_shortprojecttitle.pdf

If your project consists of more than one file, you may upload a single file in the .zip format.

If you are uable to attend (physical or through Skype), we would also like you to upload a 10-15 minute presentation of your project (or a URL to one).

The same day we will ask you to present your work in front of the class in a presentation that lasts between 10-15 minutes.

We depend on your feedback to make our courses and staff better. Please download and fill out a evaluation form and return it to us (email, or anonymous using physical mail to the address below) so we kan keep improving.

Back to top of this page

Week 48: Reading week before the exam

No lectures this week.

Back to top of this page

Week 49: Exam

There will be a 4 hour written exam with no aids on the 7th of December (Wednesday) at 10.00 - 14.00. The results of this exam will count for 60% of your final grade in this course. Passing is mandatory to get a grade in this course. Former exams are available in the Fronter LMS. More information about the exam will follow later.

Best of luck on the final exam!

Back to top of this page