Uncategorized

AVPreserve Releases Fixity And MDQC, Digital Preservation Tools

AVPreserve is pleased to announce the release of Fixity and MDQC, two new digital preservation tools available for free download on our Tools page.

Fixity is a free open source utility for the documentation and regular review of stored files. Fixity scans a folder or directory, creating a manifest of the files including their file paths and their checksums, against which a regular comparative analysis can be run.Fixity monitors file integrity through generation and validation of checksums, and file attendance through monitoring and reporting on new, missing, moved and renamed files.AVPreserve created Fixity after coming to the realization that:

1. Checksum generation and validation are universally recognized as a primary mechanism for fulfilling the goal of fixity.

2. Generating checksums without systematically and routinely validating them is of little or no use.

3. Most organizations that “use checksums” only generate checksums but don’t validate them routinely, if at all.

4. The primary reason organizations don’t systematically and routinely verify checksums is because there is no tool that allows scheduling and reporting of validation.

5. Organizations that do routinely verify checksums do not typically monitor and report on file attendance types of information.

6. Overcoming the lack of resources and/or access to IT expertise is a critical factor in enabling organizations to fulfill the goal of fixity. Organizations need a simple, low cost tool.

There are many free and open source checksum utilities out on the market but they do not offer the feature set or simplicity necessary to fulfill the need that organizations have. Fixity was created with the sole focus of fulfilling the requirements of those concerned with monitoring fixity of a collection over the long term. It is intended for use in monitoring collections of files that are “final state”, or ready for deposit into an archive or preservation oriented repository.

MDQC stands for Metadata Quality Control. It is a free open source utility that reads the embedded metadata of a file or directory and compares it against a set of rules defined by the user.AVPreserve created MDQC after doing a great deal of wrangling data using a combination of scripts, ExifTool, MediaInfo, spreadsheets and formulas. The most common use to date has been to automate and minimize the time needed to QC large batches of digitized or reformatted assets. For instance, organizations have used MDQC on files being returned from a vendor or in-house lab to verify technical parameters, such as bit depth, sample rate, codec and aspect ratio, as well as non-technical parameters such as copyright statements, title information, identifiers and more.

Although this has been the primary use so far MDQC can be used in any workflow where comparing actual metadata to expected metadata is useful. MDQC can be used on any file type supported by ExifTool and MediaInfo.

Direct downloads and User Guides of these (and other tools!) are available on our Tools page, or via the Fixity and MDQC GitHub pages.

Both of these applications are in beta. Please help refine them further by reporting all bugs to https://github.com/avpreserve/mdqc/issues or https://github.com/avpreserve/fixity/issues. User feedback is always welcome directed to info{@}avpreserve{dot}com.

9 Reasons You Understand More About Media Preservation Than You Think

Plus 1 Bad Joke

Whether you are a trained archivist or not, you probably already understand some of the basic concepts of media preservation even if the technical scope if beyond you. You probably understand more than you allow yourself to think.

1. Metadata
Chances are you or someone you tolerate takes digital photos with a camera or phone, or has an iTunes library. Okay, or a Zune. In the case of the latter, I’m sorry. In the case of the former, do the photos downloaded to your computer look like this, making it impossible to find the photo you want:

And for the power nerds, er, power users out there, have you ever scoffed at how the MP3 of Neil Young’s Harvest and Harvest Moon has embedded the same album cover?! These are simple, apparent metadata points. Metadata is not data about data, but information about an object and how you can find it quickly in a messy file structure and how you can/should use it. Some of this information is highly technical or deeply buried, but you already understand how it helps you correctly identify and use content.

2. Obsolescence
VHS-C. ‘Nuff said.

3. Format Selection
Seinfeld, Season 7, Episode 16, “The Shower Head”

When selecting target format for digitization, there is a balance of quality, sustainability, what will perform the work you need it to perform, and what your organization and systems can reasonably handle. Don’t default to low-flow just because of resource savings and don’t be scared off from doing anything at all by the high-end ideal.

4. Presentation
Ever had the experience where you were looking at one of those digital photo frames and thought, “Boy, I’ve maybe let myself go a little of late, but I didn’t think I was ever that…wide. And I thought my head was fairly average-sized”? Most likely (not in my case because my head does need a massive pillow) those are older photos shot or printed at a smaller aspect ratio (3×5, 4×6) and automatically stretched to fit the 16×9 or similar wide frame. Applying the incorrect aspect ratio to digitized film and video can destroy the image (as well as your ego).

5. Duration
Chances are you listen to albums, watch TV, and/or watch movies.

Chances are you don’t like to waste things you’ve paid for.

If you don’t know the duration of some media content, if it’s a commercial work chances are its close to 30, 60 or 120 minutes, a newsclip may be in the 1-5 minute range, and field recordings or b-roll will fill as much of the tape as possible. These inferred durations will be close enough for planning digitization. Exact run times are not a priority in most catalogs. It can be figured out after digitization, by the digitized file.

6. Degradation
That Tupperware lid that doesn’t fit anymore after being microwaved or run through the dishwasher too many times. The plastic pen clip that snaps after you fidget with it too much in meetings. The patio furniture that discolors in the sun. Those cheap plastic bags at the pharmacy they have to double up (please just make them stronger and only give us one…) so they don’t sag and stretch.

Most physical media is made up of plastics and chemical goop. It decays accordingly.

7. Searchability
“Hey Phil, can you put the game on?”
“Sure thing — what channel?”
“I dunno, it’s somewhere in the 500s or 800s. Just flip through them.”

8. Reformatting

9. Selection
I prefer Chubby Hubby to Cherry Garcia, so I will eat it first.

This pint of Chubby Hubby has been opened and this one is new. I will finish the open one first.

This pint will need hot fudge. The second pint is probably fine without it.

I can probably finish these two pints tonight, and then next week I can get two more. The Cherry Garcia can wait, or I can serve it to guests.

And 1.
Two archivists walk into a bar. One of them orders a bourbon on the rocks, the other an IPA. They begin to get into an intense discussion. The bartender says, “Hey, what do you guys do?” One of them says, “We’re archivists.” The bartender says, “Huh. What is that exactly?”

— Joshua Ranger

AVPreserve Releasing New Digital Preservation Tools To Celebrate World Day For Audiovisual Heritage

To celebrate UNESCO World Day for Audiovisual Heritage, AVPreserve is releasing two new digital preservation utilities — Fixity and MDQC — and re-announcing the recent release of the tools Interstitial and AVCC. These utilities are designed to support various aspects of the digital preservation process, from selection and planning to quality control and monitoring. Each tool is free and open to the public. Visit https://www.avpreserve.com/avpsresources/tools/ on October 28th to download the source code and user guides.

Fixity: Fixity is a utility for the documentation and regular review and monitoring of checksums of stored files. Fixity scans a folder or directory, creating a database of the files, locations, and checksums contained within, against which a regular comparative analysis can be run.

MDQC: MDQC reads the embedded metadata of a file or directory and compares it against a set of rules defined by the user, verifying that the technical and administrative specs of the files are correct. This automates and minimizes the time needed to QC large batches of digitized assets, increasing the efficiency of managing digitization projects.

Interstitial: The Interstitial Error Utility compares two streams of the digitized audio captured on separate devices. Irregularities that appear in one stream and not in the other point to issues like Interstitial Errors that relate to samples lost when writing to disc. This greatly decreases the amount of time needed for QC of the integrity of digitized audio. Interstitial is available now.

AVCC: AVCC provides a simple template for documenting audio, video, and film collections, breaking the approach down into a granular record set with recommendations for minimal capture and more complex capture. The record set focuses on technical aspects of the audiovisual object with the understanding that materials in collections are often inaccessible and have limited descriptive annotations. The data entered then feeds into automated reports that support planning for preservation prioritization, storage, and reformatting. More information at https://www.avpreserve.com/news/avpreserve-metro-release-audiovisual-cataloging-reporting-tool/

Chris Lacinak Addressing Audio Engineering Society

AVPreserve President Chris Lacinak will be speaking at this week’s Audio Engineering Society 135th International Convention in New York. Chris is part of the tutorial panel “National Recording Preservation Plan: Preservation Planning” along with David Ackerman (Harvard University), Matt Barton (Library of Congress), and Mark Hood (Indiana University).

Presented in association with the AES Technical Committee on Archiving Restoration and Digital Libraries, the tutorial is a direct response to recommendation 2.2 of the Library of Congress National Recording Preservation Plan stating the need to devise means to assist organizations in conducting comprehensive appraisals of audio collections with the goal of establishing priorities for preservation. The speakers will discuss tools and workflows their institutions have developed for preservation prioritization, and will provide demonstrations and discussion on the various approaches.

AVPreserve continues to develop free and open source tools for preservation planning and collection management (AVCC, Fixity, Interstitial, Cost of Inaction Calculator, and more) that will be a part of the discussion, and we are honored to be involved in helping other institutions such as Indiana University and the Federal Agencies Digitization Guidelines Initiative develop their own tools. We’re excited to be a part of this panel. Hit Chris up at AES if you’re attending or look for the AVPreserve crew at one of the many conferences we attend for more information on these efforts.

Things We Learned At The New York Archivists Round Table Awards Ceremony

*The awards ceremony was started in 1989 to mark the 10th anniversary of NYART. The first award for Innovative Use of Archives was given for a project based in Brooklyn, because at that point in time doing anything in Brooklyn was an innovation.

*The New York Archives Conference originally began as the Lake Ontario Archives Conference with attendees primarily from western New York and Canada. We finally got rid of the Canadians in 1999.

*The only thing you really need to learn to pass kindergarten: Correct identification of farm animals.

*The secret location of the NYART archives. This information is for NYART members only, but did you know that anyone can become a member, either as a professional or a Friend of ART?

*Wang computers.

*If you think the technology of creating digital artworks is hard, think about the technology needed to preserve them.

*NYART awards directly impact the support of archives (i.e., looks good in funding applications).

*Nat King Cole’s wax hands.

*Archivists are humble about and quick to acknowledge the importance of collaboration and the support of colleagues.

*teacharchives.org

*These people are awesome:
Peter Wosh, Director, Archives and Public History Program, NYU – Award for Archival Achievement

New York Archives Conference (Geoffrey P. Williams, University Archivist, SUNY University at Albany accepting) – Award for Outstanding Support of Archives

Rhizome Artbase (Heather Corcoran & Ben Fino-Radin accepting) – Award for Innovative Use of Archives

Brooklyn Historical Society Students and Faculty in the Archives Project (Julie Golia, Public Historian and Robin M. Katz, Outreach & Public Services Archivist accepting) – Award for Educational Use of Archives

Check out the program from the ceremony here. Congratulations to all of the winners! And thanks to all the great presenters and NYART Board for a fun evening!

METRO AND AVPRESERVE LAUNCH PODCAST SERIES ABOUT ARCHIVES AND SPECIAL COLLECTIONS

METRO and AVPreserve announce the launch of More Podcast, Less Process, a new podcast series featuring in-depth and wide ranging interviews with archivists, librarians, preservationists, technologists, and information professionals about interesting work and projects within and involving archives, special collections, and cultural heritage.

With Co-Hosts Jefferson Bailey of METRO and Josh Ranger of AVPreserve, guests will discuss their individual work and its relation to the broader archival enterprise. Topics will cover professional issues that impact archivists and special collections librarians today, including appraisal and acquisition, arrangement and description, outreach and education, collection management, physical and digital preservation, and infrastructure and technology.

Kicking off the series is CSI Special Collections: Digital Forensics and Archives featuring digital archivists Mark Matienzo of Yale University and Donald Mennerich of New York Public Library discussing digital forensics and other matters involved with acquiring and managing digital collections. Episode 1 is now available through iTunes, Internet Archive, and as a direct download via the podcast series’s website.

More Podcast, Less Process is part of METRO’s Keeping Collections programming. Keeping Collections was launched to ensure the sustainability and accessibility of New York State’s archival collections as part of the New York State Archives Documentary Heritage Program. Keeping Collections provides a variety of free and affordable services to any not-for-profit organization in the metropolitan New York area that collects, maintains, and provides access to archival materials.

More Podcast, Less Process is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License and is available via your regular media dissemination channels.

AVPreserve Participating At AMIA, MARAC Conferences

AVPreserve is excited to be participating in the upcoming Association of Moving Image Archivists Annual Conference (November 6th-9th, Richmond, Virginia) and Mid-Atlantic Regional Archives Conference (November 7th-9th, Philadelphia, Pennsylvania) in a number of ways.

* After beginning his term as a newly elected AMIA Board Member, AVPreserve Founder and President Chris Laciank will present about the Cost of Inaction (CoI) tool we have been developing on the panel Magnetic Media Stream: The End of Analog Media: The Cost of Inaction and What You Can Do About It along with fellow panelists Mike Casey (Indiana University) and Marius Snyder (PrestoCentre). The CoI tool produces reporting that show the negative cost impact — based on money already spent and the cost of losing decaying/obsolete assets — of not doing preservation reformatting in the near term.

* Senior Consultant Kara Van Malssen has helped arrange the first ever AMIA / Digital Library Federation HackDay — a full-day event bringing together audiovisual archivists with software and web developers for an intensive day of creativity and problem solving — and then will be on the panel From Zero to DAM! with Miwa Yokoyama (Carnegie Hall) and Eva Radding (Facing History and Ourselves) discussing their efforts to select and implement DAM solutions for digital video assets.

* Consultant Seth Anderson will be on the panel Navigating the Digital Archive: First, Know Thyself discussing the process of developing business cases around digital preservation needs, and then will present the stand alone paper Mastering Your Data: Tools for Metadata Management in AV Archives.

* Inventory Assistant Shira Peltzman will be discussing the preservation challenges of digital cinema packages in her talk Everything You Always Wanted to Know About DCPs (But Were Afraid to Ask) that follows up on her recently completed thesis work at NYU.

* At MARAC, Consultant Alex Duryee will be on the panel Refining Archival Data speaking about using Open Refine to clean up and normalize large data sets, and Consultant Kathryn Gronsbell will be on the panel Archiving, Preservation and Access of Complex Artworks with clients from MOMA, Cornell, and Eyebeam to help discuss their various preservation projects focused on complex media artworks.

Also be sure to keep an eye out for the AVPreserve crew to find out more about our AMIA House Party and MARAC Bar Night. Looking forward to seeing everyone!

AVPreserve President Chris Lacinak Elected To AMIA Board

Congratulations to AVPreserve Founder and President Chris Lacinak upon his election to the Association of Moving Image Archivists (AMIA) Board of Directors. The Board meets regularly to help “ensure that AMIA’s mission is fulfilled and to provide strategic leadership to move the association forward”. Chris is a long-time member of AMIA, regularly presenting at the Annual Conference and contributing to various committees, as well as the audiovisual preservation knowledge base overall. His two-year term will begin at next month’s conference in Richmond, VA, where he looks forward to being even more involved as a participant in directing the continuing development of the field and the resources needed to do our work. Congrats as well to Tom Regal (Director of the Board), Jacqueline Stewart (Director of the Board), and Caroline Frick (President of the Board) on their selections as well.

AVPreserve Collaborates On First AMIA Hack Day

The 2013 Association of Moving Image Archivists Annual Meeting will feature the first ever Digital Library Federation/AMIA Hack Day to take place on November 6th in Richmond, Virginia. Along with co-conspirators Lauren Sorensen (Bay Area Video Coalition) and Steven Villereal (University of Virginia), AVPreserve Senior Consultant Kara Van Malssen is helping to organize the event.

Hack Days are designed to provide a collaborative workspace where people with different skill sets can come together to develop technological solutions to specific problems in a condensed amount of time. The AMIA event will be a unique opportunity for practitioners and managers of digital audiovisual collections to join with developers and engineers for an intense day of collaboration to develop practical solutions around digital audiovisual preservation and access.

To be successful these events require the input of people who care for collections in addition to people with strong technical and programming skills. There is a lot of room to explore and create in the area of digital video preservation, so we hope you’ll come help! The event is free, and further information can be found on the AMIA / DLF Hack day Registration Form. See you in Richmond!

AVPreserve Celebrates Archives Week

October 6th-12th marks the 25th annual Archives Week in New York City, a celebration of the richness and diversity of archival collections across the Metropolitan region. The Archivists Round Table of Metropolitan New York is a major supporter of the event and, as usual, has helped organize a number of open houses, lectures, workshops and behind-the-scenes tours of archives throughout the city — all of which are free and open to the public.

AVPreserve is pleased to be a part of the festivities, both as attendees and participants. Kicking things off, Senior Consultant Joshua Ranger will be presenting at the Disaster Preparedness Symposium taking place at the Center for Jewish History on October 7th. Josh will be talking about the importance of metadata and documentation for disaster preparedness and recovery, both in cases of catastrophic loss and loss of institutional knowledge. The presentation will cover several case studies of AVPreserve projects, including the Eyebeam recovery effort after Hurricane Sandy and the inventory of the NJN Public Television collection after the station closed. The panel will also feature talks from Laura McCann and Rachel Searcy (NYU Libraries), Shae A. Trewin (The Image Permanence Institute), and Martin Tzanev (WITNESS).

Later in the week is the NYART Awards Ceremony, honoring the work of archivists and those who support archival programs. AVPreserve President Chris Lacinak has been a part of the group organizing this event, which will present awards to Rhizome (Innovative use of Archives), New York Archives Conference (Outstanding Support of Archives), Peter Wosh (Archival Achievement), and the Brooklyn Historical Society (Educational Use of Archives). Congratulations to all of the winners!

And finally, AVPreserve and METRO will (finally) release the first episode of the archives podcast “More Podcast, Less Process”. This first iteration features Don Mennerich (NYPL) and Mark Matienzo (Yale) discussing digital forensics and other aspects of caring for born digital collections. Look for the official release next week, and go visit your local archive or historical society!

« Previous Page — Next Page »