2002 Dan Mueth &legal; Dan Mueth

muet@alumni.uchicago.edu

Writing ScrollKeeper OMF Files V1.0 Feb 2002 Dan Mueth muet@alumni.uchicago.edu Writing ScrollKeeper OMF Files V&manrevision; &date; Malcolm Tredinnick malcolm@commsecure.com.au Brought up to date for recent releases. This manual describes how to write OMF files for use with ScrollKeeper version &appversion;. ScrollKeeper OMF Files OMF Files Writing An OMF file provides metadata for one or more documentation files. This metadata includes various useful information which may be used for cataloging the documents, including the author, title, document file location, version, locale, and various other information. ScrollKeeper uses this metadata, along with metadata extracted directly from documents, to create a documentation catalog which can be used by documentation and help browsers. This document describes the form and fields of an OMF file. It is meant to be a simple reference for an OMF file author at any level. Feel free to jump around through the sections to quickly find the information you need. The Open Metadata Framework (OMF) is an open standard for describing documentation metadata, based on the Dublin Core metadata set. The OMF was created by the Metalab project to catalog open source documentation. The OMF is described by only 16 elements to keep it simple and easy to use. OMF Standard Elements ScrollKeeper OMF Standard Elements There are 16 standard OMF elements. The authoritative definition of the OMF is on the OMF element web page. It is reproduced here (as of Feb 14, 2002) for your convenience. The 16 standard OMF elements, along with their obligation, repeatability, and standard attributes are summarized below: NUMBER LABEL OBLIGATION REPEATABILITY ATTRIBUTES 1. creator mandatory repeatable (none) 2. maintainer optional repeatable (none) 3. contributor optional repeatable (none) 4. title mandatory repeatable (none) 5. date mandatory not repeatable (none) 6. version optional repeatable id, date, description 7. subject optional repeatable (none) 8. description optional repeatable (none) 9. type optional repeatable (none) 10. format optional repeatable dtd, mime 11. identifier optional repeatable (none) 12. source optional repeatable (none) 13. language optional repeatable (none) 14. relation optional repeatable (none) 15. coverage optional repeatable geographic, distribution, kernel, architecture, os 16. rights optional repeatable type, license, license.version, holder The elements are defined as follows: NUMBER LABEL DEFINITION 1. creator The person or organization primarily responsible for creating the intellectual content of the resource. CREATOR should appear in RFC822 format (http://info.internet.isi.edu:80/in-notes/rfc/files/rfc822.txt). Preferred format: mailname@site.domain.top (Full Name) 2. maintainer The person or organization responsible for publishing the resource in its current form. If left blank, this value defaults to CREATOR. (RFC822 format) 3. contributor A person or organization not specified in a CREATOR or MAINTAINER element who has made significant intellectual contributions to the resource but whose contribution is secondary to any person or organization specified in a CREATOR or MAINTAINER element. (RFC822 format) 4. title The name given to the resource by the CREATOR or MAINTAINER. 5. date The date on which the resource was made available in its current form. (Recommended best practice is an 8 digit number in the form YYYY-MM-DD as defined in http://www.w3.org/TR/NOTE-datetime, a profile of ISO 8601. In this scheme, the date element 1994-11-05 corresponds to November 5, 1994.) 6. version VERSION is a multifaceted element, consisting of three attributes. VERSION.id consists of a string or number that distinguishes the current revision of the resource from other revisions. VERSION.date records the date the resource was made available in the form specified by VERSION.id. (Recommended best practice is an 8 digit number in the form YYYY-MM-DD as defined in http://www.w3.org/TR/NOTE-datetime, a profile of ISO 8601. In this scheme, the date element 1994-11-05 corresponds to to November 5, 1994.) VERSION.description summarizes revisions that distinguish VERSION.id from other versions of the resource. Repeated instances of VERSION comprise the revision history of a resource. 7. subject The topic of the resource. Typically, this element employs keywords that summarize the subject or content of the resource. 8. description A textual description of the content of the resource (e.g., an abstract, contents note). 9. type The category of the resource. To promote consistency, TYPE should be selected from the following list: HOWTO, mini-HOWTO, user's guide, administrator's guide, programmer's guide, installation guide. 10. format Details about the implementation of the resource. FORMAT accomodates two attributes: FORMAT.DTD and FORMAT.MIME. FORMAT.DTD describes the document type definition used in the resource (if any). FORMAT.MIME should be expressed as a MIME type, as defined in RFC 2046 (http://info.internet.isi.edu:80/in-notes/rfc/files/rfc2046.txt). e.g.: type/subtype. 11. identifier A specification of a unique ID by which the resource may be identified and from which the resource may be retrieved. Entries for this field should contain a valid URL which returns the resource in question. For example: http://www.ldp.org/docs/howto/nfs 12. source A specification of any previous or alternative publication of the resource in its current form (e.g. a larger work from which the resource is extracted, such as a chapter taken from a book). SOURCE may include a URL, ISBN or similar device. 13. language Language(s) of the intellectual content of the resource. Where practical, the content of this field should coincide with RFC 1766. See: http://ds.internic.net/rfc/rfc1766.txt 14. relation A URL that points to the IDENTIFIER element of another resource. Each instance of RELATION links the resource to other resources of similar domain or style. 15. coverage A multifaceted description of the resource's intellectual scope that consists of five attributes. COVERAGE.geographic identifies regional specificity of the resource. Where practical COVERAGE.geographic should be expressed as an ISO 3166-compliant string of two characters (See http://www.w3.org/International/O-misc-iso3166.html). COVERAGE.distribution identifies a Linux distribution explicitly specified in the resource. COVERAGE.kernel identifies the kernel version treated in the resource. COVERAGE.architecture identifies hardware described in the resource. COVERAGE.os identifies an operating system explicitly specified in the resource. 16. rights An indication of the copying policy under which the resource is distributed. Four attributes comprise the RIGHTS element. RIGHTS.type identifies the name of the resource's distribution license. To promote consistency, RIGHTS.type should be selected from the following list: GNU GPL, GNU LGPL, BSD, X Consortium, Artistic, MPL, QPL, IBMPL, ASPL, libpng, zlib, IJG JPEG, OPL, ORL. RIGHTS.license identifies the URL for the license referenced in RIGHTS.type. RIGHTS.license.version identifies the version number of the resource's license. RIGHTS.holder identifies the person or organization who holds the rights for the resource described in RIGHTS.license. Please note that these are the standard OMF elements. ScrollKeeper makes a couple small extensions to the OMF, as described in the next section. OMF ScrollKeeper Extensions ScrollKeeper OMF Extensions ScrollKeeper makes a couple small extensions to the standard OMF in order to gain some useful benefits for help and documentation browsers. This section describes these extensions and what they are used for. All OMF files which are intended to be used by ScrollKeeper should include these extensions. OMF seriesid ScrollKeeper OMF seriesid ScrollKeeper has added another attribute, "seriesid", to OMF element 14, "relation". The seriesid is used to specify a group (or series) of documents which are related. For example, multiple versions, formats, and translations of a single document are considered to be in the same series. Thus, any new version, translation, etc. of a pre-existing document should inherit the seriesid of the original document. ScrollKeeper uses the seriesid to identify documents which are equivalent to each other. For example, ScrollKeeper can merge the contents lists from two locales. ScrollKeeper uses the seriesid to guarantee that a given document is not listed twice, once in each locale. ScrollKeeper scrollkeeper-gen-seriesid To generate a new unique seriesid, use the scrollkeeper-gen-seriesid command. OMF Category ScrollKeeper OMF Category ScrollKeeper has also added an attribute to the "subject" element, called "category". The category element is used to classify the topic of the document according to subject. ScrollKeeper uses this attribute to place the document in one or more positions in the Table of Contents, which lists all the documents sorted by subject. The category list is heirarchical, consisting of main categories, subcategories, subsubcategories, etc. The category levels are seperated with pipes (|). For example, a document which is part of the core desktop of GNOME has a category of "GNOME|Core Desktop". Note that there is a controlled list of categories. The categories are installed in an XML file called scrollkeeper_cl.xml. The category file for the C locale can typically be found at /usr/share/scrollkeeper/Templates/C/scrollkeeper_cl.xml. There is an XSLT stylesheet (categories.xml) which will convert this into an HTML file to make it more readable. To do the conversion, use xsltproc -o categories.html /usr/share/scrollkeeper/stylesheets/categories.xsl /usr/share/scrollkeeper/Templates/C/scrollkeeper_cl.xml. Note that the exact paths to categories.xsl and scrollkeeper_cl.xml may vary depending on distribution. OMF Language Code ScrollKeeper OMF Language Code The standard OMF specifies the "language" element as optional and as having no attributes. For ScrollKeeper to relate documents with the locale of the user, the locale of the documents must be specified. This is done by using a new attribute, "code", of the "language" element. The language code should be a standard locale. OMF URL ScrollKeeper OMF URL The standard OMF specifies the "identifier" element as optional and as having no attributes. For ScrollKeeper to work properly, the "identifier" element is mandatory and must have the "url" attribute which points to a copy of the file on a locally accessible filesystem. OMF Mime Type ScrollKeeper OMF Mime Type The standard OMF specifies the "format" element as optional and as having an optional attribute, "mime". For ScrollKeeper to extract metadata from DocBook documents, the correct mime type must be provided. Common mime types include "text/xml", "text/sgml", and "text/html". ScrollKeeper will not attempt to extract extra metadata (such as the table of contents or index) from documents in other mime types. ScrollKeeper will catalog documents in any mime type according to the information provided in the OMF file. OMF Examples Minimal In this section, we present the simplest OMF file which obeys the OMF scheme and which allows ScrollKeeper &appversion; to properly function. Note that while the OMF is not XML-specific, ScrollKeeper requires that OMF files be in XML. Here is an example of the most minimal OMF file, containing metadata for only one document: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE omf PUBLIC "-//OMF//DTD Scrollkeeper OMF Variant V1.0//EN" "http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeeper-omf.dtd"> <omf> <resource> <creator> kenny@gnome.org (Kenny Graunke) </creator> <title> Gfloppy Manual </title> <date> 2001-11-23 </date> <subject category="GNOME|Core Desktop"/> <format mime="text/xml"/> <identifier url="file://usr/share/doc/gfloppy/gfloppy.xml"/> <language code="C"/> <relation seriesid="01ddeea4-0a42-11d6-9cf9-ee43c422358d"/> </resource> </omf> All OMF files which are to be used with ScrollKeeper should have all of these elements. Make sure that the category is valid, chosen from the controlled list. The seriesid should be unique from all other documents except different versions, locales, or formats of the same document. Multiple documents can be described in a single OMF file by placing multiple "resource" objects in the file. Note that the URL is typically not known until build or install time. Thus, a placeholder or estimated location is often used in the original OMF file. When the package is built, the correct file location is automatically substituted into the OMF file using scrollkeeper-preinstall. Note that you should always include the document's encoding in the XML declaration since there is no good way to detect a document's encoding. ScrollKeeper uses the libxml libraries, which supports the following encodings: UTF-8, UTF-16, ISO-8859-1 (ISO-Latin-1), ASCII, HTML. For more information, see Libxml Encoding Support Web Pages. OMF Examples Complete A complete OMF file will include all of the relevant elements and attributes specified by the OMF scheme, along with the extensions required by ScrollKeeper. It is a good idea to include these optional elements and attributes since ScrollKeeper will provide support for many of them in the future. Here is an example of a complete OMF file, containing metadata for almost all of the OMF elements for only one document: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE omf PUBLIC "-//OMF//DTD Scrollkeeper OMF Variant V1.0//EN" "http://scrollkeeper.sourceforge.net/dtds/scrollkeeper-omf-1.0/scrollkeeper-omf.dtd"> <omf> <resource> <creator> kenny@gnome.org (Kenny Graunke) </creator> <maintainer> kenny@gnome.org (Kenny Graunke) </maintainer> <contributor> jrb@redhat.com (Jonathan Blandford) </contributor> <contributor> hobbit@aloss.ukuu.org.uk (Telsa Gwynne) </contributor> <title> Gfloppy Manual </title> <date> 2001-11-23 </date> <version identifier="1.0.2" date="2001-11-23" description="Updated for Gfloppy 1.0.1"/> <subject category="GNOME|Core Desktop"/> <description> User manual for the Gfloppy application. </description> <type> user's guide </type> <format mime="text/xml" dtd="-//OASIS//DTD DocBook XML V4.1.2//EN"/> <identifier url="file://usr/share/doc/gfloppy/gfloppy.xml"/> <language code="C"/> <relation seriesid="01ddeea4-0a42-11d6-9cf9-ee43c422358d"/> <rights type="GNU FDL" license.version="1.1" holder="Kenny Graunke"/> </resource> </omf> OMF Testing an OMF File ScrollKeeper Testing an OMF File Once you've written an OMF file, you should test it to make sure that it works. To test an OMF file: Make sure you have ScrollKeeper installed. You can always get it from http://sourceforge.net/projects/scrollkeeper. The stable releases have an even minor revision numer, such as 0.2.x or 0.4.x. The unstable, development releases have odd minor revision numbers such as 0.1.x and 0.3.x. You probably want the latest stable release. Next you should make sure the url specified in the identifier element is accurate and points to a valid document. Now you should have ScrollKeeper process the OMF file and the document it points to. You can do this without having root access or affecting the primary ScrollKeeper catalog on your system by using the -p flag to scrollkeeper-install. For example: scrollkeeper-install -v -p testomf_db_dir testomf.omf This will read the OMF file called testomf.omf and create a new ScrollKeeper catalog in the directory testomf_db_dir. The -v flag indicates that it should be run in verbose mode, sending all warnings and errors to standard output. If you are not logged in as root, you will see an error stating that you do not have permission to make an entry in the log file. You can ignore this error. If you do not see any other errors, then no errors or warnings occurred while processing the OMF file or document. If you are curious or would like to confirm that the catalog files look alright, you may look at them directly. This step isn't necessary, but may be interesting or help debugging in the case that errors occur. The full explanation of these files are beyond the scope of this document. Briefly: testomf_db_dir/scrollkeeper_docs should contain a single line for each resource entry in the OMF file. The first document will be given an identifying number in the second column. This will be 0 since it is the first document. testomf_db_dir/TOC/0 should contain the table of contents for the first document. If the document is not in DocBook/XML, the table of contents will not exist. testomf_db_dir/index/0 should contain the index for the first document. If the document is not in DocBook/XML or does not have indexterms in it, the index will not exist. testomf_db_dir/index/locale/scrollkeeper_cl.xml should be the standard ScrollKeeper contents list for the locale, taken directly from the template for the locale, with a single entry for each document listed in the OMF file. You can verify that each document appears in the correct category. Now you are finished and can delete the temporary catalog testomf_db_dir. If you have ScrollKeeper version 0.3.7 or later, you can validate your OMF file against the ScrollKeeper-OMF DTD. This will tell you if your OMF file follows the rules specified in the DTD in terms of structure of the XML file. Note that ScrollKeeper 0.3.7 and later will ignore any OMF files which do not validate against the DTD. To validate your OMF file, issue a command similar to: xmllint --noout --valid foo-C.omf Thanks to the magic of XML catalogs, it should find the DTD specified in the OMF file on your computer (instead of going onto the internet to find the DTD) and validate the OMF file. Once you have completed and tested your OMF file, you are ready to use it. If you are packaging the OMF file with documentation in a package, see the examples and documentation in the package scrollkeeper-example1, available from http://sourceforge.net/projects/scrollkeeper. OMF Registering Locally ScrollKeeper Registering an OMF File Locally If you wish to register the OMF file and document in your local ScrollKeeper catalog, use scrollkeeper-install or scrollkeeper-update. A typical system-wide installation involves copying the OMF file into one of the directories specified by OMF_DIR in /etc/scrollkeeper.conf, or a subdirectory of any of these directories. Running scrollkeeper-update as root will update the catalog with any new or updated OMF files in these directories. Conversely, if any OMF files were removed since the last scrollkeeper-update, their entries in the catalog will be removed. Logs of the actions and any warnings or errors are written to /var/log/scrollkeeper.log. You should now be able to view the updated catalog using any ScrollKeeper-compliant documentation browser. Note that you may need to restart some help browsers to see the changes. OMF Resources ScrollKeeper OMF Resources For more information on ScrollKeeper and the OMF: OMF web pages. The OMF web pages contain the latest version of the official OMF scheme. ScrollKeeper web pages. The ScrollKeeper web pages contain all the latest news, releases, and documentation about ScrollKeeper. "ScrollKeeper: Open Source Document Management". This article on xml.com gives a nice overview of ScrollKeeper and presents an interesting perspective on its place in Unix/Linux environments. db2omf. db2omf extracts OMF metadata from DocBook documents. It can be obtained from the LDP (Linux Documentation Project).