Meta-Data-in-Ogg

Current items of interest

The following are points relevant to Ogg metadata streams or an XML-in-Ogg spec.

Referring to a logical stream within a physical ogg stream from an XML document (i.e. via a URI).

Most likely a client side operation. This suggests the use of a fragment identifier, however that seems to exclude use of a fragment to point to objects-within-objects (e.g. an ID within a given XML stream). Using a split fragment identifier: #serialno/objid to refer to objects-within-objects may work, but possibly conflicts with the CMML use of fragment identifiers for temporal locators and clip names. It may suffice to reserve IDs of the form xxxxxxxx (8 digit hexadecimal serial numbers, possibly with a prefix; maybe ogg, serial, 0x or something) within ogg files. Need to get in touch with the CMML/Annodex people, since they probably understand the issues better.

The OASIS OpenDocument 1.0 Standard describes a method for resolving relative URIs within packages (17.5, page 686). This method has a possible conflict when a name referred to in a package is the same as that of a resource in the same directory as the package, however it is unlikely a document author would want to include such a reference and URIs specifying a protocol, authority or absolute path are not affected.

Aside: the clip and temporal fragment specifiers used in CMML are potentially very useful to an Ogg metadata stream.

It may be useful if an XML stream encoder is able to translate some kind of entity reference to a serial number (or perform similar magic). The idea is that the XML generator can say 'the Vorbis stream' or 'the audio stream' and the encoder will replace the reference with the correct fragment identifier. This might be heuristic; choose the first matching file, or it might involve specifying which out of the matching streams to choose (first encountered audio, second encountered audio, e.t.c.).

Encoding of XML pages

The XML 1.0 and 1.1 Recommendations allow XML documents to have a wide range of encodings. Among the problems caused by allowing XML-in-Ogg to use the allowed range of encodings is the byte order mark required for UTF-16 and allowed for UTF-8. This would lead to several possible magic-byte sequences for the header. The allowed EBCDIC encoding would also introduce extra possible sequences. Appendix F of the XML 1.1 Recommendation (F.1 Detection Without External Encoding Information) has a frightening variety of possible sequences for the start of an XML document. It should also be noted that conforming XML parsers are not required to understand most of the allowed encodings: only UTF-8 and UTF-16 must be supported.

In creating XML streams for Ogg metadata we act as document authors. In adding them we may need to do some interpetation to delimit stanzas, but are still not processors in the XML sense. In extracting them we are working within a possibly stricter XML-in-Ogg specification. It is probably acceptable to require UTF-8 encodings for XML-in-Ogg (analogous to the UTF-8 requirement for Vorbis headers). Capable XML inserting applications may then translate from the document's encoding to UTF-8 on insertion.

There is one important start sequence (with no byte order mark), 3C 3F 78 6D; '<?xm'. Appendix F of XML 1.1:

UTF-8, ISO 646, ASCII, some part of ISO 8859, Shift-JIS, EUC, or any other 7-bit, 8-bit, or mixed-width encoding which ensures that the characters of ASCII have their normal positions

As far as a potential stanza splitting tool needs to know this means we can understand enough of the XML to not need to worry about the encoding. It may be acceptable to limit to the described 8-bit encodings.

An alternative to the above is the use of an XML codec with a seperately defined magic header as in CMML. This is unattractive as the header could only specify that XML is contained, not a possible application. In CMML the CMML magic header marks a stream with a definite use and interpretation. For generic XML, document processing is required to determine the use (and correct destination) of an XML stream. All we can say until then is that it is an XML stream, so it is reasonable to choose allowed documents such that the document declaration '<?xml' can act as the stream identifier.

Content copyright Ian Malone 2005, except for the logos on the front page and where otherwise noted. Stylesheets and code cribbed from various sources on the web, see links section for a few.