($and/ <==do note remove - switch for preventing MathML interpretation of dollar signs)

About

This page describes a proposed serialization and display format for XDI RDF called X3.

Motivations

Examples

X3 Simple

The first example, showing typical XDI RDF business card data, illustrates a version of X3 optimized for both human-readability and writeability (see the X3 Simple formatting rules below). Note that X3 Simple is not valid standard X3 because it does not display the square brackets used as X3 delimiters. However it is trivial to transform into valid standard X3, as described in a separate section below.

Note that for purposes of illustration, this X3 Simple example includes data serialized in both XML and JSON formats. See Typed Data Formats below.

 =(http://equalsdrummond.name)
        $is
                =drummond
                =drummond.reed
                =!F83.62B1.44F.2813
        $is$a
                +person
                @oasis+chair
        $has
                +home
                +work
                +friend
                @cordance
                @parity
        +email
                /
                        =drummond+home
                                +email
                        =drummond+work
                                +email
        +email+signature$a$xml
                \
                <?xml version="1.0" encoding="ISO-8859-1"?>
                <signature xmlns="http://example.com/ns">
                        <name>Drummond Reed</name>
                        <affiliation>Cordance</affiliation>
                        <affiliation>Parity</affiliation>
                        <affiliation>OASIS</affiliation>
                </signature>
                \
        +picture$a$json
                \
                {
                "Image": {
                        "Width":  800,
                        "Height": 600,
                        "Title":  "Drummond at Graduation",
                        "Thumbnail": {
                                "Url":    "http://www.example.com/image/481989943",
                                "Height": 125,
                                "Width":  "100"
                                },
                        "IDs": [116, 943, 234, 38793]
                        }
                }
                \
        =web*markus
                =drummond+friend
 =drummond+home
        +email
                "drummond@example.com"
 =drummond+work
        +email
                /
                        =drummond@cordance
                                +email
                        =drummond@parity
                                +email
 =drummond@cordance
        +email
                "drummond.reed@cordance.example.com"
 =drummond@parity
        +email
                "drummond.reed@parity.example.com"
 =drummond+friend
        $is$a
                $contract
                @identity.commons$contract
                @identity.commons+personal$contract
        $get
                /
                        =drummond+home
                        =drummond+work

X3 Whitespace

X3 Whitespace is identical to X3 Simple except it includes the opening and closing square brackets that make it valid X3. This makes it almost as readable as X3 Simple, but slightly harder to write because the author must keep track of matching square brackets. Like X3 Simple, X3 Whitespace is also trivial to transform into standard X3 per the rules in a separate section below.

Here is the same example above shown in X3 Whitespace format.

 [=(http://equalsdrummond.name)
        [$is
                [=drummond]
                [=drummond.reed]
                [=!F83.62B1.44F.2813]
        ]
        [$is$a
                [+person]
                [@oasis+chair]
        ]
        [$has
                [+home]
                [+work]
                [+friend]
                [@cordance]
                [@parity]
        ]
        [+email
                [
                        [=drummond+home
                                [+email]
                        ]
                        [=drummond+work
                                [+email]
                        ]
                ]
        ]
        [+email+signature$a$xml
                ["
                <?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>
                <signature xmlns=\"http://example.com/ns\">
                        <name>Drummond Reed</name>
                        <affiliation>Cordance</affiliation>
                        <affiliation>Parity</affiliation>
                        <affiliation>OASIS</affiliation>
                </signature>
                "]
        ]
        [+picture$a$json
                ["
                {
                \"Image\": {
                        \"Width\":  800,
                        \"Height\": 600,
                        \"Title\":  \"Drummond at Graduation\",
                        \"Thumbnail\": {
                                \"Url\":    \"http://www.example.com/image/481989943\",
                                \"Height\": 125,
                                \"Width\":  \"100\"
                                },
                        \"IDs\": [116, 943, 234, 38793]
                        }
                }
                "]
        ]
        [=web*markus
                [=drummond+friend]
        ]
 ]
 [=drummond+home
        [+email
                ["drummond@example.com"]
        ]
 ]
 [=drummond+work
        [+email
                [
                        [=drummond@cordance
                                [+email]
                        ]
                        [=drummond@parity
                                [+email]
                        ]
                ]
        ]
 ]
 [=drummond@cordance
        [+email
                ["drummond.reed@cordance.example.com"]
        ]
 ]
 [=drummond@parity
        [+email
                ["drummond.reed@parity.example.com"]
        ]
 ]
 [=drummond+friend
        [$is$a
                [$contract]
                [@identity.commons$contract]
                [@identity.commons+personal$contract]
        ]
        [$get
                [
                        [=drummond+home]
                        [=drummond+work]
                ]
        ]
 ]

X3 Standard

X3 Standard is X3 Whitespace with all insignificant whitespace removed. This is the format intended for transmission over the wire. It is the least human-readable but most machine-readable form.

Here is the same example above shown in X3 Standard format. Note: line breaks were inserted manually into this example so it would be readable on this wiki page.

[=(http://equalsdrummond.name)[$is[=drummond][=drummond.reed][=!F83.62B1.44F.2813]][$is$a[+person][@oasis+chair]]
[$has[+home][+work][+friend][@cordance][@parity]][+email[[=drummond+home[+email]][=drummond+work[+email]]]]
[+email+signature$a$xml["<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?><signature xmlns=\"http://example.com/ns\">
<name>Drummond Reed</name><affiliation>Cordance</affiliation><affiliation>Parity</affiliation><affiliation>OASIS
</affiliation></signature>"]][+picture$a$json["{\"Image\": {\"Width\":  800,\"Height\": 600,\"Title\":  
\"Drummond at Graduation\", \"Thumbnail\": {\"Url\": \"http://www.example.com/image/481989943\",\"Height\": 125,\"Width\": \"100\"},
\"IDs\": [116, 943, 234, 38793]}}"]][=web*markus[=drummond+friend]]][=drummond+home[+email["drummond@example.com"]]]
[=drummond+work[+email[[=drummond@cordance[+email]][=drummond@parity[+email]]]]][=drummond@cordance[+email
["drummond.reed@cordance.example.com"]]][=drummond@parity[+email["drummond.reed@parity.example.com"]]]
[=drummond+friend[$is$a[$contract][@identity.commons$contract][@identity.commons+personal$contract]][$get
[[=drummond+home][=drummond+work]]]]

X3J (X3 for JSON)

Many applications are now using JSON rather than XML as their primary data representation language. X3 Whitespace is similar to JSON but cannot be parsed as JSON. By contrast, X3J is native JSON.

The above X3 Whitespace example in X3J would be:

        {
        "=(http://equalsdrummond.name)": 
                {
                                "$is": ["=drummond", "=drummond.reed", "=!F83.62B1.44F.2813"],
                                "$is$a": ["+person", "@oasis+chair"],
                                "$has": ["+home", "+work", "+friend", "@cordance", "@parity" ],
                        "+email": 
                                {
                                        "_context": 
                                                {
                                                        "=drummond+home": "+email",
                                                        "=drummond+work": "+email"
                                                }
                                },
                        "+email+signature$a$xml":
                                <signature xmlns="http://example.com/ns">
                                        <name>Drummond Reed</name>
                                        <affiliation>Cordance</affiliation>
                                        <affiliation>Parity</affiliation>
                                        <affiliation>OASIS</affiliation>
                                </signature>,
                        "+picture$a$json":
                                {
                                "Image": {
                                        "Width":  800,
                                        "Height": 600,
                                        "Title":  "Drummond at Graduation",
                                        "Thumbnail": {
                                                "Url":    "http://www.example.com/image/481989943",
                                                "Height": 125,
                                                "Width":  "100"
                                                },
                                        "IDs": [116, 943, 234, 38793]
                                        }
                                },
                        "=web*markus": "=drummond+friend"
                },
        "=drummond+home": 
                {
                        "+email":  "drummond@example.com"
                },
        "=drummond+work":
                {
                        "+email":
                                {
                                        "_context": 
                                                {
                                                        "=drummond@cordance": "+email",
                                                        "=drummond@parity": "+email"
                                                }
                                }
                },
        "=drummond@cordance":
                {
                        "+email": "drummond.reed@cordance.example.com"
                },
        "=drummond@parity":
                {
                        "+email": "drummond.reed@parity.example.com"
                },
        "=drummond+friend":
                {
                        "$is$a": ["$contract", "@identity.commons$contract", "@identity.commons+personal$contract"],
                        "$get":  
                                {
                                        "_context": 
                                                {
                                                        "=drummond+home": {},
                                                        "=drummond+work": {}
                                                }
                                }
                }
        }

Examples of Comments

Like XML, X3 can contain human-readable comments that are not normatively part of the XDI graph, but which can be carried within the X3 in all formats and accessed and manipulated programmatically.

A comment may be place before ("pre-comment") or after ("post-comment") any X3 subject, predicate, or object. The following extreme examples illustrate all three options.

X3 Simple

<-- pre-sub -->
=i-name.first.example   <-- post-sub -->
        <-- pre-pred -->
        +test   <-- post-pred -->
                <-- pre-X3 -->
                /       <-- post-X3 -->
                        <-- pre-sub -->
                        =i-name.second.example  <-- post-sub -->
                                <-- pre-pred -->
                                $is     <-- post-pred -->
                                        <-- pre-obj -->
                                        =!i-number.second.example       <-- post-obj -->

X3 Whitespace

[<-- pre-sub -->
=i-name.first.example   <-- post-sub -->
        [<-- pre-pred -->
        +test   <-- post-pred -->
                [<-- pre-X3 -->
                        [<-- pre-sub -->
                        =i-name.second.example  <-- post-sub -->
                                [<-- pre-pred -->
                                $is     <-- post-pred -->
                                        [<-- pre-obj -->
                                        =!i-number.second.example       <-- post-obj -->]
                                ]
                        ]       <-- post-X3 -->
                ]
        ]
]

X3 Standard

Note: line breaks were inserted manually into this example so it would be readable on this wiki page.

[<-- pre-sub -->=i-name.first.example<-- post-sub -->[<-- pre-pred -->+test<-- post-pred -->
[<-- pre-X3 -->[<-- pre-sub -->=i-name.second.example<-- post-sub -->[<-- pre-pred -->$is<-- post-pred -->
[<-- pre-obj -->=!i-number.second.example<-- post-obj -->]]]<-- post-X3 -->]]]

X3J

Comments in X3J are standard Javascript comments so use /* to start a comment and */ to end a block comment or use // to comment everything after to the end of the line.

Syntax

Core

Using RFC 4234 syntax, the core ABNF for X3 is just six lines:

X3              = *( "[" sub *( "[" pred *( "[" obj "]" ) "]" ) "]" )
sub             = [ comment ] xri-reference [ comment ]
pred            = [ comment ] xri [ comment ]
obj             = [ comment ] ( xri-reference / literal / X3 ) [ comment ]
literal         = """ *char """
comment         = "<--" *c-char "-->"

The xri and xri-reference rules are defined in the proposed ABNF for XRI 3.0.

Characters

The ABNF rule for char is based on the same ABNF rule in JSON. This approach uses the backslash character for escaping. The only difference between the JSON and X3 escaped character sets is the addition of opening and closing square brackets.

char            = unescaped / escaped
unescaped       = %x20-21 / %x23-5A / %x5E-10FFFF
escaped         = escape (
                %x22 /          ; "    quotation mark  U+0022
                %x5B /          ; [    open sq bracket U+005B
                %x5C /          ; \    reverse solidus U+005C
                %x5D /          ; ]    clos sq bracket U+005D
                %x2F /          ; /    solidus         U+002F
                %x62 /          ; b    backspace       U+0008
                %x66 /          ; f    form feed       U+000C
                %x6E /          ; n    line feed       U+000A
                %x72 /          ; r    carriage return U+000D
                %x74 /          ; t    tab             U+0009
                %x75 4HEXDIG )  ; uXXXX                U+XXXX
escape          = %x5C          ; \

Comment Characters

The ABNF for c-char rule simply adds opening and closing angle brackets to the escaped character set:

c-char          = c-unescaped / c-escaped / escaped 
c-unescaped     = %x20-21 / %x23-3B / %x3D / %x3F-5A / %x5E-10FFFF
c-escaped       = escape (
                %x3C /          ; <    open an bracket U+003C
                %x3E )          ; <    clos an bracket U+003E

Typed Data Formats

X3 is designed to be able to carry data encoded in any popular text-based interchange format, while still retaining the ability for the data object to be identified, described, and linked using X3.

To do this, X3 uses typed data formats – a way of tagging X3 data with a data format identifier that tells the X3 processor how to handle that form of data.

X3 data format tags use the proposed XDI $a identifier. The $a identifier, when used at the start of an XDI predicate, identifies that the datatype belongs to the XDI type ontology. The $a identifier, when used as a modifier of other XDI predicates, indicates the data format type.

The first two proposed data format types are shown in the table below.

Type Identifier

Format

$json

ABNF from RFC 4627

$xml

ABNF from XML 1.0

Encoding

Encoding of X3 is identical to JSON, i.e., Unicode, with the default encoding in UTF-8.

Mime Type

The proposed mime type for X3 is text/x3. If it is necessary to use an application mime type, the proposed mime type is application/x3.

Formatting Rules

X3 Whitespace

The rules for displaying X3 in X3 Whitespace format (see example above) are:

  1. Place each X3 subject, predicate, object on a new line.
  2. Indent each X3 predicate one tab from the parent subject.
  3. Indent each X3 object one tab from the parent predicate.
  4. Put each closing bracket that immediately follows another closing bracket on a new line, indented to match its paired opening bracket.
  5. If an object is a multi-line literal, put the opening square bracket and opening quote on the first line, put the closing quote and closing square bracket on the last line, and put the literal on its own set of lines in between, maintain the same indent for all lines.
  6. If an object is a literal in a typed data format, display it as a multi-line literal (above), and observe all newlines and indents relative to the starting line.
  7. If an object is X3, place the opening square bracket alone on the object line, begin the new X3 block on a new line, indent the entire X3 block one additional tab, and iterate the rules above.
  8. Put each comment that appears after an opening square bracket immediately after the opening square bracket, then place the subject/predicate/object on a new line with the same indent as the line containing the comment.

  9. Put each comment that appears before a closing square bracket on the same line as the subject/predicate/object it describes, indented one tab.

X3 Simple

The rules for displaying X3 in X3 Simple format (see example above) are:

  1. First convert the X3 to X3 Whitespace.
  2. For any multi-line literal, replace the opening square bracket and opening quote with a backslash "\", and replace the closing quote and closing square bracket with a backslash "\".
  3. For each X3 object (at any level of nesting) that contains nested X3, replaced the opening square bracket "[" with a forward slash "/".
  4. Delete all lines that contain only closing square brackets.
  5. Delete all remaining opening and closing square brackets. (Note that escaped square brackets within XRIs or data should not be affected by this operation as they must be escaped.)

X3J

Formatting rules for X3J will be added here shortly by Feb 21st.

Canonical Form and Signatures

The basic rules for X3 canonicalization are:

  1. Transform X3 Simple or X3 Whitespace to standard X3.
  2. Remove all comments. OPEN ISSUE: is this necessary?

  3. Sort order: OPEN ISSUE: Should the canonical sort order be subjects sorted by XRI, and within each subject, predicates sorted by XRI?

Open Issues

X3J

  1. Bill proposes that X3J be adopted as the primary XDI serialization format for the following reasons: JSON adoption, existing parsers, ease of representation while maintaining high fidelity (with high fidelity any other serialization format can be converted to X3J and then if needed the X3J can be converted to another format, reducing the number of converters needed for n formats from n*n to 2n)
    1. Drummond and Markus feel that X3S should remain the primary XDI serialization format because: a) it is the most compact, and b) it can handle comments, whereas X3J cannot current handle comments. However X3J is still an excellent option where comments are not needed, and we expect most XDI endpoints will support it.
  2. Bill has written X3J messaging endpoint and will release on Github by Feb. 21

Current Proposal (Based on 2010-02-18 Telecon)

This proposal addresses all issues on the issues list below except #5 (comments). The X3J parsing rules would be:

  1. All XDI subject XRIs and predicate XRIs are expressed as JSON literal values
  2. For XDI objects:
    1. XDI literal = JSON literal
    2. XDI references = JSON list
    3. XDI sub-context = JSON object
  3. The absence of an XDI predicate is expressed using a JSON null
  4. The absence of an XDI object is expressed using a JSON null
  5. An empty XDI subcontext is expressed using an empty JSON object

Issues List

Currently all of these issues (noted by Markus) except #5 are addressed by the proposal above.

Issue #1: Isn't it problematic to have a special value called _context? Is this extra "level" of JSON structure even necessary?

Issue #2: How to distinguish between a literal and a reference? In the above example document I see the following:

        "+email":  "drummond@example.com"
...
        "=web*markus": "=drummond+friend"

In the first case, the predicate has a literal. In the second case, the predicate has a reference. But there's no difference in the JSON structure.

IDEA: Maybe for references ALWAYS use a JSON list, even if there's only one? E.g. the above would become:

        "=web*markus": [ "=drummond+friend" ]

AND: To solve the problem in Note 1, maybe for sub-contexts (inner graphs) ALWAYS use a JSON object (without the special value _context), e.g.

        "+email": 
                {
                        "=drummond+home": "+email",
                        "=drummond+work": "+email"
                },

instead of

        "+email": 
                {
                        "_context": 
                                {
                                        "=drummond+home": "+email",
                                        "=drummond+work": "+email"
                                }
                },

So the rules would be:

Would that work?

Issue #3: How to have a subject without predicates? Apparently with an empty JSON object?

        "=drummond+home": {},

Update after call: Maybe better to use JSON null, i.e.

        "=drummond+home": null,

Issue #4: How to have a predicate without literal, reference and sub-context? In the above example document I see the following:

        "=drummond+home": "+email",

But then how can you have more than 1 predicate on this subject?

Update after call: Maybe better to use JSON null, i.e.

        "=drummond+home":
                {
                        "+email": null
                },

Issue #5: How to express comments? In X3 Standard, X3 Simple, X3 Whitespace and XDI/XML we have the ability to associate "before" and "after" comments with each subject, predicate, reference, literal and sub-context.

Acknowledgements


CategoryInactive

X3Format (last edited 2013-05-02 06:28:41 by drummond-xns)